Retry GCS on 503 errors #49982

New Issue

2016-11-10T09:01:44+01:00

Sybren A. Stüvel commented

2016-11-10 09:01:44 +01:00

Google Cloud Service sometimes throws 503 Service Temporarily Unavailable errors at us. We might want to retry once or twice after such a reply, to see if the service has come up. However, we should consider this carefully, given that:

we probably want to pause for a second or two between retries, to give GCS to come up and service us again, and
during this delay the Pillar process is sleeping, blocking others from performing requests on it.

If we do this the wrong way, a hickup at GCS could cause all Pillar processes to hang in a retry loop, DOSsing ourselves.

Probably related to #48956.

Google Cloud Service sometimes throws 503 Service Temporarily Unavailable errors at us. We might want to retry once or twice after such a reply, to see if the service has come up. However, we should consider this carefully, given that: - we probably want to pause for a second or two between retries, to give GCS to come up and service us again, and - during this delay the Pillar process is sleeping, blocking others from performing requests on it. If we do this the wrong way, a hickup at GCS could cause all Pillar processes to hang in a retry loop, DOSsing ourselves. Probably related to #48956.

Sybren A. Stüvel commented

2016-11-10 09:01:44 +01:00

Changed status to: 'Open'

Sybren A. Stüvel commented

2016-11-10 09:01:44 +01:00

Added subscribers: @dr.sybren, @fsiddi, @pablovazquez

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: archive/pillar#49982

Download

What's New

Roadmap

Documentation

Blender Studio

Manual

Benchmark

Blender Conference

Development Fund

One-time Donations

Retry GCS on 503 errors #49982