Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TestApiGateway can flake due to rate limits #277

Open
blampe opened this issue Dec 3, 2024 · 2 comments
Open

TestApiGateway can flake due to rate limits #277

blampe opened this issue Dec 3, 2024 · 2 comments
Labels
kind/engineering Work that is not visible to an external user

Comments

@blampe
Copy link
Contributor

blampe commented Dec 3, 2024

What happened?

error: operation DELETE failed with "Throttling": Too Many Requests (Service: ApiGateway, Status Code: 429, Request ID: 9649383f-8ebe-4073-8cac-100df7a113cb)

We might either throttle ourselves or consider ignoring delete errors (assuming our CI cleanup script will clean up these resources).

Example

https://github.com/pulumi/pulumi-cdk/actions/runs/12140641360/job/33850878721?pr=276

Output of pulumi about

n/a

Additional context

No response

Contributing

No response
@blampe blampe added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Dec 3, 2024
@flostadler flostadler added kind/engineering Work that is not visible to an external user and removed kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Dec 3, 2024
@flostadler
Copy link
Contributor

flostadler commented Dec 3, 2024 •
edited
Loading

I'm a bit surprised that this causes throttling exceptions. We should double check the AWS SDK client configuration in aws-native, it's supposed to be configured with a lot of retries, but maybe this actually makes the situation here worse

@corymhall
Copy link
Contributor

Looks like ApiGateway has a pretty low limit for delete requests

1 request every 30 seconds per account

https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#api-gateway-control-service-limits-table

@corymhall corymhall mentioned this issue Jan 2, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/engineering Work that is not visible to an external user
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@blampe @flostadler @corymhall