You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When submitting a large number of jobs at once, some job step function executions fail when submitting the AWS Batch job with:
Too Many Requests (Service: AWSBatch; Status Code: 429; Error Code: TooManyRequestsException; Request ID: 8c6b2219-1c07-4c9d-bb18-99facd090b01; Proxy: null)
The start-execution-manager lambda invokes start-execution-worker with up to three batches of 300 jobs. The worker lambda became more efficient earlier this year; runtime for 300 jobs is now around 20 seconds. That occasionally pushes us over the 50 SubmitJobs calls per second service limit.
When submitting a large number of jobs at once, some job step function executions fail when submitting the AWS Batch job with:
The
start-execution-manager
lambda invokesstart-execution-worker
with up to three batches of 300 jobs. The worker lambda became more efficient earlier this year; runtime for 300 jobs is now around 20 seconds. That occasionally pushes us over the 50 SubmitJobs calls per second service limit.For more context, see #1272
The text was updated successfully, but these errors were encountered: