SQS failures are not immediately reported back to AWS?

[achuinard]

I am moving an older SQS consumer app that used to use JMS with https://github.com/awslabs/amazon-sqs-java-messaging-lib to now use the new Spring Cloud AWS Starter with @SqsListener annotations.

However, my understanding is that my JMS listener used to immediately report a processing failure back to SQS, which would cause SQS to re-fire / redrive the message and try again. However, this new SQS consumer seems to not report failures at all back to AWS, instead just relying on AWS to refire the message based on the SQS visibility timeout.

Doesn't it make more sense for my app to immediately report failures and retry? Is this something configurable with the new Spring Cloud starter? Or maybe I'm not understanding something 100% right.

[tomazfernandes]

Hi @achuinard, thanks for bringing this up.

SQS works a bit differently from other queuing systems, as it doesn't have a nack or similar feature. So we don't really communicate anything back to SQS for failures - if the message processing is successful we delete the message from the queue, otherwise, as you pointed out, we just wait until SQS redelivers the message.

For the default behavior I don't think we should retry immediately as it's a good practice to have some backoff applied. So I believe this is a fair behavior overall for OOTB functionality.

I agree we could support OOTB customizable behavior though, and we already do to some extent.

You should be able to implement an ErrorHandler and use the SqsAsyncClient to change the message visibility to the desired amount of time for retrial.

It's even possible to create more complex behavior such as exponential backoff delays based on ApproximateReceiveCount.

We should be able to incorporate an ErrorHandler to the project so we can provide this functionality OOTB.

I'll open an issue for this feature request.

How does that all sound? Please let me know if you have any questions or suggestions.

[achuinard]

For now, I can live with the 'wait until visibility timeout expires' behavior. I just was pretty confused since even in my SQS lambdas, when handling a batch of 'events' in my serverless app, I can use the reportBatchItemFailures functionality on the JS side. If your library supports 'instant retry', perhaps I would opt into it in the future.

It would also be good to clarify using this vs. the old JMS lib (https://github.com/awslabs/amazon-sqs-java-messaging-lib). Should Spring Boot apps lean towards your starter or use that library? Seems like your starter is better but maybe the other one needs to get deprecated.

[tomazfernandes]

and use the SqsAsyncClient to change the message visibility to the desired amount of time for retrial.

Actually, we should be able achieve this functionality by using the Visibility implementation object provided in the SQS_VISIBILITY_TIMEOUT_HEADER in the messages.