Amazon SQS FIFO Queue with Multi-Threaded Consumer - Ensuring Order Per MessageGroupId

[onyiuche]

I have a Spring Boot Java application that consumes messages from an Amazon SQS FIFO queue using the @SqsListener annotation. My consumer is multi-threaded, and I have maxConcurrentMessages set to 5, allowing multiple threads to process messages simultaneously.

I understand that SQS FIFO queues maintain the order of messages within the same MessageGroupId and deliver them in the correct sequence. However, I’m concerned that the multi-threaded nature of my consumer might break the FIFO processing guarantees, even though the messages are received in order.

For example, if I have 5 messages (message1 to message5) from the same group (groupA) in the queue, they are delivered in order and can all be in-flight at the same time in the consumer. Since my consumer uses multiple threads, these messages could be split across 5 different threads. My worry is that this setup could cause the messages to be processed out of order, breaking the FIFO guarantee of the queue.

My Questions Are:

1. Can the multi-threaded consumer potentially break FIFO processing within the same MessageGroupId, even if messages are delivered in order and are in-flight simultaneously?
2. What is the best way to ensure that messages within the same group are processed in the correct order, even with a multi-threaded consumer setup?
3. Are there any best practices, synchronization techniques, or specific configurations that can help maintain strict ordering of messages in a multi-threaded environment?

I think the issue could be that some parts of consumer logic are async (web flux), and it is using auto acknowledgement on message success. So thread1 could pick up message1 and then go in an async block and gets acknowledged. Because message1 has been acknowledged/deleted, thread2 goes on to process message2 while message1 is still running in its async block. It might need to acknowledge messages manually or introduce some synchronicity

I appreciate any insights or advice on how to manage this scenario effectively!