Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spring Cloud SQS only consume 10 messages in parallel even though maxConcurrentMessages have a greater value #851

Closed
dbeltranruiz opened this issue Jul 24, 2023 · 2 comments

Comments

@dbeltranruiz
Copy link

dbeltranruiz commented Jul 24, 2023

Type: Bug

Component:
"SQS"

Describe the bug
I used the @SQSListener annotation and defined the maxConcurrentMessages parameter to 100. When consuming messages I first produced 100 messages and then consumed them via an spring cloud app. When looking at the logs and the threads involved I never noticed more than 10 threads consuming messages in parallel.

After further investigation I noticed that the threadPool created by the default listener has a core number of threads set to 10, the max number to 100, and the queueCapacity to 100, but it never added more threads to the pool than the core number because it requires the queue to be full. That never happens because the logic to poll the sqs queue will never ask for more messages than permits and this will never be more than 100.

To work around that, I used a custom threadpool stack with the core number set to 100. With that it nicely consumed those 100 messages in parallel.

@tomazfernandes
Copy link
Contributor

Thanks @dbeltranruiz, we're aware of this, and has been addressed in this PR:
#833

3.0.2 should be out soon with this fix.

cc: @maciejwalkowiak

@dbeltranruiz
Copy link
Author

Great!, thank you very much for confirming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants