I'm using a Spring project where I have an implementation of JMS Event listener to process messages from a queue.
To be precise, I'm using a SQS (AWS) queue.
All works fine.
My point is this:
I not configured anything about the concurrency but I would like to have more threads as listener to increase the performances (speed) about the messages processing from the queue.
I'm thinking about the possibility to configure a ThreadPool (TaskExecutor) and adding the annotation #Async on my methods about the message processing.
So, I will have a onMessage method into my listener where, after message validation I will call this async methods.
Is this a good practice? Will I have some issues using this approach?
I'm looking on the web for this and I see that I's possible to configure directly the concurrency value on the listener.
I'm very confusing there are a lot of possible ways to have this and I'm not able to understand the best approach.
Are these equivalent solutions?
Do not use #Async - simply increase the concurrency of the listener container and it will be handled for you automatically by spring-jms.
Related
As per the documentation int-jms:message-driven-channel-adapter uses SimpleAsyncTaskExecutor
SimpleAsyncTaskExecutor doesn't reuse threads and creates a new thread for each task. In case of message-driven-channel-adapter what is the definition of a task?
In case of message driven channel Adapter the task is a constantly polling loop. So, this is going to be a long- living resource which keeps thread active. Therefore we don’t care too much about source of threads. See Spring JMS for more information.
In my POC, I am using Spring Cloud Config and Spring Stream Rabbit. I want to dynamically change number of listeners (concurrency). Is it possible to do that? I want to do following:
1) If there are too many messages in queue, i want to increase concurrency level.
2) In scenario where my downstream system is not available, I want to stop processing messages from queue (in short concurrency level 0).
How i can achieve this?
Thanks for help.
The listener container running in the binder supports such changes (although you can't go down to 0, but the container can be stop() ped).
However, spring-cloud-stream provides no mechanism for you to get a reference to the listener container.
You might want to consider using a #RabbitListener from Spring AMQP instead - it will give you complete control over the listener container.
My application requires that all messages with a particular group id be processed by the same thread. I tried to set this up using the DMLC, but as I am seeing messages being processed, they are being handled by separate threads within DMLC when concurrency is set > 1. My hope was that I could have multiple consumers reading from the queue each with different message groups, but I am not seeing the desired behavior with this setup. Is there a way using DMLC or SMLC (with JTA) to support multiple listeners on a queue where all messages with a particular group id are handled by the same thread, without setting the concurrency to 1? I'm nearing the point where I believe SMLC and DMLC will not meet this need and I will need to use a different implementation to handle this case. Are there any examples or advice on doing this?
Can you post your code that exhibits this behavior? I just ran a test while answering this question and it works perfectly fine for me with all messages for the same group going to the same thread.
Am using the #MessageMapping(queue_name) annotation along with the spring-aws-cloud library to consume messages from two SQS queues.
I see that the SimpleMessageListenerContainer is used as the listener container for both message queues. There is a start() and a stop() method on the container, but I believe that will start/stop consumption from both queues.
I wanted to know if there is a way to programatically start/stop queue consumption for the queues separately. How can that be achieved?
Use start(queueName) and stop(queueName) function of SimpleMessageListenerContainer to start and stop individual queues programmatically.
You need separate containers for each queue to start/stop listening independently.
From a quick look at the code, it appears to me that there is only one listener container when using the annotation to create listeners.
I suggestion you open an issue/feature suggestion in GitHub; I suggest you post the issue number back here so interested parties can follow the discussion.
I need to fire off a long running batch type job, and by long we are talking about a job that can take a couple of hours. The ejb that has the logic to run this long running job will communicate to a NoSQL store and load data etc.
So, I am using JMS MDBs to do this asynchronously. However, as each job can potentially take up to an hour or more (lets assume 4 hours max), I dont want the onMessage() method in the MDB to be waiting for so long. So I was thinking of firing off an asynchronous ejb within the onMessage() MDB method so that the MDB can be returned to the pool right after the call to the batch ejb runner.
Does it make sense to combine an asynchrous ejb method call withing an MDB? Most samples suggest using 1 or the other to achieve the same thing.
If the ejb to be invoked from the MDB is not asynchrous then the MDB will be waiting for potentially long time.
Please advise.
I would simplify things: use #Schedule to invoke #Asynchronous and forget about JMS. One less thing that can go wrong.
Whilst not yet ready for prime time, JSR 352: Batch Applications looks very promising for this sort of stuff.
https://blogs.oracle.com/arungupta/entry/batch_applications_in_java_ee
It's a matter of taste I guess.
If you have a thread from the JMS pool running your job or if you have an async ejb do it, the end result will be the same - a thread will be blocked from some pool.
It is nothing wrong with spawning an async bean from a MDB, since you might want to have the jobs triggered by a messaging interface, but you might not want to block the thread pool. Also, consider that a transaction often time out by default way before an hour, so if you do MDB transactional by some reason, you might want to consider fire of that async ejb inside the onMessage.
I think Petter answers most of the question. If you are only using mdb to get asynch behaviour, you could just fire the #Asynchronous asap.
But if you are interested in any of the other features your JMS implementation might offer in terms reliability, persistent queues, slow consumer policies, priority on jobs you should stick to mdb:s
One of the reasons behind introducing #Asynchronous in ejb 3.1 is to provide a more lightweight way to do asynchronous processing when the other JMS/MDB features are not needed.