I know Spring but I am a newbie in JMS and started reading the Spring JMS. From the Spring JMS doc Spring doc, I read the following
The number of concurrent sessions/consumers to start for each listener.
Can either be a simple number indicating the maximum number (e.g. "5")
or a range indicating the lower as well as the upper limit (e.g. "3-5").
Note that a specified minimum is just a hint and might be ignored at
runtime. Default is 1; keep concurrency limited to 1 in case of a topic
listener or if queue ordering is important; consider raising it for
general queues.
Just want to understand why should the concurrency limited to 1 in case of a topic listener? If I increase it, say to 10 instead of 1, what would happen?
Each subscriber receives a copy of each message published to a Topic. It makes no sense at all to put up multiple consumer, since all your application would do is to receive the same message 10 times, in different threads.
In the case of a Queue, messages on the queue would be distributed among 10 threads and hence, handled concurrently. That is indeed a very common scenario - load balancing.
Related
I am trying to consume messages from three IBM MQ queues using JMS.
So, there are three #JmsListener in my spring boot application.
I have a doubt about it, how will they behave if all consumer can consume from their respective queues.
Will there be any concurrency?
If not what is the best way to concurrently consume from queues as I can't afford the serial execution of the application.
Thanks in advance :)
On the JmsListener annotation you can set the concurrency behavoir:
concurrency
public abstract String concurrency
The concurrency limits for the listener, if any. Overrides the value
defined by the container factory used to create the listener
container. The concurrency limits can be a "lower-upper" String — for
example, "5-10" — or a simple upper limit String — for example, "10",
in which case the lower limit will be 1.
Note that the underlying container may or may not support all
features. For instance, it may not be able to scale, in which case
only the upper limit is used.
Source: https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/jms/annotation/JmsListener.html#concurrency--
Each Listener runs in it's own Thread.
You can easily test this by printing the message received. This will output the Thread it runs in. For example:
2020-06-06 11:26:54.339 INFO 23404 --- [enerContainer-1] c.e.d.ShippingService : Hello World!
The thread name is [enerContainer-1]
Please read more in the documentation about Spring and JMS https://docs.spring.io/spring/docs/current/spring-framework-reference/integration.html#jms-receiving
I am trying to implement a kind of Priority Channel with spring integration but I am blocked and didn't find a solution on the web.
I would like to read multiples channel (6) alternatively with a service activator. Each channel is for a priority level (CRITICAL, HIGHEST, HIGH, NORMAL, LOW, LOWEST). Message come from RabbitMQ and are distributed on correct channel with a Router.
The problem is that I would like to create a Service Activator who read alternatively in the channels using a round robin based on time.
For example, CRITICAL should have a 5 secondes running time, and then the service switch to HIGHEST for 3 seconds, and then to HIGH for 1 second, ...
Is it possible to do it properly with spring integration ?
Maybe I don't use the correct component to do it ?
Regards
The Priority Channel pattern works a bit different way.
It is a queue with sort support. When a new message arrives to the queue it is sorted to the proper place according to its priority. That absolutely does matter how your consumer of this channel works. The priority happens only in the channel. The consumer just polls messages from that queue like they are ordered for it: the CRITICAL, than HIGHEST, if CRITICAL aren't present and so on.
On the other hand, if you distribute messages by priority do different channels, why just don't have separate Service Activators for each of those channels? And each priority will be read by its own process.
There is no such a solution based on the "time to run". It just doesn't seem with a good fit for messaging architecture. Although you might can implement via scheduled task cancel() or Quartz to "perform task until...".
UPDATE
Regarding time control, I think you can come up with the solution which in the infinite loop really start()s different service activators and stop()s them after an appropriate scheduled time. All those service activators should listen to different queue channels.
I've come across a curious detail in the legacy integration solution based on WebSphere MQ 7.0.1.3 and WebSphere Message Broker 7.0.0.7. There are 2 message flows:
The 1st flow is a case of MQ Request-Reply pattern. After MQPut it has a MQGet node that gets the message by correlation ID from queue "MQ_BIS_IN".
The 2nd flow is a kind of a one-way router that starts with a MQInput node (without any filters) that listens on the queue "MQ_GW_IN".
Interestingly, "MQ_BIS_IN" is an alias for "MQ_GW_IN" queue. My first thought was that the 2 flows would interfere in a bad way, basically the "omnivorous" MQInput would ruin the Request-Reply thing. But they seem to somehow get along.
I am going to reproduce this configuration in a test environment to determine if their behaviour is stable under load. Nevertheless, does anybody know if there are some rules of precedence between concurrent read operation from the same queue? Does it matter that there's an alias to the queue?
Both the MQInput and the MQGet node can be configured to look for particular msgId's or correlation Id's only, or to pick up the items on the queue in a determined order, or only pick up complete groups of messages - so there doesn't need to be a conflict here.
If I have the 3 queues of priority 1,2 & 3 respectively. I want my consumer to consume first from queue withe priority 1, then 2 & so on. If in case queue with higher priority is empty, the consumer can consume from the queue with lower priority.
Is it possible to achieve from JMS or ActiveMQ or any other way?How?
You'd have to control that logic yourself using this method. To ActiveMQ, or any other JMS provider, you are just using another queue.
However, you can use a single queue for message priority. There are a couple different ways on how to do this as described in the documentation.
If you want your consumer to be as simple as possible then have the broker figure out the priority. Otherwise you'll need to mess with multiple consumers or inefficient single consumer logic with selectors to consume.
In both cases, your producer will just need to be smart enough to set the JMSPriority header to whatever priority the logic says it should be.
The only downside really is the fact that you have a broker side config to set up for that queue specifically rather than everything being automatic.
Concerning ActiveMQ: I have a scenario where I have one producer which sends small (around 10KB) files to the consumers. Although the files are small, the consumers need around 10 seconds to analyze them and return the result to the producer. I've researched a lot, but I still cannot find answers to the following questions:
How do I make the broker store the files (completely) in a queue?
Should I use ObjectMessage (because the files are small) or blob messages?
Because the consumers are slow processing, should I lower their prefetchLimit or use a round-robin dispatch policy? Which one is better?
And finally, in the ActiveMQ FAQ, I read this - "If a consumer receives a message and does not acknowledge it before closing then the message will be redelivered to another consumer.". So my question here is, does ActiveMQ guarantee that only 1 consumer will process the message (and therefore there will be only 1 answer to the producer), or not? When does the consumer acknowledge a message (in the default, automatic acknowledge settings) - when receiving the message and storing it in a session, or when the onMessage handler finishes? And also, because the consumers are so slow in processing, should I change some "timeout limit" so the broker knows how much to wait before giving the work to another consumer (this is kind of related to my previous questions)?
Not sure about others, but here are some thoughts.
First: I am not sure what your exact concern is. ActiveMQ does store messages in a data store; all data need NOT reside in memory in any single place (either broker or client). So you should actually be good in that regard; earlier versions did require that all ids needed to fit in memory (not sure if that was resolved), but even that memory usage would be low enough unless you had tens of millions of in-queue messages.
As to ObjectMessage vs blob; raw byte array (blob) should be most compact representation, but since all of these get serialized for storage, it only affects memory usage on client. Pre-fetch mostly helps with access latency; but given that they are slow to process, you probably don't need any prefetching; so yes, either set it to 1 or 2 or disable altogether.
As to guarantees: best that distributed message queues can guarantee is either at-least-once (with possible duplicates), or at-most-once (no duplicates, can lose messages). It is usually better to take at-least-once, and make clients to de-duping using client-provided ids. How acknowledgement is sent is defiend by JMS specification so you can read more about JMS; this is not ActiveMQ specific.
And yes, you should set timeout high enough that worker typically can finish up work, including all network latencies. This can slow down re-transmit of dropped messages (if worked dies), but it is probably not a problem for you.