If I have the 3 queues of priority 1,2 & 3 respectively. I want my consumer to consume first from queue withe priority 1, then 2 & so on. If in case queue with higher priority is empty, the consumer can consume from the queue with lower priority.
Is it possible to achieve from JMS or ActiveMQ or any other way?How?
You'd have to control that logic yourself using this method. To ActiveMQ, or any other JMS provider, you are just using another queue.
However, you can use a single queue for message priority. There are a couple different ways on how to do this as described in the documentation.
If you want your consumer to be as simple as possible then have the broker figure out the priority. Otherwise you'll need to mess with multiple consumers or inefficient single consumer logic with selectors to consume.
In both cases, your producer will just need to be smart enough to set the JMSPriority header to whatever priority the logic says it should be.
The only downside really is the fact that you have a broker side config to set up for that queue specifically rather than everything being automatic.
Related
My system has a constrain for specific consumer that messages should be handled in order, one after the other. To implement that we set the concurrency to 1.
Now we want to scale out and add more instance of this consumer.
To keep the order I want to use distributed lock manager like 'RedLock'. It can tell each consumer if it is OK to fetch the next message.
I work with RabbitMq and my question is if there is kind of observer event that comes before getting messages from the queue. In other words I need a way to enable/disable the operation of polling messages from the queue.
We are using ActiveMq in our application. We need to implement priority consumer mechanism on the queue. We have multiple consumers and we want to treat them as master/slave. So, the master consumer would have higher priority and will consume all messages and once master gets down then the consumer (with higher priority) will consume all requests from the queue.
To Implement this scenario, we came across the mechanism of priority queues. So, we initialized the queues by this way:
javax.jms.Queue queue = queueSession.createQueue("myQueue" + "?consumer.priority=" + 127);
and remaining slaves nodes were assigned lesser priorities.
But this mechanism is not working expectedly, sometimes it works fine that we get all requests on the consumer with the highest priority but sometimes consumer with lesser priority also start consuming messages while the consumer with the highest priority is working fine.
We have tried it with consumer.exclusive=true option as well, with no luck.
Note: We are running all components (ActiveMQ, producer and consumer application) locally on the same machine for now, so no network delays involved. And we are running consumers with default prefetch policy.
Is there any other approach to implement this scenario using activeMq or are we missing any configuration.
I've come across a curious detail in the legacy integration solution based on WebSphere MQ 7.0.1.3 and WebSphere Message Broker 7.0.0.7. There are 2 message flows:
The 1st flow is a case of MQ Request-Reply pattern. After MQPut it has a MQGet node that gets the message by correlation ID from queue "MQ_BIS_IN".
The 2nd flow is a kind of a one-way router that starts with a MQInput node (without any filters) that listens on the queue "MQ_GW_IN".
Interestingly, "MQ_BIS_IN" is an alias for "MQ_GW_IN" queue. My first thought was that the 2 flows would interfere in a bad way, basically the "omnivorous" MQInput would ruin the Request-Reply thing. But they seem to somehow get along.
I am going to reproduce this configuration in a test environment to determine if their behaviour is stable under load. Nevertheless, does anybody know if there are some rules of precedence between concurrent read operation from the same queue? Does it matter that there's an alias to the queue?
Both the MQInput and the MQGet node can be configured to look for particular msgId's or correlation Id's only, or to pick up the items on the queue in a determined order, or only pick up complete groups of messages - so there doesn't need to be a conflict here.
I have implemented a priority queue in ActiveMQ. If the queue is being continuously flooded with the high priority messages, the low priority messages will never get processed. How does ActiveMQ handle such situations or how can this situation be avoided or handled?
ActiveMQ doesn't attempt to do anything to prevent this as it's up to you to solve it based on the needs of your application. If you have such a situation you might want to consider instead using a Queue per priority to allow for load balancing across the Queues.
Extending to Tim Bish's answer, there are some features in ActiveMQ you can use to handle this situation.
You can setup a virtual destination to filter out high and low prio messages, like this (inside a virtualDestinationInterceptor tag).
<virtualDestinations>
<compositeQueue name="ALL">
<forwardTo>
<filteredDestination selector="JMSPriority < 5" queue="LOW.PRIO"/>
<filteredDestination selector="JMSPriority > 4" queue="HIGH.PRIO"/>
</forwardTo>
</compositeQueue>
</virtualDestinations>
Then you can follow the alternative strategy presented here.
You put one consumer on the LOW.PRIO queue and multiple consumers on the HIGH.PRIO queue. Then the LOW.PRIO messages will be handled, but with less threads than high prio messages.
You can also read messages directly from the "ALL" queues with said selectors in your consumer application.
I know Spring but I am a newbie in JMS and started reading the Spring JMS. From the Spring JMS doc Spring doc, I read the following
The number of concurrent sessions/consumers to start for each listener.
Can either be a simple number indicating the maximum number (e.g. "5")
or a range indicating the lower as well as the upper limit (e.g. "3-5").
Note that a specified minimum is just a hint and might be ignored at
runtime. Default is 1; keep concurrency limited to 1 in case of a topic
listener or if queue ordering is important; consider raising it for
general queues.
Just want to understand why should the concurrency limited to 1 in case of a topic listener? If I increase it, say to 10 instead of 1, what would happen?
Each subscriber receives a copy of each message published to a Topic. It makes no sense at all to put up multiple consumer, since all your application would do is to receive the same message 10 times, in different threads.
In the case of a Queue, messages on the queue would be distributed among 10 threads and hence, handled concurrently. That is indeed a very common scenario - load balancing.