My requirement is that:
I have an IBM MQ which is shared across 20 servers and runs the JMS client. Now there will be a specific message in the queue which is intended for a particular thread. The thread need to use a correlationID to fetch the message from all the messages in the MQ.
When I am using onMessage() it is uncertain which thread will Listen to the message. Suppose server-1 is waiting for the message but server-15 listens it. Server-1 gets eventually timed out even though there was a message intended for the thread in server-1.
Please suggest how we are going to handle this scenario without introducing major performance issue.
Use a MessageSelector on the listener container(s). If the correlationId is in the standard JMSCorrelationID header the selector would be JMSCorrelationID=foo to receive all foo messages.
Related
I'm evaluating NATS for migrating an existing msg based software
I did not find documentation about msg timeout exception and overload.
For Example:
After Subscriber has been chosen , Is it aware of timeout settings posted by Publisher ? Is it possible to notify an additional time extension ?
If the elected subscriber is aware that some DBMS connection is missing and cannot complete It could be possible to bounce the message
NATS server will pickup another subscriber and will re-post the same message ?
Ciao
Diego
For your first question: It seems to me that you are trying to publish a request message with a timeout (using the nc.Request). If so, the timeout is managed by the client. Effectively the client publishes the request message and creates a subscription on the reply subject. If the subscription doesn't get any messages within the timeout it will notify you of the timeout condition and unsubscribe from the reply subject.
On your second question - are you using a queue group? A queue group in NATS is a subscription that specifies a queue group name. All subscriptions having the same queue group name are treated specially by the server. The server will select one of the queue group subscriptions to send the message to rotating between them as messages arrive. However the responsibility of the server is simply to deliver the message.
To do what you describe, implement your functionality using request/reply using a timeout and a max number of messages equal to 1. If no responses are received after the timeout your client can then resend the request message after some delay or perform some other type of recovery logic. The reply message should be your 'protocol' to know that the message was handled properly. Note that this gets into the design of your messaging architecture. For example, it is possible for the timeout to trigger after the request recipient received the message and handled it but before the client or server was able to publish the response. In that case the request sender wouldn't be able to tell the difference and would eventually republish. This hints that such type of interactions need to make the requests idempotent to prevent duplicate side effects.
Is it possible to configure the topic to store a copy of just the last message and send this to new connections without knowing client identifiers or other info?
Update:
From the info provided by Shashi I found this two pages where they describe a use case similar to mine (applied over stock prices) by using retroactive consumer and a subscription recovery policy. How ever I'm not getting the desired behaviour. What I currently do is:
Include in the activemq the folowing lines in the policyEntry for topic=">"
<subscriptionRecoveryPolicy>
<fixedCountSubscriptionRecoveryPolicy maximumSize="1"/>
</subscriptionRecoveryPolicy>
Add to the URL used to connect to the brocker (using activemq-cpp) consumer.retroactive=true.
Set the consumer has durable. (But I strongly think this is not want since I only need the last one, but without it I didn't get any message when starting the consumer for the second time)
Start up the broker.
Start the consumer.
Send a message to the topic using the activemq web admin console. (I receive it in the consumer, as expected)
Stop consumer.
Send another message to the topic.
Start consumer. I receive the message, also as expected.
However, if the consumer receives a message, then it goes offline (stop process) and then I restart it, it doesn't get the last message back.
The goal is to whenever the consumer starts get the last message, no mater what (obviously, except when there weren't messages sent to the topic).
Any ideas on what I'm missing?
Background:
I have a device which publishes his data to a topic when ever its data changes. A variable number of consumer may be connected to this topic, from 0 to less than 10. There is only one publisher in the topic and always publish all of his data as a single message (little data, just a couple of fields of a sensor reading). The publication rate of this information is variable, not necessarily time based, when something changes a new updated message is sent to the broker.
The problem is that when a new consumer connects to the topic it has no data of the device readings until a new message is send to the topic by the device. This could be solve by creating an additional queue so new connections can subscribe to the topic and then request the device for the current reading through the queue (the device would consume the queue message which would be a request for data, and then response in the same queue).
But Since the messages send to the topic are always information complete I was wondering if is it possible to configure the topic to store a copy of just the last message and send this to new connections without know client identifiers or other info?
Current broker in use is ActiveMQ.
What you want is to have retroactive consumers and to set the lastImageSubscriptionRecoveryPolicy subscription recovery policy on the topic. Shashi is correct in saying that the following syntax for setting a consumer to be retroactive works only with Openwire
topic = new ActiveMQTopic("TEST.Topic?consumer.retroactive=true");
In your case, what you can do is to configure all consumers to be retroactive in broker config with alwaysRetroactive="true". I tested that this works even for the AMQP protocol (library qpid-jms-client) and I suspect it will work for all protocols.
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic="FOO.>" alwaysRetroactive="true">
<subscriptionRecoveryPolicy>
<lastImageSubscriptionRecoveryPolicy />
</subscriptionRecoveryPolicy>
</policyEntry>
The configuration example is taken from https://github.com/apache/activemq/blob/master/activemq-unit-tests/src/test/resources/org/apache/activemq/test/retroactive/activemq-message-query.xml
Messaging providers (WebSphere MQ for example) have a feature called Retained Publication. With this feature the last published message on a topic is retained by the messaging provider and delivered to a new consumer who comes in after a message has been published on a given topic.
Retained Publication may be supported by Active MQ in it's native interface. This link talks about consumer.retroactive which is available for OpenWire only.
A publisher will tell the messaging provider to retain a publication by setting a property on the message before publishing. Below is how it is done using WebSphere MQ.
// set as a retained publication
msg.setIntProperty(JmsConstants.JMS_IBM_RETAIN, JmsConstants.RETAIN_PUBLICATION)
One of our customers has a JMS based implementation in which there are queues for reading/writing messages. The JMS client needs to write to an outbound queue and the it will read the response from an inbound queue. The JMS client will be deployed across multiple sites and will talk to a single outbound queue for writing messages and will read from a inbound queue (one only) for the responses. Consider the scenario in which there are 100 unique outbound requests and then the consumer gets 100 different responses for the sent requests (assume the messages got delivered correctly). How do I ensure that the messages that the consumer is reading from the inbound queue is for the designated recipient? Do we have to write our own logic to map the request/response? or does JMS have any delivery mechanism based on connection id … etc so that message get delivered to correct requester. Thank you very much in advance, need your expert inputs to design the application correctly. The JMS provider I am using is Apache ActiveMQ.
Regards,
Sumeet C
It sounds like you need REQUEST/REPLY...
Request/Reply - Synchronous
A queue sender sends a REQUEST message, then in the same thread, receives a REPLY. The sending thread blocks until the receiver sends back a reply message, ensuring the reply is for the original request. It's a basic set up that uses temporary queues, REPLY_TO addressing, and JMSCorrelationID...
Apache ActiveMQ Request/Reply
EAI Patterns for JMS Request/Reply
Point-to-Point - Async
If the customer's JMS implementation provides distinct queues for sending requests and receiving replies, you can send messages asynchronously with a unique JMSCorrelationID, and provided the customer sends back a response with that same id, you can receive the response message in a different thread and correlate them with the original request based on the JMSCorrelationID. Technically-speaking, REQUEST/REPLY does the same thing except it blocks and uses temporary queues for sending response messages back to the requestor instead of explicitly-named queues.
I have set up uniform distributed queue with weblogic server 12c. I am trying to achieve order of delivery and high availability with jms distributed queue. In my prototpe testing deployment I have two managed servers in the cluster, let us say managed_server1 and managed_server2. Each of this managed server hosts jms server namely jms server1 and jms server2. I have configured the jms servers with jdbc persistent store. I have enabled server affinity.
I have a producer running such as java queuproducer t3::/managed_server1. I send out 4 messages. From the weblogic monitoring console I see there are 4 messages in the queu since there are no consumers to the queue yet.
Now I shut down managed_server1.
Bring up a consumer to listen on java queuconsumer t3://managed_server2. This consumer cannot consume message since the producer send all the messages to jms server1, and it is down.
Bring up managed server 1, start a consumer to listen to t3://managed_server1 I can get all messages.
Here is my problem say if the managed_server1 went down then there it never came back up, do i loose all my messages. Also if there is another producer sending messages to java queuproducer t3://managed_server2 then order of messages based on the time between these producers are not guanranteed.
I am a little lost, am I missing something. Can unit of order help me to overcome this. Or should I use distributed topic instead of distributed queue, where all the jms server will receive all the messages from producers, but if one jms server where my consumre is listening fails there is only one consumer in my application, when I switch over to other jms server, I might be starting to get messages from the beginning not from where I left off.
Any suggestions regarding the same will be helpful.
Good Question !
" Here is my problem say if the managed_server1 went down then there it never came back up, do i loose all my messages. "
Ans - no you do not loose all your messages, they are stored in the JDBC store configured for the JMS server deployed on managed server 1. If you want the Messages sent to managed_server1 to be consumed from managed_server2 you need to configure JMS migration.
" Also if there is another producer sending messages to java queuproducer t3://managed_server2 then order of messages based on the time between these producers are not guanranteed. Can unit of order help me to overcome this."
Ans - If you want the messages to be consumed strictly in a certain order, then you will have to make use of unit of order (UOO). when messages are sent using UOO, they are sent to one of the several UDQ destinations, if midway that destination fails, and migration is enabled the messages are migrated to the next UDQ destination and new UDQ messages are also delivered to the new destination.
Useful links -
http://www.youtube.com/watch?v=B9J7q5NbXag
http://www.youtube.com/watch?v=_W3EJ8p35lI
Hope this helps.
I'm dealing with a standalone MQ JMS application, our app need to "aware" that client already consumed the message producer put on the queue. Because client app is not responsible by us. So we cannot let them to write something like "msg.acknowledge();" thing on their side (msg.acknowledge() is not the right approach on my condition.). I search the history answer in the stackoverflow. Find following is quite the same what I want:
https://stackoverflow.com/questions/6521117/how-to-guarantee-delivery-of-the-message-in-jms
Do the JMS spec or the various implementations support delivery confirmation of messages?
My question is, is there any other way to archive this in the MQ API or JMS API? I need to do the coding only on the msg produce side, it is can be queue or topic.
Another question is in the JMS the acknowledge mode CLIENT_ACKNOWLEDGE, is that produce irrelevant? I always believe that this mode can block the application when call send() method until the client consume the message and call the msg.acknowledge(), but seems not like that. The produce just exit the app after message be delivered, and the message just store in the queue until client call the acknowledge(). Is that possible let the producer app hang there wait until the message be acknowledged by the client?
If my concept is not right, just correct me, thanks.
The main intention of message queuing is to decouple producer and consumer. Producer does not need to wait for the message to be consumed by the consumer, it can continue it's job. Ideally if producer needs to know if the message has been processed by consumer or not, it should wait for consumer to send a response message on another queue.
Message acknowledgement has nothing to do with producer. Message acknowledgement is the way a consumer tells the messaging provider to remove the message from a queue after the message has been delivered to an application.
There is auto acknowledge where the JMS providers (like MQ JMS), after delivering message to an application, tell the messaging provider to remove the message from queue. Then there is client acknowledge where, after receiving a message, the application explicitly tells the messaging provider to remove message from a queue.
Is there is a reason why the producer has to wait for consumer to receive the message? One way, though not elegant, could be: Once the message is sent, use the message id of the sent message and try to browse for that message. If message is not found, you can assume it has been consumed