I am trying to build with amqmdnet a IBM MQ receiver with 2 functionalities:
ReceiveReply(byte correlationId) – reads REPLY messages for supplied correlation identifiers. Is implemented by a GET with a correlation MATCH.
event EventHandler NewMessage – notifies the subscriber of new REQUEST messages. Is implemented by a GET with unlimited wait in infinite while loop.
The problem is that I cannot limit the second GET to read only REQUEST messages and so the correlation GET is never successful. If I only do one GET I will have to implement correlation on my own.
Anyone tried to implement something similar?
Thanks,
Radu
The simplest option is to use two queues. There are ugly ways of achieving what you are after (imagine all request messages putting something in the correlid so you could get by correlid there too, for example) but in principle you have a request queue and a reply queue - dont share!
Related
I'm new to JMS and HornetQ.
I'm wondering if there is a way to implement Message Translator Pattern using HornetQ to split data from a message in a set of smaller data and send them. I explored Bridge and Divert solutions but I can't get how to do it using org.hornetq.core.server.cluster.Transformer and org.hornetq.core.server.ServerMessage. Where can I find some docs about it? Am I looking in the right direction?
In short no(I've no Idea on camel). You cannot modify the jms body once sent until its consumed by a client(body is immutable). However you can change message headers and message properties. The org.hornetq.core.server.cluster.Transformer interface is used for modifying the headers/properties. Hence you are left with two options.
Consume the message, chunk the message based on your algorithem and send to other queues or put back to the queue(but be careful to avoid loop, by having suitable selector).
Other approach is chunk the message then send with message property to differentiate the message. And use the diverter with filter based on the message property(you can use exclusive/non exclusive strategy to send only/send copy of message to the other queue.)
Say I have one JMS message FooCompleted
{"businessId": 1,"timestamp": "20140101 01:01:01.000"}
and another JMS message BazCompleted
{"businessId": 1,"timestamp": "20140101 01:02:02.000"}
The use case is that I want some action triggered when both messages have been received for the business id in question - essentially a join point of reception of the two messages. The two messages are published on two different queues and order between reception of FooCompleted and BazCompleted may change. In reality, I may need to have join of reception of several different messages for the businessId in question.
The naive approach was that to store the reception of the message in a db and check if message(s) its dependent join arm(s) have been received and only then kick off the action desired. Given that the problem seems generic enough, we were wondering if there is a better way to solve this.
Another thought was to move messages from these two queues into a third queue on reception. The listener on this third queue will be using a special avataar of DefaultMessageListenerContainer which overrides the doReceiveAndExecute to call receiveMessage for all outstanding messages in the queue and adding messages back to the queue whose all dependent messages have not yet arrived - the remaining ones will be acknowledged and hence removed. Given that the quantum of messages will be low, probing the queue over and adding messages again should not be a problem. The advantage would be avoiding the DB dependency and the associated scaffolding code. Wanted to see if there is something glaringly bad with this
Gurus, please critique and point out better ways to achieve this.
Thanks in advance!
Spring Integration with a JMS message-driven adapter and an aggregator with custom correlation and release strategies, and a peristent (JDBC) message store will provide your first solution without writing much (or any) code.
Since it does not seem to be possible to query/inspect the underlying ZeroMQ queues/buffers sockets to see how much they are utilized, is there some way to detect when a message is dropped due to full buffers in a Publisher socket when sent/queued?
For example, if the publisher queue is full, the zmq_send operation will simply drop the message.
Basically, what I want to achieve is a way to detect situations where the queues are getting stressed and/or full to be able to (later on) tune the solution to work better. One alternative way would be to add a sequence number to each message and do a simple calculation in the subscriber but I can never be sure that a message was lost due to full buffers in the publisher.
There is an example for this in the ZeroMQ Guide (which you should read and digest if you want to use 0MQ happily): http://zguide.zeromq.org/page:all#Slow-Subscriber-Detection-Suicidal-Snail-Pattern
The mechanism is as you answered yourself, to add a sequence number in the message, and allow the subscriber to detect gaps and take appropriate action. For most pubsub scenarios you can raise the default HWM, which is 1,000, to something much higher; it depends on your average message size.
I know this is an old post but here is what I did when recently facing the same issue.
I opted to use a DEALER/ROUTER and set the ZMQ_SNDHWM option to 1. Also I provided the timeout parameter on each zmq_send(). The timeout could be anything between 10 ms to 3 seconds, depending on what your scenario is ( a local or remote send ).
If the message is not sent within the timeout or the send-buffer is full the zmq_send() will return false. That enabled me to set up a retry queue in front of zmq. I know it's not a perfect solution but for me it worked just fine. What puzzles me though is the meaning of true/false returned by the DEALER-socket zmq_send(). I have not been able to find the answer to that question. Whether it indicates that the message has been buffered or that the message has been delivered to the ROUTER has eluded me. In my case I got the results needed anyway.
Just for the record this was done using netmq but I guess it applies to ZeroMQ as well.
I do agree wtih james though. ZeroMQ ( and netmq ) should at least provide a way to inspect the queue ( and get the messages out ) and also a way to tell the various sockets not to drop messages. The best option would be to send messages not delivered in timely fashion according to the configured options to some sort of deadletter queue. The deadletter queue could then be handled separately.
I have many instances of my client application. These clients send requests to a server application via messaging and receive a reply. Normally the reply would be sent using a temporary queue.
Unfortunately I have to use the Stomp protocol which has no concept of temporary queues or topics. (Although the message broker has)
What's the best way to ensure only the original requestor receives the reply? Are there any best-practices for this unfortunate situation?
The customary solution when several requestors listen for replies on the same queue is to use correlation IDs to select messages. On the client side it looks like this:
Place a message on the request queue and commit.
Retrieve the JMSMessageID from the outbound message (the value is determined by the broker and updates the message object as a result of the send).
Receive a message from the reply queue specifying the JMSMessageID from the outbound message as the correlation ID in the selector.
Process and commit.
On the server side it looks like this:
Receive a message under syncpoint.
Process the request and prepare the response.
Set the JMSCorrelationID on the response to the value of JMSMessageID from the request.
Send the message.
Commit.
The consumer would set the selector something like this: activemq.selector:JMSCorrelationID=.
Since the broker creates a message ID that is supposed to be globally unique, the pattern of using it as the correlation ID prevents collisions that are possible when each requestor is allowed to specify it's own value.
The best way to implement this pattern with JMS (that I've found, anyway) is to create a pre-configured topic for the response messages, and use correlation selectors on the response message so that the client can get the correct one.
In more detail, this means setting a random ID on the request message (using setJMSCorrelationID()), and putting that message on the request Queue. The consumer of that request message processes it, creates the response message, sets the same correlation ID on the response message, and puts it on the response Topic. The client, meanwhile, is listening on the response topic with a selector expression which specifies the correlation ID that it's expecting.
The danger is that the response message is sent before the client can get around to listening for it, although that's probably unlikely. You can try using a pre-configured Queue for the responses rather than a topic, but I've found that topics tend to work more reliably (my JMS provider of choice is HornetQ - your mileage may vary).
All this is tell me that JMS is a very poor fit for the request/response model. The API just doesn't support it properly. This is hardly surprising, since that was never a use-case for JMS.
Something like a compute grid (Terracotta, Gigaspaces, Infinispan, etc) would likely yield better results, but that's not really an option for you.
We have quartz process that polls a ActiveMQ JMS queue.
We know that we could get several messages a minute would like to only respond to the most current message at a configured polling rate of a minute or more.
We don't need to process any of the previous messages.
Is there a way to configure the queue to get this behavior?
Its seems like a topic has the ability to do this via the subscription recovery policy using a count of 1. We would like to do this using a queue to guarantee (more or less) a single delivery of the message.
Or is there a conceptual flaw in our assumptions...
Thanks
In my opinion there is no standard operation for this, so you will have to write some code....
One possible solution would be to use a QueueBrowser together with a QueueReceiver:
Through the QueueReceiver you would get an Enumeration of the messages in the queue. For each message you can now perform a receive with a MessageSelector on the JMSMessageID as long as hasMoreElements() returns true. The last message will be the one you want to have....
When using activemq, you can use "image caching" on topics. One of the settings there is to always keep the last mesage sent..
Take a look at the Subscription recovery Policy settings:
http://activemq.apache.org/subscription-recovery-policy.html