I created a AQ in oracle and wrote 2 JMS consumers in Java to listen to the queue. I have observed sometimes that if I produce some message in to queue; the count of dequeued messages from queue is greater than what enqueued. It means that some messages are consumed twice.
I have created queue with property:- multiple_consumers => FALSE
And JMS consumers are working in CLIENT_ACKNOWLEDGE mode
Please help me learn the possible reasons for such behavior and it's solution. So, that I can replicate the problem and solve above issue and ensure that the number of message enqueued is equal to number of message dequeued in case of multiple JMS consumers listening to same AQ .
Without having seen your code, CLIENT_ACKNOWLEDGE typically says you are sending acknowledgements manually. If you do not send an ack, the message won't get deleted and the broker will try to redeliver it at a later stage (like when you restart the connection or similar). This might be the cause of your concern.
Related
I am using Nifi to get data from IBM MQ. It is working fine. My question is once the message is read from an MQ queue, does it get deleted from the queue? How to just read messages from the queue without deleting them from the queue?
My question is once the message is read from an MQ queue, does it get
deleted from the queue?
Yes, that is the default behavior.
How to just read messages from the queue without deleting them from
the queue?
You use the option: MQGMO_BROWSE_FIRST followed by MQGMO_BROWSE_NEXT on the MQGET API calls.
You can also open the queue for browse only. i.e. MQOO_BROWSE option for MQOPEN API call.
It sounds as if you would like to use a "publish/subscribe" model rather than a "point-to-point" model.
From ActiveMQ:
Topics In JMS a Topic implements publish and subscribe semantics. When
you publish a message it goes to all the subscribers who are
interested - so zero to many subscribers will receive a copy of the
message. Only subscribers who had an active subscription at the time
the broker receives the message will get a copy of the message.
Queues A JMS Queue implements load balancer semantics. A single
message will be received by exactly one consumer. If there are no
consumers available at the time the message is sent it will be kept
until a consumer is available that can process the message. If a
consumer receives a message and does not acknowledge it before closing
then the message will be redelivered to another consumer. A queue can
have many consumers with messages load balanced across the available
consumers.
If you have a queue, when a consumer consumes that message, it is removed from the queue so that the next consumer consumes the next message. With a topic, multiple consumers can be subscribed to that topic and retrieve the same message without being exclusive.
If neither of these work for you, I'm not sure what semantics you're looking for -- a "queue" which doesn't delete the message when it is consumed will never let a consumer access any but the first message.
I have a HornetQ based JMS provider and the consumer is attached to the provider in the CLIENT_ACKNOWLEDGE_MODE . The message.acknowledge () snippet though is under an if else . Thus the consumer would sometimes would not send an acknowledgement to the server in case there was an application layer failure in processing the message .So there are two questions here -
1)Will more messages that are queued in the server will keep on
flowing to the consumer even though the consumer did not acknowledge
on of the messages as stated earlier
2)Will the unacknowledged message flow down again on restarting the
consumer .
These are some of my observations on the questions I have asked
1)The messages keep on flowing down to the consumer as per the
consumer logs even though it did not acknowledge one of the messages
to the server due to an application layer failure (Note , there was
no uncaught exception as such , just that the consumer did not
acknowledge) .
2)Secondly , on restarting the consumer as well the message did not
flow down again from the server which is surprising .
Can someone please clarify this behaviour?
It has hard to fully determine what your application is doing in terms of message acknowledgement but my guess is that you are continuing to acknowledge messages after the failed attempt to acknowledge the message in question. In that case because you are using the Client Acknowledge mode the next Acknowledge will also apply to the previous message as that is how client mode works.
Session.CLIENT_ACKNOWLEDGE: A client acknowledges a message by calling the message’s acknowledge method. In this mode, acknowledgment takes place on the session level: Acknowledging a consumed message automatically acknowledges the receipt of all messages that have been consumed by its session. For example, if a message consumer consumes ten messages and then acknowledges the fifth message delivered, all ten messages are acknowledged.
So if you read that carefully you will see how consumers in a Session acknowledging messages while in client Acknowledge mode could even affect one another.
For a better answer you'd need to break down the chain of event further so that in became more clear what is going on.
We have a JMS queue and multiple competing clients are reading from this queue.
Once the message is taken and successfully processed, we want to send the acknowledge to delete ( i.e. CLIENT ACKNOWLEDGE )
However, we want to make sure that if one client has picked the message another client should not take it from the queue.
Does activeMQ provide this feature out of the box using some configuration ?
Moreover:
If the message processing failed after picking the message, so it could not be acknowledged back, in this scenario we should like other client thread to pickup the message. Is it possible out of the box with configuration , may be specifying timeout values ?
Regards,
JE
You need to take some time to understand the difference between a Topic and a Qeueue in order to understand why the first question is not an issue.
For the second question it depends a bit on the ACK mode you are using and how you are processing messages sync or async. Normally for processing where you want to control redeliveries you would do the work inside of a transaction and if the processing fails the message would be redelivered when the TX is rolled back. ActiveMQ supports redelivery policies both client side and broker side that control how many time a message will be redelivered before sent to a DLQ.
As I understand (http://activemq.apache.org/message-redelivery-and-dlq-handling.html) a redelivery can be done by either a consumer or a broker.
I have some questions though:
How does the redelivery by a consumer work underneath? Does the consumer cache the message from a broker and redeliver it locally? What does happen if a consumer terminates inbetween? Such message will be lost? I think that as long as the consumer does not acknowledge the message it shouldn't be. But in such case, the message will be still available on the broker?
Are there any guidelines when to use broker vs consumer redelivery? Any recommendations?
The consumer does cache and redeliver the message to the client locally, until the redelivery count is met, then automatically acks the message as bad (posin ack). A consumer can control if it gets marked as redelivered depending on the acknowledgment mode. If for whatever reason a consumer can't or does not want to process a message, it can also kick it back and it will be available for consumption again if it closes the session.
The broker will hold onto the message until it gets an ack from the consumer. If your consumer is set to AUTO_ACKNOWLEDGE then it is possible you could lose the message if an unhandled exception occurs or the consumer ends unexpectedly for example.
Otherwise, if your consumer is using transactions or CLIENT_ACKNOWLEDGE it will give you the control on when that occurs.
With transactions, if the consumer drops prior to a commit it will be available for the next consumer or whenever that consumer reconnects.
I've always used transaction over CLIENT_ACKNOWLEDGE so I don't want to say for sure that the message will be lost in if the Session.recover() is not called before the consumer goes down or not.
From a consumer stand point, this is also known as retry logic.
Regarding broker vs consumer redelivery: By default, the broker just keeps giving the consumer the same message until the redelivery count is met. If you tell the broker not to redeliver it after a given amount of time, then your consumer can work on consuming other messages that may be able to be processed.
When to do this is really up to what is going on in your application. Maybe a specific message needs to be put to a database and that database is not available at the moment and you want to move on to messages that go elsewhere/have another purpose?
I have a persistent queue, non-transacted, client-acknowledge, the consumers read with jms.prefetchPolicy.queuePrefetch=1&wireFormat.maxInactivityDuration=50000
and once a consumer processes a message, it ack's the message.
If the consumer reads the message, and before it can send an ack, the process terminates abruptly, what happens in ActiveMq? (What ActiveMq parameters come into play here?)
How is that different than if the the consumer will take 10 minutes to process the message (so the consumer task is alive and working), how does ActiveMq know the message is still being worked on? (Does it monitor the TCP/IP connection, if the connection dies, it assumes the message will not be Ack'ed?)
How do I determine if a message is a "poison pill", i.e. it makes the consumers crash? (the redelivery count seems to be valid if the consumer task does not die; is there an internal counter in the message that says "it was been read n times without being successfully ack'ed?")
As an experiment, I sent 6 messages, one of them being a "poison pill" (kills the consumer before the consumer can send the ack), with 2 simultaneous consumers running (and automatically restarting consumers to bring the count to 2 whenever a consumer dies). Looking at the queue (using jconsole, I enable jmx using broker.setUseJmx(true)), 4 messages were delivered, 2 are in-flight. Why would there be 2 in-flight instead of just one?
I've been reading the ActiveMq and JMS specs for a while without clear/conclusive answers, so any insights on what parameters come into play, and if there are any known bugs, will be greatly useful.
This is purely based on my understanding of JMS - may not be completely correct:
If the consumer reads the message, and before it can send an ack, the process terminates abruptly, what happens in ActiveMq
My understanding is that since this happens in the context of a session with the JMS provider, JMS provider knows if the session is no longer active or has failed and any message not acknowledged as part of the session will be redelivered when the session is re-established.
How do I determine if a message is a "poison pill", i.e. it makes the consumers crash?
Like you have mentioned, the JMS provider keeps track of the # of times the message was redelivered possibly in the header of the message
4 messages were delivered, 2 are in flight
Not sure about this point