What does JMS session exactly mean?
What does it mean that a JMS session can be or not "transacted"?
What does it mean that a JMS session can be or not with "auto-acknowledge"?
You could view a JMS Session as the link between a connection and a thread's work on that connection. A JMS Session is a 'single-threaded context', so each thread must use a different session. In contrast, a JMS connection can be shared between multiple threads. Also, one thread could own multiple sessions.
The send/receive operations on a Session are either acknowledged or transacted.
Imagine a consumer crash: usually you want to redeliver a message if the consumer said it hasn't processed a message yet. So an acknowledgment (ACK) can be send from consumer to Broker to tell the JMS Broker that the message has been processed. Not just received but also stored in the Database, or whatever work the consumer is doing. If the consumer is restarted after the crash it will receive all messages that have not yet been acknowledged.
There are different forms of ACKs: auto-acknowledge means that the API will call acknowledge() for you after you've returned from your callback. In the other cases, you will have to call acknowledge() yourself.
Transactions group send or receive operations on that session (all-or-nothing). So you do send/send/send and then commit and know all 3 msgs are made available to the consumer at the time of the commit, or none. Advantages of transacted delivery: highest guarantee of delivery and you can group operations - Disadvantage: latency and possibly performance.
One main purpose of the Session is to keep a transaction's state. If your session was created to be transacted (SESSION_TRANSACTED) it knows all the messages that you are planning to send within your transaction and if that transaction is committed or rolled-back.
All the information is there in JMS Specification here
A JMS Session is an object that maintains connection to a JMS Provider for sending and receiving messages.
A transacted session supports a single series of transactions. Each transaction groups a set of produced messages and a set of consumed messages into an atomic unit of work. In effect, transactions organize a session’s input message stream and output message stream into series of atomic units. When a transaction
commits, its atomic unit of input is acknowledged and its associated atomic
unit of output is sent. If a transaction rollback is done, its produced messages
are destroyed and its consumed messages are automatically recovered.
On the other hand in a JMS Session with auto-acknowledgement, received messages, after they are delivered to application, are automatically removed from the JMS Provider without the need for application to call commit/rollback. Auto-acknowledge has no effect on a transacted session.
Related
Hi I am new to Camel and have a design question related to JMS queues.
I am receiving set of data. These data have a reference date. These data are sent every 15 minutes by a batch process.
I have to process the received data and forward them to another route.
If a given data cannot be processed, I need to reprocess it. And I have to ensure it is processed before the next data set is processed.
So I was thinking of creating a JMS route to receive these data before processing. Then process it. Then send it to another queue.
FTP --> Process data rows (A) --> JMS Queue --> Processor (B) --> direct:call
If processor B fails I want the data to be processed before the next data set is sent by FTP. (because second data set may contain an update of the data of the first dataset)
So I was thinking using a queue, to make sure they are always processed in the order they are being received.
But my experience with JMS, without Camel, is that once the object is consumed from the queue it is not in the queue anymore.
Is it also the case with Camel?
In this case to I have to retry to process the data, or put them back in the queue?
This "recovery" part is not clear to me and I'd like to understand the patterns that do support this.
Many thanks for your help
Gilles
This part "once the object is consumed from the queue it is not in the queue anymore." is not fully correct. Actually, when you are subscribing to the queue and getting a message you need to process it and send acknowledge back to the JMS broker. If acknowledge is successful then the message will be removed from the queue. But if acknowledge will be not successful or if your process will die and connection to the broker will break then the message will not be removed from the queue and will be passed to another consumer.
Often most of the JMS libraries are using mode when acknowledgement is sent right when message was received by consumer but you always have possibility to change this mode and send acknowledgement manually when your processing part will be finished successfully.
What about camel jms (http://camel.apache.org/jms.html) you can use endpoint option "acknowledgementModeName" which has some different possible values like:
AUTO_ACKNOWLEDGE (default) - acknowledgement will be sent right after corresponded "from" in your route
CLIENT_ACKNOWLEDGE - allows the application to control when the acknowledgment is sent and if there are no exceptions will be thrown during exchange processing then message will be acknowledged and removed from queue.
As I understand (http://activemq.apache.org/message-redelivery-and-dlq-handling.html) a redelivery can be done by either a consumer or a broker.
I have some questions though:
How does the redelivery by a consumer work underneath? Does the consumer cache the message from a broker and redeliver it locally? What does happen if a consumer terminates inbetween? Such message will be lost? I think that as long as the consumer does not acknowledge the message it shouldn't be. But in such case, the message will be still available on the broker?
Are there any guidelines when to use broker vs consumer redelivery? Any recommendations?
The consumer does cache and redeliver the message to the client locally, until the redelivery count is met, then automatically acks the message as bad (posin ack). A consumer can control if it gets marked as redelivered depending on the acknowledgment mode. If for whatever reason a consumer can't or does not want to process a message, it can also kick it back and it will be available for consumption again if it closes the session.
The broker will hold onto the message until it gets an ack from the consumer. If your consumer is set to AUTO_ACKNOWLEDGE then it is possible you could lose the message if an unhandled exception occurs or the consumer ends unexpectedly for example.
Otherwise, if your consumer is using transactions or CLIENT_ACKNOWLEDGE it will give you the control on when that occurs.
With transactions, if the consumer drops prior to a commit it will be available for the next consumer or whenever that consumer reconnects.
I've always used transaction over CLIENT_ACKNOWLEDGE so I don't want to say for sure that the message will be lost in if the Session.recover() is not called before the consumer goes down or not.
From a consumer stand point, this is also known as retry logic.
Regarding broker vs consumer redelivery: By default, the broker just keeps giving the consumer the same message until the redelivery count is met. If you tell the broker not to redeliver it after a given amount of time, then your consumer can work on consuming other messages that may be able to be processed.
When to do this is really up to what is going on in your application. Maybe a specific message needs to be put to a database and that database is not available at the moment and you want to move on to messages that go elsewhere/have another purpose?
I have a multiple consumer listening to a queue and when a message arrives its onMessage() function in invoked. Since there are multiple consumers, each consumer has its own session. The requirement is that the message should be acknowledged only if there is no issues and no exception is thrown...)
AUTO_ACK mode: My understanding is: the message is acknowledged after onMessage completes successfully. If there is an exception in onMessage(), the message will be redelivered.
Client ack: At the end of onMessage(), I explicitly call acknowledge(). In case of error the acknowledge() method will not be called so message will be redelivered.
Transacted Session: I call session.commit() at the end of the onMessage() function, in case of some exception I catch it and call session.rollback and hence the message will be redelivered.
The consumer is ABLE to detect duplicate messages and handle it appropriately. My question is that all the 3 modes are doing the same thing and solving my purpose, so which method is better than the other and why? In short why should I use transacted sessions over client ack or auto mode.
I do not want to use JTA/XA as it is not supported by all jms providers example activeMQ and it will make my application slow.
If my consumer was NOT able to handle duplicate messages, then I understand that the only option I have is to use JTA/XA because all other options can send me the message again which will result in duplicate processing. In JTA/XA also I can get the message again but it would not be considered as duplicate processing as the previous transactions would have been rollbacked.
Be careful here about the difference between ack modes as they can be more subtle than you might think.
When using auto acknowledge and throwing an exception in onMessage your message might get redelivered to another client, the same client, or go into a DLQ depending on your broker and how it's configured.
The thing that many people overlook with client acknowledge is that the client ack is actually acking all previous messages delivered for the parent Session, so if you had more than one consumer in the same session it's message could also be acked by your acknowledge call. Since in your case you have a consumer per session then you sort of fall back to the auto ack case where messages will get redelivered or might go to a DLQ.
In the local transaction case the messages are often times redelivered some number of times to the same client based on either client or Broker configuration before either being re-dispatched to some other consumer or are routed to a DLQ.
To some degree the mode you choose depends on bit on your expectations of the outcome of multiple failures and how you can configure your chosen Broker implementation to handle that.
Assume some spring bean has transactional method and it does two things:
asynchronously send JMS messages
updates DB
The problem is that when the message is consumed, the DB transaction sometimes not have committed, and handler failed to find in DB expected value.
The evident solution - is to send in transaction, but I use async send available on ActiveMQ to increase throughput. If I make send transactional, I forfeit asynchrocity and also risk getting OuOfMemory.
How would you solve this problem?
You need to use an XA-enabled transaction manager, and send the JMS message and do the update to the DB in the same, distributed transaction.
This doesn't change anything to asynchronicity: the send call will return before the receiver receives the message. But when it receives it, the transaction will be committed, and the receiver will find the updates in the database.
I don't see what OutOfMemoryErrors have to do with this.
I have a persistent queue, non-transacted, client-acknowledge, the consumers read with jms.prefetchPolicy.queuePrefetch=1&wireFormat.maxInactivityDuration=50000
and once a consumer processes a message, it ack's the message.
If the consumer reads the message, and before it can send an ack, the process terminates abruptly, what happens in ActiveMq? (What ActiveMq parameters come into play here?)
How is that different than if the the consumer will take 10 minutes to process the message (so the consumer task is alive and working), how does ActiveMq know the message is still being worked on? (Does it monitor the TCP/IP connection, if the connection dies, it assumes the message will not be Ack'ed?)
How do I determine if a message is a "poison pill", i.e. it makes the consumers crash? (the redelivery count seems to be valid if the consumer task does not die; is there an internal counter in the message that says "it was been read n times without being successfully ack'ed?")
As an experiment, I sent 6 messages, one of them being a "poison pill" (kills the consumer before the consumer can send the ack), with 2 simultaneous consumers running (and automatically restarting consumers to bring the count to 2 whenever a consumer dies). Looking at the queue (using jconsole, I enable jmx using broker.setUseJmx(true)), 4 messages were delivered, 2 are in-flight. Why would there be 2 in-flight instead of just one?
I've been reading the ActiveMq and JMS specs for a while without clear/conclusive answers, so any insights on what parameters come into play, and if there are any known bugs, will be greatly useful.
This is purely based on my understanding of JMS - may not be completely correct:
If the consumer reads the message, and before it can send an ack, the process terminates abruptly, what happens in ActiveMq
My understanding is that since this happens in the context of a session with the JMS provider, JMS provider knows if the session is no longer active or has failed and any message not acknowledged as part of the session will be redelivered when the session is re-established.
How do I determine if a message is a "poison pill", i.e. it makes the consumers crash?
Like you have mentioned, the JMS provider keeps track of the # of times the message was redelivered possibly in the header of the message
4 messages were delivered, 2 are in flight
Not sure about this point