I don't quite understand what this means:
Each message may have multiple subscribers. There are time
dependencies that exist between publishers and subscribers;
subscribers to a topic may only consume messages
published after a subscription was created.
What does it mean that there exists "time dependencies"? The above says that "subscribers may only consume messages after a subscription was created", I think I'm missing something vital here because I understood that as a subscriber cannot consume a message that is not yet created (but isn't this common sense).
I couldn't find any explanation on Google, so thank you for any help!
Edit: I found an excellent channel on youtube explaining spring boot and other technologies. In particular, this video helped me understand the difference between Queue and Topic (also what durable subscriber is).
What does it mean that there exists "time dependencies"?
To understand this better, compare it with a JMS Queue in which the message broker preserves all the messages (till TimeToLive expiration) published by the producer even if there are NO consumers CREATED.
Now, coming to a JMS topic, the message broker DOES NOT preserve the messages published by the producer if there are NO subscribers CREATED.
During the creation of topic subscription, you can tell the Message Broker that I don't want to loose the messages for this topic published FROM NOW ON by creating the Topic subscription as non-durable (using topicSession.createDurableSubscriber() method). Again, even in the case of non-durable subscription, the broker preserves the messages published after (subscription creation time) till TimeToLive expiration.
Simply that if you publish messages in to the topic before the consumer has subscribed, they will not see the messages.
Similarly, this is related to persistent subscribers. Normal subscribers only see messages while they are connected to the broker. Any messages before or after the subscriber has disconnected, will not be see by the subscriber.
However, with a persistent subscription, a subscriber can disconnect, yet the broker will continue to keep messages destined for that subscriber. When the subscriber reconnects, it will fetch all of those stored messages as well as any new ones.
Related
I am using Nifi to get data from IBM MQ. It is working fine. My question is once the message is read from an MQ queue, does it get deleted from the queue? How to just read messages from the queue without deleting them from the queue?
My question is once the message is read from an MQ queue, does it get
deleted from the queue?
Yes, that is the default behavior.
How to just read messages from the queue without deleting them from
the queue?
You use the option: MQGMO_BROWSE_FIRST followed by MQGMO_BROWSE_NEXT on the MQGET API calls.
You can also open the queue for browse only. i.e. MQOO_BROWSE option for MQOPEN API call.
It sounds as if you would like to use a "publish/subscribe" model rather than a "point-to-point" model.
From ActiveMQ:
Topics In JMS a Topic implements publish and subscribe semantics. When
you publish a message it goes to all the subscribers who are
interested - so zero to many subscribers will receive a copy of the
message. Only subscribers who had an active subscription at the time
the broker receives the message will get a copy of the message.
Queues A JMS Queue implements load balancer semantics. A single
message will be received by exactly one consumer. If there are no
consumers available at the time the message is sent it will be kept
until a consumer is available that can process the message. If a
consumer receives a message and does not acknowledge it before closing
then the message will be redelivered to another consumer. A queue can
have many consumers with messages load balanced across the available
consumers.
If you have a queue, when a consumer consumes that message, it is removed from the queue so that the next consumer consumes the next message. With a topic, multiple consumers can be subscribed to that topic and retrieve the same message without being exclusive.
If neither of these work for you, I'm not sure what semantics you're looking for -- a "queue" which doesn't delete the message when it is consumed will never let a consumer access any but the first message.
If a publisher publish some messages to a WMQ topic, but the subsciber didn't take it, then where the messages are saved? is there any way to know the message count?
As MQ is JMS compliant, the answer is mostly a JMS answer.
If the subscription is not durable and no subscription is registered, the messages for that subscriber are discarded.
If the subscription is durable, MQ creates a queue (or uses a predefined one if specified by the subscriber) to deliver the messages. The messages will collect there if the subscriber is not consuming them.
The 3rd case as Dave points out int he comments is that the non-durable subscriber is holding the subscription open but not consuming the messages. Since a queue is created to receive these that queue depth can be queried to determine if there's a back-up.
Based on there being a queue for every subscription (durable or otherwise) just look in the durable subscriber's queue to determine the number of messages outstanding.
Please also see Publish/subscribe lifecycles in the MQ Knowledge Center for more description of the behavior and specification of durable subscriber queues.
Of course, if that queue fills up the behavior changes. Depending on the settings either the publishers block or the publications continue but the messages are routed to an exception queue (if specified), the DLQ, or discarded.
Thanks Dave Ware for the comments about non-durable subscriptions.
I'm wondering from the question if you're asking if MQ keeps a store of all the messages published to a topic, independent of any registered subscriptions?
If that's the question, then no, it doesn't. When messages are published they are matched to each existing subscription and a copy is sent to each of their associated queues as T.Rob describes.
So the only queue depths to worry about are those of the subscriptions.
(There is a caveat in that MQ supports "retained publications", - it means MQ keeps just the most recent publication on that topic string for late subscriptions if you choose to do that).
I try to explain all this here (slides/video), which may help... http://www.slideshare.net/DavidWare1/ame-2271-mq-publish-subscribe-pdf
I am still learning about this activemq and jms stuff.
I already tried some example and now I can produce and consuming message from the queue/topic.
Now I have a problem, when my client/consumer lost the connection, the message in queue/topic still send out that message, that message become lost and not kept in the queue/topic. So my question is how I can keep that failed message and how to make the broker resend that message again?
thanks
You are mixing up terminology a bit.
Queues will hold messages until consumed or the broker is restarted, unless the message has been marked as persistent, in which case they will stick around even after a broker restart.
Topics only deliver the current message to any current subscriber. However there are several methods you can use to persist messages published to a topic:
Durable subscribers.
Virtual Destinations .
Virtual Topics tend to be popular for many reasons over durable subscribers, but it really depends on the use-case.
How you create a durable subscriber depends on what you are using to create the subscriber (Spring, POJO, some other API?). All methods will at some point call the Session.createDurableSubscriber method, but I suggest reading up on how they behave before choosing this over Virtual Topic or Composite Queues.
The thing which you are looking for might be Durable subscription
You can find documentation for same at http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
Let's suppose I have several subscribers consuming from a topic. After a message has been delivered to all the subscribers I'd like to trigger a job that would use this message in input.
So the easy way to do that would be to move messages that have been succesfully delivered to all the sucscribers to a queue from which my job would consume messages.
Is it part of JMS?
Is there any message broker able to do that directly?
If not is there a simple solution to solve this problem?
You should be able to do this using activemq's advisories.
See here for more about advisory messages: http://activemq.apache.org/advisory-message.html
So what you want to do, for the topic in question, is track:
the number of consumers
when a message is dispatched to them
when the message has been ack'd by each of the consumers
to get the number of consumers, listen to the "ActiveMQ.Advisory.Consumer.Topic." advisory topic
to get when a message is dispatched, listen to the "ActiveMQ.Advisory.MessageDelivered.Topic."
to get when a message has been ack'd, listen to "ActiveMQ.Advisory.MessageConsumed.Topic."
you could easily use Apache Camel to help out with this (listening to the topics) and aggregating whether or not all consumers have processed (ack'd) the message.. then that could kick off your further processing..
You could just create another durable subscription to route the message from the topic to queue directly. From that queue your job can consume messages. This is much easier than creating a trigger to route the messages to a queue.
So the easy way to do that would be to move messages that have been
succesfully delivered to all the sucscribers to a queue from which my
job would consume messages. Is it part of JMS?
No, this is not part of JMS specification.
I'm using ActiveMQ JMS implementation with Spring, and just switched from queue model to topic model, because my JMS clients increased from one to many. Topic subscribers should be durable, because clients may become unavailable in some circumstances.
I don't understand when does a persistent message get removed from a topic. Underlying engine is not aware how many subscribers may receive from that topic, so when does it remove the message from its internal database? Is it done in a time-based manner?
If you register a new durable subscriber to a topic, the broker server will keep track of that subscription and keep the messages around until every single subscriber (with a unique subscriber id) has successfully consumed the message.
It's pretty similar to have the message copied to a unique queue per receiver.
Underlying engine is not aware how many subscribers may receive from that topic
-- Yes it is aware, since the subscriptions are durable