I'm investigating ActiveMQ to see if it will work for a project. The current use case I need to demonstrate is that late-joining subscribers will receive topics published prior to the creation of the subscription. It seemed that ActiveMQ Retroactive Consumers would satisfy this need, but I can't get the code to work.
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("[url]");
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createTopic("testAddress?consumer.retroactive=true");
MessageProducer producer = session.createProducer(destination);
TextMessage message = session.createTextMessage("Hello, World!");
producer.send(message);
Thread.sleep(5000);
session.createConsumer(destination).setMessageListener(message2 -> processMessage(message2));
session.close();
connection.close();
connectionFactory.close();
All I'm trying to demonstrate here is that a topic can be published, and then some arbitrary amount of time later (eg. 5 seconds) a consumer can subscribe to the topic and receive the previous message.
As far as I can tell, the issue seems to be that creating the topic creates an address but doesn't create any associated queues. If I send a topic to the address before the queue is made (either in code or manually via the web interface to the broker), the message seems to be ignored and the "un routed message count" is immediately incremented.
The ActiveMQ documentation ( https://activemq.apache.org/retroactive-consumer ) doesn't provide any greater detail on how to set up a retroactive consumer than appending "?consumer.retroactive=true" when making the topic, so I wonder if there are some other configuration aspects I'm missing.
To my knowledge ActiveMQ Artemis doesn't support the retroactive consumer feature that 5.x does. The client side option just tells the broker you want it, but since Artemis doesn't handle that you won't see any difference from sending it. The feature itself in 5.x shouldn't be relied upon as a 100% stand in for a durable consumer, broker restart for instance will cause all those messages (of which the amount stored is finite) to be lost.
If you want to guarantee that you get messages sent when the topic consumer is offline then a durable consumer is the safe way to do this
To accomplish the desired behavior (a subscriber receives topics that were published before the subscription was made) in ActiveMQ Artemis, I used a Last Value Queue with non-destructive reads. This has the limitation that I'm only receiving only the most recent copy of a topic published, but that will work for my situation.
Related
I am using Nifi to get data from IBM MQ. It is working fine. My question is once the message is read from an MQ queue, does it get deleted from the queue? How to just read messages from the queue without deleting them from the queue?
My question is once the message is read from an MQ queue, does it get
deleted from the queue?
Yes, that is the default behavior.
How to just read messages from the queue without deleting them from
the queue?
You use the option: MQGMO_BROWSE_FIRST followed by MQGMO_BROWSE_NEXT on the MQGET API calls.
You can also open the queue for browse only. i.e. MQOO_BROWSE option for MQOPEN API call.
It sounds as if you would like to use a "publish/subscribe" model rather than a "point-to-point" model.
From ActiveMQ:
Topics In JMS a Topic implements publish and subscribe semantics. When
you publish a message it goes to all the subscribers who are
interested - so zero to many subscribers will receive a copy of the
message. Only subscribers who had an active subscription at the time
the broker receives the message will get a copy of the message.
Queues A JMS Queue implements load balancer semantics. A single
message will be received by exactly one consumer. If there are no
consumers available at the time the message is sent it will be kept
until a consumer is available that can process the message. If a
consumer receives a message and does not acknowledge it before closing
then the message will be redelivered to another consumer. A queue can
have many consumers with messages load balanced across the available
consumers.
If you have a queue, when a consumer consumes that message, it is removed from the queue so that the next consumer consumes the next message. With a topic, multiple consumers can be subscribed to that topic and retrieve the same message without being exclusive.
If neither of these work for you, I'm not sure what semantics you're looking for -- a "queue" which doesn't delete the message when it is consumed will never let a consumer access any but the first message.
I don't quite understand what this means:
Each message may have multiple subscribers. There are time
dependencies that exist between publishers and subscribers;
subscribers to a topic may only consume messages
published after a subscription was created.
What does it mean that there exists "time dependencies"? The above says that "subscribers may only consume messages after a subscription was created", I think I'm missing something vital here because I understood that as a subscriber cannot consume a message that is not yet created (but isn't this common sense).
I couldn't find any explanation on Google, so thank you for any help!
Edit: I found an excellent channel on youtube explaining spring boot and other technologies. In particular, this video helped me understand the difference between Queue and Topic (also what durable subscriber is).
What does it mean that there exists "time dependencies"?
To understand this better, compare it with a JMS Queue in which the message broker preserves all the messages (till TimeToLive expiration) published by the producer even if there are NO consumers CREATED.
Now, coming to a JMS topic, the message broker DOES NOT preserve the messages published by the producer if there are NO subscribers CREATED.
During the creation of topic subscription, you can tell the Message Broker that I don't want to loose the messages for this topic published FROM NOW ON by creating the Topic subscription as non-durable (using topicSession.createDurableSubscriber() method). Again, even in the case of non-durable subscription, the broker preserves the messages published after (subscription creation time) till TimeToLive expiration.
Simply that if you publish messages in to the topic before the consumer has subscribed, they will not see the messages.
Similarly, this is related to persistent subscribers. Normal subscribers only see messages while they are connected to the broker. Any messages before or after the subscriber has disconnected, will not be see by the subscriber.
However, with a persistent subscription, a subscriber can disconnect, yet the broker will continue to keep messages destined for that subscriber. When the subscriber reconnects, it will fetch all of those stored messages as well as any new ones.
I am developing a windows service which will read the messages from MQ using IBM.XMS listeners.
I need to read only messages which are older than 120 seconds. I have successfully created a listener which is reading all the messages coming into the queue but I am not able to put a filter on the listener.
Below is my code which is reading all the messages
ISession sess = connection.CreateSession(false, AcknowledgeMode.AutoAcknowledge);
IDestination readqueue = sess.CreateQueue("XYZ");
IMessageConsumer consumer = sess.CreateConsumer(readqueue);
MessageListener list = new MessageListener(OnMessage);
consumer.MessageListener = list;
connection.Start();
This code is reading all messages, which I do not want.
I think a same question was asked on IBM developerWorks forum also.
I am not sure what your business logic is but I would suggest you explore the option of sender of the messages setting message expiry to value you want, 2 minutes in this case. When message expiry is set, any message not consumed before the expiry will not be delivered to application. So you don't need another application to clear the messages older than 120 seconds.
I am still learning about this activemq and jms stuff.
I already tried some example and now I can produce and consuming message from the queue/topic.
Now I have a problem, when my client/consumer lost the connection, the message in queue/topic still send out that message, that message become lost and not kept in the queue/topic. So my question is how I can keep that failed message and how to make the broker resend that message again?
thanks
You are mixing up terminology a bit.
Queues will hold messages until consumed or the broker is restarted, unless the message has been marked as persistent, in which case they will stick around even after a broker restart.
Topics only deliver the current message to any current subscriber. However there are several methods you can use to persist messages published to a topic:
Durable subscribers.
Virtual Destinations .
Virtual Topics tend to be popular for many reasons over durable subscribers, but it really depends on the use-case.
How you create a durable subscriber depends on what you are using to create the subscriber (Spring, POJO, some other API?). All methods will at some point call the Session.createDurableSubscriber method, but I suggest reading up on how they behave before choosing this over Virtual Topic or Composite Queues.
The thing which you are looking for might be Durable subscription
You can find documentation for same at http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
I'm using ActiveMQ JMS implementation with Spring, and just switched from queue model to topic model, because my JMS clients increased from one to many. Topic subscribers should be durable, because clients may become unavailable in some circumstances.
I don't understand when does a persistent message get removed from a topic. Underlying engine is not aware how many subscribers may receive from that topic, so when does it remove the message from its internal database? Is it done in a time-based manner?
If you register a new durable subscriber to a topic, the broker server will keep track of that subscription and keep the messages around until every single subscriber (with a unique subscriber id) has successfully consumed the message.
It's pretty similar to have the message copied to a unique queue per receiver.
Underlying engine is not aware how many subscribers may receive from that topic
-- Yes it is aware, since the subscriptions are durable