We are using Websphere MQ8 and thinking about using durable Topics to implement a Publish and Subscribe pattern.
We are using Queues already and Hermes JMS-Browser to correct erroneous Queue-Messages.
I know that you can use Hermes to subscribe to topics but i think you cannot access and modify messages that are already on hold for a specific client.
So I would like to know - how do you handle problems when there are durable messages for a specific client application that the client can't consume, e.g. because the message has wrong format?
Do you have to delete alle undelivered messages for this client? Or is there some tooling that can do this?
How do you handle problems when there are durable messages for a
specific client application that the client can't consume, e.g.
because the message has wrong format ?
Consuming Application (Client) should have proper exception handling
and it should throw out all bad/poison messages.
Do you have to delete all undelivered messages for this client? Or is there some tooling that can do this?
The messages will be deleted automatically by the MQ server once the
TTL (Time To Live) expires. We can't manually delete those messages as
they belong/meant for the original client (which is a durable
subscriber).
The below link explains well on the concept of JMS durable subscriptions:
https://docs.oracle.com/cd/E19798-01/821-1841/bncgd/index.html
Also I have provided the below sample code for the tool, which logs (doesn't delete) the messages from a durable subscription topic:
InitialContext ctx = new InitialContext();
Topic topic = (Topic)ctx.lookup("myJMSTopic1");
TopicConnectionFactory connFactory = (TopicConnectionFactory) ctx.lookup("topicConnFactory");
TopicConnection topicConn = connFactory.createTopicConnection();
topicConn.setClientID("myToolId1"); //Use a different client id than original subscriber
TopicSession topicSession = topicConn.createTopicSession(false, Session.AUTO_ACKNOWLEDGE);
TopicSubscriber topicSubscriber = topicSession.createDurableSubscriber(topic, "myClientExistingDuraSubName"); // this is where you have to provide the original client's durable subscription name
while(true) {
Message message = topicSubscriber.receive();
TextMessage txtMsg = (TextMessage)message;
logger.info(txtMsg.getText());
}
Related
Is it possible to configure the topic to store a copy of just the last message and send this to new connections without knowing client identifiers or other info?
Update:
From the info provided by Shashi I found this two pages where they describe a use case similar to mine (applied over stock prices) by using retroactive consumer and a subscription recovery policy. How ever I'm not getting the desired behaviour. What I currently do is:
Include in the activemq the folowing lines in the policyEntry for topic=">"
<subscriptionRecoveryPolicy>
<fixedCountSubscriptionRecoveryPolicy maximumSize="1"/>
</subscriptionRecoveryPolicy>
Add to the URL used to connect to the brocker (using activemq-cpp) consumer.retroactive=true.
Set the consumer has durable. (But I strongly think this is not want since I only need the last one, but without it I didn't get any message when starting the consumer for the second time)
Start up the broker.
Start the consumer.
Send a message to the topic using the activemq web admin console. (I receive it in the consumer, as expected)
Stop consumer.
Send another message to the topic.
Start consumer. I receive the message, also as expected.
However, if the consumer receives a message, then it goes offline (stop process) and then I restart it, it doesn't get the last message back.
The goal is to whenever the consumer starts get the last message, no mater what (obviously, except when there weren't messages sent to the topic).
Any ideas on what I'm missing?
Background:
I have a device which publishes his data to a topic when ever its data changes. A variable number of consumer may be connected to this topic, from 0 to less than 10. There is only one publisher in the topic and always publish all of his data as a single message (little data, just a couple of fields of a sensor reading). The publication rate of this information is variable, not necessarily time based, when something changes a new updated message is sent to the broker.
The problem is that when a new consumer connects to the topic it has no data of the device readings until a new message is send to the topic by the device. This could be solve by creating an additional queue so new connections can subscribe to the topic and then request the device for the current reading through the queue (the device would consume the queue message which would be a request for data, and then response in the same queue).
But Since the messages send to the topic are always information complete I was wondering if is it possible to configure the topic to store a copy of just the last message and send this to new connections without know client identifiers or other info?
Current broker in use is ActiveMQ.
What you want is to have retroactive consumers and to set the lastImageSubscriptionRecoveryPolicy subscription recovery policy on the topic. Shashi is correct in saying that the following syntax for setting a consumer to be retroactive works only with Openwire
topic = new ActiveMQTopic("TEST.Topic?consumer.retroactive=true");
In your case, what you can do is to configure all consumers to be retroactive in broker config with alwaysRetroactive="true". I tested that this works even for the AMQP protocol (library qpid-jms-client) and I suspect it will work for all protocols.
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic="FOO.>" alwaysRetroactive="true">
<subscriptionRecoveryPolicy>
<lastImageSubscriptionRecoveryPolicy />
</subscriptionRecoveryPolicy>
</policyEntry>
The configuration example is taken from https://github.com/apache/activemq/blob/master/activemq-unit-tests/src/test/resources/org/apache/activemq/test/retroactive/activemq-message-query.xml
Messaging providers (WebSphere MQ for example) have a feature called Retained Publication. With this feature the last published message on a topic is retained by the messaging provider and delivered to a new consumer who comes in after a message has been published on a given topic.
Retained Publication may be supported by Active MQ in it's native interface. This link talks about consumer.retroactive which is available for OpenWire only.
A publisher will tell the messaging provider to retain a publication by setting a property on the message before publishing. Below is how it is done using WebSphere MQ.
// set as a retained publication
msg.setIntProperty(JmsConstants.JMS_IBM_RETAIN, JmsConstants.RETAIN_PUBLICATION)
I am new to world of JMS and stuck in an issue. Here it is;
Suppose I have one topic and all the clients are publishing and subscribing to same topic. What i want to achieve is this if ClientA publishes a message to the topic then all the other clients should receive that message but ClientA (Sender should not receive his own message returned back).
You need to configure the subscriber to ignore messages published from the current connection.
This is done by setting the noLocal option when creating a subscriber.
See How can I prevent receiving JMS messages that I have produced?.
Spring Message Listener Container doc says:
http://static.springsource.org/spring/docs/2.5.x/api/org/springframework/jms/listener/AbstractMessageListenerContainer.html#setDurableSubscriptionName%28java.lang.String%29
The durable subscription name needs to be unique within this client's
JMS client id. Default is the class name of the specified message
listener. Note: Only 1 concurrent consumer (which is the default of
this message listener container) is allowed for each durable
subscription.
I thought, we can handle concurrent messages at the same time. Am I missing something?
Yes, the documentation is correct. At any point of time there can be only one consumer receiving messages for a durable subscription. All durable subscriptions are identified by a unique id. If one consumer is already receiving messages using an id, another attempt create a consumer with the same id for that durable subscription will fail.
I've had this problem before and our log was full of warnings complain about client id is already in use.
Virtual Topic is an option for ActiveMQ: the idea is similar to what #brainOverflow described, it's a combination of topic and queues. The producer sends the message to a topic which is subscribed by queues and each queue receives a copy of the message.
http://activemq.apache.org/virtual-destinations.html
I am using ActiveMQ as a JMS implementation server in my application. Scenario is like, there is a topic over which I have many durable subscribers which consumes the published message and a message listener which save the data from message object to central DB server. There is a producer thread which keeps on publishing persistent message over the same topic. I am using KahaDB for persistent Message Store. As soon as a message is published, kahaDB creates a data log file in message store to persist message until all durable subscriber consume it. I want to know if at any point, I shutdown the JMS server and delete all the data log files, what would be the impact. Will it be just that few durable subscriers will not receive a message which was there in data log files for them to be consumed or is there a possibility that few message didn't got saved in central database which is done by message listener over this topic.
Any hint or help is greatly appreciated......
Thanks in advance.
If you stop and start your broker, regardless of whether you delete your data files or not, topic consumer that have not already received a published message will no longer receive it. The reason behind this is that messages sent to a topic will not be written out to the persistent message store.
Durability and persistence are not the same things. A durable subscription tells the broker to preserve the subscription state in case the subscriber disconnects - any messages sent while the consumer is disconnected will be kept around. A non-durable subscription on the other hand is finite; if a subscriber disconnects, it missed any messages sent in the interim. All messages are stored in memory, and will not survive a broker restart.
Message persistence on the other hand stores messages for eventual delivery. This guards against catastrophic failure, or for later delivery to consumers who might not yet be active.
If you want to broadcast messages using pub-sub, and have the subscriptions appear durable and survive broker restarts you should use virtual destinations instead of durable subscriptions.
No messages, persistent or non-persistent, will survive switching the broker off and deleting the data directory.
I am sending messages to a remote queue, to which I have no control.
I send a xml file as message but when the application reads the message it gets a message header like
<mcd><Msd>jms_text</Msd></mcd> \0\0\0l<jms><Dst>queue:///TEST</Dst><Tms>1281475843707</Tms><Cid></Cid><Dlv>1</Dlv></jms>
i don't want this message header to be present and my code for sending this message is as follows:
Properties props = new Properties();
props.setProperty("java.naming.factory.initial",this.initialFactory);
props.setProperty("java.naming.provider.url", url);
Context context = new InitialContext(props);
QueueConnectionFactory qcf = (QueueConnectionFactory) context.lookup(this.context);
qConn = qcf.createQueueConnection();
queue = (Queue)context.lookup(name);
qSession = qConn.createQueueSession(false, QueueSession.AUTO_ACKNOWLEDGE);
qConn.start();
QueueSender send = qSession.createSender(queue);
String text = "My xml file";
TextMessage tm = qSession.createTextMessage(text);
send.send(tm);
send.close();
How do I avoid this ?
It appears that you are sending a jms message to a non jms destination. How is the message being consumed on the destination? Is it expecting a native MQ message? The receiver is not understanding the MQRFH2 header that stores the JMS header properties.
You should either configure the destination to understand jms or your can do something like the following to tell the mq jms that your receiver is a non-jms client.
((com.ibm.mq.jms.MQQueue) queue).setTargetClient(JMSC.MQJMS_CLIENT_NONJMS_MQ);
Have a look at the properties for JMS objects as listed in the docs. On the administered object there is a property called TARGCLIENT which should be set to 'MQ'. Although you may have no control over the administered object, it is the responsibility of the person who administers the managed objects to set this property correctly. If the destination does not understand the RFH2 headers (which WMQ v6 uses to hold JMS properties) then any WMQ JMS applications which send messages to that destination must have that property set.
Incidentally, the fact that you are having this problem tends to indicate that the application consuming messages is still at v6. Please be aware that v6.0 of WMQ is end-of-life as of Sept 2011. If you switch to v7 now at both the QMgr and the client side, you can manage this with simple settings on the queue itself. The legacy app will understand the messages regardless of whether they have an RFH2 attached and the client app will see the responses as JMS messages regardless of whether the legacy app adds RFH2 headers. Move to v7 now, save your self a whole lot of trouble developing this app and also avoid having to migrate to v7 next year.
WMQ v7 client downloads are available here
Update: End-of-life for WMQ V6 was pushed back to September 2012.