Virtual Destinations Active and client id wildcard - jms

I have been reading the documentation for virtual destinations here: http://activemq.apache.org/virtual-destinations.html
But I hit a bit of a snag, when I send to a topic it does not seem to follow the client id name as described on the document
My setup on the active mq is:
<destinationInterceptors>
<virtualDestinationInterceptor>
<virtualDestinations>
<virtualTopic name="Destination.>" prefix="Target.*." selectorAware="false" />
</virtualDestinations>
</virtualDestinationInterceptor>
</destinationInterceptors>
The code above describes that when I send to a Destination.Status topic with a ClientId of CustomerA.
It should send only to Target.CustomerA.Destination.Status if understand correctly, but what's happening is it's sending to Target.CustomerA.Destination.Status and Target.CustomerB.Destination.Status so basically fanning out messages to queues and ignoring the client id.
I did not see any further documentation about how to configure it, i was wondering if anyone else encountered this ?
Am I missing something here ?
Below is my producer if it's helpful.
public static class HelloWorldProducer implements Runnable {
public void run() {
try {
// Create a ConnectionFactory
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61617");
// Create a Connection
Connection connection = connectionFactory.createConnection();
connection.setClientID("CustomerA");
connection.start();
// Create a Session
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
// Create the destination (Topic or Queue)
Destination destination = session.createTopic("Destination.Status");
// Create a MessageProducer from the Session to the Topic or Queue
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
// Create a messages
String text = "Hello world! From: " + Thread.currentThread().getName() + " : " + this.hashCode();
TextMessage message = session.createTextMessage(text);
// Tell the producer to send the message
System.out.println("Sent message: "+ message.hashCode() + " : " + Thread.currentThread().getName());
producer.send(message);
// Clean up
session.close();
connection.close();
}
catch (Exception e) {
System.out.println("Caught: " + e);
e.printStackTrace();
}
}
}
Any inputs will be beneficial.

The sender in this scenario has no real effect on the routing at the broker whether or not you've set a ClientID as it is just sending to a named Topic, in this case "Destination.Status". The configuration on the broker controls the routing and in your case you've configured "Destination.>" so any Queue consumer that comes along and subscribes to a Queue that matches the configuration you've set. So in your case I'd guess you have one consumer subscribing to Queue (Target.CustomerA.Destination.Status) and one to Queue (Target.CustomerB.Destination.Status) which then causes any message sent to the Topic to be fanned out to both.
If you want competing consumers then you'd need to subscribe both to Target.CustomerA.Destination.Status and then the broker would round-robin dispatch the sent message to either of the active subscribers.

Related

ActiveMQ how to resend/retry DLQ messages programmatically

I would like to create a simple code snippet that fetches all messages from the DLQ and re-sends them to the original destination (AKA resend/retry)
It can be done easily by the ActiveMQ UI (but for a single message at a time).
There is no direct JMS API for re-sending a message from a DLQ to its original queue. In fact, the JMS API doesn't even discuss dead-letter queues. It's merely a convention used by most brokers to deal with messages that can't be consumed.
You'd need to create an actual JMS consumer to receive the message from the DLQ and then create a JMS producer to send the message back to its original queue.
It's important that you use Session.TRANSACTED mode to avoid potential message loss or duplication.
If you use Session.AUTO_ACKNOWLEDGE and there is a problem between the time the message is consumed and sent (e.g the application crashes, hardware failure, etc.) then the message could be lost due to the fact that it was already acknowledged before it was sent successfully.
If you use Session.CLIENT_ACKNOWLEDGE and there is a problem between the time the message is sent and acknowledged then the message could ultimately be duplicated due to the fact that it was already sent before it was acknowledged successfully.
Both operations should be part of the JMS transaction so that the work is atomic.
Lastly, I recommend you either invoke commit() on the transacted session for each message sent or after a small batch of messages (e.g. 10). Given that you have no idea how many messages are in the DLQ it would be unwise to process every message in a single transaction. Generally you want the transaction to be as small as possible in order to minimize the window during which an error might occur and the transaction's work will need to be performed again. Also, the larger the transaction is the more heap memory will be required on the broker to keep track of the work in the transaction. Keep in mind that you can invoke commit() on the same session as many times as you want. You don't need to create a new session for each transaction.
Retrying all messages on the DLQ is already implemented in activemq as an mbean.
You can trigger the retry method with jmxterm/jolokia
e.g
Replaying all messages on queue ActiveMQ.DLQ with jolokia
curl -XGET --user admin:admin --header "Origin: http://localhost" http://localhost:8161/api/jolokia/exec/org.apache.activemq:brokerName=localhost,destinationName=ActiveMQ.DLQ,destinationType=Queue,type=Broker/retryMessages
NOTE: You can only use this method on a queue that is marked as a DLQ. It will not work for regular queues.
Also the DLQ queue can have its 'DLQ' flag set to false if the server is restarted. It is automatically set to true when a new message is sent to the DLQ
After Justin's reply I've manually implemented the retry mechanism like so:
public void retryAllDlqMessages() throws JMSException {
logger.warn("retryAllDlqMessages starting");
logger.warn("Creating a connection to {}", activemqUrl);
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("test", "test", activemqUrl);
HashMap<String, MessageProducer> messageProducersMap = new HashMap<>();
MessageConsumer consumer = null;
try (ActiveMQConnection connection = (ActiveMQConnection) connectionFactory.createConnection();
ActiveMQSession session = (ActiveMQSession) connection.createSession(true, Session.SESSION_TRANSACTED)) {
String dlqName = getDlqName();
logger.warn("Creating a session to {}", dlqName);
ActiveMQQueue queue = (ActiveMQQueue) session.createQueue(dlqName);
logger.warn("Starting JMS Connection");
connection.start();
logger.warn("Creating a DLQ consumer");
consumer = session.createConsumer(queue);
logger.warn("Consumer start receiving");
Message message = consumer.receive(CONSUMER_RECEIVE_TIME_IN_MS);
int retriedMessages = 0;
while (message != null) {
try {
retryMessage(messageProducersMap, session, message);
retriedMessages++;
} catch (Exception e) {
logger.error("Error calling retryMessage for message = {}", message);
logger.error("Rolling back the JMS transaction...");
session.rollback();
return;
}
message = consumer.receive(CONSUMER_RECEIVE_TIME_IN_MS);
}
logger.warn("Consumer finished retrying {} messages", retriedMessages);
logger.warn("Commiting JMS Transactions of retry");
session.commit();
} finally {
if (!messageProducersMap.isEmpty()) {
logger.warn("Closing {} messageProducers in messageProducersMap", messageProducersMap.size());
for (MessageProducer producer : messageProducersMap.values()) {
producer.close();
}
}
if (consumer != null) {
logger.warn("Closing DLQ Consumer");
consumer.close();
}
}
}
private void retryMessage(HashMap<String, MessageProducer> messageProducersMap, ActiveMQSession session, Message message) {
ActiveMQObjectMessage qm = (ActiveMQObjectMessage) message;
String originalDestinationName = qm.getOriginalDestination().getQualifiedName();
logger.warn("Retry message with JmsID={} to original destination {}", qm.getJMSMessageID(), originalDestinationName);
try {
if (!messageProducersMap.containsKey(originalDestinationName)) {
logger.warn("Creating a new producer for original destination: {}", originalDestinationName);
messageProducersMap.put(originalDestinationName, session.createProducer(qm.getOriginalDestination()));
}
logger.info("Producing message to original destination");
messageProducersMap.get(originalDestinationName).send(qm);
logger.info("Message sent");
} catch (Exception e) {
logger.error("Message retry failed with exception", e);
}
}

Jms runtime selector not working with artemis (for Topic)

I am using Stringboot, springboot starter artemis and camel
here is my dependencies for them:
compile('org.springframework.boot:spring-boot-starter-artemis')
compile 'org.springframework:spring-jms'
compile 'org.apache.activemq:artemis-jms-client:1.5.6'
compile 'org.apache.activemq:artemis-jms-server:1.5.6'
// https://mvnrepository.com/artifact/org.apache.camel/camel-jms
compile group: 'org.apache.camel', name: 'camel-jms', version: '2.20.2'
Here is my code(I am not using jms template here due to some reason)
Connection connection = connectionFactory.createConnection();
try {
connection.setClientID(clientId);
connection.start();
//Create a JMS session
Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
Topic topic = ActiveMQJMSClient.createTopic(processedHeaders.get(jmsdestination));
******JmsMessageUUID will dynamic, i have added JmsMessageUUID value = ID:142536687 as a example*********
String selector = "JmsMessageUUID = 'ID:142536687'";
//Create the subscription and the subscriber.
MessageConsumer subscriber = session.createConsumer(topic,selector);
//Consume the message from the durable subscription
TextMessage messageReceived = (TextMessage) subscriber.receive();
System.out.println("Received message: " + messageReceived.getText());
// Acknowledge message
messageReceived.acknowledge();
} finally {
if (connection != null) {
// Close our JMS resources!
connection.close();
} }
}
i am trying to select message with JmsMessageUUID, it is basically a random string which is set during send message, Jms selector is unable to find out message, but when i am not using selector than successfully consumed message also I have checked the jms properties in received message, and the JmsMessageUUID is available in the properties with same string value.
I am unable to figure out why selector is not working ? Please help me here ..
I have also found JMSMessageID filter worked with Queue but not with Topoc:(

JMS and IBM WebSphere not generating COD

I am creating an application that consumes messages from a MQ using JMS. My MQ manager is IBM WebSphere MQ and I am using the IBM jms implementation to consume the messages.
The messasges are coming and going fine. I receive the messages from the other part and I can send messages to them. The problem is that they are not receiving the COD after I consume the message from the queue. They receive the COA, but no COD.
Here is my receive message code:
public byte[] readMsgFromClient() throws JMSException {
byte[] message = null;
QueueReceiver reader = null;
try {
connection = getQueueConnection();
connection.start();
session = connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue(config.getQueueRsp());
((MQQueue) queue).setTargetClient(JMSC.MQJMS_CLIENT_NONJMS_MQ);
reader = session.createReceiver(queue);
JMSBytesMessage byteMessage = (JMSBytesMessage) reader.receive(3000);
if (byteMessage != null) {
message = new byte[(int) byteMessage.getBodyLength()];
byteMessage.readBytes(message);
}
} finally {
if (reader != null) {
reader.close();
}
if (session != null) {
session.close();
}
if (connection != null) {
connection.close();
}
}
return message;
}
Do I have to manually send the COD? DO I have to configure my WebSphere to automatically send the COD? Do I have to notify the WebSphere that my application has consumed the message?
The COD messages are probably ending up the the dead letter queue (DLQ) with an RC (Reason Code) of 2035 (not authorized).
Here is another of those things you learn the hard way:
COA messages are generated under the queue manager's UserId
COD messages are generated under the UserId of the sender's message.
If the sending application's UserId is appl001 then the COD will be generated using the UserId of appl001. If that UserId does not have permission to write to the particular queue then the message will end up in the DLQ.
Generally, the permission issue happens when the sender application is connected to 1 queue manager and the receiver application is connected to another queue manager. i.e. messages are hoping between queue managers.
Hence, the sender's UserId does not have permission to put a message on the remote queue manager.
As stated by #Roger, the permissions to put the COD are based on the UserId in the MQMD of the message that is sent.
If you do not want to add the remote user to your local system you can use the itsoME exit provided in the IBM Redbook "Secure Messaging Scenarios with WebSphere MQ". The latest version is found under the "Additional Material" link.
With this exit you need to have a MCAUSER set on your RCVR or RQSTR channel and configure that channel with the following attributes:
MSGEXIT('itsoME(MsgExit)')
MSGDATA('MCA/')
The result is that UserIdentifier field of the MQMD will be changed to the value of the MCAUSER that is configured on the channel. You would then give that MCAUSER +put and +passid to the XMITQ that returns to the remote queue manager.
The exit can be used for other things such as removing the reporting options if you do not want to allow COA/COD.

How to know which queue is allocated to which consumer-RocketMQ?

Consumer queues are allocated in client side, broker knows nothing about this.
So how can we monitor which queue is allocated to which consumer client?
Though there is no exiting command, for each message queue per consumer group, You can find out the client using provided admin infrastructure. Here is the snippet achieving this:
private Map<MessageQueue, String> getClientConnection(DefaultMQAdminExt defaultMQAdminExt, String groupName){
Map<MessageQueue, String> results = new HashMap<MessageQueue, String>();
try{
ConsumerConnection consumerConnection = defaultMQAdminExt.examineConsumerConnectionInfo(groupName);
for (Connection connection : consumerConnection.getConnectionSet()){
String clinetId = connection.getClientId();
ConsumerRunningInfo consumerRunningInfo = defaultMQAdminExt.getConsumerRunningInfo(groupName, clinetId, false);
for(MessageQueue messageQueue : consumerRunningInfo.getMqTable().keySet()){
results.put(messageQueue, clinetId + " " + connection.getClientAddr());
}
}
}catch (Exception e){
}
return results;
}
In case you have not used the RocketMQ-Console project, please try and run it: https://github.com/rocketmq/rocketmq-console-ng
In the Consumer tab, Click "consumer detail" button, you will see message queue allocation result visually as below:
Message queues allocation result

MQ JMS setClientReconnectOptions doesn't work as expected?

I have a simple code to put 2 messages into a queue.
1) I set the connectionNameList with two servers.
2) Those two servers are independent, but have the same Queue Manager and Queue defined with same name, such as "QMgr" and "TEST.IN"
3) I set the setClientReconnectOptions(WMQConstants.WMQ_CLIENT_RECONNECT);
I hope when the first server is down, it should send the messages to 2nd one.
The test I did:
a) I send first message, sender.send(message); It worked.
b) sleep 30 seconds.
During this time, I shutdown the first server
c) then sleep done, try to send 2nd message, but it failed to send immediately
Further more, I tried more, I did try{} catch{} for 2nd message, and in the catch{}, I try to sender.send(message), it still fails.
Any idea why it is different than what I expected. I will really appreciate your reply.
public static void main(String[] args) throws Exception
{
MQQueueConnectionFactory cf = new MQQueueConnectionFactory();
cf.setConnectionNameList("10.230.34.191(1418),10.230.34.169(1418)");
cf.setQueueManager("QMgr");
cf.setTransportType(WMQConstants.WMQ_CM_CLIENT);
cf.setClientReconnectOptions(WMQConstants.WMQ_CLIENT_RECONNECT);
cf.setClientReconnectTimeout(600);
System.out.println("connect list " + cf.getConnectionNameList());
MQQueueConnection connection = (MQQueueConnection) cf
.createQueueConnection("mqm", "passwd");
MQQueueSession session = (MQQueueSession) connection.createQueueSession(false,
Session.AUTO_ACKNOWLEDGE);
MQQueue queue = (MQQueue) session.createQueue("queue:///TEST.IN");
MQQueueSender sender = (MQQueueSender) session.createSender(queue);
long uniqueNumber = System.currentTimeMillis() % 1000;
JMSTextMessage message = (JMSTextMessage) session.createTextMessage("SimplePTP "
+ uniqueNumber);
// Start the connection
connection.start();
sender.send(message);
System.out.println("Sent message:\\n" + message);
System.out.println("sleep 30 seconds");
Thread.sleep(30000);
uniqueNumber = System.currentTimeMillis() % 1000;
message = (JMSTextMessage) session.createTextMessage("SimplePTP " + uniqueNumber);
sender.send(message);
sender.close();
session.close();
connection.close();
System.out.println("\\nSUCCESS\\n");
}
Well this is the simplest test case and should have worked. How did you bring down the first queue manager? Did you down it with a -r option. Remember, without the -r option the clients will not reconnect when queue manager is ended with endmqm command.
endmqm -r <qm name>
Assuming you used -r option and it still did not work, then my suggestion would be to try the following:
Set an exception listener to know what is going on with reconnection. Exception listener would be invoked when the connection is broken and reconnection attempt starts till either reconnection is successful or fails. Exception listener sample code would be something like this:
conn.setExceptionListener(new ExceptionListener() {
public void onException(JMSException e) {
System.out.print(e);
}
});

Resources