I am creating an application that consumes messages from a MQ using JMS. My MQ manager is IBM WebSphere MQ and I am using the IBM jms implementation to consume the messages.
The messasges are coming and going fine. I receive the messages from the other part and I can send messages to them. The problem is that they are not receiving the COD after I consume the message from the queue. They receive the COA, but no COD.
Here is my receive message code:
public byte[] readMsgFromClient() throws JMSException {
byte[] message = null;
QueueReceiver reader = null;
try {
connection = getQueueConnection();
connection.start();
session = connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue(config.getQueueRsp());
((MQQueue) queue).setTargetClient(JMSC.MQJMS_CLIENT_NONJMS_MQ);
reader = session.createReceiver(queue);
JMSBytesMessage byteMessage = (JMSBytesMessage) reader.receive(3000);
if (byteMessage != null) {
message = new byte[(int) byteMessage.getBodyLength()];
byteMessage.readBytes(message);
}
} finally {
if (reader != null) {
reader.close();
}
if (session != null) {
session.close();
}
if (connection != null) {
connection.close();
}
}
return message;
}
Do I have to manually send the COD? DO I have to configure my WebSphere to automatically send the COD? Do I have to notify the WebSphere that my application has consumed the message?
The COD messages are probably ending up the the dead letter queue (DLQ) with an RC (Reason Code) of 2035 (not authorized).
Here is another of those things you learn the hard way:
COA messages are generated under the queue manager's UserId
COD messages are generated under the UserId of the sender's message.
If the sending application's UserId is appl001 then the COD will be generated using the UserId of appl001. If that UserId does not have permission to write to the particular queue then the message will end up in the DLQ.
Generally, the permission issue happens when the sender application is connected to 1 queue manager and the receiver application is connected to another queue manager. i.e. messages are hoping between queue managers.
Hence, the sender's UserId does not have permission to put a message on the remote queue manager.
As stated by #Roger, the permissions to put the COD are based on the UserId in the MQMD of the message that is sent.
If you do not want to add the remote user to your local system you can use the itsoME exit provided in the IBM Redbook "Secure Messaging Scenarios with WebSphere MQ". The latest version is found under the "Additional Material" link.
With this exit you need to have a MCAUSER set on your RCVR or RQSTR channel and configure that channel with the following attributes:
MSGEXIT('itsoME(MsgExit)')
MSGDATA('MCA/')
The result is that UserIdentifier field of the MQMD will be changed to the value of the MCAUSER that is configured on the channel. You would then give that MCAUSER +put and +passid to the XMITQ that returns to the remote queue manager.
The exit can be used for other things such as removing the reporting options if you do not want to allow COA/COD.
Related
Greetings of the day.
Please help on the below requirement:
Requirement:
We want to delete message from MQ only after it is processed successfully.
Use event based message detection technique and avoid loop
So, to achieve above:
I have created message listener and consumer class below:
{
sessionIn = connectionIn.CreateSession(false, AcknowledgeMode.ClientAcknowledge);
// Create message listener and assign it to consumer
messageListener = new MessageListener(OnMessageCallback);
consumerAsync.MessageListener = messageListener;
Console.WriteLine("Message Listener set. Starting the connection now.");
// Start the connection to receive messages.
connectionWMQ.Start();
}
Reading the message from the call back event and push the message into other system:
OnMessageCallback(Message) {
if (xmsMessage is IBytesMessage)
{
IBytesMessage bytesMessage = (IBytesMessage)xmsMessage;
byte[] arrayMessage = new byte[bytesMessage.BodyLength];
bytesMessage.ReadBytes(arrayMessage);
string message = System.Text.Encoding.Default.GetString(arrayMessage);
}
}
Once the message processed, external system will fire the below over ride method:
Response method override:
protected override Task OnResponse(ReponseMessage message)
{
//Read the message and get the message id and correlation id.
//Delete the message from the queue.
//I am trying to do like this, but Its not working:
messageConsumerDelete = sessionDelete.CreateConsumer(destinationDelete, query);
if (messageConsumerDelete != null)
{
IMessage m = messageConsumerDelete.Receive(1000);
LogWrite("Receive Message=" + m);
m.Acknowledge();
}
}
Please suggest a best solution for this requirement.
I am trying to find a solution for this since weeks, but no breakthrough.
Thanks,
Balaji
I have been reading the documentation for virtual destinations here: http://activemq.apache.org/virtual-destinations.html
But I hit a bit of a snag, when I send to a topic it does not seem to follow the client id name as described on the document
My setup on the active mq is:
<destinationInterceptors>
<virtualDestinationInterceptor>
<virtualDestinations>
<virtualTopic name="Destination.>" prefix="Target.*." selectorAware="false" />
</virtualDestinations>
</virtualDestinationInterceptor>
</destinationInterceptors>
The code above describes that when I send to a Destination.Status topic with a ClientId of CustomerA.
It should send only to Target.CustomerA.Destination.Status if understand correctly, but what's happening is it's sending to Target.CustomerA.Destination.Status and Target.CustomerB.Destination.Status so basically fanning out messages to queues and ignoring the client id.
I did not see any further documentation about how to configure it, i was wondering if anyone else encountered this ?
Am I missing something here ?
Below is my producer if it's helpful.
public static class HelloWorldProducer implements Runnable {
public void run() {
try {
// Create a ConnectionFactory
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61617");
// Create a Connection
Connection connection = connectionFactory.createConnection();
connection.setClientID("CustomerA");
connection.start();
// Create a Session
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
// Create the destination (Topic or Queue)
Destination destination = session.createTopic("Destination.Status");
// Create a MessageProducer from the Session to the Topic or Queue
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
// Create a messages
String text = "Hello world! From: " + Thread.currentThread().getName() + " : " + this.hashCode();
TextMessage message = session.createTextMessage(text);
// Tell the producer to send the message
System.out.println("Sent message: "+ message.hashCode() + " : " + Thread.currentThread().getName());
producer.send(message);
// Clean up
session.close();
connection.close();
}
catch (Exception e) {
System.out.println("Caught: " + e);
e.printStackTrace();
}
}
}
Any inputs will be beneficial.
The sender in this scenario has no real effect on the routing at the broker whether or not you've set a ClientID as it is just sending to a named Topic, in this case "Destination.Status". The configuration on the broker controls the routing and in your case you've configured "Destination.>" so any Queue consumer that comes along and subscribes to a Queue that matches the configuration you've set. So in your case I'd guess you have one consumer subscribing to Queue (Target.CustomerA.Destination.Status) and one to Queue (Target.CustomerB.Destination.Status) which then causes any message sent to the Topic to be fanned out to both.
If you want competing consumers then you'd need to subscribe both to Target.CustomerA.Destination.Status and then the broker would round-robin dispatch the sent message to either of the active subscribers.
I would like to create a simple code snippet that fetches all messages from the DLQ and re-sends them to the original destination (AKA resend/retry)
It can be done easily by the ActiveMQ UI (but for a single message at a time).
There is no direct JMS API for re-sending a message from a DLQ to its original queue. In fact, the JMS API doesn't even discuss dead-letter queues. It's merely a convention used by most brokers to deal with messages that can't be consumed.
You'd need to create an actual JMS consumer to receive the message from the DLQ and then create a JMS producer to send the message back to its original queue.
It's important that you use Session.TRANSACTED mode to avoid potential message loss or duplication.
If you use Session.AUTO_ACKNOWLEDGE and there is a problem between the time the message is consumed and sent (e.g the application crashes, hardware failure, etc.) then the message could be lost due to the fact that it was already acknowledged before it was sent successfully.
If you use Session.CLIENT_ACKNOWLEDGE and there is a problem between the time the message is sent and acknowledged then the message could ultimately be duplicated due to the fact that it was already sent before it was acknowledged successfully.
Both operations should be part of the JMS transaction so that the work is atomic.
Lastly, I recommend you either invoke commit() on the transacted session for each message sent or after a small batch of messages (e.g. 10). Given that you have no idea how many messages are in the DLQ it would be unwise to process every message in a single transaction. Generally you want the transaction to be as small as possible in order to minimize the window during which an error might occur and the transaction's work will need to be performed again. Also, the larger the transaction is the more heap memory will be required on the broker to keep track of the work in the transaction. Keep in mind that you can invoke commit() on the same session as many times as you want. You don't need to create a new session for each transaction.
Retrying all messages on the DLQ is already implemented in activemq as an mbean.
You can trigger the retry method with jmxterm/jolokia
e.g
Replaying all messages on queue ActiveMQ.DLQ with jolokia
curl -XGET --user admin:admin --header "Origin: http://localhost" http://localhost:8161/api/jolokia/exec/org.apache.activemq:brokerName=localhost,destinationName=ActiveMQ.DLQ,destinationType=Queue,type=Broker/retryMessages
NOTE: You can only use this method on a queue that is marked as a DLQ. It will not work for regular queues.
Also the DLQ queue can have its 'DLQ' flag set to false if the server is restarted. It is automatically set to true when a new message is sent to the DLQ
After Justin's reply I've manually implemented the retry mechanism like so:
public void retryAllDlqMessages() throws JMSException {
logger.warn("retryAllDlqMessages starting");
logger.warn("Creating a connection to {}", activemqUrl);
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("test", "test", activemqUrl);
HashMap<String, MessageProducer> messageProducersMap = new HashMap<>();
MessageConsumer consumer = null;
try (ActiveMQConnection connection = (ActiveMQConnection) connectionFactory.createConnection();
ActiveMQSession session = (ActiveMQSession) connection.createSession(true, Session.SESSION_TRANSACTED)) {
String dlqName = getDlqName();
logger.warn("Creating a session to {}", dlqName);
ActiveMQQueue queue = (ActiveMQQueue) session.createQueue(dlqName);
logger.warn("Starting JMS Connection");
connection.start();
logger.warn("Creating a DLQ consumer");
consumer = session.createConsumer(queue);
logger.warn("Consumer start receiving");
Message message = consumer.receive(CONSUMER_RECEIVE_TIME_IN_MS);
int retriedMessages = 0;
while (message != null) {
try {
retryMessage(messageProducersMap, session, message);
retriedMessages++;
} catch (Exception e) {
logger.error("Error calling retryMessage for message = {}", message);
logger.error("Rolling back the JMS transaction...");
session.rollback();
return;
}
message = consumer.receive(CONSUMER_RECEIVE_TIME_IN_MS);
}
logger.warn("Consumer finished retrying {} messages", retriedMessages);
logger.warn("Commiting JMS Transactions of retry");
session.commit();
} finally {
if (!messageProducersMap.isEmpty()) {
logger.warn("Closing {} messageProducers in messageProducersMap", messageProducersMap.size());
for (MessageProducer producer : messageProducersMap.values()) {
producer.close();
}
}
if (consumer != null) {
logger.warn("Closing DLQ Consumer");
consumer.close();
}
}
}
private void retryMessage(HashMap<String, MessageProducer> messageProducersMap, ActiveMQSession session, Message message) {
ActiveMQObjectMessage qm = (ActiveMQObjectMessage) message;
String originalDestinationName = qm.getOriginalDestination().getQualifiedName();
logger.warn("Retry message with JmsID={} to original destination {}", qm.getJMSMessageID(), originalDestinationName);
try {
if (!messageProducersMap.containsKey(originalDestinationName)) {
logger.warn("Creating a new producer for original destination: {}", originalDestinationName);
messageProducersMap.put(originalDestinationName, session.createProducer(qm.getOriginalDestination()));
}
logger.info("Producing message to original destination");
messageProducersMap.get(originalDestinationName).send(qm);
logger.info("Message sent");
} catch (Exception e) {
logger.error("Message retry failed with exception", e);
}
}
I want a method to browse all messages from a messsage queue and can send it to another queue using jmstemplate with using Websphere queues(NOT MQ). I have tried using receive and it is able to retrieve all the messages from the queue but it is still waiting for another message. And the messages are being lost. It must be in a transaction
The Code I have Tried:
**String message = (String) jmsTemplate.receiveAndConvert();
System.out.print(message);
while ((message = (String) jmsTemplate.receiveAndConvert()) != null) {
messages.add(message);
}
return messages;
}**
The JMStemplate should be used for only synchronous read or sending message. For asychronous read use one of the listener implementation. Read here
everyone. I have an HTTP API for posting messages in a RabbitMQ broker and I need to implement the request-response pattern in order to receive the responses from the server. So I am something like a bridge between the clients and the server. I push the messages to the broker with specific routing-key and there is a Consumer for that messages, which is publishing back massages as response and my API must consume the response for every request. So the diagram is something like this:
So what I do is the following- For every HTTP session I create a temporary responseQueue(which is bound to the default exchange, with routing key the name of that queue), after that I set the replyTo header of the message to be the name of the response queue(where I will wait for the response) and also set the template replyQueue to that queue. Here is my code:
public void sendMessage(AbstractEvent objectToSend, final String routingKey) {
final Queue responseQueue = rabbitAdmin.declareQueue();
byte[] messageAsBytes = null;
try {
messageAsBytes = new ObjectMapper().writeValueAsBytes(objectToSend);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
MessageProperties properties = new MessageProperties();
properties.setHeader("ContentType", MessageBodyFormat.JSON);
properties.setReplyTo(responseQueue.getName());
requestTemplate.setReplyQueue(responseQueue);
Message message = new Message(messageAsBytes, properties);
Message receivedMessage = (Message)requestTemplate.convertSendAndReceive(routingKey, message);
}
So what is the problem: The message is sent, after that it is consumed by the Consumer and its response is correctly sent to the right queue, but for some reason it is not taken back in the convertSendAndReceived method and after the set timeout my receivedMessage is null. So I tried to do several things- I started to inspect the spring code(by the way it's a real nightmare to do that) and saw that is I don't declare the response queue it creates a temporal for me, and the replyTo header is set to the name of the queue(the same what I do). The result was the same- the receivedMessage is still null. After that I decided to use another template which uses the default exchange, because the responseQueue is bound to that exchange:
requestTemplate.send(routingKey, message);
Message receivedMessage = receivingTemplate.receive(responseQueue.getName());
The result was the same- the responseMessage is still null.
The versions of the amqp and rabbit are respectively 1.2.1 and 1.2.0. So I am sure that I miss something, but I don't know what is it, so if someone can help me I would be extremely grateful.
1> It's strange that RabbitTemplate uses doSendAndReceiveWithFixed if you provide the requestTemplate.setReplyQueue(responseQueue). Looks like it is false in your explanation.
2> To make it worked with fixed ReplyQueue you should configure a reply ListenerContainer:
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory);
container.setQueues(responseQueue);
container.setMessageListener(requestTemplate);
3> But the most important part here is around correlation. The RabbitTemplate.sendAndReceive populates correlationId message property, but the consumer side has to get deal with it, too: it's not enough just to send reply to the responseQueue, the reply message should has the same correlationId property. See here: how to send response from consumer to producer to the particular request using Spring AMQP?
BTW there is no reason to populate the Message manually: You can just simply support Jackson2JsonMessageConverter to the RabbitTemplate and it will convert your objectToSend to the JSON bytes automatically with appropriate headers.