I am trying to retrieve messages from an ActiveMQ queue. I set the session to Session.CLIENT_ACKNOWLEDGE. When I receive the message from the server it is auto-acknowledged although I have not called acknowledge on the message. Below is sample code of what I am doing:
connection = connectionFactory.createConnection();
Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
Destination destination = session.createQueue(queue);
consumer = session.createConsumer(destination);
connection.start();
Message message = consumer.receive(1);
The code snippet above works and I do get the message from the queue. The problem is that messages disappear from the queue although I have not acknowledged the message. Any pointers why this is happening. I have been debugging this issue almost all this afternoon with no luck. This is a Spring boot application.
I got the client_ack to work in jmsTemplate. Below is the code.
public Object retrieve() {
return this.jmsTemplate.execute(session -> {
session.recover();
MessageConsumer consumer = session.createConsumer(
this.jmsTemplate.getDestinationResolver().resolveDestinationName(session, queue, false));
try {
Message received = consumer.receive(1);
if (received != null) {
return this.messageConverter.fromMessage(received);
}
}
catch (Exception e) {
return null;
}
finally {
consumer.close();
}
return null;
}, true);
}
The issue I am now seeing is that my app does not see these messages after it restarts. I tried running session.recover() but it is not making any difference. Thanks for your help.
The working code using jmsTemplate is as below:
public Object retrieve() {
return this.jmsTemplate.execute(session -> {
MessageConsumer consumer = session.createConsumer(
this.jmsTemplate.getDestinationResolver().resolveDestinationName(session, queue, false));
try {
Message received = consumer.receive(1);
session.recover();
if (received != null) {
return this.messageConverter.fromMessage(received);
}
}
catch (Exception e) {
return null;
}
finally {
consumer.close();
}
return null;
}, true);
}
This appears to be a problem with the springframework Constants class. It uses introspection to get the integer value of CLIENT_ACKNOWLEDGE but always return a value of 1, so session is always AUTO_ACKNOWLEDGE
To anybody who might run into this issue, I did the following to resolve the issue:
Set the session to non-transacted. This is the default and only option if you are using jmsTemplate. If you are creating your own consumer, you need to create the session as non-transacted when creating the jms connection
If you are creating your own consumer, you need to set the session to CLIENT_ACKNOWLEDGE.
Check the maximum number of redeliveries to the required value. This will differ depending on the product: ActiveMQ, IBM MQ, etc.
call session.recover() after retrieving the message.
I updated the working code that is using jmsTemplate in the original post.
Related
Greetings of the day.
Please help on the below requirement:
Requirement:
We want to delete message from MQ only after it is processed successfully.
Use event based message detection technique and avoid loop
So, to achieve above:
I have created message listener and consumer class below:
{
sessionIn = connectionIn.CreateSession(false, AcknowledgeMode.ClientAcknowledge);
// Create message listener and assign it to consumer
messageListener = new MessageListener(OnMessageCallback);
consumerAsync.MessageListener = messageListener;
Console.WriteLine("Message Listener set. Starting the connection now.");
// Start the connection to receive messages.
connectionWMQ.Start();
}
Reading the message from the call back event and push the message into other system:
OnMessageCallback(Message) {
if (xmsMessage is IBytesMessage)
{
IBytesMessage bytesMessage = (IBytesMessage)xmsMessage;
byte[] arrayMessage = new byte[bytesMessage.BodyLength];
bytesMessage.ReadBytes(arrayMessage);
string message = System.Text.Encoding.Default.GetString(arrayMessage);
}
}
Once the message processed, external system will fire the below over ride method:
Response method override:
protected override Task OnResponse(ReponseMessage message)
{
//Read the message and get the message id and correlation id.
//Delete the message from the queue.
//I am trying to do like this, but Its not working:
messageConsumerDelete = sessionDelete.CreateConsumer(destinationDelete, query);
if (messageConsumerDelete != null)
{
IMessage m = messageConsumerDelete.Receive(1000);
LogWrite("Receive Message=" + m);
m.Acknowledge();
}
}
Please suggest a best solution for this requirement.
I am trying to find a solution for this since weeks, but no breakthrough.
Thanks,
Balaji
The requirement is like to process the messages from dead letter queue by exposed a REST service API(Spring Boot).
So that once REST service is called, one message will be consumed from the DL queue and will publish in the main queue again for processing.
#RabbitListener(queues = "QUEUE_NAME") consumes the message immediately which is not required as per the scenario. The message only has to be consumed by the REST service API.
Any suggestion or solution?
I do not think RabbitListener will help here.
However you could implement this behaviour manually.
Spring Boot automatically creates RabbitMq connection factory so you could use it. When http call is made just read single message from the queue manually, you could use basic.get to synchronously get just one message:
#Autowire
private ConnectionFactory factory
void readSingleMessage() {
Connection connection = null;
Channel channel = null;
try {
connection = factory.newConnection();
channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
GetResponse response = channel.basicGet(QUEUE_NAME, true);
if (response != null) {
//Do something with the message
}
} finally {
//Check if not null
channel.close();
connection.close();
}
}
If you are using Spring; you can avoid all the boilerplate in the other answer using RabbitTemplate.receive(...).
EDIT
To manually ack/reject the message, use the execute method instead.
template.execute(channel -> {
GetResponse got = channel.basicGet("foo", false);
// ...
channel.basicAck(got.getEnvelope().getDeliveryTag(), false);
return null;
});
It's a bit lower level, but again, most of the boilerplate is taken care of for you.
I would like to create a simple code snippet that fetches all messages from the DLQ and re-sends them to the original destination (AKA resend/retry)
It can be done easily by the ActiveMQ UI (but for a single message at a time).
There is no direct JMS API for re-sending a message from a DLQ to its original queue. In fact, the JMS API doesn't even discuss dead-letter queues. It's merely a convention used by most brokers to deal with messages that can't be consumed.
You'd need to create an actual JMS consumer to receive the message from the DLQ and then create a JMS producer to send the message back to its original queue.
It's important that you use Session.TRANSACTED mode to avoid potential message loss or duplication.
If you use Session.AUTO_ACKNOWLEDGE and there is a problem between the time the message is consumed and sent (e.g the application crashes, hardware failure, etc.) then the message could be lost due to the fact that it was already acknowledged before it was sent successfully.
If you use Session.CLIENT_ACKNOWLEDGE and there is a problem between the time the message is sent and acknowledged then the message could ultimately be duplicated due to the fact that it was already sent before it was acknowledged successfully.
Both operations should be part of the JMS transaction so that the work is atomic.
Lastly, I recommend you either invoke commit() on the transacted session for each message sent or after a small batch of messages (e.g. 10). Given that you have no idea how many messages are in the DLQ it would be unwise to process every message in a single transaction. Generally you want the transaction to be as small as possible in order to minimize the window during which an error might occur and the transaction's work will need to be performed again. Also, the larger the transaction is the more heap memory will be required on the broker to keep track of the work in the transaction. Keep in mind that you can invoke commit() on the same session as many times as you want. You don't need to create a new session for each transaction.
Retrying all messages on the DLQ is already implemented in activemq as an mbean.
You can trigger the retry method with jmxterm/jolokia
e.g
Replaying all messages on queue ActiveMQ.DLQ with jolokia
curl -XGET --user admin:admin --header "Origin: http://localhost" http://localhost:8161/api/jolokia/exec/org.apache.activemq:brokerName=localhost,destinationName=ActiveMQ.DLQ,destinationType=Queue,type=Broker/retryMessages
NOTE: You can only use this method on a queue that is marked as a DLQ. It will not work for regular queues.
Also the DLQ queue can have its 'DLQ' flag set to false if the server is restarted. It is automatically set to true when a new message is sent to the DLQ
After Justin's reply I've manually implemented the retry mechanism like so:
public void retryAllDlqMessages() throws JMSException {
logger.warn("retryAllDlqMessages starting");
logger.warn("Creating a connection to {}", activemqUrl);
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("test", "test", activemqUrl);
HashMap<String, MessageProducer> messageProducersMap = new HashMap<>();
MessageConsumer consumer = null;
try (ActiveMQConnection connection = (ActiveMQConnection) connectionFactory.createConnection();
ActiveMQSession session = (ActiveMQSession) connection.createSession(true, Session.SESSION_TRANSACTED)) {
String dlqName = getDlqName();
logger.warn("Creating a session to {}", dlqName);
ActiveMQQueue queue = (ActiveMQQueue) session.createQueue(dlqName);
logger.warn("Starting JMS Connection");
connection.start();
logger.warn("Creating a DLQ consumer");
consumer = session.createConsumer(queue);
logger.warn("Consumer start receiving");
Message message = consumer.receive(CONSUMER_RECEIVE_TIME_IN_MS);
int retriedMessages = 0;
while (message != null) {
try {
retryMessage(messageProducersMap, session, message);
retriedMessages++;
} catch (Exception e) {
logger.error("Error calling retryMessage for message = {}", message);
logger.error("Rolling back the JMS transaction...");
session.rollback();
return;
}
message = consumer.receive(CONSUMER_RECEIVE_TIME_IN_MS);
}
logger.warn("Consumer finished retrying {} messages", retriedMessages);
logger.warn("Commiting JMS Transactions of retry");
session.commit();
} finally {
if (!messageProducersMap.isEmpty()) {
logger.warn("Closing {} messageProducers in messageProducersMap", messageProducersMap.size());
for (MessageProducer producer : messageProducersMap.values()) {
producer.close();
}
}
if (consumer != null) {
logger.warn("Closing DLQ Consumer");
consumer.close();
}
}
}
private void retryMessage(HashMap<String, MessageProducer> messageProducersMap, ActiveMQSession session, Message message) {
ActiveMQObjectMessage qm = (ActiveMQObjectMessage) message;
String originalDestinationName = qm.getOriginalDestination().getQualifiedName();
logger.warn("Retry message with JmsID={} to original destination {}", qm.getJMSMessageID(), originalDestinationName);
try {
if (!messageProducersMap.containsKey(originalDestinationName)) {
logger.warn("Creating a new producer for original destination: {}", originalDestinationName);
messageProducersMap.put(originalDestinationName, session.createProducer(qm.getOriginalDestination()));
}
logger.info("Producing message to original destination");
messageProducersMap.get(originalDestinationName).send(qm);
logger.info("Message sent");
} catch (Exception e) {
logger.error("Message retry failed with exception", e);
}
}
I'm new to JMS and am trying to setup Apache Active MQ for a messaging application as an alternative to Azure Service Bus that I'm very familiar with. I would like to setup topics and durable subscribers as and administrative task, and would like the runtime process to consume messages from those existing durable subscriber only based upon its name and, possibly, client id.
How do I retrieve an existing durable subscriber, without knowing the selector?
All the documentation and the samples I've read show that the only way to consume a message is to call the session.createDurableSubscriber() method.
Additionaly, I prefer to use the AMQP abstraction over JMS. So I found the following code to retrieve an existing subscriber:
public static ReceiverLink RecoverDurableSource(Session session, string topicPath, string subscriptionName)
{
Source recovered = null;
using (var attached = new ManualResetEvent(false))
{
void OnAttached(ILink link, Attach Attach)
{
recovered = (Source)Attach.Source;
attached.Set();
}
ReceiverLink receiver = null;
try
{
receiver = new ReceiverLink(session, subscriptionName, (Source)null, OnAttached);
if (!attached.WaitOne(TimeSpan.FromSeconds(5)))
return null;
CloseReceiverLink(receiver);
return recovered != null
? new ReceiverLink(session, subscriptionName, recovered, null)
: null
;
}
finally
{
if (recovered == null)
CloseReceiverLink(receiver);
}
}
}
private static void CloseReceiverLink(ReceiverLink receiver)
{
if (receiver == null)
return;
if (receiver.Error == null || Equals(receiver.Error.Condition, new Symbol("amqp:not-found")))
receiver.Close();
}
However, this code has the nasty side effect to re-create and default durable subscriber (manifested in this code by the ReceiverLink object) with the same name and then, if it exists, re-creating it with the correct Sourceobject.
But this may disrupt the reception of messages at the time this method is called.
I'm currently having a problem with jms synchronous request/reply approach, this is what happens:
1.) ProgramA create a jms message, a temporary queue and set it as a replyTo.
2.) ProgramB has a listener to the message created from ProgramA, process the message and reply to it. But ProgramB needs to communicate to a 3rd party web service that sometimes takes more than 10seconds to reply, and that is the problem I set the consumer to listen for 5000 (5s) and of course it will timeout afterwards. So the message is not received.
My observation:
1.) Even though ProgramA is done reading (no reply yet, at that instant I try to delete the temporary queue). It's not able to and ProgramB was still able to write to the reply queue, but nobody's going to read that message (too late).
When I try to change 5s to 20s listen time the problem was solved, but is it the right approach?
Also is it possible for the ProgramB to not try to write to the queue when ProgramA has stop reading?
Partial codes:
Destination replyQueue = send(jmsUtil, actionDTO);
SalesOrderResponseDTO responseDTO = readReply(jmsUtil, replyQueue, actionDTO);
public Destination send(JmsSessionUtil jmsUtil, SalesOrderActionDTO soDTO) {
try {
utx.begin();
jmsUtil.send(soDTO, null, 0L, 1, Long.parseLong(configBean.getProperty("jms.payrequest.timetolive")), true);
utx.commit();
return jmsUtil.getReplyQueue();
} catch (Exception e) {
try {
utx.rollback();
} catch (Exception e1) {
}
}
return null;
}
public SalesOrderResponseDTO readReply(JmsSessionUtil jmsUtil, Destination replyQueue, SalesOrderActionDTO actionDTO) {
SalesOrderResponseDTO responseDTO = null;
try {
utx.begin();
responseDTO = (SalesOrderResponseDTO) jmsUtil.read(replyQueue);
if (responseDTO != null) {
// fires the response event
SalesOrderResponsePayload eventPayload = new SalesOrderResponsePayload();
eventPayload.setResponseDTO(responseDTO);
responseEvent.fire(eventPayload);
} else { // timeout
((TemporaryQueue) replyQueue).delete();
jmsUtil.dispose();
}
utx.commit();
return responseDTO;
} catch (Exception e) {
try {
utx.rollback();
} catch (Exception e1) {
}
}
return responseDTO;
}
public String send(MessageDTO messageDTO,
JMSQueueEnum resultNotificationQueue, Long parentProcessId,
int JMSPriority, long timeToLive, boolean hasReply)
throws JMSException, InvalidDTOException, NamingException {
try {
// Process optional parameters
messageDTO.setResultNotificationQueue(resultNotificationQueue);
messageDTO.setParentProcessId(parentProcessId);
// Wrap MessageDTO in a JMS ObjectMessage
ObjectMessage msg = MessageDTOHelper.serialize(session, messageDTO);
msg.setJMSType(messageDTO.getClass().getSimpleName());
msg.setStringProperty("DTOType", messageDTO.getClass()
.getSimpleName());
requestProducer = session.createProducer(queue);
if (hasReply) {
replyQueue = session.createTemporaryQueue();
replyConsumer = session.createConsumer(replyQueue);
msg.setJMSReplyTo(replyQueue);
}
if (JMSPriority > -1) {
requestProducer.send(msg, DeliveryMode.PERSISTENT, JMSPriority,
timeToLive);
} else {
// Send the JMS message
requestProducer.send(msg);
}
return msg.getJMSMessageID();
} catch (Exception e) {
}
return null;
}
public MessageDTO read(Destination replyQueue) throws JMSException,
NamingException {
if (replyQueue instanceof Queue) {
Message msg = replyConsumer.receive(20000);
if (msg == null) {
return null;
}
MessageDTO messageDTO = MessageDTOHelper
.deserialize((ObjectMessage) msg);
return messageDTO;
} else {
}
return null;
}
Actual question here is whether you need synchronous or asynchronous communication.
I would always prefer asynchronous, and it seems from your question that there is no need for synchronous communication neither in your case. However, if there is some reason for synchronous then you are stuck with temporary queues - you'll have to specify timeout interval and you'll face problems expressed in your question. If Program A can wait, raise the timeout interval although that's far from optimal. As far as I know, there is no possibility for Program B to check if A still listens.
In case of asynchronous communication, you have (at least) two JMS options:
Using different message queues - Program A sends the message on Queue1 and finishes, but listens (e.g. through Message Driven Bean) on Queue2 where Program B puts its response when it's done. Small drawback is usage of one extra pair of producer and consumer.
Using same message queue - Program A and Program B both send and receive messages on Queue1, but with different message selector (see description here). Basically, message selectors will filter messages for specific listener and thus enable using same queue for bidirectional communication.
See also:
JMS Synchronous Message Consumption
You could have A add a header to its message with the current timestamp + 5 secs. When B receives the response from the 3rd party, if the current time is greater than the header, it should drop the result and not send. You could use the time-to-live jms message property for this, although that is not its express purpose.