I figured I would toss a question on here incase anyone has ideas. My MQ Admin created a new queue and alias queue for me to write messages to. I have one application writing to the queue, and another application listening on the alias queue. I am using spring jmsTemplate to write to my queue. We are seeing a behavior where the message is being written to the queue but then instantly being discarded. We disabled gets and to see if an expiry parameter was being set somehow, I used the jms template to set the expiry setting (timeToLive). I set the expiry to 10 minutes but my message still disappears instantly. A snippet of my code and settings are below.
public void publish(ModifyRequestType response) {
jmsTemplate.setExplicitQosEnabled(true);
jmsTemplate.setTimeToLive(600000);
jmsTemplate.send(CM_QUEUE_NAME, new MessageCreator() {
public Message createMessage(Session session) throws JMSException {
String responseXML = null;
try {
responseXML myJAXBContext.getInstance().toXML(response);
log.info(responseXML);
TextMessage message = session.createTextMessage(responseXML);
return message;
} catch (myException e) {
e.printStackTrace();
log.info(responseXML);
return null;
}
}
});
}
/////////////////My settings
QUEUE.PUB_SUB_DOMAIN=false
QUEUE.SUBSCRIPTION_DURABLE=false
QUEUE.CLONE_SUPPORT=0
QUEUE.SHARE_CONV_ALLOWED=1
QUEUE.MQ_PROVIDER_VERSION=6
I found my issue. I had a parent method that had the #Transactional annotation. I do not want my new jms message to be part of that transaction so I am going to add jmsTemplate.setSessionTransacted(false); before performing a jmsTemplate.send. I have created a separate jmsTempalte for sending my new message instead of reusing the one that was existing, which needs to be managed.
Related
Working versions in the app
IBM AllClient version : 'com.ibm.mq:com.ibm.mq.allclient:9.1.1.0'
org.springframework:spring-jms : 4.3.9.RELEASE
javax.jms:javax.jms-api : 2.0.1
My requirement is that in case of the failure of a message processing due to say, consumer not being available (eg. DB is unavailable), the message remains in the queue or put back on the queue (if that is even possible). This is because the order of the messages is important, messages have to be consumed in the same order that they are received. The Java app is single-threaded.
I have tried the following
#Override
public void onMessage(Message message)
{
try{
if(message instanceOf Textmessage)
{
}
:
:
throw new Exception("Test");// Just to test the retry
}
catch(Exception ex)
{
try
{
int temp = message.getIntProperty("JMSXDeliveryCount");
throw new RuntimeException("Redlivery attempted ");
// At this point, I am expecting JMS to put the message back into the queue.
// But it is actually put into the Bakout queue.
}
catch(JMSException ef)
{
String temp = ef.getMessage();
}
}
}
I have set this in my spring.xml for the jmsContainer bean.
<property name="sessionTransacted" value="true" />
What is wrong with the code above ?
And if putting the message back in the queue is not practical, how can one browse the message, process it and, if successful, pull the message (so it is consumed and no longer on the queue) ? Is this scenario supported in IBM provider for JMS?
The IBM MQ Local queue has BOTHRESH(1).
To preserve message ordering, one approach might be to stop the message listener temporarily as part of your rollback strategy. Looking at the Spring Boot doc for DefaultMessageListenerContainer there is a stop(Runnable callback) method. I've experimented with using this in a rollback as follows.
To ensure my Listener is single threaded, on my DefaultJmsListenerContainerFactory I set containerFactory.setConcurrency("1").
In my Listener, I set an id
#JmsListener(destination = "DEV.QUEUE.2", containerFactory = "listenerTwoFactory", concurrency="1", id="listenerTwo")
And retrieve the DefaultMessageListenerContainer instance.
JmsListenerEndpointRegistry reg = context.getBean(JmsListenerEndpointRegistry.class);
DefaultMessageListenerContainer mlc = (DefaultMessageListenerContainer) reg.getListenerContainer("listenerTwo");
For testing, I check JMSXDeliveryCount and throw an exception to rollback.
retryCount = Integer.parseInt(msg.getStringProperty("JMSXDeliveryCount"));
if (retryCount < 5) {
throw new Exception("Rollback test "+retryCount);
}
In the Listener's catch processing, I call stop(Runnable callback) on the DefaultMessageListenerContainer instance and pass in a new class ContainerTimedRestart as defined below.
//catch processing here and decide to rollback
mlc.stop(new ContainerTimedRestart(mlc,delay));
System.out.println("#### "+getClass().getName()+" Unable to process message.");
throw new Exception();
ContainerTimedRestart extends Runnable and DefaultMessageListenerContainer is responsible for invoking the run() method when the stop call completes.
public class ContainerTimedRestart implements Runnable {
//Container instance to restart.
private DefaultMessageListenerContainer theMlc;
//Default delay before restart in mills.
private long theDelay = 5000L;
//Basic constructor for testing.
public ContainerTimedRestart(DefaultMessageListenerContainer mlc, long delay) {
theMlc = mlc;
theDelay = delay;
}
public void run(){
//Validate container instance.
try {
System.out.println("#### "+getClass().getName()+"Waiting for "+theDelay+" millis.");
Thread.sleep(theDelay);
System.out.println("#### "+getClass().getName()+"Restarting container.");
theMlc.start();
System.out.println("#### "+getClass().getName()+"Container started!");
} catch (InterruptedException ie) {
ie.printStackTrace();
//Further checks and ensure container is in correct state.
//Report errors.
}
}
I loaded my queue with three messages with payloads "a", "b", and "c" respectively and started the listener.
Checking DEV.QUEUE.2 on my queue manager I see IPPROCS(1) confirming only one application handle has the queue open. The messages are processed in order after each is rolled five times and with a 5 second delay between rollback attempts.
IBM MQ classes for JMS has poison message handling built in. This handling is based on the QLOCAL setting BOTHRESH, this stands for Backout Threshold. Each IBM MQ message has a "header" called the MQMD (MQ Message Descriptor). One of the fields in the MQMD is BackoutCount. The default value of BackoutCount on a new message is 0. Each time a message rolled back to the queue this count is incremented by 1. A rollback can be either from a specific call to rollback(), or due to the application being disconnected from MQ before commit() is called (due to a network issue for example or the application crashing).
Poison message handling is disabled if you set BOTHRESH(0).
If BOTHRESH is >= 1, then poison message handling is enabled and when IBM MQ classes for JMS reads a message from a queue it will check if the BackoutCount is >= to the BOTHRESH. If the message is eligible for poison message handling then it will be moved to the queue specified in the BOQNAME attribute, if this attribute is empty or the application does not have access to PUT to this queue for some reason, it will instead attempt to put the message to the queue specified in the queue managers DEADQ attribute, if it can't put to either of these locations it will be rolled back to the queue.
You can find more detailed information on IBM MQ classes for JMS poison message handling in the IBM MQ v9.1 Knowledge Center page Developing applications>Developing JMS and Java applications>Using IBM MQ classes for JMS>Writing IBM MQ classes for JMS applications>Handling poison messages in IBM MQ classes for JMS
In Spring JMS you can define your own container. One container is created for one Jms Destination. We should run a single-threaded JMS listener to maintain the message ordering, to make this work set the concurrency to 1.
We can design our container to return null once it encounters errors, post-failure all receive calls should return null so that no messages are polled from the destination till the destination is active once again. We can maintain an active state using a timestamp, that could be simple milliseconds. A sample JMS config should be sufficient to add backoff. You can add small sleep instead of continuously returning null from receiveMessage method, for example, sleep for 10 seconds before making the next call, this will save some CPU resources.
#Configuration
#EnableJms
public class JmsConfig {
#Bean
public JmsListenerContainerFactory<?> jmsContainerFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory() {
#Override
protected DefaultMessageListenerContainer createContainerInstance() {
return new DefaultMessageListenerContainer() {
private long deactivatedTill = 0;
#Override
protected Message receiveMessage(MessageConsumer consumer) throws JMSException {
if (deactivatedTill < System.currentTimeMillis()) {
return receiveFromConsumer(consumer, getReceiveTimeout());
}
logger.info("Disabled due to failure :(");
return null;
}
#Override
protected void doInvokeListener(MessageListener listener, Message message)
throws JMSException {
try {
super.doInvokeListener(listener, message);
} catch (Exception e) {
handleException(message);
throw e;
}
}
private long getDelay(int retryCount) {
if (retryCount <= 1) {
return 20;
}
return (long) (20 * Math.pow(2, retryCount));
}
private void handleException(Message msg) throws JMSException {
if (msg.propertyExists("JMSXDeliveryCount")) {
int retryCount = msg.getIntProperty("JMSXDeliveryCount");
deactivatedTill = System.currentTimeMillis() + getDelay(retryCount);
}
}
#Override
protected void doInvokeListener(SessionAwareMessageListener listener, Session session,
Message message)
throws JMSException {
try {
super.doInvokeListener(listener, session, message);
} catch (Exception e) {
handleException(message);
throw e;
}
}
};
}
};
// This provides all boot's default to this factory, including the message converter
configurer.configure(factory, connectionFactory);
// You could still override some of Boot's default if necessary.
return factory;
}
}
I'm creating a topic with multiple consumer, each of them identified by a clientId.
The behaviour I'm seeing is :
A message come in
I throw a runtime exception in one of my consumer
I would like this consumer to try to consume again the same message but it goes straight to the next one.
Is there a way to stop the consumption after 3 try for instance ?
You could create a transacted JMS Session:
// create JMS Session from JMS Connection
session = connection.createSession(true, Session.AUTO_ACKNOWLEDGE);
and use the Session.rollback() method to indicate that you need to see that message again:
public void onMessage(Message message)
{
msgsReceived++;
System.err.println("received: " + message);
if( msgsReceived<3 ) { // simulating an error case
session.rollback();
} else {
session.commit();
}
you should then see this message 3 times until you commit it the last time.
On the client side javascript I have
stomp.subscribe("/topic/path", function (message) {
console.info("message received");
});
And on the server side
public class Controller {
private final MessageSendingOperations<String> messagingTemplate;
ï¼ Autowired
public Controller(MessageSendingOperations<String> messagingTemplate) {
this.messagingTemplate = messagingTemplate;
}
#SubscribeMapping("/topic/path")
public void subscribe() {
LOGGER.info("before send");
messagingTemplate.convertAndSend(/topic/path, "msg");
}
}
From this setup, I am occasionally (around once in 30 page refreshes) experiencing message dropping, which means I can see neither "message received" msg on the client side nor the websocket traffic from Chrome debugging tool.
"before send" is always logged on the server side.
This looks like that the MessageSendingOperations is not ready when I call it in the subscribe() method. (if I put Thread.sleep(50); before calling messagingTemplate.convertAndSend the problem would disappear (or much less likely to be reproduced))
I wonder if anyone experienced the same before and if there is an event that can tell me MessageSendingOperations is ready or not.
The issue you are facing is laying in the nature of clientInboundChannel which is ExecutorSubscribableChannel by default.
It has 3 subscribers:
0 = {SimpleBrokerMessageHandler#5276} "SimpleBroker[DefaultSubscriptionRegistry[cache[0 destination(s)], registry[0 sessions]]]"
1 = {UserDestinationMessageHandler#5277} "UserDestinationMessageHandler[DefaultUserDestinationResolver[prefix=/user/]]"
2 = {SimpAnnotationMethodMessageHandler#5278} "SimpAnnotationMethodMessageHandler[prefixes=[/app/]]"
which are invoked within taskExecutor, hence asynchronously.
The first one here (SimpleBrokerMessageHandler (or StompBrokerRelayMessageHandler) if you use broker-relay) is responsible to register subscription for the topic.
Your messagingTemplate.convertAndSend(/topic/path, "msg") operation may be performed before the subscription registration for that WebSocket session, because they are performed in the separate threads. Hence the Broker handler doesn't know you to send the message to the session.
The #SubscribeMapping can be configured on method with return, where the result of this method will be sent as a reply to that subscription function on the client.
HTH
Here is my solution. It is along the same lines. Added a ExecutorChannelInterceptor and published a custom SubscriptionSubscribedEvent. The key is to publish the event after the message has been handled by AbstractBrokerMessageHandler which means the subscription has been registered with the broker.
#Override
public void configureClientInboundChannel(ChannelRegistration registration) {
registration.interceptors(new ExecutorChannelInterceptorAdapter() {
#Override
public void afterMessageHandled(Message<?> message, MessageChannel channel, MessageHandler handler, Exception ex) {
SimpMessageHeaderAccessor accessor = SimpMessageHeaderAccessor.wrap(message);
if (accessor.getMessageType() == SimpMessageType.SUBSCRIBE && handler instanceof AbstractBrokerMessageHandler) {
/*
* Publish a new session subscribed event AFTER the client
* has been subscribed to the broker. Before spring was
* publishing the event after receiving the message but not
* necessarily after the subscription occurred. There was a
* race condition because the subscription was being done on
* a separate thread.
*/
applicationEventPublisher.publishEvent(new SessionSubscribedEvent(this, message));
}
}
});
}
A little late but I thought I'd add my solution. I was having the same problem with the subscription not being registered before I was sending data through the messaging template. This issue happened rarely and unpredictable because of the race with the DefaultSubscriptionRegistry.
Unfortunately, I could not just use the return method of the #SubscriptionMapping because we were using a custom object mapper that changed dynamically based on the type of user (attribute filtering essentially).
I searched through the Spring code and found SubscriptionMethodReturnValueHandler was responsible for sending the return value of subscription mappings and had a different messagingTemplate than the autowired SimpMessagingTemplate of my async controller!!
So the solution was autowiring MessageChannel clientOutboundChannel into my async controller and using that to create a SimpMessagingTemplate. (You can't directly wire it in because you'll just get the template going to the broker).
In subscription methods, I then used the direct template while in other methods I used the template that went to the broker.
ActiveMQ 5.10.0
Spring 4.1.2
I'm using Spring to access activeMQ and trying to peek at the queue before adding a new message onto the queue. The message is added successfully, but it does not show anything in the queue. Through the web interface, I see my messages are pending in the queue.
Thanks!
#Service
public class MessageQueueService{
private static final Logger logger = LoggerFactory.getLogger(MessageQueueService.class);
#Inject
JmsTemplate jmsTemplate;
#SuppressWarnings({ "rawtypes", "unchecked" })
public void testAddJob(){
jmsTemplate.send(new MessageCreator() {
public Message createMessage(Session session) throws JMSException {
IndexJob j1=new IndexJob();
j1.setOperation("post");
ObjectMessage om=session.createObjectMessage();
om.setObject(j1);
QueueBrowser qb=session.createBrowser((javax.jms.Queue) jmsTemplate.getDefaultDestination());
Enumeration<Message> messages=qb.getEnumeration();
logger.info("browsing "+qb.getQueue().getQueueName());
int i=0;
while(messages.hasMoreElements()) {
i++;
Message message=messages.nextElement();
logger.info(message+"");
}
logger.info("total record:"+i);
return om;
}
});
}
output:
2014-12-07 00:03:43.874 [main] INFO c.b.b.s.MessageQueueService - browsing indexJob
2014-12-07 00:03:43.878 [main] INFO c.b.b.s.MessageQueueService - total record:0
UPDATE: execute has a not yet well-documented parameter boolean startConnection. When it is set to "true", it seem to work. This is not a solution though -
String result=jms.execute(new SessionCallback<String>() {
#Override
public String doInJms(Session session) throws JMSException {
QueueBrowser queue=session.createBrowser((Queue)session.createQueue("indexJob"));
Enumeration<Message> messages=queue.getEnumeration();
String result="";
logger.info("Browse Queue: "+queue.getQueue().getQueueName());
while(messages.hasMoreElements()) {
Message message=messages.nextElement();
result+=message;
}
logger.info(result);
return result;
}
}, true);
Looking at org.springframework.jms.core.JmsTemplate.class source, most of the send methods are using execute() method with startConnection=false.
If the connection was not started, then how did the messages get added to the queue?
Does anyone know what this #param startConnection whether to start the Connection means?
This can be a somewhat confusing bit of JMS. The Connection start only refers to consumption of messages from the connection, not to producing. You are free to produce messages whenever you like, started or not, but if you want to consume or browse a destination you need to start the connection otherwise you will not get any messages dispatched to your consumers.
This purpose behind this is to allow you to create all your JMS resources prior to receiving any messages which might otherwise catch you in an state where you app isn't quite ready for them.
So in short, if you want to browse that message, you need to ensure the connection gets started.
I'm using JMS to perform some long-running import processes, but running into some duplicate message problems I don't quite understand.
The flow is as follows. Some backing bean code sends a message to a Message Bean. The message bean receives this message, polls a third party service, and then if there is new data to import, commits these rows into a database. The problem is that JMS is sending my message twice in instances where my import process is taking some time, approx 60 seconds or so. If there are no new rows to import and the process takes 30 seconds or so, the message is only sent once.
I thought it had something to do with the acknowledgement of the message receipt so the first thing I did (since the auto acknowledge wasn't working) was set the QueueSession to Session.CLIENT_ACKNOWLEDGE and put msg.acknowledge() in my onMessage method. Sadly, the messages were still being sent twice.
No error codes or Exceptions are thrown during any of this.
Here's the code.
In Backing Bean
public MyBackingBean(){
try {
InitialContext ctx = new InitialContext();
qconFactory = (QueueConnectionFactory) ctx.lookup(JMS_FACTORY);
qcon = qconFactory.createQueueConnection();
qsession = qcon.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
queue = (javax.jms.Queue) ctx.lookup(IMPORT_QUEUE);
qsender = qsession.createSender(queue);
qsender.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
qcon.start();
} catch (JMSException e) {
System.out.println(e);
} catch (NamingException e) {
System.out.println(e);
}
public void startImport() {
try {
// send a JMS message for the long running job
MapMessage mapMessage = qsession.createMapMessage();
mapMessage.setObjectProperty("ipAddress", "127.0.0.1");
qsender.send(mapMessage);
} catch (JMSException e) {
e.printStackTrace();
} catch (Throwable te) {
te.printStackTrace();
}
}
In Import Message Bean
#Override
public void onMessage(Message msg) {
try {
// create an ADF Application Module
// poll a third party for some data
// copy these rows (if new) and then
// commit via the ADF Application Module
}
catch (Exception e){
// no errors are being thrown
}
}
For now, I'm disregarding the duplicate by checking to see if the message is a redelivery (msg.getJMSRedelivered()) and disregarding the duplicate, but I'm not too happy with the band-aid.
Does anyone have some pointers on this?
Will, I can't be sure what's happening, but realize that, with AUTO_ACKNOWLEDGE, the acknowledgement doesn't actually happen until the thread returns from the onMessage() call. Therefore, if you are spending 60 seconds in your onMessage() call, I'm not surprised at the redelivery.
With CLIENT_ACKNOWLEDGE, by calling msg.acknowledge(), you should be explicitly acknowledging the message. However, at what point are you doing that? After spending 60 seconds in onMessage()? Then you'd get the same behavior, I'd expect.
You can call msg.acknowledge() as soon as you enter your onMessage() call, but realize that means the message won't be redelivered in the event that something later in your onMessage() method crashes. But it looks like you're not using persistent delivery anyway, so perhaps you don't care.
Here's a good reference;
http://www2.sys-con.com/itsg/virtualcd/Java/archives/0604/chappell/index.html#s1