I am new to the messaging queue implementation.
I have implemented IBM messaging queue(MQ) in my application.
Problem statement:
When this MQ is not able to handle certain number of messages MQ throws timeout exception.
Due to a technical limitation of my system I am unable to catch the exact exception class.
Means that I simply declare catch(Exception e) ..but I would like to know exactly which exception class should be use to handle timeout error.
I think you need some MQ training or you need to do a lot of reading on MQ.
There is no such thing as a timeout on an MQPUT. I would say you have some poorly written code and you are confusing MQ with your poorly written code. Are you logging ALL interactions?
If your code is Java/JMS then you should have the following exception:
catch (JMSException e)
{
System.err.println(e.getLocalizedMessage());
if (e != null)
System.err.println("getLinkedException()=" + e.getLinkedException());
}
If your code is plain Java then you should have the following exception:
catch (MQException e)
{
System.err.println(e.getLocalizedMessage());
System.err.println("CC = " + e.completionCode + " : RC = " + e.reasonCode + " [" + MQConstants.lookup(e.reasonCode, "MQRC_.*") +"]");
}
Related
I would like to create a simple code snippet that fetches all messages from the DLQ and re-sends them to the original destination (AKA resend/retry)
It can be done easily by the ActiveMQ UI (but for a single message at a time).
There is no direct JMS API for re-sending a message from a DLQ to its original queue. In fact, the JMS API doesn't even discuss dead-letter queues. It's merely a convention used by most brokers to deal with messages that can't be consumed.
You'd need to create an actual JMS consumer to receive the message from the DLQ and then create a JMS producer to send the message back to its original queue.
It's important that you use Session.TRANSACTED mode to avoid potential message loss or duplication.
If you use Session.AUTO_ACKNOWLEDGE and there is a problem between the time the message is consumed and sent (e.g the application crashes, hardware failure, etc.) then the message could be lost due to the fact that it was already acknowledged before it was sent successfully.
If you use Session.CLIENT_ACKNOWLEDGE and there is a problem between the time the message is sent and acknowledged then the message could ultimately be duplicated due to the fact that it was already sent before it was acknowledged successfully.
Both operations should be part of the JMS transaction so that the work is atomic.
Lastly, I recommend you either invoke commit() on the transacted session for each message sent or after a small batch of messages (e.g. 10). Given that you have no idea how many messages are in the DLQ it would be unwise to process every message in a single transaction. Generally you want the transaction to be as small as possible in order to minimize the window during which an error might occur and the transaction's work will need to be performed again. Also, the larger the transaction is the more heap memory will be required on the broker to keep track of the work in the transaction. Keep in mind that you can invoke commit() on the same session as many times as you want. You don't need to create a new session for each transaction.
Retrying all messages on the DLQ is already implemented in activemq as an mbean.
You can trigger the retry method with jmxterm/jolokia
e.g
Replaying all messages on queue ActiveMQ.DLQ with jolokia
curl -XGET --user admin:admin --header "Origin: http://localhost" http://localhost:8161/api/jolokia/exec/org.apache.activemq:brokerName=localhost,destinationName=ActiveMQ.DLQ,destinationType=Queue,type=Broker/retryMessages
NOTE: You can only use this method on a queue that is marked as a DLQ. It will not work for regular queues.
Also the DLQ queue can have its 'DLQ' flag set to false if the server is restarted. It is automatically set to true when a new message is sent to the DLQ
After Justin's reply I've manually implemented the retry mechanism like so:
public void retryAllDlqMessages() throws JMSException {
logger.warn("retryAllDlqMessages starting");
logger.warn("Creating a connection to {}", activemqUrl);
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("test", "test", activemqUrl);
HashMap<String, MessageProducer> messageProducersMap = new HashMap<>();
MessageConsumer consumer = null;
try (ActiveMQConnection connection = (ActiveMQConnection) connectionFactory.createConnection();
ActiveMQSession session = (ActiveMQSession) connection.createSession(true, Session.SESSION_TRANSACTED)) {
String dlqName = getDlqName();
logger.warn("Creating a session to {}", dlqName);
ActiveMQQueue queue = (ActiveMQQueue) session.createQueue(dlqName);
logger.warn("Starting JMS Connection");
connection.start();
logger.warn("Creating a DLQ consumer");
consumer = session.createConsumer(queue);
logger.warn("Consumer start receiving");
Message message = consumer.receive(CONSUMER_RECEIVE_TIME_IN_MS);
int retriedMessages = 0;
while (message != null) {
try {
retryMessage(messageProducersMap, session, message);
retriedMessages++;
} catch (Exception e) {
logger.error("Error calling retryMessage for message = {}", message);
logger.error("Rolling back the JMS transaction...");
session.rollback();
return;
}
message = consumer.receive(CONSUMER_RECEIVE_TIME_IN_MS);
}
logger.warn("Consumer finished retrying {} messages", retriedMessages);
logger.warn("Commiting JMS Transactions of retry");
session.commit();
} finally {
if (!messageProducersMap.isEmpty()) {
logger.warn("Closing {} messageProducers in messageProducersMap", messageProducersMap.size());
for (MessageProducer producer : messageProducersMap.values()) {
producer.close();
}
}
if (consumer != null) {
logger.warn("Closing DLQ Consumer");
consumer.close();
}
}
}
private void retryMessage(HashMap<String, MessageProducer> messageProducersMap, ActiveMQSession session, Message message) {
ActiveMQObjectMessage qm = (ActiveMQObjectMessage) message;
String originalDestinationName = qm.getOriginalDestination().getQualifiedName();
logger.warn("Retry message with JmsID={} to original destination {}", qm.getJMSMessageID(), originalDestinationName);
try {
if (!messageProducersMap.containsKey(originalDestinationName)) {
logger.warn("Creating a new producer for original destination: {}", originalDestinationName);
messageProducersMap.put(originalDestinationName, session.createProducer(qm.getOriginalDestination()));
}
logger.info("Producing message to original destination");
messageProducersMap.get(originalDestinationName).send(qm);
logger.info("Message sent");
} catch (Exception e) {
logger.error("Message retry failed with exception", e);
}
}
Is the nifi.processor.scheduling.timeout really defaulted to infinite as described in the admin guide? When I looked at the code, it looks like it is timing out after 60 seconds. We have a processor that takes a bit to start up (load resources) and are encountering the "Timed out while waiting for OnScheduled" error. Just trying to figure out why it sometimes fails on startup and then will also continue to fail with the same error.
Really strange. Turning off all of the processors, bouncing the instance and starting the processor individually seems to eliminate the issue. However, if they are all on and the instance is restarted, we encounter the error.
Could easily be something else, but the startup sequence seems to work.
NIFI Admin
NIFI Processor Code
Code Snippet From NIFI Github where I found the timeout error
String timeoutString = NiFiProperties.getInstance().getProperty(NiFiProperties.PROCESSOR_SCHEDULING_TIMEOUT);
long onScheduleTimeout = timeoutString == null ? 60000
: FormatUtils.getTimeDuration(timeoutString.trim(), TimeUnit.MILLISECONDS);
Future<?> taskFuture = callback.invokeMonitoringTask(task);
try {
taskFuture.get(onScheduleTimeout, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
LOG.warn("Thread was interrupted while waiting for processor '" + this.processor.getClass().getSimpleName()
+ "' lifecycle OnScheduled operation to finish.");
Thread.currentThread().interrupt();
throw new RuntimeException("Interrupted while executing one of processor's OnScheduled tasks.", e);
} catch (TimeoutException e) {
taskFuture.cancel(true);
LOG.warn("Timed out while waiting for OnScheduled of '"
+ this.processor.getClass().getSimpleName()
+ "' processor to finish. An attempt is made to cancel the task via Thread.interrupt(). However it does not "
+ "guarantee that the task will be canceled since the code inside current OnScheduled operation may "
+ "have been written to ignore interrupts which may result in runaway thread which could lead to more issues "
+ "eventually requiring NiFi to be restarted. This is usually a bug in the target Processor '"
+ this.processor + "' that needs to be documented, reported and eventually fixed.");
throw new RuntimeException("Timed out while executing one of processor's OnScheduled task.", e);
} catch (ExecutionException e){
throw new RuntimeException("Failed while executing one of processor's OnScheduled task.", e);
} finally {
callback.postMonitor();
}
I have a scenario where user submits an order for a fund and a security. And if he submits the another order with same fund and security DB throws back SqlIntegrityVoilationException with error code ora-00001.
But my question here is this exception is not getting caught under java.sql.sqlintegrityconstraintviolationexception. I am able to catch this under org.springframework.dao.DataIntegrityViolationException, but I am not able to retrieve the error code (to check for ora-00001).
Anybody has any ideas how to catch that exception and retrieve the error code to check so that I can put back a custom error message like "you cannot submit a dup order for same fund and security combination".
EDIT
catch (SQLException e) {
ex = e; inOrder.setActionMessage(ex.getMessage());
}
catch (DataIntegrityViolationException e) {
ex = e; inOrder.setActionMessage(ex.getMessage());
}
I'm currently having a problem with jms synchronous request/reply approach, this is what happens:
1.) ProgramA create a jms message, a temporary queue and set it as a replyTo.
2.) ProgramB has a listener to the message created from ProgramA, process the message and reply to it. But ProgramB needs to communicate to a 3rd party web service that sometimes takes more than 10seconds to reply, and that is the problem I set the consumer to listen for 5000 (5s) and of course it will timeout afterwards. So the message is not received.
My observation:
1.) Even though ProgramA is done reading (no reply yet, at that instant I try to delete the temporary queue). It's not able to and ProgramB was still able to write to the reply queue, but nobody's going to read that message (too late).
When I try to change 5s to 20s listen time the problem was solved, but is it the right approach?
Also is it possible for the ProgramB to not try to write to the queue when ProgramA has stop reading?
Partial codes:
Destination replyQueue = send(jmsUtil, actionDTO);
SalesOrderResponseDTO responseDTO = readReply(jmsUtil, replyQueue, actionDTO);
public Destination send(JmsSessionUtil jmsUtil, SalesOrderActionDTO soDTO) {
try {
utx.begin();
jmsUtil.send(soDTO, null, 0L, 1, Long.parseLong(configBean.getProperty("jms.payrequest.timetolive")), true);
utx.commit();
return jmsUtil.getReplyQueue();
} catch (Exception e) {
try {
utx.rollback();
} catch (Exception e1) {
}
}
return null;
}
public SalesOrderResponseDTO readReply(JmsSessionUtil jmsUtil, Destination replyQueue, SalesOrderActionDTO actionDTO) {
SalesOrderResponseDTO responseDTO = null;
try {
utx.begin();
responseDTO = (SalesOrderResponseDTO) jmsUtil.read(replyQueue);
if (responseDTO != null) {
// fires the response event
SalesOrderResponsePayload eventPayload = new SalesOrderResponsePayload();
eventPayload.setResponseDTO(responseDTO);
responseEvent.fire(eventPayload);
} else { // timeout
((TemporaryQueue) replyQueue).delete();
jmsUtil.dispose();
}
utx.commit();
return responseDTO;
} catch (Exception e) {
try {
utx.rollback();
} catch (Exception e1) {
}
}
return responseDTO;
}
public String send(MessageDTO messageDTO,
JMSQueueEnum resultNotificationQueue, Long parentProcessId,
int JMSPriority, long timeToLive, boolean hasReply)
throws JMSException, InvalidDTOException, NamingException {
try {
// Process optional parameters
messageDTO.setResultNotificationQueue(resultNotificationQueue);
messageDTO.setParentProcessId(parentProcessId);
// Wrap MessageDTO in a JMS ObjectMessage
ObjectMessage msg = MessageDTOHelper.serialize(session, messageDTO);
msg.setJMSType(messageDTO.getClass().getSimpleName());
msg.setStringProperty("DTOType", messageDTO.getClass()
.getSimpleName());
requestProducer = session.createProducer(queue);
if (hasReply) {
replyQueue = session.createTemporaryQueue();
replyConsumer = session.createConsumer(replyQueue);
msg.setJMSReplyTo(replyQueue);
}
if (JMSPriority > -1) {
requestProducer.send(msg, DeliveryMode.PERSISTENT, JMSPriority,
timeToLive);
} else {
// Send the JMS message
requestProducer.send(msg);
}
return msg.getJMSMessageID();
} catch (Exception e) {
}
return null;
}
public MessageDTO read(Destination replyQueue) throws JMSException,
NamingException {
if (replyQueue instanceof Queue) {
Message msg = replyConsumer.receive(20000);
if (msg == null) {
return null;
}
MessageDTO messageDTO = MessageDTOHelper
.deserialize((ObjectMessage) msg);
return messageDTO;
} else {
}
return null;
}
Actual question here is whether you need synchronous or asynchronous communication.
I would always prefer asynchronous, and it seems from your question that there is no need for synchronous communication neither in your case. However, if there is some reason for synchronous then you are stuck with temporary queues - you'll have to specify timeout interval and you'll face problems expressed in your question. If Program A can wait, raise the timeout interval although that's far from optimal. As far as I know, there is no possibility for Program B to check if A still listens.
In case of asynchronous communication, you have (at least) two JMS options:
Using different message queues - Program A sends the message on Queue1 and finishes, but listens (e.g. through Message Driven Bean) on Queue2 where Program B puts its response when it's done. Small drawback is usage of one extra pair of producer and consumer.
Using same message queue - Program A and Program B both send and receive messages on Queue1, but with different message selector (see description here). Basically, message selectors will filter messages for specific listener and thus enable using same queue for bidirectional communication.
See also:
JMS Synchronous Message Consumption
You could have A add a header to its message with the current timestamp + 5 secs. When B receives the response from the 3rd party, if the current time is greater than the header, it should drop the result and not send. You could use the time-to-live jms message property for this, although that is not its express purpose.
I have to update my existing JMS Receiver program to as follows.
Existing Functionality:
My receiver class will read a message and calls a web service to process the job in one of the server once the message is received as xml.
New Functionality:
The receiver should wait for sometime until the job server is free to process a job. I tried using MessageSelectors but which is only applicable for message headers.I tried this option "message = (JMSTextMessage) mqQueueReceiver.receive(100000000000000);" but whenever i posted a message those message is read after posted into queue. But i want to keep receiver to wait for some interval which i am getting from Job server through web service call.
My Code is below:
connectionFactory = new MQQueueConnectionFactory();
connectionFactory.setHostName(config.getValue("host"));
connectionFactory.setPort(Integer.parseInt(config.getValue("port")));
connectionFactory.setTransportType(JMSC.MQJMS_TP_CLIENT_MQ_TCPIP);
connectionFactory.setQueueManager(config.getValue("manager"));
connectionFactory.setChannel(config.getValue("channel"));
queueConnection = (MQQueueConnection) connectionFactory.createQueueConnection();
queueSession = (MQQueueSession) queueConnection.createQueueSession(true, Session.CLIENT_ACKNOWLEDGE);
queue = (MQQueue) queueSession.createQueue(config.getValue("queue"));
mqQueueReceiver = (MQQueueReceiver) queueSession.createReceiver(queue);
while(true) {
if(this.stopListener) {
System.out.println("stopListener variable is changed ");
break;
}
try {
message = (JMSTextMessage) mqQueueReceiver.receive(1000);
String response = "";
if(this.nullCheckJMSTextObject(message)) {
response= soapClient.invokeWebService(message.getText(),message.getJMSCorrelationID());
if(this.nullCheckSoapResponse(response)) {
queueSession.commit();
} else {
queueSession.rollback();
queueSession.commit();
Thread.sleep(receiverWaitTime);
}
}
} catch (JMSException e) {
System.err.println("Linked Exception");
e.getLinkedException();
System.err.println("Error Code");
e.getErrorCode();
System.err.println("Cause ");
e.getCause();
System.err.println("fillTrackTrace ");
e.fillInStackTrace();
e.printStackTrace();
break;
}catch(IllegalStateException e) {
e.printStackTrace();
break;
}catch(InterruptedException e) {
e.printStackTrace();
break;
}catch(Exception e) {
e.printStackTrace();
break;
}
}
The receive(timeout) method will wait for the specified timeout period for a message to arrive on a queue. If a message arrives on a queue before the timeout, the method will return immediately with a message otherwise the method will wait till the timeout period and then return with no message. You will see a 2033 exception.
The timeout specified for the receive() call indicates how long the receive method must wait for messages before it can return. The timeout specified is not to delay the message delivery. If there is a message, the method will return immediately.
I think your logic can be modified to alter the order of execution. Change the code to receive messages only when your web service is ready to process messages.