I have a JMS 2.0 MessageListener which seems to be sporadically reprocessing messages even when they have been successfully processed (confirmed via a log). I am suspecting that a session.commit() needs to be done but I am unsure because in a vast majority of cases, the messages are NOT retried. From what I understand, AUTO_ACKNOWLEDGE is the default but again, I am not so sure how it works for SessionAwareMessageListener.
The relevant spring.xml section looks something like this
<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
:
:
<property name="messageListener" ref="myMessageListener" />
<property name="maxConcurrentConsumers" value="1" />
<property name="receiveTimeout" value="5000" />
<property name="sessionTransacted" value="true" />
</bean>
MessageListener implementation is as follows
public class MyMessageListener implements SessionAwareMessageListener {
// All spring autowired objects
:
:
#Override
public void onMessage(Message message, Session session)
{
logger.debug("JMSMessage ID : " + message.JMSMessageId, "Entering onMessage() ...");
logger.debug("JMSMessage ID : " + message.JMSMessageId, "Retry Count : " + message.getIntProperty("JMSXDeliveryCount"));
try
{
}
catch(JMSException e)
{
// Log something
throw new RuntimeException(e);
}
catch(Exception ex)
{
if(certain types of exceptions)
{
session.rollback();
System.Exit(1);
}
throw new RuntimeException(ex);
}
// THE FOLLOWING IS THE LAST LINE IN onMessage()
logger.debug("JMSMessage ID : " + message.JMSMessageId,"Completed successfully !");
}
}
So, almost all the messages that I see now have this in the logs
:
JMSMessage Id : 1234, Entering onMessage()
JMSMessage Id : 1234, Retry count : 1
:
JmsMessage Id : 1234, Completed successfully!
JmsMessage Id : 3344, Entering onMessage() // New message taken up for processing.
JMSMessage Id : 3344, Retry count : 1
The problem is that once in a while (after thousands of messages), I see this in the logs
:
JMSMessage Id : 5566, Entering onMessage()
JMSMessage Id : 5566, Retry count : 1
:
JmsMessage Id : 5566, Completed successfully!
JMSMessage Id : 5566, Entering onMessage() // WHY IS JMS PROCESSING THE SAME MESSAGE (MESSAGEID : 5566) AGAIN ?
JMSMessage Id : 5566, Retry count : 2
:
:
When you have sessionTransacted set to true acknowledge mode is ignored, there is even a special value that can be set to denote that it is not being used, from other examples I see this:
<property name="sessionAcknowledgeModeName" value="SESSION_TRANSACTED"/>
According to Gary Russell's answer to the stackoverflow question Spring DMLC message consumption: auto_ack vs Transacted Session, if you have sessionTransacted set to true with a DMLC, the session is committed by the DMLC after the listener is called, if the listener throws an exception the transaction is rolled back.
Related
In my application i am processing messages from queue using camel and process it in multiple threads.
I tried to persist the data to a table during the process with PlatformTransactionManager, with Propagation type "REQUIRED_NEW", but on using the commit the transaction seems to be closed. and connection not available for other process.
The application context.xml looks as in below snippet.
<!-- other definitions -->
<context:property-placeholder location="classpath:app.properties"/>
<bean id="appDataSource" class="org.apache.commons.dbcp2.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.OracleDriver"/>
<property name="url" value="${dburl}"/>
<property name="username" value="${dbUserName}"/>
<property name="password" value="${dbPassword}"/>
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="appDataSource" />
</bean>
<!-- Other bean reference. -->
<bean id="itemDao" class="app.item.dao.ItemDao">
<property name="dataSource" ref="appDataSource"/>
</bean>
<bean id="orderProcess" class="app.order.process.OrderProcess" scope="prototype">
<property name="itemDao" ref="itemDao"/>
</bean>
I have a DAO classes something like below, also there are other Dao's.
public class ItemDao{
private NamedParameterJdbcTemplate namedParameterJdbcTemplate;
private PlatformTransactionManager transactionManager;
private TransactionStatus transactionStatus;
//Setter injection of datasource
public void setDataSource(DataSource dataSource) {
this.namedParameterJdbcTemplate = new NamedParameterJdbcTemplate(dataSource);
this.transactionManager = new DataSourceTransactionManager(dataSource);
}
//setterInjection
public void setTransactionManager(PlatformTransactionManager transactionManager) {
this.transactionManager = transactionManager;
}
public void createAndStartTransaction()
{
DefaultTransactionDefinition transDef = new DefaultTransactionDefinition();
transDef.setPropagationBehavior(Propagation.REQUIRES_NEW.ordinal());
if (transactionManager != null)
{
transactionStatus = transactionManager.getTransaction(transDef);
} // if transactionManager null log something went incorrect
}
public void commit() throws Exception
{
if (transactionManager != null && transactionStatus != null)
{
transactionManager.commit(transactionStatus);
}
}
public void rollBack() throws Exception
{
if (transactionManager != null && transactionStatus != null)
{
transactionManager.rollback(transactionStatus);
}
}
}
Finally in the code flow, once the context is defined and using those beans process the message.
Parse the message from a queue
validate the message, check if the metadata information in database, insert the data to the database.
I am trying to persist the data to database immediately at this time
After that the flow will be processing further.
The challange is that when we tried to use the
Below is what I did to persist the data to database. Refer the code snippet.
But this is working when i perform a a testing with single instance.
//....
//.. fetch info from data base using other dao's
//.. insert into another table
// Below code i added where i need to persist the data to database
try{
orderProcess.itemDao.createAndStartTransaction();
orderProcess.itemDao.
}catch(Exception exe){
orderProcess.itemDao.rollBack();
}finally{
//within try catch
orderProcess.commit();
}
//.. other dao's used to fetch the data from different table database
//.. still the process is not completed
When the process try to fetch the next message from queue, it was not able to get the connection and throws connection null exception.
What is observed is the process closes the connection abruptly, so when the process picks the next message it is not having connection defined.
SQL state [null]; error code [0]; Connection is null.; nested exception is java.sql.SQLException: Connection is null.
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84)
Any idea how to persist the transaction independently during the process.
The design is not maintainable, but was able to modify the code for my requirement. Didn't notice any side effect
The DAO call was done from different layer.
I extracted the insert/update/delete to Specific DAO class.
And created a sperate method to call the insert(), etc. in this DAO.
public void checkAndValidate(Object input){
// check data exsits in DB
boolean exists = readDao.checkForData(input);
if(!exists){
// the method which was annotated with transactional
insertDataToDB(input);
}
//.. other process..
}
#Transactional
public Object insertDataToDB(Object data) throws exception {
try{
writeDao.insertData(data);
} catch(Exception exe)
{
//handle exception
}
}
In our application around 1% of the transactions times out daily.
We clearly can see in the log that messages is being processed and sent back using reply-to mechanism but client not receiving the message and wait till the timeout then throws an error.
I'm unable to identify the issue that why these messages getting lost occasionally. Any help much appreciated.
Clint -> Rabbit -> Server -> Rabbit -> Client (timeout - 1% of transactions)
log lines:
client:
2021-11-28 23:03:43.317 WARN <sending msg to vhost <IP> with correlationId: 5cc5c40b-7193-1cf9-c66f-7c5d72901bdf , message properties correlationId: 10976>
Server:
2021-11-28 23:03:43.318 WARN <Received msg with correlationId: 5cc5c40b-7193-1cf9-c66f-7c5d72901bdf , message properties correlationId: 10976>
Server:
2021-11-28 23:03:57.830 WARN <sending msg to vhost <IP> with correlationId: 112f5771-93b9-6f8f-a353-b9a79bdd1438 , message properties correlationId: 10976>
client:
2021-11-28 23:04:43.317 ERROR - <An exception has occured: No reply received from 'assign' with arguments '[]' - perhaps a timeout in the template? org.springframework.remoting.RemoteProxyFailureException: No reply received from 'assign' with arguments '[]' - perhaps a timeout in the template?>
Client and Listener configuration:
<!-- client config -->
<rabbit:queue id="application.queue" name="API.request.queue"
declared-by="rabbitAdminConnectionFactory1"/>
<rabbit:direct-exchange name="#{ API +'.RequestDirectExchange'}" id="requestDirectExchange"
declared-by="rabbitAdminConnectionFactory1">
<rabbit:bindings>
<rabbit:binding queue="application.queue" key="API"/>
</rabbit:bindings>
</rabbit:direct-exchange>
<bean id="amqpTemplate" class="com.api.APIRabbitTemplate">
<property name="connectionFactory" ref="rabbitConnectionFactory"/>
<property name="routingKey" value="API"/>
<property name="exchange" value="#{ API +'.RequestDirectExchange'}"/>
<property name="queue" value="API.request.queue"/>
<property name="messageConverter" ref="rmessageConverter"/>
<property name="replyTimeout" value="60000"/>
<property name="receiveTimeout" value="60000"/>
<property name="retryTemplate" ref="retryTemplate"/>
</bean>
<!-- server listener config -->
<bean id="remotingAmqpTemplate1" class="com.API.APIRabbitTemplate">
<property name="connectionFactory" ref="rabbitConnectionFactory"/>
<property name="messageConverter" ref="rmessageConverter"/>
<property name="queue" value="$app{appName}.request.queue"/>
<property name="retryTemplate" ref="retryTemplate1"/>
</bean>
<bean id="amqpRemotingListener1" class="com.API.remoting.amqpRemotingListener">
<property name="amqpTemplate" ref="remotingAmqpTemplate1"/>
</bean>
<rabbit:listener-container connection-factory="rabbitConnectionFactory" auto-startup="false"
requeue-rejected="false" task-executor="taskExecutor"
concurrency="6" max-concurrency="6"
acknowledge="auto" prefetch="1"
transaction-size="1" channel-transacted="true">
<rabbit:listener ref="amqpRemotingListener1" queue-names="API.request.queue"
admin="rabbitAdminConnectionFactory1"/>
</rabbit:listener-container>
<rabbit:connection-factory id="rabbitConnectionFactory"
thread-factory="tfCommon1"
connection-factory="clientConnectionFactory"
addresses="$amqp{connectionURL1}"
username="$amqp{username}"
password="$amqp{password}"
channel-cache-size="$amqp{listeners.session.cache.size}"/>
<util:properties id="spring.amqp.global.properties">
<prop key="smlc.missing.queues.fatal">false</prop>
</util:properties>
EDIT:
public class ARabbitTemplate extends RabbitTemplate {
protected static Logger log = Logger.getLogger( AARabbitTemplate.class );
#Override
public <T> T execute(ChannelCallback<T> action) {
try {
return super.execute(action);
}catch (AmqpException ex){
//log
throw ex;
}
}
#Override
public void doSend(Channel channel, String exchange, String routingKey, Message message,
boolean mandatory, CorrelationData correlationData) throws Exception {
try {
super.doSend(channel, exchange, routingKey, message, mandatory, correlationData);
if(channel != null && channel.getConnection() != null && channel.getConnection().getAddress() != null) {
String IP = channel.getConnection().getAddress().getHostAddress();
String correlationId = message.getMessageProperties().getHeaders().get(MessagingConsts.CORRELATION_ID);
String message_CorrelationId = message.getMessageProperties().getCorrelationId();
log("sending msg to vhost " + IP + " with correlationId: " + correlationId + " , message properties correlationId: " + message_CorrelationId);
}
} catch (Exception ex) {
log.error("got exception while sending msg to ampq "+ ex.getLocalizedMessage());
throw ex;
}
}
#Override
public <R, S> boolean receiveAndReply(String queueName, ReceiveAndReplyCallback<R, S> callback,
ReplyToAddressCallback<S> replyToAddressCallback) throws AmqpException {
try {
return super.receiveAndReply(queueName, callback, replyToAddressCallback);
} catch (AmqpException amqpEx) {
throw amqpEx;
}
}
#Override
public Message receive() throws AmqpException {
try {
return super.receive();
} catch (AmqpException amqpEx) {
throw amqpEx;
}
}
#Override
public void onMessage(Message message) {
super.onMessage(message);
}
}
public class AAmqpRemotingListener implements MessageListener {
private AmqpTemplate amqpTemplate;
#Autowired
private MessageConverter messageConverter;
#Override
public void onMessage(Message message) {
String correlationId = message.getMessageProperties().getHeaders().get(MessagingConsts.CORRELATION_ID);
String message_CorrelationId = message.getMessageProperties().getCorrelationId();
log("Received msg with correlationId: " + correlationId + " , message properties correlationId: " + message_CorrelationId);
//rest of the code
//exporter.onMessage(message);
}
}
//Message Converter
public class SpringAmqpMessageConverter implements MessageConverter {
#Autowired
protected ObjectSerializer serializer;
#Override
public Message toMessage(Object object, MessageProperties messageProperties) throws MessageConversionException {
byte[] data = serializer.writeObject(object);
// build message
MessageBuilder messageBuilder = MessageBuilder.withBody(data);
String correlationId = context.getCorrelationId();
if (StringUtils.isEmpty(correlationId)) {
correlationId = (new UUID(ThreadLocalRandom.current().nextLong(), ThreadLocalRandom.current().nextLong())).toString();
}
messageBuilder.setHeader(MessagingConsts.CORRELATION_ID, correlationId);
messageBuilder.copyHeaders(messageProperties.getHeaders());
messageBuilder.setCorrelationIdIfAbsent(correlationId);
messageBuilder.setDeliveryModeIfAbsentOrDefault(MessageDeliveryMode.NON_PERSISTENT);
Message message = messageBuilder.build();
return message;
}
#Override
public Object fromMessage(Message message) throws MessageConversionException {
//
}
}
ActiveMQ/Spring experts,
I am running into a very strange problem with ActiveMQ and DefaultMessageListenerContainer/SimpleMessageListenerContainer combination. We have a web application built using Spring (we are at 4.x). One of the transactions is related to processing a bulk upload of files and each file has a number of lines. Each line will become a message for processing.
A publisher publishes each line as a message to a persistent Queue. When examined through the ActiveMQ console we can see the message in the queue. To the same queue, we have a group of listeners configured using DefaultMessageListenerContainer(DMLC)/SimpleMessageListenerContainer(SMLC) (tried both) for consuming messages.
When the publisher publishes 100 messages sometimes only 99 or 98 messages are delivered but the rest are stuck in queue. The configuration is as follows:
ActiveMQ broker is running in standalone mode not networked or embedded in WildFly.
In the Spring application, we tried both DMLC and SMLC but both ran into this issue. Tried simpleMQConnectionFactory as well as PooledConnectionFactory and both times ran into same problem.
Tried setting the prefetch limit to "1" on the PooledConnectionFactory and ran into the same problem. Spring SMLC is throwing an exception when set to "0".
Max concurrent consumers is set to 50
When messages are stuck, if we restart WildFly the remaining messages in the queue are delivered to the consumers.
We are not using transacted sessions rather set the acknowledgModeName = "CLIENT_ACKNOWLEDGE"
We initialize the queue using a spring bean and use that for initializing the SMLC or DMLC
I am running out of options to try at this. If you share your experiences in this regard it is highly appreciated. This application is in production and the problem happens almost every other day sometimes multiple times in a day.
private void publishDMRMessage(DmrDTO p_dmrDTO, long jobID, int numDMRs) {
//Create a DMR message for each of the unique keys and publish it to
try {
DMRImportMessage message = new DMRImportMessage();
message.setDmrDTO(p_dmrDTO);
message.setDmrKey(p_dmrDTO.toString());
message.setDmrImportJobID(new Long(jobID));
message.setTask(Command.INITIALIZE_DMR_FORM);
message.setNumDMRForms(new Long(numDMRs));
sender.sendMessage(message);
} catch (Exception e) {
System.out.println(" JMS Exception = " + e.getMessage());
e.printStackTrace();
}
}
public class DMRMessageListener implements MessageListener {
private DMRImportManager manager;
private JMSMessageSender sender;
private DmrFormInitService formService;
private ProcessDMRValidationMessage validateService;
private ImportDmrService dmrService;
private static final Logger log = Logger.getLogger(DMRMessageListener.class);
public ImportDmrService getDmrService() {
return dmrService;
}
public void setDmrService(ImportDmrService dmrService) {
this.dmrService = dmrService;
}
public ProcessDMRValidationMessage getValidateService() {
return validateService;
}
public void setValidateService(ProcessDMRValidationMessage validateService) {
this.validateService = validateService;
}
public DmrFormInitService getFormService() {
return formService;
}
public void setFormService(DmrFormInitService formService) {
this.formService = formService;
}
public JMSMessageSender getSender() {
return sender;
}
public void setSender(JMSMessageSender sender) {
this.sender = sender;
}
public DMRImportManager getManager() {
return manager;
}
public void setManager(DMRImportManager manager) {
this.manager = manager;
}
public void onMessage(Message message) {
if (message instanceof ObjectMessage) {
try {
ObjectMessage objectMessage = (ObjectMessage) message;
DMRImportMessage dmrMessage = (DMRImportMessage)objectMessage.getObject();
log.info("============= Message Received =========================");
log.info("Message Type = " + dmrMessage.getTask() + " for JOB ID = " + dmrMessage.getDmrImportJobID());
log.info("Message Received === DMR ID = " + dmrMessage.getDmrID());
log.info("Message Received === DMR key = " + dmrMessage.getDmrKey());
log.info("============= Message Received =========================");
//Process the message
processDMRMessage(dmrMessage);
DMRProcessingStatus status = manager.updateStatus(dmrMessage);
if (status.isStatus()) {
log.info(" One stage is complete, the next stage should start for JOB ID = " + dmrMessage.getDmrImportJobID());
publishMessageForNextStepOfProcessing(dmrMessage, status);
}
}
catch (Exception ex) {
ex.printStackTrace();
throw new RuntimeException(ex);
}
}
else {
log.error(" ***** Received an invalid message -- NOT AN Object message so cannot be processed and will result in stuck jobs **** ");
throw new IllegalArgumentException("Message must be of type ObjectMessage");
}
//Send the next message in the chain
}
/**
* It will examine the message content and based on the message type it will invoke the appropriate
* service.
*
* #param dmrMessage DMRImportMessage
*/
private void processDMRMessage(DMRImportMessage dmrMessage) {
if (dmrMessage.getTask() == Command.INITIALIZE_DMR_FORM) {
Map<String, String> dmrInitResults = formService.initDmrForm(dmrMessage.getDmrDTO());
//Indicate in message that this DMR Key is not in ICIS
if (dmrInitResults != null) {
if (StringUtils.equalsIgnoreCase(dmrInitResults.get("wsUnscheduleDmrError"), "truee")) {
log.info("DMR Key is not in ICIS: " + dmrMessage.getDmrDTO().toString());
dmrMessage.setDmrKeyInICIS(false);
} else if (StringUtils.equalsIgnoreCase(dmrInitResults.get("wsDBDown"), "truee")) {
log.error("Web Service call failed for DMR Key: " + dmrMessage.getDmrDTO().toString());
}
}
}
try {
if (dmrMessage.getTask() == Command.IMPORT_DMR_PARAMETER) {
//Process the Parameter line
ParameterProcessingStatus status = dmrService.processLine(dmrMessage.getDmrImportJobID(), dmrMessage.getDmrParameterSubmission(), new Integer(dmrMessage.getLineNumber()), dmrMessage.getDmrKeysNotInICIS());
System.out.println("LINE = " + dmrMessage.getLineNumber() + " Status = " + status.isStatus());
dmrMessage.setProcessingStatus(status.isStatus());
dmrMessage.setDmrID(status.getDmrID());
dmrMessage.setDmrComment(status.getDmrComment());
return;
}
} catch(Exception e) {
log.error("An exception occurred during processing of line " + dmrMessage.getLineNumber() + " in job " + dmrMessage.getDmrImportJobID());
e.printStackTrace();
dmrMessage.setProcessingStatus(false);
dmrMessage.setDmrID(0L);
}
try {
if (dmrMessage.getTask() == Command.END_DMR_PARAMETER_IMPORT) {
//Process the Parameter line
//ParameterProcessingStatus status = dmrService.processLine(dmrMessage.getDmrImportJobID(), dmrMessage.getDmrParameterSubmission(), 100);
dmrMessage.setProcessingStatus(true);
dmrMessage.setDmrID(0L);
return;
}
} catch(Exception e) {
e.printStackTrace();
dmrMessage.setProcessingStatus(false);
dmrMessage.setDmrID(0L);
}
try {
if (dmrMessage.getTask() == Command.POST_PROCESS_DMR) {
//Validate DMRs
validateService.validateDMR(dmrMessage);
}
} catch (Exception e) {
e.printStackTrace();
}
}
private void publishMessageForNextStepOfProcessing(DMRImportMessage dmrMessage, DMRProcessingStatus status) throws JMSException {
log.info(" =========== Publish a message for next step of processing for JOB ID = " + dmrMessage.getDmrImportJobID());
if (dmrMessage.getTask() == Command.INITIALIZE_DMR_FORM) {
//Start the DMR Parameter Processing
sender.sendDMRControlMessage(this.createControlMessage(ProcessPhase.START_PARAMETER_PROCESSING, dmrMessage.getDmrImportJobID(), status.getDmrKeysNotInICIS()));
return;
}
if ((dmrMessage.getTask() == Command.IMPORT_DMR_PARAMETER)
|| (dmrMessage.getTask() == Command.END_DMR_PARAMETER_IMPORT)) {
//Start the DMR Validation Process
dmrService.postProcessParameters(dmrMessage.getDmrImportJobID(), status.getSuccessfulLines(), status.getErroredLines());
DMRImportControlMessage message = this.createControlMessage(ProcessPhase.START_DMR_VALIDATION, dmrMessage.getDmrImportJobID());
message.setDmrIDsWithComments(status.getDmrIDsWithComments());
sender.sendDMRControlMessage(message);
return;
}
if (dmrMessage.getTask() == Command.POST_PROCESS_DMR) {
//Start the next DMR import process
sender.sendDMRControlMessage(this.createControlMessage(ProcessPhase.START_DMR_FORM_INIT, dmrMessage.getDmrImportJobID()));
return;
}
log.info(" =========== End Publish a message for next step of processing for JOB ID = " + dmrMessage.getDmrImportJobID());
}
private DMRImportControlMessage createControlMessage(DMRImportControlMessage.ProcessPhase phase, Long jobID) {
return createControlMessage(phase, jobID, null);
}
private DMRImportControlMessage createControlMessage(DMRImportControlMessage.ProcessPhase p_phase, Long p_jobID, Set<DmrDTO> p_dmrDTOs) {
DMRImportControlMessage message = new DMRImportControlMessage();
message.setDmrImportJobID(p_jobID);
message.setPhase(p_phase);
if (p_dmrDTOs != null) {
message.setDmrKeysNotInICIS(p_dmrDTOs);
}
return message;
}
//Bean Configs.
<bean id="prefetchPolicy" class="org.apache.activemq.ActiveMQPrefetchPolicy">
<property name="queuePrefetch" value="0"/>
</bean>
<bean id="jmsFactoryPub" class="org.apache.activemq.ActiveMQConnectionFactory">
<constructor-arg index="0" value="tcp://localhost:61616" />
</bean>
<bean id="jmsFactoryReceive" class="org.apache.activemq.ActiveMQConnectionFactory">
<constructor-arg index="0" value="tcp://localhost:61616" />
<property name="prefetchPolicy" ref="prefetchPolicy" />
</bean>
<bean id="jmsFactoryControlMsg" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>tcp://localhost:61616</value>
</property>
</bean>
<bean id="dmrQueue"
class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="DMRQueue" />
</bean>
<bean id="dmrControlQueue"
class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="DMRControlQueue" />
</bean>
<bean id="jmsQueueTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="jmsFactoryPub" />
</bean>
<bean id="jmsQueueTemplateControlMsg" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="jmsFactoryControlMsg" />
</bean>
<bean id="messageCreator" class="net.exchangenetwork.netdmr.service.DMRMessageCreator">
</bean>
<bean id="dmrMessageListener" class="net.exchangenetwork.netdmr.service.DMRMessageListener">
<property name="manager" ref="dmrImportManager"/>
<property name="sender" ref="messagePublisher"/>
<property name="formService" ref="dmrFormInit"/>
<property name="validateService" ref="dmrValidator"/>
<property name="dmrService" ref="importDmrService"/>
</bean>
<bean id="messageSender" class="net.exchangenetwork.netdmr.service.JMSMessageSender">
<property name="jmsTemplate" ref="jmsQueueTemplate" />
<property name="sendQueue" ref="dmrQueue" />
<property name="creator" ref="messageCreator" />
</bean>
<bean id="messagePublisher" class="net.exchangenetwork.netdmr.service.JMSMessageSender">
<property name="jmsTemplate" ref="jmsQueueTemplateControlMsg" />
<property name="sendQueue" ref="dmrControlQueue" />
<property name="creator" ref="messageCreator" />
</bean>
<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsFactoryReceive"/>
<!-- this is the queue we will listen on -->
<property name="destination" ref="dmrQueue" />
<property name="messageListener" ref="dmrMessageListener"/>
<property name="concurrentConsumers" value="60"/>
<property name="sessionAcknowledgeModeName" value="CLIENT_ACKNOWLEDGE"/>
<property name="errorHandler" ref="jmsErrorHandler"/>
<property name="exceptionListener" ref="jmsExceptionHandler"/>
<property name="receiveTimeout" value="0"/>
</bean>
This is my client side code :
public class ABCServlet extends HttpServlet {
protected void doGet(HttpServletRequest request,
HttpServletResponse response){
//do blah blah
String msg = null;
java.io.OutputStream os = response.getOutputStream();
java.io.ObjectOutputStream oos = new java.io.ObjectOutputStream(os);
oos.writeObject(msg);
msg = null;
oos.flush();
oos.close();
}
I don't know how using the above code my listener gets kicked off -
public class ABCListener implements MessageListener {
#Override
public void onMessage(Message arg0) {
AbstractJDBCFacade façade = null;
try{
façade = something;
throw new UserException();
}catch(UserException ex){
log.error("ABC Exception " + ex);
}
Configuration :
<bean id="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">....
<bean id="jmsQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
<bean id="listenerContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer102">
I have 3 questions :
1. without putting it on the queue explicitly , how a listener gets invoked?
2. When onMessage method throws UserException, instead of logging I want to pass the message to the client. How can I do that ?
3. Why would someone use JndiObjectFactoryBean instead of ActiveMQ...
JMS by design was supposed to be asynchronous and one-way. Even "synchronous" jms with using receive method of consumer will internally turn into creating a new temporary queue. And here we come to the second point about it's one-way nature. JMS queue was supposed to be one-way and that's why it is called point-to-point (http://www.enterpriseintegrationpatterns.com/patterns/messaging/PointToPointChannel.html). Of course technically with some dancing you will manage to achieve what you want but it is bad practice which will also lead to performance degradation due to the fact that you will need filtering.
To get this thing work fast the best way will be to have exactly one logical receiver (of course you can use concurrent cosumers for one receiver but that should be one logical consumer without any need of filtering the message).
without putting it on the queue explicitly , how a listener gets invoked?
Listener get invoked only when a message comes to a queue. Thats the only way to get it work as it was supposed to work.
In general there are two types of message consuming models: push (also known as event-driven consuming) and poll. In case of using push model all listeners (according to canonical observer pattern) got registered somewhere in the broker and then, when broker receive new message in some queue, it executes listener's method. On the others side in polling model consumer take care itself about receiving messages. So with some interval it comes to a broker and checks the queue for new messages.
Push model: http://www.enterpriseintegrationpatterns.com/patterns/messaging/EventDrivenConsumer.html
Poll model: http://www.enterpriseintegrationpatterns.com/patterns/messaging/PollingConsumer.html
When onMessage method throws UserException, instead of logging I want to pass the message to the client. How can I do that ?
Thats a very bad practice. Of course technically you can achieve it with dirty tricks but thats not the right way of using jms. When onMessage throws the exception then message wont be taken from the queue (of course if u did not reconfigured acknowledge mods or used another tricks). So the best way of solving your probem fmpv is to use redelivery limit on message and a dead letter queue(http://www.enterpriseintegrationpatterns.com/patterns/messaging/DeadLetterChannel.html). If system was not able to process the message after some attempts (redelivery limit shows exactly this) then broker remove message from the queue and send it to a so-called dead letter queue where all failed (from the point of broker) messages are stored. And then client can read that queue and decide what to do with message.
In amq: http://activemq.apache.org/message-redelivery-and-dlq-handling.html
If you want to use so-called "synchronous" features in JMS and really there is no way of using dead letter queue or smth like that then actually you can use consumer.recieve method on the client. But in this case you should send response on every message. In case of success you can send one message and in case of failure error messages. And so a client will be able to understand what is going on. But i dont think that you need such a huge overhead cause actually you need only failure messages. Also in this case you will have to take care about appropriate receive timeouts.
Why would someone use JndiObjectFactoryBean instead of ActiveMQ...
That's cause you are using Spring and there are additional features especially for spring.
PS:
1. For consuming:
How can I send a message using just this piece of code? Don't I need
to put this on a queue? java.io.OutputStream os =
response.getOutputStream(); java.io.ObjectOutputStream oos = new
java.io.ObjectOutputStream(os); oos.writeObject(msg);
For receiving smth like this:
`
<bean id="connectionFactory" class="org.springframework.
jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="baseJNDITemplate"/>
<property name="jndiName"
value="weblogic.jms.ConnectionFactory"/>
</bean>
<bean id="queue" class="org.springframework.
jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="baseJNDITemplate"/>
<property name="jndiName" value="#{properties.queueName}"/>
</bean>
<bean id="messageListenerContainer"
class="org.springframework.jms.listener.
DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destination" ref="queue"/>
<property name="messageListener" ref="messageListener"/>
<property name="sessionTransacted" value="true"/>
</bean>
<bean id="messageListener" class="com.example.ABCListener"/>
And then simply all logic for message processing will be in the listener.
For sending smth like this in config:
<bean id="jmsQueueTemplate"
class="org.springframework.
jms.core.JmsTemplate">
<property name="connectionFactory">
<ref bean="jmsConnectionFactory"/>
</property>
<property name="destinationResolver">
<ref bean="jmsDestResolver"/>
</property>
...
</bean>
<bean id="jmsDestResolver"
class=" org.springframework.jms.support.destination.
JndiDestinationResolver"/>
<bean id="jmsConnectionFactory"
class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jms/myCF"/>
<property name="lookupOnStartup" value="false"/>
<property name="cache" value="true"/>
<property name="proxyInterface" value="amq con fact here"/>
</bean>
and in code simply use jmsTemplate.send(queue, messageCreator) method:
#Autowired
ConnectionFactory connectionFactory;
#Test(enabled = false)
public void testJmsSend(final String msg) throws Exception {
JmsTemplate template = new JmsTemplate(connectionFactory);
template.send("test_queue", new MessageCreator() {
#Override
public Message createMessage(Session session)
throws JMSException {
return session.createTextMessage(msg);
}
});
}
https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/cspr_data_access_jms.html
I believe Dead channel comes in picture only when the message is not properly received by the receiver. In my case, the receiver
received it and processed it, however while processing it failed with
some exception. I want to let the sender know that there was a
exception and the message did not process successfully. I can do this
using a response queue but I don't want to do that, can the receiver
receive a message from the sender on the same queue ? How?
Dead letter channel is a kind of error handling for message processing also. If message processing had failed then after the limit end it got transferred there. It is not actually only for transport issues but also for processing issues. If the message processing got failed with exception then message will stay in the queue and wont be acked by default. So what we should do with this message? For example if it failed due to our database error or smth like this? We should initiate error-handling process, notify assurance systems and stakeholders, collect all necessary info and preserve the message. Due to this kind of queues, which was create d exactly for that, it is much easier. And then customer support team will investigate error queue for further analysis of what has happened. Also we have monitoring tools for notifications and statistics collection on such errors. After understanding what has happened message got removed from the queue and archived.
After processing a message, the consumer is responsible for deleting
the message. If the consumer doesn't delete the message, for example
because because it crashed while processing the message, the message
becomes visible again after the message's Visibility Timeout expires.
Each time this happens, the message's receive count is increased.
When this count reaches a configured limit, the message is placed in a
designated Dead Letter Queue.
http://www.enterpriseintegrationpatterns.com/patterns/messaging/DeadLetterChannel.html
I can do this using a response queue but I don't want to do that, can
the receiver receive a message from the sender on the same queue ?
How?
For you it will look like it's the same queue but internally new temporary queue will be created. To achieve that you should use jms request\reply message pattern. More here: http://activemq.apache.org/how-should-i-implement-request-response-with-jms.html
The only part still confuses me is : If I expect my JMS listener
(receiver) to listen to the queue, then my sender should also
implement JMS and connect to the same queue and send a message. But in
the ABCListener application that I am supporting does not have any
configuration where the sender is configured to the queue. All the
sender does is 3 lines of code : java.io.OutputStream os =
response.getOutputStream(); java.io.ObjectOutputStream oos = new
java.io.ObjectOutputStream(os); oos.writeObject(msg); Literally, that
is it. I don't know how it still works!
Of course 3 lines of code with outputstream do nothing except populating msg string. To send any jms message to the queue you anyway will have to use JMS Api or some library like Spring which wrap it by adding additional features.
I've wrote simple samples to get it more clear.
Modified servlet for asynchronous processing with dead letter queue (for dlq you should create also another listener ofc)
public class AsynchronousJmsSenderServlet extends HttpServlet {
#Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String msg = null;
try (java.io.OutputStream os = response.getOutputStream()) {
try(java.io.ObjectOutputStream oos = new java.io.ObjectOutputStream(os)) {
oos.writeObject(msg);
}
}
sendJmsMessage(msg);
}
private void sendJmsMessage(final String msg) {
ConnectionFactory connectionFactory = null; //here get it in some way from spring
JmsTemplate template = new JmsTemplate(connectionFactory);
template.send("your_queue_name", new MessageCreator() {
#Override
public Message createMessage(Session session)
throws JMSException {
return session.createTextMessage(msg);
}
});
}
}
And here is the code for "synchronous" processing and status reply messages
public class SynchronousJmsSenderServlet extends HttpServlet {
#Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String msg = null;
try (java.io.OutputStream os = response.getOutputStream()) {
try(java.io.ObjectOutputStream oos = new java.io.ObjectOutputStream(os)) {
oos.writeObject(msg);
}
}
sendJmsMessage(msg);
}
private void sendJmsMessage(final String msg) {
ConnectionFactory connectionFactory = null; //here get it in some way from spring
JmsTemplate template = new JmsTemplate(connectionFactory);
Message reply = template.sendAndReceive("your_queue_name", new MessageCreator() {
#Override
public Message createMessage(Session session)
throws JMSException {
return session.createTextMessage(msg);
}
});
if(reply instanceof TextMessage) {
try {
String status = ((TextMessage) reply).getText();
//do error handling if status is error
} catch (JMSException ex) {
throw new RuntimeException("Unable to get status message", ex);
}
} else {
throw new RuntimeException("Only text messages are supported");
}
}
}
public class SynchronousJmsMessageListener implements SessionAwareMessageListener {
#Override
public void onMessage(Message request, Session session) throws JMSException {
try {
//do some processing
sendReply(request, session, "OK");
} catch (Exception ex) {
sendReply(request, session, "Error: " + ex.toString());
}
}
private void sendReply(Message request, Session session, String status) {
try {
TextMessage reply = null; //for example you can use ActiveMQTextMessage here
reply.setJMSCorrelationID(request.getJMSCorrelationID());
reply.setText(status);
MessageProducer producer = session.createProducer(reply.getJMSReplyTo());
producer.send(reply);
} catch (JMSException exception) {
throw new RuntimeException("Unable to send reply", exception);
}
}
}
You will need Spring 5 to have sendAndReceive method on jmsTemplate. Or you will have to do all that manually.
PS1: Please let me know if that will work
I am new to rabbitmq and am trying the following scenario
--> producer sends message
--> consumer receives the message
-- Execute my own logic
if the logic fails - requeue
--> requeue the message if the consumer fails(machine goes down)
I have implemented the basic sender using Spring rabbitTemplate
rabbitTemplate.convertAndSend(.....);
and for consumer i implemented a message listener
public class CustomMessageListener implements MessageListener {
#Override
public void onMessage(Message message) {
//** my own logic**
}
}
and added it to the container through spring
<bean id="aListener" class="com.sample.CustomMessageListener" autowire="byName"/>
<rabbit:listener-container id="myListenerContainer" connection-factory="connectionFactory" acknowledge="auto" prefetch="750" concurrency="5" >
<rabbit:listener ref="aListener" queues="reportQueue"/>
</rabbit:listener-container>
Its working fine till this part.
now if ** my own logic** mentioned in the listener fails. i want to requeue the message. how can i implement this. From the blogs that i have gone through it looks like returnedMessage needs to overridden. But am not sure how it can be done through listener.
With acknowledge="auto", the message won't be ack'd until the listener exits normally, so there's nothing extra that you need to do; if your listener throws an exception or the server crashes, the message will remain in the queue.