Camel jms to external activeMQ with request-reply gives javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: - jms

We have third party applications that make restful calls to camel cxfrs endpoint which is then routed to an external activeMQ. There are applications consuming these JMS messages and provide XML responses. This is all done synchronously using camel InOut exchangePattern. The architecture is very straight forward and simple. We were using activeMQ 5.5.0-fuse, camel-jms 2.8.x and activemq-pool 5.6.
Using this configuration, we see this exception at random times:
javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp- queue://ID:testserver-37266-1366126830205-0:0:1
at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1696)
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231)
at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:74)
at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:55)
When this happens the server just stalls and none of our services respond until we restart activeMQ, tomcat and all other services.
camel config:
<import resource="classpath:META-INF/cxf/cxf.xml"/>
<bean id="routeBuilder" class="gov.nasa.arc.tmi.route.TMIServiceRoute"/>
<bean id="jaxbProvider" class="org.apache.cxf.jaxrs.provider.JAXBElementProvider">
<property name="marshallerProperties" ref="propertiesMap"/>
</bean>
<util:map id="propertiesMap">
<entry key="jaxb.formatted.output">
<value type="java.lang.Boolean">true</value>
</entry>
</util:map>
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<routeBuilder ref="routeBuilder"/>
</camelContext>
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://localhost:61616"/>
Camel Router class:
//reroute
from("cxfrs:/rr?resourceClasses=x.y.z.route.RerouteResource")
.setExchangePattern(ExchangePattern.InOut)
.process(new RerouteProcessor())
.to("activemq:queue:x.y.z.tmi.request");
Here is the spring configuration of the application that listens on queue:x.y.z.tmi.request consumes JMS messages:
<context:annotation-config/>
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>tcp://localhost:61616? wireFormat.maxInactivityDurationInitalDelay=30000</value>
</property>
</bean>
<bean id="pooledConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="8" />
<property name="connectionFactory" ref="connectionFactory" />
</bean>
<bean id="destination" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="gov.nasa.arc.tmi.request"/>
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="pooledConnectionFactory"/>
<property name="defaultDestination" ref="destination" />
</bean>
<jms:listener-container connection-factory="pooledConnectionFactory" concurrency="10">
<jms:listener destination="gov.nasa.arc.tmi.request" ref="tmiQueryListener" />
After googling, I came across these bugs:
https://issues.apache.org/jira/browse/CAMEL-6229
https://issues.apache.org/jira/browse/AMQ-3457
Based on these, we upgraded to camel 2.10.4, activeMq 5.7 and activemq-pool 5.7. Even so, the problem remains.
I am really stuck and don't know how to resolve this issue. Can some one kindly point out what could be wrong?
Thanks.

I wonder if its because the processing the JMS message on the other side takes too long, and then the inactive monitor of ActiveMQ deletes the temp destination as its been inactive for > 30 seconds. http://activemq.apache.org/activemq-inactivitymonitor.html
Maybe try to set the timeout to a higher value, or disable it.
Another option is to use fixed queues for reply queues instead of temp queues.

Related

ActiveMQ 5.15.5, Camel ActiveMQ component brokerURL to connect two standalone brokers

I have two standalone broker consuming message from another system.
The client consumer uses Camel to consume messages, below is the part of the configuration where it used camel-activemq component.
I am trying to provide both the host information within the borkerURL but throwing some exception.
using jars of
spring-core 4.2.4,
spring-boot: 1.3.1,
activemq-camel: 5.12.1,
activemq-broker: 5.12.1, etc.
...
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="(tcp://host1:61616,tcp://host2:61616)?wireFormat.maxInactivityDuration=500000" />
</bean>
<bean id="jmsPooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="10" />
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="jmsPooledConnectionFactory" />
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
</bean>
<bean id="jms" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig" />
<property name="transacted" value="true" />
</bean>
...
The process throw exception, when i run the camel main with the context.
:PropertyAccessException 1: org.springframework.beans.MethodInvocationException: Property 'brokerURL' threw exception; nested exception is java.lang.IllegalArgumentException: Invalid broker URI: (tcp://host1:61616,tcp://host2:61616)?wireFormat.maxInactivityDuration=500000
Is there any way to pass in the borker URL of two standalone brokers?
If the brokers are part of a failover system, use the "failover" protocol of the ActiveMQ client.
If the brokers have different queues, create two different commponents with different configurations and broker URLs.
refer to jms1 or jms2 in your route.

Bean definition example for JndiDestinationResolver using jndi look up

I'm working on sending and receiving messages to/from an IBM MQ queue using JmsTemplate. My application is installed on a WebSphere application server 8.0 and, in order to retrieve the connection, I use a jndi lookup.
I have 6 queues from where I need to pick/drop xmls depending on scenarios. Also I have added these queue in WAS. I need help to understand two things:
I should use DynamicDestinationResolver or JndiDestinationResolver?
As per my understanding I should use JndiDestinationResolver; if that is right how I can define that in my context file and refer jndi-lookup for each queue so that I can retrieve it from my code while using jmsTemplate send/receive?
Please see below my application context file:
<bean id="jmsQueueConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jms/CPC.TapQueueConnCPC" />
<property name="lookupOnStartup" value="false" />
<property name="cache" value="true" />
<property name="proxyInterface" value="javax.jms.QueueConnectionFactory" />
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="jmsQueueConnectionFactory" />
<property name="receiveTimeout" value="10000" />
<property name="sessionAcknowledgeMode" value="1" />
<property name="destinationResolver" ref="jmsDestResolver"/>
</bean>
<bean id="fileTransferServiceImpl" class="org.kp.cpc.service.FileTransferServiceImpl" >
<constructor-arg name="jmsTemplate" ref="jmsTemplate" />
</bean>
<bean id="jmsDestResolver" class=" org.springframework.jms.support.destination.JndiDestinationResolver"/>
<jee:jndi-lookup id="drop278" jndi-name="jms/CPC.SEND.AUTHREQ278" />
<jee:jndi-lookup id="drop275" jndi-name="jms/CPC.SEND.AUTHREQ275" />
<jee:jndi-lookup id="recev278" jndi-name="jms/CPC.RECE.AUTHREQ278" />
<jee:jndi-lookup id="recev275" jndi-name="jms/CPC.RECE.AUTHREQ275" />
<jee:jndi-lookup id="preAuthStatus" jndi-name="jms/CPC.RECE.PREAUTH.STSUPD278"/>
<jee:jndi-lookup id="succ278" jndi-name="jms/CPC.RECE.SUCC.AUTHRESP278" />
The whole point of the JndiDestinationResolver is that you don't need to do manual lookups. In other words when using the JndiDestinationResolver you don't need the <jee:jndi-lookup /> as that is al handled by the DestinationResolver.
The name of the destination would be the JNDI name. So in your JMS code you would use the following.
jmsTemplate.convertAndSend("jms/CPC.SEND.AUTHREQ278", "Your-Message-Here");
The JndiDestinationResolver will use the name of the destination to do a JNDI lookup.
If you really want to keep the JNDI names out of your code and want to use the <jee:jndi-lookup /> then use the [BeanFactoryDestinationResolver]. This will use the name of the destination to lookup a bean from the BeanFactory (in this case the ApplicationContext). Your JMS code would then point to the bean name instead of the JNDI name.
jmsTemplate.convertAndSend("drop278", "Your-Message-Here");
So which one to use depends on your preferences.

DefaultMessageListenerContainer recoveryInterval with specific retry count

We are using Spring Integration in our project and we have a requirement where If IBM MQ goes down then we will have to auto connect to IBM MQ when it is up. We have done this implementation using recoveryInterval option of org.springframework.jms.listener.DefaultMessageListenerContainer class. We have given recovery interval as 6 seconds so every 6 seconds system try to recover the MQ connection but now we have a requirement where we will have to do the autorecover twice only and after that if still MQ is down then stop the inbound adapter.
Is there any way in Spring Integration to mention the auto recovery retry count so that system will try to recover only for that retry count?
Below is my existing configuration.
<bean id="inQ" class="com.ibm.mq.jms.MQQueue">
<constructor-arg value="${mq.inbound.queue}" />
</bean>
<int:channel id="inbound" />
<int-jms:message-driven-channel-adapter
id="jmsIn" channel="inbound" container="messageListenerContainer"
acknowledge="transacted" auto-startup="false">
</int-jms:message-driven-channel-adapter>
<int:service-activator id="mainService"
input-channel="inbound" ref="messageListener" method="onMessage">
<int:request-handler-advice-chain>
<ref bean="retryWithBackoffAdviceSession" />
</int:request-handler-advice-chain>
</int:service-activator>
<bean id="messageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="mqConnectionFactory" />
<property name="destination" ref="inQ" />
<property name="sessionTransacted" value="true" />
<property name="maxConcurrentConsumers" value="${maxConcurrentConsumers}" />
<property name="concurrentConsumers" value="${concurrentConsumers}" />
<property name="receiveTimeout" value="${receiveTimeout}" />
<property name="recoveryInterval" value="60000" />
<property name="autoStartup" value="${autoStartup}" />
</bean>
Thanks
Sach
As an alternative to the recoveryInterval, you can now specify a Backoff instead (see the docs).
It doesn't provide a mechanism to stop the container but an appropriate backoff can effectively do what you want.
You would then need to programmatically stop/start to kick it off again.

Use ActiveMQ in Travis CI

My build in Travis CI failed after add configuration on Active MQ.
The log of Travis CI shows:
Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.ConnectException: Connection refused
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:360)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:305)
at org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:245)
at org.springframework.jms.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:365)
at org.springframework.jms.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:305)
at org.springframework.jms.connection.SingleConnectionFactory.getConnection(SingleConnectionFactory.java:283)
at org.springframework.jms.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:224)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:180)
at org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContainer.java:413)
at org.springframework.jms.listener.AbstractJmsListeningContainer.establishSharedConnection(AbstractJmsListeningContainer.java:381)
at org.springframework.jms.listener.AbstractJmsListeningContainer.doStart(AbstractJmsListeningContainer.java:285)
at org.springframework.jms.listener.SimpleMessageListenerContainer.doStart(SimpleMessageListenerContainer.java:209)
at org.springframework.jms.listener.AbstractJmsListeningContainer.start(AbstractJmsListeningContainer.java:270)
... 44 more
I'm using Spring MVC, here's the configuration for testing:
<beans "Some DTDs here">
<!-- Base Components -->
<context:annotation-config />
<context:component-scan base-package="com.trunkshell.voj" />
<!-- Some other configuration for JDBC and other service -->
<!-- Message Service -->
<bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<constructor-arg index="0" value="tcp://localhost:61616"/>
</bean>
<bean id="connectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<constructor-arg ref="amqConnectionFactory" />
</bean>
<bean id="defaultDestination" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="vojMessageQueue"/>
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory" />
<property name="defaultDestination" ref="defaultDestination" />
</bean>
<bean class="org.springframework.jms.listener.SimpleMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory" />
<property name="destinationName" value="vojMessageQueue" />
<property name="messageListener" ref="messageReceiver" />
</bean>
<bean id="messageSender" class="com.trunkshell.voj.util.MessageSender"></bean>
<bean id="messageReceiver" class="com.trunkshell.voj.util.MessageReceiver"></bean>
</beans>
I also noticed that Travis CI only support RabbitMQ and ZeroMQ.
Here's the reference: link
So what should I do to fix the error in Travis CI?
Or how to skip this problem in Travis CI.
I don't see that you actually start an ActiveMQ broker at port 61616. Either, do that, or preferably use the VM transport during the integration tests. I.e. change tcp://localhost:61616 to vm://localhost?broker.persistent=false
Of course, you have to be able to change that ActiveMQ URL when you deploy to a production environment. Maybe you can use spring property placeholders?

Camel JMS Component with Spring on Weblogic Server

I could not found any decent, clear example to learn JMS component with Spring configuration. So I wrote something like this:
<bean id="weblogic" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
</bean>
<bean id="jmsJndiTemplate" class="org.springframework.jndi.JndiTemplate">
<property name="environment">
<props>
<prop key="java.naming.factory.initial">weblogic.jndi.WLInitialContextFactory</prop>
<prop key="java.naming.provider.url">t3://${ip}:${port}</prop>
</props>
</property>
</bean>
<bean id="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="jmsJndiTemplate"/>
<property name="jndiName" value="jms/cdrPreMO-connfact-jndi"/>
</bean>
<camelContext id="camelContext" xmlns="http://camel.apache.org/schema/spring">
<camel:endpoint id="jmsQueue" uri="weblogic:queue:jms/cdrPreMO-queue-jndi"/>
<route>
<from ref="jmsQueue"/>
<bean ref="test" method="writeFile"/>
<to uri="log:errors?level=ERROR"/>
</route>
</camelContext>
But it gives the exception :
Caused by: javax.naming.NameNotFoundException: Unable to resolve 'weblogic.jms.backend.jms'. Resolved 'weblogic.jms.backend'; remaining name 'jms'
at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException(BasicNamingNode.java:1139)
at weblogic.jndi.internal.BasicNamingNode.lookupHere(BasicNamingNode.java:252)
at weblogic.jndi.internal.ServerNamingNode.lookupHere(ServerNamingNode.java:182)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:206)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:214)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:214)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:214)
at weblogic.jndi.internal.WLEventContextImpl.lookup(WLEventContextImpl.java:254)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:393)
at weblogic.jms.frontend.FEManager.destinationCreate(FEManager.java:287)
... 8 more
JMS Conf is true(ip port and jndi names). It is getting the connection, and the session but gives the exception
Could someone tell me what are my mistakes?
Thanx
EDIT: fix applied according to the comment of Claus.
The Camel endpoints should refer to "weblogic" as the component name.
<camel:endpoint id="jmsQueue" uri="jms:queue:jms/cdrPreMO-queue-jndi"/>
Should be
<camel:endpoint id="jmsQueue" uri="weblogic:queue:jms/cdrPreMO-queue-jndi"/>
As that is the name you gave the JMS component in the id attribute of the < bean > tag, eg
<bean id="weblogic" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
</bean>
Solved but with a workaround. I still do not know what the problem really is but, instead of weblogic create its own destination by itself, I gave the responsibility to spring like:
<bean id="weblogic" class="org.apache.camel.component.jms.JmsComponent">
<property name="configuration" ref="jmsConfiguration" />
</bean>
<bean id="jmsJndiTemplate" class="org.springframework.jndi.JndiTemplate">
<property name="environment">
<props>
<prop key="java.naming.factory.initial">weblogic.jndi.WLInitialContextFactory</prop>
<prop key="java.naming.provider.url">${ip}:${port}</prop>
</props>
</property>
</bean>
<bean id="jndiDestinationResolver" class="org.springframework.jms.support.destination.JndiDestinationResolver">
<property name="jndiTemplate" ref="jmsJndiTemplate"/>
</bean>
<bean id="jmsConfiguration" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
<property name="destinationResolver" ref="jndiDestinationResolver"/>
</bean>
<bean id="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="jmsJndiTemplate"/>
<property name="jndiName" value="jms/cdrPreMO-connfact-jndi"/>
</bean>
It is working now.
Although question is pretty old, posting this solution for other users if they get this error. I ran into the this issue on Camel 2.19 version while trying to look up weblogic queue in weblogic 12c version. The lookup that happens here is most likely not a typical JNDI lookup when resolving queue destination in Weblogic.
The end point uri should contain the queue details as follows :
jms_server_name/jms_module_name!queue_name
For example if queue name is TestRequestQueue, created under a module named sample_jms_module and the JMS server is named sample_jms_server in Weblogic, then the uri would need to be as follows :
<camel:endpoint id="jmsQueue" uri="weblogic:queue:sample_jms_server/sample_jms_module!TestRequestQueue"/>
In the prefix portion of the uri weblogic:queue:, the component name can be anything as long as it refers to the created component. It needn't necessarily be weblogic. For example, if the JmsComponent is named as test-jms, the uri will need to be declared with prefix as test-jms:queue: .
Name of the JMS server in weblogic can be obtained from Weblogic console by navigating to the below path :
Services -> Messaging -> JMS Servers
Name of the queue, not the JNDI name, can be obtained from Weblogic console by navigating to the queue and clicking on Monitoring tab. This tab shows the destination name of the queue in the name column.
I've just run apache camel jms component in weblogic 12c with next configuration. The key is the jndi name for the queue.
It must be ./module_name!queueName. './' means local jms server.
<jee:jndi-lookup id="connectionFactoryBean" jndi-name="dpxcomConnectionFactory"/>
<bean id="dpxcomJMSConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<constructor-arg ref="connectionFactoryBean"/>
<property name="sessionCacheSize" value="50"/>
</bean>
Route definition:
from("jms:queue:./DPXCOM!"+config.getMainQueueName()+"?concurrentConsumers="+config.getConcurrentConsumers()
+"&maxConcurrentConsumers="+config.getConcurrentConsumers()+"&jmsMessageType=Text&testConnectionOnStartup=true&connectionFactory=dpxcomJMSConnectionFactory&consumerType=Simple")
.setHeader("jmsMessage").body()
.process(new VerifyJMSMessageProcessor())
.choice()
..
..
end();

Resources