We are using WebSphere MQ and mmx , however we are facing issues while trying to connect with queue:
[2/10/12 13:24:51:861 CST] 00000011 SystemOut O 13:24:51,861 INFO [ListenerThread] - Retry [=1] reconnecting to JMS Queue/Topic
[2/10/12 13:24:51:864 CST] 00000012 SystemOut O 13:24:51,864 INFO [ListenerThread] - Retry [=1] reconnecting to JMS Queue/Topic
[2/10/12 13:24:51:874 CST] 00000012 SystemOut O 13:24:51,874 INFO [JMSListener] - init() failed with JMSException during initializing JMS access: xxsvclnk.queue.ISEEOutboundQueue
[2/10/12 13:24:51:875 CST] 00000011 SystemOut O 13:24:51,875 INFO [JMSListener] - init() failed with JMSException during initializing JMS access: xxreqctr.queue.ISEEInboundQueue
Please let us know possible causes for this issue . we have done all the relevant changes (host name:port ) etc.
Per the update in the comments, the apps don't print linked exceptions and you do not have access to fix them. JMS exceptions are a multi-level data structure. The linked exceptions exist to hold vendor-specific diagnostic codes. If your apps do not print the entire JMS exception then the apps should be reported to the vendor or programmers as containing sev-1 defects. There is no possible justification for a JMS application to not print all levels of a multi-level diagnostic data structure. This post is a brilliant example of exactly why such practices are unjustified. Without a stack trace and linked exceptions, there is no data with which to diagnose production problems. The week (if that) saved on delivery of the code will be paid for many times over with an extended outage.
Your only other option here is to do tracing. Exactly what options are available depends on the versions of WMQ client and WMQ server that are installed. You do have enough access and/or support to find that out, yes?
Related
We have an application that uses JMS with Tibco messaging server. It’s implemented with Spring Boot.
Our sessions are transacted sessions and the acknowledge mode is set to auto.
We can receive the messages sent to the queue but for some reason that commit() method fails.
This is the error:
2018-07-13 15:50:35.858 WARN 2576 --- [enerContainer-8] o.s.j.l.DefaultMessageListenerContainer : Execution of JMS message listener failed, and no ErrorHandler has been set.
javax.jms.TransactionRolledBackException: Commit failed
2018-07-13 15:50:35.914 WARN 2576 --- [enerContainer-8] o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'queue' - trying to recover. Cause: Commit failed; nested exception is javax.jms.JMSException: operation='ack' cause='Illegal state' queue='queue'
I should mention two things:
I see the commit() failure and stack trace, only when I run in debug mode. If I let the war to just run, I don’t see any trace of commit failure in the logs. I can tell that we received the messages, but can’t tell whether the commit() failed or not.
If we use the default JMS implementations of Spring Boot which is ActiveMQ, then the commit() does not fail whether its in debug mode or not, it only happens with Tibco.
Does anybody have any idea that why the commit() fails? When the commit fails, the transaction is rolled back and the messages won’t be cleared from the queue.
Why I see this error only in debug mode? OR: it indeed, does happen when I run the war but just not logged?
Below please find the complete stack trace:
15:50:35.858 [DefaultMessageListenerContainer-8] WARN o.s.j.l.DefaultMessageListenerContainer - Execution of JMS message listener failed, and no ErrorHandler has been set.
javax.jms.TransactionRolledBackException: Commit failed
at com.tibco.tibjms.Tibjmsx.buildException(Tibjmsx.java:596)
at com.tibco.tibjms.TibjmsxSessionImp._confirmTransacted(TibjmsxSessionImp.java:3251)
at com.tibco.tibjms.TibjmsxSessionImp._confirm(TibjmsxSessionImp.java:3643)
at com.tibco.tibjms.TibjmsxSessionImp._commit(TibjmsxSessionImp.java:2898)
at com.tibco.tibjms.TibjmsxSessionImp.commit(TibjmsxSessionImp.java:4860)
at org.springframework.jms.support.JmsUtils.commitIfNecessary(JmsUtils.java:218)
at org.springframework.jms.listener.AbstractMessageListenerContainer.commitIfNecessary(AbstractMessageListenerContainer.java:776)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:680)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
at java.lang.Thread.run(Unknown Source)
2018-07-13 15:50:35.914 WARN 2576 --- [enerContainer-8] o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'queue' - trying to recover. Cause: Commit failed; nested exception is javax.jms.JMSException: operation='ack' cause='Illegal state' queue='queue'
2018-07-13 15:50:36.543 INFO 2576 --- [enerContainer-8] o.s.j.l.DefaultMessageListenerContainer : Successfully refreshed JMS Connection
15:50:36.543 [DefaultMessageListenerContainer-8] INFO o.s.j.l.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
This EMS exception happens when some of your messages within that transaction expired in the meantime. Just tried this one myself.
When you run outside of debugging mode, things seem to be quick enough. But when you are in debugging mode, your time before you commit() is much longer. So if your producer's time-to-live expires in the meantime, the commit assumes that something is wrong (EMS knows that there should be e.g. 5 messages in this transaction, but one of them already expired) and throws this exception.
To verify your producer's time-to-live, see the last option of your MessageProducer.send() call or msgProducer.setTimeToLive() and note, this is a time in millseconds, not seconds. Another way to check if this is the issue, is to run in debug mode and be 'really quick' with continuing your thread. That might change your behaviour.
I see tonnes of logs for all header properties that have hyphen -
[30-5-18 17:13:14:321 CEST] 00001622 SystemOut O WARN
[integration.jms.DefaultJmsHeaderMapper] failed to map Message header
'Keep-Alive' to JMS property javax.jms.MessageFormatException:
CWSIA0112E: The property name Keep-Alive is not a valid Java
identifier.
[30-5-18 17:13:14:321 CEST] 00001622 SystemOut O WARN
[integration.jms.DefaultJmsHeaderMapper] failed to map Message header
'Transfer-Encoding' to JMS property javax.jms.MessageFormatException:
CWSIA0112E: The property name Transfer-Encoding is not a valid Java
identifier.
How can I resolve all these errors and warnings
Does a custom header mapper help, any example?
Using Spring boot 1.4.3 and integration version 4.3.6. Application running on websphere default JMS provider 8.5.x
Thanks
I believe that the error means that you can't use dash in the property name.
You need to re-map this Keep-Alive into something like KeepAlive using HeaderEnricher and then remove it using HeaderFilter.
I'm deploying an app that makes use of Spring 4.2.5, Hibernate 4.2.8, and JMS 1.1 on to WebSphere 8.5.5 and Oracle 12.
Resources such as the database connection manager, and JMS connection factory are set in the server and wired into the Spring app using JNDI.
When the app starts I see this in the logs:
[3/18/16 15:18:32:717 EST] 0000008b SystemOut O [B#631dd237/Set;
at org.springframework.jms.annotation.JmsListenerAnnotationBeanPostProcessor$1.inspect(JmsListenerAnnotationBeanPostProcessor.java:202)
at org.springframework.jms.annotation.JmsListenerAnnotationBeanPostProcessor$1.inspect(JmsListenerAnnotationBeanPostProcessor.java:198)
at org.springframework.core.MethodIntrospector$1.doWith(MethodIntrospector.java:72)
at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:495)
at org.springframework.core.MethodIntrospector.selectMethods(MethodIntrospector.java:68)
at org.springframework.jms.annotation.JmsListenerAnnotationBeanPostProcessor.postProcessAfterInitialization(JmsListenerAnnotationBeanPostProcessor.java:197)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsAfterInitialization(AbstractAutowireCapableBeanFactory.java:421)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1559)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
... 58 more
[3/18/16 15:18:32:718 EST] 0000008b SystemOut O [FIAT-CSP-NA] [WebContainer : 0] 2016-03-18 15:18:32,718 [INFO ] org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean: Closing JPA EntityManagerFactory for persistence unit default
As you can see it appears there is some sort of error in the JMSListenerAnnotationBeanPostProcessor, followed by a message indicating that the JPA entity manager is shutting down.
I presume from this that there is a JMS problem which is shutting down the app.
Has anyone seen this? Do you know what might be the issue?
I'm really stuck on this.
The answer turned out to be class path issues. (Isn't everything on Websphere!)
I needed to remove several jars from the war file that where conflicting with jars web sphere supplied.
I have an application deployed on WebSphere ND 7.0.0.21. Within that I have an Oracle XA datasource configured. Recently the application crashed and when I restarted it, I started getting the following exception in SystemOut.log every second.
[11/6/12 13:55:38:650 GMT] 0000000c InternalGener I DSRA8205I: JDBC driver name : Oracle JDBC driver
[11/6/12 13:55:38:651 GMT] 0000000c InternalGener I DSRA8206I: JDBC driver version : 11.2.0.2.0
[11/6/12 13:55:38:661 GMT] 0000000c InternalOracl I DSRA8212I: DataStoreHelper name is: com.ibm.websphere.rsadapter.Oracle11gDataStoreHelper.
[11/6/12 13:55:38:662 GMT] 0000000c WSRdbDataSour I DSRA8208I: JDBC driver type : ""
[11/6/12 13:55:38:685 GMT] 0000000c WSRdbXaResour E DSRA0304E: XAException occurred. XAException contents and details are: The cause is : null.
[11/6/12 13:55:38:837 GMT] 0000000c WSRdbXaResour E DSRA0302E: XAException occurred. Error code is: XAER_RMERR (-3). Exception is: <null>
[11/6/12 13:55:38:837 GMT] 0000000c XARminst E WTRN0037W: The transaction service encountered an error on an xa_recover operation. The resource was com.ibm.ws.rsadapter.spi.WSRdbXaResourceImpl#7b867b86. The error code was XAER_RMERR. The exception stack trace follows: javax.transaction.xa.XAException
at oracle.jdbc.xa.OracleXAResource.recover(OracleXAResource.java:709)
at com.ibm.ws.rsadapter.spi.WSRdbXaResourceImpl.recover(WSRdbXaResourceImpl.java:1088)
at com.ibm.ws.Transaction.JTA.XARminst.recover(XARminst.java:141)
at com.ibm.ws.Transaction.JTA.XARecoveryData.recover(XARecoveryData.java:626)
at com.ibm.tx.jta.PartnerLogTable.recover(PartnerLogTable.java:389)
at com.ibm.tx.jta.RecoveryManager.resync(RecoveryManager.java:1530)
at com.ibm.tx.jta.RecoveryManager.performResync(RecoveryManager.java:2265)
at com.ibm.ws.tx.jta.RecoveryManager.performResync(RecoveryManager.java:114)
at com.ibm.tx.jta.RecoveryManager.run(RecoveryManager.java:2218)
at java.lang.Thread.run(Thread.java:736)
This seems to be a standard problem when appropriate permissions have not been granted on the database, to allow the db user access to the transaction requiring recovery - http://www-01.ibm.com/support/docview.wss?uid=swg21196663
However, I then ran the SQL on that page and restarted the application server, but the problem persisted. The issue only went away when I removed the transaction logs (described as a workaround on the IBM page).
Why would granting the permissions not fix the problem? Can something else cause the XAER_RMERR problem?
I would suggest to enable the traces as described in the relevant MustGather (http://www-01.ibm.com/support/docview.wss?uid=swg21153216) in order to see the actual error returned by the database.
removing transaction logs and partnerlogs solves the issue. of course one must be cautious to do it in a production environment with complex transaction schemes, yet without that WebSphere is unable to recover them itself.
I have a fresh install of Wesphere 6.1 Fixpack 23. I have an app deployed that requires an MQSeries JMS Queue. I set up an MQSeries provider-based request and reply queue and an MQSeries provider-based queue connection factory. When the deployed app tries to access the queue, I receive the following error.
Any assistance would be appreciated. Thanks!
[5/28/09 10:33:42:538 EDT] 00000033 ServletWrappe E SRVE0068E: Uncaught exception thrown in one of the service methods of the servlet: espaapp. Exception thrown : org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.UnsatisfiedLinkError: mqjbnd05 (Not found in java.library.path)
Caused by: java.lang.UnsatisfiedLinkError: mqjbnd05 (Not found in java.library.path)
at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:953)
at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:922)
at java.lang.System.loadLibrary(System.java:451)
at com.ibm.mq.MQSESSION.loadLib(MQSESSION.java:1028)
at com.ibm.mq.server.MQSESSION$1.run(MQSESSION.java:246)
at java.security.AccessController.doPrivileged(AccessController.java:192)
at com.ibm.mq.server.MQSESSION.(MQSESSION.java:243)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:177)
at com.ibm.mq.MQSESSIONServer.getMQSESSION(MQSESSIONServer.java:68)
at com.ibm.mq.MQSESSION.getSession(MQSESSION.java:508)
at com.ibm.mq.MQManagedConnectionJ11.(MQManagedConnectionJ11.java:213)
at com.ibm.mq.MQBindingsManagedConnectionFactoryJ11._createManagedConnection(MQBindingsManagedConnectionFactoryJ11.java:186)
at com.ibm.mq.MQBindingsManagedConnectionFactoryJ11.createManagedConnection(MQBindingsManagedConnectionFactoryJ11.java:225)
at com.ibm.mq.StoredManagedConnection.(StoredManagedConnection.java:84)
at com.ibm.mq.MQSimpleConnectionManager.allocateConnection(MQSimpleConnectionManager.java:173)
at com.ibm.mq.MQQueueManagerFactory.obtainBaseMQQueueManager(MQQueueManagerFactory.java:795)
at com.ibm.mq.MQQueueManagerFactory.procure(MQQueueManagerFactory.java:709)
at com.ibm.mq.MQQueueManagerFactory.constructQueueManager(MQQueueManagerFactory.java:664)
at com.ibm.mq.MQQueueManagerFactory.createQueueManager(MQQueueManagerFactory.java:160)
at com.ibm.mq.MQQueueManager.(MQQueueManager.java:550)
at com.ibm.mq.MQSPIQueueManager.(MQSPIQueueManager.java:62)
at com.ibm.mq.jms.MQConnection.createQM(MQConnection.java:2427)
at com.ibm.mq.jms.MQConnection.createQMXA(MQConnection.java:1806)
at com.ibm.mq.jms.MQQueueConnection.(MQQueueConnection.java:105)
at com.ibm.mq.jms.MQQueueConnection.(MQQueueConnection.java:66)
at com.ibm.mq.jms.MQXAQueueConnection.(MQXAQueueConnection.java:59)
at com.ibm.mq.jms.MQXAQueueConnectionFactory.createXAQueueConnection(MQXAQueueConnectionFactory.java:82)
at com.ibm.ejs.jms.JMSManagedQueueConnection.createConnection(JMSManagedQueueConnection.java:123)
at com.ibm.ejs.jms.JMSManagedConnection.(JMSManagedConnection.java:315)
at com.ibm.ejs.jms.JMSManagedQueueConnection.(JMSManagedQueueConnection.java:71)
... More
Does this help?
java.lang.UnsatisfiedLinkError occurs when connecting to a queue manager
Also, within the JMS -> Queue Connection Factories section, select your Queue Connection Factory and check if your "Transport Type" is set to 'BINDINGS' or 'CLIENT'
I swapped mine to CLIENT and that seemed to help a lot.
Such error often happens as a result of passing null to port, host or QManager to connection factory. Try to check all parameters during execution. Normally MQ does not require mqjbnd05 library.
Try to find this file
libmqjbnd05.so
Add that to the LIBPATH for your JVM and try again.
GO to WebSphere Admin console. Environment -> WebSphere variables. Look for MQ_INSTALL_ROOT and modify its value to your MQ installation directory [MQ Home].