I have a spring-boot service that reads and writes to the same IBM MQ message broker. The process is stand-alone and does not run inside an application container. I want to implement the "Shared Transaction Resource" pattern so that I don't need to configure any JTA/XA transaction management. I have the happy path working, however the following edge case is not rolling-back the message publishing. The read is rolled-back, but the publish is still committed.
Given the MessageListener receives a message
And the message is published to another queue using the same JMS ConnectionFactory
When an Exception is thrown in onMessage() after the messages is published
Then the message is rollback onto the READ queue and is not published to the WRITE queue
My code looks like this...
#Component
public class MyJmsReceiver implements MessageListener
{
#Autowired MyJmsSender myJmsSender;
#Override
public void onMessage(Message message)
{
myJmsSender.sendMessage("some-payload");
if(true) throw new RuntimeException("BOOM!");
}
}
#Component
public class MyJmsSender
{
#Transactional(propagation = Propagation.MANDATORY)
public void sendMessage(final String payload)
{
jmsTemplate.convertAndSend("QUEUE.OUT", payload);
}
}
#Configuration
#EnableJms
#EnableTransactionManagement
public class Config
{
#Bean
public JmsTemplate jmsTemplate(ConnectionFactory connectionFactory)
{
// using a SingleConnectionFactory gives us one reusable connection rather than opening a new one for each message published
JmsTemplate jmsTemplate = new JmsTemplate(new SingleConnectionFactory(connectionFactory));
jmsTemplate.setSessionTransacted(true);
return jmsTemplate;
}
#Bean
public DefaultMessageListenerContainer defaultMessageListenerContainer(
ConnectionFactory connectionFactory,
PlatformTransactionManager transactionManager,
MyJmsReceiver myJmsReceiver)
{
DefaultMessageListenerContainer dmlc = new DefaultMessageListenerContainer();
dmlc.setConnectionFactory(connectionFactory);
dmlc.setSessionAcknowledgeMode(Session.SESSION_TRANSACTED);
dmlc.setSessionTransacted(true);
dmlc.setTransactionManager(transactionManager);
dmlc.setConcurrency(concurrency);
dmlc.setDestinationName("QUEUE.IN");
dmlc.setMessageListener(myJmsReceiver);
return dmlc;
}
#Bean
public PlatformTransactionManager transactionManager(ConnectionFactory connectionFactory) {
return new JmsTransactionManager(connectionFactory);
}
#Bean
public ConnectionFactory connectionFactory(
#Value("${jms.host}") String host,
#Value("${jms.port}") int port,
#Value("${jms.queue.manager}") String queueManager,
#Value("${jms.channel}") String channel
) throws JMSException
{
MQConnectionFactory ibmMq = new MQConnectionFactory();
ibmMq.setHostName(host);
ibmMq.setPort(port);
ibmMq.setQueueManager(queueManager);
ibmMq.setTransportType(WMQConstants.WMQ_CM_CLIENT);
ibmMq.setChannel(channel);
return ibmMq;
}
}
When I enable the logging of the JmsTransactionManager I see that the publish is "Participating in existing transaction", no new txn is created, and the DMLC has rolled back the transaction. However I still see the messages as having been published, while the read message is placed back onto the queue.
2020-09-07_13:21:33.000 [defaultMessageListenerContainer-1] DEBUG o.s.j.c.JmsTransactionManager - Creating new transaction with name [defaultMessageListenerContainer]: PROPAGATION_REQUIRED,ISOLATION
_DEFAULT
2020-09-07_13:21:33.015 [defaultMessageListenerContainer-1] DEBUG o.s.j.c.JmsTransactionManager - Created JMS transaction on Session [com.ibm.mq.jms.MQQueueSession#6934ab89] from Connection [com.ibm.mq.jms.MQQueueConnection#bd527da]
2020-09-07_13:21:33.034 [defaultMessageListenerContainer-1] INFO c.l.c.c.r.MyJmsReceiver - "Read message from QUEUE.IN for messageId ID:414d51204c43482e434c4b2e545354205f49ea352c992702
2020-09-07_13:21:33.054 [defaultMessageListenerContainer-1] DEBUG o.s.j.c.JmsTransactionManager - Participating in existing transaction
2020-09-07_13:21:33.056 [defaultMessageListenerContainer-1] INFO c.l.c.c.p.r.MyJmsSender - Sending message to queue: QUEUE.OUT
2020-09-07_13:21:33.077 [defaultMessageListenerContainer-1] ERROR c.l.c.c.r.MyJmsReceiver - Failed to process messageId: ID:414d51204c43482e434c4b2e545354205f49ea352c992702 with RuntimeException: BOOM!
2020-09-07_13:21:33.096 [defaultMessageListenerContainer-1] WARN o.s.j.l.DefaultMessageListenerContainer - Execution of JMS message listener failed, and no ErrorHandler has been set.
com.xxx.receive.MessageListenerException: java.lang.RuntimeException: BOOM!
at com.xxx.MyJmsReceiver.onMessage(MyJmsReceiver.java:83)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:245)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: BOOM!
at com.xxx.MyJmsReceiver.onMessage(MyJmsReceiver.java:74)
... 9 common frames omitted
2020-09-07_13:21:33.097 [defaultMessageListenerContainer-1] DEBUG o.s.j.c.JmsTransactionManager - Transactional code has requested rollback
2020-09-07_13:21:33.097 [defaultMessageListenerContainer-1] DEBUG o.s.j.c.JmsTransactionManager - Initiating transaction rollback
2020-09-07_13:21:33.097 [defaultMessageListenerContainer-1] DEBUG o.s.j.c.JmsTransactionManager - Rolling back JMS transaction on Session [com.ibm.mq.jms.MQQueueSession#6934ab89]
2020-09-07_13:21:33.107 [defaultMessageListenerContainer-1] DEBUG o.s.j.c.JmsTransactionManager - Creating new transaction with name [defaultMessageListenerContainer]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
2020-09-07_13:21:33.123 [defaultMessageListenerContainer-1] DEBUG o.s.j.c.JmsTransactionManager - Created JMS transaction on Session [com.ibm.mq.jms.MQQueueSession#8d93093] from Connection [com.ibm.mq.jms.MQQueueConnection#610b3b42]
Is there a way to get this working without implementing a formal XA library like Atomikos?
My understanding is that a ChainedTransactionManager won't solve my problem either because once the inner transaction is committed (i.e. the publish) the outer transaction is unable to roll back that commit.
The publish of the message is the practically last thing that onMessage() executes.
Defining the SingleConnectionFactory in the JmsTemplate is the problem. You will get a new connection and therefore a new session in the sender, which makes reusing the running transaction from the listener impossible.
Use a CachingDestinationResolver instead of the SingleConnectionFactory to improve performance:
#Bean
public CachingDestinationResolver cachingDestinationResolver()
{
JndiDestinationResolver destinationResolver = new JndiDestinationResolver();
destinationResolver.setFallbackToDynamicDestination(true);
return destinationResolver;
}
#Bean
public JmsTemplate jmsTemplate(ConnectionFactory connectionFactory,
CachingDestinationResolver destinationResolver)
{
JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory);
jmsTemplate.setDestinationResolver(destinationResolver);
jmsTemplate.setSessionTransacted(true);
return jmsTemplate;
}
#Bean
public DefaultMessageListenerContainer defaultMessageListenerContainer(
ConnectionFactory connectionFactory,
PlatformTransactionManager transactionManager,
MyJmsReceiver myJmsReceiver,
CachingDestinationResolver destinationResolver)
{
DefaultMessageListenerContainer dmlc = new DefaultMessageListenerContainer();
dmlc.setConnectionFactory(connectionFactory);
dmlc.setSessionAcknowledgeMode(Session.SESSION_TRANSACTED);
dmlc.setSessionTransacted(true);
dmlc.setTransactionManager(transactionManager);
dmlc.setConcurrency(concurrency);
dmlc.setDestinationName("MY.QUEUE.IN");
dmlc.setDestinationResolver(destinationResolver);
dmlc.setMessageListener(myJmsReceiver);
return dmlc;
}
Related
I am trying to use Spring integration to connect to JMS client , but i am getting :-
[WARN ] 2018-08-22 10:57:20.378 [DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D514353414D5030303144202020206CF77A5B9E4A5E21]]] SimpleMessageListenerContainer - Execution of JMS message listener failed, and no ErrorHandler has been set.
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'app-name:local:9010.inputChannel'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers, failedMessage=GenericMessage
and below is my spring integration configuration class
Any idea why i am getting this exception .
Many thanks in advance
The problem is exactly what the message says.
Dispatcher has no subscribers for channel 'app-name:local:9010.inputChannel'.
You have no subscriber on this bean
#Bean
public MessageChannel inputChannel() {
return new DirectChannel();
}
EDIT
#ServiceActivator(inputChannel = "inputChannel")
public void handle(String in) {
...
}
or
#Transformer(inputChannel = "inputChannel", outputChannel = "transformed")
public String handle(String in) {
retur in.toUpperCase();
}
#ServiceActivator(inputChannel = "transformed")
public void handle(String in) {
...
}
I have my JMS configuration like below (Spring boot 1.3.8);
#Configuration
#EnableJms
public class JmsConfig {
#Autowired
private AppProperties properties;
#Bean
TopicConnectionFactory topicConnectionFactory() throws JMSException {
return new TopicConnectionFactory(properties.getBrokerURL(), properties.getBrokerUserName(),
properties.getBrokerPassword());
}
#Bean
CachingConnectionFactory connectionFactory() throws JMSException {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(topicConnectionFactory());
connectionFactory.setSessionCacheSize(50);
return connectionFactory;
}
#Bean
JmsTemplate jmsTemplate() throws JMSException {
JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory());
jmsTemplate.setPubSubDomain(Boolean.TRUE);
return jmsTemplate;
}
#Bean
DefaultJmsListenerContainerFactory defaultContainerFactory() throws JMSException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setPubSubDomain(Boolean.TRUE);
factory.setRecoveryInterval(30 * 1000L);
return factory;
}
}
This should work fine. But i am worried about whats written on the doc of CachingConnectionFactory
Specially, these parts;
NOTE: This ConnectionFactory requires explicit closing of all Sessions obtained from its shared Connection
Note also that MessageConsumers obtained from a cached Session won't get closed until the Session will eventually be removed from the pool. This may lead to semantic side effects in some cases.
I thought the framework handled the closing session and connection part? If it does not; how should i close them properly?
or maybe i am missing something?
Any help is appreciated :)
F.Y.I : I Use SonicMQ as the broker
Yes, the JmsTemplate will close the session; the javadocs refer to direct use outside of the framework.
So I got a advanced queue working with a Connectionfactory:
ConnectionFactory jmsQueueConnectionFactory() throws JMSException, SQLException {
final OracleDataSource dataSource = new OracleDataSource();
dataSource.setUser(username);
dataSource.setPassword(password);
dataSource.setURL(url);
dataSource.setImplicitCachingEnabled(true);
dataSource.setFastConnectionFailoverEnabled(true);
return AQjmsFactory.getConnectionFactory(dataSource);
}
This is running on a shared Database which might be restarted or sometimes the network just has a short hickup. Which results in no more messages from the Queue.
I use a spring MessageListener to retrieve messages and there is actually no indicator or what so ever that the queue is not running anymore. After restarting the application I then get a load of older messages that should have been processed already.
Is there a way or specific data source implementation which reconnects or something?
Update: Listener Impl
#Bean
OracleAqQueueFactoryBean etlQueueFactory() throws JMSException, SQLException {
final OracleAqQueueFactoryBean bean = new OracleAqQueueFactoryBean();
bean.setConnectionFactory(jmsQueueConnectionFactory());
bean.setOracleQueueUser("USER");
bean.setOracleQueueName("QUEUE");
return bean;
}
#Bean
DefaultMessageListenerContainer jmsContainer() throws JMSException, SQLException {
final DefaultMessageListenerContainer bean = new DefaultMessageListenerContainer();
bean.setConnectionFactory(jmsQueueConnectionFactory());
bean.setDestination(etlQueueFactory().getObject());
bean.setMessageListener(new MyListener());
bean.setSessionTransacted(false);
return bean;
}
public class MyListener implements MessageListener {
#Override
public void onMessage(Message message) {
...
}
}
I guess you have to do it on the JMS level, not DB level.
Not sure what type of listener you are using, but a DefaultMessageListenerContainer in Spring is implemented with a consumer.receive(timeout) loop. It's more robust than using a plain listener as it will attempt to reconnect on each poll cycle (if needed).
I am trying the Srpring AMQP features regarding transactional message processing.
I have the following setup - I have a message consumer that is annotated as #Transactional
#Transactional
public void handleMessage(EventPayload event) {
Shop shop = new Shop();
shop.setName(event.getName());
Shop savedShop = shopService.create(shop);
log.info("Created shop {} from event {}", shop, event);
}
In shopService.create I save the shop and send another message about the creation:
#Transactional(propagation = REQUIRED)
#Component
public class ShopService {
...
public Shop create(Shop shop) {
eventPublisher.publish(new EventPayload(shop.getName()));
return shopRepository.save(shop);
}
}
I want to achieve the following - the message sent in the create method should just go to the broker if the database action succeeded. If it fails the message is not sent and the received message is rolled back.
I also have a Retry configured - so I expect each message to be retried 3 times before it is rejected:
#Bean
public RetryOperationsInterceptor retryOperationsInterceptor() {
return RetryInterceptorBuilder.stateless()
.maxAttempts(3)
.backOffOptions(1000, 2.0, 10000)
.build();
}
I am observing the following behaviour:
When I configure the container as follows the message is retried 3 times but every time the message in shopService.create is sent to the broker:
#Bean
SimpleMessageListenerContainer messageListenerContainer(ConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(testEventSubscriberQueue().getName());
container.setMessageListener(listenerAdapter);
container.setChannelTransacted(true);
container.setAdviceChain(new Advice[]{retryOperationsInterceptor()});
return container;
}
So I tried passing the PlatformTransactionManager to the container -
#Bean
SimpleMessageListenerContainer messageListenerContainer(ConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter,
PlatformTransactionManager transactionManager) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(testEventSubscriberQueue().getName());
container.setMessageListener(listenerAdapter);
container.setChannelTransacted(true);
container.setTransactionManager(transactionManager);
container.setAdviceChain(new Advice[]{retryOperationsInterceptor()});
return container;
}
Now the message sent in shopService.create is only send to the broker if the database transaction succeeded - which is what I want - but the message is retried indefinitely now - and not discarded after 3 retires as configured. But it seems that the backOff settings are applied - so there is some time between the retries.
The setup described does not really make sense from a business point of view - I am trying to understand and evaluate the transaction capabilities.
I am use spring-amqp 1.5.1.RELEASE
Thanks for any hints.
I had the same requirements, an #RabbitListener annotated with #Transactional, I wanted retry with backoff. It works even stateless with the following config:
#Bean
public RetryOperationsInterceptor retryOperationsInterceptor( ) {
return RetryInterceptorBuilder.stateless()
.maxAttempts( 3 )
.recoverer( new RejectAndDontRequeueRecoverer() )
.backOffOptions(1000, 2, 10000)
.build();
}
#Bean
public Jackson2JsonMessageConverter producerJackson2MessageConverter( ObjectMapper objectMapper ) {
Jackson2JsonMessageConverter jackson2JsonMessageConverter = new Jackson2JsonMessageConverter( objectMapper );
jackson2JsonMessageConverter.setCreateMessageIds( true );
return jackson2JsonMessageConverter;
}
#Bean
SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory( ConnectionFactory connectionFactory,
PlatformTransactionManager transactionManager,
Jackson2JsonMessageConverter converter) {
SimpleRabbitListenerContainerFactory container = new SimpleRabbitListenerContainerFactory ();
container.setConnectionFactory(connectionFactory);
container.setChannelTransacted(true);
container.setTransactionManager(transactionManager);
container.setAdviceChain( retryOperationsInterceptor() );
container.setMessageConverter( converter );
return container;
}
To use stateless(), using RejectAndDontRequeueRecoverer was important because otherwise the retry will work but the consumer will then by default put the message back on the queue. Then the consumer will retrieve it again, applying the retry policy and then putting it back on the queue infinitely.
I try to get XA transactions involving a jdbc and jms DataSource working in a Spring webapp deployed to Weblogic.
Using a local Atomikos TransactionManager, this works - I see XA debug messages in ActiveMQ, and stuff stays consistent. In Weblogic however, the database and ActiveMQ are not transactionally consistent.
I have added a foreign JMS server in Weblogic
JNDI Initial Context Factory:
org.apache.activemq.jndi.ActiveMQInitialContextFactory
JNDI Connection URL:
tcp://localhost:61616
JNDI Properties:
connectionFactoryNames=XAConnectionFactory
To that server, I have added a ConnectionFactory (Remote JNDI Name = XAConnectionFactory). Lookups work, so far so good.
In my code, this is how I setup the Spring JTA:
#Override
#Bean
#Profile(AppConfig.PROFILE_WEBLOGIC)
public JtaTransactionManager transactionManager()
{
WebLogicJtaTransactionManager tx = new WebLogicJtaTransactionManager();
tx.afterPropertiesSet();
return tx;
}
And this is my JMS config:
#Bean
#Profile(AppConfig.PROFILE_WEBLOGIC)
public ConnectionFactory connectionFactory()
{
Properties props = new Properties();
props.put(Context.INITIAL_CONTEXT_FACTORY, env.getProperty(Context.INITIAL_CONTEXT_FACTORY));
props.setProperty(Context.PROVIDER_URL, env.getProperty(Context.PROVIDER_URL));
try
{
InitialContext ctx = new InitialContext(props);
ActiveMQXAConnectionFactory connectionFactory = (ActiveMQXAConnectionFactory) ctx
.lookup(env.getProperty("jms.connectionFactory"));
return connectionFactory;
}
catch(NamingException e)
{
throw new RuntimeException("XAConnectionFactory lookup failed", e);
}
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() throws JMSException
{
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
factory.setTransactionManager(txConfig.transactionManager());
factory.setBackOff(new FixedBackOff());
return factory;
}
#Bean(name = "jmsTemplate")
#Override
public JmsTemplate jmsTemplate() throws JMSException
{
JmsTemplate t = new JmsTemplate();
t.setConnectionFactory(connectionFactory());
t.setMessageTimestampEnabled(true);
t.setMessageIdEnabled(true);
return t;
}
My JMS consumer is annotated with:
#Transactional
#JmsListener(destination = "test.q1")
Is there anything I am missing?
Turns out this only works via the Resource Adapter, its not possible solely via the JNDI ConnectionFactory.
It is possible, using the undocumented ActiveMQ JNDI property "xa=true" in the foreign JMS server definition, see here:
Deployment of ActiveMQ resource adapter fails
ActiveMQInitialConnectionFactory cannot return an
XAConnectionFactory
ActiveMQInitialConnectionFactory returns XA
connection factory