I have a spring application with a consumer to consume messages and write them to a database. I'm using the spring DefaultMessageListenerContainer. Is there a way to consume a message and upon a database exception being thrown put the message back onto the queue?
setSessionTransacted(true)
If you are using the namespace to configure the container, use
<jms:listener-container acknowledge="transacted" ...>
<jms:listener ... />
</jms:listener-container>
You also might want to synchronize the database and JMS transactions by adding the JDBC transaction manager to the container configuration.
See Dave Syer's Javaworld Article about Distributed transactions in Spring, with and without XA
Related
I have a spring integration project. Which reads message from the qpid queue and starts some processing. To mu understanding once message is read it should be cleared from the queue.
But Even after the processing is complete. The message stays in queue in acquired state. Even if i clear the queue and drop new message the message is not picked up by the adapter.
Following is my configuration.
<jms:message-driven-channel-adapter destination-name="incoming.queue" channel="transform-jms-message-channel" connection-factory="qpidConnectionFactory"
concurrent-consumers="1" error-channel="errorChannel" acknowledge="transacted" />
<int:transformer input-channel="transform-jms-message-channel" id="reportRequestTransformer" ref="reportRequestMessageTransformer" method="transform"
output-channel="validate-parameters-channel"/>
Any help is appreciated.
I am consuming messages from JMS ActiveMQ using the following code:
<jms:message-driven-channel-adapter
id="helloJMSAdapater" destination="helloJMSQueue" connection-factory="jmsConnectionfactory"
channel="helloChannel" extract-payload="true" />
<integration:channel id="helloChannel" />
My requirement is to consume from here and post it to Kafka outbound adapter. Using the below config:
<int-kafka:outbound-channel-adapter
id="kafkaOutboundChannelAdapter"
kafka-producer-context-ref="kafkaProducerContext"
channel="inputToKafka">
</int-kafka:outbound-channel-adapter>
Here are what i want to achieve:
My queue is a durable topic and dont want to acknowledge the records unless it is successfully published to Kafka. In short, i want to have a transaction behaviour from consuming message from jms to publishing it to Kafka.
I noticed that my messages are immediately dequeued and if processing encounters some exception, i am unable to reprocess it. I dont want that to happen.
Also, when kafka encounters some issue, i want it to be returned to some method so that i can persist the failure message and as said before does not want to acknowledge it.
I am really struggling to get it to work. Can someone please help me out?
You really can have transaction-manager on the <jms:message-driven-channel-adapter> to start TX.
When <int-kafka:outbound-channel-adapter> throws an exception it causes the TX to be ralled back and therefore the message will be requeued.
If you are interested in the persisting errors, there is an error-channel option on the <jms:message-driven-channel-adapter>, but you still have to re-throw exception to let TX to rallback.
To make all that to work you should be sure that there is only single thread from the begging to the end. No <queue> or executor channel in the flow.
Also it isn't clear why do you use so old Apache Kafka still...
I use Spring Integration to read messages from JMS, handle them and then persist them to the database using my own dbPersistor's persist method which has a return type void. I wrote a test case to verify the messages published to the JMS were successfully persisted in the db. My SI and JMS configuration are as follows for this test -
<int:poller fixed-delay="500" default="true"/>
<int:channel id="inputChannel">
<int:queue/>
</int:channel>
<int:channel id="errorChannel">
<int:queue/>
</int:channel>
<jms:message-driven-channel-adapter id="jmsInboudAdapter"
connection-factory="connectionFactory" destination-name="MessageQueue"
channel="inputChannel" error-channel="errorChannel" transaction-manager="dbTxManager"
acknowledge="transacted"/>
<int:chain id="handlerChain" input-channel="inputChannel">
<int:service-activator ref="jmsMessageHandler" method="handleMessage" />
<int:service-activator ref="dbPersistor" method="persist" />
</int:chain>
Then in the test I do the following -
jmsTemplate.send()
verifyMessageWasPersistedToDB
This works great when I just publish a single message to the db. But when I loop through jmsTemplate.send() to publish multiple messages, the main thread completes the operation while the SI threads are still executing and tries to verify the messages in DB and fails as some of the messages have not been persisted yet. My questions are -
How do I make the main thread to wait for the SI threads to finish and then invoke the verify method?
In case of a db exception and rollback how do I verify that the failed message is back in the original queue?
Thanks
AJ
inputChannel should not be a queue channel - the JMS transaction will commit when the message is inserted in the queue - the DB transaction won't be performed in the scope of the JMS transaction. You must use direct channels for that (remove the poller and <queue/>). See the documentation on transactions in Spring Integration.
You will have to poll the database for results; you could probably add an interceptor and some CountDownLatch but it's easier to just poll the DB until the results appear or some time expires.
My consumer, e.g. service activator that is consuming messages coming from ActiveMQ fromChannel should be restarted when exception occurs or ActiveMQ fails. How to do it for the following spring integration context ?
<!-- RECEIVER. message driven adapter -> jmsInChannel -> activator. -->
<si:channel id="fromChannel"/>
<int-jms:message-driven-channel-adapter id="messageDrivenAdapter"
channel="fromChannel" destination="forward" connection-factory="connectionFactory"
max-concurrent-consumers="2" auto-startup="true" acknowledge="auto" extract-payload="false" />
<si:service-activator id ="activator"
input-channel="fromChannel"
ref="messageService"
method="process"/>
<bean id="messageService" class="com.ucware.ucpo.forward.jms.MessageService"/>
My first idea was to use Retry Advice and add to a service but am not sure if this a right solution for unhandled exceptions. I also would like the receiver to restart if the ActiveMQ server is down.
The listener container within the message-driven-channel-adapter will automatically keep trying to reconnect when it loses connectivity to the broker.
If you set `acknowledge="transacted"' the message will be rolled back on an exception and the broker will resubmit it.
A stateful retry advice would allow you to give up and take some other action after some number of retries (but you can also configure that into ActiveMQ itself where it will send the message to a DLQ after some number of delivery attempts).
Reading your post I instantly thought of this video. Which gives a good insight on how to monitor and control SI application through itself.
Additionally you should have a look at ApplicationEvent documentation of SI.
Glueing that all together you could monitor the JMS message adapter with JMX and stop and restart it through sending an ApplicationEvent on issues. Regarding catching exceptions it depends on what Exceptions you actually want to handle. I'd create an errorChannel that receives exceptions being thrown by components and create a new service that restarts these components after receiving errors.
Following your idea leveraging Spring Retry's capabilites in SI.
I configured spring xml based interceptor, which sends a jms message to activemq queue on each invokation of some transactional method after it is commited. It's happening with the following xml code.
<jms:outbound-channel-adapter channel="filteredStakesChannel" destination="stakesQueue" delivery-persistent="true" explicit-qos-enabled="true" />
But if the activemq server is down i get connection refused exception, which is propagated and i don't want this to happen even if the jms delivery fails. Is this possible?
Should i use some error-channel?
The simplest solution is to make fileredStakesChannel an Executor channel and the send will run on a different thread.
http://static.springsource.org/spring-integration/reference/html/messaging-channels-section.html#executor-channel
http://static.springsource.org/spring-integration/reference/html/messaging-channels-section.html#channel-configuration-executorchannel
Use the <task/> namespace to define an executor to use.