Spring poller roll back on exception after a particular retry count - spring

My poller fetches data from DB and pass it to a service activator.
If any exception happens in the service activator method,i should roll back the data fetched to its previous state and should again send the same data to the service activater only for a specific retry-count(say 3). **Is it possible to do this in a xml configuration. ? For details i will share the poller configurations and the service activator.
poller.xml
<int-jdbc:inbound-channel-adapter id="datachannel"
query="select loyalty_id,process_id,mobile_uid from TBL_RECEIPT where r_cre_time
=(select min(r_cre_time) from TBL_RECEIPT where receipt_status=0)"
data-source="dataSource" max-rows-per-poll="1"
update="update TBL_RECEIPT set receipt_status=11 where process_id in (:process_id)">
<int:poller fixed-rate="5000">
<int:transactional/>
</int:poller>
</int-jdbc:inbound-channel-adapter>
<int:channel id="errors">
<int:queue/>
</int:channel>
<bean id="poller" class="main.java.com.as.poller.PollerService" />
<int:channel id="executerchannel">
<int:dispatcher task-executor="taskExecutor" />
</int:channel>
<task:executor id="taskExecutor" pool-size="2" />
<int:service-activator input-channel="datachannel"
output-channel="executerchannel" ref="poller" method="getRecordFromPoller">
<int:request-handler-advice-chain>
<int:retry-advice recovery-channel="errors" />
</int:request-handler-advice-chain>
</int:service-activator>
<int:service-activator input-channel="executerchannel"
ref="poller" method="getDataFromExecuterChannel">
</int:service-activator>
service activator method
#SuppressWarnings({ "unchecked", "rawtypes" })
#ServiceActivator
public void processMessage(Message message) throws IOException {
int capLimit = Integer.parseInt(env.getProperty("capping_limit"));
List<Map<String, Object>> rows = (List<Map<String, Object>>) message
.getPayload();
for (Map<String, Object> row : rows) {
String loyaltyId = (String) row.get("loyalty_id");
String processId = (String) row.get("process_id");
String xid=(String)row.get("mobile_uid");
i have heard about int:transactional being used in poller configuration.But when i added it,its taking the same record even after successful transaction.(means its getting rolled back everytime).
Can anyone please help me on this ?

You can add a retry request-handler-advice to the service activator.
To make retry stateful (throw the exception so the transaction will rollback) you need to provide a RetryStateGenerator. Otherwise the thread will simply be suspended during the retries.
Regardless of using stateful or stateless retry, you really should use a transactional poller so that the update is only applied after success.
means its getting rolled back everytime
The transactional poller will only rollback if an exception is thrown. So if you're seeing the same row, something must be failing.
Turn on DEBUG logging for all of org.springframework to follow the message flow and transaction activity.

Related

RabbitMQ listener stops listening messages when MessageListener throws exception

I am using Spring AMQP for processing the messages in RabbitMQ. Below is the issue:
1. (say) there are 3 messages in ready state inside RabbitMQ
2. First one is picked up by MessageListener and starts processing. (say) It ends up throwing an exception
3. In this case, the container remains up but the remaining 2 messages are not processed until i restart the container. Also the first messages stays in unacknowledged state.
Is it the expected behavior? If not, how to make sure that other 2 messages will be processed irrespective first one failed processing?
MQ configuraion:
<rabbit:connection-factory id="connectionFactory" host="localhost" username="guest" password="guest" />
<rabbit:admin connection-factory="connectionFactory" />
<rabbit:listener-container
connection-factory="connectionFactory"
concurrency="1"
acknowledge="auto">
<rabbit:listener queue-names="testQueue" ref="myProcessorListener " />
</rabbit:listener-container>
MessageListener class:
public class MyProcessorListener implements MessageListener{
....
#Override
public void onMessage(Message message) {
try{
...Some logic...
} catch (Exception e) {
throw new RuntimeException(e.getMessage(), e);
}
The message is redelivered over and over again; in order to reject it (and discard or route to a dead letter queue), you need to throw AmqpRejectAndDontRequeueException or set the container's requeue-rejected property to false. When configuring with Java it's defaultRequeueRejected.
You can also use a custom error handler.
This is all explained in the reference manual.

Problems with unit testing a spring integration program

I am trying to unit test an xpath router, but having problems with it. here is my context file:
<int:channel id="toTransactionTypeRouterChannel" />
<int-xml:xpath-router id="transactionsTypeRouter"
input-channel="toTransactionTypeRouterChannel" resolution-required="false"
evaluate-as-string="true" default-output-channel="errorChannel">
<!-- Select node name of the first child -->
<int-xml:xpath-expression
expression="name(/soapNs:Envelope/soapNs:Body/schNs:processArchiveRequest/schNs:fulfillmentRequest/schNs:requestDetail/*[1])"
namespace-map="archiveNamespaceMap" />
<int-xml:mapping value="sch:bulkRequestDetail"
channel="bulkChannel" />
<int-xml:mapping value="sch:transactionalRequestDetail"
channel="transactionChannel" />
</int-xml:xpath-router>
<int:channel id="bulkChannel" />
<int:channel id="transactionChannel" />
<int:transformer input-channel="bulkChannel"
output-channel="consoleOut" expression="'Bulk channel has received the payload' " />
<int:transformer input-channel="transactionChannel"
output-channel="consoleOut" expression="'Transaction channel has received payload' " />
<int:transformer input-channel="errorChannel"
output-channel="consoleOut" expression="'Error channel has received payload' " />
As you can see here, there are 2 different routes(bulk,trans.) + error channel.Here is my unit test case for trans channel route:
#Test
public void testTransactionFlow() throws Exception {
try {
Resource bulkRequest = new FileSystemResource("src/main/resources/mock-message-examples/SampleProcessArchiveTransRequest.xml");
String transRequestStr= extractResouceAsString(bulkRequest);
toTransactionTypeRouterChannel.send(MessageBuilder.withPayload(transRequestStr).build());
Message<?> outMessage = testChannel.receive(0);
assertNotNull(outMessage);
context file for junit
<int:bridge input-channel="transactionChannel"
output-channel="testChannel"/>
<int:channel id="testChannel">
<int:queue/>
</int:channel>
As you can see, in the junit context file, I am connecting transactional channel to the test channel.in the junit test case, I am sending a payload to the router in the junit method, and trying to receive it from the input channel and use it for assertion. however the assertion fails, as the message from transaction channel directly goes to consoleOut before getting routed to inputChannel as given in the junit coontext file. How do I intercept the message before it goes to consoleOut? I also tried adding wireTap interceptors but they didnt work:
WireTap wireTap = new WireTap(someChannel);
boolean w = wireTap.isRunning();
transactionChannel.addInterceptor(wireTap);
Basically, I need a separate flow for unit testing.
With that configuration, you are just adding a second consumer to transactionChannel - messages will be round-robin distributed to the transformer and bridge.
You can unsubscribe the transformer for your test case by autowiring it by id as an EventDrivenConsumer and stop() it before sending your message.

DeferredResult from JMS Request/Reply with ListenableFuture

I was looking for opportunities to improve the Rest Api we have exposed to external clients. During that exercise, I found I am barely taking advantage of the decision our integration team took to have backend integrations with JMS request/reply instead of traditional blocking SOAP request/reply.
Currently all the interactions to message broker are done using jmsOutboundGateway, because of which requesting thread has to wait for completion. In order to scale RestAPI, I want to send JMS reply using DeferredResult from Spring MVC controller. The controller interaction with message broker is depicted below:
Controller --> GatewayProxy --> JMSOutboundGateway
I am looking for opportunities to use ListenableFuture as return type of GatewayProxy, but I am unable to find a proper mean of achieving it using spring integration.
Below is the integration flow I am calling from controller:
<int:gateway
service-interface="ae.emaratech.ngx.service.PermitSearchService"
default-request-channel="permit_search_input_channel"
default-reply-timeout="${broker.jms.gateway.min.consumers}"/>
<int:channel id="permit_search_input_channel" />
<int:chain input-channel="permit_search_input_channel">
<int:header-enricher>
<int:header name="person_number" expression="payload"/>
</int:header-enricher>
<int:transformer expression="#formatString(#api_messages['FIND_PERM_BY_PERSNO_MSG'],headers)"/>
<int:header-filter header-names="JMS_*,jms_*,priority" pattern-match="true" />
<int:header-enricher>
<int:header name="jms_type" type="java.lang.String" value="1" overwrite="true"/>
</int:header-enricher>
<jms:outbound-gateway
request-destination="permitsInboundQueue"
reply-destination="permitsOutboundQueue"
receive-timeout="${broker.jms.gateway.timeout}"
correlation-key="Correlation_ID"
connection-factory="brokerConnectionFactory">
<jms:reply-listener concurrent-consumers="${broker.jms.gateway.min.consumers}" max-concurrent-consumers="${broker.jms.gateway.max.consumers}"/>
</jms:outbound-gateway>
<int-xml:xpath-filter throw-exception-on-rejection="true">
<int-xml:xpath-expression expression="not(boolean(/*/ErrorDetails))"/>
</int-xml:xpath-filter>
<int-xml:xslt-transformer
xsl-resource="classpath:/META-INF/spring/integration/permit-to-json.xsl"
result-type="StringResult" >
</int-xml:xslt-transformer>
<int:transformer expression="payload.toString()"/>
</int:chain>
Not sure what problem you have, but the feature looks like:
ListenableFuture<String> result = this.asyncGateway.async("foo");
result.addCallback(new ListenableFutureCallback<String>() {
#Override
public void onSuccess(String result) {
...
}
#Override
public void onFailure(Throwable t) {
...
}
});
It is available since version 4.1.

how to write output channel from itemWriter in spring batch and spring integration?

firstly thanks for attention, i combined spring batch and spring integration, i defined a job flow and retrieve files from ftp adapter and sent to jobChannel, and process on it with spring batch , i want to write to output channel and consume the channel after processing, my code is:
<int-ftp:outbound-gateway id="gatewayGET"
local-directory-expression="'./backup/' +#remoteDirectory"
session-factory="ftpSessionFactory"
request-channel="toGetChannel"
reply-channel="toProcessChannel"
command="get"
temporary-file-suffix=".writing"
command-options="-P"
expression="payload.remoteDirectory + '/' + payload.filename"/>
<int:channel id="toProcessChannel">
<int:interceptors>
<int:wire-tap channel="logger2"/>
</int:interceptors>
</int:channel>
<int:channel id="outboundJobRequestChannel"/>
<int:channel id="replyJobChannel"/>
<service-activator input-channel="jobLaunchReplyChannel"/>
<int:transformer input-channel="toProcessChannel" output-channel="outboundJobRequestChannel">
<bean class="ir.isc.macna.configuration.FileMessageToJobRequest">
<property name="fileParameterName" value="fileName"/>
</bean>
</int:transformer>
<batch-int:job-launching-gateway request-channel="outboundJobRequestChannel"
reply-channel="jobLaunchReplyChannel"/>
and my writer code is:
#Component
#StepScope
public class MacnaFileWriter implements ChunkMessageChannelItemWriter<FileInfo> {
#Autowired
#Qualifier("replyJobChannel")
private MessageChannel messageChannel;
#Override
public void write(List<? extends FileInfo> list) throws Exception {
// send Message to replyJobChannel channel with Send method
}
}
and use ftp adapter to write files on the server:
<int-ftp:outbound-gateway session-factory="ftpSessionFactory"
request-channel="replyJob"
command="mput"
auto-create-directory="true"
expression="payload"
remote-directory-expression="payload.remoteDirectory + '/' + payload.filename + '.out'"
reply-channel="output"/>
Is this standard way to run batch job and consume result?
MessageChannel has send method. For this purpose you can use MessageBuilder to create a Message<?> for your payload.
Since it is just a <channel> there is no difference with any other MessageChannel to consume. The standard way define a consumer like this:
<service-activator input-channel="jobLaunchReplyChannel"/>
However it isn't clear why you are going to send messages to the jobLaunchReplyChannel, when this channel is specific for the result from <batch-int:job-launching-gateway> (JobLaunchingGateway class):
jobExecution = this.jobLaunchingMessageHandler.launch(jobLaunchRequest);
So, that jobExecution is sent to the jobLaunchReplyChannel.
See Spring Batch Integration Documentation.
Maybe do you need AsyncItemWriter or ChunkMessageChannelItemWriter?

Spring splitter/aggregator handling exceptions

Version : spring-integration-core - 2.2.3
Here is the simplified version of my splitter/aggregator setup.
<task:executor id="taskExecutor" pool-size="${pool.size}"
queue-capacity="${queue.capacity}"
rejection-policy="CALLER_RUNS" keep-alive="120"/>
<int:channel id="service-requests"/>
<int:channel id="service-request"/>
<int:channel id="channel-1">
<int:dispatcher task-executor="taskExecutor" failover="false"/>
</int:channel>
<int:channel id="channel-2">
<int:dispatcher task-executor="taskExecutor" failover="false"/>
</int:channel>
<int:gateway id="myServiceRequestor" default-reply-timeout="${reply.timeout}"
default-reply-channel="service-aggregated-reply"
default-request-channel="service-request"
service-interface="com.blah.blah.MyServiceRequestor"/>
<int:splitter input-channel="service-request"
ref="serviceSplitter" output-channel="service-requests"/>
<!-- To split the request and return a java.util.Collection of Type1 and Type2 -->
<bean id="serviceSplitter" class="com.blah.blah.ServiceSplitter"/>
<int:payload-type-router input-channel="service-requests" resolution-required="true">
<int:mapping
type="com.blah.blah.Type1"
channel="channel-1"/>
<int:mapping
type="com.blah.blah.Type2"
channel="channel-2"/>
</int:payload-type-router>
<!-- myService is a bean where processType1 & processType2 method is there to process the payload -->
<int:service-activator input-channel="channel-1"
method="processType1" output-channel="service-reply" requires-reply="true"
ref="myService"/>
<int:service-activator input-channel="channel-2"
method="processType2" output-channel="service-reply" requires-reply="true"
ref="myService"/>
<int:publish-subscribe-channel id="service-reply" task-executor="taskExecutor"/>
<!-- myServiceAggregator has a aggregate method which takes a Collection as argument(aggregated response from myService) -->
<int:aggregator input-channel="service-reply"
method="aggregate" ref="myServiceAggregator"
output-channel="service-aggregated-reply"
send-partial-result-on-expiry="false"
message-store="myResultMessageStore"
expire-groups-upon-completion="true"/>
<bean id="myResultMessageStore" class="org.springframework.integration.store.SimpleMessageStore" />
<bean id="myResultMessageStoreReaper" class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="myResultMessageStore" />
<property name="timeout" value="2000" />
</bean>
<task:scheduled-tasks>
<task:scheduled ref="myResultMessageStoreReaper" method="run" fixed-rate="10000" />
</task:scheduled-tasks>
If the processType1/processType2 method in mySevice throws a RuntimeException, then it tries to send the message to an error channel(i believe spring does it by default) and the message payload in error channel stays on in heap and not getting garbage collected.
Updated More Info:
For my comment on error channel. I debugged the code and found that ErrorHandlingTaskExecutor is trying to use a MessagePublishingErrorHandler which inturn sending the message to the channel returned by MessagePublishingErrorHandler.resolveErrorChannel method.
Code snippet from ErrorHandlingTaskExecutor.java
public void execute(final Runnable task) {
this.executor.execute(new Runnable() {
public void run() {
try {
task.run();
}
catch (Throwable t) {
errorHandler.handleError(t); /// This is the part which sends the message in to error channel.
}
}
});
}
Code snipper from MessagePublishingErrorHandler.java
public final void handleError(Throwable t) {
MessageChannel errorChannel = this.resolveErrorChannel(t);
boolean sent = false;
if (errorChannel != null) {
try {
if (this.sendTimeout >= 0) {
sent = errorChannel.send(new ErrorMessage(t), this.sendTimeout);
.....
When i take a heap dump, I always see the reference to the payload message(which i believe is maintained in the above channel) and not getting GC'ed.
Would like to know what is the correct way to handle this case or if i'm missing any in my config?
Also is it possible to tell spring to discard the payload(instead of sending it to error channel) in case of any exception thrown by the service activator method?
Looking forward for your inputs.
Thanks.
You don't have an error-channel defined on your gateway so we won't send it there, we'll just throw an exception to the caller.
However, the partial group is sitting in the aggregator and will never complete. You need to configure a MessageGroupStoreReaper as shown in the reference manual (or set a group-timeout in Spring Integration 4.0.x) to discard the partial group.

Resources