DeferredResult from JMS Request/Reply with ListenableFuture - spring

I was looking for opportunities to improve the Rest Api we have exposed to external clients. During that exercise, I found I am barely taking advantage of the decision our integration team took to have backend integrations with JMS request/reply instead of traditional blocking SOAP request/reply.
Currently all the interactions to message broker are done using jmsOutboundGateway, because of which requesting thread has to wait for completion. In order to scale RestAPI, I want to send JMS reply using DeferredResult from Spring MVC controller. The controller interaction with message broker is depicted below:
Controller --> GatewayProxy --> JMSOutboundGateway
I am looking for opportunities to use ListenableFuture as return type of GatewayProxy, but I am unable to find a proper mean of achieving it using spring integration.
Below is the integration flow I am calling from controller:
<int:gateway
service-interface="ae.emaratech.ngx.service.PermitSearchService"
default-request-channel="permit_search_input_channel"
default-reply-timeout="${broker.jms.gateway.min.consumers}"/>
<int:channel id="permit_search_input_channel" />
<int:chain input-channel="permit_search_input_channel">
<int:header-enricher>
<int:header name="person_number" expression="payload"/>
</int:header-enricher>
<int:transformer expression="#formatString(#api_messages['FIND_PERM_BY_PERSNO_MSG'],headers)"/>
<int:header-filter header-names="JMS_*,jms_*,priority" pattern-match="true" />
<int:header-enricher>
<int:header name="jms_type" type="java.lang.String" value="1" overwrite="true"/>
</int:header-enricher>
<jms:outbound-gateway
request-destination="permitsInboundQueue"
reply-destination="permitsOutboundQueue"
receive-timeout="${broker.jms.gateway.timeout}"
correlation-key="Correlation_ID"
connection-factory="brokerConnectionFactory">
<jms:reply-listener concurrent-consumers="${broker.jms.gateway.min.consumers}" max-concurrent-consumers="${broker.jms.gateway.max.consumers}"/>
</jms:outbound-gateway>
<int-xml:xpath-filter throw-exception-on-rejection="true">
<int-xml:xpath-expression expression="not(boolean(/*/ErrorDetails))"/>
</int-xml:xpath-filter>
<int-xml:xslt-transformer
xsl-resource="classpath:/META-INF/spring/integration/permit-to-json.xsl"
result-type="StringResult" >
</int-xml:xslt-transformer>
<int:transformer expression="payload.toString()"/>
</int:chain>

Not sure what problem you have, but the feature looks like:
ListenableFuture<String> result = this.asyncGateway.async("foo");
result.addCallback(new ListenableFutureCallback<String>() {
#Override
public void onSuccess(String result) {
...
}
#Override
public void onFailure(Throwable t) {
...
}
});
It is available since version 4.1.

Related

Tracing input & output of message channels of Spring Integration

I have build a small application using Spring Integration. I have also written some jUnit for more service and other classes.
I have used XML configuration for the channel and en-point configuration, I am wondering if I can test the input and output of a perticular channel.
Is there a way we can test the input and output of the channels?..
Update
I am trying to the below flow. How shall i proceed?
<int:channel id="getPresciption" />
<int:channel id="respPrescription" />
<int:channel id="storePrcedureChannell" />
<int-http:inbound-gateway
request-channel="getPresciption" reply-channel="respPrescription"
supported-methods="GET" path="/getAllPresciption">
<int-http:request-mapping
consumes="application/json" produces="application/json" />
</int-http:inbound-gateway>
<int:service-activator
ref="prescriptionServiceActivator" method="buildPrescription"
input-channel="getPresciption" output-channel="storePrcedureChannell" />
<int:service-activator
ref="prescriptionServiceActivator" method="storePrescription"
input-channel="storePrcedureChannell"></int:service-activator>
SO how can i write the Test contex?
Let below are the methods which are called by channel flow.
public Message<List<Prescription>> buildPrescription() {
//Do some processing
}
public Message<List<Prescription>> storePrescription(Message<List<Prescription>> msg) {
//Do something and return the List.
}
First of all you can take a look into the Spring Integration Testing Framework: https://docs.spring.io/spring-integration/reference/html/testing.html#test-context
And then use a MockIntegration.mockMessageHandler() together with the MockIntegrationContext.substituteMessageHandlerFor() to replace a real handler in the endpoint and verify incoming data from the channel.
If that is still hard for you, you always can inject those channels into your test class and add to them a ChannelInterceptor and verify messages in its preSend() implementation.

Problems with unit testing a spring integration program

I am trying to unit test an xpath router, but having problems with it. here is my context file:
<int:channel id="toTransactionTypeRouterChannel" />
<int-xml:xpath-router id="transactionsTypeRouter"
input-channel="toTransactionTypeRouterChannel" resolution-required="false"
evaluate-as-string="true" default-output-channel="errorChannel">
<!-- Select node name of the first child -->
<int-xml:xpath-expression
expression="name(/soapNs:Envelope/soapNs:Body/schNs:processArchiveRequest/schNs:fulfillmentRequest/schNs:requestDetail/*[1])"
namespace-map="archiveNamespaceMap" />
<int-xml:mapping value="sch:bulkRequestDetail"
channel="bulkChannel" />
<int-xml:mapping value="sch:transactionalRequestDetail"
channel="transactionChannel" />
</int-xml:xpath-router>
<int:channel id="bulkChannel" />
<int:channel id="transactionChannel" />
<int:transformer input-channel="bulkChannel"
output-channel="consoleOut" expression="'Bulk channel has received the payload' " />
<int:transformer input-channel="transactionChannel"
output-channel="consoleOut" expression="'Transaction channel has received payload' " />
<int:transformer input-channel="errorChannel"
output-channel="consoleOut" expression="'Error channel has received payload' " />
As you can see here, there are 2 different routes(bulk,trans.) + error channel.Here is my unit test case for trans channel route:
#Test
public void testTransactionFlow() throws Exception {
try {
Resource bulkRequest = new FileSystemResource("src/main/resources/mock-message-examples/SampleProcessArchiveTransRequest.xml");
String transRequestStr= extractResouceAsString(bulkRequest);
toTransactionTypeRouterChannel.send(MessageBuilder.withPayload(transRequestStr).build());
Message<?> outMessage = testChannel.receive(0);
assertNotNull(outMessage);
context file for junit
<int:bridge input-channel="transactionChannel"
output-channel="testChannel"/>
<int:channel id="testChannel">
<int:queue/>
</int:channel>
As you can see, in the junit context file, I am connecting transactional channel to the test channel.in the junit test case, I am sending a payload to the router in the junit method, and trying to receive it from the input channel and use it for assertion. however the assertion fails, as the message from transaction channel directly goes to consoleOut before getting routed to inputChannel as given in the junit coontext file. How do I intercept the message before it goes to consoleOut? I also tried adding wireTap interceptors but they didnt work:
WireTap wireTap = new WireTap(someChannel);
boolean w = wireTap.isRunning();
transactionChannel.addInterceptor(wireTap);
Basically, I need a separate flow for unit testing.
With that configuration, you are just adding a second consumer to transactionChannel - messages will be round-robin distributed to the transformer and bridge.
You can unsubscribe the transformer for your test case by autowiring it by id as an EventDrivenConsumer and stop() it before sending your message.

Camel setBody using Spring configuration

I have built a Java Camel Timer-JMS route using:
context.addRoutes(new RouteBuilder() {
public void configure() {
from("timer:foo?period=1s").setBody(body().
append("Message at ${date:now:yyyy-MM-dd HH:mm:ss}")).to("jms:queue:activemq/queue/TestQueue");
}
});
Now I need to turn it to Spring.
<camel:route>
<camel:from uri="timer:foo?period=1s" />
<camel:to uri="jms:queue:activemq/queue/TestQueue" />
</camel:route>
I'm missing the equivalent in Spring XML configuration of the expression:
setBody(body().append("Message at ${date:now:yyyy-MM-dd HH:mm:ss}")
In Spring you can use the simple language, to build such messages. In fact you can also do this in Java, it the same.
<setBody>
<simple>${body}Message at ${date:now:yyyy-MM-dd HH:mm:ss}</simple>
</setBody>
Mind that the timer sends an empty/null body. So you may see "null" in the message.
About simple see: http://camel.apache.org/simple

Spring splitter/aggregator handling exceptions

Version : spring-integration-core - 2.2.3
Here is the simplified version of my splitter/aggregator setup.
<task:executor id="taskExecutor" pool-size="${pool.size}"
queue-capacity="${queue.capacity}"
rejection-policy="CALLER_RUNS" keep-alive="120"/>
<int:channel id="service-requests"/>
<int:channel id="service-request"/>
<int:channel id="channel-1">
<int:dispatcher task-executor="taskExecutor" failover="false"/>
</int:channel>
<int:channel id="channel-2">
<int:dispatcher task-executor="taskExecutor" failover="false"/>
</int:channel>
<int:gateway id="myServiceRequestor" default-reply-timeout="${reply.timeout}"
default-reply-channel="service-aggregated-reply"
default-request-channel="service-request"
service-interface="com.blah.blah.MyServiceRequestor"/>
<int:splitter input-channel="service-request"
ref="serviceSplitter" output-channel="service-requests"/>
<!-- To split the request and return a java.util.Collection of Type1 and Type2 -->
<bean id="serviceSplitter" class="com.blah.blah.ServiceSplitter"/>
<int:payload-type-router input-channel="service-requests" resolution-required="true">
<int:mapping
type="com.blah.blah.Type1"
channel="channel-1"/>
<int:mapping
type="com.blah.blah.Type2"
channel="channel-2"/>
</int:payload-type-router>
<!-- myService is a bean where processType1 & processType2 method is there to process the payload -->
<int:service-activator input-channel="channel-1"
method="processType1" output-channel="service-reply" requires-reply="true"
ref="myService"/>
<int:service-activator input-channel="channel-2"
method="processType2" output-channel="service-reply" requires-reply="true"
ref="myService"/>
<int:publish-subscribe-channel id="service-reply" task-executor="taskExecutor"/>
<!-- myServiceAggregator has a aggregate method which takes a Collection as argument(aggregated response from myService) -->
<int:aggregator input-channel="service-reply"
method="aggregate" ref="myServiceAggregator"
output-channel="service-aggregated-reply"
send-partial-result-on-expiry="false"
message-store="myResultMessageStore"
expire-groups-upon-completion="true"/>
<bean id="myResultMessageStore" class="org.springframework.integration.store.SimpleMessageStore" />
<bean id="myResultMessageStoreReaper" class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="myResultMessageStore" />
<property name="timeout" value="2000" />
</bean>
<task:scheduled-tasks>
<task:scheduled ref="myResultMessageStoreReaper" method="run" fixed-rate="10000" />
</task:scheduled-tasks>
If the processType1/processType2 method in mySevice throws a RuntimeException, then it tries to send the message to an error channel(i believe spring does it by default) and the message payload in error channel stays on in heap and not getting garbage collected.
Updated More Info:
For my comment on error channel. I debugged the code and found that ErrorHandlingTaskExecutor is trying to use a MessagePublishingErrorHandler which inturn sending the message to the channel returned by MessagePublishingErrorHandler.resolveErrorChannel method.
Code snippet from ErrorHandlingTaskExecutor.java
public void execute(final Runnable task) {
this.executor.execute(new Runnable() {
public void run() {
try {
task.run();
}
catch (Throwable t) {
errorHandler.handleError(t); /// This is the part which sends the message in to error channel.
}
}
});
}
Code snipper from MessagePublishingErrorHandler.java
public final void handleError(Throwable t) {
MessageChannel errorChannel = this.resolveErrorChannel(t);
boolean sent = false;
if (errorChannel != null) {
try {
if (this.sendTimeout >= 0) {
sent = errorChannel.send(new ErrorMessage(t), this.sendTimeout);
.....
When i take a heap dump, I always see the reference to the payload message(which i believe is maintained in the above channel) and not getting GC'ed.
Would like to know what is the correct way to handle this case or if i'm missing any in my config?
Also is it possible to tell spring to discard the payload(instead of sending it to error channel) in case of any exception thrown by the service activator method?
Looking forward for your inputs.
Thanks.
You don't have an error-channel defined on your gateway so we won't send it there, we'll just throw an exception to the caller.
However, the partial group is sitting in the aggregator and will never complete. You need to configure a MessageGroupStoreReaper as shown in the reference manual (or set a group-timeout in Spring Integration 4.0.x) to discard the partial group.

Spring Integration, how can I pass inbound http request through outbound gateway?

I'm trying to implement some sort of proxy as part of my data flow, I want to receive a http-request on my inbound gateway and pass it through outbound gateway. I want preserve all query string parameters. My gateways configuration is:
<int:channel id="searchRequestChannel" />
<int:channel id="searchReplyChannel" />
<int-http:inbound-gateway id="searchRequestInboundGateway"
supported-methods="GET"
request-channel="searchRequestChannel"
reply-channel="searchReplyChannel"
path="/services/normalization"
reply-timeout="50000"
/>
<int-http:outbound-gateway id="searchServiceGateway"
http-method="GET"
request-channel="searchRequestChannel"
url="http://localhost:8080/query"
extract-request-payload="false"
expected-response-type="java.lang.String"
reply-timeout="50000"
charset="UTF-8"
/>
I expected that it would work as follows:
Client send request to the inbound gateway /services/normalization:
GET /services/normalization q=cat&exclude=black
Inbound gateway receives request and send it through searchRequestChannel to the outbound gateway.
Outbound gateway sends whole request to the external service:
GET /query q=cat&exclude=black
But on practice, outbound gateway sends empty request that does not contains any query arguments:
GET /query
So my question, what's easiest way to send the http-request that was accepted on inbound gateway through outbound gateway. In other words how can I implement simple proxy by spring integration tools?
This is a bit of a kludge, but works; the DispatcherServlet binds the request to the thread...
<int-http:inbound-gateway id="searchRequestInboundGateway"
supported-methods="GET"
request-channel="searchRequestEnricherChannel"
reply-channel="searchReplyChannel"
path="/services/normalization{queryString}"
reply-timeout="50000"
/>
<int:header-enricher input-channel="searchRequestEnricherChannel" output-channel="searchRequestChannel">
<int:header name="queryString"
expression="T(org.springframework.web.context.request.RequestContextHolder).requestAttributes.request.queryString" />
</int:header-enricher>
and then on the outbound side, use
<int-http:outbound-gateway id="searchServiceGateway"
http-method="GET"
request-channel="searchRequestChannel"
url="http://localhost:8080/query?{queryString}"
encode-uri="false"
extract-request-payload="false"
expected-response-type="java.lang.String"
reply-timeout="50000"
charset="UTF-8">
<uri-variable name="queryString" expression="headers.queryString" />
</int-http:outbound-gateway>
However, this won't work with 2.2.x and earlier because the query string is encoded on the outbound side (foo=bar&baz=qux becomes foo%3Dbar%26baz%3Dqux). In 3.0 we have added the ability to not encode the URI using an attribute by using encode-uri="false". This is not yet available in a release, but it's available in 3.0.0.BUILD-SNAPSHOT.
EDIT:
The above is a general solution that will work for all query strings; if you know the actual parameters, another solution would be to extract each parameter separately and rebuild the query string on the outbound side...
<int-http:inbound-gateway ... >
<int-http:header name="foo" expression="#requestParams.foo.get(0)"/>
<int-http:header name="baz" expression="#requestParams.baz.get(0)"/>
</int-http:inbound-gateway>
<int-http:outbound-gateway request-channel="requestChannel"
url="http://localhost:18080/http/receiveGateway?foo={foo}&baz={baz}"
http-method="POST"
expected-response-type="java.lang.String">
<int-http:uri-variable name="foo" expression="headers.foo"/>
<int-http:uri-variable name="baz" expression="headers.baz"/>
</int-http:outbound-gateway>
On the inbound side, it would be better if we offered the queryString as a first class expression variable #queryString.
Please feel free to open an 'Improvement' JIRA Issue
My own workaround solution is use a transformer that transforms parameters in the message payload (map of query string parameters) to prepared query string and use an url-expression in an outbound-gateway to avoid a query string encoding:
<bean id="payloadToQueryString"
class="com.dph.integration.PayloadToQueryStringTransformer" />
<int-http:inbound-gateway id="searchRequestInboundGateway"
supported-methods="GET"
request-channel="searchRequestChannel"
path="/services/normalization"
reply-timeout="50000" />
<int:transformer input-channel="searchRequestChannel"
output-channel="searchGatewayChannel"
ref="payloadToQueryString" method="transform" />
<int-http:outbound-gateway id="searchServiceGateway"
http-method="GET"
request-channel="searchGatewayChannel"
url-expression="'http://localhost:8080/query?' + payload"
expected-response-type="java.lang.String"
reply-timeout="50000"
charset="UTF-8">
</int-http:outbound-gateway>
PayloadToQueryStringTransformer class is:
public class PayloadToQueryStringTransformer extends AbstractTransformer {
#Override
protected Object doTransform(final Message<?> message) throws Exception {
return MessageBuilder
.withPayload(urlEncodeUTF8(((MultiValueMap) message.getPayload()).toSingleValueMap()))
.copyHeaders(message.getHeaders())
.build();
}
private static String urlEncodeUTF8(final String s) {
try {
return URLEncoder.encode(s, "UTF-8");
} catch (final UnsupportedEncodingException e) {
throw new UnsupportedOperationException(e);
}
}
private static String urlEncodeUTF8(final Map<?,?> map) {
final StringBuilder sb = new StringBuilder();
for (final Map.Entry<?,?> entry : map.entrySet()) {
if (sb.length() > 0) {
sb.append("&");
}
sb.append(String.format("%s=%s",
urlEncodeUTF8(entry.getKey().toString()),
urlEncodeUTF8(entry.getValue().toString())
));
}
return sb.toString();
}
}

Resources