Spring Integration, how can I pass inbound http request through outbound gateway? - spring

I'm trying to implement some sort of proxy as part of my data flow, I want to receive a http-request on my inbound gateway and pass it through outbound gateway. I want preserve all query string parameters. My gateways configuration is:
<int:channel id="searchRequestChannel" />
<int:channel id="searchReplyChannel" />
<int-http:inbound-gateway id="searchRequestInboundGateway"
supported-methods="GET"
request-channel="searchRequestChannel"
reply-channel="searchReplyChannel"
path="/services/normalization"
reply-timeout="50000"
/>
<int-http:outbound-gateway id="searchServiceGateway"
http-method="GET"
request-channel="searchRequestChannel"
url="http://localhost:8080/query"
extract-request-payload="false"
expected-response-type="java.lang.String"
reply-timeout="50000"
charset="UTF-8"
/>
I expected that it would work as follows:
Client send request to the inbound gateway /services/normalization:
GET /services/normalization q=cat&exclude=black
Inbound gateway receives request and send it through searchRequestChannel to the outbound gateway.
Outbound gateway sends whole request to the external service:
GET /query q=cat&exclude=black
But on practice, outbound gateway sends empty request that does not contains any query arguments:
GET /query
So my question, what's easiest way to send the http-request that was accepted on inbound gateway through outbound gateway. In other words how can I implement simple proxy by spring integration tools?

This is a bit of a kludge, but works; the DispatcherServlet binds the request to the thread...
<int-http:inbound-gateway id="searchRequestInboundGateway"
supported-methods="GET"
request-channel="searchRequestEnricherChannel"
reply-channel="searchReplyChannel"
path="/services/normalization{queryString}"
reply-timeout="50000"
/>
<int:header-enricher input-channel="searchRequestEnricherChannel" output-channel="searchRequestChannel">
<int:header name="queryString"
expression="T(org.springframework.web.context.request.RequestContextHolder).requestAttributes.request.queryString" />
</int:header-enricher>
and then on the outbound side, use
<int-http:outbound-gateway id="searchServiceGateway"
http-method="GET"
request-channel="searchRequestChannel"
url="http://localhost:8080/query?{queryString}"
encode-uri="false"
extract-request-payload="false"
expected-response-type="java.lang.String"
reply-timeout="50000"
charset="UTF-8">
<uri-variable name="queryString" expression="headers.queryString" />
</int-http:outbound-gateway>
However, this won't work with 2.2.x and earlier because the query string is encoded on the outbound side (foo=bar&baz=qux becomes foo%3Dbar%26baz%3Dqux). In 3.0 we have added the ability to not encode the URI using an attribute by using encode-uri="false". This is not yet available in a release, but it's available in 3.0.0.BUILD-SNAPSHOT.
EDIT:
The above is a general solution that will work for all query strings; if you know the actual parameters, another solution would be to extract each parameter separately and rebuild the query string on the outbound side...
<int-http:inbound-gateway ... >
<int-http:header name="foo" expression="#requestParams.foo.get(0)"/>
<int-http:header name="baz" expression="#requestParams.baz.get(0)"/>
</int-http:inbound-gateway>
<int-http:outbound-gateway request-channel="requestChannel"
url="http://localhost:18080/http/receiveGateway?foo={foo}&baz={baz}"
http-method="POST"
expected-response-type="java.lang.String">
<int-http:uri-variable name="foo" expression="headers.foo"/>
<int-http:uri-variable name="baz" expression="headers.baz"/>
</int-http:outbound-gateway>
On the inbound side, it would be better if we offered the queryString as a first class expression variable #queryString.
Please feel free to open an 'Improvement' JIRA Issue

My own workaround solution is use a transformer that transforms parameters in the message payload (map of query string parameters) to prepared query string and use an url-expression in an outbound-gateway to avoid a query string encoding:
<bean id="payloadToQueryString"
class="com.dph.integration.PayloadToQueryStringTransformer" />
<int-http:inbound-gateway id="searchRequestInboundGateway"
supported-methods="GET"
request-channel="searchRequestChannel"
path="/services/normalization"
reply-timeout="50000" />
<int:transformer input-channel="searchRequestChannel"
output-channel="searchGatewayChannel"
ref="payloadToQueryString" method="transform" />
<int-http:outbound-gateway id="searchServiceGateway"
http-method="GET"
request-channel="searchGatewayChannel"
url-expression="'http://localhost:8080/query?' + payload"
expected-response-type="java.lang.String"
reply-timeout="50000"
charset="UTF-8">
</int-http:outbound-gateway>
PayloadToQueryStringTransformer class is:
public class PayloadToQueryStringTransformer extends AbstractTransformer {
#Override
protected Object doTransform(final Message<?> message) throws Exception {
return MessageBuilder
.withPayload(urlEncodeUTF8(((MultiValueMap) message.getPayload()).toSingleValueMap()))
.copyHeaders(message.getHeaders())
.build();
}
private static String urlEncodeUTF8(final String s) {
try {
return URLEncoder.encode(s, "UTF-8");
} catch (final UnsupportedEncodingException e) {
throw new UnsupportedOperationException(e);
}
}
private static String urlEncodeUTF8(final Map<?,?> map) {
final StringBuilder sb = new StringBuilder();
for (final Map.Entry<?,?> entry : map.entrySet()) {
if (sb.length() > 0) {
sb.append("&");
}
sb.append(String.format("%s=%s",
urlEncodeUTF8(entry.getKey().toString()),
urlEncodeUTF8(entry.getValue().toString())
));
}
return sb.toString();
}
}

Related

Spring Integration : send json as request body and headers in http outbound gateway

I want to invoke an external REST endpoint POST request with a JSON payload which would be getting called from another service through the http inbound gateway.
I am using the below configuration for my application :
<int:channel id="xappSearchRequest" />
<int:channel id="xappSearchResponse" />
<int:channel id="xappFilterChannelOutput"/>
<int:channel id="discardFilterChannel"/>
<int:channel id="mutableMessageChannel"/>
<int:filter input-channel="mutableMessageChannel" output-channel="xappFilterChannelOutput" discard-channel="discardFilterChannel" ref="structureValidationFilter"/>
<int:transformer input-channel="xappSearchRequest" output-channel="mutableMessageChannel"
ref="mutableMessageTransformer" />
<int-http:inbound-gateway id="inboundxappSearchRequestGateway"
supported-methods="POST"
request-channel="xappSearchRequest"
reply-channel="xappSearchResponse"
mapped-response-headers="Return-Status, Return-Status-Msg, HTTP_RESPONSE_HEADERS"
path="${xapp.request.path}"
reply-timeout="50000"
request-payload-type="standalone.CFIRequestBody">
</int-http:inbound-gateway>
<int:service-activator id="xappServiceActivator"
input-channel="xappFilterChannelOutput"
output-channel="xappSearchResponse"
ref="xappSearchService"
method="handlexappRequest"
requires-reply="true"
send-timeout="60000"/>
<int:service-activator id="dicardPayloadServiceActivator"
input-channel="discardFilterChannel"
output-channel="xappSearchResponse"
ref="invalidPayloadService"
method="getInvalidMessage"
requires-reply="true"
send-timeout="60000"/>
<int-http:outbound-gateway id="get.outbound.gateway"
request-channel="get_send_channel" url="${cms.stub.request.url}"
http-method="POST" reply-channel="get_receive_channel"
expected-response-type="standalone.StubResponseBody">
</int-http:outbound-gateway>
Not able to figure out how to send a JSON payload and custom headers to call the POST endpoint.
Custom headers can be mapped to HTTP headers using an appropriate property - mapped-request-headers.
There is a full documentation on the matter: https://docs.spring.io/spring-integration/docs/5.2.3.RELEASE/reference/html/http.html#http-header-mapping
For JSON request the <int-http:outbound-gateway> is supplied with the RestTemplate which has a MappingJackson2HttpMessageConverter configured if you have a jackson-databind on classpath. Only what you need from your application is to send a POJO which can be serialized into a JSON and what is important - a MessageHeaders.CONTENT_TYPE header with an application/json value.
Figured that out .
I am using a outbound gateway like below :
<int-http:outbound-gateway id="get.outbound.gateway"
request-channel="postCmsChannel" url="${cms.stub.request.url}"
http-method="POST" reply-channel="zappSearchResponse"
expected-response-type="standalone.StubResponseBody">
</int-http:outbound-gateway>
The 'postCmsChannel' is the output-channel of service activator :
<int:service-activator id="zappServiceActivator"
input-channel="xappFilterChannelOutput"
output-channel="postCmsChannel"
ref="xappSearchService"
method="handleXappRequest"
send-timeout="60000"/>
The 'xappSearchService' is like below :
#Autowired
#Qualifier("postCmsChannel")
MessageChannel postCmsChannel;
public void handleXappRequest(Message<CFIRequestBody> inMessage){
/*
* Map<String, Object> responseHeaderMap = new HashMap<String, Object>();
*
* //MessageHeaders headers = inMessage.getHeaders(); StubResponseBody
* info=inMessage.getPayload();
*
* setReturnStatusAndMessage(responseHeaderMap); Message<StubResponseBody>
* message = new GenericMessage<StubResponseBody>(info, responseHeaderMap);
* return message;
*/
inMessage.getHeaders();
Map<String,String> headerMap=new HashMap<String,String>();
headerMap.put("X-JWS-SIGNATURE", "dshdgshdgasshgdywtwtqsabh232wgd7wdyt");
headerMap.put("X-PARTICIPANT-ID", "CMDRV2112BB");
postCmsChannel.send(MessageBuilder.withPayload(inMessage.getPayload())
.copyHeadersIfAbsent(inMessage.getHeaders()).copyHeadersIfAbsent(headerMap).build());
}
}

Problems with unit testing a spring integration program

I am trying to unit test an xpath router, but having problems with it. here is my context file:
<int:channel id="toTransactionTypeRouterChannel" />
<int-xml:xpath-router id="transactionsTypeRouter"
input-channel="toTransactionTypeRouterChannel" resolution-required="false"
evaluate-as-string="true" default-output-channel="errorChannel">
<!-- Select node name of the first child -->
<int-xml:xpath-expression
expression="name(/soapNs:Envelope/soapNs:Body/schNs:processArchiveRequest/schNs:fulfillmentRequest/schNs:requestDetail/*[1])"
namespace-map="archiveNamespaceMap" />
<int-xml:mapping value="sch:bulkRequestDetail"
channel="bulkChannel" />
<int-xml:mapping value="sch:transactionalRequestDetail"
channel="transactionChannel" />
</int-xml:xpath-router>
<int:channel id="bulkChannel" />
<int:channel id="transactionChannel" />
<int:transformer input-channel="bulkChannel"
output-channel="consoleOut" expression="'Bulk channel has received the payload' " />
<int:transformer input-channel="transactionChannel"
output-channel="consoleOut" expression="'Transaction channel has received payload' " />
<int:transformer input-channel="errorChannel"
output-channel="consoleOut" expression="'Error channel has received payload' " />
As you can see here, there are 2 different routes(bulk,trans.) + error channel.Here is my unit test case for trans channel route:
#Test
public void testTransactionFlow() throws Exception {
try {
Resource bulkRequest = new FileSystemResource("src/main/resources/mock-message-examples/SampleProcessArchiveTransRequest.xml");
String transRequestStr= extractResouceAsString(bulkRequest);
toTransactionTypeRouterChannel.send(MessageBuilder.withPayload(transRequestStr).build());
Message<?> outMessage = testChannel.receive(0);
assertNotNull(outMessage);
context file for junit
<int:bridge input-channel="transactionChannel"
output-channel="testChannel"/>
<int:channel id="testChannel">
<int:queue/>
</int:channel>
As you can see, in the junit context file, I am connecting transactional channel to the test channel.in the junit test case, I am sending a payload to the router in the junit method, and trying to receive it from the input channel and use it for assertion. however the assertion fails, as the message from transaction channel directly goes to consoleOut before getting routed to inputChannel as given in the junit coontext file. How do I intercept the message before it goes to consoleOut? I also tried adding wireTap interceptors but they didnt work:
WireTap wireTap = new WireTap(someChannel);
boolean w = wireTap.isRunning();
transactionChannel.addInterceptor(wireTap);
Basically, I need a separate flow for unit testing.
With that configuration, you are just adding a second consumer to transactionChannel - messages will be round-robin distributed to the transformer and bridge.
You can unsubscribe the transformer for your test case by autowiring it by id as an EventDrivenConsumer and stop() it before sending your message.

DeferredResult from JMS Request/Reply with ListenableFuture

I was looking for opportunities to improve the Rest Api we have exposed to external clients. During that exercise, I found I am barely taking advantage of the decision our integration team took to have backend integrations with JMS request/reply instead of traditional blocking SOAP request/reply.
Currently all the interactions to message broker are done using jmsOutboundGateway, because of which requesting thread has to wait for completion. In order to scale RestAPI, I want to send JMS reply using DeferredResult from Spring MVC controller. The controller interaction with message broker is depicted below:
Controller --> GatewayProxy --> JMSOutboundGateway
I am looking for opportunities to use ListenableFuture as return type of GatewayProxy, but I am unable to find a proper mean of achieving it using spring integration.
Below is the integration flow I am calling from controller:
<int:gateway
service-interface="ae.emaratech.ngx.service.PermitSearchService"
default-request-channel="permit_search_input_channel"
default-reply-timeout="${broker.jms.gateway.min.consumers}"/>
<int:channel id="permit_search_input_channel" />
<int:chain input-channel="permit_search_input_channel">
<int:header-enricher>
<int:header name="person_number" expression="payload"/>
</int:header-enricher>
<int:transformer expression="#formatString(#api_messages['FIND_PERM_BY_PERSNO_MSG'],headers)"/>
<int:header-filter header-names="JMS_*,jms_*,priority" pattern-match="true" />
<int:header-enricher>
<int:header name="jms_type" type="java.lang.String" value="1" overwrite="true"/>
</int:header-enricher>
<jms:outbound-gateway
request-destination="permitsInboundQueue"
reply-destination="permitsOutboundQueue"
receive-timeout="${broker.jms.gateway.timeout}"
correlation-key="Correlation_ID"
connection-factory="brokerConnectionFactory">
<jms:reply-listener concurrent-consumers="${broker.jms.gateway.min.consumers}" max-concurrent-consumers="${broker.jms.gateway.max.consumers}"/>
</jms:outbound-gateway>
<int-xml:xpath-filter throw-exception-on-rejection="true">
<int-xml:xpath-expression expression="not(boolean(/*/ErrorDetails))"/>
</int-xml:xpath-filter>
<int-xml:xslt-transformer
xsl-resource="classpath:/META-INF/spring/integration/permit-to-json.xsl"
result-type="StringResult" >
</int-xml:xslt-transformer>
<int:transformer expression="payload.toString()"/>
</int:chain>
Not sure what problem you have, but the feature looks like:
ListenableFuture<String> result = this.asyncGateway.async("foo");
result.addCallback(new ListenableFutureCallback<String>() {
#Override
public void onSuccess(String result) {
...
}
#Override
public void onFailure(Throwable t) {
...
}
});
It is available since version 4.1.

Spring Integration Http with Spring Boot and #RequestMapping

i try to develop a SpringBoot Rest Server with Spring Integration HTTP -> inboundGateway.
I've an Controller, annotated with "#Controller" and "#RequestMapping" and try to create following flow:
GET Request "/" -> Channel: httpRequestChannel -> Run IndexController -> Channel: httpReplyChannel -> back to Browser
But it's not working.
My Integration Xml:
<int:channel id="httpRequestChannel">
<int:interceptors>
<int:wire-tap channel="logHttpRequestChannel" />
</int:interceptors>
</int:channel>
<int:channel id="httpReplyChannel">
<int:interceptors>
<int:wire-tap channel="logHttpReplyChannel" />
</int:interceptors>
</int:channel>
<int:logging-channel-adapter id="logHttpRequestChannel" level="INFO" logger-name="httpRequestChannel" log-full-message="true" />
<int:logging-channel-adapter id="logHttpReplyChannel" level="INFO" logger-name="httpReplyChannel" log-full-message="true" />
<int-http:inbound-gateway id="inboundGateway"
request-channel="httpRequestChannel" reply-channel="httpReplyChannel"
auto-startup="true" supported-methods="GET" path="/">
<int-http:request-mapping produces="application/json" />
</int-http:inbound-gateway>
The error is:
Dispatcher has no subscribers
But in my opinion, the controller should be an subscriber via the RequestMapping Annotation...
I upload a sample github project: https://github.com/marcelalburg/spring-boot-integration-rest-server
Thanks for you help
Marcel
UPDATE
Hello,
i see something in the documentation:
The parsing of the HTTP Inbound Gateway or the HTTP Inbound Channel Adapter registers an integrationRequestMappingHandlerMapping bean of type IntegrationRequestMappingHandlerMapping, in case there is none registered, yet. This particular implementation of the HandlerMapping delegates its logic to the RequestMappingInfoHandlerMapping. The implementation provides similar functionality as the one provided by the org.springframework.web.bind.annotation.RequestMapping annotation in Spring MVC.
So, i changed following:
<int-http:inbound-gateway id="indexGateway"
request-channel="httpRequestChannel" reply-channel="httpReplyChannel"
auto-startup="true" supported-methods="GET" path="/, /test" reply-timeout="100" />
and my controller
#ServiceActivator( inputChannel = "httpRequestChannel", outputChannel = "httpReplyChannel" )
#RequestMapping( value = "/", method = RequestMethod.GET, produces = "application/json" )
public String testGateway( LinkedMultiValueMap payload, #Headers Map<String, Object> headerMap )
{
// IntegrationRequestMappingHandlerMapping
System.out.println( "Starting process the message [reciveing]" );
return "{HelloMessage: \"Hello\"}";
}
#ServiceActivator( inputChannel = "httpRequestChannel", outputChannel = "httpReplyChannel" )
#RequestMapping( value = "/test", method = RequestMethod.GET, produces = "application/json" )
public String testGateway2( LinkedMultiValueMap payload, #Headers Map<String, Object> headerMap )
{
// IntegrationRequestMappingHandlerMapping
System.out.println( "Starting process the message [reciveing]" );
return "{HelloMessage: \"Test\"}";
}
now, i get an response but it returns randomized "Test" and "Hello" ...
Thanks
No; you seem to have a basic misunderstanding.
With Spring Integration, the inbound-gateway replaces the#Controller, and sends the inbound (possibly converted) object as the payload of a message to the requestChannel.
Some other component (not a controller) subscribes to that channel to receive the message.
So, instead of configuring an #Controller you can either configure your POJO as a <service-activator input-channel="httpRequestChannel" .../> or annotate the method as a #ServiceActivator.
It will then consume the message and, optionally, send the reply to the output channel (omitting the output channel will cause it to be routed back to the gateway).
See the http sample for an example.

Spring splitter/aggregator handling exceptions

Version : spring-integration-core - 2.2.3
Here is the simplified version of my splitter/aggregator setup.
<task:executor id="taskExecutor" pool-size="${pool.size}"
queue-capacity="${queue.capacity}"
rejection-policy="CALLER_RUNS" keep-alive="120"/>
<int:channel id="service-requests"/>
<int:channel id="service-request"/>
<int:channel id="channel-1">
<int:dispatcher task-executor="taskExecutor" failover="false"/>
</int:channel>
<int:channel id="channel-2">
<int:dispatcher task-executor="taskExecutor" failover="false"/>
</int:channel>
<int:gateway id="myServiceRequestor" default-reply-timeout="${reply.timeout}"
default-reply-channel="service-aggregated-reply"
default-request-channel="service-request"
service-interface="com.blah.blah.MyServiceRequestor"/>
<int:splitter input-channel="service-request"
ref="serviceSplitter" output-channel="service-requests"/>
<!-- To split the request and return a java.util.Collection of Type1 and Type2 -->
<bean id="serviceSplitter" class="com.blah.blah.ServiceSplitter"/>
<int:payload-type-router input-channel="service-requests" resolution-required="true">
<int:mapping
type="com.blah.blah.Type1"
channel="channel-1"/>
<int:mapping
type="com.blah.blah.Type2"
channel="channel-2"/>
</int:payload-type-router>
<!-- myService is a bean where processType1 & processType2 method is there to process the payload -->
<int:service-activator input-channel="channel-1"
method="processType1" output-channel="service-reply" requires-reply="true"
ref="myService"/>
<int:service-activator input-channel="channel-2"
method="processType2" output-channel="service-reply" requires-reply="true"
ref="myService"/>
<int:publish-subscribe-channel id="service-reply" task-executor="taskExecutor"/>
<!-- myServiceAggregator has a aggregate method which takes a Collection as argument(aggregated response from myService) -->
<int:aggregator input-channel="service-reply"
method="aggregate" ref="myServiceAggregator"
output-channel="service-aggregated-reply"
send-partial-result-on-expiry="false"
message-store="myResultMessageStore"
expire-groups-upon-completion="true"/>
<bean id="myResultMessageStore" class="org.springframework.integration.store.SimpleMessageStore" />
<bean id="myResultMessageStoreReaper" class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="myResultMessageStore" />
<property name="timeout" value="2000" />
</bean>
<task:scheduled-tasks>
<task:scheduled ref="myResultMessageStoreReaper" method="run" fixed-rate="10000" />
</task:scheduled-tasks>
If the processType1/processType2 method in mySevice throws a RuntimeException, then it tries to send the message to an error channel(i believe spring does it by default) and the message payload in error channel stays on in heap and not getting garbage collected.
Updated More Info:
For my comment on error channel. I debugged the code and found that ErrorHandlingTaskExecutor is trying to use a MessagePublishingErrorHandler which inturn sending the message to the channel returned by MessagePublishingErrorHandler.resolveErrorChannel method.
Code snippet from ErrorHandlingTaskExecutor.java
public void execute(final Runnable task) {
this.executor.execute(new Runnable() {
public void run() {
try {
task.run();
}
catch (Throwable t) {
errorHandler.handleError(t); /// This is the part which sends the message in to error channel.
}
}
});
}
Code snipper from MessagePublishingErrorHandler.java
public final void handleError(Throwable t) {
MessageChannel errorChannel = this.resolveErrorChannel(t);
boolean sent = false;
if (errorChannel != null) {
try {
if (this.sendTimeout >= 0) {
sent = errorChannel.send(new ErrorMessage(t), this.sendTimeout);
.....
When i take a heap dump, I always see the reference to the payload message(which i believe is maintained in the above channel) and not getting GC'ed.
Would like to know what is the correct way to handle this case or if i'm missing any in my config?
Also is it possible to tell spring to discard the payload(instead of sending it to error channel) in case of any exception thrown by the service activator method?
Looking forward for your inputs.
Thanks.
You don't have an error-channel defined on your gateway so we won't send it there, we'll just throw an exception to the caller.
However, the partial group is sitting in the aggregator and will never complete. You need to configure a MessageGroupStoreReaper as shown in the reference manual (or set a group-timeout in Spring Integration 4.0.x) to discard the partial group.

Resources