Related
this is my first camel HTTP4 implementation and having trouble. I get no response as if the route to URI is wrong and message goes ??? it doesn't make the HTTP4 call it appears?
using camel-version 2.17.0.redhat-630310, java 8
my pom has dependency
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-http4</artifactId>
</dependency>
my route
<route
id="core.getToken.route"
autoStartup="true" >
<from id="getToken" ref="getToken" />
<process ref="uAARequestTokenProcessor" />
<log message="Message after uAARequestTokenProcessor: ${body}" loggingLevel="INFO"/>
<setHeader headerName="CamelHttpMethod">
<constant>POST</constant>
</setHeader>
<to uri="http4://dummyhost" />
<log message="After HTTP4 POST: ${body}" loggingLevel="INFO"/>
<to uri="{{accessToken}}" />
</route>
I call the uAARequestTokenProcessor to condition the outbound message.
public class UAARequestTokenProcessor implements Processor {
private static final Logger LOG = LoggerFactory.getLogger(UAARequestTokenProcessor.class);
#Override
public void process(Exchange exchange) throws Exception {
LOG.info("Entering UAA Request Token Processor: start " );
final String clientId = (String)exchange.getIn().getHeader("CLIENTID");
final byte[] ClientId = clientId.getBytes("UTF-8");
final String userName = (String)exchange.getIn().getHeader("USERNAME");
final String password = (String)exchange.getIn().getHeader("PASSWORD");
final String tokenUrl = (String)exchange.getIn().getHeader("TOKENURL");
LOG.info("ClientId: " + new String(ClientId));
LOG.info("userName: " + userName);
LOG.info("password: " + password);
LOG.info("tokenUrl: " + tokenUrl);
LOG.info("Processing UAA token request for Client ID: " + clientId + " and User Name: " + userName);
LOG.info("Before Base64 encryption of Client ID: " + new String(ClientId));
StringBuilder authHeader = new StringBuilder("Basic ");
authHeader.append(Base64.getEncoder().encodeToString(ClientId));
LOG.info("after Base64 encryption of Client ID: " + authHeader.toString());
String body = String.format("grant_type=password&username=%s&password=%s",
URLEncoder.encode(userName, "UTF-8"), //Translates a string into x-www-form-urlencoded format
URLEncoder.encode(password, "UTF-8"));
LOG.info("HTTP BODY: " + body);
exchange.getIn().setHeader(Exchange.HTTP_URI, tokenUrl);
exchange.getIn().setHeader(Exchange.CONTENT_TYPE, "MediaType.APPLICATION_FORM_URLENCODE");
exchange.getIn().setHeader("Authorization", authHeader.toString());
exchange.getIn().setBody(body);
LOG.info("Exchange.HTTP_URI: " + exchange.getIn().getHeader("Exchange.HTTP_URI"));
LOG.info("Exchange.CONTENT_TYPE: " + exchange.getIn().getHeader("Exchange.CONTENT_TYPE"));
LOG.info("Authorization: " + exchange.getIn().getHeader("Authorization"));
LOG.info("Exiting UAA Request Token Processor: Finish " );
}
}
it looks like to me that the Exchange.HTTP_URI or HTTP headers are not getting set unless the headers for HTTP4 are different? but I'm following the HTTP4 doc, https://github.com/apache/camel/blob/master/components/camel-http4/src/main/docs/http4-component.adoc
what happens is inside the processor I try to read the HTTP headers and it shows they are null, after being set.
Entering UAA Request Token Processor: start
ClientId: ingestor.57e72dd3-6f9e-4931-b4bc-cd04eaaff3e3.1f7dbe12-2372-439e-8104-06a5f4098ec9
userName: 5c0642fe-a495-44db-93f7-67056fa2c061_ingestor
password: 154f0252d166f27b5e21ef171b02a79f41a0daf3
tokenUrl: http4://d1e53858-2903-4c21-86c0-95edc7a5cef2.predix-uaa.run.aws-usw02-pr.ice.predix.io/oauth/token
Processing UAA token request for Client ID: ingestor.57e72dd3-6f9e-4931-b4bc-cd04eaaff3e3.1f7dbe12-2372-439e-8104-06a5f4098eff9 and User Name: 5c0232fe-a495-44db-94f7-67156fa2c061_ingestor
Before Base64 encryption of Client ID: ingestor.57e72dd3-6f9e-4931-b4bc-cd04eaaff3e3.1f7dbe12-2372-439e-8104-06a5f4098eff9
after Base64 encryption of Client ID: Basic aW5nZXN0b3IuNTdlNzJkZDMtNmY5ZS00OTMxLWI0YmMtY2QwNGVhYWZmM2UzLjFmN2RiZTEyLTIzNzItNDM5ZS04MTA0LTA2YTVmNDA5OGVjOQ==
HTTP BODY: grant_type=password&username=5c0232fe-a495-44db-94f7-67156fa2c061_ingestor&password=154f0252d166f27b5e21ef171b02beebop
Exchange.HTTP_URI: null
Exchange.CONTENT_TYPE: null
Authorization: Basic aW5nZXN0b3IuNTdlNzJkZDMtNmY5ZS00OTMxLWI0YmMtY2QwNGVhYWZmM2UzLjFmN2RiZTEyLTIzNzItNDM5ZS04MTA0LTA2YTVmNDA5OGVjOQ==
Exiting UAA Request Token Processor: Finish
The stack trace shows headers but not as EXCHANGE.HTTP_URI
2018-04-23 13:17:29,203 | INFO | ile://ge-ip/core | Tracer | 231 - org.apache.camel.camel-core - 2.17.0.redhat-630310 | ID-alphprdfuse2i-44477-1524503724804-3-52 >>>
(core.getToken.route) setHeader[CamelHttpMethod, POST] --> http4://predhost <<< Pattern:InOnly,
Headers:{Authorization=Basic aW5nZXN0b3IuNTdlNzJkZDMtNmY5ZS00OTMxLWI0YmMtY2QwNGVhYWZmM2UzLjFmN2RiZTEyLTIzNzItNDM5ZS04MTA0LTA2YTVmNDA5OGVjOQ==,
breadcrumbId=ID-alphprdfuse2i-44477-1524503724804-3-50,
CamelFileAbsolute=false,
CamelFileAbsolutePath=/app/iprctest/jboss-fuse-6.3.0.redhat-310/ge-ip/core/Noble/nst_dge-1_2018-04-10_19-55-10.CSV, CamelFileLastModified=1524503800000,
CamelFileLength=3706, CamelFileName=Noble/nst_dge-1_2018-04-10_19-55-10.CSV, CamelFileNameConsumed=Noble/nst_dge-1_2018-04-10_19-55-10.CSV, CamelFileNameOnly=nst_dge-1_2018-04-10_19-55-10.CSV,
CamelFileNameProduced=ge-ip/upload/Noble/nst_dge-1_2018-04-10_19-55-10.CSV, CamelFileParent=ge-ip/core/Noble,
CamelFilePath=ge-ip/core/Noble/nst_dge-1_2018-04-10_19-55-10.CSV, CamelFileRelativePath=Noble/nst_dge-1_2018-04-10_19-55-10.CSV, CamelHttpMethod=POST,
CamelHttpUri=http4://d1e53858-2903-4c21-86c0-95edc7a5cef2.pred-uaa.run.aws-usw02-pr.ice.pred.io/oauth/token,
CLIENTID=ingestor.57e72dd3-6f9e-4931-b4bc-cd04eaaff3e3.1f7dbe12-2372-439e-8104-06a5f4098ec9,
Content-Type=MediaType.APPLICATION_FORM_URLENCODE,
CUSTKEY=Noble, PASSWORD=154f0252d166f27b5e21a79f41a0daf3,
TENANTUUID=c0642fe-a495-44db-93f7-67056fa2c061,
TOKENURL=http4://d1e53858-2903-4c21-86c0-95edc7f2.pred-uaa.run.aws-usw02-pr.ice.pred.io/oauth/token,
UPLOADURL=http4://apm-times-query-svc-prod.app-api.aws-usw02-pr.pred.io/v2/time_series/upload,
USERNAME=5c0642fe-a495-44db-93f7-67056fa2c061_ingestor},
BodyType:String, Body:grant_type=password&username=5c0642fe-a495-44db-93f7-2c061_ingestor&password=154f0252d166f2ef171b02a79f41a0daf3
THE http4://predhost is a dummy URI in the spring XML and should be over ridden by the exchange.getIn().setHeader(Exchange.HTTP_URI, tokenUrl);
I never reach the URL endpoint and never reach the log message after making the to HTTP4://
<to uri="http4://predixhost" />
<log message="After HTTP4 POST: ${body}" loggingLevel="INFO"/>
I have tried the following but still get no response, no exceptions, nothing. setting the header CamelHttpUri to overirde the default dummy uri, and a dynamic toD, ???
<route
id="core.getToken.route"
autoStartup="true" >
<from id="getToken" ref="getToken" />
<process ref="uAARequestTokenProcessor" />
<!-- <log message="Message after uAARequestTokenProcessor: ${body}" loggingLevel="INFO"/> -->
<setHeader headerName="CamelHttpMethod">
<constant>POST</constant>
</setHeader>
<!-- <setHeader headerName="CamelHttpUri">
<simple>${header.TOKENURL}</simple>
</setHeader> -->
<toD uri="${header.TOKENURL}" />
<log message="After HTTP4 POST: ${body}" loggingLevel="INFO"/>
<to uri="{{accessToken}}" />
</route>
I don't know what is wrong? I can't get a response, an exception, it looks like I cannot send to the HTTP4 endpoint. I am reaching now, verified my proxy settings in the environment, and also tried in the camel context, I don't know what is going on or what to do, I get no logging info to go on, ...
<camelContext
id="com.passthru.coreCamelContext"
trace="true"
xmlns="http://camel.apache.org/schema/blueprint"
allowUseOriginalMessage="false"
streamCache="true"
errorHandlerRef="deadLetterErrorHandler" >
<properties>
<property key="http.proxyHost" value="PITC-Zscaler-Americas-Cin.proxy.corporate.comp.com"/>
<property key="http.proxyPort" value="80"/>
</properties>
<streamCaching id="CacheConfig"
spoolUsedHeapMemoryThreshold="70"
anySpoolRules="true"/>
I've tried with CURL and it works from command line, I've tried with Postman and it works. thanks for your help.
What this is to do, simply POST a message to an UAA(https) service with a body of:
grant_type=password&username=myUserName&password=myPassword
and headers:
Content-Type=application/x-www-form-urlencoded
Authorization=Basic aW5nZXN0b3IuNTdlNzJkZDMt==
and the server will issue an access token.
like this postman example:
OkHttpClient client = new OkHttpClient();
MediaType mediaType = MediaType.parse("application/x-www-form-urlencoded");
RequestBody body = RequestBody.create(mediaType, "username=93f7c061_ingestor&password=15a79f41a0daf3&grant_type=password");
Request request = new Request.Builder()
.url("https://d1e53858-2903-4c21-86c0-95edc7a5cef2.uaa.run.aws-usw02-pr.ice.io/oauth/token")
.post(body)
.addHeader("Content-Type", "application/x-www-form-urlencoded")
.addHeader("Authorization", "Basic aW5nZXN0b3IuNTdlNzJkZDMtNmY5ZS00OTMxLWI0YmMtY2QwNGVhYWZmM2UzLjFmN2RiZTEyLTIzNzItNDM5ZS04MTA0LTA2YTVmNDA5OGVjOTo=")
.addHeader("Cache-Control", "no-cache")
.addHeader("Postman-Token", "ad8ba9b0-6215-4c53-a6ef-b3a5bf7901f6")
.build();
Response response = client.newCall(request).execute();
Hello I hope there is someone out there that can help, I'm stuck.
I have confirmed, I believe that the HTTP4 endpoint is not working. I have a deadletter queue
<bean id="deadLetterErrorHandler" class="org.apache.camel.builder.DeadLetterChannelBuilder">
<property name="deadLetterUri" value="${deadLetterQueue}"/>
<property name="redeliveryPolicy" ref="redeliveryPolicyConfig"/>
</bean>
<bean id="redeliveryPolicyConfig" class="org.apache.camel.processor.RedeliveryPolicy">
<property name="maximumRedeliveries" value="3"/>
</bean>
and each exchange sent to this endpoint is tried 3x and then sent to the deadletter queue. the content of the message in the deadletter queue is what should be sent to the HTTP4 endpoint and thus the UAA service.
├── deadletterqueue
│ ├── ID-alphprdfuse2i-36326-1524606956414-3-65
grant_type=password&username=5c0642fe_ingestor&password=02a79f41a0daf3
the message headers leading in to this look good too.
core.getToken.route) log[HTTP4 POST body: ${body}] --> http4://d1e53858-2903-4c21-86c0-95edc7a5cef2.uaa.run.aws-usw02.io:443/oauth/token?throwExceptionOnFailure=false <<< Pattern:InOnly, Headers:{Authorization=Basic aW5nZXN0b3IuNTdlNzJkZDMtNmY5ZS00OTMxLWI0YmMtY2QwNGVhYWZmM2UzLjFmN2RiZTEyLTIzNzItNDM5ZS04MTA0LTA2YTVmNDA5OGVjOQ==, CamelHttpCharacterEncoding=UTF-8, CamelHttpMethod=POST, CamelHttpUri=http4://d1e53858-2903-4c21-86c0-95edc7a5cef2.uaa.run.aws.io:443/oauth/token, Content-Type=MediaType.APPLICATION_FORM_URLENCODE, TOKENURL=http4://d95edc7a5cef2.uaa.run.aws-usw02-pr.ice.io:443/oauth/token}, BodyType:String, Body:grant_type=password&username=5c06-a495-44db-93f7-67056fa2c061_ingestor&password=7b5e21ef171b02a79f41a0daf3
I'm convinced it must be something wrong with the way I'm implementing HTTP4. I do not need to expose a web service I only need to push to it. I can't get this to work. please help.
ok, this was a multi faceted problem.
1st my proxy setting was incorrect. I found it has to be set with either the IP or the name, cononical name without the protocol http://.
I had it like:
<camelContext
id="com.ge.digital.passthru.coreCamelContext"
trace="true"
xmlns="http://camel.apache.org/schema/blueprint"
allowUseOriginalMessage="false"
streamCache="true"
errorHandlerRef="deadLetterErrorHandler" >
<properties>
<property key="http.proxyHost" value="http://PITC-Zscaler-Americas.proxy.corporate.com"/>
<property key="http.proxyPort" value="80"/>
</properties>
but as mentioned this apparently is incorrect and do not use the protocol in the def. like:
<properties>
<property key="http.proxyHost" value="PITC-Zscaler-Americas.proxy.corporate.com"/>
<property key="http.proxyPort" value="80"/>
</properties>
or
<properties>
<property key="http.proxyHost" value="123.123.123.123"/>
<property key="http.proxyPort" value="80"/>
</properties>
this gave me a response, not the one I was hoping for but one that gave me hope.
this gave me a 504 gateway timeout, wow, i was getting something back. :)
2nd, The endpoint over ride wasn't working...
m.setHeader(Exchange.HTTP_URI, tokenUrl);
using
<toD uri="${header.TOKENURL}?throwExceptionOnFailure=false" />
<toD uri="${header.TOKENURL}" />
so I just tried a simple to in the route
<to uri="http4://d1e53858-uaa.run.aws-usw02-pr.ice.io:443/oauth/token?throwExceptionOnFailure=false" />
with the over ride
m.setHeader(Exchange.HTTP_URI, tokenUrl);
where the tokenUrl was defined as
http4://d1e53858-uaa.run.aws-usw02-pr.ice.io:443/oauth/token
this gives me a 504, I was hoping this was something I could take care of on my side, usually isn't but sometimes... so I kept trying things... until... I used the actual address in the HTTP_URI like...
https://d1e53858-uaa.run.aws-usw02-pr.ice.io/oauth/token
and now I am getting a CamelHttpResponseCode=401, CamelHttpResponseText=Unauthorized
I'm one happy camper, this is something I can work with... I then tested the same substitution setting the URI in the route in xml. and this worked also.
<setHeader headerName="CamelHttpMethod">
<constant>POST</constant>
</setHeader>
<log message="HTTP4 POST headers: ${headers}" loggingLevel="DEBUG"/>
<setHeader headerName="CamelHttpUri">
<simple>${header.TOKENURL}?throwExceptionOnFailure=false</simple>
</setHeader>
<to uri="http4://d1e53858-uaa.run.aws-usw02-pr.ice.io/oauth/token?throwExceptionOnFailure=false" />
I am still trying to get the dynamic route working, but I'm satisfied with what I have and will focus more on that later.
<toD uri="${header.TOKENURL}?throwExceptionOnFailure=false" />
<toD uri="${header.TOKENURL}" /> -->
thank you everyone, I hope my struggles as a newbie help others, I can't be the only one with these silly happenings. anyway, cheers all.
We have the below XML configuration for a rest service and Oracle Stored Proc. The use case is about handling Oracle SP errors.
When the Oracle SP throws error, we need to achieve 2 things:
we need to send a json response back to the client in the format
{"status": false,"message": "Oracle SP error payload"}
Send an email out.
The configuration is working as expected when Oracle SP throws error. But when the email send fails ( gave a wrong host), client is receiving error in the below format. The Oracle SP error message is not sent back.
---- Json message received by client when email send fails:------
{
"timestamp": 1501639940143,
"status": 500,
"error": "Internal Server Error",
"exception": "org.springframework.messaging.MessagingException",
"message": "failure occurred in error-handling flow; nested exception is org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.handler.MessageHandlerChain#0$child#2.handler]; nested exception is org.springframework.mail.MailSendException: Mail server connection failed; nested exception is javax.mail.MessagingException: Unknown SMTP host: xxx.com;..",
"path": "/init/5c82d1f4-e550-4bb1-be9c-8d0ddbc7f4401/NEW"
}
How do I send response in the expected json format - {"status": false,"message": Oracle SP error payload}?
Also how to handle the email send error.i.e may be how to store the email error in a DB table?
XML Configuration:
<int:channel id="spOutClobChannel"/>
<int:publish-subscribe-channel id="gatewayErrorChannel"/>
<int-http:inbound-gateway
request-channel="gatewayInitChannel"
error-channel="gatewayErrorChannel"
supported-methods="GET"
path="/init/{refId}/{refStatus}"
payload-expression="#pathVariables.refStatus">
<int-http:header name="UNIQUE_PROCESS_ID" expression="#pathVariables.refId"/>
</int-http:inbound-gateway>
<int-jdbc:stored-proc-outbound-gateway
id="sp-get-data"
data-source="dataSource"
request-channel="gatewayInitChannel"
reply-channel="spOutClobChannel".... >
...
...
</int-jdbc:stored-proc-outbound-gateway>
<int:transformer input-channel="spOutClobChannel" expression='{"status": true, "message": "RECEIVED"}'/>
<int:transformer input-channel="gatewayErrorChannel" expression='{"status": false,"message": payload.message}'/>
<int:transformer expression="payload.failedMessage.headers['UNIQUE_PROCESS_ID']"
input-channel="gatewayErrorChannel" output-channel="appEmailChannel"/>
<int:chain input-channel="appEmailChannel" >
<int-mail:header-enricher>
<int-mail:from value="${email.from}"/>
<int-mail:to value="${email.to}"/>
<int-mail:subject value="${email.subject}"/>
<int-mail:content-type value="text/html"/>
</int-mail:header-enricher>
<int-mail:outbound-channel-adapter host="${email.host}" />
</int:chain>
To avoid errors thrown from the appEmailChannel flow, plus get a gain do not wait for the successful email shipment, it would be better to add to your gatewayErrorChannel an executor configuration. This way all the subscribers to this publish-subscriber channel will be performed in parallel and in their own threads. Therefore any exception in those threads won't impact the reply to the <int-http:inbound-gateway> which you've done with the
<int:transformer input-channel="gatewayErrorChannel" expression='{"status": false,"message": payload.message}'/>
Also how to handle the email send error.i.e may be how to store the email error in a DB table?
With this goal you can use ExpressionEvaluatingRequestHandlerAdvice with its trapException = true and failureChannel properties.
This advice can be configured with the request-handler-advice-chain sub-element.
See Reference Manual and Sample on the matter.
I'm trying to pass some custom HTTP headers using <int-http:outbound-gateway> and <int:header-enricher> components of Spring Integration, but it doesn't work on my configuration.
I have tried two configurations. The first one with the "mapped-request-headers" attribute:
<int:chain input-channel="requestChannel" output-channel="replyChannel">
<int:header-enricher>
<int:header name="test" value="test"></int:header>
</int:header-enricher>
<int-http:outbound-gateway id="gateway"
encode-uri="true" url="http://localhost:8080/webapp/service/{request}"
http-method="GET" mapped-request-headers="test, HTTP_REQUEST_HEADERS">
<int-http:uri-variable name="request" expression="payload"/>
</int-http:outbound-gateway>
</int:chain>
The second one with the "header-mapper" attribute, with the relative DefaultHttpHeaderMapper configuration:
<int:chain input-channel="requestChannel" output-channel="replyChannel">
<int:header-enricher>
<int:header name="test" value="test"></int:header>
</int:header-enricher>
<int-http:outbound-gateway id="gateway"
encode-uri="true" url="http://localhost:8080/webapp/service/{request}"
http-method="GET" header-mapper="headerMapper">
<int-http:uri-variable name="request" expression="payload"/>
</int-http:outbound-gateway>
</int:chain>
<bean id="headerMapper" class="org.springframework.integration.http.support.DefaultHttpHeaderMapper">
<property name="outboundHeaderNames" value="HTTP_REQUEST_HEADERS, test" />
<property name="userDefinedHeaderPrefix" value="" />
</bean>
But in both cases, the message's headers loaded by the remote application are the following:
GenericMessage[
payload={},
headers={
http_requestMethod=GET,
replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#4c8b7a27,
errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#4c8b7a27,
host=localhost: 8080,
http_requestUrl=http://localhost:8080/webapp/service/hello,
connection=keep-alive,
id=3c2a21ba-b7f5-e5e9-c821-45251d406318,
cache-control=no-cache,
pragma=no-cache,
user-agent=Java/1.8.0_65,
accept=[
text/html,
image/gif,
image/jpeg,
*/*;q=.2,
*/*;q=.2],
timestamp=1469009855797
}
]
There is no trace of the "test" header that I'm trying to add to the request message.
What's the right way to add a custom header to the <int-http:outbound-gateway> ?
It's also good any other working solution.
UPDATE
As from Gary indication, I turned on the DEBUG mode.
This is a very interesting thing: the log says that the header [test] "will be mapped":
message sent: GenericMessage [payload=hello, headers={id=79f98422-418b-f00f-1e21-09461b1ff80c, timestamp=1469021452494}]
2016-07-20 15:53:57 DEBUG org.springframework.integration.http.support.DefaultHttpHeaderMapper.fromHeaders:402 (executor-4) - outboundHeaderNames=[test, HTTP_REQUEST_HEADERS]
2016-07-20 15:53:57 DEBUG org.springframework.integration.http.support.DefaultHttpHeaderMapper.shouldMapHeader:537 (executor-4) - headerName=[test] WILL be mapped, matched pattern=test
2016-07-20 15:53:57 DEBUG org.springframework.integration.http.support.DefaultHttpHeaderMapper.fromHeaders:420 (executor-4) - setting headerName=[X-test], value=test
2016-07-20 15:53:57 DEBUG org.springframework.http.client.support.HttpAccessor.createRequest:79 (executor-4) - Created GET request for "http://localhost:8080/webapp/service/hello"
I have implemented a simple Filter on the server side that intercepts every HTTP request, and it prints on the log every header of that request.
This is the result:
x-test=test
cache-control=no-cache
pragma=no-cache
user-agent=Java/1.8.0_65
host=localhost:8080
accept=text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
connection=keep-alive
So, the header "x-test" is present on the HttpServletRequest instance.
But after that, in the log I can see this other message (I have reported only those significant for this context):
2016-07-20 15:53:57 DEBUG org.springframework.web.servlet.DispatcherServlet.doService:865 (http-nio-8080-exec-1) - DispatcherServlet with name 'dispatcherServer' processing GET request for [/webapp/service/hello]
2016-07-20 15:53:57 DEBUG org.springframework.integration.http.support.DefaultHttpHeaderMapper.toHeaders:436 (http-nio-8080-exec-1) - inboundHeaderNames=[Accept, Accept-Charset, Accept-Encoding, Accept-Language, Accept-Ranges, Authorization, Cache-Control, Connection, Content-Length, Content-Type, Cookie, Date, Expect, From, Host, If-Match, If-Modified-Since, If-None-Match, If-Range, If-Unmodified-Since, Max-Forwards, Pragma, Proxy-Authorization, Range, Referer, TE, Upgrade, User-Agent, Via, Warning]
2016-07-20 15:53:57 DEBUG org.springframework.integration.http.support.DefaultHttpHeaderMapper.shouldMapHeader:561 (http-nio-8080-exec-1) - headerName=[x-test] WILL NOT be mapped
2016-07-20 15:53:57 DEBUG org.springframework.integration.handler.AbstractMessageHandler.handleMessage:115 (executor-4) - ServiceActivator for [org.springframework.integration.handler.MethodInvokingMessageProcessor#6ae60a] (channelServiceActivator) received message: GenericMessage [payload={}, headers={http_requestMethod=GET, replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#565fe0d8, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#565fe0d8, host=localhost:8080, http_requestUrl=http://localhost:8080/webapp/service/hello, connection=keep-alive, id=9578f82d-7fc2-fe94-d8bf-3225ec955b22, cache-control=no-cache, pragma=no-cache, user-agent=Java/1.8.0_65, accept=[text/html, image/gif, image/jpeg, */*;q=.2, */*;q=.2], timestamp=1469022837172}]
Now the log says that the header "[x-test] WILL NOT be mapped". How is possibile? It's because it doesn't appear in the inboundHeaderNames ?
In fact, the header seems to be "disappeared" in the GenericMessage:
GenericMessage [
payload={},
headers={
http_requestMethod=GET,
replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#16b10fb6,
errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#16b10fb6,
host=localhost:8080,
http_requestUrl=http://localhost:8080/webapp/service/hello,
connection=keep-alive,
id=ba2af515-9e29-4d84-d176-5b9d0d066cb0,
cache-control=no-cache,
pragma=no-cache,
user-agent=Java/1.8.0_65,
accept=[text/html,
image/gif,
image/jpeg,
*/*;q=.2,
*/*;q=.2],
timestamp=1469021452542
}
]
I don't understand why the header in present in the HTTP Request, but it's no present in the headers of the Message channel.
I have found the answer for this issue.
The <int-http:outbound-gateway> maps the headers that had to be inserted in the "exiting" HTTP requests.
The <int-http:inbound-gateway> maps the headers that are contained in the "entering" HTTP requests.
So, it has to be defined a "header-mapper" that:
maps the "inboundHeaderNames" for the <int-http:inbound-gateway>
maps the "outboundHeaderNames" for the <int-http:outbound-gateway>
In my example case, the solution for the "OUTBOUND SIDE" (the client application) was:
<int:chain input-channel="requestChannel" output-channel="replyChannel">
<int:header-enricher>
<int:header name="test" value="test"></int:header>
</int:header-enricher>
<int-http:outbound-gateway id="gateway"
encode-uri="true" url="http://localhost:8080/webapp/service/{request}"
http-method="GET" header-mapper="headerMapper">
<int-http:uri-variable name="request" expression="payload"/>
</int-http:outbound-gateway>
</int:chain>
<bean id="headerMapper" class="org.springframework.integration.http.support.DefaultHttpHeaderMapper">
<property name="outboundHeaderNames" value="HTTP_REQUEST_HEADERS, test" />
<property name="userDefinedHeaderPrefix" value="" />
</bean>
The "outboundHeaderNames" has to be setted with the header names that has to be mapped in the "outbound requests" ("test" in this case).
While the solution for the "INBOUND SIDE" (the receiver application) was:
<int-http:inbound-gateway id="gateway" request-channel="requestChannel"
path="/service/**" supported-methods="GET"
reply-channel="outputChannel" header-mapper="headerMapper">
</int-http:inbound-gateway>
<bean id="headerMapper" class="org.springframework.integration.http.support.DefaultHttpHeaderMapper">
<property name="inboundHeaderNames" value="*" />
</bean>
The "inboundHeaderNames" setted to "*" ensure that all the headers of the source request will be mapped in the Message headers.
With this approach, we can pass custom headers to any URL in HTTP requests, and we can recover them on the server side in the message channel context.
Turn on DEBUG logging; you should see messages about header mapping like this...
08:46:44.987 DEBUG [main] ... headerName=[id] WILL NOT be mapped
08:46:44.987 DEBUG [main] ... headerName=[test] WILL be mapped, matched pattern=test
08:46:44.987 DEBUG [main] ... setting headerName=[X-test], value=foo
(This is with your first example - custom headers currently get the X- prefix by default).
My requirement is to authenticate the user (with an external ws) as part of my Mule Flow and use the result of the authentication in a cache. however, if the user credentials change, then the cache should be automatically invalidated and user auth must be done with the external ws. Basically the cache key should be based on the user credentials. Is this possible ?
Here's my mule flow and i see that Mule is caching the results after the first request and irrespective of whether or not the payload changes in subsequent requests ( which is where the credentials are sent ) mule always returns the results from the cache. So when the first request has incorrect credentials, user auth fails and mule caches the response. From this point onwards, irrespective of sending correct credentials in subsequent requests, it always refers to the cache and returns user auth failure. How do I achieve what I wanted to achieve ?
Here's my mule flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.6.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd">
<http:listener-config name="HTTP-Inbound-Endpoint" host="0.0.0.0" port="8888" doc:name="HTTP Listener Configuration"/>
<http:request-config name="HTTP_Request_Configuration" host="10.10.10.10" port="8080" doc:name="HTTP Request Configuration"/>
<ee:object-store-caching-strategy name="Auth-Cache-Strategy" doc:name="Caching Strategy">
<in-memory-store name="UserAuthCache" maxEntries="100" entryTTL="3600" expirationInterval="3600"/>
</ee:object-store-caching-strategy>
<flow name="cacheauthenticationFlow">
<http:listener config-ref="HTTP-Inbound-Endpoint" path="*" doc:name="HTTP"/>
<object-to-string-transformer doc:name="Object to String"/>
<ee:cache cachingStrategy-ref="Auth-Cache-Strategy" doc:name="Cache">
<logger message="Incoming: #[message.payload]" level="INFO" doc:name="Logger"/>
<scripting:transformer doc:name="Python">
<scripting:script engine="jython"><![CDATA[import base64
authorization = message.getInboundProperty("authorization")
#print "Authorization is: \"" + authorization + "\""
authstring = authorization.split()
#print authstring
credentials = authstring[-1]
#print "Credentials => " + credentials
decodedAuth = credentials.decode('base64')
#print decodedAuth
if (decodedAuth.find("#") > 0):
(id, password) = decodedAuth.split(":")
(username, project) = id.split("#")
print username + ":" + password + ", Project: " + project
else:
(username, password) = decodedAuth.split(":")
print username + ":" + password
message.payload = { "username" : username + "#" + project , "password" : password }
result = message]]></scripting:script>
</scripting:transformer>
<json:object-to-json-transformer doc:name="Object to JSON"/>
<logger message="Incoming payload: #[message.payload]" level="INFO" doc:name="Logger"/>
<http:request config-ref="HTTP_Request_Configuration" path="/wservices/authenticate/user" method="POST" doc:name="HTTP-Dev-Box-Authentication"/>
<object-to-string-transformer doc:name="Object to String"/>
</ee:cache>
<logger message="Response From Cache: #[message.payload]" level="INFO" doc:name="Logger"/>
<set-payload value="#[message.payload.'status']" doc:name="Response"/>
</flow>
</mule>
Since the default key generation strategy for the cache scope is based on the message payload an option in your particular case would be to simply move the scripting:transformer to be executed before the ee:cache scope.
For a more general solution, where you do not want to overwrite your request payload, you can define the keyGenerationExpression or keyGenerator-refattribute on the ee:cache element to control how the cache key i generated.
For example to use the complete HTTP authorization header as key you could use:
<ee:cache cachingStrategy-ref="Auth-Cache-Strategy" doc:name="Cache"
keyGenerationExpression="#[message.inboundProperties.authorization]">
<!-- Your flow here -->
</ee:cache>
See the cache scope documentation for more information.
If you want to use this approach you could do is move parts of your jython code outside the cache scope, change it so that it sets the user and password as separate flow variables on the message and then use them in the keyGenerationExpression and then again later inside the cache scope in a simple set-payload transformer.
You can define a global configuration "Caching_Strategy":
<ee:object-store-caching-strategy name="Caching_Strategy" keyGenerationExpression="#[flowVars.userID]" doc:name="Caching Strategy"/>
and refer the global configuration in cache flow:
<ee:cache doc:name="Cache" cachingStrategy-ref="Caching_Strategy">
<!-- flow -->
</ee:cache>
You can control your keyGenerationExpression by the flow variable #[flowVars.userID]
For complete detail with example refer
http://www.tutorialsatoz.com/caching-in-mule-cache-scope/
I'm using spring integration to create TCP server and also to test if it works with junit. The problem is that i'm receiving an error: org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply. Please help me to fix it. Here is more info. I have unit test that send some data to server, server has to reply with "success" on each portion of data. but after second portion of read data, TcpOutboundGateway (on unit test side) write error to log.
So Server configuration file:
<int-ip:tcp-connection-factory id="crLfServer"
type="server"
port="5000"
single-use="false"
so-timeout="10000"
/>
<task:executor id="pool" pool-size="16"/>
<int-ip:tcp-inbound-gateway id="gatewayCrLf"
connection-factory="crLfServer"
request-channel="serverBytes2StringChannel"
error-channel="errorChannel"/>
<int:channel id="toSA" >
<int:dispatcher task-executor="pool" />
</int:channel>
<int:service-activator input-channel="toSA"
ref="connectionHandler"
method="handleData" />
<bean id="connectionHandler" class="com.pc.tracker.utils.ConnectionHandler" />
<bean id="objectMapper" class="org.codehaus.jackson.map.ObjectMapper"/>
<int:transformer id="serverBytes2String"
input-channel="serverBytes2StringChannel"
output-channel="toSA"
expression="new String(payload).trim()"/>
<int:transformer id="errorHandler"
input-channel="errorChannel"
expression="payload.failedMessage.payload + ':' + payload.cause.message"/>
Client configuration file:
<int:gateway id="gw"
service-interface="com.pc.tracker.tcp.ConnectionHandlerTestHellperGateway"
default-request-channel="input"/>
<int-ip:tcp-connection-factory id="client"
type="client"
host="localhost"
port="5000"
single-use="false"
so-timeout="10000"/>
<int:channel id="input" />
<int-ip:tcp-outbound-gateway id="outGateway"
request-channel="input"
reply-channel="clientBytes2StringChannel"
connection-factory="client"
request-timeout="10000"
reply-timeout="10000"/>
<int:transformer id="clientBytes2String"
input-channel="clientBytes2StringChannel"
expression="new String(payload)"/>
and test that produce already mentioned error.
#Test
public void testRecivedBackupedData() {
String testData =
"[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+ "[{\"s\":13,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":24,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+ "[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+"[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n";
String result = gw.send(testData);
here is log with error
2014-04-11 23:12:23,059 [pool-1] ERROR com.pc.tracker.utils.ConnectionHandler - recived data: [{"s":1,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"},{"s":12,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"}]
2014-04-11 23:12:23,263 [pool-1] ERROR com.pc.tracker.utils.ConnectionHandler - parsisted
2014-04-11 23:12:23,265 [pool-2] ERROR com.pc.tracker.utils.ConnectionHandler - recived data: [{"s":13,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"},{"s":24,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"}]
2014-04-11 23:12:23,330 [pool-2] ERROR com.pc.tracker.utils.ConnectionHandler - parsisted
2014-04-11 23:12:23,331 [pool-2-thread-1] ERROR org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply
2014-04-11 23:12:23,332 [pool-3] ERROR com.pc.tracker.utils.ConnectionHandler - recived data: [{"s":1,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"},{"s":12,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"}]
2014-04-11 23:12:23,409 [pool-3] ERROR com.pc.tracker.utils.ConnectionHandler - parsisted
2014-04-11 23:12:23,409 [pool-2-thread-1] ERROR org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply
2014-04-11 23:12:23,410 [pool-4] ERROR com.pc.tracker.utils.ConnectionHandler - recived data: [{"s":1,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"},{"s":12,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"}]
2014-04-11 23:12:23,487 [pool-4] ERROR com.pc.tracker.utils.ConnectionHandler - parsisted
2014-04-11 23:12:23,488 [pool-2-thread-1] ERROR org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply
2014-04-11 23:12:23,489 [pool-5] ERROR com.pc.tracker.utils.ConnectionHandler - recived data:
2014-04-11 23:12:23,490 [pool-2-thread-1] ERROR org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply
I've spent two days solving the problem.
sorry for long question.
Thanks.
You are sending data with embedded \r\n...
public void testRecivedBackupedData() {
String testData =
"[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+ "[{\"s\":13,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":24,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+ "[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+"[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n";
String result = gw.send(testData);
Remove them all; the framework will add one at the end.
Your server is sending multiple replies on the socket for one request. Search for connection id localhost:5000:51420:9f93417a-c879-4753-a61e-0b9485940d14. You will see that you sent your request at
2014-04-12 11:27:11,307
The reply is received at
2014-04-12 11:27:11,340
(onMessage() call) and sent to the reply channel at
2014-04-12 11:27:11,343
the next activity on that socket was the reception of another message
2014-04-12 11:27:11,346
for which there was nobody waiting hence the error message.
Now, looking at the connection id, we can see that the remote socket is 51420, so let's take a look on the server side...
The corresponding connection id on the server is localhost:51420:5000:66862ff3-3f40-4291-938f-0694bf3727be. The reply success to the original request was sent at
2014-04-12 11:27:11,339
However, that same thread reads another message (without another send) - notice the message Available to read:525.
So, the bottom line is you are using the default (de)serializer which expects the message to be terminated with \r\n but you are sending requests that have \r\n embedded in them so the server side is "seeing" multiple requests when the sender only sent one message.
TCP is a streaming protocol - you need to frame your data somehow so the server side knows when a message is complete. If you need to send data that contains \r\n you will need to use a different deserializer to detect the end of the data (perhaps the length header implementation, which is the most efficient).
The available standard serializers and information about customizing them are documented here.