Using Publish Subscribe channel in Spring Integration - spring

We have the below XML configuration for a rest service and Oracle Stored Proc. The use case is about handling Oracle SP errors.
When the Oracle SP throws error, we need to achieve 2 things:
we need to send a json response back to the client in the format
{"status": false,"message": "Oracle SP error payload"}
Send an email out.
The configuration is working as expected when Oracle SP throws error. But when the email send fails ( gave a wrong host), client is receiving error in the below format. The Oracle SP error message is not sent back.
---- Json message received by client when email send fails:------
{
"timestamp": 1501639940143,
"status": 500,
"error": "Internal Server Error",
"exception": "org.springframework.messaging.MessagingException",
"message": "failure occurred in error-handling flow; nested exception is org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.handler.MessageHandlerChain#0$child#2.handler]; nested exception is org.springframework.mail.MailSendException: Mail server connection failed; nested exception is javax.mail.MessagingException: Unknown SMTP host: xxx.com;..",
"path": "/init/5c82d1f4-e550-4bb1-be9c-8d0ddbc7f4401/NEW"
}
How do I send response in the expected json format - {"status": false,"message": Oracle SP error payload}?
Also how to handle the email send error.i.e may be how to store the email error in a DB table?
XML Configuration:
<int:channel id="spOutClobChannel"/>
<int:publish-subscribe-channel id="gatewayErrorChannel"/>
<int-http:inbound-gateway
request-channel="gatewayInitChannel"
error-channel="gatewayErrorChannel"
supported-methods="GET"
path="/init/{refId}/{refStatus}"
payload-expression="#pathVariables.refStatus">
<int-http:header name="UNIQUE_PROCESS_ID" expression="#pathVariables.refId"/>
</int-http:inbound-gateway>
<int-jdbc:stored-proc-outbound-gateway
id="sp-get-data"
data-source="dataSource"
request-channel="gatewayInitChannel"
reply-channel="spOutClobChannel".... >
...
...
</int-jdbc:stored-proc-outbound-gateway>
<int:transformer input-channel="spOutClobChannel" expression='{"status": true, "message": "RECEIVED"}'/>
<int:transformer input-channel="gatewayErrorChannel" expression='{"status": false,"message": payload.message}'/>
<int:transformer expression="payload.failedMessage.headers['UNIQUE_PROCESS_ID']"
input-channel="gatewayErrorChannel" output-channel="appEmailChannel"/>
<int:chain input-channel="appEmailChannel" >
<int-mail:header-enricher>
<int-mail:from value="${email.from}"/>
<int-mail:to value="${email.to}"/>
<int-mail:subject value="${email.subject}"/>
<int-mail:content-type value="text/html"/>
</int-mail:header-enricher>
<int-mail:outbound-channel-adapter host="${email.host}" />
</int:chain>

To avoid errors thrown from the appEmailChannel flow, plus get a gain do not wait for the successful email shipment, it would be better to add to your gatewayErrorChannel an executor configuration. This way all the subscribers to this publish-subscriber channel will be performed in parallel and in their own threads. Therefore any exception in those threads won't impact the reply to the <int-http:inbound-gateway> which you've done with the
<int:transformer input-channel="gatewayErrorChannel" expression='{"status": false,"message": payload.message}'/>
Also how to handle the email send error.i.e may be how to store the email error in a DB table?
With this goal you can use ExpressionEvaluatingRequestHandlerAdvice with its trapException = true and failureChannel properties.
This advice can be configured with the request-handler-advice-chain sub-element.
See Reference Manual and Sample on the matter.

Related

how to retrieve the same error response received by webclient in spring reactive

I reveive a request from client, and to process that I have to make request to azure resource management app.
Now, if the client passes me incorrect info, and if I make direct call from postman to azure API, it returns me very useful info. refer below (below call contains incorrect query params) :
i get response like below in case of incorrect param passed :
{
"error": {
"code": "ResourceNotFound",
"message": "The Resource 'Microsoft.MachineLearningServices/workspaces/workspace_XYZ/onlineEndpoints/Endpoint_XYZ' under resource group 'resourceGroupXYZ' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
}
}
Now, I make this query using springboot reactive webclient API.
But I am not sure how to pass the same error back as it contains very useful info. The exception handling methods calls like onErrorReturn etc did not help me here to get the original error msg. :
response = getWebClient()
.post()
.uri(apiUrl)
.headers(h -> authToken)
.retrieve()
.bodyToMono(String.class)
// .onErrorReturn(response)
.block();

How to propagate 4XX errors to end-user which occurred during internal communication between upstream micro-services?

Let's say I have three micro-services A, B, C.
Assume, that the client calls
client--> A --> B --> C
Client call to micro-service A , which then calls to B and B then calls to C
If any of B or C returns 5XX, A will show a 502 Error to user.
client --> A --> 5XX
or client --> A --> B--> 5XX
The client receives the 502 Response with the actual error in the details message.
Assuming that there was a bad request error(say in B or C) or 4XX error, what should A show to the user? A 502 Bad Gateway or propagate a 4XX error to the user?
client --> A --> 4XX
or client --> A --> B--> 4XX
What should the client see?
Should it be a 502 Bad Gateway containing a 4xx error with error details in response body?
For example:
502 Bad Gateway
{
"errors" : {
"code" : "InternalError",
"message" : "Internal error occured please try after some time.",
"details" : "Error in validating file 123-23-234 format while unzipping."
}
}
or
should it be 4XX error with error details in response body?
4XX Bad Request
{
"errors" : {
"code" : "InvalidFileFormat",
"message" : "Error in validating file 123-23-234 format while unzipping."
}
}
What about 401 Auth errors or time out errors ?

Unable to map the error getting from the application deployed using lambda function

I am having springboot application deployed using a lambda function. Please find the below sample.
Controller
#RequestMapping(method = RequestMethod.GET, value = "/bugnlow/findByRegionId/{regionId}", produces = "application/json")
public ResponseEntity<List<Bunglow>> findAllBunglowsByRegionId(#PathVariable int regionId, #RequestParam int page, #RequestParam int size) {
Page<Bunglow> bunglows = bunglowsService.findAllBunglowsByRegionId(regionId, page, size);
if (bunglows.getContent().size() == 0){
return ResponseEntity.notFound().build();
}
return ResponseEntity.ok(bunglows.getContent());
}
Service
if the "regionid" is invalid, I am throwing a runtime exception that contains message "region id is invalid".
throw new RuntimeException(Constant.INVALID_REGION_ID);
I am getting the below response when testing it locally by sending the invalid region id.
[1]{
"timestamp": 1519577956092,
"status": 500,
"error": "Internal Server Error",
"exception": "java.lang.RuntimeException",
"message": "Region Id is invalid",
"path": "/***/bugnlow/findByRegionId/343333"
}
I deployed above application AWS using lambda function. When I send the request from the AWS API gateway to the deployed application I am getting the below error with Amazon response headers.
[2] Request: /bugnlow/findByRegionId/342324?page=0&size=5 Status: 500 Latency: 166 ms Response Body
{ "message": "Internal server error" }
In the particular endpoint, integration responses have already configured for Error 500. But didn't use a template configuring the content-type as application/json.
I able to get the localized error by setting it in the controller class with
ResponseEntity<?>
But then the List Bunglow not display as the example response value in Swagger UI.
I need to get exact response[1] from the AWS console. How to do it.
Instead of error 500, how can I send the "Region id is invalid" with the Http status 400 (bad request).
It's a great help, if someone can help me on this.
Thanks
I able to resolve my problem by creating a class with
#ControllerAdvice
and handle the Exception using
#ExceptionHandler
Each point I need to validate and respond the error, I created an custom Exception "BunglowCustomExceptionResponse" and catch the exception in the Advice class "BunglowControllerAdvice". Then send the response with the exception message with bad request response as below.
#ControllerAdvice
public class BunglowControllerAdvice {
#ExceptionHandler
public ResponseEntity handleCustomBunglowException(Exception e){
logger.info("***Exception occurred :" + e.getLocalizedMessage());
return new ResponseEntity<BunglowCustomExceptionResponse>(new
BunglowCustomExceptionResponse(e.getLocalizedMessage()), HttpStatus.BAD_REQUEST);
}
}
Then, I able to get the expected response similar to below with bad request status code 400.
{"responseMessage": "error message"}
I don't know this is the best way but able to resolve my problem. Anyway, thanks a lot for your time who viewed this and tried to help me.

Does Spring Cloud Stream Kafka supports embedded headers?

According to this topic:
Kafka Spring Integration: Headers not coming for kafka consumer -
this is no headers support for Kafka
But documentation says:
spring.cloud.stream.kafka.binder.headers
The list of custom headers that will be transported by the binder.
Default: empty.
I can't get it working with spring-cloud-stream-binder-kafka: 1.2.0.RELEASE
SENDING LOG:
MESSAGE (e23885fd-ffd9-42dc-ebe3-5a78467fee1f) SENT :
GenericMessage [payload=...,
headers={
content-type=application/json,
correlationId=51dd90b1-76e6-4b8d-b667-da25f214f383,
id=e23885fd-ffd9-42dc-ebe3-5a78467fee1f,
contentType=application/json,
timestamp=1497535771673
}]
RECEIVING LOG:
MESSAGE (448175f5-2b21-9a44-26b9-85f093b33f6b) RECEIVED BY HANDLER 1:
GenericMessage [payload=...,
headers={
kafka_offset=36,
id=448175f5-2b21-9a44-26b9-85f093b33f6b,
kafka_receivedPartitionId=0,
contentType=application/json;charset=UTF-8,
kafka_receivedTopic=new_patient, timestamp=1497535771715
}]
MESSAGE (448175f5-2b21-9a44-26b9-85f093b33f6b) RECEIVED BY HANDLER 2 :
GenericMessage [payload=...,
headers={
kafka_offset=36,
id=448175f5-2b21-9a44-26b9-85f093b33f6b,
kafka_receivedPartitionId=0,
contentType=application/json;charset=UTF-8,
kafka_receivedTopic=new_patient, timestamp=1497535771715
}]
I expect to see the same message id and get correlationId on receiving side.
application.properties:
spring.cloud.stream.kafka.binder.headers=correlationId
spring.cloud.stream.bindings.newTest.destination=new_test
spring.cloud.stream.bindings.newTestCreated.destination=new_test
spring.cloud.stream.default.consumer.headerMode=embeddedHeaders
spring.cloud.stream.default.producer.headerMode=embeddedHeaders
SENDING MESSAGE:
#Publisher(channel = "testChannel")
public Object newTest(Object param) {
...
return myObject;
}
Yes, it does: http://docs.spring.io/spring-cloud-stream/docs/Chelsea.SR2/reference/htmlsingle/index.html#_consumer_properties
headerMode
When set to raw, disables header parsing on input. Effective only for messaging middleware that does not support message headers natively and requires header embedding. Useful when inbound data is coming from outside Spring Cloud Stream applications.
Default: embeddedHeaders
But that is already Spring Cloud Stream story, not Spring Kafka per se.

TcpOutboundGateway - Cannot correlate response - no pending reply

I'm using spring integration to create TCP server and also to test if it works with junit. The problem is that i'm receiving an error: org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply. Please help me to fix it. Here is more info. I have unit test that send some data to server, server has to reply with "success" on each portion of data. but after second portion of read data, TcpOutboundGateway (on unit test side) write error to log.
So Server configuration file:
<int-ip:tcp-connection-factory id="crLfServer"
type="server"
port="5000"
single-use="false"
so-timeout="10000"
/>
<task:executor id="pool" pool-size="16"/>
<int-ip:tcp-inbound-gateway id="gatewayCrLf"
connection-factory="crLfServer"
request-channel="serverBytes2StringChannel"
error-channel="errorChannel"/>
<int:channel id="toSA" >
<int:dispatcher task-executor="pool" />
</int:channel>
<int:service-activator input-channel="toSA"
ref="connectionHandler"
method="handleData" />
<bean id="connectionHandler" class="com.pc.tracker.utils.ConnectionHandler" />
<bean id="objectMapper" class="org.codehaus.jackson.map.ObjectMapper"/>
<int:transformer id="serverBytes2String"
input-channel="serverBytes2StringChannel"
output-channel="toSA"
expression="new String(payload).trim()"/>
<int:transformer id="errorHandler"
input-channel="errorChannel"
expression="payload.failedMessage.payload + ':' + payload.cause.message"/>
Client configuration file:
<int:gateway id="gw"
service-interface="com.pc.tracker.tcp.ConnectionHandlerTestHellperGateway"
default-request-channel="input"/>
<int-ip:tcp-connection-factory id="client"
type="client"
host="localhost"
port="5000"
single-use="false"
so-timeout="10000"/>
<int:channel id="input" />
<int-ip:tcp-outbound-gateway id="outGateway"
request-channel="input"
reply-channel="clientBytes2StringChannel"
connection-factory="client"
request-timeout="10000"
reply-timeout="10000"/>
<int:transformer id="clientBytes2String"
input-channel="clientBytes2StringChannel"
expression="new String(payload)"/>
and test that produce already mentioned error.
#Test
public void testRecivedBackupedData() {
String testData =
"[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+ "[{\"s\":13,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":24,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+ "[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+"[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n";
String result = gw.send(testData);
here is log with error
2014-04-11 23:12:23,059 [pool-1] ERROR com.pc.tracker.utils.ConnectionHandler - recived data: [{"s":1,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"},{"s":12,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"}]
2014-04-11 23:12:23,263 [pool-1] ERROR com.pc.tracker.utils.ConnectionHandler - parsisted
2014-04-11 23:12:23,265 [pool-2] ERROR com.pc.tracker.utils.ConnectionHandler - recived data: [{"s":13,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"},{"s":24,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"}]
2014-04-11 23:12:23,330 [pool-2] ERROR com.pc.tracker.utils.ConnectionHandler - parsisted
2014-04-11 23:12:23,331 [pool-2-thread-1] ERROR org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply
2014-04-11 23:12:23,332 [pool-3] ERROR com.pc.tracker.utils.ConnectionHandler - recived data: [{"s":1,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"},{"s":12,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"}]
2014-04-11 23:12:23,409 [pool-3] ERROR com.pc.tracker.utils.ConnectionHandler - parsisted
2014-04-11 23:12:23,409 [pool-2-thread-1] ERROR org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply
2014-04-11 23:12:23,410 [pool-4] ERROR com.pc.tracker.utils.ConnectionHandler - recived data: [{"s":1,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"},{"s":12,"t":0,"b":0,"sp":0,"long":0,"sa":0,"lat":0,"a":0,"i":"device_for_unit_tests"}]
2014-04-11 23:12:23,487 [pool-4] ERROR com.pc.tracker.utils.ConnectionHandler - parsisted
2014-04-11 23:12:23,488 [pool-2-thread-1] ERROR org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply
2014-04-11 23:12:23,489 [pool-5] ERROR com.pc.tracker.utils.ConnectionHandler - recived data:
2014-04-11 23:12:23,490 [pool-2-thread-1] ERROR org.springframework.integration.ip.tcp.TcpOutboundGateway - Cannot correlate response - no pending reply
I've spent two days solving the problem.
sorry for long question.
Thanks.
You are sending data with embedded \r\n...
public void testRecivedBackupedData() {
String testData =
"[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+ "[{\"s\":13,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":24,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+ "[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n"
+"[{\"s\":1,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"},"
+ "{\"s\":12,\"t\":0,\"b\":0,\"sp\":0,\"long\":0,\"sa\":0,\"lat\":0,\"a\":0,\"i\":\"device_for_unit_tests\"}]\r\n";
String result = gw.send(testData);
Remove them all; the framework will add one at the end.
Your server is sending multiple replies on the socket for one request. Search for connection id localhost:5000:51420:9f93417a-c879-4753-a61e-0b9485940d14. You will see that you sent your request at
2014-04-12 11:27:11,307
The reply is received at
2014-04-12 11:27:11,340
(onMessage() call) and sent to the reply channel at
2014-04-12 11:27:11,343
the next activity on that socket was the reception of another message
2014-04-12 11:27:11,346
for which there was nobody waiting hence the error message.
Now, looking at the connection id, we can see that the remote socket is 51420, so let's take a look on the server side...
The corresponding connection id on the server is localhost:51420:5000:66862ff3-3f40-4291-938f-0694bf3727be. The reply success to the original request was sent at
2014-04-12 11:27:11,339
However, that same thread reads another message (without another send) - notice the message Available to read:525.
So, the bottom line is you are using the default (de)serializer which expects the message to be terminated with \r\n but you are sending requests that have \r\n embedded in them so the server side is "seeing" multiple requests when the sender only sent one message.
TCP is a streaming protocol - you need to frame your data somehow so the server side knows when a message is complete. If you need to send data that contains \r\n you will need to use a different deserializer to detect the end of the data (perhaps the length header implementation, which is the most efficient).
The available standard serializers and information about customizing them are documented here.

Resources