I have 2 flows implemented in Spring Integration using DSL:
REST -> AMQP -> Consumer -> Service
AMQP -> Consumer -> Service
The first flow is just a HTTP Message Publisher for clients who cannot publish directly to AMQP.
How can I let the Publisher know when the message processing failed in Service?
I was looking at Publisher Confirms and Service Acks pattern with DirectChannel, so that the Publisher can synchronously receive the error message, if I understand this correctly. However
this will block the Publisher until Service returns (or throws Exception).
What are the options in Spring Integration (since it is EIP based) to handle such situation where the Publisher should be informed of message processing failures without being blocked? This is also more of a design question.
REST -> AMQPoutboundGateway -> AmqpInboundGateway -> Service
By using gateways, the outbound gateway will block awaiting a 'success' reply from the service.
Add an error channel to the inbound gateway; if the service fails, the error flow will be invoked, and you can configure that flow to return the failure to the calling gateway.
You can use an async outbound gateway if you don't want to block, but then you'll need some other mechanism to return the result to the caller.
Related
I need to have an Apache kafka producer-consumer rest microservice application wherein the once I trigger the producer rest end point i should immediately get an acknowledgement message and the workflow which triggers this service then waits at the next step (a wait event) which is to be triggered by the consumer application.
How can I implement this ?
Thanks and Regards,
Albin
The tech-stack your are looking at is
1. Dropwizard Framework : For spinning up a REST Service to serve the incoming requests
2. Apache Kafka java clients to Produce and Consume based on Triggers
Step #1. Follow this getting-started dropwizard tutorial https://www.dropwizard.io/0.9.2/docs/getting-started.html to build a Rest Service
Step #2. In the initialization step of the Dropwizard start Kafka Producer and Kafka Consumer Threads
Step #3. The API supports a POST endpoint to receive the trigger events. Based on whether its a Produce or Consume, the Api resource can proceed with the handling the event.
One of our Spring Integration DSL flows is meant to be a simple poller based on a cron expression. Normally you would configure a PollableChannel implementation (e.g. QueueChannel) or by using an InboundAdapter (e.g. FTP, HTTPS, S3 etc.) to process inbound messages which have a polling property.
In our case, we won't have any inbound Messages to process. We simply want to start the SI DSL based IntegrationFlow on a nightly basis preferably using a cron expression. Is there any way that we can start a SI Flow with a poller or "fake" a message based on a cron job?
IntegrationFlows
.from(() -> new GenericMessage<>(""),
e -> e.poller(p -> p.cron("0 0 0 * * ?")))
Since there is no something like NullMessage or Message with null payload, we just send a "fake" message with empty string as payload. You can just ignore that message downstream.
The first Lambda is implementation of the MessageSource<T> exactly what you mentioned about (S)FTP, S3, JDBC etc.
It will be still the same Inbound Channel Adapter, a-la equivalent for the <int:inbound-channel-adapter ref="">
I am consuming SOAP web service using inbound-gateway.We need to put message in kafka topic and return a synchronous acknowledgement to requestor using declarative way of spring integration. Is this possible ?
public Acknowledgement process(#RequestPayload MessagePayload payload) {
// perform validation & logic
// need to send message to kafka topic using declarative way
// sending synchronous ack to request originator
return new Acknowledgement();
}
The kafka outbound channel adapter has a sync property setSync(true) when using java config sync="true" when using XML.
The calling thread (web container) will block until kafka assumes responsibility. If you use a publish-subscribe channel, make the kafka adapter the first consumer, and a service to build the Acknowledgement the second consumer (use the order property in the consumers to ensure proper ordering).
Or you can use a KafkaTemplate directly from your controller.
In Spring integration, I have the to deal with dynamic channels creation but when I debug the application I am seeing a “blocking” problem between different channels.
In order to obtain the dynamic channels, I divided the application in parent/children contexts and I have the following Spring integration infrastructure:
Gateway (parent) --> Transformer(Parent) --> Router (Parent/Child) --> TCP outbound (child)
This configuration works fine when all the TCP connections are OK. For testing purposes, I am stopping the different servers where the clients are connected and I could see how the errorChannel only receives errors (connection refused) but the rest of the adapters are also stopped. I would like to redirect/discard/separate these error and avoid the propagation to the common errorChannel.
I understand the errorChannel is a common channel, shared in the parent context but how can I develop a complete separated scenario for each child context?. Is the common Gateway the problem?
I see the post Error handling in Spring integration flow async but I have a complete separated environment for each child and I would like to take the advantage of these dynamic separation. Is this possible?
You can add another gateway in each child with its own error channel.
--> Router (Parent/Child) -> service-activator (child) -> gateway(child) -> TCP Out.
The gateway service interface needs a void return because no reply is expected.
There are multiple servers that are listening to activemq. The chain is configured to make the http [outbound gateway] call. Suppose one of the server picks up the message and in-between if the http call fails for some reason. The message should be put back to the queue, so that another server can pick up the message and process. Can this be achieved using Spring Integration. I read lot on Transaction, however unable to find workable way.
Yes, simply set acknowledge="transacted" on the <int-jms:message-driven-channel-adapter/> and, as long as you use only direct channels (no <queue/> on the channel or task-executor on the channel's dispatcher) then any failure will cause the message to roll back.