I would like to use streams to handle an HTTP request/response exchange. I didn't see any Spring Cloud Stream App Starters with HTTP sink functionality. Will I need to build a custom sink to handle the response? If so, do I pass the request through my processing pipeline, then use the request in my sink to form the response? I don't think I've misunderstood the use case of Spring Cloud DataFlow and Spring Cloud Stream. Perhaps there are app starters available for this pattern.
Spring Cloud Stream/Dataflow is for unidirectional (stream) processing; it is not intended for request/reply processing.
You could, however, utilize a Stream from a Spring Integration Application; for example, with the rabbitmq binder...
http-inbound-gateway -> amqp-outbound-gateway
Where the outbound gateway is configured to expect the reply from a specific queue and then your stream could be...
:requestQueue > processor1 | ... | processorn > :replyQueue
Spring Integration doesn't currently have an outbound gateway for Kafka. I opened an issue.
Related
I started developing an spring cloud stream project. I'm successfully received message from Kafka through #Streamlistener annotation. Before sending the message to any output channel, I have to convert the payload by calling an externalservice or by DB call. I don't want to call the external service or DB method from the same streamlistener method. My question is , can we create internal channels (like Spring Integration DSL flow) in spring cloud stream?
Yes you can. In Spring Cloud Stream, the channel binding to the binder would only be enabled based on what binder you use for the channel binding.
I'm developing an application that consumes messages from an exchange and it can publish to one of the multiple exchanges based on the input message transformation result.
I am trying to decide whether to go with sprrimg amqp or spring cloud stream.
What would be apt for this scenario?
Spring Cloud Stream (its Rabbit Binder) is a higher-level abstraction on top of Spring AMQP.
It is more opinionated and performs some configuration automatically.
After make a search about different ways to implement it, im stuck.
What im looking for is to realize this example (https://www.rabbitmq.com/tutorials/tutorial-six-spring-amqp.html) with Spring Integration.
I had found interesting post as this (Spring integration with Rabbit AMQP for "Client Sends Message -> Server Receives & returns msg on return queue --> Client get correlated msg") but didn't help me with what i need.
My case mill be a system where a client call the "convertSendAndReceive" method and a server (basede on Spring Integration) will response.
Thanks
According to your explanation it sounds like Outbound Gateway on the Client side and Inbound Gateway on the Server side pair is what you need.
Spring Integration AMQP support provides those implementations for you with built-in correlation functionality: https://docs.spring.io/spring-integration/docs/5.0.0.RELEASE/reference/html/amqp.html
Using Kafka as a messaging system in a microservice architecture what are the benefits of using spring-kafka vs. spring-cloud-stream + spring-cloud-starter-stream-kafka ?
The spring cloud stream framework supports more messaging systems and has therefore a more modular design. But what about the functionality ? Is there a gap between the functionality of spring-kafka and spring-cloud-stream + spring-cloud-starter-stream-kafka ?
Which API is better designed?
Looking forward to read about your opinions
Spring Cloud Stream with kafka binder rely on Spring-kafka. So the former has all functionalities supported by later, but the former will be more heavyweight. Below are some points help you make the choice:
If you might change kafka into another message middleware in the future, then Spring Cloud stream should be your choice since it hides implementation details of kafka.
If you want to integrate other message middle with kafka, then you should go for Spring Cloud stream, since its selling point is to make such integration easy.
If you want to enjoy the simplicity and not accept performance overhead, then choose spring-kafka
If you plan to migrate to public cloud service such as AWS Kensis, Azure EventHub, then use spring cloud stream which is part of spring cloud family.
Use Spring Cloud Stream when you are creating a system where one channel is used for input does some processing and sends it to one output channel. In other words it is more of an RPC system to replace say RESTful API calls.
If you plan to do an event sourcing system, use Spring-Kafka where you can publish and subscribe to the same stream. This is something that Spring Cloud Stream does not allow you do do easily as it disallows the following
public interface EventStream {
String STREAM = "event_stream";
#Output(EventStream.STREAM)
MessageChannel publisher();
#Input(EventStream.STREAM)
SubscribableChannel stream();
}
A few things that Spring Cloud Stream helps you avoid doing are:
setting up the serializers and deserializers
I am currently trying to write an adapter which will consume messages from ActiveMQ and publish it to Kafka.
I am thinking of using spring integration to integrate these two messaging systems.
My problem is that my application will not maintain registry of the Models using which many applications will publish the records to activeMQ. I want to receive these javax jms message and want to perform some transformation like adding jmscorrelationId into kafka message.
ALso, another requirement is to send acknowledgement to active mq only when kafka send/publish is successfull.
Can ack be send back to activemq using spring integration?
Will spring integration be a good option?
Kindly note my tech architect is not in favor of using Camel/Mule. Also, he does not want to use Kafka Connect as i was planning to use Kafka connect source.
Please suggest.
The Spring Integration Kafka extension project has a sync mode for publishing, which will block the thread until Kafka confirms delivery (or throw an exception on a failure).
The JMS inbound gateway can be used to return a reply to a JMS queue.
You can add transformers (or whatever) in the flow to modify the message.