Spring Cloud Stream topic per message for different consumers - spring

The topology I am looking for is
So far I have not seen a way to define the topic per message in Cloud Stream. I understand that the consumers will be bound to specific topic but how does the producer sets the topic per message before sending the message to the exchange?
source.output().send(MessageBuilder.withPayload(myMessage).build());
Does not provide any way to set the topic for the exchange to route to the proper consumer.
Or maybe I don't understand something correctly?
UPDATE
I would expect not to receive the message in the consumer due to the bindingRoutingKey being 2222 and I am sending with routeTo 1111. But I still receive it on the consumer.
Producer Properties:
spring.rabbitmq.host=localhost
spring.rabbitmq.port=5672
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest
spring.cloud.stream.bindings.output.content-type=application/json
spring.cloud.stream.bindings.output.destination=messageExchange
spring.cloud.stream.rabbit.bindings.output.producer.routing-key-expression=headers['routeTo']
#EnableBinding(Source.class)
#SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
Sender:
source.output().send(MessageBuilder.withPayload(mo).setHeader("routeTo", "1111").build());
And the Consumer:
spring.rabbitmq.host=localhost
spring.rabbitmq.port=5672
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest
spring.cloud.stream.bindings.input.destination=messageExchange
spring.cloud.stream.rabbit.bindings.input.consumer.bindingRoutingKey=2222
Application:
#SpringBootApplication
#EnableBinding(Sink.class)
public class Application {
private static final Logger log = LoggerFactory.getLogger(Application.class);
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#StreamListener(Sink.INPUT)
public void ReceiveMo(String moDTO) {
log.info("Message received moDTO: {}", moDTO);
}
}
SECOND UPDATE
With the suggestions in the accepted answer below. I was able to make it work. Needed to remove the exchanges and queues from RabbitMQ using its UI and restart the RabbitMQ docker image.

The the routingKeyExpression rabbitmq producer property.
e.g. ...producer.routing-key-expression=headers['routeTo']
then
source.output().send(MessageBuilder.withPayload(myMessage)
.setHeader("routeTo", "Booking.new")
.build());
Note that the destination is the exchange name. By default, the binder expects a Topic exchange. If you wish to use a Direct exchange instead, you must set the exchangeType property.

Related

Configure Spring Boot #KafkaListener for listening to the latest messages

I am using Spring Boot's #KafkaListener to monitor some server's heartbeat messages as:
#KafkaListener(topics = "heartbeat-topic", groupId = "monitor")
public void listenToHeartbeatMsg(String message) {}
Issue is when the listener subscriber application started, even though the server is down, subscriber application will be receiving those previous heartbeat messages.
How can I fix this issue and listen only for real-time heartbeat messages?
Implement ConsumerSeekAware and, in onPartitionsAssigned call seekToBeginning on the callback.
See https://docs.spring.io/spring-kafka/docs/current/reference/html/#seek
public class MyListener implements ConsumerSeekAware {
...
#Override
public void onPartitionsAssigned(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback) {
callback.seekToEnd(assignments.keySet());
}
}

JMS with spring boot, sender and receiver on same package: what is its use?

I am learning JMS with spring boot and nice to know that spring boot comes with embed Active MQ JMS broker.
I started from spring page on how to achieve this and it works like charm. Now i went little further and create two separate spring boot application one containing jms sender code and another containing receiver code.
I tried starting and application failed as both application are using same port for JMS. I fixed this by including this on one application
#Bean
public BrokerService broker() throws Exception {
final BrokerService broker = new BrokerService();
broker.addConnector("tcp://localhost:61616");
broker.addConnector("vm://localhost");
broker.setPersistent(false);
return broker;
}
But now sender is sending message successfully but receiver is doing nothing. I search on stackoverflow and look at this and this. And they are saying:
If you want to use JMS in production, it would be much wiser to avoid using Spring Boot embedded JMS brokers and host it separately. So 3 node setup would be preferred for PROD.
So my questions are:
1. What is the purpose of putting both jms sender and receiver on same application? Is there any practical example
2. Is it really not possible to use spring boot embedded JMS to communicate two separate application.
You might have sender and receiver in the same application if requests arrive in bursts and you want to save them somewhere before they are processed, in case of a server crash. You typically still wouldn't use an embedded broker for that.
Embedded brokers are usually used for testing only.
You can, however, run an embedded broker that is accessible externally; simply fire up a BrokerService as you have, but the other app needs to connect with the tcp://... address, not the vm://....
EDIT
App1:
#SpringBootApplication
#RestController
public class So52654109Application {
public static void main(String[] args) {
SpringApplication.run(So52654109Application.class, args);
}
#Bean
public BrokerService broker() throws Exception {
final BrokerService broker = new BrokerService();
broker.addConnector("tcp://localhost:61616");
broker.setPersistent(false);
broker.start();
return broker;
}
#Autowired
private JmsTemplate template;
#RequestMapping(path = "/foo/{id}")
public String foo(#PathVariable String id) {
template.convertAndSend("someQueue", id);
return id + ": thank you for your request, we'll send an email to the address on file when complete";
}
}
App2:
application.properties
spring.activemq.broker-url=tcp://localhost:61616
and
#SpringBootApplication
public class So526541091Application {
public static void main(String[] args) {
SpringApplication.run(So526541091Application.class, args);
}
#JmsListener(destination = "someQueue")
public void process(String id) {
System.out.println("Processing request for id");
}
}
Clearly, for a simple app like this you might just run the listener in the first app.
However, since there is no persistence of messages with this configuration, you would likely use an external broker for a production app (or enable persistence).

Spring boot stream bind queue with multiple routing keys

I need to bind single queue with multiple routing keys.
I have configuration in application.properties:
spring.cloud.stream.bindings.some-channel1.destination=exch
spring.cloud.stream.bindings.some-channel1.group=a-queue
spring.cloud.stream.rabbit.bindings.some-channel1.consumer.binding-routing-key=event.domain1
spring.cloud.stream.bindings.some-channel2.destination=exch
spring.cloud.stream.bindings.some-channel2.group=a-queue
spring.cloud.stream.rabbit.bindings.some-channel2.consumer.binding-routing-key=event.domain2
This creates queue and bindings properly in rabbit, but finally after running application I got:
org.springframework.cloud.stream.binder.BinderException: Exception thrown while starting consumer:
After all above configuration i still bad because I need single channel. But queue binded with list of routing keys.
Any Ideas how to configure it?
You can't do it with stream properties, but you can always add extra bindings with normal Spring AMQP declarations...
#SpringBootApplication
#EnableBinding(Sink.class)
public class So50526298Application {
public static void main(String[] args) {
SpringApplication.run(So50526298Application.class, args);
}
#StreamListener(Sink.INPUT)
public void listen(String in) {
System.out.println(in);
}
// extra bindings...
#Bean
public TopicExchange exch() {
return new TopicExchange("exch");
}
#Bean
public Queue queue() {
return new Queue("exch.a-queue");
}
#Bean
public Binding extraBinding1() {
return BindingBuilder.bind(queue()).to(exch()).with("event-domain2");
}
}
There is also a third party "advanced" boot starter that allows you to add declarations in a yaml file. I haven't tried it, but it looks interesting.

Spring Cloud Stream Rabbit Delivery acknowledgment

How can one ensure in Spring Cloud Stream Rabbit guaranteed delivery. My code below : -
class Source {
MessageChannel output;
Repository repo;
#Transactional
void publisher(Command command){
repo.save(command);
output.send(MessageBuilder
.withPayload(new Event()).build());
}
}
class Sink {
#StreamListener(Event.class)
void eventListener(Event event){
// method body
}
}
Any help is appreciated.
You can use the Rabbit consumer property spring.cloud.stream.rabbit.bindings.<channelName>.consumer.acknowledgeMode on how do you want to acknowledge. The acknowledgeMode comes from Spring AMQP and you can refer more documentation on this here

Spring cloud aws (messaging) - not getting message

I am attempting to get spring cloud to work with messaging using auto configure.
My properties file contains:
cloud.aws.credentials.accessKey=xxxxxxxxxx
cloud.aws.credentials.secretKey=xxxxxxxxxx
cloud.aws.region.static=us-west-2
My Configuration class is as follows:
#ComponentScan
#EnableAutoConfiguration
public class Application {
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class, args);
}
}
My Listener class:
#RestController
public class OrderListener {
#MessageMapping("orderQueue")
public void orderListener(Order order){
System.out.println("Order Name " + order.getName());
System.out.println("Order Url" + order.getUrl());
}
}
However, nothing seems to print out. I've verified that my queue is in the right region and there is a message on the queue thats ready to be received.
Can someone please provide some guidance?
I see three possible reasons why it is not working
The OrderListener class is not scanned by the component scanner. In order to be scanned this class must be in the same package as your Application class or in a sub-package.
The spring-cloud-aws-autoconfigure artifact is missing on you classpath and therefore the AmazonSQS client is not automatically configured and the queues are not registered.
The message you have in your queue is missing a message header contentType with value application/json (for more information about this see question Spring Cloud - SQS).

Resources