I have an application (Microservice like) which should send and receives messages from other applications (Microservices). The application has several publishers with every publisher publishing to a specific queue as well as several subscriber classes with each subscriber subscribing to only one queue. Unfortunately, my subscriber classes are consuming the same messages I publish. How should I go about it?
Here is my code:
a) Publisher 1 - does not have a listener method since it only publishes to my.queues.queue1
#Configuration
public class RabbitQueue1Publisher{
private static final String QUEUE_NAME = "my.queues.queue1";
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("http://127.0.0.1:1675");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
}
b) Publisher 2 - also does not have a listener method since it only publishes to my.queues.queue2
#Configuration
public class RabbitQueue2Publisher{
private static final String QUEUE_NAME = "my.queues.queue2";
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("http://127.0.0.1:1675");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
}
c) Consumer 1 - consumes from queue3. Has a listener method
#Configuration
public class RabbitQueue3Subscriber{
private static final String QUEUE_NAME = "my.queue.queue3";
#Autowired
private Queue3Listener Queue3Listener;
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("http://127.0.0.1:15672");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean
public SimpleMessageListenerContainer userListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueues(simpleQueue());
listenerContainer.setMessageConverter(jsonMessageConverter());
listenerContainer.setMessageListener(Queue3Listener);
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
}
d) Consumer 2 - consumes from queue4. Has a listener method
#Configuration
public class RabbitQueue4Subscriber{
private static final String QUEUE_NAME = "my.queue.queue4";
#Autowired
private Queue4Listener Queue4Listener;
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("http://127.0.0.1:15672");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean
public SimpleMessageListenerContainer userListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueues(simpleQueue());
listenerContainer.setMessageConverter(jsonMessageConverter());
listenerContainer.setMessageListener(Queue4Listener);
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
}
Though I am publishing and consuming to/from different queues, I end up consuming the same messages I produce. Can someone point out what I am doing wrong or suggest the way to do it?
Here is how it works for me. I have publisher and a consumer of Rabbitmq. Doesn't mater if they are part of the same project or different.
Publisher:
Publisher Configuration
#Configuration
class PublisherConfig{
String queueName = "com.queueName";
String routingKey = "com.routingKey";
String exchange = "com.exchangeName";
#Bean
Queue queue() {
return new Queue(queueName, false);
}
#Bean
TopicExchange exchange() {
return new TopicExchange(exchange);
}
#Bean
Binding binding(Queue queueFoo, TopicExchange exchange) {
return BindingBuilder.bind(queueFoo).to(exchange).with(routingKey);
}
//Required only if you want to pass custom object as part of payload
#Bean
public MappingJackson2MessageConverter jackson2Converter() {
return new MappingJackson2MessageConverter();
}
}
Publish Message
#Autowired private RabbitMessagingTemplate rabbitMessagingTemplate;
#Autowired private MappingJackson2MessageConverter mappingJackson2MessageConverter;
rabbitMessagingTemplate.setMessageConverter(this.mappingJackson2MessageConverter);
rabbitMessagingTemplate.convertAndSend(exchange, routingKey, employObj)
Consumer
Consumer Configuration
#Configuration
public class RabbitMQConfiguration implements RabbitListenerConfigurer {
public MappingJackson2MessageConverter jackson2Converter() {
return new MappingJackson2MessageConverter();
}
#Bean
public DefaultMessageHandlerMethodFactory handlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(jackson2Converter());
return factory;
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(handlerMethodFactory());
}
}
Listen a Message
#RabbitListener(queues = "com.queueName")
public void receiveMessage(Employee employee) {
// More code
}
You can encapsulate Publisher and Listener configurations in two different #configuration files.
Hope this helps you
P.S.
OP asked for explanation. Here it is:
Exchange and Routing Key
Publisher publishes a message to an exchange with a particular routing key. Routing key helps to differentiate the type of message it is.
Suppose:
Send all user logged in messages with routing key of 'user_logged_in'.
Send all email sent messages with 'email_sent'.
Queue:
Once the routing key is attached with the exchange there comes a queue.
Queue is attached a exchange and routing key and all the published messages will sit in this queue.
Now consumer explicitly, connects to such queues and listen messages.
So queue name in publisher config and consumer config has to be the same.
Once your publisher is up you can actually visit RabbitMq dashboard
and see the exchange, routing key and queue to see how it works.
Related
I have this RabbitMQ Spring Boot Configuration:
#Configuration
public class RabbitConfiguration {
// Main queue configuration
#Value("${rabbitmq.main.messages.queue}")
private String mainQueueName;
#Value("${rabbitmq.main.exchange.queue}")
private String mainExchangeName;
#Value("${rabbitmq.main.routing.key}")
private String mainRoutingKey;
// DLQ configuration
#Value("${rabbitmq.dlq.messages.queue}")
private String dlqQueueName;
#Value("${rabbitmq.dlq.exchange.queue}")
private String dlqExchangeName;
#Value("${rabbitmq.dlq.routing.key}")
private String dlqRoutingKey;
// Connectivity
#Value("${spring.rabbitmq.host}")
private String rabbitmqHost;
#Value("${spring.rabbitmq.port}")
private int rabbitmqPort;
#Value("${spring.rabbitmq.username}")
private String rabbitmqUsername;
#Value("${spring.rabbitmq.password}")
private String rabbitmqPassword;
// Not delivered messages, will be used eventually
#Value("${rabbitmq.not.delivered.messages.queue}")
private String notDeliveredMessagesQueue;
// status with delivered messages (callback default)
#Value("${rabbitmq.delivered.messages.queue}")
private String deliveredMessagesQueue;
#Bean
DirectExchange deadLetterExchange() {
return new DirectExchange(dlqExchangeName);
}
#Bean
DirectExchange exchange() {
return new DirectExchange(mainExchangeName);
}
#Bean
Queue dlq() {
return QueueBuilder.durable(dlqQueueName).build();
}
#Bean
Queue queue() {
return QueueBuilder
.durable(mainQueueName)
.withArgument("x-dead-letter-exchange", "deadLetterExchange")
.withArgument("x-dead-letter-routing-key", dlqQueueName).build();
}
#Bean
Binding deadLetterBinding() {
return BindingBuilder.bind(dlq()).to(deadLetterExchange()).with(dlqRoutingKey);
}
#Bean
Binding binding() {
return BindingBuilder.bind(queue()).to(exchange()).with(mainQueueName);
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(rabbitmqHost);
connectionFactory.setUsername(rabbitmqUsername);
connectionFactory.setPassword(rabbitmqPassword);
return connectionFactory;
}
#Bean
public RabbitAdmin rabbitAdmin() {
RabbitAdmin admin = new RabbitAdmin(connectionFactory());
admin.declareQueue(queue());
admin.declareExchange(exchange());
admin.declareBinding(binding());
return admin;
}
public AmqpTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(jsonMessageConverter());
return rabbitTemplate;
}
}
The problem is that when some exception is launched the message is not sent to DLQ queue.
Consumer:
#RabbitListener(queues = { "${rabbitmq.main.messages.queue}" })
public void recievedMessage(#Payload Mensagem item, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) long tag) throws InvalidMessageException {
if (item.getIdCliente().equals("69")) {
logger.info("Something went wrong to: " + item);
throw new InvalidMessageException();
} else {
logger.info("==> Message consumed successfully: " + item);
}
}
This is the configuration I have on my application.properties:
spring.rabbitmq.listener.simple.retry.enabled=true
spring.rabbitmq.listener.simple.retry.initial-interval=3s
spring.rabbitmq.listener.simple.retry.max-attempts=2
spring.rabbitmq.listener.simple.retry.multiplier=2
spring.rabbitmq.listener.simple.retry.max-interval=10s
When I throw an exception on purpose just to see the message moving to DLQ nothing happens. What's wrong here? What am I forgetting here?
Try adding
spring.rabbitmq.listener.simple.default-requeue-rejected=false
to your application.properties. I think the problem is that the failed deliveries are being requeued instead of being sent to the DLQ.
I am using RabbitMQ with Spring Boot to broker messages between two services. I am able to receive the message and format it but when I call a service class in the onMessage method, I get a null pointer exception error. Here is my message listener class which receives the message
public class QueueListener implements MessageListener{
#Autowired
private QueueProcessor queueProcessor;
#Override
public void onMessage(Message message) {
String msg = new String(message.getBody());
String output = msg.replaceAll("\\\\", "");
String jsonified = output.substring(1, output.length()-1);
JSONArray obj = new JSONArray(jsonified);
queueProcessor.processMessage(obj);
}
}
Calling the method processMessage throws null pointer exception
Can someone point to me what I ma doing wrong?
I found out the issue was in the RabbitMqConfig class. Here is the code which was causing the error:
#Configuration
public class RabbitMqConfig {
private static final String QUEUE_NAME = "my.queue.name";
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("<url.to.rabbit>");
connectionFactory.setUsername("<username>");
connectionFactory.setPassword("<password>");
return connectionFactory;;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean
public SimpleMessageListenerContainer userListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueues(simpleQueue());
listenerContainer.setMessageConverter(jsonMessageConverter());
listenerContainer.setMessageListener(new QueueListener());
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
}
The line listenerContainer.setMessageListener(new QueueListener()); was the source of the error. I solved it by Autowiring the class instead of using new. Here is the working code
#Configuration
public class RabbitMqConfig {
private static final String QUEUE_NAME = "my.queue.name";
#Autowired
private QueueListener queueListener;
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("<url.to.rabbit>");
connectionFactory.setUsername("<username>");
connectionFactory.setPassword("<password>");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
/*#Bean
public SimpleMessageListenerContainer userListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueues(simpleQueue());
listenerContainer.setMessageConverter(jsonMessageConverter());
listenerContainer.setMessageListener(queueListener);
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
}
Hope this helps someone else
Make sure the QueueListener is a component class or service class that can be managed by the Spring IoC. Otherwise, the config class cannot make this a bean out of the box, since this is just a normal Java class that need to be in the container #runtime.
So when u write new QueueListener() in yr config class, then the Java class is not in the SpringContext at the time when the config class is instantiated and is therefore null.
Hope this helps clear out some of this issue!
In my project i am having 3 Classes App(Publisher), PojoListener(Receiver) , Config(bind both App and PojoListner). But i need to separate out the PojoListener from my project and deploy somewhere else so that my Listener class will continuous listen to the Rabbit Mq and ack the messages back to queue and then to App Class for any messages published by my App class.
As my config file is common for both Publisher and Receiver. Is there any way to seperate out them. I need to deploy my receiver on different server and Publisher on different. Both will listen to common queues.
**App.java** :
public class App {
public static void main(String[] args) throws UnsupportedEncodingException {
ApplicationContext context = new AnnotationConfigApplicationContext(RabbitMQConfig.class);
RabbitTemplate rabbitTemplate = context.getBean(RabbitTemplate.class);
String response = (String) rabbitTemplate.convertSendAndReceive("message from publisher");
System.out.println("message sent");
System.out.println("response from :" + response);
}
}
PojoListener.java :
public class PojoListener {
public String handleMessage(String msg) {
System.out.println("IN POJO RECEIVER!!!");
return msg.toUpperCase();
}
}
RabbitMQConfig.java :
#Configuration
public class RabbitMQConfig {
#Bean
public ConnectionFactory rabbitConnectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setHost("localhost");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public RabbitTemplate fixedReplyQRabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(rabbitConnectionFactory());
template.setExchange(ex().getName());
template.setRoutingKey("test");
template.setReplyQueue(replyQueue());
template.setReplyTimeout(60000);
return template;
}
#Bean
public SimpleMessageListenerContainer replyListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueues(replyQueue());
container.setMessageListener(fixedReplyQRabbitTemplate());
return container;
}
#Bean
public SimpleMessageListenerContainer serviceListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueues(requestQueue());
container.setMessageListener(new MessageListenerAdapter(new PojoListener()));
return container;
}
#Bean
public DirectExchange ex() {
return new DirectExchange("rabbit-exchange", false, true);
}
#Bean
public Binding binding() {
return BindingBuilder.bind(requestQueue()).to(ex()).with("test");
}
#Bean
public Queue requestQueue() {
return new Queue("request-queue-spring");
}
#Bean
public Queue replyQueue() {
return new Queue("response-queue-spring");
}
#Bean
public RabbitAdmin admin() {
return new RabbitAdmin(rabbitConnectionFactory());
}
}
I am creating a rabbitmq messageListener and would like to be able to access the connectionfactory configuration while in the onMessage method, is that possible? It would be useful for logging and other details. Being able to log the vhost from which the message was delivered would be helpful and it is not available in the message itself. Here is my consumer and config
public class Consumer implements MessageListener {
#Override
public void onMessage(Message message) {
//how can I get the connection factory configuration when a message is sent?
}
here is the config
{
#Configuration
#EnableAutoConfiguration
public class RabbitConfig {
private static final String SIMPLE_MESSAGE_QUEUE = "qDLX2.dlq";
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("server");
connectionFactory.setUsername("admin");
connectionFactory.setPassword("ad,om");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("vhost1");
return connectionFactory;
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
return template;
}
#Autowired
private Consumer consumer;
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueueNames(SIMPLE_MESSAGE_QUEUE);
listenerContainer.setMessageListener(consumer);
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
}
Thanks
Gregg
There's no standard way of doing that.
You could inject the connection factory into the consumer field in your listenerContainer bean definition...
listenerContainer.setMessageListener(this.consumer);
this.consumer.setConnectionFactory(connectionFactory());
You could probably #Autowire it too.
You can then use getRabbitConnectionFactory().getVirtualHost() to access the vhost.
EmailQueueListener
#Component
public class EmailQueueListener{
public String handleMessage(String string) {
System.out.println("Message printing"); // this was printed several times
System.out.println(rabbitTemplate.receiveAndConvert()); //received null here
return string;
}
}
configuration
#Configuration
public class RabbitMQConfiguration {
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory =
new CachingConnectionFactory("localhost");
return connectionFactory;
}
#Bean
public AmqpAdmin amqpAdmin() {
RabbitAdmin admin=new RabbitAdmin(connectionFactory());
admin.declareQueue(queue());
return admin;
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate rabbitTemplate=new RabbitTemplate(connectionFactory());
rabbitTemplate.setRoutingKey("eventsQueue");
rabbitTemplate.setQueue("eventsQueue");
return rabbitTemplate;
}
#Bean
public Queue queue() {
return new Queue("eventsQueue");
}
#Bean
#Autowired
public SimpleMessageListenerContainer messageListenerContainer(EmailQueueListener listener){
SimpleMessageListenerContainer container=new SimpleMessageListenerContainer(connectionFactory());
MessageListenerAdapter adapter=new MessageListenerAdapter(listener, "handleMessage");
container.setMessageListener(adapter);
container.addQueues(queue());
return container;
}
}
Sender
rabbitTemplate.convertAndSend("hello");
I updated the code basing on what you said. But this is not working. i could not see the message which i printed to the console in Listener method. Is their anything wrong in my configuration
Here is another option to register any POJO listener:
#Bean
public SimpleMessageListenerContainer serviceListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueues(requestQueue());
container.setMessageListener(new MessageListenerAdapter(new PojoListener()));
return container;
}
Where PojoListener is:
public class PojoListener {
public String handleMessage(String foo) {
return foo.toUpperCase();
}
}
For the MessageListener implementation you should use org.springframework.amqp.support.converter.MessageConverter to extract Message body and convert it to desired domain object.