I am implementing a consumer class that binds to fanout exchange in RabbitMQ and receives the message published as json. For some reason, the handleMessage within the Consumer class is not being invoked when its argument is a custom object. Same code works when the handleMessage is changed to take Object. Would appreciate your help in identity the missing piece.
Here is the configuration and consumer classes. This is not a SpringBoot application. My Configuration class has #Configuration annotation and not #SpringBootApplication.
#Bean
public SimpleMessageListenerContainer messageListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueueNames(QUEUE_NAME);
container.setMessageListener(listenerAdapter());
container.setMessageConverter(new Jackson2JsonMessageConverter());
container.setMissingQueuesFatal(false);
return container;
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(rabbitConnectionFactory());
}
#Bean
public Queue queue() {
return new Queue(QUEUE_NAME, false, false, false);
}
#Bean
public FanoutExchange exchange() {
return new FanoutExchange(EXCHANGE_NAME, false, false);
}
#Bean
public Binding inboundEmailExchangeBinding() {
return BindingBuilder.bind(queue()).to(exchange());
}
#Bean
public ConnectionFactory rabbitConnectionFactory() {
return new CachingConnectionFactory("localhost");
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(rabbitConnectionFactory());
rabbitTemplate.setExchange(EXCHANGE_NAME);
return rabbitTemplate;
}
#Bean
MessageListenerAdapter listenerAdapter() {
return new MessageListenerAdapter(new Consumer(), "receiveMessage");
}
Here is the consumer ...
public class Consumer {
// This works
/*
public void receiveMessage(Object message) {
System.out.println("Received <" + message + ">");
}
*/
// This does not works, whereas I expect this to work.
public void receiveMessage(CustomObject message) {
System.out.println("Received <" + message + ">");
}
}
where CustomObject class is a plain POJO.
Here is an example of what is being published in RabbitMQ.
{
"state": "stable",
"ip": "1.2.3.4"
}
Its being published as json content-type
exchange.publish(message_json, :content_type => "application/json")
Appreciate all your help in making me understand the problem. Thanks.
The Jackson2JsonMessageConverter needs to be told what object to map the json to.
This can be provided via information in a __TypeId__ header (which would be the case if Spring was on the sending side); the header can either contain the full class name, or a token that is configured to map to the class name.
Or, you need to configure the converter with a class mapper.
For convenience there is a DefaultClassMapper that be configured with your target class:
ClassMapper classMapper = new DefaultClassMapper();
classMapper.setDefaultType(CustomObject.class);
converter.setClassMapper(classMapper);
Related
I am trying to consume a JSON message using spring kafka. The message which is consumed by the consumer is like this.
{
"EventHeader": {
"entityName": "Account",
"changedFields": ["Id", "Name"]
},
"NewFields": {
"Id": "001",
"Name": "Test Account",
},
"OldFields": {}
}
So far I have created classes for "EventHeader", "NewFields","OldFields" ,and for "KafkaPayload". And also I have created a custom deserializer to deserialize this JSON payload.Here is my custom deserializer.
public class CustomDeserailizer <T extends Serializable> implements Deserializer<T> {
private ObjectMapper objectMapper = new ObjectMapper();
public static final String VALUE_CLASS_NAME_CONFIG = "value.class.name";
#Override
public void configure(Map<String, ?> configs, boolean isKey) {
Deserializer.super.configure(configs, isKey);
}
#Override
public T deserialize(String topic, byte[] objectData) {
return (objectData == null) ? null : (T) SerializationUtils.deserialize(objectData);
}
#Override
public T deserialize(String topic, Headers headers, byte[] data) {
return Deserializer.super.deserialize(topic, headers, data);
}
#Override
public void close() {
Deserializer.super.close();
}
}
I have set the consumer configurations as below.
public class KafkaConfig {
#Bean
public KafkaConsumer<String, KafkaPayload> consumerFactory(){
Properties config = new Properties();
config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
config.put(ConsumerConfig.GROUP_ID_CONFIG, "groupId");
config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, CustomDeserializer.class);
return new KafkaConsumer<>(config);
}
}
Now I need to show the consumed message through a #KafkaListener setting the consumer into ConsumerFactory. But I don't understand how to do that. This is my first time using kafka.So could anyone give me some idea about this?
This is how I am trying to do that.
#Bean
public ConcurrentKafkaListenerContainerFactory<String, KafkaPayload> kafkaListener(){
ConcurrentKafkaListenerContainerFactory factory = new ConcurrentKafkaListenerContainerFactory();
factory.setConsumerFactory(consumerFactory());
return factory;
}
This is my KafkaListener
public class ConsumerService {
#KafkaListener(topics = "Topic", groupId = "sample-group",containerFactory = "kafkaListener")
public void consume(KafkaPayload kafkaPayload){
System.out.println("Consumed Message :"+ kafkaPayload);
}
}
Since you are using Spring Boot, just set the value deserializer class name as a property and Boot will automatically wire it into the container factory for your #KafkaListener. No need to define your own consumer factory or container factory.
spring.kafka.consumer.value-deserializer=com.acme.CustomDeserializer
https://docs.spring.io/spring-boot/docs/current/reference/html/application-properties.html#application-properties.integration.spring.kafka.consumer.value-deserializer
I need to bind a queue to a topic exchange, but:
Only if the topic exists
If the topic exists, use the existing settings (e.g. durable, auto-delete, etc)
Reason is, I need a 3rd party application to create the exchange with whatever settings they want to use, I don't want to modify the topic settings.
I put the code below together by reading RabbitMQ Spring AMQP tutorial. It works, but creates an exchange if doesn't exist.
#Configuration
public class BeanConfiguration {
#Bean
public TopicExchange topic() {
return new TopicExchange("MyTopicExchange", true, false);
}
#Bean
public Queue queue() {
return QueueBuilder.durable("MyQueue").build();
}
#Bean
public Binding binding(TopicExchange topicExchange, Queue queue) {
return BindingBuilder.bind(queue).to(topicExchange).with("purchases.*");
}
}
I found a way by using superclass method setShouldDeclareFalse:
#Bean
public TopicExchange topic() {
TopicExchange topicExchange = new TopicExchange("MyTopicExchange", true, false);
topicExchange.setShouldDeclare(false);
return topicExchange;
}
Skip the exchange declaration bean and ignore the binding declaration failure.
#SpringBootApplication
public class So59994152Application {
public static void main(String[] args) {
SpringApplication.run(So59994152Application.class, args);
}
#Bean
public Queue queue() {
return QueueBuilder.durable("MyQueue").build();
}
#Bean
public Binding binding(Queue queue, AmqpAdmin admin) {
((RabbitAdmin) admin).setIgnoreDeclarationExceptions(true);
return new Binding("MyQueue", DestinationType.QUEUE, "MyTopicExchange", "purchases.*", null);
}
#Bean
public ApplicationRunner runner(CachingConnectionFactory cf) {
return args -> {
cf.createConnection();
cf.destroy();
};
}
}
If you are not using Spring Boot; set the admin property in the admin bean.
I try to convert the "Hello World example" from Spring Integration samples (https://github.com/spring-projects/spring-integration-samples/tree/master/basic/helloworld) from XML, to Java Configuration, (so with the #Configuration annotation).
The configuration class looks like this :
#Configuration
#EnableIntegration
public class BasicIntegrationConfig{
#Bean
public DirectChannel inputCHannel() {
return new DirectChannel();
}
#Bean
public QueueChannel outputChannel() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel= "inputChannel", outputChannel= "outputChannel" )
public MessageHandler fileWritingMessageHandler() {
MessageHandler mh = new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
System.out.println("Message payload: " + message.getPayload());
}
};
return mh;
}
}
To test it, I use the main() supplied from sample project :
DirectChannel fileChannel = applicationContext.getBean("inputChannel", DirectChannel.class);
QueueChannel outputChannel = applicationContext.getBean("outputChannel", QueueChannel.class);
System.out.println("********** SENDING MESSAGE");
fileChannel.send(new GenericMessage<>("test"));
System.out.println(outputChannel.receive(0).getPayload());
I see in the console "Message payload: test", but unfortunately, I don't receive the message on the outputchannel (I have a NullPointerException on outputChannel.receive(0).
Do you have an idea why the Service Activator does not send the message to the output channel?
Your MessageHandler returns void.
You need to subclass AbstractReplyProducingMessageHandler instead.
Thank you Gary, it works perfectly after switching to :
#Bean
#ServiceActivator(inputChannel= "inputChannel")
public AbstractReplyProducingMessageHandler fileWritingMessageHandler() {
AbstractReplyProducingMessageHandler mh = new AbstractReplyProducingMessageHandler() {
#Override
protected Object handleRequestMessage(Message<?> message) {
String payload= (String)message.getPayload();
return "Message Payload : ".concat(payload);
}
};
mh.setOutputChannelName("outputChannel");
return mh;
}
As a side note, I had to remove the output channel attribute in #ServiceActivator annotation, and put it in method body instead (Bean Validation Exception if not).
I have an application (Microservice like) which should send and receives messages from other applications (Microservices). The application has several publishers with every publisher publishing to a specific queue as well as several subscriber classes with each subscriber subscribing to only one queue. Unfortunately, my subscriber classes are consuming the same messages I publish. How should I go about it?
Here is my code:
a) Publisher 1 - does not have a listener method since it only publishes to my.queues.queue1
#Configuration
public class RabbitQueue1Publisher{
private static final String QUEUE_NAME = "my.queues.queue1";
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("http://127.0.0.1:1675");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
}
b) Publisher 2 - also does not have a listener method since it only publishes to my.queues.queue2
#Configuration
public class RabbitQueue2Publisher{
private static final String QUEUE_NAME = "my.queues.queue2";
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("http://127.0.0.1:1675");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
}
c) Consumer 1 - consumes from queue3. Has a listener method
#Configuration
public class RabbitQueue3Subscriber{
private static final String QUEUE_NAME = "my.queue.queue3";
#Autowired
private Queue3Listener Queue3Listener;
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("http://127.0.0.1:15672");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean
public SimpleMessageListenerContainer userListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueues(simpleQueue());
listenerContainer.setMessageConverter(jsonMessageConverter());
listenerContainer.setMessageListener(Queue3Listener);
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
}
d) Consumer 2 - consumes from queue4. Has a listener method
#Configuration
public class RabbitQueue4Subscriber{
private static final String QUEUE_NAME = "my.queue.queue4";
#Autowired
private Queue4Listener Queue4Listener;
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("http://127.0.0.1:15672");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue simpleQueue() {
return new Queue(QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(QUEUE_NAME);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean
public SimpleMessageListenerContainer userListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueues(simpleQueue());
listenerContainer.setMessageConverter(jsonMessageConverter());
listenerContainer.setMessageListener(Queue4Listener);
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
}
Though I am publishing and consuming to/from different queues, I end up consuming the same messages I produce. Can someone point out what I am doing wrong or suggest the way to do it?
Here is how it works for me. I have publisher and a consumer of Rabbitmq. Doesn't mater if they are part of the same project or different.
Publisher:
Publisher Configuration
#Configuration
class PublisherConfig{
String queueName = "com.queueName";
String routingKey = "com.routingKey";
String exchange = "com.exchangeName";
#Bean
Queue queue() {
return new Queue(queueName, false);
}
#Bean
TopicExchange exchange() {
return new TopicExchange(exchange);
}
#Bean
Binding binding(Queue queueFoo, TopicExchange exchange) {
return BindingBuilder.bind(queueFoo).to(exchange).with(routingKey);
}
//Required only if you want to pass custom object as part of payload
#Bean
public MappingJackson2MessageConverter jackson2Converter() {
return new MappingJackson2MessageConverter();
}
}
Publish Message
#Autowired private RabbitMessagingTemplate rabbitMessagingTemplate;
#Autowired private MappingJackson2MessageConverter mappingJackson2MessageConverter;
rabbitMessagingTemplate.setMessageConverter(this.mappingJackson2MessageConverter);
rabbitMessagingTemplate.convertAndSend(exchange, routingKey, employObj)
Consumer
Consumer Configuration
#Configuration
public class RabbitMQConfiguration implements RabbitListenerConfigurer {
public MappingJackson2MessageConverter jackson2Converter() {
return new MappingJackson2MessageConverter();
}
#Bean
public DefaultMessageHandlerMethodFactory handlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(jackson2Converter());
return factory;
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(handlerMethodFactory());
}
}
Listen a Message
#RabbitListener(queues = "com.queueName")
public void receiveMessage(Employee employee) {
// More code
}
You can encapsulate Publisher and Listener configurations in two different #configuration files.
Hope this helps you
P.S.
OP asked for explanation. Here it is:
Exchange and Routing Key
Publisher publishes a message to an exchange with a particular routing key. Routing key helps to differentiate the type of message it is.
Suppose:
Send all user logged in messages with routing key of 'user_logged_in'.
Send all email sent messages with 'email_sent'.
Queue:
Once the routing key is attached with the exchange there comes a queue.
Queue is attached a exchange and routing key and all the published messages will sit in this queue.
Now consumer explicitly, connects to such queues and listen messages.
So queue name in publisher config and consumer config has to be the same.
Once your publisher is up you can actually visit RabbitMq dashboard
and see the exchange, routing key and queue to see how it works.
In my project i am having 3 Classes App(Publisher), PojoListener(Receiver) , Config(bind both App and PojoListner). But i need to separate out the PojoListener from my project and deploy somewhere else so that my Listener class will continuous listen to the Rabbit Mq and ack the messages back to queue and then to App Class for any messages published by my App class.
As my config file is common for both Publisher and Receiver. Is there any way to seperate out them. I need to deploy my receiver on different server and Publisher on different. Both will listen to common queues.
**App.java** :
public class App {
public static void main(String[] args) throws UnsupportedEncodingException {
ApplicationContext context = new AnnotationConfigApplicationContext(RabbitMQConfig.class);
RabbitTemplate rabbitTemplate = context.getBean(RabbitTemplate.class);
String response = (String) rabbitTemplate.convertSendAndReceive("message from publisher");
System.out.println("message sent");
System.out.println("response from :" + response);
}
}
PojoListener.java :
public class PojoListener {
public String handleMessage(String msg) {
System.out.println("IN POJO RECEIVER!!!");
return msg.toUpperCase();
}
}
RabbitMQConfig.java :
#Configuration
public class RabbitMQConfig {
#Bean
public ConnectionFactory rabbitConnectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setHost("localhost");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public RabbitTemplate fixedReplyQRabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(rabbitConnectionFactory());
template.setExchange(ex().getName());
template.setRoutingKey("test");
template.setReplyQueue(replyQueue());
template.setReplyTimeout(60000);
return template;
}
#Bean
public SimpleMessageListenerContainer replyListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueues(replyQueue());
container.setMessageListener(fixedReplyQRabbitTemplate());
return container;
}
#Bean
public SimpleMessageListenerContainer serviceListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueues(requestQueue());
container.setMessageListener(new MessageListenerAdapter(new PojoListener()));
return container;
}
#Bean
public DirectExchange ex() {
return new DirectExchange("rabbit-exchange", false, true);
}
#Bean
public Binding binding() {
return BindingBuilder.bind(requestQueue()).to(ex()).with("test");
}
#Bean
public Queue requestQueue() {
return new Queue("request-queue-spring");
}
#Bean
public Queue replyQueue() {
return new Queue("response-queue-spring");
}
#Bean
public RabbitAdmin admin() {
return new RabbitAdmin(rabbitConnectionFactory());
}
}