RabbitMQ separate listeners by type - spring

I have POJO which represents a message to Rabbit MQ. There is an integer which is responsible for the type of the message(whether it's update, remove, add and so on):
public class Message {
private String field1;
private String field2;
private Integer type;
...
<some other fields>
}
I have a consumer which accepts such messages in my spring boot app. So in order to handle each type separately, I have to add some switch/case construction in my code.
Are there any more clear solutions for such case?

You can use Spring Integration with a router...
Rabbit Inbound channel adapter -> router ->
Where the router routes to a different service activator (method) based on the type.
EDIT
Here's an example:
#SpringBootApplication
public class So47272336Application {
public static void main(String[] args) {
SpringApplication.run(So47272336Application.class, args);
}
#Bean
public ApplicationRunner runner(RabbitTemplate rabbitTemplate) {
return args -> {
rabbitTemplate.convertAndSend("my.queue", new Domain(1, "foo"));
rabbitTemplate.convertAndSend("my.queue", new Domain(2, "bar"));
rabbitTemplate.convertAndSend("my.queue", new Domain(3, "baz"));
};
}
#Bean
public Queue queue() {
return new Queue("my.queue");
}
#Bean
public IntegrationFlow flow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Amqp.inboundAdapter(connectionFactory, "my.queue"))
.route("payload.type", r -> r
.subFlowMapping("1", f -> f.handle("bean", "add"))
.subFlowMapping("2", f -> f.handle("bean", "remove"))
.subFlowMapping("3", f -> f.handle("bean", "update")))
.get();
}
#Bean
public MyBean bean() {
return new MyBean();
}
public static class MyBean {
public void add(Domain object) {
System.out.println("Adding " + object);
}
public void remove(Domain object) {
System.out.println("Removing " + object);
}
public void update(Domain object) {
System.out.println("Updating " + object);
}
}
public static class Domain implements Serializable {
private final Integer type;
private final String info;
public Domain(Integer type, String info) {
this.type = type;
this.info = info;
}
public Integer getType() {
return this.type;
}
public String getInfo() {
return this.info;
}
#Override
public String toString() {
return "Domain [type=" + this.type + ", info=" + this.info + "]";
}
}
}

Related

Listener not getting message in REDIS PubS/ub with Spring Boot

I am relatively new to Redis Pub/Sub. I have integrated this recently in my Spring Boot application.
Redis Pub/Sub configuration is as follows:
#Configuration
public class RedisPubSubConfiguration {
#Bean
public RedisMessageListenerContainer messageListenerContainer(RedisConnectionFactory
connectionFactory,
#Qualifier("topicAdapterPair")
List<Pair
<Topic,
MessageListenerAdapter>>
channelAdaperPairList) {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
for (Pair<Topic, MessageListenerAdapter> chanelAdapterPair : channelAdaperPairList) {
container.addMessageListener(chanelAdapterPair.getValue(),
chanelAdapterPair.getKey());
}
container.setConnectionFactory(connectionFactory);
return container;
}
#Bean("msg-listener-adptr-1")
public MessageListenerAdapter messageListnerAdapter1(
#Qualifier("message-listener-1")
MessageListener listener) {
return new MessageListenerAdapter(listener, REDIS_RECEIVER_METHOD_NAME);
}
#Bean("message-listener-1")
public MessageListener messageListener1(ManagerProxy managerProxy) {
return new MessageListener1(managerProxy);
}
#Bean("message-sender-1")
public MessageSender messageSender1(RedisTemplate redisTemplate,
#Value("${chnlTopicName1}")
String channelTopicName) {
return new MessageSender1(redisTemplate, channelTopicName);
}
#Bean
#Qualifier("topicAdapterPair")
public Pair<Topic, MessageListenerAdapter> getTopicListenerAdapterpair1(
#Value("${chnlTopicName1}") String channelTopicName,
#Qualifier("msg-listener-adptr-1")
MessageListenerAdapter messageListenerAdapter) {
return Pair.of(new ChannelTopic(channelTopicName), messageListenerAdapter);
}
#Bean("msg-listener-adptr-2")
public MessageListenerAdapter messageListnerAdapter2(
#Qualifier("message-listener-2")
MessageListener listener) {
return new MessageListenerAdapter(listener, REDIS_RECEIVER_METHOD_NAME);
}
#Bean("message-listener-2")
public MessageListener messageListener2(NotificationServiceImpl notificationService) {
return new MessageListener2(notificationService);
}
#Bean("message-sender-2")
public MessageSender messageSender2(RedisTemplate redisTemplate,
#Value("${chnlTopicName2}")
String channelTopicName) {
return new MessageSender2(redisTemplate, channelTopicName);
}
#Bean
#Qualifier("topicAdapterPair")
public Pair<Topic, MessageListenerAdapter> getTopicListenerAdapterPair2(
#Value("${chnlTopicName2}") String channelTopicName,
#Qualifier("msg-listener-adptr-2")
MessageListenerAdapter messageListenerAdapter) {
return Pair.of(new ChannelTopic(channelTopicName), messageListenerAdapter);
}
}
MessageSender2 is as follows:
public class MessageSender2 implements MessageSender<MyDTO> {
private final RedisTemplate<String, Object> redisTemplate;
private final String chanelName;
public MessageSender2(
RedisTemplate<String, Object> redisTemplate,
String chanelName) {
this.redisTemplate = redisTemplate;
this.chanelName = chanelName;
}
#Override
public void send(MyDTO myDTO) {
redisTemplate.convertAndSend(chanelName, myDTO);
}
}
MessageListener2 is as follows:
public class MessageListener2 implements MessageListener<EventDTO> {
private static final Logger LOGGER = LoggerFactory
.getLogger(MessageListener2.class);
private final NotificationService notificationService;
public MessageListener1(NotificationServiceImpl notificationService) {
this.notificationService = notificationService;
}
#Override
public void receiveMessage(MyDTO message) {
LOGGER.info("Received message : {} ", message); <--HERE MESSAGE IS NOT COMING EVEN AFTER PUBLISHING MESSAGE TO THE ASSOCIATED TOPIC FROM PUBLISHER
Type type = message.getType();
...
}
}
MessageSender1 is as follows:
public class MessageSender1 implements MessageSender<String> {
private final RedisTemplate<String, Object> redisTemplate;
private final String chanelName;
public MessageSender1(
RedisTemplate<String, Object> redisTemplate,
String chanelName) {
this.redisTemplate = redisTemplate;
this.chanelName = chanelName;
}
#Override
public void send(String message) {
redisTemplate.convertAndSend(chanelName, message);
}
}
Associated listener is follows:
public class MessageListener1 implements MessageListener<String> {
private static final Logger LOGGER = LoggerFactory
.getLogger(MessageListener1.class);
private final ManagerProxy managerProxy;
public MessageListener1(ManagerProxy managerProxy) {
this.managerProxy = managerProxy;
}
public void receiveMessage(String message) {
LOGGER.info("Received message : {} ", message);
managerProxy.refresh();
}
}
Here though MessageSender1 and associated message listener are working fine, I don't understand what I did with MessageSender2 and associated listener, because of which I am not able to receive message in the listener.

Spring-boot 2 rabbitmq MessageConverter not working as excepted

I am using the springboot and rabitmq. I have kind of similar scenario here.
I would like to map my message to custom java object
I also want pass delivery tag which is message proprieties
I wann to pass Channel as well beacuse I need to manual ack messages
My code is like this
#RunWith(SpringRunner.class)
public class EPPQ2SubscriberTest {
#Autowired
private RabbitListenerEndpointRegistry registry;
public final String sampleMessage = "{" + "\"header\": {" + "\"RETRY_COUNT\":0," + "\"PUBLISH_EVENT_TYPE\":\"AUTH\""
+ "}," + "\"payLoad\":{" + "\"MTI\": \"120\"," + "\"MTI_REQUEST\": \"120\","
+ "\"PAN\": \"6011000000000000\"" + "}" + "}";
#Test
public void message_converter_test() throws Exception {
SimpleMessageListenerContainer container = (SimpleMessageListenerContainer) this.registry
.getListenerContainer("messageListener");
ChannelAwareMessageListener listener = (ChannelAwareMessageListener) container.getMessageListener();
Message message = MessageBuilder.withBody(sampleMessage.getBytes())
.andProperties(MessagePropertiesBuilder.newInstance().setContentType("application/json").build())
.build();
listener.onMessage(message, mock(Channel.class));
}
#Configuration
#EnableRabbit
public static class config {
#Bean
public ConnectionFactory mockConnectionFactory() {
return mock(ConnectionFactory.class);
}
#Bean
public MessageConverter messageConverter() {
Jackson2JsonMessageConverter messageConverter = new Jackson2JsonMessageConverter();
/*
* DefaultClassMapper classMapper = new DefaultClassMapper();
* classMapper.setDefaultType(com.discover.dftp.scrubber.domain.Message.class);
* messageConverter.setClassMapper(classMapper);
*/
return messageConverter;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(mockConnectionFactory());
factory.setMessageConverter(messageConverter());
factory.setAutoStartup(false);
return factory;
}
#Bean
public EPPQ2Subscriber messageListener() {
return new EPPQ2Subscriber();
}
}
}
#Component
public class EPPQ2Subscriber {
private static final Logger LOGGER = LoggerFactory.getLogger(EPPQ2Subscriber.class);
// #RabbitListener(queues = "#{queue.getName()}") #TODO I wann to use this in
// later point in time.. !
#RabbitListener(id = "messageListener", queues = "TestQueue")
public void receiveMessage(Message message, Channel channel/* ,#Header(AmqpHeaders.DELIVERY_TAG) long tag */) {
LOGGER.info("Method receiveMessage invoked");
message.getMessageProperties().getDeliveryTag();
LOGGER.info("Result:" + message.getClass() + ":" + message.toString());
}
}
public class Message implements Serializable {
private static final long serialVersionUID = 1L;
private Map<String, Object> header;
private Map<String, Object> payLoad;
public Map<String, Object> getHeader() {
return header;
}
public void setHeader(Map<String, Object> header) {
this.header = header;
}
public Map<String, Object> getPayLoad() {
return payLoad;
}
public void setPayLoad(Map<String, Object> payLoad) {
this.payLoad = payLoad;
}
#Override
public String toString() {
return "Header [header=" + this.header + ", payLoad=" + this.payLoad + "]";
}
}
#RabbitListener(id = "messageListener", queues = "TestQueue")
public void receiveMessage(Message message, Channel channel/* ,#Header(AmqpHeaders.DELIVERY_TAG) long tag */) {
LOGGER.info("Method receiveMessage invoked");
message.getMessageProperties().getDeliveryTag();
LOGGER.info("Result:" + message.getClass() + ":" + message.toString());
}
I looks like that method receives the raw (unconverted) Spring AMQP Message (you are importing the wrong Message class in EPPQ2Subscriber).

Pass list of topics from application yml to KafkaListener

I have the following application.yml:
service:
kafka:
groupId: 345
consumer:
topics:
-
name: response
producer:
topics:
-
name: request1
num-partitions: 5
replication-factor: 1
-
name: request2
num-partitions: 3
replication-factor: 1
How can I access the list of topic names using spel for passing to KafkaListener annotation?
#KafkaListener(topics = "#{'${service.kafka.consumer.topics.name}'}", containerFactory = "kafkaListenerContainerFactory")
public void receive(String payload, #Header(KafkaHeaders.RECEIVED_TOPIC)String topic) {
Use configuration properties and collection projection...
#ConfigurationProperties("service.kafka.producer")
#Component
public class ConfigProps {
List<Topic> topics = new ArrayList<>();
public List<Topic> getTopics() {
return this.topics;
}
public void setTopics(List<Topic> topics) {
this.topics = topics;
}
#Override
public String toString() {
return "ConfigProps [topics=" + this.topics + "]";
}
public static class Topic {
private String name;
private int numPartitions;
private short replicationFactor;
public String getName() {
return this.name;
}
public void setName(String name) {
this.name = name;
}
public int getNumPartitions() {
return this.numPartitions;
}
public void setNumPartitions(int numPartitions) {
this.numPartitions = numPartitions;
}
public short getReplicationFactor() {
return this.replicationFactor;
}
public void setReplicationFactor(short replicationFactor) {
this.replicationFactor = replicationFactor;
}
#Override
public String toString() {
return "Topic [name=" + this.name + ", numPartitions=" + this.numPartitions + ", replicationFactor="
+ this.replicationFactor + "]";
}
}
}
and
#SpringBootApplication
public class So52741016Application {
public static void main(String[] args) {
SpringApplication.run(So52741016Application.class, args);
}
#KafkaListener(groupId = "${service.kafka.groupId}", topics = "#{configProps.topics.![name]}")
public void listener(String in) {
}
#Bean
public SmartLifecycle createTopics(KafkaAdmin admin, ConfigProps props) {
return new SmartLifecycle() {
#Override
public int getPhase() {
return Integer.MIN_VALUE;
}
#Override
public void stop() {
}
#Override
public void start() {
try (AdminClient client = AdminClient.create(admin.getConfig())) {
CreateTopicsResult createTopics = client.createTopics(props.topics.stream()
.map(t -> new NewTopic(t.getName(), t.getNumPartitions(), t.getReplicationFactor()))
.collect(Collectors.toList()));
createTopics.all().get();
}
catch (Exception e) {
// e.printStackTrace();
}
}
#Override
public boolean isRunning() {
return false;
}
#Override
public void stop(Runnable callback) {
}
#Override
public boolean isAutoStartup() {
return true;
}
};
}
}
and
2018-10-10 11:20:25.813 INFO 14979 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer : partitions assigned: [request1-4, request2-0, request1-0, request2-1, request1-1, request2-2, request1-2, request1-3]
Of course, this is only the producer topics, but you can handle them all this way.

Maybe not public or not valid? Using Spring's Websocket and Kafka

As I am trying to consume data from a topic (the topic name is based on user) and during runtime I am trying to consume message from the topic but I am getting the following error.
Caused by:
org.springframework.expression.spel.SpelEvaluationException: EL1008E:
Property or field 'consumerProperties' cannot be found on object of
type 'org.springframework.beans.factory.config.BeanExpressionContext'
- maybe not public or not valid?
Here is my code
#Service
public class kafkaConsumerService {
private SimpMessagingTemplate template;
KafkaConsumerProperties consumerProperties;
#Autowired
public kafkaConsumerService(KafkaConsumerProperties consumerProperties, SimpMessagingTemplate template) {
this.consumerProperties=consumerProperties;
this.template=template;
}
#KafkaListener(topics = {"#{consumerProperties.getTopic()}"})
// #KafkaListener(topics="Chandan3706")
public void consume(#Payload Message message) {
System.out.println("from kafka topic::" + message);
template.convertAndSend("/chat/getMessage", message);
}
}
My KafkaConsumerProperties.class
#Component
#ConfigurationProperties(prefix="kafka.consumer")
public class KafkaConsumerProperties {
private String bootStrap;
private String group;
private String topic;
public String getBootStrap() {
return bootStrap;
}
public void setBootStrap(String bootStrap) {
this.bootStrap = bootStrap;
}
public String getGroup() {
return group;
}
public void setGroup(String group) {
this.group = group;
}
public String getTopic() {
return topic;
}
public void setTopic(String topic) {
this.topic = topic;
}
#Override
public String toString() {
return "KafkaConsumerProperties [bootStrap=" + bootStrap + ", group=" + group + ", topic=" + topic + "]";
}
}
Thanks in advance
Since you don’t provide any bean name for your KafkaConsumerProperties component, the default one is de-capitalized class name. That’s one.
The expression you use in the #KafkaListener is regular bean definition phase expression, therefore a root object is some BeanExpressionContext , but not your listener bean as you try to get access through the property.
Not sure if you need that KafkaConsumerProperties property in this listener, but expression must ask for the kafkaConsumerProperties bean:
#Service
public class kafkaConsumerService {
private SimpMessagingTemplate template;
#Autowired
public kafkaConsumerService(SimpMessagingTemplate template) {
this.template=template;
}
#KafkaListener(topics = {"#{kafkaConsumerProperties.topic}"})
// #KafkaListener(topics="Chandan3706")
public void consume(#Payload Message message) {
System.out.println("from kafka topic::" + message);
template.convertAndSend("/chat/getMessage", message);
}
}
The following code worked for me, notice to #DependsOn("KafkaConsumerProperties") and #Component("KafkaConsumerProperties") annotations.
KafkaConsumerService class:
#Service
#DependsOn("KafkaConsumerProperties")
public class KafkaConsumerService {
#KafkaListener(topics = "#{#KafkaConsumerProperties.getTopic()}")
public void consume(#Payload Message message) {
System.out.println("from kafka topic::" + message);
template.convertAndSend("/chat/getMessage", message);
}
}
KafkaConsumerProperties class:
#Component("KafkaConsumerProperties")
#ConfigurationProperties(prefix="kafka.consumer")
public class KafkaConsumerProperties {
private String topic;
public String getTopic() {
return topic;
}
}

Jackson configuration to consume list of records in rabbitmq

I am using spring boot amqp in which I will be consuming a list of Employee objects from a queue. My listener method looks like this:
#RabbitListener(queues = "emp_queue")
public void processAndPortEmployeeData(List<Employee> empList) {
empList.forEach(emp -> { some logic })
}
However, when I try to consume the message, I get a class cast exception: For some reason, I'm getting a LinkedHashMap.
Caused by: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to com.integration.domain.Employee
If I change my listener method to consume a single employee object, it works fine and I'm using the following jackson configurations for it:
#Configuration
#EnableRabbit
public class RabbitConfiguration implements RabbitListenerConfigurer {
#Bean
public MappingJackson2MessageConverter jackson2Converter() {
return new MappingJackson2MessageConverter();
}
#Bean
public DefaultMessageHandlerMethodFactory handlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(jackson2Converter());
return factory;
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(handlerMethodFactory());
}
}
Is there some other jackson configuration that I need to do to consume the list of employee objects?
Thanks a lot!
Sample Input Json message which I will be consuming:
[
{
"name" : "Jasmine",
"age" : "24",
"emp_id" : 1344
},
{
"name" : "Mark",
"age" : "32",
"emp_id" : 1314
}
]
What version of Spring AMQP are you using?
If 1.6 or greater, the framework passes the argument type to the message converter.
Before 1.6 you either need type information in the message headers, or you need to configure the converter with type information.
That said, since the converter created a map, it implies that was what received (not a list).
Please show a sample of the JSON in a message.
EDIT
Note that boot auto-configures the message converter if there's a single bean of that type...
#SpringBootApplication
public class So40491628Application {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = SpringApplication.run(So40491628Application.class, args);
Resource resource = new ClassPathResource("data.json");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
FileCopyUtils.copy(resource.getInputStream(), baos);
context.getBean(RabbitTemplate.class).send("foo", MessageBuilder.withBody(baos.toByteArray())
.andProperties(MessagePropertiesBuilder.newInstance().setContentType("application/json").build()).build());
Thread.sleep(10000);
context.close();
}
#Bean
public Jackson2JsonMessageConverter converter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public Queue foo() {
return new Queue("foo");
}
#RabbitListener(queues = "foo")
public void listen(List<Employee> emps) {
System.out.println(emps);
}
public static class Employee {
private String name;
private String age;
private int empId;
public String getName() {
return this.name;
}
public void setName(String name) {
this.name = name;
}
public String getAge() {
return this.age;
}
public void setAge(String age) {
this.age = age;
}
public int getEmpId() {
return this.empId;
}
public void setEmpId(int empId) {
this.empId = empId;
}
#Override
public String toString() {
return "Employee [name=" + this.name + ", age=" + this.age + ", empId=" + this.empId + "]";
}
}
}
Result:
[Employee [name=Jasmine, age=24, empId=0], Employee [name=Mark, age=32, empId=0]]

Resources