How to bind to topic with Spring AMQP only if exchange exists? - spring

I need to bind a queue to a topic exchange, but:
Only if the topic exists
If the topic exists, use the existing settings (e.g. durable, auto-delete, etc)
Reason is, I need a 3rd party application to create the exchange with whatever settings they want to use, I don't want to modify the topic settings.
I put the code below together by reading RabbitMQ Spring AMQP tutorial. It works, but creates an exchange if doesn't exist.
#Configuration
public class BeanConfiguration {
#Bean
public TopicExchange topic() {
return new TopicExchange("MyTopicExchange", true, false);
}
#Bean
public Queue queue() {
return QueueBuilder.durable("MyQueue").build();
}
#Bean
public Binding binding(TopicExchange topicExchange, Queue queue) {
return BindingBuilder.bind(queue).to(topicExchange).with("purchases.*");
}
}

I found a way by using superclass method setShouldDeclareFalse:
#Bean
public TopicExchange topic() {
TopicExchange topicExchange = new TopicExchange("MyTopicExchange", true, false);
topicExchange.setShouldDeclare(false);
return topicExchange;
}

Skip the exchange declaration bean and ignore the binding declaration failure.
#SpringBootApplication
public class So59994152Application {
public static void main(String[] args) {
SpringApplication.run(So59994152Application.class, args);
}
#Bean
public Queue queue() {
return QueueBuilder.durable("MyQueue").build();
}
#Bean
public Binding binding(Queue queue, AmqpAdmin admin) {
((RabbitAdmin) admin).setIgnoreDeclarationExceptions(true);
return new Binding("MyQueue", DestinationType.QUEUE, "MyTopicExchange", "purchases.*", null);
}
#Bean
public ApplicationRunner runner(CachingConnectionFactory cf) {
return args -> {
cf.createConnection();
cf.destroy();
};
}
}
If you are not using Spring Boot; set the admin property in the admin bean.

Related

How to build a nonblocking Consumer when using AsyncRabbitTemplate with Request/Reply Pattern

I'm new to rabbitmq and currently trying to implement a nonblocking producer with a nonblocking consumer. I've build some test producer where I played around with typereference:
#Service
public class Producer {
#Autowired
private AsyncRabbitTemplate asyncRabbitTemplate;
public <T extends RequestEvent<S>, S> RabbitConverterFuture<S> asyncSendEventAndReceive(final T event) {
return asyncRabbitTemplate.convertSendAndReceiveAsType(QueueConfig.EXCHANGE_NAME, event.getRoutingKey(), event, event.getResponseTypeReference());
}
}
And in some other place the test function that gets called in a RestController
#Autowired
Producer producer;
public void test() throws InterruptedException, ExecutionException {
TestEvent requestEvent = new TestEvent("SOMEDATA");
RabbitConverterFuture<TestResponse> reply = producer.asyncSendEventAndReceive(requestEvent);
log.info("Hello! The Reply is: {}", reply.get());
}
This so far was pretty straightforward, where I'm stuck now is how to create a consumer which is non-blocking too. My current listener:
#RabbitListener(queues = QueueConfig.QUEUENAME)
public TestResponse onReceive(TestEvent event) {
Future<TestResponse> replyLater = proccessDataLater(event.getSomeData())
return replyLater.get();
}
As far as I'm aware, when using #RabbitListener this listener runs in its own thread. And I could configure the MessageListener to use more then one thread for the active listeners. Because of that, blocking the listener thread with future.get() is not blocking the application itself. Still there might be the case where all threads are blocking now and new events are stuck in the queue, when they maybe dont need to. What I would like to do is to just receive the event without the need to instantly return the result. Which is probably not possible with #RabbitListener. Something like:
#RabbitListener(queues = QueueConfig.QUEUENAME)
public void onReceive(TestEvent event) {
/*
* Some fictional RabbitMQ API call where i get a ReplyContainer which contains
* the CorrelationID for the event. I can call replyContainer.reply(testResponse) later
* in the code without blocking the listener thread
*/
ReplyContainer replyContainer = AsyncRabbitTemplate.getReplyContainer()
// ProcessDataLater calls reply on the container when done with its action
proccessDataLater(event.getSomeData(), replyContainer);
}
What is the best way to implement such behaviour with rabbitmq in spring?
EDIT Config Class:
#Configuration
#EnableRabbit
public class RabbitMQConfig implements RabbitListenerConfigurer {
public static final String topicExchangeName = "exchange";
#Bean
TopicExchange exchange() {
return new TopicExchange(topicExchangeName);
}
#Bean
public ConnectionFactory rabbitConnectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setHost("localhost");
return connectionFactory;
}
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(rabbitConnectionFactory());
rabbitTemplate.setMessageConverter(producerJackson2MessageConverter());
return rabbitTemplate;
}
#Bean
public AsyncRabbitTemplate asyncRabbitTemplate() {
return new AsyncRabbitTemplate(rabbitTemplate());
}
#Bean
public Jackson2JsonMessageConverter producerJackson2MessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
Queue queue() {
return new Queue("test", false);
}
#Bean
Binding binding() {
return BindingBuilder.bind(queue()).to(exchange()).with("foo.#");
}
#Bean
public SimpleRabbitListenerContainerFactory myRabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(rabbitConnectionFactory());
factory.setMaxConcurrentConsumers(5);
factory.setMessageConverter(producerJackson2MessageConverter());
factory.setAcknowledgeMode(AcknowledgeMode.MANUAL);
return factory;
}
#Override
public void configureRabbitListeners(final RabbitListenerEndpointRegistrar registrar) {
registrar.setContainerFactory(myRabbitListenerContainerFactory());
}
}
I don't have time to test it right now, but something like this should work; presumably you don't want to lose messages so you need to set the ackMode to MANUAL and do the acks yourself (as shown).
UPDATE
#SpringBootApplication
public class So52173111Application {
private final ExecutorService exec = Executors.newCachedThreadPool();
#Autowired
private RabbitTemplate template;
#Bean
public ApplicationRunner runner(AsyncRabbitTemplate asyncTemplate) {
return args -> {
RabbitConverterFuture<Object> future = asyncTemplate.convertSendAndReceive("foo", "test");
future.addCallback(r -> {
System.out.println("Reply: " + r);
}, t -> {
t.printStackTrace();
});
};
}
#Bean
public AsyncRabbitTemplate asyncTemplate(RabbitTemplate template) {
return new AsyncRabbitTemplate(template);
}
#RabbitListener(queues = "foo")
public void listen(String in, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) long tag,
#Header(AmqpHeaders.CORRELATION_ID) String correlationId,
#Header(AmqpHeaders.REPLY_TO) String replyTo) {
ListenableFuture<String> future = handleInput(in);
future.addCallback(result -> {
Address address = new Address(replyTo);
this.template.convertAndSend(address.getExchangeName(), address.getRoutingKey(), result, m -> {
m.getMessageProperties().setCorrelationId(correlationId);
return m;
});
try {
channel.basicAck(tag, false);
}
catch (IOException e) {
e.printStackTrace();
}
}, t -> {
t.printStackTrace();
});
}
private ListenableFuture<String> handleInput(String in) {
SettableListenableFuture<String> future = new SettableListenableFuture<String>();
exec.execute(() -> {
try {
Thread.sleep(2000);
}
catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
future.set(in.toUpperCase());
});
return future;
}
public static void main(String[] args) {
SpringApplication.run(So52173111Application.class, args);
}
}

Spring-Rabbitmq MessageConverter - not invoking custom object handleMessage

I am implementing a consumer class that binds to fanout exchange in RabbitMQ and receives the message published as json. For some reason, the handleMessage within the Consumer class is not being invoked when its argument is a custom object. Same code works when the handleMessage is changed to take Object. Would appreciate your help in identity the missing piece.
Here is the configuration and consumer classes. This is not a SpringBoot application. My Configuration class has #Configuration annotation and not #SpringBootApplication.
#Bean
public SimpleMessageListenerContainer messageListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueueNames(QUEUE_NAME);
container.setMessageListener(listenerAdapter());
container.setMessageConverter(new Jackson2JsonMessageConverter());
container.setMissingQueuesFatal(false);
return container;
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(rabbitConnectionFactory());
}
#Bean
public Queue queue() {
return new Queue(QUEUE_NAME, false, false, false);
}
#Bean
public FanoutExchange exchange() {
return new FanoutExchange(EXCHANGE_NAME, false, false);
}
#Bean
public Binding inboundEmailExchangeBinding() {
return BindingBuilder.bind(queue()).to(exchange());
}
#Bean
public ConnectionFactory rabbitConnectionFactory() {
return new CachingConnectionFactory("localhost");
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(rabbitConnectionFactory());
rabbitTemplate.setExchange(EXCHANGE_NAME);
return rabbitTemplate;
}
#Bean
MessageListenerAdapter listenerAdapter() {
return new MessageListenerAdapter(new Consumer(), "receiveMessage");
}
Here is the consumer ...
public class Consumer {
// This works
/*
public void receiveMessage(Object message) {
System.out.println("Received <" + message + ">");
}
*/
// This does not works, whereas I expect this to work.
public void receiveMessage(CustomObject message) {
System.out.println("Received <" + message + ">");
}
}
where CustomObject class is a plain POJO.
Here is an example of what is being published in RabbitMQ.
{
"state": "stable",
"ip": "1.2.3.4"
}
Its being published as json content-type
exchange.publish(message_json, :content_type => "application/json")
Appreciate all your help in making me understand the problem. Thanks.
The Jackson2JsonMessageConverter needs to be told what object to map the json to.
This can be provided via information in a __TypeId__ header (which would be the case if Spring was on the sending side); the header can either contain the full class name, or a token that is configured to map to the class name.
Or, you need to configure the converter with a class mapper.
For convenience there is a DefaultClassMapper that be configured with your target class:
ClassMapper classMapper = new DefaultClassMapper();
classMapper.setDefaultType(CustomObject.class);
converter.setClassMapper(classMapper);

Spring Boot: how to use FilteringMessageListenerAdapter

I have a Spring Boot application which listens to messages on a Kafka queue. To filter those messages, have the following two classs
#Component
public class Listener implements MessageListener {
private final CountDownLatch latch1 = new CountDownLatch(1);
#Override
#KafkaListener(topics = "${spring.kafka.topic.boot}")
public void onMessage(Object o) {
System.out.println("LISTENER received payload *****");
this.latch1.countDown();
}
}
#Configuration
#EnableKafka
public class KafkaConfig {
#Autowired
private Listener listener;
#Bean
public FilteringMessageListenerAdapter filteringReceiver() {
return new FilteringMessageListenerAdapter(listener, recordFilterStrategy() );
}
public RecordFilterStrategy recordFilterStrategy() {
return new RecordFilterStrategy() {
#Override
public boolean filter(ConsumerRecord consumerRecord) {
System.out.println("IN FILTER");
return false;
}
};
}
}
While messages are being processed by the Listener class, the RecordFilterStrategy implementation is not being invoked. What is the correct way to use FilteringMessageListenerAdapter?
Thanks
The solution was as follows:
No need for the FilteringMessageListenerAdapter class.
Rather, create a ConcurrentKafkaListenerContainerFactory, rather than relying on what Spring Boot provides out of the box. Then, set the RecordFilterStrategy implementation on this class.
#Bean
ConcurrentKafkaListenerContainerFactory<Integer, String>
kafkaListenerContainerFactory() {
ConcurrentKafkaListenerContainerFactory<Integer, String> factory =
new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
factory.setRecordFilterStrategy(recordFilterStrategy());
return factory;
}

spring boot rabbitmq dead letter queue config not work

I config spring boot rabbit' dead letter queue, but ErrorHandler never receive any message. I search all the questiones about dead letter queue, but could not figure out. Can anyone help me ?
RabbitConfig.java to config dead letter queue/exchange:
#Configuration
public class RabbitConfig {
public final static String MAIL_QUEUE = "mail_queue";
public final static String DEAD_LETTER_EXCHANGE = "dead_letter_exchange";
public final static String DEAD_LETTER_QUEUE = "dead_letter_queue";
public static Map<String, Object> args = new HashMap<String, Object>();
static {
args.put("x-dead-letter-exchange", DEAD_LETTER_EXCHANGE);
//args.put("x-dead-letter-routing-key", DEAD_LETTER_QUEUE);
args.put("x-message-ttl", 5000);
}
#Bean
public Queue mailQueue() {
return new Queue(MAIL_QUEUE, true, false, false, args);
}
#Bean
public Queue deadLetterQueue() {
return new Queue(DEAD_LETTER_QUEUE, true);
}
#Bean
public FanoutExchange deadLetterExchange() {
return new FanoutExchange(DEAD_LETTER_EXCHANGE);
}
#Bean
public Binding deadLetterBinding() {
return BindingBuilder.bind(deadLetterQueue()).to(deadLetterExchange());
}
}
ErrorHandler.java to process DEAD LETTER QUEUE:
#Component
#RabbitListener( queues = RabbitConfig.DEAD_LETTER_QUEUE)
public class ErrorHandler {
#RabbitHandler
public void handleError(Object message) {
System.out.println("xxxxxxxxxxxxxxxxxx"+message);
}
}
MailServiceImpl.java to process MAIL_QUEUE:
#Service
#RabbitListener(queues = RabbitConfig.MAIL_QUEUE)
#ConditionalOnProperty("spring.mail.host")
public class MailServiceImpl implements MailService {
#Autowired
private JavaMailSender mailSender;
#RabbitHandler
#Override
public void sendMail(TMessageMail form) {
//......
try {
mailSender.save(form);
}catch(Exception e) {
logger.error("error in sending mail: {}", e.getMessage());
throw new AmqpRejectAndDontRequeueException(e.getMessage());
}
}
}
thx god, I finanlly find the answer!
all the configuration are correct, the problem is all the queues like mail_queue are created before I configure dead letter queue. So when I set x-dead-letter-exchange to the queue after the queue is created, it does not take effect.
中文就是,修改队列参数后,要删除队列重建!!!这么简单的一个tip,花了我几小时。。。。。。
How to delete queue, I follow the answer.
Deleting queues in RabbitMQ

Spring MQTT integration: loss of messages

I'm trying use spring integration to receive a big amount of MQTT messages, process them and then store in a db.
Here is the code:
#Bean
public MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();
factory.setServerURIs("tcp://localhost:1883");
return factory;
}
#Bean
public DefaultPahoMessageConverter messageConverter(){
DefaultPahoMessageConverter converter = new DefaultPahoMessageConverter();
converter.setPayloadAsBytes(true);
return converter;
}
#Bean
public MessageChannel mqttInputChannel() {
return new DirectChannel();
}
#Bean
public MessageProducer mqttInbound() {
MqttPahoMessageDrivenChannelAdapter adapter =
new MqttPahoMessageDrivenChannelAdapter("clientID", mqttClientFactory(), "topic1");
adapter.setCompletionTimeout(5000);
adapter.setConverter(messageConverter());
adapter.setQos(2);
adapter.setOutputChannel(mqttInputChannel());
return adapter;
}
#Bean
#ServiceActivator(inputChannel = "mqttInputChannel")
public MessageHandler handler(){
return new MessageHandler() {
#Override
public void handleMessage(Message<?> arg0) throws MessagingException {
//process messages and storing operations
}
};
}
My problem is that I'm not able to receive all messages and losing some of them, probably because I spend a lot of resources and time inside the handler method. I've tried also to use a QueueChannel instead of the DirectChannel, but when the queue is full the problem remains.
A possible solution could be this, stop the reception till the message is completely handled and then restarts it, but I don't know how. Any advice?
Thanks in advance.

Resources