Request-response pattern using Spring amqp library - spring

everyone. I have an HTTP API for posting messages in a RabbitMQ broker and I need to implement the request-response pattern in order to receive the responses from the server. So I am something like a bridge between the clients and the server. I push the messages to the broker with specific routing-key and there is a Consumer for that messages, which is publishing back massages as response and my API must consume the response for every request. So the diagram is something like this:
So what I do is the following- For every HTTP session I create a temporary responseQueue(which is bound to the default exchange, with routing key the name of that queue), after that I set the replyTo header of the message to be the name of the response queue(where I will wait for the response) and also set the template replyQueue to that queue. Here is my code:
public void sendMessage(AbstractEvent objectToSend, final String routingKey) {
final Queue responseQueue = rabbitAdmin.declareQueue();
byte[] messageAsBytes = null;
try {
messageAsBytes = new ObjectMapper().writeValueAsBytes(objectToSend);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
MessageProperties properties = new MessageProperties();
properties.setHeader("ContentType", MessageBodyFormat.JSON);
properties.setReplyTo(responseQueue.getName());
requestTemplate.setReplyQueue(responseQueue);
Message message = new Message(messageAsBytes, properties);
Message receivedMessage = (Message)requestTemplate.convertSendAndReceive(routingKey, message);
}
So what is the problem: The message is sent, after that it is consumed by the Consumer and its response is correctly sent to the right queue, but for some reason it is not taken back in the convertSendAndReceived method and after the set timeout my receivedMessage is null. So I tried to do several things- I started to inspect the spring code(by the way it's a real nightmare to do that) and saw that is I don't declare the response queue it creates a temporal for me, and the replyTo header is set to the name of the queue(the same what I do). The result was the same- the receivedMessage is still null. After that I decided to use another template which uses the default exchange, because the responseQueue is bound to that exchange:
requestTemplate.send(routingKey, message);
Message receivedMessage = receivingTemplate.receive(responseQueue.getName());
The result was the same- the responseMessage is still null.
The versions of the amqp and rabbit are respectively 1.2.1 and 1.2.0. So I am sure that I miss something, but I don't know what is it, so if someone can help me I would be extremely grateful.

1> It's strange that RabbitTemplate uses doSendAndReceiveWithFixed if you provide the requestTemplate.setReplyQueue(responseQueue). Looks like it is false in your explanation.
2> To make it worked with fixed ReplyQueue you should configure a reply ListenerContainer:
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory);
container.setQueues(responseQueue);
container.setMessageListener(requestTemplate);
3> But the most important part here is around correlation. The RabbitTemplate.sendAndReceive populates correlationId message property, but the consumer side has to get deal with it, too: it's not enough just to send reply to the responseQueue, the reply message should has the same correlationId property. See here: how to send response from consumer to producer to the particular request using Spring AMQP?
BTW there is no reason to populate the Message manually: You can just simply support Jackson2JsonMessageConverter to the RabbitTemplate and it will convert your objectToSend to the JSON bytes automatically with appropriate headers.

Related

Spring AMQP AsyncRabbitTemplate Doesn't Send Message In Delay Time

I'm trying to send delayed messages on RabbitMQ with Spring AMQP.
I'm defining MessageProperties like this:
MessageProperties delayedMessageProperties = new MessageProperties();
delayedMessageProperties.setDelay(45000);
I'm defining the message which should be send in delay time like this:
org.springframework.amqp.core.Message amqpDelayedMessage = org.springframework.amqp.core.MessageBuilder.withBody(objectMapper.writeValueAsString(reversalMessage).getBytes())
.andProperties(reversalMessageProperties).build();
And then, If I send this message with RabbitTemplate, there is no problem. Message is being sent in defined delay time.
rabbitTemplate.convertSendAndReceiveAsType("delay-exchange",delayQueue, amqpDelayedMessage, new ParameterizedTypeReference<org.springframework.amqp.core.Message>() {
});
But I need to send this message asynchronously because I need not to block any other message in the system and to get more performance and if I use asyncRabbitTemplate, message is being delivered immediately. There is no delay.
asyncRabbitTemplate.convertSendAndReceiveAsType("delay-exchange",delayQueue, amqpDelayedMessage, new ParameterizedTypeReference<org.springframework.amqp.core.Message>() {
});
How can I obtain the delay with asnycRabbitTemplate?
This is probably a bug; please open an issue on GitHub.
The convertSendAndReceive() methods are not intended to send and receive raw Message objects.
In the case of the RabbitTemplate the conversion is skipped if the object is already a Message; there are some cases where this skip is not performed with the async template; please edit the question to show your template configuration.
However, since you are dealing with Message directly, don't use the convert... methods at all, simply use
public RabbitMessageFuture sendAndReceive(String exchange, String routingKey, Message message) {

Listener for NATS JetStream

Can some one help how to configure NATS jet stream subscription in spring boot asynchronously example: looking for an equivalent annotation like #kafkalistener for Nats jetstream
I am able to pull the messages using endpoint but however when tried to pull messages using pushSubscription dispatcherhandler is not invoked. Need to know how to make the listener to be active and consume messages immediately once the messages are published to the subject.
Any insights /examples regarding this will be helpful, thanks in advance.
I don't know what is your JetStream retention policy, neither the way you want to subscribe. But I have sample code for WorkQueuePolicy push subscription, wish this will help you.
public static void subscribe(String streamName, String subjectKey,
String queueName, IMessageHandler iMessageHandler) throws IOException,
InterruptedException, JetStreamApiException {
long s = System.currentTimeMillis();
Connection nc = Nats.connect(options);
long e = System.currentTimeMillis();
logger.info("Nats Connect in " + (e - s) + " ms");
JetStream js = nc.jetStream();
Dispatcher disp = nc.createDispatcher();
MessageHandler handler = (msg) -> {
try {
iMessageHandler.onMessageReceived(msg);
} catch (Exception exc) {
msg.nak();
}
};
ConsumerConfiguration cc = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.maxDeliver(3)
.ackWait(Duration.ofMinutes(2))
.build();
PushSubscribeOptions so = PushSubscribeOptions.builder()
.stream(streamName)
.configuration(cc)
.build();
js.subscribe(subjectKey, disp, handler, false, so);
System.out.println("NatsUtil: " + durableName + "subscribe");
}
IMessageHandler is my custom interface to handle nats.io received messages.
First, configure the NATS connection. Here you will specify all your connection details like server address(es), authentication options, connection-level callbacks etc.
Connection natsConnection = Nats.connect(
new Options.Builder()
.server("nats://localhost:4222")
.connectionListener((connection, eventType) -> {})
.errorListener(new ErrorListener(){})
.build());
Then construct a JetStream instance
JetStream jetStream = natsConnection.jetStream();
Now you can subscribe to subjects. Note that JetStream consumers can be durable or ephemeral, can work according to push or pull logic. Please refer to NATS documentation (https://docs.nats.io/nats-concepts/jetstream/consumers) to make the appropriate choice for your specific use case. The following example constructs a durable push consumer:
//Subscribe to a subject.
String subject = "my-subject";
//queues are analogous to Kafka consumer groups, i.e. consumers belonging
//to the same queue (or, better to say, reading the same queue) will get
//only one instance of each message from the corresponding subject
//and only one of those consumers will be chosen to process the message
String queueName = "my-queue";
//Choosing delivery policy is analogous to setting the current offset
//in a partition for a consumer or consumer group in Kafka.
DeliverPolicy deliverPolicy = DeliverPolicy.New;
PushSubscribeOptions subscribeOptions = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.deliverPolicy(deliverPolicy)
.buildPushSubscribeOptions();
Subscription subscription = jetStream.subscribe(
subject,
queueName,
natsConnection.createDispatcher(),
natsMessage -> {
//This callback will be called for incoming messages
//asynchronously. Every subscription configured this
//way will be backed by its own thread, that will be
//used to call this callback.
},
true, //true if you want received messages to be acknowledged
//automatically, otherwise you will have to call
//natsMessage.ack() manually in the above callback function
subscribeOptions);
As for the declarative API (i.e. some form of #NatsListener annotation analogous to #KafkaListener from Spring for Apache Kafka project), there is none available out of the box in Spring. If you feel like you absolutely need it, you can write one yourself, if you are familiar with Spring BeanPostProcessor-s or other extension mechanism that can help to do that. Alternatively you can refer to 3rd party libs, it looks like a bunch of people (including myself) felt a bit uncomfortable when switching from Kafka to NATS, so they tried to bring the usual way of doing things with them from the Kafka world. Some examples can be found on github:
https://github.com/linux-china/nats-spring-boot-starter,
https://github.com/dstrelec/nats
https://github.com/amalnev/declarative-nats-listeners
There may be others.

How to configure Spring's Rabbit Template to throw exception on timeout

I've been looking on google for a bit by now and I can't find a solution to my problem.
The problem is the default behavior of RabbitTemplate's methods, namely convertSendAndReceive() and convertSendAndReceiveAsType().
When you invoke these methods and they are not processed and replied to (default direct reply-to with queue=amq.rabbitmq.reply-to) the RabbitTemplate simply returns null response instead of indicating that message was not replied to.
That is pretty important when you send almost empty body on queue and expect to receive like user's books or something similar, with null response you can't tell if user has no books or if message wasn't processed in time.
Example invocation
final List<String> messages = rabbitTemplate.convertSendAndReceiveAsType("getMessagesQueue", 0, new ParameterizedTypeReference<>() {});
I found a workaround for this - using AsyncRabbitTemplate as it's RabbitConverterFuture throws exception on method .get(timeout), but that's not my go-to. I don't want to have to use AsyncRabbitTemplate just to get notified on unprocessed message.
Example
final AsyncRabbitTemplate.RabbitConverterFuture<List<String>> messages = asyncRabbitTemplate.convertSendAndReceiveAsType("getMessagesQueue", 0, new ParameterizedTypeReference<>() {});
try {
messages.get(5000, TimeUnit.MILLISECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
// message not processed
}
My problem is how to configure RabbitTemplate (configure template itself, not wrap template calls with aspects, decorator, proxy or similar) to actually throw some exception instead of returning null values.
There is currently no such feature; feel free to open a new feature request on GitHub. https://github.com/spring-projects/spring-amqp/issues

How to list all the jms headers attributes in apache camel?

I'm trying to read the jms header in apache-camel route. The following is the route in which I'm reading body & header.
String endPointTopic = "activemq:topic:geoLoc";
String endPointTopicOut = endPointTopic + "_outbox";
from(endPointTopic)
.log("Message from Topic is ${body} & header is ${header.Action}")
.to(endPointTopicOut);
Basically the And from the logs I'm able to see the following, which means I'm able to read the body but not the id in header.
Message from Topic is GeoLocationInfoDTO{id=2, geoLocationUUId='null',
geoLocationName='null', geoLocationDesc='null',
geoLocationPolygon='null', geoLocationCenterLatitude='null',
geoLocationCenterLongitude='null'} & header is
And the following is the code in which I'm publishing the message to activeMQ through jms template.
private MessageHeaders getMessageHeaders(HttpMethod action) {
log.debug("DomainPublisher : getMessageHeaders");
Map <String, Object> headerMap = new HashMap<>();
headerMap.put("Action", action);
return new MessageHeaders(headerMap);
}
public void publish(BaseDTO dto, HttpMethod action) {
log.debug("DomainPublisher : type is : {} : ", dto.getClass().getName());
getJmsMessagingTemplate().convertAndSend(topicMap.get(dto.getClass().getName()), dto, getMessageHeaders(action));
}
Note: I also tried to log the header id like ${header.id} instead of ${header.Action} but nothing is getting printed.
And I also wanted to know all the headers that are available to the jms message.
You can log exchange with all headers and properties as shown in this example:
.to("log:like-to-see-all?level=INFO&showAll=true&multiline=true")
http://camel.apache.org/log.html
More information about JMS headers can be found here: http://camel.apache.org/jms.html
List of possible headers:
JMSCorrelationID - The JMS correlation ID.
JMSDeliveryMode - The JMS delivery mode.
JMSDestination - The JMS destination.
JMSExpiration - The JMS expiration.
JMSMessageID - The JMS unique message ID.
JMSPriority - The JMS priority (with 0 as the lowest priority and 9 as the highest).
JMSRedelivered - Is the JMS message redelivered.
JMSReplyTo - The JMS reply-to destination.
JMSTimestamp - The JMS timestamp.
JMSType - The JMS type.
JMSXGroupID - The JMS group ID.
As per the Claus Ibsen comment looks like JMS headers only allow certain types as headers and camel will drop invalid headers. And it looks like HttpMethod (type Enum) is been dropped by Camel. All I have to do in my code is convert the Enum as String while constructing the header.
headerMap.put("Action", action);
to
headerMap.put("Action", action.toString());
The JMS headers can be viewed from the karaf client console by running the following command:
activemq:browse --amqurl tcp://localhost:61616 --msgsel JMSMessaageID='1' -Vheader TEST.FOO
Note: The above are all example values, change according to your config.

Receive method in JMS waiting for messages

I want a method to browse all messages from a messsage queue and can send it to another queue using jmstemplate with using Websphere queues(NOT MQ). I have tried using receive and it is able to retrieve all the messages from the queue but it is still waiting for another message. And the messages are being lost. It must be in a transaction
The Code I have Tried:
**String message = (String) jmsTemplate.receiveAndConvert();
System.out.print(message);
while ((message = (String) jmsTemplate.receiveAndConvert()) != null) {
messages.add(message);
}
return messages;
}**
The JMStemplate should be used for only synchronous read or sending message. For asychronous read use one of the listener implementation. Read here

Resources