I am new to rabbitmq and am trying the following scenario
--> producer sends message
--> consumer receives the message
-- Execute my own logic
if the logic fails - requeue
--> requeue the message if the consumer fails(machine goes down)
I have implemented the basic sender using Spring rabbitTemplate
rabbitTemplate.convertAndSend(.....);
and for consumer i implemented a message listener
public class CustomMessageListener implements MessageListener {
#Override
public void onMessage(Message message) {
//** my own logic**
}
}
and added it to the container through spring
<bean id="aListener" class="com.sample.CustomMessageListener" autowire="byName"/>
<rabbit:listener-container id="myListenerContainer" connection-factory="connectionFactory" acknowledge="auto" prefetch="750" concurrency="5" >
<rabbit:listener ref="aListener" queues="reportQueue"/>
</rabbit:listener-container>
Its working fine till this part.
now if ** my own logic** mentioned in the listener fails. i want to requeue the message. how can i implement this. From the blogs that i have gone through it looks like returnedMessage needs to overridden. But am not sure how it can be done through listener.
With acknowledge="auto", the message won't be ack'd until the listener exits normally, so there's nothing extra that you need to do; if your listener throws an exception or the server crashes, the message will remain in the queue.
Related
I have a simple Kafka producer in my spring cloud stream application. As my spring application starts, I have a #PostConstruct method which performs some reconciliation and tries sending events to the Kafka producer.
Issue is, my Kafka Producer is not yet ready when the reconciliation starts sending the enets into it, leading to the below:
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'orderbook-service-1.orderbook'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers, failedMessage=GenericMessage ..
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:445)
Is there is a way to get a notification during my application's startup that Kafka channel is initialized, so that I only kick off the rec job post it.
Here is my code snippets:
public interface OrderEventChannel {
String TOPIC_BINDING = "orderbook";
#Output(TOPIC_BINDING)
SubscribableChannel outboundEvent();
}
#Configuration
#EnableBinding({OrderEventChannel.class})
#ConditionalOnExpression("${aix.core.stream.outgoing.kafka.enabled:false}")
public class OutgoingKafkaConfiguration {
}
#Service
public class OutgoingOrderKafkaProducer {
#Autowired
private OrderEventChannel orderEventChannel;
public void onOrderEvent( ClientEvent clientEvent ) {
try {
Message<KafkaEvent> kafkaMsg = mapToKafkaMessage( clientEvent );
SubscribableChannel subscribableChannel = orderEventChannel.outboundEvent();
subscribableChannel.send( kafkaMsg );
} catch ( RuntimeException rte ) {
log.error( "Error while publishing Kafka event [{}]", clientEvent, rte );
}
}
..
..
}
#PostConstruct is MUCH too early in the context lifecycle to start using beans; they are still being created, configured and wired together.
You can use an ApplicationListener (or #EventListener) to listen for an ApplicationReadyEvent (be sure to compare the even's applicationContext to the main application context because you may get other events).
You can also implement SmartLifecycle and put your code in start(); put your bean in a late Phase so it is started after everything is wired up.
Output bindings are started in phase Integer.MIN_VALUE + 1000, input bindings are started in phase Integer.MAX_VALUE - 1000.
So if you want to do something before messages start flowing, use a phase in-between these (e.g. 0, which is the default).
I have a listener
#JmsListener(destination = "QUEUE_NAME")
public void checkStatus(Message message, JmsMessageHeaderAccessor jmsMessageHeaderAccessor) throws Exception {
// TO DO
}
I need to receive a message from the queue once a minute. How can I do this?
Add a sleep() or use JmsTemplate.receive() instead of a listener.
Scenario 1:
I have Message producer and consumer and the flow of the application is as follows:
producer -> Queue -> Consumer
Scenario 2:
Now we have introduced Camel to re-sequence the messages.So The flow of application is as follows :
producer -> Queue1 -> Camel(Resequence) ->Queue2 -> consumer
Question:
Can we have do the Scenario 2 without using the Queue2 in camel. I want the messages to be consumed directly by consumer after the camel re-sequence step so the application flow will be as follows:
producer -> Queue1 -> Camel(Resequence) -> consumer
To send message:
jmsTemplate.convertAndSend("mailbox", new Email("info#example.com", "Hello"));
Camel re-sequence
from("jms:queue1").resequence(header("myprop")).batch().to("queue2");
PS: I have used Message groups so that the messages to would be consumed by specific consumers, the solution should maintain this as well
In that case you wouldn't implement the JMS consumer yourself but delegate message consumption to Camel's JMS component - you already did that with from("jms:queue1").
The logic that you would invoke in your consumer in "scenario 2" would then be moved to a Camel processor:
from("jms:queue1")
.resequence(header("myprop")).batch()
.process(new MessageProcessor());
The Camel processor working with the received message:
public class MessageProcessor implements Processor {
#Override
public void process(Exchange exchange) throws Exception {
Message in = exchange.getIn();
Object body = in.getBody();
// body contains the content of the received JMS message
...
}
}
or shorter using Java 8 lambda syntax:
from("jms:queue1")
.resequence(header("myprop")).batch()
.process().message(message -> {
Object body = message.getBody();
// body contains the content of the received JMS message
...
});
This consumer-side logic should be transparent regarding message groups. The broker automatically choses the right consumer owning a certain message group when dispatching the message, you don't need to worry in your consumer-side code.
This is my client side code :
public class ABCServlet extends HttpServlet {
protected void doGet(HttpServletRequest request,
HttpServletResponse response){
//do blah blah
String msg = null;
java.io.OutputStream os = response.getOutputStream();
java.io.ObjectOutputStream oos = new java.io.ObjectOutputStream(os);
oos.writeObject(msg);
msg = null;
oos.flush();
oos.close();
}
I don't know how using the above code my listener gets kicked off -
public class ABCListener implements MessageListener {
#Override
public void onMessage(Message arg0) {
AbstractJDBCFacade façade = null;
try{
façade = something;
throw new UserException();
}catch(UserException ex){
log.error("ABC Exception " + ex);
}
Configuration :
<bean id="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">....
<bean id="jmsQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
<bean id="listenerContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer102">
I have 3 questions :
1. without putting it on the queue explicitly , how a listener gets invoked?
2. When onMessage method throws UserException, instead of logging I want to pass the message to the client. How can I do that ?
3. Why would someone use JndiObjectFactoryBean instead of ActiveMQ...
JMS by design was supposed to be asynchronous and one-way. Even "synchronous" jms with using receive method of consumer will internally turn into creating a new temporary queue. And here we come to the second point about it's one-way nature. JMS queue was supposed to be one-way and that's why it is called point-to-point (http://www.enterpriseintegrationpatterns.com/patterns/messaging/PointToPointChannel.html). Of course technically with some dancing you will manage to achieve what you want but it is bad practice which will also lead to performance degradation due to the fact that you will need filtering.
To get this thing work fast the best way will be to have exactly one logical receiver (of course you can use concurrent cosumers for one receiver but that should be one logical consumer without any need of filtering the message).
without putting it on the queue explicitly , how a listener gets invoked?
Listener get invoked only when a message comes to a queue. Thats the only way to get it work as it was supposed to work.
In general there are two types of message consuming models: push (also known as event-driven consuming) and poll. In case of using push model all listeners (according to canonical observer pattern) got registered somewhere in the broker and then, when broker receive new message in some queue, it executes listener's method. On the others side in polling model consumer take care itself about receiving messages. So with some interval it comes to a broker and checks the queue for new messages.
Push model: http://www.enterpriseintegrationpatterns.com/patterns/messaging/EventDrivenConsumer.html
Poll model: http://www.enterpriseintegrationpatterns.com/patterns/messaging/PollingConsumer.html
When onMessage method throws UserException, instead of logging I want to pass the message to the client. How can I do that ?
Thats a very bad practice. Of course technically you can achieve it with dirty tricks but thats not the right way of using jms. When onMessage throws the exception then message wont be taken from the queue (of course if u did not reconfigured acknowledge mods or used another tricks). So the best way of solving your probem fmpv is to use redelivery limit on message and a dead letter queue(http://www.enterpriseintegrationpatterns.com/patterns/messaging/DeadLetterChannel.html). If system was not able to process the message after some attempts (redelivery limit shows exactly this) then broker remove message from the queue and send it to a so-called dead letter queue where all failed (from the point of broker) messages are stored. And then client can read that queue and decide what to do with message.
In amq: http://activemq.apache.org/message-redelivery-and-dlq-handling.html
If you want to use so-called "synchronous" features in JMS and really there is no way of using dead letter queue or smth like that then actually you can use consumer.recieve method on the client. But in this case you should send response on every message. In case of success you can send one message and in case of failure error messages. And so a client will be able to understand what is going on. But i dont think that you need such a huge overhead cause actually you need only failure messages. Also in this case you will have to take care about appropriate receive timeouts.
Why would someone use JndiObjectFactoryBean instead of ActiveMQ...
That's cause you are using Spring and there are additional features especially for spring.
PS:
1. For consuming:
How can I send a message using just this piece of code? Don't I need
to put this on a queue? java.io.OutputStream os =
response.getOutputStream(); java.io.ObjectOutputStream oos = new
java.io.ObjectOutputStream(os); oos.writeObject(msg);
For receiving smth like this:
`
<bean id="connectionFactory" class="org.springframework.
jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="baseJNDITemplate"/>
<property name="jndiName"
value="weblogic.jms.ConnectionFactory"/>
</bean>
<bean id="queue" class="org.springframework.
jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="baseJNDITemplate"/>
<property name="jndiName" value="#{properties.queueName}"/>
</bean>
<bean id="messageListenerContainer"
class="org.springframework.jms.listener.
DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destination" ref="queue"/>
<property name="messageListener" ref="messageListener"/>
<property name="sessionTransacted" value="true"/>
</bean>
<bean id="messageListener" class="com.example.ABCListener"/>
And then simply all logic for message processing will be in the listener.
For sending smth like this in config:
<bean id="jmsQueueTemplate"
class="org.springframework.
jms.core.JmsTemplate">
<property name="connectionFactory">
<ref bean="jmsConnectionFactory"/>
</property>
<property name="destinationResolver">
<ref bean="jmsDestResolver"/>
</property>
...
</bean>
<bean id="jmsDestResolver"
class=" org.springframework.jms.support.destination.
JndiDestinationResolver"/>
<bean id="jmsConnectionFactory"
class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jms/myCF"/>
<property name="lookupOnStartup" value="false"/>
<property name="cache" value="true"/>
<property name="proxyInterface" value="amq con fact here"/>
</bean>
and in code simply use jmsTemplate.send(queue, messageCreator) method:
#Autowired
ConnectionFactory connectionFactory;
#Test(enabled = false)
public void testJmsSend(final String msg) throws Exception {
JmsTemplate template = new JmsTemplate(connectionFactory);
template.send("test_queue", new MessageCreator() {
#Override
public Message createMessage(Session session)
throws JMSException {
return session.createTextMessage(msg);
}
});
}
https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/cspr_data_access_jms.html
I believe Dead channel comes in picture only when the message is not properly received by the receiver. In my case, the receiver
received it and processed it, however while processing it failed with
some exception. I want to let the sender know that there was a
exception and the message did not process successfully. I can do this
using a response queue but I don't want to do that, can the receiver
receive a message from the sender on the same queue ? How?
Dead letter channel is a kind of error handling for message processing also. If message processing had failed then after the limit end it got transferred there. It is not actually only for transport issues but also for processing issues. If the message processing got failed with exception then message will stay in the queue and wont be acked by default. So what we should do with this message? For example if it failed due to our database error or smth like this? We should initiate error-handling process, notify assurance systems and stakeholders, collect all necessary info and preserve the message. Due to this kind of queues, which was create d exactly for that, it is much easier. And then customer support team will investigate error queue for further analysis of what has happened. Also we have monitoring tools for notifications and statistics collection on such errors. After understanding what has happened message got removed from the queue and archived.
After processing a message, the consumer is responsible for deleting
the message. If the consumer doesn't delete the message, for example
because because it crashed while processing the message, the message
becomes visible again after the message's Visibility Timeout expires.
Each time this happens, the message's receive count is increased.
When this count reaches a configured limit, the message is placed in a
designated Dead Letter Queue.
http://www.enterpriseintegrationpatterns.com/patterns/messaging/DeadLetterChannel.html
I can do this using a response queue but I don't want to do that, can
the receiver receive a message from the sender on the same queue ?
How?
For you it will look like it's the same queue but internally new temporary queue will be created. To achieve that you should use jms request\reply message pattern. More here: http://activemq.apache.org/how-should-i-implement-request-response-with-jms.html
The only part still confuses me is : If I expect my JMS listener
(receiver) to listen to the queue, then my sender should also
implement JMS and connect to the same queue and send a message. But in
the ABCListener application that I am supporting does not have any
configuration where the sender is configured to the queue. All the
sender does is 3 lines of code : java.io.OutputStream os =
response.getOutputStream(); java.io.ObjectOutputStream oos = new
java.io.ObjectOutputStream(os); oos.writeObject(msg); Literally, that
is it. I don't know how it still works!
Of course 3 lines of code with outputstream do nothing except populating msg string. To send any jms message to the queue you anyway will have to use JMS Api or some library like Spring which wrap it by adding additional features.
I've wrote simple samples to get it more clear.
Modified servlet for asynchronous processing with dead letter queue (for dlq you should create also another listener ofc)
public class AsynchronousJmsSenderServlet extends HttpServlet {
#Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String msg = null;
try (java.io.OutputStream os = response.getOutputStream()) {
try(java.io.ObjectOutputStream oos = new java.io.ObjectOutputStream(os)) {
oos.writeObject(msg);
}
}
sendJmsMessage(msg);
}
private void sendJmsMessage(final String msg) {
ConnectionFactory connectionFactory = null; //here get it in some way from spring
JmsTemplate template = new JmsTemplate(connectionFactory);
template.send("your_queue_name", new MessageCreator() {
#Override
public Message createMessage(Session session)
throws JMSException {
return session.createTextMessage(msg);
}
});
}
}
And here is the code for "synchronous" processing and status reply messages
public class SynchronousJmsSenderServlet extends HttpServlet {
#Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String msg = null;
try (java.io.OutputStream os = response.getOutputStream()) {
try(java.io.ObjectOutputStream oos = new java.io.ObjectOutputStream(os)) {
oos.writeObject(msg);
}
}
sendJmsMessage(msg);
}
private void sendJmsMessage(final String msg) {
ConnectionFactory connectionFactory = null; //here get it in some way from spring
JmsTemplate template = new JmsTemplate(connectionFactory);
Message reply = template.sendAndReceive("your_queue_name", new MessageCreator() {
#Override
public Message createMessage(Session session)
throws JMSException {
return session.createTextMessage(msg);
}
});
if(reply instanceof TextMessage) {
try {
String status = ((TextMessage) reply).getText();
//do error handling if status is error
} catch (JMSException ex) {
throw new RuntimeException("Unable to get status message", ex);
}
} else {
throw new RuntimeException("Only text messages are supported");
}
}
}
public class SynchronousJmsMessageListener implements SessionAwareMessageListener {
#Override
public void onMessage(Message request, Session session) throws JMSException {
try {
//do some processing
sendReply(request, session, "OK");
} catch (Exception ex) {
sendReply(request, session, "Error: " + ex.toString());
}
}
private void sendReply(Message request, Session session, String status) {
try {
TextMessage reply = null; //for example you can use ActiveMQTextMessage here
reply.setJMSCorrelationID(request.getJMSCorrelationID());
reply.setText(status);
MessageProducer producer = session.createProducer(reply.getJMSReplyTo());
producer.send(reply);
} catch (JMSException exception) {
throw new RuntimeException("Unable to send reply", exception);
}
}
}
You will need Spring 5 to have sendAndReceive method on jmsTemplate. Or you will have to do all that manually.
PS1: Please let me know if that will work
i am asked to create a 2 projects, project A and B, that makes use of JMS. B is a WEB project with a simple page containing a form where i input two dates(start date- end date). After submitting the form, B will ask A to process a task (Query a files) based on the two dates. B will then display the result in a web page. the caveat is that i need to do this in jms.
in my B controller im thinking about this (simplified. i'm using spring )
#controller
Mycontroller{
MyMessageProducer mp;
#RequestMapping(....)
public String(...){
mp.sendMessage(...);
//wait for the response here and render?
}
}
now i'm stuck with how to implement project A. if a use a point-to-point messaging (using queue), then that means that A will have to explicitly get the message from the queue(im using activemq). Thats bad because A should be automatically listening for request, shouldn't it? However, if i use publisher-subscriber, in this case the publisher would be B, the client (because B sends message to A), which i think is a bad solution. which strategy should i use?
Now suppose that A successfully receives the message and query the file, how will i send the result back to B such the B will be able to display the result in a web page? is there a way to do this?
(PS i'm new to JMS though i've already implement a simple producers and receivers based on tutorials)
The easiest way, IMO, is to use Spring Integration which has the concept of Gateways (see also Enterprise Integration Patterns). You can just specify a service-interface that has a method with a return value. Something like
public interface MessageProducer {
#Gateway
public String sendMessageAndGetReply(String name);
}
A proxy of the interface will be created. when you reference it from the <int:gateway> element. Something like
<int:channel id="requestChannel"/>
<int:channel id="replyChannel"/>
<int:gateway id="messageProducerGateway" default-request-channel="requestChannel"
default-reply-channel="replyChannel"
service-interface="demo.MessageProducer">
</int:gateway>
<int-jms:outbound-gateway id="outboundJmsGateway"
connection-factory="connectionFactory" reply-channel="replyChannel"
request-channel="requestChannel" reply-destination-name="reply.queue"
request-destination-name="request.queue">
</int-jms:outbound-gateway>
The <int-jms:outbound-gateway> will send the message out the jms queue and receive a reply. connectionFactory is just your usual ConnectionFactory instances (i.e. ActiveMQConnectionFactory, CachingConnectionFactory)
On the "server" side of the jms interaction, you'll use a <int-jms:inbound-gateway>, Something like
<int-jms:inbound-gateway id="inboundJmsGateway"
request-channel="requestChannel" acknowledge="client"
connection-factory="connectionFactory" request-destination-name="request.queue" />
<int:service-activator id="messageHandler" ref="serverMessageHandler"
input-channel="requestChannel">
</int:service-activator>
The serverMessageHandler is just a simple component with a method to handle and return a reply back to the gateway.
#Component
public class ServerMessageHandler {
#ServiceActivator
public String handleMessage(String message) {
return "Hello, " + message;
}
}
Running a simple demo
AbstractApplicationContext client = new ClassPathXmlApplicationContext("demo-gateway.xml");
AbstractApplicationContext server = new ClassPathXmlApplicationContext("demo-gateway-server.xml");
MessageProducer producer = client.getBean(MessageProducer.class);
String returnedMessage = producer.sendMessageAndGetReply("StackOverflow");
System.out.println(returnedMessage);
you get "Hello, StackOverflow". There's not really much to it, once you get a basic understanding of the framework. The example I gave a synchronous example. You can see a full example (along with other examples) at spring-integration-samples at github. I'd take some time to go over the reference guide to get familiar with the basics.