I am working on an existing Spring jms application that pulls messages from ActiveMQ using jms:inbound-gateway. This application is a job processor that takes jobs off a queue and sends results back to the queue. Everything currently works great.
I would like to modify this application to accept one and only one job, process it, and then exit, but I have not been able to find a way to cleanly do this. The method that is called must return the results, and the results are automatically placed back onto the queue by Spring. Is there any way to tell Spring to stop accepting messages at 1? How would you know when it's done sending the reply message so you can exit.
In a more general case, if you had an application that wanted to stop accepting messages and finish processing them all so you could exit cleanly, how can that be done?
Thanks in advance for any advice.
It sounds like you should be using JmsTemplate.receive . Inbound gateway is meant for a message driven model. It sounds like while you are using a message queue for transport, you're not really doing a message driven model of processing. Obviously that's not a problem but if your aim is to process a single message you should just use JmsTemplate.
Another way to do this is to use the default message listener container, ensure threads is set to one and then make sure your activemq setup has no read-ahead setup (i.e. reads one message at a time and no more). Then, at the end of your onMessage method, call the DMLC.stop method.
Related
I am new to Microservices and have a question with RabbitMQ / EasyNetQ.
I am sending messages from one microservice to another microservice.
Each Microservice are Web API's. I am using CQRS where my Command Handler would consume message off the Queue and do some business logic. In order to call the handler, it will need to make a request to the API method.
I would like to know without having to explicit call the API endpoint to hit the code for consuming messages. Is there an automated way of doing it without having to call the API endpoint ?
Suggestion could be creating a separate solution which would be a Console App that will execute the RabbitMQ in order to start listening. Create a while loop to read messages, then call the web api endpoint to handle business logic every time a new message is sent to the queue.
My aim is to create a listener or a startup task where once messages are in the queue it will automatically pick it up from the Queue and continue with command handler but not sure how to do the "Automatic" way as i describe it. I was thinking to utilise Azure Webjob that will continuously be running and it will act as the Consumer.
Looking for a good architectural way of doing it.
Programming language being used is C#
Much Appreciated
The recommended way of hosting RabbitMQ subscriber is by writing a windows service using something like topshelf library and subscribe to bus events inside that service on its start. We did that in multiple projects with no issues.
If you are using Azure, the best place to host RabbitMQ subscriber is in a "Worker Role".
I am using CQRS where my Command Handler would consume message off
the Queue and do some business logic. In order to call the handler, it
will need to make a request to the API method.
Are you sure this is real CQRS? CQRS occures when you handle queries and commands differently in your domain logic. Receiving a message via a calss, that's called CommandHandler and just reacting to it is not yet CQRS.
My aim is to create a listener or a startup task where once messages
are in the queue it will automatically pick it up from the Queue and
continue with command handler but not sure how to do the "Automatic"
way as i describe it. I was thinking to utilise Azure Webjob that will
continuously be running and it will act as the Consumer. Looking for
a good architectural way of doing it.
The easier you do that, the better. Don't go searching for complex solutions until you tried out all the simple ones. When I was implementing something similar, I was just running a pool of message handler scripts using Linux cron. A handler poped a message off the queue, processed it and terminated. Simple.
I think using the CQRS pattern, you will have events as well and corresponding event handlers. As you are using RabbitMQ for asynchronous communication between command and query then any message put on specific channel on RabbitMQ, can be listened by a callback method
Receiving messages from the queue is more complex. It works by subscribing a callback function to a queue. Whenever we receive a message, this callback function is called by the Pika library.
I have a dead-letter queue which stores failed messages. Is there have a way to manually trigger a listener to process the messages in the dead-letter queue? I’m using Spring AMQP.
Like I said, it depends on your use case, but I can assume you are looking for something from spring amqp out-of-the-box.
just have a look at BlockingQueueConsumer, you can i.e. prefetch one message if you want, then start it manually.
Please be aware that you need choose solution according to your case, maybe you just need a flag in db to mark the job is failed, or save the msg in db.
Sorry, if it is a duplicate question.
I have a legacy web application which uses Queues (yes. normal Java Queue) and custom polling (every 500ms). A REST web service (/message) will be called, which will return the message if any otherwise empty string.
My need: If any message is available in Queue, in Real-Time, the client should get the message. So I can save 500ms.
Is there any advantage to moving to JMS from current approach? From this link JMS MessageConsumer's messageListener makes push or pull? it seems, MessageListener (process is asynchronous) uses polling which is no different from current approach.
If it is vendor based, how HornetQ/ActiveMQ supports MessageListener?
EDIT:
The queue is used for integration of two systems. A web app & standlone java program.
Either receive or a MessageListener will be asynchronous and will be called as soon as you receive a message.
you could control the pre-fetch size of your client.
Now, if all you need is to avoid the delay of poling every 500 ms, using a Queue system may be an overkill? It's perfect fine to use java.util.Queue (or any other subclass).
If all you need is to block until an element of a java.util.Queue is available, and you don't need distributed messaging, persistence or anything like you could simply using BlockingDequeue and your thread would unblock as soon as you have a message..
Look at this:
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/BlockingDeque.html
The Async MessageListener is implemented using a push based model. In ActiveMQ the broker sends a number of messages to the client based in it's set prefetch value so that messages are ready for consumption. Whether or not this helps with your particular use case is a question you need to answer for yourself.
I'm pretty new to Spring Integration and still trying to get my head around it. Right now I'm just trying to understand if the example I've found here is actually safe across multiple threads:
https://github.com/spring-projects/spring-integration-samples/blob/master/basic/jms/src/test/java/org/springframework/integration/samples/jms/ChannelAdapterDemoTest.java
My use case is as follows:
Send request to queue with JMS Reply-to as a temporary queue
Wait for response to be received on the temporary queue
Need this to happen synchronously within a method -- I don't want to split it up and make it asynchronous across several methods
Will the above example work for this? If not, am I barking up the wrong tree?
Thanks in advance.
That sample is pretty simple; it just sends the message to stdout so, yes, it's perfectly thread safe.
For the request/reply scenario you are talking about, you need to use a <gateway/> - see the other example in that sample project. In that case, you can see that the message is handled by 'demoBean' which, again, is perfectly thread safe.
For a real application, the thread-safetyness depends on the code in the services invoked by the flow receiving the message.
If you wish, you can use Spring Integration on the client side too (with an outbound gateway).
I'm using a message listener to process some messages from MQ based on Spring's DefaultMessageListenerContainer. After I receive a message, I have to make a Web Service (WS) call. However, I don't want to do this in the onMessage method because it would block the onMessage method until the invocation of WS is successful and this introduces latency in dequeuing of messages from the queue. How can I decouple the invocation of the Web Service by calling it outside of the onMesage method or without impacting the dequeuing of messages?
Thanks,
I think you might actually want to invoke the web service from your onMessage. Why do you want to dequeue messages quickly, then delay further processing? If you do what you're saying, you'd probably have to introduce another level of queueing, or some sort of temporary "holding" collection, which is redundant. The point of the queue is to hold messages, and your message listener will pull them off and process them as quickly as possible.
If you are looking for a way to maximize throughput on the queue, you might think about making it multi-threaded, so that you have multiple threads pulling messages off the queue to invoke the web service. You can easily do this by setting the "concurrentConsumers" configuration on the DefaultMessageListenerContainer. If you set concurrentConsumers to 5, you'll have 5 threads pulling messages off the queue to process. It does get tricky if you have to maintain ordering on the messages, but there may be solutions to that problem if that's the case.
I agree with answer provided before me , however I can see a usecase similar to this very common in practice. I'm adding my two cents It might be valid in some cases that you don't want to do time consuming work in your onMessage Thread (which is pulling message from Q)
We have something similar in one workflow, where if user selects some XYZ option on GUI that means at server we need to connect to another external webservice to get ABCD in this case we do not make call to webservice in onMessage Thread and use ThreadPool to dispatch and handle that call.
If something wrong happens during webservice call we broadcast that to GUI as separate Message , there is concept of request id which is preserved across messages so that GUI can relate error messages. You can use ExecutorService implementation to submit task.
hope it helps.