I am using the Spring DefaultMessageListenerContainer to gain some dynamic benefits in setting the MessageSelector value since I am using the Glassfish OpenMQ which is not that advanced in that regards.
Let's have a JMS message. The listener issues a specific failure that means: retry after x seconds. It tries again with failure: retry after x*y seconds, and so on the time grows exponentially. If you cannot handle it after z retries, consider it as a poison JMS message.
DefaultMessageListenerContainer dmlc;
dmlc.stop();
dmlc.setMessageSelector(String.format("retries < %d AND retryTime <= %d", z, System.currentTimeMillis()));
dmlc.start();
I am not that satisfied with this solution, especially, when the Spring docs raise warning here:-). However, for the moment things meet our needs.
Now, I have a number of EJBs message consumers on different applications. Some of them need such dynamic changes of the messageSelector. Unfortunately, and to-my-best-knowledge, EJB MDBs do not support such dynamic "features". For example, see this.
Is that correct? is there a workaround for an EJB solution? I would appreciate any help.
To achieve dynamic changes to the message selector, you'd need to implement it straight in JMS, e.g.
ConnectionFactory cf;
Connection connection = cf.createConnection();
session = connection.createSession(transactional, acknowledgeMode);
MessageConsumer messageConsumer = session.createConsumer(destination, "message selector");
Additionally, you'd need to place this code some place it executes on its own, perhaps in an asynchronous task? But you'd be reinventing the wheel, as Spring DMLC does that better.
I don't know why you're doing this:
for load balancing? The message broker should take care of this.
for handling temporary downtimes? The queue should be configured to be able to store appropriate number of messages, or switch delivery to other node in cluster.
Related
I can't find this information anywhere. I have two queues, #JmsListener(destination = "p1"), #JmsListener(destination = "p2"). How can I make sure I only process 1 message at a time, even though I am listening to 2 queues, and also how do I configure the polling of what queue I get messages from first, that is after processing a message I want to poll p1 first. Or do weighted polling: p1:90%, p2:10%. Etc.
Basically I am asking how to implement priority processing of messages for Spring. I'm using SQS which doesn't support priorities.
Use one of the JmsTemplate receive() or receiveAndConvert() methods instead of the message-driven model.
Use transactions if you want to ensure no message loss.
I have a basic JMS related question in spring.
Rather than having to consume a single message at a time, it would be convenient to batch messages for a short duration (say a few seconds) and process them in bulk (thereby doing things in bulk). I see that java only provides an onMessage call that gives a single message at a time. I came across BatchMessageListenerContainer which seems to do this exactly. The recipe is ported to spring-batch where it is being used.
I wanted to know if there are any fundamental problem in the approach itself? If there are no problems, we can propose to the spring folks to add this in the spring-jms artifact itself (without needing to resort to use spring-batch whatsoever).
Thanks!
If your need is to process the messages in parallel you can use DefaultMessageListenerContainer in your spring project without the necessity for spring batch. You set the attribute concurrent consumers to the number of partitions you want.
#Bean
public DefaultMessageListenerContainer messageListener() {
DefaultMessageListenerContainer listener = new DefaultMessageListenerContainer();
**listener.setConcurrentConsumers(Integer.valueOf(env.getProperty(JmsConstant.CONCURRENT_CONSUMERS_SIZE)));**
// listener.setMaxConcurrentConsumers(maxConcurrentConsumers);
listener.setConnectionFactory((ConnectionFactory) queueConnectionFactory().getObject());
listener.setDestination((Destination) jmsQueue().getObject());
listener.setMessageListener(this);
listener.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
listener.setSessionTransacted(true);
return listener;
}
Otherwise, if you're using spring batch, you can use remote chunking and BatchMessageListenerContainer, you can find an example here https://github.com/spring-projects/spring-batch/tree/master/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotechunking
I have one doubt during my work with JMS. As I know it's possible to create synchrous message consumer. However, I must launch it with a frequency, because of the fact that there is no listener. Next, to consume messages synchrously from a queue I can create a MDB and set the pool to 1. I think it is not a good solution.
My aim is to consume messages synchrously when they appear on the queue. From my point of view above mentioned solutions are not good:
1. Consumer which is launched from time to time.
2. MDB (asynchrous normally) and pool is set to 1.
Are there any solutions for my purpose?
Not sure why you don't like MDBs... but if you want to avoid them, you could use the Spring JMS listener:
http://docs.spring.io/spring-framework/docs/current/spring-framework-reference/html/jms.html
I plan to have persistent message Queues based on some implementation of AMQP and JMS API. I would like to know whether is ok (from architectural point of view) to have messages staying in the queues for hours. A day is max.
I plan to use the message broker as another persistence layer basically. Is this viable?
The technologies that I am evaluating are ActiveMQ, RabbitMQ or qupid.
I plan to use the message broker as another persistence layer
basically. Is this viable?
The broker's persistence mechanism for message retention is usually file-based, or JDBC; either one will work. It is viable? Sure, its a feature of the broker, nothing wrong with using it for the intended purpose, assuming temporary message retention is your goal; 1 day is not a big deal.
But if you're planning to retain messages for 1 day, or more, I recommend doing some calculations based on average message size and total messages per day that may end up sitting in a queue. Queue depth, by default, is usually a low number, like 10Mb, and if exceeded, the broker will probably drop subsequent messages; you want to prevent this from happening. Vendors handle this differently, so check with RabbitMq and ActiveMQ for specifics and what configuration parameters are used to control depth. I know SonicMq has what's known as the "DeadMessage" queue, a destination for expired or undeliverable messages; other products might have something similar.
It's OK to have persistent queues, and it's OK if messages are hanging around in the queues: Clients might be disconnected because of updates, network problems etc. That's one benefit of queues to decouple sender from receiver, and the queue is the buffer. However these use cases are not the normal mode of operation, it's rather an exceptional situation.
Using a messaging broker as "another persistence layer" is technically speaking possible, but in this case a database is probably more suitable, because quick message delivery/messaging and long term storage/database are different tools/scenarios. So ask yourself the question: Is it still messaging or is it already a database?
If in your use case the normal message delay (= period between sending and reception) is always beyond an hour, a database might be better, because JMS selectors are normally slower and less comfortable than database queries using where clauses.
There is another aspect: Consider the need for an online backup of your messages in a JMS provider, especially in a HA cluster mode. It might be easier to do this using a database.
I have a JMS queue published by a third party.
I want to setup multiple consumers on different machines, with only one particular machine's consumer, acknowledging messages on that queue. In short, if a particular machine's consumer does not receive the message, then that message should not be removed from queue.
Is this achievable ?
Okay, you might have your reasons for this setup and it's easy to achieve.
I would go with local session transactions. It is rather easy to commit or rollback the transactions acording to some critera, such as which server is consuming the message. If rolled back, the message will end up first in the queue again.
Sample code might look like this:
public class MyConsumer implements MessageListener{
Session sess;
public void init(Connection conn, Destination dest){
// connection and destination from JNDI, or some other method.
sess = conn.createSession(true, Session.AUTO_ACKNOWLEDGE);
MessageConsumer cons = sess.createConsumer(dest);
cons.setMessageListener(this);
conn.start();
}
#Override
public void onMessage(Message msg) {
// Do whatever with message
if(isThisTheSpecialServer()){
sess.commit();
}else{
sess.rollback();
}
}
private boolean isThisTheSpecialServer(){
// figure out if this server should delete messages or not
}
}
If you are doing this inside a Java EE container with JTA and you are using UserTransactions, you could just call UserTransaction.setRollBack();
or if you are using declarative transactions you could just throw a Runtime exception to make the transaction fail and rollback the message to the queue, once you have read the message and done things. Note that database changes will roll back as well with this approach (if you are using JTA and not local JMS transactions).
UPDATE:
You should really do this using transactions, not acknowledgement.
A summary of this topic (for ActiveMQ, but written generally for JMS) is found here.
http://activemq.apache.org/should-i-use-transactions.html
I don't know if this behaviour is consistent with all JMS implementations, but for ActiveMQ if you try to use a non transacted session with Session.CLIENT_ACKNOWLEDGEMENT, then it will not really behave as you expect. A message that has been read, but not acknowledged, is still on the queue, but will not get "released" and delivered to other JMS consumers until the connection is broken to the first consumer (i.e. connection.close(), a crash or similar).
Using local transactions, you can controll this by session.commit() and session.rollback() explicitly. I see no real point in not using transactions. Acknowledgement is just there to guarantee delivery.
Another way to look at this is in the case of a forwarding queue. You could apply it to your design by doing the following:
Create a consumer on the published queue from the third party.
This consumer has one job - distribute every message to other queues.
Create additional queues that your real subscribers will listen to.
Code your message listener to take each message and forward it to the various destinations.
Change each of your listeners to read from their specific queue.
By doing this, you ensure that every listener sees every message, every transaction works as expected, and you don't make any assumptions about how the message is being sent (for example, what if the publisher side is doing AUTO_ACKNOWLEDGE ?)