I am working with someone who is trying to achieve a load-balancing behavior using JMS Queues with IBM Websphere MQ. As such, they have multiple Camel JMS consumers configured to read from the same Queue. Despite that this behavior is undefined according to the JMS spec (last time I looked anyway), they expect a sort of round-robin / load-balancing behavior. And, while the spec leaves this undefined, I'm led to believe that the normal behavior of Websphere MQ is to deliver the message to only one of the consumers, and that it may do some type of load-balancing. See here, for example: When multi MessageConsumer connect to same queue(Websphere MQ),how to load balance message-consumer?
But in this particular case, it appears that both consumers are receiving the same message.
Can anyone who is more of an expert with Websphere MQ shed any light on this? Is there any situation where this behavior is expected? Is there any configuration change that can alleviate this?
I'm leaning towards telling everyone here to use the native Websphere MQ clustering facility and go away from having multiple consumers pointing at the same Queue, but that will be a big change for them, so I'd love to discover a way to make this work.
Not that I'm a fan of relying on anything that's undefined, but if they're willing to rely on IBM specific behavior, I'll leave that up to them.
The only way for them to both receive the same messages are:
There are multiple copies of the message.
The apps are browsing the message without a lock, then circling back to delete it.
The apps are backing out a transaction and making the message available again.
The connection is severed before the app acknowledges the message.
Having multiple apps compete for messages in a queue is a recommended practice. If one app goes down the queue is still served. In a cluster this is crucial because the cluster will continue to direct messages to the un-served queue instance until it fills up.
If it's a Dev system, install SupportPac MA0W and tell it to trace just that one queue and you will be able to see exactly what is happening.
See the JMS spec in section 4.4. The provider must never deliver a second copy of an acknowledged message. Exception is made for session handling in 4.4.13 which I cover in #4 above. That's pretty unambiguous and part of the official spec so not an IBM-specific behavior.
Related
I am currently exploring de-duplication strategies within Active MQ. Artemis supports duplicate detection, but I'm not sure about ActiveMQ 5
Is it possible to prevent a message from being placed on a queue if it currently exists on the queue in ActiveMQ 5?
Messages which are no longer on the queue and have been so in the past will be allowed back on the queue.
The underlying capability I am trying to achieve is flow control in which multiple messages of the same value are not placed on the queue as to remove duplicate processing.
Based on the documentation, I have tried using the message property defined _AMQ_DUPL_ID, but I am still experiencing duplication. I suspect this may not be supported in ActiveMQ 5 and am unsure what alternative option is. I'm open to suggestions.
NOTE: The Active MQ instance being used is provided by Amazon MQ.
As you suspect, ActiveMQ 5.x doesn't support automatic duplicate detection. This is only supported in ActiveMQ Artemis. That said, messages are not removed from the broker's duplicate ID cache when the message is consumed from the queue. This is because in most cases a duplicate sent after the message is consumed is still considered a duplicate.
You may be able to implement some kind of duplicate detection in a broker plugin, but I have no idea of Amazon MQ supports adding custom plugins. It's more likely that you'll have to implement duplicate detection in the clients themselves.
We are migrating from Tibco to start using ActiveMQ Artemis. There are several ack settings that are available on Tibco, but we haven't found anything that's simply similar to this in Artemis. We are using the amqpnetlite .NET library to interface with Artemis, and as part of our code using DistributionMode to either move or copy based on the boolean value we are assigning to a configuration flag that we are calling as UseAutoAcknowledge. I haven't found much documentation about DistributionMode but for one that isn't very clear here - http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-messaging-v1.0.html.
My question is if DistributionMode is set to move - does Artemis send an acknowledgement to the client and doesn't when it is set to copy?
I can't talk to Tibco but I can try to explain AMQP DistributionMode. Essentially the DistributionMode is a setting as to the behaviour of the receiver - a receiver with a move mode is expecting the messages to be sent only to it, not to other receivers - this is the normal behaviour of a consumer on a queue. A receiver with a copy mode is expecting other receivers to also receive the message (like a queue browser, or - sort of - like a subscriber to a topic). In a traditional Client-Broker topology, the DistributionMode is only really interesting when receiving messages from the Broker, and is unlikely to have effect when sending messages to the Broker.
Acknowledgement is separate from the DistributionMode. AMQP has the concept of Disposition which is similar to but not the same as Acknowledgement. Disposition is ultimately the action that the sender will apply at the completion of the message transfer (and so interacts with DistributionMode for messages sent by the Broker). Conceptually for each message transfer a Broker might decide that the transfer has completed successfully; that it has failed - but in a way that retrying might succeed; that it has failed in a way that will not succeed on retry; or some other more subtle outcome. Here the behaviour at the Broker is probably different depending upon whether the DistributionMode was move or copy (the specification left this vague to allow flexibility in implementations). If the receiver is asking for messages to be moved, and it declares that the transfer was unsuccessful, a broker is likely to make that message available for all competing consumers. If the receiver was asking for copy, then it never held an exclusive lock on the message, and so the choice is only whether to retry sending the copy to that same consumer.
Perhaps the simplest thing here is if you can describe the behaviour that you desire, and experts on Apache Artemis can weigh in on if/how that can be achieved.
Is it possible to keep a history of messages (with message content would be perfect) that have already been retrieved and are no longer on a queue?
In the application I can see when the sender attempts to put the message in the queue and when the receiver attempts to pick the messages up, but I'd like to see when the message really arrived into the queue and when the messages were really received.
Does MQ Explorer have this function? How would I use it?
What you are looking for is a message tracking/auditing software for IBM MQ. You can find a list of what is available here.
It is possible to use an API exit to make copies of messages in a queue or to audit both PUT and GET operations.
It is also possible to put messages to a topic, then create as many administrative subscriptions to destination queues as required. Something can then GET and log messages from one of those destination queues. The problem with this is that MQ changes the message ID between publication and consumption whereas in a queue it remains static.
There is no native MQ function to capture messages. It's possible to use linear logs and later scrape the logs but these do not necessarily capture all messages due to optimization. (A message PUT to a waiting getter outside of syncpoint for example.) However there is at least one commercial product to scrape linear transaction logs to audit message activity.
The philosophy of MQ in general is that it is the delivery mechanism and deals with envelope data to route and deliver but does not deal with payload data. WAS, IIB and other broker/transformation engines are where IBM has put all of the functions that deal with message payloads.
We are using WebLogic 10.3.6.0 and IBM MQ 7.5.
Application design is to send messages to a dead letter queue (in WebLogic) on re-delivery. The re-delivery happens as the first delivery has failed due to some network issue or database data source failure.
My Client wants a way to browse the messages in the dead letter queue from the application GUI and pull them for processing when the network issue or data source issue has been resolved.
What is the best way to go about this?
I cam across QueueBrowser coupled with activemq or some other implementation. Is QueueBrowser possible with WebLogic? Please suggest on best ways to achieve this requirement.
Kindly pardon if my question is too naive. I am only a PL/SQL programmer.
Valerie is referring to the SYSTEM DLQ and application should never ever write to it. Application's should have there own DLQ.
i.e. If your application queue is called 'TEST.Q1' then your application DLQ should be called 'TEST.Q1.DLQ'.
There is a whole long list of MQ tools here to view messages and manage your MQ environment.
Is the application actually designed to write to the DLQ? If so, that is a very poor design. The DLQ is for the queue manager and MQ software to place messages which can not be delivered. The application should not be writing to the DLQ.
As for how to view messages on DLQ, that can be done with the MQ Explorer GUI. Or to write a script, use the DLQ handler (runmqdlq) with a rules table for processing messages.
Im a web developer ended up in some j2ee development (newbie). I sincerely need this theory confirmed.
I been given the privilege to deliver a message from our system (producer) to the SOA Enterprice service bus (consumer) when the user hits the save button. The information can not be missed or not delivered and the delivery order must be kept.
Environment:
Jboss eap 5.1 as the producer.
JNDI server is the ESB (maybe standard).
Jboss ESB as the consumer.
My weapon of choice is JMS, p2p, due to the asynchronous nature.
When the producer is abut to send the message some problems can occur:
ESB is down causing JNDI exception
Queue manager is for some reason not awake or wrongly configured. This should cause some JMS exception.
Network hickup, causing a JMS error.
So Im looking for some failover pattern. Here is my suggestion:
Add a internal JMS queue to which the message is initially added.
Add a MDB that listen to the internal queue and tries to send it to the target queue (ESB).
If failing in any way log fatal and send email to cool support people.
This should generate a reliable pattern where a message remains on the internal que until processed by the MDB.
Please advice.
Best Regards
ds
Well a 'temporary' queue is not a totally bad idea, but during the time from moving data from one queue to putting it on another you'll have a potential window of risk. Even though that window is close to nothing, what would happen if you got some failure right there and then? -You'd have to put the message back on the queue (and there you'd get into the problem with getting it in the correct order - nasty stuff!) or hold on to it in some way until you put it the other queue (which in turn can be cumbersome if you'd e g get into some failure-situaton.
A more stable solution would be to put data in a db with a queue-order column. You can then select your data in the correct order, send it to the new queue, and finally flag it as 'done' or something or even (better?) remove the data in the db.