I have a TIBCO BusinessWorks process that is publishing to a JMS topic -- lets call it TOPIC.A -- there is one process that is subscribing to that topic with a name SUBSCRIBE.A.
The problem I have is that the first server that starts listening to SUBSCRIBE.A hooks in fine. The other 3 servers, running the exact same process get an error of "WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2429' ('MQRC_SUBSCRIPTION_IN_USE')"
This can't be reasonable behavior for enterprise software, and I know WebsphereMQ, JMS and TIBCO Businessworks all scale well, so I must be missing something. I only want each event processed once, but a single box just isn't going to do it, both for fail-over reasons and shear volume reasons.
What do I have to do to let all 4 servers in the cluster to service the subscription SUBSCRIBE.A?
I agree it does not seem like reasonable behavior for enterprise software - but this restriction is imposed by the JMS specification. The JMS 1.1 spec, section 6.66.1 says "Only one session at a time can have a TopicSubscriber for a particular durable subscription".
That said, WebSphere MQ does provide a vendor-specific option that allows you to do what you want: see the CLONESUPP Connection Factory property. This is documented in the Infocenter at the CLONESUPP properties page.
While it's specific to MQ, if you specify this using administered objects you code will not need to use any vendor-specific methods.
The reason for MQRC 2429 is because all of your 4 subscribers are using the same client ID and all are trying to use the same durable subscription. When a subscriber is already actively listening on a durable subscription, no other subscribers can listen on the same. If you want all subscribers to listen concurrently, then have separate client ids for each of your subscribers.
But you must note that all subscribers will get a copy of the same message published by a publisher (TIBCO BusinessWorks in your case). So if all subscribers are active, all will process the event.
For fail-over you can look at using WebSphere MQ Multi-instance queue manager feature or any other HA solutions. For load sharing you can look at WebSphere MQ clustering feature.
Related
I'm an IBM MQ novice, however have used other messaging systems in the past (Solace, RabbitMQ, BizTalk). I apologize if some of my MQ terminology is incorrect when it comes to local/remote/transmission queues etc.
I am integrating with a 3rd Party using MQ. I have no control over the use of MQ when it comes to this 3rd party. They define the set integration pattern.
I have my own local Queue Manager.
The 3rd party supports many 100s of different message types.
Typically each message sent to them will result in a response
Responses on overage will be delivered back to my QM within 0.5s (agreed SLA)
The basic model is as follows;
For all outbound messaging I publish to a single queue
All responses come to a single inbound queue on my QM
I do not believe they respect/use the ReplyToQ or ReplyToQmgr header properties
High level overview of the use case;
The MQ usage will be very active. Perhaps 500-1000 outbound messages per second
For the vast majority of these - there will be a user waiting for a response
The current design approach is to put a message onto the outbound queue and then create a subscription to the inbound queue with a JMS selector using the correlation ID.
Here is my challenge;
I am not sure if having so many concurrently subscriptions all with a unique filter on the single inbound queue will perform well. I would appreciate insights into this. I have a 2 node cluster running in a docker container. I don't have details on the spec yet.
My preference would be have messages arrive into the inbound queue and then be routed to many "function" specific local queues on the QM.
I would still use JMS selectors on the message ID in this model - however I would hope the load would be spread from a single deep queue to many shorter queues
It does not seem that I could route messages natively within MQ without an addon such as MQ Integration Broker or ESB (some other products within the WebSphere suite).
It is on this last point I could use some guidance. Can I route within my QM? If so - what options do I have. Alternatively - perhaps there are much better approaches that I have not considered?
Any guidance would be heartily appreciated!!
I am wondering if MQ can be used as a state cache for monitoring? And is this a good idea or not?
In theory you can have many sources (monitoring agents) that detect problem states and distribute them to subscribers via an MQ system such as RabbitMQ. But has anyone heard of using MQ systems to cache the state, so when clients initialize, they read from the state queue before subscribing to new state messages? Is that a bad way to use MQ?
So to recap, a monitor would read current state from a state queue then setup a subscription queue to receive any new updates. And the state queue would be maintained by removing any alerts that are no longer valid by the monitoring agents that put the alert there to begin with.
Advantage would be decentralized notification and theoretically very salable by adding more mq systems to relay events.
I have a use case for Rabbit MQ that holds the last valid status of a system. When a new client of that system connects it receives the current status.
It is so simple to do!
You must use the Last Value Cache custom exchange https://github.com/simonmacmullen/rabbitmq-lvc-plugin
Once installed you send all your status messages to that exchange. Each client that needs the status information will create a queue that will have the most recent status delivered to that queue on instantiation. After that it will continue to receive status updates.
IBM MQ FTE uses such way for storing logs.
I think it is good idea, if you can prevent destination queue from overflow, because IBM MQ for example remove overdue messages only during GET call.
I am working with someone who is trying to achieve a load-balancing behavior using JMS Queues with IBM Websphere MQ. As such, they have multiple Camel JMS consumers configured to read from the same Queue. Despite that this behavior is undefined according to the JMS spec (last time I looked anyway), they expect a sort of round-robin / load-balancing behavior. And, while the spec leaves this undefined, I'm led to believe that the normal behavior of Websphere MQ is to deliver the message to only one of the consumers, and that it may do some type of load-balancing. See here, for example: When multi MessageConsumer connect to same queue(Websphere MQ),how to load balance message-consumer?
But in this particular case, it appears that both consumers are receiving the same message.
Can anyone who is more of an expert with Websphere MQ shed any light on this? Is there any situation where this behavior is expected? Is there any configuration change that can alleviate this?
I'm leaning towards telling everyone here to use the native Websphere MQ clustering facility and go away from having multiple consumers pointing at the same Queue, but that will be a big change for them, so I'd love to discover a way to make this work.
Not that I'm a fan of relying on anything that's undefined, but if they're willing to rely on IBM specific behavior, I'll leave that up to them.
The only way for them to both receive the same messages are:
There are multiple copies of the message.
The apps are browsing the message without a lock, then circling back to delete it.
The apps are backing out a transaction and making the message available again.
The connection is severed before the app acknowledges the message.
Having multiple apps compete for messages in a queue is a recommended practice. If one app goes down the queue is still served. In a cluster this is crucial because the cluster will continue to direct messages to the un-served queue instance until it fills up.
If it's a Dev system, install SupportPac MA0W and tell it to trace just that one queue and you will be able to see exactly what is happening.
See the JMS spec in section 4.4. The provider must never deliver a second copy of an acknowledged message. Exception is made for session handling in 4.4.13 which I cover in #4 above. That's pretty unambiguous and part of the official spec so not an IBM-specific behavior.
I have a .NET application that needs to monitor a queue in WebSphere MQ. I need to react to each message without impacting the current process. The client application can't explicity send me the same message.
Can I read a message without removing it from the queue? Can I be notified for each message? Can I configure the MQ to duplicate the current queue?
Is there another solution?
If you are using WMQ v7 then you can do this without any impact to the existing applications other than to change the queue name for one of them.
Currently the message producer and consumer use the same queue. In v7 of WMQ you can create an alias over a topic so that the message producer thinks it's a queue. Then you can create two administrative, durable subscriptions such that one points to the existing input queue and another points to a queue dedicated to your new application.
Of course you are already using v7 since v6 goes out of service next year, right? You can upgrade the QMgr to v7 which enables this behavior while still using v6 client code for the apps.
If you are using WMQ v6 then the MirrorQ program might work for you.
You could change from using a queue to a durable topic and have both your reader and your browser subscribe to it.
You could also create a distribution list on your queue manager. A distribution list is used to send a copy of the same message to multiple queues. You would then have a processing queue and a browsing/monitoring queue.
I'm queuing messages to a WebSphere MQ queue (NB: A point-to-point queue -- not a topic) using a stored procedure in my Oracle database. Is there a way to publish each message to multiple queues with a single call? What I would like is to find a solution that would incur zero additional latency on my database compared to sending the message to a single queue.
Solutions that involve changing my WebSphere MQ settings are certainly welcome! What I had in mind was somehow creating a "clone" queue that got all the same messages as the original one, but I've been unable to locate anything like this in the documentation.
Thanks,
Jeff
With WMQ v7 you can do this easily and with administration only. You would create a topic object and then an alias over the topic. The Oracle app writes to the alias and does not know that it is actually publishing.
Meanwhile, you make two administrative subscriptions on the topic so that publications are delivered to your two destination queues. The apps consuming them have no idea that the messages were published as opposed to delivered through point-to-point queues.
If you are not familiar with the new WMQ v7 features, take a look at the Infocenter. In particular, the "What's New in V7" section and the sections on Pub/Sub.
You can accomplish this using "Distribution Lists" in WebsphereMQ. These have to be configured on your queue manager.
Take a look at the Wesbphere MQ Application Programming Guide for more info.