Is it possible in WAS or on the MQ side to only ever have 1 active connection to a MQ and other passive connections?
That way I can avoid competing consumers for a queue but if a consumer fails, it would still allow for another competing consumer to connect. Like a heartbeat monitor for consumer health.
Scenerio:
Multiple WAS Server Consumers in a cluster
Only 1 MQ Queue Manager
Require only 1 active consumer due to message processing correlation. Order of messages is not important.
Anyway to send messages to only 1 active consumer while keeping other consumers passive?
I thought maybe a max open input count property on a queue would exist but did not see it.
Related
I have two ActiveMQ brokers (A and B) that are configured as a cluster of network of brokers with a static list.
Normal message consumption and store and forward works as expected. That is: messages on A with no consumers connected will be picked up by a consumer connected to broker B.
I do see that this is not the case for advisory messages.
In my current setup I have an application that is monitoring the DLQ by connecting to the advisory: ActiveMQ.Advisory.MessageDLQd.Queue. I need this to trigger certain actions when messages arrive in the DLQ.
When testing the cluster I don't see any advisory messages for DLQ messages on broker A coming through to my monitoring application connected to broker B.
Any idea how I can have advisory messages also stored and forwarded just like normal queue messages?
Advisory Messages are not really meant for that sort of thing, they are Topic based event information specific to a given broker. The brokers can use them to discover activity on another broker or a client can listen to the broker it is connected to for information on activity there but broadcasting advisories across networks would lead to confusion as you have no context on where the advisory came from or how it was acted on.
I have a master/slave AMQ broker setup for JMS messaging. I have two servers that I would like to setup as a master/slave durable consumers using Apache Camel. We've been achieving this by having both servers attempt to connect with the same client ID. One node handles all of the work but if it goes down the other node connects and picks right back up on the work. This has been working fine for having a single consumer at a time but it makes noise in disconnected server's log files with the message
ERROR org.apache.camel.component.jms.DefaultJmsMessageListenerContainer]
(Camel (spring-context) thread #0 - JmsConsumer[global.topic.event]) Could
not refresh JMS Connection for destination 'global.topic.event' - retrying
using FixedBackOff{interval=5000, currentAttempts=12,
maxAttempts=unlimited}. Cause: Broker: broker - Client: client already
connected from tcp://xxx.xx.xx.xxx:xxxx
Is there a proper way to get the functionality that I'm looking to achieve? I was considering having the slave server ping the master to coordinate which one is connected but I'd like to keep the implementation as simple as possible.
Convert your usage of topics on the consumer side to Virtual Topics. Virtual Topics allow you to continue to have existing message flows produce and consume from the topic, but also have consumers listen on specially named queues.
Once you are consuming from a queue, you can implement all the consumer patterns-- exclusive consumer (which allows that hot-standby backup consumer), message groups, parallel consumers, etc.
I read following quote from ActiveMQ documentation here:
The downside to using synchronous message delivery is that the
producer is more likely to block if there is a slow consumer that he
is dispatching messages to.
But I don't understand how the producers, in general, can be blocked because of slow consumer, because my understanding is that a JMS client (producer) will send a message to a queue of a ActiveMQ instance (broker) and that's it, now producer is free or in other words producer thread which sent the message is free to complete/die. Then there will be some other JMS clients (consumers) listening for that queue, now as soon as message arrives to the broker, broker will send it to the listening consumers.
Now, in all this where producer is dependent upon consumer?
When a producer send a persistent message to the broker the broker must write that message to disk before sending back an acknowledge to the producer to let it know what it's message is safely stored away for later dispatch to a consumer. In the case of a slow consumer the producer might eventually run into a case where the broker has reached a resource limit and cannot store that message either on the Queue in memory or on disk, it all depends on the broker's configuration.
The the producer isn't so much tied to the consumer as it is to the resources on the broker needed to hold the message until it has been consumed.
I'm pretty new to websphere MQ, so please pardon me if I am not using the right terms. We are doing a project in which we need to setup a MQ cluster for high availability.
The client application maintains a pool of connection with the Queue Manager for subscribers and publishers. Suppose we have two Queue Managers in a cluster hosting the queues with the same names. Each of the queue has its own set of subscribers and publishers which are cached by the client application. Suppose one of the queue manager goes down, the subscribers and publishers of the queues on that queue manager will die making the objects on client application defunct.
In this case can the following scenarios taken care of?
1] When first QueueManager crashes, the messages on its queues are transferred to other queuemanager in the cluster
2] When QueueManager comes up again, is there any mechanism to restore the publishers and subscribers. Currently we have written an automated recovery thread in the client application which tries to reconnect the failed publishers and subscriber. But in case of cluster setup, we fear that the publishers and subscribers will reconnect to the other running qmanager. And when the crashed queuemanager is restored, there will be no publishers and subscribers to it.
Can anybody please explain how to take care of above two scenarios?
WMQ Clustering is an advanced topic. You must first do a good amount of read up of WMQ and understand what clustering in WMQ world means before attempting anything.
WMQ Cluster differs in many ways from the traditional clusters. Unlike the traditional clusters, say in a Active/Passive cluster, data will be shared between active and passive instances of an application. At any point in time, the active instance of application will be processing data. When the active instance goes down, the passive instance takes over and starts processing. This is not the case in WMQ clusters where queue managers in a cluster are unique and hence queues/topics hosted by those queue managers are not shared. You might have the same queues/topics in both queue managers but since queue managers are different, messages, topics, subscriptions etc won't be shared.
Answering to your questions.
1) No. Messages,if persistent, will remain in the crashed queue manager. They will not be transferred to other queue manager. Since the queue manager itself is not available nothing can be done till the queue manager is brought up.
2)No. Queue manager can't do that. It's the duty of the application to check for queue manager availability and reconnect. WMQ provides automatic client reconnection feature where in the WMQ client libraries automatically reconnect to queue manager when they detect connection broken errors. This feature is available from WMQ v7.x and above with C and Java clients. C# client supports the feature from v7.1.
For your high availability requirement, you could look at using Multi instance queue manager feature of WMQ. This feature enables an Active/Passive instances of the same queue manager running on two different machines. Active instance of the queue manager will be handling client connections while the passive instance will be in sleep mode. Both instances will be sharing data and logs. Once the active instance goes down, the passive instance becomes active. You will have access to all the persistent messages that were in the queues before the active queue manager went down.
Read through the WMQ InfoCenter for more on Multi instance queue manager.
To add to Shashi's answer, to get the most out of WMQ clustering you need to have a network hop between senders and receivers of messages. WMQ clustering is about how QMgrs talk among themselves. It has nothing to do with how client apps talk to QMgrs and does not replicate messages. In a cluster when a message has to get from one QMgr to another, the cluster figures out where to route it. If there are multiple clustered instances of a single destination queue, the message is eligible to be routed to any of them. If there is no network hop between senders and receivers, then messages don't need to leave the local QMgr and therefore WMQ clustering behavior is never invoked, even though the QMgrs involved may participate in the cluster.
In a conventional WMQ cluster architecture, the receivers all listen on multiple instances of the same queue, with the same name, spread across multiple QMgrs. The senders have one or more QMgrs where they can connect and send requests (fire-and-forget), possibly awaiting replies (request-reply). Since the receivers of the messages provide some service, I call their QMgrs "Service Provider QMgrs." The QMgrs where the senders of messages live are "Service Consumer" QMgrs because these apps are consumers of services.
The slide below is from a presentation I use on WMQ Architecture consulting engagements.
Note that consumers of services - the things sending request messages - fail over. Things listening on service endpoint queues and providing services do NOT fail over. This is because of the need to make sure every active service endpoint queue is always served. Typically each app instance holds an input handle on two or more queue instances. This way a QMgr can go down and all app instances remain active. If an app instance goes down, some other app instance continues to serve its queues. This affinity of service providers to specific QMgrs also enables XA transactionality if needed.
The best way I've found to explain WMQ HA is a slide from the IMPACT conference:
A WebSphere MQ cluster ensures that a service remains available, even though an instance of a clustered queue may be unavailable. New messages in the cluster will route to the remaining queue instances. A hardware cluster or multi-instance QMgr (MIQM) provides access to existing messages. When one side of the active/passive pair goes down, there is a brief outage on that QMgr only while the failover occurs, then the secondary node takes over and makes any messages on the queues available again. A network that combines both WMQ clusters and hardware clusters/MIQM provides the highest level of availability.
Keep in mind that in none of these configurations are messages replicated across nodes. A WMQ message always has a single physical location. For more on this aspect, please see Thoughts on Disaster Recovery.
I have set up uniform distributed queue with weblogic server 12c. I am trying to achieve order of delivery and high availability with jms distributed queue. In my prototpe testing deployment I have two managed servers in the cluster, let us say managed_server1 and managed_server2. Each of this managed server hosts jms server namely jms server1 and jms server2. I have configured the jms servers with jdbc persistent store. I have enabled server affinity.
I have a producer running such as java queuproducer t3::/managed_server1. I send out 4 messages. From the weblogic monitoring console I see there are 4 messages in the queu since there are no consumers to the queue yet.
Now I shut down managed_server1.
Bring up a consumer to listen on java queuconsumer t3://managed_server2. This consumer cannot consume message since the producer send all the messages to jms server1, and it is down.
Bring up managed server 1, start a consumer to listen to t3://managed_server1 I can get all messages.
Here is my problem say if the managed_server1 went down then there it never came back up, do i loose all my messages. Also if there is another producer sending messages to java queuproducer t3://managed_server2 then order of messages based on the time between these producers are not guanranteed.
I am a little lost, am I missing something. Can unit of order help me to overcome this. Or should I use distributed topic instead of distributed queue, where all the jms server will receive all the messages from producers, but if one jms server where my consumre is listening fails there is only one consumer in my application, when I switch over to other jms server, I might be starting to get messages from the beginning not from where I left off.
Any suggestions regarding the same will be helpful.
Good Question !
" Here is my problem say if the managed_server1 went down then there it never came back up, do i loose all my messages. "
Ans - no you do not loose all your messages, they are stored in the JDBC store configured for the JMS server deployed on managed server 1. If you want the Messages sent to managed_server1 to be consumed from managed_server2 you need to configure JMS migration.
" Also if there is another producer sending messages to java queuproducer t3://managed_server2 then order of messages based on the time between these producers are not guanranteed. Can unit of order help me to overcome this."
Ans - If you want the messages to be consumed strictly in a certain order, then you will have to make use of unit of order (UOO). when messages are sent using UOO, they are sent to one of the several UDQ destinations, if midway that destination fails, and migration is enabled the messages are migrated to the next UDQ destination and new UDQ messages are also delivered to the new destination.
Useful links -
http://www.youtube.com/watch?v=B9J7q5NbXag
http://www.youtube.com/watch?v=_W3EJ8p35lI
Hope this helps.