Need to have two outbound queues on two different servers and queue managers act as primary and secondary respectively if the sending to primary fails want to connect to secondary but as soon as primary become up application must send to primary any spring boot configuration can help? using websphere MQ.
For a Java JMS messaging application in Spring Boot, a connection name list allows for setting multiple host(port) endpoints as comma separated pairs.
The connection factory will then try these endpoints in turn to finds an available queue manager. Unsetting WMQConstants.WMQ_QUEUE_MANAGER means the connection factory will connect to the available queue manager on a given host(port) endpoint regardless of its name.
You'll need to think about the lifecycle of the connection factory as the connection will remain valid once it has been established. In the scenario where the connection factory is bound to the queue manager on hostB(1414) (the second in the list) and then the queue manager on hostA(1414) (the first in the list) becomes available again, nothing would change until the next connection attempt.
It's important to note that where the queue manager endpoints in the connection name list are unrelated, then the queues and messages available will not be the same. Multi-Instance queue managers allow a queue manager instance to failover between two host(port) endpoints. For container deployments, IBM MQ Native HA ensures your messaging applications are routed to an active instance of the queue manager
A CCDT allows for much more sophisticated connection matching options with IBM MQ over the connection name list method outlined above. More information on CCDT is available here. For JMS applications you'll need to set the connection factory to use CCDT e.g., cf.setStringProperty(WMQConstants.WMQ_CCDTURL, "") to the location of the CCDT JSON file.
It's probably worth me putting up an answer rather than more comments, though most of the ground has been covered by Rich's answer, augmented by Rich's, Morag's and my comments.
Client Reconnection seems the most natural fit for the first part of the use-case, and I'd suggest using a connection name list rather than going to the complexity of using a CCDT for this job. This will ensure that the application connects to your primary queue manager if it's available and the secondary queue manager if the primary isn't available, and will also move applications to the secondary if the primary fails.
Once a connection has been made, in general it won't move. The two exceptions are that in client reconnection configuration, a connection will be migrated to a different queue manager if communication to the first is broken, and in the context of uniform clusters connections may be requested to migrate to a queue manager to balance load over the cluster. There is no automatic mechanism that I can think of which would force all the connections back from your secondary queue manger to the primary - you'd need to either do something with the application, or in a client reconnect setup you could bounce the secondary queue manager.
(You could use this forum to request such a feature, but I can't promise either that the request would be accepted or that it would be rapidly acted on.)
If you want to discuss this further, I'd suggest that more dialogue is probably needed to help us understand your scenario properly, so that we can make the most helpful recommendations. The MQ discussion forum may be a useful place for such dialogue, as it doesn't fit well (IMHO) to the StackOverflow model.
Related
We have multiple application environments (development, QA, UAT, etc) that need to connect to fewer provider environments through MQ. For example, the provider only has one test (we'll call it TEST1) environment to which all of the client application environments need to interact. It is imperative that each client environment only receives MQ responses to the messages sent by that respective environment. This is a high volume scenario so correlating message IDs has been ruled out.
Right now TEST1 has a queue set up and is functional, but if one of the client app's environments wants to use it the others have to be shut off so that messaging doesn't overlap.
Does MQ support a model having multiple clients connect to a single queue while preserving the client-specific messaging? If so, where is that controlled (i.e. the channel, queue manager, etc)? If not, is the only solution to set up additional queues for each corresponding client?
Over the many years I have worked with IBM MQ, I have gone back and forth on this issue. I've come to the conclusion that sharing a queue just makes life more difficult. Queues should be handed out like candy on Halloween. If an application team says that they have 10 components to their application then the MQAdmin should give them 10 queues. To the queue manager or server or CPU or hard disk, there is no difference in resource usage.
Also, use an MQ naming standard that makes sense and is easy to apply security to. i.e. for HR (Human Resource) department
HR.PAYROLL.SALARY
HR.PAYROLL.DEDUCTIONS
HR.PAYROLL.BENEFITS
HR.EMPLOYEE.DETAILS
HR.EMPLOYEE.REVIEWS
etc...
You could use a selector such as MQGET(where applname="myapp") or based on a specific user-defined property assuming the sender populates such a property but that's likely to be worse performance than any retrieval by msgid or correlid. Though you've not given any information to demonstrate that get-by-correlid is actually problematic.
And of course any difference between a test and production environment - whether it involves code or configuration - is going to be very risky.
You would not normally share a single destination queue between multiple different application types - multiple queues is far more standard.
we are running an in-house EAI system using ActiveMQ as the message broker using JDBC persistence.
There we have a cold-standby failover solution each one having an own database schema (due to several reasons).
Now if the primary goes down and we want to startup the backup we would like to transfer all undelivered messages on database level from the one node to the other.
Having a look at the table "ACTIVEMQ_MSGS" made us unsure if we can do this without any drawbacks or side effects:
There is a column "ID" without any DB-sequence behind - can the backup broker handle this?
The column "MSGID_PROD" contains the host name of the primary server - is there a problem if the message should be processed by a broker with a different name?
There is a column "MSGID_SEQ" (which seems to be "1" all the time) - what does this mean? Can we keep it?
Thanks and kind regards,
Michael
I would raise a big red flag about this idea. Well, yes, in theory you could well succeed with this, but you are not supposed to touch the JDBC data piece by piece.
ActiveMQ has a few different patterns for master/slave HA setups. Either using a shared store for both the master and the slave, or use a replicated store (LevelDB+ZooKeeper).
Even a shared JDBC store could be replicated, but on the database level.
Ok, so you somehow want another setup than the official ones, fine. There is a way, but not using raw SQL commands.
By "Primary goes down", I assume you somehow assumes the primary database is still alive to copy data from. Fine. Then have a spare installation of ActiveMQ ready (on a laptop, on the secondary server or anywhere safe). You can configure that instance to connect to the "primary database" and ship all messages over to the secondary node using "network of brokers". From the "spare" broker, configure a network connection to the secondary broker and make sure you specify the "staticBrige" option to true. That will make the "spare" broker hand over all unread messages to the secondary broker. Once the spare broker is done, it can be shut down and the secondary should have all messages. This way, you can reuse the logic in whatever ActiveMQ version you have and need not to worry about ID sequences and so forth.
Issue:
Having multiple consumer applications active specification attached to a single MQ on distributed VM servers is causing a null payload in a MQ Message.
Note: See notes at the bottom. No issue with mq.
Details:
I have 3 Websphere applications deployed across 2 VM servers. 1 application is a publisher and the other 2 applications are consumers attached to a single MQ Manager, and MQ.
2 consumer applications are pulling off the messages and processing them. The consumer application on the separate server receives a null payload. I have confirmed that its seems to be an issue having multiple application server instances attached to MQ. Confirmed by deploying the publisher on server 2 with consumer 2, then consumer 1 fails.
Question:
Has anyone tried attaching multiple MDB applications deployed on separate server instances bind to one Queue Manager and one MQ?
Specifications:
Websphere 7, EJB 3.0 MDB's,Transactions turned off,Queue in a queue installed on another machine.
Goal:
Distributed computing, scaling up against large number of messages.
I'm thinking this is a configuration issue but not 100% sure where to look. I had read that you could use MQLink, but I don't see why I would need to use service bus integration.
Supporting Doucmentation:
[MQ Link][1
UPDATE: I fixed the problem and it was a related to a combination of class loader issue with a duplicate classes. See Solution Notes below I added.
EDIT HISTORY:
- Clarified specifications, clarified question and added overall goal.
- reference notes to solution.
Has anyone tried attaching multiple MDB applications deployed on
separate server instances bind to one Local MQ?
Multiple MDB applications deployed on separate servers, connecting to one Queue Manager(but different queue) is a normal scenario and we have it everywhere and all applications work fine.
But, I suspect what you are doing is: Multiple MDB applications deployed on separate servers, connecting to one Queue Manager and listening to the same queue.
In this case one message will be received by one consumer only.
You need to create separate queue for each application and create subscriptions for each for the topic being published by your publisher.
Addition:
I suspect, for load balancing the problem you may be facing is that, when your first application gets the message, it doesn't issue a commit. So, there will be an uncommited message in the queue, which may be stopping your other application from getting message from the queue. When your first application finishes its processing, it issues a commit, but then again it is ready to pick the message and hence it again issues a get.
In my architecture, we have implemented load balancing using multiple queue managers like below:
You create 3 queue managers, say GatewayQM, App1QM and App2QM.
Keep the three queue managers in the same cluster.
Create an alias queue(shared in cluster) in GatewayQM and ask your putting app to put message in the gateway queue.
Now create one local cluster queue in each of App1QM and App2QM. Read from these queues via your applications App1 and App2 respectively.
This implementation provides you better security and serves a perfect load balancer.
This specific problem was caused by a code issue and the combination of class loading being set to "Parent First" in the Websphere console. On one node it would work and the other nodes in a cluster would fail, I think this was caused by the "Parent First" setting.
More important, in terms of my configuration in a cluster binding multiple active specifications to a single MQ to provide distributed computing is a correct solution.
However "points" due go to "nitgeek" solution references above if you are looking for a extremely high volume solution. Its important to understand that a single MQ can have a very high depth and takes a lot to fully utilize one. My current configuration is a good starting point for quick configuration and distributed processing using Multiple MDB's.
I have two MQ queue manager with same queue names configured. Both are configured to send data to different servers. Currently queue manager(QM1) is stopped(status Ended Immediately) and QM2 is running
Now my program opens the queue and sends data. It doesnot specify queue manager name. When I execute the program, MQ connection request returns error 2059.
My questions are:
What happens when multiple queue managers have same queue name?
How to tackle situation without changing the code?
Please forgive if the description is vague. It would be helpful if anyone provide links so that newbie like me can learn something.
Thanks
It would be helpful if could provide details on your application. Whether it's using server bindings or client mode connection to queue manager. What version of MQ are you using?
The below information is valid for MQ v7.x:
If you are using client mode then you can use multiple CONNNAMEs to connect. If one queue manager is down, your application will connect to next queue manager in CONNAME list. One of the simplest way to do when using client mode connection is to define MQSERVER environment variable and specify multiple CONNNAMEs.
SET MQSERVER=<channel name>/TCP/host1(port1), host2(port2)
For example when both queue managers are on local host:
SET MQSERVER=MYSVRCONCHN/TCP/localhost(1414),localhost(1415)
In server bindings mode if queue manager name is not specified, then application will attempt to connect to the default queue manager. If the default queue manager is down, then 2059 is thrown.
Your explaination doesn't provide clarity about your requirements.
You wrote:
My questions are 1. What happens when multiple queue managers have same queue name.
Nothing. Its a normal scenario. Different queue managers may have queues with same name and it doesn't create any ambiguity. Although, scenario will be a little different when the queue managers are in same cluster and the queue is also a cluster queue. Then everything will depend on requirements and design.
You wrote:
2. How to tackle situation without changing the code
Run the queue manager which is stopped.
You wrote:
Now my program opens the queue and sends data. It doesnot specify
queue manager name.
What application are you using?For a client application, you access a queue using an object of queue manager.
I am asssuming that you are using an application(client) which doesn't take queue manager details from you, only takes queue details. And may be the queue manager is hard coded within the code. And it sends the message first to the queue of Queue manager 1 and then to queue manager 2. But, in your case queue manager 1 is down.
If above is the case, then the application's code needs to be changed. You should have exception handling in such a way that it executes the code for sending the message to the second queue manager even though the first lines of code throws error.
When I read about HA in Websphere MQ I always come to the point, when the best practise is to create two Queue Managers handling the same queue and use out-of-the-box load balancing. Therefore, when one is down, the other takes over his job.
Well, this is great but what about the messages in the queue that belong to the Queue Manager that went down? I mean do these messages reside there (when queue is persistent of course) until QM is up and running again?
Furthermore, is it possible to create a common storage for this doubled Queue Managers? Then no message would wait for the QM to be up. Every message would be delivered in the proper order. Is this correct?
WebSphere MQ provides different capabilities for HA, depending on your requirements. WebSphere MQ clustering uses parallelism to distribute load across multiple instances of a queue. This provides availability of the service but not for in-flight messages.
Hardware clustering and Multi-Instance Queue Manager (MIQM) are both designs using multiple instances of a queue manager that see a single disk image of that queue manager's state. These provide availability of in-flight messages but the service is briefly unavailable while the cluster fails over.
Using these in combination it is possible to provide recovery of in-flight messages as well as availability of the service across multiple queue instances.
In hardware cluster model the disk is mounted to only one server and the cluster software monitors for failure and swaps the disk, IP address and possibly other resources to the secondary node. This requires a hardware cluster monitor such as PowerHA to manage the cluster.
The Multi-Instance QMgr is implemented entirely within WebSphere MQ and needs no other software. It works by having two running instances of the QMgr pointing to the same NFS 4 shared disk mount. Both instances compete for locks on the files. The first one to acquire a lock becomes the active QMgr. Because there is no hardware cluster monitor to perform IP address takeover this type of cluster will have multiple IP addresses. Any modern version of WMQ allows for this using multi-instance CONNAME where you can supply a comma-separated list of IP or DNS names. Client applications that previously used Client Channel Definition Tables (CCDT) to manage failover across multiple QMgrs will continue to work and CCDT continues to be supported in current versions of WMQ.
Please see the Infocenter topic Using WebSphere MQ with high availability configurations for details of hardware cluster and MIQM support.
Client Channel Definition Table files are discussed in the Infocenter topic Client Channel Definition Table file.