Just want to confirm the correct way the MQ delivers messages to the MQOutput node. Recently came across a situation where i a felt bit confused. Here is the scenario.
I have a local queue on Qmgr,say(A) which receives messages from applications and have a local broker associated with this qmgr(A) with a message flow deployed, which consumes messsages from this queue and drops it to another local(L.B) queue on Queue manager (B).
To successfully deliver the messages to qmgr(B) do i have to
Create a remote queue definition on Qmgr(A) with transmission queue name matching the remote queue manager name, here(B)
MQOutput node value set as, queue->remote queue definition name on (A) and queue manager value as blank
or
to create only the transmission queue that matches with the name of the remote queue manager name, here(B).
MQOutput node value set as queue-> target local queue (L.B) and Queue manager value as (B).
When i follow with the first process, noticed messages reaching the destination and when i follow with the later one, noticed messages sitting up in the local queue itself.
Is there any necessity to always create 'n' number of remote queue definitions when it needs to drop messages to 'n' number of local queues?
Kindly guide me to better understand this. Thanks in advance to each of you.
There is no necessity to create n remote queue definitions, MQ is happy to accept output marked as destined for "Queue Name" on "Queue Manager Name".
You say that when using method 2. that your messages are "sitting up in the local queue". There are a few things you must check to solve this problem.
I assume the named queue L.B is defined on QMgr B and not QMgr A?
I assume the local queue the messages are sitting on is a transmission queue?
Have you defined a channel to read messages from the transmission queue they are stuck on?
Have you started the channel which should be moving the messages off the transmission queue to QMgr B?
Related
I have implemented the example from the RabbitMQ website:
RabbitMQ Example
I have expanded it to have an application with a button to send a message.
Now I started two consumer on two different computers.
When I send the message the first message is sent to computer1, then the second message is sent to computer2, the thrid to computer1 and so on.
Why is this, and how can I change the behavior to send each message to each consumer?
Why is this
As noted by Yazan, messages are consumed from a single queue in a round-robin manner. The behavior your are seeing is by design, making it easy to scale up the number of consumers for a given queue.
how can I change the behavior to send each message to each consumer?
To have each consumer receive the same message, you need to create a queue for each consumer and deliver the same message to each queue.
The easiest way to do this is to use a fanout exchange. This will send every message to every queue that is bound to the exchange, completely ignoring the routing key.
If you need more control over the routing, you can use a topic or direct exchange and manage the routing keys.
Whatever type of exchange you choose, though, you will need to have a queue per consumer and have each message routed to each queue.
you can't it's controlled by the server check Round-robin dispatching section
It decides which consumer turn is. i'm not sure if there is a set of algorithms you can pick from, but at the end server will control this (i think round robin algorithm is default)
unless you want to use routing keys and exchanges
I would see this more as a design question. Ideally, producers should create the exchanges and the consumers create the queues and each consumer can create its own queue and hook it up to an exchange. This makes sure every consumer gets its message with its private queue.
What youre doing is essentially 'worker queues' model which is used to distribute tasks among worker nodes. Since each task needs to be performed only once, the message is sent to only one node. If you want to send a message to all the nodes, you need a different model called 'pub-sub' where each message is broadcasted to all the subscribers. The following link shows a simple pub-sub tutorial
https://www.rabbitmq.com/tutorials/tutorial-three-python.html
I would like to know if it is possible to point a MQ Sender channel to a F5 Load balancer VIP address, rather than to a concrete MQ Server IP address and have the message delivered to one of two MQ servers in the F5 Cluster resources group. There are two MQ Servers in the F5 cluster.
What I'm trying to do, is determine if I could use this method in lieu of creating a MQ Cluster Gateway Queue manager, on yet more hardware and use the F5 LB feature to deliver the message to a cluster queue.
If I could capitalize on the F5 Load balancing, I'm thinking it would simulate a MQ Cluster Gateway Queue manager.
Would it work? Pitfalls?
You are looking at a couple of issues with this configuration:
With persistent messages the Sender and the corresponding Receiver channels increment a sequence number with each persistent message sent across the channel. If this sequence number does not match then the channel will not start unless it is reset to match on one end or the other (usually the Sender). This means that if the sender channel connects to QMGR1 behind the F5, the Receiver on QMGR1 will increment the sequence number, if the next time the sender channel connects it is routed to QMGR2, the sequence number of the Receiver will be lower than it is on the sender and it will not start.
Even if you were only sending non-persistent messages which do not increment the sequence number, you would still not achieve the same results as you would with having a cluster gateway in front of the two queue managers. Typically with a cluster configuration you would get a round robin of the messages between the two clustered queue managers. With a sender channel it is normally configured to start when a message is put to the associated transmission queue (TRIGGER) and to keep running until messages have not been sent for the length of time specified on the disconnect interval (DISCINT). Because of this you would not see workload balancing of messages between the two queue managers behind the F5. Depending on how you have the F5 configured and how long the disconnect interval is set to you would see a group of messages to go one queue manager and then a group of messages go to the other queue manager. The number of messages in each group would depend on the traffic pattern compared to the sender channel settings.
Note that even if the Sender channel is configured to connect only to one of the two clustered queue managers, if you set the cluster workload use queue (CLWLUSEQ) to the value of ANY for the clustered queue, you can have messages still round robin between the two instances of the queue. This would require that you have the remote queue (QREMOTE) on the Sender channel queue manager specify a Remote Queue Manager Alias (RQMA) as the remote queue manager name (RQMNAME) value. The RQMA would then allow the messages to resolve to any instance of the clustered queue including the local instance. Examples of objects are below for the Sender queue manager (SQMGR) and Receiver (first clustered) queue manager (CQMGR1) and the second clustered queue manager (CQMGR2):
SQMGR:
DEFINE QREMOTE(TO.CQLOCAL) RNAME(CQLOCAL) RQMNAME(CLUSTER.RQMA) XMITQ(CQMGR1)
DEFINE QLOCAL(CQMGR1) USAGE(XMITQ) INITQ(SYSTEM.CHANNEL.INITQ) TRIGDATA(SQMGR.CQMGR1) TRIGGER .....
DEFINE CHL(SQMGR.CQMGR1) CHLTYPE(SDR) XMITQ(CQMGR1) CONNAME(10.20.30.40) .....
CQMGR1:
DEFINE CHL(SQMGR.CQMGR1) CHLTYPE(RCVR) MCAUSER('notmqm') .....
DEFINE QREMOTE(CLUSTER.RQMA) RNAME('') RQMNAME('') XMITQ('')
DEFINE QLOCAL(CQLOCAL) CLUSTER('YOURCLUSTER') CLWLUSEQ(ANY)
CQMGR2:
DEFINE QLOCAL(CQLOCAL) CLUSTER('YOURCLUSTER') CLWLUSEQ(ANY)
I've come across a curious detail in the legacy integration solution based on WebSphere MQ 7.0.1.3 and WebSphere Message Broker 7.0.0.7. There are 2 message flows:
The 1st flow is a case of MQ Request-Reply pattern. After MQPut it has a MQGet node that gets the message by correlation ID from queue "MQ_BIS_IN".
The 2nd flow is a kind of a one-way router that starts with a MQInput node (without any filters) that listens on the queue "MQ_GW_IN".
Interestingly, "MQ_BIS_IN" is an alias for "MQ_GW_IN" queue. My first thought was that the 2 flows would interfere in a bad way, basically the "omnivorous" MQInput would ruin the Request-Reply thing. But they seem to somehow get along.
I am going to reproduce this configuration in a test environment to determine if their behaviour is stable under load. Nevertheless, does anybody know if there are some rules of precedence between concurrent read operation from the same queue? Does it matter that there's an alias to the queue?
Both the MQInput and the MQGet node can be configured to look for particular msgId's or correlation Id's only, or to pick up the items on the queue in a determined order, or only pick up complete groups of messages - so there doesn't need to be a conflict here.
When a local queue manager receives the following message in it's AMQ error log:
09/13/12 08:00:19 - Process(3017.20) User(mqm) Program(amqrmppa_nd)
AMQ9544: Messages not put to destination queue.
EXPLANATION: During the processing of channel 'TO_QM_QD2T1_C1' one or
more messages could not be put to the destination queue and attempts
were made to put them to a dead-letter queue. The location of the
queue is 2, where 1 is the local dead-letter queue and 2 is the remote
dead-letter queue.
... what is the mechanism by which MQ exchanges such information? Is there a built in facility within the channel program API itself, or is the info exchanges as discrete messages placed on the SYSTEM.CLUSTER.COMMAND.QUEUE (in the case of a cluster)? Given that this could occur in a remote queue definition situation, with only simple sender/receiver channel pairs, and no corresponding COMMAND QUEUE necessarily, I could imagine that it would be handled via the channel process communications... just wondering...
The channel agents have a bi-directional communication between them, even though messages flow in only one direction. When a message fails to find the destination at the remote end there are several possibilities for what happens next. The channel will only continue to run if the message can be successfully put somewhere and the first place to try is the remote DLQ. If that fails, the local MCA must either relocate the message or stop the channel. Therefore, the two message channel agents work out between them what happens and what the status of the channel should be.
The peculiar wording of the error message reflects that the different dispositions of the message derive from within the same code path and exception handling and the optimization of WMQ. The MCA knows the message was put to a DLQ at that point and rather than having two different error messages or logic to work out the wording on which DLQ was used, it just drops a number into a template. Thus a single error message and streamlined logic are used for both possibilities.
In MQ explorer, If I open up the properties for a particular queue (X), what property will tell me if the queue is defined as an error handling queue for another queue (Y)? i.e. If Y fails to process the message and the transaction rolls back, it will put the message on X.
Any queue designated as an error or exception queue (backout queue in WebSphere MQ terminology) is an ordinary queue. The BOQNAME on a primary queue points to the backout queue but there is no attribute of the backout queue that points back to the primary queue. This might be a one-to-many relationship because any number of primary queues might point to a single backout queue.
One way to do this in WMQ Explorer would be to make sure that BOQNAME is visible in the display and then sort the queue list on that column. Then look for all instances with your backout queue name in them.