I would like to know if it is possible to point a MQ Sender channel to a F5 Load balancer VIP address, rather than to a concrete MQ Server IP address and have the message delivered to one of two MQ servers in the F5 Cluster resources group. There are two MQ Servers in the F5 cluster.
What I'm trying to do, is determine if I could use this method in lieu of creating a MQ Cluster Gateway Queue manager, on yet more hardware and use the F5 LB feature to deliver the message to a cluster queue.
If I could capitalize on the F5 Load balancing, I'm thinking it would simulate a MQ Cluster Gateway Queue manager.
Would it work? Pitfalls?
You are looking at a couple of issues with this configuration:
With persistent messages the Sender and the corresponding Receiver channels increment a sequence number with each persistent message sent across the channel. If this sequence number does not match then the channel will not start unless it is reset to match on one end or the other (usually the Sender). This means that if the sender channel connects to QMGR1 behind the F5, the Receiver on QMGR1 will increment the sequence number, if the next time the sender channel connects it is routed to QMGR2, the sequence number of the Receiver will be lower than it is on the sender and it will not start.
Even if you were only sending non-persistent messages which do not increment the sequence number, you would still not achieve the same results as you would with having a cluster gateway in front of the two queue managers. Typically with a cluster configuration you would get a round robin of the messages between the two clustered queue managers. With a sender channel it is normally configured to start when a message is put to the associated transmission queue (TRIGGER) and to keep running until messages have not been sent for the length of time specified on the disconnect interval (DISCINT). Because of this you would not see workload balancing of messages between the two queue managers behind the F5. Depending on how you have the F5 configured and how long the disconnect interval is set to you would see a group of messages to go one queue manager and then a group of messages go to the other queue manager. The number of messages in each group would depend on the traffic pattern compared to the sender channel settings.
Note that even if the Sender channel is configured to connect only to one of the two clustered queue managers, if you set the cluster workload use queue (CLWLUSEQ) to the value of ANY for the clustered queue, you can have messages still round robin between the two instances of the queue. This would require that you have the remote queue (QREMOTE) on the Sender channel queue manager specify a Remote Queue Manager Alias (RQMA) as the remote queue manager name (RQMNAME) value. The RQMA would then allow the messages to resolve to any instance of the clustered queue including the local instance. Examples of objects are below for the Sender queue manager (SQMGR) and Receiver (first clustered) queue manager (CQMGR1) and the second clustered queue manager (CQMGR2):
SQMGR:
DEFINE QREMOTE(TO.CQLOCAL) RNAME(CQLOCAL) RQMNAME(CLUSTER.RQMA) XMITQ(CQMGR1)
DEFINE QLOCAL(CQMGR1) USAGE(XMITQ) INITQ(SYSTEM.CHANNEL.INITQ) TRIGDATA(SQMGR.CQMGR1) TRIGGER .....
DEFINE CHL(SQMGR.CQMGR1) CHLTYPE(SDR) XMITQ(CQMGR1) CONNAME(10.20.30.40) .....
CQMGR1:
DEFINE CHL(SQMGR.CQMGR1) CHLTYPE(RCVR) MCAUSER('notmqm') .....
DEFINE QREMOTE(CLUSTER.RQMA) RNAME('') RQMNAME('') XMITQ('')
DEFINE QLOCAL(CQLOCAL) CLUSTER('YOURCLUSTER') CLWLUSEQ(ANY)
CQMGR2:
DEFINE QLOCAL(CQLOCAL) CLUSTER('YOURCLUSTER') CLWLUSEQ(ANY)
Related
I have implemented the example from the RabbitMQ website:
RabbitMQ Example
I have expanded it to have an application with a button to send a message.
Now I started two consumer on two different computers.
When I send the message the first message is sent to computer1, then the second message is sent to computer2, the thrid to computer1 and so on.
Why is this, and how can I change the behavior to send each message to each consumer?
Why is this
As noted by Yazan, messages are consumed from a single queue in a round-robin manner. The behavior your are seeing is by design, making it easy to scale up the number of consumers for a given queue.
how can I change the behavior to send each message to each consumer?
To have each consumer receive the same message, you need to create a queue for each consumer and deliver the same message to each queue.
The easiest way to do this is to use a fanout exchange. This will send every message to every queue that is bound to the exchange, completely ignoring the routing key.
If you need more control over the routing, you can use a topic or direct exchange and manage the routing keys.
Whatever type of exchange you choose, though, you will need to have a queue per consumer and have each message routed to each queue.
you can't it's controlled by the server check Round-robin dispatching section
It decides which consumer turn is. i'm not sure if there is a set of algorithms you can pick from, but at the end server will control this (i think round robin algorithm is default)
unless you want to use routing keys and exchanges
I would see this more as a design question. Ideally, producers should create the exchanges and the consumers create the queues and each consumer can create its own queue and hook it up to an exchange. This makes sure every consumer gets its message with its private queue.
What youre doing is essentially 'worker queues' model which is used to distribute tasks among worker nodes. Since each task needs to be performed only once, the message is sent to only one node. If you want to send a message to all the nodes, you need a different model called 'pub-sub' where each message is broadcasted to all the subscribers. The following link shows a simple pub-sub tutorial
https://www.rabbitmq.com/tutorials/tutorial-three-python.html
I am creating NATS go lang Queue Subscriber client as follows,
nc.QueueSubscribe("foo", "my_queue", func(msg *nats.Msg) {
log.Printf("Message :%s", string(msg.Data))
})
So whenever i publish any message to "foo" subject then some time it is receiving and some time not.
e.g let say i sent 10 messages to above "foo" subject then it will receive 2 or 3 max.
My requirement is as follows,
There should be Queue Subscription.
All input events should be processed.
How to implement Queue Subscribe in concurrent mode.
Any help appreciated.
If you start multiple queue subscribers with the same name (in your example my_queue), then a message published on "foo" goes to only one of those queue subscribers.
I am not sure from your statement if you imply that the queue subscriber sometimes misses messages or not. Keep in mind one thing: there is no persistence in NATS (there is in NATS Streaming). So if you publish messages before the subscriber is created, and if there is no other subscriber on that subject, the messages will be lost.
If you were experimenting and starting the queue subscriber from one connection and then in the same application sending messages from another connection, it is possible that the server did not register the queue subscription before it started to receive messages (again, if you were using 2 connections). If that is the case, you would need to flush the connection after creating the subscription and before starting sending: nc.Flush().
Finally, there is nothing special to use queue subscribers in concurrent mode. This is what they are for: load balancing processing of messages on the same subject for subscribers belonging to the same group. The only thing you have to be careful of if you are creating multiple queue subscribers in the same application is either to not share the message handler or if you do, you need to use locking since the message handler would be concurrently invoked if messages arrive fast enough.
Just want to confirm the correct way the MQ delivers messages to the MQOutput node. Recently came across a situation where i a felt bit confused. Here is the scenario.
I have a local queue on Qmgr,say(A) which receives messages from applications and have a local broker associated with this qmgr(A) with a message flow deployed, which consumes messsages from this queue and drops it to another local(L.B) queue on Queue manager (B).
To successfully deliver the messages to qmgr(B) do i have to
Create a remote queue definition on Qmgr(A) with transmission queue name matching the remote queue manager name, here(B)
MQOutput node value set as, queue->remote queue definition name on (A) and queue manager value as blank
or
to create only the transmission queue that matches with the name of the remote queue manager name, here(B).
MQOutput node value set as queue-> target local queue (L.B) and Queue manager value as (B).
When i follow with the first process, noticed messages reaching the destination and when i follow with the later one, noticed messages sitting up in the local queue itself.
Is there any necessity to always create 'n' number of remote queue definitions when it needs to drop messages to 'n' number of local queues?
Kindly guide me to better understand this. Thanks in advance to each of you.
There is no necessity to create n remote queue definitions, MQ is happy to accept output marked as destined for "Queue Name" on "Queue Manager Name".
You say that when using method 2. that your messages are "sitting up in the local queue". There are a few things you must check to solve this problem.
I assume the named queue L.B is defined on QMgr B and not QMgr A?
I assume the local queue the messages are sitting on is a transmission queue?
Have you defined a channel to read messages from the transmission queue they are stuck on?
Have you started the channel which should be moving the messages off the transmission queue to QMgr B?
I have a following scenario:
Producer P produces messages posts all of them to AMQ 'A' (which is local to 'P').
Is it possible for me to route the messages from ActiveMQ A to remote Active MQs B or C?
Basically Iam looking for filters in 'A' side configuration to route these messages.
Thanks in Advance,
Madhav
Gaurav,
What I mean is I have 3 instances of activeMQs at location A, B and C respectively, I have a producer bean 'P' in location A which locally places the messages in AMQ instance #A, locally because I relieve the headache of connection maintenance(lessen) and probability of message loss in 'P' if I were to connect to AMQ instances B or C remotely.
if you want to interconnect the 3 brokers, then just create a network of brokers out of them...then producers and consumers can connect to any broker and messages can flow to the appropriate broker based on demand, etc.
otherwise, if you want more explicit control, then you can use Camel to perform basic (or complex) routing from brokerA queues to broker B/C queues using separate connection factories, etc.
When a local queue manager receives the following message in it's AMQ error log:
09/13/12 08:00:19 - Process(3017.20) User(mqm) Program(amqrmppa_nd)
AMQ9544: Messages not put to destination queue.
EXPLANATION: During the processing of channel 'TO_QM_QD2T1_C1' one or
more messages could not be put to the destination queue and attempts
were made to put them to a dead-letter queue. The location of the
queue is 2, where 1 is the local dead-letter queue and 2 is the remote
dead-letter queue.
... what is the mechanism by which MQ exchanges such information? Is there a built in facility within the channel program API itself, or is the info exchanges as discrete messages placed on the SYSTEM.CLUSTER.COMMAND.QUEUE (in the case of a cluster)? Given that this could occur in a remote queue definition situation, with only simple sender/receiver channel pairs, and no corresponding COMMAND QUEUE necessarily, I could imagine that it would be handled via the channel process communications... just wondering...
The channel agents have a bi-directional communication between them, even though messages flow in only one direction. When a message fails to find the destination at the remote end there are several possibilities for what happens next. The channel will only continue to run if the message can be successfully put somewhere and the first place to try is the remote DLQ. If that fails, the local MCA must either relocate the message or stop the channel. Therefore, the two message channel agents work out between them what happens and what the status of the channel should be.
The peculiar wording of the error message reflects that the different dispositions of the message derive from within the same code path and exception handling and the optimization of WMQ. The MCA knows the message was put to a DLQ at that point and rather than having two different error messages or logic to work out the wording on which DLQ was used, it just drops a number into a template. Thus a single error message and streamlined logic are used for both possibilities.