Simple queue distribution scenario with pub/sub WMQ - ibm-mq

Using WebSphere MQ I want to setup up a topic that uses queues, so that when an application or inbound cluster connection attempts to put a message to a “queue”, it
actually uses a topic, and publishes it to 2 subscriptions, which are
themselves 2 separate queues. In essence, I want to go from an inbound
queue by name, but map it to 2 separate queues, like
AF_TO_DAAS is the inbound topic/which today is an actual
cluster queue alias
=>Goes to AF_TO_APP1 and
=>Goes to AF_TO_APP2
Sort of like if these were queues on a distribution list I suppose.
Those two things are local queues.
I’m getting lost in the /topic/node business mapping it to subscriptions
and model queues and what not …

WebSphere MQ allows an alias to point to a queue or a topic. It also provides a means to create a durable subscription administratively. To accomplish this setup, you connect the dots by replacing the existing alias with one that points to a topic. Then use two administrative subscriptions to route publications to the two (or more) queues.
* First, define the topic
DEFINE TOPIC('AF_TO_DAAS.TOPIC') +
TOPICSTR('AF_TO_DAAS') +
REPLACE
* Now, create an alias over the topic.
* Sending apps think this is a queue.
DEFINE QALIAS('AF_TO_DAAS') +
TARGET('AF_TO_DAAS.TOPIC') +
TARGTYPE(TOPIC) +
REPLACE
* Queues for the two recvr apps
DEFINE QLOCAL('AF_TO_APP1') +
REPLACE
DEFINE QLOCAL('AF_TO_APP2') +
REPLACE
* Now set up adminsitrative subs to route
* messages to the two app queues.
DEFINE SUB('AF_TO_DAAS.SUB') +
TOPICSTR('') +
TOPICOBJ('AF_TO_DAAS.TOPIC') +
DEST('AF_TO_APP1') +
PSPROP(NONE) +
REPLACE
DEFINE SUB('AF_TO_APP2.SUB') +
TOPICSTR('') +
TOPICOBJ('AF_TO_DAAS.TOPIC') +
DEST('AF_TO_APP2') +
PSPROP(NONE) +
REPLACE
Because this is a publication, the messages will contain a property Top which contains the topic string. The PSPROP(NONE) in the subscription entries causes this to be suppressed so the messages look like the original publication.
Note also that the MQMD.MsgID on the publications is different than it is on the original message.

Related

JMSListener escapes '.' for durable subscriptions

With Spring JMS 4.3.19 as well as with 5.2.5 I am trying to set up a JMSListener for durable subscriptions:
#JmsListener(destination = "test", subscription = "Consumer.Test", connectionFactory = "factory")
public void receiveFromDurableSub(String message) {
System.out.println("receiveFromTest: " + message);
}
But it ends up in Consumer\\.Test. For addresses it works somehow.
How can I avoid those backslashes?
JMS topic subscriptions are implemented on ActiveMQ Artemis as queues. Each subscription gets it own queue.
The name of the queue depends on how the subscription is configured. The broker will take the JMS client ID (if one is configured), the JMS subscription name (if one is configured), and the JMS subscription's durability to construct the name of the underlying subscription queue. The broker uses the . character to concatenate all these piece of information together to compose the final name. See the related source code for more details on that bit.
In order to be able to decompose this name into its constituent parts later any uses of the . character in the client ID or subscription name have to be escaped.
Since you're using Consumer.Test for the JMS subscription's name this will ultimately be escaped to Consumer\\.Test for use in the underlying subscription queue's name. The broker's use of the . character is not configurable in this case.
If you don't want your subscription's name to be escaped then I recommend you don't use a name with a . character in it (e.g. Consumer-Test).

Overlapping MQ Cluster and reply message delivery with IBM MQ

I have two clusters CLUSMES and CLUSHUBS. Each cluster has two queue managers.
Cluster CLUSMES has QMGRS: QMGR1A and QMGR1B
Cluster CLUSHUBS has QMGRS: QMGR3A and QMGR3B
There is a Gateway QMGR: QMGR2, which forms the overlap and is a partial repository in each MQ cluster.
Request messages are sent out from either QMGR1A/B to either QMGR3A/B via QMGR2 which acts as a cluster Load balance to QMGR3A/B (This works fine) and a reply is expected back to the sending QMGR.
All channel connectivity is in place and fully functional. The problem is how to return the message from where it came from. The replying QMGR connects to QMGR3A/B and issues a put. I will either get a REMOTE_QMGR not found (MQRC 2087) or a MQ Object not found (MQRC 2085) depending on how I have it configured. The message header of the message contains the ReplytoQueue and ReplyToQMgr properly. I would like to have the replying application just issue a put and have it delivered to the proper queue in CLUSMES, but this is proving to be extremely difficult. I have played with Remote QMGR Alias and QAlias on the GateWay Qmgr: QMGR2, but no luck. There is got to be a simple trick to this and there are plenty of examples, but I have not been able to implement one successfully. A clear cut example of what my return path should be would be most helpful. Keep in mind, that the ReplyToQMgris in the MQMD and resolution needs to occur from that. I need resolution to occur at the QMGR2 level, where both clusters are known. Concrete full suggestions appreciated.
MQ Definitions on the QMGR1A/B, where the REPLY is expected:
DEFINE QLOCAL('SERVER.REPLYQ') CLUSTER('CLUSMES')
On QMGR2 ( The Gateway for message hoping)
DEFINE NAMELIST(CLUSTEST) NAMES(CLUSMES,CLUSHUBS)
DEFINE QALIAS(SERVER.ALIAS.REPLYQ) TARGQ(SERVER.REPLYQ) CLUSTER(CLUSTEST) DEFBIND(NOTFIXED)
DEFINE QREMOTE(QMGR1A) RNAME(' ') RQMNAME(QMGR1A) XMITQ('') CLUSTER(CLUSHUBS)
DEFINE QREMOTE(QMGR1B) RNAME(' ') RQMNAME(QMGR1B) XMITQ('') CLUSTER(CLUSHUBS)
On MQMGR3A/B QALIAS(SERVER.ALIAS.REPLYQ) cluster queue. The Gateway QMGR could not resolve the baseQ: mqrc_unknown_alias_base_q 2082
This was the configuration when trying to resolve it using the cluster.
When the request message is sent by the application it would specify ReplyToQMgr of either QMGR1A and QMGR1B and ReplytoQueue with the name of the queue that is present on QMGR1A and QMGR1B, the reply queue need not be clustered.
On gateway queue manager QMGR2 you would define the following objects:
DEFINE QREMOTE(QMGR1A) RNAME('') RQMNAME(QMGR1A) XMITQ('') CLUSTER(CLUSHUBS)
DEFINE QREMOTE(QMGR1B) RNAME('') RQMNAME(QMGR1B) XMITQ('') CLUSTER(CLUSHUBS)
This would allows any queue manager in the cluster CLUSHUBS to route reply messages back to QMGR1A and QMGR1B via gateway queue manager QMGR2.
If you want to limit queues on QMGR1A and QMGR1B that queue managers in the CLUSHUBS cluster can put to you would need to take a different approach. Let me know if that is what you need and I will update my answer with some suggestions.

Too many consumers in Rabbit MQ (Bunny)

I'm sending lots of data to my app through JMeter.
My subscribe block and the publisher look like this:
BunnyStarter.start_bunny_components
cons = BunnyStarter.queue.subscribe do |delivery_info, metadata, payload|
method_calling ( payload )
cons.cancel
end
BunnyStarter.exchange.publish(body.to_json, routing_key: BunnyStarter.queue.name)
And my BunnyStarter class:
def self.start_bunny_components
if ##conn.nil?
##conn = Bunny.new
##conn.start
##ch = ##conn.create_channel
##queue = ##ch.queue("dump_probe_queue")
##exchange = ##ch.default_exchange
end
end
The problem is, although I call consumer.cancel after method_calling, in my Rabbit MQ admin I still see that I get like one thousand consumers created in about 6 minutes.
Is that because of the rate and the amount of data I'm sending?
How can I improve this?
I have seen this issue before. The reason its creating 1000 of consumers is because you are creating channel per connection. Eventually your consumer will shut down after a while because of this.
The number of consumers getting created are not because of the data but its because in the consumer its creating one connection per subscription.
Solution:
Instead of creating multiple channels, create only one channel and use number of connections using the same channel.
I mean one instance of Connection and multiple instances of Model so you can share the same connection for multiple model.

Testing remote WebSphereMQ queue managers

I'd like set up a battery of tests for testing new installations of QMs. Most of them will be in a "remote" setup. I'd like to be able to send a message to the new QM and receive a response. Is there any way to make a "loop" just with normal configuration (i.e. without using exits) ?
My former employer was a bank and we had many connections to clearing
houses and other vendors. On many of these I was able to have the business partner set up loopback queues on their end. These consisted of a QRemote on my QMgr that pointed to a QRemote on the other side, which pointed to a QLocal on the originating QMgr on my side. When a message is PUT to the QRemote it flowed to the other side, landed on the QRemote there, then was directed back to the QLocal.
If the message didn't make it back, we would then start our inbound channel (which was defined as a RQSTR on our side). If this caused the channel to start and the message arrived then we knew the channel initiator on the other side was down but at least we could keep the channel running until it was fixed. But if this didn't work we knew there was some problem on their side other than channel triggering.
And, yes, this was production. We figured that non-transactional instrumentation messages are valid in Production environments. As far as we were concerned, this was the same as a ping or cluster control message. Of course, we had a web screen and other instrumentation to kick them off so usage of the utility was controlled and were recorded.
There are similar methods in which QMgr aliasing is used and this can greatly cut down on the number of loopback queues needed in a large network. Do NOT be tempted to do this! When we use QRemotes that specify a fully-qualified destination, all they can do is send and receive loopback messages on the prescribed queue. But if a QRemote is used without an RQNAME then it can be used to send messages to any queue on the adjacent QMgr.
Let's say you have QMA and QMB:
runmqsc QMA
DEFINE QLOCAL(QMB.LOOPBACK) REPLACE
DEFINE QREMOTE(QMB.LOOPBACK.RMT) +
XMITQ(QMB) +
RQMNAME(QMB) +
RNAME(QMB.QMA.LOOPBACK.RMT) +
REPLACE
END
runmqsc QMB
DEFINE QREMOTE(QMB.QMA.LOOPBACK.RMT) +
XMITQ(QMA) +
RQMNAME(QMA) +
RNAME(QMB.LOOPBACK) +
REPLACE
Now you can go to the server hosting QMA and do:
echo Hello World | amqsput QMB.LOOPBACK.RMT QMA
Note that the name of the loopback reflector on QMB has both QMgr names in it. This is because you probably want to set up lopback from both sides and cannot use QMB.LOOPBACK.RMT both as a reflector and to initiate loopbacks. In that case the object inventory looks more like this:
runmqsc QMA
DEFINE QLOCAL(QMB.LOOPBACK) REPLACE
DEFINE QREMOTE(QMB.LOOPBACK.RMT) +
XMITQ(QMB) +
RQMNAME(QMB) +
RNAME(QMB.QMA.LOOPBACK.RMT) +
REPLACE
DEFINE QREMOTE(QMA.QMB.LOOPBACK.RMT) +
XMITQ(QMB) +
RQMNAME(QMB) +
RNAME(QMA.LOOPBACK) +
REPLACE
END
runmqsc QMB
DEFINE QREMOTE(QMB.QMA.LOOPBACK.RMT) +
XMITQ(QMA) +
RQMNAME(QMA) +
RNAME(QMB.LOOPBACK) +
REPLACE
DEFINE QLOCAL(QMA.LOOPBACK) REPLACE
DEFINE QREMOTE(QMA.LOOPBACK.RMT) +
XMITQ(QMA) +
RQMNAME(QMA) +
RNAME(QMA.QMB.LOOPBACK.RMT) +
REPLACE
END
Note that all of the objects sort out based on the name of the remote QMgr. Some people prefer names like LOOPBACK.QMB.RMT to make all the loopback queues cluster together in a list of objects or a backup.
All of this is a great target for automation since the names of the objects can all be derived from the names of the QMgrs.
I can think of a couple of ways to do this. It rather depends on whether you want to include the MQPUT and MQGET on the remote system target queue.
Define the target queue as a QREMOTE pointing back to the originating system again with a real QLOCAL as the final destination back on the original system.
Use something like SupportPac MA01 (Q program) to echo the message back when it lands on a real QLOCAL queue on the remote system. The -e option let's you echo the message back to the ReplyToQ and ReplyToQMgr fields.

What is the preferred way to do a "Destination Bridge" featuring Message selectors in Websphere MQ?

In TIBCO EMS, which I am familiar with, there is a feature called "destination bridges".
Queues and Topic can be bridged (linked) so that the 2nd destination can become client of the the first. (Queue to queue, Topic to queue, Queue to topic, topic to topic)
For instance, a topic can be bridged to a queue, which in essence will become a durable subscriber of the messages submitted to the topic. Clients can subscribe to the topic OR read from the queue. This example is a way to load balance the reading of a pub/sub for multiple clients (readers of the queue).
This "bridge" feature can also involve message selectors and destination name wilcards.
So, a QUEUE X can be the client of TOPIC.* with condition CUST_ID(a JMS attribute)>30.
In that case, all message submitted to TOPIC.A OR TOPIC.B fitting the criteria would end up in QUEUE X. All this does not involve anything but simple EMS configuration.
I don't know enough about Websphere MQ, and I need a similar behavior. Will I have to develop a processing program outside of MQ, or can the feature within the product sufice ?
Note : I have already gone through MQ documentation and found about the "Alias queues" feature. Since the feature should really be called "Shortcut queue" and does not involve 2 destinations... I don't think it could help me...
Thanks!
Edit : For reference, the command (DEF SUB) enabling this in MQ is documented here
Edit 2 : The selected answer cover the "Topic -> Queue" pattern from TIBCO EMS "Destination bridge" featuire. Please note that the "Q->Q", T->T and Q->T" patterns are not covered here.
Easy! Define your queue to receive the subscription and then define a durable administrative subscription.
DEF QL(MY.SUSCRIBER.QUEUE)
DEF SUB('MY.SUBSCRIPTION') +
TOPICSTR('SOME/TOPIC/#') +
DEST('MY.SUSCRIBER.QUEUE') +
SELECTOR('JMSType = 'car' AND color = 'blue' AND weight > 2500') +
REPLACE
The Infocenter has a section on Selector Syntax and a page for the DEFINE SUB command.

Resources