I would truly appreciate some help with developing a simple pub/sub flow using message broker 7.0 and MQ 7.0
My flow is supposed to accept a certain message with no header, filter it based on
the field (process, if the value is "yes")
And then publish the body to all the queues, listed in nodes of the message
<pub>
<header>
<topics>
<topic> Topic1 </topic>
<topic> Topic2 </topic>
</topics>
<properties>
<property>
<publish>yes</publish>
</property>
</properties>
</header>
<body>
<a>
<b>The publication </b>
</a>
</body>
</pub>
This is my flow:
I registered a Topic and a Subscription in MQ, but I'm pretty much lost what am I supposed to do next.
I used RFHUtil for testing point-to-point application,
but have no idea, how to make use of the it while developing publish-subscribe.
questions:
1. Is it correct to use just a simple queue as a publisher ( in MQ input i just set "IN", the queue, i have in MQ)
2. How do I register subscriber/multiple subscribers in this flow? What is the subscription point?
It's just a learning task.
Any help is welcome!
For a normal pub-sub flow you can have something like below:
Set, the queue name of MQInput node to your input queue. Let's name it as "inputQ".
Now the message has been read by MQInput node from the "inputQ" and it has been passed on to the compute node.
In the compute node you need to set the message type to publish and also set the topic name, before you pass it to the Publication node.
You can use below code for the same:
SET OutputRoot.MQRFH2.psc.Command = 'Publish';
SET OutputRoot.MQRFH2.psc.Topic = 'YourTopicString';
"How do I register subscriber/multiple subscribers in this flow? "
I am assuming your problem is "How to publish message with different topics from same flow".
Now suppose you have multiple topics to be published from the same flow. You can't do it all at once. One message can have one topic.
But, you can achieve it as below (Suppose you have 3 topics):
SET OutputRoot.MQRFH2.psc.Command = 'Publish';
SET OutputRoot.MQRFH2.psc.Topic = 'Topic1';
PROPAGATE TO TERMINAL 'out' DELETE NONE;
SET OutputRoot.MQRFH2.psc.Topic = 'Topic2';
PROPAGATE TO TERMINAL 'out' DELETE NONE;
SET OutputRoot.MQRFH2.psc.Topic = 'Topic3';
PROPAGATE TO TERMINAL 'out' DELETE NONE;
RETURN FALSE;
However, if your requirement is publishing single topic but multiple queues should pick it up, then its simpler.
You just need to create subscriptions for all those queues for your topic.
Related
I've been trying my hands on Spring AMQP. And I have a couple of questions:
I'd like to know what is Publisher returns and how is it different from Publisher Confirm. Of my understanding, we have a Publisher Confirm Callback that checks the status of acks. Now I looked at the documentation in Spring AMQP and Rabbit MQ. didn't really find or understand much on this.
And also why is it that if the message is tried to send to a non-existing queue, I don't get any sort of acknowledgement (ack/nack
) nor do I get any errors. Is there a way to setTimeouts for non-acknowledgements?
Short answer from the link https://www.rabbitmq.com/confirms.html :
"For unroutable messages, the broker will issue a confirm once the exchange verifies a message won't route to any queue (returns an empty list of queues). If the message is also published as mandatory, the basic.return is sent to the client before basic.ack."
In Spring AMQP if you set 'spring.rabbitmq.publisherReturns' to true this will mean messages will be 'mandatory' (unless you set mandatory to false) because of the following code:
private boolean determineMandatoryFlag() {
Boolean mandatory = this.properties.getTemplate().getMandatory();
return (mandatory != null ? mandatory : this.properties.isPublisherReturns());
}
I suggest you to read this article. There is a good description of all possible acknowledgments scenarios, including returns for the unrouted messages, like your non-existing queue.
From the Spring AMQP perspective you should bear in mind: https://docs.spring.io/spring-amqp/docs/2.0.3.RELEASE/reference/html/_reference.html#template-confirms
This feature requires a CachingConnectionFactory that has its publisherReturns property set to true.
I have two clusters CLUSMES and CLUSHUBS. Each cluster has two queue managers.
Cluster CLUSMES has QMGRS: QMGR1A and QMGR1B
Cluster CLUSHUBS has QMGRS: QMGR3A and QMGR3B
There is a Gateway QMGR: QMGR2, which forms the overlap and is a partial repository in each MQ cluster.
Request messages are sent out from either QMGR1A/B to either QMGR3A/B via QMGR2 which acts as a cluster Load balance to QMGR3A/B (This works fine) and a reply is expected back to the sending QMGR.
All channel connectivity is in place and fully functional. The problem is how to return the message from where it came from. The replying QMGR connects to QMGR3A/B and issues a put. I will either get a REMOTE_QMGR not found (MQRC 2087) or a MQ Object not found (MQRC 2085) depending on how I have it configured. The message header of the message contains the ReplytoQueue and ReplyToQMgr properly. I would like to have the replying application just issue a put and have it delivered to the proper queue in CLUSMES, but this is proving to be extremely difficult. I have played with Remote QMGR Alias and QAlias on the GateWay Qmgr: QMGR2, but no luck. There is got to be a simple trick to this and there are plenty of examples, but I have not been able to implement one successfully. A clear cut example of what my return path should be would be most helpful. Keep in mind, that the ReplyToQMgris in the MQMD and resolution needs to occur from that. I need resolution to occur at the QMGR2 level, where both clusters are known. Concrete full suggestions appreciated.
MQ Definitions on the QMGR1A/B, where the REPLY is expected:
DEFINE QLOCAL('SERVER.REPLYQ') CLUSTER('CLUSMES')
On QMGR2 ( The Gateway for message hoping)
DEFINE NAMELIST(CLUSTEST) NAMES(CLUSMES,CLUSHUBS)
DEFINE QALIAS(SERVER.ALIAS.REPLYQ) TARGQ(SERVER.REPLYQ) CLUSTER(CLUSTEST) DEFBIND(NOTFIXED)
DEFINE QREMOTE(QMGR1A) RNAME(' ') RQMNAME(QMGR1A) XMITQ('') CLUSTER(CLUSHUBS)
DEFINE QREMOTE(QMGR1B) RNAME(' ') RQMNAME(QMGR1B) XMITQ('') CLUSTER(CLUSHUBS)
On MQMGR3A/B QALIAS(SERVER.ALIAS.REPLYQ) cluster queue. The Gateway QMGR could not resolve the baseQ: mqrc_unknown_alias_base_q 2082
This was the configuration when trying to resolve it using the cluster.
When the request message is sent by the application it would specify ReplyToQMgr of either QMGR1A and QMGR1B and ReplytoQueue with the name of the queue that is present on QMGR1A and QMGR1B, the reply queue need not be clustered.
On gateway queue manager QMGR2 you would define the following objects:
DEFINE QREMOTE(QMGR1A) RNAME('') RQMNAME(QMGR1A) XMITQ('') CLUSTER(CLUSHUBS)
DEFINE QREMOTE(QMGR1B) RNAME('') RQMNAME(QMGR1B) XMITQ('') CLUSTER(CLUSHUBS)
This would allows any queue manager in the cluster CLUSHUBS to route reply messages back to QMGR1A and QMGR1B via gateway queue manager QMGR2.
If you want to limit queues on QMGR1A and QMGR1B that queue managers in the CLUSHUBS cluster can put to you would need to take a different approach. Let me know if that is what you need and I will update my answer with some suggestions.
In my RabbitMQ, I have an topic exchanger called room-topic-exchange and the bindings are like this
When I send a message to an specific queue, using the exchanger, everything works fine. I'm sending as follow:
template.convertAndSend(ROOM_TOPIC_EXCHANGE, roomId, message);
but when I try to send to ALL queues, nothing happens. I'm trying as this
template.convertAndSend(ROOM_TOPIC_EXCHANGE, "room*", message);
I declared the exchanger and the bind as follow
TopicExchange allRooms = new TopicExchange(ROOM_TOPIC_EXCHANGE, false, true);
admin.declareExchange(allRooms);
admin.declareBinding(BindingBuilder.bind(q).to(allRooms).with(roomId));
I can't see what I'm doing wrong. I read the documentantion, and tried with routing key room# too and nothing happened.
The topic exchange doesn't work that way; you bind with wildcards, you don't use a wildcard in the routing key.
A queue bound with room.* will get messages sent to room.123 or room.124.
You can achieve what you want by adding a second binding to each room, say room.splat; then sending to room.splat will go to both queues.
Or, you can add a second fanout exchange. Bind both queues to both exchanges (no routing key needed for the fanout) and send broadcasts to the fanout exchange and directed messages to the topic.
So I have request/response queues that I am putting messages on and reading messages off from.
The problem is that I have multiple local instances that are reading/feeding off the same queues, and what happens sometimes is that one instance can read some other instance's reply message.
So is there a way I can configure my JMS, using spring that actually makes the instances read the messages that are only requested by them and not read other instance's messages.
I have very little knowledge about JMS and related stuff. So if the above question needs more info then I can dig around and provide it.
Thanks
It's easy!
A JMS message have two properties you can use - JMSMessageID and JMSCorrelationID.
A JMSMessageId is supposed to be unique for each message, so you could do something like this:
Let the client send a request, then start to listen for responses where the correlation id = the sent message id. The server side is then responsible for copying the message id of the request to the correlation id of the response. Something like: responseMsg.setJMSCorrelationID(requestMsg.getJMSMessageID());
Example client side code:
Session session = getSession();
Message msg = createRequest();
MessageProducer mp = session.createProducer(session.createQueue("REQUEST.QUEUE"));
mp.send(msg,DeliveryMode.NON_PERSISTENT,0,TIMEOUT);
// If session is transactional - commit now.
String msgID = msg.getJMSMessageID();
MessageConsumer mc = session.createConsumer(session.createQueue("REPLY.QUEUE"),
"JMSCorrelationID='" + msgId + "'");
Message response = mc.receive(TIMEOUT);
A more performant solution would be to use dedicated reply queues per destination. Simply set message.setJMSReplyTo(session.createQueue("REPLY.QUEUE."+getInstanceId())); and make sure the server side sends response to requestMsg.getJMSReplyTo() and not to a hard coded value.
In TIBCO EMS, which I am familiar with, there is a feature called "destination bridges".
Queues and Topic can be bridged (linked) so that the 2nd destination can become client of the the first. (Queue to queue, Topic to queue, Queue to topic, topic to topic)
For instance, a topic can be bridged to a queue, which in essence will become a durable subscriber of the messages submitted to the topic. Clients can subscribe to the topic OR read from the queue. This example is a way to load balance the reading of a pub/sub for multiple clients (readers of the queue).
This "bridge" feature can also involve message selectors and destination name wilcards.
So, a QUEUE X can be the client of TOPIC.* with condition CUST_ID(a JMS attribute)>30.
In that case, all message submitted to TOPIC.A OR TOPIC.B fitting the criteria would end up in QUEUE X. All this does not involve anything but simple EMS configuration.
I don't know enough about Websphere MQ, and I need a similar behavior. Will I have to develop a processing program outside of MQ, or can the feature within the product sufice ?
Note : I have already gone through MQ documentation and found about the "Alias queues" feature. Since the feature should really be called "Shortcut queue" and does not involve 2 destinations... I don't think it could help me...
Thanks!
Edit : For reference, the command (DEF SUB) enabling this in MQ is documented here
Edit 2 : The selected answer cover the "Topic -> Queue" pattern from TIBCO EMS "Destination bridge" featuire. Please note that the "Q->Q", T->T and Q->T" patterns are not covered here.
Easy! Define your queue to receive the subscription and then define a durable administrative subscription.
DEF QL(MY.SUSCRIBER.QUEUE)
DEF SUB('MY.SUBSCRIPTION') +
TOPICSTR('SOME/TOPIC/#') +
DEST('MY.SUSCRIBER.QUEUE') +
SELECTOR('JMSType = 'car' AND color = 'blue' AND weight > 2500') +
REPLACE
The Infocenter has a section on Selector Syntax and a page for the DEFINE SUB command.