How would I set up MQ so that every message received is immediately written to file system?
I have the "redbooks", but at least need someone at least point me to a chapter or heading in the book to figure it out.
We are a .NET shop. I have written C# via API to read the queue, and we currently use BizTalk MQ adapter. Our ultimate goal is to write same message to multiple directories in file system to "clone" the feed for our various test environments (DEV, STAGE, TRAINING, etc..). The problem with BizTalk is that when we consume the message, we map it at the same time to a new message, so the message is already changed, and we want the original raw message to be cloned, not the morphed one. Our vendors don't offer multiple copies of the feed, for example, they offer DEV and PROD, but we have 4 systems internally.
I suppose I could do a C# Windows Service to do it, but I would rather use built-in features of MQ if possible.
There is no configuration required. If the message is persistent, WMQ writes it to disk. However, I don't think that's going to help you because they are not written as discrete messages. There's no disk file to copy and replication only works if the replicated QMgr is identical to the primary and is offline during the replication.
There are a number of solutions to this problem but as of WMQ V7, the easiest one is to use the built-in Pub/Sub functionality. This assumes that the messages are arriving over a QMgr-to-QMgr channel and landing on a queue where you then consume them.
In that case, it is possible to delete the queue and create an alias of the same name over a topic. You then create a new queue and define an administrative subscription that delivers messages on the topic into the new queue. Your app consumes from the new queue.
When you need to send a feed to another QMgr or application, define a new subscription and point it at the new destination queue. Since this is Pub/Sub, MQ will replicate the original message as many times as there are subscriptions and the first application and its messages are not affected. If the destination you need to send to isn't accessible over MQ channels (perhaps DEV and QA are not connected, for example), you can deliver the messages to the new queue, use QLoad from SupportPac MO03 to write them to a file and then use another instance of QLoad to load them onto a different QMgr. If you wanted to move them in real time, you could set up the Q program from SupportPac MA01 to move them direct from the new subscription queue on QMgr1 to the destination queue on QMgr2. And you can replicate across as many systems as you need.
The SupportPacs main page is here.
If all you are using is the Redbooks, you might want to have a look at the Infocenters. Be sure to use the Infocenter matching the version of WMQ you are using.
WMQ V7.0 Infocenter
WMQ V7.1 Infocenter
WMQ V7.5 Infocenter
Related
Question:
Is it possible to copy MQ Messages from one Queue Manager/Queue to a different Queue Manager/Queue?
Scenario:
I have a "PROD" Queue Manager and when it receives a Message on it's Queues I would like to "copy" the Message to a queue on a "TEST" Queue Manager.
Requirements
The original message must be left on the PROD queue to be processed as normal.
This must be an automated process (lots of messages during a day). I could not intervene on a Message by Message basis.
If at all possible I would like this to be implemented by some native MQ functionality rather than an ad hoc program/script.
The copying must be as near to real time as possible
Must work with MQ version is 7.0.2.1(!). This cannot be changed.
Must run on Red Hat Enterprise Linux Server release 5.11 (Tikanga). Again, can't be changed.
I'm no MQ expert so use small words please
Thanks in advance
The only problem with the technote pointed out by gouda is that MQ will modify/changed the MsgId and CorrelId of each message replicated.
If the MsgId and/or CorrelId fields are important then the only other option is an MQ API Exit that replicates the message. You may need a commercial product like MQ Message Replication.
The next question is how are you going to move the message from a PROD queue to a TEST queue? You definitely do NOT want to create channels between a PROD queue manager and a TEST queue manager.
There are lots of tools that can off offload PROD messages to a file then you can move the file to your TEST environment and then load the messages into a TEST queue. Here is a list of MQ tools that can do it. The 2 tools you should try out are: MQ Batch Toolkit and QLoad.
Personally, I would create a schedule task (CRONTAB) to be run every night at midnight to off the messages and I would make the filename include the date and time. The last steps of the script would be to zip/compress the file and delete the original file (because the data you offload could be massive).
Hence, any time you want a particular day's PROD messages, just copy the file to your TEST server and unzip/uncompress it and load it into the queue.
all you need is mqadmin staff and this technote
I have 2 queues say Q1 and Q2. When I insert messages using MQPUT into Q1, is there any way that this message be replicated into Q2?
Does WMQ have support for queue copy?
Note : The queues reside on different queue managers.
Thanks,
Hudson
If you have multiple consumers(Queues), which needs the same message, then Publish-Subscribe is made exactly for this purpose.
You can refer here for details about pub-sub.
If you are using Websphere MQ 7.XX, then you can use an Alias queue to publish the message.
You can follow below steps:
Create a new Topic(Lets say "MyTopic" is your topic).
Create an Alias queue(Lets call it as "MyPublisher"), with Base Type as Topic and give Base Object as "MyTopic"(Name of your topic).
Now say Q1, Q2, Q3 have to get the message. Create subscriptions for these queues for Topic "MyTopic".
Now, the messages posted in queue "MyPublisher", will reach to all queues subscribing to the topic "MyTopic"(Q1,Q2,Q3).
Even remote queues can be given in destination while creating subscriptions. So, different queue managers will not be a problem as far as you have connectivity between them.
If you don't want to bother with all of the Pub/Sub stuff, then have a look at the free open source project Message Multiplexer (MMX) at http://www.capitalware.biz/mmx_overview.html
Its purpose is to read messages from a source queue and put the exact same message to 'n' target queues (up to 99).
The download contains builds for AIX, HP-UX, Linux, IBM i (OS/400), Solaris, Windows and z/OS (mainframe).
MMX is a very simple and straight-forward program to use.
I am wondering if MQ can be used as a state cache for monitoring? And is this a good idea or not?
In theory you can have many sources (monitoring agents) that detect problem states and distribute them to subscribers via an MQ system such as RabbitMQ. But has anyone heard of using MQ systems to cache the state, so when clients initialize, they read from the state queue before subscribing to new state messages? Is that a bad way to use MQ?
So to recap, a monitor would read current state from a state queue then setup a subscription queue to receive any new updates. And the state queue would be maintained by removing any alerts that are no longer valid by the monitoring agents that put the alert there to begin with.
Advantage would be decentralized notification and theoretically very salable by adding more mq systems to relay events.
I have a use case for Rabbit MQ that holds the last valid status of a system. When a new client of that system connects it receives the current status.
It is so simple to do!
You must use the Last Value Cache custom exchange https://github.com/simonmacmullen/rabbitmq-lvc-plugin
Once installed you send all your status messages to that exchange. Each client that needs the status information will create a queue that will have the most recent status delivered to that queue on instantiation. After that it will continue to receive status updates.
IBM MQ FTE uses such way for storing logs.
I think it is good idea, if you can prevent destination queue from overflow, because IBM MQ for example remove overdue messages only during GET call.
I have a .NET application that needs to monitor a queue in WebSphere MQ. I need to react to each message without impacting the current process. The client application can't explicity send me the same message.
Can I read a message without removing it from the queue? Can I be notified for each message? Can I configure the MQ to duplicate the current queue?
Is there another solution?
If you are using WMQ v7 then you can do this without any impact to the existing applications other than to change the queue name for one of them.
Currently the message producer and consumer use the same queue. In v7 of WMQ you can create an alias over a topic so that the message producer thinks it's a queue. Then you can create two administrative, durable subscriptions such that one points to the existing input queue and another points to a queue dedicated to your new application.
Of course you are already using v7 since v6 goes out of service next year, right? You can upgrade the QMgr to v7 which enables this behavior while still using v6 client code for the apps.
If you are using WMQ v6 then the MirrorQ program might work for you.
You could change from using a queue to a durable topic and have both your reader and your browser subscribe to it.
You could also create a distribution list on your queue manager. A distribution list is used to send a copy of the same message to multiple queues. You would then have a processing queue and a browsing/monitoring queue.
I'm queuing messages to a WebSphere MQ queue (NB: A point-to-point queue -- not a topic) using a stored procedure in my Oracle database. Is there a way to publish each message to multiple queues with a single call? What I would like is to find a solution that would incur zero additional latency on my database compared to sending the message to a single queue.
Solutions that involve changing my WebSphere MQ settings are certainly welcome! What I had in mind was somehow creating a "clone" queue that got all the same messages as the original one, but I've been unable to locate anything like this in the documentation.
Thanks,
Jeff
With WMQ v7 you can do this easily and with administration only. You would create a topic object and then an alias over the topic. The Oracle app writes to the alias and does not know that it is actually publishing.
Meanwhile, you make two administrative subscriptions on the topic so that publications are delivered to your two destination queues. The apps consuming them have no idea that the messages were published as opposed to delivered through point-to-point queues.
If you are not familiar with the new WMQ v7 features, take a look at the Infocenter. In particular, the "What's New in V7" section and the sections on Pub/Sub.
You can accomplish this using "Distribution Lists" in WebsphereMQ. These have to be configured on your queue manager.
Take a look at the Wesbphere MQ Application Programming Guide for more info.