I have a application where I am using IBM MQ in a Pseudo-Synchronous way. I have request Q and Response Q. This will called through service. Now there is a time out period for my service to show the response.
But the actual response can be pushed to the response Q after the timeout also. Now I want to move such all delayed messages(timed out) to a separate message Q, such that they can be consumed by a different process.
Is this is possible through setting by time to live? Is any approach available in IBM MQ for such movement.
Please refer below image for the scenario.
Don't use time to live because the message will expire and be deleted - unless that is what you want. From your description/picture, it sounds like you want another application to process late responses.
There is nothing in MQ that will move messages older then "x" seconds/minutes to another queue. You will need to write a program to do that. If you write a program, make sure you do the move under a unit of work.
There are several commercial programs that can do it: (1) GUI program: MQ Visual Edit and (2) Command Line (shell): MQ Batch Toolkit.
Related
We have a requirement in where we need to send only one message at a time to a backend process. The call back of this process takes around an hour, only after the call back can we send another request to the process.
I am trying to achieve this by using a manager bpel process that will hold the messages first if there is already something being processed in the backend, and then send it once it realizes that the backend is free. This approach will work, but our architect wants a cleaner solution. He suggested using JMS queues. The idea is for the jms queue to messages to be read by a amanger one at a time, only moving on to the next one once we receive the callback from the backend and we know that the composite and bpel instance is finished. I've been scouring the internet for weeks, but I couldn't find a working jms based solution for my requirement.
I've tried the suggestions for this link but turning on unit of order and acknowledgement properties does nothing.
Try this approach!!
Use a event driven bpel process.
Use a database flag as your next trigger. (flag is TRUE)
jms Adapter receives first message from the queue. Here use a delay in the adapter since you are expecting the bpel to be long running. use below setting.
<binding.jca config="MyServiceInboundQueue_jms.jca">
<property name="minimumDelayBetweenMessages">10000</property>
<property name="singleton">true</property>
</binding.jca>
if flag == TRUE in the db causes the db adapter to proceed with the bpel process,
else skip the bpel.
mark flag==FALSE
call the backend system
callback is received after an hour.
set flag==TRUE
Hi Jonar,
At my company we always use JMS queues for Asynchronous messaging. You could do with a delay timer build in your composite set to 1 hour and 15 minutes for example, and it will work most of the time, but its hella messy. The whole idea is for any asynchronous process to kick off when a message is put upon its queue target (specified by the JMS queue). The JMS adapter in the composite of your project will pick up the message from the queue when it is free to process the queue. The goal for you would be to put the message on the queue and pick it up from it using the adapter. It will know which message to pick up because you specify which queues it listens to in the adapter.
The following blog post by John-Brown Evans eplains the whole process from step one. It might be a bit tedious, but I found it very helpful. Its using SOa Suite 11g instead of the nowadays more commonly used 12c, but its fundamentals remain the same.
Awesome JMS queue tutorial
I hope this works for you!
Cheers,
Jesper
Hoping someone with MQ knowledge will know if the following is possible ...
Background : we have one IBM MQ 7.5 server currently but happy to upgrade if that helps.
Scenario : We have a dedicated channel and queues for a processing application. I'd like a way, on a single MQ Server, to be able to have new messages delivered and for them to queue up in "Messages_IN" during a release then let them through afterwards. I cannot stop the channel that is used as the consuming application will error (lose MQ access) and not work. i.e. sending applications and the processing application currently use the same channel / queues.
Diagram linked below to help explain ...
MQ options?
Is there anyway to have a flow control on a queue that would allow us to control messages not being consumed by an application without stopping the channel? Any new queues / channels / config etc are fine but currently limited to one MQ Server.
Many thanks for anyone who's taken the time to read this and hopefully its technically possible.
Regards
David
Hi the comment of JoshMC is your answer. You ned to set the "GET" propiertie of the QUEUES "MESSAGES_IN" to "DISABLED". With that you prevent any aplication of get message from the QUEUE, whe you end your maintenance you need to set the propertie to "ENABLED" for enabled applicaiton to gets messages from the queue.
Question:
Is it possible to copy MQ Messages from one Queue Manager/Queue to a different Queue Manager/Queue?
Scenario:
I have a "PROD" Queue Manager and when it receives a Message on it's Queues I would like to "copy" the Message to a queue on a "TEST" Queue Manager.
Requirements
The original message must be left on the PROD queue to be processed as normal.
This must be an automated process (lots of messages during a day). I could not intervene on a Message by Message basis.
If at all possible I would like this to be implemented by some native MQ functionality rather than an ad hoc program/script.
The copying must be as near to real time as possible
Must work with MQ version is 7.0.2.1(!). This cannot be changed.
Must run on Red Hat Enterprise Linux Server release 5.11 (Tikanga). Again, can't be changed.
I'm no MQ expert so use small words please
Thanks in advance
The only problem with the technote pointed out by gouda is that MQ will modify/changed the MsgId and CorrelId of each message replicated.
If the MsgId and/or CorrelId fields are important then the only other option is an MQ API Exit that replicates the message. You may need a commercial product like MQ Message Replication.
The next question is how are you going to move the message from a PROD queue to a TEST queue? You definitely do NOT want to create channels between a PROD queue manager and a TEST queue manager.
There are lots of tools that can off offload PROD messages to a file then you can move the file to your TEST environment and then load the messages into a TEST queue. Here is a list of MQ tools that can do it. The 2 tools you should try out are: MQ Batch Toolkit and QLoad.
Personally, I would create a schedule task (CRONTAB) to be run every night at midnight to off the messages and I would make the filename include the date and time. The last steps of the script would be to zip/compress the file and delete the original file (because the data you offload could be massive).
Hence, any time you want a particular day's PROD messages, just copy the file to your TEST server and unzip/uncompress it and load it into the queue.
all you need is mqadmin staff and this technote
How would I set up MQ so that every message received is immediately written to file system?
I have the "redbooks", but at least need someone at least point me to a chapter or heading in the book to figure it out.
We are a .NET shop. I have written C# via API to read the queue, and we currently use BizTalk MQ adapter. Our ultimate goal is to write same message to multiple directories in file system to "clone" the feed for our various test environments (DEV, STAGE, TRAINING, etc..). The problem with BizTalk is that when we consume the message, we map it at the same time to a new message, so the message is already changed, and we want the original raw message to be cloned, not the morphed one. Our vendors don't offer multiple copies of the feed, for example, they offer DEV and PROD, but we have 4 systems internally.
I suppose I could do a C# Windows Service to do it, but I would rather use built-in features of MQ if possible.
There is no configuration required. If the message is persistent, WMQ writes it to disk. However, I don't think that's going to help you because they are not written as discrete messages. There's no disk file to copy and replication only works if the replicated QMgr is identical to the primary and is offline during the replication.
There are a number of solutions to this problem but as of WMQ V7, the easiest one is to use the built-in Pub/Sub functionality. This assumes that the messages are arriving over a QMgr-to-QMgr channel and landing on a queue where you then consume them.
In that case, it is possible to delete the queue and create an alias of the same name over a topic. You then create a new queue and define an administrative subscription that delivers messages on the topic into the new queue. Your app consumes from the new queue.
When you need to send a feed to another QMgr or application, define a new subscription and point it at the new destination queue. Since this is Pub/Sub, MQ will replicate the original message as many times as there are subscriptions and the first application and its messages are not affected. If the destination you need to send to isn't accessible over MQ channels (perhaps DEV and QA are not connected, for example), you can deliver the messages to the new queue, use QLoad from SupportPac MO03 to write them to a file and then use another instance of QLoad to load them onto a different QMgr. If you wanted to move them in real time, you could set up the Q program from SupportPac MA01 to move them direct from the new subscription queue on QMgr1 to the destination queue on QMgr2. And you can replicate across as many systems as you need.
The SupportPacs main page is here.
If all you are using is the Redbooks, you might want to have a look at the Infocenters. Be sure to use the Infocenter matching the version of WMQ you are using.
WMQ V7.0 Infocenter
WMQ V7.1 Infocenter
WMQ V7.5 Infocenter
What is the best approach to connect to websphere mq v7.1 and clear all the messages of one or more specified queues using Java and JMS? Do I need to use Websphere MQ specific java API? Thanks.
Like all good questions, "it depends."
The queue can be cleared with a command only if there are no open handles on the queue. In that case sending a PCF command to clear the queue is quite effective, but if there are open handles you get back an error. PCF commands are of course a Java feature and not JMS because they are proprietary to WebSphere MQ.
On the other hand, any program authorized to perform destructive gets off a queue can clear the queue. In this case, just loop over a get until you get the 2033 return code indicating the queue is empty. This can be performed using JMS or Java but both of these manage the input buffer for you. If the queue is REALLY deep then you end up moving all that data and if the app is client connected, you are moving it at network speed instead of in memory.
To get around this, you need to specify a minimal amount of buffer and as one of the GET options also specify MQGMO.TRUNCATED_MSG_ACCEPTED. This moves only the message header during the get calls and can be significantly faster.
Finally, if you are doing this programamtically and regardless of which method you use, spin off several threads and don't use syncpoint. You actually have to go out of your way to get exclusive input on a queue so once you get a session, just spawn many threads off of it. Close each thread gracefully and shut down the the session once all the threads are closed.