I tried doing the following:
alt ql(somequeue) statq(on)
alt qmgr statq(on)
alt qmgr statint(30)
RESET QMGR TYPE(STATISTICS)
But nothing appears in the SYSTEM.ADMIN.STATISTICS.QUEUE. I even tried to set the Inhibit GET on the queue to make sure noone is stealing the messages.
Why are the statistics not showing?
If your description lists all that you did, and there are no error messages in your queue manager error log, then best guess is that there was nothing to report!
If I do exactly that, I get no messages on the SYSTEM.ADMIN.STATISTICS.QUEUE either.
However, if before issuing the RESET command, I put a couple of messages to the queue in question, then I get some statistics on the SYSTEM.ADMIN.STATISTICS.QUEUE.
Related
I saw a question & its answer below;
https://stackoverflow.com/a/46128844/7419921
Although I understood that I cannot do anything for the error queue via MassTransit, what should I handle the error queue?
Error messages would be accumlating. It's pressing storage capacity.
It seems that I have nothing to do for the error queue. Is there no choice but to remove them?
If so, I cannot imagine a meaning of the error queue.
The meaning of error queues is very simple. Messages come to error queues because, well, of errors! When you fix issues in your application, you can move messages from the error queue back to the regular queue using Shovel plugin, and voila - you recovered lost data. We do this very often.
If you cannot move them back because these messages aren't actual anymore or they contain wrong data - this is also very valuable since using these messages you can reproduce the issue and see if you can fix the sender.
First of all, let me admit that I am completely new to MQ, so my question might be trivial.
We are using MQs configured on Websphere and I am trying to use MQExplorer 8 to monitor the same. After connecting using the Queue Managers, I can see the Current Queue Depth, Open Input Count, etc. But I don't get any option to see the actual messages lying in a queue.
I tried googling it, but did not find anything useful.
Here is what I see:
Can some enlighten me how can I see the actual messages from a queue?
The context menu is your friend. Right-click on the queue name and select Browse.
If you are dealing with JMS messages, you can use other tools like JMSToolBox to browse and manage the messages in MQ 8 and see the JMS headers etc.
I've been studying the vagaries of channel statuses, how they get to those states and what to do to get them stopped or started. I've got a pretty solid understanding now, but a colleague brought up the topic of channel resets.
I've done them occasionally when I couldn't explain what was going on, but now I understand things a bit better I'm not sure his advice to "always reset" when stopping troublesome channels is the right advice.
Searching for info online, it's clear that when recreating channels it is obvious a reset would be needed but in the case if stuff just breaking – whether a queue manager is unexpectedly dropped or the network breaks or stuff like that – is a reset a good idea in general or should I only bother if I see sequence errors or it otherwise refuses to start when I know it should?
FYI, if you are resetting from the sending side of the channel, its OK to set the sequence number to 1. The receiving side will then also go back to 1. QED :-)
If you are resetting from the receiving side of the channel, you must use the sequence number that the sender was expecting.
These numbers are in the queue manager error logs on both sides.
If the channel is in RETRY state, it will try to use the new sequence numbers when it does the next retry. This could be up to 20 minutes away if you are using the default retry attributes on the sender channel. A simple way to bump this is to STOP the channel and then START it again straight away.
HTH, G.
Channels get sequence errors for a few reasons:
The local and remote MCAs got out of sync on a batch. Usually the remote MCA committed the batch but the local one did not. If you know the remote side delivered the batch, issue a RESOLVE ACTION(COMMIT) on the channel, otherwise issue RESOLVE ACTION(COMMIT). After resolving, issue RESET.
The channel points to a new QMgr. Perhaps after failover at the DNS, circuit or firewall NAT, a different QMgr of the same name is now attached to the channel. These should be well known because the failover (hopefully) doesn't happen without some alerts going off.
The contents of the channel sync queue are in error. Sometimes the QMgr can cause this but those issues are resolved (so far as I know) in recent versions. Sometimes people accidentally mess up the sync queue, usually by browsing it with a lock while the channels are trying to use it. This is a little harder to resolve and may require clearing the sync queue but check with IBM Support first.
When the channel is out of sync because of a known exception like failover, go ahead and reset it. Otherwise, you'd be well advised to find out why it's out of sync. You might reset it just to get it up and running, but hopefully not until you've saved off the <QMGR>/errors/AMQERR*.LOG files and any FDCs so you can diagnose the cause.
The logic flow is like this
A message is sent to an input queue
A ProcessorMDB's onMessage() is invoked. Within this method several operations/validations are done
In case of a poison message(msg that application code cannot handle) a RuntimeException is thrown.
This should rollback the transaction. We are seeing evidence in the log file.
There is a backout threshold defined with a backout queue name
once threshold is reached, the message is sent to backout queue
But immediately it starts going back and forth between the input queue and backout queue.
We are using MQMON tool to observe this weird behavior. It continues for ever almost even after the app server(where MDB is running) is shutdown.
We are using Weblogic 10.3.1 and WebSphere MQ 6.02
Any help will be much appreciated, looks like we are running out of ideas.
This sounds like a syncpoint issue. If the QMgr were to issue a COMMIT when a message is requeued inside of a unit of work it would affect all messages under syncpoint inside of that thread. This would cause serious problems if an application had performed several PUT or GET calls prior to hitting the poison message. Rather than issue a COMMIT outside of the program's control, the QMgr just leaves the message on the backout queue inside the unit of work and waits for the program to issue the COMMIT. This can lead to some unexpected behavior such as what you are seeing where a message lands back on the input queue.
If another message is in the queue behind the "bad" one and it is processed successfully by the same thread, everything works out perfectly. The app issues a COMMIT on the new message and this also affects the poison message on the Backout Queue. However if the thread were to exit uncleanly (without an explicit disconnect or COMMIT) then the transaction is rolled back and the poison message is returned to the input queue.
The usual way of dealing with this is that the next good message (or batch of messages if transactions are batched) in the input queue will force the COMMIT. However in some cases where the owning thread gets no new work (perhaps it was performing a GET by Correlation ID) there is nothing to push the bad message through. In these cases, it is important to make sure that the application issues a COMMIT before ending. One way to do this is to write the code to perform the GET by CORRELID with a wait interval. If the wait interval expires, the application would get a return code of 2033 and then issue a COMMIT before closing the thread. If the reply message is legitimately late for whatever reason, the COMMIT will have no effect. But if the message arrived and had been backed out and requeued, the COMMIT will cause it to stay in the Backout Queue.
One way to see exactly what is going on is to run a trace against the queue in question. You can use the built-in trace function - strmqtrc - which has a few more options in V7 than does the V6 version. However if you want very fine grained control you can use the trace exit in SupportPac MA0W. With MA0W you can see exactly what API calls are made by the program and those made on its behalf.
[EDIT] Updating the response with some info from the PMR:
The following is from the WMQ V7 Infocenter:
MessageConsumers are single threaded below the Session level, and
any requeuing of poison messages
takes place within the current unit of
work. This does not affect the
operation of the application, however
when poison messages are requeued
under a transacted or
Client_acknowledge Session, the
requeue action itself will not be
committed until the current unit of
work is committed by the application
code or, if appropriate, the
application container code."
Hence, if it is important for the customer to have poison messages
committed immediately after they are
backed out, it is recommended they
either make use of the Application
Server Facilities
(ConnectionConsumer) which can commit
the message immediately, or
another mechanism to move poison
messages from the queue.
Here is the link to this information in the V6 and V7 Information Centers. Since you are using the V6 client so you would want to refer to the V6 Infocenter. Note that with the V6 client, there is no mention in the Infocenter of ASF being able to commit the poison message immediately, even when using a ConnectionConsumer. The way I read it, this means you probably will need to upgrade to the V7 client to get the behavior you are looking for. Will be interested to see if the PMR results in a similar recommendation.
I use MQ for send/receive message between my system and other system. Sometime I found that no response message in response queue, yet other system have already put response message into response queue (check from log). So, how to check which point is cause of problem, how to prove message is not arrive to my response queue.
In addition, when message arrive my queue it will be written to log file.
You can view this in real-time using the QStats interface. The MO71 SupportPac is a desktop client that you can configure to connect similar to WebSphere MQ Explorer. One of the options it has is queue statistics. Each time you view the queue stats, WMQ resets them to zero. So the procedure is this:
Start MO71 and browse the queues.
Filter on the one queue of interest.
View the queue stats a couple of times.
You will see them reset to zero.
Now run your test.
View the queue stats again.
If the remote program actually put a message, you will see that the queue now shows one or more messages PUT.
If your program successfully executed a GET of the message, you will see GET counts equal to the number of PUT counts.
If GET and PUT both zero, the remote program never PUT the response message.
There are a few other approaches to this but this is the easiest. The opposite end of the spectrum is SupportPac MA0W which will show you every API call against that queue, or by PID, or whatever. It shows all the options so if a program tries to open the queue with the wrong options (i.e. open a remote queue for input) it shows that. But MA0W is a installed as an exit and requires the QMgr to be bounced so it's a little invasive.