I am new to IIB technology, I am facing an issue and seeing below error very frequently when there is failure in the messageflow.
We have MQ input node which all terminals are connected, Failure and Catch terminals are connected to CommonErrorHandling in which it route the message to backout queue.
Have verified backout queue which is created and present, Can someone help me provide some insights.
Message flow - MQ Input Node(Domain-JSON) ->Compute ->Https Request node
<Catalog>BIPmsgs</Catalog>
<Severity>3</Severity>
<Number>2652</Number>
<Text>Dequeued failed message. Propagating a message to the failure terminal</Text>
Debugged the flow and could observe it is coming out from the Failure terminal of Input MQ node but not able to understand much why it is giving this message.
Please share more details about your flow. From which node is the exception happening at first ? Did you check the MQOutput node configuration ?
It is also helpful to connect a trace node to the failure or catch terminals of the MQOutput node. It should give a clearer description of why its bombing out before rolling the message all the way back to the MQInput node.
Check your JSON String for instance control the type of variable.
In IIB it's necessary control everything step by step.
Related
My instance of OSB is attempting to process a message from a JMS queue that doesn't exist - I believe this has already been processed and removed, but my current concern is the multiple failures each second it is trying to continue. The error logs are now useless as they're flooded with failures for one particular message.
I have rebooted the managed servers and admin server, but each time, it is immediately reattempting to process the same message. I believe this is having knock-on effects to performance, and I have had to remove all logs as the file system is continuing to overflow.
Where is this "currently processing" message being picked up from, and how can I progress this so that it will not keep trying to reprocess this?
As far as I understand, a problematic message is processing continuous failure within a JMS queue. There are 2 important actions.
Identify failure root cause. For me to help you with this item
depends on the error message. If error details are provided, I might
provide suggestions.
Protect environment by JMS Queue Configuration for Delivery Failure
such as "Expiration Policy", "Redelivery Limit", "Error Destination"
etc.
Please check following Oracle documentation for these configurations.
I'm newbie on IBM tool.
On my laptop, I have installed IIB v10 using a local MQ manager. I created 2 local queues on MQ: "Queue.In" and "Queue.Out".
I have a very simple scenario that I work on: I send a 'csv' file on the "Queue.In", get the message using 'MQInput node' transform the message into xml using 'mapping node', and then put the xml message to 'Queue.Out'.
Using the ‘flow exercicser’ I’m able to send my ‘csv’ file and I can see that the message get consumed and transformed to xml and send to “Queue.Out”
Problem: My xml message is successfully put in "Queue.Out", but disappear right after.
I want it to remain in the "Queue.Out".
I use “MQ explorer” to browse my queues.
Please advice.
I would always recommend using Rfhutil for Delivery and Subscription of messages onto/out of a queue. In Rfhutil you can also browse messages without taking them off the queue which can be useful.
I am unsure why the flow exerciser removes messages - this is a new feature in IIB10 so it might be that they wanted queues to be cleared after the flow execution has been completed to avoid clogging queues. I believe the point of the exerciser is to track the logic of the flows themselves not to get messages.
Hope this helps :)
I have a list of stores with both Incoming and Outgoing Failed Message Queues. It uses Message Driven Beans (to read and write) into those Queues (Java Message Service(JMS) in WildFly 8.0)
I want to know
how to pinpoint (from the application) how and where these messages are written into the Queues
how to reduce the number of failed messages to a minimum
Messages fail because they timeout, increase the timetolive config value to avoid the messages ending up in the error queue.
To answer your first question, I dont have a specific answer, you may have to use logging statements just before you send the message to the queue along with the time. If your message has time also, later in the future you can cross verify the logs to that of the error messages.
This is in Cluster Environment. Queue Manager lost its identity in cluster and it is unable to connect to other servers. All channels to repository and others were retrying state.
CPU usage is optimal in this server. This is a UNIX box.
When I checked the logs below is it,
AMQ9532: Program cannot set queue
attributes.
EXPLANATION: The attempt to set the
attributes of queue
'SYSTEM.CLUSTER.TRANSMIT.QUEUE' on
queue manager 'QMGR.SERVER6A' failed
with reason code 2102.
ACTION: Ensure
that the queue is available and retry
the operation.
----- amqrmssa.c : 690 --------------------------------------------------------
AMQ9999: Channel program ended
abnormally.
EXPLANATION: Channel program
'Channel.Coord00' ended abnormally.
ACTION: Look at previous error
messages for channel program
'Channel.Coord00' in the error files to
determine the cause of the failure.
----- amqrccca.c : 883 --------------------------------------------------------
03/06/11 08:24:26 AMQ9544: Messages
not put to destination queue.
EXPLANATION: During the processing of
channel 'Channel.Server6A' one or more
messages could not be put to the
destination queue and attempts were
made to put them to a dead-letter
queue. The location of the queue is
1, where 1 is the local dead-letter
queue and 2 is the remote dead-letter
queue.
ACTION: Examine the contents of
the dead-letter queue. Each message
is contained in a structure that
describes why the message was put to
the queue, and to where it was
originally addressed. Also look at
previous error messages to see if the
attempt to put messages to a
dead-letter queue failed. The program
identifier (PID) of the processing
program was '1372200'.
----- amqrmrca.c : 1318 -------------------------------------------------------
Then I did recycled the queue manager it is now fine?
My question here is how did the MQ resource problem occurr? CPU usage of this server is not more than 15%. Please advise.
There are three different and unrelated problems shown in the log.
AMQ9532: Program cannot set queue
attributes.
EXPLANATION: The attempt to set the
attributes of queue
'SYSTEM.CLUSTER.TRANSMIT.QUEUE' on
queue manager 'QMGR.SERVER6A' failed
with reason code 2102.
The 2102 is MQRC_RESOURCE_PROBLEM and presumably the resource issue referred to in the posting. The 2102 can be any kind of scarce resource, including semaphores, user processes, queue handles, etc. Since the QMgr was attempting to set an attribute of the queue, it would have already had a thread instantiated but it would have required additional queue handles. When something like this occurs, use your admin tool (WMQ Explorer, mqmon or one of the many 3rd party tools) to look into the number of open queue handles, open channels, etc. Note that for a resource error, it will be necessary to maintain an open connection to the QMgr or else the tool will be unable to make a new connection when the resource shortage occurs.
AMQ9999: Channel program ended
abnormally.
EXPLANATION: Channel program
'Channel.Coord00' ended abnormally.
ACTION: Look at previous error
messages for channel program
'C00.US.MP00' in the error files to
determine the cause of the failure.
This error appears to actually be two different errors since it references two different channels. One of these appears to be an outbound cluster channel and the other appears to be a point-to-point channel. Neither channel mentioned in this error are associated with the first and last error message.
03/06/11 08:24:26 AMQ9544: Messages
not put to destination queue.
EXPLANATION: During the processing of
channel 'Channel.Server6A' one or more
messages could not be put to the
destination queue and attempts were
made to put them to a dead-letter
queue. The location of the queue is 1,
where 1 is the local dead-letter queue
and 2 is the remote dead-letter queue.
ACTION: Examine the contents of the
dead-letter queue. Each message is
contained in a structure that
describes why the message was put to
the queue, and to where it was
originally addressed. Also look at
previous error messages to see if the
attempt to put messages to a
dead-letter queue failed. The program
identifier (PID) of the processing
program was '1372200'.
The last error appears to be an inbound cluster channel. Since the first error was trying to set attributes of the cluster transmit queue, it could only have been associated with an outbound channel. Therefore the first and last error messages are unrelated. This error message appears to show an inbound message that was destined for a queue and that queue was full, PUT-disabled, or otherwise unable to accept the message. The message was therefore routed to the dead letter queue.
For the resource error, I would suggest reviewing the performance report appropriate to your platform. Go to the SupportPacs page and look for those SupportPacs named MP* and then look for the one for your platform. The Performance Reports give you specific tuning advice.
You may also want to review the Problem Determination chapter in the System Administration manual for additional advice on how to identify resource issues.
The WebSphere MQ cluster design and operation article in the developerWorks Mission:Messaging series has specific advice about keeping clusters healthy.
Last but not least, the WebSphere MQ MustGather page has sections on troubleshooting for all major platforms and categorized by problem area.
To increase the MAXMSGL to 100 MB in IBMMQ,
(Reason code-2102 - MQRC_RESOURCE_PROBLEM) after setting the MAXMSGL to 100 mb
Category: IBM WebSphere MQ
If you are receiving error Reason code:2102 - MQRC_RESOURCE_PROBLEM, then try
Queue manager->properties->Extended->Increase Log->Log primary files and Log->Log secondary files->value to 20
I use MQ for send/receive message between my system and other system. Sometime I found that no response message in response queue, yet other system have already put response message into response queue (check from log). So, how to check which point is cause of problem, how to prove message is not arrive to my response queue.
In addition, when message arrive my queue it will be written to log file.
You can view this in real-time using the QStats interface. The MO71 SupportPac is a desktop client that you can configure to connect similar to WebSphere MQ Explorer. One of the options it has is queue statistics. Each time you view the queue stats, WMQ resets them to zero. So the procedure is this:
Start MO71 and browse the queues.
Filter on the one queue of interest.
View the queue stats a couple of times.
You will see them reset to zero.
Now run your test.
View the queue stats again.
If the remote program actually put a message, you will see that the queue now shows one or more messages PUT.
If your program successfully executed a GET of the message, you will see GET counts equal to the number of PUT counts.
If GET and PUT both zero, the remote program never PUT the response message.
There are a few other approaches to this but this is the easiest. The opposite end of the spectrum is SupportPac MA0W which will show you every API call against that queue, or by PID, or whatever. It shows all the options so if a program tries to open the queue with the wrong options (i.e. open a remote queue for input) it shows that. But MA0W is a installed as an exit and requires the QMgr to be bounced so it's a little invasive.