I'm stuck with the following issue with message Broker 7.0.0.5
Here is my flow:
what I want it to do:
1. accept an XML, parse, using XMLNSC
2. then I want some business logic to be performed, but let's skip it and focus on the following - I want to throw two user exceptions -
one in module GoodReport, another one - in module BadReport,
therefore, after being processed in both Out and Catch flows,
I assume my message to land to Backout Queue (I have created one for my queue, and set Threshold to 10) and be sent back to MQInput in order to be processed again.
So I expect 10 messages in the Backout queue, but instead - I get NOTHING
I can see both errors being thrown in my flow, but what wonders me most is the last lines in the trace - from where have that "XMLNS" domain appeared?
I use only XMLNSC domain.
And why the message does not appear in the Backout queue?
Thank you in advance!
Tatiana.
here is the trace:
2012-12-17 19:25:54.692283 5756 RecoverableException BIP2488E: ('.Esql6_1Flow_Report.Main', '19.4') Error detected whilst executing the SQL statement ''THROW EXCEPTION MESSAGE 3 VALUES( 'NO_SUCH_SOURCE');''.
The message broker detected an error whilst executing the given statement. An exception has been thrown to cut short the SQL program.
See the following messages for details of the error.
2012-12-17 19:25:54.692302 5756 UserException ?????????? 3 ?????????? BIPmsgs.properties
2012-12-17 19:26:52.830982 5756 Error BIP2628E: Exception condition detected on input node 'Esql6_1Flow.MQInput'.
The input node 'Esql6_1Flow.MQInput' detected an error whilst processing a message. The message flow has been rolled-back and, if the message was being processed in a unit of work, it will remain on the input queue to be processed again. Following messages will indicate the cause of this exception.
Check the error messages which follow to determine why the exception was generated, and take action as described by those messages.
2012-12-17 19:26:52.831005 5756 RecoverableException BIP2230E: Error detected whilst processing a message in node 'Esql6_1Flow.BadReport'.
The message broker detected an error whilst processing a message in node 'Esql6_1Flow.BadReport'. An exception has been thrown to cut short the processing of the message.
See the following messages for details of the error.
2012-12-17 19:26:52.831012 5756 RecoverableException BIP2488E: ('.Esql6_1Flow_Compute.Main', '13.4') Error detected whilst executing the SQL statement ''THROW EXCEPTION MESSAGE 3 VALUES( 'NO_SUCH_SOURCE');''.
The message broker detected an error whilst executing the given statement. An exception has been thrown to cut short the SQL program.
See the following messages for details of the error.
2012-12-17 19:26:52.831020 5756 UserException ?????????? 3 ?????????? BIPmsgs.properties
2012-12-17 19:26:53.831737 5756 Error BIP2648E: Message backed out to a queue; node 'Esql6_1Flow.MQInput'.
Node 'Esql6_1Flow.MQInput' has received a message which has previously been backed out one or more times because of a processing error in the main path of the message flow. The failure terminal is not attached, so the message broker is putting the message directly to the requeue or dead letter backout queue associated with this node. The MQMD 'backoutCount' of the message now equals the 'backoutThreshold' defined for the WebSphere MQ input queue.
Examine previous messages and the message flow to determine why the message is being backed out. Correct this situation if possible. Perform any local error recovery processing required.
2012-12-17 19:26:53.832435 5756 UserTrace BIP2638I: The MQ output node 'Esql6_1Flow.MQInput' attempted to write a message to queue ''SYSTEM.DEAD.LETTER.QUEUE'' connected to queue manager ''testQueueManagerName''. The MQCC was '0' and the MQRC was '0'.
2012-12-17 19:26:53.832466 5756 UserTrace BIP2615I: The WebSphere MQ input node 'Esql6_1Flow.MQInput' has backed out the message to the backout requeue or the dead letter queue.
Message backout processing has been invoked, and the message has either been backed out by being written to the backout requeue or dead letter queue, as determined by the WebSphere MQ queue manager and queue configuration.
No user action required.
2012-12-17 19:27:31.087949 4380 UserTrace BIP2632I: Message received and propagated to 'out' terminal of MQ input node '.InputNode'.
2012-12-17 19:27:31.088045 4380 UserTrace BIP6060I: Parser type ''Properties'' created on behalf of node '.InputNode' to handle portion of incoming message of length 0 bytes beginning at offset '0'.
2012-12-17 19:27:31.088066 4380 UserTrace BIP6061I: Parser type ''MQMD'' created on behalf of node '.InputNode' to handle portion of incoming message of length '364' bytes beginning at offset '0'. Parser type selected based on value ''MQHMD'' from previous parser.
2012-12-17 19:27:31.088092 4380 UserTrace BIP6069W: The broker is not capable of handling a message of data type ''''.
The message broker received a message that requires the handling of data of type '''', but the broker does not have the capability to handle data of this type.
Check both the message being sent to the message broker and the configuration data for the node. References to the unsupported data type must be removed if the message is to be processed by the broker.
2012-12-17 19:27:31.088113 4380 UserTrace BIP6061I: Parser type ''XMLS'' created on behalf of node '.InputNode' to handle portion of incoming message of length '236' bytes beginning at offset '364'. Parser type selected based on value ''XMLS'' from previous parser.
BACKOUT count determines the number of times the flow should retry processing the message. Here the threshold is set as 10, that means the flow tries the processing 10 times and if it still failed, MQInput node will put the message to backout queue or DLQ if backout queue is not configured. Flow will put only one message to the backout queue, not 10 messages as exepected by you
If the messages are processed in non-transactional mode, then the flow will not put the message to backout queue. Check whether you have configured Transactional property as 'NO' in the MQInput node. If the configured value is 'Automatic' then the message should be a persistent one to get it processed under a transaction. But the trace snippet shows the message is put to SYSTEM.DEAD.LETTER.QUEUE. You may need to check whether the message is in DLQ and also confirm backout queue property is correctly configured in the input node.
Have a read on
http://publib.boulder.ibm.com/infocenter/wmbhelp/v7r0m0/topic/com.ibm.etools.mft.doc/ac00414_.htm
this will address all your queries.
Related
E 02/07/19-07:51:49.358 [-5] JMSListener exception : java.lang.ClassCastException: com.ibm.jms.JMSMessage cannot be cast to javax.jms.TextMessage
This is a ticky one. We put an xml message onto a queue and our code consumed it no problem. The next day, we put another xml message onto the same queue, and the same code threw this error.
I'm looking for suggestions on how to get more info out of what's actually happening, or if anyone else has had intermittent class cast exceptions like this with JMS.
Please note that not all 'XML' messages are sent as JMS TextMessage. I've come across the same thing. Some applications will decide to send you the XML content as JMS BytesMessage (and put the text inside the byte array body). So make sure your sending is really sending you a TextMessage.
One way to see this is to stop your consumer and look at the queued messages with MQExplorer: the Named Property mcd.Msd should show jms_text if it was sent as proper MQ JMS TextMessage. If the data is received from a non-JMS application, the MQ format matters. If the format is MQFMT_STRING, the message is received as a JMS TextMessage. Otherwise, it is received as a JMS BytesMessage !
So either change your publisher to set the value of format to be MQFMT_STRING or change your consumer application to remove the cast to TextMessage and instead also accept BytesMessages and construct the text from the body's byte array.
If I acknowledge the same message twice using the Delivery.Ack method, my consumer channel just closes by itself.
Is this expected behaviour? Has anyone experienced this ?
The reason I am acknowledging the same message twice is a special case where I have to break the original message into copies and process them on the consumer. Once the consumer processes everything, it loops and acks everything. Since there are copies of the entity, it acks the same message twice and my consumer channel shuts down
According to the AMQP reference, a channel exception is raised when a message gets acknowledged for the second time:
A message MUST not be acknowledged more than once. The receiving peer
MUST validate that a non-zero delivery-tag refers to a delivered
message, and raise a channel exception if this is not the case.
Second call to Ack(...) for the same message will not return an error, but the channel gets closed due to this exception received from server:
Exception (406) Reason: "PRECONDITION_FAILED - unknown delivery tag ?"
It is possible to register a listener via Channel.NotifyClose to observe this exception.
We are using spring integration and daily in our logs we can see below stacktrace. Other JMS adapters are working fine, we think only below one is missing something:
Spring integration configuration:
<jms:message-driven-channel-adapter concurrent-consumers="1" id="jmsInLOAN" destination="queueLOAN" channel="LOANCommonDataChannel" acknowledge="transacted" />
Please find below MQ statistics of Put and Msgs read count, there should be exact count of Message read by adapter. I am worried about spring integration's message-driven-channel-adapter of reading extra messages from queue. Any help would be appreciated.
WARN 07/Jan/2016 09:04:15,438 [org.springframework.jms.listener.DefaultMessageListenerContainer#23-1] springframework.jms.listener.DefaultMessageListenerContainer - [SYSTEM_ID=HBUSLOANIQ] [MESSAGE_ID=null] Execution of JMS message listener failed, and no ErrorHandler has been set.
org.springframework.integration.MessagingException: unsupported payload type [com.ibm.jms.JMSMessage]
at org.springframework.integration.xml.DefaultXmlPayloadConverter.convertToDocument(DefaultXmlPayloadConverter.java:76)
at org.springframework.integration.xml.DefaultXmlPayloadConverter.convertToNode(DefaultXmlPayloadConverter.java:88)
at org.springframework.integration.xml.router.XPathRouter.getChannelIdentifiers(XPathRouter.java:119)
at org.springframework.integration.router.AbstractMessageRouter.determineTargetChannels(AbstractMessageRouter.java:247)
at org.springframework.integration.router.AbstractMessageRouter.handleMessageInternal(AbstractMessageRouter.java:211)
It looks like you are passing the unconverted JMS message (com.ibm.jms.JMSMessage) to the XML Payload converter...
org.springframework.integration.MessagingException: unsupported payload type [com.ibm.jms.JMSMessage]
at org.springframework.integration.xml.DefaultXmlPayloadConverter.convertToDocument(DefaultXmlPayloadConverter.java:76)
Perhaps you have set extract-payload to false ?
Although it's not in the configuration you show.
Turning on DEBUG logging will show the payload type of messages passing through the system.
The issue was because of valid and poisonous(which has a payload type [com.ibm.jms.JMSMessage]) messages we were getting onto queue. Valid messages processed well, but poisounous messages not able to digest by application and send to the BackoutQueue.
In our case, BOQ threshold is 3 that means 3 times my application will try to consume a perticular message and if the message backout 3 times then it will be moved to BOQ queue and (msgs read - msgs put)/3 on LOAIQ == the msgs put on to BOQ queue at that sampling interval. From msgs put on BOQ queue, we can see how many messages are backout from LOAIQ queue. That is why the message read count is more than that of msg received.
I am working on real-time event-based application using Spring WebSockets, Messaging and RabbitMQ. In this application, messages need to be delivered to clients in the exact order they were inserted into RabbitMQ.
"EDITED"
Our goal is to recieve a message from a browser, process it on the server in order (against a unique object determined by route paramteter), enrich the message and broadcast it to all subscribing browsers via external STOMP MQ (RabbitMQ).
Our MessageMapping method is as follows:
#MessageMapping(/commands.{route}.{data})
public CommandMessage receiveCommand(CommandMessage message, Principal principal) {
try {
// Get object to synch on using route
Object o = ...
syncrhonized(o) {
// Perform command on object
// Set message server sequence
message.setServerSequence(o.getAutoIncrementSequence());
// Log server sequence
log.debug("Message server sequence:" + message.getServerSequence());
// Send to external MQ for broadcasting to all subscribers
return message;
}
} catch (Exception e) {
...
}
return null;
}
If we configure the ClientInboundChannel and ClientOutboundChannels with a corePoolSize and maxPoolSize of 1 each, all messages are in order.
If we increase the corePoolSize and maxPoolSize on the ClientInboundChannel, messages get to MQ in the incorrect order; if we do the same increase for ClientOutboundChannel, messages get to browser in incorrect order.
]
Tests were done using a single browser client.
We turned on tracing for StompBrokerRelayMessageHandler and received logs entries like:
2014-05-06 14:26:39 TRACE [clientInboundChannel-6] o.s.m.s.s.StompBrokerRelayMessageHandler [StompBrokerRelayMessageHandler.java:412] Processing message=[Payload byte[303]][Headers={stompCommand=SEND, nativeHeaders={content-type=[application/json;charset=UTF-8], destination=[/topic/commands.BKN01.20140318]}, simpMessageType=MESSAGE, simpDestination=/topic/commands.BKN01.20140318, contentType=application/json;charset=UTF-8, simpSessionId=ehjcoxb3, id=a31d0e3d-12cc-f562-1ec2-e2d7ba0899eb, timestamp=1399400799940}]
2014-05-06 14:26:39 DEBUG [clientInboundChannel-6] o.s.m.s.s.StompBrokerRelayMessageHandler [StompBrokerRelayMessageHandler.java:658] Forwarding message to broker
2014-05-06 14:26:39 TRACE [clientInboundChannel-3] o.s.m.s.s.StompBrokerRelayMessageHandler [StompBrokerRelayMessageHandler.java:406] Ignoring message to destination=/app/commands.BKN01.20140318
2014-05-06 14:26:39 TRACE [clientInboundChannel-7] o.s.m.s.s.StompBrokerRelayMessageHandler [StompBrokerRelayMessageHandler.java:406] Ignoring message to destination=/app/commands.BKN01.20140318
2014-05-06 14:26:39 TRACE [clientInboundChannel-1] o.s.m.s.s.StompBrokerRelayMessageHandler [StompBrokerRelayMessageHandler.java:412] Processing message=[Payload byte[314]][Headers={stompCommand=SEND, nativeHeaders={content-type=[application/json;charset=UTF-8], destination=[/topic/commands.BKN01.20140318]}, simpMessageType=MESSAGE, simpDestination=/topic/commands.BKN01.20140318, contentType=application/json;charset=UTF-8, simpSessionId=ehjcoxb3, id=3cc7b4ae-8ea4-ef8a-6c4d-c3bc1ed23bcd, timestamp=1399400799947}]
2014-05-06 14:26:39 DEBUG [clientInboundChannel-1] o.s.m.s.s.StompBrokerRelayMessageHandler [StompBrokerRelayMessageHandler.java:658] Forwarding message to broker
We also turned tracing on for StompSubProtocolHandler in the org.springframework.web.socket.messaging package and received messages like:
2014-05-07 10:58:58 TRACE [http-nio-8080-exec-5] o.s.w.s.m.StompSubProtocolHandler [StompSubProtocolHandler.java:180] Received message from client session=u8wrnsr6
None of the information provides an easy way to map our message.serverSequence property (which is set prior to sending to MQ) with the various ids the log details.
Is there any way to increase inbound/outbound channel threads so ordering is intact? For example, could the channels be tied to a "route" or could a thread be pegged to a "route"?
Please help.
Thank you,
Dan
[EDITED]
Thanks, I see now. Indeed at present there is no way to ensure messages from the external broker are delivered to a client in the exact same order and that messages from a client will be sent to the external broker in the exact same order.
We could assign an index to every Message within a session and then check and buffer if necessary to enforce the order but that would be a new feature. Feel free to create a request in JIRA.
As for now you could look into creating a ChannelInterceptor that checks every message to see what session it belongs to and then set some incremental index header on it before having it sent (on the clientInboundChannel or clientOutboundChannel). Then extend StompSubProtocolHandler and StompBrokerRelayMessageHandler to check the index header and try to enforce the order of messages.
I think I tried to start a channel that is already running or whatever. Whenever I start the sender channel, the receiver channel goes to a PAUSED state. I looked it up and found something about AdoptNewMCA configuration, not sure how to set it at the queue manager level. How do I fix this smoothly. Merely stopping and restarting the channels does not do it.
Error log says:
/02/2012 12:38:41 PM - Process(19161.269) User(mqm) Program(amqrmppa)
Host() Installation(Installation1)
VRMF(7.1.0.0) QMgr(QM_TEST2)
AMQ9514: Channel 'QM_TEST1.TO.QM_TEST2' is in use.
EXPLANATION: The requested operation failed because channel
''QM_TEST1.TO.QM_TEST2' is currently active. ACTION: Either end the channel
manually, or wait for it to close, and retry the operation.
----- amqrcsia.c : 1042 -------------------------------------------------------
08/02/2012 12:38:41 PM - Process(19161.269) User(mqm) Program(amqrmppa)
Host(...) Installation(Installation1)
VRMF(7.1.0.0) QMgr(QM_TEST2)
AMQ9999: Channel ''QM_TEST1.TO.QM_TEST2' to host '17.2.33.44' ended abnormally.
EXPLANATION: The channel program running under process ID 19161 for
channel ''QM_TEST1.TO.QM_TEST2' ended abnormally. The host name is
'17.2.33.44'; in some cases the host name cannot be
determined and so is shown as '????'. ACTION: Look at previous error
messages for the channel program in the error logs to determine the
cause of the failure. Note that this message can be excluded
completely or suppressed by tuning the "ExcludeMessage" or
"SuppressMessage" attributes under the "QMErrorLog" stanza in qm.ini.
Further information can be found in the System Administration Guide.
----- amqrmrsa.c : 887 --------------------------------------------------------
When looking these things up, I'd start first with the product manuals. In this case, the Infocenter topic on channel states says that a channel in PAUSED state is waiting on a retry interval. The sub-topic on channel errors explains why sending or receiving channels can be in retry:
If a channel is unable to put a message to the target queue because
that queue is full or put inhibited, the channel can retry the
operation a number of times (specified in the message-retry count
attribute) at a time interval (specified in the message-retry interval
attribute). Alternatively, you can write your own message-retry exit
that determines which circumstances cause a retry, and the number of
attempts made. The channel goes to PAUSED state while waiting for the
message-retry interval to finish.
So if you stop your channels, you should see a message in the XMitQ on the sending side. If you GET-enable that queue you can browse the message, look at the header and see which queue it is destined for. On the receiving side, look to see if that queue is full.
Classic fast-sender, slow-consumer problem here. If the consumer can't keep up, the messages back up on the receiving QMgr, then the channel goes to retry and they begin to back up on the sending QMgr. Got to monitor depth and input handles on request queues.
Make sure a DLQ is set.
Try reducing the message retry count to 1 to speed up use of the DLQ.