I have setup a topic to MQ messaging. The setup is done like in the documentation "MQ Connectivity scenario 2: Fan in per device notification" it appears as though Messagesight adds data the the message when it arrives at MQ destination. (possibly JMS header data). Is there something that has to be setup either on messagesight or MQ manager to remove the data and only leave the message?
Related
I'm newbie on IBM products, and I would like to know what would be the best approach to achieve the following integration solution:
A third party application that uses only AMQP to exchange messages, will put a message on an inbound queue (managed by IBM MQ)
IBM integration bus(version 10) flow will retrieve that message
Parse the message and invoke a rest api, get a response.
Parse the response, and send it back to the output queue so the third party can consume it.
My MQ colleague (amqp in new for him) created the corresponding "AMQP queue manager; channel and topic" using IBM MQ 9.0.4, and provided me of the connection info.
In my understanding I thought about creating a flow that will use "MQInput node" that will connect to the queue used by the third party to drop the message, and continue my processing.
Unfortunately I’m not able to connect to that AMQP channel, neither using: The "MQInput node" on IIB v10 Nor using "MQ explorer"
I strongly believe I miss something in that integration solution: either on the mq side or on the iib side, but not able to know what.
The challenge is to know, in order to achieve the above scenario, does the component IBM MQ and IBM Integration bus are enough, or I'll some other components?
Any documentation or tutorial that could describe/explain such scenario?
I have a Java JMS application that reads messages from a MQ queue. My application can successfully read a message and pull out the JMS headers (e.g. JMSDeliveryMode, JMSPriority etc) and the message body, but I can not access the <usr> folder part of the message.
I am placing messages on a queue using RFHutil. Under the RFH tab I am ticking 'Include RFH V2 Headers' and 'usr', and under the usr tab I have test1=1. I can see the <usr><test1>1</test1></usr> in the message if I browse the queue using MQExplorer, so I think I'm setting the usr folder correctly.
I have outputted the whole of message.getPropertyNames() enumeration (as discussed at topic JMS passing entire Application data <usr> block, but my usr fields are not under there. My message body also does not contain my usr fields.
I've also read on the MQ v7 info center you need to set WMQ_MESSAGE_BODY to WMQ_MESSAGE_BODY_MQ, but I am using v6MQ and do not know if this is relevant/how to set this value.
How can I get access to the usr folder using the JMS API?
Any help/pointers would be much appreciated as I am fairly new to JMS programming.
A first question is why is the USR folder of importance? Are you attempting to get your JMS application to interoperate with messages that put using another application that writes custom RFH2 headers.
RFHUtil is primarily used for test purposes.
MQ JMS originally used the RFH2 heasder to specifically send the JMS properties. Therefore the MQ JMS code reads the message, removes the RFH2 and processes it into a full JMS Message.
If you want the JMS API to fully read the RFH2, then the properties for reading the MQ message you've highlighted are correct. However those are not in MQ v6. Be aware that mq v6 and V7.0.1 are now out of support. Would strongly recommend that you investigate upgrading to MQ v8 - at minimum MQ 7.5.
CLients are freely downloadable if you are a developer and connecting to an estbalished QueueManager. Client versions don't have to match the QM.
Is it possible to configure the topic to store a copy of just the last message and send this to new connections without knowing client identifiers or other info?
Update:
From the info provided by Shashi I found this two pages where they describe a use case similar to mine (applied over stock prices) by using retroactive consumer and a subscription recovery policy. How ever I'm not getting the desired behaviour. What I currently do is:
Include in the activemq the folowing lines in the policyEntry for topic=">"
<subscriptionRecoveryPolicy>
<fixedCountSubscriptionRecoveryPolicy maximumSize="1"/>
</subscriptionRecoveryPolicy>
Add to the URL used to connect to the brocker (using activemq-cpp) consumer.retroactive=true.
Set the consumer has durable. (But I strongly think this is not want since I only need the last one, but without it I didn't get any message when starting the consumer for the second time)
Start up the broker.
Start the consumer.
Send a message to the topic using the activemq web admin console. (I receive it in the consumer, as expected)
Stop consumer.
Send another message to the topic.
Start consumer. I receive the message, also as expected.
However, if the consumer receives a message, then it goes offline (stop process) and then I restart it, it doesn't get the last message back.
The goal is to whenever the consumer starts get the last message, no mater what (obviously, except when there weren't messages sent to the topic).
Any ideas on what I'm missing?
Background:
I have a device which publishes his data to a topic when ever its data changes. A variable number of consumer may be connected to this topic, from 0 to less than 10. There is only one publisher in the topic and always publish all of his data as a single message (little data, just a couple of fields of a sensor reading). The publication rate of this information is variable, not necessarily time based, when something changes a new updated message is sent to the broker.
The problem is that when a new consumer connects to the topic it has no data of the device readings until a new message is send to the topic by the device. This could be solve by creating an additional queue so new connections can subscribe to the topic and then request the device for the current reading through the queue (the device would consume the queue message which would be a request for data, and then response in the same queue).
But Since the messages send to the topic are always information complete I was wondering if is it possible to configure the topic to store a copy of just the last message and send this to new connections without know client identifiers or other info?
Current broker in use is ActiveMQ.
What you want is to have retroactive consumers and to set the lastImageSubscriptionRecoveryPolicy subscription recovery policy on the topic. Shashi is correct in saying that the following syntax for setting a consumer to be retroactive works only with Openwire
topic = new ActiveMQTopic("TEST.Topic?consumer.retroactive=true");
In your case, what you can do is to configure all consumers to be retroactive in broker config with alwaysRetroactive="true". I tested that this works even for the AMQP protocol (library qpid-jms-client) and I suspect it will work for all protocols.
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic="FOO.>" alwaysRetroactive="true">
<subscriptionRecoveryPolicy>
<lastImageSubscriptionRecoveryPolicy />
</subscriptionRecoveryPolicy>
</policyEntry>
The configuration example is taken from https://github.com/apache/activemq/blob/master/activemq-unit-tests/src/test/resources/org/apache/activemq/test/retroactive/activemq-message-query.xml
Messaging providers (WebSphere MQ for example) have a feature called Retained Publication. With this feature the last published message on a topic is retained by the messaging provider and delivered to a new consumer who comes in after a message has been published on a given topic.
Retained Publication may be supported by Active MQ in it's native interface. This link talks about consumer.retroactive which is available for OpenWire only.
A publisher will tell the messaging provider to retain a publication by setting a property on the message before publishing. Below is how it is done using WebSphere MQ.
// set as a retained publication
msg.setIntProperty(JmsConstants.JMS_IBM_RETAIN, JmsConstants.RETAIN_PUBLICATION)
How do you get lost messages when MQ Server was down?
How do you make sure messages should not lost event though MQ is down?
JMS messages can be delivered in two modes: Persistent and Non-persistent modes. In persistent mode, messages are stored to database/permanent storage so that the message will not get lost in case of provider shutdown/failure. With persistent mode, messages can be retrieved when the provider goes back to service. The default mode is Persistent mode. You can use setDeliveryMode method in MessageProducer class to set the JMS delivery mode. For more details please refer to:
http://activemq.apache.org/what-is-the-difference-between-persistent-and-non-persistent-delivery.html
When MQ server is down, nothing can be done, neither put nor get messages. To get or put messages, MQ server must be up and running.
Messages are of two types, Persistent and non-persistent messages. Non-persistent messages are lost when the MQ server goes down. These can not be recovered. Persistent messages are not lost even if the server goes down as they are saved to a storage and retrieved when server comes up. In WebSphere MQ world, there is a third type, Semi-persistent. This type of messages are not lost if the server is shutdown gracefully.
You can read more on WebSphere MQ here: http://www.redbooks.ibm.com/redpapers/pdfs/redp0021.pdf and InfoCenter: http://pic.dhe.ibm.com/infocenter/wmqv7/v7r5/index.jsp
What is the difference between a Websphere Message Broker and a Queue Manager. I guess the queue manager puts messages in the queue, takes messages out of the queue, moves messages to backout queues etc. So what is the job of the broker?
Does it sit between the publisher and the Queue Manager or between the consumer and the Queue Manager?
Websphere MQ is a software which uses the AMQ(Asynchronous messaging protocol). You can achieve asynchronous messaging between your applications via Websphere MQ, which will make your infrastructure loosely coupled(Applications can keep working even though other applications are down in the infrastructure).
But the applications in your infrastructure may not be able to understand each others' message formats, and hence just sending the message to the target application may not be enough. You may require transformation of the message.
You can do it by writing your own program using the Websphere MQ API.
Your program should be able to do the below things:
Pick message from a specific queue (using MQGET)
Should be able to understand the message. That is say it's an XML message. Then your program must be able to parse the XML and read the
data in it.
After reading the input message you will make your output message based on the requirements.
Then you will either publish the message or put the message in some specific queue(say TargetQ), so that the target application can get
the message. Target application will then get the message either by
issuing MQGET on the TargetQ or subscribing to the topic which was
published from your application.
But writing your own program will take a lot of development time and effort and also may be a bit complex.
So, IBM provided its own software to do the job, which is "Websphere Message Broker".
WMB allows you to create programs very easily and a lot faster.
Appropriate nodes in WMB will do all above steps for you. In fact it provides lot many features than the above steps.
Websphere MQ still doesn't have an HTTP listener. But, a message broker does. It allows you to host web services and have HTTP based flows etc that too in a secure way(Supports SSL).
MQ is providing you the infrastructure for messaging: queues and topics - IBM MQ
IBM Integration Bus (formerly known as WebSphere Message Broker) allows you to apply the common EAI patterns, e.g. Routing, Transformation
Hope that helps.
Best,
Patrick
I want to add just two points: Message Broker (now IIB) includes a set of optimized and fast parsers (XML, CSV, etc) and useful mapping nodes (msg-msg, msg-db). MQ is also used for internal configuration messages coming from the Configuration Manager.
WebSphere MQ is a solution for application-to-application communication services regardless of where your applications or data reside. Whether on a single server, separate servers of the same type, or separate servers of different architecture types, WebSphere MQ facilitates communications between applications by sending and receiving message data via messaging queues. Applications then use the information in these messages to interact with Web browsers, business logic, and databases. WebSphere MQ provides a secure and reliable transport layer for moving data unchanged in the form of messages between applications but it is not aware of the content of the messages. WebSphere MQ uses a set of small and standard application programming interfaces (APIs) that support a number of programming languages, including Visual Basic, NATURAL, COBOL, Java, and C across all platforms.
WebSphere Message Broker is built to extend WebSphere MQ, and it is capable of understanding the content of each message that it moves through the Broker. Customers can define the set of operations on each message depending on its content. The message processing nodes supplied with WebSphere Message Broker are capable of processing messages from various sources, such as Java Message Service (JMS) providers, HyperText Transfer Protocol (HTTP) calls, or data read from files. By connecting these nodes with each other, customers can define linked operations on a message as it flows from one application to its destination.
Message Broker can do the following:
Matches and routes communications between services
Converts between different transport protocols
Transforms message formats between requestor and service
Identifies and distributes business events from disparate sources
Together, WebSphere MQ and WebSphere Message Broker deliver a comprehensive publish and subscribe facility, connecting Message Broker’s broad transport and format support to WebSphere MQ’s messaging backbone. WebSphere Message Broker extends the WebSphere MQ publish and subscribe functionality with advanced function such as content-based publish and subscribe by means of an enhanced Publication node. The two products share a common publish and subscribe domain for topic- and content-based operations
MQ is mainly for the transforming the messages from on system to another system.
WMB(websphere message broker) will sit between QMGR's and transforming message along with change content of the message format as per the system requirement/Business Logic implementation.
Srinu D