We are creating new application, which is going to use IBM's MQ as a JMS provider for a short term and switch to Tibco EMS within a year.
My question is how involving the changes would be from the Application code perspective.
So far reading from JMS documentation, my impression is it should only require minimal changes. Does anyone have experience with it and provide some input on the work involved in switching between JMS provider.
I've done POCs where I swapped out connection factories and used WMQ JMS Classes to send to various providers (TIBCO, ActiveMQ, etc), to prove out the interchangeability. I've also done full swaps from one vendors JMS to another. In theory, it should be very simple.
The biggest change will be with the connection factories. Everything JMS specific will be the same between providers. The more tightly coupled the code is to the connection factories, the more complex it will be to change the app itself. Outside of this, you may need to change vendor specific implementation of objects, such as MQQueue vs Queue.
One addition thing to keep in mid is dependent on the IBM endpoints. If you are using "target queue managers" on any producers, these will need to change. WMQ has a specific URI to reach queues on specific Queue Managers in a cluster ( "queue://target_qm/queue_name/" ). If any application uses this URI it will need to ensure it makes the proper changes here as well.
Related
I am trying to understand JMS.
What is the difference between ActiveMQ and JMS
can pool the data from NON ActiveMQ with ActiveMQ plugin in Spring?
Thanks ,In advance
JMS is a specification. JMS has three main parts to it. The first is the producer, which is nothing more than a bean that submits a "message" to a JMS broker (#2) (the system that manages messages between producers and consumers). In this case, ActiveMQ is the broker. Once the broker receives a message, the consumer (#3), or Message-Driven Bean (MDB), processes the message.
If you want to work with JMS, you'll just write both your producer/consumer code using the JMS API, but behind the scenes there is a "resource adapter" that is a special ActiveMQ driver that will connect to an ActiveMQ instance and do the management for you.
Have a look at this post I made recently. I'm still trying to figure out the best way to write JMS beans, but I've got the basics down.
The accepted answer emphasizes what is the structure of JMS is. Not disagreeing just want to add to it in case anyone else wants to know. ActiveMQ could be a JMS supplier. A JMS supplier shapes the computer program system for encouraging the utilize of JMS concepts interior an application. A single node of ActiveMQ which permits clients to associate to it and utilize these informing concepts is called an "ActiveMQ Broker."
Enterprises feel this disparity with business actions such as mergers and acquisitions. This creates the need to maintain an increasingly heterogeneous collection of business applications. As your enterprise grows, so does the need to allow all of these platforms to share data. A number of architectural patterns exist today which help to solve this problem.
Some other examples of JMS providers are:
HornetQ.
RabbitMQ.
SonicMQ.
Winsows Azure Messaging
The following example shows a simple configuration of an ActiveMQ connection:
<jms:config name="JMS_Config">
<jms:active-mq-connection >
<jms:factory-configuration brokerUrl="tcp://localhost:61616" />
</jms:active-mq-connection>
</jms:config>
This post explains a detailed difference between the ActiveMQ and JMS (or maybe about the details of their specifications). Hope it clears your concepts.
I would like to deploy ActiveMQ in order to connect and communicate with other MOM implementations of jms such as WebsphereMQ, Tibco EMS, Oracle WebLogic JMS etc.
As far as I have researched online this seems to be possible by building a so-called bridge on top of activemq that will be able to communicate with the other end, regardless of the underlying wire protocol each jms implementation uses.
Examples though are scarce and often times minimal and so a point of confusion is whether I need to include jms client jars from each such implementation (Weblogic, WebsphereMQ, TIBCO EMS etc) in my classpath and bridge configuration. Can this be done without any such jars?
Since the goal is to connect to separate organizations MOMs I would like to avoid using any 3rd party, potentially licensed, jar.
...a point of confusion is whether I need to include JMS client jars
from each such implementation (Weblogic, Websphere MQ, TIBCO EMS etc)
in my classpath and bridge configuration. Can this be done without any
such jars?
TL;DR: You need the vendor's proprietary classes.
JMS defines the API and leaves the wire format and other implementation details to the transport vendor. Furthermore, in the proprietary implementations the wire formats can change significantly version to version. (One of the advantages of proprietary code is the ability to make such large structural changes when doing so improves performance, reliability or function.)
The one place where two JMS implementations have commonality is when the messages are in memory. Since all of the transport vendor's implementations inherit from the same Java JMS classes, it is usually possible to instantiate message objects from two different transport vendors and copy from one to the other. To do this, you must have the jars of each supported transport provider.
In fact, this is how most JMS bridge products that I've used work. The bridge code uses fully-qualified class names to reference the classes from two different vendors. It then creates two connection factories, one for each transport, and two different destinations, also one for each transport. It then reads a message from a queue on one provider, copies it to the message object on the other transport provider, then writes the message.
There are of course many "gotchas" with this approach. I'll list a few of the obvious ones:
Mapping destinations across providers is manual.
Identity propagation is not enforceable and requires the bridge to have quasi- or full-admin rights.
Transactionality across the providers is at best difficult, at worst impossible.
Correlation identifiers must be mapped by the bridge if they are mapped at all. This makes the bridge more complex and requires it to use a database or other persistent storage.
The vendor-specific classes often have a superset of the attributes and methods specified by the JMS spec. If used, these are stripped away during the copy.
Due to message segmentation, grouping and other affinities, there may not be a 1:1 correlation across the bridge.
Due to differences in implementation of such things as high availability, the physical network structure of one transport provider may not map 1:1 to that of another transport provider. This tends to limit the bridge function to the simplest common denominator of the two transports.
The issues raised by a general-purpose bridge include all of the issues of a 2-party bridge, raised as the exponent of the number of transports supported. So if a 2-transport bridge is of difficulty x, then for planning purposes assume a 5 transport bridge is of difficulty (x)**5.
That's not to say "don't code a bridge" since these turn out to be very useful. Just be aware, it's not a trivial task.
I am afraid that's not possible at least on the IBM MQ end. JMS is just an API and each provider is free to implement it the way they see it fits (as you rightly said its not a wire protocol). provider jars are essential to connect to a JMS product.
We have concept called Virtual destinations, try that
I am not expert in JMS and recently came across a situation in my project where they wanted to use JMS to communicate between applications. Is it possible to make use of JMS without an underlying MQ software like Websphere MQ / TIBCO EMS to communicate asynchronously. My thinking is that just like we use JDBC API to speak to a physical Database product (like Oracle/DB2 DB) underneath, we would use JMS to speak to a physical queue. But my team says just JMS and no queue. Can anyone help me understand? We use Websphere 6.0 server and use spring framework.
Thanks in advance.
You are correct: JMS is solely an interface, it contains no implementation. The comparison with JDBC is quite fitting.
When using JMS you always need a JMS provider. A provider is the JMS implementation, which interfaces a message oriented middleware.
Most application servers have built in JMS providers, so maybe your team meant using JMS without an external provider.
Is it possible to make use of JMS without an underlying MQ software like Websphere MQ / TIBCO EMS to communicate asynchronously
This makes no sense. Few points to note -
JMS is just specs. By specs you can imagine it as set of APIs/interfaces/ method signatures and governing rules.
There are various organization/companies that provide implementation of these APIs.Apache's ActiveMQ,IBM's Websphere MQ or TIBCO EMS are some of the examples.
Now when you say I want to use JMS then you need a JMS compatible server i.e server which understands the rules laid down by specs. This is generally know as MQ server and is provided by the provider.
My thinking is that just like we use JDBC API to speak to a physical Database product (like Oracle/DB2 DB) underneath, we would use JMS to speak to a physical queue. But my team says just JMS and no queue
Two things to note here. When you say you want to use databases and JDBC APIs to communicate JMS comes no where in the picture. Please understand why JMS is used. In short it is used to transfer messages between two decoupled system. What storage type JMS server uses(and if it is configurable) will depend on the provider. Generally all have DB storage type and can be configured to use your DB.
Next JMS has two types of communications - PTP(peer to peer) which uses queues and PUBSUB(publish subscribe) which uses Topics. When you say you are using JMS everything boils down to either of these two and some of it's variants.Now when you say JMS without a queue perhaps your team meant using PUBSUB. But again it is not something you decide randomly and use it in your application. This decision is takes as per what your requirement is as both of them behave differently.
First you will need queues/topics to use JMS.
Second (your team probably means not using WebSphere MQ because my team was saying the same thing)
This diagram in IBM's Red Book will fully answer your question (it is the same as WS6):
http://www.redbooks.ibm.com/redbooks/pdfs/sg247770.pdf
page 4.
Probably what you will be using is the WebSphere Default Messaging Provider.
Please consider the scenario as shown in the attached image :
The Portal(producer) will send some message to the bus to which has to be processed by multiple applications(consumer) – PAYROLLAPP, HELPDESK etc.
Multiple instances of consumer applications may be running, also these instances can be added/
removed dynamically
Now, it is critical to ensure that message is processed only once, per application i.e if
PAYROLLAPP -1 processes the message, PAYROLLAPP -2 should NOT process it; of course, in the
above diagram, HELPDESK – 1 must process it. In short, in case of multiple instances, exactly one
must process the message, once
When I searched for answers, most of the stuff was about creating a 'selective consumer' - a consumer that accepts/rejects a message based on some logic - please note that no changes/additions/wrapping can be done for the applications shown in the diagram; the logic has to reside somewhere in the provider that manages the bus
Please guide about the same.
Adding more details after Petter's answer :
The items to the to the left of the left-dotted line are the 'approaches' - Pure JMS,ESB,EAI
The items to the to the right of the right-dotted line are the 'implementations'
Now, the big part - QUERIES :
Irrespective of the solution(pure JMS, ESB, EAI), does the part
below the horizontal dotted line(application-specific queues) needs
to be implemented?
How does the usage of ESB(JBoss ESB etc.), instead of ‘pure’ JMS(Active MQ etc.), help/
hamper? Does ESB provide any advantage over JMS which is ‘java-only’(?). I am hell confused
– ‘ESB or JMS’, even after referring threads like these : JMS and ESB - how they are related?.
It has one reply which says “JMS is not well suited
for the integration of REST services, File systems, S/FTP, Email, Hessian, SOAP etc. which are
better handled with an ESB that supports these types natively. For example, if you have a process
that dumps a CSV file of 500MB at midnight, and you want another system to pickup the file,
parse CSV and import into a database, this can easily be accomplished by an ESB - whereas a
solution with just JMS will be bad. Similarly, integration of REST services, with load balancing/
failover to multiple backend instances can be done better with an ESB supporting HTTP/S
natively.” It only added to my confusion !!!
Is the usage of EAI framework (Apache Camel etc.) an approach entirely different from the pure
JMS or ESB approach? If yes, how and what are the pros/cons?
I was told that ESB alone won’t help, BPM(or something else?) needs to be used to define and
store the ‘routing’ logic – is this true?
I see the point. This might be a bit tricky with "pure" JMS.
What you essentially want to do is to let the portal publish messages to a topic, but not let the PAYROLLAPPs subscribe to that topic (since all of them would get a copy of the message). So what you would need is some logic in between that distributes the message from the topic subscription to one queue per application type. From that queue, normal load balanced (the competing consumer pattern) can be implemented with JMS.
Different JMS providers have special implementations that can acomplish this task
ActiveMQ has its Virtual Destinations, WebSphere MQ has its server side subscriptions that can subscribe from a topic to a queue. In the case your JMS provider does not have any way to handle this, you might want to look at adding some routing middleware to your topology. Apache Camel is a nice, lightweight one, but there are lots of others that can setup some routing in the middle without affecting the real applications.
Update for detailed questions
The Queues below the line has to be there for sure (if your applications uses messaging). The "Some distrib. logic" box shouldn't be needed. The "Some routing logic" box could be an ESB or in this very case, be implemented in the messaging server, for instance ActiveMQ with virtual destinations (or WebSphere MQ or perhaps RabbitMQ among others).
There are a lot of buzwords in the domain of integration. Simplified (depending on who you ask - ESB can also be seen as an architectural pattern, but let's keep it simple), an ESB is a server application (or a topology of multiple servers in practice) that is a centerpiece of an integration landscape. The ESB server simply contain logic and small message flows that takes messages (files, or whatever) from one application and routes them to many applications, transform them to other formats, encrypts, converts from one transport protocol such as HTTP/SOAP to File etc.
JMS is a rather confusing and missused word. Java has to some extent dominated the enterprise messaging domain in the last years, so JMS is sometimes used pretty much as a synonym to Messaging. However, Messaging, (or message queueing, asynchronous messaging, MOM=message oriented middleware, etc.) is to be simply considered as a family of similar transport protocols that features a central relaying server. Is is not at all a Java only thing. Many successful ESBs setups I worked with actually leverage on a Messaging backbone
In your situation, I would not go too deep into the academical/philosophical differences between ESB and EAI software. They will most likely do pretty much the same things for you. Instead, look at the hard facts such as price, support, resource footprint, monitoring, tech. features, learning curve etc. Be it Camel/ServiceMix, Mule, JBoss ESB, Microsoft BizTalk, IBM Message Broker, Tibco etc.
Hah! Was it perhaps a salesman? An ESB will do just fine. A Messaging server will do in your case as well, such as ActiveMQ as has been pointed out already. BPM suits are fine for orchestrating semi-automatized business processes or if there is major business logic in the integration layer. Otherwise, avoid that added complexity.
Irrespective of the solution(pure JMS, ESB, EAI), does the part below the horizontal dotted line(application-specific queues) needs to be implemented?
The consumers need to be implemented in such a way that work with you chosen solution but you shouldn't have to worry about the creation of the queue per consumer or the distribution logic (assuming that the consumers can consume directly from the chosen tech)
How does the usage of ESB(JBoss ESB etc.), instead of ‘pure’ JMS(Active MQ etc.), help/ hamper? Does ESB provide any advantage over JMS which is ‘java-only’(?). I am hell confused – ‘ESB or JMS’, even after referring threads like these : JMS and ESB - how they are related?. It has one reply which says “JMS is not well suited for the integration of REST services, File systems, S/FTP, Email, Hessian, SOAP etc. which are better handled with an ESB that supports these types natively. For example, if you have a process that dumps a CSV file of 500MB at midnight, and you want another system to pickup the file, parse CSV and import into a database, this can easily be accomplished by an ESB - whereas a solution with just JMS will be bad. Similarly, integration of REST services, with load balancing/ failover to multiple backend instances can be done better with an ESB supporting HTTP/S natively.” It only added to my confusion !!!
My opinion is that ESB would overcomplicate this solution. It's designed (amongst other things) to assist integration with different technologies, but simpler solutions do this too - e.g - Apache Camel provides a very easy way of communicating using a huge variety of transports (including ActiveMQ).
Not all JMS implementations cater for connectivity from other languages, but ActiveMQ does using it's STOMP connector.
Is the usage of EAI framework (Apache Camel etc.) an approach entirely different from the pure JMS or ESB approach? If yes, how and what are the pros/cons?
Apache Camel and JMS are complementary technologies, as are JMS and ESB. Camel (& Spring Integration) are lightweight, simple and portable. ESB's are much more heavyweight and will normally lead to greater coupling with the ESB/application server.
I was told that ESB alone won’t help, BPM(or something else?) needs to be used to define and store the ‘routing’ logic – is this true?
It depends what your 'routing' logic is, it looks to me like you don't require routing logic, you just require guaranteed delivery to 1payroll consumer and 1 helpdesk consumer. BPM would be more useful where you want to selectively public data/invoke a service based on some characteristic of that data.
I strongly suggest reading http://activemq.apache.org/virtual-destinations.html, using these you would:
Send messages to the ActiveMQ broker, onto a VirtualTopic, e.g. VirtualTopic.X
Register the Payroll and Helpdesk consumers, as consumers on queues that ActiveMQ dynamically creates on the topic - e.g. Consumer.Payroll.VirtualTopic.X. Both Payroll consumers should be registered with the same string.
ActiveMQ will automatically retain a marker that represents what each set of consumers hasn't consumed. This means that 100% of messages will be processed by a Payroll consumer but a message will never be sent to > 1 payroll consumer.
Add/remove consumers at will.
N.B.I believe that other products, e.g. Apache QPID provide similar functionality - I'm just most aware of ActiveMQ, and have had success with this approach
Want to know how to create physical queue in JMS at run time.
when I search for this I got Creating JMS Queues at runtime
But when I read http://activemq.apache.org/how-do-i-create-new-destinations.html I come to know queue which mention in Creating JMS Queues at runtime is not creating any physical queue at server side.
Please correct me if I m wrong. If any one know to create physical queue at run time please replay.
Thanks in advance.
The creation of "normal" queues is not adressed by the JMS standard. Depending on what you want to do there are two approaches:
use temporary queues -> however they have many restrictions, most commonly they are used forrequest-reply scenarios
use the API of the JMS provider - however your solution will be depending on this specific provider then
The JMS standard only addresses sending and receiving data from objects like queues and topics. Creation of JMS artefacts is vendor specific and most often requires using:
1)specific vendor APIs (not JMS)
2)command/admin messages aimed at the JMS server (command agents on activemq)
3)JMX API
I have used JMX method, which is the most powerful, but also the most work.
JMX Method for activemq (version 5.0+)
a) JMS Server Setup
1) Enable JMX in activemq startup scripts and activemq.xml files
2) If you are authenticating to to the server, make sure your user has admin privileges setup in activemq.xml (see http://activemq.apache.org/security.html)
3)restart activemq server
b) Your Client Code
1) create an instance of org.apache.activemq.broker.jmx.BrokerViewMBean (you will need to connect with some JMX connectivity code which is a bit messy)
2) use its addQueue method. This will create a queue on the server
(The process is similar for hornetq but since you mentioned Activemq I have omitted hornetq details here.)
I have used this method myself and it works.
An alternative is to use Command Agents in Activemq, but I have no personal experience with these. These are special messages contain admin commands and may do what you want as well.