I have two applications SystemA and SystemB communicating via request and response messages and using correlation IDs.
I would like to be able to test SystemA in isolation without having to rely on the availability of our IBM MQ broker and SystemB. Ideally I would like to do it in our continuous integration environment (Jenkins). I am looking for common approaches to this problem, before I code a solution myself in-house.
How can I create a mockup of the IBM MQ broker and SystemB for the purpose of testing SystemA in isolation?
Why mock the queue? Have you considered mocking the consumer from the queue instead? This would be simpler and would leave the queues in place for when the full featured service became available.
If you are looking for both basic and advanced mocking and simulation for developers and testers and would like to do black box testing have a look at Traffic Parrot and this mocking and simulating JMS IBM WebSphere MQ tutorial. It will work in Jenkins as well.
If you do not need to do black box testing, and can mock stuff in unit tests, just use mockito or a similar java mocking library.
Related
Java.
I had to port away from the Proton MQ library to the IBM one. I had written a dummy MQ server to run jUnit tests against but I cannot see any way to achieve something similar with IBM MQ. To me it seems that it requires a standalone server and a license. Is that the case? Are there any lightweight alternatives usable for jUnit?
They mention that there is an IVTRun application which is actually just a wrapper over MQJMSIVT.class. Decompiling inside, I don't see anything that can start an actual server/queue manager. See this: https://www.ibm.com/docs/en/ibm-mq/9.2?topic=jms-point-point-ivt-mq-classes
You can run MQ natively, in a container or in the cloud. The container option lends itself to automated testing as the set up and start can be automated.
For detailed instructions see https://developer.ibm.com/series/mq-ready-set-connect/
We are designing a solution that will consume messages from IBM MQ using JMS. The plan is to use WAS Liberty, so JMS is the technology of choice. We will create Message-Drive beans that will listen for messages in MQ queues.
We are considering both WAS Liberty and OpenLiberty as well.
The trick here is that we must implement it with fail-over, so that if one of our server fail, the other will keep consuming messages from MQ automatically. Like in a ative/passive mechanism.
I'm aware that the MQ adapter needs to be installed as it is not provided out-of-the-box.
I have the following questions:
Does WAS Liberty messaging implementation supports fail-over? Meaning that if the ative message consumer node fails, the stand-by node will automatically migrate and start consuming messages from MQ? What about OpenLiberty?
How can I configure the message system to work that way? Can you point out to the documentation?
Or is this feature only provided by WebSphere?
There is no such functionality in WebSphere Liberty or Open Liberty yet. You can create RFE here https://www.ibm.com/developerworks/rfe/?PROD_ID=544 .
There are ways to do it manually, check these links:
JMS Activation spec on Liberty: “WAS_EndpointInitialState” full profile equivalent property?
Controlling the state of endpoints at runtime
Solution that you could do:
create a script/application that will monitor your servers and call that API to enable/disable endpoint in specific server
or use Dynamic cluster/ auto scaling feature of Liberty and divide you app to two clusters - one with MDBs, one without. And then define policy that MDBs cluster has 1 instance always available. So once the server dies it is automatically restarted somewhere in the cluster
or use Kubernetes/ICP platform in the same way - so deploying 2 versions of app, and defining different replicasets parameters.
I am not expert in JMS and recently came across a situation in my project where they wanted to use JMS to communicate between applications. Is it possible to make use of JMS without an underlying MQ software like Websphere MQ / TIBCO EMS to communicate asynchronously. My thinking is that just like we use JDBC API to speak to a physical Database product (like Oracle/DB2 DB) underneath, we would use JMS to speak to a physical queue. But my team says just JMS and no queue. Can anyone help me understand? We use Websphere 6.0 server and use spring framework.
Thanks in advance.
You are correct: JMS is solely an interface, it contains no implementation. The comparison with JDBC is quite fitting.
When using JMS you always need a JMS provider. A provider is the JMS implementation, which interfaces a message oriented middleware.
Most application servers have built in JMS providers, so maybe your team meant using JMS without an external provider.
Is it possible to make use of JMS without an underlying MQ software like Websphere MQ / TIBCO EMS to communicate asynchronously
This makes no sense. Few points to note -
JMS is just specs. By specs you can imagine it as set of APIs/interfaces/ method signatures and governing rules.
There are various organization/companies that provide implementation of these APIs.Apache's ActiveMQ,IBM's Websphere MQ or TIBCO EMS are some of the examples.
Now when you say I want to use JMS then you need a JMS compatible server i.e server which understands the rules laid down by specs. This is generally know as MQ server and is provided by the provider.
My thinking is that just like we use JDBC API to speak to a physical Database product (like Oracle/DB2 DB) underneath, we would use JMS to speak to a physical queue. But my team says just JMS and no queue
Two things to note here. When you say you want to use databases and JDBC APIs to communicate JMS comes no where in the picture. Please understand why JMS is used. In short it is used to transfer messages between two decoupled system. What storage type JMS server uses(and if it is configurable) will depend on the provider. Generally all have DB storage type and can be configured to use your DB.
Next JMS has two types of communications - PTP(peer to peer) which uses queues and PUBSUB(publish subscribe) which uses Topics. When you say you are using JMS everything boils down to either of these two and some of it's variants.Now when you say JMS without a queue perhaps your team meant using PUBSUB. But again it is not something you decide randomly and use it in your application. This decision is takes as per what your requirement is as both of them behave differently.
First you will need queues/topics to use JMS.
Second (your team probably means not using WebSphere MQ because my team was saying the same thing)
This diagram in IBM's Red Book will fully answer your question (it is the same as WS6):
http://www.redbooks.ibm.com/redbooks/pdfs/sg247770.pdf
page 4.
Probably what you will be using is the WebSphere Default Messaging Provider.
For me JMS and ESB seem to be very related things and I'm trying to understand how exactly they are related.
I've seen a sentence that JMS can be used as a transport for ESB - then what else except the transport should be present in such an ESB? Is JMS a simple ESB or if not, then what it lacks from the real ESB?
JMS offers a set of APIs for messaging: put a message on a queue, someone else, sometime later, perhaps geographically far away takes the message off the queue and processes it. We have decoupled in time and location of the message provider and consumer. Even if the message consumer happens to be down for a time we can keep producing messages.
JMS also offers a publish/subscribe capability where the producer puts the message to a "topic" and any interested parties can subscribe to that topic, receiving messages as and when they are produced, but for the moment focus just on the queue capabilty.
We have decoupled some aspects of the relationship between provider and consumer. However some coupling remains. First, as things stand every message is processed in the same way. Suppose we want to introduce different kinds of processing for different kinds of messages:
if ( message.customer.type == Platinum )
do something special
Obviously we can write code like that, but an alternative would be to have a messaging system that can send different messages to different places we set up three queues:
Request Queue, the producer(s) puts their requests here
Platinum Queue, platinum consumer processing reads from here
Standard Queue, a standard consumer reads messages from here
And then all we need is a little bit of cleverness in the queue system itself to transfer then messsage from the Request Queue to the Platinum Queue or Standard Queue.
So this is a Content-Based Routing capability, and is something that an ESB provides. Note that the ESB uses the fundamental Queueing capabilities offered by JMS.
A second kind of couppling is that the consumer and producer must agree about the message format. In simple cases that's fine. But when you start to have many producers all putting message to the same queue you start to hit versioning problems. New message formats are introduced but you don't want to change all the existing providers.
Request Version 1 Queue Existing providers write here
Request Version 2 Queue New provider write here, New Consumer Reads here
And the ESB picks up the Version 1 Queue messages and transforms them into Version 2 messages and puts them onto the Version 2 queue.
Message transformation is another possible ESB capability.
Have a look at ESB products, see what they can do. As I work for IBM, I'm most familiar with WebSphere ESB
I would say ESB is like a facade into a number of protocals....JMS being one of them.
An addition to the above list is the latest Open Source ESB - UltraESB
JMS is not well suited for the integration of REST services, File systems, S/FTP, Email, Hessian, SOAP etc. which are better handled with an ESB that supports these types natively. For example, if you have a process that dumps a CSV file of 500MB at midnight, and you want another system to pickup the file, parse CSV and import into a database, this can easily be accomplished by an ESB - whereas a solution with just JMS will be bad. Similarly, integration of REST services, with load balancing/failover to multiple backend instances can be done better with an ESB supporting HTTP/S natively.
This Transformation does not happen automatically. You need to configure the mapping or write transformation service
Look at https://access.redhat.com/knowledge/docs/en-US/JBoss_Enterprise_SOA_Platform/4.2/html/SOA_ESB_Message_Transformation_Guide/ch02s03.html
Regards,
Raja Nagendra Kumar,
C.T.O
www.tejasoft.com
ESB offers integration with a lot of different protocols in addition to JMS.
Most use JMS behind the scenes to transfer, stor and move messages. One such solution OpenESB, uses XML format messages.
There are open source ESB which you could checkout -
OpenESB
Apache Camel
MuleESB
WSO2 ESB
JMS implementation like ActiveMQ come with Camel inbuilt into them.
JMS is a protocol for communicating with an underlying messaging layer. ESB operates at a higher level, offering integration with multiple technologies and protocols, one of which would be JMS, in a uniform way that makes management of complex flows much simpler.
There are JMS message brokers , that you can easily configure with ESB. https://docs.wso2.com/display/ESB470/JMS+Transport
JMS and ESB both provide a way of communication between different applications. But the context for JMS and ESB are different. JMS is for simple need. JMS is implemented by JMS Provider. It is Java specific.
Examples of JMS Providers are: Apache Active MQ, IBM MQ, HornetQ etc.
ESB is for complex need. ESB is a component in EAI providing communication facility to various applications. It is generic & not specific to Java. JMS is one of the supported protocols.
Examples of ESB provider are: MuleESB, Apache Camel, OpenESB
Use Case: It may be an overhead to use ESB, if all our communicating applications are in Java and are using the same message format. Here JMS may be sufficient.
How does RabbitMQ compare to Mule, I am going to build an application using message oriented architecture and AMQP (RabbitMQ) provides everything i want, but i am perplexed with so many related technology choice and similar concepts like ESB. I am having a doubt if i am making a choice without considering other alternatives.
I am mostly clear that RabbitMQ is a message broker and it helps me in mediating message between producer and consumer (all forms or publish subscribe and i could understand how its used from real examples like twitter , or Facebook updates, etc)
What is Mule, if i could achieve what i do in RabbitMQ using mule, should i consider mule similar to RabbitMQ?
Does mule has a different objective than that of a message broker?
Does mule assumes that underlying it there is a message broker that delivers message to the appropriate mule listeners (i could easily write a listener in RabbitMQ)
Is mule a complete Java bases system ( The current experiment i did with RabbitMQ took me less than 30 Min to write a simple RPC Client Server with client as C# and Server as Java , will such things be done in Mule easily).
Mule is an ESB (Enterprise Service Bus). RabbitMQ is a message broker.
An ESB provides added layers atop of a message broker such as routing, transformations and business process management. It is a mediator between applications, integrating Web Services, REST endpoints, database connections, email and ftp servers - you name it. It is a high-level integration backbone which orchestrates interoperability within a network of applications that speak different protocols.
A message broker is a lower level component which enables you as a developer to relay raw messages between publishers and subscribers, typically between components of the same system but not always. It is used to enable asynchronous processing to keep response times low. Some tasks take longer to process and you don't want them to hold things up if they're not time-sensitive. Instead, post a message to a queue (as a publisher) and have a subscriber pick it up and process it "later".
Mule is a "higher level" service implemented with message broker. From the docs
The messaging backbone of the ESB is
usually implemented using JMS, but any
other message server implementation
could be used
You can build an ESB with rabbit; however, you're going to be limited to sending byte[] packages, and you'll have to build your system out of messaging primitives like topics and queues. It might be a bit faster (based on absolutely no benchmarking, testing or data) because there are fewer layers of translation. Mule provides an abstraction on top of this, speaks a variety of transports, and can handle some routing logic.
Mule is a Enterprise service bus providing end to end integration solution where as Rabbit is message broker for queueing messages between subscriber and receiver.
RabbitMQ, a open source message broker software is written in Erlang programming language and is built on Open Telecom Platform for clustering and failover. It is easy to use, supports a huge number of developer platforms and runs on all major operating systems. It works on a concept called Exchange.
Mule connects RabbitMQ with AMQP connector.