How does RabbitMQ compare to Mule, I am going to build an application using message oriented architecture and AMQP (RabbitMQ) provides everything i want, but i am perplexed with so many related technology choice and similar concepts like ESB. I am having a doubt if i am making a choice without considering other alternatives.
I am mostly clear that RabbitMQ is a message broker and it helps me in mediating message between producer and consumer (all forms or publish subscribe and i could understand how its used from real examples like twitter , or Facebook updates, etc)
What is Mule, if i could achieve what i do in RabbitMQ using mule, should i consider mule similar to RabbitMQ?
Does mule has a different objective than that of a message broker?
Does mule assumes that underlying it there is a message broker that delivers message to the appropriate mule listeners (i could easily write a listener in RabbitMQ)
Is mule a complete Java bases system ( The current experiment i did with RabbitMQ took me less than 30 Min to write a simple RPC Client Server with client as C# and Server as Java , will such things be done in Mule easily).
Mule is an ESB (Enterprise Service Bus). RabbitMQ is a message broker.
An ESB provides added layers atop of a message broker such as routing, transformations and business process management. It is a mediator between applications, integrating Web Services, REST endpoints, database connections, email and ftp servers - you name it. It is a high-level integration backbone which orchestrates interoperability within a network of applications that speak different protocols.
A message broker is a lower level component which enables you as a developer to relay raw messages between publishers and subscribers, typically between components of the same system but not always. It is used to enable asynchronous processing to keep response times low. Some tasks take longer to process and you don't want them to hold things up if they're not time-sensitive. Instead, post a message to a queue (as a publisher) and have a subscriber pick it up and process it "later".
Mule is a "higher level" service implemented with message broker. From the docs
The messaging backbone of the ESB is
usually implemented using JMS, but any
other message server implementation
could be used
You can build an ESB with rabbit; however, you're going to be limited to sending byte[] packages, and you'll have to build your system out of messaging primitives like topics and queues. It might be a bit faster (based on absolutely no benchmarking, testing or data) because there are fewer layers of translation. Mule provides an abstraction on top of this, speaks a variety of transports, and can handle some routing logic.
Mule is a Enterprise service bus providing end to end integration solution where as Rabbit is message broker for queueing messages between subscriber and receiver.
RabbitMQ, a open source message broker software is written in Erlang programming language and is built on Open Telecom Platform for clustering and failover. It is easy to use, supports a huge number of developer platforms and runs on all major operating systems. It works on a concept called Exchange.
Mule connects RabbitMQ with AMQP connector.
Related
What exactly is Solace JCSMP?
Is it just a more suited JMS API for Solace? What kind of other benefits does it have?
For example, I would need to create a Java application using SolaceMQ that needs to send/receive messages over to/from AMQP protocol so that all different microservices written in different languages would be able to consume them. Is JCSMP right for that? or is it just another JMS API that it only works between Java applications?
What exactly is Solace JCSMP?
JCSMP is the Classic Java API for Solace's SMF (Solace Message Format) protocol. Note that Solace recently introduced a more modern messaging API for java devs using their event brokers that is just referred to as the "PubSub+ Messaging API for Java". If you are new to Solace and using Java it may make sense to use that instead of JCSMP.
Is it just a more suited JMS API for Solace? What kind of other benefits does it have?
JCSMP allows developers to take advantage of the full feature set offered by the Solace Broker whereas the Solace JMS implementation only covers the features defined in the JMS specification itself. For example, use of Solace features such as Replay or having Queues subscribe to Topics are not programmatically possible via the JMS API and would have to be done administratively if using JMS.
For example, I would need to create a Java application using SolaceMQ that needs to send/receive messages over to/from AMQP protocol so that all different microservices written in different languages would be able to consume them. Is JCSMP right for that? or is it just another JMS API that it only works between Java applications?
Solace PubSub+ Event Brokers provide protocol translation between any of the protocols supported by the broker. It doesn't matter whether you are using SMF (which both Solace JMS and JCSMP use), MQTT, AMQP 1.0, etc. or what programming language you are using. For example when you send a message using JCSMP you can receive it using a Java app using JMS, a Python app using AMQP and a Go app using MQTT. The Solace Event Broker even has support for calling a RESTful webhook using it (Check out Solace "Rest Delivery Endpoints"). Just a heads up that if you're using headers you'll want to checkout the Solace docs to see how they're mapped during protocol translation.
Newbee to Spring world. I have some knowledge on ActiveMQ. Recently used in one of my projects. While reading about Spring Events raised a doubt.
Spring Events: Publisher -> Listener. We do publish events and we would have created some listeners for that.
ActiveMQ: Publisher -> Listener. We do publish events and we would have created some listeners for that.
So anyone helps me to understand the use cases or difference between these two APIs.
As far as I can tell, Spring Events are an application level events mechanism, so that different parts inside our application can communicate/coordinate. The scope and functionality appear to be quite narrow and small respectively. You can publish events and deal with those events either synchronously (default behaviour) or asynchronously (using #EnableAsync and #Async). There is no broker. This functionality may be a perfect fit for your application if this is all it needs.
On the other hand, ActiveMQ is a full-featured message broker. Generally speaking, it runs as an independent server process (although it can be embedded in your application). It supports industry-standard protocols like AMQP, MQTT, & STOMP which have client implementations on numerous platforms and in various languages. For example, you could send STOMP messages via Websockets from a Javascript client and process those messages with an AMQP client written in .NET on Windows. It provides both a JMS & JNDI client implementation. It supports both publish-subscribe and point-to-point semantics. You can use it as an integration platform and scale it up to multi-node clusters with high-availability for the message data and thousands of remote clients using various protocols or you can embed it into your application and just use it for local events.
I want to know the major difference between VM and JMS component of Mule ESB. Can someone help me to know it.
As per Mule documentation, VM transport is for intra-JVM communication between Mule flows. So, that means when you use a VM in your flow, you can communicate between different flows in the application.
A flow containing VM inbound cannot be called externally from external application as thus the flow is equivalent to a private flow used within the application. By default uses in-memory queues.
Please go through the documentation :- https://docs.mulesoft.com/mule-user-guide/v/3.8/vm-transport-reference
On the other hand as per Mule documentation, JMS is an external host, allows communication between different components of a distributed application and JMS transport lets you easily send and receive messages to queues and topics for any message service which implements the JMS specification.
A flow, which has JMS inbound can be called from externally unlike VM. Documentation is here :- https://docs.mulesoft.com/mule-user-guide/v/3.8/jms-transport-reference
Within the application, if you send the control from one flow to another flow we use VM.VM can be used as both inbound and outbound.
Outside the application, for example, A application want to send something to B application(external application) there we use JMS.
Please consider the scenario as shown in the attached image :
The Portal(producer) will send some message to the bus to which has to be processed by multiple applications(consumer) – PAYROLLAPP, HELPDESK etc.
Multiple instances of consumer applications may be running, also these instances can be added/
removed dynamically
Now, it is critical to ensure that message is processed only once, per application i.e if
PAYROLLAPP -1 processes the message, PAYROLLAPP -2 should NOT process it; of course, in the
above diagram, HELPDESK – 1 must process it. In short, in case of multiple instances, exactly one
must process the message, once
When I searched for answers, most of the stuff was about creating a 'selective consumer' - a consumer that accepts/rejects a message based on some logic - please note that no changes/additions/wrapping can be done for the applications shown in the diagram; the logic has to reside somewhere in the provider that manages the bus
Please guide about the same.
Adding more details after Petter's answer :
The items to the to the left of the left-dotted line are the 'approaches' - Pure JMS,ESB,EAI
The items to the to the right of the right-dotted line are the 'implementations'
Now, the big part - QUERIES :
Irrespective of the solution(pure JMS, ESB, EAI), does the part
below the horizontal dotted line(application-specific queues) needs
to be implemented?
How does the usage of ESB(JBoss ESB etc.), instead of ‘pure’ JMS(Active MQ etc.), help/
hamper? Does ESB provide any advantage over JMS which is ‘java-only’(?). I am hell confused
– ‘ESB or JMS’, even after referring threads like these : JMS and ESB - how they are related?.
It has one reply which says “JMS is not well suited
for the integration of REST services, File systems, S/FTP, Email, Hessian, SOAP etc. which are
better handled with an ESB that supports these types natively. For example, if you have a process
that dumps a CSV file of 500MB at midnight, and you want another system to pickup the file,
parse CSV and import into a database, this can easily be accomplished by an ESB - whereas a
solution with just JMS will be bad. Similarly, integration of REST services, with load balancing/
failover to multiple backend instances can be done better with an ESB supporting HTTP/S
natively.” It only added to my confusion !!!
Is the usage of EAI framework (Apache Camel etc.) an approach entirely different from the pure
JMS or ESB approach? If yes, how and what are the pros/cons?
I was told that ESB alone won’t help, BPM(or something else?) needs to be used to define and
store the ‘routing’ logic – is this true?
I see the point. This might be a bit tricky with "pure" JMS.
What you essentially want to do is to let the portal publish messages to a topic, but not let the PAYROLLAPPs subscribe to that topic (since all of them would get a copy of the message). So what you would need is some logic in between that distributes the message from the topic subscription to one queue per application type. From that queue, normal load balanced (the competing consumer pattern) can be implemented with JMS.
Different JMS providers have special implementations that can acomplish this task
ActiveMQ has its Virtual Destinations, WebSphere MQ has its server side subscriptions that can subscribe from a topic to a queue. In the case your JMS provider does not have any way to handle this, you might want to look at adding some routing middleware to your topology. Apache Camel is a nice, lightweight one, but there are lots of others that can setup some routing in the middle without affecting the real applications.
Update for detailed questions
The Queues below the line has to be there for sure (if your applications uses messaging). The "Some distrib. logic" box shouldn't be needed. The "Some routing logic" box could be an ESB or in this very case, be implemented in the messaging server, for instance ActiveMQ with virtual destinations (or WebSphere MQ or perhaps RabbitMQ among others).
There are a lot of buzwords in the domain of integration. Simplified (depending on who you ask - ESB can also be seen as an architectural pattern, but let's keep it simple), an ESB is a server application (or a topology of multiple servers in practice) that is a centerpiece of an integration landscape. The ESB server simply contain logic and small message flows that takes messages (files, or whatever) from one application and routes them to many applications, transform them to other formats, encrypts, converts from one transport protocol such as HTTP/SOAP to File etc.
JMS is a rather confusing and missused word. Java has to some extent dominated the enterprise messaging domain in the last years, so JMS is sometimes used pretty much as a synonym to Messaging. However, Messaging, (or message queueing, asynchronous messaging, MOM=message oriented middleware, etc.) is to be simply considered as a family of similar transport protocols that features a central relaying server. Is is not at all a Java only thing. Many successful ESBs setups I worked with actually leverage on a Messaging backbone
In your situation, I would not go too deep into the academical/philosophical differences between ESB and EAI software. They will most likely do pretty much the same things for you. Instead, look at the hard facts such as price, support, resource footprint, monitoring, tech. features, learning curve etc. Be it Camel/ServiceMix, Mule, JBoss ESB, Microsoft BizTalk, IBM Message Broker, Tibco etc.
Hah! Was it perhaps a salesman? An ESB will do just fine. A Messaging server will do in your case as well, such as ActiveMQ as has been pointed out already. BPM suits are fine for orchestrating semi-automatized business processes or if there is major business logic in the integration layer. Otherwise, avoid that added complexity.
Irrespective of the solution(pure JMS, ESB, EAI), does the part below the horizontal dotted line(application-specific queues) needs to be implemented?
The consumers need to be implemented in such a way that work with you chosen solution but you shouldn't have to worry about the creation of the queue per consumer or the distribution logic (assuming that the consumers can consume directly from the chosen tech)
How does the usage of ESB(JBoss ESB etc.), instead of ‘pure’ JMS(Active MQ etc.), help/ hamper? Does ESB provide any advantage over JMS which is ‘java-only’(?). I am hell confused – ‘ESB or JMS’, even after referring threads like these : JMS and ESB - how they are related?. It has one reply which says “JMS is not well suited for the integration of REST services, File systems, S/FTP, Email, Hessian, SOAP etc. which are better handled with an ESB that supports these types natively. For example, if you have a process that dumps a CSV file of 500MB at midnight, and you want another system to pickup the file, parse CSV and import into a database, this can easily be accomplished by an ESB - whereas a solution with just JMS will be bad. Similarly, integration of REST services, with load balancing/ failover to multiple backend instances can be done better with an ESB supporting HTTP/S natively.” It only added to my confusion !!!
My opinion is that ESB would overcomplicate this solution. It's designed (amongst other things) to assist integration with different technologies, but simpler solutions do this too - e.g - Apache Camel provides a very easy way of communicating using a huge variety of transports (including ActiveMQ).
Not all JMS implementations cater for connectivity from other languages, but ActiveMQ does using it's STOMP connector.
Is the usage of EAI framework (Apache Camel etc.) an approach entirely different from the pure JMS or ESB approach? If yes, how and what are the pros/cons?
Apache Camel and JMS are complementary technologies, as are JMS and ESB. Camel (& Spring Integration) are lightweight, simple and portable. ESB's are much more heavyweight and will normally lead to greater coupling with the ESB/application server.
I was told that ESB alone won’t help, BPM(or something else?) needs to be used to define and store the ‘routing’ logic – is this true?
It depends what your 'routing' logic is, it looks to me like you don't require routing logic, you just require guaranteed delivery to 1payroll consumer and 1 helpdesk consumer. BPM would be more useful where you want to selectively public data/invoke a service based on some characteristic of that data.
I strongly suggest reading http://activemq.apache.org/virtual-destinations.html, using these you would:
Send messages to the ActiveMQ broker, onto a VirtualTopic, e.g. VirtualTopic.X
Register the Payroll and Helpdesk consumers, as consumers on queues that ActiveMQ dynamically creates on the topic - e.g. Consumer.Payroll.VirtualTopic.X. Both Payroll consumers should be registered with the same string.
ActiveMQ will automatically retain a marker that represents what each set of consumers hasn't consumed. This means that 100% of messages will be processed by a Payroll consumer but a message will never be sent to > 1 payroll consumer.
Add/remove consumers at will.
N.B.I believe that other products, e.g. Apache QPID provide similar functionality - I'm just most aware of ActiveMQ, and have had success with this approach
For me JMS and ESB seem to be very related things and I'm trying to understand how exactly they are related.
I've seen a sentence that JMS can be used as a transport for ESB - then what else except the transport should be present in such an ESB? Is JMS a simple ESB or if not, then what it lacks from the real ESB?
JMS offers a set of APIs for messaging: put a message on a queue, someone else, sometime later, perhaps geographically far away takes the message off the queue and processes it. We have decoupled in time and location of the message provider and consumer. Even if the message consumer happens to be down for a time we can keep producing messages.
JMS also offers a publish/subscribe capability where the producer puts the message to a "topic" and any interested parties can subscribe to that topic, receiving messages as and when they are produced, but for the moment focus just on the queue capabilty.
We have decoupled some aspects of the relationship between provider and consumer. However some coupling remains. First, as things stand every message is processed in the same way. Suppose we want to introduce different kinds of processing for different kinds of messages:
if ( message.customer.type == Platinum )
do something special
Obviously we can write code like that, but an alternative would be to have a messaging system that can send different messages to different places we set up three queues:
Request Queue, the producer(s) puts their requests here
Platinum Queue, platinum consumer processing reads from here
Standard Queue, a standard consumer reads messages from here
And then all we need is a little bit of cleverness in the queue system itself to transfer then messsage from the Request Queue to the Platinum Queue or Standard Queue.
So this is a Content-Based Routing capability, and is something that an ESB provides. Note that the ESB uses the fundamental Queueing capabilities offered by JMS.
A second kind of couppling is that the consumer and producer must agree about the message format. In simple cases that's fine. But when you start to have many producers all putting message to the same queue you start to hit versioning problems. New message formats are introduced but you don't want to change all the existing providers.
Request Version 1 Queue Existing providers write here
Request Version 2 Queue New provider write here, New Consumer Reads here
And the ESB picks up the Version 1 Queue messages and transforms them into Version 2 messages and puts them onto the Version 2 queue.
Message transformation is another possible ESB capability.
Have a look at ESB products, see what they can do. As I work for IBM, I'm most familiar with WebSphere ESB
I would say ESB is like a facade into a number of protocals....JMS being one of them.
An addition to the above list is the latest Open Source ESB - UltraESB
JMS is not well suited for the integration of REST services, File systems, S/FTP, Email, Hessian, SOAP etc. which are better handled with an ESB that supports these types natively. For example, if you have a process that dumps a CSV file of 500MB at midnight, and you want another system to pickup the file, parse CSV and import into a database, this can easily be accomplished by an ESB - whereas a solution with just JMS will be bad. Similarly, integration of REST services, with load balancing/failover to multiple backend instances can be done better with an ESB supporting HTTP/S natively.
This Transformation does not happen automatically. You need to configure the mapping or write transformation service
Look at https://access.redhat.com/knowledge/docs/en-US/JBoss_Enterprise_SOA_Platform/4.2/html/SOA_ESB_Message_Transformation_Guide/ch02s03.html
Regards,
Raja Nagendra Kumar,
C.T.O
www.tejasoft.com
ESB offers integration with a lot of different protocols in addition to JMS.
Most use JMS behind the scenes to transfer, stor and move messages. One such solution OpenESB, uses XML format messages.
There are open source ESB which you could checkout -
OpenESB
Apache Camel
MuleESB
WSO2 ESB
JMS implementation like ActiveMQ come with Camel inbuilt into them.
JMS is a protocol for communicating with an underlying messaging layer. ESB operates at a higher level, offering integration with multiple technologies and protocols, one of which would be JMS, in a uniform way that makes management of complex flows much simpler.
There are JMS message brokers , that you can easily configure with ESB. https://docs.wso2.com/display/ESB470/JMS+Transport
JMS and ESB both provide a way of communication between different applications. But the context for JMS and ESB are different. JMS is for simple need. JMS is implemented by JMS Provider. It is Java specific.
Examples of JMS Providers are: Apache Active MQ, IBM MQ, HornetQ etc.
ESB is for complex need. ESB is a component in EAI providing communication facility to various applications. It is generic & not specific to Java. JMS is one of the supported protocols.
Examples of ESB provider are: MuleESB, Apache Camel, OpenESB
Use Case: It may be an overhead to use ESB, if all our communicating applications are in Java and are using the same message format. Here JMS may be sufficient.