A lot of materials about Service Bus mention that it uses AMQP 1.0. However, examples I have found use libraries that hide away AMQP layer, so it is not obvious how would one use generic AMQP 1.0 client and do things
that are listed in this Service Bus example
Being able to use independent AMQP client instead of supplied higher level clients might be important in several scenarios. For instance supplied clients might not be available at some platforms, or generic clients might provide more suitable way of working for particular problem.
Possibly related question
Little bit further from .net example I had linked in question I have found following example of how to access Service Bus with Proton AMQP 1.0 Python library. It shows how to send and receive messages.
What is lacking in that example, is how to create and manage queues and topics. It appears that this is not possible from AMQP interface, and needs to be done out of the band through either web azure console, or through REST interface.
Related
I am investigating solution to implement microservice Saga pattern in platform hosted in K8S in GCP.
There are 2 options: Eventulate Tram and Axon. However, these frameworks seem not to support message broker managed by cloud provider such as google-cloud-Pubsub whereas I do not want to deploy either Kafka or RabbitMQ to K8S since GCP support PubSub already.
So is there any way to integrate either Eventulate or Axon to use google cloud PubSub?
Thanks
Uncertain about Eventuate's angle on this, but Axon works with extensions as message brokers other than Axon Server. Throughout Axon's lifecycle (read: last 10 years), some of these have been provided, but none are currently used for all types of messages defined by Axon Framework. So, you wouldn't be able to use Kafka for sending commands in Axon for example.
Reasoning for this? Commands, events and queries have different routing requirements which should be reflected by using the right tool for the job.
To be a bit more specific on Axon's side, the following extensions can be used for distributing your messages:
AMQP -> for Events
Kafka -> for Events
JGroups -> for Commands
Spring Cloud Discovery -> for Commands
As you can tell, there currently is no Pub/Sub extension out there to allow you to distribute your messages. Added on top of that, my gut would tell me if it was available, then it would likely only be used for Event messages due to Pub/Sub's intent when it comes to being a message broker.
Luckily this actually makes it rather straightforward to create just such a extension yourself. Going into all the details to build this would be a little much, so I would recommend to have a look at Axon's AMQP extension first when it comes to achieving this. Hints on the matter are that for publication, you should add a component to handle Axon's events and publish them on Pub/Sub. For handling events, you are required to build a StreamableMessageSource or SubscribableMessageSource. These interfaces are used respectively by the TrackingEventProcessor and SubscribingEventProcessor, which in turn are the component in charge of dealing with the technical aspect of handling events.
By the way, if you would be building such an extension and you need a hand, it would be best to request this at AxonIQ forum, which you can find here.
Last note, and rather important I'd say, is the argument that such a connector would not be able to deal with all types of messages. If you would require a more full fledged Axon application to run in a distributed fashion, I would highly recommend to give Axon Server a try prior to building your own solution from the ground up.
I have a small stock-market application with Spring boot and if any product updated I want to serve an updated product to the clients in realtime
does it make sense to use message queues like RabbitMQ and Sse(Server Sent Events) for this, or is there a more sensible solution?
Solution
Publish your updated data to some channel
Your clients should subscribe to that channel to get updated feed in real-time.
Tools
Use in-house setup for RabbitMQ, ActiveMQ, Kafka or other open-source tools and implement WebSocket (For Front end applications)
Use commercial service like Google Cloud PubSub
Readymade and fully packaged solution with supported SDK for backend and frontend, https://www.pubnub.com/.
For this you can use either of
Spring Integration
Web Sockets
JMS
Spring Integration is an implementation of Enterprise Integration Patterns and is ideal for asynchronous processing data at realtime.
However, looking at your scope, it is only about publisher-subscriber pattern. Hence can be solved with JMS.
With JMS the subscribers/consumers can register/de-register dynamically. Also it provides ways to have fall-backs and tracking.
I'm thinking about introducing QPid Broker-J as an AMQP broker in the architecture we're developing, for it's wide support of different versions of the AMQP protocol.
We would be only producers, while third-party software over which we wouldn't have much control would act as consumer(s).
Our producers would be written in C++, so the most sensible thing to do at first seemed to adopt Proton C++ as a client library, which uses the latest AMQP 1.0.
I then read that the 1.0 specification is a lot different from all previous ones and this raises a concern about the interoperability. In example, it doesn't say anything about exchanges.
So, my question is... can the two versions communicate? Do I still have access even from AMQP 1.0 clients to all the concepts I need like exchanges and queue management that are from AMQP 0-* clients?
Or should I rather drop the idea of using Proton C++ and adopt another client for an older version (0-9-1 seems like the most commonly supported). Which one, in this case?
I found this vaguely related question and some interoperability documentation about RabbitMQ Clients with QPid Broker-J, but neither of them unfortunately answers my question. Proton's documentation is not that good as well, and I couldn't find anything interesting there either.
Thanks in advance for your time.
In general Qpid Broker-J provides full interoperability between clients of the different AMQP versions as much as it is possible to do so.
In terms of concepts such as exchanges, as you observe, these are not defined in AMQP 1.0 itself, however Qpid Broker-J allows your AMQP 1.0 clients to send messages to exchanges (and also to "consume" directly from an exchange by - in the background - creating a temporary queue and binding it to the exchange).
In terms of queue/exchange management, 0-x clients can continue to use operations like queue.declare or exchange.declare. For AMQP 1.0 clients these commands are not available, however Qpid Broker-J does offer a couple of ways to manage queues for AMQP 1.0 clients. One option would be to use Qpid Broker-J's REST API to create/bind exchanges/queues etc. The other (undocumented) option is to use management messages to the special $management address. This is an implementation of the Draft OASIS AMQP Management specification (which is subject to change, hence it is not documented currently). You can find an example of its use in some of Qpid Broker-J's tests, e.g. org.apache.qpid.systests.jms_1_1.extensions.management.AmqpManagementTest with the test method testCreateQueueOnVirtualHostManagement() giving an example as to how to create a queue over AMQP management for example.
If you have any more questions on Qpid Broker-J I'd suggest mailing the users list for Qpid (users#qpid.apache.org) where the developers would be happy to answer any questions.
I would like to deploy ActiveMQ in order to connect and communicate with other MOM implementations of jms such as WebsphereMQ, Tibco EMS, Oracle WebLogic JMS etc.
As far as I have researched online this seems to be possible by building a so-called bridge on top of activemq that will be able to communicate with the other end, regardless of the underlying wire protocol each jms implementation uses.
Examples though are scarce and often times minimal and so a point of confusion is whether I need to include jms client jars from each such implementation (Weblogic, WebsphereMQ, TIBCO EMS etc) in my classpath and bridge configuration. Can this be done without any such jars?
Since the goal is to connect to separate organizations MOMs I would like to avoid using any 3rd party, potentially licensed, jar.
...a point of confusion is whether I need to include JMS client jars
from each such implementation (Weblogic, Websphere MQ, TIBCO EMS etc)
in my classpath and bridge configuration. Can this be done without any
such jars?
TL;DR: You need the vendor's proprietary classes.
JMS defines the API and leaves the wire format and other implementation details to the transport vendor. Furthermore, in the proprietary implementations the wire formats can change significantly version to version. (One of the advantages of proprietary code is the ability to make such large structural changes when doing so improves performance, reliability or function.)
The one place where two JMS implementations have commonality is when the messages are in memory. Since all of the transport vendor's implementations inherit from the same Java JMS classes, it is usually possible to instantiate message objects from two different transport vendors and copy from one to the other. To do this, you must have the jars of each supported transport provider.
In fact, this is how most JMS bridge products that I've used work. The bridge code uses fully-qualified class names to reference the classes from two different vendors. It then creates two connection factories, one for each transport, and two different destinations, also one for each transport. It then reads a message from a queue on one provider, copies it to the message object on the other transport provider, then writes the message.
There are of course many "gotchas" with this approach. I'll list a few of the obvious ones:
Mapping destinations across providers is manual.
Identity propagation is not enforceable and requires the bridge to have quasi- or full-admin rights.
Transactionality across the providers is at best difficult, at worst impossible.
Correlation identifiers must be mapped by the bridge if they are mapped at all. This makes the bridge more complex and requires it to use a database or other persistent storage.
The vendor-specific classes often have a superset of the attributes and methods specified by the JMS spec. If used, these are stripped away during the copy.
Due to message segmentation, grouping and other affinities, there may not be a 1:1 correlation across the bridge.
Due to differences in implementation of such things as high availability, the physical network structure of one transport provider may not map 1:1 to that of another transport provider. This tends to limit the bridge function to the simplest common denominator of the two transports.
The issues raised by a general-purpose bridge include all of the issues of a 2-party bridge, raised as the exponent of the number of transports supported. So if a 2-transport bridge is of difficulty x, then for planning purposes assume a 5 transport bridge is of difficulty (x)**5.
That's not to say "don't code a bridge" since these turn out to be very useful. Just be aware, it's not a trivial task.
I am afraid that's not possible at least on the IBM MQ end. JMS is just an API and each provider is free to implement it the way they see it fits (as you rightly said its not a wire protocol). provider jars are essential to connect to a JMS product.
We have concept called Virtual destinations, try that
I have just read about CORBA and JMS, they both seem to be used to implement
Broker Architecture/Pattern.
I have few questions regarding them
1.The differences between them are still not clear to me, anybody please explain ?
2.Is CORBA is used in today's IT Solutions ? Or is it losing charm ?
3.Does JMS can replace every aspect of CORBA ?
Ramon Gil Moreno is right in stating that
JMS is the Java API that allows building applications to send and
receive messages. IBM MQ or ActiveMQ are samples of JMS vendors that
implements this API.
CORBA on the other hand is a specification that specifies how objects can interact with each other over a network across programming languages and run-time platforms.
The standard includes many APIs and infrastructure definitions (language bindings, marshalling, naming etc.), that are needed to support this. CORBA is still being used, and is Open Source as well as commercial (hard to find!)
Implementations exist, but I doubt if any of them covers 10% of the standard. Ramon's statement that CORBA is closer to RMI is a bit too simple - CORBA 2.4+ definitions include a CORBA Messaging definition that allows asynchronous and (reliable) queued communication.
CORBA, which is not hot nowdays, allows objects to be used remotely by different systems. It is more similar to RMI.
JMS is the Java API that allows building applications that send and receive messages. IBM MQ or ActiveMQ are samples of products that implements this API.