I'm developing a messaging system and I used JBoss Netty + Google protobuf for the POC. protobuf was chosen for its swiftness of serialization/deserialization, relatively low traffic cost and availability in several languages.
Still, when it comes to production under heavy load a self-coded server application can never be as good as well-established and tested frameworks.
The problem is, I can't find such a framework that would allow me using protobuf as a transport protocol. Apache ActiveMQ and ActiveBlaze are the closest things I could find but the documentation is nearly absent.
I stumbled upon something that is ActiveMQ protobuf implementation but there is no reminding of it in official ActiveMQ documentation (its not among the supported protocols).
So my question is whether AMQ supports protobuf and if it does how it can be integrated?
No, ActiveMQ uses its own OpenWire protocol or the Stomp protocol. The protobuf bits are used for the underlying KahaDB Message store not the wire level portion. You can store your protobuf data in a BytesMessage and transmit it that way to allow you to marshal and unmarshal the data on either end.
Related
What exactly is Solace JCSMP?
Is it just a more suited JMS API for Solace? What kind of other benefits does it have?
For example, I would need to create a Java application using SolaceMQ that needs to send/receive messages over to/from AMQP protocol so that all different microservices written in different languages would be able to consume them. Is JCSMP right for that? or is it just another JMS API that it only works between Java applications?
What exactly is Solace JCSMP?
JCSMP is the Classic Java API for Solace's SMF (Solace Message Format) protocol. Note that Solace recently introduced a more modern messaging API for java devs using their event brokers that is just referred to as the "PubSub+ Messaging API for Java". If you are new to Solace and using Java it may make sense to use that instead of JCSMP.
Is it just a more suited JMS API for Solace? What kind of other benefits does it have?
JCSMP allows developers to take advantage of the full feature set offered by the Solace Broker whereas the Solace JMS implementation only covers the features defined in the JMS specification itself. For example, use of Solace features such as Replay or having Queues subscribe to Topics are not programmatically possible via the JMS API and would have to be done administratively if using JMS.
For example, I would need to create a Java application using SolaceMQ that needs to send/receive messages over to/from AMQP protocol so that all different microservices written in different languages would be able to consume them. Is JCSMP right for that? or is it just another JMS API that it only works between Java applications?
Solace PubSub+ Event Brokers provide protocol translation between any of the protocols supported by the broker. It doesn't matter whether you are using SMF (which both Solace JMS and JCSMP use), MQTT, AMQP 1.0, etc. or what programming language you are using. For example when you send a message using JCSMP you can receive it using a Java app using JMS, a Python app using AMQP and a Go app using MQTT. The Solace Event Broker even has support for calling a RESTful webhook using it (Check out Solace "Rest Delivery Endpoints"). Just a heads up that if you're using headers you'll want to checkout the Solace docs to see how they're mapped during protocol translation.
I'm thinking about introducing QPid Broker-J as an AMQP broker in the architecture we're developing, for it's wide support of different versions of the AMQP protocol.
We would be only producers, while third-party software over which we wouldn't have much control would act as consumer(s).
Our producers would be written in C++, so the most sensible thing to do at first seemed to adopt Proton C++ as a client library, which uses the latest AMQP 1.0.
I then read that the 1.0 specification is a lot different from all previous ones and this raises a concern about the interoperability. In example, it doesn't say anything about exchanges.
So, my question is... can the two versions communicate? Do I still have access even from AMQP 1.0 clients to all the concepts I need like exchanges and queue management that are from AMQP 0-* clients?
Or should I rather drop the idea of using Proton C++ and adopt another client for an older version (0-9-1 seems like the most commonly supported). Which one, in this case?
I found this vaguely related question and some interoperability documentation about RabbitMQ Clients with QPid Broker-J, but neither of them unfortunately answers my question. Proton's documentation is not that good as well, and I couldn't find anything interesting there either.
Thanks in advance for your time.
In general Qpid Broker-J provides full interoperability between clients of the different AMQP versions as much as it is possible to do so.
In terms of concepts such as exchanges, as you observe, these are not defined in AMQP 1.0 itself, however Qpid Broker-J allows your AMQP 1.0 clients to send messages to exchanges (and also to "consume" directly from an exchange by - in the background - creating a temporary queue and binding it to the exchange).
In terms of queue/exchange management, 0-x clients can continue to use operations like queue.declare or exchange.declare. For AMQP 1.0 clients these commands are not available, however Qpid Broker-J does offer a couple of ways to manage queues for AMQP 1.0 clients. One option would be to use Qpid Broker-J's REST API to create/bind exchanges/queues etc. The other (undocumented) option is to use management messages to the special $management address. This is an implementation of the Draft OASIS AMQP Management specification (which is subject to change, hence it is not documented currently). You can find an example of its use in some of Qpid Broker-J's tests, e.g. org.apache.qpid.systests.jms_1_1.extensions.management.AmqpManagementTest with the test method testCreateQueueOnVirtualHostManagement() giving an example as to how to create a queue over AMQP management for example.
If you have any more questions on Qpid Broker-J I'd suggest mailing the users list for Qpid (users#qpid.apache.org) where the developers would be happy to answer any questions.
We currently use JMS API with ActiveMQ broker, looking over to move to RabbitMQ.
Compared to openWire vs amqp which one would give best performance with java client-producer. I Couldn't find a comparison study on amqp(RabbitMQ) vs openwire (activeMQ) native protocol. I'm looking in terms of raw performance and ease of scalability.
We currently use Spring Integration for ActiveMQ, I would like to know if its a drastic change to move to RabbitMQ(AMQP) even with Spring Integration. Is there any bridge similar to what ActiveMQ uses to do JMS<->AMQP forwarding ?
ActiveMQ also supports AMQP: http://activemq.apache.org/amqp.html
Both are binary protocols. Openwire is going to be more full-featured when using ActiveMQ.
For "raw performance" you'll have to nail down your use cases first. Chances are the protocol you choose (ampq vs openwire) is not going to make any difference from a "perfomance" standpoint.
ActiveMQ now implements AMQP.
You can specify multiple communication protocols and let ActiveMQ auto detect determine which to implement:
http://activemq.apache.org/auto.html
I heard a guy from IBM give a talk about MQTT and he said that MQTT is the most lightweight messaging protocol as of today. His argument was that the smallest overhead produced for a single message is 2 bytes. However, I've heard the very same (2 bytes overhead) about HTML5 WebSockets too? Anyways, as I plan to use Open MQ as a JMS provider for a messaging application, which protocol my provider use became of interest to me. I cannot anywhere find the answer to that question and I even googled the Open MQ documentation and the documentation of GlassFish which use Open MQ as a JMS provider. Some blog posts and the like on Internet says AMQP but I cannot find a solid reference to back that statement up.
Which protocol does Open MQ use, and how would you know?
I think you mix a few things up: MQTT is an application level protocol and uses TCP as transport protocol. MQTT can also be used with Websockets as transport. MQTT is, if you will, a lightweight alternative to JMS and AMQP.
AMQP is a popular and reliable alternative to JMS and is well suited for business and mission critical messaging. It is very feature rich and widely used. Note that AMQP is a messaging protocol while JMS is an API (which can even use AMQP as transport). You can use AMQP directly in Java Applications with Libraries such as the RabbitMQ Java Client.
MQTT on the other hand is perfect fit for telemetry data and scenarios where you have many clients which communicate with a single message broker and where low bandwidth usage, memory efficiency and battery life on the clients is key.
JMS does not define any transport protocol (in contrast to MQTT) and anything could be used here. I personally think TCP is a good fit here, too.
I do not know about OpenMQ but for JMS and AMQP I can recommend ActiveMQ. For MQTT there are a few brokers out there including HiveMQ and Mosquitto.
Obligatory Disclaimer: I am a developer of HiveMQ, so I am probably a bit biased ;-)
I think OpenMQ is using STOMP as Application Level Protocl. There is no other documentation which protocol OpenMQ supports than STOMP.
Edit: found the UMS Protocol and it seems OpenMQ is using this per default. https://mq.java.net/4.3-content/ums/umsIntro.html
OpenMQ is not able to use AMQP so I would recommend RabbitMQ.
Does anyone has a good comparison between the two technologies: Thrift vs JMS, in Java for messaging services? Thanks!
Thrift is serialization/rpc framework whereas JMS is full-featured messaging service, so they don't compare directly.
JMS uses stardart java object serialization for ObjectMessage's. It's much slower than thrift and can't be deserialized without java. However, it's possible to serialize objects with any other framework and send them in the form of BytesMessage. For example, ActiveMQ encourages usage of protocol buffers for this task, but it's possible to use any other framework, like thrift, avro, kryo or smile.
This page might give an insight on speed/serialization size of different technologies:
http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking
In general, each of modern serialization frameworks has its pros and cons, but they provide approximately the same performance.