I have two apps that run on the same server. One is a c++ app and another is a java web server running on top of vertx. The webserver wants to send request to the C++ part and obtains response. ZeroMq seems a performing solution to do the inter process communication. And it exists a bridge to vertx (https://github.com/dano/vertx-zeromq), but no so well documented.
I'm wondering if what i think can be done with this bridge:
C++ zeroMq socket type is a dealer, it registers to the event bus by sending the appropriate message that contains the handler adress.
Webserver send data to the socket event bus handler address and get response in its callback.
Does it have an opportunity to work or i misunderstand the zeroMq bridge ?
That sounds correct to me but you don't need ZeroMQ - you can just used regular TCP - https://vertx.io/docs/vertx-tcp-eventbus-bridge/java/ and that has good documentation and support.
I'm currently looking into the benefits of using ZeroMQ for my project and suspect it is useful for more complex topologies like "broadcasting an event without knowing who wants it (don't require handlers to register)" but Vertx doesn't support this from what I can see.
Related
I have a concox GT06 device from which I want to send tracking data to my AWS Server.
The coding protocol manual that comes with it only explains the data structure and protocol.
How does my server receive the GPS data collected by my tracker?
Verify if your server allows you to open sockets, which most low cost solutions do NOT allow for security reasons (i recommend using an Amazon EC2 virtual machine as your platform).
Choose a port on which your application will listen to incoming data, verify if it is open (if not open it) and code your application (i use C++) to listen to that port.
Compile and run your application on the server (and make sure that it stays alive).
Configure your tracker (usually by sending an sms to it) to send data to your server's IP and to the port which your application is listening to.
If you are, as i suspect you are, just beginning, consider that you will invest 2 to 3 weeks to develop this solution from scratch. You might also consider looking for a predeveloped tracking platform, which may or may not be acceptable in terms of data security.
You can find examples and tutorials online. I am usually very open with my coding and would gladly send a copy of the socket server, but, in this case, for security reasons, i cannot do so.
Instead of direct parsing of TCP or UDP packets you may use simplified solution putting in-between middleware backends specialized in data parsing e.g. flespi.
In such approach you may use HTTP REST API to fetch each new portion of data from trackers sent to you dedicated IP:port (called channel) or even send standardized commands with HTTP REST to connected devices.
At the same time it is possible to open MQTT connection using standard libraries and receive converted into JSON messages from devices as MQTT in real time, which is even better then REST due to almost zero latency.
If you are using python you may take a look at open-source flespi_receiver library. In this approach with 10 lines of code you may have on your EC2 whole parsed into JSON messages from Concox GT06.
I'd like to compare somehow capabilities of grpc vs. zeromq & its patterns: and I'd like to create some comparsion (feature set) - somehow - 0mq is "better" sockets - but anyways - if I apply 0mq patterns - I get comparable 'frameworks' I think - and here 0mq seems to be much more flexible ...
The main requirements are:
async req / res communication (inproc or remote) between nodes
flexible messages routing
loadbalancing support
well documented
any ideas?
thanks!
async req / res communication (inproc or remote) between nodes
Both libraries allow for synchronous or asynchronous communication depending on how to implement the communication. See this page for gRPC: http://www.grpc.io/docs/guides/concepts.html. Basically gRPC allow for typical HTTP synchronous request/response or a 'websocket-like' bidirectional streaming. For 0mq you can setup a simple REQ-REP connection which is basically a synchronous communication path, or you can create async ROUTER-DEALER type topologies.
flexible messages routing
'Routing' essentially means that a message gets from A to B via some broker. This is trivially done in 0mq and there are a number of topologies that support stuff like this (http://zguide.zeromq.org/page:all#Basic-Reliable-Queuing-Simple-Pirate-Pattern). In gRPC the same sort of topology could be created with a 'pub-sub' like connection over a stream. gRPC supports putting metadata in the message (https://github.com/grpc/grpc-go/blob/master/Documentation/grpc-metadata.md) which will allow you to 'route' a message into a queue that a 'pub-sub' connection could pull from.
loadbalancing support
gRPC has a health check support (https://github.com/grpc/grpc/blob/master/doc/health-checking.md) but because it's HTTP/2 you'd have to have a HTTP/2 load balancer to support the health check. This isn't a huge big deal, however, because you can tie the health check to a HTTP/1.1 service which the load balancer calls. 0mq is a tcp connection which means that a load balancer would likely check a 'socket connect' in tcpmode to verify the connection. This works but it's not as nice. Again you could get slick and front-end the 0mq service with a HTTP/1.1 webserver that the load balancer reads from.
well documented
both are well documented. 0mq's documentation must be read to throughly understand the technology and is more of a higher lift.
Here's the big differences:
0mq is a tcp protocol whereas gRPC is HTTP with a binary payload.
0mq requries you design a framing protocol (frame 1 = verison, frame 2 = payload, etc.), whereas much of this work is done for you in gRPC
gRPC is transparently coverted to REST (https://github.com/grpc-ecosystem/grpc-gateway) whereas 0mq requires a middleware application if you want to talk REST to it.
gRPC uses standard tls x509 certificates (think websites) whereas 0mq uses a custom encryption/authentication protocol (http://curvezmq.org/). Prior to 4.x there was no encryption support in 0mq and if you really wanted it you had to dive into this crap: https://wiki.openssl.org/index.php/BIO. (trust me don't do it)
0mq can create some pretty sick topologies (https://github.com/zeromq/majordomo) (https://rfc.zeromq.org/spec:7/MDP/) whereas gRPC is basically client/server
0mq requires much more time to build and get running whereas gRPC is basically compiling a protobuf messages and importing the service into your code.
Not quite the same. gRPC is primarily for heterogeneous service interoperability, ZeroMQ (ZMQ/0MQ/ØMQ) is a lower level messaging framework. ØMQ doesn't specify payload serialization beyond passing binary blobs whereas gRPC chooses Protocol Buffers by default. ØMQ is pretty much stuck on the same machine or machines between datacenters/clouds, whereas gRPC could potentially work on a real client too (ie mobile or web, it already supports iOS). gRPC using ØMQ could be significantly faster and more efficient for in-cloud/-datacenter services than the overhead, latency and complexity of http2 request/response chain. I'm not sure how (or even if) gRPC TLS security is adequate for public cloud and mobile/web usage, but one could always inject end-to-end security requirements (ie libsodium) at the router/controller level of the app/app framework and run in plaintext mode (which would also remove OpenSSL fork BoringSSL from causing maintenance headaches because of upstream flaws).
For very high latency / low bandwidth services (ie mission to mars), one would think about RPC using a transport like SMTP (ie Active Directory alternate replication) or MQTT (ie Facebook Messenger, ZigBee, SCADA)
Bonus (off-topic): It would be nice if gRPC had alternate pluggable transports like ØMQ (which also itself supports UNIX sockets, TCP, PGM and inproc), because HTTP/2 isn't stable in all languages yet and it's slower than ØMQ. And, it's worth looking at nanomsg (especially in the HFT world) because it could be extended with RDMA/SDP/MPI and made crazy low-latency/zero-copy/Infiniband-ready.
I'm new to ServiceStack and want some validation on a pattern we're thinking about using.
We want to use ServiceStack with Xamarin and Message Queues. While I understand how REST works under the covers, I'm not sure how the Message Queues on ServiceStack work and if its appropriate for mobile devices.
Specifically we know that all mobile devices are essentially behind a NAT firewall setup by the Telco. Meaning Clients can talk to servers, but servers cant talk directly to clients, without the client talking first.
While the concept of a ServiceBus is designed specifically to handle this case, i'm not sure if its "mobile network friendly".
I would assume that the client side implementation, would need to work in one of two ways: polling, blocking get.
Polling would have the client side frequently runing a Http GET to ask the server if anything is available on a queue. A Blocking Get would, perform a Http GET but have the server return nothing until data is ready. Or is there another technique that i'm missing?
If it is a poll, is there any way to control the Poll frequencies in service stack. If its a blocking get how is this configured..
What happens when the app goes to the background, do we need to cancel the connections manually. etc.etc.
We tool an old version of the ServiceStack client library and ported them to xamarin. We now see that the latest ServiceStack client side library is Xamarin compatible.
So, basically my question is: Had anyone used Message Queues from a Xamarin Mobile to ServiceStack with RedisMQ or other server side message queue.
I'm evaluating some messaging libraries and protocols (e.g. ZeroMQ, WAMP). One of my main requirements is that sending messages from client to server and vice verse (two way communication) must be absolute safe with respect to client/server crashes. This means to me that e.g. the client must continue sending all not delivered messages after a spontaneous reboot. So the library should implement some kind of file based buffering. Is there anything there I can use out of the box?
[EDIT]
Some note on my use case:
In my scenario there are around 1000 clients communicating with one server. There is no direct client to client communication required. But I need a two-way communication, so both, the clients can push some data to the server and vice versa. The clients are connected via 3G mobile network. Both, client and server are written in C#. I focused on using ZeroMQ, Apache Thrift or WAMP. But one of the main requirements is to ensure asynchronous but safe messaging with respect to system crashes. So when the client starts an asynchronous data push to the server, and it will crash before the message can be delivered to the server, it is required that the client will continue sending the message after a reboot.
You might look into the Apache.org's Kafka project.
The problem is harder than it looks, and most people don't want to pay the price to make it happen.
Also, there is a UX issue with old queued up messages replaying without the user's understanding.
I want to create a messaging service that uses the XMPP protocol. How would I implement the server-side as well as the client side aspects of this service? I know I would need a server (like Jabberd 2) that runs the messaging framework. How hard would this be to set up and get running? Also what would be the best way to hook up a client program into this service? How would i start pushing messages from one client, through the server, to another client?
Server: there are many out there, see http://xmpp.org/software/servers.shtml for a list.
I've used OpenFire in the past, it's fairly straightforward to set up.
You can add a library like xmppframework to your Cocoa project to make it a client, and configure it to talk to your XMPP server.
Each client gets an identifier (called a 'jid') of the form: uniquetext#xmppserver.name, and you send messages from one client to the other by addressing them to the jid of the intended recipient.
If you want to play around with simple examples in a scripting language, you can use something like the examples in the python xmpp library to see how it all works. Use an xmpp client like psi to connect as one jid and use the examples to connect as another jid to send/receive messages through the server.