I am reading about microservices communication style, and there is a style here called "domain-specific protocol" which i don't understand.
Can anyone please describe it with simple words?
"domain-specific protocol" means a protocol that's purpose-built for the task at hand (for example SMTP is built for sending email, RTSP is built for streaming video).
The alternative would be a generic message-passing system (such as a message queue or a generic RPC system like gRPC, DCOM, ...) where only the messages themselves are domain-specific, but the protocol itself is built in a way to flexibly adapt to different purposes.
Related
I have been toying with the idea of using gRPC for 'IoT' type devices; not very constrained things like sensors; more like single board computer inbuilt devices like robots, drones and the like. What is needed a interface between device and centralized controller as the devices are developed separately and possible by other companies. So a versioned interface language is a must; and it should not be just in a word document; something programmable like a header file or WSDL or IDL or ProtocolBuffer. Also between device and controller the behavior should be specified for common use case like registration, re-registration etc.This can be in word file or in some non technical document.
The (rpc) interface specification in Protocol Buffer (ver 3) along with the efficient implementation via gRPC is leading me to choose this over CoAP/ LWM2M (Leshan Java and C++ implementation).
Having used both LWM2M and grPC, I would say that gRPC is more developer friendly; interface definition and implementation is fast,compared to OMA LWM2M process.Of course there is no Observer-Notify in gRPC, but for that MQTT should suffice.
Strictly one cannot compare LWM2M to gRPC. LWM2M is not just the interface but defines behavior in lot of IoT cases like BootStrap, Registration, KeepAlive , SW Upgrade etc and its universal HTTP like GET, PUT on an URL type addressable resource makes it very complete. However most of these behaviors can be custom defined with some effort.
Some of the IoT things which we plan to orchestrate are far from little brained devices like bulbs, and more like robots. Has anyone used gRPC for similar purposes. Any success of failure stories to share
The thing I would miss in gRPC for IoT are the MQTT MQ capabilities like queueing of messages, broker bridging QoS Parameter. Or for CoAP the Out of Band messages over SMS Transport. In this context gRPC is "just" an RPC framework which assumes to be always connected over TCP.
We were thinking in the same solution as Joe's, which includes CoAP + Protobuf:
Protobuf & its IDL are a really helpful way to define and govern our APIs in a centralized way. Google's api-linter as well as AIPs are high quality resources for keeping everything sane.
CoAP for transport: Lightweight protocol for IoT devices. It's available in practically every RTOS/embedded/platform and it's adapted to the low bandwitdh we usually face in this world. Being the protocol chosen by Thread and the support Project Connected Home Over IP is giving to it does also help.
Use Protobuf for the messages no matter they are Request/Response or even event messages pushed by the IoT device. An already solved problem by nanopb which is a protobuf wire compatible C code generator for embedded systems.
Generate our own stubs based on the IDL to create our own wrapper over CoAP and nanopb code. Unary calls are supported and also server streaming by leveraging this to the CoAP observable mechanisms.
I have taken the plunge and used it in a project with connected 'Devices'; These are small computers like Raspberry-pi. Overall it has been good experience; and languages used are C++ and Java mainly and also JavaScript in Node.js . We have used this as Dockerized microservices; Load Balancing is what we have not done; and I read that HTTP/2 based load balancing is tricky; Will update that part; planning to use Kubernetes for that. Overall Container technology with versioned interfaces - GRPC seems like a good fit for (micro) services
I use an esp32 and R_Pi with CoAP and protobufs. As far as I know gRPC is not supported for esp32/8266. I'm pretty happy with it, but didn't do any concrete testing against lwm2m. Implementation is here
I'm trying to have a messaging service(over TCPIP) between windows on PC(running C# app) and linux on an SoC board(running C++ app). I went through Google's protocol buffers and thought that I can serialize the structure (data struct) into a buffer and write this buffer over sockets(saw an example too).
My question, is there another way to do this? What does protocol buffer-RPC do? I went through their documentation but they don't specify on how to do it with TCPIP. Maybe someone has examples for this or detailed documentation?
Thank you,
Karthik.
My question, is there another way to do this?
There are many ways to do this. Google Protocol Buffers is library for cross-platform object serialization. You can share this serialized data in many different ways. My recommendation for you is using ZMQ (ZMQ GUIDE) which is perfectly suited for applications like yours.
I have a similar use case to you. I wrote Linux C++ ZMQ server on Raspberry Pi and Python ZMQ client dedicated for telemetry and remote management purposes. This library is efficient, lightweight, works perfectly with Google Protocol Buffers and has binding to all major programming languages. If you decide to use this library do not hesitate to ask me about working code example.
See if the Google Developers page on how the protocol buffers wire format is encoded helps with your question: https://developers.google.com/protocol-buffers/docs/encoding
I am making a turn-based card game that will have clients, a lobby server and a game server. What methologies are there that are both cross-language and bi-directional (e.g. client request -> server server response-> client, as well as server request-> client client response -> server)?
I have looked into JMS but believe it is too heavyweight for my needs (this program will just be small scale, and I don't think the complexities make this solution suitable). I have briefly looked into REST but I believe that wouldn't fit the bi-directional requirement. Of course, there is RMI but I would like to be able to develop clients in C++ and other languages as another learning exercise.
If I'm honest, I'm at a bit of loss because I don't want to use JMS as I think it is too complex for this, but I don't think just using TCP sockets and say using a basic XML based protocol for the messages will provide a good structure of communication for the program.
The research lab that I do some work with develops a system called "Object Oriented Distributed Semantic Services."
We leverage some work that we do with cross-language serialization to allow you to write clients/servers in different languages, and the underlying messages to be a format that be serialized and deserialized by clients/servers regardless of their implementation language.
Right now we mostly support Java/ObjectiveC. You can take a look at the chat room tutorial, which should give you a basic idea of how requests / responses work.
http://ecologylab.net/research/simplGuide/oodss/index.html
OODSS is designed to work well for game scenarios... the system was originally written to support a game one of the researchers in our lab was working on. The original paper on OODSS discusses the development of a game from the ground up. That may work out well for you: http://ecologylab.net/technicalReports/oodss_TR_10_01.pdf
You could apply a similar idea to allow for multiple clients in languages that aren't supported yet. (you may have to write some serialization/deserialization code on your own, to start.)
Good luck! Hope that helps!
EDIT: I forgot to include the prime candidate for web applications: JSON over HTTP/REST + Comet. It combines the best features of the others (below)
Persevere basically bundles everything I need in a server
The focus for Java and such is definitely on Comet servers, but it can't be too hard to use/write a client.
I'm embarking on an application with a server holding data, and clients executing operations which would affect this data, and thus require some sort of notification across all interested/subscribed clients.
The first client will probably be written in WPF, but we'll probably need to add clients written in other languages, e.g. a Java (Swing?) client, and possibly, a web client.
The actual question(s): What protocol should I use to implement this? How easy would it be to integrate with JS, Java and .NET (precisely, C#) clients?
I could use several interfaces/protocols, but it'd be easier overall to use one that is interoperable. Given that interoperability is important, I have researched a few options:
JSON-RPC
lightweight
supports notifications
The only .NET lib I could find, Jayrock doesn't support notifications
works well with JS
also true of XML-based stuff (and possibly, even binary protocols) BUT this would probably be more efficient, thanks to native support
Protobuf/Thrift
IDL makes it easy to spit out model classes in each language
doesn't seem to support notifications
Thrift comes with RPC out of the box, but protobufs don't
not sure about JS
XML-RPC
simple enough, but doesn't support notifications
SOAP: I'm not even sure about this one; I haven't grokked this yet.
seems rather complex
Message Queues/PubSub approach: Not strictly a protocol, but might be fitting
I hardly know anything about them, and got lost amongst the buzzwords`-- JMS? **MQ?
Perhaps combined with some RPC mechanism above, although that might not be strictly necessary, and possibly, overkill.
Other options are, of course, welcome.
I am partial to the pub/sub design you've suggested. I'd take a look at ZeroMQ. It has bindings to C#, Java, and many other platforms.
Bindings list: http://www.zeromq.org/bindings:clr
I also found this conversation on the ZeroMQ dev listing that may answer some questions you have about multiple clients and ZeroMQ: http://lists.zeromq.org/pipermail/zeromq-dev/2010-February/002146.html
As XMPP was mentioned, SIP has a similar functionality. This might be more accessible for you.
We use Servoy for this. It does automatic data broadcasting to web-clients and java-clients. I'm not sure if broadcasts can be sent to other platforms, you might be able to find an answer to that on their forum.
If you want to easily publish events to clients across networks, you may wish to look at a the XMPP standard. (Used by, amongst other things, Jabber and Google Talk.)
See the extension for publish-subscribe functionality.
There are a number of libraries in different languages including C#, Java and Javascript.
You can use SOAP over HTTP to modify the data on the server and SOAP over SMTP to notify the subscribed clients.
OR
The server doesn't know anything about the subscription and the clients call the server by timeout to track updates they are interested in, using XML-RPC, SOAP (generated using WSDL), or simply HTTP GET if there is no need to pass back complex data on tracking.
Can XMPP be applied in a similar way to the reverse Ajax pattern? Can it be used to implement http long wait like Comet? Is there an example of using such a technique with XMPP?
Yes. Take a look at XMPP over BOSH. It's a long-lived non-polling technique. From the BOSH specification:
This specification defines a transport protocol that emulates the semantics of a long-lived, bidirectional TCP connection between two entities (such as a client and a server) by efficiently using multiple synchronous HTTP request/response pairs without requiring the use of frequent polling or chunked responses.
jbosh is an open source Java implementation. Tigase.org has a number of projects, including BOSH support, and appears to be mostly Java/GWT. JAXMPP is a Java library in the framework, although it's not clear what feature set it supports. I've not tried any of these myself.
Like Ire told you can use bosh for that. On the client-side(javascript) you could use strophe.js.
Prosody is an easy to install XMPP-server which supports BOSH.
To learn how to create programs using strophe.js you could read Professional XMPP programming with javascript and Jquery