I'm trying to have a messaging service(over TCPIP) between windows on PC(running C# app) and linux on an SoC board(running C++ app). I went through Google's protocol buffers and thought that I can serialize the structure (data struct) into a buffer and write this buffer over sockets(saw an example too).
My question, is there another way to do this? What does protocol buffer-RPC do? I went through their documentation but they don't specify on how to do it with TCPIP. Maybe someone has examples for this or detailed documentation?
Thank you,
Karthik.
My question, is there another way to do this?
There are many ways to do this. Google Protocol Buffers is library for cross-platform object serialization. You can share this serialized data in many different ways. My recommendation for you is using ZMQ (ZMQ GUIDE) which is perfectly suited for applications like yours.
I have a similar use case to you. I wrote Linux C++ ZMQ server on Raspberry Pi and Python ZMQ client dedicated for telemetry and remote management purposes. This library is efficient, lightweight, works perfectly with Google Protocol Buffers and has binding to all major programming languages. If you decide to use this library do not hesitate to ask me about working code example.
See if the Google Developers page on how the protocol buffers wire format is encoded helps with your question: https://developers.google.com/protocol-buffers/docs/encoding
Related
I currently have a primitive RPC setup relying on JSON transferred over secured sockets, but I would like to switch to gRPC. Unfortunately I also need access to AF_UNIX on windows (Which Microsoft recently started supporting, but gRPC has not implemented).
Since I have an existing working connection (managed with a different library), my preference would be to just use that in conjunction with GRPC to send/receive commands in place of my JSON parsing, but I am struggling to identify the best way to do that.
I have seen Plugging custom transport into gRPC but this question differs in the following ways (As well as my hope for a more recent answer)
I am wanting to avoid making changes to the core of gRPC. I'd prefer to extend it if possible from within my library, but the answer here implies adding a new transport to gRPC.If I did need to do this at the transport level, is there a mechanism to register it with gRPC after the core has been built?
I am unsure if I need to define this as a full custom transport, since I do already have an existing connection established and ready. I have seen some things that imply I could simply extend Channel, but I might be wrong.
I need to be able to support Windows, or at least modern versions of it (Which means that the from_fd options gRPC provides are not available since they are currently only implemented for POSIX)
Has anyone solved similar problems with gRPC?
I may have figured out my own answer. I seem to have been overly focused on gRPC, when the service definition component of Protobuf is not dependent on that.
How can i write my own RPC Implementation for Protocol Buffers utilizing ZeroMQ is very similar to my use case, with https://developers.google.com/protocol-buffers/docs/proto#services seeming to resolve my issue (And this also explains why I seem to have been mixing up the different kinds of "Channels" involved
I welcome any improvements/suggestions, and hope that maybe this can be found in future searches by people that had the same confusion.
I have already looked at some options such as cellophane but found that the sample app crashes. Also it does way more than just implement socket.io protocol because it uses json and curl. What I am looking for is a simple library in C (not c++, so no boost) which does the encoding and decoding of data using socket.io protocol, so that it can be used with any other library that does network i/o. The purpose is to implement a native client for linux developed in C to talk to node.js server.
Any libraries/solutions that would help is appreciated. Also any documentation on socket.io protocol would help, so that I can look at implementing it.
Thanks.
I searched on the internet but couldn't find anything useful. First, I was thinking to use Protocol Buffers but it doesn't provide built in feature to track multiple messages (where one message finish and second starts) or message self delimiting, but I read about this feature in Thrift white paper and it seems good to me. Now I am thinking to use Thrift instead of Protocol Buffers.
I am working on custom protocol for that I don't require RPC, could someone suggest if I can use Thrift without RPC (as its in the Protocol Buffers, one simply use the streams function) and some starting point as thrift documentation is a bit cumbersome.
Thanks!
Yes, It is possible. A similar answer is given Here. Apache thrift can be used without RPC you can simply use transport and protocol layers related libraries as they are defined in the documentation.
Apache Thrift is indeed a RPC- and serialization framework. The serialization part is used as part of the RPC mechanism, but can be used standalone. For the various languages there are samples and/or supporting helper classes available. If this is not the case for your particular language, the necessary code pretty much boils down to this (pseudo code):
var data = InitializeMyDataStructure(...);
var trans = new TStreamTransport(...);
var prot = new TJSONProtocol(trans);
data.write(prot);
Both transport(s) and protocol are pluggable, so instead JSON and a stream you are free to use your own protocol, and (for example) a file transport. Or whatever else combination makes sense for your use case and is supported for your target language.
as thrift documentation is a bit cumbersome.
You are free to ask any question, be it here or in the mailing list. Furthermore, we have a nice tutorial and the Test server/client pairs are also good examples for typical use cases.
I've taken a look at the basic websocket capabilities in Dart, using this simple example:
https://github.com/financeCoding/chat-websocket-dart
But I was wondering if there's a nice library I could use to build a realtime online game using websockets. I've had experience in this using node.js with socket.io, which worked out quite well. I need to be able to have "rooms", join rooms, leave rooms, broadcast to clients in a room, etc. as well as some nice notion of connection "health", reconnection etc. So what I'm asking is if there's a nice library for dart that has similar functionality? Even cooler would be a library on top of that library that could enable nice RPC functionality with variable syncing etc. such as http://nowjs.com/ which achieves this using socket.io. But I guess that might be too ambitious.
If anyone's had any experience or found a project which is similar to what I'm talking about, let me know :)
Duct is clone of Socket.IO in Dart which aims to be protocol-level compatible with the original implementation.
https://github.com/petrhosek/duct
Sorry, at the time of this writing, I'm not aware of a socket.io port for Dart. socket.io is nice because it has a bunch of implementation options for browsers that don't support Web sockets.
Sounds like a good idea for a hackathon project!
I am currently working on a project that embeds a flash game, that uses Smartfoxserver for the flash communication. That communication is mostly just synchronizing the cursor and object movements between the two players.
Since I am not a flash guy, but a ruby programmer, I got curious: What kind of communication options does flash offer for this kind of time sensitive data exchange? I was thinking of writing a ruby-eventmachine based communication server to minimize the dependencies on external programs. Would that even be feasible?
Although I don't have a complete answer for you, because I'm not done myself, I have found myself in almost the exact same position as you.
My current approach is for my ruby server to essentially just be a socket server that handles all of the communication between clients, however I personally intend on keeping any of the logic outside of the server ( unless I run across a reason to change that idea).
If you haven't done any socket programming in ruby, I recommend the following as a jumping point. This is an IBM document on Ruby Socket programming and discusses an approach for asynchronous data:
Ruby Sockets - IBM
I think what it ultimately comes down to as well will be performance. I currently use a version of my ruby server in my daily work, but the data doesn't have to be updated in a time sensitive manner.
You can create a server in (almost?) any language that supports sockets and manage your clients with that.
From a flash perspective you could use ruby, but I don't know how fitted ruby would be for the task...I mean, you can make a PHP socket server but it would not handle much stress.
An alternative to SmartFoxServer could be open sourced Red5, written in Java.
If you want to know about flash built in capabilities in handling p2p:
Flash p2p: Everett Church
In Flash Player 10 adobe added Live media (RTMFP) support- Media was always sourced from the publishing peer. In Flash Player 10.1 beta they introduced groups and application-level multicast.
However, you would still need a server (Stratus) to handle introductions and manage active connections. ( Also, I regard it as geared towards flash video, Stratus does not support shared objects or scripting. So with Stratus, your applications will be clients communicating directly with each other.)
Edit:
After re-reading your question and the answer provided by Beanish, I think the first time I got your question wrong. So I just re-wrote my answer...