I have two go apps running on different machines.
I want them to communicate with each other in a native fast manner.
How do I do it better? I worked with Erlang and find very handy the way it's implemented there.
What are preferred ways to communicate between Go apps?
net/rpc - out of the box & fast
gRPC - fast and universal (imo the way to go)
net/rpc/jsonrpc - good when you need human readable messages or want to integrate
go with client side js code
plain HTTP - overkill for communication between go programs, you'll have to plan routes and data encoding yourself, which is unnecessary effort
message queue - a bit slower approach, but you can benefit from advanced routing and persistence which they provide
Related
I need a embeddable messaging component for Golang application.I looked into distributed messaging system like NSQ etc, but prefer one which is much simpler, stable and embeddable (no separate daemon). Systems like NSQ would be overkill for my requirement and I don't think it is easily embeddable.
I need such a component to distribute the payload/data reliably (delivered once atleast) among my processing distributed engines and embeddable would make installation and configurations easy.
Any information would be appreciated.
I use RabbitMQ in production with this library https://github.com/streadway/amqp. This is not an official library but the implementation is quite good.
I use it to improve fault tolerance and scalability for IOT.
Pro:
Easy to use
Great administration tools
Many languages implementations
Many patterns availables https://www.rabbitmq.com/getstarted.html
Cons:
Not really efficient for big payloads
Probably not the fastest (but fast enough to my projects)
I need to do something relatively simple, and I don't really want to install a MOM like RabittMQ etc.
There are several programs that "register" with a central
"service" server through TCP. The only function of the server is to
call back all the registered clients when they all in turn say
"DONE". So it is a kind of "join" (edit: Barrier) for distributed client processes.
When all clients say "DONE" (they can be done at totally different times), the central server messages
them all saying "ALL-COMPLETE". The clients "block" until asynchronously called back.
So this is a kind of distributed asynchronous Observer Pattern. The server has to keep track of where the clients are somehow. It is ok for the client to pass its IP address to the server etc. It is constructable with things like Boost::Signal, BOOST::Asio, BOOST::Dataflow etc, but I don't want to reinvent the wheel if something simple already exists. I got very close with ZeroMQ, but non of their patterns support this use-case very well, AFAIK.
Is there a very simple system that does this? Notice that the server can be written in any language. I just need C++ bindings for the clients.
After much searching, I used this library
https://github.com/actor-framework
It turns out that doing this with this framework is relatively straightforward. The only real "impediment" to using it is that the library seems to have gotten an API transition recently and the documentation .pdf file has not completely caught up with the source. No biggie since the example programs and the source (.hpp) files get you over this hump. However, they need to bring the docs in sync with the source. In addition, IMO they need to provide more interesting examples on how to use c++ Actors for extreme performance. For my case it is not needed, but the idea of actors (shared nothing) in this use-case is one of the reasons people use it instead shared memory communication when using threads.
Also, getting used to the syntax that the library enforces (get used to lambdas!) if one is not used to state of the art c++11 programs it can be a bit of a mind-twister at first. Then, the triviality of remembering all the clients that registered with the server was the only other caveat.
STRONGLY RECOMMENDED.
We have a traditional VB application which are used for Organization operations. Now we are building a Hybrid application developed by using HTML5,CSS and Javascript which is targeted on Google Chromium desktop container. Now we are planning to provide a way to exchange large data like employees records between both of these 2 applications. Now my specific question is
What are the different ways to achieve communication between Chromium desktop container and VB application to exchange large chunks of data?
Sounds a bit painful no matter what.
Chrome Apps Architecture
All external processes are isolated from the app.
This would seem to suggest the obvious course is to use cloud data services, whether on public or private clouds.
I suspect that for political as well as practical reasons no cloud vendor goes to the trouble to provide VB/VBA-friendly APIs for their services. Mainly nobody wants to deal with support issues from the teeming hordes of casual coders the VB community is saddled with.
The VB6 community hasn't stepped up and taken care of this themselves either.
If you can limp along with the burdens of ".Net Inter Clop" (the usual MS answer) that might be a way to exploit existing API implementations.
Otherwise you might roll your own cloud. I see a few obvious services you'd want to implement in your cloud with lightweight APIs easily implemented in both of your development ecosystems:
Bulk Storage. I suggest WebDAV, which IIS supports. If you eschew the locking features then WebDAV API implementations are pretty easy in both JS and VB. Or buy (or scrounge open source) implementations of a more complete WebDAV client library.
DBMS. Pick any, implement a simple REST-like XML over HTTP API. Relatively easy to implement.
Push Notifications. I'd write a custom service accepting long-duration TCP connections from all clients, and with protocols and workflow à la Amazon SNS or Google Cloud Messaging. Such a service would be generally light in resource consumption but you'd probably want a dedicated box with OS tweaks to support a large number of active TCP connections.
Maybe optionally a message queue service?
Nothing novel here, these are all well established patterns.
All of the tools to do that are pretty off-the-shelf whether you want your cloud servers to be based on Windows, Linux, or generically Java anywhere.
Most of the effort will probably go into developing a consistent authentication model, access control model, and of course an integrated administration interface, monitoring, and logging to help keep operating overhead low and uptime high. Well, that and developer docs and training.
Ok, still a lot of work. Too bad there isn't a "cloud in the box" with the API libraries you'd need that you can buy off the shelf today.
Or perhaps I'm missing something obvious?
I am making a turn-based card game that will have clients, a lobby server and a game server. What methologies are there that are both cross-language and bi-directional (e.g. client request -> server server response-> client, as well as server request-> client client response -> server)?
I have looked into JMS but believe it is too heavyweight for my needs (this program will just be small scale, and I don't think the complexities make this solution suitable). I have briefly looked into REST but I believe that wouldn't fit the bi-directional requirement. Of course, there is RMI but I would like to be able to develop clients in C++ and other languages as another learning exercise.
If I'm honest, I'm at a bit of loss because I don't want to use JMS as I think it is too complex for this, but I don't think just using TCP sockets and say using a basic XML based protocol for the messages will provide a good structure of communication for the program.
The research lab that I do some work with develops a system called "Object Oriented Distributed Semantic Services."
We leverage some work that we do with cross-language serialization to allow you to write clients/servers in different languages, and the underlying messages to be a format that be serialized and deserialized by clients/servers regardless of their implementation language.
Right now we mostly support Java/ObjectiveC. You can take a look at the chat room tutorial, which should give you a basic idea of how requests / responses work.
http://ecologylab.net/research/simplGuide/oodss/index.html
OODSS is designed to work well for game scenarios... the system was originally written to support a game one of the researchers in our lab was working on. The original paper on OODSS discusses the development of a game from the ground up. That may work out well for you: http://ecologylab.net/technicalReports/oodss_TR_10_01.pdf
You could apply a similar idea to allow for multiple clients in languages that aren't supported yet. (you may have to write some serialization/deserialization code on your own, to start.)
Good luck! Hope that helps!
EDIT: I forgot to include the prime candidate for web applications: JSON over HTTP/REST + Comet. It combines the best features of the others (below)
Persevere basically bundles everything I need in a server
The focus for Java and such is definitely on Comet servers, but it can't be too hard to use/write a client.
I'm embarking on an application with a server holding data, and clients executing operations which would affect this data, and thus require some sort of notification across all interested/subscribed clients.
The first client will probably be written in WPF, but we'll probably need to add clients written in other languages, e.g. a Java (Swing?) client, and possibly, a web client.
The actual question(s): What protocol should I use to implement this? How easy would it be to integrate with JS, Java and .NET (precisely, C#) clients?
I could use several interfaces/protocols, but it'd be easier overall to use one that is interoperable. Given that interoperability is important, I have researched a few options:
JSON-RPC
lightweight
supports notifications
The only .NET lib I could find, Jayrock doesn't support notifications
works well with JS
also true of XML-based stuff (and possibly, even binary protocols) BUT this would probably be more efficient, thanks to native support
Protobuf/Thrift
IDL makes it easy to spit out model classes in each language
doesn't seem to support notifications
Thrift comes with RPC out of the box, but protobufs don't
not sure about JS
XML-RPC
simple enough, but doesn't support notifications
SOAP: I'm not even sure about this one; I haven't grokked this yet.
seems rather complex
Message Queues/PubSub approach: Not strictly a protocol, but might be fitting
I hardly know anything about them, and got lost amongst the buzzwords`-- JMS? **MQ?
Perhaps combined with some RPC mechanism above, although that might not be strictly necessary, and possibly, overkill.
Other options are, of course, welcome.
I am partial to the pub/sub design you've suggested. I'd take a look at ZeroMQ. It has bindings to C#, Java, and many other platforms.
Bindings list: http://www.zeromq.org/bindings:clr
I also found this conversation on the ZeroMQ dev listing that may answer some questions you have about multiple clients and ZeroMQ: http://lists.zeromq.org/pipermail/zeromq-dev/2010-February/002146.html
As XMPP was mentioned, SIP has a similar functionality. This might be more accessible for you.
We use Servoy for this. It does automatic data broadcasting to web-clients and java-clients. I'm not sure if broadcasts can be sent to other platforms, you might be able to find an answer to that on their forum.
If you want to easily publish events to clients across networks, you may wish to look at a the XMPP standard. (Used by, amongst other things, Jabber and Google Talk.)
See the extension for publish-subscribe functionality.
There are a number of libraries in different languages including C#, Java and Javascript.
You can use SOAP over HTTP to modify the data on the server and SOAP over SMTP to notify the subscribed clients.
OR
The server doesn't know anything about the subscription and the clients call the server by timeout to track updates they are interested in, using XML-RPC, SOAP (generated using WSDL), or simply HTTP GET if there is no need to pass back complex data on tracking.