Maybe I missed this in the docs, but how do you use ZeroMQ for simple signaling between multiple nodes? Something like REQ to REQ, with no REPs.
Example: I sometimes want to tell all other nodes to invalidate cache pages or notify them that something happened.
Request-reply won't work because I don't want the requester to block waiting for an empty response. I want to allow multiple signals to build up at the server.
Publish-subscribe feels wrong because I'd have to subscribe to everything, and start two sockets, one for each direction of communication.
PAIRs don't support automatically reconnecting and have other limitations.
Is pub-sub the best way to go? Or am I better just to use a traditional socket, write to both ends, and handle disconnections/reconnecting?
What you want is a dealer socket in your client side and a dealer socket on your server side. If you want to be able to send message to a specific node from the server side, then you better use a router socket in the server.
Related
In golang, when you make a client/server combination with Listen/Accept on the server and Dial on the client, as far as I can tell writing to the client doesn't actually guarantee the full transmission chain. IE when you say:
_len, _err := conn.Write([]byte("sent"))
it is possible that the text you send may reach the buffer of the client machine but not the client itself. In other words, if the client becomes unavailable, _err may still be unset and _len may still show the correct length of bytes being sent. I noticed this by killing the connection between my server and client and monitoring the return status of conn.Write() manually and seeing that it didn't show an error.
In other words, just using these tools out of the box won't guarantee delivery, and I was hoping that there was a go library that implemented a client acknowledgement to insure this further guarantee. I'd like to say something like:
_len, _err := _write(conn, "sent\n")
and have guarantees that if the client goes away that _err will be set accordingly - assuming a specific timeout between the send and an acknowledgement from the client.
Is there a standard library like this? I could write my own wrapper to do this, but I have a feeling the logic to do this correctly would be somewhat intricate.
thanks much again for any info,
Ed
Is there a standard library like this?
There is no standard library for this. How application level acknowledgements are done is specified in the specific application protocol, i.e. a protocol like HTTP (web), SMTP (mail delivery), SIP (VoIP) etc. And this protocol must be spoken in both client and server. There is no way for a client to enforce some application level acknowledgement if it is not part of the protocol specification and the server will not explicitly send it.
Therefore you either need to pick an application protocol which supports the semantics you need and use it on both client and server. Or you need to define your own application protocol which supports the exact semantics you need and implement it yourself on both sides.
I am building an application which can have multiple gRPC servers and definitely will have multiple gRPC clients, I wanted to know, how to identify on server side that this is the client I am talking to and only send data to that client. I am using bidirectional streaming RPC and right now the data gets broadcasted to every client and I don't want that. What functions in go gRPC make it possible or how can I implement it?
There are two ways to read this question. One way is to read it as the auth problem as answered before. The second way is how I read it, as a connection/session problem.
When the client connects, the grpc server will invoke a function to implement the call in its own goroutine, and that function will be only talking to the client that initiated that call. So, the struct you registered as your grpc server will be shared among many connections, but each connection will run in its own goroutine, and will only talk to the client that initiated it. That also means you have to make sure the grpc server implementation is thread-safe.
You mentioned data is being broadcasted to every client? There is no broadcast in grpc, are you sure that's what's happening?
This sounds like a common authentication/authorization problem that ultimately won't have much to do with gRPC or Go.
You need a way for a client to indicate who they are. Personally I'm a fan of JWTs. In a standard HTTP request, there are authorization headers that can indicate who is making the request. Similarly, gRPC supports meta data attached to each remote call. In my current work project, every call must have a JWT in the meta data or else I don't process the request. Every call except the login endpoint that is.
I haven't looked into how to get details like a gRPC client's IP address or other information about a client's connection but chances are that anything provided by gRPC's generated code is something potentially faked by the client. When architected correctly, JWTs can offer cryptographic confidence that the client is who they claim to be.
I'm building a distributed system and I would like asynchronous send and recv from both sides with blocking after high water mark.
PUSH/PULL sockets works great, but I wasn't able to bind a PUSH socket. Meaning I can't have a client-PUSH to server-PULL and a server-PUSH to client-PULL, if the client is behind a firewall, since the server can't connect to the client.
In the book, the following is written, but I can't find an example of it.
"REQ to DEALER: you could in theory do this, but it would break if you added a second REQ because DEALER has no way of sending a reply to the original peer. Thus the REQ socket would get confused, and/or return messages meant for another client." http://zguide.zeromq.org/php:chapter3
I only need a one-to-one connection, so this would in theory work for me.
My question is, what is the best practice to obtain asynchronous send and recv with ZeroMQ without dropping packets?
Most ZeroMQ sockets can both bind (listen on a specific port, acting as a server) and connect (acting as a client). It is usually not related to the data flow. See the guide for more info.
Try to bind on your servers PUSH socket and connect from your clients PULL socket.
My network has one server and possibly thousands of clients. In most cases, the server sends a command to one of the clients and the client immediately sends a response. But there are cases wherein the client initiates the communication to the server, that is the client sends a status update to the server but then does not need to wait for the server's reply.
I am quite new to zeromq, I would like to ask what kind of pattern suits this kind of communication?
I think The Asynchronous Client/Server Pattern is exactly what you need.
How can a client both subscribe and listen to replies with zeromq?
That is, on the client side I'd like to run a loop which only receives messages and selectively sends requests, and on the server side I'd like to publish most of the time, but to sometimes receive requests as well.
It looks like I'll have to have two different sockets - one for each mode of communication. Is it possible to avoid that and on the server side receive "request notifications" from the socket on a zeromq callback thread while pushing messages to the socket in my own thread?
I am awfully new to ZeroMQ, so I'm not sure if what you want is considered best-practice or not. However, a solution using multiple sockets is pretty simple using zmq_poll.
The basic idea would be to have both client and server:
open a socket for pub/sub
open a socket for req/rep
multiplex sends and receives between the two sockets in a loop using zmq_poll in an infinite loop
process req/rep and pub/sub events within the loop as they occur
Using zmq_poll in this manner with multiple sockets is nice because it avoids threads altogether. The 0MQ guide has a good example here. Note that in that example, they use a timeout of -1 in zmq_poll, which causes it to block until at least one event occurs on any of the multiplexed sockets, but it's pretty common to use a timeout of x milliseconds or something if your loop needs to do some other work as well.
You can use 2 threads to handle the different sockets. The challenge is that if you need to share data between threads, you need to synchronize it in a safe way.
The alternative is to use the ZeroMQ Poller to select the sockets that have new data on them. The process would then use a single loop in the way bjlaub explained.
This could be accomplished using a variation/subset of the Majordomo Protocol. Here's the idea:
Your server will be a router socket, and your clients will be dealer sockets. Upon connecting to the server, the client needs to send some kind of subscription or "hello" message (of your design). The server receives that packet, but (being a router socket) also receives the ID of that client. When the server needs to send something to that client (through your design), it sends it to that ID. The client can send and receive at will, since it is a dealer socket.