I have several dealers connecting to one Router socket. The dealers send data asynchronously and the Router gathers/processes the data.
What I need to do is find a way to know when a dealer has stopped sending data/disconnected from the router socket.
I build a map of all the connection identities.
I have a monitor connected the the router port and I receive notifications for connects/disconnects.
The problem is I can't find a way to identify which dealer the monitor notifications are for. The notifications only give me a FD which is of little use.
Is there a way to map between notifications and connection IDs?
If not mistaken, the zeroMq allows you to determine only the fact of connection/disconnection.
In my opinion, a good solution would be something like this: when the notification is received, ROUTER must send to all connected dealers a heartbeat message with a timeout and the one who did not respond - disconnected.
Related
I'm building a distributed system and I would like asynchronous send and recv from both sides with blocking after high water mark.
PUSH/PULL sockets works great, but I wasn't able to bind a PUSH socket. Meaning I can't have a client-PUSH to server-PULL and a server-PUSH to client-PULL, if the client is behind a firewall, since the server can't connect to the client.
In the book, the following is written, but I can't find an example of it.
"REQ to DEALER: you could in theory do this, but it would break if you added a second REQ because DEALER has no way of sending a reply to the original peer. Thus the REQ socket would get confused, and/or return messages meant for another client." http://zguide.zeromq.org/php:chapter3
I only need a one-to-one connection, so this would in theory work for me.
My question is, what is the best practice to obtain asynchronous send and recv with ZeroMQ without dropping packets?
Most ZeroMQ sockets can both bind (listen on a specific port, acting as a server) and connect (acting as a client). It is usually not related to the data flow. See the guide for more info.
Try to bind on your servers PUSH socket and connect from your clients PULL socket.
Could anyone please provide a real-world example of using zmq with the router/dealer pattern, and explain its advantage over the more simple publish/subscribe pattern? Thanks.
A 'real-world' example: Stock market simulation
Router socket is the server (or 'Market'), Dealer sockets are the clients (or 'Traders').
Traders place buy and sell orders by sending relevant 'order'
messages to the Market.
The Market responds immediately with an 'acknowledge' message
Sometime later when an order is fulfilled the Market sends
'order complete' messages to all involved traders.
This sort of behavior would be quite cumbersome to implement with pub/sub as you would need both Market and Traders to run a Publisher and a Subscriber socket to allow the 2 way communication. There would also be privacy concerns if all completed transactions were 'published' rather than sent direct to the relevant traders. (Trader B should not get to know that Trader A bought or sold something).
What makes Router sockets different
Sending and receiving from Router sockets are slightly more complicated to allow the asynchronous responses:
Any incoming message has an identity frame prepended to the incoming message upon receipt, this indicates which client the message came from.
The first frame of any sent message is removed and used to identify which client to send the response to.
The 'identity' is a string and will be set to something unique per connected client by default, but you can set custom identities on client sockets via socket options.
Dealers send their client ID with each message. Router is able to accept connections from multiple dealers and each time a message is received, it is able to discern which client sent the message. It is also able to send messages to specific clients by referring to their IDs.
Real life example: game server lets clients join a room until it is full. It keeps track of each client ID in the room. When room is full, it loops over each client ID within that room and sends them a "game started" message.
I'm using the pattern: Router-Dealer
Router send messages to Dealer(A) all the time.
Dealer just receive message (SOCKOPT_IDENTITY: A).
Work fine in this time.
Then
i press Ctrl-C to stop the Dealer , and reconnect.
Now i got nothing ~~
Is there anything wrong?
How can i fix it?
Thanks.
You are using the wrong kind of sockets and/or direction of messages.
I would be easier to understand what you would like to do if you write what you are using the socket for.
Assuming that you want to use Dealer and Router sockets then you should send the message in the other direction (Dealer => Router).
The Router socket connects many inputs to a single output or a single input to many outputs. This is done by adding/removing an identifier to the message, that describes where it came from or where it should go.
So in your case when you are trying to send a message from the router, the router socket will try to remove the identifier from the message, so that it can understand which connection it should forward the message on. So if you really want this then you need to associate your dealer connection with an identifier and then add that to the message that your sending through the router.
But I'm pretty sure that you actually want to use another combination of zmq sockets.
I have a single publisher application (PUB) which has N number of subscribers (SUB)
These subscribers need to be able to catch up if they are restarted, or fall down and miss messages.
We have implemented a simple event store that the publisher writes to.
We have implemented a CatchupService which can query the event store and send missed messages to the subscriber.
We have implemented in the subscriber a PUSH socket which sends a request for missed messages.
The subscriber also has a PULL socket which listens for missed messages on a seperate port.
The subscriber will:
Detect a gap
Send a request for missed messages to our CatchupService, the request also contains the address on which to send the results to.
The catchup service has a PULL socket on which it listens for requests
When the CatchupService receives a request it starts a worker thread which:
Gets the missed messages
Opens a PUSH socket connecting to the subscribers PULL socket
Sends the missed messages to the subscriber.
This seems to work quite well however we are unsure if we are using the right socket types for this sort of application. Are these correct or should be using a different pattern.
Sounds okay. Otherwise 0MQ is able to recovery from message loss when peers go offline for a short time. Take a look at the Socket Options and specifically option ZMQ_SNDHWM.
I don't know just how guaranteed the 0MQ recovery mechanisms are so maybe you're best to stay with what you've got, but it is something to be aware of.
I'm working on an app that notifies multiple Workers when a reboot is about to happen and then waits for ALL the Workers to do perform some tasks then send an ACK before rebooting. The number of Workers can change so somehow my app will need to know how many Workers are currently subscribed so that it knows that every Worker has sent an ACK.
Is a pub/sub approach the best way for doing this? Does it provide a way to figure out how many subscribers are currently connected? Should my app use a REP socket to listen for ACK from the Workers? Is there a more elegant way of designing this?
Thanks
Is a pub/sub approach the best way for doing this?
Using pub/sub from the server to broadcast a "server reboot" message is fine for the workers who get the message, but it's not full-proof. Slow-joiner syndrome may prevent a worker (or workers) from receiving the message. To address that, the server, once it publishes a reboot message, should continue publishing that message until all workers respond with ACK, but that creates a new problem: how does the server keep track of all workers to ensure it receives all necessary ACK's?
Does it provide a way to figure out how many subscribers are
currently connected?
No. Exposing that information breaks ZeroMq's abstraction model which hides the physical details of the connection and connected peers. You can send heartbeat messages periodically from server to workers over pub/sub; workers respond with a logical node id (WorkerNode1, etc), and the server keeps track of each worker in a hashtable along with a future expiration time. When a worker responds to a hearbeat, the server simply resets the future expiration for that worker; the server should periodically check the hashtable and remove expired workers.
That's the best you can do for keeping track of workers. The shorter the expiration, the more accurate the worker list reflects.
Should my app use a REP socket to listen for ACK from the Workers? Is
there a more elegant way of designing this?
REQ/REP sockets have limited uses. I'd use PUB on the server for sending reboot and heartbeat messages; ROUTER to receive ACK's. The workers should use DEALER for sending ACK's (and anything else), and SUB for receiving heartbeats/reboots. ROUTER and DEALER are bi-directional and fully asynchronous, and the most versatile; can't go wrong.
Hope it helps!