ZeroMQ choose recipient - zeromq

I'm new to ZeroMQ (and to networking in general), and have a question about using ZeroMQ in a setup where multiple clients connect to a single server. My situation is as follows:
--1 server
--multiple clients
--Clients send messages to server: I've already figured out how to do this part.
--Server sends messages to a specific client: This is the part I'm having trouble with. When certain events get handled on the server, the server will need to send a message to a specific client -- not all clients. In other words, the server will need to be able to choose which client to send a given message to.
Right now, this is my server code:
using (NetMQContext ctx = NetMQContext.Create())
{
using (var server = ctx.CreateResponseSocket())
{
server.Bind(#"tcp://127.0.0.1:5555");
while (true)
{
string fromClientMessage = server.ReceiveString();
Console.WriteLine("From Client: {0}", fromClientMessage);
server.Send("ack"); // There is no overload for the 'Send'
method that takes an IP address as an argument!
}
}
}
I have a feeling that the problem is that my design is wrong, and that the ResponseSocket type isn't meant to be used in the way that I want to use it. Since I'm new to this, any advice is very much appreciated!

when using the Response socket you always replying to the client that sent you the message. So the Request-Response socket types together are just simple request response.
To more complicated scenarios you probably want to use Dealer-Router.
With router the first frame of each message is the routing id (the identity of the client that sent you the message)
so your example with router will look like:
using (NetMQContext ctx = NetMQContext.Create())
{
using (var server = ctx.CreateRouterSocket())
{
server.Bind(#"tcp://127.0.0.1:5555");
while (true)
{
byte[] routingId = server.Receive();
string fromClientMessage = server.ReceiveString();
Console.WriteLine("From Client: {0}", fromClientMessage);
server.SendMore(routingId).Send("ack");
}
}
}
I also suggest to read the zeromq guide, it will probably answer most of your questions.

Related

Detect server disconnect in gRPC Go client

I have a gRPC service simmilar to below
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
I need the client to maintain a long living gRPC connection to the server so that if the server goes down, the client can reconnect and issue SayHello() again.
Based on my understanding there are a few options:
Pass in a statsHandler to grpc.Dial and add retry logic in HandleConn()
Add a new ClientStreaming API that maybe sends a message every few seconds. Check for server side stream close errors and implement retry logic.
Not sure if there is a recommended way for my use case and would appreciate any help.

Connection time out in jpos client

I am using jpos client (In one of the class of java Spring MVC Program) to connect the ISO8585 based server, however due to some reason server is not able to respond back, due to which my program keeps waiting for the response and results in hanging my program. So what is the proper way to implement connection timeout?
My client program look like this:
public FieldsModal sendFundTransfer(FieldsModal field){
try {
JposLogger logger = new JposLogger(ISO_LOG_LOCATION);
org.jpos.iso.ISOPackager customPackager = new GenericPackager(ISO_PACKAGER);
ISOChannel channel = new PostChannel(ISO_SERVER_IP, Integer.parseInt(ISO_SERVER_PORT), customPackager);// live
logger.jposlogconfig(channel);
channel.connect();
log4j.info("Connection established using PostChannel");
ISOMsg m = new ISOMsg();
m.set(0, field.getMti());
//m.set(2, field.getField2());
m.set(3, field.getField3());
m.set(4, field.getField4());
m.set(11, field.getField11());
m.set(12, field.getField12());
m.set(17, field.getField17());
m.set(24, field.getField24());
m.set(32, field.getField32());
m.set(34, field.getField34());
m.set(41, field.getField41());
m.set(43, field.getField43());
m.set(46, field.getField46());
m.set(49, field.getField49());
m.set(102,field.getField102());
m.set(103,field.getField103());
m.set(123, field.getField123());
m.set(125, field.getField125());
m.set(126, field.getField126());
m.set(127, field.getField127());
m.setPackager(customPackager);
System.out.println(ISOUtil.hexdump(m.pack()));
channel.send(m);
log4j.info("Message has been send");
ISOMsg r = channel.receive();
r.setPackager(customPackager);
System.out.println(ISOUtil.hexdump(r.pack()));
channel.disconnect();
}catch (Exception err) {
System.out.println("sendFundTransfer : " + err);
}
return field;
}
Well the real proper way would be to use Q2. Given you don't need a persistent connection you coud just set a timeout for the channel.
PostChannel channel = new PostChannel(ISO_SERVER_IP, Integer.parseInt(ISO_SERVER_PORT), customPackager);// live
channel.setTimeout(timeout); //timeout in millies.
This way channel will autodisconnect if nothing happens during the time specified by timeout , and your call to receive will throw an exception.
The alternative is using Q2 and a mux (see QMUX, for which you need to run Q2, or ISOMUX which is kind of deprecated).

Websocket connection. Why are we making a call to ws://echo.websocket.org?

I am writing some websocket code and I have this so far:
window.onload = function() {
// Get references to elements on the page.
var form = document.getElementById('message-form');
var messageField = document.getElementById('message');
var messagesList = document.getElementById('messages');
var socketStatus = document.getElementById('status');
var closeBtn = document.getElementById('close');
var socket = new WebSocket('ws://echo.websocket.org');
// Show a connected message when the WebSocket is opened.
socket.onopen = function(event) {
socketStatus.innerHTML = 'Connected to: ' + event.currentTarget.url;
socketStatus.className = 'open';
};
// Handle any errors that occur.
socket.onerror = function(error) {
console.log('WebSocket Error: ' + error);
};
form.onsubmit = function(e) {
e.preventDefault();
// Retrieve the message from the textarea.
var message = messageField.value;
// Send the message through the WebSocket.
socket.send(message);
// Add the message to the messages list.
messagesList.innerHTML += '<li class="sent"><span>Sent:</span>' + message +
'</li>';
// Clear out the message field.
messageField.value = '';
return false;
};
socket.onmessage = function(event) {
var message = event.data;
messagesList.innerHTML += '<li class="received"><span>Received:</span>' +
message + '</li>';
};
closeBtn.onclick = function(e) {
e.preventDefault();
// Close the WebSocket.
socket.close();
return false;
};
socket.onclose = function(event) {
socketStatus.innerHTML = 'Disconnected from WebSocket.';
socketStatus.className = 'closed';
};
};
What is this code doing:
var socket = new WebSocket('ws://echo.websocket.org');
What url is that? When I visit there with my browser it does not exist but it seems to be important as I can't simply jus replace that url with random strings. What does it do? Is Websocket an external API?
I'm looking at the network tab and I see this:
Request URL: ws://echo.websocket.org/
Request Method: GET
Status Code: 101 Web Socket Protocol Handshake
conceptually, what is going on? Why do I need to make a request to an external site to use Websockets?
echo.websocket.org provides a webSocket server that lets you make a webSocket connection to it and then it simply echos back to you anything that you send it. It's there primarily for testing and demo purposes.
The code you show looks like a demo/testing app designed to run in a browser web page for a webSocket connection which you can access something similar here: https://websocket.org/echo.html.
A URL starting with ws:// indicates a connection that intends to use the webSocket protocol.
What is this code doing:
var socket = new WebSocket('ws://echo.websocket.org');
It is making a webSocket connection to a webSocket server at echo.websocket.org.
What url is that?
That is a webSocket URL that indicates the intent to use the webSocket protocol to connect and talk to that host. This is not something you type in the URL bar of a browser. It's something that is used by a programming language (such as Javascript in your browser).
Is Websocket an external API?
It's a protocol that specifies a means of connecting, a security scheme, a packet data format, etc... You could say that the http protocol is to the webSocket protocol as the English language is to Japanese. They are different means of communicating. The specification for the webSocket protocol is here: https://www.rfc-editor.org/rfc/rfc6455.
It's also designed to fit well into the http/browser world and to be friendly with infrastructure that was originally designed for http requests. Just searching for "what is websocket" on Google will turn up all sorts of descriptive articles. The Wikipedia page for webSocket provides a pretty good overview.
There is tons written on the web about what the webSocket protocol is and is useful for so I won't repeat that here. You can see a tutorial on webSocket clients here and a tutorial on webSocket servers here.
In a nutshell, it's designed to be a long lasting, continuous connection (supported in all modern browsers) that allows a client to connect to a server and then maintain a continuous connection for (potentially) a long duration. While that connection is open, data can be easily sent both ways over the webSocket. The primary reason people use it is when they want the server to be able to send data directly to the client in a timely fashion without making the client continuously ask the server over and over again if it has any new data. Once a webSocket connection is established, the server can just "push" data to the client at any time.
I'm looking at the network tab and I see this. Conceptually, what is going on?
Request URL: ws://echo.websocket.org/
Request Method: GET Status Code:
101 Web Socket Protocol Handshake
Those are the first steps of establishing a webSocket connection. You can see a more complete description of how that connection works here: How socket.io works. That post talks about socket.io which is another layer built on top of webSocket, but the underlying protocol is webSocket.
Why do I need to make a request to an external site to use Websockets?
A webSocket's purpose is to connect a client to a server (so data can then be sent between them) so it would only be used when connecting to some server somewhere.

connected users' list in sails socket

i've currently started using sailsJS with angularJs at frontend alognwith socket for realtime communiction.
Sailsjs gives built-in support to websocket through "sails.io.js".On client side after adding this library this code is added to angular's chat controller.
Client side code
io.socket.get('/chat',{token:token},function(users){
console.log(users);
});
chatController's action on sails side is like this.
Server side code
chat: function (req, res) {
console.log(req.isSocket);
//this gives true when called through client.
})
infact very new to sails so i want suggestion that how to maintain connected user's list because m not using redis as storage purpose.adapter is memory.array is not a good idea because it'll vanish when restart a server.m using sails version of 0.11.0.
thanx in advance.
I'm somewhat new but learning fast, these suggestions should get you there unless someone else responds with greatness...
They changed it in 11 but in 10.5 I use sockets.js in config folder and on connect I store the session data in an array with their socket.
I created a service in APIs/service that contains the array and socket associate function.
For v11 you can't do that exactly the same, but you can make your first 'hello' from the client call a function in a controller that calls the associate function.
A couple tips would be don't let the client just tell you who they are, as in don't just take the username from the params but get it from req.session
(This assumes you have user auth setup)
In my case I have
in api/services/Z.js (putting the file here makes it's functions globally accessible)
var socketList = [];
module.exports = {
associateSocket: function(session, socket) { // send in your username(string) socket(object) id(mongoId) and this will push to the socketlist for lookups
sails.log.debug("associate socket called!",socketList.length)
var iHateYou = socketList
//DEBUG
var sList = socketList
var util = require('util')
if (session.authenticated){
var username = session.user.auth.username
var userId = session.user.id
// sails.log.debug("Z: associating new user!",username,userId,socket)
if (username && socket && userId) {
sList[sList.length]= {
username: session.user.auth.username,
socket: socket,
userId: session.user.id,
};
sails.log.debug('push run!!! currentsocketList length',socketList.length)
} else sails.log("Z.associateSocket called with invalid data", username, userId, authId, socket)
}else{sails.log.warn("Z.associateSocket: a socket attempted to associate itself without being logged in")}
},
}
in my config/sockets.js
onConnect: function(session, socket) {
Z.associateSocket(session,socket)
if (session.user && session.user.auth){
sails.log("config/sockets.js: "+session.user.auth.username+" CONNECT! session:",session)
}else sails.log.warn('connect called on socket without an auth, the client thinks it already has a session, so we need to fix this')
// By default, do nothing.
},
Then you can make add some functions to your services file to do lookups based on username and passwords, remove sockets that are disconnecting and the like (I'm using waterlock for my auth at the moment, although debating the switch back to sails-generate-auth)
Remove your onConnect and dicconnect function from config/sockets.js.

zeromq message pattern for many to many request/reply

I'm very new to ZeroMQ. I've read the guide and am currently going through the examples as well as looking at other relevant info around the web. I'm having some indecision with what message pattern(s) to use or if I should use a combination of 2 patterns.
I have an existing software application that has a home grown messaging system that is in need of replacement. I have a fairly simple architecture:
|Client|<----->|driver1|
|
|---|driverN|
Only one "client" connects to a driver at a time currently, and there may be many drivers.
(in actuality, the client, in this case, isn't truly my client application, but a middleman of sorts. For this discussion, it can be treated as a client)
Messaging:
The client issues commands to the driver.
The drivers return status/state information in response to commands.
The drivers produce data elements (i.e. Not status/state information)
Some Client messages go to all connected devices, some are directed only to a single driver.
Drivers may exist on the same system or remotely on a LAN. This is not a public network.
I'm currently thinking that I would have a pub and sub socket on each driver and a sub/pub socket on the client. Messages shouldn't be dropped once a connection is made. I assume that the client would subscribe to the different driver data types and the driver would then subscribe to the clients command messages.
Important considerations: low latency, lowest possible bandwidth overhead.
I would appreciate any suggestions or recommendations! Thanks in advance!
You picked a great learning exercise, that's for sure!
Read up on these, they provide the basic implementation for request/reply using a custom router-to-router proxy with polling and should address your client-to-device problem.
https://github.com/imatix/zguide/blob/master/examples/C/lbbroker.c
https://github.com/imatix/zguide/blob/master/examples/C/lbbroker2.c
The solution is synchronous, so any request sent from the client blocks until it gets a response. Personally I would use async for both request and reply for total flexibility, but that solution is way more complex. There are, however, examples in the book called Freelance and Dealer/Router that illustrate async request/reply.
Here's an example for synchronous many-to-many request/reply. You MUST know how ZeroMq enveloping works to fully understand the mechanics of this approach; see example lbbroker1.
Client
Set the client identity with setIdentity(); important for response routing.
Client sends requests for device1, device2, etc, in a loop; if device exists, status messages returned from specific device, otherwise, "no device" returned to client.
Socket client = context.socket(ZMQ.REQ);
client.setIdentity("client1".getBytes());
client.connect("tcp://localhost:5550");
for( int i = 0; i < 5; i++){
client.send("device" + i);
String reply = client.recvStr();
log("Received message: " + reply);
Thread.currentThread().sleep(1000);
}
Device
Device sets id just like client for unique routing.
Device sends device.send("DEVICEREADY") to server to signal online availability.
Device does recvStr() three times to read full envelope from server.
String deviceId = "device1"
Socket device = context.socket(ZMQ.REQ);
device.setIdentity(deviceId.getBytes());
device.connect( "tcp://localhost:5560");
device.send( "DEVICEREADY");
while (!Thread.currentThread().isInterrupted()) {
String clientAddress = device.recvStr();
String empty = device.recvStr();
String clientRequest = device.recvStr();
//create envelope to send reply to same client who made request
device.sendMore(clientAddress);
device.sendMore("");
device.send( "stauts on " + deviceId + " is ok");
}
Server (Router/Router)
A custom proxy using ROUTER sockets; clients connect to frontend ROUTER socket while devices connect to backend router. Server polls on both sockets for messages.
Context context = ZMQ.context(1);
Socket frontend = context.socket(ZMQ.ROUTER);
Socket backend = context.socket(ZMQ.ROUTER);
frontend.bind( "tcp://localhost:5550");
backend.bind( "tcp://localhost:5560");
Poller poller = new Poller(2);
poller.register(frontend, Poller.POLLIN);
poller.register(backend, Poller.POLLIN);
while (!Thread.currentThread().isInterrupted()) {
poller.poll();
//frontend poller
if (poller.pollin(0)) {
String clientId = frontend.recvStr();
String empty = frontend.recvStr(); //empty frame
String deviceId = frontend.recvStr();
//if client is requesting to talk to nonexistent deviceId,
//return message "no device", otherwise, create envelope and send
//request on backend router to device.
if( deviceMap.get( deviceId) == null ){
frontend.sendMore(clientId);
frontend.sendMore("");
frontend.send("no deviceId: " + deviceId);
} else {
//request envelope addressed to specific device
backend.sendMore(deviceId);
backend.sendMore("");
backend.sendMore(clientId);
backend.sendMore("");
backend.send("hello from " + clientId);
}
}
//backend poller
if(poller.pollin(1)){
String deviceId = backend.recvStr();
String empty = backend.recvStr();
String clientId = backend.recvStr();
//device signaling it's ready
//store deviceId in map, don't send a response
if( clientId.equals("DEVICEREADY"))
deviceMap.put(deviceId, deviceId);
else {
//the device is sending a response to a client
//create envelope addressed to client, send on frontend socket
empty = backend.recvStr();
String reply = backend.recvStr();
frontend.sendMore(clientId);
frontend.sendMore("");
frontend.send(reply);
}
}
}

Resources