How to use ZMQ when Server on IP other than the client - zeromq

I learned how to use ZeroMQ on a localhost, but I failed to do it on a remote IP.
Q1: Do I need a broker?If so,Q2: which broker and how to do it.?
Update:
OK. I'm using the ZMQ Weather Update example but with a remote IP ( not the localhost ). Here is what I do using C# ZMQ bindings ( however, I'm OK to use any other language ):
ZMQ Server:
using (var context = new ZContext())
using (var publisher = new ZSocket(context, ZSocketType.PUB))
{
string address = "tcp://*:5001";
publisher.Bind(address);
publisher.Send("msg")
}
Proxy:
using (var context = new ZContext())
using (var frontend = new ZSocket(context, ZSocketType.XSUB))
using (var backend = new ZSocket(context, ZSocketType.XPUB))
{
// Frontend is where the weather server sits
string localhost = "tcp://127.0.0.1:5001";
Console.WriteLine("I: Connecting to {0}", localhost);
frontend.Connect(localhost);
// Backend is our public endpoint for subscribers
string remoteIP = "216.123.23.98"; // For example
var tcpAddress = string.Format("tcp://{0}:8100", remoteIP); // I also tried localhost address here
Console.WriteLine("I: Binding on {0}", tcpAddress);
backend.Bind(tcpAddress);
var epgmAddress = string.Format("epgm://localhost;{0}:8100", remoteIP);
Console.WriteLine("I: Binding on {0}", epgmAddress);
backend.Bind(epgmAddress);
using (var subscription = ZFrame.Create(1))
{
subscription.Write(new byte[] { 0x1 }, 0, 1);
backend.Send(subscription);
}
// Run the proxy until the user interrupts us
ZContext.Proxy(frontend, backend);
}
Client:
using (var context = new ZContext())
using (var subscriber = new ZSocket(context, ZSocketType.SUB))
{
string remoteIP = "tcp://216.123.23.98"; //For example
Console.WriteLine("I: Connecting to {0}…", remoteIP);
subscriber.Connect(connect_to);
// Subscribe to zipcode
string zipCode = args[0];
Console.WriteLine("I: Subscribing to zip code {0}…", zipCode);
subscriber.Subscribe(zipCode);
// Process 10 updates
int i = 0;
long total_temperature = 0;
for (; i < 20; ++i)
{
ZError err;
using (var replyFrame = subscriber.ReceiveFrame(out err))
{
string reply = replyFrame.ReadString(Encoding.ASCII);
Console.WriteLine(reply);
total_temperature += Convert.ToInt64(reply.Split(' ')[1]);
}
}
Console.WriteLine("Average temperature for zipcode '{0}' was {1}", zipCode, (total_temperature / i));
}
When I run this I get error in Server and error in proxy - server gets
Invalid end point
and proxy gets EINVAL(22):
Invalid argument at ZeroMQ.ZSocket.Bind(String endpoint)

A1: No, ZeroMQ is a Broker-less messaging framework.
A2: N/A
How to repair the code?
All the services need to obey respective transport-class addressing rules, for the TCP/IP case - both the .bind() / .connect() methods have to state both parts of the IP:PORT# specification ( with some aids from DNS-resolution for the IP-part, but the :PORT#-part is still mandatory )
( which the source-code does not meet in client, ref.:
subscriber.Connect(connect_to);
whereas there ought be also a Proxy-side matching :PORT#, i.e.:8100, specified, for a correct .connect() ).
For the clarity and for avoiding a port#-collision, remove the epgm transport class from the code.

Related

AMQPNETLITE - ActiveMQ Artemis (Red Hat AMQ) - autocreate multi-consumer multicast queue

This qeuestion is on consuming the messages using AMQP in .Net. The documentation recommends amqpnetlite: https://access.redhat.com/documentation/en-us/red_hat_amq/7.0/html-single/using_the_amq_.net_client/index
On subscribing to an address using AMQPNetLite, the address and the queue will be auto-created. The auto-created queue is always "unicast" though. I have not been able to auto-create
a multicast queue
that allowed any number of consumers.
Code:
private async Task RenewSession()
{
Connect = await Connection.Factory.CreateAsync(new Address("amqp://admin:admin#localhost:5672"), new Open() {ContainerId = "client-1"});
MqSession = new Session(Connect);
var receiver = new ReceiverLink(MqSession, DEFAULT_SUBSCRIPTION_NAME, GetSource("test-topic"), null);
receiver.Start(100, OnMessage);
}
private Source GetSource(string address)
{
var source = new Source
{
Address = address,
ExpiryPolicy = new Symbol("never"),
Durable = 2,
DefaultOutcome = new Modified
{
DeliveryFailed = true,
UndeliverableHere = false
}
};
return source;
}
Maybe I am missing some flags?
in AMQP, you choose between autocreating a queue (anycast routing) or a topic (multicast routing) by setting a capability.
The capability should be either new Symbol("queue") or new Symbol("topic").
public class SimpleAmqpTest
{
[Fact]
public async Task TestHelloWorld()
{
Address address = new Address("amqp://guest:guest#localhost:5672");
Connection connection = await Connection.Factory.CreateAsync(address);
Session session = new Session(connection);
Message message = new Message("Hello AMQP");
Target target = new Target
{
Address = "q1",
Capabilities = new Symbol[] { new Symbol("queue") }
};
SenderLink sender = new SenderLink(session, "sender-link", target, null);
await sender.SendAsync(message);
Source source = new Source
{
Address = "q1",
Capabilities = new Symbol[] { new Symbol("queue") }
};
ReceiverLink receiver = new ReceiverLink(session, "receiver-link", source, null);
message = await receiver.ReceiveAsync();
receiver.Accept(message);
await sender.CloseAsync();
await receiver.CloseAsync();
await session.CloseAsync();
await connection.CloseAsync();
}
}
Have a look at https://github.com/Azure/amqpnetlite/issues/286, where the code comes from.
You can choose whether the default routing will be multicast or anycast by setting default-address-routing-type in broker.xml, everything documented at https://activemq.apache.org/artemis/docs/2.6.0/address-model.html
The broker's multicastPrefix and anycastPrefix feature is not implemented for AMQP. https://issues.jboss.org/browse/ENTMQBR-795

NetMQ client to client messaging

I'm trying to create an rpc program to communicate hosts located on different networks and chose Router-Dealer configuration of NetMQ provided here: http://netmq.readthedocs.io/en/latest/router-dealer/#router-dealer
But the problem is that router always selects a random dealer when routing a message to backend.
Code which I used :
using (var frontend = new RouterSocket(string.Format("#tcp://{0}:{1}", "127.0.0.1", "5556")))//"#tcp://10.0.2.218:5559"
using (var backend = new DealerSocket(string.Format("#tcp://{0}:{1}", "127.0.0.1", "5557")))//"#tcp://10.0.2.218:5560"
{
// Handler for messages coming in to the frontend
frontend.ReceiveReady += (s, e) =>
{
Console.WriteLine("message arrived on frontEnd");
NetMQMessage msg = e.Socket.ReceiveMultipartMessage();
string clientAddress = msg[0].ConvertToString();
Console.WriteLine("Sending to :" + clientAddress);
//TODO: Make routing here
backend.SendMultipartMessage(msg); // Relay this message to the backend };
// Handler for messages coming in to the backend
backend.ReceiveReady += (s, e) =>
{
Console.WriteLine("message arrived on backend");
var msg = e.Socket.ReceiveMultipartMessage();
frontend.SendMultipartMessage(msg); // Relay this message to the frontend
};
using (var poller = new NetMQPoller { backend, frontend })
{
// Listen out for events on both sockets and raise events when messages come in
poller.Run();
}
}
Code for Client:
using (var client = new RequestSocket(">tcp://" + "127.0.0.1" + ":5556"))
{
var messageBytes = UTF8Encoding.UTF8.GetBytes("Hello");
var messageToServer = new NetMQMessage();
//messageToServer.AppendEmptyFrame();
messageToServer.Append("Server2");
messageToServer.Append(messageBytes);
WriteToConsoleVoid("======================================");
WriteToConsoleVoid(" OUTGOING MESSAGE TO SERVER ");
WriteToConsoleVoid("======================================");
//PrintFrames("Client Sending", messageToServer);
client.SendMultipartMessage(messageToServer);
NetMQMessage serverMessage = client.ReceiveMultipartMessage();
WriteToConsoleVoid("======================================");
WriteToConsoleVoid(" INCOMING MESSAGE FROM SERVER");
WriteToConsoleVoid("======================================");
//PrintFrames("Server receiving", clientMessage);
byte[] rpcByteArray = null;
if (serverMessage.FrameCount == 3)
{
var clientAddress = serverMessage[0];
rpcByteArray = serverMessage[2].ToByteArray();
}
WriteToConsoleVoid("======================================");
Console.ReadLine();
}
Code for Dealer:
using (var server = new ResponseSocket())
{
server.Options.Identity = UTF8Encoding.UTF8.GetBytes(confItem.ResponseServerID);
Console.WriteLine("Server ID:" + confItem.ResponseServerID);
server.Connect(string.Format("tcp://{0}:{1}", "127.0.0.1", "5557"));
using (var poller = new NetMQPoller { server })
{
server.ReceiveReady += (s, a) =>
{
byte[] response = null;
NetMQMessage serverMessage = null;
try
{
serverMessage = a.Socket.ReceiveMultipartMessage();
}
catch (Exception ex)
{
Console.WriteLine("Exception on ReceiveMultipartMessage : " + ex.ToString());
//continue;
}
byte[] eaBody = null;
string clientAddress = "";
if (serverMessage.FrameCount == 2)
{
clientAddress = serverMessage[0].ConvertToString();
Console.WriteLine("ClientAddress:" + clientAddress);
eaBody = serverMessage[1].ToByteArray();
Console.WriteLine("Received message from remote computer: {0} bytes , CurrentID : {1}", eaBody.Length, confItem.ResponseServerID);
}
else
{
Console.WriteLine("Received message from remote computer: CurrentID : {0}", confItem.ResponseServerID);
}
};
poller.Run();
}
}
Is it possible to choose a specific backend on frontend.ReceiveReady?
Thanks!
Your backend should be router as well. You need the worker to register or you need to know all the available workers and their identity. When send on the backend push the worker identity at the beginning of the server.
Take a look at the Majordomo example in the zeromq guide:
http://zguide.zeromq.org/page:all#toc72
http://zguide.zeromq.org/page:all#toc98

NetMQ why is "SendReady" needed for Req-Rep?

I have a problem that I managed to fix... However I'm a little concerned as I don't really understand why the solution worked;
I am using NetMQ, and specifically a NetMQ poller which has a number of sockets, one of which is a REQ-REP pair.
I have a queue of requests which get dequeued into requests, and the server handles each request type as required and sends back an appropriate response. This had been working without issue, however when I tried to add in an additional request type the system stopped working as expected; what would occur is that the request would reach the server, the server would send the response... and the client would not receive it. The message would not be received at the client until the server was shut down (unusual behavior!).
I had been managing the REQ-REP pair with a flag that I set before sending a request, and reset on receipt of a reply. I managed to fix the issue by only triggering replies within the "SendReady" event of the REQ socket - this automagically fixed all of my issues, however I can't really find anything in the documentation that tells me why the socket might not have been in the "sendready" state, or what this actually does.
Any information that could be shed on why this is working now would be great :)
Cheers.
Edit: Source
Client:
"Subscribe" is run as a separate thread to the UI
private void Subscribe(string address)
{
using (var req = new RequestSocket(address + ":5555"))
using (var sub = new SubscriberSocket(address + ":5556"))
using (var poller = new NetMQPoller { req, sub })
{
// Send program code when a request for a code update is received
sub.ReceiveReady += (s, a) =>
{
var type = sub.ReceiveFrameString();
var reply = sub.ReceiveFrameString();
switch (type)
{
case "Type1":
manager.ChangeValue(reply);
break;
case "Type2":
string[] args = reply.Split(',');
eventAggregator.PublishOnUIThread(new MyEvent(args[0], (SimObjectActionEventType)Enum.Parse(typeof(MyEventType), args[1])));
break;
}
};
req.ReceiveReady += Req_ReceiveReady;
poller.RunAsync();
sub.Connect(address + ":5556");
sub.SubscribeToAnyTopic();
sub.Options.ReceiveHighWatermark = 10;
reqQueue = new Queue<string[]>();
reqQueue.Enqueue(new string[] { "InitialiseClient", "" });
req_sending = false;
while (programRunning)
{
if (reqQueue.Count > 0 && !req_sending)
{
req_sending = true;
string[] request = reqQueue.Dequeue();
Console.WriteLine("Sending " + request[0] + " " + request[1]);
req.SendMoreFrame(request[0]).SendFrame(request[1]);
}
Thread.Sleep(1);
}
}
}
private void Req_ReceiveReady(object sender, NetMQSocketEventArgs e)
{
var req = e.Socket;
var messageType = req.ReceiveFrameString();
Console.WriteLine("Received {0}", messageType);
switch (messageType)
{
case "Reply1":
// Receive action
break;
case "Reply2":
// Receive action
break;
case "Reply3":
// Receive action
break;
}
req_sending = false;
}
Server:
using (var rep = new ResponseSocket("#tcp://*:5555"))
using (var pub = new PublisherSocket("#tcp://*:5556"))
using (var beacon = new NetMQBeacon())
using (var poller = new NetMQPoller { rep, pub, beacon })
{
// Send program code when a request for a code update is received
rep.ReceiveReady += (s, a) =>
{
var messageType = rep.ReceiveFrameString();
var message = rep.ReceiveFrameString();
Console.WriteLine("Received {0} - Content: {1}", messageType, message);
switch (messageType)
{
case "InitialiseClient":
// Send
rep.SendMoreFrame("Reply1").SendFrame(repData);
break;
case "Req2":
// do something
rep.SendMoreFrame("Reply2").SendFrame("RequestOK");
break;
case "Req3":
args = message.Split(',');
if (args.Length == 2)
{
// Do Something
rep.SendMoreFrame("Reply3").SendFrame("RequestOK");
}
else
{
rep.SendMoreFrame("Ack").SendFrame("RequestError - incorrect argument format");
}
break;
case "Req4":
args = message.Split(',');
if (args.Length == 2)
{
requestData = //do something
rep.SendMoreFrame("Reply4").SendFrame(requestData);
}
else
{
rep.SendMoreFrame("Ack").SendFrame("RequestError - incorrect argument format");
}
break;
default:
rep.SendMoreFrame("Ack").SendFrame("Error");
break;
}
};
// setup discovery beacon with 1 second interval
beacon.Configure(5555);
beacon.Publish("server", TimeSpan.FromSeconds(1));
// start the poller
poller.RunAsync();
// run the simulation loop
while (serverRunning)
{
// todo - make this operate for efficiently
// push updated variable values to clients
foreach (string[] message in pubQueue)
{
pub.SendMoreFrame(message[0]).SendFrame(message[1]);
}
pubQueue.Clear();
Thread.Sleep(2);
}
poller.StopAsync();
}
You are using the Request socket from multiple threads, which is not supported. You are sending on the main thread and receiving on the poller thread.
Instead of using regular queue try to use NetMQQueue, you can add it to the poller and enqueue from the UI thread. Then the sending is happening on the poller thread as well as the receiving.
You can read the docs here:
http://netmq.readthedocs.io/en/latest/queue/
Only thing I can think of is that the REP socket is ready to send only after you actually received a message fully (all parts).

Get the server name and ip address in C# 2010

Get the server name and ip address in C# 2010
I want to get the IP address of the server. The following code comes from:
public static void DoGetHostEntry(string hostname)
{
IPHostEntry host;
host = Dns.GetHostEntry(hostname);
MessageBox.Show("GetHostEntry({0}) returns:"+ hostname);
foreach (IPAddress ip in host.AddressList)
{
MessageBox.Show(" {0}"+ ip.ToString());
}
}
This code must know the name of the server computer.
AddressFamily in System.Net.IPAddress
System.Net.IPAddress i;
string HostName = i.AddressFamily.ToString();
Error ------------->Use of unassigned local variable 'i'
How can I get the name of the server computer?
To get the host name you can do the following:
string name = System.Net.Dns.GetHostName();
If you want the hostname and (first IPv4) IP of your computer use the following:
string name = System.Net.Dns.GetHostName();
host = System.Net.Dns.GetHostEntry(name);
System.Net.IPAddress ip = host.AddressList.Where(n => n.AddressFamily == System.Net.Sockets.AddressFamily.InterNetwork).First();
The name and the ip will hold the info for the local computer.
The server could then send out the ip via a udp multicast and the client on the network would just join a known multicast address that is not specific to the server.
multicast example.
First of all, you need to figure out for yourself that error(unassigned local variable) and learn why it is coming(it is very basic), before looking for some magical code that will do the job for you.
And secondly, there is no magical code. I am no socket programmer but it seems to me that in your application running on the client machines, you need to hardcode the name of your server. If you don't want to do that, program in such a way that only your server machine will listen on a particular port and all client machines will listen on a different port. Thus, each machine in the LAN can enumerate over the available machines and establish/determine the client server connection/relation for the first time. and that approach is still very ugly unless you are writing a virus or something.
public string[] ServerName()
{
string[] strIP = DisplayIPAddresses();
int CountIP = 0;
for (int i = 0; i < strIP.Length; i++)
{
if (strIP[i] != null)
CountIP++;
}
string[] name = new string[CountIP];
for (int i = 0; i < strIP.Length; i++)
{
if (strIP[i] != null)
{
try
{
name[i] = System.Net.Dns.GetHostEntry(strIP[i]).HostName;
}
catch
{
continue;
}
}
}
return name;
}
public string[] DisplayIPAddresses()
{
StringBuilder sb = new StringBuilder();
// Get a list of all network interfaces (usually one per network card, dialup, and VPN connection)
NetworkInterface[] networkInterfaces = NetworkInterface.GetAllNetworkInterfaces();
int i = -1;
string[] s = new string[networkInterfaces.Length];
foreach (NetworkInterface network in networkInterfaces)
{
i++;
if (network.OperationalStatus == OperationalStatus.Up)
{
if (network.NetworkInterfaceType == NetworkInterfaceType.Tunnel) continue;
if (network.NetworkInterfaceType == NetworkInterfaceType.Tunnel) continue;
//GatewayIPAddressInformationCollection GATE = network.GetIPProperties().GatewayAddresses;
// Read the IP configuration for each network
IPInterfaceProperties properties = network.GetIPProperties();
//discard those who do not have a real gateaway
if (properties.GatewayAddresses.Count > 0)
{
bool good = false;
foreach (GatewayIPAddressInformation gInfo in properties.GatewayAddresses)
{
//not a true gateaway (VmWare Lan)
if (!gInfo.Address.ToString().Equals("0.0.0.0"))
{
s[i] = gInfo.Address.ToString();
good = true;
break;
}
}
if (!good)
{
continue;
}
}
else
{
continue;
}
}
}
return s;
}

Why ServiceStack.Redis throw error instead of trying to connect to another read instance?

I successfully installed Redis on two machines and made then work as Master-Slave.
I tested some code to check if replication work and everything is ok.
My client manager looks like
var manager = new PooledRedisClientManager(new[] { "MasterIP:6379" }, new[] { "MasterIP:6379", "SlaveIP:6379" });
But now i shutdown my master instance and when i test my code again i get an error like client cant connect to Master server.
p.s For read i use GetReadOnlyCacheClient();
I repeated my code and i notice that client first is getting Master (error cant connect), then when i run my code again client is getting Slave, then again when i run my code client is getting master and so on.
I downloaded source code on ServiceStack.Redis client. I just wanted to check when that error happens and here is the code.
private void Connect()
{
socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
{
SendTimeout = SendTimeout,
ReceiveTimeout = ReceiveTimeout
};
try
{
if (ConnectTimeout == 0)
{
socket.Connect(Host, Port);
}
else
{
var connectResult = socket.BeginConnect(Host, Port, null, null);
connectResult.AsyncWaitHandle.WaitOne(ConnectTimeout, true);
}
if (!socket.Connected)
{
socket.Close();
socket = null;
return;
}
Bstream = new BufferedStream(new NetworkStream(socket), 16 * 1024);
if (Password != null)
SendExpectSuccess(Commands.Auth, Password.ToUtf8Bytes());
db = 0;
var ipEndpoint = socket.LocalEndPoint as IPEndPoint;
clientPort = ipEndpoint != null ? ipEndpoint.Port : -1;
lastCommand = null;
lastSocketException = null;
LastConnectedAtTimestamp = Stopwatch.GetTimestamp();
if (ConnectionFilter != null)
{
ConnectionFilter(this);
}
}
catch (SocketException ex)
{
if (socket != null)
socket.Close();
socket = null;
HadExceptions = true;
var throwEx = new RedisException("could not connect to redis Instance at " + Host + ":" + Port, ex);
log.Error(throwEx.Message, ex);
throw throwEx;
}
}
I really dont understand this code, couze project is really big, but i think there is no If this server fails, then try to get from another host in read only server list if any other exist
I can make some kind of mine custom logic to check if fail to try to get another read only instance..but isnt this client supported to be ready about this issue?

Resources