ZMQ PUB/SUB socket high latency under low load - zeromq

Question

I am testing ZMQ PUB/SUB socket performance with NetMQ.
In the test program below, SUB socket is used as a server to receive messages and PUB socket is used as a client to receive messages. The topic of a message is simply a sequence number and the content of a message is the time when it is sent by the PUB client. The SUB server measures the latency by comparing between the time when the message is received time and the time (which is the message sent time) inside the message.
For 1,000 messages sent over ~1 second, the performance results I find are the average latency ~4ms and maximum latency ~40ms. However, based on http://wiki.zeromq.org/results:more-precise-0mq-tests, ZMQ's latency should be in the magnitude of microseconds which is much less than the latency I measured.
Do I use ZMQ PUB/SUB sockets wrongly? Is there any way to reduce the latency? Any ideas will be welcome and appreciated.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using NetMQ;
using NetMQ.Sockets;
namespace NetMQ_Hello
{
class Program
{
static void Main(string[] args)
{
if (args.Length == 0)
{
Console.WriteLine("Please use \"server\" or \"client\" as the first argument.");
return;
}
string mode = args[0];
if (mode == "server")
{
RunSubAsServer();
}
else if (mode == "client")
{
RunPubAsClient();
}
}
static void RunPubAsClient()
{
string address = "tcp://localhost:1234";
PublisherSocket pubSocket = new PublisherSocket();
pubSocket.Connect(address);
Thread.Sleep(10);
Console.WriteLine("Starting to send messages...");
DateTime startTime = DateTime.UtcNow;
for (int i = 0; i < 1000; i++)
{
long data = DateTime.UtcNow.ToBinary();
byte[][] frames = new byte[2][] { BitConverter.GetBytes(i), BitConverter.GetBytes(data) };
pubSocket.SendMultipartBytes(frames);
Console.WriteLine(string.Format("Sent: {0}, {1}", i, data));
}
DateTime endTime = DateTime.UtcNow;
Console.WriteLine(string.Format("1,000 messages sent in {0}", endTime - startTime));
Console.WriteLine("Press enter to exit...");
Console.ReadLine();
pubSocket.Disconnect(address);
}
static void RunSubAsServer()
{
string address = "tcp://*:1234";
SubscriberSocket subSocket = new SubscriberSocket();
subSocket.Bind(address);
subSocket.SubscribeToAnyTopic();
Thread.Sleep(10);
// Stats
List<TimeSpan> latencies = new List<TimeSpan>();
Console.WriteLine("Starting to receive messages...");
List<byte[]> frames = new List<byte[]>();
while (true)
{
if (subSocket.TryReceiveMultipartBytes(
timeout: TimeSpan.FromSeconds(1),
frames: ref frames,
expectedFrameCount: 2))
{
DateTime now = DateTime.UtcNow;
int topic = BitConverter.ToInt32(frames[0], 0);
DateTime sentTime = DateTime.FromBinary(BitConverter.ToInt64(frames[1], 0));
TimeSpan latency = now - sentTime;
Console.WriteLine(string.Format("Received: {0}, {1}, delay {2}",
topic, sentTime, latency));
latencies.Add(latency);
if (topic == 1000 - 1)
{
break;
}
}
}
int n = latencies.Count;
double max = latencies.Max().TotalMilliseconds;
double mean = latencies.Sum(s => s.TotalMilliseconds) / n;
Console.WriteLine(String.Format("Latency\nMax: {0}ms\nMean: {1}ms", max, mean));
Console.WriteLine("Press enter to exit...");
Console.ReadLine();
subSocket.Unbind(address);
}
}
}
Sample output from Sub server:
Received: 990, 9/30/2020 12:53:11 PM, delay 00:00:00.0010009
Received: 991, 9/30/2020 12:53:11 PM, delay 00:00:00
Received: 992, 9/30/2020 12:53:11 PM, delay 00:00:00.0019993
Received: 993, 9/30/2020 12:53:11 PM, delay 00:00:00.0010016
Received: 994, 9/30/2020 12:53:11 PM, delay 00:00:00
Received: 995, 9/30/2020 12:53:11 PM, delay 00:00:00.0019993
Received: 996, 9/30/2020 12:53:11 PM, delay 00:00:00.0010007
Received: 997, 9/30/2020 12:53:11 PM, delay 00:00:00
Received: 998, 9/30/2020 12:53:11 PM, delay 00:00:00
Received: 999, 9/30/2020 12:53:11 PM, delay 00:00:00.0030002
Latency
Max: 39ms
Mean: 4.49101730769231ms