Minimizing ZeroMQ round trip latency - zeromq

Question

My question is about minimizing the latency between a ZMQ client and server
I have the following modified Hello World ZMQ (JeroMQ 0.5.1)
import org.zeromq.SocketType;
import org.zeromq.ZContext;
import org.zeromq.ZMQ;
public class server {
public static void main(String[] args) {
try (ZContext context = new ZContext()) {
// Socket to talk to clients
ZMQ.Socket socket = context.createSocket(SocketType.REP);
socket.bind("tcp://*:5555");
while (!Thread.currentThread().isInterrupted()) {
byte[] reply = socket.recv(0);
System.out.println(
"Received " + ": [" + reply.length+ "]"
);
String response = "world";
socket.send(response.getBytes(ZMQ.CHARSET), 0);
}
}
}
}
and client:
import org.zeromq.SocketType;
import org.zeromq.ZContext;
import org.zeromq.ZMQ;
public class client {
public static void main(String[] args)
{
try (ZContext context = new ZContext()) {
// Socket to talk to server
System.out.println("Connecting to hello world server" + args[0] + args[1] + args[2] );
ZMQ.Socket socket = context.createSocket(SocketType.REQ);
socket.connect("tcp://"+args[0]+":"+args[1]);
for (int requestNbr = 1; requestNbr != 10; requestNbr++) {
byte[] request = new byte[requestNbr*(Integer.parseInt(args[2]))];
System.out.println("Sending Hello " + requestNbr);
long time = System.nanoTime();
socket.send(request, 0);
byte[] reply = socket.recv(0);
double restime = (System.nanoTime() - time)/1000000.0;
System.out.println(
"Received " + new String(reply, ZMQ.CHARSET) + " " +
requestNbr + " " + restime
);
}
}
}
}
I'm running the server and the client over a network with latency (160ms round trip). I create the latency using tc on both the client and the server:
tc qdisc del dev eth0 root
tc class add dev eth0 parent 1: classid 1:155 htb rate 1000mbit
tc filter add dev eth0 parent 1: protocol ip prio 1 u32 flowid 1:155 match ip dst 192.168.181.1/24
tc qdisc add dev eth0 parent 1:155 handle 155: netem delay $t1 $dt1 distribution normal
Now when I run java -jar client.jar 192.168.181.3 5555 100000 I get the following output:
Sending Hello 1
Received world 1 1103.392783
Sending Hello 2
Received world 2 322.553512
Sending Hello 3
Received world 3 478.10143
Sending Hello 4
Received world 4 606.396567
Sending Hello 5
Received world 5 641.465041
Sending Hello 6
Received world 6 772.961712
Sending Hello 7
Received world 7 910.848674
Sending Hello 8
Received world 8 966.694224
Sending Hello 9
Received world 9 940.645636
which means that as we increase the size of the message, it takes more round trips to send the message and receive the ack (you can play with the message size to see for yourself). I was wondering what I need to do to prevent that from happening, that is: send everything in one go and minimize the latency to the roundtrip time.
Note: In my original application, I'm using a REQ-ROUTER pattern as I have multiple clients, but the issue with the latency and large messages lingers on