Why won't the socket open in bulk at once? - macos

Question

OSX socket programming
Why won't the socket open in bulk at once?
I am using Intel macOS Big Sur 11.5.1
The connection test is being conducted with the local docker nginx server.
We are using Golang to conduct tests with the following codes:
func TestBulkConnection(t *testing.T) {
var worker = 1000
var wg sync.WaitGroup
for i := 0; i < worker; i++ {
wg.Add(1)
//time.Sleep(time.Millisecond * 10)
go func(id int) {
conn, err := net.Dial("tcp", "localhost:9000")
if err != nil {
log.Fatal(err)
}
defer conn.Close()
defer wg.Done()
fmt.Println("waiting... ", id)
time.Sleep(time.Second * 30)
}(i)
}
wg.Wait()
}
1000 goroutine connecting to nginx only.
After the connection, the sleep() function was used to make sure that nothing was done.
The client created 1000 goroutines, but found that only 200 to 300 nginx and connections worked and the rest did not (we confirmed with netstat-anv | grep 9000).
When connecting, it was confirmed that all connections are well established when the sleep() function is executed.
With nginx and client code, when spun from private ubuntu 18.04, the connection was confirmed at once.
I think it's a problem on the nginx server side, but I don't know the cause of the problem.
Is there a difference between Mac and Ubuntu in this test?
Added
let net = require('net');
for (let i = 0; i < 1000; i++) {
const socket = net.connect({ port: 9000 });
socket.on('connect', function () {
console.log('connected to server!');
});
}
netstat -anv | grep 9000 | wc -l
2000 connection ok
Added
The following links are used to increase the file descriptors of OSX.
https://wilsonmar.github.io/maximum-limits/
In recovery mode, 'csrutil disable' was also executed.
$ ulimit -a
-t: cpu time (seconds) unlimited
-f: file size (blocks) unlimited
-d: data seg size (kbytes) unlimited
-s: stack size (kbytes) 8192
-c: core file size (blocks) 0
-v: address space (kbytes) unlimited
-l: locked-in-memory size (kbytes) unlimited
-u: processes 2048
-n: file descriptors 524288
But still.
$ netstat -anv | grep 9000 | wc -l
287

Answer 1

Every unix based OS has limits on the number of file descriptors a process can open. Also, every socket consumes a file descriptor, just like opening a file on disk or your stdin, stdout & stderr.
MacOS by default sets the limit of file descriptors to 256 per process. Therefore your statement that your Go process stops at around 200-300 connections sounds right. In theory it should stop being able to open sockets after 253 connections (3 file descriptors already assigned to stdin, stdout & stderr).
Ubuntu on the other hand sets the default limit to 1024. You will still have this issue on Ubuntu but it will be able to open more sockets before you hit the wall.
On both systems you can check this limit by running the following command:
ulimit -n
Note: you can run ulimit -a to see all the limits.
On MacOS you can change this limit temporarily system-wide (it will reset after reboot) with the following command:
sudo launchctl limit maxfiles 1024 1024
On Ubuntu you can change this limit in your current shell with the following command:
ulimit -n 1024
This should let your 1000 connections to succeed. Note that the number does not have to be powers of 2. You can pass 1500 for example. Just remember that your process use file descriptors for things other than sockets.