Release UDP port used by dead process on OS X - macos

Question

I am on OS X 10.11.6 and trying to run a program that normally listens on UDP port 8008 upon startup.
This program normally also spawns a couple of helper child processes during its operation, but the port is bound by the parent process.
Unfortunately when exiting the program, sometimes the port remains open, even though the program (parent + children) no longer exist.
When this happens, if I try to run the program again it naturally fails with a EADDRINUSE error, and in these cases no matter what I try, the only solution I found was to reboot the machine.
I'm having a hard time believing that I cannot release the port without a reboot.
Here are some diagnostics that I ran so far (I ran all of these with and without sudo):
Find the process using port 8008 with lsof:
$ lsof -i -n -P | grep UDP | grep 8008
But surprisingly doesn't return any results.
However, I had more luck with netstat:
$ netstat -tulnvp udp | grep 8008
udp4 0 0 *.8008 *.* 196724 9216 47205 0
So, the port is indeed bound, and the culprit is pid 47205, however:
$ ps aux | grep 47205
Doesn't return anything. The same thing for PIDs 47206 and 47207 (most certainly the PIDs assigned to the children). I also tried other variations of the grep (program name, path, etc).
I also looked for any process reporting 47205 as its parent:
$ ps -axo pid,ppid,command | grep 47205
So the children processes are also clearly dead.
Not being able to kill anything, I tried to SIGHUP launchd in the hope that it might remove any zombie child processes:
$ sudo kill HUP 1
$ sudo kill -s HUP 1
But alas, netstat still shows the port bound.
Lastly, I tried to restart the loopback interface:
$ sudo ifconfig lo down
$ sudo ifconfig lo up
But again, to no effect.
I have waited several hours since the program last ran, so I'm pretty sure any timeout would have happened by now, but the port just won't get released.
Any ideas on how to force release the port without a reboot?
Edit:
The program in question is the electron-wrapped Patchwork.
This question originates from this github issue.
Although finding a solution/bugfix that prevents the issue from ocurring in the first place would be ideal, I'm also interested in ways to manually close that port from the terminal

Answer 1

In your code, after you create the socket, but before the bind call, invoke the following:
int val = 1;
setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val));
Then call bind. The above will allow the socket bind to succeed even if the port is in use.
Two processes, attempting a recvfrom on the same port, will result in one of the processes receiving the packet, but not the other. And it's not deterministic which one will. So make sure you don't actually have two processes legitimately running and sharing the port.

Answer 2

one related question: mac changed the Behavior of SO_REUSEADDR and SO_REUSEPORT:
Behavior of SO_REUSEADDR and SO_REUSEPORT changed?
and I am the maintainer of iptux[1], if I use SO_REUSEPORT, the program can start, but I can't receive msg from this port, all the message go to unclosed port as a black hole.
[1] https://github.com/iptux-src/iptux

Answer 3

It is indeed possible to close the port manually w/o restarting the machine. On various linux flavors this is usually done w/ GDB by issuing syscalls masquerading as a the process (for example close(fd) syscall on the sockets file descriptor).
The process for that:
Open a UDP port: netcat -u 127.0.0.1 33333.
Check the UDP port: netstat -npu (u for UDP), which will give you the PID that occupies that port.
Run: lsof -np $pid for that PID to get the filedescriptor for the socket.
Then run GDB for that PID: sudo gdb -p 73599
When inside GDB run call close(file_descriptor)
Example:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
netcat 73599 ubunt cwd DIR 259,2 4096 13895497 /home/ubunt/Downloads
netcat 73599 ubunt rtd DIR 259,2 4096 2 /
netcat 73599 ubunt txt REG 259,2 31248 28835938 /bin/nc.openbsd
netcat 73599 ubunt mem REG 259,2 47600 23990813 /lib/x86_64-linux-gnu/libnss_files-2.23.so
netcat 73599 ubunt mem REG 259,2 1868984 23990714 /lib/x86_64-linux-gnu/libc-2.23.so
netcat 73599 ubunt mem REG 259,2 101200 23990866 /lib/x86_64-linux-gnu/libresolv-2.23.so
netcat 73599 ubunt mem REG 259,2 81040 23990710 /lib/x86_64-linux-gnu/libbsd.so.0.8.2
netcat 73599 ubunt mem REG 259,2 162632 23990686 /lib/x86_64-linux-gnu/ld-2.23.so
netcat 73599 ubunt 0u CHR 136,19 0t0 22 /dev/pts/19
netcat 73599 ubunt 1u CHR 136,19 0t0 22 /dev/pts/19
netcat 73599 ubunt 2u CHR 136,19 0t0 22 /dev/pts/19
netcat 73599 ubunt 3u IPv4 22142418 0t0 UDP 127.0.0.1:45255->127.0.0.1:33333
Then GDB:
$sudo gdb -p 73599
...
(gdb) call close(3u)
$1 = 0
You will see that the port is no longer there:
ubunt#ubunt-MS-7A94:~$ lsof -np 73599
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
netcat 73599 ubunt cwd DIR 259,2 4096 13895497 /home/ubunt/Downloads
netcat 73599 ubunt rtd DIR 259,2 4096 2 /
netcat 73599 ubunt txt REG 259,2 31248 28835938 /bin/nc.openbsd
netcat 73599 ubunt mem REG 259,2 47600 23990813 /lib/x86_64-linux-gnu/libnss_files-2.23.so
netcat 73599 ubunt mem REG 259,2 1868984 23990714 /lib/x86_64-linux-gnu/libc-2.23.so
netcat 73599 ubunt mem REG 259,2 101200 23990866 /lib/x86_64-linux-gnu/libresolv-2.23.so
netcat 73599 ubunt mem REG 259,2 81040 23990710 /lib/x86_64-linux-gnu/libbsd.so.0.8.2
netcat 73599 ubunt mem REG 259,2 162632 23990686 /lib/x86_64-linux-gnu/ld-2.23.so
netcat 73599 ubunt 0u CHR 136,19 0t0 22 /dev/pts/19
netcat 73599 ubunt 1u CHR 136,19 0t0 22 /dev/pts/19
netcat 73599 ubunt 2u CHR 136,19 0t0 22 /dev/pts/19
GDB is available for MacOS, so it should work for your case as well.

Answer 4

System may keep socket open till I/O process still in progress. Even when the process died but not explicitly closed the socket. If your socket not closed at hours most probably you're missing something. Try to use low-level kernel investigation instead of top-level utilites like netstat or lsof.
Disclaimer
I'm not OS X expert, and most commands for linux. I still leave it there if someone else will have same problem.
1. Try to see if socket still alive (optional)
I may suggest to check socket communication.
tcpdump -A -s0 port 8080 and tcpdump -A -s0 -ilo port 8080
If you see any data transfered over socket you can be sure the process active. Or may be one of its childs. Later you can catch the pid with strace
2. Check the process and its status
Linux have wonderful procfs. You can get so many things from there. And sure you can see all opened file descriptors
ls -al /proc/47205/fd
If you see output and /proc/47205 exists the pid not released nevertheless ps shows. You will see all opened files and its fds.It looks like
133 -> socket:[32242509]
Where 133 is a fd number
Unfortunately OS X don't have /proc filesystem. The alternative command I found.
procexp 47205 fds
But I'm not sure its 100% working.
3. Closing the file descriptor (socket) in another process
In linux there is nice command
fuser -k -n udp 8080
This will explicitly close all processes blocking port. It seems OS X may have fuser too
Another real hackers way is to connect to process with gdb and run commands inside the process, because file descriptor numbers valid only with in process environment,exactly as #Mindaugas Bernatavičius wrote:
gdb -p 47205
>call shutdown([fd_number],2)
>call close([fd_number])
There is third way, when possible you can just restart whole network. Plese note, down and up just loopback interface is not enough. In linux run
systemctl restart network
4.What to do to prevent socket stuck in system
You should always ensure socked closed before your program exits. I seen many issues with nodejs that sockets stays opened. Calling Socket.destroy() will solve the problem
May be put your socket destroy code here, before exiting the app:
app.on('close', function (code) {
// User closed the app. Kill the host process.
process.exit();
});

Answer 5

Your question look similar to :
release-renew-ip-address-via-terminal-in-os-x
How to reset network stack
Reset TCP/IP is OSX
As you said:
Lastly, I tried to restart the loopback interface:
$ sudo ifconfig lo down
$ sudo ifconfig lo up
Did you try to retart all avaible network interfaces (lan or wlan) and not only the loopback) ?
Instead of ifconfig you may use also native MacOS command utility (from here) to power off then power on the device itself (adapt en0 to your device name):
networksetup -setairportpower en0 off
networksetup -setairportpower en0 on
You may also finally try to release and renew DHCP with:
sudo dhclient -v -r
Regards