What is the proper way to create a well-behaved Unix or Linux daemon in Ruby?
What is the definition of a well-behaved daemon anyway, and how would one write such a program in Ruby?
According to Stevens's Advanced Programming in the UNIX Environment chapter 13, this is the procedure to make a well-behaved Unix daemon:
Fork and have the parent exit. This makes the shell or boot script think the command is done. Also, the child process is guaranteed not to be a process group leader (a prerequisite for setsid next)
Call setsid to create a new session. This does three things:
The process becomes a session leader of a new session
The process becomes the process group leader of a new process group
The process has no controlling terminal
Optionally fork again and have the parent exit. This guarantes that the daemon is not a session leader nor can it acquire a controlling terminal (under SVR4)
Change the current working directory to / to avoid interfering with mounting and unmounting
Set file mode creation mask to 000 to allow creation of files with any required permission later.
Close unneeded file descriptors inherited from the parent (there is no controlling terminal anyway): stdout, stderr, and stdin.
Nowadays there is a file to track the PID which is used heavily by Linux distribution boot scripts. Be sure to write out the PID of the grandchild, either the return value of the second fork (step 3) or the value of getpid() after step 3.
Here is a Ruby implementation, mostly translated from the book, but with the double-fork and writing out the daemon PID.
# Example double-forking Unix daemon initializer.
raise 'Must run as root' if Process.euid != 0
raise 'First fork failed' if (pid = fork) == -1
exit unless pid.nil?
Process.setsid
raise 'Second fork failed' if (pid = fork) == -1
exit unless pid.nil?
puts "Daemon pid: #{Process.pid}" # Or save it somewhere, etc.
Dir.chdir '/'
File.umask 0000
STDIN.reopen '/dev/null'
STDOUT.reopen '/dev/null', 'a'
STDERR.reopen STDOUT
Following on from Jason's awesome response I have written a fuller implementation here:
https://gist.github.com/1372491/b76b60fb1842bf0507f47869ab19ad50a045b214
I have implemented logging in addition to the double fork and writing of the pid to file.
Another interesting implementation is in Unicorn:
https://github.com/defunkt/unicorn/blob/master/lib/unicorn/launcher.rb
Related
I'm trying to write a ruby script that:
Run a command/script
Stores the command's process pid in a file so I can check if it's still running later, and
the command should keep running after the ruby code exits.
I'm successful in steps 1 and 2, but it looks like the started script (i.e, the child process) terminates once the ruby code is finished.
This is the last version of what I could think about (super simplified):
pid = fork do
exec "/my/fancy/daemon/style/script"
end
File.open('tmp/process.pid', 'w') { |file| file.write(pid.to_s) }
Can you please tell me what am I doing wrong? The ultimate goal is to keep the other script (i.e, the child process) running after the ruby code exits.
You can "detach" your child process:
Process.detach(pid)
See Process#detach for more info.
If you're running your script on a shell, and if your script is the last interactive process, your virtual terminal may exit and cause your child process to hangup as well. If you consider not sending output to the terminal, you can use Process.daemon before running exec.
See Process#daemon.
In Ruby, I'm running a system("command here") that is constantly watching changes for files, similar to tail. I'd like my program to continue to run and not halt at the system() call. Is there a way in Ruby to create another process so both can run independently, output results to the terminal, and then when you exit the program all processes the application created are removed?
Just combine spawn and waitall:
spawn 'sleep 6'
spawn 'sleep 8'
Process.waitall
You don't want to use system as that waits for the process to complete. You could use spawn instead and then wait for the processes (to avoid zombies). Then, when you want to exit, send a SIGTERM to your spawned processes. You could also use fork to launch your child processes but spawn is probably easier if you're using external programs.
You could also use process groups instead of tracking all the process IDs, then a single Process.kill('TERM', -process_group_id) call would take care of things. Your child processes should end up in the same process group but there is Process.setpgid if you need it.
Here's an example that uses fork (easier to get it all wrapped in one package that way).
def launch(id, sleep_for)
pid = Process.fork do
while(true)
puts "#{id}, pgid = #{Process.getpgid(Process.pid())}, pid = #{Process.pid()}"
sleep(sleep_for)
end
end
# No zombie processes please.
Process.wait(pid, Process::WNOHANG)
pid
end
# These just forward the signals to the whole process group and
# then immediately exit.
pgid = Process.getpgid(Process.pid())
Signal.trap('TERM') { Process.kill('TERM', -pgid); exit }
Signal.trap('INT' ) { Process.kill('INT', -pgid); exit }
launch('a', 5)
launch('b', 3)
while(true)
puts "p, pgid = #{Process.getpgid(Process.pid())}, pid = #{Process.pid()}"
sleep 2
end
If you run that in one terminal and then kill it from another (using the shell's kill command)you'll see that the children are also killed. If you remove the "forward this signal to the whole process group" Signal.trap stuff, then a simple SIGTERM will leave the children still running.
All of this assumes that you're working on some sort of Unixy system (such as Linux or OSX), YMMV anywhere else.
One more vote for using Spawn. We use it in Production a lot and it's very stable.
Given a very simple ruby script:
child = fork do
system 'sleep 10000'
end
5.times do
sleep 1
puts "send kill to #{child}"
Process.kill("QUIT", child)
end
QUIT signal is just lost. Where does it go? Something with default handler which just ignores it?
How to send signal to all processes created by that fork? Is it possible to do that without searching for all child processes?
The problem is that the system call creates yet another child process running the given command in a subshell, so there are actually three processes running in your example. Additionally, the Ruby Kernel#system command is implemented via the standard C function system(3), which calls fork and exec to create the new process and (on most systems) ignores SIGINT and SIGQUIT, and blocks SIGCHLD.
If you simply call sleep(10000) instead of system("sleep 10000") then things should work as you expect. You can also trap SIGQUIT in the child to handle it gracefully:
child = fork do
Signal.trap("QUIT") { puts "CHILD: ok, quitting time!"; exit }
sleep(10000)
end
If you really need to use a "system" call from the child process then you might be better off using an explicit fork/exec pair (instead of the implicit ones in the system call), so that you can perform your own signal handling in the third forked child.
I think that you are sending signal to fork process corectly. I think that the problem is with the system command. System command creates new fork and waits until it ends and I think that this waiting is blocking your quit signal. If you run your example as test.rb you'll see three processes:
test.rb
test.rb
sleep 10000
If you send signal "TERM" or "KILL" instead of "QUIT" the second test.rb will die but sleep 10000 will continue!
i'm trying to set up a server for integration tests (specs actually) via ruby and can't figure out how to control the process.
so, what i'm trying to do is:
run a rake task for my gem that executes the integration specs
the task needs to first start a server (i use webrick) and then run the specs
after executing the specs it should kill the webrick so i'm not left with some unused background process
webrick is not a requirement, but it's included in the ruby standard library so being able to use it would be great.
hope anyone is able to help!
ps. i'm running on linux, so having this work for windows is not my main priority (right now).
The standard way is to use the system functions fork (to duplicate the current process), exec (to replace the current process by an executable file), and kill (to send a signal to a process to terminate it).
For example :
pid = fork do
# this code is run in the child process
# you can do anything here, like changing current directory or reopening STDOUT
exec "/path/to/executable"
end
# this code is run in the parent process
# do your stuffs
# kill it (other signals than TERM may be used, depending on the program you want
# to kill. The signal KILL will always work but the process won't be allowed
# to cleanup anything)
Process.kill "TERM", pid
# you have to wait for its termination, otherwise it will become a zombie process
# (or you can use Process.detach)
Process.wait pid
This should work on any Unix like system. Windows creates process in a different way.
I just had to do something similar and this is what I came up with. #Michael Witrant's answer got me started, but I changed some things like using Process.spawn instead of fork (newer and better).
# start spawns a process and returns the pid of the process
def start(exe)
puts "Starting #{exe}"
pid = spawn(exe)
# need to detach to avoid daemon processes: http://www.ruby-doc.org/core-2.1.3/Process.html#method-c-detach
Process.detach(pid)
return pid
end
# This will kill off all the programs we started
def killall(pids)
pids.each do |pid|
puts "Killing #{pid}"
# kill it (other signals than TERM may be used, depending on the program you want
# to kill. The signal KILL will always work but the process won't be allowed
# to cleanup anything)
begin
Process.kill "TERM", pid
# you have to wait for its termination, otherwise it will become a zombie process
# (or you can use Process.detach)
Process.wait pid
rescue => ex
puts "ERROR: Couldn't kill #{pid}. #{ex.class}=#{ex.message}"
end
end
end
# Now we can start processes and keep the pids for killing them later
pids = []
pids << start('./someprogram')
# Do whatever you want here, run your tests, etc.
# When you're done, be sure to kill of the processes you spawned
killall(pids)
That's about all she wrote, give it a try and let me know how it works.
I have tried fork, but it has kind of problems when ActiveRecord is involved in both the processes. I would suggest Spawn plugin (http://github.com/tra/spawn). It does fork only but takes care of ActiveRecord.
I wanted to know why i am seeing a different behaviour in the background process in Bash shell
Case 1: Logged in to Unix server using Putty(SSH)
By default it uses csh shell
I changed to bash shell
typed sleep 2000 &
press enter
It gave me the job number. Now i killed my session by clicking the x in the putty window
Now open another session and tried to lookup the process..the process died.
Case 2:Case 1: Logged in to Unix server using Putty(SSH)
By default it uses csh shell
I changed to bash shell
vi mysleep.sh
sleep 2000 & Saved mysleep.sh
./mysleep.sh
Diff here is..instead of executing the sleep command directly i am storing the sleep command in a file and executing the file.
Now i killed my session by clicking the x in the putty window
Now open another session and tried to lookup the process..the process is still there
Not sure why this is happening. I thought i need to do disown in bash to run the process even after logging out.
One diff i see in the parent process id..In the second case..the parent process id for the sleep 2000 becomes 1. Looks like as soon as process for mysleep.sh died the kernel assigned the parent process to 1.
The difference here is indeed the intervening process.
When you close the terminal window, a HUP signal (related to "nohup" as an0nymo0usc0ward mentioned) is sent to the processes running in it. The default action on receiving HUP is to die - from the signal(3) manpage,
No Name Default Action Description
1 SIGHUP terminate process terminal line hangup
In your first example, the sleep process directly receives this HUP signal and dies because it isn't set to do anything else. (Some processes catch HUP and use it to perform some action, e.g. reread some configuration files)
In the second example, the shell process running your shell script has already died, so the sleep process never gets the signal. In UNIX, every process must have a parent process due to the internals of how the wait(2) family of calls works and indeed processes in general. So when the parent process dies, the kernel gives it to init (pid 1, as you note) as a foster child.
Orphan process (on wikipedia) has some more information available about it, also see Zombie process for some additional technical details.
Already running process?
^z
bg
disown %<jobid>
New process/script (on local machine's console)?
nohup script.sh &
New process/script (on remote machine's console)?
Depending on your need,
there are two options [ there will be more ;-) ]
ssh remotehost 'nohup /path/to/script.sh </dev/null > nohup.out 2>&1 &'
OR
use 'screen'
Try "nohup cmd args..."
Steven's answer is correct, but I'd like to highlight the tricky part here again:
=> Using a bash script that just executes sleep in the background
The effect of this is that the "script" exits almost immediately (since it's done all its commands). However, it did create a child process (sleep) during its lifetime. The effect of this is that:
The "script" cannot be the parent anymore, and sleep is orphaned to init (which shows nicely in a pstree)
The bash shell where you started the script from has no underlying jobs anymore
Note that this stuff all happens when you executed the script, and has nothing to do with any ssh logout/putty closing.
When you then finally close your putty session, bash receives a "SIGHUP", but doesn't forward it to any other process (since there are no jobs left)
In the other case, bash did still have a job left, which it then sent the SIGHUP to, causing it to end (as you noticed)
Hope this helps