The way I normally start a long-running shell script is
% (nohup ./script.sh </dev/null >script.log 2>&1 & )
The redirections close stdin, and reopen stdout and stderr; the nohup stops HUP reaching the process when the owning process exits (I realise that the 2>&1 is somewhat redundant, since the nohup does something like this anyway); and the backgrounding within the subshell is the double-fork which means that the ./script.sh process's parent has exited while it's still running, so it acquires the init process as its parent.
That doesn't completely work, however, because when I exit the shell from which I've invoked this (typically, of course, I'm doing this on a remote machine), it doesn't exit cleanly. I can do ^C to exit, and this is OK – the process does carry on in the background as intended. However I can't work out what is/isn't happening to require the ^C, and that's annoying me.
The actions above seem to tick most of the boxes in the unix FAQ (question 1.7), except that I'm not doing anything to detach this process from a controlling terminal, or to make it a session leader. The setsid(2) call exists on FreeBSD, but not the setsid command; nor, as far as I can see, is there an obvious substitute for that command. The same is true on macOS, of course.
So, the questions are:
Is there a differently-named caller of setsid on this platform, that I'm missing?
What, precisely, is happening when I exit the calling shell, that I'm killing with the ^C? Is there any way this could bite me?
Related questions (eg 1, 2) either answer a slightly different question, or assume the presence of the setsid command.
(This question has annoyed me for years, but because what I do here doesn't actually not work, I've never before got around to investigating, getting stumped, and asking about it).
In FreeBSD, out of the box you could use daemon -- run detached from the controlling terminal. option -r could be useful:
-r Supervise and restart the program after a one-second delay if it
has been terminated.
You could also try a supervisor, for example immortal is available for both platforms:
pkg install immortal # FreeBSD
brew install immortal # macOS
To daemonize your script and log (stdout/stderr) you could use:
immortal /path/to/your/script.sh -l /tmp/script.log
Or for more options, you could create a my-service.yml for example:
cmd: /path/to/script
cwd: /your/path
env:
DEBUG: 1
ENVIROMENT: production
log:
file: /tmp/app.log
stderr:
file: /tmp/app-error.log
And then run it with immortal -c my-service.yml
More examples can be found here: https://immortal.run/post/examples
If just want to use nohup and save the stdout & stderr into a file, you could add this to your script:
#!/bin/sh
exec 2>&1
...
Check more about exec 2>&1 in this answers https://stackoverflow.com/a/13088401/1135424
And then simply call nohup /your/script.sh & and check the file nohup.out, from the man
FILES
nohup.out The output file of the nohup execution if stan-
dard output is a terminal and if the current
directory is writable.
$HOME/nohup.out The output file of the nohup execution if stan-
dard output is a terminal and if the current
directory is not writable.
Related
According to this docker tutorial
What's the difference between
./my_first_process -D
./my_main_process &
They both seem unblocking to bash script and run in background
& tells the shell to put the command that precedes it into the background. -D is simply a flag that is passed to my_first_process and is interpreted by it; it has absolutely nothing whatsoever to do with the shell.
You will have to look into the documentation of my_first_process to see what -D does … it could mean anything. E.g. in npm, -D means "development", whereas in some other tools, it may mean "directory". In diff, it means "Output merged file to show `#ifdef NAME' diffs."
Some programs, by convention, take -D as an instruction to self-daemonize. Doing this looks something like the following:
Call fork(), and exit if it returns 0 (so only the child survives).
Close stdin, stdout and stderr if they are attached to the console (ideally, replacing their file descriptors with handles on /dev/null, so writes don't trigger an error).
Call setsid() to create a new session.
Call fork() again, and exit if it returns 0 again.
That's a lot more work than what just someprogram & does! A program that has self-daemonized can no longer log to the terminal, and will no longer be impacted if the terminal itself closes. That's not true of a program that's just started in the background.
To get something similar to the same behavior from bash, correct code would be something like:
someprogram </dev/null >/dev/null 2>&1 & disown -h
...wherein disown -h tells the shell not to pass along a SIGHUP to that process. It's also not uncommon to see the external tool nohup used for this purpose (though by default, it redirects stdout and stderr to a file called nohup.out if they're pointed at the TTY, the end purpose -- of making sure they're not pointed at the terminal, and thus that writes to them don't start failing if the terminal goes away -- is achieved):
nohup someprogram >/dev/null &
Ive got a script that takes a quite a long time to run, as it has to handle many thousands of files. I want to make this script as fool proof as possible. To this end, I want to check if the user ran the script using nohup and '&'. E.x.
me#myHost:/home/me/bin $ nohup doAlotOfStuff.sh &. I want to make 100% sure the script was run with nohup and '&', because its a very painful recovery process if the script dies in the middle for whatever reason.
How can I check those two key paramaters inside the script itself? and if they are missing, how can I stop the script before it gets any farther, and complain to the user that they ran the script wrong? Better yet, is there way I can force the script to run in nohup &?
Edit: the server enviornment is AIX 7.1
The ps utility can get the process state. The process state code will contain the character + when running in foreground. Absence of + means code is running in background.
However, it will be hard to tell whether the background script was invoked using nohup. It's also almost impossible to rely on the presence of nohup.out as output can be redirected by user elsewhere at will.
There are 2 ways to accomplish what you want to do. Either bail out and warn the user or automatically restart the script in background.
#!/bin/bash
local mypid=$$
if [[ $(ps -o stat= -p $mypid) =~ "+" ]]; then
echo Running in foreground.
exec nohup $0 "$#" &
exit
fi
# the rest of the script
...
In this code, if the process has a state code +, it will print a warning then restart the process in background. If the process was started in the background, it will just proceed to the rest of the code.
If you prefer to bailout and just warn the user, you can remove the exec line. Note that the exit is not needed after exec. I left it there just in case you choose to remove the exec line.
One good way to find if a script is logging to nohup, is to first check that the nohup.out exists, and then to echo to it and ensure that you can read it there. For example:
echo "complextag"
if ( $(cat nohup.out | grep "complextag" ) != "complextag" );then
# various commands complaining to the user, then exiting
fi
This works because if the script's stdout is going to nohup.out, where they should be going (or whatever out file you specified), then when you echo that phrase, it should be appended to the file nohup.out. If it doesn't appear there, then the script was nut run using nohup and you can scold them, perhaps by using a wall command on a temporary broadcast file. (if you want me to elaborate on that I can).
As for being run in the background, if it's not running you should know by checking nohup.
I have a VM that I want running indefinitely. The server is always running but I want the script to keep running after I log out. How would I go about doing so? Creating a cron job?
In general the following steps are sufficient to convince most Unix shells that the process you're launching should not depend on the continued existence of the shell:
run the command under nohup
run the command in the background
redirect all file descriptors that normally point to the terminal to other locations
So, if you want to run command-name, you should do it like so:
nohup command-name >/dev/null 2>/dev/null </dev/null &
This tells the process that will execute command-name to send all stdout and stderr to nowhere (instead of to your terminal) and also to read stdin from nowhere (instead of from your terminal). Of course if you actually have locations to write to/read from, you can certainly use those instead -- anything except the terminal is fine:
nohup command-name >outputFile 2>errorFile <inputFile &
See also the answer in Petur's comment, which discusses this issue a fair bit.
I have a bash script server.sh which is maintained by an external source and ideally should not be modified. This script writes to stdout and stderr.
In fact, this server.sh itself is doing an exec tclsh immediately:
#!/bin/sh
# \
exec tclsh "$0" ${1+"$#"}
so in fact, it is just a wrapper around a Tcl script. I just mention this in case you think that this matters.
I need a Tcl script setup.tcl which is supposed to do some preparatory work, then invoke server.sh (in the background), then do some cleanup work (and display the PID of the background process), and terminate.
server.sh is supposed to continue running until explicitly killed.
setup.tcl is usually invoked manually, either from a Cygwin bash shell or from a Windows cmd shell. In the latter case, it is ensured that Cygwin's bash.exe is in the PATH.
The environment is Windows 7 and Cygwin. The Tcl is either Cygwin's (8.5) or ActiveState 8.4.
The first version (omitting error handling) went like this:
# setup.tcl:
# .... preparatory work goes here
set childpid [exec bash.exe server.sh &]
# .... clean up work goes here
puts $childpid
exit 0
While this works when started as ActiveState Tcl from a Windows CMD shell, it does not work in a pure Cygwin setup. The reason is that as soon as setup.tcl ends, a signal is sent to the child process and this is killed too.
Using nohup would not help here, because I want to see the output of server.sh as soon as it occurs.
My next idea would be to created an intermediate bash script, mediator.sh, which uses disown -h to detach the child process and keep it from being killed:
#!/usr/bin/bash
# mediator.sh
server.sh &
child=$!
disown -h $child
and invoke mediator.sh from setup.tcl. But aside from the fact that I don't see an easy way to pass the child PID up to setup.tcl, the main problem is that it doesn't work either: While mediator.sh indeed keeps the child alive when called from the Cygwin command line directly, we have the same behaviour again (server.sh being killed when setup.tcl exits), when I call it via setup.tcl.
Anybody knowing a solution for this?
You'll want to set a trap handler in your server script so you can handle/ignore certain signals.
For example, to ignore HUP signals, you can do something like the following:
#!/bin/bash
handle_signal() {
echo "Ignoring HUP signal"
}
trap handle_signal SIGHUP
# Rest of code goes here
In the example case, if the script receives a HUP signal it will print a message and continue as normal. It will still die to Ctrl-C as that's the INT signal which is unhandled.
I logged in to a remote server via ssh and started a php script. Appereantly, it will take 17 hours to complete, is there a way to break the connection but the keep the script executing? I didn't make any output redirection, so I am seeing all the output.
Can you stop the process right now? If so, launch screen, start the process and detach screen using ctrl-a then ctrl-d. Use screen -r to retrieve the session later.
This should be available in most distros, failing that, a package will definitely be available for you.
ctrl + z
will pause it. Than type
bg
to send it to background. Write down the PID of the process for later usage ;)
EDIT: I forgot, you have to execute
disown -$PID
where $PID is the pid of your process
after that, and the process will not be killed after you close the terminal.
you described it's important to protect script continuation. Unfortunately I don't know, you make any interaction with script and script is made by you.
continuation protects 'screen' command. your connection will break, but screen protect pseudo terminal, you can reconnect to this later, see man.
if you don't need operators interaction with script, you simply can put script to background at the start, and log complete output into log file. Simply use command:
nohup /where/is/your.script.php >output.log 2&>1 &
>output.log will redirect output into log file, 2&>1 will append error stream into output, effectively into log file. last & will put command into background. Notice, nohup command will detach process from terminal group.
At now you can safely exit from ssh shell. Because your script is out of terminal group, then it won't be killed. It will be rejoined from your shell process, into system INIT process. It is unix like system behavior. Complete output you can monitor using command
tail -f output.log #allways breakable by ^C, it is only watching
Using this method you do not need use ^Z , bg etc shell tricks for putting command to the background.
Notice, using redirection to nohup command is preferred. Otherwise nohup will auto redirect all outputs for you to nohup.out file in the current directory.
You can use screen.