I'm wondering what's the difference between these two grammar in bash: ( &) and ( ) &.
The only difference that I noticed is, (tty &) will return "not a tty" while (tty) & will return the current tty name, but why?
To give an example, should I run (setsid startx &) or (setsid startx) &?
In the case of
(tty &)
a subshell is started which starts another tty process in the background without job control and terminal, hence there is a "not a tty" error. The tty process becomes detached with PPID 1
In the case of
(tty) &
a subshell is started and runs in the background. This background shell starts a tty process and after tty finishes and reports to the terminal, the subshell finishes in the background.
--
tty is a simple command. Whether or not a particular command (like startx) needs a ( ... &) construct to become detached / disowned from a parent process depends on the command itself. There are a number of ways for a process to in turn start a subprocess and detach that, so the command may not need it.
Related
Just switched from bash to zsh.
In bash, background tasks continue running when the shell exits. For example here, dolphin continues running after the exit:
$ dolphin .
^Z
[1]+ Stopped dolphin .
$ bg
[1]+ dolphin . &
$ exit
This is what I want as the default behavior.
In contrast, zsh's behavior is to warn about running jobs on exit, then close them if you exit again. For example here, dolphin is closed when the second exit-command actually exits the shell:
% dolphin .
^Z
zsh: suspended dolphin .
% bg
[1] + continued dolphin .
% exit
zsh: you have running jobs.
% exit
How do I make zsh's default behavior here like bash's?
Start the program with &!:
dolphin &!
The &! (or equivalently, &|) is a zsh-specific shortcut to both background and disown the process, such that exiting the shell will leave it running.
From the zsh documentation:
HUP
... In zsh, if you have a background job running when the shell exits, the shell will assume you want that to be killed; in this case it is sent a particular signal called SIGHUP... If you often start jobs that should go on even when the shell has exited, then you can set the option NO_HUP, and background jobs will be left alone.
So just set the NO_HUP option:
% setopt NO_HUP
I have found that using a combination of nohup, &, and disown works for me, as I don't want to permanently cause jobs to run when the shell has exited.
nohup <command> & disown
While just & has worked for me in bash, I found when using only nohup, &, or disown on running commands, like a script that calls a java run command, the process would still stop when the shell is exited.
nohup makes the command ignore NOHUP and SIGHUP signals from the shell
& makes the process run in the background in a subterminal
disown followed by an argument (the index of the job number in your jobs list) prevents the shell from sending a SIGHUP signal to child processes. Using disown without an argument causes it to default to the most recent job.
I found the nohup and disown information at this page, and the & information in this SO answer.
Update
When I originally wrote this, I was using it for data processing scripts/programs. For those kinds of use cases, something like ts (task-spooler), works nicely.
I typically use screen for keeping background jobs running.
1) Create a screen session:
screen -S myScreenName
2) Launch your scripts,services,daemons or whatever
3) Exit (detach) screen-session with
screen -d
or
shortcut ALT+A then d
After few hundreds of years - if you want to resume your session (reattach):
screen -r myScreenName
If you want to know if there's a screen-session, its name and its status (attached or detached):
screen -ls
This solution works on all terminal interpreters like bash, zsh etc.
See also man screen
The scenario is that I need my main command to run in current shell, this is required or losing all environment stuff, etc.
So, I can't just run my pipe this way:
#command-line 1
mainCommand | (
...subshell commands...
) &
#this wait works, but main command is in child process
wait $!
I have to run main command in current shell:
#command-line 2
mainCommand &> >(
...subshell commands...
) &
#this wait is waiting for mainCommand, not subshell
wait $!
However, in the command line 2, it's just a single command and I can't just send it to background, only the subshell should go to background then I can get its PID.
How to let
The main command be in current shell
And the 'wait' command does actually wait for the subshell?
I have the lock file solution but I prefer not using file as the whole script runs continuously and writing/modifying a file again and again is like penetrating the file system.
Newer versions of bash allow waiting on a process substitution, but until then, I would recommend simply using a named pipe.
mkfifo p
( ... subshell commands ... ) < p &
mainCommand > p
wait
Give a try to this. you need to add a kill in the subshell commands.
sleep 100 &
export BACKGROUNDPID=$!
mainCommand &> >(
...subshell commands...
kill "${BACKGROUNDPID}"
) &
wait ${BACKGROUNDPID}"
# execution continue here ...
I can't figure what happen on:
exec CMD &
especially what value for pid is:
echo $!
With exec CMD &, the & takes effect first, so everything is run in background; then the command is evaluated, and exec'd, but that would have happened without the exec, so in fact that is the same as writing CMD & (but causes people to worry about why it was written thus, so it is bad style).
The value in $! should be the PID of the command that is executed in the background.
Just switched from bash to zsh.
In bash, background tasks continue running when the shell exits. For example here, dolphin continues running after the exit:
$ dolphin .
^Z
[1]+ Stopped dolphin .
$ bg
[1]+ dolphin . &
$ exit
This is what I want as the default behavior.
In contrast, zsh's behavior is to warn about running jobs on exit, then close them if you exit again. For example here, dolphin is closed when the second exit-command actually exits the shell:
% dolphin .
^Z
zsh: suspended dolphin .
% bg
[1] + continued dolphin .
% exit
zsh: you have running jobs.
% exit
How do I make zsh's default behavior here like bash's?
Start the program with &!:
dolphin &!
The &! (or equivalently, &|) is a zsh-specific shortcut to both background and disown the process, such that exiting the shell will leave it running.
From the zsh documentation:
HUP
... In zsh, if you have a background job running when the shell exits, the shell will assume you want that to be killed; in this case it is sent a particular signal called SIGHUP... If you often start jobs that should go on even when the shell has exited, then you can set the option NO_HUP, and background jobs will be left alone.
So just set the NO_HUP option:
% setopt NO_HUP
I have found that using a combination of nohup, &, and disown works for me, as I don't want to permanently cause jobs to run when the shell has exited.
nohup <command> & disown
While just & has worked for me in bash, I found when using only nohup, &, or disown on running commands, like a script that calls a java run command, the process would still stop when the shell is exited.
nohup makes the command ignore NOHUP and SIGHUP signals from the shell
& makes the process run in the background in a subterminal
disown followed by an argument (the index of the job number in your jobs list) prevents the shell from sending a SIGHUP signal to child processes. Using disown without an argument causes it to default to the most recent job.
I found the nohup and disown information at this page, and the & information in this SO answer.
Update
When I originally wrote this, I was using it for data processing scripts/programs. For those kinds of use cases, something like ts (task-spooler), works nicely.
I typically use screen for keeping background jobs running.
1) Create a screen session:
screen -S myScreenName
2) Launch your scripts,services,daemons or whatever
3) Exit (detach) screen-session with
screen -d
or
shortcut ALT+A then d
After few hundreds of years - if you want to resume your session (reattach):
screen -r myScreenName
If you want to know if there's a screen-session, its name and its status (attached or detached):
screen -ls
This solution works on all terminal interpreters like bash, zsh etc.
See also man screen
I have a process that is already running for a long time and don't want to end it.
How do I put it under nohup (that is, how do I cause it to continue running even if I close the terminal?)
Using the Job Control of bash to send the process into the background:
Ctrl+Z to stop (pause) the program and get back to the shell.
bg to run it in the background.
disown -h [job-spec] where [job-spec] is the job number (like %1 for the first running job; find about your number with the jobs command) so that the job isn't killed when the terminal closes.
Suppose for some reason Ctrl+Z is also not working, go to another terminal, find the process id (using ps) and run:
kill -SIGSTOP PID
kill -SIGCONT PID
SIGSTOP will suspend the process and SIGCONT will resume the process, in background. So now, closing both your terminals won't stop your process.
The command to separate a running job from the shell ( = makes it nohup) is disown and a basic shell-command.
From bash-manpage (man bash):
disown [-ar] [-h] [jobspec ...]
Without options, each jobspec is removed from the table of active jobs. If the -h option is given, each jobspec is not
removed from the table, but is marked so that SIGHUP is not sent to the job if the shell receives a SIGHUP. If no jobspec is
present, and neither the -a nor the -r option is supplied, the current job is used. If no jobspec is supplied, the -a option
means to remove or mark all jobs; the -r option without a jobspec argument restricts operation to running jobs. The return
value is 0 unless a jobspec does not specify a valid job.
That means, that a simple
disown -a
will remove all jobs from the job-table and makes them nohup
These are good answers above, I just wanted to add a clarification:
You can't disown a pid or process, you disown a job, and that is an important distinction.
A job is something that is a notion of a process that is attached to a shell, therefore you have to throw the job into the background (not suspend it) and then disown it.
Issue:
% jobs
[1] running java
[2] suspended vi
% disown %1
See http://www.quantprinciple.com/invest/index.php/docs/tipsandtricks/unix/jobcontrol/
for a more detailed discussion of Unix Job Control.
Unfortunately disown is specific to bash and not available in all shells.
Certain flavours of Unix (e.g. AIX and Solaris) have an option on the nohup command itself which can be applied to a running process:
nohup -p pid
See http://en.wikipedia.org/wiki/Nohup
Node's answer is really great, but it left open the question how can get stdout and stderr redirected. I found a solution on Unix & Linux, but it is also not complete. I would like to merge these two solutions. Here it is:
For my test I made a small bash script called loop.sh, which prints the pid of itself with a minute sleep in an infinite loop.
$./loop.sh
Now get the PID of this process somehow. Usually ps -C loop.sh is good enough, but it is printed in my case.
Now we can switch to another terminal (or press ^Z and in the same terminal). Now gdb should be attached to this process.
$ gdb -p <PID>
This stops the script (if running). Its state can be checked by ps -f <PID>, where the STAT field is 'T+' (or in case of ^Z 'T'), which means (man ps(1))
T Stopped, either by a job control signal or because it is being traced
+ is in the foreground process group
(gdb) call close(1)
$1 = 0
Close(1) returns zero on success.
(gdb) call open("loop.out", 01102, 0600)
$6 = 1
Open(1) returns the new file descriptor if successful.
This open is equal with open(path, O_TRUNC|O_CREAT|O_RDWR, S_IRUSR|S_IWUSR).
Instead of O_RDWR O_WRONLY could be applied, but /usr/sbin/lsof says 'u' for all std* file handlers (FD column), which is O_RDWR.
I checked the values in /usr/include/bits/fcntl.h header file.
The output file could be opened with O_APPEND, as nohup would do, but this is not suggested by man open(2), because of possible NFS problems.
If we get -1 as a return value, then call perror("") prints the error message. If we need the errno, use p errno gdb comand.
Now we can check the newly redirected file. /usr/sbin/lsof -p <PID> prints:
loop.sh <PID> truey 1u REG 0,26 0 15008411 /home/truey/loop.out
If we want, we can redirect stderr to another file, if we want to using call close(2) and call open(...) again using a different file name.
Now the attached bash has to be released and we can quit gdb:
(gdb) detach
Detaching from program: /bin/bash, process <PID>
(gdb) q
If the script was stopped by gdb from an other terminal it continues to run. We can switch back to loop.sh's terminal. Now it does not write anything to the screen, but running and writing into the file. We have to put it into the background. So press ^Z.
^Z
[1]+ Stopped ./loop.sh
(Now we are in the same state as if ^Z was pressed at the beginning.)
Now we can check the state of the job:
$ ps -f 24522
UID PID PPID C STIME TTY STAT TIME CMD
<UID> <PID><PPID> 0 11:16 pts/36 S 0:00 /bin/bash ./loop.sh
$ jobs
[1]+ Stopped ./loop.sh
So process should be running in the background and detached from the terminal. The number in the jobs command's output in square brackets identifies the job inside bash. We can use in the following built in bash commands applying a '%' sign before the job number :
$ bg %1
[1]+ ./loop.sh &
$ disown -h %1
$ ps -f <PID>
UID PID PPID C STIME TTY STAT TIME CMD
<UID> <PID><PPID> 0 11:16 pts/36 S 0:00 /bin/bash ./loop.sh
And now we can quit from the calling bash. The process continues running in the background. If we quit its PPID become 1 (init(1) process) and the control terminal become unknown.
$ ps -f <PID>
UID PID PPID C STIME TTY STAT TIME CMD
<UID> <PID> 1 0 11:16 ? S 0:00 /bin/bash ./loop.sh
$ /usr/bin/lsof -p <PID>
...
loop.sh <PID> truey 0u CHR 136,36 38 /dev/pts/36 (deleted)
loop.sh <PID> truey 1u REG 0,26 1127 15008411 /home/truey/loop.out
loop.sh <PID> truey 2u CHR 136,36 38 /dev/pts/36 (deleted)
COMMENT
The gdb stuff can be automatized creating a file (e.g. loop.gdb) containing the commands and run gdb -q -x loop.gdb -p <PID>. My loop.gdb looks like this:
call close(1)
call open("loop.out", 01102, 0600)
# call close(2)
# call open("loop.err", 01102, 0600)
detach
quit
Or one can use the following one liner instead:
gdb -q -ex 'call close(1)' -ex 'call open("loop.out", 01102, 0600)' -ex detach -ex quit -p <PID>
I hope this is a fairly complete description of the solution.
Simple and easiest steps
Ctrl + Z ----------> Suspends the process
bg --------------> Resumes and runs background
disown %1 -------------> required only if you need to detach from the terminal
To send running process to nohup (http://en.wikipedia.org/wiki/Nohup)
nohup -p pid , it did not worked for me
Then I tried the following commands and it worked very fine
Run some SOMECOMMAND,
say /usr/bin/python /vol/scripts/python_scripts/retention_all_properties.py 1.
Ctrl+Z to stop (pause) the program and get back to the shell.
bg to run it in the background.
disown -h so that the process isn't killed when the terminal closes.
Type exit to get out of the shell because now you're good to go as the operation will run in the background in its own process, so it's not tied to a shell.
This process is the equivalent of running nohup SOMECOMMAND.
ctrl + z - this will pause the job (not going to cancel!)
bg - this will put the job in background and return in running process
disown -a - this will cut all the attachment with job (so you can close the terminal and it will still run)
These simple steps will allow you to close the terminal while keeping process running.
It wont put on nohup (based on my understanding of your question, you don't need it here).
On my AIX system, I tried
nohup -p processid>
This worked well. It continued to run my process even after closing terminal windows. We have ksh as default shell so the bg and disown commands didn't work.