I'm using bash, and as I understand it, exec followed by a command is supposed to replace the shell without creating a new process. For example,
exec echo hello
has the appearance of printing "hello" and then immediately exiting, because after echo is done, the shell process isn't there to return to anymore.
If I put this as part of a pipeline - for instance,
exec echo hello | sed 's/hell/heck/'
or
echo hello | exec sed 's/hell/heck/'
my expectation is that, similarly, the shell would terminate as a result of its process being replaced away. This is not what happens in reality, though - both the commands above print "hecko" and return to the shell normally, just as if the word "exec" wasn't there. Why is this?
There is sentence in bash manual:
Each command in a pipeline is executed as a separate process (i.e., in
a subshell).
So in both examples two processes are spawned by the pipeline first and 'exec' is executed inside one of spawned process - without impact on shell executing the pipeline.
Related
When running commands from a bash script, does bash always wait for the previous command to complete, or does it just start the command then go on to the next one?
ie: If you run the following two commands from a bash script is it possible for things to fail?
cp /tmp/a /tmp/b
cp /tmp/b /tmp/c
Yes, if you do nothing else then commands in a bash script are serialized. You can tell bash to run a bunch of commands in parallel, and then wait for them all to finish, but doing something like this:
command1 &
command2 &
command3 &
wait
The ampersands at the end of each of the first three lines tells bash to run the command in the background. The fourth command, wait, tells bash to wait until all the child processes have exited.
Note that if you do things this way, you'll be unable to get the exit status of the child commands (and set -e won't work), so you won't be able to tell whether they succeeded or failed in the usual way.
The bash manual has more information (search for wait, about two-thirds of the way down).
add '&' at the end of a command to run it parallel.
However, it is strange because in your case the second command depends on the final result of the first one. Either use sequential commands or copy to b and c from a like this:
cp /tmp/a /tmp/b &
cp /tmp/a /tmp/c &
Unless you explicitly tell bash to start a process in the background, it will wait until the process exits. So if you write this:
foo args &
bash will continue without waiting for foo to exit. But if you don't explicitly put the process in the background, bash will wait for it to exit.
Technically, a process can effectively put itself in the background by forking a child and then exiting. But since that technique is used primarily by long-lived processes, this shouldn't affect you.
In general, unless explicitly sent to the background or forking themselves off as a daemon, commands in a shell script are serialized.
They wait until the previous one is finished.
However, you can write 2 scripts and run them in separate processes, so they can be executed simultaneously. It's a wild guess, really, but I think you'll get an access error if a process tries to write in a file that's being read by another process.
I think what you want is the concept of a subshell. Here's one reference I just googled: http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/subshells.html
I am reading The TTY demystified. In the "Jobs and sessions" section there is an example of a user using an xterm:
$ cat
hello
hello
^Z
[1]+ Stopped cat
$ ls | sort
And there is a table listing the processes involved: xterm, bash (child of the xterm), and the three last processes (cat, ls and sort) all have the same PPID (parent process ID) -- they are all children of the same bash process.
Now, I know that pipelines in bash are executed in subshells. I have always thought that this subshell thing meant that there was an extra bash process for each subshell. My question is: shouldn't there be another two bash processes, both children of the first bash, and then ls would be a child of the first bash, and sort would be a child of the second bash? Is the table in the article simplified, or is my understanding of subshells wrong?
Programs are executed in child processes, but these are not subshells. The shell forks a child, redirects standard input/output/error as necessary, and then immediately calls execv() to execute the program.
For a very brief period the child process is still running bash, but we don't consider this a subshell because it's not doing any shell command processing -- that was all done in the original shell, and the child is just starting up the external program (as if via an explicit exec for commands like ls).
In the case of a pipeline, if any of the commands are shell built-ins, they run in a subshell. So if you do:
ls | read var
it will create two child processes. One child will run ls, the other will be a subshell executing read var.
Invoking an executable, whether directly or via a pipe, does not spawn a subshell. Only explicitly invoking it within a subshell (via (...), $(...), and so on) does so.
Running a bash script in the background with job control enabled and stdin closed will exit the PARENT shell. How can that happen?
To demonstrate make this background_bash_script:
#!/bin/bash
set -m
ruby -e "puts :here"
Then run it in bash - it will exit the shell you ran it in. The ruby command does not matter although it appears it must be a command and not a bash built-in (for example awk --version works but true does not). To get a better look I've been running it in yet another instance of bash. A full session looks like this.
parent: PS1='child: ' bash
child: ./background_bash_script <&- &
[1] 3893
child: here
exit
parent:
Confusing!
What seems like is happening is that after set -m is run in the script, the next command that is run is forced to be in the foreground process group, which takes the original shell out of the foreground process group. Once that process exits, the shell running the script is now in the foreground process group, but once that shell exits, the original shell doesn't put itself back into the foreground process group because it ran the script in the background. So you now have an interactive shell that is in a background process group.
You can see some weird behavior here if you put a sleep at the end of your script so that it doesn't exit immediately. When you run the script in the background you get the terminal prompt back, but now your interactive shell isn't in the foreground process group! As soon as you try to type anything the shell exits. I'm not sure exactly what mechanism causes the exit. Since the shell is in the background, any attempts to read or write characters to the terminal should result in SIGTTIN OR SIGTTOU, but these signals don't cause the shell to exit in my tests.
Summary: I have a bash script that runs a process in background, and is supposed to work as a normal command and inside a command substitution block such as $(...). The script itself spawns a process that forks to background. It can be reduced to this test case:
#!/bin/sh
echo something
sleep 5 &
Running this script in a shell will return immediately (and print "something"), running it inside $(...) will hang for 5 seconds, waiting for the backgrounded "sleep" to finish.
Applies to anything that is started inside the command substitution shell and spawns processes in background, including any children in that process tree apparently. Seems to affect both bash and zsh, haven't tried others.
Original question: I have a bash script that is supposed to print a value to stdout and also copy it to the X clipboard every time it runs.
#!/bin/sh
echo something
echo something | xclip -selection clipboard
This script (let's call it "something") is meant to be used to get this word (which is actually the output of another command) and be used in different ways such as:
$ something
something
$ xclip -o -selection clipboard
something
$ echo $(something)
^C
Prints to normal stdout, copies the output to the clipboard to be used in normal X applications, and should also be able to use the stdout with bash command substitution, to insert this word in the middle of any command.
However the bash command substitution seems to force xclip to stay alive in foreground. xclip normally daemonizes itself since the X clipboard requires that a client provides the clipboard contents, and the default behavior is to make it quit once the clipboard contents are replaced.
After having this issue with xclip I made the minimal test case that I wrote at the beginning of this question, so it seems to apply that anything that daemonizes inside the $(...) shell
Can anyone explain this behavior? Is there any way I can avoid it?
If you want the backgrounded process to not interfere with command substitution, you have to disconnect its stdout. This will return immediately:
$ cat bg.sh
#!/bin/sh
echo before
sleep 5 >/dev/null &
echo after
$ date; x=$(./bg.sh); date; echo "$x"
Sat Jun 1 13:02:26 EDT 2013
Sat Jun 1 13:02:26 EDT 2013
before
after
You will lose the ability to capture the backgrounded process's stdout, but if you're running it in the background you probably don't care. the bg.sh process can always write to disk.
When running commands from a bash script, does bash always wait for the previous command to complete, or does it just start the command then go on to the next one?
ie: If you run the following two commands from a bash script is it possible for things to fail?
cp /tmp/a /tmp/b
cp /tmp/b /tmp/c
Yes, if you do nothing else then commands in a bash script are serialized. You can tell bash to run a bunch of commands in parallel, and then wait for them all to finish, but doing something like this:
command1 &
command2 &
command3 &
wait
The ampersands at the end of each of the first three lines tells bash to run the command in the background. The fourth command, wait, tells bash to wait until all the child processes have exited.
Note that if you do things this way, you'll be unable to get the exit status of the child commands (and set -e won't work), so you won't be able to tell whether they succeeded or failed in the usual way.
The bash manual has more information (search for wait, about two-thirds of the way down).
add '&' at the end of a command to run it parallel.
However, it is strange because in your case the second command depends on the final result of the first one. Either use sequential commands or copy to b and c from a like this:
cp /tmp/a /tmp/b &
cp /tmp/a /tmp/c &
Unless you explicitly tell bash to start a process in the background, it will wait until the process exits. So if you write this:
foo args &
bash will continue without waiting for foo to exit. But if you don't explicitly put the process in the background, bash will wait for it to exit.
Technically, a process can effectively put itself in the background by forking a child and then exiting. But since that technique is used primarily by long-lived processes, this shouldn't affect you.
In general, unless explicitly sent to the background or forking themselves off as a daemon, commands in a shell script are serialized.
They wait until the previous one is finished.
However, you can write 2 scripts and run them in separate processes, so they can be executed simultaneously. It's a wild guess, really, but I think you'll get an access error if a process tries to write in a file that's being read by another process.
I think what you want is the concept of a subshell. Here's one reference I just googled: http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/subshells.html