This question already has answers here:
SIGINT to cancel read in bash script?
(2 answers)
Closed 2 years ago.
I'm playing around with bash read functionality. I like what I have so far as a simple layer on top of my current shell. The read -e does tab-complete and previous commands, and sending EOF with ctrl+d gets me back to my original shell. Here's my reference:
Bash (or other shell): wrap all commands with function/script
I'd like some help handling SIGINT, ctrl+c. In a normal shell, if you start typing and hit ^C halfway, it immediately ends the line. For this simple example, after ^C, I still have to hit return before it's registered.
How do I keep the nice things that readline does, but still handle SIGINT correctly? Ideally, it would send a continue statement to the while read loop, or somehow send a \n to STDIN where my read is waiting.
Example code:
#!/bin/bash
# Emulate bash shell
gtg=1
function handleCtrl-C {
# What do I do here?
gtg=0
return
}
trap handleCtrl-C INT
while read -e -p "> " line
do
if [[ $gtg == 1 ]] ; then
eval "$line"
fi
gtg=1
done
I think I came up with something finally I liked. See SIGINT to cancel read in bash script? for that answer.
Reading man 7 signal tells that some system calls have a restartable flag set as a result will return back to the command
For some system calls, if a signal is caught while the call is
executing and the call is prematurely terminated, the call is
auto-matically restarted. Any handler installed with signal(3) will
have the SA_RESTART flag set, meaning that any restartable system call
will not return on receipt of a signal. The affected system calls
include read(2), write(2), sendto(2), recvfrom(2),sendmsg(2), and
recvmsg(2) on a communications channel or a low speed device and
during a ioctl(2) or wait(2). However, calls that have already
committed are not restarted, but instead return a partial success (for
example, a short read count). These semantics could be changed with
siginterrupt(3).
You can try printing the value input to line and verify that the read is resumed after CtrlC return until new line is hit. Type in something like "exit", followed by Ctrl-C and then "exit" the output comes out as "exitexit". Make the following change and run for the above test case
echo ">$line<"
if [ $gtg == 1 ] ; then
You'll the output as
You can verify this with a C program as well.
Related
Question
I am using Bash 5 and have a long-running loop which needs to check occasionally for various keys the user may have hit. I know how to do this using stty — see my answer below — but it's more ugly than ought.
Essentially, I'm looking for a clean way to do this:
keyhit-p() {
if "read -n1 would block"; then
return false
else
return true
fi
}
Non-solution: read -t 0
I have read the bash manual and know about read -t 0. That does not do what I want, which is to detect if any input is available. Instead, it only returns true if the user hits ENTER (a complete line of input).
For example:
while true; do
if read -n1 -t0; then
echo "This only works if you hit enter"
break
fi
done
A working answer, albeit ugly
While the following works, I am hoping someone has a better answer.
#!/bin/bash
# Reset terminal's stty to previous values on exit.
trap 'stty $(stty --save)' EXIT
keyhit-p() {
# Return true if input is available on stdin (any key has been hit).
local sttysave=$(stty --save)
stty -icanon min 1 time 0 # ⎫
read -t0 # ⎬ Ugly: This ought to be atomic so the
local status=$? # ⎪ terminal's stty is always restored.
stty $sttysave # ⎭
return $status
}
while true; do
echo -n .
if ! keyhit-p; then
continue
else
while keyhit-p; do
read -n1
echo Key: $REPLY
done
break
fi
done
This alters the user's terminal settings (stty) before the read and attempts to write them back afterward, but does so non-atomically. It's possible for the script to get interrupted and leave the user's terminal in an incorrect state. I'd like to see an answer which solves that problem, ideally using only the tools built in to bash.
A faster, even uglier answer
Another flaw in the above routine is that it takes a lot of CPU time trying to get everything right. It requires calling an external program (stty) three times just to check that nothing has happened. Forks can be expensive in loops. If we dispense with correctness, we can get a routine that runs two orders of magnitude (256×) faster.
#!/bin/bash
# Reset terminal's stty to previous values on exit.
trap 'stty $(stty --save)' EXIT
# Set one character at a time input for the whole script.
stty -icanon min 1 time 0
while true; do
echo -n .
# We save time by presuming `read -t0` no longer waits for lines.
# This may cause problems and can be wrong, for example, with ^Z.
if ! read -t0; then
continue
else
while read -t0; do
read -n1
echo Key: $REPLY
done
break
fi
done
Instead of changing to non-canonical mode only during the read test, this script sets it once at the beginning and uses an exception handler when the script exits to undo it.
While I like that the code looks cleaner, the atomicity flaw of the original version is exacerbated because the SUSPEND signal isn't handled. If the user's shell is bash, icanon is enabled when the process is suspended, but NOT disabled when the process is foregrounded. That makes read -t0 return FALSE even when keys (other than Enter) are hit. Other user shells may not enable icanon on ^Z as bash does, but that's even worse as entering commands will no longer work as usual.
Additionally, requiring non-canonical mode to be left on all the time may cause other problems as the script gets longer than this trivial example. It is not documented how non-canonical mode is supposed to affect read and other bash built-ins. It seems to work in my tests, but will it always? Chances of running into problems would multiply when calling — or being called by — external programs. Maybe there would be no issues, but it would require tedious testing.
I'm looking at https://stackoverflow.com/a/10225050/1737158
And in same Q there is an answer with timeout command but it's not in all OSes, so I want to avoid it.
What I try to do is:
demo="$(top)" &
TASK_PID=$!
sleep 3
echo "TASK_PID: $TASK_PID"
echo "demo: $demo"
And I expect to have nothing in $demo variable while top command never ends.
Now I get an empty result. Which is "acceptable" but when i re-use the same thing with the command which should return value, I still get an empty result, which is not ok. E.g.:
demo="$(uptime)" &
TASK_PID=$!
sleep 3
echo "TASK_PID: $TASK_PID"
echo "demo: $demo"
This should return uptime result but it doesn't. I also tried to kill the process by TASK_PID but I always get. If a command fails, I expect to have stderr captures somehow. It can be in different variable but it has to be captured and not leaked out.
What happens when you execute var=$(cmd) &
Let's start by noting that the simple command in bash has the form:
[variable assignments] [command] [redirections]
for example
$ demo=$(echo 313) declare -p demo
declare -x demo="313"
According to the manual:
[..] the text after the = in each variable assignment undergoes tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal before being assigned to the variable.
Also, after the [command] above is expanded, the first word is taken to be the name of the command, but:
If no command name results, the variable assignments affect the current shell environment. Otherwise, the variables are added to the environment of the executed command and do not affect the current shell environment.
So, as expected, when demo=$(cmd) is run, the result of $(..) command substitution is assigned to the demo variable in the current shell.
Another point to note is related to the background operator &. It operates on the so called lists, which are sequences of one or more pipelines. Also:
If a command is terminated by the control operator &, the shell executes the command asynchronously in a subshell. This is known as executing the command in the background.
Finally, when you say:
$ demo=$(top) &
# ^^^^^^^^^^^ simple command, consisting ONLY of variable assignment
that simple command is executed in a subshell (call it s1), inside which $(top) is executed in another subshell (call it s2), the result of this command substitution is assigned to variable demo inside the shell s1. Since no commands are given, after variable assignment, s1 terminates, but the parent shell never receives the variables set in child (s1).
Communicating with a background process
If you're looking for a reliable way to communicate with the process run asynchronously, you might consider coprocesses in bash, or named pipes (FIFO) in other POSIX environments.
Coprocess setup is simpler, since coproc will setup pipes for you, but note you might not reliably read them if process is terminated before writing any output.
#!/bin/bash
coproc top -b -n3
cat <&${COPROC[0]}
FIFO setup would look something like this:
#!/bin/bash
# fifo setup/clean-up
tmp=$(mktemp -td)
mkfifo "$tmp/out"
trap 'rm -rf "$tmp"' EXIT
# bg job, terminates after 3s
top -b >"$tmp/out" -n3 &
# read the output
cat "$tmp/out"
but note, if a FIFO is opened in blocking mode, the writer won't be able to write to it until someone opens it for reading (and starts reading).
Killing after timeout
How you'll kill the background process depends on what setup you've used, but for a simple coproc case above:
#!/bin/bash
coproc top -b
sleep 3
kill -INT "$COPROC_PID"
cat <&${COPROC[0]}
Editor's note: The OP is ultimately looking to package the code from this answer
as a script. Said code creates a stay-open FIFO from which a background command reads data to process as it arrives.
It works if I type it in the terminal, but it won't work if I enter those commands in a script file and run it.
#!/bin/bash
cat >a&
pid=$!
it seems that the program is stuck at cat>a&
$pid has no value after running the script, but the cat process seems to exist.
cdarke's answer contains the crucial pointer: your script mustn't run in a child process, so you have to source it.
Based on the question you linked to, it sounds like you're trying to do the following:
Open a FIFO (named pipe).
Keep that FIFO open indefinitely.
Make a background command read from that FIFO whenever new data is sent to it.
See bottom for a working solution.
As for an explanation of your symptoms:
Running your script NOT sourced (NOT with .) means that the script runs in a child process, which has the following implications:
Variables defined in the script are only visible inside that script, and the variables cease to exist altogether when the script finishes running.
That's why you didn't see the script's $myPid variable after running the script.
When the script finishes running, its background tasks (cat >a&) are killed (as cdarke explains, the SIGHUP signal is sent to them; any process that doesn't explicitly trap that signal is terminated).
This contradicts your claim that the cat process continues to exist, but my guess is that you mistook an interactively started cat process for one started by a script.
By contrast, any FIFO created by your script (with mkfifo) does persist after the script exits (a FIFO behaves like a file - it persists until you explicitly delete it).
However, when you write to that FIFO without another process reading from it, the writing command will block and thus appear to hang (the writing process blocks until another process reads the data from the FIFO).
That's probably what happened in your case: because the script's background processes were killed, no one was reading from the FIFO, causing an attempt to write to it to block. You incorrectly surmised that it was the cat >a& command that was getting "stuck".
The following script, when sourced, adds functions to the current shell for setting up and cleaning up a stay-open FIFO with a background command that processes data as it arrives. Save it as file bgfifo_funcs:
#!/usr/bin/env bash
[[ $0 != "$BASH_SOURCE" ]] || { echo "ERROR: This script must be SOURCED." >&2; exit 2; }
# Set up a background FIFO with a command listening for input.
# E.g.:
# bgfifo_setup bgfifo "sed 's/^/# /'"
# echo 'hi' > bgfifo # -> '# hi'
# bgfifo_cleanup
bgfifo_setup() {
(( $# == 2 )) || { echo "ERROR: usage: bgfifo_setup <fifo-file> <command>" >&2; return 2; }
local fifoFile=$1 cmd=$2
# Create the FIFO file.
mkfifo "$fifoFile" || return
# Use a dummy background command that keeps the FIFO *open*.
# Without this, it would be closed after the first time you write to it.
# NOTE: This call inevitably outputs a job control message that looks
# something like this:
# [1]+ Stopped cat > ...
{ cat > "$fifoFile" & } 2>/dev/null
# Note: The keep-the-FIFO-open `cat` PID is the only one we need to save for
# later cleanup.
# The background processing command launched below will terminate
# automatically then FIFO is closed when the `cat` process is killed.
__bgfifo_pid=$!
# Now launch the actual background command that should read from the FIFO
# whenever data is sent.
{ eval "$cmd" < "$fifoFile" & } 2>/dev/null || return
# Save the *full* path of the FIFO file in a global variable for reliable
# cleanup later.
__bgfifo_file=$fifoFile
[[ $__bgfifo_file == /* ]] || __bgfifo_file="$PWD/$__bgfifo_file"
echo "FIFO '$fifoFile' set up, awaiting input for: $cmd"
echo "(Ignore the '[1]+ Stopped ...' message below.)"
}
# Cleanup function that you must call when done, to remove
# the FIFO file and kill the background commands.
bgfifo_cleanup() {
[[ -n $__bgfifo_file ]] || { echo "(Nothing to clean up.)"; return 0; }
echo "Removing FIFO '$__bgfifo_file' and terminating associated background processes..."
rm "$__bgfifo_file"
kill $__bgfifo_pid # Note: We let the job control messages display.
unset __bgfifo_file __bgfifo_pid
return 0
}
Then, source script bgfifo_funcs, using the . shell builtin:
. bgfifo_funcs
Sourcing executes the script in the current shell (rather than in a child process that terminates after the script has run), and thus makes the script's functions and variables available to the current shell. Functions by definition run in the current shell, so any background commands started from functions stay alive.
Now you can set up a stay-open FIFO with a background process that processes input as it arrives as follows:
# Set up FIFO 'bgfifo in the current dir. and process lines sent to it
# with a sample Sed command that simply prepends '# ' to every line.
$ bgfifo_setup bgfifo "sed 's/^/# /'"
# Send sample data to the FIFO.
$ echo 'Hi.' > bgfifo
# Hi.
# ...
$ echo 'Hi again.' > bgfifo
# Hi again.
# ...
# Clean up when done.
$ bgfifo_cleanup
The reason that cat >a "hangs" is because it is reading from the standard input stream (stdin, file descriptor zero), which defaults to the keyboard.
Adding the & causes it to run in background, which disconnects from the keyboard. Normally that would leave a suspended job in background, but, since you exit your script, its background tasks are killed (sends a SIGHUP signal).
EDIT: although I followed the link in the question, it was not stated originally that the OP was actually using a FIFO at that stage. So thanks to #mklement0.
I don't understand what you are trying to do here, but I suspect you need to run it as a "sourced" file, as follows:
. gash.sh
Where gash.sh is the name of your script. Note the preceding .
You need to specify a file with "cat":
#!/bin/bash
cat SOMEFILE >a &
pid=$!
echo PID $pid
Although that seems a bit silly - why not just "cp" the file (cp SOMEFILE a)?
Q: What exactly are you trying to accomplish?
I have a script that is supposed to trap SIGTERM and SIGTSTP. This is what I have in the main block:
trap 'killHandling' TERM
And in the function:
killHandling () {
echo received kill signal, ignoring
return
}
... and similar for SIGINT. The problem is one of user interface. The script prompts the user for some input, and if the SIGTERM or SIGINT occurs when the script is waiting for input, it's confusing. Here is the output in that case:
Enter something: # SIGTERM received
received kill signal, ignoring
# shell waits at blank line for user input, user gets confused
# user hits "return", which then gets read as blank input from the user
# bad things happen because of the blank input
I have definitely seen scripts which handle this more elegantly, like so:
Enter something: # SIGTERM received
received kill signal, ignoring
Enter something: # re-prompts user for user input, user is not confused
What is the mechanism used to accomplish the latter? Unfortunately I can't simply change my trap code to do the re-prompt as the script prompts the user for several things and what the prompt says is context-dependent. And there has to be a better way than writing context-dependent trap functions.
I'd be very grateful for any pointers. Thanks!
These aren't terribly robust methods--there are some issues with the way it handles CTRL-C as a character after the first trap, for example--but they both handle the use case you defined.
Use BASH_COMMAND to re-run the last command (e.g read).
prompt () {
read -p 'Prompting: '
}
reprompt () {
echo >&2
eval "$BASH_COMMAND"
}
trap "reprompt" INT
prompt
In this case, *BASH_COMMAND* evaluates to read -p 'Prompting: '. The command then needs to be reprocessed with eval. If you don't eval it, you can end up with weird quoting problems. YMMV.
Use FUNCNAME to re-run previous function in call stack.
prompt () {
read -p 'Prompting: '
}
reprompt () {
echo >&2
"${FUNCNAME[1]}"
}
trap "reprompt" INT
prompt
In this example, FUNCNAME[1] expands to prompt, which is the previous function in the stack. We just call it again recursively, as many times as needed.
The answer CodeGnome gave works, but as he points out, it is not robust; a second control-c causes undesirable behavior. I ultimately got around the problem by making better use of existing input validation in the code. So my interrupt handling code now looks like this:
killHandling () {
echo received kill signal, ignoring
echo "<<Enter>> to continue"
return
}
Now the cursor still waits at a blank line for user input, but the user is not confused, and hits the "Enter" key as prompted. Then the script's input validation detects that a blank line has been entered, which is treated as invalid input, and the user is re-prompted to enter something.
I remain grateful to CodeGnome for his suggestions, from which I learned a couple of things. And I apologize for the delay in posting this answer.
A simple and seemingly reliable way to do locking under bash is:
exec 9>>lockfile
flock 9
However, bash notoriously propagates such a fd lock to all forked stuff including executed programs etc.
Is there any way to tell bash not to duplicate the fd? It's great that the lock is attached to a fd which gets removed when the program terminates, no matter how it gets terminated.
I know I can do stuff like:
run_some_prog 9>&-
But that is quite tedious.
Is there any better solution?
You can use the -o command line option to flock(1) (long option --close, which might be better for writing in scripts for the self-documenting nature) to specify that the file descriptor should be closed before executing commands via flock(1):
-o, --close
Close the file descriptor on which the lock is held
before executing command. This is useful if command
spawns a child process which should not be holding
the lock.
Apparently flock -o FD does not fix the problem. A trick to get rid of the superfluous FD for later commands in the same shell script is to wrap the remaining part into a section which closes the FD, like this:
var=outside
exec 9>>lockfile
flock -n 9 || exit
{
: commands which do not see FD9
var=exported
# exit would exit script
# see CLUMSY below outside this code snippet
} 9<&-
# $var is "exported"
# drop lock closing the FD
exec 9<&-
: remaining commands without lock
This is a bit CLUMSY, because the close of the FD is so far separated from the lock.
You can refactor this loosing the "natural" command flow but keeping things together which belong together:
functions_running_with_lock()
{
: commands which do not see FD9
var=exported
# exit would exit script
}
var=outside
exec 9>>lockfile
flock -n 9 || exit
functions_running_with_lock 9<&-
# $var is "exported"
# drop lock closing the FD
exec 9<&-
: remaining commands without lock
A little nicer writing which keeps the natural command flow at the expense of another fork plus an additional process and a bit different workflow, which often comes handy. But this does not allow to set variables in the outer shell:
var=outside
exec 9>>lockfile
flock -n 9 || exit
(
exec 9<&-
: commands which do not see FD9
var=exported
# exit does not interrupt the whole script
exit
var=neverreached
)
# optionally test the ret if the parentheses using $?
# $var is "outside" again
# drop lock closing the FD
exec 9<&-
: remaining commands without lock
BTW, if you really want to be sure that bash does not introduce additional file descriptors (to "hide" the closed FD and skip a real fork), for example if you execute some deamon which then would hold the lock forever, the latter variant is recommended, just to be sure. lsof -nP and strace your_script are your friends.
There is no way to mark a FD as close-on-exec within bash, so no, there is no better solution.
-o doesn't work with file descriptors, it only works with files. You have to use -u to unlock the file descriptor.
What I do is this:
# start of the lock sections
LOCKFILE=/tmp/somelockfile
exec 8>"$LOCKFILE"
if ! flock -n 8;then
echo Rejected # for testing, remove this later
exit # exit, since we don't have lock
fi
# some code which shouldn't run in parallel
# End of lock section
flock -u 8
rm -f "$LOCKFILE"
This way the file descriptor will be closed by the process that made the lock, and since every other process will exit, that means only the process holding the lock will unlock the file descriptor and remove the lock file.