I wrote a small Go app that reads from stdin using the technique outlined here: https://flaviocopes.com/go-shell-pipes/
I invoke it like this: cmd | fsjl (where fsjl is my app)
Trouble is, the exit code is always 0 even when cmd exits 1.
How can I "forward" the exit code from cmd?
Here is my source code
That's not a job for your Go program. That's a job for whoever's managing the overall pipeline. Pipeline elements don't see each other's exit statuses.
For example, in bash, you can use pipefail:
If set, the return value of a pipeline is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands in the pipeline exit successfully. This option is disabled by default.
Here's an example, using a subshell to restrict the effects of pipefail to a single line:
(set -o pipefail && cmd | fsjl)
You can't always assume bash, but there are ways to do similar things in other shells.
Related
Minimized test case for the problem:
I have following Makefile:
test:
bash test.sh || true
echo OK
and the test.sh contains
#!/bin/bash
while read -p "Enter some text or press Ctrl+C to exit > " input
do
echo "Your input was: $input"
done
When I run make test and press Ctrl+C to exit the bash read the make will emit
Makefile:2: recipe for target 'test' failed
make: *** [test] Interrupt
How can I tell make to ignore the exit status of the script? I already have || true after the script which usually is enough to get make to keep going but for some reason, the SIGINT interrupting the read will cause make to behave different for this case.
I'm looking for a generic answer that works for processes other than while read loop in bash, too.
This has nothing to do with the exit status of the script. When you press ^C you're sending an interrupt signal to the make program, not just to your script. That causes the make program to stop, just like ^C always does.
There's no way to have make ignore ^C operations; whenever you press ^C at the terminal, make will stop.
ctrl+c sends a signal to the program to tell it to stop. What you want is ctrl+d which sends the signal EOT (end of transmission). You will need to send ctrl+d twice unless you are at the beginning of a line.
some text<c-d><c-d>
or
some text<return>
<c-d>
I found a way to make this work. It's a bit tricky so I'll explain the solution first. The important thing to understand that Ctrl+C is handled by your terminal and not by the currently running process in the terminal as I previously thought. When the terminal catches your Ctrl+C it will check the foreground process group and then send SIGINT to all processes in that group immediately. When you run something via Makefile and press Ctrl+C the SIGINT be immediately sent to Makefile and all processes that it started because those all belong in the foreground process group. And GNU Make handles SIGINT by waiting for any currently executed child process to stop and then exit with a message
Makefile:<line number>: recipe for target '<target>' failed
make: *** [<target>] Interrupt
if the child exited with non-zero exit status. If child handled the SIGINT by itself and exited with status 0, GNU Make will exit silently. Many programs exit via status code 130 on SIGINT but this is not required. In addition, kernel and wait() C API interface can differentiate with status code 130 and status code 130 + child received SIGINT so if Make wanted to behave different for these cases, it would be possible regardless of exit code. bash doesn't support testing for child process SIGINT status but only supports exit status codes.
The solution is to setup processes so that your foreground process group does not include GNU Make while you want to handle Ctrl+C specially. However, as POSIX doesn't define a tool to create any process groups, we have to use bash specific trick: use bash job control to trigger bash to create a new process group. Be warned that this causes some side-effects (e.g. stdin and stdout behaves slightly different) but at least for my case it was good enough.
Here's how to do it:
I have following Makefile (as usual, nested lines must have TAB instead of spaces):
test:
bash -c 'set -m; bash ./test.sh'
echo OK
and the test.sh contains
#!/bin/bash
int_handler()
{
printf "\nReceived SIGINT, quitting...\n" 1>&2
exit 0
}
trap int_handler INT
while read -p "Enter some text or press Ctrl+C to exit > " input
do
echo "Your input was: $input"
done
The set -m triggers creating a new foreground process group and the int_handler takes care of returning successful exit code on exit. Of course, if you want to have some other exit code but zero on Ctrl+C, feel free to any any value suitable. If you want to have something shorter, the child script only needs trap 'exit 0' INT instead of separate function and setup for it.
For additional information:
https://unix.stackexchange.com/a/99134/20336
https://unix.stackexchange.com/a/386856/20336
https://stackoverflow.com/a/18479195/334451
https://www.cons.org/cracauer/sigint.html
I want to run several commands in a bash loop, how I can achieve this properly? You can see what I tried below but after executing the first command, it exited the loop. What is the correct way of doing it? Thanks in advance for your help.
for sample in 2 3 27 28 32
do
command1
command2
command3
done
Since the pseudo-code in the question works, here are a number of possible issues based on the environment and the contents of command1:
The script is running with errexit set, and the exit code of command1 is non-zero. To test this, try echo "$-" - if there's an e in there, errexit is set.
The shell is FUBAR. Could anyone else have modified the shell?
There's a crazy alias somewhere, making command1 run something other than what you think it is running.
command1 runs break, exit or continue (or return if it's in a function).
If a POSIX-compliant shell fails to execute a command it immediately exits with status 127.
Is it possible to immediately determine that such a shell has succeeded in executing a command without waiting for the command (and thus the shell) to exit? (NOT the exit status of the command!) In particular I'm thinking of commands like xterm that don't exit until the user chooses to terminate them.
If this can't be done portably can it be done non-portably? Under Linux, or just with Bash or some other subset of shells?
Bash is quite verbose when running jobs in the background:
$ echo toto&
toto
[1] 15922
[1]+ Done echo toto
Since I'm trying to run jobs in parallel and use the output, I'd like to find a way to silence bash. Is there a way to remove this superfluous output?
You can use parentheses to run a background command in a subshell, and that will silence the job control messages. For example:
(sleep 10 & )
Note: The following applies to interactive Bash sessions. In scripts, job-control messages are never printed.
There are 2 basic scenarios for silencing Bash's job-control messages:
Launch-and-forget:
CodeGnome's helpful answer answer suggests enclosing the background command in a simple subshell - e.g, (sleep 10 &) - which effectively silences job-control messages - both on job creation and on job termination.
This has an important side effect:
By using control operator & inside the subshell, you lose control of the background job - jobs won't list it, and neither %% (the spec. (ID) of the most recently launched job) nor $! (the PID of the (last) process launched (as part of) the most recent job) will reflect it.[1]
For launch-and-forget scenarios, this is not a problem:
You just fire off the background job,
and you let it finish on its own (and you trust that it runs correctly).
[1] Conceivably, you could go looking for the process yourself, by searching running processes for ones matching its command line, but that is cumbersome and not easy to make robust.
Launch-and-control-later:
If you want to remain in control of the job, so that you can later:
kill it, if need be.
synchronously wait (at some later point) for its completion,
a different approach is needed:
Silencing the creation job-control messages is handled below, but in order to silence the termination job-control messages categorically, you must turn the job-control shell option OFF:
set +m (set -m turns it back on)
Caveat: This is a global setting that has a number of important side effects, notably:
Stdin for background commands is then /dev/null rather than the current shell's.
The keyboard shortcuts for suspending (Ctrl-Z) and delay-suspending (Ctrl-Y) a foreground command are disabled.
For the full story, see man bash and (case-insensitively) search for occurrences of "job control".
To silence the creation job-control messages, enclose the background command in a group command and redirect the latter's stderr output to /dev/null
{ sleep 5 & } 2>/dev/null
The following example shows how to quietly launch a background job while retaining control of the job in principle.
$ set +m; { sleep 5 & } 2>/dev/null # turn job-control option off and launch quietly
$ jobs # shows the job just launched; it will complete quietly due to set +m
If you do not want to turn off the job-control option (set +m), the only way to silence the termination job-control message is to either kill the job or wait for it:
Caveat: There are two edge cases where this technique still produces output:
If the background command tries to read from stdin right away.
If the background command terminates right away.
To launch the job quietly (as above, but without set +m):
$ { sleep 5 & } 2>/dev/null
To wait for it quietly:
$ wait %% 2>/dev/null # use of %% is optional here
To kill it quietly:
{ kill %% && wait; } 2>/dev/null
The additional wait is necessary to make the termination job-control message that is normally displayed asynchronously by Bash (at the time of actual process termination, shortly after the kill) a synchronous output from wait, which then allows silencing.
But, as stated, if the job completes by itself, a job-control message will still be displayed.
Wrap it in a dummy script:
quiet.sh:
#!/bin/bash
$# &
then call it, passing your command to it as an argument:
./quiet.sh echo toto
You may need to play with quotes depending on your input.
Interactively, no. It will always display job status. You can influence when the status is shown using set -b.
There's nothing preventing you from using the output of your commands (via pipes, or storing it variables, etc). The job status is sent to the controlling terminal by the shell and doesn't mix with other I/O. If you're doing something complex with jobs, the solution is to write a separate script.
The job messages are only really a problem if you have, say, functions in your bashrc which make use of job control which you want to have direct access to your interactive environment. Unfortunately there's nothing you can do about it.
One solution (in bash anyway) is to route all the output to /dev/null
echo 'hello world' > /dev/null &
The above will not give you any output other than the id for the bg process.
Solved
I need to spawn background processes in MakeFile and also consider their exit codes.
Scenario:
several processes are spawned in background.
MakeFile continue evaluation (and do not want to check spawned processes PIDs in some loop an so forth)
Some process exits with non zero exit code
make utility exits with non zero exit code
Naturally, I am consider to use command & to spawn a process in background.
Problem: If command is specified like command & then make process does not track it's exit code.
Sample 1
do:
#false & \
echo "all is normal"
%make -f exit_status_test.mk
all is normal
Sample 2
do:
#false && \
echo "all is normal"
%make -f exit_status_test.mk
*** Error code 1
Stop in /usr/home/scher/tmp/lock_testing.
Sample 1 shows that make utility does not consider exit code of the background process.
P.S. Please do not advice to store spawned processes PIDs and to check them in a loop with some sleep delay and so forth. A need to continue evaluation of MakeFile and exit with non zero code automatically.
Solution
do:
#(echo "background command" ; (echo "[HANDLER] Prev command exits with $$?")) & \
echo "doing something"
So we can create a sequence of commands to handle exit status of background process.
This seems like an ill-conceived attempt to create a Makefile that can run multiple jobs in parallel, when in fact make can generally do this for you.
All you need to do is give each job a separate command in make:
target: job1 job2
job1:
some_command
job2:
some_other_command
If you use something like this in your Makefile and then run make -j2 target, then both some_command and some_other_command will be run in parallel.
See if you can find a way to get make to run your work in parallel like this.