Solved
I need to spawn background processes in MakeFile and also consider their exit codes.
Scenario:
several processes are spawned in background.
MakeFile continue evaluation (and do not want to check spawned processes PIDs in some loop an so forth)
Some process exits with non zero exit code
make utility exits with non zero exit code
Naturally, I am consider to use command & to spawn a process in background.
Problem: If command is specified like command & then make process does not track it's exit code.
Sample 1
do:
#false & \
echo "all is normal"
%make -f exit_status_test.mk
all is normal
Sample 2
do:
#false && \
echo "all is normal"
%make -f exit_status_test.mk
*** Error code 1
Stop in /usr/home/scher/tmp/lock_testing.
Sample 1 shows that make utility does not consider exit code of the background process.
P.S. Please do not advice to store spawned processes PIDs and to check them in a loop with some sleep delay and so forth. A need to continue evaluation of MakeFile and exit with non zero code automatically.
Solution
do:
#(echo "background command" ; (echo "[HANDLER] Prev command exits with $$?")) & \
echo "doing something"
So we can create a sequence of commands to handle exit status of background process.
This seems like an ill-conceived attempt to create a Makefile that can run multiple jobs in parallel, when in fact make can generally do this for you.
All you need to do is give each job a separate command in make:
target: job1 job2
job1:
some_command
job2:
some_other_command
If you use something like this in your Makefile and then run make -j2 target, then both some_command and some_other_command will be run in parallel.
See if you can find a way to get make to run your work in parallel like this.
Related
Minimized test case for the problem:
I have following Makefile:
test:
bash test.sh || true
echo OK
and the test.sh contains
#!/bin/bash
while read -p "Enter some text or press Ctrl+C to exit > " input
do
echo "Your input was: $input"
done
When I run make test and press Ctrl+C to exit the bash read the make will emit
Makefile:2: recipe for target 'test' failed
make: *** [test] Interrupt
How can I tell make to ignore the exit status of the script? I already have || true after the script which usually is enough to get make to keep going but for some reason, the SIGINT interrupting the read will cause make to behave different for this case.
I'm looking for a generic answer that works for processes other than while read loop in bash, too.
This has nothing to do with the exit status of the script. When you press ^C you're sending an interrupt signal to the make program, not just to your script. That causes the make program to stop, just like ^C always does.
There's no way to have make ignore ^C operations; whenever you press ^C at the terminal, make will stop.
ctrl+c sends a signal to the program to tell it to stop. What you want is ctrl+d which sends the signal EOT (end of transmission). You will need to send ctrl+d twice unless you are at the beginning of a line.
some text<c-d><c-d>
or
some text<return>
<c-d>
I found a way to make this work. It's a bit tricky so I'll explain the solution first. The important thing to understand that Ctrl+C is handled by your terminal and not by the currently running process in the terminal as I previously thought. When the terminal catches your Ctrl+C it will check the foreground process group and then send SIGINT to all processes in that group immediately. When you run something via Makefile and press Ctrl+C the SIGINT be immediately sent to Makefile and all processes that it started because those all belong in the foreground process group. And GNU Make handles SIGINT by waiting for any currently executed child process to stop and then exit with a message
Makefile:<line number>: recipe for target '<target>' failed
make: *** [<target>] Interrupt
if the child exited with non-zero exit status. If child handled the SIGINT by itself and exited with status 0, GNU Make will exit silently. Many programs exit via status code 130 on SIGINT but this is not required. In addition, kernel and wait() C API interface can differentiate with status code 130 and status code 130 + child received SIGINT so if Make wanted to behave different for these cases, it would be possible regardless of exit code. bash doesn't support testing for child process SIGINT status but only supports exit status codes.
The solution is to setup processes so that your foreground process group does not include GNU Make while you want to handle Ctrl+C specially. However, as POSIX doesn't define a tool to create any process groups, we have to use bash specific trick: use bash job control to trigger bash to create a new process group. Be warned that this causes some side-effects (e.g. stdin and stdout behaves slightly different) but at least for my case it was good enough.
Here's how to do it:
I have following Makefile (as usual, nested lines must have TAB instead of spaces):
test:
bash -c 'set -m; bash ./test.sh'
echo OK
and the test.sh contains
#!/bin/bash
int_handler()
{
printf "\nReceived SIGINT, quitting...\n" 1>&2
exit 0
}
trap int_handler INT
while read -p "Enter some text or press Ctrl+C to exit > " input
do
echo "Your input was: $input"
done
The set -m triggers creating a new foreground process group and the int_handler takes care of returning successful exit code on exit. Of course, if you want to have some other exit code but zero on Ctrl+C, feel free to any any value suitable. If you want to have something shorter, the child script only needs trap 'exit 0' INT instead of separate function and setup for it.
For additional information:
https://unix.stackexchange.com/a/99134/20336
https://unix.stackexchange.com/a/386856/20336
https://stackoverflow.com/a/18479195/334451
https://www.cons.org/cracauer/sigint.html
I wrote a small Go app that reads from stdin using the technique outlined here: https://flaviocopes.com/go-shell-pipes/
I invoke it like this: cmd | fsjl (where fsjl is my app)
Trouble is, the exit code is always 0 even when cmd exits 1.
How can I "forward" the exit code from cmd?
Here is my source code
That's not a job for your Go program. That's a job for whoever's managing the overall pipeline. Pipeline elements don't see each other's exit statuses.
For example, in bash, you can use pipefail:
If set, the return value of a pipeline is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands in the pipeline exit successfully. This option is disabled by default.
Here's an example, using a subshell to restrict the effects of pipefail to a single line:
(set -o pipefail && cmd | fsjl)
You can't always assume bash, but there are ways to do similar things in other shells.
This question already has answers here:
How to wait in bash for several subprocesses to finish, and return exit code !=0 when any subprocess ends with code !=0?
(35 answers)
Closed 6 years ago.
Background
I'm working on a bash script to automate the process of building half a dozen projects that live in the same directory. Each project has two scripts to run in order to build it:
npm install
npm run build
The first line will fetch all of the dependencies from npm. Since this step takes the longest, and since the projects can fetch their dependencies simultaneously, I'm using a background job to fetch everything in parallel. (ie: npm install &)
The second line will use those dependencies to build the project. Since this must happen after all the Step 1s finish, I'm running the wait command in between. See code snippet below.
The Question
I would like to have my script exit as soon as an error occurs in any of the background jobs, or the npm run build step that happens afterward.
I'm using set -e, however this does not apply to the background jobs, and thus if one project fails to install it's dependencies, everything else keeps going.
Here is an simplified example of how my script looks right now.
build.sh
set -e
DIR=$PWD
for dir in ./projects/**/
do
echo -e "\033[4;32mInstalling $dir\033[0m"
cd $dir
npm install & # takes a while, so do this in parallel
cd $DIR
done
wait # continue once the background jobs are completed
for dir in ./projects/**/
do
cd $dir
echo -e "\033[4;32mBuilding $dir\033[0m"
npm run build # Some projects use other projects, so build them in series
cd $DIR
echo -e "\n"
done
Again, I don't want to continue doing anything in the script if an error occurs at any point, this applies to both the parent and background jobs. Is this possible?
Collect the PIDs for the background jobs; then, use wait to collect the exit status of each, exiting the first time any PID polled over in that loop is nonzero.
install_pids=( )
for dir in ./projects/**/; do
(cd "$dir" && exec npm install) & install_pids+=( $! )
done
for pid in "${install_pids[#]}"; do
wait "$pid" || exit
done
The above, while simple, has a caveat: If an item late in the list exits nonzero prior to items earlier in the list, this won't be observed until that point in the list is polled. To work around this caveat, you can repeatedly iterate through the entire list:
install_pids=( )
for dir in ./projects/**/; do
(cd "$dir" && exec npm install) & install_pids+=( $! )
done
while (( ${#install_pids[#]} )); do
for pid_idx in "${!install_pids[#]}"; do
pid=${install_pids[$pid_idx]}
if ! kill -0 "$pid" 2>/dev/null; then # kill -0 checks for process existance
# we know this pid has exited; retrieve its exit status
wait "$pid" || exit
unset "install_pids[$pid_idx]"
fi
done
sleep 1 # in bash, consider a shorter non-integer interval, ie. 0.2
done
However, because this polls, it incurs extra overhead. This can be avoided by trapping SIGCHLD and referring to jobs -n (to get a list of jobs whose status changed since prior poll) when the trap is triggered.
Bash isn't made for parallel processing such as this. To accomplish what you want, I had to write a function library. I'd suggest seeking a language more readily suited to this if possible.
The problem with looping through the pids, such as this...
#!/bin/bash
pids=()
f() {
sleep $1
echo "no good"
false
}
t() {
sleep $1
echo "good"
true
}
t 3 &
pids+=$!
f 1 &
pids+=$!
t 2 &
pids+=$!
for p in ${pids[#]}; do
wait $p || echo failed
done
The problem is that "wait" will wait on the first pid, and if the other pids finish before the first one does, you'll not catch the exit code. The code above shows this problem on bash v4.2.46. The false command should produce output that never gets caught.
I am looking to launch a debugger program in the background from a Makefile. I want the make call to complete and return and leave the debugger running. It works fine from a shell but when I put it in the Makefile make waits around for all triggered processes to complete.
I have tried the following test target both with and without the -.
sleep:
-sleep 5 &
Regardless, make sits around and waits for sleep to complete.
I do understand that this is not a normal usage of make but it would allow me to present the debugger launcher using the same sort of setup as the build targets.
Have you tried adding an echo "before" before the sleep and an echo "after" after it?
What I think you'll notice is that Make does not wait for sleep 5 & itself to complete, but it waits for all processes started by a target to complete before considering that target to be complete. You will need to spawn a new process -- try nohup <programname> instead of programname &.
The answer to the question I should have asked would be:
sleep:
cygstart sleep 5
Eclipse now considers the build finished immediately.
I'm wondering if there's a way to implement trap in GNU make, similar to that built into BASH?
If the user presses CTRL-C, or if make itself fails (non-zero exit), I'd like to call a particular target or macro.
At this point in time, GNU make doesn't have native support.
There is a reliable workaround however:
.PHONY: internal-target external-target
external-target:
bash -c "trap 'trap - SIGINT SIGTERM ERR; <DO CLEANUP HERE>; exit 1' SIGINT SIGTERM ERR; $(MAKE) internal-target"
internal-target:
echo "doing stuff here"
This catches interruptions, terminations AND any non-zero exit codes.
Note the $(MAKE) so cmdline overrides and make options get passed to submake.
On trap:
clear trap handler (with -)
do the cleanup
exit with non-zero exit status, so build automation tools report the failed build.
DELETE_ON_ERROR does NOT work for directories, so this is key for cleaning up after mktemp -d, for example
Replace <DO CLEANUP HERE> with valid CMD.
A simplified version of #kevinf’s answer which seems good enough for basic cases:
run:
bash -c "trap 'docker-compose down' EXIT; docker-compose up --build"
(This example is for a reason: docker-compose up does say
When
the command exits, all containers are stopped.
but it does not rm the stopped containers like docker run --rm would, so you can still see them with docker ps -a.)
No. GNU make’s signal handling already leaves a lot to be desired. From within its signal handler, it calls functions like printf that are not safe to be called from within a signal handler. I have seen this cause problems, for example .DELETE_ON_ERROR rules don’t always run if stderr is redirected to stdout.
For example, on a CentOS 7.4 box:
Create the following Makefile:
.DELETE_ON_ERROR:
foo:
touch $#
sleep 10
Open it in vim and run :make,
While it is sleeping, hit Ctrl-C
Vim/make prints
Press ENTER or type command to continue
touch foo
sleep 10
^C
shell returned 130
Interrupt: Press ENTER or type command to continue
Make was sent an interrupt signal, but foo still exists.
Make does not support it, but using BASH tricks you can accomplish something similar.
default: complete
complete: do_mount
echo "Do something here..."
do_mount:
mkdir -p "$(MOUNTPOINT)"
( while ps -p $$PPID >/dev/null ; do \
sleep 1 ; \
done ; \
unmount "$(MOUNTPOINT)" \
) &
mount "$(MOUNTSOURCE)" "$(MOUNTPOINT)" -o bind
The "unmount" will run after the "make" completes. This is usually a satisfactory solution if you are attempting to cleanup operations that may occur during the build but are not cleaned up normally on "make" exit.
No. As far as I know there is no such functionality.
make produces return codes. As far as I can remember right now, it returns 0 for success, 2 for failure (please check the documentation). Therefore, would it be enough for you to wrap make inside a shell script for example?