This question already has answers here:
Make: how to continue after a command fails?
(8 answers)
Closed 6 years ago.
I'm looking for a way to continue a make command in the event of a error failure.
I need a way of wrapping a command so it doesn't respond with a exit code 1.
test:
exit 1 ;\
echo 'hi' ;\
I need a way to wrap something like this:
example:
somecommand && othercommand ;\
echo 'hi' ;\
Where somecommand can exit with a 1 (error) and not run othercommand or a 0 which would run othercommand.
This should do:
test:
commandThatMayFail && otherCommand || true
echo hi
You can try it like this:
test:
rm fileDoesNotExist && echo foo || true
echo bar
You can also use make -i ... to ignore all errors. Per the man page:
-i, --ignore-errors
Ignore all errors in commands executed to remake files.
Prefixing a recipe with - tells make to ignore any errors returned by that line, the only other thing you need to do is run the two recipes separately.
example:
-somecommand && othercommand
echo 'hi'
Related
I have a multi-line shell command in a Makefile:
format:
. $(venv_activate_path) ;\
echo "++++++++++ ISORT ++++++++++" ;\
isort -rc . ;\
echo $? ;\
echo "+++ isort status " ;\
echo "++++++++++ BLACK ++++++++++" ;\
black $(package_name)/ --check ;\
echo $? ;\
echo "+++ black package status " ;\
black tests/ --check ;\
echo $? ;\
echo "+++ black test status "
I want this whole thing to return non-zero exist status if either of these scripts has exit code 1. Nonetheless, I want to run all commands, regardless of whether the first fails as I call this on github actions.
The echo $? are all empty despite the processes (black and isort) returning exit code 1 when run individually and the echo $? of the entire make format command is 0.
How do I get the make format command to return exit code 1 if at least one of these scripts has non-zero exit code?
You can run both in the background and then wait.
This is pretty ugly as a make target; perhaps refactor it to a separate script.
format:
. $(venv_activate_path);\
isort -rc . & isort=$$!; \
black $(package_name)/ --check & black=$$!; \
wait $$isort; isortresult=$$?; \
wait $$black; blackresult=$$?; \
exit $$((isortresult+blackresult))
If these commands are rewriting the files, you will need some additional checks to force them to wait for each other. Ultimately, perhaps a better solution will be to run each separately.
(Stack Overflow renders tabs as spaces, so you will not be able to copy/paste this recipe directly.)
Supposing that your condition "if either of these scripts has exit code 1" can be interpreted as "if any of these commands exit with nonzero status", this is is a fairly clean and clear solution:
format:
. $(venv_activate_path) ;\
failed=0;\
isort -rc . || failed=1;\
black $(package_name)/ --check || failed=1;\
black tests/ --check || failed=1;\
test "$$failed" -eq 0
It runs the commands sequentially, using variable failed to track whether any command fails. At the end, it uses the test command to exit with status 0 if $failed still has the value 0, or with a failure status otherwise. You can restore some or all of the echo commands if you like, other than after the test command, which must be last. However, you still will not get useful information from the $? variable to feed to them.
I have a target such as this:
.PHONY: do-symlink
do-symlink:
ln -s $(HERE_ONE_FILE) $(THERE_ONE_DIR)/
ln -s $(HERE_TWO_FILE) $(THERE_ONE_DIR)/
# check if those succeed or failed. I want to make sure both have passed
if succeeded:
do_something
else:
do_something_else
How can i check that they are both succeed? and if so, do something based on it?
So first of all, if a make recipe line fails, make fails, and stops executing the recipe. So if you put a recipe line at the end #echo "succeeded", then this will on run if all previous lines worked. As far as printing if something specific if one fails, you can use bash || for that
all:
#command1 || { echo "command1 failed"; exit 1 }
#command2 || { echo "command2 failed"; exit 1 }
#echo "command1 and command2 passed"
Notice the exit 1's in there. Normally, false || echo "false" would have a return status of 0 (pass), because the exit status is taken from the last command run (echo "false"), which will always succeed. This would cause make to always continue running the next recipe line. You may want this, however, you can preserve the failure by compounding the statement and doing an exit 1 at the end.
For running both commands regardless of exit status, and then handling the exits after, prefix the recipe lines with -. This will cause make to not stop running if the command fails. Notice however that each recipe line is run in its own shell, so one recipe line cannot directly access the return code from another line. You could output to a file and then access that file in a later recipe line:
all:
-#command1; echo $? > .commands.result
-#command2; echo $? >> .commands.result
#if grep -q "^00$" .commands.result; then \
echo both commands passed; \
else \
echo "at least one command failed"
(or you could always concatenate all the recipes lines into a single bash command and pass the variables around as well).
For example, I can do this with a subshell:
VAL=$( do_something )
but how do I achieve the same thing with curly braces so the command is NOT executing in a subshell? I.e. this does not work:
VAL={ do_something; }
TIA.
I'm not sure I understand the reasoning for what you're trying to accomplish, but if you can elaborate a bit more I might be able to help you.
I do recommend reading this fantastic write up about what's actually going on though, and why I don't think you want to invoke a process without a subshell.
However, to try and answer what you've asked:
You can't really run a command inside ${}, except in the fallback clause for when a value is not set (in POSIX sh or bash; might be feasible in zsh, which allows all manner of oddball syntax).
However, you can call cd like this if you really wanted this:
cdr() {
if (( $# )); then
command cd "$#"
else
local home
home=$(git rev-parse --show-toplevel 2>/dev/null) || home=$HOME
command cd "$home"
fi
}
Note
Using a function lets us test our argument list, use branching logic, have local variables, &c.
command cd is used to call through to the real cd implementation rather than recursing.
set -e is kinda stiff. Try something like
trap 'err=$?;
echo >&2 "ERROR $err in $0 at line $LINENO, Aborting";
exit $err;' ERR
This is a lot more informative when reading through your logs, and you can put a similar command inside the subshell. Yes, it means adding it inside the subshell... but I often do this sort of thing in function definitions that get called in subshells. Works well.
In use:
$ trap 'echo BOOM' ERR # parent shell trap for demo
$ false # trigger manually for demo
BOOM
$ x="$( trap 'err=$?;
> echo >&2 "ERROR $err in $0 at line $LINENO, Aborting";
> exit $err;' ERR
> date
> pwd
> false
> echo "I shan't"
> )"
ERROR 1 in bash at line 7, Aborting
BOOM
$ echo "$x"
Thu, Jan 10, 2019 8:35:57 AM
/c/Users/P2759474/repos/Old/deploy_microservices
$
If the outer shell had the same or a similar trap, it would have aborted too, with another message. (It's usually helpful to make the messages different.)
If you just don't like that, then as a clumsy workaround you can drop the data to a tempfile. Here's a script that will do it.
set -ex
{ pwd
date
false
echo "will this happen?"
} > foo
x=$(<foo)
echo "$x"
Put that in a script, it successfully bails.
$: ./sete
+ pwd
+ date
+ false
$: echo $?
1
I'd still use the trap, but the logic works.
I'd also use mktemp, and a trap to delete the temp on exit, etc.... but you get the idea.
i have a script that I accidentally ran without an underlying file present, and my script doesn't have a check for this file, because the script should stop when the command that requires that file exits 1.
i got caught out because it went ahead and skipped the sleep command and the ||exit 0 if test that I have as some protection protection. i would really like to know why. the if test and exit works if the preceding command doesn't fail.
if i strip the script down I can see some unexpected behaviour where the script doesn't stop at the && and skips the next sleep command.
is this not the correct way to use &&?
you can test this here:
#!/bin/bash
mkdir /root/simulatecomplexcommandthatreturns1 &&
sleep 5m
echo "let's go ahead and delete all the stuff"
find /blah/ -delete
this is on debian 9
EDIT:
for clarity, I want the script to stop when it encounters an error and I have &&. I just thought it was odd that it didn't run the sleep command.
The && only apply to next command, for a sequence, braces must be added:
#!/bin/bash
mkdir /root/simulatecomplexcommandthatreturns1 && {
sleep 5m
echo "let's go ahead and delete all the stuff"
find /blah/ -delete
}
or to avoid indent level the condition can be inverted
#!/bin/bash
mkdir /root/simulatecomplexcommandthatreturns1 || {
echo "something goes wrong"
exit 1
}
# ok, continue
sleep 5m
echo "let's go ahead and delete all the stuff"
find /blah/ -delete
If you want a script to abort/exit as soon as a command pipeline exists with a non-zero status (that means the last command in the pipeline, unless pipefail enabled), you might consider using:
set -e
In your example:
#!/bin/bash
set -e
mkdir /root/simulatecomplexcommandthatreturns1
sleep 5m
echo "let's go ahead and delete all the stuff"
find /blah/ -delete
when any of the commands fails, your script will exit.
Note however, this can sometimes lead to unwanted exits. For example it's normal for grep to exit with error if no match was found (you might "silence" such commands with grep .. || true ensuring the pipeline exits with success).
You'll probably be safer with manually testing for failure. For example:
if ! mkdir /root/simulatecomplexcommandthatreturns1; then
echo "Error description."
exit 1
fi
The usage of shortcircuiting && and || is best reserved for simple command sequences, when the execution of the next depends on successful exit of the previous. For example, the command pipeline:
mkdir /somedir && cp file /somedir && touch /somedir/file
will try to create a directory, if created successfully, it will try to copy the file; and if the file was copied successfully, it will touch the file.
Example with OR:
cp file /somedir || exit 1
where we try to copy the file and we exit if copy failed.
But you should be very careful when combining the two, since the result can be unexpected. For example:
a && b || c
is not equal to:
if a; then b; else c; fi
because c in the former expression will get executed whenever either of a or b fails (exits with a non-zero status). In the latter expression, c is executed only if a fails. For example:
true && false || echo "This also gets executed."
&& is like the AND operator with the property :
fail && <anything> equals fail
<anything> && fail equals fail
success && success equals success
So, if the first operand (or command) fails, there is no point in resolving the second command.
Hence,
when mkdir /root/simulatecomplexcommandthatreturns1 fails in
mkdir /root/simulatecomplexcommandthatreturns1 &&
sleep 5m
It skips the second command.
What you want here is || or the OR operator
fail || fail equals fail
fail || success equals success
success || <anything> equals success
So, using if the mkdir /root/simulatecomplexcommandthatreturns1 fails in"
mkdir /root/simulatecomplexcommandthatreturns1 || sleep 5m
It will have to evaluate the second operand ie the sleep 5m command.
EDIT :
Note that bash script do not exit if one of its command fails. It only exits when it reaches the end of the script or when exit is called.
if you want to exit when a certain command fails, you would do something like :
$ theCommandThatCanFail || exit 1 # the first command returns fail and
# since its `OR` operator, the second
# command will be resolved
You're using the wrong operator. What you need is || (example 2 below).
Explanation:
Note:
`A && B` # => Run `A`, and then `B` if A ran successfully.
`A || B` # => Run `A`, and then `B` if A did not run successfully.
I would like to check the result of a grep search in a Makefile. Contrary to this solution, I do not wish to use the shell command.
Also, I don't want the Makefile to raise an error when grep do not find the string (exit code of 1 is treated as an error).
The following tries to ignore the error and check the exit code :
all:
-grep term log*
echo $$?
#case "$$?" in \
0)\
echo "found";; \
*) \
echo "not found";;\
esac;
Unfortunately, the exit code is always 0.
The separate lines of a series of actions in a makefile are normally executed in separate sub-shells. To code what you're after, then:
all:
if grep term log*; \
then echo found; \
else echo not found; \
fi
That's a single command; it tests the exit status of grep directly. Note the liberal use of semi-colons; that's necessary because it all gets flattened when passed to the shell. Note too that the - is not needed; the statement as a whole exits with status 0 because one of the echo commands is executed, succeeds, and that is the status returned from the sub-shell. But there's another part to the trick; IIRC, the script is invoked with /bin/sh -e so the script exits on the first error (non-zero) status from a shell command — except in explicit conditionals such as an if.
If you want to explicitly capture the status of grep (if only to be sure it's being done right), then:
all:
-grep term log*; \
status=$$?; echo $$status; \
if [ $$status = 0 ]; \
then echo found; \
else echo not found; \
fi
You probably need the - this time because the grep is not executed as part of a shell conditional and a non-zero exit status could trigger the -e processing. I don't recommend futzing with this.
You might note that you can do cd commands in an action and because each action is executed separately, you have to do it repeatedly.
install: ${PROG}
cd ${INSTBIN}; ${RM_F} ${PROG}
${CP} ${PROG} ${INSTBIN}
cd ${INSTBIN}; ${CHOWN} ${OWNER}:${GROUP} ${PROG}; ${CHMOD} ${PERMS} ${PROG}
Yes, you can do it differently — I'm demonstrating a point, not advocating a style of installing programs.