I would like to check the result of a grep search in a Makefile. Contrary to this solution, I do not wish to use the shell command.
Also, I don't want the Makefile to raise an error when grep do not find the string (exit code of 1 is treated as an error).
The following tries to ignore the error and check the exit code :
all:
-grep term log*
echo $$?
#case "$$?" in \
0)\
echo "found";; \
*) \
echo "not found";;\
esac;
Unfortunately, the exit code is always 0.
The separate lines of a series of actions in a makefile are normally executed in separate sub-shells. To code what you're after, then:
all:
if grep term log*; \
then echo found; \
else echo not found; \
fi
That's a single command; it tests the exit status of grep directly. Note the liberal use of semi-colons; that's necessary because it all gets flattened when passed to the shell. Note too that the - is not needed; the statement as a whole exits with status 0 because one of the echo commands is executed, succeeds, and that is the status returned from the sub-shell. But there's another part to the trick; IIRC, the script is invoked with /bin/sh -e so the script exits on the first error (non-zero) status from a shell command — except in explicit conditionals such as an if.
If you want to explicitly capture the status of grep (if only to be sure it's being done right), then:
all:
-grep term log*; \
status=$$?; echo $$status; \
if [ $$status = 0 ]; \
then echo found; \
else echo not found; \
fi
You probably need the - this time because the grep is not executed as part of a shell conditional and a non-zero exit status could trigger the -e processing. I don't recommend futzing with this.
You might note that you can do cd commands in an action and because each action is executed separately, you have to do it repeatedly.
install: ${PROG}
cd ${INSTBIN}; ${RM_F} ${PROG}
${CP} ${PROG} ${INSTBIN}
cd ${INSTBIN}; ${CHOWN} ${OWNER}:${GROUP} ${PROG}; ${CHMOD} ${PERMS} ${PROG}
Yes, you can do it differently — I'm demonstrating a point, not advocating a style of installing programs.
Related
I have a multi-line shell command in a Makefile:
format:
. $(venv_activate_path) ;\
echo "++++++++++ ISORT ++++++++++" ;\
isort -rc . ;\
echo $? ;\
echo "+++ isort status " ;\
echo "++++++++++ BLACK ++++++++++" ;\
black $(package_name)/ --check ;\
echo $? ;\
echo "+++ black package status " ;\
black tests/ --check ;\
echo $? ;\
echo "+++ black test status "
I want this whole thing to return non-zero exist status if either of these scripts has exit code 1. Nonetheless, I want to run all commands, regardless of whether the first fails as I call this on github actions.
The echo $? are all empty despite the processes (black and isort) returning exit code 1 when run individually and the echo $? of the entire make format command is 0.
How do I get the make format command to return exit code 1 if at least one of these scripts has non-zero exit code?
You can run both in the background and then wait.
This is pretty ugly as a make target; perhaps refactor it to a separate script.
format:
. $(venv_activate_path);\
isort -rc . & isort=$$!; \
black $(package_name)/ --check & black=$$!; \
wait $$isort; isortresult=$$?; \
wait $$black; blackresult=$$?; \
exit $$((isortresult+blackresult))
If these commands are rewriting the files, you will need some additional checks to force them to wait for each other. Ultimately, perhaps a better solution will be to run each separately.
(Stack Overflow renders tabs as spaces, so you will not be able to copy/paste this recipe directly.)
Supposing that your condition "if either of these scripts has exit code 1" can be interpreted as "if any of these commands exit with nonzero status", this is is a fairly clean and clear solution:
format:
. $(venv_activate_path) ;\
failed=0;\
isort -rc . || failed=1;\
black $(package_name)/ --check || failed=1;\
black tests/ --check || failed=1;\
test "$$failed" -eq 0
It runs the commands sequentially, using variable failed to track whether any command fails. At the end, it uses the test command to exit with status 0 if $failed still has the value 0, or with a failure status otherwise. You can restore some or all of the echo commands if you like, other than after the test command, which must be last. However, you still will not get useful information from the $? variable to feed to them.
In our project we have a shell script which is to be sourced to set up environment variables for the subsequent build process or to run the built applications.
It contains a block which checks the already set variables and does some adjustment.
# part of setup.sh
for LIBRARY in "${LIBRARIES_WE_NEED[#]}"
do
echo $LD_LIBRARY_PATH | \grep $LIBRARY > /dev/null
if [ $? -ne 0 ]
then
echo Adding $LIBRARY
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LIBRARY
else
echo Not adding $LIBRARY
fi
done
i.e. it checks if a path to a library is already in $LD_LIBRARY_PATH and if not, adds it.
(To be fair, this could be written differently (like here), but assume the script is supposed to achieve something which is very hard to do without calling a program, checking $? and then either doing one thing or doing another thing).
The .gitlab-ci.yml then contains
before_script:
- yum install -y <various packages>
- source setup.sh
but the runner decides to stop the before script the very moment $? is non-zero, i.e. when the if-statement decides to add a path to $LD_LIBRARY_PATH.
Now it is nice that the gitlab runner checks $? after each line of my script, but here it'd be great if the lines in .gitlab-ci.yml were considered atomic.
Is there a way to avoid the intermediate checks of $? in a script that's sourced in .gitlab-ci.yml?
Use command_that_might_fail || true to mask the exit status of said command.
Also note that you can use grep -q to prevent output:
echo "$LD_LIBRARY_PATH" | grep -q "$LIBRARY" || true
This will however also mask $? which you might not want. If you want to check if the command exits correct you might use:
if echo "$LD_LIBRARY_PATH" | grep -q "$LIBRARY"; then
echo "Adding $LIBRARY"
else
...
fi
I suspect that gitlab-ci sets -e which you can disabled with set +e:
set +e # Disable exit on error
for library in "${LIBRARIES_WE_NEED[#]}"; do
...
done
set -e # Enable exit on error
Future reading: Why double quotes matter and Pitfalls with set -e
Another trick that I am using is a special kind of "|| true", combined with having access to previous exit code.
- exit_code=0
- ./myScript.sh || exit_code=$?
- if [ ${exit_code} -ne 0 ]; then echo "It failed!" ; else echo "It worked!"; fi
The $exit_code=$? always evaluates to "true" so you get a non failing command but you also receive exit_code and you can do whatever you want with it.
Note please, that you shouldn't skip the first line or exit_code will be uninitialized (since on successful run of script, the or'ed part is never executed and the if ends up being)
if [ -ne 0 ];
instead of
if [ 0 -ne 0 ];
Which causes syntax error.
I would like to convert and execute
if egrep -r 'my_pattern' ./template_builder
then exit 1
elif egrep -r 'my_second_pattern' ./template_builder
then exit 1
fi
in a Makefile, without success for now.
To build this:
cd /tmp;
mkdir template_builder;
echo "not_pattern" >> ./template_builder/test.txt
# Do the command at the top, nothing happens
echo "my_pattern" >> ./template_builder/test.txt
# Do the command at the top, terminal stops
touch Makefile
In a Makefile, I thought this would work :
check:
if egrep -r 'my_pattern' ./template_builder
then exit 1
elif egrep -r 'my_second_pattern' ./template_builder
then exit 1
fi
make check
if egrep -r 'my_pattern' ./template_builder
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [template] Error 2
How can I fix this?
Your attempt was not far from working!
Add backslashes at the end of every line, and ;s as explicit command separators (and of course use real tabs instead of the 8-space indents below):
check:
if egrep -r 'my_pattern' ./template_builder; \
then exit 1; \
elif egrep -r 'my_second_pattern' ./template_builder; \
then exit 1; \
fi
If I understand you correctly, if the directory template_builder located in /tmp does not contain a file matching the string 'my_pattern' or 'my_second_pattern', you want to exit from make with an error code.
You can achieve this with this rule in Makefile:
check:
egrep -r -v 'my_pattern' /tmp/template_builder || egrep -r -v 'my_second_pattern' /tmp/template_builder
Explanation: the first egrep is going to return an error in the case he finds a match. Due to the presence of the || operator, the second egrep will be invoked. The result of this second command will be the result that make will see. If it returns an error, the execution of make is aborted, which seems to be the behaviour you are expecting.
Caution: I edited my answer. The right boolean operator is || and not &&.
As others have already noted, make runs each separate line in a recipe in a new shell subprocess. (For the record, it uses sh out of the box, not Bash.) The trivial fix is to add a backslash to escape the newline at the end of each line which should be executed in the same shell as the next one. (You need to add a semicolon as well in some places, like before then and else and fi.) But you really want to refactor to use the facilities and idioms of make.
The default logic of make is to terminate the recipe if any line fails. So, your code can be reduced to simply
check: template_builder
! egrep -r 'my_pattern' $<
! egrep -r 'my_second_pattern' $<
The explicit exit 1 is not necessary here (negating a zero exit code produces exactly that); but if you wanted to force a particular exit code, you could do that with
egrep -r 'my_pattern' $< && exit 123 || true
Modern POSIX prefers grep -E over legacy egrep; of course, with these simple patterns, you can use just grep, or even grep -F (née fgrep).
Moreover, if you want to search for both patterns in the same set of files, it's much more efficient to search for them both at once.
check: template_builder
! egrep -e 'my_pattern' -e 'my_second_pattern' -r $<
... or combine them into a single regex my_(second_)?pattern (which requires egrep / grep -E).
Notice also how I factored out the dependency into $< and made it explicit; but you probably want to make this recipe .PHONY anyway, so that it gets executed even if nothing has changed.
(You can't directly copy/paste this code, because Stack Overflow stupidly renders the literal tabs in the markdown source as spaces.)
Given a Makefile that's often times run with the -j flag for parallel builds. I want it to terminate with a result message. I would like this message to say if the build failed, and if it failed, what the error was. It doesn't have to say anything if the build succeeded (although it could) but it must warn the user when a target failed to build and why.
This behavior is already there during sequential builds, but not during parallel builds. Parallel builds interweaves the output and an error message is often overlooked because output from other targets might push the failed target's error off screen. A careless developer might see no errors on his/her screen and assume the build succeeded.
It's quite an intuitive feature and I've searched for an answer, but it doesn't seem like there's any straight forward solutions. Any ideas?
You basically run
make -j 8 2> >(tee /tmp/error.log)
test $? -ne 0 && echo "build errors:"
cat /tmp/error.log
and you get all of stderr after the build finishes.
-- EDIT --
Updating to use tee, to output on stdout and into file:
Make returns non-zero if one of its recipe's fails so you could do something like this from the command line (assuming bash shell):
make 2>&1 | tee build.log
[ ${PIPESTATUS}[0] -eq 0 ] || ( echo "MAKE FAILED!"; grep --color build.log "Error:" )
The ${PIPESTATUS}[0] gives you the exit code of the first command (make 2>&1) as opposed to the exit status of the entire command (which would the exit status of tee if the make failed). It is bash specific, so it won't work in zsh for example.
Alternatively you could add the same logic as the top level target of a recursive make.
ifndef IN_RECURSION
export IN_RECURSION:=1
$(info At top level -- defining default target)
_default:
#echo "doing recursive call of make"
#$(MAKE) $(MAKECMDGOALS) IN_RECURSION=1 2>&1 | tee build.log; \
[ ${PIPESTATUS}[0] -eq 0 ] || ( echo "MAKE FAILED!"; grep --color "Error:" build.log )
.PHONY: _default
endif
all:
....
Note that in this case the \ used to catinate the two recipe lines is crucial, as the second command must run in the same shell instance as the first.
I have a target such as this:
.PHONY: do-symlink
do-symlink:
ln -s $(HERE_ONE_FILE) $(THERE_ONE_DIR)/
ln -s $(HERE_TWO_FILE) $(THERE_ONE_DIR)/
# check if those succeed or failed. I want to make sure both have passed
if succeeded:
do_something
else:
do_something_else
How can i check that they are both succeed? and if so, do something based on it?
So first of all, if a make recipe line fails, make fails, and stops executing the recipe. So if you put a recipe line at the end #echo "succeeded", then this will on run if all previous lines worked. As far as printing if something specific if one fails, you can use bash || for that
all:
#command1 || { echo "command1 failed"; exit 1 }
#command2 || { echo "command2 failed"; exit 1 }
#echo "command1 and command2 passed"
Notice the exit 1's in there. Normally, false || echo "false" would have a return status of 0 (pass), because the exit status is taken from the last command run (echo "false"), which will always succeed. This would cause make to always continue running the next recipe line. You may want this, however, you can preserve the failure by compounding the statement and doing an exit 1 at the end.
For running both commands regardless of exit status, and then handling the exits after, prefix the recipe lines with -. This will cause make to not stop running if the command fails. Notice however that each recipe line is run in its own shell, so one recipe line cannot directly access the return code from another line. You could output to a file and then access that file in a later recipe line:
all:
-#command1; echo $? > .commands.result
-#command2; echo $? >> .commands.result
#if grep -q "^00$" .commands.result; then \
echo both commands passed; \
else \
echo "at least one command failed"
(or you could always concatenate all the recipes lines into a single bash command and pass the variables around as well).