Given a Makefile that's often times run with the -j flag for parallel builds. I want it to terminate with a result message. I would like this message to say if the build failed, and if it failed, what the error was. It doesn't have to say anything if the build succeeded (although it could) but it must warn the user when a target failed to build and why.
This behavior is already there during sequential builds, but not during parallel builds. Parallel builds interweaves the output and an error message is often overlooked because output from other targets might push the failed target's error off screen. A careless developer might see no errors on his/her screen and assume the build succeeded.
It's quite an intuitive feature and I've searched for an answer, but it doesn't seem like there's any straight forward solutions. Any ideas?
You basically run
make -j 8 2> >(tee /tmp/error.log)
test $? -ne 0 && echo "build errors:"
cat /tmp/error.log
and you get all of stderr after the build finishes.
-- EDIT --
Updating to use tee, to output on stdout and into file:
Make returns non-zero if one of its recipe's fails so you could do something like this from the command line (assuming bash shell):
make 2>&1 | tee build.log
[ ${PIPESTATUS}[0] -eq 0 ] || ( echo "MAKE FAILED!"; grep --color build.log "Error:" )
The ${PIPESTATUS}[0] gives you the exit code of the first command (make 2>&1) as opposed to the exit status of the entire command (which would the exit status of tee if the make failed). It is bash specific, so it won't work in zsh for example.
Alternatively you could add the same logic as the top level target of a recursive make.
ifndef IN_RECURSION
export IN_RECURSION:=1
$(info At top level -- defining default target)
_default:
#echo "doing recursive call of make"
#$(MAKE) $(MAKECMDGOALS) IN_RECURSION=1 2>&1 | tee build.log; \
[ ${PIPESTATUS}[0] -eq 0 ] || ( echo "MAKE FAILED!"; grep --color "Error:" build.log )
.PHONY: _default
endif
all:
....
Note that in this case the \ used to catinate the two recipe lines is crucial, as the second command must run in the same shell instance as the first.
Related
Having this simple makefile rule:
exe:
for i in *; do [ -x "$$i" ] && echo "$$i"; done
Will output:
for i in *; do [ -x "$i" ] && echo "$i"; done
executablefile
make: *** [makefile:6: exe] Error 1
So it does, what I want, but even then, error with no other message. But not only for this particular example (which I still do not get), I would like to know, how to get some more info from bugs in makefile (is there a makefile debugger?). From makefile manual the *** is for fatal error, which ends compilation, but yet it outputs the executablefile (so it did compiled to that point). Apart from fatal error, - warnings give more info, so why do not do fatal errors as well?
explanation of this example
some advices how to debug makefile scripts
This is not an error from make, that's why there's no other information.
Make runs a shell and gives the shell your recipe to invoke. If the shell exits with success (exit code 0) then make assumes that the command it ran worked. If the shell exits with failure (any exit code other than 0), then make assumes the command it ran failed. Make doesn't know why it failed, make assumes that whatever command failed will have printed some information about why. All make knows is the exit code, so that's all make can tell you:
make: *** [makefile:6: exe] Error 1
This means that make ran the recipe for target exe at makefile line number 6, and that command exited with an error code 1 (which is not 0, hence a failure).
Why did this happen? Let's look at your shell script:
for i in *; do [ -x "$$i" ] && echo "$$i"; done
Let's suppose the last file matching * (so the last time we go through the loop) the file is not executable. That means the test of the last file [ -x "$$i" ] will fail. Since that's the last command that the shell runs before it exits, that will be the exit code of the shell, and you have a failure.
You need to be sure that the shell exits with success. One way to do that is ensure the last command the shell runs is always success; maybe something like this:
for i in *; do [ -x "$$i" ] && echo "$$i"; done; true
I have a target such as this:
.PHONY: do-symlink
do-symlink:
ln -s $(HERE_ONE_FILE) $(THERE_ONE_DIR)/
ln -s $(HERE_TWO_FILE) $(THERE_ONE_DIR)/
# check if those succeed or failed. I want to make sure both have passed
if succeeded:
do_something
else:
do_something_else
How can i check that they are both succeed? and if so, do something based on it?
So first of all, if a make recipe line fails, make fails, and stops executing the recipe. So if you put a recipe line at the end #echo "succeeded", then this will on run if all previous lines worked. As far as printing if something specific if one fails, you can use bash || for that
all:
#command1 || { echo "command1 failed"; exit 1 }
#command2 || { echo "command2 failed"; exit 1 }
#echo "command1 and command2 passed"
Notice the exit 1's in there. Normally, false || echo "false" would have a return status of 0 (pass), because the exit status is taken from the last command run (echo "false"), which will always succeed. This would cause make to always continue running the next recipe line. You may want this, however, you can preserve the failure by compounding the statement and doing an exit 1 at the end.
For running both commands regardless of exit status, and then handling the exits after, prefix the recipe lines with -. This will cause make to not stop running if the command fails. Notice however that each recipe line is run in its own shell, so one recipe line cannot directly access the return code from another line. You could output to a file and then access that file in a later recipe line:
all:
-#command1; echo $? > .commands.result
-#command2; echo $? >> .commands.result
#if grep -q "^00$" .commands.result; then \
echo both commands passed; \
else \
echo "at least one command failed"
(or you could always concatenate all the recipes lines into a single bash command and pass the variables around as well).
I'm trying to pipe to the output of the compiler to the tee command in windows, but I've ran into an issue where if the compiler fails within make it'll continue compiling the next file when I want to it to stop. Is there a way to have the exit status of the first command be the exit status of the second command?
$(ODIR)/%.o: %.c $(DEPS)
$(CC) -c -o $# $< $(CFLAGS) 2>&1 | tee build_log.txt
First off, I would leave any logging to the caller of make. Second off, this sort of piping is untidy in make. Third off, not a fan of losing the stderr stream inside make.
That said, this is a shell question. If you are using bash then see pipefail in the manual. Unfortunately I think it's quite tricky to turn on. (Yeah, I know you said windows, but I assume you aren't using the execrable cmd.)
SHELL := /bin/bash
passes:
(exit 1) |& cat
fails:
bash -c 'set -o pipefail; (exit 1) |& cat'
After Struggling a lot, I came to this solution ...
.ONESHELL:
$(ODIR)/%.o: %.c $(DEPS)
$(CC) -c -o $# $< $(CFLAGS) 2> temp_err_file
set EXIT_STATUS=%ERRORLEVEL%
type temp_err_file >> build_log.txt
type temp_err_file 1>&2
del /q temp_err_file
exit /b %EXIT_STATUS%
Here .ONESHELL allows make to run entire recipe in the single shell command instead of running each line in separate cmd and collect return status of each separately. Overall exit status depends on the main compilation command so in end it is necessary to exit with the status of compilation.
I know its not a clean solution involving the temp_err_file and if something goes wrong after compilation command, make would not be able to catch it but this I think is the best I can find to work with windows without losing stderr stream and logging.
A method that is shell independent and may be feasible in some cases is the following:
assume you have a recipe:
target:
try_making_target |& tee target.log
What I did was convert it to:
target:
(try_making_target || rm -f $#) |& tee target.log
test -e $#
The the piped command fails, the "fallback" (command after ||) will delete the goal file, and the final test will fail. Note that this example assumes the OS is Linux ('rm' for deletions) and that your shell supports the || operator.
This assumes that you are not interested in partial results when try_making_target fails. If you want to keep partial result in 'target', you can use other "marker" files to designate the success or failure of try_making_target. Something like this may work:
target:
touch $#.succeeded # Assume success
(try_making_target || rm -f $#.succeeded) |& tee target.log # Delete to mark failure
test -e $#.succeeded # Fail if marker
rm $#.succeeded # Remove unneeded marker
This last code would also work for phony goals, although you really should minimize the use of these (I try only using them as mnemonics for real files, which may have longish names).
I would like to check the result of a grep search in a Makefile. Contrary to this solution, I do not wish to use the shell command.
Also, I don't want the Makefile to raise an error when grep do not find the string (exit code of 1 is treated as an error).
The following tries to ignore the error and check the exit code :
all:
-grep term log*
echo $$?
#case "$$?" in \
0)\
echo "found";; \
*) \
echo "not found";;\
esac;
Unfortunately, the exit code is always 0.
The separate lines of a series of actions in a makefile are normally executed in separate sub-shells. To code what you're after, then:
all:
if grep term log*; \
then echo found; \
else echo not found; \
fi
That's a single command; it tests the exit status of grep directly. Note the liberal use of semi-colons; that's necessary because it all gets flattened when passed to the shell. Note too that the - is not needed; the statement as a whole exits with status 0 because one of the echo commands is executed, succeeds, and that is the status returned from the sub-shell. But there's another part to the trick; IIRC, the script is invoked with /bin/sh -e so the script exits on the first error (non-zero) status from a shell command — except in explicit conditionals such as an if.
If you want to explicitly capture the status of grep (if only to be sure it's being done right), then:
all:
-grep term log*; \
status=$$?; echo $$status; \
if [ $$status = 0 ]; \
then echo found; \
else echo not found; \
fi
You probably need the - this time because the grep is not executed as part of a shell conditional and a non-zero exit status could trigger the -e processing. I don't recommend futzing with this.
You might note that you can do cd commands in an action and because each action is executed separately, you have to do it repeatedly.
install: ${PROG}
cd ${INSTBIN}; ${RM_F} ${PROG}
${CP} ${PROG} ${INSTBIN}
cd ${INSTBIN}; ${CHOWN} ${OWNER}:${GROUP} ${PROG}; ${CHMOD} ${PERMS} ${PROG}
Yes, you can do it differently — I'm demonstrating a point, not advocating a style of installing programs.
Criteria: Makefile is a GNU Make Makefile - I'm not interested in makepp, qmake, cmake, etc. They're all nice (especially cmake), but this is for work and at work we use GNU Make. The optimal solution is a pure Makefile solution rather than a shell script that parses make for you.
I also don't want to do a 'continue on failure' solution - if it's broken, it's broken and needs to be fixed.
The situation is this, I've got a makefile that builds several directories in parallel - if one of them fails, of course the whole build fails, but not until all the running makes run to completion (or failure). This means that the reason why make actually failed is buried somewhere arbitrarily far from the end of make's output.
Here's an example of what I've got:
all: $(SUBDIRS)
SUBDIRS = \
apple \
orange \
banana \
pineapple \
lemon \
watermelon \
grapefruit
$(SUBDIRS):
cd $# && $(MAKE) $(MFLAGS) 2>&1 | sed -e "s/^/$(notdir $(#)): /g"
If I run 'make -j 5' and 'orange' happens to fail - I'd like to see a table like this at the end
of the make process
apple - passed
orange - FAILED
banana - passed
pineapple - passed
lemon - passed
I've considered having an && echo "passed" >.result || echo "FAILED" >.result, but make still needs some sort of TRAP or __onexit() cleanup command to print at them on exit.
Any Makefile ninjas out there have a pure-makefile solution for this?
un-edit - my solution wasn't actually working the way I had hoped.. STYMIED!
When you want make to abort at the first failure, end immediately and kill all in-flight jobs instead of waiting for them to finish, you need to patch GNU Make like this
http://lists.gnu.org/archive/html/bug-make/2009-01/msg00035.html
Then you need to set a trap for every shell that make invokes (as well as set -o pipefail if you use a pipe), as described in this post http://lists.gnu.org/archive/html/help-make/2009-02/msg00011.html
In a nutshell:
target1:
trap 'kill $$(jobs -p)'; command && something || something-else
target2:
trap 'kill $$(jobs -p)'; set -o pipefail; command | sed '...'
The only way I see is self-execution with a sub-make:
all : subdirs
subdirs :
$(MAKE) -f $(lastword $(MAKEFILE_LIST)) subdirs-recursive || cat log
subdirs-recursive: $(SUBDIRS)