Say I have the following files:
buggy_program:
#!/bin/sh
echo "wops, some bug made me exit with failure"
exit 1
Makefile:
file.gz:
buggy_program | gzip -9 -c >$#
Now if I type make, GNU make will happily build file.gz even though buggy_program exited with non-zero status.
In bash I could do set -o pipefail to make a pipeline exit with failure if at least one program in the pipeline exits with failure. Is there a similar method in GNU make? Or some workaround that doesn't involve temporary files? (The reason for gzipping here is precisely to avoid a huge temporary file.)
Try this
SHELL=/bin/bash -o pipefail
file.gz:
buggy_program | gzip -9 -c >$#
You could do:
SHELL=/bin/bash
.DELETE_ON_ERROR:
file.gz:
set -o pipefail; buggy_program | gzip -9 -c >$#
but this only work with bash.
Here's a possible solution that doesn't require bash. Imagine you have two programs thisworks and thisfails that fail or work fine, respectively. Then the following will only leave you with work.gz, deleting fail.gz, ie. create the gzipped make target if and only if the program executed correctly:
all: fail.gz work.gz
work.gz:
( thisworks && touch $#.ok ) | gzip -c -9 >$#
rm $#.ok || rm $#
fail.gz:
( thisfails && touch $#.ok ) | gzip -c -9 >$#
rm $#.ok || rm $#
Explanation:
In the first line of the work.gz rule, thisworks will exit with success, and a file work.gz.ok will be created, and all stdout goes through gzip into work.gz. Then in the second line, because work.gz.ok exists, the first rm command also exits with success – and since || is short-circuiting, the second rm does not get run and so work.gz is not deleted.
OTOH, in the first line of the fail.gz rule, thisfails will exit with failure, and fail.gz.ok will not be created. All stdout still goes through gzip into fail.gz. Then in the second line, because fail.gz.ok does not exist, the first rm command exits with failure, so || tries the second rm command which deletes the fail.gz file.
To easily check that this works as it should, simply replace thisworks and thisfails with the commands true and false, respectively, put it in a Makefile and type make.
(Thanks to the kind people in #autotools for helping me with this.)
Related
I tried to suppress an error from rm command by writing
Makefile:
...
clean: $(wildcard *.mod)
-rm $^ 2>/dev/null
...
I ran:
$ make clean
rm 2>/dev/null
make: [clean] Error 64 (ignored)
I still had gotten an error.
Anyway, when I tried
$ rm [some non-existent files] 2>/dev/null
on the bash shell, it just works fine.
How can I use 2>/dev/null inside a makefile?
2>dev/null will redirect the error output so you don't see it, it will not prevent the shell to raise the error level. And the - sign in front of your shell command will tell GNU make to continue even if the error level is raised but it will not either prevent the shell to raise it.
What you want is the shell not to raise the error level and this can be done like this :
Unix (credits to this answer)
-rm $^ 2>/dev/null ; true
Windows
-rm $^ 2>NUL || true
or if you don't have rm on Windows
-del /F /Q $^ 2>NUL || true
The message make: [clean] Error 64 (ignored) is being printed by make after it sees that your shell command has failed.
It will therefore not be affected by any redirection that you use in the recipe.
Two fixes:
Use the -f rm flag. rm -f never returns an error.
(Well, hardly ever anyway, and if it does you probably want to know about it!)
Stop the shell command returning an error: simply append || : to the command.
Say what? Well if the rm succeeds your job is done and make is happy. OTOH if rm fails, the shell runs the second command in the or.
: is a shell built-in that always succeeds, and is much preferable to true IMHO.
The first of these is best in this case,
though the second is a general, if somewhat less efficient, pattern.
.PHONY: clean
clean: ; rm -rf *.mod
In the following example I would like foo to be deleted in case of error. Unfortunately it doesn't work.
foo:
perl -e 'die()' > $# || [rm $# -a true]
What is it wrong?
GNU make can do that for you.
Special Built-in Target Names:
.DELETE_ON_ERROR
If .DELETE_ON_ERROR is mentioned as a target anywhere in the makefile, then make will delete the target of a rule if it has changed and its recipe exits with a nonzero exit status, just as it does when it receives a signal.
It is a general problem that creating a file is a non-atomic operation. And not always you can delete an incomplete or corrupted file on termination, for example, when the program is killed with SIGKILL or by the OOM-killer. In other words, all solutions involving removing the file are prone to failures.
The robust generic solution is:
Create the file with a temporary filename.
Once the file is complete and have correct permissions, rename it to the final name. Renaming a file is an atomic operations in UNIX, as long as the file stays in the same filesystem.
E.g.:
foo:
perl -e 'die()' > $#~
mv --force $#~ $#
This works for me…
foo:
#perl -e 'die()' > $# || { echo "removing $# because exit code was $${?}"; rm $#; }
Output
Died at -e line 1.
removing foo because exit code was 255
I'm trying to pipe to the output of the compiler to the tee command in windows, but I've ran into an issue where if the compiler fails within make it'll continue compiling the next file when I want to it to stop. Is there a way to have the exit status of the first command be the exit status of the second command?
$(ODIR)/%.o: %.c $(DEPS)
$(CC) -c -o $# $< $(CFLAGS) 2>&1 | tee build_log.txt
First off, I would leave any logging to the caller of make. Second off, this sort of piping is untidy in make. Third off, not a fan of losing the stderr stream inside make.
That said, this is a shell question. If you are using bash then see pipefail in the manual. Unfortunately I think it's quite tricky to turn on. (Yeah, I know you said windows, but I assume you aren't using the execrable cmd.)
SHELL := /bin/bash
passes:
(exit 1) |& cat
fails:
bash -c 'set -o pipefail; (exit 1) |& cat'
After Struggling a lot, I came to this solution ...
.ONESHELL:
$(ODIR)/%.o: %.c $(DEPS)
$(CC) -c -o $# $< $(CFLAGS) 2> temp_err_file
set EXIT_STATUS=%ERRORLEVEL%
type temp_err_file >> build_log.txt
type temp_err_file 1>&2
del /q temp_err_file
exit /b %EXIT_STATUS%
Here .ONESHELL allows make to run entire recipe in the single shell command instead of running each line in separate cmd and collect return status of each separately. Overall exit status depends on the main compilation command so in end it is necessary to exit with the status of compilation.
I know its not a clean solution involving the temp_err_file and if something goes wrong after compilation command, make would not be able to catch it but this I think is the best I can find to work with windows without losing stderr stream and logging.
A method that is shell independent and may be feasible in some cases is the following:
assume you have a recipe:
target:
try_making_target |& tee target.log
What I did was convert it to:
target:
(try_making_target || rm -f $#) |& tee target.log
test -e $#
The the piped command fails, the "fallback" (command after ||) will delete the goal file, and the final test will fail. Note that this example assumes the OS is Linux ('rm' for deletions) and that your shell supports the || operator.
This assumes that you are not interested in partial results when try_making_target fails. If you want to keep partial result in 'target', you can use other "marker" files to designate the success or failure of try_making_target. Something like this may work:
target:
touch $#.succeeded # Assume success
(try_making_target || rm -f $#.succeeded) |& tee target.log # Delete to mark failure
test -e $#.succeeded # Fail if marker
rm $#.succeeded # Remove unneeded marker
This last code would also work for phony goals, although you really should minimize the use of these (I try only using them as mnemonics for real files, which may have longish names).
Criteria: Makefile is a GNU Make Makefile - I'm not interested in makepp, qmake, cmake, etc. They're all nice (especially cmake), but this is for work and at work we use GNU Make. The optimal solution is a pure Makefile solution rather than a shell script that parses make for you.
I also don't want to do a 'continue on failure' solution - if it's broken, it's broken and needs to be fixed.
The situation is this, I've got a makefile that builds several directories in parallel - if one of them fails, of course the whole build fails, but not until all the running makes run to completion (or failure). This means that the reason why make actually failed is buried somewhere arbitrarily far from the end of make's output.
Here's an example of what I've got:
all: $(SUBDIRS)
SUBDIRS = \
apple \
orange \
banana \
pineapple \
lemon \
watermelon \
grapefruit
$(SUBDIRS):
cd $# && $(MAKE) $(MFLAGS) 2>&1 | sed -e "s/^/$(notdir $(#)): /g"
If I run 'make -j 5' and 'orange' happens to fail - I'd like to see a table like this at the end
of the make process
apple - passed
orange - FAILED
banana - passed
pineapple - passed
lemon - passed
I've considered having an && echo "passed" >.result || echo "FAILED" >.result, but make still needs some sort of TRAP or __onexit() cleanup command to print at them on exit.
Any Makefile ninjas out there have a pure-makefile solution for this?
un-edit - my solution wasn't actually working the way I had hoped.. STYMIED!
When you want make to abort at the first failure, end immediately and kill all in-flight jobs instead of waiting for them to finish, you need to patch GNU Make like this
http://lists.gnu.org/archive/html/bug-make/2009-01/msg00035.html
Then you need to set a trap for every shell that make invokes (as well as set -o pipefail if you use a pipe), as described in this post http://lists.gnu.org/archive/html/help-make/2009-02/msg00011.html
In a nutshell:
target1:
trap 'kill $$(jobs -p)'; command && something || something-else
target2:
trap 'kill $$(jobs -p)'; set -o pipefail; command | sed '...'
The only way I see is self-execution with a sub-make:
all : subdirs
subdirs :
$(MAKE) -f $(lastword $(MAKEFILE_LIST)) subdirs-recursive || cat log
subdirs-recursive: $(SUBDIRS)
I'm making a Makefile that moves an output file (foo.o) to a different directory (baz).
The output file moves as desired to the directory. However since make won't recompile the output file if I type make again, mv gets an error when it tries to move the non-existent empty file to the directory baz.
So this is what I have defined in my rule make all after all compilation:
-test -e "foo.o" || mv -f foo.o ../baz
Unfortunately, I'm still getting errors.
Errors in Recipes (from TFM)
To ignore errors in a recipe line, write a - at the beginning of the
line's text (after the initial tab).
So the target would be something like:
moveit:
-mv foo.o ../baz
I notice nobody has actually answered the original question itself yet, specifically how to ignore errors (all the answers are currently concerned with only calling the command if it won't cause an error).
To actually ignore errors, you can simply do:
mv -f foo.o ../baz 2>/dev/null; true
This will redirect stderr output to null, and follow the command with true (which always returns 0, causing make to believe the command succeeded regardless of what actually happened), allowing program flow to continue.
+#[ -d $(dir $#) ] || mkdir -p $(dir $#)
is what I use to silently create a folder if it does not exist. For your problem something like this should work
-#[ -e "foo.o" ] && mv -f foo.o ../baz
-test -e "foo.o" || if [ -f foo.o ]; then mv -f foo.o ../baz; fi;
That should work
Something like
test -e "foo.o" && mv -f foo.o ../baz
should work: the operator should be && instead of ||.
You can experiment with this by trying these commands:
test -e testfile && echo "going to move the file"
test -e testfile || echo "going to move the file"
I faced the same problem and as I am generating files, they always have different time. Workaround is set the same time to the files: touch -d '1 June 2018 11:02' file. In that case, gzip generates the same output and same md5sum. In my scenario, I don't need the time for the files.