Make seems to compare a file modification time with some sort of internal cache of modification times. Does anyone know where is this cache located or how to access it?
Use a utility such as strace to see what it's doing. Here we can follow all file actions (-e trace=file) that the make program performs.
Let's say that we have foo.c and a foo which is built from it using a simple Makefile that looks like:
$ cat Makefile
foo: foo.c
Let's run make:
$ strace -e trace=file make
execve("/usr/bin/make", ["make"], [/* 20 vars */]) = 0
...
open("Makefile", O_RDONLY) = 3
stat("Makefile", {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
...
stat("foo", 0x7ffd4373c8c0) = -1 ENOENT (No such file or directory)
stat("foo.c", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
cc foo.c -o foo
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=27717, si_status=0, si_utime=0, si_stime=0} ---
stat("foo", {st_mode=S_IFREG|0755, st_size=8549, ...}) = 0
+++ exited with 0 +++
Here you see that it checks for the existence of foo and fails to find it, it checks for the existence of foo.c, and it then calls the rule to compile it.
Now you can call make again, and you get slightly different results:
$ strace -e trace=file make
...
stat("foo", {st_mode=S_IFREG|0755, st_size=8549, ...}) = 0
stat("foo.c", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
make: `foo' is up to date.
+++ exited with 0 +++
If there was some kind of a cache file being used, we'd expect to see an open() and a read() against it. If this cache was being used between runs, then we'd expect to see a write() to it as well.
The cache as such is your file system:
The make program reads the makefile,
determines the list of targets and dependencies,
observes the timestamps for these targets and dependencies,
works backward from the target you specify (or the default target, i.e., the first one in the makefile), to find a target whose dependencies are newer.
it runs the rule for the out-of-date target, and repeats until everything is up to date.
As it performs rules, the target it builds is expected to be updated (newer), but it moves on to the next out-of-date target from its initial inspection of the filesystem even if the target is not actually updated.
Re-executing make forces it to start over with the analysis, using the current state of your file system.
Related
I've got a makefile that has a 'contingency step' to bootstrap the environment if the task is run without prior, critical setup:
bootstrap: ensure_opam_switch_once build
.ONESHELL:
$(FRONTEND_OPAM_SWITCH):
#echo ''
#echo "$(bold)[make bootstrap]$(sgr0) An opam-switch doesn't exist for frontend/, creating one ..."
OPAMYES=true $(MAKE) init
opam exec --sw="${FRONTEND_DIR}" -- $(MAKE) bootstrap
#EXIT_CODE=$$?
#exit $$EXIT_CODE
ensure_opam_switch_once: | $(FRONTEND_OPAM_SWITCH)
init:
# this does critical first-time setup
build:
# ... everything else happens here
The issue is that several shell-variables need to be set for the rest of the makefile — and I'm not guaranteed to know those variables. (opam sets up some stuff with eval; think rbenv, nvm, pyenv, etc.)
My plan/hack here is to re-execute the make-task after setup, with those shell-variables configured in the environment make is invoked from. (In the above example, that's the opam exec line.)
However, my issue at the moment is that, if the sub-make exits with success, then my #exit $$EXIT_CODE doesn't result in Make exiting! It continues to run and attempt to execute other dependents, which obviously fails because the outer Make invocation doesn't have any of the necessary shell-environment.
Is there a way to stop Make with a successful exit-code?
You could use conditionals to hide or not parts of your Makefile. Example with a PASS make variable that defines what part of the Makefile shall be used:
$ cat Makefile
.PHONY: all
all:
ifeq ($(PASS),)
#echo 'PASS = $(PASS), VAR = $(VAR)'
export VAR=VAL; $(MAKE) PASS=1
else
#echo 'PASS = $(PASS), VAR = $(VAR)'
endif
$ make
PASS = , VAR =
export VAR=VAL; make PASS=1
make[1]: Entering directory 'foo'
PASS = 1, VAR = VAL
make[1]: Leaving directory 'foo'
I have a number of makefiles that build and run tests. I would like to create a script that makes each one and notes whether the tests passed or failed. Though I can determine test status within each make file, I am having trouble finding a way to communicate that status to the caller of the make command.
My first thought is to somehow affect the return value of the make command, though this does not seem possible. Can I do this? Is there some other form of communication I can use to express the test status to the bash script that will be calling make? Perhaps by using environment variables?
Thanks
Edit: It seems that I cannot set the return code for make, so for the time being I will have to make the tests, run them in the calling script instead of the makefile, note the results, and then manually run a make clean. I appreciate everyone's assistance.
Make will only return one of the following according to the source
#define MAKE_SUCCESS 0
#define MAKE_TROUBLE 1
#define MAKE_FAILURE 2
MAKE_SUCCESS and MAKE_FAILURE should be self-explanatory; MAKE_TROUBLE is only returned when running make with the -q option.
That's pretty much all you get from make, there doesn't seem to be any way to set the return code.
The default behavior of make is to return failure and abandon any remaining targets if something failed.
for directory in */; do
if ( cd "$directory" && make ); then
echo "$0: Make in $directory succeeded" >&2
else
echo "$0: Make in $directory failed" >&2
fi
done
Simply ensure each test leaves its result in a file unique to that test. Least friction will be to create test.pass if thes test passes, otherwise create test.fail. At the end of the test run gather up all the files and generate a report.
This scheme has two advantages that I can see:
You can run the tests in parallel (You do us the -jn flag, don't you? (hint: it's the whole point of make))
You can use the result files to record whether the test needs to be re-run (standard culling of work (hint: this is nearly the whole point of make))
Assuming the tests are called test-blah where blah is any string, and that you have a list of tests in ${tests} (after all, you have just built them, so it's not an unreasonable assumption).
A sketch:
fail = ${#:%.pass=%.fail}
test-passes := $(addsuffix .pass,${tests})
${test-passes}: test-%.pass: test-%
rm -f ${fail}
touch $#
$* || mv $# ${fail}
.PHONY: all
all: ${test-passes}
all:
# Count the .pass files, and the .fail files
echo '$(words $(wildcard *.pass)) passes'
echo '$(words $(wildcard *.fail)) failures'
In more detail:
test-passes := $(addsuffix .pass,${tests})
If ${tests} contains test-1 test-2 (say), then ${test-passes} will be test-1.pass test-2.pass
${test-passes}: test-%.pass: test-%
You've just gotta love static pattern rules.
This says that the file test-1.pass depends on the file test-1. Similarly for test-2.pass.
If test-1.pass does not exist, or is older than the executable test-1, then make will run the recipe.
rm -f ${fail}
${fail} expands to the target with pass replaced by fail, or test-1.fail in this case. The -f ensures the rm returns no error in the case that the file does not exist.
touch $# — create the .pass file
$< || mv $# ${fail}
Here we run the executable
If it returns success, our work is finished
If it fails, the output file is deleted, and test-1.fail is put in its place
Either way, make sees no error
.PHONY: all — The all target is symbolic and is not a file
all: ${test-passes}
Before we run the recipe for all, we build and run all the tests
echo '$(words $(wildcard *.pass)) passes'
Before passing the text to the shell, make expands $(wildcard) into a list of pass files, and then counts the files with $(words). The shell gets the command echo 4 passes (say)
You run this with
$ make -j9 all
Make will keep 9 jobs running at once — lovely if you have 8 CPUs.
I'm incrementally verifying my build output and I want to be able to exit after a given recipe is executed.
If the original recipe is
$(HEADER_BUILD)/mpversion.h: FORCE | $(HEADER_BUILD)
$(Q)$(PYTHON) $(PY_SRC)/makeversionhdr.py $#
I want to be able to add one line at the end like so
$(HEADER_BUILD)/mpversion.h: FORCE | $(HEADER_BUILD)
$(Q)$(PYTHON) $(PY_SRC)/makeversionhdr.py $#
some_command
and some_command should merely stop the execution of the makefile without interfering with the rest of the recipe.
If I set some_command as exit 1, I get
../py/py.mk:269: recipe for target 'build-gnu/genhdr/mpversion.h'
failed make: * [build-gnu/genhdr/mpversion.h] Error 1 make: *
Deleting file 'build-gnu/genhdr/mpversion.h'
If I set some_command as $(error), the recipe isn't even executed even though it's BEFORE the $(error)
Is there such a command that stops executing the makefile but doesn't delete the target?
UPDATE
I've found this hack: make .PRECIOUS depend on the target and add exit 1 as the last line in the recipe.
If the file you want to keep is an intermediate file (not mentioned as target or dependency of a rule - but possibly implied via a pattern rule), then you'll need to make it a dependency of .PRECIOUS.
Otherwise, it should be sufficient to temporarily remove or comment out the .DELETE_ON_ERROR: target that we all put in every Makefile.
I'm starting to use GNU Make as my frontend build tool, and most things work great. Only thing annoying is that compilation doesn't seem to stop when one of the steps reaches an error. The relevant portion of the Makefile:
js_files=$(filter-out $(ignore_js),$(wildcard \
js/ll/*.js js/ll/**/*.js))
ignore_js=js/ll/dist% js/ll/%.min.js
%.min.js: %.js
#echo ">>> Uglifying $?"
#$(BABELJS) $(BABELJSFLAGS) $? | $(UGLIFYJS) --source-map $(UGLIFYJSFLAGS) > $#
min_js_files=$(js_files:%.js=%.min.js)
main.js: $(min_js_files)
#echo ">>> Concatenating JavaScript"
mkdir -p $(DIST_DIR)
cat $^ > $(DIST_DIR)$#
prod: main.js clean
The output I get from running make prod looks something like this:
>>> Uglifying js/ll/DateEx.js
SyntaxError: js/ll/DateEx.js: Invalid number (22:36)
20 | day = today.getDate();
21 | }
> 22 | return new Date(year, month, day, 01, 0, 0);
| ^
23 | }
24 |
25 | function newDateS(s)
>>> Uglifying js/ll/Anonymization.js
>>> Uglifying js/ll/DummyStorage.js
(...)
I have the impression that this happens because the steps run in parallel, but I know nothing about Make to back that up. How can I have compilation stop when one of the steps returns non-zero?
What you are asking for is the well-established default behavior of Make. Something in your build chain is not properly setting a nonzero exit code on failure, or you are masking it out.
In particular, the exit code from a shell pipeline is always the exit code from the final command in the pipeline. In other words, any error from BABELJS in your recipe will be lost.
Maybe refactor to not use a pipe, perhaps something like this:
%.min.js: %.js.tmp
$(UGLIFYJS) --source-map $(UGLIFYJSFLAGS) <$< >$#
%.js.tmp: %.js
$(BABELJS) $(BABELJSFLAGS) $< >$#
.PHONY: clean
clean:
rm *.js.tmp
The use of a temporary file is a bit of a wart, and the choice whether to use a separate recipe for the first step is kind of arbitrary. I prefer this style because it better adheres to the spirit of Make (explicitly declare dependencies, let Make keep track) but if you want fine-grained control over the flow, you could go the other way.
I also removed the # prefix from the BABELJS rule. This is a bit of a private crusade of mine -- littering your Makefile with these makes debugging hard, and the proper solution if you want peace and quiet is to use make -s.
I have a Makefile of the following content:
NUMBERS = 1 2 3 4
lib:
$(foreach var,$(NUMBERS),./a.out $(var);)
And this is the command that I run ( in the same directory as the Makefile)
make -f Makefile
But I got an error message saying that "The system cannot find the file specified".
Following the suggestion of one of the answers, I created the following file inside the same directory as the Makefile:
a.out
1.out
2.out
3.out
4.out
Now the error becomes:
./a.out 1; ./a.out 2; ./a.out 3;
./a.out 4; make (e=-1): Error -1 make:
*** [lib] Error -1
Note: I am running on Windows XP platform
The purpose of make is to create (and update) target files that depends on source files by running commands.
Here, the problem is with the command that is run. You are trying to run (through make) the command a.out but it does not exist, or is not an executable command. Try to replace a.out in your makefile by the actual executable command you want to run.
On Windows/DOS, use && instead of ; to join multiple commands on one line. You have to manually include a final command or the trailing && will throw a syntax error. Try something like:
NUMBERS = 1 2 3 4
lib:
$(foreach var,$(NUMBERS),.\a.out $(var) && ) echo.
It seems to me that the error comes because the file a.out cannot be located and not because the makefile could not be found.
Also if the name of your makefile is "Makefile" just invoking "make" is enough (without using -f option) as make by default would look for a file by names: GNUmakefile, makefile, and Makefile in that order.
Just what are you trying to do?
It seems to me that a plain script would be better suited rather than using make.
I found the answer by bta most useful, but it didn't work for me on both Windows and Linux, so I found a way to remove the final &&, which avoids the need for a no-op command that works on both platforms:
NUMBERS = 1 2 3 4
lib:
$(filter-out &&EOL, $(foreach var,$(NUMBERS), .\a.out $(var) &&)EOL)
Of course be careful of elements within your array matching &&EOL, but in my case, this isn't a problem.