I can't understand chapter 3 of official gmake documentation starting from section 3.5
https://www.gnu.org/software/make/manual/make.html#Remaking-Makefiles
Docs say:
... after reading in all makefiles make will consider each as a goal target and attempt to update it. If a makefile has a rule which says how to update it ...., it will be updated if necessary. After all makefiles have been checked, if any have actually been changed, make starts with a clean slate and reads all the makefiles over again. (It will also attempt to update each of them over again, but normally this will not change them again, since they are already up to date.) Each restart will cause the special variable MAKE_RESTARTS to be updated.
So I try with this mkfile ( presence of the -include as G.M. has suggested changes nothing ):
all:
echo all $(MAKE_RESTARTS) aaa
#-include: mkfile
mkfile:
echo touch
touch mkfile
I expect that after mkfile runs first time mkfile modification time changed and make must run it again and again.
But nothing happens:
$ make -f mkfile
echo all aaa
all aaa
$
Moreover, target "mkfile" not called as expecting from docs: after reading in all makefiles make will consider each as a goal target and attempt to update it.
What I'm misunderstand ? Can you, please, give an example of what docs mean about "remading" makefiles?
UPD: I've tried Andreas' suggestions but got strange results again :
original mkfile but run with --always-make
$ make --always-make -f mkfile
echo touch
touch
touch mkfile
echo all 1 aaa
all 1 aaa
here target "mkfile" runs once and target "all" runs once too. The results of consequence runs are repeatable.
If I change target "mkfile" to force run, results a strange and differ from invocation to invocation:
mkfile:
all:
echo all $(MAKE_RESTARTS) aaa
mkfile: force
echo touch
touch mkfile
force: ;
results:
$ make -f mkfile
echo touch
touch
touch mkfile
echo touch
touch
touch mkfile
echo touch
touch
touch mkfile
................
touch mkfile
echo touch
touch
touch mkfile
echo all 16 aaa
all 16 aaa
$ make -f mkfile
echo touch
touch
touch mkfile
echo touch
touch
touch mkfile
echo touch
touch
touch mkfile
echo all 2 aaa
all 2 aaa
$ make -f mkfile
echo touch
touch
touch mkfile
...............
touch mkfile
echo all 13 aaa
all 13 aaa
$
UPD2 - about MAKE_RESTARTS
--always-make respects MAKE_RESTARTS only if makefile's target has no prerequisites.
mkfile:
all:
echo all $(MAKE_RESTARTS) aaa
mkfile:
echo touch $(MAKE_RESTARTS)
sleep 0.5
touch mkfile
result:
$ make --always-make -f mkfile
echo touch
touch
sleep 0.5
touch mkfile
echo all 1 aaa
all 1 aaa
$
if makefile's target has force prerequisite, loop is infinite (I even try to set MAKE_RESTARTS by force to value greater than 0 )
mkfile:
MAKE_RESTARTS := 1
all:
echo all $(MAKE_RESTARTS) aaa
mkfile: force
echo touch $(MAKE_RESTARTS)
sleep 0.5
touch mkfile
force: ;
result:
$ make --always-make -f mkfile
echo touch 1
touch 1
sleep 0.5
touch mkfile
echo touch 1
touch 1
sleep 0.5
touch mkfile
echo touch 1
touch 1
sleep 0.5
touch mkfile
^C
For this:
all:
echo all $(MAKE_RESTARTS) aaa
mkfile:
echo touch
touch mkfile
When make -f mkfile runs it tries to build mkfile. The make manual says that if a target has no prerequisites, then if it exists it's considered up-to-date (how can it be out of date when it doesn't depend on anything?) Since the mkfile file exists (obviously), this rule is really a no-op. If you want to see the behavior of rebuilding makefiles then the makefile has to have some prerequisite which shows that it's out of date.
If you run it with --always-make you'll run across this statement in the GNU make manual:
GNU make proceeds to
consider targets and their prerequisites using the normal algorithms;
however, all targets so considered are always remade regardless of the
status of their prerequisites. To avoid infinite recursion, if
MAKE_RESTARTS is set to a number greater than 0 this option is
disabled when considering whether to remake makefiles.
(Emphasis added).
If you run with a force target, then it will be updated as you've seen. For some operating systems this will run forever. However for others, at some point the touch command will run fast enough that it won't actually change the modification time of the mkfile file; it will be so fast that the modification time before and after the command runs will be the same. When a recipe runs but doesn't change the modification time of that target, then the target is not considered to be modified and any rules that depend on it will not be considered out-of-date because of that target.
When that happens, make believes that the target was not changed and it no longer re-execs itself.
Related
The working example below always runs touch fileA with each run of make. How can I stop this?
all: analysis.txt
analysis.txt: countries
touch $#
countries: usa.txt mexico.txt
$(countries): %.txt
%.txt:
echo "bash ./cityGenerator $*" > $#
fileA depends on workflowB. So as long as workflowB is out of date, fileA will be out of date and the touch will be run.
workflowB, as shown here, is marked .PHONY so it will ALWAYS be considered out of date by make, so fileA will ALWAYS be rebuilt.
Even if it weren't .PHONY, there is no rule to create a file named workflowB so it would still always be out of date.
As for how you can stop it, that depends entirely on what these things do which you haven't told us, and why you structured workflowB that way which you also haven't told us.
One way to solve it would be to remove the .PHONY and give workflowB a recipe that touches a file, then it would be up to date.:
workflowB: workflowB_files
touch $#
But of course, that might defeat the purpose of this (which, again, you haven't told us).
Your analysis.txt depends on countries which is considered as a file, but such file is never created, hence the rule is always run. You can find out by the debug output:
$ make -dr
...
Finished prerequisites of target file 'analysis.txt'.
Prerequisite 'countries' of target 'analysis.txt' does not exist.
Must remake target 'analysis.txt'.
touch analysis.txt
...
It seems like you are treating countries as a variable, so maybe this could be reworked:
$ cat Makefile
all: analysis.txt
countries := usa.txt mexico.txt
analysis.txt: $(countries)
touch $#
%.txt:
echo "bash ./cityGenerator $*" > $#
Now there is no countries file considered and make behaves as expected:
$ make
echo "bash ./cityGenerator usa" > usa.txt
echo "bash ./cityGenerator mexico" > mexico.txt
touch analysis.txt
$ make
make: Nothing to be done for 'all'.
#!make
.PRECIOUS: a/b/c/%_pqr
a/b/c/%_pqr:
echo $#
touch $#
.PRECIOUS: a/b/c/%_abc
a/b/c/%_abc: a/b/c/pqr_pqr
echo $#
touch $#
When I run first time, it honors its dependency:
make a/b/c/pqr_abc -f 1.mk
echo a/b/c/pqr_pqr
a/b/c/pqr_pqr
touch a/b/c/pqr_pqr
echo a/b/c/pqr_abc
a/b/c/pqr_abc
touch a/b/c/pqr_abc
But if I delete "a/b/c/pqr_pqr" file and then run this target again, it doesn't do the needful:
rm -f a/b/c/pqr_pqr; make a/b/c/pqr_abc -f 1.mk
make: `a/b/c/pqr_abc' is up to date.
As per my understanding, it should run both targets again.
Please somebody help me.
Both your targets are implicit and therefore intermediate. As per documentation:
The first difference is what happens if the intermediate file does not exist. [...] But if b is an intermediate file, then make can leave well enough alone. It won’t bother updating b, or the ultimate target, unless some prerequisite of b is newer than that target or there is some other reason to update that target.
Therefore since pqr_abc already exists and pqr_pqr is an intermediate file and has no prerequisites (so no prerequisites have changed), it will not be regenerated.
As per make's understanding pqr_pqr would need to be created only to generate pqr_abc from it (this is the definition of intermediate file), and since pqr_abc already exists, there is no need to regenerate it.
If you want the file to be regenerated every time, it has to be mentioned explicitly, so that it is no longer considered an intermediate file, i.e.:
$ cat Makefile
a/b/c/%_pqr:
touch $#
a/b/c/%_abc: a/b/c/pqr_pqr
touch $#
a/b/c/pqr_pqr:
The last line defines explicit target, but it does not define recipe, which will still be invoked from implicit rule. Note that it will no longer be considered intermediate and will not be automatically removed, so .PRECIOUS directives are also not needed.
Output:
$ make a/b/c/pqr_abc
touch a/b/c/pqr_pqr
touch a/b/c/pqr_abc
$ make a/b/c/pqr_abc
make: 'a/b/c/pqr_abc' is up to date.
$ rm a/b/c/pqr_pqr
$ make a/b/c/pqr_abc
touch a/b/c/pqr_pqr
touch a/b/c/pqr_abc
I am reading this https://makefiletutorial.com/ makefile tutorial.
With this exmaple:
some_file:
echo "This line will only print once"
touch some_file
This file will make some_file the first time, and the second time notice it’s already made, resulting in make: 'some_file' is up to date.
While with this example:
some_binary: ../headers blah.h
touch some_binary
../headers:
mkdir ../headers
blah.h:
touch ../headers/blah.h
clean:
rm -rf ../headers
rm -f some_binary
Why don't I get 'xxxx' is up to date notice? My output is:
tianhe#tianhe-windy:~/Desktop/learnMake/lala$ make
mkdir ../headers
touch ../headers/blah.h
touch some_binary
tianhe#tianhe-windy:~/Desktop/learnMake/lala$ make
touch ../headers/blah.h
touch some_binary
tianhe#tianhe-windy:~/Desktop/learnMake/lala$
I expect to get xxx is up to date notice the second time I run make, just like the first example.
Very simple: because blah.h is not the same thing as ../headers/blah.h.
So a rule like this:
blah.h:
touch ../headers/blah.h
is wrong: the target tells make "this recipe will create a file named blah.h", but the recipe actually creates a file named ../headers/blah.h.
So the next time make runs, it looks for blah.h again: if it was found then you'd get a message saying that things were up to date. But since it's not found, make will try to make it again because it's clearly not up to date (since it doesn't exist).
I have a Makefile that does performs a task if it hasn't happened in the last hour. It does so like this:
HOUR_FROM_NOW = $(shell perl -e '($$s,$$m,$$h,$$d,$$M)=localtime(time()+3600); printf("%02d%02d%02d%02d\n",$$M+1,$$d,$$h,$$m);')
NOW_FILE = $(shell mkdir -p .make; touch .make/now; echo .make/now )
.PHONY: externals
externals: $(PROJECTS:%=.make/proj_%)
.make/proj_%: $(NOW_FILE)
$(MAKE) -s $(*F)
touch -t $(HOUR_FROM_NOW) $#
.PHONY: $(PROJECTS)
$(PROJECTS):
# do stuff, specifically, clone git-repo if not exists, else pull latest
That part works great, except that I now get warnings:
make: Warning: File `.make/proj' has modification time 3.5e+03 s in the future
make: Nothing to be done for `externals'.
make: warning: Clock skew detected. Your build may be incomplete.
Anyone know how to suppress those warnings? (Or to do a periodic task in a makefile)
Most versions of touch I have come across can do some date time maths which allows for setting the timestamp of a file directly via the --date option.
That and the fact that variables assigned with := are only "evaluated once" makes this a bit easier to read.
HOUR_AGO := .make/hour_ago
__UGLY := $(shell mkdir -p .make && touch --date='1hour ago' $(HOUR_AGO))
# The preceding line will be executed once
.make/proj_%: .make/hour_ago | .make
$(MAKE) -s $(*F)
#touch $#
.make:
mkdir -p $#
I'm using something very similar to this to periodically refresh login tokens.
Never would have thought of it if it wasn't for Dave's answer though.
The directory is created by specifying it as a order-only-prerequisite
I suspect that the + 3600 is at fault. What happens if you remove it?
I thought and thought, and then the stupid-obvious solution hit me ...
Instead of setting timestamps in the future with HOUR_FROM_NOW, I use the real time and compare with HOUR_AGO_FILE ...
HOUR_AGO = $(shell perl -e '($$s,$$m,$$h,$$d,$$M)=localtime(time()-3600); printf("%02d%02d%02d%02d\n",$$M+1,$$d,$$h,$$m);')
HOUR_AGO_FILE = $(shell mkdir -p .make; touch -t $(HOUR_AGO) .make/hour_ago; echo .make/hour_ago )
.PHONY: externals
externals: $(PROJECTS:%=.make/proj_%)
.make/proj_%: $(HOUR_AGO_FILE)
$(MAKE) -s $(*F)
#touch $#
I'm trying to create a Makefile that will download and process file a file to generate targets, this is a simplified version:
default: all
.PHONY: all clean filelist.d
clean:
#rm -fv *.date *.d
#The actual list comes from a FTP file, but let's simplify things a bit
filelist.d:
#echo "Getting updated filelist..."
#echo "LIST=$(shell date +\%M)1.date $(shell date +\%M)2.date" > $#
#echo 'all: $$(LIST)' >> $#
%.date:
touch $#
-include filelist.d
Unfortunately the target all doesn't get updated properly on the first run, it needs to be run again to get the files. This is the output I get from it:
$ make
Getting updated filelist...
make: Nothing to be done for `default'.
$ make
Getting updated filelist...
touch 141.date
touch 142.date
touch 143.date
I'm using GNU Make 3.81 whose documentation states that it reloads the whole thing if the included files get changed. What is going wrong?
You have specified filelist.d as a .PHONY target, so make believes making that target doesn't actually update the specified file. However, it does, and the new contents are used on the next run. For the first run, the missing file isn't an error because include is prefixed with the dash.
Remove filelist.d from .PHONY. However, remember it won't be regenerated again until you delete it (as it doesn't depend on anything).
By the same token, you should include "default" in .PHONY.
I wrote a shell script rather than lump all this in the makefile:
#!/bin/bash
# Check whether file $1 is less than $2 days old.
[ $# -eq 2 ] || {
echo "Usage: $0 FILE DAYS" >&2
exit 2
}
FILE="$1"
DAYS="$2"
[ -f "$FILE" ] || exit 1 # doesn't exist or not a file
TODAY=$(date +%s)
TARGET=$(($TODAY - ($DAYS * 24 * 60 * 60)))
MODIFIED=$(date -r "$FILE" +%s)
(($TARGET < $MODIFIED))
Replace X with the max number of days that can pass before filelist.d is downloaded again:
filelist.d: force-make
./less-than-days $# X || command-to-update
.PHONY: force-make
force-make:
Now filelist.d depends on a .PHONY target, without being a phony itself. This means filelist.d is always out of date (phony targets are always "new"), but its recipe only updates the file periodically.
Unfortunately, this requires you to write the update command as a single command, and space may be a problem if it is long. In that case, I would put it in a separate script as well.