Given a makefile:
# Create the following sequence of files, in the following order: 1)'old' then 2)'all', finally 3)'new'.
$(shell touch 'old')
$(shell sleep 1)
$(shell touch 'all')
$(shell sleep 1)
$(shell touch 'new')
all: new
echo '$#'
# Let the modification-time of 'new', to be like 'old' ("older" than 'all').
new : phony
cp -p old new
.PHONY : phony
.INTERMEDIATE: new
Running, we get:
$ make -Wnew
cp -p old new
echo 'all'
all
rm new
Now, although all was older than the file new before parsing the build, things changed after Make finished building the new file, with the recipe cp -p old new, which basically copied the modification-time of old into new, hence: all is now "newer" than the file new.
Then, why does build the up-to-date file all, given, that running through its dependencies - after their respective build - we find that all modification-time is "after" that of its dependency "new".
Explanation of Effect of -Wnew
The -Wnew option tells make to assume that new has been modified and that we must update the targets of which it is a prerequisite. Regardless of what happens to the file new in our makefile make must update all of the targets new is a prerequisite of.
I have provided an explanation and demonstration, using your example, that make does not keep the old modification time but that it is the effect of the -Wnew option which causes make to rebuild all, below.
Process of Execution with -Wnew
So in executing make -Wnew make first reads your makefile and sees
all: new
echo '$#'
so it knows regardless of what happens to new later on it has to update the target all because we've specified the -Wnew option.
make then proceeds to execute your shell commands and creates the files old, all and new. It now traverses through your targets and prerequisites noting that phony does not exist so we must remake it. After making phony we then make new as its target phony is newer than it. This actually modifies new's modification time to be older than all. However, we specified -Wnew so regardless of what happened to new we've told make we want it to rebuild targets that have new as a prerequisite such as all. This is why make then proceeds to build all.
Demonstration That make Does Not Keep Old Modification Times
If we were to execute your example with make, (note no -Wnew option), the output would be
$ make
cp -p old new
Aha! make didn't update all. It realised that by remaking new the modification time of new changed and it is now older than all thus all is up to date and we can finish.
Related
I have the following Makefile for running pdflatex on tex source files:
MAKEFLAGS += --warn-undefined-variables
deps := mydoc.tex mydoc.cls
mydoc.pdf: $(deps)
.PHONY: build
build: $(deps)
pdflatex mydoc.tex mydoc.pdf
.PHONY: build
clean: ## Delete misc
rm -f mydoc.out mydoc.pdf mydoc.aux mydoc.log
When I run make build it always runs pdflatex even though mydoc.tex has not changed.
My understanding is make build should say there is nothing to do if mydoc.tex has not changed. What am I doing wrong?
First, you've declared the target build to be .PHONY. The entire point of a phony target is that it will always be considered out of date and its recipe will be invoked. So, of course the recipe is always run.
Even if you remove the .PHONY though, the recipe will always be run. You say make should do nothing is mydoc.tex has not changed... well, how can make know that mydoc.tex has not changed? Not changed compared to what? Make doesn't have its own database that tells it the last time it ran, and what all the timestamps on the files were at some time in the past. It simply relies on comparing timestamps of files on the filesystem as they exist now.
When you write a rule it tells make, if any of the prerequisites have a newer modification time than the target, then the target is out of date and the recipe should be run to bring it up to date.
So if you write a rule build: mydoc.tex make will look to see if the prerequisite mydoc.tex is newer than the target build. Since there is no file build and one is never created, mydoc.tex will always be considered newer than a non-existent file, and the recipe will always be run.
You need to be sure that the target of the rule is the file that is updated by the recipe. Best practice is to ensure that every recipe you write (that updates a file) updates the file contained in the $# automatic variable:
mydoc.pdf: $(deps)
pdflatex mydoc.tex $#
I have a situation where I need to execute a rule in make depending on a remote file. This is an example of what I'm trying to do (the actual Makefile is a lot more complicated in ways that aren't relevant here):
URL = http://cdn.sstatic.net/stackoverflow/img/favicon.ico
stackoverflow.png: favico.ico modified | check_modified
convert favicon.ico $#
check_modified: modified
#echo Icon was modified. Downloading.
rm -f favicon.ico
wget $(URL)
.PHONY: check_modified
favico.ico: check_modified
modified:
touch -d "`wget --spider -S $(URL) 2>&1 | sed -n 's/.*Modified: //p'`" $#
The idea is:
The rule to build modified should always be run. The function of that rule is to change the modification time of the file to be the same as the modification time of what the URL points to.
After that, I want modified to behave normally in terms of dependencies. If modified is more recent than favicon.ico, I want to retrieve the new file and then let the dependency on it cause the target file to be remade.
As a wrinkle, in some applications, I have to retrieve the file manually. In those cases, I want to just have a rule that fires and tells me to manually download the file, but that doesn't otherwise affect building the target. As an example, if the source image that I'm converting were behind a site login, I'd need to manually login to the site and download it to a fixed location and then rerun make.
Everything I've tried either:
Fails to check the URL if the target is up to date, or
Always checks the URL and rebuilds the target, even if modified is not more recent than the target.
Any words of wisdom?
Your problem is probably that make builds the dependency graph before running any recipe. So, when your modified tag file is created / updated, it is too late for make to consider its last modification date and use it to decide whether other targets shall be built or not.
Your first observed behaviour (fails to check the URL if the target is up to date) probably corresponds to attempts where you did not use the .PHONY special target. As soon as modified has been created, make always considers it as up-to-date because it has no prerequisites and thus cannot be outdated.
The second behaviour (always checks the URL and rebuilds the target, even if modified is not more recent than the target) corresponds to what you posted: check_modified is a prerequisite of the .PHONY special target, which forces make to always consider it as outdated. stackoverflow.png and favico.ico, directly or not, depend on check_modified and are thus also always considered as outdated.
One option to solve your problem would be to use a recursive, two-passes, make invocation. On a first run make would build the all phony target (always), update a modified tag file, and then invoke itself again to build other targets that have the tag file as prerequisite and are outdated. As, for the second invocation, make rebuilds its dependency graph, it will take the last modification date of the tag file into account. Something like:
URL = http://cdn.sstatic.net/stackoverflow/img/favicon.ico
.PHONY: all
all:
touch -d "`wget --spider -S $(URL) 2>&1 | sed -n 's/.*Modified: //p'`" modified
$(MAKE) stackoverflow.ico
stackoverflow.ico: modified
#echo Icon was modified. Downloading.; \
rm -f favicon.ico; \
wget $(URL); \
convert favicon.ico $#
Explanations:
I replaced your conversion to png by a useless conversion to ico because the favicon.ico icon of SO is composite and its conversion to png creates two files instead of one, named stackoverflow-0.png and stackoverflow-1.png, which uselessly complicates things.
all is a true phony target and the default goal so, each time you invoke make (or make all), it is built. It first updates the modified tag file and then invokes make again to build stackoverflow.ico.
The second make invocation does nothing if stackoverflow.ico is more recent than modified, else it downloads and converts.
Your second question (get a message about required manual operations) is completely different and simpler to solve. Let's first define a message and echo it in the file's recipe:
define DIY_message
Dear user, you should first:
- do this
- and that.
Unless you know it is useless, of course.
endef
export DIY_message
the_remote_file:
#echo "$$DIY_message"
The message will be printed if make is invoked with this goal (make the_remote_file) or with a goal that somehow depends on the_remote_file and either:
the_remote_file does not exist,
or the_remote_file exists but is out-of-date with respect to its prerequisites (if you declared prerequisites for it),
or the_remote_file is a prerequisite of .PHONY.
Note: using an intermediate make variable assigned by a define-endef makes things easier for formatted multi-lines messages.
I have a Makefile similar to this:
.PRECIOUS: do/%.build
do/%.install: do/%.build
touch $#
do/%.build:
touch $#
My intention is that all do/.install targets depend on the do/.build target, and that the rule for the build target is run if the stamp file for the build target is missing, and that the install target is run if the build stamp is newer. This works fine during the first run:
$ make do/foo.install
touch do/foo.build
touch do/foo.install
It also works fine if the build stamp is newer:
$ touch do/foo.build
$ make do/foo.install
touch do/foo.install
However, it doesn't work as intended if the install stamp is present and the build stamp is missing:
$ rm do/foo.build
$ make do/foo.install
make: `do/foo.install' is up to date.
The install target is not run. What should I do? Does this have something to do with the fact that I have to add the .PRECIOUS line to avoid the automatic deletion of the build stamp?
Regards,
Tino
From the manual:
"If an ordinary file b does not exist, and make considers a target that depends on b, it invariably creates b and then updates the target from b. But if b is an intermediate file [i.e inferred from a pattern rule], then make can leave well enough alone. It won't bother updating b, or the ultimate target, unless some prerequisite of b is newer than that target or there is some other reason to update that target."
There are a couple of ways to solve this problem, but I know of no really clean way. Do any other targets depend on the build or install files?
(this is similar to GNU make: Execute target but take dependency from file but slightly different).
When I try to force to build a target, make automatically thinks that this target is out of date and forces a run of all targets which depend on it.
In my case, the target is a recursive make call which does a lot of work and might just return with "nothing to be done":
.PHONY: intermediate.dat
intermediate.dat:
$(MAKE) expensive_chain_which_finally_creates_intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
In this case, "make all" runs for 40 seconds although intermediate.dat might not have changed (recursive make returned "nothing to be done"). However, if the recursive make updated intermediate.dat, the target shall be out of date.
Is there really no way to do this?
Make intermediate.dat depend on a phony target instead of being phony itself.
.PHONY : always-remake
intermediate.dat : always-remake
IIRC, the last time I solved the problem, the .PHONY didn't work as intended and I used:
always-remake :
#true
instead. However, I can't recall why, so try the .PHONY first.
The problem with making intermediate.dat itself phony is that make never checks the existence/date of a phony file, which is behaviour that you want. You only need to trigger the rebuild rule, which is done by a prerequisite that is out of date; a phony prerequisite is always out of date, so it does the job.
Another trick is to use an order-only prerequisite, plus a recursive call to make. This can be useful when you want to implement some additional logic around when the intermediate.dat file needs to be created, or when you want to decouple a target from the usual exists && newer than check that make does.
For example:
Let's say that the file contains some date-sensitive material that subsequent tasks depend on, and that it expires after 12 hours. We only want to recreate the file when any of these is true:
The file is older than 12 hours
The file does not exist
To implement the make target & recipes, we'll create a .PHONY check-intermediate.dat target, which runs the time check, conditionally removes the file if it's expired, and then re-runs make for that file.
Note that the check-intermediate.dat target will be defined via the order-only prerequisite syntax (e.g. intermediate.dat: | check-intermediate.dat) which causes it to run always before the target (e.g. intermediate.dat). The time check is inexpensive, and we always want it to run and manage the file for us, but not invalidate the original make check on intermediate.dat if it already exists like a normal prerequisite .PHONY target would.
.PHONY: check-intermediate.dat
check-intermediate.dat:
if [ -e intermediate.dat ]; then find intermediate.dat -mmin +720 -exec bash -c 'rm -f "{}"; $(MAKE) intermediate.dat' \; ; fi
intermediate.dat: | check-intermediate.dat
echo run-expensive-tasks-here-to-create-intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
Now, anything that depends on the intermediate.dat target will first run the check, remove the file if it's expired, and re-run make intermediate.dat before running the rest of the dependent targets. If the intermediate.dat file already exists, it will run the check, and if the check passes, it will continue without running the time expensive recreation task.
The problem with depending on a .PHONY target is that it will always run the tasks which depend on it, because of the way GNU Make handles .PHONY targets. This can cause the make run to always perform the time expensive operations even when it does not need to:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file. As long as a phony target is never a prerequisite of a real target, the phony target recipe will be executed only when the phony target is a specified goal (see Arguments to Specify the Goals).
Here is the context :
I am working on a makefile to create a .h at every build, including another makefile that will use this header. I can't edit the second one.
Using a target all depending on my file, it compiles the first time, creating the missing MyHeader.h. The problem is, when I recompile, the header is not regenerated...
My makefile looks like this :
all: myHeader.h
myHeader.h:
scriptToBuildMyHeader.sh
include obscureAndPrivateMakefile.make
I also tried with a .phony target at the beginning. Same result : once created, it won't be regenerated at every build.
PS : I can't call a script before make.
Do some makfile-Masters have any ideas how to deal with that ?
Thanks!
Because myHeader.h has no dependencies, it will never be rebuilt once it exists. You can work around this by creating a dependency from myHeader.h to a phony target, eg:
forcebuild:
# dummy; do nothing and don't create this file
.PHONY: forcebuild
myHeader.h: forcebuild
scriptToBuildMyHeader.sh
This will however slow down your build considerably, as the header (and any source files including it) will need to be rebuilt every time.
The trouble is that because myHeader.h does not depend on anything, it exists and is therefore up to date on the second build. To make sure it is built each time, it has to depend on a non-existent file:
myHeader.h: .FORCE
scriptToBuildMyHeader.sh
.FORCE:
The name '.FORCE' (or, sometimes, FORCE) is used classically.
If you use GNU Make, you could make the 'non-existent' file into a phony target:
.PHONY: .FORCE
The advantage of this is that (GNU) make does not create the file .FORCE even if you run make -t - which would break the automatic rebuild of the header because that rule depends on there not being a file .FORCE that actually exists.
Here's another possible approach:
all: clean foo.txt
clean:
rm foo.txt
foo.txt:
echo > foo.txt
where I'm using echo > foo.txt to simulate creation of your header.