A collegue just came across this, and I thought I'd ask if there's any neat solution to the following problem: I have a GNU makefile:
a: | b
touch $#
b:c
touch $#
c:
touch $#
Then I run:
~/tmp> make a
touch c
touch b
touch a
~/tmp> make a
make: `a' is up to date.
~/tmp> touch b
~/tmp> make a
make: `a' is up to date.
~/tmp> touch c
~/tmp> make a
touch b
It rebuilds b but not a if I touch c. Is there a way to NOT rebuild b if it is being invoked by an order-only prerequisite? ( in the real-life case b is a file with hundreds of dependencies, which may not be around when make a is invoked. I can't make b's prerequisites be order-only as that would break make b)
The behavior is fully correct. You instructed make to ignore the tight dependency between a and b and depend only on b presence. Whilst a and c share a date dependency.
This is the difference between a: b and a: | b.
If one wants to investigate that case (or other make magic :) ):
Try the following, you'll see that make is keen and tells you what it is doing for b target:
% touch b
% LANG=C make -rd | awk '/^Considering/,/^$/ {print}'
Considering target file `a'.
Considering target file `b'.
Considering target file `c'.
Finished prerequisites of target file `c'.
No need to remake target `c'.
Finished prerequisites of target file `b'.
Prerequisite `c' is older than target `b'.
No need to remake target `b'.
Finished prerequisites of target file `a'.
Prerequisite `b' is order-only for target `a'. <--- "order-only"
No need to remake target `a'.
Now, add a tight dependency:
% echo "a:b" >> Makefile
And compare results:
% touch b
% LANG=C make -rd | awk '/^Considering/,/^$/ {print}'
Considering target file 'a'.
Considering target file 'b'.
Considering target file 'c'.
Finished prerequisites of target file 'c'.
No need to remake target 'c'.
Finished prerequisites of target file 'b'.
Prerequisite 'c' is older than target 'b'.
No need to remake target 'b'.
Pruning file 'b'.
Finished prerequisites of target file 'a'.
Prerequisite 'b' is order-only for target 'a'.
Prerequisite 'b' is newer than target 'a'. <--- additional "date dependency"
Must remake target 'a'.
touch a <--- 'a' is rebuilt
Putting child 0x1b09ec0 (a) PID 5940 on the chain.
Live child 0x1b09ec0 (a) PID 5940
Reaping winning child 0x1b09ec0 PID 5940
Removing child 0x1b09ec0 PID 5940 from chain.
Successfully remade target file 'a'.
I used make -r (no implicit rules) as this example doesn't use them and it's boring to read.
Comments aside, your problem statement is extremely generic, it's very far from describing an actual use case. This makes it very short and easy to read and understand. On the other hand, understanding and solving or working around your particular problem is more uncertain.
I've seen your exact same, generic problem statement apply to this other use case:
nimble_exe: nimble_srcs | big_lib
touch $#
big_lib: biglib_many_sources_and_headers
touch $#
sleep 1 # simulate a long [no-op] build time
biglib_many_sources_and_headers
touch $#
This code means: "build big_lib when it's missing for nimble_exe, but don't care when it's out of date" which superficially matches the definition of an "order-only dependency" and even used to match ninja's behavior until 2011
Unfortunately, if anyone changes anything in the biglib_many_sources_and_headers, make nimble_exe rebuilds big_lib but not nimble_exe! This shows that "order-only dependencies" were simply not designed for that use case. So I recommend this instead:
nimble_exe: nimble_srcs
test -e big_lib || $(MAKE) big_lib
touch $#
Related
(Yes, I know this question gets asked a lot. I think this case is different.)
Makefile:
dependency := ./dependency
target_file := ./target
.PHONY: target
target: $(target_file)
$(dependency):
touch $#
$(target_file): $(dependency)
touch $#
Running make --trace once yields this output:
Makefile:8: target 'dependency' does not exist
touch dependency
make: Circular target <- target dependency dropped.
Makefile:11: update target 'target' due to: dependency
touch target
That's pretty much as expected, although the circular target bit is odd.
Subsequent invocations of the make --trace yield this output:
make: Circular target <- target dependency dropped.
Makefile:11: update target 'target' due to: dependency
touch target
There's still that circular target, and it recreates the target file even though it was already up-to-date. It also claims the dependency file was out of date, but it doesn't recreate that.
What is causing this behavior? One possible clue: if you replace ./target with ./foo it behaves as you'd expect.
Traditionally for make, and generally speaking for GNU make, target and prerequisite names are flat strings, in that make does not know about directories or paths, and it does not attempt to canonicalize target names as paths or any such thing.
Except, GNU make does provide special handling in the particular case of leading sequences of ./ in target and prerequisite names. Per the manual, such sequences are stripped, with the effect that ./file and file are equivalent as target and / or prerequisite names to this make implementation. That's apparently a GNU invention, and it is the source of the odd behavior you observe.
Safest, especially if you want portability to other makes, is to avoid target and prerequisite names starting with ./.
I've been trying to get a makefile, a, to include another makefile, b, if the target specified is not found in file a. I'm using this snippet to try and achieve this, but from echos I've put into the file I can see that makefile b is being accessed even when the target is found in a and run.
The snippet I'm using from the link above is:
foo:
frobnicate > foo
%: force
#echo "No target found locally, running default makefile"
#$(MAKE) -f Makefile $#
force: ;
Specifically I'm getting "Nothing to be done" outputs when makefile b is being used, and makefile a is behaving as expected. This is shown below:
$ make all # all target appears in both make files
No target found locally, running default makefile
make[1]: Entering directory `/home/user/currdir' # (b)
make[1]: Nothing to be done for `Makefile'.
make[1]: Leaving directory `/home/user/currdir'
Local all # (a)
Is there a better way to be doing this?
addition: After adding another echo to the % rule, I've found that $# is "Makefile", when it should be the target trying to be built.
I don't really understand your question based on the example you gave; there is no "a" or "b" in that example, just one Makefile.
However, the behavior you're seeing is due to GNU make's re-making makefiles capability. When you create match-anything pattern rules as you've done, you have to consider that every single target or prerequisite that make wants to build will match that rule. That's a large burden.
You can avoid having remade makefiles match by creating explicit rules for them, such as:
Makefile: ;
I have a problem with order-only prerequisites. These do not execute first at all. Am I mis-understanding the way order-only prerequisites work?
The following make script:
.PHONY: mefirst mefirst2
mefirst:
#echo "I'm first!"
mefirst2:
#echo "I'm first too!"
normaltarget: normaltarget2 | mefirst2
#echo "normaltarget done"
normaltarget2: a b c
#echo "normaltarget2 done"
helloworld: normaltarget | mefirst
#echo "helloworld done"
.DEFAULT_GOAL := go
go: helloworld
#echo "go done"
a:
#echo a
b:
#echo b
c:
#echo c
...prints out the following:
a
b
c
normaltarget2 done
I'm first too!
normaltarget done
I'm first!
helloworld done
go done
...instead of what I would expect:
I'm first!
I'm first too!
a
b
c
normaltarget2 done
normaltarget done
helloworld done
go done
What am I doing wrong?
Am I mis-understanding the way order-only prerequisites work?
Yes, that is what it looks like.
The name "order-only" is somewhat confusing. The prerequisites behind the | are called "order-only prerequisites" not because they change the order of recipe execution inside the list of prerequisites for a single target, but because their only purpose is to have certain targets created before others, like a bootstrap. As accurately explained by user bobbogo below ( -- thanks for correcting): if make decides to rebuild a prerequisite of a target, it will run the recipe for that prerequisite. Now, for an ordinary prerequisite this update implies that the target is now out-of-date, and make will have to run the target's recipe. For an order-only prerequisite on the other hand, make does not mark the target as needing an update.
For example see the section Types of Prerequisites for a use case where a directory should be created before the objects in that directory are created.
Take this example makefile:
a: b
touch a
b: c
touch b
c:
touch c
x: | y
touch x
y: | z
touch y
z:
touch z
As you can see, b and c are normal prerequisites of a and b, whereas y and z are order-only prerequisites of x and y. Starting from a clean slate, they look the same:
:~$ make a
touch c
touch b
touch a
:~$ make x
touch z
touch y
touch x
:~$ make a
make: `a' is up to date.
:~$ make x
make: `x' is up to date.
However, if we now manually "update" the prerequisites at the end of the chain (c and z), we see the difference:
:~$ touch c
:~$ make a
touch b
touch a
:~$ touch z
:~$ make x
make: `x' is up to date.
This shows how order-only prerequisites that exist do not invalidate any targets, independent of their time-stamp. Deleting the order-only target does result in rebuilding though (but only rebuilding of that missing file):
:~$ rm c
:~$ make a
touch c
touch b
touch a
:~$ rm z
:~$ make x
touch z
Having that said, the correct way to change the order in which your recipes are run is by correcting the dependencies between the targets. For example, if you want mefirst to be built before a, then you need to make mefirst a prerequisite for a, as in
a: mefirst
#echo a
It is not possible to give the entire solution to your question since you did not describe in detail in which order you expect recipes to run.
There is a shortcut to your answer which is not the solution but still interesting to know. Although not documented, it seems that the prerequisites of a single target are processed in the order that they appear. The | sign does not change that. In your simple case, you can take advantage of that to achieve the output that you are looking for:
normaltarget: mefirst2 normaltarget2
#echo "normaltarget done"
and
helloworld: mefirst normaltarget
#echo "helloworld done"
However, as pointed out by yourself, this "solution" breaks as soon as the -j flag is used to run recipes in parallel. Also, as pointed out by user bobbogo, relying on this ordering mechanism is bad practice. Introducing new dependencies can interfere with the ordering. So don't do this :-)
I have a Makefile similar to this:
.PRECIOUS: do/%.build
do/%.install: do/%.build
touch $#
do/%.build:
touch $#
My intention is that all do/.install targets depend on the do/.build target, and that the rule for the build target is run if the stamp file for the build target is missing, and that the install target is run if the build stamp is newer. This works fine during the first run:
$ make do/foo.install
touch do/foo.build
touch do/foo.install
It also works fine if the build stamp is newer:
$ touch do/foo.build
$ make do/foo.install
touch do/foo.install
However, it doesn't work as intended if the install stamp is present and the build stamp is missing:
$ rm do/foo.build
$ make do/foo.install
make: `do/foo.install' is up to date.
The install target is not run. What should I do? Does this have something to do with the fact that I have to add the .PRECIOUS line to avoid the automatic deletion of the build stamp?
Regards,
Tino
From the manual:
"If an ordinary file b does not exist, and make considers a target that depends on b, it invariably creates b and then updates the target from b. But if b is an intermediate file [i.e inferred from a pattern rule], then make can leave well enough alone. It won't bother updating b, or the ultimate target, unless some prerequisite of b is newer than that target or there is some other reason to update that target."
There are a couple of ways to solve this problem, but I know of no really clean way. Do any other targets depend on the build or install files?
.PHONY: b
c: a
#touch c
#echo "Changed"
a: b
b:
#date +%s > a
Running make with the sample make file causes "Changed" to be printed the 1st time it is run; but "Changed" is only printed then on the 3rd, 5th, etc execution. This is because make doesn't seem to recognize that executing the recipe for target "b" updates a.
Changing the rule with "a" as the target into an empty recipe causes "Changed" to be printed for each time make is run (as you would expect - where phony targets are always considered "out of date"). E.g.
a: b ;
Make should skip the implicit rule search for PHONY targets, but "a" is not PHONY. If no implicit rule is found for "a", is make correct to not consider that "a" may have been changed by its PHONY dependency "b"?
Make can't analyze the effects of commands, so it is the user's responsibility to organize the rules correctly.
Consider a slightly different case:
d: c b
c: a
#touch c
#echo "Changed"
a:
b:
#date +%s > a
This has the same behavior as your example; there's no way Make could be expected to know that c "really" depends on b. The author of the makefile is at fault.
Now the way it should be written:
c: a
#touch c
#echo "Changed"
.PHONY: a
a:
#date +%s > a
The a rule modifies the file a (and PHONY is there just to force the a rule to run). This is the way to tell make that the #date ... command modifies a. This makefile works correctly.
Your example is midway between these two. If a rule modifies a file which is the target of another rule, the makefile is badly organized, and Make is not at fault. Yes, Make could assume that a target that depends on a PHONY rule may have been updated when that rule is run, but it could just as well assume that any target may have been updated when any rule is run. And if Make were that paranoid, it wouldn't be very efficient.