The task at hand is the following: an external tool called in a recipe produces a makefile that should be included by make immediately. Then, another rule, using the data included generates further include files. Make should restart again, and only then process the further rules. Consider the following example:
$(info ------- Restart $(MAKE_RESTARTS))
all :
include a
include b
a : p
touch a
b : a
touch b
Works like this:
touch p
make
------- Restart
touch a
touch b
------- Restart 1
make: Nothing to be done for 'all'.
My problem is that the rule b needs the data included from a, but b is executed BEFORE including the updated version of a.
Make should be restarted before executing b. How can this be achieved? I'd like to see this:
touch p
make
------- Restart
touch a
------- Restart 1
touch b
------- Restart 2
make: Nothing to be done for 'all'.
It's easy to detect whether a was included or not, and the rule for b can be hidden when a is not included. This works for a clean build, but does not when a already exists on the disk from a previous build, and the rule is triggered because p was updated.
Only make knows, whether the rule a:p is up-to-date, it's not possible to check that with conditional expressions.
Is there a solution for this?
Update: based on the advice from #MadScientist, I made it working this way:
$(info ------- Restart $(MAKE_RESTARTS))
all :
include b
include a
a : p
#echo rule A
touch a
$(eval upd=1)
b : a
#echo rule B
$(if $(upd),#echo b skipped,touch b)
And the output:
touch p
make
------- Restart
rule A
touch a
rule B
b skipped
------- Restart 1
rule B
touch b
------- Restart 2
Perfect! Thanks guys, Merry Xmas everybody.
One solution is to invoke a sub-make to build b. That sub-make will include the newer a and so will be correct. I believe something like this will work (untested):
$(info ------- Restart $(MAKE_RESTARTS))
all :
include a
include b
a : p
touch a
ifeq ($(filter real_b,$(MAKECMDGOALS)),)
b : a
$(MAKE) real_b
endif
.PHONY: real_b
real_b:
touch b
Another solution would be to ensure that b is not updated when a is updated. Maybe something like this (again, not tested):
$(info ------- Restart $(MAKE_RESTARTS))
all :
include a
include b
BUILT_A =
a : p
touch a
$(eval BUILT_A = true)
b : a
$(if $(BUILT_A),,touch b)
(a rare legitimate use of eval in a recipe!) In this version if a is built, then b will not be touched. This way after make re-execs itself it will include a and include b, then see a is up to date but see that b is out of date (because we skipped the build step the first time) and rebuild b: this time b will be updated because a was updated in the previous pass, then make will re-exec itself again.
There's more than one way to do it (as there usually is with Make).
I'd do it this way: put the the include b statement in a.
$(info ------- Restart $(MAKE_RESTARTS))
all :
-include a
a : p
touch a
#echo -include b >> a
b : a
touch b
Related
I have a build procedure roughly described by the following Makefile example:
a: b
#echo "Build a, just using b. Don't care about c."
touch a
b: c
#echo "Constructing b from c is cheap..."
touch b
#echo "Once accomplished, I no longer need c."
c:
#echo "Constructing c is very expensive..."
#echo "Work work work..."
touch c
clean:
$(RM) a b c
example: clean
make a
$(RM) c
make a
The point is: I need c to build b, but once I have b, I never again need c. When I do make example, make makes c, b, and a (as expected), deletes c, and then, in the last make a invocation, just remakes c (and does NOT re-make b and a, even though, I'd have thought they were stale now). But, since my goal is a and b hasn't changed, I don't want to remake c. Forget about it! Who cares! a should be considered up-to-date.
Another peculiar thing, is that when I
make a
rm c
make a
(rather than make example), in the second invocation make rebuilds everything (while in make example the second invocation just rebuilds c).
How do I prevent make from building c when its goal is a and all of a's immediate prerequisites exist and is fresher than they are (a isn't stale compared to b), even though the prerequisites of the prerequisites do not?
Edit: I think that what I may want is to treat every file as old (eg. with --old-file) unless that file doesn't exist.
It looks like you want make to treat the file c as an intermediate file, a file that does not have any importance to you other than as an intermediate result when generating another file or other files. This concept is explained in section 10.4 Chains of Implicit Rules of the manual. Since your example does not use any implicit rules, you can manually mark your file c as .INTERMEDIATE.
This makefile shows c as an intermediate file.
a: b
#echo "Build a, just using b. Dont care about c."
touch a
b: c
#echo "Constructing b from c is cheap..."
touch b
#echo "Once accomplished, I no longer need c."
c: d
#echo "Constructing c is very expensive..."
#echo "Work work work..."
touch c
.INTERMEDIATE: c
.PRECIOUS: c
I added a file d, based on your comment, although it is not needed for this example to work.
Before invoking make, the file d has to exist, it is the starting point of the chain. When invoking make, the following happens:
$ touch d
$ make
Constructing c is very expensive...
Work work work...
touch c
Constructing b from c is cheap...
touch b
Once accomplished, I no longer need c.
Build a, just using b. Dont care about c.
touch a
Now deleting c will not have any impact on the build:
$ rm c
$ make
make: `a' is up to date.
Other than that, the update behavior based on dependencies is "the same as usual".
The .PRECIOUS target is optional. It is a built-in that instructs make not to delete the intermediate file named c. You can see for yourself what happens if you remove that line.
b might be built from c, but you don't want to tell Make that b depends on c — if b merely exists, then that's good enough. So you might write b's recipe as
b:
$(MAKE) c
#echo "Constructing b from c is cheap..."
touch b
#echo "Once accomplished, I no longer need c."
or if c is only used in making b, you could just fold the commands for making c into the recipe for b and not expose the existence of c to Make at all.
Maybe there are more elegant ways of expressing this, without invoking sub-makes. And if c has some prerequisites that would cause it to be rebuilt if they were updated, I guess they would need to be listed as reprequisites of b as well.
A collegue just came across this, and I thought I'd ask if there's any neat solution to the following problem: I have a GNU makefile:
a: | b
touch $#
b:c
touch $#
c:
touch $#
Then I run:
~/tmp> make a
touch c
touch b
touch a
~/tmp> make a
make: `a' is up to date.
~/tmp> touch b
~/tmp> make a
make: `a' is up to date.
~/tmp> touch c
~/tmp> make a
touch b
It rebuilds b but not a if I touch c. Is there a way to NOT rebuild b if it is being invoked by an order-only prerequisite? ( in the real-life case b is a file with hundreds of dependencies, which may not be around when make a is invoked. I can't make b's prerequisites be order-only as that would break make b)
The behavior is fully correct. You instructed make to ignore the tight dependency between a and b and depend only on b presence. Whilst a and c share a date dependency.
This is the difference between a: b and a: | b.
If one wants to investigate that case (or other make magic :) ):
Try the following, you'll see that make is keen and tells you what it is doing for b target:
% touch b
% LANG=C make -rd | awk '/^Considering/,/^$/ {print}'
Considering target file `a'.
Considering target file `b'.
Considering target file `c'.
Finished prerequisites of target file `c'.
No need to remake target `c'.
Finished prerequisites of target file `b'.
Prerequisite `c' is older than target `b'.
No need to remake target `b'.
Finished prerequisites of target file `a'.
Prerequisite `b' is order-only for target `a'. <--- "order-only"
No need to remake target `a'.
Now, add a tight dependency:
% echo "a:b" >> Makefile
And compare results:
% touch b
% LANG=C make -rd | awk '/^Considering/,/^$/ {print}'
Considering target file 'a'.
Considering target file 'b'.
Considering target file 'c'.
Finished prerequisites of target file 'c'.
No need to remake target 'c'.
Finished prerequisites of target file 'b'.
Prerequisite 'c' is older than target 'b'.
No need to remake target 'b'.
Pruning file 'b'.
Finished prerequisites of target file 'a'.
Prerequisite 'b' is order-only for target 'a'.
Prerequisite 'b' is newer than target 'a'. <--- additional "date dependency"
Must remake target 'a'.
touch a <--- 'a' is rebuilt
Putting child 0x1b09ec0 (a) PID 5940 on the chain.
Live child 0x1b09ec0 (a) PID 5940
Reaping winning child 0x1b09ec0 PID 5940
Removing child 0x1b09ec0 PID 5940 from chain.
Successfully remade target file 'a'.
I used make -r (no implicit rules) as this example doesn't use them and it's boring to read.
Comments aside, your problem statement is extremely generic, it's very far from describing an actual use case. This makes it very short and easy to read and understand. On the other hand, understanding and solving or working around your particular problem is more uncertain.
I've seen your exact same, generic problem statement apply to this other use case:
nimble_exe: nimble_srcs | big_lib
touch $#
big_lib: biglib_many_sources_and_headers
touch $#
sleep 1 # simulate a long [no-op] build time
biglib_many_sources_and_headers
touch $#
This code means: "build big_lib when it's missing for nimble_exe, but don't care when it's out of date" which superficially matches the definition of an "order-only dependency" and even used to match ninja's behavior until 2011
Unfortunately, if anyone changes anything in the biglib_many_sources_and_headers, make nimble_exe rebuilds big_lib but not nimble_exe! This shows that "order-only dependencies" were simply not designed for that use case. So I recommend this instead:
nimble_exe: nimble_srcs
test -e big_lib || $(MAKE) big_lib
touch $#
I have a problem with order-only prerequisites. These do not execute first at all. Am I mis-understanding the way order-only prerequisites work?
The following make script:
.PHONY: mefirst mefirst2
mefirst:
#echo "I'm first!"
mefirst2:
#echo "I'm first too!"
normaltarget: normaltarget2 | mefirst2
#echo "normaltarget done"
normaltarget2: a b c
#echo "normaltarget2 done"
helloworld: normaltarget | mefirst
#echo "helloworld done"
.DEFAULT_GOAL := go
go: helloworld
#echo "go done"
a:
#echo a
b:
#echo b
c:
#echo c
...prints out the following:
a
b
c
normaltarget2 done
I'm first too!
normaltarget done
I'm first!
helloworld done
go done
...instead of what I would expect:
I'm first!
I'm first too!
a
b
c
normaltarget2 done
normaltarget done
helloworld done
go done
What am I doing wrong?
Am I mis-understanding the way order-only prerequisites work?
Yes, that is what it looks like.
The name "order-only" is somewhat confusing. The prerequisites behind the | are called "order-only prerequisites" not because they change the order of recipe execution inside the list of prerequisites for a single target, but because their only purpose is to have certain targets created before others, like a bootstrap. As accurately explained by user bobbogo below ( -- thanks for correcting): if make decides to rebuild a prerequisite of a target, it will run the recipe for that prerequisite. Now, for an ordinary prerequisite this update implies that the target is now out-of-date, and make will have to run the target's recipe. For an order-only prerequisite on the other hand, make does not mark the target as needing an update.
For example see the section Types of Prerequisites for a use case where a directory should be created before the objects in that directory are created.
Take this example makefile:
a: b
touch a
b: c
touch b
c:
touch c
x: | y
touch x
y: | z
touch y
z:
touch z
As you can see, b and c are normal prerequisites of a and b, whereas y and z are order-only prerequisites of x and y. Starting from a clean slate, they look the same:
:~$ make a
touch c
touch b
touch a
:~$ make x
touch z
touch y
touch x
:~$ make a
make: `a' is up to date.
:~$ make x
make: `x' is up to date.
However, if we now manually "update" the prerequisites at the end of the chain (c and z), we see the difference:
:~$ touch c
:~$ make a
touch b
touch a
:~$ touch z
:~$ make x
make: `x' is up to date.
This shows how order-only prerequisites that exist do not invalidate any targets, independent of their time-stamp. Deleting the order-only target does result in rebuilding though (but only rebuilding of that missing file):
:~$ rm c
:~$ make a
touch c
touch b
touch a
:~$ rm z
:~$ make x
touch z
Having that said, the correct way to change the order in which your recipes are run is by correcting the dependencies between the targets. For example, if you want mefirst to be built before a, then you need to make mefirst a prerequisite for a, as in
a: mefirst
#echo a
It is not possible to give the entire solution to your question since you did not describe in detail in which order you expect recipes to run.
There is a shortcut to your answer which is not the solution but still interesting to know. Although not documented, it seems that the prerequisites of a single target are processed in the order that they appear. The | sign does not change that. In your simple case, you can take advantage of that to achieve the output that you are looking for:
normaltarget: mefirst2 normaltarget2
#echo "normaltarget done"
and
helloworld: mefirst normaltarget
#echo "helloworld done"
However, as pointed out by yourself, this "solution" breaks as soon as the -j flag is used to run recipes in parallel. Also, as pointed out by user bobbogo, relying on this ordering mechanism is bad practice. Introducing new dependencies can interfere with the ordering. So don't do this :-)
.PHONY: b
c: a
#touch c
#echo "Changed"
a: b
b:
#date +%s > a
Running make with the sample make file causes "Changed" to be printed the 1st time it is run; but "Changed" is only printed then on the 3rd, 5th, etc execution. This is because make doesn't seem to recognize that executing the recipe for target "b" updates a.
Changing the rule with "a" as the target into an empty recipe causes "Changed" to be printed for each time make is run (as you would expect - where phony targets are always considered "out of date"). E.g.
a: b ;
Make should skip the implicit rule search for PHONY targets, but "a" is not PHONY. If no implicit rule is found for "a", is make correct to not consider that "a" may have been changed by its PHONY dependency "b"?
Make can't analyze the effects of commands, so it is the user's responsibility to organize the rules correctly.
Consider a slightly different case:
d: c b
c: a
#touch c
#echo "Changed"
a:
b:
#date +%s > a
This has the same behavior as your example; there's no way Make could be expected to know that c "really" depends on b. The author of the makefile is at fault.
Now the way it should be written:
c: a
#touch c
#echo "Changed"
.PHONY: a
a:
#date +%s > a
The a rule modifies the file a (and PHONY is there just to force the a rule to run). This is the way to tell make that the #date ... command modifies a. This makefile works correctly.
Your example is midway between these two. If a rule modifies a file which is the target of another rule, the makefile is badly organized, and Make is not at fault. Yes, Make could assume that a target that depends on a PHONY rule may have been updated when that rule is run, but it could just as well assume that any target may have been updated when any rule is run. And if Make were that paranoid, it wouldn't be very efficient.
I am using GNU make 3.81. Here is a test makefile that demonstrates the problem:
define BOZO
a$(1): b c
touch a$(1)
endef
$(foreach i,1 2 3,$(call BOZO,$(i)))
The idea here is to use a macro template (BOZO) to generate rules that follow a predictable pattern.
Problem: when I run make on this makefile I get an error saying:
Makefile.fake:10: *** multiple target patterns. Stop.
(where line 10 is the line with the foreach).
Now, I know what that error normally indicates. Let's see what that line expands to by using the info function to send the expansion to standard out. I change line 10 to be:
$(info $(foreach i,1 2 3,$(call BOZO,$(i))))
and I run:
$ make -n
a1: b c
touch a1
a2: b c
touch a2
a3: b c
touch a3
make: *** No targets. Stop.
Note that the "no targets" message is expected, since the $(info ...) function evaluates to empty but causes make to print the generated rules.
Let's run those rules then shall we?
$make -n > out.txt
make: *** No targets. Stop.
$make -f out.txt a1 a2 a3
touch a1
touch a2
touch a3
$
AAARGH! The rules work fine. So... is the bug in make, or in my understanding?
One final clue that might help diagnose: if I change the foreach line to:
$(foreach i,1,$(call BOZO,$(i)))
(so that foreach has only one iteration)
and then do
$make a1
I get a different error:
make: *** No rule to make target `a1'. Stop.
I don't know of any way to "see" the expansion of $(foreach ) that make sees except for $(info ), and its output is legal, so I'm quite stumped.
$(foreach i,1 2 3,$(eval $(call BOZO,$(i))))
The eval function tells Make to parse the structures as makefile syntax, to "enact" them. I'm not sure why Make objected to the un-eval'd rules this particular way, but that's kind of academic.
Beta's answer is correct but I wanted to address the comment, "I'm not sure why Make objected to un un-eval'd rules".
The reason the un-eval'd rules don't work is that a makefile is ultimately line-based, and lines are chopped BEFORE variables are expanded. So it's just not possible for an expansion of a variable to turn into a multiline result: even if the expansion contains newlines make treats the entire thing as one "line". When make finishes expanding the foreach loop and parses the results it basically sees this:
a1: b c touch a1 a2: b c touch a2 a3: b c touch b3
which is why you get the "multiple target patterns" error.
The eval causes make to re-interpret the result of the expansion from the beginning as a complete snippet of makefile syntax, including the line-chopping etc., and that's why a multi-line expansion works there.
Maybe old, but always relevant.
There is no need to make this macro at all.
The solution for your problem is by simply defining this rule:
a%: b c
touch a%
The percent acts as a wildcard. See more info here:
https://stackoverflow.com/a/7406471/2522849