I have a build procedure roughly described by the following Makefile example:
a: b
#echo "Build a, just using b. Don't care about c."
touch a
b: c
#echo "Constructing b from c is cheap..."
touch b
#echo "Once accomplished, I no longer need c."
c:
#echo "Constructing c is very expensive..."
#echo "Work work work..."
touch c
clean:
$(RM) a b c
example: clean
make a
$(RM) c
make a
The point is: I need c to build b, but once I have b, I never again need c. When I do make example, make makes c, b, and a (as expected), deletes c, and then, in the last make a invocation, just remakes c (and does NOT re-make b and a, even though, I'd have thought they were stale now). But, since my goal is a and b hasn't changed, I don't want to remake c. Forget about it! Who cares! a should be considered up-to-date.
Another peculiar thing, is that when I
make a
rm c
make a
(rather than make example), in the second invocation make rebuilds everything (while in make example the second invocation just rebuilds c).
How do I prevent make from building c when its goal is a and all of a's immediate prerequisites exist and is fresher than they are (a isn't stale compared to b), even though the prerequisites of the prerequisites do not?
Edit: I think that what I may want is to treat every file as old (eg. with --old-file) unless that file doesn't exist.
It looks like you want make to treat the file c as an intermediate file, a file that does not have any importance to you other than as an intermediate result when generating another file or other files. This concept is explained in section 10.4 Chains of Implicit Rules of the manual. Since your example does not use any implicit rules, you can manually mark your file c as .INTERMEDIATE.
This makefile shows c as an intermediate file.
a: b
#echo "Build a, just using b. Dont care about c."
touch a
b: c
#echo "Constructing b from c is cheap..."
touch b
#echo "Once accomplished, I no longer need c."
c: d
#echo "Constructing c is very expensive..."
#echo "Work work work..."
touch c
.INTERMEDIATE: c
.PRECIOUS: c
I added a file d, based on your comment, although it is not needed for this example to work.
Before invoking make, the file d has to exist, it is the starting point of the chain. When invoking make, the following happens:
$ touch d
$ make
Constructing c is very expensive...
Work work work...
touch c
Constructing b from c is cheap...
touch b
Once accomplished, I no longer need c.
Build a, just using b. Dont care about c.
touch a
Now deleting c will not have any impact on the build:
$ rm c
$ make
make: `a' is up to date.
Other than that, the update behavior based on dependencies is "the same as usual".
The .PRECIOUS target is optional. It is a built-in that instructs make not to delete the intermediate file named c. You can see for yourself what happens if you remove that line.
b might be built from c, but you don't want to tell Make that b depends on c — if b merely exists, then that's good enough. So you might write b's recipe as
b:
$(MAKE) c
#echo "Constructing b from c is cheap..."
touch b
#echo "Once accomplished, I no longer need c."
or if c is only used in making b, you could just fold the commands for making c into the recipe for b and not expose the existence of c to Make at all.
Maybe there are more elegant ways of expressing this, without invoking sub-makes. And if c has some prerequisites that would cause it to be rebuilt if they were updated, I guess they would need to be listed as reprequisites of b as well.
Related
The task at hand is the following: an external tool called in a recipe produces a makefile that should be included by make immediately. Then, another rule, using the data included generates further include files. Make should restart again, and only then process the further rules. Consider the following example:
$(info ------- Restart $(MAKE_RESTARTS))
all :
include a
include b
a : p
touch a
b : a
touch b
Works like this:
touch p
make
------- Restart
touch a
touch b
------- Restart 1
make: Nothing to be done for 'all'.
My problem is that the rule b needs the data included from a, but b is executed BEFORE including the updated version of a.
Make should be restarted before executing b. How can this be achieved? I'd like to see this:
touch p
make
------- Restart
touch a
------- Restart 1
touch b
------- Restart 2
make: Nothing to be done for 'all'.
It's easy to detect whether a was included or not, and the rule for b can be hidden when a is not included. This works for a clean build, but does not when a already exists on the disk from a previous build, and the rule is triggered because p was updated.
Only make knows, whether the rule a:p is up-to-date, it's not possible to check that with conditional expressions.
Is there a solution for this?
Update: based on the advice from #MadScientist, I made it working this way:
$(info ------- Restart $(MAKE_RESTARTS))
all :
include b
include a
a : p
#echo rule A
touch a
$(eval upd=1)
b : a
#echo rule B
$(if $(upd),#echo b skipped,touch b)
And the output:
touch p
make
------- Restart
rule A
touch a
rule B
b skipped
------- Restart 1
rule B
touch b
------- Restart 2
Perfect! Thanks guys, Merry Xmas everybody.
One solution is to invoke a sub-make to build b. That sub-make will include the newer a and so will be correct. I believe something like this will work (untested):
$(info ------- Restart $(MAKE_RESTARTS))
all :
include a
include b
a : p
touch a
ifeq ($(filter real_b,$(MAKECMDGOALS)),)
b : a
$(MAKE) real_b
endif
.PHONY: real_b
real_b:
touch b
Another solution would be to ensure that b is not updated when a is updated. Maybe something like this (again, not tested):
$(info ------- Restart $(MAKE_RESTARTS))
all :
include a
include b
BUILT_A =
a : p
touch a
$(eval BUILT_A = true)
b : a
$(if $(BUILT_A),,touch b)
(a rare legitimate use of eval in a recipe!) In this version if a is built, then b will not be touched. This way after make re-execs itself it will include a and include b, then see a is up to date but see that b is out of date (because we skipped the build step the first time) and rebuild b: this time b will be updated because a was updated in the previous pass, then make will re-exec itself again.
There's more than one way to do it (as there usually is with Make).
I'd do it this way: put the the include b statement in a.
$(info ------- Restart $(MAKE_RESTARTS))
all :
-include a
a : p
touch a
#echo -include b >> a
b : a
touch b
I wrote a simple Makefile as below:
.PHONY: all A B
A:
#echo "in target A"
B:
#echo "in target B"
all: A B
the output is
in target A
but if I modified the Makefile as below:
.PHONY: all A B
all: A B
A:
#echo "in target A"
B:
#echo "in target B"
the output become
in target A
in target B
what part of Makefile manual should I read more carefully to understand the difference?
what part of Makefile manual should I read more carefully to understand the difference?
Which part of the manual to study depend on which make manual you are reading, but the explanation for the behavior you observe is simple: when you run make without designating any targets to build, make builds the default target, which is the first target appearing in the makefile whose name does not consist of a period ('.') followed by uppercase letters.
In particular, the target name "all" has no special significance to make. Its use is simply a widespread convention, and to make the all target work as you expect (that is, for it to be the default target), it must be positioned first in the makefile. There are makefiles that have an "all" target that is intentionally positioned elsewhere so that a different one is the default. When it's not the default, you can still build it explicitly, of course, by naming it on the command line: make all.
when running make without any goal, the first goal in out Makefile will be exectued.
You can manage the default goal using .DEFAULT_GOAL variable.
Read https://www.gnu.org/software/make/manual/html_node/Goals.html for more information
I have a problem with order-only prerequisites. These do not execute first at all. Am I mis-understanding the way order-only prerequisites work?
The following make script:
.PHONY: mefirst mefirst2
mefirst:
#echo "I'm first!"
mefirst2:
#echo "I'm first too!"
normaltarget: normaltarget2 | mefirst2
#echo "normaltarget done"
normaltarget2: a b c
#echo "normaltarget2 done"
helloworld: normaltarget | mefirst
#echo "helloworld done"
.DEFAULT_GOAL := go
go: helloworld
#echo "go done"
a:
#echo a
b:
#echo b
c:
#echo c
...prints out the following:
a
b
c
normaltarget2 done
I'm first too!
normaltarget done
I'm first!
helloworld done
go done
...instead of what I would expect:
I'm first!
I'm first too!
a
b
c
normaltarget2 done
normaltarget done
helloworld done
go done
What am I doing wrong?
Am I mis-understanding the way order-only prerequisites work?
Yes, that is what it looks like.
The name "order-only" is somewhat confusing. The prerequisites behind the | are called "order-only prerequisites" not because they change the order of recipe execution inside the list of prerequisites for a single target, but because their only purpose is to have certain targets created before others, like a bootstrap. As accurately explained by user bobbogo below ( -- thanks for correcting): if make decides to rebuild a prerequisite of a target, it will run the recipe for that prerequisite. Now, for an ordinary prerequisite this update implies that the target is now out-of-date, and make will have to run the target's recipe. For an order-only prerequisite on the other hand, make does not mark the target as needing an update.
For example see the section Types of Prerequisites for a use case where a directory should be created before the objects in that directory are created.
Take this example makefile:
a: b
touch a
b: c
touch b
c:
touch c
x: | y
touch x
y: | z
touch y
z:
touch z
As you can see, b and c are normal prerequisites of a and b, whereas y and z are order-only prerequisites of x and y. Starting from a clean slate, they look the same:
:~$ make a
touch c
touch b
touch a
:~$ make x
touch z
touch y
touch x
:~$ make a
make: `a' is up to date.
:~$ make x
make: `x' is up to date.
However, if we now manually "update" the prerequisites at the end of the chain (c and z), we see the difference:
:~$ touch c
:~$ make a
touch b
touch a
:~$ touch z
:~$ make x
make: `x' is up to date.
This shows how order-only prerequisites that exist do not invalidate any targets, independent of their time-stamp. Deleting the order-only target does result in rebuilding though (but only rebuilding of that missing file):
:~$ rm c
:~$ make a
touch c
touch b
touch a
:~$ rm z
:~$ make x
touch z
Having that said, the correct way to change the order in which your recipes are run is by correcting the dependencies between the targets. For example, if you want mefirst to be built before a, then you need to make mefirst a prerequisite for a, as in
a: mefirst
#echo a
It is not possible to give the entire solution to your question since you did not describe in detail in which order you expect recipes to run.
There is a shortcut to your answer which is not the solution but still interesting to know. Although not documented, it seems that the prerequisites of a single target are processed in the order that they appear. The | sign does not change that. In your simple case, you can take advantage of that to achieve the output that you are looking for:
normaltarget: mefirst2 normaltarget2
#echo "normaltarget done"
and
helloworld: mefirst normaltarget
#echo "helloworld done"
However, as pointed out by yourself, this "solution" breaks as soon as the -j flag is used to run recipes in parallel. Also, as pointed out by user bobbogo, relying on this ordering mechanism is bad practice. Introducing new dependencies can interfere with the ordering. So don't do this :-)
.PHONY: b
c: a
#touch c
#echo "Changed"
a: b
b:
#date +%s > a
Running make with the sample make file causes "Changed" to be printed the 1st time it is run; but "Changed" is only printed then on the 3rd, 5th, etc execution. This is because make doesn't seem to recognize that executing the recipe for target "b" updates a.
Changing the rule with "a" as the target into an empty recipe causes "Changed" to be printed for each time make is run (as you would expect - where phony targets are always considered "out of date"). E.g.
a: b ;
Make should skip the implicit rule search for PHONY targets, but "a" is not PHONY. If no implicit rule is found for "a", is make correct to not consider that "a" may have been changed by its PHONY dependency "b"?
Make can't analyze the effects of commands, so it is the user's responsibility to organize the rules correctly.
Consider a slightly different case:
d: c b
c: a
#touch c
#echo "Changed"
a:
b:
#date +%s > a
This has the same behavior as your example; there's no way Make could be expected to know that c "really" depends on b. The author of the makefile is at fault.
Now the way it should be written:
c: a
#touch c
#echo "Changed"
.PHONY: a
a:
#date +%s > a
The a rule modifies the file a (and PHONY is there just to force the a rule to run). This is the way to tell make that the #date ... command modifies a. This makefile works correctly.
Your example is midway between these two. If a rule modifies a file which is the target of another rule, the makefile is badly organized, and Make is not at fault. Yes, Make could assume that a target that depends on a PHONY rule may have been updated when that rule is run, but it could just as well assume that any target may have been updated when any rule is run. And if Make were that paranoid, it wouldn't be very efficient.
I want to depend on a virtual target that only runs once.
Makefile with what I tried so far:
a: b
b: c d
touch b
c:
# time consuming task that only needs to run once
d:
# time consuming task that only needs to run once
Is there a way to stop the dependency chain when b already exists? I'm okay with making a manual clean to get rid of b to trigger a re-run of c and d. I want to be able to run a many times without triggering the long running tasks if b exists.
I have a lot of tasks like c and d, so I want to avoid touching a file per separate task and I don't want the file system to be cluttered with unnecessary files.
You can write your Makefile as:
a: b
b c:
# time consuming task that only needs to run once
touch b
make c will call your task unconditionally.
make b only executes your task when the file b does not exist.
make a depends on b, so the task is only executed when b does not exist,