I have a makefile with many independent targets, each of which depends on one intermediate target. Here is an example simplified makefile:
.PHONY: %.world %.hello
all: 0.world 1.world 2.world 3.world 4.world 5.world 6.world
%.world: %.hello
#echo $#
%.hello:
#echo $#
.INTERMEDIATE: %.hello
When I run make serially, it processes in the following order:
0.hello
0.world
1.hello
1.world
2.hello
2.world
3.hello
3.world
4.hello
4.world
5.hello
5.world
6.hello
6.world
But when I run with -j2, it runs all the intermediate files first, then the final ones:
0.hello
1.hello
2.hello
3.hello
4.hello
5.hello
6.hello
0.world
1.world
2.world
3.world
4.world
5.world
6.world
Both orderings are correct, but when processing many 100's of targets, I'd rather have the final files (*.world) sooner rather than waiting for all the intermediate files. Is there any way to tweak the execution ordering, such that it would execute more like this:
0.hello
1.hello
0.world
1.world
2.hello
3.hello
2.world
3.world
4.hello
5.hello
4.world
5.world
6.hello
6.world
I know I could artificially enforce any generic ordering with dependencies, but I need a more general solution since I'm working with pattern rules and 100's of auto-generated targets.
No, there's no way you can change this behavior. Make walks the graph in one direction and it doesn't start over until it's done. Even if it sees that some prerequisite has finished building it will keep going through the graph until it's done, before restarting at the beginning and building things which are now ready to be built.
Related
I have a Makefile which, for a particular target, calls another Makefile. Suppose that the main Makefile contains
some_dir/some_target:
cd some_dir && make some_target
and that some_dir/Makefile contains
some_target: file1 file2
do_stuff
Here's my conundrum: What should the dependencies be for the target in the main Makefile? If I put no dependencies, then, according to the GNU make manual, some_dir/Makefile will only be invoked if some_dir/some_target doesn't exist. I could instead copy the dependencies from some_dir/Makefile. However, that creates the danger of later changing the dependencies in the subdirectory's Makefile and forgetting to update the main Makefile.
Is there a way to tell the main Makefile, "I don't know if the target is out of date or not. Go ask the other Makefile"?
Is there a way to tell the main Makefile, "I don't know if the target is out of date or not. Go ask the other Makefile"?
There is no provision specifically for delegating to another makefile, but you can achieve a similar result by ensuring that the recipe for the target in question is always run. There are mechanisms and conventions for that.
The old-school approach is to declare a dependency on a target that does not exist and is never actually built. You may see such a target being named FORCE, though that's only a convention. The name is not directly significant. Example:
some_dir/some_target: FORCE
cd some_dir && make some_target
# Dummy target
FORCE:
As long is there is not, in fact, a file named FORCE in the working directory, the FORCE target will be considered initially out of date, so every target that directly or indirectly depends on it will be built (see also below).
There is, of course, a weakness in that: what if a file named FORCE actually is created? However unlikely that may be, it screws up the whole scheme if it happens. Some make implementations, notably GNU's, have an implementation-specific way to address that. GNU's approach is to recognize a special, built-in target named .PHONY (do not overlook the leading .). All prerequisites of .PHONY are considered out of date on every build, notwithstanding anything on the filesystem. Inasmuch as make implementations that do not recognize that convention are unlikely to be troubled by its use, there is little drawback to putting it in play:
.PHONY: FORCE
You could also skip FORCE and just directly declare some_dir/some_target itself to be phony, as another answer suggests, but there are at least two problems with that:
it's not really phony in the usual sense. You expect that target to be built. Declaring it phony is therefore confusing.
if you happen to try to use that approach with a make that does not recognize .PHONY, then the whole scheme falls apart. If you instead use an intermediate phony target (such as FORCE, above) then your makefile still works even with such makes, except in the unlikely event that a file named the same as the dummy target is created.
But note well that however implemented, any such scheme has a significant drawback: if you force some_dir/some_target be considered out of date on every build, so that the sub-make will be run unconditionally, then every other target that depends directly or indirectly on some_dir/some_target will also be rebuilt every time. On the other hand, if you do not force it to be rebuilt, then it might not be rebuilt when it ought to be, as you already recognize. This is the topic of the well-known paper Recursive Make Considered Harmful. As an alternative, then, you should consider not using recursive make.
One option consists in forcing a sub-make:
dummy := $(shell $(MAKE) -C some_dir some_target)
top_target: some_dir/some_target
...
"I don't know if the target is out of date or not" - you can use .PHONY for this:
.PHONY: some_dir/some_target
some_dir/some_target:
cd some_dir && make some_target
https://www.gnu.org/software/make/manual/html_node/Special-Targets.html#Special-Targets:
The prerequisites of the special target .PHONY are considered to be phony targets. When it is time to consider such a target, make will run its recipe unconditionally, regardless of whether a file with that name exists or what its last-modification time is.
Just out of curiosity, what is the order of executing targets in a makefile with
${OBJ_DIR}/%.o: ${SRC_DIR}/%.cpp
I noticed it is not lexicographic (like ls -l).
Is it just random?
They are built in the order in which make walks the prerequisite graph.
In the simple case where you don't have parallel jobs (no -j option), then if you have a target like:
prog: foo.o bar.o. baz.o
make will first try to build foo.o, then bar.o, then baz.o, then finally prog.
If you do enable parallel jobs, then make will still try to start builds in the same order but because some builds finish faster than others, you may get different targets building at the same time.
I haven't found an answer so far, so I think this is not a repeat question.
I have some Makefile along the lines of:
include prerequisite_2
all: prerequisite_1 prerequisite_2
clean:
rm *.mod
prerequisite_1:
mkdir somedir
prerequisite_2:
re-write existing file
The issue is that I want the prerequisite_2 to rebuild whenever the default goal is called (all) or when prerequisite_2 is called from the command line, and I know I can use touch prerequisite_2, FORCE or .PHONY to achieve this. However, I DO NOT want it to run every time (the written file contains dependency information for the Fortran files involved) as it doesn't make sense to also rebuild this when calling: make clean
Is it possible to emulate the effects of FORCE or .PHONY only when the depending targets are called?
You can see what the goal targets are by looking at the MAKECMDGOALS variable.
So you can do something like:
ifeq (,$(if $(MAKECMDGOALS),$(filter-out all prerequisite-2,$(MAKECMDGOALS))))
include prerequisite-2
endif
The if condition will be true if MAKECMDGOALS is the empty string, or if it contains only all and/or prerequisite-2 but not if it contains any other target.
Usually, this is not what you want though. Usually you want to disable the include only if certain targets (clean is the classic example) are used.
This exact situation is even discussed in the GNU make manual.
When a project depends on some other project which has its own makefile, recursive make is used like this:
LIBDIR := path/to/lib
LIBNAME := library.a
LIBPATH := $(LIBDIR)/$(LIBNAME)
$(LIBPATH):
$(MAKE) -C $(LIBDIR) $(LIBNAME)
However the obvious problem with this is that make is unable to determine the dependencies of the $(LIBPATH) because it's defined in the recursive makefile in $(LIBDIR).
What I'm currently doing is using a .PHONY target to force the check if the sub-project needs rebuilding:
$(LIBPATH): always_build
$(MAKE) -C $(LIBDIR) $(LIBNAME)
.PHONY: always_build
While this allow me to trigger rebuild when it needed it still needs to walk through a lot of directories and invoke make a lot of times, just to find out nothing needs to be done.
Is there a way to get the dependencies out of the sub-makefile so I can add them as a dependencies of the $(LIBPATH) so the sub-makefile is only invoked when it really need to be invoked?
If you mean, in an automated way then no. Even if there were it wouldn't make any sense.
In order to get those prerequisites you'd have to invoke make to compute them. Once that sub-make had computed them it would inform the parent make and the parent make would check the prerequisites then if any were out of date it would invoke the sub-make again which would re-compute the prerequisites to actually build the target.
Far from being MORE efficient, you'd actually be doing about three times as much work!
In a recursive make scenario, your current method of delegating the out-of-date computation to a sub-make is the best you can do.
What you're really asking is to use a non-recursive make environment where a single instance of make knows all the prerequisites and determines what is out of date. Note, however, that this does not really solve the problem of "not reading lots of makefiles".
Ultimately you can't know that your project is completely up to date, without checking that it's completely up to date... which means checking all the dependency relationships.
I know what a .PHONY does.
If in the folder where my Makefile is, I add an empty file called clean and after I run make clean all of the clean target will not be executed since there was not any change in the file, so the target will not run and this is correct.
If I add .PHONY: clean, than the clean is seen as a command and this is also correct.
My question is why this behavior does not happen the same to all target, since I added a all file in the folder.So basically the all target still executes like if it was a .PHONY: all
I have the fallowing makefile code.
all: test1 test2
test1: test1.o
test1.o: test1.c
test2: test2.o
test2.o: test2.c
clean:
rm -rf *.o test1 test2
How do you know that the all rule is "still executing"? That rule has no recipe, so there's no way it can be "executed".
If what you mean is that even though the all file exists in the local directory, make is still building the targets test1 and test2, that's how make works (this doesn't have anything to do with phony vs. non-phony targets). When make decides whether or not build a particular target first it tries to build all the prerequisites of that target, and all the prerequisites of those targets, etc. Only after all that is complete, can make know whether or not to build the first target (all in this case).
make clean here doesn't have any dependencies, so putting a file named clean there is enough for the target to be considered built.
make all on the other hand has dependencies. Even if you put a file named all there, Make has to check whether the all file is newer than test1 and test2. This process triggers builds of test1 and test2, and it happens to have the same effect as if all was a phony target.
The basis is that all: test1 test2 is a recipe for building a file named all, that depends on the files test1 and test2.
If you ran make all, Make would do something like this:
Analyse the Makefile.
Find out that all depends on test1 and test2.
Check the timestamp of all and see if it is "up to date".
It is "up to date" if none of the dependencies are newer than itself.
In other words, Make can skip building a file if it's newer than all it's dependencies.
Build outdated or missing files.
Now, if you would like to prevent Make from considering the targets as files, you could specify them as phony targets. That is best practice for non-file targets like all.
(This answer isn't disagreeing with either of the existing answers, but suggesting another way of thinking about this).
When you have a rule like
dst: src
action
you're saying two things (as you know):
if dst doesn't exist, or is older than src, then do action; and
when action completes, the file dst will exist.
With targets such as all or clean, the second statement is of course not true. Make doesn't hold you to the promise in (2), so when you say make all, it'll compute and generate the required dependencies, and not complain that there's no file all in place afterwards. You're lying to Make, but it doesn't mind (it's cool with that...). That is, this is basically a makefile hack.
Where this goes wrong, of course, is if for some reason there happens to be a file called all or clean. Then Make will take the modification date of the file all into account when calculating the dependencies, and possibly come to a conclusion you didn't expect.
So what .PHONY: all does is legitimise the hack, and tells Make ‘even if a file all exists, pretend that it doesn't’; you're basically cancelling promise (2).
Therefore, as the other answers state, mentioning .PHONY isn't necessary. It simply forestalls an error – easy to make but easy to miss – when a file matching a phony target is accidentally created.