In a GNU Makefile, if I use * in a prerequisite or $(wildcard) in an assignment it will use shell expansion to find matching files.
Is there a way to do the equivalent for phony targets? For example:
.PHONY: compile-1 compile-2
compile-all: compile-*
This is similar to this question but I don't want to have to manually list the targets as in AVAILABLE_MODELS.
"No, you have to list them" is an acceptable answer.
You don't have to quite list them. You can compute them from their differences,
if that's sufficiently different:-
compiles := $(patsubst %,compile-%,1 2)
PHONY: $(compiles)
compile-all: $(compiles)
See 8.2 Functions for String Substitution and Analysis
However, if compile-N is a phony target for some compilation you ought not to
have a phony target for that at all. Compilation makes real files, and they're
the targets, no need for phony ones.
On the other hand, compile-all clearly should to be a phony target.
Related
I have a rule for the target
data/processed/21.12.2021/experiment6/written_piv21122021.005.exp6.mp4
in my makefile such that it has a dependency
data/raw/21.12.2021/experiment6/piv21122021.005.exp6.mov
Using subst functions, I'm trying to create the dependency by pattern matching as
%/written_*.mp4: \
$(subst processed,raw, $$*)/*.mov \
<do something>
However, the above rule can't find the *.mov dependency. i have tried many versions of $(subst processed,raw, $$*)/*.mov to match the dependency but didn't work.
How to do this? What is the correct syntax?
First, you can't do this:
%/written_*.mp4:
You can't combine a pattern % with a wildcard *. You have to realize that make works in two very discrete steps: first, all the makefiles are parsed and an internal representation of all the targets and prerequisites are constructed into a graph. Then, make walks that graph figuring out what needs to be built and how to build it, and running recipes.
Make variables, functions, and wildcards in targets and prerequisites are expanded when makefiles are parsed (in the first step). Automatic variables like $* are not set until a recipe is invoked (the second step), and patterns like % are not matched/expanded until make tries to decide how to build something (again in the second step).
So, a rule like:
%/written_*.mp4: $(subst processed,raw, $$*)/*.mov
can't work because the wildcard will expand to files matching the literal filename %/written_*.mp4 which clearly has no matches since you won't have a directory named %. In any event you can't use wildcards in targets because when make is parsing the makefile those targets won't exist (since that's what you want make to build) so the wildcards won't match anything. Also, $$* is the literal string $* and there is no processed string in that so the subst function will do nothing. And, even if /*.mov did match something it would put ALL the files matching that wildcard as prerequisites of every target so they'd all get rebuilt whenever any one changed.
And finally, you definitely should not use a backslash after your prerequisites: this just turns your recipe into prerequisites.
Your problem is very difficult to solve because your target and prerequisite differ in multiple distinct places and make doesn't support multiple % matching. You can get most of the way there with this:
data/processed/%.mp4: data/raw/%.mov
<do something>
However this is not quite right because the % in the target is written_... while in the prerequisite it's just ... and this is not possible to represent in make.
If you can rework the filenames so that instead of written_piv21122021.005.exp6.mp4 you can use piv21122021.005.exp6.written.mp4 (or even better you don't need the written_ prefix at all) then you can easily do this. If not you'll need to get very fancy to make this work.
Here is an imperfect kludge.
Delegate the work to another makefile I'll call adjunct.mk. In the main makefile:
data/processed/%.mp4:
#$(MAKE) -f adjunct.mk $#
And in adjunct.mk this ugly transformation:
.SECONDEXPANSION:
$(MAKECMDGOALS): $$(patsubst data/processed/%,data/raw/%,$$(subst written_,,$$(patsubst %.mp4,%.mov,$$#)))
...do whatever with $< and $#...
This incurs the usual cost of recursive Make: it blinds the higher-level Make to the dependency relations being tracked by the lower one. So if your Make must build that mov file, or you try to build the mp4 when the mov does not exist, then this solution will require some more careful pipefitting before it will work correctly.
The Makefile style guide mentions that "No file targets should be prerequisites of .PHONY."
I assume you could reword that as "Don't declare file targets PHONY"?
To express dependencies, for the human reader as well as for Make, I guess you sometimes need file targets be prerequisites of PHONY targets.
Should those be order-only targets, then?
As far as I understand, in this case the only difference between order-only and not is if they appear in $^ or in $|, so it probably depends on what behavior you want in that matter, respectively which kind of dependency you want to express.
Am I correct here? (If there is no clear case for one or the other,) are there any reasons to choose one over the other (order-only or not)?
I guess you sometimes need file targets be prerequisites of PHONY targets
Most of the time, in fact. Consider that "standard" stuff:
.PHONY: all
all: myprog other_stuff
myprog: $(OBJECTS)
...
But "Don't declare file targets PHONY / No file targets should be prerequisites of .PHONY" is a totally different thing. It means .PHONY: myprog is bad and should be avoided. The reason is that (1) it unconditionally triggers rebuild of myprog which is abnormal; (2) it could fool a human into thinking that myprog is "not a file".
If you need to force rebuild of myprog once you do make -B myprog. If it really needs to be rebuilt every time make runs, then you can do:
myprog: $(OBJECTS) FORCE
...
FORCE:;
Or something like that.
As far as I understand, in this case the only difference between order-only and not is if they appear in $^ or in $|
The order-only prerequisites are rarely used GNU extension. There is no need to put them in just to show that "phony stuff" on the left side will be rebuilt anyway. So no one ever writes all: | myprog, although it would work as good as all: myprog.
Using GNU make, I am trying to solve a problem similar to make recipe execute twice — that is, to have a Makefile recipe run twice. In my case, however, the recipe is run under the .SECONDEXPANSION target, and the two different runs will be called with different parameters to generate different versions of the output file from the same input file. That is, with input file foo, this example Makefile should be callable via make foo.pdf or make foo.expanded.pdf to build one .pdf file, or make all to build both .pdf files:
.PHONY: all
all: foo.pdf foo.expanded.pdf
.SECONDEXPANSION:
%.expanded.pdf %.pdf: %
#echo building $(basename $#)
Of the two solutions given in that answer, the first is unsuitable because it always runs the rule twice; I want it run twice when the user asks for it.
The second solution posted there is conceptually what I am looking for and have implemented in the above example Makefile, with only the small problem that it doesn't work: although the all target lists both .pdf files as dependencies, only one is built when make all is run.
Is there a way to tell GNU make to build two different files using the same rule under a .SECONDEXPANSION?
EDIT: Clarified in problem description that the same input file is used to build both versions of the output file, and modified sample Makefile to include this dependency.
EDIT: I would like a solution as scalable as possible; that is, it should work if the input filename contains dots, specifying additional output file foo.reduced.pdf should require only adjusting the targets and recipe as appropriate, etc. This limits performing string surgery that relies on the filenames appearing exactly as given in this narrow example (e.g., changing the rule to %.pdf: $$(firstword $$(subst ., ,$$*)) fails if the input file could be either foo or foo.bar).
You are probably looking for Pattern-specific Variable Values. Let's assume your recipe depends on a make variable named BUILDFLAGS that takes value normal by default and special for the "expanded" targets. Then this Makefile:
BUILDER := builder
BUILDFLAGS := normal
.PHONY: all
all: foo.pdf foo.expanded.pdf
%.expanded.pdf: BUILDFLAGS := special
%.pdf:
$(BUILDER) $(BUILDFLAGS) $#
should do about what you want with the same rule for all targets, plus one pattern-specific variable value declaration. Replace builder, normal and special with what makes sense in your case. Demo:
$ make foo.pdf
builder normal foo.pdf
$ make foo.expanded.pdf
builder special foo.expanded.pdf
$ make
builder normal foo.pdf
builder special foo.expanded.pdf
Your problem has nothing to do with .SECONDEXPANSION. You can just drop that and the problem will be the same.
Your problem is that you are using a pattern rule with multiple target patterns, and expecting that it works similar to an explicit rule with multiple targets. But it does not (and in fact you cannot have a rule with both pattern and explicit targets).
For a pattern rule with multiple target patterns, Make matches the same pattern to all the %, including multiple times in the targets, and then assumes that it just has to execute the recipe with that pattern once, and it will make all the matched targets.
In your case the best way is to use multiple rules (I changed your recipe because using echo as a Make recipe is a bad idea):
.PHONY: all
all: foo.expanded.pdf foo.pdf
RECIPE = touch $#
%.expanded.pdf:
$(RECIPE)
%.pdf:
$(RECIPE)
I'm trying to build several executables in one make instance, as suggested by
Recursive Make Considered Harmful.
The arguments of that paper apply to my project because some of my source files are generated, and go into multiple executables.
The object files for each of these executables go into separate directories, because they are compiled with different preprocessor flags.
How can I specify separate vpath settings for each of these executables, so that source files with duplicate filenames in separate directories go into the executables where I want them to go?
Setting vpath before the rules to build the object files for one executable, and erasing it afterwards (by not giving any directories) doesn't have the desired effect. Apparently, the last setting is used for all the rules.
Solutions I see currently:
Rename source filenames to use unique names so that I can use a global vpath setting
Instead of vpath, use separate rules for each source directory, with the source directory in the prerequisite pattern (*)
Recursive make, with separate vpath settings in each make instance, and somehow deal with the resulting trouble.
Use something different than make.
Is there a better solution, or which one of the above would you prefer?
(*) The solution with separate rules looks like this:
build/$(PROGRAM)/%.o: %.c
$(COMPILE_RECIPE)
build/$(PROGRAM)/%.o: $($(PROGRAM)_SOURCE_DIR)/%.c
$(COMPILE_RECIPE)
Ok for my current project, but would become ugly fast, if there where more than one directory different in the source paths of the executables
Edit: Test for suggestion of #Etan, which shows that $^ is empty, when the prerequisites appear only in pattern rules (y/y.c) - this only works if each dependency is given directly, as for x/x.c.
.PHONY: all
all: build/x.o build/y.o
build/x.o: x/x.c
# the following rule is ignored:
build/%.o: y/%.c
# because there is a matching pattern rule with a recipe:
build/%.o:
#echo \"$#\" from \"$^\"
#touch $#
The files x/x.c, y/y.c and the directory build exist.
output:
"build/x.o" from "x/x.c"
"build/y.o" from ""
Tested with GNU Make 3.82.90
First, it's never correct to use VPATH or vpath to find derived files (files that are built by make)... that includes "source files that are generated" (if they are generated by make, and not before make is invoked somehow). VPATH/vpath can only be used to find source files (files that are not built by make). See How not to use VPATH for more information.
Second, vpath is a global setting, not a per-target setting. Whenever make wants to find a prerequisite and that prerequisite doesn't exist "normally", make will use the vpath settings to find it. There's no facility in vpath to say "for this target use these vpath settings, for that target use those". In an ideal world, you would be able to set VPATH as a target-specific variable but this doesn't work.
There are various ways to generate prerequisites based on a target: you might find some of the suggestions on metaprogramming in make helpful.
Lastly, the makefile example you give regarding $^ is not correct:
# the following rule is ignored:
build/%.o: y/%.c
# because there is a matching pattern rule with a recipe:
build/%.o:
#echo \"$#\" from \"$^\"
#touch $#
You cannot specify a pattern rule without any recipe and have it "add" prerequisite patterns to some other existing pattern rule. It works for explicit rules, to add more prerequisites to a target, but specifying a pattern rule with no recipe simply deletes the pattern rule.
So, $^ is empty for build/y.o because there are no prerequisites defined in the pattern rule and you've not defined any explicit prerequisites for build/y.o like you did for build/x.o.
ETA: The reference to metaprogramming was to suggest that you can use it to make your second option, which is the one I would use, be less typing and easier maintenance. Pick the method you prefer and auto-generate the various pattern rules. This might be termed a "fifth option".
Other than that any of the four you suggest are fine, and there isn't another option you're missing that I'm aware of.
I have a Makefile, where none of the targets reference files, so if this can be done in some loop it would be
convenient.
Is there some way to set every target in a makefile as .PHONY ?
The reason I'm asking this is I never want any files in the same directory as the makefile to conflict with the targets (where the makefile will automatically associate any target with a directory of file).
You can ensure targets always build by defining one empty "FORCE" target and have all you're other targets depend on it
target1: FORCE
./foobar $#
target2: FORCE
./dongle $#
FORCE:
(as suggested in the make manual)
You can also run "make -B" instead of "make"?
`-B'
`--always-make'
Consider all targets out-of-date. GNU `make' proceeds to consider
targets and their prerequisites using the normal algorithms;
however, all these targets are remade, regardless of the status of
their prerequisites.