I have a chain of pattern dependencies in a makefile, and in the end they should come together in one file, e.g.: *.x -> *.y -> onefile.z
So I made the files like this:
$ touch a.x b.x
and the rules:
%.y: %.x some-other-script
touch $#
onefile.z: %.y second-other-script
touch $#
This rule does not work:
$ make onefile.z
make: *** No rule to make target '%.y', needed by 'onefile.z'. Stop.
Using a wildcard:
%.y: %.x some-other-script
touch $#
z: $(wildcard *.y) second-other-script
touch $#
This does not work either: it sees there are no *.y files and proceeds making onefile.z skipping the first rule.
$ make onefile.z
touch onefile.z
$ ls
a.x b.x onefile.z Makefile
I probably could merge two rules into one, but in the real application, there are more steps, and some are making requests over HTTP and take much time, and in fact should not be repeated without a reason.
Is there a way to make such a dependency?
onefile.z: %.y second-other-script is a regular rule, not a pattern rule or a static pattern rule, so the % is interpreted literally. Even if it were a pattern rule, how is make supposed to infer what the stem is supposed to match?
$(wildcard *.y) tells make to find all the files that match *.y, but of course there are none yet so it returns an empty string.
The following should work, if I've understood your question correctly:
xfiles := $(wildcard *.x)
yfiles := $(xfiles:.x=.y)
%.y: %.x some-other-script
touch $#
onefile.z: $(yfiles) second-other-script
touch $#
See the Gnu Make documentation on
Wildcard functions $(wildcard ...)
Substitution functions $(foo:from=to)
Related
In GNU Make, what exactly are the semantics of a pattern rule that depends on a non-pattern file?
I have the following snippet in a Makefile. When foo.a exists, GNU Make doesn't seem to create foo.b when make foo.b is called.
.SUFFIXES:
%.b: move_a_to_b.artifact
move_a_to_b.artifact:
mv foo.a foo.b
touch move_a_to_b.artifact
The following, however, works fine and moves the file from foo.a to foo.b.
.SUFFXIES:
%.b: %.a
mv $< $#
As does this, with a pattern rule depending on a pattern rule
.SUFFIXES:
%.b: %.intermediate
mv $< $#
%.intermediate: %.a
mv $< $#
It doesn't have anything to do with pattern rules depending on non-patterns. That's fine and it has the expected semantics: for any file ending in .b if it's out of date with respect to the file move_a_to_b.artifact then the recipe will be run.
Your issue is that you're not defining a pattern rule, you're deleting a pattern rule. A pattern rule must always have a recipe. A pattern rule without a recipe deletes the pattern rule. See Canceling Pattern Rules.
You have to add a recipe, then it will do something:
%.b : move_a_to_b.artifact
#echo do something to create $#
I have a directory structure that looks like this:
$ tree
.
|-- dir
| `-- subdir
| `-- data
`-- makefile
where data is a file. My makefile looks like this:
all: dir/analysis
.SECONDEXPANSION:
%/analysis: %/subdir
%/analysis: $(addsuffix /data, $$^)
#echo TARGET: $# DEPS: $^
touch $#
When I run make, I would expect the result to look like this:
TARGET: dir/analysis DEPS: dir/subdir/data dir/subdir
touch dir/analysis
Instead, it just reports
make: *** No rule to make target `dir/analysis', needed by `all'. Stop.
If I change the first rule to dir/analysis: dir/subdir then it works as I expected. So I suspect that make ignores the first rule and skips straight to the second when the first rule is %/analysis: %/subdir. It also works as expected when both rules have dir/analysis as their target instead of just the first rule. MadScientists's answer to a different question here seemed to apply to my problem. I tried adding rules like
dummy_target: dir/subdir
mkdir -p dir/subdir
and
dir/subdir:
mkdir -p dir/subdir
to the end of makefile to try to make the dependency on the first rule an explicit target but this didn't change anything. I'm pretty new to make, so I'm probably missing something pretty stupid, but I can't for the life of me figure it out. How do I get the first and second rules to execute in the order they're written? I'm using Make version 3.81 in case that matters.
--EDIT--
If I actually add a command after the first rule, like #echo RULE 1 then that command executes and not the second one.
I think that you understand the makefile:
# Original
all: dir/analysis
.SECONDEXPANSION:
%/analysis: %/subdir
%/analysis: $(addsuffix /data, $$^)
#echo TARGET: $# DEPS: $^
touch $#
like this:
By default we make dir/analysis
We request secondary expansion
We say that whatever/analysis depends on whatever/subdir. So
that makes dir/subdir a prerequisite of dir/analysis
We say also that whatever/analysis depends on the thing we get by
suffixing /data to the prior prerequisites of whatever/analysis. (Secondary
expansion of $$^ gives us the prior prerequisites of the target). And
since the prior prerequisites of dir/analysis are dir/subdir, that now
makes dir/subdir dir/subdir/data the new prerequisites of dir/analysis.
Hence the output of the recipe ought to be:
TARGET: dir/analysis DEPS: dir/subdir/data dir/subdir
touch dir/analysis
This understanding is badly wrong. You're confusing and conflating the operation
of pattern rules with the operation or ordinary rules.
If a makefile says, e.g:
# A
xx: xa
xx: xb
xx: xc
touch $#
or even:
# B
xx: xa
xx: xb
%x: %c
touch $#
then the prerequisites of all the empty rules for the target xx are combined
with those of the (one) non-empty rule when xx is considered as a target. So
# A is equivalent to:
xx: xa xb xc
touch $#
and # B is equivalent to:
xx: xa xb
%x: %c
touch $#
and in both cases the prerequisites of xx are xa xb xc.
However:
%x: %a
%x: %b
touch $#
is not equivalent to:
%x: %a %b
touch $#
If you ever write an empty pattern rule, its effect is simply to delete
any prior pattern rule with the same target and prerequisite patterns. And if you
ever write a non-empty pattern rule, you simply replace
any prior pattern rule with the same target and prerequisite patterns. See
10.5.6 Canceling Implicit Rules
So:
.SECONDEXPANSION:
%/analysis: %/subdir
%/analysis: $(addsuffix /data, $$^)
#echo TARGET: $# DEPS: $^
touch $#
is equivalent to:
.SECONDEXPANSION:
%/analysis: $(addsuffix /data, $$^)
#echo TARGET: $# DEPS: $^
touch $#
Then when this pattern rule is considered with respect to dir/analysis,
there are no prior prerequisites: $^ will expand to the empty string, and
in secondary expansion, so will $$^. So finally the recipe is equivalent to:
# Residual
%/analysis: /data
#echo TARGET: $# DEPS: $^
touch $#
You can satisfy yourself of this by running both of the # Original and
# Residual makefiles in debug mode (-d) and comparing the outputs.
This explains why the outputs in both cases conclude:
make: *** No rule to make target 'dir/analysis', needed by 'all'. Stop.
A pattern rule will only be instantiated to make a target if doing so offers a way for the target to be made. Why else select it? Since the prerequisite /data does not exist, the pattern rule
is not viable and is discarded. Just as if you tried to make foo.o with the makefile:
%.o: %c
touch $#
when there is no foo.c in the directory.
If you really want to see the output you were expecting, then you can get it from:
.PHONY: all clean
all: dir/analysis
%/analysis: %/subdir %/subdir/data
#echo TARGET: $# DEPS: $^
touch $#
clean:
rm -f dir/analysis
But it is hard to believe that is what you really want to see. This makefile
makes the analysis dependent on the data and also on the directory where
the data resides. What can be the point of that?
%/analysis: %/subdir/data
presumably expresses the desired relationship between the data and the analysis:-
If the data doesn't exist then the analysis is out of date and cannot be made (till you get some data).
If the data exists and is older than the analysis then the analysis does not
need to be made.
If the data exists and is newer than the analysis then the analysis can and will be
made.
By making dir/subdir an independent prerequisite all that you achieve is to introduce
a requirement to remake the analysis if it becomes older than the directory in
which the data resides - regardless of the data. So if, e.g. I run touch dir/subdir,
that will require make to re-run the analysis, even though the data hasn't changed.
I can make sense of your motivation only by supposing that in
reality you also want make to make the data, and the directory where it resides, if
they happen not exist when an analysis is required. If that is the situation then
you want a makefile on the lines of:
.PHONY: all clean
all: dir/analysis
.SECONDARY:
%/subdir:
mkdir -p $#
%/subdir/data: | %/subdir
touch $#
%/analysis: %/subdir/data
#echo TARGET: $# DEPS: $^
touch $#
clean:
rm -f dir/analysis dir/subdir/data
Here, the pattern rule:
%/subdir/data: | %/subdir
makes %/subdir an order-only prerequisite
of %/subdir/data. It means that /dir/subdir is not considered when determining whether /dir/subdir/data needs
to be made, but if it is determined that /dir/subdir/data is to be made, then /dir/subdir will be made first. This
way, /dir/subdir will be created, if it does not exist, when you need it to put the data there, but will have no influence
on whether you need to make the data or the analysis.
The special target .SECONDARY is a technicality. It directs make not to automatically delete any subsequent targets
that it deduces to be intermediate artefacts that emerge from chained pattern rules. Without it, in this example make
would deduce that dir/subdir/data and dir/subdir/ are just waste-products of making dir/analysis and auto-delete them
at the end.
If you run this makefile initially with no dir/subdir/data and no dir/subdir/ you get:
$ make
mkdir -p dir/subdir
touch dir/subdir/data
TARGET: dir/analysis DEPS: dir/subdir/data
touch dir/analysis
And subsequently:
$ make clean
rm -f dir/analysis dir/subdir/data
$ make
touch dir/subdir/data
TARGET: dir/analysis DEPS: dir/subdir/data
touch dir/analysis
And then:
$ make
make: Nothing to be done for 'all'.
$ touch dir/subdir
$ make
make: Nothing to be done for 'all'.
Consider this makefile:
%: %.c
#echo making $#
#touch $#
.SECONDEXPANSION:
%.pid: $$(basename $$#)
$(<D)/$(<F) --pidfile=$<.pid
Here, the first rule builds a program and second rule starts it, producing a pid-file. Note: I know that secondary expansion is unnecessary in this example; the real makefile is more complex and I really need secondary expansion there.
So, typing make foo.pid, I expect Make to build foo from foo.c by the first rule and then run ./foo --pidfile=foo.pid by the second one.
However, it does not seem to work:
$ make -f Makefile.test foo.pid
make: *** No rule to make target 'foo.pid'. Stop.
This somehow relates to secondary-expanded dependencies which are provided by pattern rules. If I write either
%.pid: % in the second rule (i. e. get rid of secondary-expansion), or
foo: %: %.c in the first rule (i. e. write an explicit static pattern rule),
it all suddenly works. Why? Is this a limitation of GNU Make? In my case, I'd like to avoid pre-listing all possible programs in the first rule.
I think your situation is described in section 10.5.5 Match-Anything Pattern Rules. Whay you have here is a non-terminal match-anything rule, which is not allowed to match a file name that indicates a specific type of data.
If you had let's say
%:: %.c
#echo making $#
#touch $#
it would work.
I want to make a language depend target. In Particular: I have one source-file and I want to create different Objects which where add to the corresponding language folder. That single source file will differ in the C-Flags, the compiler will get. As long as I used it in a static way, it works quite fine.
de/info.o en/info.o es/info.o : info.c
$(ECHO) (DEP) $< for $#
Now I thought, it would be great if it is a bit more dynamic, in case i'll add a new language depending file. So I used a wildcard as followed:
de/%.o en/%.o es/%.o : %.c
$(ECHO) (DEP) $< for $#
But now it just make the first target and ignores the rest. The Make-Debug prints the following thing:
Successfully remade target file `de/info.o'.
Considering target file `en/info.o'.
File `en/info.o' was considered already.
Just in case: No, the objects do not exist. So there is no target, but an existing dependencie, so make should execute the rules.
EDIT: Found a solution for that Problem.
define FOO
$(1)/%.o : %.c
$(ECHO) $$< for $(1)
endef
$(foreach lang,$(LANGUAGE_LIST), $(eval $(call FOO,$(lang))))
Inspired by: http://www.gnu.org/software/make/manual/make.html#Eval-Function
Pattern rules work differently than implicit rules. While an implicit rule such as
a b c: d
command
is equivalent to the longer notation
a: d
command
b: d
command
c: d
command
this does NOT hold for pattern rules. Pattern rules with multiple targets are explicitly required to build all of their targets in a single invocation of command. Thus you would have to write
$ cat GNUmakefile
all: de/x.o en/x.o es/x.o
de/%.o: %.c
#echo $# from $<
en/%.o: %.c
#echo $# from $<
es/%.o: %.c
#echo $# from $<
$ gmake
de/x.o from x.c
en/x.o from x.c
es/x.o from x.c
The relevant documentation is found in 10.5.1 Introduction to Pattern Rules of the GNU make manual:
Pattern rules may have more than one target. Unlike normal rules, this does not act as many different rules with the same prerequisites and recipe. If a pattern rule has multiple targets, make knows that the rule’s recipe is responsible for making all of the targets. The recipe is executed only once to make all the targets. When searching for a pattern rule to match a target, the target patterns of a rule other than the one that matches the target in need of a rule are incidental: make worries only about giving a recipe and prerequisites to the file presently in question. However, when this file’s recipe is run, the other targets are marked as having been updated themselves.
I have the following makefile I use to generate files from some templates, the generated files have two possible extensions:
%.tex: %.tex*_tpl
./generate $#_tpl -o $#
%.xml: %.xml*_tpl
./generate $#_tpl -o $#
The dependency list will here match things like a.tex_tpl, a.tex-subpart1_tpl, a.tex-subpart2_tpl.
While this works, is there a way to avoid repetition? For example by matching *.{tex,xml} in the rule name and use the whole matched name in the dependency list? Something that would look like that:
%.{tex,xml}: $#_tpl
./generate $< -o $#
(Though I know %.{tex,xml} is not a valid rule name and you can't use $# in the dependency list)
Or any other (cleaner?) way.
Seems to me this does what you are looking for:
#
# I've assumed that files of the form:
#
# a.xml_tpl
# b.tex_tpl
#
# determine what targets you want to build
#
TARGETS:=$(patsubst %_tpl,%,$(wildcard *.xml_tpl *.tex_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $^ -o $#
The key is to use .SECONDEXPANSION to allow $$(wildcard %*_tpl) to be evaluated in a second expansion phase. The double $ is not a typo, by the way; it protects the expression from being evaluated at the time of the first expansion.
If I populate a directory with these files:
a.tex-subpart1_tpl
a.tex_tpl
a.xml-subpart1_tpl
a.xml-subpart2_tpl
a.xml_tpl
and run make -n, I get this on the console:
./generate a.xml_tpl a.xml-subpart1_tpl a.xml-subpart2_tpl -o a.xml
./generate a.tex_tpl a.tex-subpart1_tpl -o a.tex
Why the Second Expansion?
Without the second expansion, you'd have to have $(wildcard %*_tpl) in the dependency because with the $$ the wildcard function would never execute. Instead, make would treat $$(wildcard..) literally as the dependency, which is obviously wrong.
Ok, so $(wildcard %*_tpl) would be evaluated at the time make first runs across that line (this is the "first expansion"). At that time % has no value yet so wildcard would roughly be doing something like what would be ls %*_tpl at the command line.
For reasons of speed, make does not by default give you the opportunity to do any evaluation later than during the first expansion. If you want a later opportunity you have to specify .SECONDEXPANSION, which turns on the second expansion processing. Make still performs the firts expansion as usual. This is why you need to have $$(wildcard: it is transformed to $(wildcard during the first expansion. At the time of the second expansion make sees $(wildcard %*_tpl), replaces % with the actual stem and then executes the wildcard function with the actual stem rather than with a literal %.
Why $(TARGETS) in the Pattern Rule?
The pattern rule could be written:
%: $$(wildcard %*_tpl)
./generate $^ -o $#
without $(TARGETS). However, this rule would do nothing, as it would be a "match-anything rule". Basically, if make took such a rule at face value, then the computation cost would be significant, and most likely it is not the case that the author of the Makefile really means to apply this rule to any file whatsoever. So such a rule comes with restrictions, which in the Makefile here make it useless.
Adding $(TARGETS) makes it into a static pattern rule, which is not a match-anything rule. The addition of $(TARGETS) in front of the target pattern tells make that the rule applies only to these targets, and nothing else.