Why GNU Make's secondary expansion does not work with pattern rules for dependencies? - makefile

Consider this makefile:
%: %.c
#echo making $#
#touch $#
.SECONDEXPANSION:
%.pid: $$(basename $$#)
$(<D)/$(<F) --pidfile=$<.pid
Here, the first rule builds a program and second rule starts it, producing a pid-file. Note: I know that secondary expansion is unnecessary in this example; the real makefile is more complex and I really need secondary expansion there.
So, typing make foo.pid, I expect Make to build foo from foo.c by the first rule and then run ./foo --pidfile=foo.pid by the second one.
However, it does not seem to work:
$ make -f Makefile.test foo.pid
make: *** No rule to make target 'foo.pid'. Stop.
This somehow relates to secondary-expanded dependencies which are provided by pattern rules. If I write either
%.pid: % in the second rule (i. e. get rid of secondary-expansion), or
foo: %: %.c in the first rule (i. e. write an explicit static pattern rule),
it all suddenly works. Why? Is this a limitation of GNU Make? In my case, I'd like to avoid pre-listing all possible programs in the first rule.

I think your situation is described in section 10.5.5 Match-Anything Pattern Rules. Whay you have here is a non-terminal match-anything rule, which is not allowed to match a file name that indicates a specific type of data.
If you had let's say
%:: %.c
#echo making $#
#touch $#
it would work.

Related

How to set a Makefile target depend on pattern prerequisites?

I have a chain of pattern dependencies in a makefile, and in the end they should come together in one file, e.g.: *.x -> *.y -> onefile.z
So I made the files like this:
$ touch a.x b.x
and the rules:
%.y: %.x some-other-script
touch $#
onefile.z: %.y second-other-script
touch $#
This rule does not work:
$ make onefile.z
make: *** No rule to make target '%.y', needed by 'onefile.z'. Stop.
Using a wildcard:
%.y: %.x some-other-script
touch $#
z: $(wildcard *.y) second-other-script
touch $#
This does not work either: it sees there are no *.y files and proceeds making onefile.z skipping the first rule.
$ make onefile.z
touch onefile.z
$ ls
a.x b.x onefile.z Makefile
I probably could merge two rules into one, but in the real application, there are more steps, and some are making requests over HTTP and take much time, and in fact should not be repeated without a reason.
Is there a way to make such a dependency?
onefile.z: %.y second-other-script is a regular rule, not a pattern rule or a static pattern rule, so the % is interpreted literally. Even if it were a pattern rule, how is make supposed to infer what the stem is supposed to match?
$(wildcard *.y) tells make to find all the files that match *.y, but of course there are none yet so it returns an empty string.
The following should work, if I've understood your question correctly:
xfiles := $(wildcard *.x)
yfiles := $(xfiles:.x=.y)
%.y: %.x some-other-script
touch $#
onefile.z: $(yfiles) second-other-script
touch $#
See the Gnu Make documentation on
Wildcard functions $(wildcard ...)
Substitution functions $(foo:from=to)

GNU Make pattern rule depending on non-pattern file

In GNU Make, what exactly are the semantics of a pattern rule that depends on a non-pattern file?
I have the following snippet in a Makefile. When foo.a exists, GNU Make doesn't seem to create foo.b when make foo.b is called.
.SUFFIXES:
%.b: move_a_to_b.artifact
move_a_to_b.artifact:
mv foo.a foo.b
touch move_a_to_b.artifact
The following, however, works fine and moves the file from foo.a to foo.b.
.SUFFXIES:
%.b: %.a
mv $< $#
As does this, with a pattern rule depending on a pattern rule
.SUFFIXES:
%.b: %.intermediate
mv $< $#
%.intermediate: %.a
mv $< $#
It doesn't have anything to do with pattern rules depending on non-patterns. That's fine and it has the expected semantics: for any file ending in .b if it's out of date with respect to the file move_a_to_b.artifact then the recipe will be run.
Your issue is that you're not defining a pattern rule, you're deleting a pattern rule. A pattern rule must always have a recipe. A pattern rule without a recipe deletes the pattern rule. See Canceling Pattern Rules.
You have to add a recipe, then it will do something:
%.b : move_a_to_b.artifact
#echo do something to create $#

GNU Makefile - Pattern rule with multiple targets with one dependency ignores all targets but the first

I want to make a language depend target. In Particular: I have one source-file and I want to create different Objects which where add to the corresponding language folder. That single source file will differ in the C-Flags, the compiler will get. As long as I used it in a static way, it works quite fine.
de/info.o en/info.o es/info.o : info.c
$(ECHO) (DEP) $< for $#
Now I thought, it would be great if it is a bit more dynamic, in case i'll add a new language depending file. So I used a wildcard as followed:
de/%.o en/%.o es/%.o : %.c
$(ECHO) (DEP) $< for $#
But now it just make the first target and ignores the rest. The Make-Debug prints the following thing:
Successfully remade target file `de/info.o'.
Considering target file `en/info.o'.
File `en/info.o' was considered already.
Just in case: No, the objects do not exist. So there is no target, but an existing dependencie, so make should execute the rules.
EDIT: Found a solution for that Problem.
define FOO
$(1)/%.o : %.c
$(ECHO) $$< for $(1)
endef
$(foreach lang,$(LANGUAGE_LIST), $(eval $(call FOO,$(lang))))
Inspired by: http://www.gnu.org/software/make/manual/make.html#Eval-Function
Pattern rules work differently than implicit rules. While an implicit rule such as
a b c: d
command
is equivalent to the longer notation
a: d
command
b: d
command
c: d
command
this does NOT hold for pattern rules. Pattern rules with multiple targets are explicitly required to build all of their targets in a single invocation of command. Thus you would have to write
$ cat GNUmakefile
all: de/x.o en/x.o es/x.o
de/%.o: %.c
#echo $# from $<
en/%.o: %.c
#echo $# from $<
es/%.o: %.c
#echo $# from $<
$ gmake
de/x.o from x.c
en/x.o from x.c
es/x.o from x.c
The relevant documentation is found in 10.5.1 Introduction to Pattern Rules of the GNU make manual:
Pattern rules may have more than one target. Unlike normal rules, this does not act as many different rules with the same prerequisites and recipe. If a pattern rule has multiple targets, make knows that the rule’s recipe is responsible for making all of the targets. The recipe is executed only once to make all the targets. When searching for a pattern rule to match a target, the target patterns of a rule other than the one that matches the target in need of a rule are incidental: make worries only about giving a recipe and prerequisites to the file presently in question. However, when this file’s recipe is run, the other targets are marked as having been updated themselves.

Complex pattern rule in Makefile

I have the following makefile I use to generate files from some templates, the generated files have two possible extensions:
%.tex: %.tex*_tpl
./generate $#_tpl -o $#
%.xml: %.xml*_tpl
./generate $#_tpl -o $#
The dependency list will here match things like a.tex_tpl, a.tex-subpart1_tpl, a.tex-subpart2_tpl.
While this works, is there a way to avoid repetition? For example by matching *.{tex,xml} in the rule name and use the whole matched name in the dependency list? Something that would look like that:
%.{tex,xml}: $#_tpl
./generate $< -o $#
(Though I know %.{tex,xml} is not a valid rule name and you can't use $# in the dependency list)
Or any other (cleaner?) way.
Seems to me this does what you are looking for:
#
# I've assumed that files of the form:
#
# a.xml_tpl
# b.tex_tpl
#
# determine what targets you want to build
#
TARGETS:=$(patsubst %_tpl,%,$(wildcard *.xml_tpl *.tex_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $^ -o $#
The key is to use .SECONDEXPANSION to allow $$(wildcard %*_tpl) to be evaluated in a second expansion phase. The double $ is not a typo, by the way; it protects the expression from being evaluated at the time of the first expansion.
If I populate a directory with these files:
a.tex-subpart1_tpl
a.tex_tpl
a.xml-subpart1_tpl
a.xml-subpart2_tpl
a.xml_tpl
and run make -n, I get this on the console:
./generate a.xml_tpl a.xml-subpart1_tpl a.xml-subpart2_tpl -o a.xml
./generate a.tex_tpl a.tex-subpart1_tpl -o a.tex
Why the Second Expansion?
Without the second expansion, you'd have to have $(wildcard %*_tpl) in the dependency because with the $$ the wildcard function would never execute. Instead, make would treat $$(wildcard..) literally as the dependency, which is obviously wrong.
Ok, so $(wildcard %*_tpl) would be evaluated at the time make first runs across that line (this is the "first expansion"). At that time % has no value yet so wildcard would roughly be doing something like what would be ls %*_tpl at the command line.
For reasons of speed, make does not by default give you the opportunity to do any evaluation later than during the first expansion. If you want a later opportunity you have to specify .SECONDEXPANSION, which turns on the second expansion processing. Make still performs the firts expansion as usual. This is why you need to have $$(wildcard: it is transformed to $(wildcard during the first expansion. At the time of the second expansion make sees $(wildcard %*_tpl), replaces % with the actual stem and then executes the wildcard function with the actual stem rather than with a literal %.
Why $(TARGETS) in the Pattern Rule?
The pattern rule could be written:
%: $$(wildcard %*_tpl)
./generate $^ -o $#
without $(TARGETS). However, this rule would do nothing, as it would be a "match-anything rule". Basically, if make took such a rule at face value, then the computation cost would be significant, and most likely it is not the case that the author of the Makefile really means to apply this rule to any file whatsoever. So such a rule comes with restrictions, which in the Makefile here make it useless.
Adding $(TARGETS) makes it into a static pattern rule, which is not a match-anything rule. The addition of $(TARGETS) in front of the target pattern tells make that the rule applies only to these targets, and nothing else.

Force make not to re-order pre-requisites in automatic variable expansion

Here a is my makefile (GNU make) to compile a small OCaml program:
SUFFIXES:=
OCAML=ocamlopt
LD=ocamlopt
OFLAGS=
.PHONY: all clean
all: playground
playground.cmx: playground.ml lstream.cmi
playground: lstream.cmx playground.cmx
%.cmi: %.ml
$(OCAML) $(OFLAGS) -c $<
%.cmx: %.ml
$(OCAML) $(OFLAGS) -c $<
%: %.cmx
$(LD) -o $# $^
playground uses functions from the Lstream module. In this case, the ocaml linker requires the files to link to be specified in order of dependency (eg: ocamlopt -o playground lstream.cmx playground.cmx).
Despite the fact that I defined the playground rule's dependencies in the right order,
make consistently re-orders them and executes ocamlopt -o playground playground.cmx lstream.cmx which causes a linker error.
Is there a way to enforce the correct behaviour ? I would like to avoid specifying the link command explicitely and let make infer it from the dependencies.
Implicit rules always force the pattern matching prerequisite to be first, regardless of the order in which they're defined elsewhere. This is almost always what you want, because in most rules the pattern matching prerequisite is special.
For example when compiling an object file the prerequisites consist of one source file and a bunch of header files; the source file is special and needs to be listed on the command line. Make ensures that for a pattern rule %.o : %.c (for example) the prerequisite matching %.c is first in the list, and so is assigned to the $< automatic variable, and it can be treated differently.
In any event the short answer is no, you cannot modify this behavior. An implicit rule % : %.cmx matching a target playground will always force the prerequisite playground.cmx to be listed first in the prerequisite list. The other prerequisites will maintain their order.
If you really need the prerequisites to maintain their order then I recommend using a static pattern rule:
TARGETS = playground
$(TARGETS) : % :
$(LD) -o $# $^
(you can also use a "match anything" pattern rule but this can be a real performance degrader). Here since you have no pattern in the prerequisite list, nothing will be reordered.

Resources