I have two wildcard rules like so:
%important_results/output.pdf: input2.o
SETTING=$* myimportantcommand > $#
%results/output.pdf: input1.o
SETTING=$* mycommand > $#
with simple prerequisites:
input1.o: input1.c
touch input1.o
input2.o: input2.c
touch input2.o
However, I am getting varying behaviour when I run make important_results:
if input2.o exists, it uses the %important_results rule
if input2.o does not exists, but input1.o exists, it uses the %results rule instead of building input2.o. However, this runs the wrong command.
How can I give make a rule preference, so it always goes with %important_results rule even if it has to build prerequisites?
Searching for "gnu make rule order" or preference did not yield anything.
I suppose the simplest solution is to use non-overlapping wildcards.
edit: https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html suggests that stems cannot be applied to directories in this way.
GNU make manual:
The target is considered a pattern for matching file names; the ‘%’
can match any nonempty substring, while other characters match only
themselves.
Try the following, maybe:
%mportant_results/output.pdf: input2.o
SETTING=$(patsubst %i,%,$*) myimportantcommand > $#
Demo:
$ ls
Makefile input1.c input1.o input2.c
$ make important_results/output.pdf
touch input2.o
SETTING= myimportantcommand > important_results/output.pdf
Related
Using make for generic purposes (not compiling)
Suppose I have a set of files with full path names, and I would like to do something with that file.
something --file a/b/c/X > $< # (for example)
And I have made a rule:
something-%:
something --file $*
Which matches fine against, say, "something-foo" but does not catch "something-a/b/c/foo". Is there a way to write a wildcard rule for this latter case?
The manual describes how patterns match:
When the target pattern does not contain a slash (and it usually does not), directory names in the file names are removed from the file name before it is compared with the target prefix and suffix.
In your case when calling as make something-a/b/c/foo, something-a/b/c/ is treated as directory and removed, so the rest does not match your rule. This can be easily checked:
$ cat Makefile
something-%:
echo something --file $<
f%o:
echo $*
Output:
$ make something-OtherDirectory/src/foo -dr
GNU Make 4.2.1
...
Considering target file 'something-OtherDirectory/src/foo'.
File 'something-OtherDirectory/src/foo' does not exist.
Looking for an implicit rule for 'something-OtherDirectory/src/foo'.
Trying pattern rule with stem 'o'.
Found an implicit rule for 'something-OtherDirectory/src/foo'.
Finished prerequisites of target file 'something-OtherDirectory/src/foo'.
Must remake target 'something-OtherDirectory/src/foo'.
echo something-OtherDirectory/src/o
...
Note that it matched the other pattern rule with the stem of o.
You can make it work your way if your pattern does include a slash. For sake of completeness I would also define a prerequisite if your rule is based on a file and declare target as phony if it does not generate a real output file:
$ cat Makefile
.PHONY: something/%
something/%: %
echo something --file $<
Output:
$ make something/OtherDirectory/src/foo.c
echo something --file OtherDirectory/src/foo.c
something --file OtherDirectory/src/foo.c
Suppose I have a Makefile like this
B1.txt: A1.txt
python big_long_program.py A1.txt > $#
correct1.txt: B1.txt reference.txt
diff -q B1.txt reference.txt
touch $#
Then the output when I make correct1.txt is pretty well what I would expect:
python big_long_program.py A1.txt > B1.txt
diff -q B1.txt reference.txt
touch correct1.txt
Now if I have lots of files, B1.txt, B2.txt, B3.txt etc, so create an implicit rule:
B%.txt: A%.txt
python big_long_program.py A$*.txt > $#
correct%.txt: B%.txt reference.txt
diff -q B$*.txt reference.txt
touch $#
Instead this happens when I make correct1.txt:
python big_long_program.py A1.txt > B1.txt
diff -q B1.txt reference.txt
touch correct1.txt
rm B1.txt
i.e. there difference is that now the file B1.txt has been deleted, which in many cases is really bad.
So why are implicit rules different? Or am I doing something wrong?
You are not doing anything wrong. The behavior you observe and analyze is documented in 10.4 Chains of Implicit Rules. It states that intermediate files are indeed treated differently.
The second difference is that if make does create b in order to update
something else, it deletes b later on after it is no longer needed.
Therefore, an intermediate file which did not exist before make also
does not exist after make. make reports the deletion to you by
printing a rm -f command showing which file it is deleting.
The documentation does not explicitly explain why it behaves like this. Looking in the file ChangeLog.1, there is a reference to the remove_intermediates function as far back as 1988. At that time, disk space was expensive and at a premium.
If you do not want this behavior, mention the targets you want to keep somewhere in the makefile as an explicit prerequisite or target or use the .PRECIOUS or the .SECONDARY special built-in targets for that.
With thanks to MadScientist for the additional comments, see below.
I'm trying to limit $(all_possible_inputs) to $(relevant_inputs). $(all_possible_inputs) is a concatenation of multiple files from other included makefiles. The following functions correctly (the perl scripts know how to ignore the extra inputs), but everything is rebuilt if a single input changes:
$(step2_outputs): $(data)/%.step2: $(routines)/step2.%.pl $(all_possible_inputs)
perl $^ > $#
UPDATE: Filter must match more than one *.step1 file. If step1 produced:
A.foo.step1
A.bar.step1
B.foo.step1
B.bar.step1
B.baz.step1
Then step2's rules should expand to:
A.step2: routines/step2.A.pl A.foo.step1 A.bar.step1
B.step2: routines/step2.B.pl B.foo.step1 B.bar.step1 B.baz.step1
Logically, this is what I want to work:
$(step2_outputs): $(data)/%.step2: $(routines)/step2.%.pl $(filter $(data)/%.*.step1,$(all_possible_inputs))
perl $^ > $#
The % is supposed to match the static pattern rule stem. The * is supposed to be a wildcard (which I'm aware won't work). I believe the problem is that filter repurposes '%', so the filter expression fails. I thought it might be solvable with Secondary Expansion, but I tried this, and the filter still returned the empty string:
UPDATE: I switched the examples to use $$* based on Beta's good suggestion:
.SECONDEXPANSION:
$(step2_outputs): $(data)/%.step2: $(routines)/step2.%.pl $$(filter $(data)/$$*.%.step1,$(all_possible_inputs))
perl $^ > $#
This is running on gnu make 3.81 in a linux environment.
Your third method works for me, but you can try this: instead of % (which is expanded in the first phase) use $$*
.SECONDEXPANSION:
$(step2_outputs): $(data)/%.step2: $(routines)/step2.%.pl $$(filter $(data)/$$*.step1,$(all_possible_inputs))
perl $^ > $#
So i have a makefile here that looks something like
ALL_FILES=$(shell find . -name '*')
install : $(ALL_FILES)
$(INSTALL_LOCATION)/%.sh : %.sh ; /bin/usr/install -D $^ $#
$(INSTALL_LCOATION)/% : % ; /usr/bin/install -D $^ $#
$(INSTALL_LOCATION)/dir1/% : dir1/% ; /usr/bin/install -D $^ $#
my question here is , the second install there is obviously a superset of the first one,
but i guess the first one is ran first so the second one is ignored.
however, wouldnt the second one also be a full superset of the third one?
so how does make choose which rule to use? does it choose the most speicifc rule?
Depends on the version of make. For example, in GNU make prior to 3.82, it would search the patterns in the order they were declared and simply use the first pattern that matched, regardless of specificity. In 3.82 and later, GNU make uses the most specific rule, regardless of declaration order. Here's a simple example:
all: sub/foo.x
%.x:
#echo "Prefer first match (stem is $*)."
sub/%.x:
#echo "Prefer most specific match (stem is $*)."
Compare the output with gmake 3.81 and gmake 3.82:
$ gmake-3.81
Prefer first match (stem is sub/foo).
$ gmake-3.82
Prefer most specific match (stem is foo).
I've got this makefile:
ALL = ../lib/Mo.pm \
../lib/Mo/builder.pm \
../lib/Mo/default.pm \
../lib/Mo/has.pm \
all: $(ALL)
$(ALL): Mo.pm compress.pl Makefile
perl compress.pl $(#:../lib/%=%) > $#
What it's meant to do is something like this:
$ make -n
perl compress.pl Mo.pm > ../lib/Mo.pm
perl compress.pl Mo/builder.pm > ../lib/Mo/builder.pm
perl compress.pl Mo/default.pm > ../lib/Mo/default.pm
perl compress.pl Mo/has.pm > ../lib/Mo/has.pm
However with dmake on Windows this happens:
d:\mo-pm\src>dmake -n
perl compress.pl ..\lib\Mo.pm > ..\lib\Mo.pm
perl compress.pl ..\lib\Mo\builder.pm > ..\lib\Mo\builder.pm
perl compress.pl ..\lib\Mo\default.pm > ..\lib\Mo\default.pm
perl compress.pl ..\lib\Mo\has.pm > ..\lib\Mo\has.pm
I've been trying out various combinations of s/// and subst to make it work in dmake, and found out that it wants the path to have \s, which means a double substitution against both variants of the path (../lib/ and ..\lib) could work, but i can't figure out how to make it work for both make variants.
Any ideas or other ways to do this?
It's not only that the dir separator chars are different for both versions, moreover the dmake syntax seems to be deliberately designed to be incompatible with GNU make. The only part of the syntax that is actually compatible is pattern substitution, so this is the way to go:
all: $(ALL)
$(ALL) : Makefile compress.pl
../lib/%.pm : %.pm
perl compress.pl $< > $#
dmake actually substitutes the / for directory separator chars for you here. I've tested this Makefile with an echo instead, and it writes to the right directory.
Explanation: The pattern rules define rules for a particular file to be re-made when it matches a regular expression (the ../lib/%.pm part) and a prerequisite of a similar name is found (%.pm). The % in the prerequisite is replaced by the matching part of the % in the target. The extra rule with Makefile and compress.pl is needed because dmake doesn't like extra prerequisites in a pattern rule. As usual, $< and $# are make's special variables for source and target file.
So, the core difference is that your original rule said "the files named in this list can be made with the following rule), while the pattern rule says "any file looking like ../lib/%.pm can be made from a matching file in the current directory" and then gives a list of pm files to make.
Pattern rules are actually quite powerful, useful to know. Unfortunately, some makes don't know them, only the older suffix rules.
Further details of what's going on can be obtained by running
make -rdn