Make with with multiple targets with pattern - makefile

I thought that having multiple targets in a rule was the same as defining multiple rules with the same recipe for each of the targets; however, I seem to have a problem when the target/dependency have patterns.
Here's a simplified sample makefile with two different methods for building files.
all: $(foreach n, 1 2 3, $(foreach l, a b, out.$(l).$(n).txt))
1.txt 2.txt 3.txt:
touch $#
#Method One
out.a.%.txt out.b.%.txt: %.txt
touch $#
#Method Two
out.a.%.txt: %.txt
touch $#
out.b.%.txt: %.txt
touch $#
Using method one with a "make -n" I get
touch 1.txt
touch out.a.1.txt
touch 2.txt
touch out.a.2.txt
touch 3.txt
touch out.a.3.txt
but with method two I get the desired output
touch 1.txt
touch out.a.1.txt
touch out.b.1.txt
touch 2.txt
touch out.a.2.txt
touch out.b.2.txt
touch 3.txt
touch out.a.3.txt
touch out.b.3.txt
I didn't see any exceptions to the multiple target in a rule in the on-line docs. Did i miss a part of the documentation that explains why this doesn't work? Is there an easy way to make it work? (In reality, I have many more combinations of files to build than in this simple example).
(Using GNU Make 3.81)

Implicit (pattern) rules and explicit rules work in the opposite way from each other. For explicit rules, multiple targets define multiple separate rules, one for each target. For pattern rules, multiple targets means that a single invocation of the pattern rule recipe builds all the targets.
See the documentation on pattern rules.
Oh and BTW, no, there is no alternative to writing the pattern rule multiple times. You can use various methods to automatically generate the pattern rules, but that's about it.

Related

GNU make rule order/preference

I have two wildcard rules like so:
%important_results/output.pdf: input2.o
SETTING=$* myimportantcommand > $#
%results/output.pdf: input1.o
SETTING=$* mycommand > $#
with simple prerequisites:
input1.o: input1.c
touch input1.o
input2.o: input2.c
touch input2.o
However, I am getting varying behaviour when I run make important_results:
if input2.o exists, it uses the %important_results rule
if input2.o does not exists, but input1.o exists, it uses the %results rule instead of building input2.o. However, this runs the wrong command.
How can I give make a rule preference, so it always goes with %important_results rule even if it has to build prerequisites?
Searching for "gnu make rule order" or preference did not yield anything.
I suppose the simplest solution is to use non-overlapping wildcards.
edit: https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html suggests that stems cannot be applied to directories in this way.
GNU make manual:
The target is considered a pattern for matching file names; the ‘%’
can match any nonempty substring, while other characters match only
themselves.
Try the following, maybe:
%mportant_results/output.pdf: input2.o
SETTING=$(patsubst %i,%,$*) myimportantcommand > $#
Demo:
$ ls
Makefile input1.c input1.o input2.c
$ make important_results/output.pdf
touch input2.o
SETTING= myimportantcommand > important_results/output.pdf

How to avoid overriding targets, or suppress the warning

I want to create a makefile to be included in others, kind of as a "library", let's call it library.mak:
TARGET_FILES = a.txt b.txt c.txt d.txt
SRC_FOLDER = somewhere/else
.PHONY: all
all:
for target in ${TARGET_FILES} ; do \
echo -e "\nMaking $${target}:" ;\
${MAKE} ${MFLAGS} --no-print-directory $${target} ;\
done
.PHONY: ${TARGET_FILES}
${TARGET_FILES}:
cp ${SRC_FOLDER}/$# $#
The two rules are there to make all, as well as one specific target, respectively; using a default rule (which is the purpose of library.mak).
In my "user makefile" called Makefile, I want to then do this:
include library.mak
# special handling of c.txt
c.txt:
grep -v 'all except this' ${SRC_FOLDER}/$# > $#
As you can see, the user wants to be able to override the behaviour for some special cases. While this works, it always greets the user with the dreaded warning: overriding recipe for target and warning: ignoring old commands for target messages, even though the behaviour is as intended.
So here's the question: Can this be done in a different way that avoids these warnings, or is there a means to suppress them?
The warning says you overwrite a recipe for an explicit rule. And this is really wrong. It makes sense to use a pattern rule instead, like: %.txt: ${SRC_FOLDER}/%.txt. Then it's ok to have some explicit rule (c.txt: ${SRC_FOLDER}/c.txt) overwriting a recipe. Of course, it's not 100% the same, but, I think that should not be a problem. Anyway, putting an explicit rule into a reusable file is a crime.
Next, you constantly use "phonies" and rules w/o prerequisites even when you definitely should have them. This is bad. You're trying to make "make" work as a non-branching shell script. That's not only inefficient but also is a misuse. Basically, make can be viewed as "shell extension" capable of "branching on file timestamps in a sophisticated way". If you don't need it, don't use make at all.
Next, I see absolutely no reason to go into recursive make. Whatever, you're going to do, I believe, you can do without it. And even if you really need recursion, then write simply $(MAKE). $(MFLAGS) shoud not be used anymore (read this).
P.S. There's yet another (more flexible) option to define parameterized variables (macros). For example:
define nl :=
endef
define myrule.text
$1: $2/$1
#cp $$< $$#
endef
myrule = $(eval $(foreach foo$0,$1,$(call myrule.text,$(foo$0),$2)$(nl)))
...
# user makefile
# for everything except c.txt call predefined macro
$(call myrule,$(filter-out c.txt,$(TARGET_FILES)),$(SRC_FOLDER))
# for c.txt only
c.txt: $(SRC_FOLDER)/c.txt
grep -v 'all except this' $< >$#
But as you see, it's a bit of a "cryptic" stuff, and I don't feel like recommending it for a beginner.

why does "make" delete target files only if implicit

Suppose I have a Makefile like this
B1.txt: A1.txt
python big_long_program.py A1.txt > $#
correct1.txt: B1.txt reference.txt
diff -q B1.txt reference.txt
touch $#
Then the output when I make correct1.txt is pretty well what I would expect:
python big_long_program.py A1.txt > B1.txt
diff -q B1.txt reference.txt
touch correct1.txt
Now if I have lots of files, B1.txt, B2.txt, B3.txt etc, so create an implicit rule:
B%.txt: A%.txt
python big_long_program.py A$*.txt > $#
correct%.txt: B%.txt reference.txt
diff -q B$*.txt reference.txt
touch $#
Instead this happens when I make correct1.txt:
python big_long_program.py A1.txt > B1.txt
diff -q B1.txt reference.txt
touch correct1.txt
rm B1.txt
i.e. there difference is that now the file B1.txt has been deleted, which in many cases is really bad.
So why are implicit rules different? Or am I doing something wrong?
You are not doing anything wrong. The behavior you observe and analyze is documented in 10.4 Chains of Implicit Rules. It states that intermediate files are indeed treated differently.
The second difference is that if make does create b in order to update
something else, it deletes b later on after it is no longer needed.
Therefore, an intermediate file which did not exist before make also
does not exist after make. make reports the deletion to you by
printing a rm -f command showing which file it is deleting.
The documentation does not explicitly explain why it behaves like this. Looking in the file ChangeLog.1, there is a reference to the remove_intermediates function as far back as 1988. At that time, disk space was expensive and at a premium.
If you do not want this behavior, mention the targets you want to keep somewhere in the makefile as an explicit prerequisite or target or use the .PRECIOUS or the .SECONDARY special built-in targets for that.
With thanks to MadScientist for the additional comments, see below.

In make, can we use $^ to get the dependency list for just the current rule?

I've been learning make from the software carpentry tutorial (make patterns) and it says that we can use $^ to get the dependency list for our rule but have extra dependencies for our target by writing extra empty rules. For example
all:
touch f1.txt f2.txt a.txt
result: a.txt
result: f*.txt
#echo $^
I thought that this would print f1.txt f2.txt but instead I see f1.txt f2.txt a.txt. Am I missing something or is the tutorial wrong?
Clearly, I should have been explicit, in the example I run make to create the files for the test, then I run make result to get the result shown.
Not sure I understand your problem and what you would like to do but if you type just make it is like if you were typing make all because all is the first target and thus the default goal. It should create the 3 files (or update their last modification date if they already exist), echo the recipe and you should see:
touch f1.txt f2.txt a.txt
If you then type make result you should see the result of #echo $^, that is the list of all dependencies of result:
f1.txt f2.txt a.txt
It is absolutely normal as you declared them all as dependencies of result.
Finally, if you type make result before make or make all and if a.txt does not exist or if there is no file matching f*.txt, you will get an error because make needs a.txt and at least one file matching f*.txt to make result and it does not know how to make them.
I cannot imagine a use case where the behaviour you expected ($^ being expanded as dependencies of the current rule only) would be useful, but if really you need this feature you can use the rather obscure Double-Colon Rules (DCR):
all:
touch f1.txt f2.txt a.txt
result:: a.txt
#echo $^
result:: f*.txt
#echo $^
Note that if there is a DCR for a target, all rules of this target must also be DCR. Note also that all DCR must have a recipe. A DCR recipe is applied if one of the dependencies of this rule is newer than the target. If several DCR apply, they are executed in order of appearance in the Makefile. And finally, what you were interested in: the $^ automatic variable logically expands as the dependencies of the particular DCR.
No, $^ refers to the prerequisites for the current target, not the current rule.
Also, the results you are reporting are not correct; for make result I get make: *** No rule to make target `f*.txt', needed by `result'. Stop. like I very much expected.

Make : Get dependencies of a target inside another one

I often find myself trying to reference the dependency of a target (Target1) inside another one (Target2).
Consider the following Makefile. How nice would be a $(deps target-name) function !
Rule1 : file1 file2 file3
echo $^ # Careful, these are whitespaces, not a tab !
Rule2 : file4 file5 file6
echo $^ # Careful, these are whitespaces, not a tab !
clean-Rule-1-2 :
rm $(deps Rule1) $(deps Rule2) # Careful, these are whitespaces, not a tab !
I found this link mentioning that one could build himself his own dependency list, but it's looks rather tedious.
Does any one have a better solution (assuming none are natively implemented in the Makefile) and/or workflow tips referring to this issue ?
Why do you want to list prerequisites of a clean-type rule? That just forces make to build those dependencies if they're out of date, only to delete them again.
There is no way to do what you want because it is not possible to be consistent about it. For example, your rules could be written like this:
Rule1 : file1 file2 file3
echo $^ # Careful, these are whitespaces, not a tab !
clean-Rule-1-2 : $(deps Rule1)
rm $^ # Careful, these are whitespaces, not a tab !
Rule1 : file10 file11 file12
Oops! When $(deps ...) is expanded make doesn't know about the extra three prerequisites and so they won't be listed. And that's not even considering implicit rules, where make doesn't know what the full prerequisite list is when it parses the makefile; it only computes them when it's trying to build the target.
You don't give a real example of what you want to do, but generally the way makefiles are written is that you put the prerequisites into variables, then you can use the variables in multiple places.

Resources