In a Makefile I have:
images/schematic.pdf: images/schematic.svg
inkscape -D -z --file=$^ --export-pdf=$# --export-latex
sed -i "s:schematic:images/schematic:g" $#_tex
What this rule does is:
Use inkscape to convert to a latex-ready .pdf PDF file + its corresponding .pdf_tex text file (see this answer: https://tex.stackexchange.com/a/2107/104581)
Modify the mentioned .pdf_tex file so that it will not break latex compiling from a "higher" directory (namely . when the .pdf_tex file is in ./images)
My problem:
I have many rules in this form i. e. where only schematic is changed. I would like to use a pattern-rule that replaces schematic by %. And to use % in the recipe (in the sed command).
However the rule:
images/%.pdf: images/%.svg
inkscape -D -z --file=$^ --export-pdf=$# --export-latex
sed -i "s:%:images/%:g" $#_tex
does not works: % is interpreted literally in the recipe.
I also tried to replace % in the recipe by $% but this variable seems to be empty.
Unsatisfactory solution:
Add a line in the recipe to create a (make) variable that will hold the result of notdir(removeprefix($<)) (using this question or a call to bash because there is no removeprefix in GNU Make).
You want $*.
From https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html :
$*:
The stem with which an implicit rule matches (see How Patterns Match).
If the target is dir/a.foo.b and the target pattern is a.%.b
then the stem is dir/foo. The stem is useful for constructing names of
related files.
In a static pattern rule, the stem is part of the file name that matched the ‘%’ in the target pattern.
In an explicit rule, there is no stem; so ‘$*’ cannot be determined in that way. Instead, if the target name ends with a
recognized suffix (see Old-Fashioned Suffix Rules), ‘$*’ is set to the
target name minus the suffix. For example, if the target name is
‘foo.c’, then ‘$*’ is set to ‘foo’, since ‘.c’ is a suffix. GNU make
does this bizarre thing only for compatibility with other
implementations of make. You should generally avoid using ‘$*’ except
in implicit rules or static pattern rules.
If the target name in an explicit rule does not end with a recognized suffix, ‘$*’ is set to the empty string for that rule.
$% is an automatic variable, but it's for archive (.a library) members, and almost never useful.
Related
I'm currently learning Make and am struggling to wrap my head around the wildcard concept. Specifically, it seems like there are two symbols that can represent wildcards: * and %
For example, say I want to to generate a variable that identifies all .c source files in the working directory. I would use the traditional * symbol in *.c However, if I use the patsubst function, I am required to use the % symbol instead of * symbol:
// WORKS
SRC = $(wildcard *.c) # list of source files
OBJS = $(patsubst %.c, %.o, $(SRC)) # list of object files
// DOES NOT WORK!!!!
SRC = $(wildcard *.c) # list of source files
OBJS = $(patsubst *.c, *.o, $(SRC)) # list of object files
Can someone explain the difference between * and % in the context of Make wildcards?
Can someone explain the difference between * and % in the context of Make wildcards?
TL;DR:
This is all specific to GNU make.
* and a few other characters are special to the wildcard function, but not to patsubst. This kind of pattern is expanded to the names of existing files and directories that match the pattern, possibly more than one for each pattern.
% is special to the patsubst function. This kind of pattern is used to select matching strings provided by the makefile or directly by make, in the process capturing the part matching the % for later use.
Both kinds have application to the target and prerequisite lists of rules, but their significance there is somewhat different from each other.
None of these are significant to make in recipes, but wildcard-style patterns are significant to the shell, and the shell will interpret them in recipes that it executes.
General
Understand first that $(wildcard) and $(patsubst) are features specific to GNU's implementation of make, and GNU make also attributes special significance to %, *, and a few other characters in rule targets and prerequisites. The POSIX specifications for make say nothing about any of that. GNU make is widely used these days, and with good reason, but it is not the only make you might encounter. If you want maximum portability among make implementations then you must avoid these altogether.
Understand also that a complete, albeit rather basic response to the question would be simply "yes, the wildcard and patsubst functions recognize different special characters." These functions do different things, so it is potentially useful that the special characters of one can be used as ordinary characters in the other.
Wildcards
The asterisk (*) is among the special characters recognized by the Bourne shell for "pathname expansion", which replaces patterns with the names of possibly-many existing files and directories matching the pattern. There are more characters than just * significant in pathname expansion, but % is not among them. Look up that term for a full description.
Additionally, there is the tilde (~), which make and some shells recognize for "tilde expansion", which involves interpreting the first segments of affected paths as specified users' home directories.
The GNU make documentation describes the characters and constructs it recognizes for pathname and tilde expansion as "wildcards", and it has this to say about them:
Wildcard expansion is performed by make automatically in targets and in prerequisites. In recipes, the shell is responsible for wildcard expansion. In other contexts, wildcard expansion happens only if you request it explicitly with the wildcard function.
(GNU make manual, section 4.4)
And that's where the wildcard function appearing in the question comes in -- if you want to perform the same wildcard expansion that make performs automatically on target and prerequisite names in some other context, such as a variable definition, then you can use the wildcard function to get it. Thus,
SRC = $(wildcard *.c)
results variable SRC representing a list of all the existing files in the working directory at the time the makefile is parsed whose names end with .c.
On the other hand, % is not significant for pathname or tilde expansion, so
SRC = $(wildcard %.c)
will expand to literally %.c.
Patterns
The shell documentation uses the term "pattern" to mean shell input that is interpreted to be subject to pathname expansion, rather than being literal. However, GNU make reserves the term for a different kind of pattern in which the % character features. This is relevant in two main areas: the patsubst function and pattern rules.
The patsubst function
The patsubst function computes one string or series of strings from another by replacing those that match a given pattern with a specified replacement. In a pattern, the first % character, if any, matches any number of characters, including zero. In this sense, it can be described as a wildcard, and the manual does, somewhat confusingly, use that term. If the replacement also contains at least one % then the first is replaced by the substring that was matched by the % in the pattern.
Note well that this has nothing inherently to do with file names, existing or otherwise.
Thus, with
SRC = main.c other.c
OBJ = $(patsubst %.c,%.o,$(SRC))
the %.c pattern matches first main.c (with % matching main) and then other.c (with % matching other), and the result for OBJ is the same as this:
OBJ = main.o other.o
regardless of whether any files named main.c, other.c, main.o, or other.o exist.
Pattern rules
If the target of a rule contains a %, then it is interpreted by GNU make as a pattern rule. Pattern rules use % in much the same way that patsubst does. When make needs to build a target but does not have a rule for that specific target, it will check whether the target name matches the target pattern of any pattern rule (including some built-in ones). There does not need to be any existing file of that name. If it finds a match, then it will use that rule to build the target. And in that case, for the purpose of building the target in question, the first % character, if any, in any prerequisite names specified in that rule will be replaced by the stem that matched the % in the target, much like in patsubst.
Combinations
Usually, only one of these kinds of pattern matching / expansion is desired in any given context, but sometimes they can usefully be combined. For example, consider this pattern rule:
%.o: %.c *.h
$(CC) $(CPPFLAGS) $(CFLAGS) -c -o $# $<
That says that any .o file can be built from a corresponding .c file and all the .h files in the working directory by applying the provided recipe.
The wildcard function expects wildcard characters (~, *, ? and [...] which match on file names) while the patsubst function expects a pattern (%) that operates on text. As such $(patsubst *.c, *.o, $(SRC)) means replace the literal string *.c with the string *.o and SRC probably does not contain any literal *.c strings.
In a Makefile, I saw this:
all: .made
# ... some other code
.made: $(Program) bashbug
#echo "$(Program) last made for a $(Machine) running $(OS)" >.made
Is the .made rule a suffix rule as per: https://www.gnu.org/software/make/manual/make.html#Suffix-Rules?
Also, are all rules with targets that have a . in front suffix rules?
Otherwise, what is the significance of targets that start with .? It seems like there's a significance as per this in https://www.gnu.org/software/make/manual/make.html#How-Make-Works:
By default, make starts with the first target (not targets whose names
start with ‘.’).
But its significance is not mentioned.
Also, if it is a suffix rule, how come .made can be used as a prerequisite in the all rule? (It's not mentioned that the targets of suffix rules can be used as prerequisites in other rules.)
P.S.: This question is related to but different from what is the meaning for . (dot) before the target in makefile. This question asks explicitly for the difference between a target with . and a target of a suffix rule.
Is the .made rule a suffix rule
No, because .made is not a "known suffix".
Also, are all rules with targets that have a . in front suffix rules?
No, only those where the word after the dot is a "known suffix":
Your first two questions are answered by https://www.gnu.org/software/make/manual/make.html#Suffix-Rules:
Suffix rule definitions are recognized by comparing each rule’s target against a defined list of known suffixes. When make sees a rule whose target is a known suffix, this rule is considered a single-suffix rule. When make sees a rule whose target is two known suffixes concatenated, this rule is taken as a double-suffix rule.
In your example .made is an actual filename. Your Makefile has a rule for it that creates/updates the file:
#vvvv
.made: $(Program) bashbug
#echo "$(Program) last made for a $(Machine) running $(OS)" >.made
# ^^^^^
All pretty normal; it would work the same way with any other name.
The only significance of the leading dot is that it makes the file "hidden" by convention, i.e. ls won't show it (without specifying -a), plain * won't match it, etc.
Consider the following:
%.foo: %.bar
echo $< > $#
Assuming we have one file 1.bar, the command executed is simply echo 1.bar > 1.foo. However, when % contains a path, rather than just a file name, it start becoming finicky. My problem is that I want to prepend another path to %.bar, the pattern becomes completely mangled. I.e., when %.bar is nice/path/1.bar, this becomes impossible:
%.foo: /project/something/%.bar
echo $< > $#
This will run, but it executes echo nice/path//project/something/1.bar > 1.foo in stead of echo /project/something/nice/path1.bar > 1.foo
The reason for this is in how make does its pattern rules. From the docs:
When the target pattern does not contain a slash (and it usually does not), directory names in the file names are removed from the file name before it is compared with the target prefix and suffix. [...] The directories are ignored only for the purpose of finding an implicit rule to use, not in the application of that rule. Thus, ‘e%t’ matches the file name src/eat, with ‘src/a’ as the stem. When prerequisites are turned into file names, the directories from the stem are added at the front, while the rest of the stem is substituted for the ‘%’. The stem ‘src/a’ with a prerequisite pattern ‘c%r’ gives the file name src/car
Is there any way I can turn this off for a specific rule?
You may like to read up on How Patterns Match:
When the target pattern does not contain a slash (and it usually does not), directory names in the file names are removed from the file name before it is compared with the target prefix and suffix. After the comparison of the file name to the target pattern, the directory names, along with the slash that ends them, are added on to the prerequisite file names generated from the pattern rule’s prerequisite patterns and the file name. The directories are ignored only for the purpose of finding an implicit rule to use, not in the application of that rule. Thus, ‘e%t’ matches the file name src/eat, with ‘src/a’ as the stem. When prerequisites are turned into file names, the directories from the stem are added at the front, while the rest of the stem is substituted for the ‘%’. The stem ‘src/a’ with a prerequisite pattern ‘c%r’ gives the file name src/car.
The above explains why nice/path/ is prepended to prerequisite /project/something/1.bar.
One fix would be to use full file names in rules, e.g.:
${CURDIR}/nice/path/%.foo: /project/something/%.bar
I have a makefile which generates a bunch of versions of an image in different places:
website/img/logo_256.png
website/img/logo_152.png
/tmp/logo_64.png
and so on (the /tmp/ generation is so I can later use those files to later generate a multiresolution .ico, the details of that aren't important).
I'd like a rule of the form
logo_%.png: ${SRC}
convert $^ -thumbnail $*x$* $#
but, $* brings in the matched directory too, so I get a command of the form:
convert logo_1024.png -thumbnail /tmp/64x/tmp/64 /tmp/logo_64.png
which is incorrect (I need 48x48, not /tmp/48x/tmp/48).
Or I can write
/tmp/logo_%.png: ${SRC}
convert $^ -thumbnail $*x$* $#
website/img/logo_%.png: ${SRC}
convert $^ -thumbnail $*x$* $#
which seems ugly.
I'm sure there are ways to break down and pattern match $# to get what I want, but I'm not a makefile guru, so this would take some research.
What's the easiest way to do this?
See the second half of the Automatic Variables in the GNU Make Manual:
Of the variables listed above, four have values that are single file names, and three have values that are lists of file names. These seven have variants that get just the file's directory name or just the file name within the directory. The variant variables' names are formed by appending ‘D’ or ‘F’, respectively. These variants are semi-obsolete in GNU make since the functions dir and notdir can be used to get a similar effect (see Functions for File Names). Note, however, that the ‘D’ variants all omit the trailing slash which always appears in the output of the dir function. Here is a table of the variants:
‘$(#D)’
The directory part of the file name of the target, with the trailing slash removed. If the value of ‘$#’ is dir/foo.o then ‘$(#D)’ is dir. This value is . if ‘$#’ does not contain a slash.
‘$(#F)’
The file-within-directory part of the file name of the target. If the value of ‘$#’ is dir/foo.o then ‘$(#F)’ is foo.o. ‘$(#F)’ is equivalent to ‘$(notdir $#)’.
‘$(*D)’
‘$(*F)’
The directory part and the file-within-directory part of the stem; dir and foo in this example.
‘$(%D)’
‘$(%F)’
The directory part and the file-within-directory part of the target archive member name. This makes sense only for archive member targets of the form archive(member) and is useful only when member may contain a directory name. (See Archive Members as Targets.)
‘$(<D)’
‘$(<F)’
The directory part and the file-within-directory part of the first prerequisite.
‘$(^D)’
‘$(^F)’
Lists of the directory parts and the file-within-directory parts of all prerequisites.
‘$(+D)’
‘$(+F)’
Lists of the directory parts and the file-within-directory parts of all prerequisites, including multiple instances of duplicated prerequisites.
‘$(?D)’
‘$(?F)’
Lists of the directory parts and the file-within-directory parts of all prerequisites that are newer than the target.
Edit:
As prompted by #Ian's comment I looked again and realized that this was not a complete solution. A complete solution follows.
The above F modifiers (and the $(notdir) function) will strip the path from the target filename. That's part of what is necessary.
Additional manipulation is required to extract only the numerical component from target like /some/path/logo_64.png.
The $(basename) function will strip off suffixes (as will $(patsubst %.png,%,$#) or $(#:%.png=) in a more specific fashion).
Combining those we get from /some/path/logo_64.png to logo_64. Handling things at this point depends heavily on what the data is going to look like and what assertions about it can be made. If logo_ is a static prefix then a simple $(patsubst logo_%,%,...) will work (as will the matching substitution reference like before).
If that is not guaranteed but the guarantee can be made that the dimension will be the last underscore separated component then $(lastword $(subst _, ,...)) can be used.
The rule needed is:
logo_%.png: ${SRC}
convert $^ -thumbnail $(*F)x$(*F) $#
The $(*F), is documented very briefly in the Make manual, as quoted in Etan's answer.
‘$(*F)’ The file-within-directory part of the stem; foo in this example.
The 'stem' ($*) is anything not explicit in the pattern. That includes the wildcard, and any implicit directories. So hence in the question it had the value /tmp/48, /tmp/ from the implicit directory, and 48 from the wildcard in the pattern. So of this combined stem, I need to select just the filename part, $(*F).
Alternatively, noting that the manual states:
These variants are semi-obsolete in GNU make since the functions dir and notdir can be used to get a similar effect
we can instead do:
logo_%.png: ${SRC}
convert $^ -thumbnail $(notdir $*)x$(notdir $*) $#
In a comment, Etan also linked to the How Patterns Match section of the manual, to help understand how the stem is constructed. I found this useful and wanted to bubble it up into an answer.
Given this bit of Makefile:
# for pattern matching
$(OBJDIR) := build
# just to see if a level of indirection will work
my_dir = $(dir $(1))
$(OBJECTS) : $(OBJDIR)/% : $(HEADERS) $(SRCDIR)/% | % $(dir %) $(call my_dir,%)
#echo output-only = $|
This is a static pattern rule with order-only prerequisites.
Consider the target "build/utility/debug.js". The output of the above rule will be this:
output-only = utility/debug.js ./
The first component, "utility/debug.js", is properly copied from the stem (%).
The second component, "./", is the output of calling the dir function in the prerequisites list.
The third component, an empty string, is the output of calling my my_dir function in the prerequisites list.
If I change my_dir to this:
my_dir = $(1)
The output remains the same. If I change it to this:
my_dir = "foo"
Then make complains there is no rule to make "foo" (which is expected). It appears, then, that $(1) is not getting bound in the call to my_dir.
What's going on? Why can't I pass the stem to a function? I have a workaround that uses secondary expansion, but I want to know why I can't write my rule this way.
EDIT: I'm new to stackoverflow, forgive me if this is not the way things are done here.
I want $(1) because I am passing the stem as an argument to my_dir, as Alex pointed out.
I don't know why it was suggested I want "$". I don't believe $ by itself expands to anything in any context.
I know that automatic variables are only available in the recipe. I am not using an automatic variable in the prerequisites - I am using the stem:
Each target is matched against the target-pattern to extract a part of the target name, called the stem. This stem is substituted into each of the prereq-patterns to make the prerequisite names (one from each prereq-pattern). - the manual
The fact that the stem is available is demonstrated by the example: the stem expands to the correct value when used alone, but not when passed to a function.
As can be seen in this section of the GNU make manual, variable and function references in the list of prerequisites are immediately expanded, during the read in phase. This means, before any pattern matching is done, so the % has no special meaning yet; it is interpreted as a literal character in the two function references, $(dir %) and $(call my_dir,%), both having ./ as a result, which get merged in the reference to $| in the recipe.
I don't know of any other workaround than the one you already found, i.e. secondary expansion.
Note $1 is not a special variable that expands to anything interesting related to pattern rules (or static pattern rules). The $1 variable only has unique behavior within the context of a user-defined macro invoked by the $(call ...) function.
You wanted to use $*, not $1; $* is an automatic variable which expands to the stem of the target of the rule.
However, in all versions of make (including the POSIX standard definition of make), automatic variables (including $*, $<, $#, $^, etc.) are only available in the context of the recipe. They are not available in the context of the target or prerequisite lists. See the GNU make manual section on Automatic Variables for more details.
As you suggest, there is a GNU make-specific feature enabled by the .SECONDEXPANSION pseudo target which provides a way to avoid this limitation.