I'm maintaining a (horrendously complicated) Makefile, and in some recipes I saw the following:
$(#:.h=.h.d)
I have absolutely no clue as to how to interpret this, or whether there's any documentation on those characters. Obviously, Google won't work because it thinks I'm typing gibberish.
I saw a related question about #:H, but this is GNU make instead of BSD make.
This is a variable reference with a substitution: $(VAR:FROM=TO). It means the value of the variable VAR, but for each whitespace-separated word in the value, if the word ends with the suffix FROM, it is replaced by the suffix TO.
In this case, the variable is #, the filename of the target of the rule (with special handling for archive members). If the target of the rule ends with .h, then .d is added at the end.
A common file naming convention is to use .d for a list of dependencies. The file foo.h.d presumably contains dependencies for rules to compile source files that include foo.h (so, in practice, foo.d.h would contains foo.h and the headers that it includes).
By the way, this is portable syntax. There is another slightly more wordy syntax which is common (supported by both GNU and BSD make) but not POSIX: $(#:%.h=%.h.d) where the % acts as a wildcard; this syntax allows a prefix to be substituted in addition to a suffix. There is yet another syntax to do the same thing in GNU make: call the function patsubst, written $(patsubst %.h,%.h.d,$#) — it's arguably less cryptic, but because the portable syntax has existed for decades, it's commonly used even in makefiles that otherwise require GNU make.
Related
I'm currently learning Make and am struggling to wrap my head around the wildcard concept. Specifically, it seems like there are two symbols that can represent wildcards: * and %
For example, say I want to to generate a variable that identifies all .c source files in the working directory. I would use the traditional * symbol in *.c However, if I use the patsubst function, I am required to use the % symbol instead of * symbol:
// WORKS
SRC = $(wildcard *.c) # list of source files
OBJS = $(patsubst %.c, %.o, $(SRC)) # list of object files
// DOES NOT WORK!!!!
SRC = $(wildcard *.c) # list of source files
OBJS = $(patsubst *.c, *.o, $(SRC)) # list of object files
Can someone explain the difference between * and % in the context of Make wildcards?
Can someone explain the difference between * and % in the context of Make wildcards?
TL;DR:
This is all specific to GNU make.
* and a few other characters are special to the wildcard function, but not to patsubst. This kind of pattern is expanded to the names of existing files and directories that match the pattern, possibly more than one for each pattern.
% is special to the patsubst function. This kind of pattern is used to select matching strings provided by the makefile or directly by make, in the process capturing the part matching the % for later use.
Both kinds have application to the target and prerequisite lists of rules, but their significance there is somewhat different from each other.
None of these are significant to make in recipes, but wildcard-style patterns are significant to the shell, and the shell will interpret them in recipes that it executes.
General
Understand first that $(wildcard) and $(patsubst) are features specific to GNU's implementation of make, and GNU make also attributes special significance to %, *, and a few other characters in rule targets and prerequisites. The POSIX specifications for make say nothing about any of that. GNU make is widely used these days, and with good reason, but it is not the only make you might encounter. If you want maximum portability among make implementations then you must avoid these altogether.
Understand also that a complete, albeit rather basic response to the question would be simply "yes, the wildcard and patsubst functions recognize different special characters." These functions do different things, so it is potentially useful that the special characters of one can be used as ordinary characters in the other.
Wildcards
The asterisk (*) is among the special characters recognized by the Bourne shell for "pathname expansion", which replaces patterns with the names of possibly-many existing files and directories matching the pattern. There are more characters than just * significant in pathname expansion, but % is not among them. Look up that term for a full description.
Additionally, there is the tilde (~), which make and some shells recognize for "tilde expansion", which involves interpreting the first segments of affected paths as specified users' home directories.
The GNU make documentation describes the characters and constructs it recognizes for pathname and tilde expansion as "wildcards", and it has this to say about them:
Wildcard expansion is performed by make automatically in targets and in prerequisites. In recipes, the shell is responsible for wildcard expansion. In other contexts, wildcard expansion happens only if you request it explicitly with the wildcard function.
(GNU make manual, section 4.4)
And that's where the wildcard function appearing in the question comes in -- if you want to perform the same wildcard expansion that make performs automatically on target and prerequisite names in some other context, such as a variable definition, then you can use the wildcard function to get it. Thus,
SRC = $(wildcard *.c)
results variable SRC representing a list of all the existing files in the working directory at the time the makefile is parsed whose names end with .c.
On the other hand, % is not significant for pathname or tilde expansion, so
SRC = $(wildcard %.c)
will expand to literally %.c.
Patterns
The shell documentation uses the term "pattern" to mean shell input that is interpreted to be subject to pathname expansion, rather than being literal. However, GNU make reserves the term for a different kind of pattern in which the % character features. This is relevant in two main areas: the patsubst function and pattern rules.
The patsubst function
The patsubst function computes one string or series of strings from another by replacing those that match a given pattern with a specified replacement. In a pattern, the first % character, if any, matches any number of characters, including zero. In this sense, it can be described as a wildcard, and the manual does, somewhat confusingly, use that term. If the replacement also contains at least one % then the first is replaced by the substring that was matched by the % in the pattern.
Note well that this has nothing inherently to do with file names, existing or otherwise.
Thus, with
SRC = main.c other.c
OBJ = $(patsubst %.c,%.o,$(SRC))
the %.c pattern matches first main.c (with % matching main) and then other.c (with % matching other), and the result for OBJ is the same as this:
OBJ = main.o other.o
regardless of whether any files named main.c, other.c, main.o, or other.o exist.
Pattern rules
If the target of a rule contains a %, then it is interpreted by GNU make as a pattern rule. Pattern rules use % in much the same way that patsubst does. When make needs to build a target but does not have a rule for that specific target, it will check whether the target name matches the target pattern of any pattern rule (including some built-in ones). There does not need to be any existing file of that name. If it finds a match, then it will use that rule to build the target. And in that case, for the purpose of building the target in question, the first % character, if any, in any prerequisite names specified in that rule will be replaced by the stem that matched the % in the target, much like in patsubst.
Combinations
Usually, only one of these kinds of pattern matching / expansion is desired in any given context, but sometimes they can usefully be combined. For example, consider this pattern rule:
%.o: %.c *.h
$(CC) $(CPPFLAGS) $(CFLAGS) -c -o $# $<
That says that any .o file can be built from a corresponding .c file and all the .h files in the working directory by applying the provided recipe.
The wildcard function expects wildcard characters (~, *, ? and [...] which match on file names) while the patsubst function expects a pattern (%) that operates on text. As such $(patsubst *.c, *.o, $(SRC)) means replace the literal string *.c with the string *.o and SRC probably does not contain any literal *.c strings.
I'm working on a rather large project in pure C, and for various reasons we are not using CMake.
We have a good system which uses various shell commands to require very little maintanence. It will automatically find new headers and C files and add the headers to the dependencies and will compile the C files and include them in the output. However, we've gotten to the point where any changes we make to the header files becomes a problem because it requires a recompilation of the entire project, rather than just the C files that include it directly or indirectly.
I've worked on a system to ensure that the only header files which are added as dependencies are ones which are required by the C file itself (recursively up the tree).
I had thought I would be able to solve this myself, but it seems that make has rather inconsistent rules for expansion of variables.
The solution I came up with was to try to use ag and get all the includes out of the file. This works as intended, and I pipe it into tr so that I can ensure that no newlines are added, thereby messing up make. The issue that I'm having though is in determining which file to search in. This is the recipe that it's using for the section in question:
$(GAMEOBJDIR)/%.o : $(GAMEDIR)/%.c $(shell ag -o '(?<=(^#include "))(.*?)(?=("$$))' $(GAMEDIR)/%.c | tr '\n' ' ')
$(CC) $(CFLAGS) $(if $(RELEASE),$(RELFLAGS),$(DBFLAGS)) -c -o $# $<
The problem is that $(GAMEDIR)/%.c is expanding to src/game/%.c and not src/game/<filename>.c. I'm unsure how to fix this issue based on the expansion rules of make.
Once I can figure this out I'll be able to make sure this walks up the chain of header files too, but until I can get this figured out I have no reason to work on that.
Make's rules for expansion are completely consistent... but sometimes they aren't what people want. That's not the same thing :)
Rules for expansion are explained in the manual. In your case you're working with a prerequisite list which means the variables are expanded as the makefile is parsed. That includes the shell command and all other variables. Because you are working with a pattern rule, it means that at the time the makefile is parsed it doesn't match any particular file, it's just defining the pattern rule itself. So, % can't be expanded here. It will be expanded later, when make starts walking the dependency graph and trying to locate ways to build targets.
In general your method cannot work efficiently. You could defer the expansion of these parts of the prerequisites list using Secondary Expansion. However, that means every time make wants to TRY to use this pattern it will run these commands--that will just be slow.
Have you considered using a more standard way to manage detecting prerequisites? Try reading this description for example to see if it would work for you.
My main Makefile call config.mk
include $(TOPDIR)/config.mk
then config.mk include some sentences like this:
ifdef CPU
sinclude $(TOPDIR)/cpu/$(CPU)/config.mk
endif
ifdef SOC
sinclude $(TOPDIR)/cpu/$(CPU)/$(SOC)/config.mk
endif
I have prepared these two tree and necessary config.mks. But for "SOC", whose value is "versatile", there is a problem. If I put "versatile" directly here, it could find the file and everything is fine; but when I use $(SOC), il will meet an error, and say
/../../../cpu/arm926ejs/versatile: is a folder, stop
Anyone know what the problem is ??
Are you sure you gave the exact error message? What version of make are you using? That error doesn't look like anything GNU make would print.
Anyway, I'll bet the problem is that your assignment of the SOC variable has trailing whitespace. According to the POSIX definition of make, leading whitespace before a variable value is removed, but trailing whitespace is preserved. That means, for example, if you write your makefile like this:
SOC = versatile # this is the versatile SOC
then make will remove the comment, but keep the space, so the value will be 'versatile' (space at the end). This means when the value is expanded in the sinclude line you get:
sinclude $(TOPDIR)/cpu/$(CPU)/versatile /config.mk
which make interprets as trying to include two different values, the first of which is a directory.
Even if you don't have a comment there, any trailing whitespace will be preserved. When editing makefiles you should try to put your editor into a mode where it flags trailing whitespace, or even better removes it automatically. GNU Emacs, for example, can do this.
I have a makefile that has C INCLUDES with spaces in them. There is no way for me to get around having to have the spaces in the file names. Is there any way to have spaces in file names with gnu make?
Make has some basic support for this by escaping spaces in filenames, in that the following Makefile will correctly compile and recompile the C file foo bar.c:
foo\ bar: foo\ bar.c
gcc -o "${#}" "${<}"
However, you have to be super-careful in quoting every command you run, and variables that are space-separated lists of files—e.g., SRCS, LIBS—won’t work, although it’ß possible that with enough hacking using Make text functions you can parse out the quotes and get everything working…
So while there is rudimentary support for spaces in filenames in rules and patterns, anything complicated is going to be an awful lot of very hard and frustrating work.
I'm using GNUmake to generate a deployment version of some web content from a source directory. I want to be rules that will run some files through compression tools (e.g. YUI compressor), and then for anything that doesn't have a rule, just copy it.
So for example:
# Rule with all the $(WWW_OUT_DIR)/*.* files as a prerequisite
all: $(WWW_OUT_FILES)
# Generic rule to perform compression on Javascript files.
$(WWW_OUT_DIR)/%.js: $(WWW_SRC_DIR)/%.js
$(YUI_COMP) $(YUI_COMP_OPTS) $< > $#
# Generic rule to perform compression on CSS files.
$(WWW_OUT_DIR)/%.css: $(WWW_SRC_DIR)/%.css
$(YUI_COMP) $(YUI_COMP_OPTS) $< > $#
# TDB Rule to magically handle everything else? (This doesn't work)
$(WWW_OUT_DIR)/%.%: $(WWW_SRC_DIR)/%.%
cp $< $#
How do I accomplish what that last rule is trying to do? I.e. For everything in $(WWW_OUT_FILES) that's not a .js or .css, just copy it? If possible, I want to retain the dependency on the corresponding input file.
You are almost right, the only thing to fix is the last pattern rule, just remove redundant percent symbol:
$(WWW_OUT_DIR)/%: $(WWW_SRC_DIR)/%
cp $< $#
Also keep in mind that as of GNU Make 3.82 pattern search algorithm has been modified a bit (from Changelog):
The pattern-specific variables and pattern rules are now applied in the
shortest stem first order instead of the definition order (variables
and rules with the same stem length are still applied in the definition
order). This produces the usually-desired behavior where more specific
patterns are preferred.
It is exactly what you want in case if you use the most recent version of Make. In order to get your Makefile compatible with other versions of GNU Make (including versions earlier than 3.82), the rule must be defined after other ones (as in the original question).
UPD.
A good example from here:
Prior to version 3.82, when gmake finds multiple matches during a pattern search, it prefers patterns declared earlier in the makefile over patterns declared later. As of 3.82, gmake instead prefers the pattern that results in the shortest stem. That sounds a bit confusing thanks to the jargon, but I think this will actually cause gmake to better adhere to the principle of least astonishment. Here’s an example:
all: sub/foo.x
%.x:
#echo "Prefer first match (stem is $*)."
sub/%.x:
#echo "Prefer most specific match (stem is $*)."
Compare the output from gmake 3.81 and 3.82:
gmake 3.81
Prefer first match (stem is sub/foo).
gmake 3.82
Prefer most specific match (stem is foo).
gmake 3.82 prefers the second pattern because it is a more specific match than the first. Note that this is a significant backwards-incompatibility compared with previous versions of gmake!