Could somebody provide a real-world example of using recursively expanded variables (REV)? In the docs or various blog posts people give only useless toy examples, like
foo = $(bar)
bar = $(ugh)
ugh = Huh?
I cannot find a real use for REV besides creating custom functions with $(call). I also found that in the past people were using REV to supply additional parameters to a compiler for specific targets but that trick is considered outdated now because GNU Make has target-specific variables.
Both recursively expanded variables and simply expanded variables recurse their expansions. The major difference is when that expansion happens.
So your example above works just fine with simply expanded variables if you invert the assignments:
ugh := Huh?
bar := $(ugh)
foo := $(bar)
So the major thing that recursively expanded variables get you is the freedom to assign values in whatever order you need (which means you don't need to worry about inclusion order for included makefiles, etc.).
In a project at work we have a dozen or so included makefiles that have inter-dependent relationships. These are expressed through the usage of known-format variable names (e.g. module A generates an A_provides variable, etc.) Modules that need to utilize the things that module A provides can then list $(A_provides) in their makefiles.
With simply expanded variables (which we had been using until somewhat recently) this meant that the inclusion of the generated makefiles required a manually sorted order to force the inclusion of assigning makefiles before consuming makefiles (into other variables).
With recursively expanded variables this order does not matter. (This would not be the case if the variables were used in any immediately evaluated context in these makefiles but luckily they are not, they only set variables that are used later in the main makefiles.)
Well, one simple example are variables which contain commands for recipes; perhaps:
buildc = $(CC) $(CPPFLAGS) $(CFLAGS) -c -o $# $<
%.o: %.c
$(buildc)
I probably wouldn't write a compile rule like this this way, but if you have a much more complex recipe it can be very useful.
Personally I don't consider "additional parameters ... for specific targets" (by which I assume you mean recursively defined variables such as $($*_FLAGS)) to be outdated or obsoleted by target-specific variables, by any stretch. If nothing else recursively defined variables are much more portable than target-specific variables.
It just means that recursively defined variables are set at the time of definition only!
The best example I can find is this
x := foo
y := $(x) bar
x := later
and is equivalent to
y := foo bar
x := later
Related
I've been debugging this problem for 2 days straight, and I'm about to lose my mind here. We have a master makefile that includes a bunch of sub makefiles. Among other things, these sub makefiles add stuff to a variable EXTERNAL_LIBS.
The adding to the variable happens like this:
EXTERNAL_LIBS += foo
or this
EXTERNAL_LIBS := bar $(EXTERNAL_LIBS)
The problem I'm having is that the values added to this EXTERNAL_LIBS variable by some sub makefiles are gone when I look at the EXTERNAL_LIBS variable in the master makefile (which happens AFTER all the sub makefiles have been included), while the values added by other sub makefiles are still there. I don't understand what's happening to the "lost" values.
I've grepped our entire codebase for occurrences of EXTERNAL_LIBS to make sure it doesn't get emptied out or otherwise overwitten somewhere, and it doesn't. Also, I know for sure that the sub makefiles in question do get evaluated as I've both added debug printouts and done a trace via the -d option to GNU Make.
Does anyone have an idea what I might be doing wrong here? We're using GNU Make 4.2 for Windows on a Windows 7 machine, by the way.
EDIT: I got it!!!! The root cause was my incomplete knowledge of how GNU Make functions. This link ( https://www.gnu.org/software/make/manual/html_node/Reading-Makefiles.html ) describes how variables initialized like this
EXTERNAL_LIBS = bla
will used deferred expansion when stuff is appended via +=, whereas variables initialized like this
EXTERNAL_LIBS := bla
will use immediate expansion. That was exactly the problem! Some makefiles used syntax like this:
MY_LIB_NAME = <some_fancy_function_call>
...
EXTERNAL_LIBS += $(MY_LIB_NAME)
and many makefiles reused the MY_LIB_NAME variable in that fashion. So I think what happened is that all these Makefiles didn't add the current value of $(MY_LIB_NAME) but the literal string "$(MY_LIB_NAME)", which then later got expanded to whatever the last makefile had set $(MY_LIB_NAME) to. I have now switched to immediate expansion by initializing the variable like this:
EXTERNAL_LIBS :=
hoping that this would enforce immediate expansion. I've also switched from = to := when assigning the value to the variable that later shows up on the right hand side of EXTERNAL_LIBS, just to be sure. So far, this seems to have solved the problem for me. God, Make is going to drive me to an early grave one of these days.
I have a makefile that basically looks like this:
DIRS = a
all : $(DIRS)
DIRS += b
a b:
# echo $#
I was surprised to discover that only a gets printed. Why? I thought the whole advantage of recursive expansion was that I can expand my variables in arbitrary order for convenience. Is there a way to get the behavior I want for the prerequisites of all?
The problem is that make has already expanded $(DIRS) by the time you append to it.
From 3.7 How make Reads a Makefile:
GNU make does its work in two distinct phases. During the first phase it reads all the makefiles, included makefiles, etc. and internalizes all the variables and their values, implicit and explicit rules, and constructs a dependency graph of all the targets and their prerequisites. During the second phase, make uses these internal structures to determine what targets will need to be rebuilt and to invoke the rules necessary to do so.
It’s important to understand this two-phase approach because it has a direct impact on how variable and function expansion happens; this is often a source of some confusion when writing makefiles. Here we will present a summary of the phases in which expansion happens for different constructs within the makefile. We say that expansion is immediate if it happens during the first phase: in this case make will expand any variables or functions in that section of a construct as the makefile is parsed. We say that expansion is deferred if expansion is not performed immediately. Expansion of a deferred construct is not performed until either the construct appears later in an immediate context, or until the second phase.
...
Rule Definition
A rule is always expanded the same way, regardless of the form:
immediate : immediate ; deferred
deferred
That is, the target and prerequisite sections are expanded immediately, and the recipe used to construct the target is always deferred. This general rule is true for explicit rules, pattern rules, suffix rules, static pattern rules, and simple prerequisite definitions.
You can always add prerequisites to targets.
So you could add all: b when you know what b is to add it to the prerequisite list for all.
You could also put all: at the top (for the default target selection) and then put all: $(DIRS) at the bottom to use the full DIRS value as prerequisites.
Lastly you could use Secondary Expansion to force an extra expansion phase that should do what you want here.
.SECONDEXPANSION:
all: $$(DIRS)
In make file, for one example, $# is the name of the file being generated,
Find it difficult to remember these special variables
Is there a systematic way that this been coined?
or How this can be remembered?
No, there is no mnemonic that I can think of. The list of automatic variables is documented here. There are really only three that are typically used, so they're not that hard to remember once you get used to them: $# for the target, $< for the first prerequisite, and $^ for all the prerequisites. The others are more for special situations.
What you really need is a cheat-sheet. There are plenty available (search for makefile cheat sheet , and maybe specify the variant, eg gnu).
As an example: http://www.cheatography.com/bavo-van-achte/cheat-sheets/gnumake/
So I have a makefile with a target dependency like this:
all: $(foreach lang, $(LANGS), $(foreach models,$(MODELS),targetName$(model).xml$(lang)))
and the targetName target looks like this:
targetName%.xml%: MODEL\=% targetName.xml*
But it doesn't work. I get this error:
make[1]: *** No rule to make target `targetNameMYMODEL.xmlen', needed by `all'. Stop.
however, calling 'all' with a hardcoded language in the targetName target like this works:
all: $(foreach lang, $(LANGS), $(foreach models,$(MODELS),targetName$(model).xmlen))
and calling 'all' without the language in either all or targetName works fine as well. I'm thinking perhaps Makefile doesn't like having two percent signs in the target name.
Also, since I sort of inherited this makefile, can someone tell me what that MODEL\=% means? There is another target that looks like this: MODEL%: ; but i'm not sure what the \=% means.
Edit:
So to clarify further based on this reponse:
I have a list of, say, 5 Models and a list of say 5 Languages, the goal is to generate 25 files, one for each combination of languages and models.
targetName%.xml target would originally build a file called targetName.xml but now I need it to build something like targetName.xml That's why I needed the two % signs.
I'm cycling through these two lists with a nested foreach as shown above, but I'm having a hard time passing those variables around to targets. I've tried exporting the language variable from the foreach statement, I've tried putting the contents of the targetName%.xml in a for loop to loop through the languages there. The latter doesn't work because there are Makefile commands (like eval for example) in the 'do' portion of the loop.
You're right, Make doesn't like two % symbols in the target name. That's just one of the ways in which Make's wildcard handling isn't all one might wish for. There is a way to generate all of the pattern rules iterating over the values of one variable (and using % for the other) automatically; it's a little ugly, but if that's what you want we can show you.
But this line is a mess:
targetName%.xml%: MODEL\=% targetName.xml*
Apart from the problem of two wildcards in the target, the last preq, targetName.xml* will mean exactly that: a target (or file) called targetName.xml*. The asterisk will not be expanded, not to a list of all existing files that start with targetName.xml, or anything else.
Then there's MODEL\=%, which indicates that the author of this makefile was a little confused (and didn't believe in testing). If the value of the wildcard is foo, then this will indicate a prerequisite which is a target (or file) named MODEL=foo. The \ "escapes" the =, so that Make will not abort, but that is simply suppression of a healthy error message. I think what the author had in mind was something like this:
targetName%.xml: MODEL=%
targetName%.xml: $(wildcard targetName.xml*)
do some things
The first rule is not specifying a prerequisite, it is setting a target-specific variable. So when Make tries to build targetNamefoo.xml by doing some things, the variable MODEL will have the value foo in the context of that command. The second rule has a preq list, generated by wildcard; you can't mix target-specific variables and preqs in one line.
We can give further advice, but it would help if we could see a little more clearly what you're trying to do.
Worth noting the official documentation on the matter:
10.5 Defining and Redefining Pattern Rules
You define an implicit rule by writing a pattern rule. A pattern rule looks like an ordinary rule, except that its target contains the character % (exactly one of them). The target is considered a pattern for matching file names; the % can match any nonempty substring, while other characters match only themselves. The prerequisites likewise use % to show how their names relate to the target name.
Thus, a pattern rule %.o : %.c says how to make any file stem.o from another file stem.c.
Note that expansion using % in pattern rules occurs after any variable or function expansions, which take place when the makefile is read. See How to Use Variables, and Functions for Transforming Text.
Source: https://www.gnu.org/software/make/manual/html_node/Pattern-Rules.html
I'm working on re-writing an old build that was originally "designed" (or not, as the case may be) to be recursive. To preface, there will come a day when we'll move to something more modern, expressive, and powerful (eg, scons); however, that day is not now.
As part of this effort I'm in the process of consolidating what should be generic variables/macros & targets/recipes into a few concise rulefiles that'll be included as part of the primary build. Each sub-section of the build will use a small makefile that adds targets & dependencies with little in the way of variables being added in these sub-makefiles. The top level makefile will then include all the makefiles, allowing everything to contribute to the dependency tree.
I must admit I'm not at all confident that people will use good judgement in modifying makefiles. As an example of what I'm worried about:
CFLAGS = initial cflags
all: A.so
%.so %.o:
#echo "${#}: ${CFLAGS} : ${filter 5.o,${^}} ${filter %.c,${^}"
%.c :
true
%.o : %.c
A.so : B.so a1.o a2.o a3.o
B.so : b1.o b2.o b3.o
A.so : CFLAGS += flags specific to building A.so
Provided I didn't screw up copying that example, the situation is thus: A.so would link to B.so, and A.so's objects need special flags to be built; however, B.so and B's objects will inherit the changes to CFLAGS.
I would prefer to have a single target for building most if not all object files, even to the extent of modifying CFLAGS specifically for those those objects that need slightly different flags, in order to promote re-use of more generic targets (makes debugging easier if there's only one target/recipe to worry about).
After I finish re-architecting this build, I'm not at all confident someone won't do something stupid like this; worse, it's likely to pass peer review if I'm not around to review it.
I've been kicking around the idea of doing something like this:
% : CFLAGS = initial cflags
... which would prevent dependency poisoning unless someone then updates it with:
% : CFLAGS += some naive attempt at altering CFLAGS for a specific purpose
However, if there's, just 1000 targets (an extremely conservative estimate), and approximately 1k in memory allocated to variables, then we're up around 1mb of overhead which could significantly impact the time it takes to lookup the CFLAGS value when working through recipes (depending on gmake's architecture of course).
In short, I suppose my question is: what's a sane/good way to prevent dependency poisoning in a makefile? Is there a better strategy than what I've outlined?
edit
If anyone out there attempts to go down the path of scoping variables as described above, I ran into a nuance that wasn't entirely obvious at first.
% : INCLUDES :=
# ...
SOMEVAR := /some/path
% : INCLUDES += -I${SOMEVAR}
...
SOMEVAR :=
When a variable is created using :=, everything to the right of := should be evaluated immediately, whereas if it just used = it would delay evaluation until the target recipe evaluates INCLUDES.
However, SOMEVAR evaluates to nothing when a target recipe is evaluated. If you change the definition to:
% : INCLUDES := whatever
# ...
SOMEVAR := /some/path
% : INCLUDES := ${INCLUDES} -I${SOMEVAR}
...
SOMEVAR :=
... then it forces SOMEVAR to be evaluated immediately instead of delaying evaluation, but INCLUDES doesn't evaluate to its previously scoped definition, rather to the global definition.
$(flavor ...) says INCLUDES is simple, and $(origin ...) returns file; this occurs whether you use := or +=.
In short, if you use += on scoped variables, it'll only use the definition of the variable scoped to that target; it doesn't look at globals. If you use :=, it only uses globals.
If you abstain from unusual characters in your filenames, you can select target-specific variables with variable name substitution:
A.so_CFLAGS = flags specific to building A.so
%.so %.o:
#echo "${#}: ${CFLAGS} ${$#_CFLAGS} : ${filter %.o,${^}} ${filter %.c,${^}}"
Which obviously doesn't propagate the name of the currently built archive to the objects built for that library, but I don't know if this is desired.
This approach has some obvious deficiencies, naturally, like the inability to actually override CFLAGS. However, considering that automake has the same problem to solve and resorts to stupid text substitution, I think a lot of people already failed to find a nice solution here.
As a side-note, you might consider using automake instead of re-engineering it.