I want to auto-create makefile rules based on a variable list (in my use case a set of files which I want to selectively build). However I have issues expanding the names.
I have created the following Makefile as an example (M(not)WE):
FILES_A=a1 a2 a3
FILES_B=b1 b2 b3
TARGET_BASES=A B
define test
FILES := $(value FILES_$1)
build_$1: $(addsuffix .ext, $(FILES))
build_dummy_$1: $(FILES)
endef
$(foreach target, $(TARGET_BASES), $(eval $(call test,$(target))))
Running make -p|grep build I would expect the following output:
[...]
build_A: a1.ext a2.ext a3.ext
build_dummy_A: a1 a2 a3
build_B: b1.ext b2.ext b3.ext
build_dummy_B: b1 b2 b3
but instead I get
build_A:
build_dummy_A:
build_B: a1.ext a2.ext a3.ext
build_dummy_B: a1 a2 a3
and I do not know why. Can somebody shed some light on what I am missing here?
Understanding expansion around eval and call can be confusing. That's why they're considered advanced items which should only be used if none of the simpler to understand facilities will solve the problem.
The issue here is that your macro test is evaluated first by the call function, before being passed to the eval function. During that evaluation, the variable FILES is not not set yet and so it resolves to whatever it was set to the last time the eval ran: the first time it evaluates to empty and the second time it evaluates to the result after the first run, etc.
Basically the rule is that any variable you want to be expanded by call should have only one $, and any variable you want to be expanded by the eval needs to be escaped with $$. So:
define test
FILES := $$(value FILES_$1)
build_$1: $$(addsuffix .ext, $$(FILES))
build_dummy_$1: $$(FILES)
endef
Related
I am defining a makefile variable by parsing a json file using jq.
define variable
$(eval objects := $(shell jq -c '.abc.values[]' config.json))
endef
target_a:
$(variable)
#echo "from target-a: $(objects)"
target_b: $(foreach X, $(objects), stage_$(X))
I am getting an output that's the same as the json list:
from target-a: first second
This means the variable is getting initialized correctly but the foreach is not triggering any stage_% targets. Why is that?
Basically your makefile cannot work. Make proceeds in two distinct steps: first all makefiles are entirely parsed and an internal graph of the targets and prerequisites is created. Second make walks that graph and invokes all the recipes. All prerequisites must be defined during the first step. All recipes are only expanded during the second step.
Here, you've added the expansion of the objects variable into the recipe for target_a, so that will not happen until the second step, IF make decides to rebuild the target target_a because it's out of date. But, the definition of the prerequisites of target_b has already happened a long time ago, when make was parsing the makefile in the first step.
You can do this in your example and it will work:
objects := $(shell jq -c '.abc.values[]' config.json)
target_a:
#echo "from target-a: $(objects)"
target_b: $(foreach X, $(objects), stage_$(X))
Here, objects is expanded as the makefile is parsed and so it will have a value before target_b is expanded.
Maybe there's some reason you can't do it this way, but if so you've simplified your example too much for us to help.
In a Makefile, I am trying to populate lists by transforming items from an initial list.
As in my real code, those transformations are non-trivial, I try to use a define... endef construct, to apply to each element of the initial list, containing the logic of what I want to accomplish. Then, I apply this "function" using a foreach containing a eval and call.
But something weird happens: it seems that the last element of the list is not treated by the "function".
Here is a MWE Makefile:
libraries :=
define Function
libName = lib$(1)
libraries += $(libName)
# [...more things...]
endef
libs = a b c
$(foreach lib,$(libs),$(eval $(call Function,$(lib))))
all:
$(foreach lib,$(libs),$(lib))
#echo $(libraries)
And the result of running the command make:
a b c
liba libb
I expected the second line to have an extra item libc at its end...
What did I do wrong? What did I misunderstood?
You missed the fact that the argument you provide is expanded twice: first by call, then again as part of the eval.
You can get a better idea of what is happening with eval by replacing it with info:
$(foreach lib,$(libs),$(info $(call Function,$(lib))))
This will show you the text that eval is evaluating. You'll see that here:
libraries += $(libName)
libName is being evaluated by call, before eval sees it. So it expands to the previous run's setting of libName (or the empty string in the first run).
You need to examine your define and for every variable that is a call parameter like $(1) you use it like this, so call expands it, and for every other variable or function reference you probably want to escape it with $$ so that call doesn't expand it and it's left to eval to expand:
define Function
libName = lib$(1)
libraries += $$(libName)
# [...more things...]
endef
I have a complicated set of rules I need to write to generate a rather large number of "parameterised" output files and thought that, rather than expand them all out by hand, I could repeatedly "include" a template file with sets of rules and use (GNU)make's facility for allowing "simply expanded" variables to avoid the pain.
(In the past I've always been using the "recursively expanded" variable approach, so this is new to me)
As a trivial example of what I thought would work, I tried putting the following in a Makefile
Targ:=A
Param1:=Pa
Param2:=Qa
$(Targ):
#echo expect A, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
Targ:=B
Param1:=Pb
Param2:=Qb
$(Targ):
#echo expect B, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
Targ:=C
Param1:=Pc
Param2:=Qc
$(Targ):
#echo expect C, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
The eventual plan was to replace the rules with an include file containing dozens of different rules, each referencing the various "parameter" variables.
However, what I get is...
prompt> make A
expect A, get C, Target is A. Params are Pc and Qc
prompt> make B
expect B, get C, Target is B. Params are Pc and Qc
Essentially, unlike each rule's target, which is picking up the intended definition, the $(Targ), $(Param1), and $(Param2) in each rule's command is instead being run with the final definition.
Does anyone know how to prevent this, i.e. how do you force the command to use the definition at the time it is encountered in the Makefile?
Simple vs recursive expansion makes no difference here; regardless of which you use you'll see the same behavior. A GNU make variable is global and obviously can have only one value.
You have to understand when variables are expanded. The documentation provides a detailed description of this. Targets and prerequisites are expanded when the makefile is read in, so the value of Targ as the makefile is being parsed is used.
Recipes are expanded when the recipe is to be invoked, which is not until after all makefiles are parsed and make starts to build targets. At that time of course the variable Targ has its last set value.
Without knowing what your makefile really does it's hard to suggest an alternative. One option is to use target-specific variables:
Targ := A
$(Targ): LocalTarg := $(Targ)
$(Targ):
#echo expect A, get $(LocalTarg), Target is $#
Another option is to use constructed variable names:
Targ := A
Targ_$(Targ) := $(Targ)
$(Targ):
#echo expect A, get $(Targ_$#), Target is $#
Apologies for answering my own question, but I now realised it is possible to solve the issue I was having by running make recursively.
E.g. if the parameter variables for the rules are Targ, Param1 and Param2 then
#Set up "default" values for the parameters (As #madscientist points out,
#these will safely be overridden by the defs on the #(make) commands below
Targ=XXXXXXXXXX
Param=XXXXXXXXXX
Param2=XXXXXXXXXX
Recursing=
#
# define N (==3) templated rule(s)
#
$(Targ)%a:
#echo Run Combo_a $(Targ) $(Param1) $(Param2) $#
$(Targ)%b:
#echo Run Combo_b $(Targ) $(Param2) $(Param1) reversed $#
$(Targ)%c:
#echo Run Combo_c $(Param1) $(Targ) $(Param2) mixed again $#
#
#Enumerate "M" (==2) sets of parameters,
# (Except if we are already recursing because unrecognised targets may cause
# it to descend forever)
#
ifneq ($(Recursing), Yes)
Set1%:
#$(MAKE) Targ=Set1 Param1=foo Param2=bar Recursing=Yes $#
Set2%:
#$(MAKE) Targ=Set2 Param1=ray Param2=tracing Recursing=Yes $#
endif
This then allows N*M different combos for N+M typing cost.
eg. (removing messages from make re recursion)
>make Set1.a
Run Combo_a Set1 foo bar Set1.a
>make Set2.c
Run Combo_c ray Set2 tracing mixed again Set2.c
I am trying to build a macro in Makefile which I can expand once but can still work as a macro after being expanded. This is useful to me as the first level expansion will fill in recursive parameters that won't last. Here's an example:
all: aperiod
TGT = hello
hello.TGT = world
world.TGT = period
define CREATE_TARGET
.SECONDARY: $(1)
$(3)$(1): $(4)$(2)
#echo $$$$(#)
$(foreach t,$($(1).TGT),$(call CREATE_TARGET,$(t),$(1),$$(1),$$(1)))
endef
define CREATE
$(call CREATE_TARGET,$(TGT),,$$(1),)
endef
CREATE_EXP := $(call CREATE)
TGT :=
$(eval $(call CREATE_EXP,a))
Error when running make:
make: *** No rule to make target aperiod', needed byall'. Stop.
TGT contains a changing set of values. I want CREATE_EXP to contain the full expanded creation method which accepts a parameter to give prefixes to the targets.
So optimally, I can call make aperiod and get hello world period, or call make bperiod after $(eval $(call CREATE_EXP,b)) and get the same thing
This is a highly reduced test case!
The value of CREATE_EXP is correct, but won't work for me as a macro anymore.
$(info $(value CREATE_EXP))
.SECONDARY: hello
$(1)hello:
#echo $$(#)
.SECONDARY: world
$(1)world: $(1)hello
#echo $$(#)
.SECONDARY: period
$(1)period: $(1)world
#echo $$(#)
I would like to know why Make behaves this way, as well as if there is a better way to accomplish the general gist of what I am trying to do.
EDIT: I found a solution to accomplish this, although I am still curious as to whether a call to $(call) can create a macro that still needs expansion.
define CREATE
define CREATE_EXP
$(call CREATE_TARGET,$(TGT),,$$(1),)
endef
endef
Use $(eval $(call CREATE))
The first time through, make will expand the variables inside. This allows for the recursive expansion as well as the creation of a function macro.
I would have to think more deeply about "a better way" and really understand what you're trying to do, but to answer "why make behaves this way": you are assigning CREATE_EXP as a simply-expanded variable, with :=:
CREATE_EXP := $(call CREATE)
That information is stored along with the variable and when make expands something like $(CREATE_EXP) it knows that the value of CREATE_EXP has already been expanded and it shouldn't be expanded again. That's the entire point, really, of using :=.
Here's an alternate model that might work for you:
$(foreach 1,a,$(eval $(CREATE_EXP)))
(I haven't tried this). The difference here is that first we set the variable 1 as the foreach variable, then we call eval in that context. Although the $(CREATE_EXP) expands to the value without further expansion, then eval will parse it as a makefile and expand it again, with 1=a set.
Just a note: this:
CREATE_EXP := $(call CREATE)
Is absolutely identical to this:
CREATE_EXP := $(CREATE)
If you pass no arguments to call it's the same as a simple macro expansion.
You might be interested to read the set of blog posts here: http://make.mad-scientist.net/category/metaprogramming/ (start from the oldest first).
I am using GNU make 3.81. Here is a test makefile that demonstrates the problem:
define BOZO
a$(1): b c
touch a$(1)
endef
$(foreach i,1 2 3,$(call BOZO,$(i)))
The idea here is to use a macro template (BOZO) to generate rules that follow a predictable pattern.
Problem: when I run make on this makefile I get an error saying:
Makefile.fake:10: *** multiple target patterns. Stop.
(where line 10 is the line with the foreach).
Now, I know what that error normally indicates. Let's see what that line expands to by using the info function to send the expansion to standard out. I change line 10 to be:
$(info $(foreach i,1 2 3,$(call BOZO,$(i))))
and I run:
$ make -n
a1: b c
touch a1
a2: b c
touch a2
a3: b c
touch a3
make: *** No targets. Stop.
Note that the "no targets" message is expected, since the $(info ...) function evaluates to empty but causes make to print the generated rules.
Let's run those rules then shall we?
$make -n > out.txt
make: *** No targets. Stop.
$make -f out.txt a1 a2 a3
touch a1
touch a2
touch a3
$
AAARGH! The rules work fine. So... is the bug in make, or in my understanding?
One final clue that might help diagnose: if I change the foreach line to:
$(foreach i,1,$(call BOZO,$(i)))
(so that foreach has only one iteration)
and then do
$make a1
I get a different error:
make: *** No rule to make target `a1'. Stop.
I don't know of any way to "see" the expansion of $(foreach ) that make sees except for $(info ), and its output is legal, so I'm quite stumped.
$(foreach i,1 2 3,$(eval $(call BOZO,$(i))))
The eval function tells Make to parse the structures as makefile syntax, to "enact" them. I'm not sure why Make objected to the un-eval'd rules this particular way, but that's kind of academic.
Beta's answer is correct but I wanted to address the comment, "I'm not sure why Make objected to un un-eval'd rules".
The reason the un-eval'd rules don't work is that a makefile is ultimately line-based, and lines are chopped BEFORE variables are expanded. So it's just not possible for an expansion of a variable to turn into a multiline result: even if the expansion contains newlines make treats the entire thing as one "line". When make finishes expanding the foreach loop and parses the results it basically sees this:
a1: b c touch a1 a2: b c touch a2 a3: b c touch b3
which is why you get the "multiple target patterns" error.
The eval causes make to re-interpret the result of the expansion from the beginning as a complete snippet of makefile syntax, including the line-chopping etc., and that's why a multi-line expansion works there.
Maybe old, but always relevant.
There is no need to make this macro at all.
The solution for your problem is by simply defining this rule:
a%: b c
touch a%
The percent acts as a wildcard. See more info here:
https://stackoverflow.com/a/7406471/2522849