I am working on a make-script for a hobby OS project. When playing with the script I noticed that two different variable expansions of (almost) the same variable yields different results (even though they are placed 'directly' after each other). I'll provide the important part of the makefile and the result when ran.
Makefile:
####################
# KERNEL #
####################
.PHONY: kernel
KERNEL_OBJS = $(patsubst %.c,%.o,$(wildcard kernel/*.c))
KERNEL_OBJS += $(patsubst %.asm,%.o,$(wildcard kernel/*.asm))
KERNEL_OBJS += $(DRIVER_OBJS)
KERNEL_NAME = kernel32.elf
kernel: $(KERNEL_OBJS)
#echo $^
#echo $(KERNEL_OBJS)
####################
# DRIVERS #
####################
.PHONY: drivers
DRIVER_OBJS := $(patsubst %.c,%.o,$(wildcard drivers/*/*.c))
Makefile executed via terminal in the following manner (GNU Make 4.2.1):
make kernel
And this yields the following result:
kernel/kmain.o kernel/boot.o
kernel/kmain.o kernel/boot.o drivers/vga/vga.o
The output lines are of-course from the two 'echo lines' in the kernel recipe. It is worth mentioning that all variables used in this code-snippet are used here and only here in the otherwise larger make-script. Two regular suffix rules are used to build the KERNEL_OBJS but they should not alter the output here in anyway. Apart from the suffix rules, this snippet and it's variables is completely seperated from the rest of the script.
Any ideas why the two variable expansions differ? Yours, Mikael.
The big difference between the two contexts is that the prerequisite list is expanded immediately when the makefile is parsed, but the recipe is only expanded later, when make is about to build that target.
When make first parses your makefile it finds this line:
kernel: $(KERNEL_OBJS)
It expands this variable immediately. When the variable is expanded the DRIVER_OBJS variable has not been set yet, so it is the empty string and you get this:
kernel: kernel/kmain.o kernel/boot.o
Then make finishes parsing all the makefiles and as part of that, the DRIVER_OBJS variable is set... but that doesn't matter to the above line because it's already been expanded.
Now make decides it wants to build the kernel target and in order to do that it has to expand the recipe:
#echo $^
#echo $(KERNEL_OBJS)
Here $^ is the prerequisite list: kernel/kmain.o kernel/boot.o. Now KERNEL_OBJS is expanded and now DRIVER_OBJS is set, so you get the full list.
See How make Reads a Makefile for full details on when expansion happens.
You can activate second expansion and then use it in the dependency:
.SECONDEXPANSION:
kernel: $$(KERNEL_OBJS)
Related
I am defining a makefile variable by parsing a json file using jq.
define variable
$(eval objects := $(shell jq -c '.abc.values[]' config.json))
endef
target_a:
$(variable)
#echo "from target-a: $(objects)"
target_b: $(foreach X, $(objects), stage_$(X))
I am getting an output that's the same as the json list:
from target-a: first second
This means the variable is getting initialized correctly but the foreach is not triggering any stage_% targets. Why is that?
Basically your makefile cannot work. Make proceeds in two distinct steps: first all makefiles are entirely parsed and an internal graph of the targets and prerequisites is created. Second make walks that graph and invokes all the recipes. All prerequisites must be defined during the first step. All recipes are only expanded during the second step.
Here, you've added the expansion of the objects variable into the recipe for target_a, so that will not happen until the second step, IF make decides to rebuild the target target_a because it's out of date. But, the definition of the prerequisites of target_b has already happened a long time ago, when make was parsing the makefile in the first step.
You can do this in your example and it will work:
objects := $(shell jq -c '.abc.values[]' config.json)
target_a:
#echo "from target-a: $(objects)"
target_b: $(foreach X, $(objects), stage_$(X))
Here, objects is expanded as the makefile is parsed and so it will have a value before target_b is expanded.
Maybe there's some reason you can't do it this way, but if so you've simplified your example too much for us to help.
I have a complicated set of rules I need to write to generate a rather large number of "parameterised" output files and thought that, rather than expand them all out by hand, I could repeatedly "include" a template file with sets of rules and use (GNU)make's facility for allowing "simply expanded" variables to avoid the pain.
(In the past I've always been using the "recursively expanded" variable approach, so this is new to me)
As a trivial example of what I thought would work, I tried putting the following in a Makefile
Targ:=A
Param1:=Pa
Param2:=Qa
$(Targ):
#echo expect A, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
Targ:=B
Param1:=Pb
Param2:=Qb
$(Targ):
#echo expect B, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
Targ:=C
Param1:=Pc
Param2:=Qc
$(Targ):
#echo expect C, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
The eventual plan was to replace the rules with an include file containing dozens of different rules, each referencing the various "parameter" variables.
However, what I get is...
prompt> make A
expect A, get C, Target is A. Params are Pc and Qc
prompt> make B
expect B, get C, Target is B. Params are Pc and Qc
Essentially, unlike each rule's target, which is picking up the intended definition, the $(Targ), $(Param1), and $(Param2) in each rule's command is instead being run with the final definition.
Does anyone know how to prevent this, i.e. how do you force the command to use the definition at the time it is encountered in the Makefile?
Simple vs recursive expansion makes no difference here; regardless of which you use you'll see the same behavior. A GNU make variable is global and obviously can have only one value.
You have to understand when variables are expanded. The documentation provides a detailed description of this. Targets and prerequisites are expanded when the makefile is read in, so the value of Targ as the makefile is being parsed is used.
Recipes are expanded when the recipe is to be invoked, which is not until after all makefiles are parsed and make starts to build targets. At that time of course the variable Targ has its last set value.
Without knowing what your makefile really does it's hard to suggest an alternative. One option is to use target-specific variables:
Targ := A
$(Targ): LocalTarg := $(Targ)
$(Targ):
#echo expect A, get $(LocalTarg), Target is $#
Another option is to use constructed variable names:
Targ := A
Targ_$(Targ) := $(Targ)
$(Targ):
#echo expect A, get $(Targ_$#), Target is $#
Apologies for answering my own question, but I now realised it is possible to solve the issue I was having by running make recursively.
E.g. if the parameter variables for the rules are Targ, Param1 and Param2 then
#Set up "default" values for the parameters (As #madscientist points out,
#these will safely be overridden by the defs on the #(make) commands below
Targ=XXXXXXXXXX
Param=XXXXXXXXXX
Param2=XXXXXXXXXX
Recursing=
#
# define N (==3) templated rule(s)
#
$(Targ)%a:
#echo Run Combo_a $(Targ) $(Param1) $(Param2) $#
$(Targ)%b:
#echo Run Combo_b $(Targ) $(Param2) $(Param1) reversed $#
$(Targ)%c:
#echo Run Combo_c $(Param1) $(Targ) $(Param2) mixed again $#
#
#Enumerate "M" (==2) sets of parameters,
# (Except if we are already recursing because unrecognised targets may cause
# it to descend forever)
#
ifneq ($(Recursing), Yes)
Set1%:
#$(MAKE) Targ=Set1 Param1=foo Param2=bar Recursing=Yes $#
Set2%:
#$(MAKE) Targ=Set2 Param1=ray Param2=tracing Recursing=Yes $#
endif
This then allows N*M different combos for N+M typing cost.
eg. (removing messages from make re recursion)
>make Set1.a
Run Combo_a Set1 foo bar Set1.a
>make Set2.c
Run Combo_c ray Set2 tracing mixed again Set2.c
I have created a make file with list of variables. I have added all the files list and assigned to a variable in that make file.
ROLS= \
$(wildcard *.c)
The files list is more than 10000. when I try to print this variable using below rule , it is not printing all the files list.
all:
#echo $(ROLS)
How to print entire list of "c" files in a single variable.
I am using windows 7 64bit system to run this makefile.
Windows has a limit on the complete size of a command line that it will accept. There's nothing make can do about that, it's a hard limit by the operating system.
If all you want to do is actually print the content, then you can use make's internal info function instead so it doesn't try to invoke a Windows command:
all:
$(info $(ROLS))
Of course that won't help if you're trying to do something else with those files that requires a command to be invoked.
Based on additional information in comments:
ROLS= $(wildcard *.c)
target: $(ROLS)
build-target
Or if ROLS used more than once, some efficiency can be achieved with phony target
ROLS= $(wildcard *.c)
.PHONY: all-rols
all-rols: $(ROLS)
target1: all-rols
build-target1
target2: all-rols
build-target2
target3: all-rols
build-target3
Given this bit of Makefile:
# for pattern matching
$(OBJDIR) := build
# just to see if a level of indirection will work
my_dir = $(dir $(1))
$(OBJECTS) : $(OBJDIR)/% : $(HEADERS) $(SRCDIR)/% | % $(dir %) $(call my_dir,%)
#echo output-only = $|
This is a static pattern rule with order-only prerequisites.
Consider the target "build/utility/debug.js". The output of the above rule will be this:
output-only = utility/debug.js ./
The first component, "utility/debug.js", is properly copied from the stem (%).
The second component, "./", is the output of calling the dir function in the prerequisites list.
The third component, an empty string, is the output of calling my my_dir function in the prerequisites list.
If I change my_dir to this:
my_dir = $(1)
The output remains the same. If I change it to this:
my_dir = "foo"
Then make complains there is no rule to make "foo" (which is expected). It appears, then, that $(1) is not getting bound in the call to my_dir.
What's going on? Why can't I pass the stem to a function? I have a workaround that uses secondary expansion, but I want to know why I can't write my rule this way.
EDIT: I'm new to stackoverflow, forgive me if this is not the way things are done here.
I want $(1) because I am passing the stem as an argument to my_dir, as Alex pointed out.
I don't know why it was suggested I want "$". I don't believe $ by itself expands to anything in any context.
I know that automatic variables are only available in the recipe. I am not using an automatic variable in the prerequisites - I am using the stem:
Each target is matched against the target-pattern to extract a part of the target name, called the stem. This stem is substituted into each of the prereq-patterns to make the prerequisite names (one from each prereq-pattern). - the manual
The fact that the stem is available is demonstrated by the example: the stem expands to the correct value when used alone, but not when passed to a function.
As can be seen in this section of the GNU make manual, variable and function references in the list of prerequisites are immediately expanded, during the read in phase. This means, before any pattern matching is done, so the % has no special meaning yet; it is interpreted as a literal character in the two function references, $(dir %) and $(call my_dir,%), both having ./ as a result, which get merged in the reference to $| in the recipe.
I don't know of any other workaround than the one you already found, i.e. secondary expansion.
Note $1 is not a special variable that expands to anything interesting related to pattern rules (or static pattern rules). The $1 variable only has unique behavior within the context of a user-defined macro invoked by the $(call ...) function.
You wanted to use $*, not $1; $* is an automatic variable which expands to the stem of the target of the rule.
However, in all versions of make (including the POSIX standard definition of make), automatic variables (including $*, $<, $#, $^, etc.) are only available in the context of the recipe. They are not available in the context of the target or prerequisite lists. See the GNU make manual section on Automatic Variables for more details.
As you suggest, there is a GNU make-specific feature enabled by the .SECONDEXPANSION pseudo target which provides a way to avoid this limitation.
Can I pass variables to a GNU Makefile as command line arguments? In other words, I want to pass some arguments which will eventually become variables in the Makefile.
You have several options to set up variables from outside your makefile:
From environment - each environment variable is transformed into a makefile variable with the same name and value.
You may also want to set -e option (aka --environments-override) on, and your environment variables will override assignments made into makefile (unless these assignments themselves use the override directive . However, it's not recommended, and it's much better and flexible to use ?= assignment (the conditional variable assignment operator, it only has an effect if the variable is not yet defined):
FOO?=default_value_if_not_set_in_environment
Note that certain variables are not inherited from environment:
MAKE is gotten from name of the script
SHELL is either set within a makefile, or defaults to /bin/sh (rationale: commands are specified within the makefile, and they're shell-specific).
From command line - make can take variable assignments as part of his command line, mingled with targets:
make target FOO=bar
But then all assignments to FOO variable within the makefile will be ignored unless you use the override directive in assignment. (The effect is the same as with -e option for environment variables).
Exporting from the parent Make - if you call Make from a Makefile, you usually shouldn't explicitly write variable assignments like this:
# Don't do this!
target:
$(MAKE) -C target CC=$(CC) CFLAGS=$(CFLAGS)
Instead, better solution might be to export these variables. Exporting a variable makes it into the environment of every shell invocation, and Make calls from these commands pick these environment variable as specified above.
# Do like this
CFLAGS=-g
export CFLAGS
target:
$(MAKE) -C target
You can also export all variables by using export without arguments.
The simplest way is:
make foo=bar target
Then in your makefile you can refer to $(foo). Note that this won't propagate to sub-makes automatically.
If you are using sub-makes, see this article: Communicating Variables to a Sub-make
Say you have a makefile like this:
action:
echo argument is $(argument)
You would then call it make action argument=something
From the manual:
Variables in make can come from the environment in which make is run. Every environment variable that make sees when it starts up is transformed into a make variable with the same name and value. However, an explicit assignment in the makefile, or with a command argument, overrides the environment.
So you can do (from bash):
FOOBAR=1 make
resulting in a variable FOOBAR in your Makefile.
It seems command args overwrite environment variable.
Makefile:
send:
echo $(MESSAGE1) $(MESSAGE2)
Example run:
$ MESSAGE1=YES MESSAGE2=NG make send MESSAGE2=OK
echo YES OK
YES OK
There's another option not cited here which is included in the GNU Make book by Stallman and McGrath (see http://www.chemie.fu-berlin.de/chemnet/use/info/make/make_7.html). It provides the example:
archive.a: ...
ifneq (,$(findstring t,$(MAKEFLAGS)))
+touch archive.a
+ranlib -t archive.a
else
ranlib archive.a
endif
It involves verifying if a given parameter appears in MAKEFLAGS. For example .. suppose that you're studying about threads in c++11 and you've divided your study across multiple files (class01, ... , classNM) and you want to: compile then all and run individually or compile one at a time and run it if a flag is specified (-r, for instance). So, you could come up with the following Makefile:
CXX=clang++-3.5
CXXFLAGS = -Wall -Werror -std=c++11
LDLIBS = -lpthread
SOURCES = class01 class02 class03
%: %.cxx
$(CXX) $(CXXFLAGS) -o $#.out $^ $(LDLIBS)
ifneq (,$(findstring r, $(MAKEFLAGS)))
./$#.out
endif
all: $(SOURCES)
.PHONY: clean
clean:
find . -name "*.out" -delete
Having that, you'd:
build and run a file w/ make -r class02;
build all w/ make or make all;
build and run all w/ make -r (suppose that all of them contain some certain kind of assert stuff and you just want to test them all)
If you make a file called Makefile and add a variable like this $(unittest)
then you will be able to use this variable inside the Makefile even with wildcards
example :
make unittest=*
I use BOOST_TEST and by giving a wildcard to parameter --run_test=$(unittest)
then I will be able to use regular expression to filter out the test I want my Makefile
to run
export ROOT_DIR=<path/value>
Then use the variable, $(ROOT_DIR) in the Makefile.