Given a makefile, with a target-specific defintion:
# A target-specific definition for both: 'all' and 'x'.
all : foo += target
x : foo += target
all : x ;
x ::
#echo '$(foo)'
Running, I get:
# override Makefile-level variables, with a command-line definition.
$ make foo=cmd
cmd
Now, the same makefile, as above, but with a pattern-specific defintion:
# A pattern-specific definition that matches both targets: 'all' and 'x'.
% : foo += target
all : x ;
x ::
#echo '$(foo)'
Running, I get:
# override Makefile-level variables, with a command-line definition.
$ make foo=cmd
cmd cmd cmd
Now, both makefiles are almost identical, except that:
For version 1 of Makefile: (With target-specific definition), we had:
all : foo += target
x : foo += target
For version 2 of Makefile (With pattern-specific definition), we had:
% : foo += target
In both cases, we had basically the same configuration:
A pattern/target definition that appends to a previous value.
The pattern/target definition applies both to all and x targets.
We had a command-line definition that is suppose to override the makefile-level definition.
Now, interpreting or implementing that override, Make uses a very different methodology, for version 1 (target definition) than version 2 (pattern definition).
Let's see:
For target definition, Make completely ignores ANY value defined in the makefile. Hence, the final value is: cmd.
For pattern definition, however, Make comes up with "conciliatory" approach, trying to satisfy both, the makefile-level definition and the command-line defintion, so it:
Accepts that the final value is cmd, per the command-line that overrides the makefile definition.
But, because the makefile implies that the "patterns" are supposed to "append" additional values - never mind that those same values had been overridden - Make will still "obey" those appends, but it will done with different values (i.e. the "winning" command-line value), thereby reaching a "compromise" where nor does the makefile, or the command-line have sole control over the final value of the variable.
Well, these different approaches that Make takes to implement a command-line override, (per pattern vs target specific variables), is even more obvious when we expect the very different final results we get for them, respectively:
cmd (for target-specific).
cmd cmd cmd (for pattern-specific).
Is this justified? And how so?
This appears to be a minimal complete example:
%: foo += targ
x:
#echo $(foo)
Invoked as make foo=cmd (using GNUMake v3.81), this produces
cmd cmd
This looks like a bug to me. According to the manual:
Variables provided on the command line (and in the environment if the
‘-e’ option is in force) will take precedence [over target-specific variables].
So the += statement should have no effect. But apparently if the assignment is in a pattern rule, the command-line value overrules the value which the rule tries to append, but not the act of appending it, so that Make appends $(foo) to itself.
Related
I know that I can pass variables to a Makefile in two ways:
make <target> FOO=bar
export FOO=bar
make <target>
and that both of these will make a variable FOO available in the makefile with value bar.
However, is there a way to require that the variable only comes from the command line? I want the passing of variables to be explicit in order to avoid certain potential overlaps of environment variables, so I want to ensure make only receives the variable if passed from the command line, and to disregard it if it's only set in the environment so that the value it uses must be defined by the user when calling make.
EDIT: I realize after researching it a bit more that environment variables are not actually accessed the way I thought they are, they're actually used within make as ${FOO} so as long as I don't define FOO at any point in the makefile, its only set value will be from the command line (as noted in this answer).
These methods to pass variables to GNU make aren't equivalent.
Variables that come from the environment don't override the assignments in makefile, unless make is invoked with -e option. See variables from the environment. This is because depending on environment variables is poor practice in terms of build reproducibility (someone forgets to set the environment variable and the build is different):
It is not wise for makefiles to depend for their functioning on environment variables set up outside their control, since this would cause different users to get different results from the same makefile. This is against the whole purpose of most makefiles.
Variables that come from make command line do override assignments in makefile, unless override is specified. See the override directive.
Hence, the recommended practice is to explicitly set all your variables to their default values in the makefile unconditionally, so that only the assignments from the command line override them.
As far as I know, there's no difference between ${FOO} and $(FOO), regardless of the way FOO is defined.
If you're using GNU make, there's a function origin that allows you to make the distinction: it will return command line for a variable defined on the command line and environment for a variable exported by the environment (more info in the manual)
with the following Makefile:
foo ?= foo
default:
#echo ${foo}, comes from $(origin foo)
make prints foo, comes from file
make foo=bla prints bla, comes from command line
(export foo=bar; make) prints bar, comes from environment
expanding on #Virgile answer, you could add the following kind of check at the start of the makefile. It is a lot to repeat for each variable you wish to check, although all such checks could reside in a dedicated makefile that is then included from main makefile
foo ?= foo
# check origin
ifdef foo
ifneq "$(origin foo)" "command line"
$(error foo: must come from command line)
endif
else
$(error foo not defined)
endif
default:
#echo ${foo}, comes from $(origin foo)
I've read in linux Makefile:
$(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make
$(Q)#:
What is the meaning of $(Q)#:?
I'm trying to google it, but google always crappy if the search using some weird character. So in the end i can't found any Manual about it.
After looking in the code. Q is defined somewhere after those line. Since makefile have peculiar concept of variable (which is expandable), it can be implement in anywhere. Q is used to whether show message or not (Q maybe for Quiet).
ifeq ($(KBUILD_VERBOSE),1)
quiet =
Q =
else
quiet=quiet_
Q = #
endif
And for the last #: this means do-nothing-output-nothing.
So the conclusion $(Q)#: simply do-nothing-output-nothing.
To reinforce and expand on what nafsaka found:
Sometimes Makefiles are written like this:
target:
rm -rf $(DIRECTORY)
$(Q)$(MAKE) all
And Q will be defined as # or nothing for example:
V ?= 0
ifeq ($(V), 0)
Q = #
else
Q =
endif
If a target action is preceded by # then make won't display it when run. Here's the GNU make documentation on that subject: Recipe Echoing
In this case you need to define V=1 before running make to see commands as they're run (This is very common).
Another wrinkle: Look for "include file.mk" statements in your Makefile, which is where V and Q were defined in my case. Here's the GNU make documentation on include: Including Other Makefiles
From the Make manual:
5.2 Recipe Echoing
Normally make prints each line of the recipe before it is executed. We call this echoing because it gives the appearance that you are typing the lines yourself.
When a line starts with ‘#’, the echoing of that line is suppressed. The ‘#’ is discarded before the line is passed to the shell. Typically you would use this for a command whose only effect is to print something, such as an echo command to indicate progress through the makefile:
#echo About to make distribution files
When make is given the flag ‘-n’ or ‘--just-print’ it only echoes most recipes, without executing them. See Summary of Options. In this case even the recipe lines starting with ‘#’ are printed. This flag is useful for finding out which recipes make thinks are necessary without actually doing them.
The ‘-s’ or ‘--silent’ flag to make prevents all echoing, as if all recipes started with ‘#’. A rule in the makefile for the special target .SILENT without prerequisites has the same effect (see Special Built-in Target Names).
Note: This question was originally posted as a rant by a now-deleted user, but there was a valid question behind the rant; this is my attempt to provide an answer.
Given the Makefile:
ifeq "$(MAKELEVEL)" "0"
# Override the command-line specification of "foo".
override foo=replaced
export foo
all::
#echo outer: foo is "$(foo)"
#$(MAKE)
else
# Variable 'foo' was "exported" from the top-level Makefile.
all::
#echo inner: foo is "$(foo)"
endif
The expectation is that export foo will cause make to export the value defined in the override declaration. But it doesn't:
$ make -s foo=original
outer: foo is replaced
inner: foo is original
The expectation is probably reasonable, but it turns out that this is not the way Gnu make works. It could well be that the make documentation could be improved to clarify the process, but the hints all seem to be there.
How variables get their values
A variable can be set by the programmer in three ways:
On the command line with a var=value command-line argument
Explicitly in the make file
From the environment
The above list is the normal priority order; the first definition found in the list "wins". However, you can use the override directive to swap the priorities of the first two methods. (You can also use the -e flag to make to swap the priorities of the last two methods. The -e flag is required by Posix, but its use is discouraged and it does not interact will with override.)
How variables are passed to sub-makes
make is very similar to a shell in that the environment is used to pass variable values. If a variable is marked as exported, then its value is placed into the environment for any processes initiated by make, including sub-makes. As with the shell, a variable is marked as exported if its definition came from the environment or if it is explicitly marked as exported with the export directive. Variables are also exported if they were set on the command line.
However, there is another mechanism by which variables on the command-line are passed to subprocesses: the MAKEFLAGS exported variable.. MAKEFLAGS contains (most) command-line options as well as all of the command-line variable overrides. If make finds MAKEFLAGS in the environment, it merges the settings in that variable with the ones actually specified on its command line. That means that command-line variable settings in a make will also take priority in a sub-make.
Since command-line variable settings are passed through the MAKEFLAGS variable, they are not subject to any changes in the makefile. A makefile can unexport or override a variable set on the command-line, but that will only affect the value (or presence) of the variable in the environment; it does not remove or change the value from MAKEFLAGS.
Resolution
So if the intent is to override (or modify) a command-line variable both in the make itself and in the environment of sub-makes, it is necessary to use both an override and an explicit modification of MAKEFLAGS. (As explained in the make manual, MAKEFLAGS is actually recursively composed using the MAKEOVERRIDES variable, so we actually modify that variable.)
ifeq "$(MAKELEVEL)" "0"
# Override the command-line specification of "foo".
override foo=replaced
MAKEOVERRIDES += foo=replaced
all::
#echo outer: foo is "$(foo)"
#$(MAKE) -s
else
# Variable 'foo' was "exported" from the top-level Makefile.
all::
#echo inner: foo is "$(foo)"
endif
And now we get the expected result:
$ make -s foo=original
outer: foo is replaced
inner: foo is replaced
Real-life application: dealing with whitespace
The primary intention of overrides is to allow the makefile to append words to a variable possibly provided on the command line. The example provided in the gnu make manual is insisting that CFLAGS always includes the -g flag, even if it were specified on the make command line:
override CFLAGS += -g
Passing the append through to a sub-make needs a little caution; in particular, the obvious:
MAKEOVERRIDES += CFLAGS=$(CFLAGS) # Don't do this
won't work because the whitespace inside the CFLAGS variable will not be escaped when it is added to MAKEFLAGS; the result will be that MAKEFLAGS will look something like this:
-- CFLAGS=-O3 CFLAGS=-O3 -g
instead of the desired
-- CFLAGS=-O3 CFLAGS=-O3\ -g
If the value assigned to CFLAGS on the command line included whitespace, the whitespace is escaped in MAKEFLAGS. The particular escaping mechanism used is not documented, and Posix only requires that there be some mechanism; apparently, Gnu make uses backslash. It would be possible to manually backslash escape the whitespace, resulting in something like this:
# Don't do this either
MAKEOVERRIDES += CFLAGS=$(subst $(space),\ ,$(CFLAGS))
(The definition and use of space is based on an example in the gnu make manual.)
But it is actually easier to just use an append assignment in MAKEOVERRIDES, which is undocumented but appears to work. It works on the command line, too.
override CFLAGS+=-g
MAKEOVERRIDES += CFLAGS+=-g
Important Note as of make v4.1: A bit of testing revealed that the above stanza will only work if CFLAGS (or some other variable) is actually set on the command-line. I reported this bug as Savannah issue 46013, with a very simple fix in the bug report. In the meantime, if you really want to do this, use the following workaround:
override CFLAGS+=-g
MAKEOVERRIDES += CFLAGS+=-g
# This line is necessary in case there were no command-line overrides.
# In effect, it produces a command-line override, although that value
# will not be passed on to sub-makes.
MAKEFLAGS += dummy=dummy
Update May 19, 2019: Today I was informed that a fix for the bug referenced above has been committed, so it should be fixed in the next gmake release.
First of all, I want to point out that your suggestion to add to MAKEOVERRIDES, is dangerous!
And SHOULD NEVER BE DONE!!
You simply turn a recursive variable into a simple one, you will always get false results, if recursive expansion is done.
I can not believe that you got up-voted for this clearly wrong "suggestion".
And note this:
You can not even fix it with a quoted assignment, like MAKEOVERRIDES += foo=$$(bar)!!!
But, let me return to the main point of your post.
And, with which, I couldn't disagree more.
One simple example would be, if you run the very same makefile, that you have:
This one is copied verbatim, from your post:
ifeq "$(MAKELEVEL)" "0"
# Override the command-line specification of "foo".
override foo=replaced
export foo
all::
#echo outer: foo is "$(foo)"
#$(MAKE) -s
else
# Variable 'foo' was "exported" from the top-level Makefile.
all::
#echo inner: foo is "$(foo)"
endif
And running in any modern version, 4.0 and up:
# Sub-make does NOT get the value from the root-Make's command-line.
# Instead, it "inherits" the value from the root-Make's definition in the Makefile.
$ make -s foo=original -e
outer: foo is replaced
inner: foo is replaced
Now, given your assertion above:
However, there is another mechanism by which variables on the command-line are passed to subprocesses: the [MAKEFLAGS exported variable.][3]. MAKEFLAGS contains (most) command-line options as well as all of the command-line variable overrides. If make finds MAKEFLAGS in the environment, it merges the settings in that variable with the ones actually specified on its command line. That means that command-line variable settings in a make will also take priority in a sub-make.
Since command-line variable settings are passed through the MAKEFLAGS variable, they are not subject to any changes in the makefile. A makefile can unexport or override a variable set on the command-line, but that will only affect the value (or presence) of the variable in the environment; it does not remove or change the value from MAKEFLAGS.
You should get:
outer: foo is replaced
inner: foo is original
In other words, we should get for the sub-make, the value defined on the command-line (original)!
Because, you said yourself:
A makefile can't unexport or override a variable set on the command-line.
So, here, when we empower the environment over the makefile, which means that the makefile has less "power" in the total scheme of things. Right?
Sure, for such a case, you assertion will hold even stronger.
Given a Makefile:
ifeq "$(MAKELEVEL)" "0"
0 ::
#$(MAKE)
else
1 ::
#echo 'foo is: "$(foo)"'
endif
And executing, we get:
$ make foo='$#'
make[1]: Entering directory '/home/myname'
foo is: "1"
make[1]: Leaving directory '/home/myname'
$ make foo='$#' --environment-overrides
make[1]: Entering directory '/home/myname'
foo is: "0"
make[1]: Leaving directory '/home/myname'
So we have here a recursive variable foo with the value: $#, which - of course - expands to the name of the target. Now, we have two options here:
Either, Make expands first the variable, and then export the variable to a sub-make.
With this "logic", when Make runs the first makefile (MAKELEVEL = 0), it will build the target 0, Hence: expand the variable foo(and its value:$#) to 0, and then export - this already expanded value - to the sub-make.
This result, with the sub-make running its makefile, with a variable foo that has the simple value: 0.
This is in-fact the case, when we run make --environment-overrides, as you can see in the second run of the makefile, in the example above.
Another "logic" is, for Make to pass the value "verbatim". That means, with no expansion!
Hence all recursive variables will be still intact, when passed to the second Make.
For this logic, only the sub-make is allowed to expand its variables recursively, hence: any recursive-expansion will be done in the context of the sub-make.
In our example above, that we had foo with its value $#, if we are to follow this logic, Make will pass the value $# "verbatim", with no expansion at all, so the sub-make will effectively see a value $# for its foo variable, hence: when expanding in the context of its target, that happens to be 1, it will recursively expand foo(and its value: $#) to 1.
Actually, this is the "normal" behaviour that is evident in the first run of makefile, as evident in the example above.
So, for a lack of clear methodology, we are left to conclude, that this behaviour of either expand and then export or export and then expand is inconsistent.
That is, sometimes Make will choose the first method, where sometimes it will chose the second.
In our example, it was a command-line option (--environment--overrides), that acted as a deciding factor, as to what method Make has to choose.
But, can we really justify, that these - seemingly - unrelated features (i.e export/recursive vs. environment-overrides), will end up to have such a dramatic effect on each-other?
(Versions note: 4.0 and up).
make does export foo in expanded form, as can be seen here:
target:
#echo "'$$foo'"
Output:
$make foo='$#'
'target'
However, to pass its argument to sub-makes, it does not use the environment directly -- instead it stuffs everything into the variable MAKEFLAGS. And there it passes foo unexpanded:
target:
#echo "'$$MAKEFLAGS'"
Output:
$make foo='$#'
' -- foo=$$#'
To be clear: the sub-make does import foo from the environment, but that is then overridden by the setting from MAKEFLAGS.
When you specify --environment--overrides, that means that environment variables take precedence over settings in the makefile. The GNU make documentation does not explicitly specify what happens to variables passed in via MAKEFLAGS in the presence of --environment--overrides, but apparently they are overridden too.
From the docs:
Target-specific variables have the same priority as any other
makefile variable. Variables provided on the command line (and in the
environment if the '-e' option is in force) will take precedence.
Specifying the 'override' directive will allow the target-specific
variable value to be preferred.
So, a simple makefile, like:
# A pattern-specific variable assignment.
% : foo += file
all : x ;
# Target is a double-colon w/o dependencies, so Make will ALWAYS run its commands.
x ::
#echo '$(foo)'
Running, we get:
# Override makefile-level variables, with a command-line assignment.
$ make foo=cmd
cmd cmd cmd
# Set the value in the environment, And tell Make to prefer it over any makefile-level definitions.
$ foo=env make --environment-overrides
env file file
Returning now, to the quote above, from the documentation:
Variables provided on the command line (and in the environment if the '-e' option is in force) will take precedence.
It seems, that using either:
Command-line assignment.
Environment-set variables, AND using -e (--environment-overrides).
Have both the same effect, i.e. overrides the file-level (makefile) variable.
But, the results differ greatly. Remember that the value given in the command-line was: cmd, and the value given in the environment was: env.
Now, compare the values, given for a command-line override vs. an environment override:
cmd cmd cmd (for command-line override).
env file file (for environment override).
So, whereas for command-line, Make repeats the same value, i.e. cmd, 3 times, for environment-override, the situation is different. That is, Make will "repeat" the environment-level value: env only 1 time, and then repeats - none other - than the overridden file-level value: file.
Now, not only is the situation completely different for an "override" from command-line vs. an "override" from the environment, which is strange by itself, the problem here is much bigger.
Since, Make rules to give "priority" for a command-line (or environment) value, why does it insist to append "other" values (as in the case of environment-override, where Make appends "file file"), or in the case of a command-line override (where Make repeats the same value ***3* times). Seriously?
How does it make sense at all? And what is the justification for these inconsistent and strange results?
I believe the answer here is related to the answer to this other question of yours. (And possibly a bug in the env override version.)
The global variables and the target-specific variables are distinct variables.
The cmd cmd cmd result is because when you write %: foo += file make stores that as an addition to the current value of variable foo of the target-specific value of the variable foo which is file.
However, when you set foo on the command line make overrides the value of the target-specific variable foo to be cmd instead of file. So when make concats the variable each time it gets cmd cmd cmd.
That explanation should, I think, get you env env env then and I'm not sure why it doesn't. This could be a bug or it could be some other detail about how env override variables and target-specific variable values work. I'm not sure.
(Check the output of make -p for both these cases to see what I mean about the target-specific variable's value.)