I have a makefile that runs some other make target by first setting some variables:
make -C somedir/ LE_VAR=/some/other/stuff LE_ANOTHER_VAR=/and/so/on
Now I need to unset LE_VAR (really unset, not just override the value with "").
Is there any way to do it so on GNU Make 3.81?
Thanks!
Assuming your makefile contains something like this to invoke a sub-make:
submake:
$(MAKE)
You need to modify the magical variable MAKEOVERRIDES, like this:
MAKEOVERRIDES := $(filter-out LE_VAR=%,$(MAKEOVERRIDES))
unexport LE_VAR
submake:
$(MAKE)
Check this out unexport variable.
From gnu manual
export variable
export variable-assignment
unexport variable
Tell make whether or not to export a particular variable to child processes
Refer https://www.gnu.org/software/make/manual/html_node/Quick-Reference.html
Thank you very much for your replies, this was quite tricky.
When executing make, and setting vars in the parameters, like:
make -C le/path install ONEVAR=one OTHERVAR=two
We have both ONEVAR and OTHERVAR on the env and the subtasks ran by the first command. This kind of puzzled me because I added to the task (at le/path) to execute a simple bash script that only did:
echo $ONEVAR
unset ONEVAR
And by my surprise the var $ONEVAR was actually "one" (so it was on the env) and the unset actually cleared it. But, adding an "echo $(ONEVAR)" on the makefile still outputs "one".. This is due to MAKEOVERRIDES, and in fact, as suggested by Communicating Options to a Sub-make:
The command line variable definitions really appear in the variable
MAKEOVERRIDES, and MAKEFLAGS contains a reference to this variable. If
you do want to pass flags down normally, but don't want to pass down
the command line variable definitions, you can reset MAKEOVERRIDES to
empty, like this:
MAKEOVERRIDES =
Or as MadScientist suggested above :)
But this was not enough, since this var was still being passed to the other subtasks below (in this situation some nodejs modules that were being compiled on a local folder, and by bad luck, both a js file from phantomjs and some other makefiles where using a var with the same name (e.g., $ONEVAR).
unexport variable Tell make whether or not to export a particular
variable to child processes.
GNU Make Appendix A Quick Reference
What I did was:
DESTDIR_BUFFER=$(DESTDIR)
MAKEOVERRIDES := $(filter-out DESTDIR=%,$(MAKEOVERRIDES))
unexport DESTDIR
And only then make npm install.
At the end of this task I export DESTDIR with the value at DESTDIR_BUFFER and all the other consequent tasks still work.
Thanks a lot for your help!
Related
I know that I can pass variables to a Makefile in two ways:
make <target> FOO=bar
export FOO=bar
make <target>
and that both of these will make a variable FOO available in the makefile with value bar.
However, is there a way to require that the variable only comes from the command line? I want the passing of variables to be explicit in order to avoid certain potential overlaps of environment variables, so I want to ensure make only receives the variable if passed from the command line, and to disregard it if it's only set in the environment so that the value it uses must be defined by the user when calling make.
EDIT: I realize after researching it a bit more that environment variables are not actually accessed the way I thought they are, they're actually used within make as ${FOO} so as long as I don't define FOO at any point in the makefile, its only set value will be from the command line (as noted in this answer).
These methods to pass variables to GNU make aren't equivalent.
Variables that come from the environment don't override the assignments in makefile, unless make is invoked with -e option. See variables from the environment. This is because depending on environment variables is poor practice in terms of build reproducibility (someone forgets to set the environment variable and the build is different):
It is not wise for makefiles to depend for their functioning on environment variables set up outside their control, since this would cause different users to get different results from the same makefile. This is against the whole purpose of most makefiles.
Variables that come from make command line do override assignments in makefile, unless override is specified. See the override directive.
Hence, the recommended practice is to explicitly set all your variables to their default values in the makefile unconditionally, so that only the assignments from the command line override them.
As far as I know, there's no difference between ${FOO} and $(FOO), regardless of the way FOO is defined.
If you're using GNU make, there's a function origin that allows you to make the distinction: it will return command line for a variable defined on the command line and environment for a variable exported by the environment (more info in the manual)
with the following Makefile:
foo ?= foo
default:
#echo ${foo}, comes from $(origin foo)
make prints foo, comes from file
make foo=bla prints bla, comes from command line
(export foo=bar; make) prints bar, comes from environment
expanding on #Virgile answer, you could add the following kind of check at the start of the makefile. It is a lot to repeat for each variable you wish to check, although all such checks could reside in a dedicated makefile that is then included from main makefile
foo ?= foo
# check origin
ifdef foo
ifneq "$(origin foo)" "command line"
$(error foo: must come from command line)
endif
else
$(error foo not defined)
endif
default:
#echo ${foo}, comes from $(origin foo)
When I run
export PATH := mypath
$(error $(shell echo "$${PATH}"))
it seems my PATH isn't changed on the call to shell.
Why is this and how do I actually change the PATH for shell calls?
Is this with GNU make? There is a long-standing GNU make feature request to honor exported variables with $(shell …). This is not specific to PATH at all, it affects (or does not affect) all export variables.
According to the GNU make sources, this is tricky to implement:
/* Using a target environment for 'shell' loses in cases like:
export var = $(shell echo foobie)
bad := $(var)
because target_environment hits a loop trying to expand $(var) to put it
in the environment. This is even more confusing when 'var' was not
explicitly exported, but just appeared in the calling environment.
See Savannah bug #10593.
envp = target_environment (NULL);
*/
The solution is simple: never ever use $(shell) or export.
Environment variables should be part of the recipe that needs them.
For $(shell) invocations that are supposed to fill a makefile variable you can use instead.
it also has the advantage to be more flexible, because you can fill more than one variable with one recipe
you can also define proper dependencies, whereas $(shell) is always executed, either when the makefile is parsed or the recursively expanded variable gets expanded.
you get build errors and recipes are logged, whereas $(shell) can make the DevOp engineers life a living h...
PATH := mypath
Makefile.variables:
#PATH=$(PATH) echo "This my path '$${PATH}'"
echo >$# "MY_DYNAMIC_CONTENT := abcd"
include Makefile.variables
$(info MY_DYNAMIC_CONTENT '$(MY_DYNAMIC_CONTENT)')
Example run:
$ make
MY_DYNAMIC_CONTENT ''
This my path 'mypath'
echo >Makefile.variables "MY_DYNAMIC_CONTENT := abcd"
MY_DYNAMIC_CONTENT 'abcd'
make: 'Makefile.variables' is up to date.
Note: This question was originally posted as a rant by a now-deleted user, but there was a valid question behind the rant; this is my attempt to provide an answer.
Given the Makefile:
ifeq "$(MAKELEVEL)" "0"
# Override the command-line specification of "foo".
override foo=replaced
export foo
all::
#echo outer: foo is "$(foo)"
#$(MAKE)
else
# Variable 'foo' was "exported" from the top-level Makefile.
all::
#echo inner: foo is "$(foo)"
endif
The expectation is that export foo will cause make to export the value defined in the override declaration. But it doesn't:
$ make -s foo=original
outer: foo is replaced
inner: foo is original
The expectation is probably reasonable, but it turns out that this is not the way Gnu make works. It could well be that the make documentation could be improved to clarify the process, but the hints all seem to be there.
How variables get their values
A variable can be set by the programmer in three ways:
On the command line with a var=value command-line argument
Explicitly in the make file
From the environment
The above list is the normal priority order; the first definition found in the list "wins". However, you can use the override directive to swap the priorities of the first two methods. (You can also use the -e flag to make to swap the priorities of the last two methods. The -e flag is required by Posix, but its use is discouraged and it does not interact will with override.)
How variables are passed to sub-makes
make is very similar to a shell in that the environment is used to pass variable values. If a variable is marked as exported, then its value is placed into the environment for any processes initiated by make, including sub-makes. As with the shell, a variable is marked as exported if its definition came from the environment or if it is explicitly marked as exported with the export directive. Variables are also exported if they were set on the command line.
However, there is another mechanism by which variables on the command-line are passed to subprocesses: the MAKEFLAGS exported variable.. MAKEFLAGS contains (most) command-line options as well as all of the command-line variable overrides. If make finds MAKEFLAGS in the environment, it merges the settings in that variable with the ones actually specified on its command line. That means that command-line variable settings in a make will also take priority in a sub-make.
Since command-line variable settings are passed through the MAKEFLAGS variable, they are not subject to any changes in the makefile. A makefile can unexport or override a variable set on the command-line, but that will only affect the value (or presence) of the variable in the environment; it does not remove or change the value from MAKEFLAGS.
Resolution
So if the intent is to override (or modify) a command-line variable both in the make itself and in the environment of sub-makes, it is necessary to use both an override and an explicit modification of MAKEFLAGS. (As explained in the make manual, MAKEFLAGS is actually recursively composed using the MAKEOVERRIDES variable, so we actually modify that variable.)
ifeq "$(MAKELEVEL)" "0"
# Override the command-line specification of "foo".
override foo=replaced
MAKEOVERRIDES += foo=replaced
all::
#echo outer: foo is "$(foo)"
#$(MAKE) -s
else
# Variable 'foo' was "exported" from the top-level Makefile.
all::
#echo inner: foo is "$(foo)"
endif
And now we get the expected result:
$ make -s foo=original
outer: foo is replaced
inner: foo is replaced
Real-life application: dealing with whitespace
The primary intention of overrides is to allow the makefile to append words to a variable possibly provided on the command line. The example provided in the gnu make manual is insisting that CFLAGS always includes the -g flag, even if it were specified on the make command line:
override CFLAGS += -g
Passing the append through to a sub-make needs a little caution; in particular, the obvious:
MAKEOVERRIDES += CFLAGS=$(CFLAGS) # Don't do this
won't work because the whitespace inside the CFLAGS variable will not be escaped when it is added to MAKEFLAGS; the result will be that MAKEFLAGS will look something like this:
-- CFLAGS=-O3 CFLAGS=-O3 -g
instead of the desired
-- CFLAGS=-O3 CFLAGS=-O3\ -g
If the value assigned to CFLAGS on the command line included whitespace, the whitespace is escaped in MAKEFLAGS. The particular escaping mechanism used is not documented, and Posix only requires that there be some mechanism; apparently, Gnu make uses backslash. It would be possible to manually backslash escape the whitespace, resulting in something like this:
# Don't do this either
MAKEOVERRIDES += CFLAGS=$(subst $(space),\ ,$(CFLAGS))
(The definition and use of space is based on an example in the gnu make manual.)
But it is actually easier to just use an append assignment in MAKEOVERRIDES, which is undocumented but appears to work. It works on the command line, too.
override CFLAGS+=-g
MAKEOVERRIDES += CFLAGS+=-g
Important Note as of make v4.1: A bit of testing revealed that the above stanza will only work if CFLAGS (or some other variable) is actually set on the command-line. I reported this bug as Savannah issue 46013, with a very simple fix in the bug report. In the meantime, if you really want to do this, use the following workaround:
override CFLAGS+=-g
MAKEOVERRIDES += CFLAGS+=-g
# This line is necessary in case there were no command-line overrides.
# In effect, it produces a command-line override, although that value
# will not be passed on to sub-makes.
MAKEFLAGS += dummy=dummy
Update May 19, 2019: Today I was informed that a fix for the bug referenced above has been committed, so it should be fixed in the next gmake release.
First of all, I want to point out that your suggestion to add to MAKEOVERRIDES, is dangerous!
And SHOULD NEVER BE DONE!!
You simply turn a recursive variable into a simple one, you will always get false results, if recursive expansion is done.
I can not believe that you got up-voted for this clearly wrong "suggestion".
And note this:
You can not even fix it with a quoted assignment, like MAKEOVERRIDES += foo=$$(bar)!!!
But, let me return to the main point of your post.
And, with which, I couldn't disagree more.
One simple example would be, if you run the very same makefile, that you have:
This one is copied verbatim, from your post:
ifeq "$(MAKELEVEL)" "0"
# Override the command-line specification of "foo".
override foo=replaced
export foo
all::
#echo outer: foo is "$(foo)"
#$(MAKE) -s
else
# Variable 'foo' was "exported" from the top-level Makefile.
all::
#echo inner: foo is "$(foo)"
endif
And running in any modern version, 4.0 and up:
# Sub-make does NOT get the value from the root-Make's command-line.
# Instead, it "inherits" the value from the root-Make's definition in the Makefile.
$ make -s foo=original -e
outer: foo is replaced
inner: foo is replaced
Now, given your assertion above:
However, there is another mechanism by which variables on the command-line are passed to subprocesses: the [MAKEFLAGS exported variable.][3]. MAKEFLAGS contains (most) command-line options as well as all of the command-line variable overrides. If make finds MAKEFLAGS in the environment, it merges the settings in that variable with the ones actually specified on its command line. That means that command-line variable settings in a make will also take priority in a sub-make.
Since command-line variable settings are passed through the MAKEFLAGS variable, they are not subject to any changes in the makefile. A makefile can unexport or override a variable set on the command-line, but that will only affect the value (or presence) of the variable in the environment; it does not remove or change the value from MAKEFLAGS.
You should get:
outer: foo is replaced
inner: foo is original
In other words, we should get for the sub-make, the value defined on the command-line (original)!
Because, you said yourself:
A makefile can't unexport or override a variable set on the command-line.
So, here, when we empower the environment over the makefile, which means that the makefile has less "power" in the total scheme of things. Right?
Sure, for such a case, you assertion will hold even stronger.
I am unable to prevent make from communicating any variables to a submake. I've read the manual and I've followed their advice (resetting MAKEOVERRIDES and MAKEFLAGS) but it's still not working has I think it should.
Consider the following prototype Makefile:
${warning $(MAKEOVERRIDES)}
${warning $(MAKEFLAGS)}
${warning $(VAR)}
none:
$(MAKE) -f Makefile MAKEOVERRIDES= MAKEFLAGS= all
all:
echo done!
If I make VAR=10 none, I get the following:
Makefile:2: VAR=10
Makefile:3:
Makefile:4: 10
make -f Makefile MAKEOVERRIDES= MAKEFLAGS= all
make[1]: Entering directory `/home/adriano/sandbox/makes'
Makefile:2:
Makefile:3:
Makefile:4: 10
echo done!
done!
make[1]: Leaving directory `/home/adriano/sandbox/makes'
Meaning that make is communication VAR to the submake. Is this the correct behaviour?
I've tried unexport VAR and bash -c make ... without any luck.
EDIT: I've modified none's recipe to: bash -c "echo $$MAKEOVERRIDES $$MAKEFLAGS $$VAR" ; make ...
This way I found out that VAR is actually being passed through the environment that make creates for the commands to be executed and not through the other variables (the other variables are also passed this way to make).
I think my question now is: how can I create a fresh shell/environment to run my sub make?
EDIT: Someone asked why am I trying to this; I'll try to answer to that here.
I have a "module" which uses a variable named CONFIG. In order to build this module I need to build another partially unrelated "module" which also uses CONFIG, but with a different value. The problem is that when I try to build the "sub-module" CONFIG contains the value of the "super-module." I could specify CONFIG when making the "sub-module" however both modules use many variables with the same name and trying to specify them all would make the modules tightly coupled which is something I cannot afford.
How can this be so difficult...
This is wrong:
none:
$(MAKE) -f Makefile MAKEOVERRIDES= MAKEFLAGS= all
These variables (MAKEOVERRIDES and MAKEFLAGS) are set in the environment by the parent make to be passed down to the sub-makes. Setting overrides on these values inside the recipe won't help, because make has to set the environment for the recipe before it actually starts the commands in the recipe (of course).
You have to override/remove these values in the parent makefile, so that those changes are seen by the parent make before it constructs the sub-make's environment:
MAKEOVERRIDES =
none:
$(MAKE) -f Makefile all
There's no perfect way to do this. However, you can play a trick that will work most of the time:
unexport $(shell echo '$(MAKEOVERRIDES)' | sed 's/=[^ ]*//g')
MAKEOVERRIDES =
The first line tries to unexport all the variables in MAKEOVERRIDES and the second line resets MAKEOVERRIDES. There are a few issues with this. One is that if MAKEOVERRIDES is empty, it will use "unexport" by itself which unexports everything. That can be easily worked around by sticking some bogus variable before the shell function. The other is that if any variable's value contains whitespace, the expansion will consider it a variable to be unexported. That's probably OK, but it's odd.
I can't think of any better way to do it.
You don't really say why you want to do this. Have you considered doing something different, such as running the commands where you want to have a "vanilla" environment using env; for example if you want to run a command with a limited and specific set of env vars, you can run:
test:
env -i PATH='$(PATH)' LANG='$(LANG)' runMyCommand --with --my arguments
Unfortunately some versions of env use - instead of -i; check your man page.
Alternatively, you can try to start a login shell which will re-read the user's shell setup environment from scratch:
test:
/bin/sh -lc 'runMyCommand --with --my arguments'
EDIT: It's difficult because what you're asking to do (restrict the environment of the sub-make) is tricky.
Luckily based on your description, it doesn't seem necessary. Make has a hierarchy of importance for finding variable values. The command line is the highest level (well, there's override but we'll ignore that). After that comes variables set in the makefile itself. And last and lowest comes variables imported from the environment (well, default variables are even lower but we'll ignore that too).
So if your goal is to allow the variables in the sub-makes to not be affected by command line variables given to the upper-level makes, then all this rigmarole of getting the variables out of the environment is not necessary. Variables set in the sub-makefiles will take precedence over the values in the environment. So all you have to do is get rid of the variables set on the command line, which I've already shown how to do above, by setting MAKEOVERRIDES.
Can I pass variables to a GNU Makefile as command line arguments? In other words, I want to pass some arguments which will eventually become variables in the Makefile.
You have several options to set up variables from outside your makefile:
From environment - each environment variable is transformed into a makefile variable with the same name and value.
You may also want to set -e option (aka --environments-override) on, and your environment variables will override assignments made into makefile (unless these assignments themselves use the override directive . However, it's not recommended, and it's much better and flexible to use ?= assignment (the conditional variable assignment operator, it only has an effect if the variable is not yet defined):
FOO?=default_value_if_not_set_in_environment
Note that certain variables are not inherited from environment:
MAKE is gotten from name of the script
SHELL is either set within a makefile, or defaults to /bin/sh (rationale: commands are specified within the makefile, and they're shell-specific).
From command line - make can take variable assignments as part of his command line, mingled with targets:
make target FOO=bar
But then all assignments to FOO variable within the makefile will be ignored unless you use the override directive in assignment. (The effect is the same as with -e option for environment variables).
Exporting from the parent Make - if you call Make from a Makefile, you usually shouldn't explicitly write variable assignments like this:
# Don't do this!
target:
$(MAKE) -C target CC=$(CC) CFLAGS=$(CFLAGS)
Instead, better solution might be to export these variables. Exporting a variable makes it into the environment of every shell invocation, and Make calls from these commands pick these environment variable as specified above.
# Do like this
CFLAGS=-g
export CFLAGS
target:
$(MAKE) -C target
You can also export all variables by using export without arguments.
The simplest way is:
make foo=bar target
Then in your makefile you can refer to $(foo). Note that this won't propagate to sub-makes automatically.
If you are using sub-makes, see this article: Communicating Variables to a Sub-make
Say you have a makefile like this:
action:
echo argument is $(argument)
You would then call it make action argument=something
From the manual:
Variables in make can come from the environment in which make is run. Every environment variable that make sees when it starts up is transformed into a make variable with the same name and value. However, an explicit assignment in the makefile, or with a command argument, overrides the environment.
So you can do (from bash):
FOOBAR=1 make
resulting in a variable FOOBAR in your Makefile.
It seems command args overwrite environment variable.
Makefile:
send:
echo $(MESSAGE1) $(MESSAGE2)
Example run:
$ MESSAGE1=YES MESSAGE2=NG make send MESSAGE2=OK
echo YES OK
YES OK
There's another option not cited here which is included in the GNU Make book by Stallman and McGrath (see http://www.chemie.fu-berlin.de/chemnet/use/info/make/make_7.html). It provides the example:
archive.a: ...
ifneq (,$(findstring t,$(MAKEFLAGS)))
+touch archive.a
+ranlib -t archive.a
else
ranlib archive.a
endif
It involves verifying if a given parameter appears in MAKEFLAGS. For example .. suppose that you're studying about threads in c++11 and you've divided your study across multiple files (class01, ... , classNM) and you want to: compile then all and run individually or compile one at a time and run it if a flag is specified (-r, for instance). So, you could come up with the following Makefile:
CXX=clang++-3.5
CXXFLAGS = -Wall -Werror -std=c++11
LDLIBS = -lpthread
SOURCES = class01 class02 class03
%: %.cxx
$(CXX) $(CXXFLAGS) -o $#.out $^ $(LDLIBS)
ifneq (,$(findstring r, $(MAKEFLAGS)))
./$#.out
endif
all: $(SOURCES)
.PHONY: clean
clean:
find . -name "*.out" -delete
Having that, you'd:
build and run a file w/ make -r class02;
build all w/ make or make all;
build and run all w/ make -r (suppose that all of them contain some certain kind of assert stuff and you just want to test them all)
If you make a file called Makefile and add a variable like this $(unittest)
then you will be able to use this variable inside the Makefile even with wildcards
example :
make unittest=*
I use BOOST_TEST and by giving a wildcard to parameter --run_test=$(unittest)
then I will be able to use regular expression to filter out the test I want my Makefile
to run
export ROOT_DIR=<path/value>
Then use the variable, $(ROOT_DIR) in the Makefile.