I want to do immediate expansion of a shell command within a Makefile, but I want the shell command to have access to the environment variables within the Makefile. If I use the $(shell ...), it expands immediately, but there is no access to the variables. If I use the backquotes, the expansion is not immediate, and it causes problems for me later in the Makefile. I'm wondering if there is any way to make the backquotes expand immediately, or to pass the current environment to a $(shell) command.
For example, the following makefile:
SOME_VAR := some_val
export SOME_VAR
VAR1 := `echo $$SOME_VAR`
export VAR1
VAR2 := `echo $$VAR1`
all:
#echo VAR1=$(VAR1)
#echo VAR2=$(VAR2)
Will output:
~/tmp/t2> make
VAR1=some_val
VAR2=`echo $SOME_VAR`
Where I want it to print "VAR2=some_val". The real example is a bit more complicated (environment variables are inherited from parent makefiles, and I'm trying to use a perl script to edit the variables), but the principle is the same.
Any help is appreciated.
Is this what you want?
VAR2 := $(shell VAR1="$(VAR1)" script_that_uses_var1)
What's wrong with this?
VAR1 := $(shell echo $(SOME_VAR))
VAR2 := $(shell echo $(VAR1))
You may try to use Special Built-in Target Name: .EXPORT_ALL_VARIABLES
.EXPORT_ALL_VARIABLES:
MY_VAR = foo
test:
#echo $$MY_VAR
As I mentioned in some of the comments, my actual goal was to make the script generate filenames based on the settings the object was being compiled with. I then need another script to generate a specially formatted list of all the filenames generated (the target is an embedded system which doesn't have a JIT compiler on it). At any given time, there are over thirty settings which can potentially effect the binary, and this may be used on more than one module in the future, so I'd like something scalable.
My solution is as follows. Instead of passing the variables in, I modified my script to output a makefile-parsable string based on the settings:
-include $(SOME_MK_FILE)
$(SOME_MK_FILE) : .phony
script.pl $(SETTINGS_OF_INTEREST_LIST) > $(SOME_MK_FILE)
someFilename := $(shell script2.pl $(VAR1))
script.pl outputs a string that looks something like:
VAR1 := CONFIG_X1=$(CONFIG_X1) CONFIG_X2=$(CONFIG_X2) CONFIG_X33=$(CONFIG_X33)
and script2 outputs a filename that looks something like 'someFilename.X1_y.X2_n.elf'
and then, later on, in another rule, I have:
someobj: somedep
script3.pl $(someFilename) >> builtfiles.txt
which properly builds builtfiles.txt (which in turn is the input for yet another script...). In the end this is a workaround to the fact that make cannot pass its environement to $(shell). It's not overly pretty but it works.
John
Here you go:
export FOO=bar
Here's a page with a lot more info:
http://www.cmcrossroads.com/article/basics-getting-environment-variables-gnu-make?page=0%2C1
Related
I have a shell command where it outputs multiple lines. I want to store it in a variable in makefile for later processing in the target.
A simplified example:
I have this file called zfile1
#zfile1
some text
$echo 123
more text
$$$#&^$
more text
The makefile:
a:
#$(eval v1 = $(shell cat zfile1))
# need to process the variable here, example:
#echo "$(v1)"
# I want to prevent expansion of values in the file and print in multi-line
If you have GNU make 4.2 or above you can use the $(file <zfile1) function. See https://www.gnu.org/software/make/manual/html_node/File-Function.html
If you don't have a new-enough version of GNU make, you can't do it. Of course in your example there's no real need to put the contents of the file into a make variable at all: you can just have your recipe use the file itself. But maybe your real use-case isn't so simple.
ETA
You should never use either the make function eval or the make function shell in a recipe [1].
You can just write:
v1 := $(file <zfile1)
.ONESHELL:
a:
#echo "$(v1)"
You must have the .ONESHELL because otherwise each line of the recipe (after it expands into multiple lines) is considered a separate recipe line. Note that .ONESHELL is in effect for the entire make process so could cause other recipes to break if they rely on each line being invoked in a different shell.
Another option is to export the result into the environment, and use a shell variable like this:
export v1 := $(file <zfile1)
a:
#echo "$$v1"
There are probably better ways to do it but since you've only given us this example and not what you really want to do, that's about all we can say.
[1] There are times where it can be useful but if you have a sufficiently sophisticated requirement to need this you'll probably know how to use it.
I think you're making things too complicated.
Start by writing your recipes as proper self-contained shell scripts.
You can then either store the whole script in a file and run it from make, or you can include it directly in your makefile as a single logical line, as in the following:
a:
#v1=$$(< zfile1); \
echo $$v1
Note the need to "escape" the dollar sign by repeating it.
You could also use global make variables, depending on the actual logic of your real-world use.
When I run
export PATH := mypath
$(error $(shell echo "$${PATH}"))
it seems my PATH isn't changed on the call to shell.
Why is this and how do I actually change the PATH for shell calls?
Is this with GNU make? There is a long-standing GNU make feature request to honor exported variables with $(shell …). This is not specific to PATH at all, it affects (or does not affect) all export variables.
According to the GNU make sources, this is tricky to implement:
/* Using a target environment for 'shell' loses in cases like:
export var = $(shell echo foobie)
bad := $(var)
because target_environment hits a loop trying to expand $(var) to put it
in the environment. This is even more confusing when 'var' was not
explicitly exported, but just appeared in the calling environment.
See Savannah bug #10593.
envp = target_environment (NULL);
*/
The solution is simple: never ever use $(shell) or export.
Environment variables should be part of the recipe that needs them.
For $(shell) invocations that are supposed to fill a makefile variable you can use instead.
it also has the advantage to be more flexible, because you can fill more than one variable with one recipe
you can also define proper dependencies, whereas $(shell) is always executed, either when the makefile is parsed or the recursively expanded variable gets expanded.
you get build errors and recipes are logged, whereas $(shell) can make the DevOp engineers life a living h...
PATH := mypath
Makefile.variables:
#PATH=$(PATH) echo "This my path '$${PATH}'"
echo >$# "MY_DYNAMIC_CONTENT := abcd"
include Makefile.variables
$(info MY_DYNAMIC_CONTENT '$(MY_DYNAMIC_CONTENT)')
Example run:
$ make
MY_DYNAMIC_CONTENT ''
This my path 'mypath'
echo >Makefile.variables "MY_DYNAMIC_CONTENT := abcd"
MY_DYNAMIC_CONTENT 'abcd'
make: 'Makefile.variables' is up to date.
I have a makefile that runs some other make target by first setting some variables:
make -C somedir/ LE_VAR=/some/other/stuff LE_ANOTHER_VAR=/and/so/on
Now I need to unset LE_VAR (really unset, not just override the value with "").
Is there any way to do it so on GNU Make 3.81?
Thanks!
Assuming your makefile contains something like this to invoke a sub-make:
submake:
$(MAKE)
You need to modify the magical variable MAKEOVERRIDES, like this:
MAKEOVERRIDES := $(filter-out LE_VAR=%,$(MAKEOVERRIDES))
unexport LE_VAR
submake:
$(MAKE)
Check this out unexport variable.
From gnu manual
export variable
export variable-assignment
unexport variable
Tell make whether or not to export a particular variable to child processes
Refer https://www.gnu.org/software/make/manual/html_node/Quick-Reference.html
Thank you very much for your replies, this was quite tricky.
When executing make, and setting vars in the parameters, like:
make -C le/path install ONEVAR=one OTHERVAR=two
We have both ONEVAR and OTHERVAR on the env and the subtasks ran by the first command. This kind of puzzled me because I added to the task (at le/path) to execute a simple bash script that only did:
echo $ONEVAR
unset ONEVAR
And by my surprise the var $ONEVAR was actually "one" (so it was on the env) and the unset actually cleared it. But, adding an "echo $(ONEVAR)" on the makefile still outputs "one".. This is due to MAKEOVERRIDES, and in fact, as suggested by Communicating Options to a Sub-make:
The command line variable definitions really appear in the variable
MAKEOVERRIDES, and MAKEFLAGS contains a reference to this variable. If
you do want to pass flags down normally, but don't want to pass down
the command line variable definitions, you can reset MAKEOVERRIDES to
empty, like this:
MAKEOVERRIDES =
Or as MadScientist suggested above :)
But this was not enough, since this var was still being passed to the other subtasks below (in this situation some nodejs modules that were being compiled on a local folder, and by bad luck, both a js file from phantomjs and some other makefiles where using a var with the same name (e.g., $ONEVAR).
unexport variable Tell make whether or not to export a particular
variable to child processes.
GNU Make Appendix A Quick Reference
What I did was:
DESTDIR_BUFFER=$(DESTDIR)
MAKEOVERRIDES := $(filter-out DESTDIR=%,$(MAKEOVERRIDES))
unexport DESTDIR
And only then make npm install.
At the end of this task I export DESTDIR with the value at DESTDIR_BUFFER and all the other consequent tasks still work.
Thanks a lot for your help!
I've learned about string manipulation with bash, and more especially about substring replacement:
#! /bin/bash
VAR1="aaaa.bbbb.cccc"
VAR2="bbbb*"
echo ${VAR1%${VAR2}}
This bash script prints "aaaa.". I tried to include it in my makefile, but I can't make it work..
SHELL:=/bin/bash
VAR1="aaaa.bbbb.cccc"
VAR2="bbbb*"
all:
#echo $${VAR1%$${VAR2}}
This Makefile only prints a blank line.
I think I've misunderstood something, but can't figure out what. Any help would be really appreciated.
No need to put double quotes around VAR1 and VAR2. And you need to use export if you want to put VAR1 and VAR2 above all:
SHELL:=/bin/bash
export VAR1=aaaa.bbbb.cccc
export VAR2=bbbb*
all:
#echo $${VAR1%$${VAR2}}
The problems is that VAR1 and VAR2 are not shell variables, but Makefile variables.
To complicate matters further, each line in the Makefile recipe is executed in a separate shell process, which means the following naive attempt will fail:
all:
VAR1="aaaa.bbbb.cccc"
VAR2="bbbb*"
#echo $${VAR1%$${VAR2}}
since VAR1 is defined in one shell, VAR2 in another, and the echo in a third where neither variable is defined. You could use the following:
all:
#VAR1="aaaa.bbbb.cccc"; \
VAR2="bbbb*"; \
echo $${VAR1%$${VAR2}};
to have a single bash statement (all executed in one shell) split into multiple lines in the Makefile.
B̶a̶s̶h̶ ̶a̶n̶d̶ ̶M̶a̶k̶e̶ ̶d̶o̶ ̶n̶o̶t̶ ̶h̶a̶v̶e̶ ̶t̶h̶e̶ ̶s̶a̶m̶e̶ ̶s̶y̶n̶t̶a̶x̶.̶ ̶ ̶Y̶o̶u̶ ̶n̶e̶e̶d̶ ̶t̶o̶ You can use built-in make functions like (in this case) $(substr a,b,c)
See this
I need to execute some make rules conditionally, only if the Python installed is greater than a certain version (say 2.5).
I thought I could do something like executing:
python -c 'import sys; print int(sys.version_info >= (2,5))'
and then using the output ('1' if ok, '0' otherwise) in a ifeq make statement.
In a simple bash shell script it's just:
MY_VAR=`python -c 'import sys; print int(sys.version_info >= (2,5))'`
but that doesn't work in a Makefile.
Any suggestions? I could use any other sensible workaround to achieve this.
Use the Make shell builtin like in MY_VAR=$(shell echo whatever)
me#Zack:~$make
MY_VAR IS whatever
me#Zack:~$ cat Makefile
MY_VAR := $(shell echo whatever)
all:
#echo MY_VAR IS $(MY_VAR)
Beware of recipes like this
target:
MY_ID=$(GENERATE_ID);
echo $MY_ID;
It does two things wrong. The first line in the recipe is executed in a separate shell instance from the second line. The variable is lost in the meantime. Second thing wrong is that the $ is not escaped.
target:
MY_ID=$(GENERATE_ID); \
echo $$MY_ID;
Both problems have been fixed and the variable is useable. The backslash combines both lines to run in one single shell, hence the setting of the variable and the reading of the variable afterwords, works.
I realize the original post said how to get the results of a shell command into a MAKE variable, and this answer shows how to get it into a shell variable. But other readers may benefit.
One final improvement, if the consumer expects an "environment variable" to be set, then you have to export it.
my_shell_script
echo $MY_ID
would need this in the makefile
target:
export MY_ID=$(GENERATE_ID); \
./my_shell_script;
Hope that helps someone. In general, one should avoid doing any real work outside of recipes, because if someone use the makefile with '--dry-run' option, to only SEE what it will do, it won't have any undesirable side effects. Every $(shell) call is evaluated at compile time and some real work could accidentally be done. Better to leave the real work, like generating ids, to the inside of the recipes when possible.
Wrapping the assignment in an eval is working for me.
# dependency on .PHONY prevents Make from
# thinking there's `nothing to be done`
set_opts: .PHONY
$(eval DOCKER_OPTS = -v $(shell mktemp -d -p /scratch):/output)
With GNU Make, you can use shell and eval to store, run, and assign output from arbitrary command line invocations. The difference between the example below and those which use := is the := assignment happens once (when it is encountered) and for all. Recursively expanded variables set with = are a bit more "lazy"; references to other variables remain until the variable itself is referenced, and the subsequent recursive expansion takes place each time the variable is referenced, which is desirable for making "consistent, callable, snippets". See the manual on setting variables for more info.
# Generate a random number.
# This is not run initially.
GENERATE_ID = $(shell od -vAn -N2 -tu2 < /dev/urandom)
# Generate a random number, and assign it to MY_ID
# This is not run initially.
SET_ID = $(eval MY_ID=$(GENERATE_ID))
# You can use .PHONY to tell make that we aren't building a target output file
.PHONY: mytarget
mytarget:
# This is empty when we begin
#echo $(MY_ID)
# This recursively expands SET_ID, which calls the shell command and sets MY_ID
$(SET_ID)
# This will now be a random number
#echo $(MY_ID)
# Recursively expand SET_ID again, which calls the shell command (again) and sets MY_ID (again)
$(SET_ID)
# This will now be a different random number
#echo $(MY_ID)
Here's a bit more complicated example with piping and variable assignment inside recipe:
getpodname:
# Getting pod name
#eval $$(minikube docker-env) ;\
$(eval PODNAME=$(shell sh -c "kubectl get pods | grep profile-posts-api | grep Running" | awk '{print $$1}'))
echo $(PODNAME)
I'm writing an answer to increase visibility to the actual syntax that solves the problem. Unfortunately, what someone might see as trivial can become a very significant headache to someone looking for a simple answer to a reasonable question.
Put the following into the file "Makefile".
MY_VAR := $(shell python -c 'import sys; print int(sys.version_info >= (2,5))')
all:
#echo MY_VAR IS $(MY_VAR)
The behavior you would like to see is the following (assuming you have recent python installed).
make
MY_VAR IS 1
If you copy and paste the above text into the Makefile, will you get this? Probably not. You will probably get an error like what is reported here:
makefile:4: *** missing separator. Stop
Why: Because although I personally used a genuine tab, Stack Overflow (attempting to be helpful) converts my tab into a number of spaces. You, frustrated internet citizen, now copy this, thinking that you now have the same text that I used. The make command, now reads the spaces and finds that the "all" command is incorrectly formatted. So copy the above text, paste it, and then convert the whitespace before "#echo" to a tab, and this example should, at last, hopefully, work for you.
In the below example, I have stored the Makefile folder path to LOCAL_PKG_DIR and then use LOCAL_PKG_DIR variable in targets.
Makefile:
LOCAL_PKG_DIR := $(shell eval pwd)
.PHONY: print
print:
#echo $(LOCAL_PKG_DIR)
Terminal output:
$ make print
/home/amrit/folder
From the make manual
The shell assignment operator ‘!=’ can be used to execute a shell script and set a >variable to its output. This operator first evaluates the right-hand side, then passes >that result to the shell for execution. If the result of the execution ends in a >newline, that one newline is removed; all other newlines are replaced by spaces. The >resulting string is then placed into the named recursively-expanded variable. For >example:
hash != printf '\043'
file_list != find . -name '*.c'
source