How can I use shell parameter extension in a Makefile?
I need to get the path to the leveldb database.
What I tried so far is in my Makefile:
$(shell db=$(locate leveldb/db.h); export LEVELDB_PATH=${db%%/include/leveldb/db.h})
LEVELDB_LIBS=-L$(LEVELDB_PATH) -I$(LEVELDB_PATH)/include -lleveldb
But LEVELDB_PATH is empty.
Thanks for help.
The shell cannot export stuff into your Makefile. Try something like this instead.
db := $(shell locate leveldb/db.h)
LEVELDB_PATH:=$(patsubst %/include/leveldb/db.h,%,$(db))
... or, to spare the temp variable,
LEVELDB_PATH:=$(shell locate leveldb/db.h | sed 's%/include/leveldb/db.h$$%%')
Edit: Fix to use $(patsubst) instead of $(subst), and double the dollar sign in the sed script.
Generally speaking, whatever goes on inside $(shell ...) happens in a subprocess, which (as ever) cannot modify its parent. make sees the output of the function when it finishes, but not what happened during the execution of the shell commands.
Related
I'm looking how to export from a Makefile environment variables to be exposed in the userland environment so exporting these variables from the Makefile should be accessible from the user shell.
I have tried make's export but as I understand and have tried does not export to outside of Makefile.
The idea of this is to populate Docker Compose environment variables in a elegant way and have these variables ready to use in the user shell also.
This is a fragment of what I've tried with make's export:
include docker.env
export $(shell sed -n '/=/p' docker.env)
SHELL := /bin/bash
run:
#docker-compose -f my-service.yml up -d
According with ArchWiki, each process of Bash...
Each process stores their environment in the /proc/$PID/environ file.
so once Make execute a source, export or any other command to set a new environment variable it will be applied only for that process.
As workaround I've written in the bash startup file so the variables will be in the global environment as soon as a new bash shell is loaded:
SHELL := /bin/bash
RC := ~/.bashrc
ENV := $(shell sed -n '/=/p' docker.env)
test:
#$(foreach e,$(ENV),echo $(e) >> $(RC);) \
EDIT completely reworked the answer after the OP explained in a comment that he wants the environment variables to be defined for any user shell.
If your goal is to have a set of environment variables defined for any user shell (I assume this means interactive shell), you can simply add these definitions to the shell's startup file (.bashrc for bash). From GNU make manual:
Variables in make can come from the environment in which make is run.
Every environment variable that make sees when it starts up is
transformed into a make variable with the same name and value.
However, an explicit assignment in the makefile, or with a command
argument, overrides the environment. (If the ‘-e’ flag is specified,
then values from the environment override assignments in the makefile.
See Summary of Options. But this is not recommended practice.)
Example:
$ cat .bashrc
...
export FOOBAR=foobar
export BARFOO="bar foo"
...
$ cat Makefile
all:
#printf '$$(FOOBAR)=%s\n' '$(FOOBAR)'
#printf 'FOOBAR='; printenv FOOBAR
#printf '$$(BARFOO)=%s\n' '$(BARFOO)'
#printf 'BARFOO='; printenv BARFOO
$ make
$(FOOBAR)=foobar
FOOBAR=foobar
$(BARFOO)=bar foo
BARFOO=bar foo
If you want to keep these definitions separate, you can just source the file from .bashrc:
$ cat docker.env
export FOOBAR=foobar
export BARFOO="bar foo"
$ cat .bashrc
...
source <some-path>/docker.env
...
And finally, if you don't want to add the export bash command to your file, you can parse the file in your .bashrc:
$ cat docker.env
FOOBAR=foobar
BARFOO="bar foo"
$ cat .bashrc
...
while read -r line; do
eval "export $$line"
done < <(sed -n '/=/p' <some-path>/docker.env)
...
Of course, there are some constraints for the syntax of your docker.env file (no unquoted special characters, no spaces in variable names, properly quoted values...) If your syntax is not bash-compatible it is time to ask another question about parsing this specific syntax and converting it into bash-compatible syntax.
Make cannot change the calling shell's environment without its cooperation. Of course, if you are in control, you can make the calling shell cooperate.
In broad terms, you could replace the make command with a shell alias or function which runs the real make and also sets the environment variables from the result. I will proceed to describe in more detail one way to implement this.
Whether you call this alias or function of yours make or e.g. compose is up to you really. To wrap the real make is marginally harder -- inside the function, you need to say command make, because just make would cause an infinite loop with the alias or function calling itself recursively -- so I will demonstrate this. Let's define a function (aliases suck);
make () {
# run the real make, break out on failure
command make "$#" || return
# if there is no env for us to load, we are done
test -f ./docker.env || return 0
# still here? load it
. ./docker.env
}
If you want even stricter control, maybe define a variable in the function and check inside the Makefile that the variable is set.
$(ifneq '${_composing}','function_make')
$(error Need to use the wrapper function to call make)
$(endif)
The error message is rather bewildering if you haven't read this discussion, so maybe it needs to be improved, and/or documented in a README or something. You would change the make line in the function above into
_composing='function_make' \
command make "$#" || return
The syntax var=value cmd args sets the variable var to the string value just for the duration of running the command line cmd args; it then returns to its previous state (unset, or set to its previous value).
For this particular construction, the name of the variable just needs to be reasonably unique and transparent to a curious human reader; and the value is also just a reasonably unique and reasonably transparent string which the function and the Makefile need to agree on.
Depending on what you end up storing in the environment, this could introduce complications if you need this mechanism for multiple Makefiles. Running it in directory a and then switching to a similar directory b will appear to work, but uses the a things where the poor puny human would expect the b things. (If the variables you set contain paths, relative paths fix this scenario, but complicate others.)
Extending this to a model similar to Ruby's rvm or Python's virtualenv might be worth exploring; they typically add an indicator to the shell prompt to remind you which environment is currently active, and have some (very modest) safeguards in place to warn you when your current directory and the environment disagree.
Another wart: Hardcoding make to always load docker.env is likely to produce unwelcome surprises one day. Perhaps hardcode a different file name which is specific to this hook - say, .compose_post_make_hook? It can then in turn contain something like
. ./docker.env
in this particular directory.
I am trying to create something like a global variable that I will use in order to make my project easy to deploy for other developers.
I would like to have an .sh file where there is a variable defining the location of the project.
Later on I want to export this variable and make it accessable in every makefile that I am creating so that I can use this design to keep everything constant and in one place.
This is an example of what I am trying to build:
Creating and exporting the variables in script.sh:
#!/bin/bash
DIRECTORY='some path value here'
Importing the values in multiple Makefiles:
# start script and fetch the value
VAR := $(shell ./script.sh | sed -n '/^result: /s/^.*: //p')
all:
#echo VAR=$(VAR)
I would like to see how other people are dealing with the same problem.
Being a better developer is my goal here. :)
Feedback always welcomed.
Environment variables exported in the shell are visible from make, so in a shell script like this:
#!/bin/sh
VAR=value
export VAR
make $*
The Makefile will start with VAR defined to value. That's one way to get variables from a shell script into make.
If you don't want the shell script to run make, you can have a user source it:
$ source script.sh
$ make
The variables set in the script will be visible to make this way too.
Or course there doesn't seem to be any reason you need a shell script here. Stick your configuration into a fragment of a Makefile (which would look almost exactly like your shell script, but not use quotes for multiple word values) and then include Makefile.inc in your main makefile.
Also note that syntax like this:
#!/bin/sh or another commment
VAR=value
export VAR
It equally valid included in a Makefile or sourced into a shell script. So sometimes it's possible to use the same include file in both places!
This question already has an answer here:
command substitution doesn't work with echo in makefile [duplicate]
(1 answer)
Closed 6 years ago.
Inside of a makefile, I'm trying to check if fileA was modified more recently than fileB. Using a few prior posts (this and this) as references, I've come up with this as an attempt to store the time since last file modification as a variable:
(I'd rather this be in a function outside of a make recipe, but one step at a time.)
.PHONY: all clean
all : (stuff happens here)
radio :
BASE_MOD_TIME="$( expr $(date +%s) - $(date +%s -r src/radio_interface/profile_init.c) )"
#echo "$(BASE_MOD_TIME)"
I thought that I would be assigning the output of the expr command to a variable, BASE_MOD_TIME, but the output is:
bash-4.1$
BASE_MOD_TIME=""
echo ""
What am I doing wrong here? Simple attempts to save the output of ls -l also didn't work like this.
Make variables are normally global, and you don't normally set make variables in a recipe. A recipe is simply a list of commands to be executed by a shell, and so what looks like a variable assignment in a recipe is a shell variable assignment.
However, each line in a make recipe is run in its own shell subprocess. So a variable set in one line won't be visible in another line; they are not persistent. That makes setting shell variables in recipes less useful. [Note 1]
But you can combine multiple lines of a recipe into a single shell command using the backslash escape at the end of the line, and remembering to terminate the individual commands with semicolons (or, better, link them with &&), because the backslash-escaped newline will not be passed to the shell. Also, don't forget to escape the $ characters so they will be passed to the shell, rather than being interpreted by make.
So you could do the following:
radio:
#BASE_MOD_TIME="$$( expr $$(date +%s) - $$(date +%s -r src/radio_interface/profile_init.c) )"; \
echo "$$BASE_MOD_TIME"; \
# More commands in the same subprocess
But that gets quite awkward if there are more than a couple of commands, and a better solution is usually to write a shell script and invoke it from the recipe (although that means that the Makefile is no longer self-contained.)
Gnu make provides two ways to set make variables in a recipe:
1. Target-specific variables.
You can create a target-specific variable (which is not exactly local to the target) by adding a line like:
target: var := value
To set the variable from a shell command, use the shell function:
target: var := $(shell ....)
This variable will be available in the target recipe and all dependencies triggered by the target. Note that a dependency is only evaluated once, so if it could be triggered by a different target, the target-specific variable might or might not be available in the dependency, depending on the order in which make resolves dependencies.
2. Using the eval function
Since the expansion of recipes is always deferred, you can use the eval function inside a recipe to defer the assignment of a make variable. The eval function can be placed pretty well anywhere in a recipe because its value is the empty string. However, evaling a variable assignment makes the variable assignment global; it will be visible throughout the makefile, but its value in other recipes will depend, again, on the order in which make evaluates recipes, which is not necessarily predictable.
For example:
radio:
$(eval var = $(shell ...))
Notes:
You can change this behaviour using the .ONESHELL: pseudo-target, but that will apply to the entire Makefile; there is no way to mark a single recipe as being executed in a single subprocess. Since changing the behaviour can break recipes in unexpected ways, I don't usually recommend this feature.
What's wrong with this?
fileB: fileA
#echo $< was modified more recently than $#
Instead of forcing the makefile to do all of the heavy lifting via some bash commands, I just called a separate bash script. This provided a lot more clarity for a newbie to bash scripting like myself since I didn't have to worry about escaping the current shell being used by make.
Final solution:
.PHONY: all clean
all : (stuff happens here)
radio :
./radio_init_check.sh
$(MKDIR_P) $(OBJDIR)
make $(radio_10)
with radio_init_check.sh being my sew script.
I found many answers here and elsewhere on the topic, but none that worked. Please help me out here.
I need to set some environment variables, which is partly done in some scripts, called from a master script, partly directly. Here is a minimal Makefile that shows the unwanted behaviour:
FC := ifort
SHELL := /bin/bash
some_target: load_ifort
$(FC) file.f
load_ifort:
source /usr/local2/bin/ifort-compilervars.sh ia32
export LM_LICENSE_FILE=/usr/local2/misc/intel2013/flexlm/server.lic
if I call make, I get an "ifort: command not found" error. If I execute the two comamnds by hand on the command line before calling make, ifort is found and everything is good.
What am I missing???
Each line in a recipe gets executed in a separate subshell. So you create one shell which sources the .sh file, then exits and forgets everything, then another shell which starts with a clean slate.
The straightforward solution in your case would be to collect all these commands in a single variable. I have factored out the LM_LICENSE_FILE assignment because that can be done in Make directly, but you could include that in the FC variable as well.
LM_LICENSE_FILE := /usr/local2/misc/intel2013/flexlm/server.lic
export LM_LICENSE_FILE
FC := source /usr/local2/bin/ifort-compilervars.sh ia32; \
ifort
some_target:
$(FC) file.f
If the shell commands can be straightforwardly run by Make as well, you could include them, or perhaps translate the sh file into Make commands by a simple script.
Another option would be to create a simple wrapper in your PATH; maybe call it fc:
#!/bin/sh
. /usr/local2/bin/ifort-compilervars.sh ia32
ifort "$#"
then just use fc where you currently have $(FC). (If the ifort-compilervars.sh file contains Bash constructs, in spite of the name, you should change the shebang to #!/bin/bash.)
As a rule, only one-liner shell commands "work". From the comment about "bash", it seems likely you are using GNU make. In your example, the word "source" is not found in the GNU make manual's index. (If you found this in a working example, it would be helpful to start from that). There are two types of variables of interest:
makefile variables, which live in the make program
environment variables, which are "exported"
The latter would include $PATH, which is used to find programs. For updating that, you do need shell commands. But (lacking some special provision in the make program), exported variables from a shell script are not passed up into the make program and made available for the next line of the makefile.
You could reorganize the makefile to provide a rule which combines the source command and other initialization into a shell command which then recurs (carrying those variables along) into a subprocess which would then do the compiles. Something like
build:
sh -c "source /usr/local2/bin/ifort-compilervars.sh ia32; \
export LM_LICENSE_FILE=/usr/local2/misc/intel2013/flexlm/server.lic; \
$(MAKE) some_target"
some_target: load_ifort
$(FC) file.f
So, to compile my executable, I need to have the library locations set up correctly. The problem is, the setup comes from a bunch of scripts that do the env variable exporting, and what needs to be set up may change (beyond my control) so I need to use those scripts instead of copying their functionality. To compile in regular command line, I need to do something like:
setup library1
setup library2
source some_other_setup_script.bash
g++ blah.c
# setup is a executable on my system that run some scripts
How would I write a makefile that accomplishes that? As far as I tried, the env variable exporting does not carry over (i.e. "export VAR=remember; echo $VAR" won't work)
You can also add environment variables properly with the machinery of GNU make, like so:
export TEST:="Something Good!"
test:
echo $$TEST
This (I think) has different semantics from:
TEST2:="Something not quite so useful?"
test2:
echo ${TEST2}
Which (again, I think) does the substitution within make before passing along to the shell. Note that the export command doesn't work within a target block, just unindented as an immediately executed command.
If variable exporting is not working the way it does on your command line, that suggests that Make is choosing a shell different from the one you're using, with different syntax for handling variables (export VAR=remember; echo $VAR works fine for me). Make uses /bin/sh by default, but you can override this with the SHELL variable, which Make does not import from the environment. I suggest setting SHELL (in the Makefile) to whatever you're using in your environment and trying the export VAR=remember experiment again.
Ultimately you will need to define the variable and execute the compiler in a shell list or even a script, rather than in separate make commands. There are a couple of refinements you could add, however. You could tell make about the script:
maintarget: script.sh blah.c
source script.sh; g++ blah.c
script.sh:
setup include script here
Another thing would be to just execute all that stuff in the same shell
maintarget: blah.c
run this; run that; run the other thing; g++ blah.c
I believe all make versions will run a ; list in the same shell, but you can always force a subshell with (list) or by calling specifically a shell script as a compiler command wrapper.
Don't forget to have the appropriate targets depend on your scripts themselves. BTW, some make versions (pmake aka bsd make) can execute a command when defining a make variable, and all versions of make then exports those. But I don't think gmake can do that.
You could write another shell script that executes all those commands, then prints out variable assignments that make can use. Run the script, pipe its output to a file, then include that file from your Makefile. For example:
Makefile:
all:
echo $(FOO)
test.mk: test.sh
./$< > $#
include test.mk
test.sh
echo "FOO=1"
Running "make" in the directory containing this Makefile produces:
make: Entering directory `/home/luser/build/mktest'
Makefile:7: test.mk: No such file or directory
./test.sh > test.mk
make: Leaving directory `/home/luser/build/mktest'
make: Entering directory `/home/luser/build/mktest'
echo 1
1
make: Leaving directory `/home/luser/build/mktest'
make creates test.mk by running the shell script, then includes it. test.mk contains the output of test.sh, and is parsed as a Makefile. See http://www.gnu.org/software/make/manual/make.html#Include for more details.
We use a variant of this in Mozilla's client.mk to let you define options in a "mozconfig" file:
http://mxr.mozilla.org/mozilla-central/source/client.mk#138
Restatement: How do I get a shell variable into a make file?
Something like:
MYVAR := $(shell echo $(MYVAR)) <any_makefile_additions_here>
So, this defines MYVAR inside a MAKEFILE when an environment variable named MYVAR is also set.
It might be of interest, that, in order to override an option that is already defined in a makefile, make supports (I am referring to GNU Make 3.82, but other version probably too) the option -e.
Example:
Makefile:
CC=gcc
...
Run make:
CC=gcc-4.7
make -e
will use gcc-4.7 instead of gcc.