This question already has an answer here:
command substitution doesn't work with echo in makefile [duplicate]
(1 answer)
Closed 6 years ago.
Inside of a makefile, I'm trying to check if fileA was modified more recently than fileB. Using a few prior posts (this and this) as references, I've come up with this as an attempt to store the time since last file modification as a variable:
(I'd rather this be in a function outside of a make recipe, but one step at a time.)
.PHONY: all clean
all : (stuff happens here)
radio :
BASE_MOD_TIME="$( expr $(date +%s) - $(date +%s -r src/radio_interface/profile_init.c) )"
#echo "$(BASE_MOD_TIME)"
I thought that I would be assigning the output of the expr command to a variable, BASE_MOD_TIME, but the output is:
bash-4.1$
BASE_MOD_TIME=""
echo ""
What am I doing wrong here? Simple attempts to save the output of ls -l also didn't work like this.
Make variables are normally global, and you don't normally set make variables in a recipe. A recipe is simply a list of commands to be executed by a shell, and so what looks like a variable assignment in a recipe is a shell variable assignment.
However, each line in a make recipe is run in its own shell subprocess. So a variable set in one line won't be visible in another line; they are not persistent. That makes setting shell variables in recipes less useful. [Note 1]
But you can combine multiple lines of a recipe into a single shell command using the backslash escape at the end of the line, and remembering to terminate the individual commands with semicolons (or, better, link them with &&), because the backslash-escaped newline will not be passed to the shell. Also, don't forget to escape the $ characters so they will be passed to the shell, rather than being interpreted by make.
So you could do the following:
radio:
#BASE_MOD_TIME="$$( expr $$(date +%s) - $$(date +%s -r src/radio_interface/profile_init.c) )"; \
echo "$$BASE_MOD_TIME"; \
# More commands in the same subprocess
But that gets quite awkward if there are more than a couple of commands, and a better solution is usually to write a shell script and invoke it from the recipe (although that means that the Makefile is no longer self-contained.)
Gnu make provides two ways to set make variables in a recipe:
1. Target-specific variables.
You can create a target-specific variable (which is not exactly local to the target) by adding a line like:
target: var := value
To set the variable from a shell command, use the shell function:
target: var := $(shell ....)
This variable will be available in the target recipe and all dependencies triggered by the target. Note that a dependency is only evaluated once, so if it could be triggered by a different target, the target-specific variable might or might not be available in the dependency, depending on the order in which make resolves dependencies.
2. Using the eval function
Since the expansion of recipes is always deferred, you can use the eval function inside a recipe to defer the assignment of a make variable. The eval function can be placed pretty well anywhere in a recipe because its value is the empty string. However, evaling a variable assignment makes the variable assignment global; it will be visible throughout the makefile, but its value in other recipes will depend, again, on the order in which make evaluates recipes, which is not necessarily predictable.
For example:
radio:
$(eval var = $(shell ...))
Notes:
You can change this behaviour using the .ONESHELL: pseudo-target, but that will apply to the entire Makefile; there is no way to mark a single recipe as being executed in a single subprocess. Since changing the behaviour can break recipes in unexpected ways, I don't usually recommend this feature.
What's wrong with this?
fileB: fileA
#echo $< was modified more recently than $#
Instead of forcing the makefile to do all of the heavy lifting via some bash commands, I just called a separate bash script. This provided a lot more clarity for a newbie to bash scripting like myself since I didn't have to worry about escaping the current shell being used by make.
Final solution:
.PHONY: all clean
all : (stuff happens here)
radio :
./radio_init_check.sh
$(MKDIR_P) $(OBJDIR)
make $(radio_10)
with radio_init_check.sh being my sew script.
Related
I used below sample code in makefile in MacOS to construct path, but it always not works.
all:
ROOT=$(shell pwd)
#echo $(ROOT)
AGENT:=$(ROOT)/agent
#echo $(AGENT)
#mkdir -p $(AGENT)
It will output as AGENT:=/agent, obviously, the AGENT variable is not expanded as ROOT
Several common mistakes here:
Recipe lines run in separate shells, you cannot expect a shell variable assigned in one to be still defined in another.
No need to use the $(shell...) make function in recipes: recipes are already shell scripts.
You apparently mix-up shell variables and make variables.
Shell variables are assigned with var=xxx (no spaces around the =), only in recipes (well, almost only). They are expanded with $var or ${var} but never with $(var), still almost only in recipes.
Make variables are assigned with var := xxx or var = xxx (with or without spaces around := or =), only outside from recipes (well, almost only). They are expanded with $(var) or ${var}, never with $var, anywhere. Note that = and := are different but this is out of scope.
If you are a make beginner please ignore the "almost" in the above explanation, you will not need this for now.
A Makefile is written with two different languages: the shell language in recipes and the make language everywhere else. A bit like a web page is written in HTML and in CSS. The shell language and the make language look sometimes similar but they are different. Something important to consider is that make will pass the recipes to the shell... after having expanded them first to substitute make variables or make function calls. It is thus frequently necessary to protect parts of the shell syntax from this make expansion (see below for an example).
Anyway, try something like this, maybe:
With make variables
ROOT := $(shell pwd)
AGENT := $(ROOT)/agent
all:
#echo $(ROOT)
#echo $(AGENT)
#mkdir -p $(AGENT)
ROOT and AGENT are make variables, assigned outside any recipe with := and available anywhere, including in recipes. They are expanded with $(var). Before passing the recipe to the shell make will expand it as:
echo /some/where
echo /some/where/agent
mkdir -p /some/where/agent
Each line will be executed in a separate shell but it does not matter because there is no shell variables passing between them.
Note: if you are using GNU make, the CURDIR make variable is defined and expands as the current working directory. No need to use pwd, just use $(CURDIR) instead of $(ROOT):
AGENT := $(CURDIR)/agent
all:
#echo $(CURDIR)
#echo $(AGENT)
#mkdir -p $(AGENT)
Note: if ROOT refers to where the Makefile is located and if you invoke make from elsewhere using the -f make option, $(CURDIR) does not work any more because it points to where you invoked make from, not where the Makefile is located. But you can also get the correct ROOT with:
ROOT := $(dir $(lastword $(MAKEFILE_LIST)))
at the very beginning of your Makefile (before any include statement).
With shell variables
Use only shell syntax and a one-line recipe:
all:
#ROOT="`pwd`"; AGENT="$$ROOT"/agent; echo "$$ROOT"; echo "$$AGENT"; mkdir -p "$$AGENT"
ROOT and AGENT are shell variables, assigned with = inside the recipe and available only in the very same line of the recipe. Note the double $ used to expand shell variables. They are needed to escape the first expansion that make performs before passing the result to the shell. After expansion by make the recipe is passed to the shell as:
ROOT="`pwd`"; AGENT="$ROOT"/agent; echo "$ROOT"; echo "$AGENT"; mkdir -p "$AGENT"
(make expands $$ as $ and stops there) which is what you want. It is executed in one single shell, reason why the shell variables are passed from one command of the list to the next.
If you wish, for better readability, you can use the \ line continuation to split the recipe on several lines, but still have only one single recipe executed in one single shell:
all:
#ROOT="`pwd`"; \
AGENT="$$ROOT/agent"; \
echo "$$ROOT"; \
echo "$$AGENT"; \
mkdir -p "$$AGENT"
This is strictly equivalent to the other form, just a bit easier to read. But it is completely different from:
all:
#ROOT="`pwd`"
#AGENT="$$ROOT/agent"
#echo "$$ROOT"
#echo "$$AGENT"
#mkdir -p "$$AGENT"
because with this last form the 5 lines are executed by 5 different shells. When the second one runs, for example, the ROOT shell variable assigned by the first shell is not defined any more and the result is the same as:
AGENT="/agent"
Then, the two echo lines echo nothing at all, just a newline. And finally you get an error because:
mkdir -p
is not valid.
I have a script "set_env.py" that outputs the following uppon execution:
export MY_VAR_1=some_value
export MY_VAR_2=some_other_value
I cannot change this script, it is supplied in my current environment.
Further I have a Makefile that looks like this:
SHELL := /bin/bash
all: set_env
env | grep MY_VAR
set_env:
eval ./set_env.py
With this makefile I expected the grep to list my two variables, however it seems the environment is not set.
I suspect this is because make creates a sub-environment for each line and so the variables set on the first line will not be available in the second.
So the question is, how would I go about exporting the environment from the untouchable script in my makefile?
Actually, the output of the python is valid make.
One option then is to read the output of the python directly into the makefile.
The only fly in the ointment is that $(shell) doesn't cut the mustard.
include Environment.mk
PHONY: test
test:
env | grep MY_VAR
Environment.mk:
./set_env.py >$#-tmp
mv $#-tmp $#
How does this work?
The first thing that make tries to do is to ensure the makefile itself is up-to-date.
Since we have told it to include Environment.mk,
make must ensure that is up-to-date too.
Make finds a rule for Environment.mk
The python is run, creating Environment.mk
Environment.mk is read in, creating two make variables with the export attribute
The makefile is now up-to-date, so make proceeds on to the target (test in this case)
Make runs test's recipe, exporting any variables with the export attribute.
No recursion, but you should ensure the python always spits out make compatible syntax.
EDIT
As #raspy points out, this is not the whole story.
As it stands,
once Environment.mk has been created,
it will never be regenerated.
If set_env.py ever generates different output,
you should tell make what conditions these are by adding dependencies.
If set_env.py takes a trivial time to run,
I advise a simple .PHONY.
That way it will run every time you run make,
and Environment.mk will never be stale.
.PHONY: Environment.mk
Recursive make is your friend I'm afraid.
.PHONY: all
all:
eval $$(./set_env.py) && ${MAKE} test
.PHONY: test
test:
env | grep MY_VAR
There are a few moving parts here.
make all executes the shell command eval $(./set_env.py) && make test
The shell first does the command substitution
$(./set_env.py) is replaced by the export commands
The export commands are passed to the (shell) eval command
The environment variables are defined, but only for this shell invocation
The eval succeeds, so the execution passes to the command after the &&
Make runs recursively, but this second make has an augmented environment
I'm looking how to export from a Makefile environment variables to be exposed in the userland environment so exporting these variables from the Makefile should be accessible from the user shell.
I have tried make's export but as I understand and have tried does not export to outside of Makefile.
The idea of this is to populate Docker Compose environment variables in a elegant way and have these variables ready to use in the user shell also.
This is a fragment of what I've tried with make's export:
include docker.env
export $(shell sed -n '/=/p' docker.env)
SHELL := /bin/bash
run:
#docker-compose -f my-service.yml up -d
According with ArchWiki, each process of Bash...
Each process stores their environment in the /proc/$PID/environ file.
so once Make execute a source, export or any other command to set a new environment variable it will be applied only for that process.
As workaround I've written in the bash startup file so the variables will be in the global environment as soon as a new bash shell is loaded:
SHELL := /bin/bash
RC := ~/.bashrc
ENV := $(shell sed -n '/=/p' docker.env)
test:
#$(foreach e,$(ENV),echo $(e) >> $(RC);) \
EDIT completely reworked the answer after the OP explained in a comment that he wants the environment variables to be defined for any user shell.
If your goal is to have a set of environment variables defined for any user shell (I assume this means interactive shell), you can simply add these definitions to the shell's startup file (.bashrc for bash). From GNU make manual:
Variables in make can come from the environment in which make is run.
Every environment variable that make sees when it starts up is
transformed into a make variable with the same name and value.
However, an explicit assignment in the makefile, or with a command
argument, overrides the environment. (If the ‘-e’ flag is specified,
then values from the environment override assignments in the makefile.
See Summary of Options. But this is not recommended practice.)
Example:
$ cat .bashrc
...
export FOOBAR=foobar
export BARFOO="bar foo"
...
$ cat Makefile
all:
#printf '$$(FOOBAR)=%s\n' '$(FOOBAR)'
#printf 'FOOBAR='; printenv FOOBAR
#printf '$$(BARFOO)=%s\n' '$(BARFOO)'
#printf 'BARFOO='; printenv BARFOO
$ make
$(FOOBAR)=foobar
FOOBAR=foobar
$(BARFOO)=bar foo
BARFOO=bar foo
If you want to keep these definitions separate, you can just source the file from .bashrc:
$ cat docker.env
export FOOBAR=foobar
export BARFOO="bar foo"
$ cat .bashrc
...
source <some-path>/docker.env
...
And finally, if you don't want to add the export bash command to your file, you can parse the file in your .bashrc:
$ cat docker.env
FOOBAR=foobar
BARFOO="bar foo"
$ cat .bashrc
...
while read -r line; do
eval "export $$line"
done < <(sed -n '/=/p' <some-path>/docker.env)
...
Of course, there are some constraints for the syntax of your docker.env file (no unquoted special characters, no spaces in variable names, properly quoted values...) If your syntax is not bash-compatible it is time to ask another question about parsing this specific syntax and converting it into bash-compatible syntax.
Make cannot change the calling shell's environment without its cooperation. Of course, if you are in control, you can make the calling shell cooperate.
In broad terms, you could replace the make command with a shell alias or function which runs the real make and also sets the environment variables from the result. I will proceed to describe in more detail one way to implement this.
Whether you call this alias or function of yours make or e.g. compose is up to you really. To wrap the real make is marginally harder -- inside the function, you need to say command make, because just make would cause an infinite loop with the alias or function calling itself recursively -- so I will demonstrate this. Let's define a function (aliases suck);
make () {
# run the real make, break out on failure
command make "$#" || return
# if there is no env for us to load, we are done
test -f ./docker.env || return 0
# still here? load it
. ./docker.env
}
If you want even stricter control, maybe define a variable in the function and check inside the Makefile that the variable is set.
$(ifneq '${_composing}','function_make')
$(error Need to use the wrapper function to call make)
$(endif)
The error message is rather bewildering if you haven't read this discussion, so maybe it needs to be improved, and/or documented in a README or something. You would change the make line in the function above into
_composing='function_make' \
command make "$#" || return
The syntax var=value cmd args sets the variable var to the string value just for the duration of running the command line cmd args; it then returns to its previous state (unset, or set to its previous value).
For this particular construction, the name of the variable just needs to be reasonably unique and transparent to a curious human reader; and the value is also just a reasonably unique and reasonably transparent string which the function and the Makefile need to agree on.
Depending on what you end up storing in the environment, this could introduce complications if you need this mechanism for multiple Makefiles. Running it in directory a and then switching to a similar directory b will appear to work, but uses the a things where the poor puny human would expect the b things. (If the variables you set contain paths, relative paths fix this scenario, but complicate others.)
Extending this to a model similar to Ruby's rvm or Python's virtualenv might be worth exploring; they typically add an indicator to the shell prompt to remind you which environment is currently active, and have some (very modest) safeguards in place to warn you when your current directory and the environment disagree.
Another wart: Hardcoding make to always load docker.env is likely to produce unwelcome surprises one day. Perhaps hardcode a different file name which is specific to this hook - say, .compose_post_make_hook? It can then in turn contain something like
. ./docker.env
in this particular directory.
Here is what it finally took to get my code in my makefile to work
Line 5 is the question area
BASE=50
INCREMENT=1
FORMATTED_NUMBER=${BASE}+${INCREMENT}
all:
echo $$((${FORMATTED_NUMBER}))
why do i have to add two $ and two (( )) ?
Formatted_Number if i echo it looks like "50+1" . What is the logic that make is following to know that seeing $$(("50+1")) is actually 51?
sorry if this is a basic question i'm new to make and dont fully understand it.
First, whenever asking questions please provide a complete example. You're missing the target and prerequisite here so this is not a valid makefile, and depending on where they are it could mean very different things. I'm assuming that your makefile is something like this:
BASE=50
INCREMENT=1
FORMATTED_NUMBER=${BASE}+${INCREMENT}
all:
echo $$((${FORMATTED_NUMBER}))
Makefiles are interesting in that they're a combination of two different formats. The main format is makefile format (the first five lines above), but inside a make recipe line (that's the last line above, which is indented with a TAB character) is shell script format.
Make doesn't know anything about math. It doesn't interpret the + in the FORMATTED_NUMBER value. Make variables are all strings. If you want to do math, you have to do it in the shell, in a shell script, using the shell's math facilities.
In bash and other modern shells, the syntax $(( ...expression... )) will perform math. So in the shell if you type echo $((50+1)) (go ahead and try it yourself) it will print 51.
That's why you need the double parentheses ((...)): because that's what the shell wants and you're writing a shell script.
So why the double $? Because before make starts the shell to run your recipe, it first replaces all make variable references with their values. That's why the shell sees 50+1 here: before make started the shell it expanded ${FORMATTED_NUMBER} into its value, which is ${BASE}+${INCREMENT}, then it expanded those variables so it ends up with 50+1.
But what if you actually want to use a $ in your shell script (as you do here)? Then you have to tell make to not treat the $ as introducing a make variable. You do this by doubling it, so if make sees $$ then it does not think that's a make variable, and sends a single $ to the shell.
So for the recipe line echo $$((${FORMATTED_NUMBER})) make actually invokes a shell script echo $((50+1)).
You can use this in BASH:
FORMATTED_NUMBER=$((BASE+INCREMENT))
Is using non BASH use:
FORMATTED_NUMBER=`echo "$BASE + $INCREMENT" | bc`
This won't work for me. I want to make some substitution and assign it to a variable in Makefile. An example is as follows but I prefer to do it with Perl since other substitutions can be more complex than this.
eval.%:
# make eval.exp-1.ans
# $* --> exp-1.ans
folder=`echo $* | sed -e 's/\..*//g'`
# OR
folder=`echo $* | perl -ne 'm/(.*)\.ans/; print $$1'
# I want that folder will be exp-1
echo $$folder ${folder}
Why this does not work? How can I do this kind of things in Makefile?
Your question is not clear. Are you trying to set a variable in your makefile, so that other recipes, etc. can see it? Or are you just trying to set a variable in one part of your recipe that can be used in other parts of the same recipe?
Make runs recipes in shells that it invokes. It's a fundamental feature of UNIX that no child process can modify the memory/environment/working directory/etc. of its parent process. So no variable that you assign in a recipe (subshell) of a makefile can ever set a make environment variable. There are ways to do this, but not from within a recipe. However that doesn't appear (from your example) to be what you want to do.
The next thing to note is that make runs each logical line of the recipe in a different shell. So shell variables set in one logical line will be lost when that shell exits, and the next logical line cannot see that value. The solution is to ensure that all the lines of your recipe that need access to the variable are on the same logical line, so they'll be sent to the same shell script, like this:
eval.%:
folder=`echo $* | perl -ne 'm/(.*)\.ans/; print $$1' || exit 1 ; \
echo $$folder
Have you tried the $(VARIABLE:suffix=replacement) syntax? In your case, $(*:.ans=). That will work for any suffix, even if it doesn't start with a dot.