How to use environment variables defined inside a makefile? - makefile

I currently have a Makefile that looks something like this:
some_process:
VAR1=1
VAR2=2
python3 ./main.py --arg1 /home/user1/something/ \
--arg2 $VAR1 \
--arg3 $VAR2
When I run this code I get the error:
main.py: error: argument --arg2: expected one argument
I've tried using brackets around VAR1 and VAR2, but I noticed that if I do that then the Makefile doesn't recognize(?) the arguments. I also tried escaping the $ sign (i.e. $$VAR1) but this also does not work.
What is the proper way to use a Makefile for environment variables? Thanks.
EDIT
Just in case it's unclear from the original question, I'm using the Makefile to replicate what I'd usually do in the command line:
user#thing:$ export VAR1=1
user#thing:$ export VAR2=2
user#thing:$ python3 ./main.py --arg1 /home/user1/something --arg2 $VAR1 --arg3 $VAR2

There are two basic problems with the code shown...
some_process:
VAR1=1
VAR2=2
python3 ./main.py --arg1 /home/user1/something/ \
--arg2 $VAR1 \
--arg3 $VAR2
Firstly, make runs each line from the commands section of a rule in its own shell. So VAR1=1 is run in a shell and then, effectively, lost. Secondly, tokens such as $VAR1 will be expanded by make rather than the shell. In this case -- because there's no parentheses -- $VAR1 will be interpreted by make as $(V)AR1. That is, the value of the variable V followed by the text AR1. You can use $$ to force make to ignore the expansion.
Taking all of the above into account you could try (untested)...
some_process:
VAR1=1; \
VAR2=2; \
python3 ./main.py --arg1 /home/user1/something/ \
--arg2 $$VAR1 \
--arg3 $$VAR2

Firstly, as #G.M. pointed out, make runs each command in a rule in separate sub-shells; so variable assignments in separate lines won't work. Also beware of the variable reference format. $(VAR1) or ${VAR1} if it needs to be expanded by make, $$VAR1 if it needs to be expanded in a sub-shell.
Having said that, you can define VAR1 and VAR2 as make variables and export them to make them available for your some_process rule.
VAR1 := 1
VAR2 := 2
export VAR1 VAR2
some_process:
python3 ./main.py --arg1 /home/user1/something/ \
--arg2 $$VAR1 \
--arg3 $$VAR2
This way, you can override the variable values when you invoke the rule if needed.
$ make some_process VAR2=22
By the way, as you're explicitly giving the variables as command line arguments to your python script, they don't need to be exported; so the following should work just fine.
VAR1 := 1
VAR2 := 2
some_process:
python3 ./main.py --arg1 /home/user1/something/ \
--arg2 $(VAR1) \
--arg3 $(VAR2)

Related

Makefile: Reuse environment variables

In my Makefile deploy target I create environment variables and I want to reuse those in the following lines:
SHELL=/bin/sh
deploy:
export $(shell sh this-script-generate-key-values.sh | xargs)
echo ${VAR1} #there is no variable here
echo ${VAR2} #there is no variable here
Where:
this-script-generate-key-values.sh generates this output:
VAR1="somevalue"
VAR2="somevalue"
Why the variables are not set in subsequent lines? How can I make it work?
Notes:
This line works: sh this-script-generate-key-values.sh | xargs
The shell must be /bin/sh (no bash)
All lines in a Makefile recipe run in a separate shell. You need to run the lines in a single shell. Also you need to escape the dollar sign ($) so that variable substitution is not done by make but by the shell.
SHELL=/bin/sh
deploy:
export $$(this-script-generate-key-values.sh | xargs) ;\
echo $${VAR1} ;\
echo $${VAR2}
Just to expand on my comment -- you could output to a file, and use the file to generate your output as so:
vars.txt:
this-script-generate-key-values.sh > $#
deploy : vars.txt
echo VAR1=$$(sed -n 's|VAR1=\(.*\)|\1|p' vars.txt)
echo VAR2=$$(sed -n 's|VAR2=\(.*\)|\1|p' vars.txt)
note: you may have to generate dependencies for vars.txt or declare it .PHONY, otherwise, this will not run on every invocation of make.
If the .ONESHELL special target appears anywhere in the makefile then all recipe lines for each target will be provided to a single invocation of the shell. Newlines between recipe lines will be preserved.
.ONESHELL:
deploy:
export $$(this-script-generate-key-values.sh)
echo $${VAR1}
echo $${VAR2}

pass env variables in make command in makefile

I am trying to pass a shell variable from one makefile command to another, but so far have not been successful.
target1:
curl ... ${myvar} ## using myvar from target2
target2:
export myvar=$(shell curl .....);
echo $myvar
In the above case, I am not even getting the output on echo $myvar. Is there something I'm doing wrong?
In a Makefile, every line of a target is run in a separate shell. Additionally, a command can only change the environment for itself and its children. So when you have:
target2:
export myvar=$(shell curl .....);
echo $myvar
And you run make target2, the following happens:
Make starts a shell that runs export myvar=...some value...
The shell exits.
Make runs another shell that runs echo $myvar
That shell exits.
First, there's a syntax problem here: when you write $myvar, this
will be interpreted by make as a request for the value $m followed
by the text yvar. To access shell variables, you need to escape the
$, like this:
echo $$myvar
But that won't solve this problem, because as we see from the above
sequence, the export command happens in a shell process which
immediately exits, so it's effectively invisible to anything else.
This target would work the way you expect if you were to write:
target2:
export myvar=$(shell curl .....); \
echo $$myvar
Here, because we're using the \ to escape the end-of-line, this is
all one long "virtual" line and executes in a single shell process, so
the echo statement will see the variable value set in the previous
statement.
But nothing will make an environment variable set in a shell process
in one target visible in another target, because there's no way to get
these to execute in the same process.
If you need to set variables in your Makefile that are visible
across all targets, set make variables:
myvar = $(shell curl ...)
target1:
curl ... $(myvar)
A workaround, as you have discovered, is to re-execute make as a
child process from within the process that set the environment
variable as in:
target2:
export myvar=$(shell curl .....); \
echo $$myvar; \
$(MAKE) myvar=$$myvar
But often this sort of recursive call to make results in a more
complicated Makefile.
Found the answer. Posting if anyone comes across this. I needed to use $$ instead of $
target1:
curl ... ${myvar} ## using myvar from target2
target2:
export myvar=$(shell curl .....);
echo $$myvar
$(MAKE) myvar=$${myvar} target1

How to detect if the makefile `--silent/--quiet` command line option was set?

How to detect if the makefile --silent / --quiet command line options was set?
Related questions:
how to detect if --quiet option is specified with rake
I think you need:
$(findstring s,$(word 1, $(MAKEFLAGS)))
Because MAKEFLAGS has long options, too, eg:
MAKEFLAGS=s -j80 --jobserver-auth=4,6
So, IOW:
# Set SILENT to 's' if --quiet/-s set, otherwise ''.
SILENT := $(findstring s,$(word 1, $(MAKEFLAGS)))
If you call either make --quiet or --silent, the variable {MAKEFLAGS} is set only to s. And if you add other options like --ignore-errors and --keep-going, the variable {MAKEFLAGS} is set to iks. Then, you can capture it with this:
ECHOCMD:=/bin/echo -e
SHELL := /bin/bash
all:
printf 'Calling with "%s" %s\n' "${MAKECMDGOALS}" "${MAKEFLAGS}";
if [[ "ws" == "w$(findstring s,${MAKEFLAGS})" ]]; then \
printf '--silent option was set\n'; \
fi
References:
Recursive make: correct way to insert `$(MAKEFLAGS)`
How to set MAKEFLAGS from Makefile, in order to remove default implicit-rules
Remove target from MAKECMDGOALS?
https://www.gnu.org/software/autoconf/manual/autoconf-2.61/html_node/The-Make-Macro-MAKEFLAGS.html

bash set environment variable before command in script

I compile my project with:
debug=yes make -j4 or debug=no make -j4
The debug variable changes some compiler flags in the Makefile
Instead of typing this repeatedly in the shell, I wrote this script (lets call it daemon):
#!/bin/bash
inotifywait -q -m -e close_write `ls *.c* *.h*` |
while read; do
make -j4
done
so I just do ./daemon which automatically builds whenever a file is written to.
However, I would like to be able to pass the debug=no make -j4 to the ./daemon script like this:
./daemon debug=no make -j4
So I modified the script:
#!/bin/bash
if [ $# -lt 1 ]; then
echo "Usage `basename $0` [COMMAND]"
exit 1;
fi
inotifywait -q -m -e close_write `ls *.c* *.h*` |
while read; do
"$#"
done
This works with ./daemon make -j4 but when I say daemon debug=no make -j4 I get the following error:
./daemon: line 9: debug=no: command not found
How can I make it so debug=no is recognized as a variable and not a command in the daemon script?
Thanks
The expansion of "$#" is parsed after any pre-command assignments are recognized. All you need to do is ensure that debug=... is in the environment of the command that runs make, which is your daemon script.
debug=no ./daemon make -j4
Variable expansions will only ever become arguments (including the zeroth argument: the command name).
They will never become:
Redirections, so you can't var='> file'; cmd $var
Shell keywords or operators, so you can't var='&'; mydaemon $var
Assignments, including prefix assignments, so you can't var='debug=yes'; $var make as you discovered
Command expansions, loops, process substitutions, &&/||, escape sequences, or anything else.
If you want to do this though, you're in luck: there's a standard POSIX tool that will turn leading key=value pairs into environment variables and run the program you want.
It's called env. Here's an example:
run() {
env "$#"
}
run debug=yes make -j 4
Though TBH I'd use chepner's solution
You always need to put the (env) variable settings at the beginning of the command, i.e. before "daemon".

Setting env variable from Perl and using in Makefile

I'm trying to set env variable from Perl script and using it inside Makefile doesn't work. While dumping ENV hash tree shows variable successfully set.
Makefile code
list_gen: $(TESTBENCH_PATH)/blocks/soc_tb/global/default
perl list_gen.pl; \
$(MAKE) $(RECITAL_PATH)/catalog/catalog.xml
# touch $#
# Generates catalog.xml from .xmls of VIPs
$(RECITAL_PATH)/catalog/catalog.xml: $(VIP_XMLs)
#echo "# generating catalog.xml " ; \
echo "vip list is $(VIP_LIST)" ; \
rpfCatalog --add $(VIP_XMLs); \
for vip in $(VIP_LIST); \
do \
rpfCatalog --add $$TESTBENCH_PATH/common_blocks/$$vip/global/default.xml; \
done
Perl Script code
$ENV{"VIP_LIST"} = $vip_exists;
I don't think there is any elegant way to update environment of make by running any external command (like perl script) from it. Because even a direct way wouldn't work.
/home/user> cat makefile
NUM=100
first:
NUM=200
echo $(NUM)
second:
NUM=200; echo $(NUM)
third:
NUM=200; \
echo $(NUM);
/home/user> make first
NUM=200
echo 100
100
/home/user> make second
NUM=200; echo 100
100
/home/user> make third
NUM=200; \
echo 100;
100
If you are only looking for a perl script to modify it's parent's environment, something like this works (in a very limited way):
/home/user> cat change_env.pl
#!/usr/bin/perl -w
print "NUM=100\n";
/home/user> NUM=1
/home/user> echo $NUM
1
/home/user> eval $(./change_env.pl)
/home/user> echo $NUM
100
Coming to original your question, .. can't you call make after making the changes to your shell environment?
VIP_LIST=$(perl my.pl)
Would save the output of the script in the variable of the parent shell.
HTH
Georg
When you execute your Perl script, do it like $(./myperlscript.pl) on the prompt. What you need to do is
$ENV{VIP_LIST} = $vip_exists;
Drop the quotes

Resources