I've got the following Makefile (GNU Make 3.81):
CWD:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
all:
pwd
cd "$(CWD)/.."
pwd
cd ".."
pwd
Here is the output:
$ make
pwd
/Users/kenorb/temp/foo
cd "/Users/kenorb/temp/foo/.."
pwd
/Users/kenorb/temp/foo
cd ".."
pwd
/Users/kenorb/temp/foo
It seems that cd'ing to parent directory via .. doesn't take any effect.
How do I change the current working directory to a parent directory relatively to Makefile file?
This issue has to do with the fact that each line of a rule's recipe is executed in a dedicated shell instance (i.e.: a new process). So, running cd in one line, won't have any effect for a command in another line, because these commands are executed by different processes.
Changing the working directory for a recipe
You can either use .ONESHELL (GNU Make 3.82 or later) to run all the recipe's lines in a single and the same shell instance:
CWD:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
.ONESHELL:
all:
pwd
cd "$(CWD)/.."
pwd
cd ".."
pwd
or just keep all the commands in a single line:
CWD:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
all:
pwd; cd "$(CWD)/.."; pwd; cd ".."; pwd
The change of the working directory takes effect only inside that recipe.
Note that changing the current working directory inside the makefile for the shell that invoked make is not possible, since that's a process' property, and the running make instance that will execute cd is another (sub)process. Therefore, it will only change the working directory of that subprocess.
That is the same reason you cannot change the working directory of a shell by executing a script without sourcing it: because the script runs in a new shell instance (i.e.: a subprocess).
I compile my project with:
debug=yes make -j4 or debug=no make -j4
The debug variable changes some compiler flags in the Makefile
Instead of typing this repeatedly in the shell, I wrote this script (lets call it daemon):
#!/bin/bash
inotifywait -q -m -e close_write `ls *.c* *.h*` |
while read; do
make -j4
done
so I just do ./daemon which automatically builds whenever a file is written to.
However, I would like to be able to pass the debug=no make -j4 to the ./daemon script like this:
./daemon debug=no make -j4
So I modified the script:
#!/bin/bash
if [ $# -lt 1 ]; then
echo "Usage `basename $0` [COMMAND]"
exit 1;
fi
inotifywait -q -m -e close_write `ls *.c* *.h*` |
while read; do
"$#"
done
This works with ./daemon make -j4 but when I say daemon debug=no make -j4 I get the following error:
./daemon: line 9: debug=no: command not found
How can I make it so debug=no is recognized as a variable and not a command in the daemon script?
Thanks
The expansion of "$#" is parsed after any pre-command assignments are recognized. All you need to do is ensure that debug=... is in the environment of the command that runs make, which is your daemon script.
debug=no ./daemon make -j4
Variable expansions will only ever become arguments (including the zeroth argument: the command name).
They will never become:
Redirections, so you can't var='> file'; cmd $var
Shell keywords or operators, so you can't var='&'; mydaemon $var
Assignments, including prefix assignments, so you can't var='debug=yes'; $var make as you discovered
Command expansions, loops, process substitutions, &&/||, escape sequences, or anything else.
If you want to do this though, you're in luck: there's a standard POSIX tool that will turn leading key=value pairs into environment variables and run the program you want.
It's called env. Here's an example:
run() {
env "$#"
}
run debug=yes make -j 4
Though TBH I'd use chepner's solution
You always need to put the (env) variable settings at the beginning of the command, i.e. before "daemon".
Let me first write a quick Makefile as a showcase:
#!/bin/make -f
folders := $(shell find -mindepth 1 -maxdepth 1 -type d -print)
make_dir:
#mkdir -p "test0"
pwd_test:
#cd "test0" && pwd
#pwd
pwd_all:
#for f in $(folders); do \
cd "$${f}" && pwd; \
pwd; \
cd ..; \
done
First do make make_dir and then see the different results:
➜ so make pwd_test
/data/cache/tmp/so/test0
/data/cache/tmp/so
➜ so make pwd_all
/data/cache/tmp/so/test0
/data/cache/tmp/so/test0
You see that in the for loop it is necessary to do cd ... Apparently, now there is no child process spawn for the cd X && pwd command, while that is normally the case. Is this behaviour specific to make or specific to my shell?
Make spawns a new process for each command in the rule. Since the for loop is one command you get only one process.
Take a look at Recipe Execution
Edit:
Each line in a makefile gets it own subshell. Commands that have
\ tells make that the next line should be part of the current line.
The reason the for loop get its own subshell is because make see the line as
#for f in $(folders); do cd "$${f}" && pwd; pwd; cd ..; done
MadScientist explains it fairly well. Any command that you can type in your
shell in one line will be executed by make in one subshell or process.
If you were to run this in ksh, ksh would be passed
for f in $(folders); do cd "$${f}" && pwd; pwd; cd ..; done and it would be
run in that one subshell. If ksh did not have a for loop implemented this
probably would error and make would say the command returned some error code.
Explanation of pwd_test
pwd_test:
#cd "test0" && pwd
#pwd
#cd "test0" && pwd is seen as one line so the subshell updates its current
working directory and then prints out what the current working is.
#pwd At this line make spawns a new subshell that contains the old working
directory (or the directory make was called form) and pwd prints that
directory.
I have the following gnu make script:
for hdrfile in $(_PUBLIC_HEADERS) ; do \
echo $(dir $$hdrfile) ; \
done
The _PUBLIC_HEADERS variable has a list of relative paths, like so:
./subdir/myheader1.h
./subdir/myheader2.h
The output I get from the for loop above is:
./
./
I expect to see:
./subdir/
./subdir/
What am I doing wrong? Note that if I change the code to:
echo $(dir ./subdir/myheader1.h)
it works in this case. I think maybe it has something to do with the for loop but I'm not sure.
You are confusing make variables (or functions) with shell variables when executing the for-loop. Note that $(dir ...) is a make construct that gets expanded by make before the command is executed by the shell. However, you want the shell to execute that command inside the loop.
What you could do is replace $(dir) with the corresponding command dirname which gets executed by the shell. So it becomes:
for hdrfile in $(_PUBLIC_HEADERS) ; do \
dirname $$hdrfile ; \
done
This should give the desired result.
Is there a better way to source a script, which sets env vars, from within a makefile?
FLAG ?= 0
ifeq ($(FLAG),0)
export FLAG=1
/bin/myshell -c '<source scripts here> ; $(MAKE) $#'
else
...targets...
endif
Makefile default shell is /bin/sh which does not implement source.
Changing shell to /bin/bash makes it possible:
# Makefile
SHELL := /bin/bash
rule:
source env.sh && YourCommand
To answer the question as asked: you can't.
The basic issue is that a child process can not alter the parent's environment. The shell gets around this by not forking a new process when source'ing, but just running those commands in the current incarnation of the shell. That works fine, but make is not /bin/sh (or whatever shell your script is for) and does not understand that language (aside from the bits they have in common).
Chris Dodd and Foo Bah have addressed one possible workaround, so I'll suggest another (assuming you are running GNU make): post-process the shell script into make compatible text and include the result:
shell-variable-setter.make: shell-varaible-setter.sh
postprocess.py #^
# ...
else
include shell-variable-setter.make
endif
messy details left as an exercise.
If your goal is to merely set environment variables for Make, why not keep it in Makefile syntax and use the include command?
include other_makefile
If you have to invoke the shell script, capture the result in a shell command:
JUST_DO_IT=$(shell source_script)
the shell command should run before the targets. However this won't set the environment variables.
If you want to set environment variables in the build, write a separate shell script that sources your environment variables and calls make. Then, in the makefile, have the targets call the new shell script.
For example, if your original makefile has target a, then you want to do something like this:
# mysetenv.sh
#!/bin/bash
. <script to source>
export FLAG=1
make "$#"
# Makefile
ifeq($(FLAG),0)
export FLAG=1
a:
./mysetenv.sh a
else
a:
.. do it
endif
Using GNU Make 3.81 I can source a shell script from make using:
rule:
<tab>source source_script.sh && build_files.sh
build_files.sh "gets" the environment variables exported by source_script.sh.
Note that using:
rule:
<tab>source source_script.sh
<tab>build_files.sh
will not work. Each line is ran in its own subshell.
This works for me. Substitute env.sh with the name of the file you want to source. It works by sourcing the file in bash and outputting the modified environment, after formatting it, to a file called makeenv which is then sourced by the makefile.
IGNORE := $(shell bash -c "source env.sh; env | sed 's/=/:=/' | sed 's/^/export /' > makeenv")
include makeenv
Some constructs are the same in the shell and in GNU Make.
var=1234
text="Some text"
You can alter your shell script to source the defines. They must all be simple name=value types.
Ie,
[script.sh]
. ./vars.sh
[Makefile]
include vars.sh
Then the shell script and the Makefile can share the same 'source' of information. I found this question because I was looking for a manifest of common syntax that can be used in Gnu Make and shell scripts (I don't care which shell).
Edit: Shells and make understand ${var}. This means you can concatenate, etc,
var="One string"
var=${var} "Second string"
I really like Foo Bah's answer where make calls the script, and the script calls back to make. To expand on that answer I did this:
# Makefile
.DEFAULT_GOAL := all
ifndef SOME_DIR
%:
<tab>. ./setenv.sh $(MAKE) $#
else
all:
<tab>...
clean:
<tab>...
endif
--
# setenv.sh
export SOME_DIR=$PWD/path/to/some/dir
if [ -n "$1" ]; then
# The first argument is set, call back into make.
$1 $2
fi
This has the added advantage of using $(MAKE) in case anyone is using a unique make program, and will also handle any rule specified on the command line, without having to duplicate the name of each rule in the case when SOME_DIR is not defined.
If you want to get the variables into the environment, so that they are passed to child processes, then you can use bash's set -a and set +a. The former means, "When I set a variable, set the corresponding environment variable too." So this works for me:
check:
bash -c "set -a && source .env.test && set +a && cargo test"
That will pass everything in .env.test on to cargo test as environment variables.
Note that this will let you pass an environment on to sub-commands, but it won't let you set Makefile variables (which are different things anyway). If you need the latter, you should try one of the other suggestions here.
My solution to this: (assuming you're have bash, the syntax for $# is different for tcsh for instance)
Have a script sourceThenExec.sh, as such:
#!/bin/bash
source whatever.sh
$#
Then, in your makefile, preface your targets with bash sourceThenExec.sh, for instance:
ExampleTarget:
bash sourceThenExec.sh gcc ExampleTarget.C
You can of course put something like STE=bash sourceThenExec.sh at the top of your makefile and shorten this:
ExampleTarget:
$(STE) gcc ExampleTarget.C
All of this works because sourceThenExec.sh opens a subshell, but then the commands are run in the same subshell.
The downside of this method is that the file gets sourced for each target, which may be undesirable.
Depending on your version of Make and enclosing shell, you can implement a nice solution via eval, cat, and chaining calls with &&:
ENVFILE=envfile
source-via-eval:
#echo "FOO: $${FOO}"
#echo "FOO=AMAZING!" > $(ENVFILE)
#eval `cat $(ENVFILE)` && echo "FOO: $${FOO}"
And a quick test:
> make source-via-eval
FOO:
FOO: AMAZING!
An elegant solution found here:
ifneq (,$(wildcard ./.env))
include .env
export
endif
If you need only a few known variables exporting in makefile can be an option, here is an example of what I am using.
$ grep ID /etc/os-release
ID=ubuntu
ID_LIKE=debian
$ cat Makefile
default: help rule/setup/lsb
source?=.
help:
-${MAKE} --version | head -n1
rule/setup/%:
echo ID=${#F}
rule/setup/lsb: /etc/os-release
${source} $< && export ID && ${MAKE} rule/setup/$${ID}
$ make
make --version | head -n1
GNU Make 3.81
. /etc/os-release && export ID && make rule/setup/${ID}
make[1]: Entering directory `/tmp'
echo ID=ubuntu
ID=ubuntu
--
http://rzr.online.fr/q/gnumake
Assuming GNU make, can be done using a submake. Assuming that the shell script that exports the variables is include.sh in the current directory, move your Makefile to realmake.mk. Create a new Makefile:
all:
#. ./include.sh; \
$(MAKE) -f realmake.mk $(MAKECMDGOALS)
$(MAKECMDGOALS):
+#. ./include.sh; \
$(MAKE) -f realmake.mk $(MAKECMDGOALS)
Pay attention to the ./ preceding include.sh.
Another possible way would be to create a sh script, for example run.sh, source the required scripts and call make inside the script.
#!/bin/sh
source script1
source script2 and so on
make
target: output_source
bash ShellScript_name.sh
try this it will work, the script is inside the current directory.