How to enter a parent directory in Makefile? - shell

I've got the following Makefile (GNU Make 3.81):
CWD:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
all:
pwd
cd "$(CWD)/.."
pwd
cd ".."
pwd
Here is the output:
$ make
pwd
/Users/kenorb/temp/foo
cd "/Users/kenorb/temp/foo/.."
pwd
/Users/kenorb/temp/foo
cd ".."
pwd
/Users/kenorb/temp/foo
It seems that cd'ing to parent directory via .. doesn't take any effect.
How do I change the current working directory to a parent directory relatively to Makefile file?

This issue has to do with the fact that each line of a rule's recipe is executed in a dedicated shell instance (i.e.: a new process). So, running cd in one line, won't have any effect for a command in another line, because these commands are executed by different processes.
Changing the working directory for a recipe
You can either use .ONESHELL (GNU Make 3.82 or later) to run all the recipe's lines in a single and the same shell instance:
CWD:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
.ONESHELL:
all:
pwd
cd "$(CWD)/.."
pwd
cd ".."
pwd
or just keep all the commands in a single line:
CWD:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
all:
pwd; cd "$(CWD)/.."; pwd; cd ".."; pwd
The change of the working directory takes effect only inside that recipe.
Note that changing the current working directory inside the makefile for the shell that invoked make is not possible, since that's a process' property, and the running make instance that will execute cd is another (sub)process. Therefore, it will only change the working directory of that subprocess.
That is the same reason you cannot change the working directory of a shell by executing a script without sourcing it: because the script runs in a new shell instance (i.e.: a subprocess).

Related

realpath not working on newly created files

test :
touch test
echo $(realpath test)
echo $(shell realpath test)
rm test
When I run make, $(realpath test) returns the empty string, whereas $(shell realpath test) returns the exptected result. Why is this? I've tried using .ONESHELL but it doesn't make a difference.
First, shell realpath and the GNU make realpath function are different. Shell realpath will return a path even if the file doesn't exist:
/home/me$ rm -f blahblah
/home/me$ realpath blahblah
/home/me/blahblah
But, GNU make's realpath will return the empty string if the file doesn't exist.
So, why does the file not exist? Because make will expand all lines of the recipe before running any line of the recipe.
That means that the make functions like $(realpath ...) and $(shell ...) are expanded first, before the first line of the recipe (touch test) is run... and so at the time they are expanded the test file doesn't exist.
In general you never want to use make's $(shell ...) function in a recipe and you can't use make constructs to "interact" with operations that happen inside your recipe. You should use shell functions for that:
test :
touch test;
echo $$(realpath test)
rm test

How to get parent dir in Makefile

This is printing the current dir, not the parent:
run:
#cd ..; \
echo $(shell pwd)
I need the parent dir in a command like:
run:
#cd ..; \
docker run -it --rm -p 8080:8080 -v $(shell pwd):/go/src/hello golang bash
Remember that make works by invoking a shell and sending the recipe to the shell for execution. Make doesn't have a shell "built in", so it's not running recipes directly.
The problem is that $(shell ..) is a make function. All make variables and functions are expanded before the shell is invoked (consider: the shell doesn't know how to handle make functions).
That means that a make function like $(shell ...) is first expanded and pwd is run, which gives you the current directory that the make process is running in, then the resulting string is passed to the shell for execution. So the shell sees this:
cd ..; echo /path/to/make/dir
You never need to use the $(shell ...) function inside a recipe; the recipe is already running in a shell! Instead you want to use shell syntax inside a recipe. The one caveat to this is that you have to escape dollar signs (replacing shell $ with $$) so that make doesn't interpret them as make variables. So if you write:
run:
#cd ..; echo $$(pwd)
then make expands that string and sends this command to the shell:
cd ..; echo $(pwd)
which works as you want.
Why not use the POSIXly mandated variable PWD?
run:
#cd ..; echo $$PWD
Save a process today!

How to get current relative directory of your Makefile?

I have a several Makefiles in app specific directories like this:
/project1/apps/app_typeA/Makefile
/project1/apps/app_typeB/Makefile
/project1/apps/app_typeC/Makefile
Each Makefile includes a .inc file in this path one level up:
/project1/apps/app_rules.inc
Inside app_rules.inc I'm setting the destination of where I want the binaries to be placed when built. I want all binaries to be in their respective app_type path:
/project1/bin/app_typeA/
I tried using $(CURDIR), like this:
OUTPUT_PATH = /project1/bin/$(CURDIR)
but instead I got the binaries buried in the entire path name like this: (notice the redundancy)
/project1/bin/projects/users/bob/project1/apps/app_typeA
What can I do to get the "current directory" of execution so that I can know just the app_typeX in order to put the binaries in their respective types folder?
The shell function.
You can use shell function: current_dir = $(shell pwd).
Or shell in combination with notdir, if you need not absolute path:
current_dir = $(notdir $(shell pwd)).
Update.
Given solution only works when you are running make from the Makefile's current directory.
As #Flimm noted:
Note that this returns the current working directory, not the parent directory of the Makefile. For example, if you run cd /; make -f /home/username/project/Makefile, the current_dir variable will be /, not /home/username/project/.
Code below will work for Makefiles invoked from any directory:
mkfile_path := $(abspath $(lastword $(MAKEFILE_LIST)))
current_dir := $(notdir $(patsubst %/,%,$(dir $(mkfile_path))))
As taken from here;
ROOT_DIR:=$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
Shows up as;
$ cd /home/user/
$ make -f test/Makefile
/home/user/test
$ cd test; make Makefile
/home/user/test
If you are using GNU make, $(CURDIR) is actually a built-in variable. It is the location where the Makefile resides the current working directory, which is probably where the Makefile is, but not always.
OUTPUT_PATH = /project1/bin/$(notdir $(CURDIR))
See Appendix A Quick Reference in http://www.gnu.org/software/make/manual/make.html
THIS_DIR := $(dir $(abspath $(firstword $(MAKEFILE_LIST))))
I like the chosen answer, but I think it would be more helpful to actually show it working than explain it.
/tmp/makefile_path_test.sh
#!/bin/bash -eu
# Create a testing dir
temp_dir=/tmp/makefile_path_test
proj_dir=$temp_dir/dir1/dir2/dir3
mkdir -p $proj_dir
# Create the Makefile in $proj_dir
# (Because of this, $proj_dir is what $(path) should evaluate to.)
cat > $proj_dir/Makefile <<'EOF'
path := $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
cwd := $(shell pwd)
all:
#echo "MAKEFILE_LIST: $(MAKEFILE_LIST)"
#echo " path: $(path)"
#echo " cwd: $(cwd)"
#echo ""
EOF
# See/debug each command
set -x
# Test using the Makefile in the current directory
cd $proj_dir
make
# Test passing a Makefile
cd $temp_dir
make -f $proj_dir/Makefile
# Cleanup
rm -rf $temp_dir
Output:
+ cd /tmp/makefile_path_test/dir1/dir2/dir3
+ make
MAKEFILE_LIST: Makefile
path: /private/tmp/makefile_path_test/dir1/dir2/dir3
cwd: /tmp/makefile_path_test/dir1/dir2/dir3
+ cd /tmp/makefile_path_test
+ make -f /tmp/makefile_path_test/dir1/dir2/dir3/Makefile
MAKEFILE_LIST: /tmp/makefile_path_test/dir1/dir2/dir3/Makefile
path: /tmp/makefile_path_test/dir1/dir2/dir3
cwd: /tmp/makefile_path_test
+ rm -rf /tmp/makefile_path_test
NOTE: The function $(patsubst %/,%,[path/goes/here/]) is used to strip the trailing slash.
I tried many of these answers, but on my AIX system with gnu make 3.80 I needed to do some things old school.
Turns out that lastword, abspath and realpath were not added until 3.81. :(
mkfile_path := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
mkfile_dir:=$(shell cd $(shell dirname $(mkfile_path)); pwd)
current_dir:=$(notdir $(mkfile_dir))
As others have said, not the most elegant as it invokes a shell twice, and it still has the spaces issues.
But as I don't have any spaces in my paths, it works for me regardless of how I started make:
make -f ../wherever/makefile
make -C ../wherever
make -C ~/wherever
cd ../wherever; make
All give me wherever for current_dir and the absolute path to wherever for mkfile_dir.
The simple, correct, modern way:
For GNU make >= 3.81, which was introduced in 2006
ROOT_DIR := $(dir $(realpath $(lastword $(MAKEFILE_LIST))))
MAKEFILE_LIST changes as include files come in and out of scope. The last item is the current file.
lastword plucks the last item (Makefile name, relative to pwd)
realpath is built-in to make, and resolves to a canonical path from filesystem root
dir trims off the filename, leaving just the directory.
Here is one-liner to get absolute path to your Makefile file using shell syntax:
SHELL := /bin/bash
CWD := $(shell cd -P -- '$(shell dirname -- "$0")' && pwd -P)
And here is version without shell based on #0xff answer:
CWD := $(abspath $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST))))))
Test it by printing it, like:
cwd:
#echo $(CWD)
As far as I'm aware this is the only answer here that works correctly with spaces:
space:=
space+=
CURRENT_PATH := $(subst $(lastword $(notdir $(MAKEFILE_LIST))),,$(subst $(space),\$(space),$(shell realpath '$(strip $(MAKEFILE_LIST))')))
It essentially works by escaping space characters by substituting ' ' for '\ ' which allows Make to parse it correctly, and then it removes the filename of the makefile in MAKEFILE_LIST by doing another substitution so you're left with the directory that makefile is in. Not exactly the most compact thing in the world but it does work.
You'll end up with something like this where all the spaces are escaped:
$(info CURRENT_PATH = $(CURRENT_PATH))
CURRENT_PATH = /mnt/c/Users/foobar/gDrive/P\ roje\ cts/we\ b/sitecompiler/
Example for your reference, as below:
The folder structure might be as:
Where there are two Makefiles, each as below;
sample/Makefile
test/Makefile
Now, let us see the content of the Makefiles.
sample/Makefile
export ROOT_DIR=${PWD}
all:
echo ${ROOT_DIR}
$(MAKE) -C test
test/Makefile
all:
echo ${ROOT_DIR}
echo "make test ends here !"
Now, execute the sample/Makefile, as;
cd sample
make
OUTPUT:
echo /home/symphony/sample
/home/symphony/sample
make -C test
make[1]: Entering directory `/home/symphony/sample/test'
echo /home/symphony/sample
/home/symphony/sample
echo "make test ends here !"
make test ends here !
make[1]: Leaving directory `/home/symphony/sample/test'
Explanation, would be that the parent/home directory can be stored in the environment-flag, and can be exported, so that it can be used in all the sub-directory makefiles.
use {} instead of ()
cur_dir=${shell pwd}
parent_dir=${shell dirname ${shell pwd}}}
Solution found here : https://sourceforge.net/p/ipt-netflow/bugs-requests-patches/53/
The solution is : $(CURDIR)
You can use it like that :
CUR_DIR = $(CURDIR)
## Start :
start:
cd $(CUR_DIR)/path_to_folder
update 2018/03/05
finnaly I use this:
shellPath=`echo $PWD/``echo ${0%/*}`
# process absolute path
shellPath1=`echo $PWD/`
shellPath2=`echo ${0%/*}`
if [ ${shellPath2:0:1} == '/' ] ; then
shellPath=${shellPath2}
fi
It can be executed correct in relative path or absolute path.
Executed correct invoked by crontab.
Executed correct in other shell.
show example, a.sh print self path.
[root#izbp1a7wyzv7b5hitowq2yz /]# more /root/test/a.sh
shellPath=`echo $PWD/``echo ${0%/*}`
# process absolute path
shellPath1=`echo $PWD/`
shellPath2=`echo ${0%/*}`
if [ ${shellPath2:0:1} == '/' ] ; then
shellPath=${shellPath2}
fi
echo $shellPath
[root#izbp1a7wyzv7b5hitowq2yz /]# more /root/b.sh
shellPath=`echo $PWD/``echo ${0%/*}`
# process absolute path
shellPath1=`echo $PWD/`
shellPath2=`echo ${0%/*}`
if [ ${shellPath2:0:1} == '/' ] ; then
shellPath=${shellPath2}
fi
$shellPath/test/a.sh
[root#izbp1a7wyzv7b5hitowq2yz /]# ~/b.sh
/root/test
[root#izbp1a7wyzv7b5hitowq2yz /]# /root/b.sh
/root/test
[root#izbp1a7wyzv7b5hitowq2yz /]# cd ~
[root#izbp1a7wyzv7b5hitowq2yz ~]# ./b.sh
/root/./test
[root#izbp1a7wyzv7b5hitowq2yz ~]# test/a.sh
/root/test
[root#izbp1a7wyzv7b5hitowq2yz ~]# cd test
[root#izbp1a7wyzv7b5hitowq2yz test]# ./a.sh
/root/test/.
[root#izbp1a7wyzv7b5hitowq2yz test]# cd /
[root#izbp1a7wyzv7b5hitowq2yz /]# /root/test/a.sh
/root/test
[root#izbp1a7wyzv7b5hitowq2yz /]#
old:
I use this:
MAKEFILE_PATH := $(PWD)/$({0%/*})
It can show correct if executed in other shell and other directory.
One line in the Makefile should be enough:
DIR := $(notdir $(CURDIR))

One child process per for loop in make?

Let me first write a quick Makefile as a showcase:
#!/bin/make -f
folders := $(shell find -mindepth 1 -maxdepth 1 -type d -print)
make_dir:
#mkdir -p "test0"
pwd_test:
#cd "test0" && pwd
#pwd
pwd_all:
#for f in $(folders); do \
cd "$${f}" && pwd; \
pwd; \
cd ..; \
done
First do make make_dir and then see the different results:
➜ so make pwd_test
/data/cache/tmp/so/test0
/data/cache/tmp/so
➜ so make pwd_all
/data/cache/tmp/so/test0
/data/cache/tmp/so/test0
You see that in the for loop it is necessary to do cd ... Apparently, now there is no child process spawn for the cd X && pwd command, while that is normally the case. Is this behaviour specific to make or specific to my shell?
Make spawns a new process for each command in the rule. Since the for loop is one command you get only one process.
Take a look at Recipe Execution
Edit:
Each line in a makefile gets it own subshell. Commands that have
\ tells make that the next line should be part of the current line.
The reason the for loop get its own subshell is because make see the line as
#for f in $(folders); do cd "$${f}" && pwd; pwd; cd ..; done
MadScientist explains it fairly well. Any command that you can type in your
shell in one line will be executed by make in one subshell or process.
If you were to run this in ksh, ksh would be passed
for f in $(folders); do cd "$${f}" && pwd; pwd; cd ..; done and it would be
run in that one subshell. If ksh did not have a for loop implemented this
probably would error and make would say the command returned some error code.
Explanation of pwd_test
pwd_test:
#cd "test0" && pwd
#pwd
#cd "test0" && pwd is seen as one line so the subshell updates its current
working directory and then prints out what the current working is.
#pwd At this line make spawns a new subshell that contains the old working
directory (or the directory make was called form) and pwd prints that
directory.

Define make variable at rule execution time

In my GNUmakefile, I would like to have a rule that uses a temporary directory. For example:
out.tar: TMP := $(shell mktemp -d)
echo hi $(TMP)/hi.txt
tar -C $(TMP) cf $# .
rm -rf $(TMP)
As written, the above rule creates the temporary directory at the time that the rule is parsed. This means that, even I don't make out.tar all the time, many temporary directories get created. I would like to avoid my /tmp being littered with unused temporary directories.
Is there a way to cause the variable to only be defined when the rule is fired, as opposed to whenever it is defined?
My main thought is to dump the mktemp and tar into a shell script but that seems somewhat unsightly.
In your example, the TMP variable is set (and the temporary directory created) whenever the rules for out.tar are evaluated. In order to create the directory only when out.tar is actually fired, you need to move the directory creation down into the steps:
out.tar :
$(eval TMP := $(shell mktemp -d))
#echo hi $(TMP)/hi.txt
tar -C $(TMP) cf $# .
rm -rf $(TMP)
The eval function evaluates a string as if it had been typed into the makefile manually. In this case, it sets the TMP variable to the result of the shell function call.
edit (in response to comments):
To create a unique variable, you could do the following:
out.tar :
$(eval $#_TMP := $(shell mktemp -d))
#echo hi $($#_TMP)/hi.txt
tar -C $($#_TMP) cf $# .
rm -rf $($#_TMP)
This would prepend the name of the target (out.tar, in this case) to the variable, producing a variable with the name out.tar_TMP. Hopefully, that is enough to prevent conflicts.
A relatively easy way of doing this is to write the entire sequence as a shell script.
out.tar:
set -e ;\
TMP=$$(mktemp -d) ;\
echo hi $$TMP/hi.txt ;\
tar -C $$TMP cf $# . ;\
rm -rf $$TMP ;\
I have consolidated some related tips here: Multi-line bash commands in makefile
Another possibility is to use separate lines to set up Make variables when a rule fires.
For example, here is a makefile with two rules. If a rule fires, it creates a temp dir and sets TMP to the temp dir name.
PHONY = ruleA ruleB display
all: ruleA
ruleA: TMP = $(shell mktemp -d testruleA_XXXX)
ruleA: display
ruleB: TMP = $(shell mktemp -d testruleB_XXXX)
ruleB: display
display:
echo ${TMP}
Running the code produces the expected result:
$ ls
Makefile
$ make ruleB
echo testruleB_Y4Ow
testruleB_Y4Ow
$ ls
Makefile testruleB_Y4Ow
I dislike "Don't" answers, but... don't.
make's variables are global and are supposed to be evaluated during makefile's "parsing" stage, not during execution stage.
In this case, as long as the variable local to a single target, follow #nobar's answer and make it a shell variable.
Target-specific variables, too, are considered harmful by other make implementations: kati, Mozilla pymake. Because of them, a target can be built differently depending on if it's built standalone, or as a dependency of a parent target with a target-specific variable. And you won't know which way it was, because you don't know what is already built.

Resources