Having some trouble with GNUMAKE $(subst) I cannot explain
My goal is to transform a "SRC_DIR" directory into an "OBJ_DIR"
In one case it works, the other it does not - I do not understand why.
I thought, that the two "HERE{1|2}" constructs below would be identical, it seems they are not.
I do not believe it is a "delayed execution" problem because I am using the := (evaluate now, immediately in all cases)
PROJ_ROOT :=$(HOME)/a
BUILD_DIR :=$(PROJ_ROOT)/build/debug
HERE1:=$(shell pwd)
HERE2:=`pwd`
OK1 := $(subst $(PROJ_ROOT),$(BUILD_DIR),$(HERE1))
BAD := $(subst $(PROJ_ROOT),$(BUILD_DIR),$(HERE2))
TEST=$(subst ee,EE,feet on the street)
all:
#echo PR=$(PROJ_ROOT)
#echo BR=$(BUILD_DIR)
#echo H1=$(HERE1)
#echo H2=$(HERE2)
#echo OK1 is $(OK1)
#echo BAD IS $(BAD)
#echo TEST=$(TEST)
The output is below, the OK1 is correct,
but BAD is wrong It should be the same ask OK1
PR=/home/foobar/a
BR=/home/foobar/a/build/debug
H1=/home/foobar/a/src/one/two/three
H2=/home/foobar/a/src/one/two/three
OK1 is /home/foobar/a/build/debug/src/one/two/three
BAD IS /home/foobar/a/src/one/two/three
TEST=fEEt on the strEEt
background: I have a large project (400+ source files) spread over about 10 directories, each directory has a src folder, ie: $(PROJECT_ROOT)/libfoo/src/foo.c, and $(PROJECT_ROOT)/libbar/src/bar.c - Unlike the "autoconfigure" method where SRCDIR != BUILD_DIR (you configure in the build dir, and create makefiles in the build dir, I am doing the opposite - I have a prebuilt makefile that should create/populate a build directory structure with object files.
To do that, I need to create various subdirectories under $(PROJECT_ROOT)/build" that MIRROR the source folders. For example the makefiles should create: $(PROJECT_ROOT)/build/debug/libfoo/src/foo.o, and $(PROJECT_ROOT)/build/debug/libbar/src/bar.o
thus I need to 'slice the 'lib{foo|bar}/src and paste this into/onto the root of the build directory using makefile tricks.
As mentioned in the comments:
HERE1:=$(shell pwd)
HERE2:=`pwd`
The first line runs the make shell function and sets the value of HERE1 to the result of running the pwd command, which is what you expected. The second line sets the value of HERE2 to the literal string `pwd`.
When you use subst on the string `pwd` obviously nothing happens.
Once again, people are led astray by premature addition of #. Never, ever, ever add # to your recipes until after your makefile works completely.
If you removed the # from the lines that used HERE2 then it would be immediately obvious why things worked differently. Make would print out the command it was going to run and you could see that HERE2 is not replaced by make but that it's the shell that is running the pwd command:
echo H2=`pwd`
H2=/home/foobar/a/src/one/two/three
and:
echo BAD IS `pwd`
BAD IS /home/foobar/a/src/one/two/three
Related
I need to automate a variable alignment in my Makefile. My Makefile's full file path is:
/home/e2/branchname/projectname/modulename/Makefile
In my Makefile, I have a variable BUILD_DIR, a part of which should be equal to the branchname in the full path.
So I did this:
BRANCH_NAME= $(shell cd ../.. && basename "$PWD" && cd projectname/modulename)
BUILD_DIR=$(HOME)/$(BRANCH_NAME)/build
Apparently I expected BUILD_DIR to be ~/branchname/build here, but after make I got ~/WD/build instead. I think it's most likely that I got a wrong BRANCH_NAME. Is there something wrong with what I did? And if yes I'd like to get some advice about how to do it correctly.
Thanks.
It's because $ has a special meaning to Make, so if you want to pass that up to shell you have to "escape" it. In case of Make, you escape the dollar sign by doubling. So you have to use $$PWD.
Also, what you are doing is not really the best way - it is always best to avoid the shell and use Make functionality if possible. In your case, the best way to do what you want is this:
BUILD_DIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST)))/../../build)
You have to put the above line in the makefile in question, near the top, so that it is before you include any other makefiles.
I came up with this:
ENVIRONMENT := $(shell basename $(dir $(abspath $(dir $$PWD))))
If this were executed, you'd have:
ENVIRONMENT=projectname
I am trying to compile for different software directories with different optimization levels etc. I created the following makefile to do so:
OWNER = betsy molly fred
DOG = poodle mutt doberman
COLOUR = brown red yellow
ATTR = big small
LEGS = 0 3
#we want every possible combination to be excercised
OUTPUT_STUFF = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))
.PHONY: all
all: $(OUTPUT_STUFF)
define PROGRAM_template
own = $(1)
dog = $(2)
col = $(3)
attr = $(4)
legs = $(5)
BUILD_DIR = new/$(own)/$(dog)/$(col)/$(attr)/$(legs)
#for each build directory, we are going to put a file in it containing the build dir. string
$$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $$(BUILD_DIR)
#echo "$$(BUILD_DIR)" > $$(BUILD_DIR)/dogInfo.txt
endef
#call the function many times
$(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),$(eval $(call PROGRAM_template,$(own),$(dog),$(col),$(attr),$(legs))))))))
As you can see, this simple test program loops through different combinations of owner, dog etc. The end goal is to have a directory, new, that has all owners as dirs, and in those, all dogs, etc. At the bottom is just a file with the path in it.
When I run this, the output is:
new/betsy/poodle/brown/big/0/dogInfo.txt
mkdir new/fred/doberman/yellow/small/3
mkdir: cannot create directory `new/fred/doberman/yellow/small/3': No such file or directory
make: *** [new/betsy/poodle/brown/big/0/dogInfo.txt] Error 1
So, for some reason, the target is ok, but the seemingly exact same variable is the last in my loops. Fundamentally, I don't understand what is happening that well.
Weird foreach + user-defined function behavior in Makefiles seems to answer, but I don't fully get it. In my mind, when the function is called, it fills in all instances with one $, and the escaped ones become $(BUILD_DIR). It then 'pastes' the code to the temporary file, and after it's done all the calls it evaluates the file, substituting the variables as normal.
One (ugly) solution I thought of is to make the BUILD_DIR variable different every time like so:
B_D_$(1)_$(2)_$(3)_$(4)_$(5) = ~~~
Alex is correct (although I think he means recipe, not receipt :-)). The best way to debug complex eval issues is to replace the eval function with a call to info instead. So if you have something like:
$(foreach A,$(STUFF),$(eval $(call func,$A)))
then you can rewrite this as:
$(foreach A,$(STUFF),$(info $(call func,$A)))
Now make will print out to you exactly what the eval is going to parse. It's usually pretty clear, looking at the makefile output, what the problem is. In your case you'll see something like this in the output (leaving out all the extra variable settings):
BUILD_DIR = new/betsy/poodle/brown/big/0
$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $(BUILD_DIR)
#echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt
BUILD_DIR = new/betsy/poodle/brown/big/3
$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $(BUILD_DIR)
#echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt
etc. Notice how you're setting the global variable BUILD_DIR every time. In make, variables have only one value (at a time). While make is reading the makefile it expands the target and prerequisite lists immediately, so whatever value BUILD_DIR has at that time will be used for targets/prerequisites, so this works for you.
But when make finishes reading the makefile, the value of BUILD_DIR will always be the last thing you set it to; in this case new/fred/doberman/yellow/small/3. Now make starts to invoke the recipes for each target, and when it does that it will expand BUILD_DIR in the recipes then, and so ALL the recipes will get that same value.
As Alex points out, you should ensure that your recipe uses only automatic variables like $#, which are set correctly for each rule. If you do that you'll notice that you don't really need to redefine the rule at all because it's actually the same recipe for all the targets. And if you notice THAT, you'll notice you don't need the whole eval or call complexity in the first place.
All you have to do is compute the names of all the targets, then write a single rule:
ALLDOGINFO = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))
$(ALLDOGINFO):
#echo "$#"
mkdir $(dir $#)
#echo "$(dir $#)" > $#
If you don't want the trailing slash you have to use $(patsubst %/,%,$(dir $#)) instead.
The problem is that when $$(BUILD_DIR) is evaluated in receipt, the loop is already complete. The solution is to rewrite the receipt:
$$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $$(#D)
#echo "$$(#D)" > $$#
I don't think your problem is necessarily with something to do with make. This command:
mkdir new/fred/doberman/yellow/small/3
will fail if one of the parent directories (for example, yellow) doesn't already exist. The error it spits out in this case is the one you're getting, so it seems likely this is the case. If you want a command that makes all parent directories of a given directory as needed, you should run mkdir -p, like this:
mkdir -p $$(BUILD_DIR)
See the mkdir man page for a full description of what -p does.
What is the correct way to get the directory where the currently executing makefile resides?
I'm currently using export ROOT=$(realpath $(dir $(lastword $(MAKEFILE_LIST)))) and am running into some problems where when running make with the exact same options will result in different values for ROOT. About 90%of the time it has the correct value, but in the remaining 10% there are a number of invalid paths.
realpath,abspath,lastword and a couple of more functions were only introduced in GNU Make 3.81 [See ref]. Now you can get the current filename in older versions using words and word:
THIS_MAKEFILE:=$(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
But I am not sure of a workaround for realpath without going to the shell. e.g. this works with make v3.80:
THIS_MAKEFILE_PATH:=$(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
THIS_DIR:=$(shell cd $(dir $(THIS_MAKEFILE_PATH));pwd)
THIS_MAKEFILE:=$(notdir $(THIS_MAKEFILE_PATH))
all:
#echo "This makefile is $(THIS_MAKEFILE) in the $(THIS_DIR) directory"
Which gives
$ make -f ../M
This makefile is M in the /home/sandipb directory
Ref: http://cvs.savannah.gnu.org/viewvc/make/NEWS?revision=2.93&root=make&view=markup
$(shell pwd) is not correct since the makefile might exist in a directory other than pwd (as allowed by make -f).
The OP's proposed
export ROOT=$(realpath $(dir $(lastword $(MAKEFILE_LIST))))
is fine, except, s/he probably wants to use firstword instead, especially if the top level makefile (potentially) includes other makefile(s) prior to assiging to ROOT.
The OPs 10% problem could be explained if there was a conditional include 10% of the time prior to the assignment, but, hey, that's a guess...
For your convenience, when GNU make starts (after it has processed any -C options)
it sets the variable CURDIR to the pathname of the current working directory. This value
is never touched by make again: in particular note that if you include files from other
directories the value of CURDIR does not change. The value has the same precedence it
would have if it were set in the makefile (by default, an environment variable CURDIR will
not override this value). Note that setting this variable has no impact on the operation of
make (it does not cause make to change its working directory, for example).
all:
echo $(CURDIR)
Consider, if you will, the following situation:
$ pwd
/tmp/submake_example
$ head $(find -type f)
==> ./subdir/Makefile <==
subtarget:
echo "executing subtarget from directory $$(pwd)"
==> ./Makefile <==
include subdir/Makefile
$ make subtarget
echo "executing subtarget from directory $(pwd)"
executing subtarget from directory /tmp/submake_example
Is there a way that I can have the command for the target subtarget executed from within the directory subdir? I'd like to be able to access the targets of included makefiles from the command line in the main directory.
After some digging and experimentation, I've discovered a couple of things.
The approach recommended by this reference, which refers to the paper Recursive Make Considered Harmful, is pretty complicated, and involves manually maintaining a directory stack throughout the makefile inclusion heirarchy.
I also discovered, independently of any reference, that the last element of the variable $(MAKEFILE_LIST) will, at least for some basic test cases, contain the name of the makefile that's currently being included. This means that it's possible to store its directory in a simply expanded, target-specific variable with code like
foo: dir := $(dir $(lastword $(MAKEFILE_LIST)))
That's it. Then you can just use $(dir) within that rule like any other variable; it retains the value that it assumed during makefile inclusion. cd to it, put it in command paths, whatever.
You have to be careful to put the assignment to dir before any include directives in that makefile.
This won't work if there's a space in the directory name, but I gather that pretty much nothing in make works if there are spaces in file or directory names.
It's also notable that this approach uses features only found in GNU make. Sounds like a killer app to me, though I remain a bit perplexed as to why they don't just provide the directory name in a builtin variable or function.
Anyway this seems to work, at least at first glance. I'm still pretty much just floating this one without any real backup, so comments on the practicality of this approach, and alternative suggestions, are very welcome.
I'm trying to consolidate some build information by using a common makefile. My problem is that I want to use that makefile from different subdirectory levels, which makes the working directory value (pwd) unpredictable. For example:
# Makefile.common
TOP := $(shell pwd)
COMPONENT_DIR := $(TOP)/component
COMPONENT_INC := $(COMPONENT_DIR)/include
COMPONENT_LIB := $(COMPONENT_DIR)/libcomponent.a
If I include Makefile.common from a subdirectory, like so, the $(TOP) directory is incorrect and everything else follows suit:
# other_component/Makefile
include ../Makefile.common
# $(COMPONENT_LIB) is incorrectly other_component/component
What's the best way to get Makefile.common to use its own directory path instead of the more fickle pwd?
You should be able to use the MAKEFILE_LIST variable, like this:
# This must be the first line in Makefile.common
TOP := $(dir $(firstword $(MAKEFILE_LIST)))
From the documentation:
As make reads various makefiles, including any obtained from the MAKEFILES variable, the command line, the default files, or from include directives, their names will be automatically appended to the MAKEFILE_LIST variable. They are added right before make begins to parse them. This means that if the first thing a makefile does is examine the last word in this variable, it will be the name of the current makefile. Once the current makefile has used include, however, the last word will be the just-included makefile.
Try this:
ROOT_DIR := $(dir $(realpath $(lastword $(MAKEFILE_LIST))))
Edit: Be sure to use := instead of = because the latter causes make to use late-binding and MAKEFILE_LIST may have changed due to later includes.
Have you tried doing:
# Makefile.common
TOP ?= $(shell pwd)
COMPONENT_DIR := $(TOP)/component
COMPONENT_INC := $(COMPONENT_DIR)/include
COMPONENT_LIB := $(COMPONENT_DIR)/libcomponent.a
# other_component/Makefile
TOP ?= ..
include ../Makefile.common
Using the ?= construct will keep TOP from being redefined if it is already set. You can set it to the appropriate value based on where you are in the tree when you invoke make. I confess it's been awhile since I've used GNU make so this may not work or may need some tweaks.
My solution:
cwd := $(shell readlink -en $(dir $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))))
This also works for calls like make -f /opt/some/dir/Makefile whenn your in /opt/other/path/subdir.
write the common stuff in common.mk. Then put the common.mk in the default directories that Make looks for when it encounters an include statement. See the manual for common directories Make looks for.
You could also put the common.mk in custom directory, and then type make -I customdir.
Inside the Makefile in each subfolder, you do
include common.mk
That is all. No need to worry about path and moving things around.