I am trying to compile for different software directories with different optimization levels etc. I created the following makefile to do so:
OWNER = betsy molly fred
DOG = poodle mutt doberman
COLOUR = brown red yellow
ATTR = big small
LEGS = 0 3
#we want every possible combination to be excercised
OUTPUT_STUFF = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))
.PHONY: all
all: $(OUTPUT_STUFF)
define PROGRAM_template
own = $(1)
dog = $(2)
col = $(3)
attr = $(4)
legs = $(5)
BUILD_DIR = new/$(own)/$(dog)/$(col)/$(attr)/$(legs)
#for each build directory, we are going to put a file in it containing the build dir. string
$$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $$(BUILD_DIR)
#echo "$$(BUILD_DIR)" > $$(BUILD_DIR)/dogInfo.txt
endef
#call the function many times
$(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),$(eval $(call PROGRAM_template,$(own),$(dog),$(col),$(attr),$(legs))))))))
As you can see, this simple test program loops through different combinations of owner, dog etc. The end goal is to have a directory, new, that has all owners as dirs, and in those, all dogs, etc. At the bottom is just a file with the path in it.
When I run this, the output is:
new/betsy/poodle/brown/big/0/dogInfo.txt
mkdir new/fred/doberman/yellow/small/3
mkdir: cannot create directory `new/fred/doberman/yellow/small/3': No such file or directory
make: *** [new/betsy/poodle/brown/big/0/dogInfo.txt] Error 1
So, for some reason, the target is ok, but the seemingly exact same variable is the last in my loops. Fundamentally, I don't understand what is happening that well.
Weird foreach + user-defined function behavior in Makefiles seems to answer, but I don't fully get it. In my mind, when the function is called, it fills in all instances with one $, and the escaped ones become $(BUILD_DIR). It then 'pastes' the code to the temporary file, and after it's done all the calls it evaluates the file, substituting the variables as normal.
One (ugly) solution I thought of is to make the BUILD_DIR variable different every time like so:
B_D_$(1)_$(2)_$(3)_$(4)_$(5) = ~~~
Alex is correct (although I think he means recipe, not receipt :-)). The best way to debug complex eval issues is to replace the eval function with a call to info instead. So if you have something like:
$(foreach A,$(STUFF),$(eval $(call func,$A)))
then you can rewrite this as:
$(foreach A,$(STUFF),$(info $(call func,$A)))
Now make will print out to you exactly what the eval is going to parse. It's usually pretty clear, looking at the makefile output, what the problem is. In your case you'll see something like this in the output (leaving out all the extra variable settings):
BUILD_DIR = new/betsy/poodle/brown/big/0
$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $(BUILD_DIR)
#echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt
BUILD_DIR = new/betsy/poodle/brown/big/3
$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $(BUILD_DIR)
#echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt
etc. Notice how you're setting the global variable BUILD_DIR every time. In make, variables have only one value (at a time). While make is reading the makefile it expands the target and prerequisite lists immediately, so whatever value BUILD_DIR has at that time will be used for targets/prerequisites, so this works for you.
But when make finishes reading the makefile, the value of BUILD_DIR will always be the last thing you set it to; in this case new/fred/doberman/yellow/small/3. Now make starts to invoke the recipes for each target, and when it does that it will expand BUILD_DIR in the recipes then, and so ALL the recipes will get that same value.
As Alex points out, you should ensure that your recipe uses only automatic variables like $#, which are set correctly for each rule. If you do that you'll notice that you don't really need to redefine the rule at all because it's actually the same recipe for all the targets. And if you notice THAT, you'll notice you don't need the whole eval or call complexity in the first place.
All you have to do is compute the names of all the targets, then write a single rule:
ALLDOGINFO = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))
$(ALLDOGINFO):
#echo "$#"
mkdir $(dir $#)
#echo "$(dir $#)" > $#
If you don't want the trailing slash you have to use $(patsubst %/,%,$(dir $#)) instead.
The problem is that when $$(BUILD_DIR) is evaluated in receipt, the loop is already complete. The solution is to rewrite the receipt:
$$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $$(#D)
#echo "$$(#D)" > $$#
I don't think your problem is necessarily with something to do with make. This command:
mkdir new/fred/doberman/yellow/small/3
will fail if one of the parent directories (for example, yellow) doesn't already exist. The error it spits out in this case is the one you're getting, so it seems likely this is the case. If you want a command that makes all parent directories of a given directory as needed, you should run mkdir -p, like this:
mkdir -p $$(BUILD_DIR)
See the mkdir man page for a full description of what -p does.
Related
I tried to build a nested directories following: Iterating through a list of directories in a Makefile
Here is my script
LAYER1 = a b
LAYER2 = 1 2
$(LAYER1)/foo%/run: $(DIR)
mkdir -p $#
DIR: $(addsuffix /foo$(LAYER2)/run, $(LAYER1))
The expected output is creating a/foo1/run a/foo2/run b/foo1/run b/foo2/run but the output is only 1 directory named a. How do I create the expected directory structure?
What does the rule:
$(LAYER1)/foo%/run: $(DIR)
expand to? It expands to this:
a b/foo%/run: $(DIR)
(I don't know what $(DIR) is supposed to be) which is an explicit rule that tells make how to build two things: a and b/foo%/run (this is not a pattern rule because not all the targets contain %).
Since a here is the first target in the makefile, it's the one that will be run by default when you run make.
What does $(addsuffix /foo$(LAYER2)/run, $(LAYER1)) do? It takes every word in $(LAYER1) and prefixes it by the string /foo$(LAYER2)/run. What is that string? It's /foo1 2/run. So the result of this is:
a/foo1 2/run b/foo1 2/run
which means DIR has the files a/foo1, 2/run, b/foo1, and 2/run as dependencies.
If LAYER1 has multiple words you need to loop through it, not just use it as-is.
See MadScientist's answer as to why your answer would not work. As far as something that will work, you likely want code that looks like this:
LAYER1 = a b
LAYER2 = 1 2
DIRS := $(foreach L1,$(LAYER1),\
$(foreach L2,$(LAYER2),\
$(L1)/foo$(L2)/run))
all: | $(DIRS)
$(DIRS):
mkdir -p $#
Notice the use of order-only dependencies as the prerequisite type for $(DIRS)
I have a makefile that looks something like this:
include anotherFile.mk
all:
someStuff
The file anotherFile.mk is like this:
include yetAnotherFile.mk
export SOME_VAR = 93
The problem is that anotherFile.mk and yetAnotherFile.mk are in a different directory from my Makefile. So my makefile can't just be changed to this:
include $(OTHER_PROJECT_PATH)/anotherFile.mk
all:
someStuff
The problem with this approach is that the include statement in anotherFile.mk will fail because it will be searching in the current directory.
A partial solution that I found is to pass the --include-dir=$OTHER_PROJECT_PATH flag to the invocation of make, but that's a bit user-unfriendly.
So my question is: Is there something I can put inside my makefile that will add to the directories that make searches for when executing an include? Something like MAKE_INCLUDE_DIRS += $(OTHER_PROJECT_PATH)
Surprisingly there doesn't seem to be a good answer to that question. Forcing .INCLUDE_DIR doesn't help and there doesn't seem to be any way around invoking make with --include-dir=$OTHER_PROJECT_PATH.
It is however possible to put the appropriate recursive make invocation inside the makefile but, in order to get it to work for all reasonable cases it quickly becomes too complicated to be worth it. In summary it requires:
a top level condition to check if the OTHER_PROJECT_PATH is in .INCLUDE_DIR
the appropriate target with the recipe invoking make recursively
possibly additional targets if there are multiple command goals
the real make file enclosed in the else part of the conditional
You Makefile would look like this:
OTHER_PROJECT_PATH := other
ifeq (,$(filter $(OTHER_PROJECT_PATH), $(.INCLUDE_DIRS)))
# this is the mechanism to add the include dir in a recursive make
$(or $(firstword $(MAKECMDGOALS)),all):
$(MAKE) -I$(OTHER_PROJECT_PATH) $(MAKECMDGOALS)
# add empty targets for additional goals if needed
ifneq (,$(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS)))
$(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS)):
endif
else
# this is where the real makefile starts
all more:
echo $#: $< $^
include a.mak
endif
It still does not seem possible from a makefile, but if you have a script that sets up environment variables, you can use MAKEFLAGS (e.g. export MAKEFLAGS=I/your/path ordentlich on Linux, or SET on Windows)
Here is a Makefile example that uses double slashes in paths to targets:
out/file.txt:
#mkdir -p $(dir $#)
#echo aaa > $#
out//file.txt:
#mkdir -p $(dir $#)
#echo bbb > $#
make interprets this as two different targets. If you run make out/file.txt, the first rule will be executed. If you run make out//file.txt, the second rule will be executed.
Also, if you run make out///file.txt and file.txt does not exist, you'll get the error:
make: *** No rule to make target `out///file.txt'
However if the file exists, it just says:
make: Nothing to be done for `out///file.txt'
Also make builds targets successfully if you run make .///out/file.txt or even make .////././././././////.///.////out/file.txt
So, is there any defined behavior how make works with paths that are literally different, but point to the same file in the filesystem?
I got from this answer, that operating system itself doesn't differ such paths. But for make they are different.
The problem originates from the such usage:
my_target: $(SOME_DIR)/some_file
If SOME_DIR already has trailing slash, the code above doesn't work. It expands to something like this: some_dir//some_file and the rule for the specific case with double slashes doesn't exist.
How can such problems be avoided? Is there any path canonization means in make?
Here is direct workaround for my problem - creating a macro that trims trailing slashes:
trslashes = $(if $(filter %/,$(1)),$(call trslashes,$(patsubst %/,%,$(1))),$(1))
This macro must be used in every place where double slashes can cause a problem:
my_target: $(call trslashes,$(SOME_DIR))/some_file
If $(SOME_DIR) is empty the file from root directory will be used:
/some_file. If empty variable should mean current directory then another macro should be used:
trslashes_cur = $(if $(1),$(call trslashes,$(1)),.)
Then $(call trslashes,$(SOME_DIR))/some_file will expand to ./somefile.
I have a file "ORIGINAL", which, if updated, I would like to copy, modify, and distribute to a few places on the drive. The modification is made by a little bash script which takes one parameter, a parameter unique for each spawned remote file.
In my Makefile, I can do this with a separate rule/recipe for each parameter, like so:
parameters = AWK BAT CAT DOG
$(DEST_FILE_AWK) : $(ORIGINAL)
./copyAndModify "AWK" ## Creates $(ORIGINAL)_AWK, substed copy of ORIGINAL
mv - f $(ORIGINAL)_AWK $(DEST_FILE_AWK)
$(DEST_FILE_BAT) : $(ORIGINAL)
./copyAndModify "BAT" ## Creates $(ORIGINAL)_BAT, substed copy of ORIGINAL
mv - f $(ORIGINAL)_BAT $(DEST_FILE_BAT)
The dereferenced values of DEST_FILE_AWK and DEST_FILE_BAT have nothing to do with each other, but other than that, the two recipes above are exactly the same with the only difference the parameter, so I can't help but want to merge them into one super rule/recipe with a multiple target rule line.
But I just can't make it happen. I've tried all kinds of foreach() and other stuff in the target section of the rule, but the problem is that no matter what, I can't get the value of the parameter into the recipe part.
Is there a way?
With the information provided here the best you can do (assuming you're using GNU make) is an eval/call combination. As anishsane suggests, depending on the value of the DEST_FILE_* variables it might be possible to do something simpler.
But this should work:
define COPY_TO_DEST
$$(DEST_FILE_$1) : $$(ORIGINAL)
./copyAndModify "$1"
mv - f $$(ORIGINAL)_$1 $$#
endef
parameters = AWK BAT CAT DOG
$(foreach P,$(parameters),$(eval $(call COPY_TO_DEST,$P)))
It can be done without $(eval), at least in gnu make :)
Start with one recipe that specifies all of the targets, i.e. the list of targets is on the left side of the recipe. Let's assume we have a variable that holds the names of all these targets.
Now observe that both functions and variables will be evaluated separately for a given recipe as it gets expanded for each of the targets. Recall that, say $# is just a variable, and will be substituted separately for each target. Function calls behave the same.
Provide a list of types, and a list of type:target pairs. I presume that there's no need to put the targets into separate variables like you did ($(DEST_FILE_AWK) etc).
The TARGET_FOR_TYPE function takes the pairs and the types and generates a list of destination files.
The TYPE variable is assigned once for each target, computed by the TYPE_FOR_TARGET function. That way the repeated function call doesn't pollute the recipe :)
Note that the DESTINATIONS list contains plain filenames, without any further indirection.
types = AWK BAT
ORIGINAL = an_original
DESTINATIONS = \
AWK:dest_for_awk \
BAT:dest_for_bat
TARGET_FOR_TYPE = $(patsubst $(1):%,%,$(filter $(1):%,$(DESTINATIONS)))
TYPE_FOR_TARGET = $(patsubst %:$(1),%,$(filter %:$(1),$(DESTINATIONS)))
# Usage example for the functions above:
$(info type: $(call TYPE_FOR_TARGET,dest_for_awk))
$(info target: $(call TARGET_FOR_TYPE,AWK))
$(info $())
DEST_FILES = $(foreach type,$(types),$(call TARGET_FOR_TYPE,$(type)))
all: $(DEST_FILES)
$(DEST_FILES) : TYPE=$(call TYPE_FOR_TARGET,$#)
$(DEST_FILES) : $(ORIGINAL)
#echo ./copyAndModify $(TYPE)
#echo mv - f $(ORIGINAL)_$(TYPE) $#
I setup make to build my tree using recursive make. So the setup is
A/Makefile a.c
A/B/Makefile a.c
A/B/C/Makefile a.c
where if I issue the command make all from level A/ then make will travel down the tree building everything and then come back up. Each Makefile contains a list of folders below it to build. There is a common.inc file in the root which is read in each Makefile.
This is just a standard layout for recursive make, and nothing new. The details is gives in many places. here and here are examples.
My question is this: many times I'd like to do make all but only build things in the current folder, and not actually travel down the tree, may be because I want to test some changes in the current folder at this time. So I end up editing the current folder's Makefile by commenting out the SUBDIRS=A B C which lists all folders below, or by adding new special targets for this folder only. Both are annoying things to have to keep doing.
Does any one have an idea or a small example of a recursive makefile that uses a switch to tell it if it should travel down the tree or not when called? may be there is a way to call make and pass it some flag at the command line, and this flag is used to remove SUBDIRS=A B C ..... list so it only stops at the current folder level?
Just to be clear. I am using standard SUBDIRS in the Rules.mk, which each Makefile in the tree includes. Here is the part. I copied this from the net long time ago
$(SUBDIRS)::
#if test -d $#; then \
set $(EXIT_ON_ERROR); \
echo "cd $#; make $#"; \
cd $#; make $#; \
set +e; \
else \
echo "Skipping non-directory $#..."; \
fi \
$(CLICK_STOPWATCH);
endif
and in each folder Makefile I write
SUBDIRS = A B C
include Rules.mk
all:: .......
Then I just write make all to build. If there is a way to do make all LOOP=0 where LOOP is some value I pass it or an option or a string or something and then change the above SUBDIRS logic to check for the value of this LOOP and based on the value then do the recursive make or not, then the problem is solved. The default can be to LOOP=1 if it is missing from the command line.
But I do not know enough Make to program this type of logic.
You should use power of rules' depencies. Add your sources files to the dependencies of the rule called from the "root Makefile". If these files are up to date, the recursivity in an folder will stop because the rule is 'up-to-date', and nothing will be done.
Don't add .PHONY for all your rules in the sub-directory Makefile, otherwise recursives rules will be called.
Play with the dependencies of the rules can be the key to not make recursive call, but if you modify sources in each folder and want to build only from the root Makfile, you have to create another rules. With make all, the make binary may not know if you want build all your projet or not (if all your sources has been modified).
EDIT: choice by the command line
You're near the answer, you can set env var while calling your make all and test the value to decide calling recursivly or not.
CC=g++
SUBDIR=a b
all: ${SUBDIR} main.cc
${CC} main.cc
${SUBDIR}:
ifneq ($(MK_LOOP), 0)
#echo "trust the recursivity !"
${MAKE} -C $#
endif
.PHONY: ${SUBDIR}
If you don't set the MK_LOOP var or you set to something else than 0, it will not be equal to 0 so recursive Makefile will be call; if you set to 0, $(SUBDIR) rule do nothing
42SH $ MK_LOOP=0 make # no recur
42SH $ make all
trust the recursivity !
42SH $ make all MK_LOOP=1 # recur by default; same as : make all
trust the recursivity !
42SH $