Related
I know variables can be exported to sub-makefiles: Communicating Variables to a Sub-make
Example:
Makefile:
export PWD := $(shell pwd)
target:
#echo $(PWD)
#cd somewhere; $(MAKE)
somewhere/Makefile
target:
#echo $(PWD)
Supposing that the first Makefile is located at /path/to/first/makefile, the code above will print:
/path/to/first/makefile
/path/to/first/makefile
My question is: is there a way to let the variable PWD be implicitly evaluated inside sub-makefiles?
The output should look like this:
/path/to/first/makefile
/path/to/first/makefile/somewhere
So far I can only think of:
Exporting the literal declaration and use the function eval
Do it somehow with .SECONDEXPANSION
Put the declaration into a separate file and include it
in both the first and the second Makefile
All this solution are explicits: they imply code to be added to the sub-makefiles.
What I'm searching is an implicit solution which will change only the code inside the first Makefile.
Plus, honestly...the first two solutions are so ugly I would rather declare manually PWD in every sub-makefile.
[EDIT]
Just to make it more clear: the variable PWD is just an example, I'm not trying to obtain the path of every Makefile.
Use ${MAKEFILE_LIST} variable and let make change directories for you:
[max#earth:~/tmp]$ cat Makefile
target:
#echo $(abspath $(lastword ${MAKEFILE_LIST}))
${MAKE} -C somewhere $#
[max#earth:~/tmp]$ cat somewhere/Makefile
target:
#echo $(abspath $(lastword ${MAKEFILE_LIST}))
[max#earth:~/tmp]$ make target
/home/max/tmp/Makefile
make -C somewhere target
make[1]: Entering directory '/home/max/tmp/somewhere'
/home/max/tmp/somewhere/Makefile
make[1]: Leaving directory '/home/max/tmp/somewhere'
That prints the full path to the makefile being processed. Use $(dir $(abspath $(lastword ${MAKEFILE_LIST}))) to chop off filename Makefile.
Assume I have a build-target foo:
foo:foo.c
$(CC) $(CFLAGS) $(ARGS) -c foo.c -o foo
Now, ARGS is something that I pass on the command line:
$ make ARGS:=-DX=1 foo
So, I need to bypass make's cleverness, because the foo target does not only depend on which files have changed, but also on the value of ARGS.
Is there something in make to do this? My hack (see answer) doesn't seem to be the most elegant but it works. Anything better?
Here is a general solution to your specific problem.
You want to be able to depend on a variable as a prerequisite. That is, you can make it a prerequisite to any target in your makefile, and when the value of the variable changes, you rebuild those targets.
Here is a function that does that, you use this function to declare a variable to be dependable, and then you can use it as a prerequisite.
Note that if the variable is not used on the command line, it will still mean that variable still has a value, namely, the empty string.
define DEPENDABLE_VAR
.PHONY: phony
$1: phony
#if [[ `cat $1 2>&1` != '$($1)' ]]; then \
echo -n $($1) > $1 ; \
fi
endef
#declare ARGS to be dependable
$(eval $(call DEPENDABLE_VAR,ARGS))
foo:foo.c ARGS
$(CC) $(CFLAGS) $(ARGS) -c foo.c -o foo
In fact, we could omit the need for "declaration", and just write a similar function that will make all variables dependable by default. But I don't like that. I prefer that the users that modify makefiles I write, declare their intentions explicitly. It is good for them :)
My solution was to create a dummy phony target:
.PHONY:dummy
dummy:
#:
and have foo depend on dummy if ARGS is nonempty:
foo:foo.c $(patsubst %,dummy,$(ARGS))
Note on Mark's excellent answer
The bare necessities of Mark's answer are actually very simple. It really boils down to just:
.PHONY: phony
ARGS: phony
#if [[ `cat ARGS 2>&1` != '$(ARGS)' ]]; then echo -n $(ARGS) >ARGS; fi
The rest of his code is just to let you reproduce the recipe for other variable names without repeating yourself. Useful in practice, but the above version will help you see what's going on more easily.
In fact, my answer can even be made general (like Mark's) for any variable name, but in a less complicated way as follows:
.PHONY: phony
.ARG~%: phony
#if [[ `cat .ARG~$* 2>&1` != '$($*)' ]]; then echo -n $($*) >.ARG~$*; fi
Then you simply add .ARG~MYVARNAME as a dependency to any target to make that target depend on variable MYVARNAME.
Note that the dot in front of .ARG~% simply causes it to create a dependency-tracking file that is 'hidden' in linux.
So in your case, you would do:
foo: foo.c .ARG~ARGS
$(CC) $(CFLAGS) $(ARGS) -c foo.c -o foo
I don't understand how the other solutions are supposed to work. If the ARGS target is .PHONY or depends on a .PHONY, then it will always be run, right?
Here is my solution using the $(file) function in newer versions of gmake:
.PHONY: FORCE
define DEPENDABLE_VAR
$(1):
echo -n $($(1)) > $(1)
ifneq ("$(file <$(1))","$($(1))")
$(1): FORCE
endif
endef
#declare ARGS to be dependable
$(eval $(call DEPENDABLE_VAR,ARGS))
foo: foo.c ARGS
touch foo
And the result:
~/stuff/makevars> make foo ARGS=1
echo -n 1 > ARGS
touch foo
~/stuff/makevars> make foo ARGS=1
make: 'foo' is up to date.
~/stuff/makevars> make foo ARGS=2
echo -n 2 > ARGS
touch foo
~/stuff/makevars> make foo ARGS=2
make: 'foo' is up to date.
How can I have the variable for $(MAKEFILE) be defined during target execution?
Basically I have a few make files in subdirectories that are named for a specific platform "Makefile.aix" and just Makefile in all other directories. I would like to set a variable for $(MAKEFILE) that gets defined in each subdirectory. Code would look something like this.
MAKEFILE = Makefile
SUBDIR = ./sub ./sub2
ifneq ($(wildcard Makefile),)
MAKEFILE = Makefile
else
MAKEFILE = Makefile.$(PLATFORM)
endif
all:;
#for i in $(SUBDIR);\
do (\
echo Making $$i ...;\
cd $$i;\
make -f $(MAKEFILE)\
); done
Is there just one Makefile.$(PLATFORM) in each subdirectory, or are there several, for different platforms?
In the first case, you could do something like this:
SUBDIR = ./sub ./sub2
define script
cd $(1); \
$(MAKE) -f Makefile*
endef
all:
$(foreach dir, $(SUBDIR), $(call script,$(dir)))
(The empty line inside the define is significant. It can be omitted, if you add a semicolon at the end of the line $(MAKE) ..., leading to one long command line, containing the commands for all directories, which will then be executed in one chunk.)
An alternative script would be (just a matter of personal preference which you like better):
define script
$(MAKE) -C $(1) -f $(notdir $(wildcard $(1)/Makefile*))
endef
If there are several Makefile.$(PLATFORM) files in a directory it becomes more difficult. I'll have to think about that one some more.
UPDATE: In response to your comment, something like this should work:
define script
$(MAKE) -C $(1) -f $(notdir $(wildcard $(1)/Makefile $(1)/Makefile.$(PLATFORM)))
endef
Following your logic, I'd propose update do () section with:
do (\
echo Making $$i ...;\
cd $$i;\
if [ -f Makefile.$(PLATFORM) ] \
then\
make -f Makefile.$(PLATFORM) \
else\
make -f Makefile\
fi\
); done
This is actually not a make style, but I can't suggest anything better without specific of your project
You can do most of this, including the loop over directories, using GNU make's built-in functions. Put the following in a central place, say $(TOP_DIR)/mk/subdir.mk:
makefile-for-dir = \
$(if $(wildcard $(1)/Makefile),Makefile,Makefile.$(PLATFORM))
make-recursive = \
$(foreach _d,$(1),$(MAKE) -C $(_d) -f $(call makefile-for-dir,$(_d)) && ) :
In each makefile that start recursive makes, use
include $(TOP_DIR)/mk/subdir.mk
SUBDIRS = dir1 dir2 dir3
.PHONY: all
all:
+#$(call make-recursive,$(SUBDIRS))
I have a "lib" directory in my applications main directory, which contains an arbitrary number of subdirectories, each having its own Makefile.
I would like to have a single Makefile in the main directory, that calls each subdirectory's Makefile. I know this is possible if I manually list the subdirs, but I would like to have it done automatically.
I was thinking of something like the following, but it obviously does not work. Note that I also have clean, test, etc. targets, so % is probably not a good idea at all.
LIBS=lib/*
all: $(LIBS)
%:
(cd $#; $(MAKE))
Any help is appreciated!
The following will work with GNU make:
LIBS=$(wildcard lib/*)
all: $(LIBS)
.PHONY: force
$(LIBS): force
cd $# && pwd
If there might be something other than directories in lib, you could alternatively use:
LIBS=$(shell find lib -type d)
To address the multiple targets issue, you can build special targets for each directory, then strip off the prefix for the sub-build:
LIBS=$(wildcard lib/*)
clean_LIBS=$(addprefix clean_,$(LIBS))
all: $(LIBS)
clean: $(clean_LIBS)
.PHONY: force
$(LIBS): force
echo make -C $#
$(clean_LIBS): force
echo make -C $(patsubst clean_%,%,$#) clean
There is also a way of listing sub-directories with gmake commands only, without using any shell commands:
test:
#echo $(filter %/, $(wildcard lib/*/))
This will list all sub-directories with trailing '/'. To remove it you can use the substitute pattern:
subdirs = $(filter %/, $(wildcard lib/*/))
test:
#echo $(subdirs:%/=%)
Then to actually create rules executing makefiles in each sub-directory you can use a small trick - a phony target in a non-existent directory. I think in this case an example will tell more than any explanation:
FULL_DIRS =$(filter %/, $(wildcard lib/*/))
LIB_DIRS =$(FULL_DIRS:%/=%)
DIRS_CMD =$(foreach subdir, $(LIB_DIRS), make-rule/$(subdir))
make-rule/%:
cd $* && $(MAKE)
all: DIRS_CMD
Basically, target 'all' lists all sub-directories as prerequisites. For example, if LIB_DIRS contained lib/folder1 lib/folder2 then the expansion would look like this:
all: make-rule/lib/folder1 make-rule/lib/folder2
Then 'make', in order to execute rule 'all', tries to match each prerequisite with an existing target. In this case the target is 'make-rule/%:', which uses '$*' to extract the string after 'make-rule/' and uses it as argument in the recipe. For example, the first prerequisite would be matched and expanded like this:
make-rule/lib/folder1:
cd lib/folder1 && $(MAKE)
What if you want to call different targets than all in an unknown number of subdirectories?
The following Makefile uses macros so create a forwarding dummy-target for a number of subdirectories to apply the given target from the command line to each of them:
# all direct directories of this dir. uses "-printf" to get rid of the "./"
DIRS=$(shell find . -maxdepth 1 -mindepth 1 -type d -not -name ".*" -printf '%P\n')
# "all" target is there by default, same logic as via the macro
all: $(DIRS)
$(DIRS):
$(MAKE) -C $#
.PHONY: $(DIRS)
# if explcit targets where given: use them in the macro down below. each target will be delivered to each subdirectory contained in $(DIRS).
EXTRA_TARGETS=$(MAKECMDGOALS)
define RECURSIVE_MAKE_WITH_TARGET
# create new variable, with the name of the target as prefix. it holds all
# subdirectories with the target as suffix
$(1)_DIRS=$$(addprefix $(1)_,$$(DIRS))
# create new target with the variable holding all the subdirectories+suffix as
# prerequisite
$(1): $$($1_DIRS)
# use list to create target to fullfill prerequisite. the rule is to call
# recursive make into the subdir with the target
$$($(1)_DIRS):
$$(MAKE) -C $$(patsubst $(1)_%,%,$$#) $(1)
# and make all targets .PHONY
.PHONY: $$($(1)_DIRS)
endef
# evaluate the macro for all given list of targets
$(foreach t,$(EXTRA_TARGETS),$(eval $(call RECURSIVE_MAKE_WITH_TARGET,$(t))))
Hope this helps. Really helpfull when dealing with paralelism: make -j12 clean all in a tree with makefiles having these targets... As always: playing with make is dangerous, different meta-levels of programming are too close together ,-)
I want to create directories using makefile. My project directory is like this
+--Project
+--output
+--source
+Testfile.cpp
+Makefile
I want to put all the objects and output into the respective output folder. I want to create folder structure which would be like this after compiling.
+--Project
+--output
+--debug (or release)
+--objs
+Testfile.o
+Testfile (my executable file)
+--source
+Testfile.cpp
+Makefile
I tried with several options, but could not succeed. Please help me to make directories using make file. I'm posting my Makefile for your consideration.
#---------------------------------------------------------------------
# Input dirs, names, files
#---------------------------------------------------------------------
OUTPUT_ROOT := output/
TITLE_NAME := TestProj
ifdef DEBUG
TITLE_NAME += _DEBUG
else
ifdef RELEASE
TITLE_NAME += _RELEASE
endif
endif
# Include all the source files here with the directory tree
SOURCES := \
source/TestFile.cpp \
#---------------------------------------------------------------------
# configs
#---------------------------------------------------------------------
ifdef DEBUG
OUT_DIR := $(OUTPUT_ROOT)debug
CC_FLAGS := -c -Wall
else
ifdef RELEASE
OUT_DIR := $(OUTPUT_ROOT)release
CC_FLAGS := -c -Wall
else
$(error no build type defined)
endif
endif
# Put objects in the output directory.
OUT_O_DIR := $(OUT_DIR)/objs
#---------------------------------------------------------------------
# settings
#---------------------------------------------------------------------
OBJS = $(SOURCES:.cpp=.o)
DIRS = $(subst /,/,$(sort $(dir $(OBJS))))
DIR_TARGET = $(OUT_DIR)
OUTPUT_TARGET = $(OUT_DIR)/$(TITLE_NAME)
CC_FLAGS +=
LCF_FLAGS :=
LD_FLAGS :=
#---------------------------------------------------------------------
# executables
#---------------------------------------------------------------------
MD := mkdir
RM := rm
CC := g++
#---------------------------------------------------------------------
# rules
#---------------------------------------------------------------------
.PHONY: all clean title
all: title
clean:
$(RM) -rf $(OUT_DIR)
$(DIR_TARGET):
$(MD) -p $(DIRS)
.cpp.o:
#$(CC) -c $< -o $#
$(OBJS): $(OUT_O_DIR)/%.o: %.cpp
#$(CC) -c $< -o $#
title: $(DIR_TARGET) $(OBJS)
In my opinion, directories should not be considered targets of your makefile, either in technical or in design sense. You should create files and if a file creation needs a new directory then quietly create the directory within the rule for the relevant file.
If you're targeting a usual or "patterned" file, just use make's internal variable $(#D), that means "the directory the current target resides in" (cmp. with $# for the target). For example,
$(OUT_O_DIR)/%.o: %.cpp
#mkdir -p $(#D)
#$(CC) -c $< -o $#
title: $(OBJS)
Then, you're effectively doing the same: create directories for all $(OBJS), but you'll do it in a less complicated way.
The same policy (files are targets, directories never are) is used in various applications. For example, git revision control system doesn't store directories.
Note: If you're going to use it, it might be useful to introduce a convenience variable and utilize make's expansion rules.
dir_guard=#mkdir -p $(#D)
$(OUT_O_DIR)/%.o: %.cpp
$(dir_guard)
#$(CC) -c $< -o $#
$(OUT_O_DIR_DEBUG)/%.o: %.cpp
$(dir_guard)
#$(CC) -g -c $< -o $#
title: $(OBJS)
This would do it - assuming a Unix-like environment.
MKDIR_P = mkdir -p
.PHONY: directories
all: directories program
directories: ${OUT_DIR}
${OUT_DIR}:
${MKDIR_P} ${OUT_DIR}
This would have to be run in the top-level directory - or the definition of ${OUT_DIR} would have to be correct relative to where it is run. Of course, if you follow the edicts of Peter Miller's "Recursive Make Considered Harmful" paper, then you'll be running make in the top-level directory anyway.
I'm playing with this (RMCH) at the moment. It needed a bit of adaptation to the suite of software that I am using as a test ground. The suite has a dozen separate programs built with source spread across 15 directories, some of it shared. But with a bit of care, it can be done. OTOH, it might not be appropriate for a newbie.
As noted in the comments, listing the 'mkdir' command as the action for 'directories' is wrong. As also noted in the comments, there are other ways to fix the 'do not know how to make output/debug' error that results. One is to remove the dependency on the the 'directories' line. This works because 'mkdir -p' does not generate errors if all the directories it is asked to create already exist. The other is the mechanism shown, which will only attempt to create the directory if it does not exist. The 'as amended' version is what I had in mind last night - but both techniques work (and both have problems if output/debug exists but is a file rather than a directory).
Or, KISS.
DIRS=build build/bins
...
$(shell mkdir -p $(DIRS))
This will create all the directories after the Makefile is parsed.
make in, and off itself, handles directory targets just the same as file targets. So, it's easy to write rules like this:
outDir/someTarget: Makefile outDir
touch outDir/someTarget
outDir:
mkdir -p outDir
The only problem with that is, that the directories timestamp depends on what is done to the files inside. For the rules above, this leads to the following result:
$ make
mkdir -p outDir
touch outDir/someTarget
$ make
touch outDir/someTarget
$ make
touch outDir/someTarget
$ make
touch outDir/someTarget
This is most definitely not what you want. Whenever you touch the file, you also touch the directory. And since the file depends on the directory, the file consequently appears to be out of date, forcing it to be rebuilt.
However, you can easily break this loop by telling make to ignore the timestamp of the directory. This is done by declaring the directory as an order-only prerequsite:
# The pipe symbol tells make that the following prerequisites are order-only
# |
# v
outDir/someTarget: Makefile | outDir
touch outDir/someTarget
outDir:
mkdir -p outDir
This correctly yields:
$ make
mkdir -p outDir
touch outDir/someTarget
$ make
make: 'outDir/someTarget' is up to date.
TL;DR:
Write a rule to create the directory:
$(OUT_DIR):
mkdir -p $(OUT_DIR)
And have the targets for the stuff inside depend on the directory order-only:
$(OUT_DIR)/someTarget: ... | $(OUT_DIR)
All solutions including the accepted one have some issues as stated in their respective comments. The accepted answer by #jonathan-leffler is already quite good but does not take into effect that prerequisites are not necessarily to be built in order (during make -j for example). However simply moving the directories prerequisite from all to program provokes rebuilds on every run AFAICT.
The following solution does not have that problem and AFAICS works as intended.
MKDIR_P := mkdir -p
OUT_DIR := build
.PHONY: directories all clean
all: $(OUT_DIR)/program
directories: $(OUT_DIR)
$(OUT_DIR):
${MKDIR_P} $(OUT_DIR)
$(OUT_DIR)/program: | directories
touch $(OUT_DIR)/program
clean:
rm -rf $(OUT_DIR)
I've just come up with a fairly reasonable solution that lets you define the files to build and have directories be automatically created. First, define a variable ALL_TARGET_FILES that holds the file name of every file that your makefile will be build. Then use the following code:
define depend_on_dir
$(1): | $(dir $(1))
ifndef $(dir $(1))_DIRECTORY_RULE_IS_DEFINED
$(dir $(1)):
mkdir -p $$#
$(dir $(1))_DIRECTORY_RULE_IS_DEFINED := 1
endif
endef
$(foreach file,$(ALL_TARGET_FILES),$(eval $(call depend_on_dir,$(file))))
Here's how it works. I define a function depend_on_dir which takes a file name and generates a rule that makes the file depend on the directory that contains it and then defines a rule to create that directory if necessary. Then I use foreach to call this function on each file name and eval the result.
Note that you'll need a version of GNU make that supports eval, which I think is versions 3.81 and higher.
given that you're a newbie, I'd say don't try to do this yet. it's definitely possible, but will needlessly complicate your Makefile. stick to the simple ways until you're more comfortable with make.
that said, one way to build in a directory different from the source directory is VPATH; i prefer pattern rules
OS independence is critical for me, so mkdir -p is not an option. I created this series of functions that use eval to create directory targets with the prerequisite on the parent directory. This has the benefit that make -j 2 will work without issue since the dependencies are correctly determined.
# convenience function for getting parent directory, will eventually return ./
# $(call get_parent_dir,somewhere/on/earth/) -> somewhere/on/
get_parent_dir=$(dir $(patsubst %/,%,$1))
# function to create directory targets.
# All directories have order-only-prerequisites on their parent directories
# https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html#Prerequisite-Types
TARGET_DIRS:=
define make_dirs_recursively
TARGET_DIRS+=$1
$1: | $(if $(subst ./,,$(call get_parent_dir,$1)),$(call get_parent_dir,$1))
mkdir $1
endef
# function to recursively get all directories
# $(call get_all_dirs,things/and/places/) -> things/ things/and/ things/and/places/
# $(call get_all_dirs,things/and/places) -> things/ things/and/
get_all_dirs=$(if $(subst ./,,$(dir $1)),$(call get_all_dirs,$(call get_parent_dir,$1)) $1)
# function to turn all targets into directories
# $(call get_all_target_dirs,obj/a.o obj/three/b.o) -> obj/ obj/three/
get_all_target_dirs=$(sort $(foreach target,$1,$(call get_all_dirs,$(dir $(target)))))
# create target dirs
create_dirs=$(foreach dirname,$(call get_all_target_dirs,$1),$(eval $(call make_dirs_recursively,$(dirname))))
TARGETS := w/h/a/t/e/v/e/r/things.dat w/h/a/t/things.dat
all: $(TARGETS)
# this must be placed after your .DEFAULT_GOAL, or you can manually state what it is
# https://www.gnu.org/software/make/manual/html_node/Special-Variables.html
$(call create_dirs,$(TARGETS))
# $(TARGET_DIRS) needs to be an order-only-prerequisite
w/h/a/t/e/v/e/r/things.dat: w/h/a/t/things.dat | $(TARGET_DIRS)
echo whatever happens > $#
w/h/a/t/things.dat: | $(TARGET_DIRS)
echo whatever happens > $#
For example, running the above will create:
$ make
mkdir w/
mkdir w/h/
mkdir w/h/a/
mkdir w/h/a/t/
mkdir w/h/a/t/e/
mkdir w/h/a/t/e/v/
mkdir w/h/a/t/e/v/e/
mkdir w/h/a/t/e/v/e/r/
echo whatever happens > w/h/a/t/things.dat
echo whatever happens > w/h/a/t/e/v/e/r/things.dat
See https://www.oreilly.com/library/view/managing-projects-with/0596006101/ch12.html
REQUIRED_DIRS = ...
_MKDIRS := $(shell for d in $(REQUIRED_DIRS); \
do \
[[ -d $$d ]] || mkdir -p $$d; \
done)
$(objects) : $(sources)
As I use Ubuntu, I also needed add this at the top of my Makefile:
SHELL := /bin/bash # Use bash syntax
I use the makefiles in windows environment and my simple solution is as follows,
Create a target makedir and add it as a prerequisites to where ever it is required.
# Default goal
all: gccversion makedir build finalize list sizeafter completed
The makedir target is (applicable only in windows environment)
makedir:
#IF NOT EXIST $(subst /,\,$(BUILD_DIR)) mkdir $(subst /,\,$(BUILD_DIR)) 2> NULL
#IF NOT EXIST $(subst /,\,$(OUTPUT_DIR)) mkdir $(subst /,\,$(OUTPUT_DIR)) 2> NULL
#IF NOT EXIST $(subst /,\,$(DEP_DIR)) mkdir $(subst /,\,$(DEP_DIR)) 2> NUL
#IF NOT EXIST $(subst /,\,$(OBJ_DIR)) mkdir $(subst /,\,$(OBJ_DIR)) 2> NUL
$(subst /,\,$(BUILD_DIR)) converts the directory separator / to \ and
mkdir $(subst /,\,$(BUILD_DIR)) 2> NUL redirects the error if any.
src_dir := src
obj_dir := obj
build_dir := build
dirs := $(src_dir) $(obj_dir) $(build_dir) # new variable
all: $(dirs) $(other_dependencies) # added dependency (*before* any others)
$(dirs): # rule which makes missing directories
mkdir $#
Won't clutter your terminal with "cannot create directory" error messages. If the directories exist, they don't need to be built.
Works like any other dependency, only requires one rule and one variable.