Suppose I have a project with two or more subfolders foo, bar, etc. I have a Makefile at the root of the project, and also in each subdirectory.
I would like to have certain targets (e.g. all, clean, etc) to run recursively in each subdirectory. My top-level Makefile looks like this:
all:
$(MAKE) -C foo all
$(MAKE) -C bar all
clean:
$(MAKE) -C foo clean
$(MAKE) -C bar clean
Seems to me there's a lot of duplication going on here. Is there a way I can avoid such tedious duplication in my Makefiles?
How about this:
SUBDIRS=foo bar
all clean:
for dir in $(SUBDIRS) ; do \
$(MAKE) -C $$dir $# ; \
done
A bit scary:
SUBDIRS=foo bar
SUBDIR_TARGETS=all clean
define subdir_rule
$(2): $(1)-$(2)
$(1)-$(2):
make -C $(1) $(2)
endef
$(foreach targ,$(SUBDIR_TARGETS),\
$(foreach dir,$(SUBDIRS),\
$(eval $(call subdir_rule,$(dir),$(targ)))))
Here's how I'd do it:
SUBDIRS=foo bar baz
TARGETS = clean all whatever
.PHONY:$(TARGETS)
# There really should be a way to do this as "$(TARGETS):%:TARG=%" or something...
all: TARG=all
clean: TARG=clean
whatever: TARG=whatever
$(TARGETS): $(SUBDIRS)
#echo $# done
.PHONY: $(SUBDIRS)
$(SUBDIRS):
#$(MAKE) -s -C $# $(TARG)
Related
I have a top level Makefile that define a list of subdirectories containing sub-Makefiles.
Each sub-Makefile define the same targets: all, install, clean, test, verify, style...
How do I make the top level Makefile call all my subdirectories Makefile with the same target ?
It works for the default target, like this:
SUBDIRS = dir1 dir2 dir3
all: subdirs
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $#
.PHONY: subdirs $(SUBDIRS)
But how do I extend that to the other non default targets ?
If you use GNU make you could use the MAKECMDGOALS variable and the filter function:
SUBDIRS := dir1 dir2 dir3
TARGETS := all install clean test verify style
$(TARGETS): subdirs
#echo making top $#
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(filter $(TARGETS),$(MAKECMDGOALS))
.PHONY: subdirs $(TARGETS) $(SUBDIRS)
%::
#echo making top $#
Demo:
$ make --no-print-directory all install foobar
make -C dir1 all install
make -C dir2 all install
make -C dir3 all install
making top all
making top install
making top foobar
Note that filter is needed to avoid passing top-only goals to the sub-makes.
If, for any reason, this MAKECMDGOALS based solution does not work for you, here are some other, but a bit more complex, solutions.
Target-specific variables
Target-specific variables propagate to prerequisites. So you can define a pattern-specific variable (SUBGOAL) for each of your top-level goals and use it in the subdir's recipe:
SUBDIRS := dir1 dir2 dir3
all: SUBGOAL := all
all: subdirs
#echo making top $#
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(SUBGOAL)
.PHONY: subdirs $(SUBDIRS)
If you have many such top-level goals and use GNU make you can factorize a bit with the foreach and eval GNU make functions:
SUBDIRS := dir1 dir2 dir3
TARGETS := all install clean test verify style
$(foreach t,$(TARGETS),$(eval $(t): GOAL := $(t)))
$(TARGETS): subdirs
#echo making top $#
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(GOAL)
.PHONY: subdirs $(SUBDIRS) $(TARGETS)
One important drawback is that as you use the same phony subdirectory target (dir1) for several top phony targets (all, install...) you cannot invoke make with several top goals:
$ make --no-print-directory all install
make -C dir1 all
make -C dir2 all
make -C dir3 all
making top all
making top install
will only make all in the subdirectories. If this is a problem you can use the next solution.
One phony target (all-dir1) per top-subdirectory combination
Let's do this programmatically with a 2-variables macro and a foreach-eval-call combination (GNU make only):
SUBDIRS := dir1 dir2 dir3
TARGETS := all install clean test verify style
$(TARGETS): subdirs
#echo making top $#
TARGETS_X_SUBDIRS :=
# $(1): top target
# $(2): subdirectory target
define TARGETS_X_SUBDIRS_RULES
$(1): $(1)-$(2)
$(1)-$(2):
$(MAKE) -C $(2) $(1)
TARGETS_X_SUBDIRS += $(1)-$(2)
endef
$(foreach t,$(TARGETS),$(foreach s,$(SUBDIRS),$(eval $(call TARGETS_X_SUBDIRS_RULES,$(t),$(s)))))
.PHONY: $(TARGETS) $(TARGETS_X_SUBDIRS)
Demo:
$ make --no-print-directory all install
make -C dir1 all
make -C dir2 all
make -C dir3 all
making top all
make -C dir1 install
make -C dir2 install
make -C dir3 install
making top install
Renaud's answer is more complete, but I also found this technique that worked fine for my use case:
all:
%:
make -C dir1 $#
make -C dir2 $#
make -C dir3 $#
In this case, I don't have the subdirs list in a variable, but they are spelled out in the commands for the generic target %:. It requires to add an extra make -C in front of every subdirectory, and a # in the back, but otherwise pretty maintainable and easy to read.
But it probably would not work if there was any other targets specific to the top-level Makefile.
I'm using similar code to https://stackoverflow.com/a/17845120/1375972 which includes:
TOPTARGETS := zip test clean
SUBDIRS := $(wildcard */.)
$(TOPTARGETS): $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(MAKECMDGOALS)
This runs whichever make command is passed (from TOPTARGETS) on every subdirectory, which is what I want.
However I want to have a target deploy which has this same behaviour only when an environment variable is set. Here's what I've tried:
deploy:
ifdef GITLAB_CI
#echo "GITLAB_CI variable is defined, running deploy on all subdirectories"
$(MAKE) -C $(SUBDIRS) $(MAKECMDGOALS)
else
#echo "snip"
endif
Notice the $(MAKE) line is the same as the $(SUBDIRS) one, just with the $# replaced with $(SUBDIRS) directly.
So the logic is when it runs in my CI it'll run the deploy recursively on all subdirectories, but when it's run locally it doesn't. The problem is that the $(SUBDIRS) in deploy doesn't behave as expected. When I run make deploy in a directory with 2 subdirectories:
> make deploy
GITLAB_CI variable is defined, running deploy on all subdirectories
/Library/Developer/CommandLineTools/usr/bin/make -C subdir1/. subdir2/. deploy
make[1]: *** No rule to make target `subdir2/.'. Stop.
make: *** [deploy] Error 2
Compared to make clean (one of my TOPTARGETS):
> make clean
/Library/Developer/CommandLineTools/usr/bin/make -C subdir1/. clean
/Library/Developer/CommandLineTools/usr/bin/make -C subdir2/. clean
So when the TOPTARGETS are used, the $# seems to unroll $(SUBDIRS) in a different way to when I write the same line myself with $# substituted to $(SUBDIRS). Is there any way to get that behaviour myself in the deploy line? Or do I need to write my own for loop over $(SUBDIRS) inside that target?
You could keep what works and add a minimal deploy rule:
TOPTARGETS := zip test clean
SUBDIRS := $(wildcard */.)
$(TOPTARGETS): $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(MAKECMDGOALS)
ifdef GITLAB_CI
deploy: $(SUBDIRS)
else
deploy:
#echo "snip"
endif
Or, maybe even simpler:
TOPTARGETS := zip test clean
ifdef GITLAB_CI
TOPTARGETS += deploy
else
deploy:
#echo "snip"
endif
SUBDIRS := $(wildcard */.)
$(TOPTARGETS): $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(MAKECMDGOALS)
I wrote this Makefile with a PHONY target that loops over a set of subdirectories and executes $(MAKE)
SUBDIRS:= dir_1 dir_2 dir_3 dir_n
libs:$(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $#;
.PHONY: libs $(SUBDIRS)
I have tried with this code but it doesn't work.
SUBDIRS:=dir_1 dir_2 dir_3 dir_n
CLEANDIRS:=dir_1 dir_2 dir_3 dir_n
libs:$(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $#;
clean:$(CLEANDIRS)
$(CLEANDIRS):
$(MAKE) -C $# clean;
.PHONY: libs $(SUBDIRS)
.PHONY: clean $(CLEANDIRS)
Any suggestion? Thanks
Is it possible to simplify a Makefile of the following form in order to avoid code repetition:
dir1/foo:
make -C dir1 foo
dir1/bar:
make -C dir1 bar
dir2/baz:
make -C dir2 baz
...
clean:
make -C dir1 clean
make -C dir2 clean
...
I imagine I could specify only:
MY_TARGETS=dir1/foo dir1/bar dir2/baz ...
And than have some general rules to derive targets, as presented in the Makefile above.
You haven't given us much information. Recursive Make can be a good solution, if your main makefile is as simple as your example (which I doubt).
You could do this:
%:
$(MAKE) -C $(dir $#) $(notdir $#)
clean:
$(MAKE) -C dir1 clean
$(MAKE) -C dir2 clean
...
If that clean recipe is too long, you can simplify it:
DIRS := dir1 dir2 ...
clean:
#for x in $(DIRS); do $(MAKE) -C $$x clean; done
Or if you don't like having that DIRS list there, you could have Make generate it, depending on whether you want to make clean in all subdirectories, or all that match the dir* pattern, or... you haven't given us enough information.
I have a Makefile for a C++ program that uses automatic dependency generation. The %.d recipe is taken from the GNU Make manual.
The problem is that somehow "Makefile" is being added as a target and then an implicit rule is causing it to assume it's an executable and using my src/%.cpp rule to try to compile src/Makefile.cpp. When looking at the debug info, this always happens right after the include is executed.
No need to remake target `build/Sprite.d'.
Considering target file `Makefile'.
Looking for an implicit rule for `Makefile'.
...
Trying pattern rule with stem `Makefile'.
Trying implicit prerequisite `Makefile.o'.
Looking for a rule with intermediate file `Makefile.o'.
I know include causes the given Makefiles to be rebuilt if necessary. Does it also try to rebuild the current Makefile? If so how do I stop it, and if not, then why is "Makefile" being added as a target?
Also, the include is executed, causing the .d files to be remade even if I specify a target on the command line, such as make clean. Is there any way to stop that from happening?
# $(call setsuffix,newsuffix,files)
# Replaces all the suffixes of the given list of files.
setsuffix = $(foreach file,$2,$(subst $(suffix $(file)),$1,$(file)))
# $(call twinfile,newdir,newsuffix,oldfile)
# Turns a path to one file into a path to a corresponding file in a different
# directory with a different suffix.
twinfile = $(addprefix $1,$(call setsuffix,$2,$(notdir $3)))
MAIN = main
SOURCE_DIR = src/
INCLUDE_DIR = include/
BUILD_DIR = build/
SOURCES = $(wildcard $(SOURCE_DIR)*.cpp)
OBJECTS = $(call twinfile,$(BUILD_DIR),.o,$(SOURCES))
DEPENDENCIES = $(call twinfile,$(BUILD_DIR),.d,$(SOURCES))
CXX = g++
LIBS = -lpng
CXXFLAGS = -I $(INCLUDE_DIR)
.PHONY: all
all: $(MAIN)
$(MAIN): $(OBJECTS)
$(CXX) $(LIBS) $^ -o $(MAIN)
include $(DEPENDENCIES)
%.o: $(BUILD_DIR)stamp
$(CXX) $(CXXFLAGS) -c $(call twinfile,$(SOURCE_DIR),.cpp,$#) -o $#
$(BUILD_DIR)%.d: $(SOURCE_DIR)%.cpp $(BUILD_DIR)stamp
# echo Generate dependencies for $ $#.$$$$; \
sed 's,\($*\)\.o[ :]*,$(BUILD_DIR)\1.o $# : ,g' $#; \
rm -f $#.$$$$
$(BUILD_DIR)stamp:
mkdir -p $(BUILD_DIR)
touch $#
.PHONY: clean
clean:
rm -rf $(BUILD_DIR)
.PHONY: printvars
printvars:
# echo $(SOURCES)
# echo $(OBJECTS)
# echo $(DEPENDENCIES)
Make will always try to remake the Makefile before executing the Makefile. To do so, make will look for rules which can be used to recreate the Makefile. Make will look for quite a few implicit rules and other obscure methods to (re)create the Makefile.
In your case, make somehow decided that the pattern rule %.o: $(BUILD_DIR)/stamp should be used to recreate the Makefile, which failed.
To prevent make from remaking the Makefile you can write a rule with an empty recipe:
Makefile: ;
Read the chapter Remaking Makefiles in the make manual for more explanation.
About the included Makefiles: Included Makefiles will always be included, regardless of the target. If the included makefiles are missing (or older than their prerequisites) then they will first be (re)created. That means a make clean will first generate the .d Makefiles, only to delete them again.
You can prevent the including for specific goals by wraping the include directive in a conditional:
ifneq ($(MAKECMDGOALS),clean)
include $(DEPENDENCIES)
endif
Here is your entire Makefile with some fixes. I marked the places where I changed something.
# Makefile
# $(call setsuffix,newsuffix,files)
# Replaces all the suffixes of the given list of files.
setsuffix = $(foreach file,$2,$(subst $(suffix $(file)),$1,$(file)))
# $(call twinfile,newdir,newsuffix,oldfile)
# Turns a path to one file into a path to a corresponding file in a different
# directory with a different suffix.
twinfile = $(addprefix $1/,$(call setsuffix,$2,$(notdir $3)))
MAIN = main
SOURCE_DIR = src
INCLUDE_DIR = include
BUILD_DIR = build
SOURCES = $(wildcard $(SOURCE_DIR)/*.cpp)
OBJECTS = $(call twinfile,$(BUILD_DIR),.o,$(SOURCES))
DEPENDENCIES = $(call twinfile,$(BUILD_DIR),.d,$(SOURCES))
CXX = g++
LIBS = -lpng
CXXFLAGS = -I $(INCLUDE_DIR)
.PHONY: all
all: $(MAIN)
$(MAIN): $(OBJECTS)
$(CXX) $(LIBS) $^ -o $(MAIN)
# -------> only include if goal is not clean <---------
ifneq ($(MAKECMDGOALS),clean)
include $(DEPENDENCIES)
endif
# ---------> fixed this target <--------------
$(BUILD_DIR)/%.o: $(SOURCE_DIR)/%.cpp $(BUILD_DIR)/stamp
$(CXX) $(CXXFLAGS) -c $(call twinfile,$(SOURCE_DIR),.cpp,$#) -o $#
# ---------> and this target <---------------
$(BUILD_DIR)/%.d: $(SOURCE_DIR)/%.cpp $(BUILD_DIR)/stamp
# echo Generate dependencies for $#;
#set -e; rm -f $#; \
$(CC) -M $(CPPFLAGS) $< > $#.$$$$; \
sed 's,\($*\)\.o[ :]*,$(BUILD_DIR)\1.o $# : ,g' < $#.$$$$ > $#; \
rm -f $#.$$$$
$(BUILD_DIR)/stamp:
mkdir -p $(BUILD_DIR)
touch $#
.PHONY: clean
clean:
rm -rf $(BUILD_DIR)
.PHONY: printvars
printvars:
# echo $(SOURCES)
# echo $(OBJECTS)
# echo $(DEPENDENCIES)