I have a C/C++ project that is organized like this:
project
|
+---- c
| |
| +---- subproject1
| | |
| | +---- bin
| | |
| | +---- obj
| | |
| | +---- src
| |
| +---- subproject2
| |
| +---- bin
| |
| +---- obj
| |
| +---- src
|
+-----+ cpp
| |
| +---- subproject1
| | |
| | +---- bin
| | |
| | +---- obj
| | |
| | +---- src
etcetera
At the project level, there's a Makefile that contains the following:
SUBDIRS = c cpp
all: $(SUBDIRS)
.PHONY: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $#
The c and cpp sub-directories contain similar Makefiles with different SUBDIRS definitions. Typing make at the command prompt from the project, c or cpp directories causes make to change directory to each sudb-directory in SUBDIRS and run make with that sub-directory's Makefile. This behaves exactly as described in the GNU Make Manual and I'm very happy with it. The manual also states that you can add a target like clean after the $# and it will invoke that target in the sub-directory's Makefile. Also great, but what the manual doesn't give guidance on is what to do when SUBDIRS contains more than one sub-directory, as my project and c directories do. I'm going to give you the two methods that I know work and then I'll pose my question.
Add a target for each sub-directory in the clean target's recipe:
clean:
$(MAKE) -C c $#
$(MAKE) -C cpp $#
Loop over each sub-directory in the clean target's recipe (basically the same as method 1, but easier to add more sub-projects to):
clean:
#for subdir in $(SUBDIRS); do $(MAKE) -C $$subdir $#; done
My question is, is there a way of achieving the above that requires using only existing GNU make constructs, something like this (which doesn't work btw):
$(SUBDIRS) clean:
$(MAKE) -C c $#
I would declare per-directory phony clean targets and add them as pre-requisites of clean:
CLEANSUBDIRS = $(addprefix clean-,$(SUBDIRS))
.PHONY: clean $(CLEANSUBDIRS)
clean: $(CLEANSUBDIRS)
$(CLEANSUBDIRS): clean-%:
$(MAKE) -C $* clean
Related
Since sdcc have some limitations, like compiling one file at a time, I've tried hard to write a Makefile to perfect the automation of MCS-51 development, which have some requirements:
Source file (.c) expect main.c are stored in ProjectFolder/Sources/, while main.c are stored at the root of project folder.
Headers are stored in ProjectFolder/Includes/.
Outputs through compiling, linking and locating should be stored at ProjectFolder/Builds/
Makefile should be smart enough to find all source files, instead of type their file name by hand.
Makefile should be smart enough to if there are some files in Sources/, or there's only main.c in the project.
The file structure can be depicted like:
Project Folder
|
|- Sources
| |
| |(some source files, but OPTIONAL)
|
|- Includes
| |
| |(some headers, but OPTIONAL)
|
|- Builds
| |
| |(some .rel .o .hex files. OUTPUT here)
|
|- main.c
|
|- Makefile
Here's my solution but still have a problem. It cannot be used for project only have one file main.c which means no source file in Sources/.
INCLUDES = Includes/
SOURCES = Sources/
BUILDS = Builds/
CC = sdcc
CFLAGS = -o $(BUILDS)
LOADER = stcgal
LOADER_FLAGS = -P stc89
$(BUILDS)main.ihx: main.c $(BUILDS)main.rel
# Link
#$(CC) main.c $(shell find $(BUILDS) -name "*.rel" -not -name "main.rel" -maxdepth 1) $(CFLAGS)
#echo Link & Locate Succeeded
$(BUILDS)main.rel: $(SOURCES) $(BUILDS)
# Compile
#for f in $(shell ls $(SOURCES)*.c) ; do \
$(CC) -c $${f} $(CFLAGS) ; \
done
#echo Compile Succeeded
$(SOURCES):
#mkdir $(SOURCES)
$(BUILDS):
#mkdir $(BUILDS)
clean:
# Remove all files in build folder
#rm $(BUILDS)*
#echo Build Folder Cleaned
load: $(BUILDS)main.ihx
# Load data to MCU via USB port
#$(LOADER) $(LOADER_FLAGS) -p $(shell ls /dev/tty.usbserial*) $(BUILDS)main.ihx
Let's try something. First note that I've not looked at the load target.
Let's start with the same definition as you:
INCLUDES = Includes/
SOURCES = Sources/
BUILDS = Builds/
CC = sdcc
We need a variable with the source files from Sources. GNU Make has a wildcard functions which does the same thing as your find. See that I'm using := to have an immediate expansion of the value, so the wildcard will not be executed several times.
SRCFILES := $(wildcard $(SOURCES)*.c)
Now a variable with the .rel files. It is build from main.rel and the SRCFILES value:
RELFILES := $(BUILDS)main.rel $(SRCFILES:$(SOURCES)%.c=$(BUILDS)%.rel)
Let's define another variable with the flag to pass so the Includes directory is searched:
CPPFLAGS = -I$(INCLUDES)
Now we can define pattern rules to describe how to build .rel files from .c files. I'm using an order-only prerequisite for the build directory:
$(BUILDS)%.rel: $(SOURCES)%.c | $(BUILDS)
$(CC) $(CPPFLAGS) $(CFLAGS) -c -o $# -c $<
$(BUILDS)%.rel: %.c | $(BUILDS)
$(CC) $(CPPFLAGS) $(CFLAGS) -o $# -c $<
Let's define some usability targets:
.PHONY: all clean
all: $(BUILDS)main.ihx
clean:
rm $(BUILDS)*
And finally define how to build the targets which aren't handled by the pattern rules:
$(BUILDS)main.ihx: $(RELFILES) | $(BUILDS)
$(CC) $(LDFLAGS) -o $# $^ $(LOADLIBES) $(LDLIBS)
$(BUILDS):
mkdir $(BUILDS)
I've used a few variables (CC, CPPFLAGS, CFLAGS, LDFLAGS, LOADLIBES, LDLIBS) in the same way as they are used by the built-in rules of GNU Make.
I've kept your makefile behavior. There are good reasons to have Makefiles targeting to build in the current directory. Explaining them and modifying the Makefile for that is out of scope for this answer, you may look at MadScientist's GNU Make White Papers and the GNU Make Manual.
My problem is that a prerequisite (set by a variable) is not executed and even if executed manually before, required libraries are not found when linking. My Makefiles are non recursive, but the prerequisite is the external Software googletest and therefore called recursively, as the sources are not changed between several "make" runs.
My environment is:
$ make --version
GNU Make 4.2.1
Built for x86_64-unknown-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ uname -a
Linux archlinux 4.17.3-1-ARCH #1 SMP PREEMPT Tue Jun 26 04:42:36 UTC 2018 x86_64 GNU/Linux
This is the relevant directory structure:
|-- ExampleTrunk
| |-- BUILD
| | |-- BIN
| | |-- LIB
| | `-- OBJ
| |-- buildenv
| | |-- ...
| | |-- GTEST.mk
| | |-- Makefile
| | |-- commons.mk
| | |-- module_commons.mk
| | |-- pathdefs.mk
| |-- src
| `-- HelloWorld
| `-- module.mk
| `-- test
| `-- HelloWorld
| `-- module.mk
`-- Tools
`-- GoogleTest
`-- googletest
|-- make
|-- Makefile
The folder ExampleTrunk/BUILD and its subdirectories are immediately created in pathdefs.mk and therefore exist before further rule execution.
The start Makefile is ExampleTrunk/buildenv/Makefile:
...
export SHELL := $(shell which bash)
...
.DEFAULT_GOAL := all
#some common function definitions
include commons.mk
#get $(PROJECT_ROOT), $(REL_PROJECT_ROOT); immediately create $(GLOBAL_OBJ_PATH), $(GLOBAL_LIB_PATH), $(GLOBAL_BIN_PATH)
include pathdefs.mk
export BUILD_DIRS := $(GLOBAL_BIN_PATH) $(GLOBAL_LIB_PATH) $(GLOBAL_OBJ_PATH)
include $(REL_PROJECT_ROOT)/src/HelloWorld/module.mk
...
#only include test stuff, when tests shall be built or cleaned to save dependency calculation time in all other cases
ifeq "$(call givenStringEndsWithTest,$(MAKECMDGOALS))" "0"
include GTEST.mk
include $(REL_PROJECT_ROOT)/test/HelloWorld/module.mk
endif
all : $(programs) | $(BUILD_DIRS);
all_Test: $(testPrograms) | $(BUILD_DIRS);
$(GLOBAL_OBJ_PATH)/%.o: %.cpp | $(BUILD_DIRS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $< -o $#
clean:
$(RM) $(addprefix $(GLOBAL_BIN_PATH)/,$(programs)) $(addprefix $(GLOBAL_BIN_PATH)/,$(testPrograms))
$(RM) $(objectsToClean)
$(RM) $(objectsToClean:.o=.d)
clean_Test: clean gtest_clean gmock_clean;
The rules "all", "clean" and "clean_Test" resolve and execute their prerequisites as expected and run without errors. (evaluated by make ... -p) The rule "all_Test" produces errors as described below.
GTest.mk contains the following recursive rules:
gtest: | $(BUILD_DIRS)
[ -d "$(REL_GTEST_MAKEFILE_DIR)" ] && cd "$(REL_GTEST_MAKEFILE_DIR)" && $(MAKE)
gtest_clean:
[ -d "$(REL_GTEST_MAKEFILE_DIR)" ] && cd "$(REL_GTEST_MAKEFILE_DIR)" && $(MAKE) clean
The directory "$(REL_GTEST_MAKEFILE_DIR)" exists, because it is only another path representation to the ExampleTrunk/BUILD/* directories that were immediately created before by pathdefs.mk. When I call any of both rules directly they can be executed without errors. Because I adapted the googletest Makefile, the outputs are created in the ExampleTrunk/BUILD/* directories as required.
The module.mk files are very short, as they include the module_commons.mk which contains most of the things to be executed:
ExampleTrunk/src/HelloWorld/module.mk:
additionalPrograms :=
gmockIsRequired := no
include $(PROJECT_ROOT)/buildenv/module_commons.mk
ExampleTrunk/test/HelloWorld/module.mk:
additionalPrograms := HelloWorld
gmockIsRequired := no
include $(PROJECT_ROOT)/buildenv/module_commons.mk
ExampleTrunk/buildenv/module_commons.mk generates most of the variables automatically from the module.mk file location and settings.
When I exectue
[.....ExampleTrunk/buildenv]$ make HelloWorld
everythink works fine, but with
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test
it fails with a linker error (shown below). Here are the relevant *.mk parts for HelloWorld_Test with already resolved variables as shown from make HelloWorld_Test -p (variables with files/directories or flags are not written in resolved form to keep it short):
...
LDFLAGS_TEST := -L../BUILD/LIB -lgtest -lgtest_main
CPPFLAGS_TEST := $(googleStuffCPPFLAGS) # these are -I arguments for GCC, generated in GTEST.mk
moduleObj := "this variable is generated correctly by functions..."
...
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
../BUILD/BIN/HelloWorld_Test: $(moduleObj)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
When I exectue
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test -p>log.txt
the log.txt shows that the rule variables are resolved correctly, but I get the error
/usr/bin/ld: cannot find -lgtest
/usr/bin/ld: cannot find -lgtest_main
collect2: error: ld returned 1 exit status
This is because the rule "gtest" is not run before, but even if I run it manually by
[.....ExampleTrunk/buildenv]$ make gtest
and all required files are created in the correct ../BUILD/* directory, the error still remains.
What could be the reason, why
1. The prerequisite gtest is not executed for make HelloWorld_Test and
2. even if the libraries are created manually and therefore exist, they are not found by the linker
Thanks for your help
Jasmin
It would help people answer your question if it were a short, self-contained, correct example. There's too much information here which is likely not relevant to the problem and it takes a lot of effort to read through it all. Instead, start with a simple version of what you want to do with a cut-down makefile. If it doesn't fail, then keep adding parts of your real environment until it does.
I suspect your problem is right here at the end of your question:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
That's not what you want because it means first ../BUILD/BIN/HelloWorld_Test is built, then gtest is built.
In fact what you want is for gtest to be built before ../BUILD/BIN/HelloWorld_Test otherwise the gtest libraries are not available. I think you need this instead:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
This ensures that gtest is built before you try to link your program. Ideally you'd want to have this target also list the gtest libraries as prerequisites so that if they're modified, your program is re-built.
As stated in the comment before, the answer of MadScientist solved the first problem of building gtest before the test code itself, but linking still needed some changes to work. To get gtest to be built before the test sources I changed the build rules from:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
../BUILD/BIN/HelloWorld_Test: $(moduleObj)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
to
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test;
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
g++ $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
The linking process doesn't need any header files. The required gtest libs were now built to ../BUILD/LIB before the *.o files are linked, but the libs still could not be found during the linking process:
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test -p>log.txt
still gave the error
/usr/bin/ld: cannot find -lgtest
/usr/bin/ld: cannot find -lgtest_main
collect2: error: ld returned 1 exit status
The problem could be revealed by getting verbose output of the linking process by changing the rule to:
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
ld -o $# $(LDFLAGS_TEST) $^ --verbose
The verbose output was:
GNU ld (GNU Binutils) 2.30
...
==================================================
attempt to open ../BUILD/LIB/libgtest.so failed
attempt to open ../BUILD/LIB/libgtest.a failed
...
so the problem was, that the parameter "-l..." adds the prefix "lib" to the library name. As the library names were gtest.a and gtest_main.a they could not be found. The solution was to change the variable from:
LDFLAGS_TEST := -L../BUILD/LIB/ -lgtest -lgtest_main
to the full path representation
LDFLAGS_TEST := ../BUILD/LIB/gtest.a ../BUILD/LIB/gtest_main
Now everything works fine :D
If more libs with "special" names have to be linked, it would be a better solution to change the corresponding Makefile to add the prefix "lib" to the library's output names. This would keep the command line shorter, as the path ../BUILD/LIB only appears once with the "-L" switch. Nevertheless I think for my case the full paths are okay, because the parameter length is not so much more and the manipulations to the 3rd party Makefile of googletest are less.
Thanks for your help,
Jasmin
Suppose I have some libraries that I need as prerequisites for a target, and those libraries are stored in directories that contain the library name:
.
|-- Makefile
|-- a
| |-- Makefile
| `-- lib-a
|-- b
| |-- Makefile
| `-- lib-a
`-- out
where out requires both a/lib-a and b/lib-b. Both libs can be built by simply entering the directory and executing make lib-<X> there.
My idea of a Makefile with a pattern rule to avoid repetitions was this, which would have worked if both files were in the same directory or in different directories but with the same name, ie. I had needed to use % only once:
all: out
out: a/lib-a b/lib-b
cat $^ > out
%/lib-%:
make -C $(dir $#)
This doesn't work, however, since the %/lib-% pattern is illegal.
You should never use the raw make command when invoking a sub-make. Always use $(MAKE) (or ${MAKE}) variables.
In short there is no way to write a pattern rule where multiple patterns are required.
If they all have the same recipe then the simplest thing to do is construct the target list and write them all in a single rule:
TARGETS := a b
all: out
OUT_TARGETS := $(foreach T,$(TARGETS),$T/lib-$T)
out: $(OUT_TARGETS)
$(OUT_TARGETS):
$(MAKE) -C $(#D)
.PHONY: all out $(OUT_TARGETS)
Suppose i have the following directory structure:
root
|--Makefile
|--dir1
| |--Makefile
| |--tmp1.c
|--dir2
|--Makefile
|--tmp2.c
dir1/Makefile contains target tmp1 for compiling tmp1.c:
tmp1:tmp1.c
$(CC) $(CFLAGS) %< -o %#
In a same way dir2/Makefile contains a target tmp2 for compiling tmp2.c
What i want is to simply run make tmp1 or make tmp2 from root directory without explicitly defining this targets in root/Makefile.
For recursive make i use the following:
SUBDIRS = dir1 dir2
$(SUBDIRS):
$(MAKE) -C $#
I just need some way to pass specified target to this recursive make call so it can finds Makefile which contains a target and run it.
Thanks in advance.
How to make top level Makefile to call all targets in subdirectory Makefile ?
My folder structure is like this
/Makefile
/src/Makefile
I code all targets in /src/Makefile, but now I want to write another /Makefile to ease my work. How to write this top level Makefile ?
You could write in your TOP makefile use
all:
#$(MAKE) -C src
and if you do NOT use makefile in sub dir, for example, you use somename.mk, you could use
all:
#$(MAKE) -C src -f somename.mk
I show you my example, my DIR looks like:
TOPDIR-- Makefile
|
|-- debug
| |-- debug.c
| |-- debug.h
| |-- debug.mk
| |-- instrument.c
| `-- uart_print.c
|-- driver
| |-- driver.c
| |-- driver_ddi.c
| |-- driver_ddi.h
| |-- driver.h
| `-- driver.mk
|-- include
| `-- common.h
|-- Makefile
|-- mw
| |-- manager.c
| `-- mw.mk
|-- root
| |-- main.c
| `-- root.mk
and my TOP makefile looks like:
MAKE_DIR = $(PWD)
ROOT_DIR := $(MAKE_DIR)/root
DRV_DIR := $(MAKE_DIR)/driver
INCLUDE_DIR := $(MAKE_DIR)/include
DEBUG_DIR := $(MAKE_DIR)/debug
INC_SRCH_PATH :=
INC_SRCH_PATH += -I$(ROOT_DIR)
INC_SRCH_PATH += -I$(DRV_DIR)
INC_SRCH_PATH += -I$(INCLUDE_DIR)
INC_SRCH_PATH += -I$(DEBUG_DIR)
LIB_SRCH_PATH :=
LIB_SRCH_PATH += -L$(MAKE_DIR)/libs
COLOR_ON = color
COLOR_OFF =
CC = $(COLOR_ON)gcc
#CC = $(COLOR_OFF)gcc
LD = ld
LINT = splint
LIBS := -ldriver -ldebug -lmw -lm -lpthread
CFLAGS :=
CFLAGS += $(INC_SRCH_PATH) $(LIB_SRCH_PATH)
CFLAGS += -Wall -O -ggdb -Wstrict-prototypes -Wno-pointer-sign -finstrument-functions -fdump-rtl-expand
CFLAGS += -DDEBUG -D_REENTRANT
LDFLAGS :=
export MAKE_DIR CC LD CFLAGS LDFLAGS LIBS LINT INC_SRCH_PATH
all:
#$(MAKE) -C debug -f debug.mk
#$(MAKE) -C driver -f driver.mk
#$(MAKE) -C mw -f mw.mk
#$(MAKE) -C root -f root.mk
.PHONY: clean
clean:
#$(MAKE) -C debug -f debug.mk clean
#$(MAKE) -C driver -f driver.mk clean
#$(MAKE) -C mw -f mw.mk clean
#$(MAKE) -C root -f root.mk clean
.PHONY: lint
lint:
$(MAKE) -C debug -f debug.mk lint
it will call sub DIR *.mk during the compile. The sub DIR makefile, I just write a simple example for you reference:
LIB = $(MAKE_DIR)/libs/yourmodulename.a
SRCS = $(wildcard *.c)
OBJS = $(patsubst %.c, %.o, $(SRCS))
$(LIB): $(OBJS)
#mkdir -p ../libs
#$(AR) cr $# $^
#echo " Archive $(notdir $#)"
$(OBJS): $(SRCS)
#$(CC) $(CFLAGS) -c $^
#echo " CC $(OBJS)"
.PHONY: clean
clean:
#$(RM) -f $(LIB) $(OBJS)
#$(RM) -f *.expand
#echo " Remove Objects: $(OBJS)"
#echo " Remove Libraries: $(notdir $(LIB))"
.PHONY: lint
lint:
$(LINT) $(INC_SRCH_PATH) $(SRCS)
for the makefile, which generate the target file is little bit different, because I use the sub makefile to generate LIB file, and I use a root.mk to generate target:
PROG = ../prog/DEMO
SRCS = $(wildcard *.c)
OBJS = $(patsubst %.c, %.o, $(SRCS))
$(PROG): $(SRCS)
#mkdir -p ../prog
#$(CC) $^ $(CFLAGS) -Wl,-Map=$(PROG).map $(LIBS) -o $#
#echo " Generate Program $(notdir $(PROG)) from $^"
.PHONY: clean
clean:
#$(RM) -f $(OBJS) $(PROG)
#$(RM) -f *.expand
#$(RM) -rf ../prog ../libs
#echo " Remove Objects: $(OBJS)"
#echo " Remove Libraries: $(notdir $(PROG))"
Beware, note there are, at least, two ways of doing this.
GNU Changedir Option
One way is using a specific feature of GNU Make, the -C option allow to change your compile directory and reach another one:
all:
make -C dir
The make manual says:
-C dir, --directory=dir
Change to directory dir before reading the makefiles or doing anything
else. If multiple -C options are specified, each is interpreted relative
to the previous one: -C / -C etc is equivalent to -C /etc. This is
typically used with recursive invocations of make.
You can also combine this option by calling a specific target inside the targeted directory. For example, the following target will get into the src/ directory and call make with the clean target:
clean:
#rm -f *.o
make -C src/ clean
POSIX Way
The problem with the GNU way of doing is that it only works with GNU Make, and not the standard Make. If you may use another Make (for whatever reason), you better consider doing it in a more POSIX way.
In POSIX Make, you have to rely more on the cd command, like this:
all:
cd src/ && make
Note that, I used && and not ;. It is quite important to avoid infinite recursive calls to make. Indeed, cmd1 ; cmd2 will execute sequentially cmd1 and cmd2 whatever is the result of each command, where cmd1 && cmd2 will execute sequentially cmd1, and cmd2 will be executed only if cmd1 returned an EXIT_SUCCESS. In our case, imagine that the first cd failed because the directory has been removed. Then, the initial makefile will be executed again and again in an infinite recursive loop.
Anyway, this POSIX manner, is the more robust way of descending in subdirectories and execute other Makefile. I would recommend you to use it better than relying on an option linked to GNU Make.
And if you want to do multiple dirs easily:
SUBDIRS=dir1 dir2
all::
#echo make all
$(foreach var,$(SUBDIRS),echo $(var): ; cd $(var)/ && make $# && cd ..;)
clean:
#echo make clean
$(foreach var,$(SUBDIRS),echo $(var): ; cd $(var)/ && make $# && cd ..;)