Suppose I have some libraries that I need as prerequisites for a target, and those libraries are stored in directories that contain the library name:
.
|-- Makefile
|-- a
| |-- Makefile
| `-- lib-a
|-- b
| |-- Makefile
| `-- lib-a
`-- out
where out requires both a/lib-a and b/lib-b. Both libs can be built by simply entering the directory and executing make lib-<X> there.
My idea of a Makefile with a pattern rule to avoid repetitions was this, which would have worked if both files were in the same directory or in different directories but with the same name, ie. I had needed to use % only once:
all: out
out: a/lib-a b/lib-b
cat $^ > out
%/lib-%:
make -C $(dir $#)
This doesn't work, however, since the %/lib-% pattern is illegal.
You should never use the raw make command when invoking a sub-make. Always use $(MAKE) (or ${MAKE}) variables.
In short there is no way to write a pattern rule where multiple patterns are required.
If they all have the same recipe then the simplest thing to do is construct the target list and write them all in a single rule:
TARGETS := a b
all: out
OUT_TARGETS := $(foreach T,$(TARGETS),$T/lib-$T)
out: $(OUT_TARGETS)
$(OUT_TARGETS):
$(MAKE) -C $(#D)
.PHONY: all out $(OUT_TARGETS)
Related
I have a C++ project which contains a generated file that all the other C++ files depend on. I'm trying to force that file to be generated and compiled before any other compilation begins. Usually it'd be as simple as putting that file first in the all: target, but the complication is that my Makefile is also generated by a build system, and I can only append fragments to the Makefile, not edit it in general.
So, is there a way to force that generated file target to run first, via dependencies or otherwise? I've thought of using something like this:
cpp_files := $(wildcard src/*.cpp)
$(cpp_files): generated_file.cpp
generated_file.cpp:
# generate the file here
But it doesn't seem to work. For reference, my source dir structure is like this, so I need to collect the cpp files recursively:
src/
|---file1.cpp
|---file2.cpp
|---subdir1/
|---file3.cpp
gen/
|---generated_file.cpp
If you're sure that's really what you want, here's one way to do it: have a rule for a dummy file which the makefile will include.
.PHONY: dummy
dummy:
# generate the file here
-include dummy
No matter what target you specify, Make will first run the dummy rule, then restart itself.
Actually, other cpp files don't depend on the generated one (in terms of Make rules). Dependency graph still looks like:
program
|
^- file1.o
| ^- file1.c
|
^- file2.o
| ^- file2.c
|
^- generated_file.o
^- generated_file.c
^- <generator prerequisites>
Your source files could only depend on some kind of generated header file (in case when it is #included by these sources).
If you need to generate only a cpp file then the following makefile should be sufficient:
srcs := $(wildcard src/*.cpp src/*/*.cpp)
gen_file := gen/generated_file.cpp
srcs += $(gen_file)
objs := $(srcs:.cpp=.o)
.PHONY : all
all : program
program : $(objs)
#$(LD) ...
%.o : %.c
#$(CC) ...
$(gen_file) :
# generate the file here
UPD.
A little improvement based on Beta's answer.
-include generator-task
.PHONY : generator-task
generator-task : $(gen_files)
$(gen_files) : $(gen_prerequisites)
# generate the file here
Instead of running generator on each invocation of Make, this will only regenerate a file when one of its prerequisites would change.
My problem is that a prerequisite (set by a variable) is not executed and even if executed manually before, required libraries are not found when linking. My Makefiles are non recursive, but the prerequisite is the external Software googletest and therefore called recursively, as the sources are not changed between several "make" runs.
My environment is:
$ make --version
GNU Make 4.2.1
Built for x86_64-unknown-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ uname -a
Linux archlinux 4.17.3-1-ARCH #1 SMP PREEMPT Tue Jun 26 04:42:36 UTC 2018 x86_64 GNU/Linux
This is the relevant directory structure:
|-- ExampleTrunk
| |-- BUILD
| | |-- BIN
| | |-- LIB
| | `-- OBJ
| |-- buildenv
| | |-- ...
| | |-- GTEST.mk
| | |-- Makefile
| | |-- commons.mk
| | |-- module_commons.mk
| | |-- pathdefs.mk
| |-- src
| `-- HelloWorld
| `-- module.mk
| `-- test
| `-- HelloWorld
| `-- module.mk
`-- Tools
`-- GoogleTest
`-- googletest
|-- make
|-- Makefile
The folder ExampleTrunk/BUILD and its subdirectories are immediately created in pathdefs.mk and therefore exist before further rule execution.
The start Makefile is ExampleTrunk/buildenv/Makefile:
...
export SHELL := $(shell which bash)
...
.DEFAULT_GOAL := all
#some common function definitions
include commons.mk
#get $(PROJECT_ROOT), $(REL_PROJECT_ROOT); immediately create $(GLOBAL_OBJ_PATH), $(GLOBAL_LIB_PATH), $(GLOBAL_BIN_PATH)
include pathdefs.mk
export BUILD_DIRS := $(GLOBAL_BIN_PATH) $(GLOBAL_LIB_PATH) $(GLOBAL_OBJ_PATH)
include $(REL_PROJECT_ROOT)/src/HelloWorld/module.mk
...
#only include test stuff, when tests shall be built or cleaned to save dependency calculation time in all other cases
ifeq "$(call givenStringEndsWithTest,$(MAKECMDGOALS))" "0"
include GTEST.mk
include $(REL_PROJECT_ROOT)/test/HelloWorld/module.mk
endif
all : $(programs) | $(BUILD_DIRS);
all_Test: $(testPrograms) | $(BUILD_DIRS);
$(GLOBAL_OBJ_PATH)/%.o: %.cpp | $(BUILD_DIRS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $< -o $#
clean:
$(RM) $(addprefix $(GLOBAL_BIN_PATH)/,$(programs)) $(addprefix $(GLOBAL_BIN_PATH)/,$(testPrograms))
$(RM) $(objectsToClean)
$(RM) $(objectsToClean:.o=.d)
clean_Test: clean gtest_clean gmock_clean;
The rules "all", "clean" and "clean_Test" resolve and execute their prerequisites as expected and run without errors. (evaluated by make ... -p) The rule "all_Test" produces errors as described below.
GTest.mk contains the following recursive rules:
gtest: | $(BUILD_DIRS)
[ -d "$(REL_GTEST_MAKEFILE_DIR)" ] && cd "$(REL_GTEST_MAKEFILE_DIR)" && $(MAKE)
gtest_clean:
[ -d "$(REL_GTEST_MAKEFILE_DIR)" ] && cd "$(REL_GTEST_MAKEFILE_DIR)" && $(MAKE) clean
The directory "$(REL_GTEST_MAKEFILE_DIR)" exists, because it is only another path representation to the ExampleTrunk/BUILD/* directories that were immediately created before by pathdefs.mk. When I call any of both rules directly they can be executed without errors. Because I adapted the googletest Makefile, the outputs are created in the ExampleTrunk/BUILD/* directories as required.
The module.mk files are very short, as they include the module_commons.mk which contains most of the things to be executed:
ExampleTrunk/src/HelloWorld/module.mk:
additionalPrograms :=
gmockIsRequired := no
include $(PROJECT_ROOT)/buildenv/module_commons.mk
ExampleTrunk/test/HelloWorld/module.mk:
additionalPrograms := HelloWorld
gmockIsRequired := no
include $(PROJECT_ROOT)/buildenv/module_commons.mk
ExampleTrunk/buildenv/module_commons.mk generates most of the variables automatically from the module.mk file location and settings.
When I exectue
[.....ExampleTrunk/buildenv]$ make HelloWorld
everythink works fine, but with
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test
it fails with a linker error (shown below). Here are the relevant *.mk parts for HelloWorld_Test with already resolved variables as shown from make HelloWorld_Test -p (variables with files/directories or flags are not written in resolved form to keep it short):
...
LDFLAGS_TEST := -L../BUILD/LIB -lgtest -lgtest_main
CPPFLAGS_TEST := $(googleStuffCPPFLAGS) # these are -I arguments for GCC, generated in GTEST.mk
moduleObj := "this variable is generated correctly by functions..."
...
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
../BUILD/BIN/HelloWorld_Test: $(moduleObj)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
When I exectue
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test -p>log.txt
the log.txt shows that the rule variables are resolved correctly, but I get the error
/usr/bin/ld: cannot find -lgtest
/usr/bin/ld: cannot find -lgtest_main
collect2: error: ld returned 1 exit status
This is because the rule "gtest" is not run before, but even if I run it manually by
[.....ExampleTrunk/buildenv]$ make gtest
and all required files are created in the correct ../BUILD/* directory, the error still remains.
What could be the reason, why
1. The prerequisite gtest is not executed for make HelloWorld_Test and
2. even if the libraries are created manually and therefore exist, they are not found by the linker
Thanks for your help
Jasmin
It would help people answer your question if it were a short, self-contained, correct example. There's too much information here which is likely not relevant to the problem and it takes a lot of effort to read through it all. Instead, start with a simple version of what you want to do with a cut-down makefile. If it doesn't fail, then keep adding parts of your real environment until it does.
I suspect your problem is right here at the end of your question:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
That's not what you want because it means first ../BUILD/BIN/HelloWorld_Test is built, then gtest is built.
In fact what you want is for gtest to be built before ../BUILD/BIN/HelloWorld_Test otherwise the gtest libraries are not available. I think you need this instead:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
This ensures that gtest is built before you try to link your program. Ideally you'd want to have this target also list the gtest libraries as prerequisites so that if they're modified, your program is re-built.
As stated in the comment before, the answer of MadScientist solved the first problem of building gtest before the test code itself, but linking still needed some changes to work. To get gtest to be built before the test sources I changed the build rules from:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
../BUILD/BIN/HelloWorld_Test: $(moduleObj)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
to
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test;
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
g++ $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
The linking process doesn't need any header files. The required gtest libs were now built to ../BUILD/LIB before the *.o files are linked, but the libs still could not be found during the linking process:
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test -p>log.txt
still gave the error
/usr/bin/ld: cannot find -lgtest
/usr/bin/ld: cannot find -lgtest_main
collect2: error: ld returned 1 exit status
The problem could be revealed by getting verbose output of the linking process by changing the rule to:
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
ld -o $# $(LDFLAGS_TEST) $^ --verbose
The verbose output was:
GNU ld (GNU Binutils) 2.30
...
==================================================
attempt to open ../BUILD/LIB/libgtest.so failed
attempt to open ../BUILD/LIB/libgtest.a failed
...
so the problem was, that the parameter "-l..." adds the prefix "lib" to the library name. As the library names were gtest.a and gtest_main.a they could not be found. The solution was to change the variable from:
LDFLAGS_TEST := -L../BUILD/LIB/ -lgtest -lgtest_main
to the full path representation
LDFLAGS_TEST := ../BUILD/LIB/gtest.a ../BUILD/LIB/gtest_main
Now everything works fine :D
If more libs with "special" names have to be linked, it would be a better solution to change the corresponding Makefile to add the prefix "lib" to the library's output names. This would keep the command line shorter, as the path ../BUILD/LIB only appears once with the "-L" switch. Nevertheless I think for my case the full paths are okay, because the parameter length is not so much more and the manipulations to the 3rd party Makefile of googletest are less.
Thanks for your help,
Jasmin
Suppose i have the following directory structure:
root
|--Makefile
|--dir1
| |--Makefile
| |--tmp1.c
|--dir2
|--Makefile
|--tmp2.c
dir1/Makefile contains target tmp1 for compiling tmp1.c:
tmp1:tmp1.c
$(CC) $(CFLAGS) %< -o %#
In a same way dir2/Makefile contains a target tmp2 for compiling tmp2.c
What i want is to simply run make tmp1 or make tmp2 from root directory without explicitly defining this targets in root/Makefile.
For recursive make i use the following:
SUBDIRS = dir1 dir2
$(SUBDIRS):
$(MAKE) -C $#
I just need some way to pass specified target to this recursive make call so it can finds Makefile which contains a target and run it.
Thanks in advance.
I have a C++ project which contains a generated file that all the other C++ files depend on. I'm trying to force that file to be generated and compiled before any other compilation begins. Usually it'd be as simple as putting that file first in the all: target, but the complication is that my Makefile is also generated by a build system, and I can only append fragments to the Makefile, not edit it in general.
So, is there a way to force that generated file target to run first, via dependencies or otherwise? I've thought of using something like this:
cpp_files := $(wildcard src/*.cpp)
$(cpp_files): generated_file.cpp
generated_file.cpp:
# generate the file here
But it doesn't seem to work. For reference, my source dir structure is like this, so I need to collect the cpp files recursively:
src/
|---file1.cpp
|---file2.cpp
|---subdir1/
|---file3.cpp
gen/
|---generated_file.cpp
If you're sure that's really what you want, here's one way to do it: have a rule for a dummy file which the makefile will include.
.PHONY: dummy
dummy:
# generate the file here
-include dummy
No matter what target you specify, Make will first run the dummy rule, then restart itself.
Actually, other cpp files don't depend on the generated one (in terms of Make rules). Dependency graph still looks like:
program
|
^- file1.o
| ^- file1.c
|
^- file2.o
| ^- file2.c
|
^- generated_file.o
^- generated_file.c
^- <generator prerequisites>
Your source files could only depend on some kind of generated header file (in case when it is #included by these sources).
If you need to generate only a cpp file then the following makefile should be sufficient:
srcs := $(wildcard src/*.cpp src/*/*.cpp)
gen_file := gen/generated_file.cpp
srcs += $(gen_file)
objs := $(srcs:.cpp=.o)
.PHONY : all
all : program
program : $(objs)
#$(LD) ...
%.o : %.c
#$(CC) ...
$(gen_file) :
# generate the file here
UPD.
A little improvement based on Beta's answer.
-include generator-task
.PHONY : generator-task
generator-task : $(gen_files)
$(gen_files) : $(gen_prerequisites)
# generate the file here
Instead of running generator on each invocation of Make, this will only regenerate a file when one of its prerequisites would change.
First, let me say that I am aware of the cons of using recursive Makefiles. So if you are here just to tell me don't use it, please don't.
Imagine this directory structure:
rootdir
`-- subdir
|-- a
|-- b
`-- c
Let's say the Makefile on rootdir reads like this:
.PHONY: all
all:
# build some stuff
$(MAKE) -C subdir
and the one in subdir reads like this:
.PHONY: all
all:
# nothing here except redirecting make to each of the subdirectories
$(MAKE) -C a
$(MAKE) -C b
$(MAKE) -C c
and another Makefile in each of a, b and c folders building something.
Since the Makefile in subdir serves no purpose except redirecting make, I want make not to print: Entering directory rootdir/subdir and Leaving directory rootdir/subdir to clean up the output a bit.
On the other hand, since there are commands being executed in the subfolders a, b and c, I do want make to print these outputs. Here's what I thought would work:
rootdir's Makefile:
.PHONY: all
all:
# build some stuff
$(MAKE) --no-print-directory -C subdir
subdir's Makefile:
.PHONY: all
all:
# nothing here except redirecting make to each of the subdirectories
$(MAKE) --print-directory -C a
$(MAKE) --print-directory -C b
$(MAKE) --print-directory -C c
The problem is, once the --no-print-directory is given to make when calling make for subdir, --print-directory doesn't enable it again when calling make for a, b or c.
So my question is, how can I re-enable printing directories when a parent make has disabled it?
Make command line flags get communicated to sub-makes via MAKEFLAGS variable. You may like to replace --no-print-directory (if any) from MAKEFLAGS with w manually before invoking the sub-makes:
${MAKE} MAKEFLAGS="$(subst --no-print-directory,w,${MAKEFLAGS})" -C ...