I am using the gcc compiler.
My project source tree looks like somewhat like this
test$~: tree .
.
|-- folder
| |-- hello.cpp
| `-- hello.h
`-- main.cpp
1 directory, 3 files
test$~:
The file main.cpp contains the main() function and all the functions invoked by main.cpp
lie in the directory named folder
So far in all my little projects I never had to put some source code under a sub-directory.
What I am looking for, in short, is some gcc command for recursive compilation in sub-directories and their subdirectories and so on... This command should be invoked from the
home directory of the code project.
I couldn't really test it on a c++ library as I don't have one currently, but this should work:
find . -type f -iname *.cpp -execdir g++ {} \;
I just don't remember what happens when the -o option is omitted for gcc. If its necessary then this will at least bring a you a bit closer to a solution. I would recommend using makefiles instead though
Related
Below is the folder structure for my code.
This is a very small example to understand the concept of multiple makefiles based on which I have to create makefile for bigger code structure.
work
├── code
| |
| └── main.h and test.h files here
│ └── main.c and test.c files here
| └── subdir.mk
|
├── _Build/
│ └── Makefile here
I am keeping both Makefile and subdir.mk files to be very basic and simple to grasp the concept.
Below is the code for subdir.mk
#subdir.mk
#============================================
test.o : test.c test.h
#echo Building test.c ...
gcc -Werror -Wall -c test.c -o test.o
main.o : main.c main.h test.h
#echo Building main.c ...
gcc -Werror -Wall -c main.c -o main.o
#============================================
Below is the code for main file ... Makefile
#Makefile
#============================================
include ../code/subdir.mk
main : ../code/main.o ..code/test.o
#echo Building ...
make subdir.mk # <--- What is the correct way to perform this code
#echo Linking files ...
gcc -Llib ../code/main.o ../code/test.o -lm -o main
clean:
rm -rv ../code/*.o
#============================================
The error I am getting is
make: *** No rule to make target 'test.c', needed by 'test.o'. Stop.
In subdir.mk I am trying to generate object files.
In Makefile I am trying to link the object files generated in subdir.mk
The way I am trying to execute is correct way or some different steps are followed when we have multiple subdir.mk and main Makefile.
Share your valuable comments please.
You cannot both include the subdir.mk file and also invoke it recursively.
You need to decide whether you want to use non-recursive make (which means you'd use include) or recursive make (which means you'd run a sub-make command).
If you want to use non-recursive make then your subdir.mk makefile needs to be prepared to be run when the current working directory is different than the directory that the subdir.mk file appears in.
If you want to use recursive make then you need a separate rule to build the objects, and you should not include subdirs.mk. Something like this:
main : ../code/main.o ..code/test.o
#echo Linking files ...
gcc -Llib ../code/main.o ../code/test.o -lm -o main
../code/main.o ..code/test.o : subdir ;
subdir:
#echo Building ...
cd ../code && $(MAKE) -f subdir.mk
.PHONY: subdir
Be sure to include the semicolon after the subdir in the .o rule.
When invoking sub-makes you should always use the $(MAKE) variable, never use the literal string make.
You will probably be better off having your subdir.mk build a single library out of the objects rather than having to repeat all the object files in multiple places. Then replace the list of object files in this makefile with the library.
Contrary to Andreas's assertion, this will not always rebuild main. It will only be rebuilt when one of the object files was changed.
subdir.mk has to use paths relative to the main makefile. E.g.
../code/test.o : ../code/test.c ../code/test.h
...
What you need is a recipe for creating the object files. In Makefile you'll need to remove include ../code/subdir.mk and add something like this:
.PHONY: ../code/main.o ..code/test.o
../code/main.o ..code/test.o:
$(MAKE) -C ../code/ -f subdir.mk $(#F)
When you build target main, make sees you need the object files. Then make find the above recipe for creating the object files and so runs it.
Having them .PHONY is because the top Makefile can´t tell whether or not they´re up to date. Without it the object files would never be rebuilt. This has the unfortunate consequence that main will always be rebuilt, even if subdir.mk determines the object files were indeed up to date. The solution is either to have it all in a single make (can still split into files and include), or change to a build tool that can get it right.
My problem is that a prerequisite (set by a variable) is not executed and even if executed manually before, required libraries are not found when linking. My Makefiles are non recursive, but the prerequisite is the external Software googletest and therefore called recursively, as the sources are not changed between several "make" runs.
My environment is:
$ make --version
GNU Make 4.2.1
Built for x86_64-unknown-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ uname -a
Linux archlinux 4.17.3-1-ARCH #1 SMP PREEMPT Tue Jun 26 04:42:36 UTC 2018 x86_64 GNU/Linux
This is the relevant directory structure:
|-- ExampleTrunk
| |-- BUILD
| | |-- BIN
| | |-- LIB
| | `-- OBJ
| |-- buildenv
| | |-- ...
| | |-- GTEST.mk
| | |-- Makefile
| | |-- commons.mk
| | |-- module_commons.mk
| | |-- pathdefs.mk
| |-- src
| `-- HelloWorld
| `-- module.mk
| `-- test
| `-- HelloWorld
| `-- module.mk
`-- Tools
`-- GoogleTest
`-- googletest
|-- make
|-- Makefile
The folder ExampleTrunk/BUILD and its subdirectories are immediately created in pathdefs.mk and therefore exist before further rule execution.
The start Makefile is ExampleTrunk/buildenv/Makefile:
...
export SHELL := $(shell which bash)
...
.DEFAULT_GOAL := all
#some common function definitions
include commons.mk
#get $(PROJECT_ROOT), $(REL_PROJECT_ROOT); immediately create $(GLOBAL_OBJ_PATH), $(GLOBAL_LIB_PATH), $(GLOBAL_BIN_PATH)
include pathdefs.mk
export BUILD_DIRS := $(GLOBAL_BIN_PATH) $(GLOBAL_LIB_PATH) $(GLOBAL_OBJ_PATH)
include $(REL_PROJECT_ROOT)/src/HelloWorld/module.mk
...
#only include test stuff, when tests shall be built or cleaned to save dependency calculation time in all other cases
ifeq "$(call givenStringEndsWithTest,$(MAKECMDGOALS))" "0"
include GTEST.mk
include $(REL_PROJECT_ROOT)/test/HelloWorld/module.mk
endif
all : $(programs) | $(BUILD_DIRS);
all_Test: $(testPrograms) | $(BUILD_DIRS);
$(GLOBAL_OBJ_PATH)/%.o: %.cpp | $(BUILD_DIRS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $< -o $#
clean:
$(RM) $(addprefix $(GLOBAL_BIN_PATH)/,$(programs)) $(addprefix $(GLOBAL_BIN_PATH)/,$(testPrograms))
$(RM) $(objectsToClean)
$(RM) $(objectsToClean:.o=.d)
clean_Test: clean gtest_clean gmock_clean;
The rules "all", "clean" and "clean_Test" resolve and execute their prerequisites as expected and run without errors. (evaluated by make ... -p) The rule "all_Test" produces errors as described below.
GTest.mk contains the following recursive rules:
gtest: | $(BUILD_DIRS)
[ -d "$(REL_GTEST_MAKEFILE_DIR)" ] && cd "$(REL_GTEST_MAKEFILE_DIR)" && $(MAKE)
gtest_clean:
[ -d "$(REL_GTEST_MAKEFILE_DIR)" ] && cd "$(REL_GTEST_MAKEFILE_DIR)" && $(MAKE) clean
The directory "$(REL_GTEST_MAKEFILE_DIR)" exists, because it is only another path representation to the ExampleTrunk/BUILD/* directories that were immediately created before by pathdefs.mk. When I call any of both rules directly they can be executed without errors. Because I adapted the googletest Makefile, the outputs are created in the ExampleTrunk/BUILD/* directories as required.
The module.mk files are very short, as they include the module_commons.mk which contains most of the things to be executed:
ExampleTrunk/src/HelloWorld/module.mk:
additionalPrograms :=
gmockIsRequired := no
include $(PROJECT_ROOT)/buildenv/module_commons.mk
ExampleTrunk/test/HelloWorld/module.mk:
additionalPrograms := HelloWorld
gmockIsRequired := no
include $(PROJECT_ROOT)/buildenv/module_commons.mk
ExampleTrunk/buildenv/module_commons.mk generates most of the variables automatically from the module.mk file location and settings.
When I exectue
[.....ExampleTrunk/buildenv]$ make HelloWorld
everythink works fine, but with
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test
it fails with a linker error (shown below). Here are the relevant *.mk parts for HelloWorld_Test with already resolved variables as shown from make HelloWorld_Test -p (variables with files/directories or flags are not written in resolved form to keep it short):
...
LDFLAGS_TEST := -L../BUILD/LIB -lgtest -lgtest_main
CPPFLAGS_TEST := $(googleStuffCPPFLAGS) # these are -I arguments for GCC, generated in GTEST.mk
moduleObj := "this variable is generated correctly by functions..."
...
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
../BUILD/BIN/HelloWorld_Test: $(moduleObj)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
When I exectue
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test -p>log.txt
the log.txt shows that the rule variables are resolved correctly, but I get the error
/usr/bin/ld: cannot find -lgtest
/usr/bin/ld: cannot find -lgtest_main
collect2: error: ld returned 1 exit status
This is because the rule "gtest" is not run before, but even if I run it manually by
[.....ExampleTrunk/buildenv]$ make gtest
and all required files are created in the correct ../BUILD/* directory, the error still remains.
What could be the reason, why
1. The prerequisite gtest is not executed for make HelloWorld_Test and
2. even if the libraries are created manually and therefore exist, they are not found by the linker
Thanks for your help
Jasmin
It would help people answer your question if it were a short, self-contained, correct example. There's too much information here which is likely not relevant to the problem and it takes a lot of effort to read through it all. Instead, start with a simple version of what you want to do with a cut-down makefile. If it doesn't fail, then keep adding parts of your real environment until it does.
I suspect your problem is right here at the end of your question:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
That's not what you want because it means first ../BUILD/BIN/HelloWorld_Test is built, then gtest is built.
In fact what you want is for gtest to be built before ../BUILD/BIN/HelloWorld_Test otherwise the gtest libraries are not available. I think you need this instead:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
This ensures that gtest is built before you try to link your program. Ideally you'd want to have this target also list the gtest libraries as prerequisites so that if they're modified, your program is re-built.
As stated in the comment before, the answer of MadScientist solved the first problem of building gtest before the test code itself, but linking still needed some changes to work. To get gtest to be built before the test sources I changed the build rules from:
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);
../BUILD/BIN/HelloWorld_Test: $(moduleObj)
g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
to
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test;
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
g++ $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $#
The linking process doesn't need any header files. The required gtest libs were now built to ../BUILD/LIB before the *.o files are linked, but the libs still could not be found during the linking process:
[.....ExampleTrunk/buildenv]$ make HelloWorld_Test -p>log.txt
still gave the error
/usr/bin/ld: cannot find -lgtest
/usr/bin/ld: cannot find -lgtest_main
collect2: error: ld returned 1 exit status
The problem could be revealed by getting verbose output of the linking process by changing the rule to:
../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
ld -o $# $(LDFLAGS_TEST) $^ --verbose
The verbose output was:
GNU ld (GNU Binutils) 2.30
...
==================================================
attempt to open ../BUILD/LIB/libgtest.so failed
attempt to open ../BUILD/LIB/libgtest.a failed
...
so the problem was, that the parameter "-l..." adds the prefix "lib" to the library name. As the library names were gtest.a and gtest_main.a they could not be found. The solution was to change the variable from:
LDFLAGS_TEST := -L../BUILD/LIB/ -lgtest -lgtest_main
to the full path representation
LDFLAGS_TEST := ../BUILD/LIB/gtest.a ../BUILD/LIB/gtest_main
Now everything works fine :D
If more libs with "special" names have to be linked, it would be a better solution to change the corresponding Makefile to add the prefix "lib" to the library's output names. This would keep the command line shorter, as the path ../BUILD/LIB only appears once with the "-L" switch. Nevertheless I think for my case the full paths are okay, because the parameter length is not so much more and the manipulations to the 3rd party Makefile of googletest are less.
Thanks for your help,
Jasmin
Suppose I have some libraries that I need as prerequisites for a target, and those libraries are stored in directories that contain the library name:
.
|-- Makefile
|-- a
| |-- Makefile
| `-- lib-a
|-- b
| |-- Makefile
| `-- lib-a
`-- out
where out requires both a/lib-a and b/lib-b. Both libs can be built by simply entering the directory and executing make lib-<X> there.
My idea of a Makefile with a pattern rule to avoid repetitions was this, which would have worked if both files were in the same directory or in different directories but with the same name, ie. I had needed to use % only once:
all: out
out: a/lib-a b/lib-b
cat $^ > out
%/lib-%:
make -C $(dir $#)
This doesn't work, however, since the %/lib-% pattern is illegal.
You should never use the raw make command when invoking a sub-make. Always use $(MAKE) (or ${MAKE}) variables.
In short there is no way to write a pattern rule where multiple patterns are required.
If they all have the same recipe then the simplest thing to do is construct the target list and write them all in a single rule:
TARGETS := a b
all: out
OUT_TARGETS := $(foreach T,$(TARGETS),$T/lib-$T)
out: $(OUT_TARGETS)
$(OUT_TARGETS):
$(MAKE) -C $(#D)
.PHONY: all out $(OUT_TARGETS)
Let's consider this files structure:
/project
+ Makefile
+ projectA
| + Makefile
| + foo.c
| + bar.c
| + ...
|
+ projectB
+ Makefile
+ oak.c
+ ...
How can I quickly check if a submake is up to date?
In my main make I have:
all:
make -C projectA
And in the submakefile
all: a.txt b.txt
%.txt:%.xtx
cp $< $#
The problem with this solution is that I will always call the submake whatever it has nothing to do. I want to avoid it.
The ugly solution I've found is to create a make rule in my submakefile to echo all the files that are depending on the main target. Then I can call the submake with a shell command:
listfiles=$(shell make -C projectA listfiles)
all: $(listfiles)
make -C projectA
EDIT
A better approach would be to have a shell script in each sub-projects that returns the list of all the project's dependencies. For instance:
projectc/listdep.sh:
#!/usr/bin/bash
find . -iname *.[ch]
Then I can use this script in the parent make:
projectc/a.out: $(shell projectc/listdep.sh)
make -C $(dir $#)
In this way, the parent make won't call the recipe if projectc/a.out exists and is more recent that its dependencies.
I hope your projectA and projectB are "self-contained". That means, their builds do not read any sources or build intermediate files from outside that project. They may use tools, such as gcc or, as you have it in the example above cp, but not sources outside of their directory, and not any files that are built by somebody else. In other words, if you deleted the whole rest of the tree and only build projectA and projectB by themselves with their Makefiles, they will build just fine.
In such a case, it is OK to use recursive Make.
If you use recursive Make, you have to do what you are doing,
make -C projectA
There is no better way. That is the best way to "quickly check" if the "submake" is up to date.
I dont know much makefile stuff I've been tending to learn bits as required.
The biggest failing of my makefiles is that I have been listing all the files manually, while this hasn't been a problem my current project is getting unwieldy. I have 4 directories each with sources files.
How can I get all the object file listing without having to list them manually.
This doesn't work, but it shows what I've been trying to do.
VPATH = Lib GameCode Moot/Moot Moot/Impl
OBJS = $(subst .cpp, .o, $(VPATH))
foobar: $(OBJS)
g++ -o $# $^
%.o: %.cpp
g++ -c $< -o $# -I Moot
clean:
rm main.o lib.o foo.o foobar
Personally, I never had any problem in listing all files manually. Listing a file to the makefile takes negligible time compared to adding filling it with useful content.
To get all files from different directories, one might suggest using wildcard function. So my_sources:=$(wildcard *.cpp dir1/*.cpp) will make the variable contain source files that match wildcard expression.
However, I find it less convenient than using usual Linux find command via shell:
# Find all sources
my_sources:=$(shell find -iname '*.cpp')
# Make targets out of them
OBJS=$(my_sources:%.cpp=%.o)
Find is more powerful than Make's builtin wildcard. You might also want to use other shell capabilities, such as pipelines, for example, to filter output of find (if Make's filter-out function is not enough). Or something like this, to avoid excessive variables:
OBJS:=$(shell find -iname '*.cpp' | sed 's/\.cpp$/.o/')
You name it!
Using VPATH or vpath will not work for your problem.. it provides a search path to find files but you still need to list the files. If you just need to compile all and any .c/.cpp files found in those directories then this should work:
foobar: $(shell ls Lib/*.cpp) $(shell ls GameCode/*.cpp) $(shell ls Moot/Moot/*.cpp) $(shell ls Moot/Impl/*cpp)
g++ -o $# $^
clean:
rm foobar $(shell ls Lib/*.o) $(shell ls GameCode/*.o) $(shell ls Moot/Moot/*.o) $(shell ls Moot/Impl/*o)
The VPATH info is not needed, the substitution of .o for .cpp can go as can the override of the implicit rule. Additionally, not the use of ls instead of find to look in, and only in, the specfified directory.