Good practices of Makefile - makefile

I have a following directory structure in my project:
bin/
dist/
include/
├── module_a/
└── module_b/
Makefile
src/
├── module_a/
└── module_b/
Folder include/ contains *.hpp's while *.cpp's are in src/. I would like to compile all sources to bin/ and then link them up together to dist/. Seems a pretty reasonable wish for me.
I would like to know the best practices for a Makefile for this case. All I can find is %.o: %.cpp target, but that doesn't really work, because of different source and binary folder.
I was trying to use something like this:
D_SRC = src
D_BIN=bin
F_CPP := $(shell find $(D_SRC) -iname '*.cpp' -type f)
F_OBJ := $(shell echo $(F_CPP) | sed s:\ :\\n:g | sed s:$(D_SRC):$(D_BIN): | sed 's:^\(.*\)\.cpp$$:\1\.o:')
$(F_OBJ): $(F_SRC)
$(foreach file, $(F_SRC), \
$(GXX) $(CXXFLAGS) -c $(file)\
)
This target doesn't work, because $(F_OBJ) paths start with bin/, while foreach compiles sources to current working dir. I could make it compile to bin/, but that would happen only with a few more sed expressions and it's ugly enough as it is.
It's probably so difficult for me, because I don't know make all that well, but I cannot be the only one with this project setup. In my opinion, it must be a pretty common one. I know I can write a Makefile for each module separately, but is that really the best choice here?
EDIT: I was now wondering what would I achieve with several Makefiles. If one was at root and another one in src/module_a, how would the latter know about the bin/? If you'd execute it with make -f src/module_a/Makefile, it would be the same as executing it from root directory, 'cause it's working directory would be root. Another way, I guess, would be to change directory before executing it, like so: make -C include/module_a, but in that case, how would it find bin/? I wouldn't want to have something like D_BIN = ../../bin in a Makefile.

What I normally do is have a Makefile in the src directory (which can be invoked from the top level Makefile if you like) and then use rules like this:
D_BIN = ../bin
$(D_BIN)/%.o: %.cpp
You could also experiment with just a makefile in the top level dir, and use rules that look like this:
D_BIN = bin
D_SRC = src
$(D_BIN)/%.o: $(D_SRC)/%.cpp
but I have not used such rules, so I don't know the pros/cons vs the way I normally do it. The way I normally do it works fine, I even have rules that build depends like so:
$(D_BIN)/%.d: %.cpp
and the link rule would be like:
../dist/outexe: $(F_OBJ)
Using a foreach is usually frowned upon because it does not make use of all the features built into normal makefile rules (i.e. there is no depends check on a per file basis, either you build everything or nothing), and as such foreach should only be used as a last resort, but in this case you will be able to get it to work without the foreach.
In addition to this there are much easier ways to build your file lists, you don't need to use the shell or sed.
F_CPP = $(wildcard *.cpp)
F_OBJ = $(F_CPP:.cpp=.o)
Update: This is how I normally issue recursive makes:
SUBDIRS = src
.PHONY: $(SUBDIRS)
all: $(SUBDIRS)
$(SUBDIRS):
#echo "Building $#..."
$(MAKE) -C $# $(MFLAGS)
Then indeed in your submake, you would need to use ../bin for example.
However with a project as simple as yours, you might be better off just having one makefile at the root level and using rules like this:
D_BIN = bin
D_SRC = src
$(D_BIN)/%.o: $(D_SRC)/%.cpp
recursive makefiles are ok (ok but not great) if you have a really complex directory structure, where you will be adding/removing/modifying new dir trees as time goes on. But for a simple project where you just want to have separate directories for code and objs, it is probably overkill.

Related

Symbols ($(bindir), $(sysconfdir),...) unknown in (sub) Makefiles

I'm working with autotools for the first time, for a tool that's written in perl (SQLTeX), so only installation is required, no compilation.
The toplevel contains a simple Makefile.am:
AUTOMAKE_OPTIONS = foreign
SUBDIRS = src man doc
EXTRA_DIST = README.md
.PHONY: all-am
all-am:
#echo "Done!"
If I create Makefile.am files in the sub-directories too, nothing seems to happen there so I just stick to Makefile. A snippet from src/Makefile (EDIT: this file is now renamed to Makefile.am):
SQLTeX: SQLTeX.pl
cat $^ | sed -e 's#{PERLDIR}#$(PL)#;s#{SYSCONFDIR}#$(sysconfdir)#' > $#
#chmod +x $#
The symbol PL is set as expect (defined in the same makefile), but sysconfdir is empty, although it is defined in the top-level Makefile generated by ./configure.
What am I doing wrong?
Thanks in advance!
What am I doing wrong?
Although the Autotools support, with some caveats, recursing into directories where you provide pre-built makefiles, you cannot expect those pre-built makefiles to be able to rely on autotools-provided variables such as the standard directory variables bindir and sysconfdir. Thus, although it is allowed to rely on hand-written makefiles in subdirectories, this is probably a false trail for you.
I recommend going back to this:
If I create Makefile.am files in the sub-directories too, nothing seems to happen there
and working out what's wrong. The Autotools definitely support generating recursive build systems, and one Makefile.am per directory is part of the usual approach to that. If it didn't work for you then my first guess would be that you forgot to list the extra makefiles in your AC_CONFIG_FILES list.
As an alternative, just because you have multiple directories does not mean that you need to use recursive make. It is quite possible to build such a project with the support of a single makefile, and the Autotools can help with such a makefile.

non-recursive make constrict targets

Based on this paper, I'm trying to rework a subset of my build system to be non-recursive. It's actually working pretty well. By default, I have part of my makefile include all the relevant directories via a template:
DIRECTORIES = dirA dirB ... etc ...
define import_template
dir := $(1)
include $(1)/Rules.mk
include Rules.mk
endef
$(foreach DIR,$(DIRECTORIES), \
$(eval $(call import_template,$(DIR))))
Those includes build up a variables like TGT_BIN, a la the paper, that all works.
$ make # does the right thing
However, I want to provide the user the ability to make a subset of those directories. I know I can define DIRECTORIES like:
DIRECTORIES ?= dirA ...
So that:
$ make DIRECTORIES="dirB dirF"
works. But is there a way to write my makefile such that:
$ make -j12 dirB dirF
will do the same thing?
Assuming you have per-directory variables of targets to build (e.g. TGT_dirB, TGT_dirF, etc.) then doing what you want should be as simple as adding:
$(eval $(DIR): $(TGT_$(DIR)))
to the foreach loop like this:
$(eval $(call import_template,$(DIR)))$(eval $(DIR): $(TGT_$(DIR)))
to add all the per-directory targets as pre-requisites of the directory targets.
And adding
.PHONY: $(DIRECTORIES)
somewhere in the makefile to make sure make realizes those are phony targets and don't actually mean the directories themselves.
I would recommend you a non-recursive prorab build system.
It allows you to have independent makefile's in each directory while having a master makefile for everything. So, you'll be able to build only part of your project tree by cd'ing to the right subdir and invoking make.

How to handle dependencies from recursive Makefile invocations?

I have a C project that consists of a fairly large number of source files, and to make some sense of them, I have put them into subdirectories (with subdirectories). The whole project results in only one executable file, however.
In order to build this project, then, I am using recursive Makefiles, where the Makefile in each non-toplevel directory links all the object files produced in that directory into a concatenated lib.o file (using ld -r, that is). I do have a Makefile system that can build this and works rather fine for what it is, but it cannot support parallel make, which I would like to fix.
The problem is that I cannot figure out a proper way to both force make to descend into each directory's subdirectories, but also have the local lib.o target depend on that without being forced to rebuild even when nothing has changed.
This is how it works, somewhat abbreviated (leaving out CFLAGS and whatnot):
default: build
SUBOBJECTS = $(patsubst %,%/lib.o,$(SUBDIRS))
.PHONY: $(SUBDIRS)
$(SUBDIRS):
#$(MAKE) -C $#
build: $(SUBDIRS) lib.o
lib.o: $(OBJECTS) $(SUBOBJECTS)
$(LD) $(LDFLAGS) -r -o $# $^
This is from a Makefile.common which all other Makefiles include. Every other Makefile would also define their own SUBDIRS and OBJECTS. It might look like this, for instance:
SUBDIRS = dir1 dir2
OBJECTS = object1.o object2.o
include ../Makefile.common # Or ../../Makefile.common, &c.
As you can see from this, the main target is really the build target, which depends on the subdirectories and lib.o. If I invoke parallel make on this, it won't know that lib.o cannot be built until make has already run recursively on the subdirectories and will sometimes attempt that, causing errors. However, if I make lib.o depend on the subdirectories, then lib.o will always be unnecessarily rebuilt on each invocation, in each directory.
Is there a way to solve this? I've wrecked my brains on this for quite a while now without being able to find a way out. I'm only using GNU make, so don't worry too much about being POSIX-compatible.

GNU Make -- Append to a variable all targets matching a pattern

Before I start, I'll mention that I'm not using GNU Make in this case for building a C/C++ project.
Makefile:
DEST_DIR = build/
SRC_DIR = src/
$(SRC_DIR)a/ : $(SOMETHING_ELSE)
$(DO_SOMETHING_TO_GENERATE_A_DIR)
$(DEST_DIR)% : $(SRC_DIR)%
cp -r $^ $#
ALL_DEPS += <SOMETHING>
... more code which appends to ALL_DEPS ...
.PHONY: all
all : $(ALL_DEPS)
I've got some files not generated via Make rules in $(SRC_DIR). (For the sake of this example, let's say there's a directory $(SRC_DIR)b/ and a file $(SRC_DIR)c .)
I want to append to ALL_DEPS all targets which represent files or directories in $(DEST_DIR) so that "make all" will run all of the available $(DEST_DIR)% rules.
I thought to do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(wildcard $(SRC_DIR)*)))
But of course, that doesn't catch anything that hasn't yet been made. (i.e. it doesn't append $(DEST_DIR)a/ to the list because $(SRC_DIR)a/ doesn't yet exist when the $(wildcard ...) invocation is evaluated and the shell doesn't include it in the results returned by the $(wildcard ...) invocation.)
So, rather than a function which finds all (currently-existing) files matching a pattern, I need one which finds all targets matching a pattern. Then, I could do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(targetwildcard $(SRC_DIR)*)))
If it matters any, I've got much of the GNU Make code split across multiple files and included by a "master" Makefile. The ALL_DEPS variable is appended to in any of these files which has something to add to it. This is in an attempt to keep the build process modular as opposed to dropping it all in one monster Makefile.
I'm definitely still learning GNU Make, so it's not unlikely that I'm missing something fairly obvious. If I'm just going about this all wrong, please let me know.
Thanks!
It is simply not possible to do what you're trying to do; you're trying to get make to recognise something that doesn't exist.
This is part of the reason why, in general, wildcards are bad (the other being that you can end up including stuff you didn't mean to). The right thing to do here is to explicitly create a list of source files (ls -1 | sed -e 's/\(.*\)/sources+=\1/' > dir.mk) and perform the patsubst transformation on that list.
If you have additional files that are generate as part of the build, then you can append them to that list and their rules will be found as you'd expect.

Using make to build several binaries

I want to create a Makefile (in a parent dir) to call several other Makefiles (in sub dirs) such that I can build several binaries (one per project sub dir) by invoking just the one parent Makefile.
My research has been hampered by finding loads of stuff on recursive Makefiles, but I think this is where you are trying to build several directories Makefiles into a single binary?
Maybe what I want to do is better handled by a shell script perhaps invoking make in each sub directory in turn, but I thought a Makefile might be a more elegant solution?
any pointers gratefully received
PS using linux and the GNU tool chain
The for loop solution given in the first answer above actually shouldn't be used, as-is. In that method, if one of your sub-makes fails the build will not fail (as it should) but continue on with the other directories. Not only that, but the final result of the build will be whatever the exit code of the last subdirectory make was, so if that succeeded the build succeeds even if some other subdirectory failed. Not good!!
You could fix it by doing something like this:
all:
#for dir in $(SUBDIRS); \
do \
$(MAKE) -C $${dir} $# || exit $$?; \
done
However now you have the opposite problem: if you run "make -k" (continue even if there are errors) then this won't be obeyed in this situation. It'll still exit on failure.
An additional issue with both of the above methods is that they serialize the building of all subdirectories, so if you enable parallel builds (with make's -j option) that will only happen within a single subdirectory, instead of across all subdirectories.
Eregrith and sinsedrix have solutions that are closer to what you want, although FYI you should never, ever use "make" when you are invoking a recursive make invocation. As in johfel's example you should ALWAYS use $(MAKE).
Something like this is what you want:
SUBDIRS = subdir1 subdir1 subdir3 ...
all: $(addprefix all.,$(SUBDIRS))
all.%:
# $(MAKE) -C '$*' '$(basename $#)'
.PHONY: $(addprefix all.,$(SUBDIRS))
And of course you can add more stanzas like this for other targets such as "install" or whatever. There are even more fancy ways to handle building subdirectories with any generic target, but this requires a bit more detail.
If you want to support parallel builds you may need to declare dependencies at this level to avoid parallel builds of directories which depend on each other. For example in the above if you cannot build subdir3 until after both subdir1 and subdir2 are finished (but it's OK for subdir1 and subdir2 to build in parallel) then you can add something like this to your makefile:
all.subdir3 : all.subdir1 all.subdir2
You can call targets in subdirectory makefiles via
all:
$(MAKE) -C subdirectory1 $#
$(MAKE) -C subdirectory2 $#
...
or better
SUBDIRS=subd1 subd2 subd3
all:
#for dir in $(SUBDIRS); \
do \
$(MAKE) -C $${dir} $#; \
done
you should indeed use cmake to generate the Makefile automatically from a given CMakeLists.txt configuration file.
Here's a random link to get you started. Here you can find a simple sample project, including multiple subdirectories, executables, and a shared library.
Each makefile can have several target, it's still true with recursive makefiles, usually it's written:
all: target1 target2 target3
target1 :
make -C subdir
Then make all

Resources