Based on this paper, I'm trying to rework a subset of my build system to be non-recursive. It's actually working pretty well. By default, I have part of my makefile include all the relevant directories via a template:
DIRECTORIES = dirA dirB ... etc ...
define import_template
dir := $(1)
include $(1)/Rules.mk
include Rules.mk
endef
$(foreach DIR,$(DIRECTORIES), \
$(eval $(call import_template,$(DIR))))
Those includes build up a variables like TGT_BIN, a la the paper, that all works.
$ make # does the right thing
However, I want to provide the user the ability to make a subset of those directories. I know I can define DIRECTORIES like:
DIRECTORIES ?= dirA ...
So that:
$ make DIRECTORIES="dirB dirF"
works. But is there a way to write my makefile such that:
$ make -j12 dirB dirF
will do the same thing?
Assuming you have per-directory variables of targets to build (e.g. TGT_dirB, TGT_dirF, etc.) then doing what you want should be as simple as adding:
$(eval $(DIR): $(TGT_$(DIR)))
to the foreach loop like this:
$(eval $(call import_template,$(DIR)))$(eval $(DIR): $(TGT_$(DIR)))
to add all the per-directory targets as pre-requisites of the directory targets.
And adding
.PHONY: $(DIRECTORIES)
somewhere in the makefile to make sure make realizes those are phony targets and don't actually mean the directories themselves.
I would recommend you a non-recursive prorab build system.
It allows you to have independent makefile's in each directory while having a master makefile for everything. So, you'll be able to build only part of your project tree by cd'ing to the right subdir and invoking make.
Related
I have a makefile that looks something like this:
include anotherFile.mk
all:
someStuff
The file anotherFile.mk is like this:
include yetAnotherFile.mk
export SOME_VAR = 93
The problem is that anotherFile.mk and yetAnotherFile.mk are in a different directory from my Makefile. So my makefile can't just be changed to this:
include $(OTHER_PROJECT_PATH)/anotherFile.mk
all:
someStuff
The problem with this approach is that the include statement in anotherFile.mk will fail because it will be searching in the current directory.
A partial solution that I found is to pass the --include-dir=$OTHER_PROJECT_PATH flag to the invocation of make, but that's a bit user-unfriendly.
So my question is: Is there something I can put inside my makefile that will add to the directories that make searches for when executing an include? Something like MAKE_INCLUDE_DIRS += $(OTHER_PROJECT_PATH)
Surprisingly there doesn't seem to be a good answer to that question. Forcing .INCLUDE_DIR doesn't help and there doesn't seem to be any way around invoking make with --include-dir=$OTHER_PROJECT_PATH.
It is however possible to put the appropriate recursive make invocation inside the makefile but, in order to get it to work for all reasonable cases it quickly becomes too complicated to be worth it. In summary it requires:
a top level condition to check if the OTHER_PROJECT_PATH is in .INCLUDE_DIR
the appropriate target with the recipe invoking make recursively
possibly additional targets if there are multiple command goals
the real make file enclosed in the else part of the conditional
You Makefile would look like this:
OTHER_PROJECT_PATH := other
ifeq (,$(filter $(OTHER_PROJECT_PATH), $(.INCLUDE_DIRS)))
# this is the mechanism to add the include dir in a recursive make
$(or $(firstword $(MAKECMDGOALS)),all):
$(MAKE) -I$(OTHER_PROJECT_PATH) $(MAKECMDGOALS)
# add empty targets for additional goals if needed
ifneq (,$(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS)))
$(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS)):
endif
else
# this is where the real makefile starts
all more:
echo $#: $< $^
include a.mak
endif
It still does not seem possible from a makefile, but if you have a script that sets up environment variables, you can use MAKEFLAGS (e.g. export MAKEFLAGS=I/your/path ordentlich on Linux, or SET on Windows)
I'm trying to make a rule that will generate files regarding their names but regardless of the directory.
I'm starting from this makefile (see a previous question of mine):
TARGETS:=$(patsubst %_tpl,%,$(wildcard *_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $#_tpl > $#
With this, I can do, for instance, make foo.xml. It looks if a set of foo.xml*_tpl files are there, consider them as prerequisites and call the generate script to generate the target.
What I would like to do is, for example, make ../ressources/foo.xml and have make use the rule to create foo.xml but creating it in the ../ressources/ directory, without having to explicitely specify this directory in the makefile.
What I have tried for the moment is adding this to the Makefile:
../ressources/%: $(notdir %)
mv $< $#
Which works, but I would like to avoid creating the file in the current directory before moving it to the destination folder. I would also like not having to specify the possible destination folders in the makefile (but this is less important).
But first of all, does this make any sense? Or is what I want to do just conceptually wrong?
EDIT: Some precisions regarding the _tpl files and the generate script to avoid confusions:
Each target has a main template ($#_tpl) that includes others ($#-part1_tpl, $#-part2_tpl...) and the generate script only takes the main template as argument. The templates are written with Jinja2 (the subparts included with the {% include %} jinja directive).
If you always want the targets in another directory, just say so.
TARGETS:=$(patsubst %_tpl,../resources/%,$(wildcard *_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): ../resources/%: $$(wildcard %*_tpl)
./generate $#_tpl > $#
I'm not sure if you should have generate $^ >$# instead; superficially, this would make more sense.
If there are multiple *_tpl files for each target (i.e. there are more tpl files than xml files), the TARGETS definition isn't really correct; but we don't have enough information to actually fix it.
On the other hand, if the target directory can change a lot, the sane way forward might be to cd into the target directory and use make -f ../path/to/Makefile -- just make sure your VPATH is set up so that the source files can be found.
I have a following directory structure in my project:
bin/
dist/
include/
├── module_a/
└── module_b/
Makefile
src/
├── module_a/
└── module_b/
Folder include/ contains *.hpp's while *.cpp's are in src/. I would like to compile all sources to bin/ and then link them up together to dist/. Seems a pretty reasonable wish for me.
I would like to know the best practices for a Makefile for this case. All I can find is %.o: %.cpp target, but that doesn't really work, because of different source and binary folder.
I was trying to use something like this:
D_SRC = src
D_BIN=bin
F_CPP := $(shell find $(D_SRC) -iname '*.cpp' -type f)
F_OBJ := $(shell echo $(F_CPP) | sed s:\ :\\n:g | sed s:$(D_SRC):$(D_BIN): | sed 's:^\(.*\)\.cpp$$:\1\.o:')
$(F_OBJ): $(F_SRC)
$(foreach file, $(F_SRC), \
$(GXX) $(CXXFLAGS) -c $(file)\
)
This target doesn't work, because $(F_OBJ) paths start with bin/, while foreach compiles sources to current working dir. I could make it compile to bin/, but that would happen only with a few more sed expressions and it's ugly enough as it is.
It's probably so difficult for me, because I don't know make all that well, but I cannot be the only one with this project setup. In my opinion, it must be a pretty common one. I know I can write a Makefile for each module separately, but is that really the best choice here?
EDIT: I was now wondering what would I achieve with several Makefiles. If one was at root and another one in src/module_a, how would the latter know about the bin/? If you'd execute it with make -f src/module_a/Makefile, it would be the same as executing it from root directory, 'cause it's working directory would be root. Another way, I guess, would be to change directory before executing it, like so: make -C include/module_a, but in that case, how would it find bin/? I wouldn't want to have something like D_BIN = ../../bin in a Makefile.
What I normally do is have a Makefile in the src directory (which can be invoked from the top level Makefile if you like) and then use rules like this:
D_BIN = ../bin
$(D_BIN)/%.o: %.cpp
You could also experiment with just a makefile in the top level dir, and use rules that look like this:
D_BIN = bin
D_SRC = src
$(D_BIN)/%.o: $(D_SRC)/%.cpp
but I have not used such rules, so I don't know the pros/cons vs the way I normally do it. The way I normally do it works fine, I even have rules that build depends like so:
$(D_BIN)/%.d: %.cpp
and the link rule would be like:
../dist/outexe: $(F_OBJ)
Using a foreach is usually frowned upon because it does not make use of all the features built into normal makefile rules (i.e. there is no depends check on a per file basis, either you build everything or nothing), and as such foreach should only be used as a last resort, but in this case you will be able to get it to work without the foreach.
In addition to this there are much easier ways to build your file lists, you don't need to use the shell or sed.
F_CPP = $(wildcard *.cpp)
F_OBJ = $(F_CPP:.cpp=.o)
Update: This is how I normally issue recursive makes:
SUBDIRS = src
.PHONY: $(SUBDIRS)
all: $(SUBDIRS)
$(SUBDIRS):
#echo "Building $#..."
$(MAKE) -C $# $(MFLAGS)
Then indeed in your submake, you would need to use ../bin for example.
However with a project as simple as yours, you might be better off just having one makefile at the root level and using rules like this:
D_BIN = bin
D_SRC = src
$(D_BIN)/%.o: $(D_SRC)/%.cpp
recursive makefiles are ok (ok but not great) if you have a really complex directory structure, where you will be adding/removing/modifying new dir trees as time goes on. But for a simple project where you just want to have separate directories for code and objs, it is probably overkill.
The question is about parallel making w/ GNU makefile.
Given a folder structure as below, the goal is to deliver a makefile that it supports make release/debug/clean in parallel.
project folder structure:
foo
+-foo1
+-foo2
+-foo3
The makefile may be sth like:
SUBDIR = foo1 foo2 foo3
.PHONY $(SUBDIR) release debug clean
release: $(SUBDIR)
$(SUBDIR):
$(MAKE) -C $# release
debug: $(SUBDIR)
#below is incorrect. $(SUBDIR) is overriden.
$(SUBDIR):
$(MAKE) -C $# debug
..
Sub directory list are set as phony targets for parallel making. but it lost the information of original target (release, debug, clean etc).
One method is to suffix the names for the directories and recover it in commands, but it is weird. another method might be to use variables, but not sure how to work it out.
The questions is:
How to write the rules for directories, that supports parallel making w/ different targets (release/debug/clean)?
Any hints are greatly appreciated.
Setting variables on the command line certainly works. You can also use MAKECMDGOALS (see the GNU make manual):
$(SUBDIR):
$(MAKE) -C $# $(MAKECMDGOALS)
I want to create a Makefile (in a parent dir) to call several other Makefiles (in sub dirs) such that I can build several binaries (one per project sub dir) by invoking just the one parent Makefile.
My research has been hampered by finding loads of stuff on recursive Makefiles, but I think this is where you are trying to build several directories Makefiles into a single binary?
Maybe what I want to do is better handled by a shell script perhaps invoking make in each sub directory in turn, but I thought a Makefile might be a more elegant solution?
any pointers gratefully received
PS using linux and the GNU tool chain
The for loop solution given in the first answer above actually shouldn't be used, as-is. In that method, if one of your sub-makes fails the build will not fail (as it should) but continue on with the other directories. Not only that, but the final result of the build will be whatever the exit code of the last subdirectory make was, so if that succeeded the build succeeds even if some other subdirectory failed. Not good!!
You could fix it by doing something like this:
all:
#for dir in $(SUBDIRS); \
do \
$(MAKE) -C $${dir} $# || exit $$?; \
done
However now you have the opposite problem: if you run "make -k" (continue even if there are errors) then this won't be obeyed in this situation. It'll still exit on failure.
An additional issue with both of the above methods is that they serialize the building of all subdirectories, so if you enable parallel builds (with make's -j option) that will only happen within a single subdirectory, instead of across all subdirectories.
Eregrith and sinsedrix have solutions that are closer to what you want, although FYI you should never, ever use "make" when you are invoking a recursive make invocation. As in johfel's example you should ALWAYS use $(MAKE).
Something like this is what you want:
SUBDIRS = subdir1 subdir1 subdir3 ...
all: $(addprefix all.,$(SUBDIRS))
all.%:
# $(MAKE) -C '$*' '$(basename $#)'
.PHONY: $(addprefix all.,$(SUBDIRS))
And of course you can add more stanzas like this for other targets such as "install" or whatever. There are even more fancy ways to handle building subdirectories with any generic target, but this requires a bit more detail.
If you want to support parallel builds you may need to declare dependencies at this level to avoid parallel builds of directories which depend on each other. For example in the above if you cannot build subdir3 until after both subdir1 and subdir2 are finished (but it's OK for subdir1 and subdir2 to build in parallel) then you can add something like this to your makefile:
all.subdir3 : all.subdir1 all.subdir2
You can call targets in subdirectory makefiles via
all:
$(MAKE) -C subdirectory1 $#
$(MAKE) -C subdirectory2 $#
...
or better
SUBDIRS=subd1 subd2 subd3
all:
#for dir in $(SUBDIRS); \
do \
$(MAKE) -C $${dir} $#; \
done
you should indeed use cmake to generate the Makefile automatically from a given CMakeLists.txt configuration file.
Here's a random link to get you started. Here you can find a simple sample project, including multiple subdirectories, executables, and a shared library.
Each makefile can have several target, it's still true with recursive makefiles, usually it's written:
all: target1 target2 target3
target1 :
make -C subdir
Then make all