I'm using GNU make, which I thought would make this easier.
I'm trying to build in either build/Debug or build/Release, depending on whether the target is 'debug' or 'release'.
Here's the relevant excerpt of my makefile:
# Project files
SRC_DIRS = ../source ..source/[...], etc.
SRCS := $(shell find $(SRC_DIRS) -maxdepth 1 -name *.cpp -or -name *.cxx -or -name *.c -or -name *.s)
SRCS := $(sort $(SRCS))
vpath %.cpp $(SRC_DIRS)
vpath %.cxx $(SRC_DIRS)
vpath %.c $(SRC_DIRS)
# Object files
OBJS := $(addsuffix .o, $(notdir $(basename $(SRCS))))
# Dependencies
DEPS := $(OBJS:.o=.d)
debug: DEPS_DIR = build/Debug
release: DEPS_DIR = build/Release
DEPS := $(addprefix $(DEPS_DIR)/, $(DEPS))
include $(DEPS)
The problem is that I can't get $(DEPS) to evaluate properly. It seems like the DEPS addprefix is being evaluated before the target-specific DEPS_DIR assignments, so that it's resulting in "/filename.d", since it thinks $DEPS_DIR is blank. However, if I go down further and under the actual targets add a echo "DEPS_DIR = $(DEPS_DIR)", it expands correctly.
There's probably something obvious I'm doing wrong. How can I point include at the DEPS list in either of build/Debug or build/Release, depending on target?
You've missed the part of the documentation on target-specific variables where it specifies that they are in effect only when expanding the recipes for rules.
Make works by reading in the entire makefile all at once and building its internal graph of targets and prerequisites, and only after that is complete does it look to see which targets should be built based on the command line, etc. That's why order of rule definition, etc. mostly doesn't matter: make is not "running" the makefile as it is read in, as if it were a script. So there's no way for a target-specific variable to impact what files are included by the include statement.
There are a few ways to do what you want. The simplest and most portable one is to use a recursive invocation of make, like this:
debug:
$(MAKE) DEPS_DIR=build/Debug realtarget
release:
$(MAKE) DEPS_DIR=build/Release realtarget
Alternatively the MAKECMDGOALS make variable contains whatever goals were specified on the command line, so you can check this in a make ifeq statement or similar and set DEPS_DIR that way.
Related
This is my first ever post to stackoverflow. It ought to be straightforward, but I have this problem whenever I try to write a makefile and I've never been able to figure out a satisfactory solution. Apologies if there is already a solution somewhere on the site. I couldn't find one.
What I'm trying to do is as follows:
Search my src directory for matching source files.
Compile the src code into a sandbox.
Here's my directory structure:
makefile
src1
file1.src
file2.src
src2
file3.src
subfolder
src3
file4.src
file5.src
And here's my makefile:
BUILDDIR := ./sandbox
SRC_DIRS := ./
SRCS := $(shell find $(SRC_DIRS) -name *.src)
OBJS := $(addprefix $(BUILDDIR)/o., $(notdir $(SRCS) ) )
# make print-X prints value of variable X
print-%: ; #echo $* = $($*)
.PHONY: help
help:
#echo "make <all|clean>"
.PHONY: all
all: $(OBJS)
#echo "compilation done"
$(OBJS) : $(SRCS) $(BUILDDIR)/.create
#echo "\"compiling\" $< to produce $#"
cp $< $#
$(BUILDDIR)/.create:
#echo "creating sandbox"
mkdir -p $(BUILDDIR) && cd $(BUILDDIR)
#touch $#
.PHONY: clean
clean:
#echo "deleting sandbox"
#rm -rf $(BUILDDIR)
If I type make all, the file works as expected. However, if I type make all again, instead of saying everything is up to data, I end up with the following contents in the sandbox:
o.file1.src o.file2.src o.file3.src o.file4.src o.file5.src o.o.file1.src o.o.file2.src o.o.file3.src o.o.file4.src o.o.file5.src
And the process of creating objects of objects continues recursively as many times as I type make.
Any help would be appreciated.
Incidentally, please don't post solutions that rely on the build in compile functions of make. I'm looking for a general solution that can be used for any task. For example, in this instance, I'm trying to read the source files into a tool using its command line interface.
Well, first, by having your sandbox as a subdirectory of your source directory, then using find on the source directory, every time you run it you're going to find all the files in both the source directory and all its subdirectories, including the sandbox. If the built files in the sandbox have the same names as the files in the source directories, the find will find them all.
Maybe instead of:
SRCS := $(shell find $(SRC_DIRS) -name *.src)
you want something like:
SRCS := $(shell find $(SRC_DIRS) -name $(notdir $(BUILDDIR)) -prune -o -name *.src -print)
Or, alternatively, don't make your sandbox a subdirectory of your source directory. Or, make sure that whatever name you give to the files in the sandbox directory won't match the *.src pattern you give to find.
But beyond that this is wrong:
$(OBJS) : $(SRCS) $(BUILDDIR)/.create
Suppose SRCS is foo.src bar.src, which means OBJS is sandbox/o.foo.src sandbox/o.bar.src. Then the above expands to this:
sandbox/o.foo.src sandbox/o.bar.src : foo.src bar.src sandbox/.create
This is a common mistake; people seem to think that make will go through the targets and prerequisites and match them up, so the first target depends on the first prerequisite and the second target depends on the second one etc. but of course this cannot work correctly and that's not how make works. Make treats the above as if you'd written one rule for each target, with the same prerequisites; like this:
sandbox/o.foo.src : foo.src bar.src sandbox/.create
sandbox/o.bar.src : foo.src bar.src sandbox/.create
You can see this won't do what you want at all, since the $< will always be foo.src which is clearly wrong.
You need to write a single pattern rule that will build ONE target. Then make sure the pattern applies to all the targets.
You have made things hard for yourself by trying to "flatten" a directory structure of multiple source subdirectories, into a single level of target directory (by using the $(notdir $(SRCS))). Because of this, there's no pattern that will match the same target and directory, unless you write a separate rule for every subdirectory.
Luckily there is a solution for this: VPATH. This should work for you:
VPATH := $(sort $(dir $(SRCS))
$(BUILDDIR)/o.%.src : %.src $(BUILDDIR)/.create
#echo "\"compiling\" $< to produce $#"
cp $< $#
The VPATH tells make to go look in all the directories that it found any sources in, whenever it can't find one to build.
The basic problem is that your SRCS is all files in all subdirectories that match the pattern *.src (when you run make). That means that all your object files ($(OBJS)) also match, so they copied as well.
The solution is to change your SRCS pattern so it does not match the "object" files in the build directory. Possibilities:
SRCS := $(wildcard *.src)
or
SRCS := $(shell find $(SRC_DIRS) -name $(notdir $(BUILDDIR)) -prune -false -o -name *.src)
or change the names of your "object" files so they don't end in .src
If I type make all, the file works as expected.
By which I take you to mean that directory ./sandbox exists and contains these files:
o.file1.src
o.file2.src
o.file3.src
o.file4.src
o.file5.src
However, if I type make all again, instead of saying everything is up to data, I end up with the following contents in the sandbox:
o.file1.src o.file2.src o.file3.src o.file4.src o.file5.src o.o.file1.src o.o.file2.src o.o.file3.src o.o.file4.src o.o.file5.src
Of course you do, because everything is not up to date at that point, according to the target list you create. This line ...
SRCS := $(shell find $(SRC_DIRS) -name *.src)
... defines SRCS as a list of all the files under path ./ (which is the value of SRC_DIRS) that ends in .src. That includes any such files in ./sandbox, which include all the files placed there by the first make run. You generate a corresponding target file to build for each source, and those targets corresponding to sources built by the previous make run will not, in general, exist yet. So make builds them, just as you instructed it.
The best solution, short of abandoning that automatic source identification altogether, would probably be to change your naming scheme so that outputs cannot be mistaken so easily for sources. For example, if you want the output names to have the form foo.src, then have the corresponding input named something like foo.src.in. In that particular case, you could convert from source names to target names with
OBJS := $(addprefix $(BUILDDIR)/o., $(basename $(notdir $(SRCS) ) ) )
Alternatively, you could modify the find command to skip the sandbox directory, maybe by modifying SRC_DIRS:
SRC_DIRS = src1 src2 subfolder/src3
(These specific alternatives are not mutually exclusive.)
I have the following hierarchy:
+ makefile
|
+ TT_Project1
| obj/
| makefile
+ TT_Project2
| obj/
| makefile
+ TT_Project3
| obj/
| makefile
The top level makefile calls recursively all makefiles below it, each compiles the .cpp to .o and places it under /obj
I would like to collect all .o in the top level to create a static library. And failing as my output library is too small...
TT := libTT.a
SUBDIRS := $(wildcard */.)
OBJECTS := $(wildcard $(addsuffix *.o,$(SUBDIRS)/obj))
all: $(TT) $(OBJECTS)
$(TT) : $(OBJECTS)
ar rcs $(TT) $(OBJECTS)
$(SUBDIRS):
$(MAKE) -C $#
I can clearly see that it's running the makefiles beneath it, and they are placing the *.o under their /obj as expected. But I am failing to grab all the .o I guess... any help is appreciated.
First, this is wrong:
OBJECTS := $(wildcard $(addsuffix *.o,$(SUBDIRS)/obj))
Not only are you missing a / between the obj and *.o, but you need to put the /obj in the first argument to addsuffix. Suppose SUBDIRS ends up as foo/. bar/. baz/.. Then the addsuffix will expand to:
$(addsuffix *.o,foo/. bar/. baz/./obj)
which gives:
foo/.*o bar/.*.o baz/./obj*.o
which is clearly wrong. You want this:
OBJECTS := $(wildcard $(addsuffix /obj/*.o,$(SUBDIRS)))
However, that isn't what you want. It won't work because these variables are expanded when the makefile is read, before any actions have been taken. Before the makefile actually builds anything, there will be no .o files existing, which means the wildcard function will match nothing.
In your case that's a good thing, because if it did match anything you'll get errors because make doesn't know how to build those objects: they're built by the sub-makes.
But, since you have nothing that depends on the SUBDIRS target they won't get built anyway: those recipes will never run.
The easiest thing you can do is something like this:
TT := libTT.a
SUBDIRS := $(wildcard */.)
all: $(TT)
$(TT) : $(SUBDIRS)
ar rcs $(TT) $(addsuffix /obj/*.o,$(SUBDIRS))
$(SUBDIRS):
$(MAKE) -C $#
.PHONY: $(SUBDIRS)
The downside of this is that it will rebuild the library every time you run make even if no objects have been changed. To fix that you'll have to add some complexity.
I have a makefile that is trying to do the following: identify all files under the current directory (all sub-directories included) with .c and .s extensions, for each one compile a non-linked object file and put it into a directory. All C files end up in objects/c, all assembly files end up in objects/ass.
The makefile always works as expected on the first execution (all commands are called in the right order) and no errors are produced.
However if I call make again, half of the time i get "nothing to be done for 'all'.". Which is what you would expect, since no files have been modified. But the other half of the time, make is selecting a random assembly file and compiling that file. That is to say,if I keep doing "make" I sometimes compile file1.s sometimes file2.s. and it keeps randomly swapping between the assembly files add infinitum (it never reaches a "nothing to be done") state.
How is make exhibitting non deterministic behaviour?
This is the smallest makefile I could make that reproduces the error:
SRC_C = $(wildcard *.c) $(wildcard **/*.c)
SRC_ASS = $(wildcard *.s) $(wildcard **/*.s)
OBJECTS_C = $(addprefix $(OBJECT_DIR)c/, $(notdir $(SRC_C:.c=.o)))
OBJECTS_ASS = $(addprefix $(OBJECT_DIR)ass/, $(notdir $(SRC_ASS:.s=.o)))
OBJECTS = $(OBJECTS_C) $(OBJECTS_ASS)
OBJECT_DIR = objects/
all: $(OBJECTS)
%/:
mkdir $#
$(OBJECTS_C): $(OBJECT_DIR) $(OBJECT_DIR)c/
arm-none-eabi-gcc -O0 -march=armv8-a $(wildcard */$(#F:.o=.c)) -nostartfiles -c -o $#
$(OBJECTS_ASS): $(OBJECT_DIR) $(OBJECT_DIR)ass/
arm-none-eabi-as -march=armv8-a $(wildcard */$(#F:.o=.s)) -c -o $#
.PHONY: clean
clean:
rm -rf $(OBJECT_DIR)
You have many errors here.
The biggest is a conceptual one: By flattening all your object files into one directory, there's no way to express proper dependencies using pattern rules, so your object files do not really depend on their respective source files. I'd say: just don't do that! Having object directories is fine, but they should mirror the directory structure of the source tree.
Further errors:
directly depending on directories. This will not work as expected, directories should always be order-only dependencies, as already stated in the comments
Make doesn't support recursive wildcards -- if you really need that, you could write your own function or, assuming you're always building on *nix, just call find instead
Pattern rules for creating directories are not the best idea either -- I'd suggest to collect all needed directories in a variable and loop over that.
Stylistic improvements:
Assign variables that don't need deferred evaluation with :=
Assign variables influencing the build process with ?=, so the user can override them at the command line
Use "standard" variables like CC, AS, CROSS_COMPILE
declare all phony targets in .PHONY.
Your Makefile with these changes applied would look like this:
OBJECT_DIR ?= objects
C_OBJECT_DIR ?= $(OBJECT_DIR)/c
AS_OBJECT_DIR ?= $(OBJECT_DIR)/ass
SRC_C:= $(shell find -name \*.c)
SRC_ASS:= $(shell find -name \*.s)
OBJECTS_C:= $(addprefix $(C_OBJECT_DIR)/, $(SRC_C:.c=.o))
OBJECTS_ASS:= $(addprefix $(AS_OBJECT_DIR)/, $(SRC_ASS:.s=.o))
OBJECTS:= $(OBJECTS_C) $(OBJECTS_ASS)
OUTDIRS:= $(sort $(dir $(OBJECTS)))
CROSS_COMPILE ?= arm-none-eabi-
CC ?= gcc
AS ?= as
CFLAGS ?= -O0 -march=armv8-a -nostartfiles
ASFLAGS ?= -march=armv8-a
all: $(OBJECTS)
$(OUTDIRS):
$(foreach _dir,$#,mkdir -p $(_dir);)
$(C_OBJECT_DIR)/%.o: %.c | $(OUTDIRS)
$(CROSS_COMPILE)$(CC) -c -o $# $(CFLAGS) $<
$(AS_OBJECT_DIR)/%.o: %.s | $(OUTDIRS)
$(CROSS_COMPILE)$(AS) -c -o $# $(ASFLAGS) $<
clean:
rm -rf $(OBJECT_DIR)
.PHONY: all clean
Note there is one important thing missing: automatic dependencies. With this Makefile, each object file depends on its respective source file, but completely misses any headers included. For anything other than a simple toy, you should add that, google for "gnu make gcc automatic dependencies" or something similar (not the scope of this question).
I have a directory structure like
packages/
foo/
lib/
a.js
b.js
bar/
lib/
c.js
d.js
and I'm trying to write a simple Makefile to run the source files through a compiler and output the result of compiling packages/foo/lib/a.js to packages/foo/build/a.js. In the past, I've done something like
JS = $(shell find packages/foo/lib -name "*.js")
BUILD = $(patsubst packages/foo/lib/%.js, packages/foo/build/%.js, $(JS))
.PHONY: all
all: $(BUILD)
packages/foo/lib/%.js: packages/foo/build/%.js
# Compile $^ and output to $#
which works great. However, I'm now doing
JS = $(shell find packages/*/lib -name "*.js")
BUILD = $(patsubst ...)
The problem here is that patsubst doesn't seem to like multiple % wildcards (ie packages/%/lib/%.js). However, if I use packages/%.js, then I can't know which directory to use in the substitution. I'm convinced there's a very simple way to do this that I can't find in the make docs.
To define BUILD, you could use the stupid but simple solution:
BUILD = $(subst /lib/,/build/,$(JS))
as long as you aren't worried about additional "lib" and "build" pathname components in your source.
However, this leaves you with the problem of how to define the actual Makefile rules, unless you want to manually specify a rule for each directory foo, bar, etc.
Instead, maybe try something like this (remember to replace spaces with tabs again where appropriate, if you cut and paste):
PACKAGES = $(wildcard packages/*)
ALL :=
define add_package
JS := $$(shell find $(1)/lib -name "*.js")
BUILD := $$(patsubst $(1)/lib/%.js, $(1)/build/%.js, $$(JS))
$(1)/build/%.js:: $(1)/lib/%.js
echo COMPILE $$^ '->' $$#
cp $$^ $$# # super fast compiler!
ALL += $$(BUILD)
endef
$(foreach p, $(PACKAGES), \
$(eval $(call add_package,$(p))))
all: $(ALL)
This evaluates a template for each package directory, so you can put anything in there, as long as you get the fiddly doubling of dollar signs correct.
I have a "lib" directory in my applications main directory, which contains an arbitrary number of subdirectories, each having its own Makefile.
I would like to have a single Makefile in the main directory, that calls each subdirectory's Makefile. I know this is possible if I manually list the subdirs, but I would like to have it done automatically.
I was thinking of something like the following, but it obviously does not work. Note that I also have clean, test, etc. targets, so % is probably not a good idea at all.
LIBS=lib/*
all: $(LIBS)
%:
(cd $#; $(MAKE))
Any help is appreciated!
The following will work with GNU make:
LIBS=$(wildcard lib/*)
all: $(LIBS)
.PHONY: force
$(LIBS): force
cd $# && pwd
If there might be something other than directories in lib, you could alternatively use:
LIBS=$(shell find lib -type d)
To address the multiple targets issue, you can build special targets for each directory, then strip off the prefix for the sub-build:
LIBS=$(wildcard lib/*)
clean_LIBS=$(addprefix clean_,$(LIBS))
all: $(LIBS)
clean: $(clean_LIBS)
.PHONY: force
$(LIBS): force
echo make -C $#
$(clean_LIBS): force
echo make -C $(patsubst clean_%,%,$#) clean
There is also a way of listing sub-directories with gmake commands only, without using any shell commands:
test:
#echo $(filter %/, $(wildcard lib/*/))
This will list all sub-directories with trailing '/'. To remove it you can use the substitute pattern:
subdirs = $(filter %/, $(wildcard lib/*/))
test:
#echo $(subdirs:%/=%)
Then to actually create rules executing makefiles in each sub-directory you can use a small trick - a phony target in a non-existent directory. I think in this case an example will tell more than any explanation:
FULL_DIRS =$(filter %/, $(wildcard lib/*/))
LIB_DIRS =$(FULL_DIRS:%/=%)
DIRS_CMD =$(foreach subdir, $(LIB_DIRS), make-rule/$(subdir))
make-rule/%:
cd $* && $(MAKE)
all: DIRS_CMD
Basically, target 'all' lists all sub-directories as prerequisites. For example, if LIB_DIRS contained lib/folder1 lib/folder2 then the expansion would look like this:
all: make-rule/lib/folder1 make-rule/lib/folder2
Then 'make', in order to execute rule 'all', tries to match each prerequisite with an existing target. In this case the target is 'make-rule/%:', which uses '$*' to extract the string after 'make-rule/' and uses it as argument in the recipe. For example, the first prerequisite would be matched and expanded like this:
make-rule/lib/folder1:
cd lib/folder1 && $(MAKE)
What if you want to call different targets than all in an unknown number of subdirectories?
The following Makefile uses macros so create a forwarding dummy-target for a number of subdirectories to apply the given target from the command line to each of them:
# all direct directories of this dir. uses "-printf" to get rid of the "./"
DIRS=$(shell find . -maxdepth 1 -mindepth 1 -type d -not -name ".*" -printf '%P\n')
# "all" target is there by default, same logic as via the macro
all: $(DIRS)
$(DIRS):
$(MAKE) -C $#
.PHONY: $(DIRS)
# if explcit targets where given: use them in the macro down below. each target will be delivered to each subdirectory contained in $(DIRS).
EXTRA_TARGETS=$(MAKECMDGOALS)
define RECURSIVE_MAKE_WITH_TARGET
# create new variable, with the name of the target as prefix. it holds all
# subdirectories with the target as suffix
$(1)_DIRS=$$(addprefix $(1)_,$$(DIRS))
# create new target with the variable holding all the subdirectories+suffix as
# prerequisite
$(1): $$($1_DIRS)
# use list to create target to fullfill prerequisite. the rule is to call
# recursive make into the subdir with the target
$$($(1)_DIRS):
$$(MAKE) -C $$(patsubst $(1)_%,%,$$#) $(1)
# and make all targets .PHONY
.PHONY: $$($(1)_DIRS)
endef
# evaluate the macro for all given list of targets
$(foreach t,$(EXTRA_TARGETS),$(eval $(call RECURSIVE_MAKE_WITH_TARGET,$(t))))
Hope this helps. Really helpfull when dealing with paralelism: make -j12 clean all in a tree with makefiles having these targets... As always: playing with make is dangerous, different meta-levels of programming are too close together ,-)