Based on this famous link and adapted from this gist, and supposing that all of your source files are .cpp, you get easily a solution like this to have auto-dependence generation:
SRCS := $(wildcard *.cpp */*.cpp)
DEPDIR := .d
DEPS := $(SRCS:%.cpp=$(DEPDIR)/%.d)
# Temporary .Td dependence file... ?
DEPFLAGS = -MT $# -MD -MP -MF $(DEPDIR)/$*.Td
# Isn't it better a order-only prerequisite?
$(shell mkdir -p $(dir $(DEPS)) >/dev/null)
%.o: %.cpp # Removal of implicit rules... ? Twice?
%.o: %.cpp $(DEPDIR)/%.d # Dependency on .d... ?
g++ -o $# $(DEPFLAGS) $(CXXFLAGS) $(CPPFLAGS) -c $<
# Avoid some bugs?
mv -f $(DEPDIR)/$*.Td $(DEPDIR)/$*.d && touch $#
# If the `%.d` dependency is removed, is this still necessary?
.PRECIOUS = $(DEPDIR)/%.d
$(DEPDIR)/%.d: ;
-include $(DEPS)
To don't make this question too long discussing deeply why I think that all of the lines commented in the snippet above are innecesary, I will ask it in a short form; is there any difference in behaviour if I just change this snippet to:
SRCS := $(wildcard *.cpp */*.cpp)
DEPDIR := .d
DEPS := $(SRCS:%.cpp=$(DEPDIR)/%.d)
DEPFLAGS := -MT $# -MD -MP -MF $(DEPDIR)/$*.d
$(DEPDIR)/%:
mkdir -p $#
%.o: %.cpp | $(DEPDIR)/$(dir %)
g++ -o $(DEPFLAGS) $(CXXFLAGS) $(CPPFLAGS) -c $<
# touch $(DEPDIR)/$.d # ??
-include $(DEPS)
I have still two extra doubts; in the first link above, it says that
it’s been reported that some versions of GCC may leave the object file
older than the dependency file, which causes unnecessary rebuilds.
However, in the gist (second link above), the touch command is removed, and since the dependence file is no longer prerequisite of any rule, is there any reason to keep including it? Does this "gcc bug" still apply in any form?
The second doubt is related to the directory creation, moved to an order-only rule; do I need to make the "order-only" $(DEPDIR)/% rule .PRECIOUS? I don't know if make will attempt to remove the directory if the %.o recipe fails, because I don't know the concrete features of order-only rules.
You cannot remove the %.d prerequisite. The reason this is needed is explained in the page you linked.
I don't know what you mean by your comment Removal of implicit rules... ? Twice?. The removal of the implicit rule is needed, to ensure that our new implicit rule is used, and we only remove it once.
The temporary file .Td is used in case someone uses ^C or similar to kill their make job right in the middle of creating this file. By writing to a temporary file then only atomically replacing the real file once we know it's complete, we never have to worry about partial files which could cause the next invocation of make to generate an error or, worse, not recompile source files that should be recompiled.
Regarding the comment about object files older than the dependency file, first the gist you link to uses clang not GCC and perhaps Clang doesn't have this issue (or perhaps it does but people don't realize it). Second that update to the blog post is relatively recent as people have reported this issue to me with GCC. I've not seen it myself (I only use GCC) so maybe it's only an issue with some versions of GCC.
Regarding .PRECIOUS, make never (currently) recursively deletes directories so it won't delete any non-empty directory regardless of that setting.
Related
Make chooses the same source file for different object files. Both are a list of files, only with different filenames. Make switches between the object files but not the source files.
I've already tried some of the answers on StackOverflow with related problems, though those solutions either seem too complicated for what's needed, some don't work and others need the files to be in one directory.
I've also tried compiling the files together in one go (with gcc), but this gives some problems with the linking of the rest of the file.
$(OBJFILES): $(SRCFILES)
$(CC) $(CCFLAGS) -c $< -o $#
$(OBJFILES) contains the following files:
src/kernel.o src/screen/screen_basic.o
And $(SRCFILES) contains these files:
src/kernel.c src/screen/screen_basic.c
Basically, src/kernel.c gets compiled to both src/kernel.o and src/screen/screen_basic.o, while src/screen/screen_basic.c never gets compiled.
What's run by make (I replaced the options for gcc with the variable names to keep it short):
i686-elf-gcc $(CFLAGS) $(WARNINGS) -c src/kernel.c -o src/kernel.o
i686-elf-gcc $(CFLAGS) $(WARNINGS) -c src/kernel.c -o src/screen/screen_basic.o
I don't really know what you need to see what's going wrong. So, the source files (all of them) are at https://github.com/m44rtn/vireo-kernel.
It may be nice to know that this is a rewrite of the project. In the previous 'version' I manually added the file names to the makefile, which worked perfectly, but isn't nice when you have to add new files or when you're moving them around. That makefile is on the master branch (which ISN'T the default branch anymore).
The make version is the newest:
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
So, I expected this to work absolutely great. I thought it would just compile all files in the list. Unfortunately, it didn't. I don't really know what's going wrong here.
It compiles kernel.c to both kernel.o and screen_basic.o. I, of course, had hoped it would compile kernel.c to kernel.o and screen_basic.c to screen_basic.o.
Later on, the two files get linked. However, because they are the same, the linker throws errors because everything is defined twice, which isn't ideal.
I've tried to solve it by compiling every C file in one go, but this gave some issues with linking the Assembly files with the C files (sometimes making it non GRUB multibootable, which is necessarry to have, in my case).
I do not know what's wrong with the makefile for it to behave like this.
All the solutions from stack overflow I've tried:
Some solutions involve throwing all files in the root directory and just using:
%.o: %.c
(..)
However, this project will have a lot of files. This makes having everything in the same directory very annoying, very fast. I think this didn't work as well, but I don't know if that's true or just my brain lying to me. Sorry.
I've heard something about 'static rules':
$(OBJFILES): %.o: %.c
(..)
This didn't work, however I may have used it wrong. I don't know.
I like the makefile to stay the same as much as possible, because it's a very convenient one (it just detects all files automatically).
I really hope I've provided enough information, and that this question wasn't already asked. If it has been, I'm sorry in advance.
If you need more information, please ask! :)
--EDIT--
I'm quite new to make in this way, although I've used it for five years. I've always used it wrong. It is possible that my makefile is very ugly or bad. And I did use an example to write the makefile.
Consider the rule...
$(OBJFILES): $(SRCFILES)
$(CC) $(CCFLAGS) -c $< -o $#
Let's say, for the sake of argument, that we have...
SRCFILES := a.c b.c
OBJFILES := a.o b.o
If you expand the rule manually it becomes...
a.o b.o: a.c b.c
$(CC) $(CCFLAGS) -c $< -o $#
I think (correct me if I'm wrong) you are under the impression that this will be interpreted by make as a.o depends on a.c and, separately, b.o depends on b.c. That's not the case. What it actually states is that both of a.o and b.o depend on both of a.c and b.c.
So, when make tries to update the target a.o it sees the full prerequisite list of a.c and b.c and assigns the first of these, a.c, to the builtin variable $<. That's why you always see the first source file in $(SRCFILES) being compiled.
The best way to solve this probably depends on how you intend to structure your source file hierarchy and object files but you might want to take a look at using one or more vpath directives.
The pattern rule does not put all objects in root directory, consider
CFILES := path/to/a.c b.c
OBJFILES := $(foreach f,$(CFILES),$(f:%.c=%.o))
all: $(OBJFILES)
%.o: %.c
$(CC) $(CCFLAGS) -c $< -o $#
Here is what you get:
cc -c path/to/a.c -o path/to/a.o
cc -c b.c -o b.o
The following is not a recommendation, but kind of an exercise in makefile programming.
If you have $(SRCFILES) and want to compile them one at a time, you can use:
define compile
$1.o: $1.c
$(CC) $(CCFLAGS) -c $$< -o $$#
endef
$(foreach src,$(SRCFILES),$(eval $(call compile, $(src:%.c=%))))
If the correspondence of lists of sources and objects is not by name, but only by placement in list, you can destroy the CFILES list
define compile
src := $(CFILES)
CFILES := $(wordlist 2, $(words $(CFILES)), $(CFILES))
$1: $(src)
$(CC) $(CCFLAGS) -c $$< -o $$#
endef
$(foreach obj,$(OBJFILES),$(eval $(call compile, $(obj))))
Or you may use a helper list, to keep CFILES unchanged:
helperlist := $(CFILES)
define compile
src := $(firstword $(helperlist))
helperlist := $(wordlist 2, $(words $(helperlist)), $(helperlist))
$1: $(src)
$(CC) $(CCFLAGS) -c $$< -o $$#
endef
$(foreach obj,$(OBJFILES),$(eval $(call compile, $(obj))))
This is my current makefile
CFLAGS = -Iheaders/
CC = g++
PROGRAM_NAME = sportsmanager
rwildcard = $(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))
SOURCES = $(call rwildcard,sources/,*.cpp)
OFILES = $(call rwildcard,obj-tmp/,*.o)
OBJDIR = obj-tmp/
compileAndRun:
make -s compile && make -s $(PROGRAM_NAME)
./$(PROGRAM_NAME)
compile: $(SOURCES)
mkdir -p $(OBJDIR)
$(CC) $(CFLAGS) -c $(SOURCES) && mv *.o $(OBJDIR)
$(PROGRAM_NAME): $(OFILES)
$(CC) $(CFLAGS) $(OFILES) -o $(PROGRAM_NAME)
Whenever I run $ make, target compile is triggered which compiles all .cpp files in directory sources/ to .o files which are then moved to obj-tmp/. Then the target $(PROGRAM_NAME) is triggered, which links all the .o files and outputs the executable file.
The problem is that all files are compiled each time I run make. What ideally should happen if I run 'make' twice in succession is that make should know that the program is up to date the second time. If I modify only one file, only that file should be compiled.
Heads up: I know that there exists similar questions regarding this, but I've yet to see a solution which works in conjunction with the above makefile.
Any input is greatly appreciated.
The whole point of make is to compile only those files which have been modified since the last build. The problem in your makefile is that your compile recipe has the $(SOURCES) variable as a dependency. As in, all the source files.
I would use vpath to organize the project folder like so:
vpath %.cpp src
vpath %.h include
This will tell make to look for c++ files in ./src and header files in ./include. Then, you can simplify your recipe for individual files like this:
%.o: %.cpp
$(CC) $(CFLAGS) -c -o $# $<
Having done this, you can now define an $(OBJECTS) variable with a wildcard that matches .o files and continue from there. As an aside, moving your object files into a separate folder is considered bad practice and I agree; it really adds nothing substantial of value but complicates recipes.
Remember that object files represent a dependency for the $(PROGRAM) recipe. So naturally, make looks for the necessary object files to see if they need to be rebuilt. If they've been moved, one of two things happens. Either make will determine that they don't exist and will rebuild all the object files again from scratch, thereby invalidating the very reason we use make in the first place, or you'll have to define a folder where the object files will live, and every time you handle wildcards, searches, etc., literally anything that has to do with the object files, you'll have to take this added complexity into account.
I agree that having a ton of object files in the project folder can be a little annoying, but it definitely beats waiting forever for the project to compile. Just remember to add *.o to your .gitignore or whatever source control platform you use and they'll be nothing more than an eyesore, while make will be that much easier to use.
To answer your question on handling subdirectories in the source folder, the answer is a little more complicated.
Rather than using the specific vpath <pattern> <folder> directive as above, you could just outright use the VPATH variable like this:
VPATH = include src src/sub
This would handle the job, but the first method is usually preferred because when using VPATH, make searches every directory every time when looking for a file, rather than being location-constrained by file extension.
It is possible to use make to conveniently manage large projects though, and it involves calling make itself recursively, writing makefiles for each module in the build process. This process is obviously much more complicated, and I would strongly recommend considering whether the project genuinely necessitates this, as any potential gains in build-process modularization may not be recuperated due to the complexity involved in implementation.
I'd like to point you to this and this, both of which are phenomenal resources on makefiles.
Change the dependency of compile to be the object files.
Add a pattern rule for the object files.
compile: $(OFILES)
$(OBJDIR)/%.o: sources/%.cpp
mkdir -p $(OBJDIR)
$(CC) $(CFLAGS) -c $< -o $#
Ok, a lot of good input in this thread! Here's a follow up. I've now updated the script to the following:
CC = g++
CFLAGS = -Iheaders/
PROGRAM_NAME = sportsmanager
OFILES = $(patsubst %.cpp,%.o,$(wildcard sources/*.cpp))
vpath %.cpp sources
compileAndRun:
#make -s $(PROGRAM_NAME)
#./$(PROGRAM_NAME)
$(PROGRAM_NAME): $(OFILES)
$(CC) $(CFLAGS) -o $(PROGRAM_NAME) $(OFILES)
%.o: %.cpp
$(CC) $(CFLAGS) -c -o $# $<
clean:
rm -rf $(PROGRAM_NAME) $(OFILES)
Any suggestions for further improvements are very welcome!
I'm trying to set up a Makefile to handle two different targets from one set of sources, and I'm a little out of my element. The vast majority of it works fine, but my dependency structure is hosed and thus I'm forced to do a full recompile each time. A pared down sample is as follows:
first: OBJDIR = obj
second: OBJDIR = obj-2
SRCS = $(wildcard src/*.cc)
OBJECTS = $(patsubst %.cc,$(OBJDIR)/%.o,$(SRCS))
first: CFLAGS = -g -Wall -Wextra -std=c++11 -MMD
second: CFLAGS = -g -Wall -Wextra -std=c++11 -MMD -DCOMPILE_FLAG
$(OBJDIR)/%.o: %.cc
#mkdir -p $(OBJDIR)/src
clang++ -c $(CFLAGS) -o $(OBJDIR)$# $<
#DEPENDENCIES AREN'T WORKING PROPERLY
-include $(OBJECTS:.o=.d)
first: $(OBJECTS)
clang++ -o gen/first $(OBJECTS)
second: $(OBJECTS)
clang++ -o gen/second $(OBJECTS)
If I #echo $(OBJECTS:.o=.d) under my first: executable generation (or as it's used in the compilation step), it properly expands to /obj/src/filename.d. However, when it's in the include line, it shows up simply as /src/filename.d. Obviously the .d files don't exist in that location, so it doesn't find them and does a full recompile.
Makefiles are not something I'm heavily experienced with, so if there are better ways to do anything up above, I'm all ears. The key point, though, is being able to have two sets of object files and two sets of dependencies from the same sources.
--
To clarify on the eventual goals, what I have is a set of source files that are used to build two separate executables, with the differences handled via #ifdefs.
What I want to get out of the makefile structure is a single makefile with two targets. Each target generates its own .o/.d files, so that when a change is made to the source, I can run make first and make second to generate the two new executables without having to recompile everything from scratch. I've handled this previously by having two separate makefiles, but that just seems wrong.
You've missed a critical sentence in the GNU make manual related to target-specific variables:
As with automatic variables, these values are only available within the context of a target's recipe
This means that you can't use target-specific variables in either the target or prerequisite lists: any variables used there will ALWAYS have the global value and never the target-specific value. Similarly, include lines are parsed as the makefile is read in: there's no target context at all here either so the global value is used.
Further, any time you see ANY rule in a makefile that is creating a file which is not exactly $# but is instead some modification of it, that's a red flag and you know you have a problem: -o $(OBJDIR)$# is wrong.
In general, there's a lot wrong with this makefile and it's not entirely clear exactly what you're trying to do here. Maybe if you stepped back and described the goal you want to achieve we can give you some pointers.
I believe the answer is simply to rethink the way I was doing it. I've rewritten the makefile to be as follows (trimming out anything unrelated), after reading a lot of Makefile documentation and taking into consideration some comments from MadScientist:
CC = clang++
SRCS = $(wildcard src/*.cc)
OBJECTS = $(patsubst %.cc,$(OBJDIR)/%.o,$(SRCS))
CFLAGS = -g -Wall -Wextra -std=c++11 -MMD
.PHONY: all clean run
all: $(EXECUTABLE)
-include $(OBJECTS:.o=.d)
$(EXECUTABLE): $(OBJECTS)
#mkdir -p gen
$(CC) -o gen/$(EXECUTABLE) $(OBJECTS)
$(OBJDIR)/%.o: %.cc
#mkdir -p $(#D)
$(CC) -c $(CFLAGS) $(CMDFLAGS) -o $# $<
clean:
rm -rf obj obj-2 gen
run:
cd gen && ./$(EXECUTABLE)
From there, I made a couple aliases in my .bash_profile:
alias mfirst="make OBJDIR=obj EXECUTABLE=first"
alias msecond="make OBJDIR=obj-2 CMDFLAGS=-DCOMPILE_FLAG EXECUTABLE=second"
Because the variables are now set outside of any target specification, everything plays nicely. It keeps the object files and the dependencies separate, and the aliases still allow quick usage (including mfirst run, for example).
Feel free to point out any flaws here, but I'm fairly happy with the result.
I have a domain specific language compiler (homemade) which takes a file x.inflow and generates two files: x.c and x.h. The C file is compiled in the conventional manner and the generated header file has to be included into any file that calls the functions defined within it.
The header files therefore have to be generated before any C files that use them are compiled. My current Makefile, below, works fine except for the first build from clean where it can try and compile main.c before the header file that it includes has been created.
NAME = simplest
OBJ = $(patsubst %.c,%.o,$(wildcard *.c)) \
$(patsubst %.inflow,%.o,$(wildcard *.inflow))
CC = gcc
CFLAGS = -g -Wall
$(NAME): $(OBJ)
$(CC) $(CFLAGS) -o $# $^ $(CLIBS)
# Dependencies for existing .o files.
-include $(OBJ:.o=.d)
# Compile an inflow file into both a .c and .h file.
# Note that this rule has two targets.
%.c %.h: %.inflow
inflow $<
# Compile object files and generate dependency information.
%.o: %.c
$(CC) -MD -MP -c $(CFLAGS) -o $# $<
Obviously, I can fix this for specific cases by adding, for example (where simplest.h is a generated header):
main.o: simplest.h
But is there a general way to force one type of pattern rule (%.c %.h: %.inflow) to be run before any invokations of another (%.o: %.c)?
Well, you can force any target to be run before any other target with order-only prerequisites. So for example, you can write:
%.o : %.c | simplest.h
$(CC) -MD -MP -c $(CFLAGS) -o $# $<
which will ensure that no target that uses this pattern rule to build will be invoked before the target simplest.h is created. However, I don't think you can put patterns in an order-only prerequisite. To be honest, I've never tried it so it's possible that it works, I'm not sure.
If not, you could just list all the order-only prerequisites in the %.o pattern rule; this would ensure that all the inflow files are generated before any of the object files are built. That's probably OK.
It seems the problem is twofold:
Make doesn't know that it needs to generate simplest.h before compiling main.c.
You don't want to have to explicitly tell Make about the dependency (and remember to update it when it changes).
Rather than force Make to evaluate rules in a set order, you can solve your problem by letting Make create the dependencies for you. Check out this section of the Gnu Make manual: http://www.gnu.org/software/make/manual/make.html#Automatic-Prerequisites
When you run Make, it will scan your source files and gather their dependencies for you (and you won't have to explicitly list that main.o depends on simplest.h).
just for quick terminology:
#basic makefile rule
target: dependencies
recipe
The Problem: I want to generate the dependencies automatically.
For example, I am hoping to turn this:
#one of my targets
file.o: file.cpp 1.h 2.h 3.h 4.h 5.h 6.h 7.h 8.h another.h lots.h evenMore.h
$(COMPILE)
Into this:
#one of my targets
file.o: $(GENERATE)
$(COMPILE)
and I'm not too sure if it's possible..
What I do know:
I can use this compiler flag:
g++ -MM file.cpp
and it will return the proper target and dependency.
so from the example, it would return:
file.o: file.cpp 1.h 2.h 3.h 4.h 5.h 6.h 7.h 8.h another.h lots.h evenMore.h
however, 'make' does NOT allow me to explicitly write shell code in the target or dependency section of a rule :(
I know there is a 'make' function called shell
but I can't quite plug this in as dependency and do parsing magic because it relies on the macro $# which represents the target.. or at least I think that’s what the problem is
I've even tried just replacing the "file.cpp" dependency with this makefile function and that won't work either..
#it's suppose to turn the $# (file.o) into file.cpp
THE_CPP := $(addsuffix $(.cpp),$(basename $#))
#one of my targets
file.o: $(THE_CPP) 1.h 2.h 3.h 4.h 5.h 6.h 7.h 8.h another.h lots.h evenMore.h
$(COMPILE)
#this does not work
So all over google, there appear to be two solutions. both of which I don't fully grasp.
From GNU Make Manual
Some Site that says the GNU Make Manual one is out-of-date
So my ultimate question is: Is it possible to do it the way I want to do it,
and if not, can somebody break down the code from one of these sites and explain to me in detail how they work. I'll implement it one of these ways if I have to, but I'm weary to just paste a chunk of code into my makefile before understanding it
Newer versions of GCC have an -MP option which can be used with -MD. I simply added -MP and -MD to the CPPFLAGS variable for my project (I did not write a custom recipe for compiling C++) and added an "-include $(SRC:.cpp=.d)" line.
Using -MD and -MP gives a dependency file which includes both the dependencies (without having to use some weird sed) and dummy targets (so that deleting header files will not cause errors).
To manipulate the filenames when you already know what the dependencies should be, you can use a pattern rule:
file.o: %.o : %.cpp 1.h 2.h 3.h 4.h 5.h 6.h 7.h 8.h another.h lots.h evenMore.h
$(COMPILE)
And you can reuse the rule for other targets:
# Note these two rules without recipes:
file.o: 1.h 2.h 3.h 4.h 5.h 6.h 7.h 8.h another.h lots.h evenMore.h
anotherFile.o: 4.h 9.h yetAnother.h
file.o anotherFile.o: %.o : %.cpp
$(COMPILE)
But if you want Make to figure out the list of dependencies automatically, the best way (that I know of) is Advanced Auto-Dependency Generation. It looks like this:
%.o : %.cc
#g++ -MD -c -o $# $<
#cp $*.d $*.P; \
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $*.d >> $*.P; \
rm -f $*.d
-include *.P
Basically, when it builds file.o, it also builds file.d. Then it runs file.d through a bewildering sed command that turns the list of dependencies into a rule with no recipes. The last line is an instruction to include any such rules that exist. The logic here is subtle and ingenious: you don't actually need the dependencies the first time you build foo.o, because Make already knows that foo.o must be built, because it doesn't exist. The next time you run Make, it will use the dependency list it created last time. If you change one of the files so that there is actually a new dependency which is not in the list, Make will still rebuild foo.o because you changed a file which was a dependency. Try it, it really works!
Excellent answers but in my build I put the .obj files in a subdirectory based on build type (ie: debug vs. release). So for example, if I'm building debug, I put all the object files in a build/debug folder. It was a mind-numbing task to try to get the multiline sed command above to use the correct destination folder, but after some experimentation, I stumbled on a solution that works great for my build. Hopefully it'll help someone else as well.
Here's a snippet:
# List my sources
CPP_SOURCES := foo.cpp bar.cpp
# If I'm debugging, change my output location
ifeq (1,$(DEBUG))
OBJ_DIR:=./obj/debug
CXXFLAGS+= -g -DDEBUG -O0 -std=c++0x
else
CXXFLAGS+= -s -O2
OBJ_DIR:=./obj/release
endif
# destination path macro we'll use below
df = $(OBJ_DIR)/$(*F)
# create a list of auto dependencies
AUTODEPS:= $(patsubst %.cpp,$(OBJ_DIR)/%.d,$(CPP_SOURCES))
# include by auto dependencies
-include $(AUTODEPS)
.... other rules
# and last but not least my generic compiler rule
$(OBJ_DIR)/%.o: %.cpp
## Build the dependency file
#$(CXX) -MM -MP -MT $(df).o -MT $(df).d $(CXXFLAGS) $< > $(df).d
## Compile the object file
#echo " C++ : " $< " => " $#
#$(CXX) -c $< $(CXXFLAGS) -o $#
Now for the details:
The first execution of CXX in my generic build rule is the interesting one. Note that I'm not using any "sed" commands. Newer versions of gcc do everything I needed (I'm using gcc 4.7.2).
-MM builds the main dependency rule including project headers but not system headers. If I left it like this, my .obj file would NOT have the correct path. So I use the -MT option to specify the "real" path to my .obj destination. (using the "df" macro I created).
I also use a second -MT option to make sure the resulting dependency file (ie: .d file) has the correct path, and that it is included in the target list and therefor has the same dependencies as the source file.
Last but not least is the inclusion of the -MP option. This tell gcc to also make stubbed rules for each header solving the problem that occurs if I delete a header causing make to generate an error.
I suspect that since I'm using gcc for all the dependency generation instead of piping out to sed, my build is faster (although I've yet to prove that since my build is relatively small at this point). If you see ways I can improve upon this, I'm always open to suggestions. Enjoy
For the record, this is how I generate dependencies automatically now:
CPPFLAGS = -std=c++1y -MD -MP
SRC = $(wildcard *.cpp)
all: main
main: $(SRC:%.cpp=%.o)
g++ $(CPPFLAGS) -o $# $^
-include $(SRC:%.cpp=%.d)
The compiler flags -MD and -MP help do the trick.
First, you can have THE_CPP=$(patsubst %.o,%.cpp,$#)
Then you can run make -p to understand the builtin rules of make
A usual way of doing could be to generate the makefile dependencies into *.md files:
%.o: %.c
$(COMPILE.c) $(OUTPUT_OPTION) $< -MMD -MF $(patsubst %.c,%.md,$#)
and later in your Makefile including them with something like
-include $(wildcard *.md)
But you can also consider using other builders like omake and many many others
WOOO! I did manage to get the code in Beta's post to work on a small test project.
I should note, for anyone else who may come across this,
If you're using the bash shell(which I was), you will need to add an escape character in front of the pound sign to escape from making the rest of the expression a comment. (see 4th line of code)
%.o : %.cpp
g++ -c -MD -o $# $<
cp $*.d $*.P; \
sed -e 's/\#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $*.d >> $*.P; \
rm -f $*.d
-include *.P
Now I want to share information that I found in Managing Projects with GNU Make, 3rd Edition. because it's points out some important issues on this matter, and supplies code that I still don't fully grasp yet.
A method appears in the book that is similar to the method found on the Make manual page.
It looks like this:
include $(subst .c,.d,$(SOURCES))
%.d: %.c
$(CC) -M $(CPPFLAGS) $< > $#.$$$$; \
sed 's,\($*\).o[ :]*,\1.o $# : ,g' < $#.$$$$ > $#; \
rm -f $#.$$$$
This is what I believe is happening.
Right away, 'make' wants to include a ".d" file for every source file.
Because no .d files initially exist, the chunk of code is ran again and again in order to create all the missing .d files.
This means make will start over again and again until every .d file is created and included in the makefile.
Each ".d" file is what Beta said: a target with a set of dependencies and NO recipe.
If a header file is ever changed, those rules that are included in, will need the dependencies updated first. This is what throws me off a bit, how is it that the chunk of code is able to be called again? It is used to update .d files, so if a .h file changes how does it get called? Aside from this, I realize that the default rule is used to compile the object. Any clarifications/misconceptions to this explanation are appreciated.
Later in the book it points out problems with this method, and problems that I believe also exist in the Advanced Auto-Dependency Generation implementation.
Problem 1: It's inefficient. 'make' must restart every time it makes a .d file
Problem 2: make generates warning messages for all the missing .d files- Which is mostly just a nuisance and can be hidden by adding a "-" in front of the include statement.
Problem 3: If you delete a src file because it's no longer needed, 'make' will crash the next time you try to compile because some .d file has the missing src as a dependency, and because there is no rule to recreate that src, make will refuse to go any further.
They say a fix to these issues is Tromey's method, but the code looks very different from the code on the website. Perhaps it's just because they used some macros, made it a function call, and wrote it slightly different. I'm still looking into it, but wanted to share some discoveries I've made so far. Hopefully this opens up a little bit more discussion, and gets me closer to the bottom of all this.
A simple and elegant solution, inclusive of a detailed explanation of how it works, is available here.
DEPDIR := .deps
DEPFLAGS = -MT $# -MMD -MP -MF $(DEPDIR)/$*.d
%.o : %.cpp
%.o : %.cpp $(DEPDIR)/%.d | $(DEPDIR)
g++ -c $(DEPFLAGS) $(CFLAGS) $<
$(DEPDIR): ; #mkdir -p $#
DEPFILES := $(SRCS:%.c=$(DEPDIR)/%.d)
$(DEPFILES):
include $(wildcard $(DEPFILES))
I prefer to use $(shell ...) function with find. Here is a sample of one of my Makefiles:
SRCDIR = src
OBJDIR = obj
LIBDIR = lib
DOCDIR = doc
# Get Only the Internal Structure of Directories from SRCDIR
STRUCTURE := $(shell find $(SRCDIR) -type d)
#Filter-out hidden directories
STRUCTURE := $(filter-out $(shell find $(SRCDIR)/.* -type d),$(STRUCTURE))
# Get All Files From STRUCTURE
CODEFILES := $(addsuffix /*,$(STRUCTURE))
CODEFILES := $(wildcard $(CODEFILES))
## Filter Only Specific Files
SRCFILES := $(filter %.c,$(CODEFILES))
HDRFILES := $(filter %.h,$(CODEFILES))
OBJFILES := $(subst $(SRCDIR),$(OBJDIR),$(SRCFILES:%.c=%.o))
DOCFILES := $(addprefix $(DOCDIR)/, \
$(addsuffix .md, \
$(basename $(SRCFILES))))
# Filter Out Function main for Libraries
LIBDEPS := $(filter-out $(OBJDIR)/main.o,$(OBJFILES))
In this approach, I first get all the internal directory structure, with any depth. Then I get all files inside the Structure. At this time, I can use filter, filter-out, addsuffix, etc, to get exactly what I need at each time.
This example covers *.c files, but you can change it to *.cpp as well.
Building on the content of the Auto-Dependency Generation article referenced in comments on a previous post at I've created an annotated makefile project which includes a generic Makefile annotated with comments and implemented for a simple project with 3 .c files and 2 .h files. See full Makefile content below. Simple projects should be able to just customize the TODO section
# See http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
# for the template used to start this file
# -- TODO: customize the list below for your project ---
# List of source .c files used with the project
SRCS := main.c file1.c file2.c
# The aplication generated
APPNAME = depend-generation-test
# -- End of customization section ---
# Replace .c extension on SRCS to get objfiles using gnu make pattern rules and substitution references.
# See https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html#Pattern-Intro for pattern rules and
# https://www.gnu.org/software/make/manual/html_node/Substitution-Refs.html#Substitution-Refs for substitution references overview
OBJFILES := $(SRCS:%.c=%.o)
# Build the app you've specified in APPNAME for the "all" or "default" target
all : $(APPNAME)
default : $(APPNAME)
# Remove all build intermediates and output file
clean : ; #rm -rf $(APPNAME) *.o
# Build the application by running the link step with all objfile inputs
$(APPNAME) : $(OBJFILES)
$(CC) $(LDFLAGS) $^ -o $(APPNAME)
# Add all warnings/errors to cflags default. This is not required but is a best practice
CFLAGS += -Wall -Werror
# The below content is from http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
# with the following changes:
# 1) Added comments
# 2) Removed TARGET_ARCH from COMPILE.c since it's no longer listed in the [default rules](https://www.gnu.org/software/make/manual/html_node/Catalogue-of-Rules.html#Catalogue-of-Rules) and [isn't documented](https://lists.gnu.org/archive/html/help-make/2010-06/msg00005.html)
# Original content below is:
# Copyright © 1997-2019 Paul D. Smith Verbatim copying and distribution is permitted in any medium, provided this notice is preserved.
# The directory (hidden) where dependency files will be stored
DEPDIR := .deps
# Flags passed to gcc to automatically build dependencies when compiling
# See https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html for detail about variable names
# $# references the target file of the rule and will be "main.o" when compiling "main.c"
# $* references the stem of the rule, and will be "main" when target is "main.o"
DEPFLAGS = -MT $# -MMD -MP -MF $(DEPDIR)/$*.d
# Rules for compiling a C file, including DEPFLAGS along with Implicit GCC variables.
# See https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
# and see https://www.gnu.org/software/make/manual/html_node/Catalogue-of-Rules.html#Catalogue-of-Rules
# for the default c rule
COMPILE.c = $(CC) $(DEPFLAGS) $(CFLAGS) $(CPPFLAGS) -c
# Delete the built-in rules for building object files from .c files
%.o : %.c
# Define a rule to build object files based on .c or dependency files by making the associated dependency file
# a prerequisite of the target. Make the DEPDIR an order only prerequisite of the target, so it will be created when needed, meaning
# the targets won't get rebuilt when the timestamp on DEPDIR changes
# See https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html for order only prerequesites overview.
%.o : %.c $(DEPDIR)/%.d | $(DEPDIR)
$(COMPILE.c) $(OUTPUT_OPTION) $<
# Create the DEPDIR when it doesn't exist
$(DEPDIR): ; #mkdir -p $#
# Use pattern rules to build a list of DEPFILES
DEPFILES := $(SRCS:%.c=$(DEPDIR)/%.d)
# Mention each of the dependency files as a target, so make won't fail if the file doesn't exist
$(DEPFILES):
# Include all dependency files which exist, to include the relevant targets.
# See https://www.gnu.org/software/make/manual/html_node/Wildcard-Function.html for wildcard function documentation
include $(wildcard $(DEPFILES))