I wanted to build some C++ code. I've already had a magic Makefile that could calculate dependencies automaticaly, but I wanted to keep object files in separate directory. So the structure looks like:
Makefile
source/a/
source/b/
source/c/
obj/
I was able to run Makefile that converted sources into objects using obj/ prefix, but things got messy when I wanted to backport dependency calculation.
The problem is that is almost all Makefile examples this is done in one step, creating .d from .cpp (%.d: %.cpp). I spent some time trying to understand Makefile and fix it with trials and errors. Finally I've got a working script:
DIRS := a b c
SOURCES := $(foreach dir,$(DIRS),$(wildcard source/$(dir)/*.cpp))
OBJECTS := $(patsubst %.cpp,obj/%.o,$(SOURCES))
obj/%.d: %.cpp
#mkdir -p $(#D)
#$(CXX) -E $(CXXFLAGS) -MM -MP -MF $# -MQ $(#:.d=.o) -MQ $# $<
obj/%.o: %.cpp obj/%.d
#$(CXX) $(CXXFLAGS) -o $# -c $<
project: $(OBJECTS)
...
-include $(OBJECTS:.o=.d)
But I'm still not sure why do I need to have separate steps for cpp->d and cpp&d -> o (or maybe I understand - but then I have no idea why most examples don't need this).
Suppose the source files are foo.cpp and bar.cpp, and the headers are foo.h and bar.h, such that foo.cpp includes foo.h and bar.h, and bar.cpp includes bar.h.
You can have a simple makefile that does not involve dependency files at all:
project: $(OBJECTS)
...
%.o: %.cpp
...
Now if you modify foo.cpp, Make will rebuild foo.o (and project). If you modify bar.cpp, Make will rebuild bar.o (and project). If you modify a header, Make will do nothing, because Make doesn't know that the objects depend in any way upon the headers.
You can put in a rule for dependency files:
project: $(OBJECTS)
...
%.o: %.cpp
...
%.d:
...
-include *.d
Now Make will build and use the dependency files-- if you remember to tell it to. If you do that, Make will know that when you modify foo.h it must rebuild foo.o, and when you modify bar.h it must rebuild foo.o and bar.o.
Remembering to rebuild dependency files is a pain. It would be better to let Make do it. When must foo.d be rebuilt? Well, when foo.cpp is modified, certainly:
project: $(OBJECTS)
...
%.o: %.cpp
...
%.d: %.cpp
...
-include *.d
But foo.d must also be rebuilt when any of the other prerequisites of foo.o (i.e. foo.h or bar.h) has been modified. I haven't spelled out the commands in the rules (and I can't make much sense of the command in your %.d rule), but a simple rule will build a foo.d that looks like this:
foo.o: foo.cpp foo.h bar.h
It is possible to write a command that will produce a foo.d that looks like this:
foo.o: foo.cpp foo.h bar.h
foo.d: foo.cpp foo.h bar.h
There are more sophisticated methods, but this is enough for today.
As to why many example makefiles share a certain sub-optimal feature, it's because people copy each others' makefiles without rigorously examining or understanding every feature.
Related
My makefile always recompiles everything in directory if one header is changed. It's not a problem now but since I'm adding more to my program this is becoming and issue. I don't want to wait for a whole recompile if I add a new variable to a header of a separate class object.
Here is my makefile:
CXX = g++
CPPFLAGS = -I -lm -lsfml-graphics -lsfml-audio -lsfml-window -lsfml-system
OBJ = CR_Main.o CarRental.o CR_Button.o CR_LoginMenu.o CR_TextBox.o CR_UserCreation.o CR_CheckBox.o
DEPS = CarRental.hpp CR_Button.hpp CR_LoginMenu.hpp CR_TextBox.hpp CR_UserCreation.hpp CR_CheckBox.hpp
%.o: %.cpp $(DEPS)
$(CXX) -c -o $# $< $(CPPFLAGS)
CRC.exe: $(OBJ)
$(CXX) -o $# $^ $(CPPFLAGS)
.PHONY: clean
clean:
del *.o *.exe
Thanks in advance!
EDIT:
I was wondering why is it compiling everything in my directory if only 1 out 6 .hpp files are modified on one line? Is something wrong with my makefile or is that how it is?
why is it compiling everything in my directory if only 1 out 6 .hpp files are modified on one line? Is something wrong with my makefile or is that how it is?
"Wrong" might be too strong a word, but yes, the behavior you describe is a consequence of how your makefile is written.
This rule ...
%.o: %.cpp $(DEPS)
$(CXX) -c -o $# $< $(CPPFLAGS)
... says, roughly, that you can build .o files from corresponding .cpp files plus all the files named in variable DEPS. This implies that if that's the rule make selects for building a given .o file, and any of those prerequisites is newer than the target, then the target is out of date and needs to be rebuilt. You have named all your headers in DEPS and you have not provided any other rules for building .o files, so yes, if any of your headers changes, all of the .o files will be rebuilt.
The most simple-minded alternative would be to write a separate rule for each .o, naming the prerequisites of that file only. That is, the corresponding .cpp file and whichever headers it #includes, directly or indirectly.
But you can save yourself a little typing by instead removing the $(DEPS) part from your existing rule, and adding an additional rule for each .o that does not have a recipe but names all the header prerequisites for that file.
Or if, as it appears, you have consistent relationships between source file names and header names, you might do something like this:
CXX = g++
CPPFLAGS = -I.
LIBS = -lsfml-graphics -lsfml-audio -lsfml-window -lsfml-system -lm
MAIN_OBJ = CR_Main.o
MODULE_OBJS = CarRental.o CR_Button.o CR_LoginMenu.o CR_TextBox.o CR_UserCreation.o CR_CheckBox.o
$(MAIN_OBJ): CR_Main.cpp $(MODULE_OBJS:.o=.h)
$(CXX) $(CPPFLAGS) -c -o $# $<
%.o: %.cpp %.h
$(CXX) $(CPPFLAGS) -c -o $# $<
CRC.exe: $(MAIN_OBJ) $(MODULE_OBJS)
$(CXX) -o $# $^ $(LIBS)
# Extra dependencies (guesses for the sake of example):
CarRental.o CR_LoginMenu.o CR_UserCreation.o: CR_TextBox.h CR_CheckBox.h
# No recipe here
.PHONY: clean
clean:
del $(MAIN_OBJ) $(MODULE_OBJS) CRC.exe
Ultimately, though, what you would really like to do is generate all the header dependencies automatically. That makes the project much easier to maintain once you get it initially set up. You can find lots of information about that on the web, some of it in the GNU make manual.
I have a project with this toy structure:
src:
obj1.cpp
obj2.cpp
obj:
(empty folder)
which I want to compile into a library using make. What I want to write in my makefile is something along the lines of:
mylib.a: obj1.o obj2.o
ar $# $^
And this works, however it leaves the folder messy, as it results in this structure:
obj1.o
obj2.o
mylib.a
src:
obj1.cpp
obj2.cpp
obj:
(empty folder)
What I would prefer is that the result of the call would be something like this:
mylib.a
src:
obj1.cpp
obj2.cpp
obj:
obj1.o
obj2.o
where both obj sources are compiled into .o objects in obj. I tried doing this in my makefile:
vpath %.o obj
mylib.a: obj1.o obj2.o
but this has the same result as above. So, I figures, well sure, the implicit rule for obj1.o doesn't know where to output, right? OK, so I added this rule:
%.o: %.cpp
g++ $^ -o obj/$#
to ensure that I am compiling into obj, however, now, the make command fails, and I understand why it fails. It fails because mylib finds obj1.o, as in it finds the rule for it, it executes the rule, but the rule does not result in the existence of obj1.o, but rather in the existence of obj/obj1.o, and so when mylib is composed, its rule says ar obj1.o obj2.o and it fails.
So, my question is: is there a simple and elegant way to avoid this problem?
One solution I see is to always have some file, empty if need be, named obj1.o in obj, but that seems ugly to me. Another would be to add
"obj/" to every object going to mylib.a, but that means duplicating code and is even uglier. So, what would be the best way to solve this?
vpath and VPATH are not intended for targets, they are for sources. Using them to tell make where the object files should go is not a good idea. The easiest solution is probably a pattern rule:
obj/%.o: src/%.c
<your compilation rule>
Based on this you can elaborate a bit:
SRCDIR := src
OBJDIR := obj
SRCS := $(wildcard $(SRCDIR)/*.c)
OBJS := $(patsubst $(SRCDIR)/%.c,$(OBJDIR)/%.o,$(SRCS))
mylib.a: $(OBJS)
<your library building recipe>
$(OBJDIR)/%.o: $(SRCDIR)/%.c
<your compilation recipe>
clean:
rm -f $(OBJS) mylib.a
And of course, in your recipes, do not hesitate to use the automatic variables and the variables you declared yourself like OBJS, for instance. Examples:
mylib.a: $(OBJS)
$(AR) $# $^
$(OBJDIR)/%.o: $(SRCDIR)/%.c
$(CC) $(CFLAGS) -c $< -o $#
With the following makefile snippet:
main: main.o f1.o f2.o
$(CC) $(CFLAGS) -o program main.o f1.o f2.o
main.o: main.cc
$(CC) $(CFLAGS) -c main.cc
f1.o: f1.cc
$(CC) $(CFLAGS) -c f1.cc
f2.o: f2.cc
$(CC) $(CFLAGS) -c f2.cc
If I just change one file, only that file get recompiled when I rerun make, as desired. However, I'm having a hard time generalizing this without having to list each file individually. When I try something like:
$(OBJECTS): $(SOURCES)
$(CC) $(CFLAGS) -o $# -c $(patsubst %.o,%.cc,$#)
It builds each object file individually, but each object file depends on ALL my sources, so a change in any one file causing a full recompile. What's a good way to accomplish this?
Basically,
you do have to list each .o file's dependencies individually.
For example, each .o is likely to depend on a different bunch of headers.
Taking your f1.o, you need something like:
f1.o: include/i.h
f1.o: another.h dir/and-another.h
f1.o: f1.cc
$(CC) $(CFLAGS) -c f1.cc
(you can have as many dependency lines for a target as you like).
Maintaining that list is a nightmare.
Broken dependency lists render your Makefile worse than useless—you might as well use a batch file.
All is not lost!
If you are tidy,
you can get the compiler to do it automatically,
and pretty much for free.
Makes your Makefile tidier to boot.
Win Win.
As Ismail Badawi commented, pattern rules provide a nice solution. They are a type of implicit rule for make. Basically, implicit rules are automatic recipes based off the file extension. For example, make knows how to convert .c files into .o files implicitly. By default make will run the following recipe for .c files (see the rule catalogue):
$(CC) $(CPPFLAGS) $(CFLAGS) -c
You can modify the process either by setting the variables CC, CPPFLAGS, and CFLAGS, or by defining a pattern rule:
%.o: %.c
$(CC) $(CFLAGS) -c $<
The "$<" above matches the name of the first prerequisite, which will be the .c file in this example. See Beta's comment and automatic variables.
I am new to using make and have been learning the basics through this tutorial. Here is the final example makefile example from the tutorial:
IDIR =../include
CC=gcc
CFLAGS=-I$(IDIR)
ODIR=obj
LDIR =../lib
LIBS=-lm
_DEPS = hellomake.h
DEPS = $(patsubst %,$(IDIR)/%,$(_DEPS))
_OBJ = hellomake.o hellofunc.o
OBJ = $(patsubst %,$(ODIR)/%,$(_OBJ))
$(ODIR)/%.o: %.c $(DEPS)
$(CC) -c -o $# $< $(CFLAGS)
hellomake: $(OBJ)
gcc -o $# $^ $(CFLAGS) $(LIBS)
.PHONY: clean
clean:
rm -f $(ODIR)/*.o *~ core $(INCDIR)/*~
This should work fine assuming all .c files are only including hellomake.h, but it wouldn't work if each .c file was including different headers. Is it possible to write a makefile that knows what each .c file is including, so I don't have to go in and do it manually like:
foo.o: foo.c something.h
...
bar.o: bar.c somethingelse.h
...
because that seems like it would be a big waste of time.
Suppose foo.c has a line:
#include "something.h"
You'd like a line in the makefile:
foo.o: foo.c something.h
The gcc compiler can construct that line for you. The command
gcc -MMD -c -o foo.o foo.c
will build foo.o and foo.d which contains the line. (Try it.)
So just modify your makefile to produce these *.d files and include them, and you're done:
$(ODIR)/%.o: %.c $(DEPS)
$(CC) -MMD -c -o $# $< $(CFLAGS)
-include $(ODIR)/*.d
(Further refinements are possible, like specifying where the *.d files should go.)
Traditional makes are rather limited and force you to do all that basic stuff yourself. If you rightly expect a build tool to find dependencies and know what to link, try makepp. You may not need a makefile at all, or just a minimal one like
CFLAGS = -O3
myprog: # just a default target to know what to build
The linking part would require a little help on your side, in that it is based on source-header pairs. If myprog.cpp includes a.h and b.hpp it'll look if it can build a.o and/or b.o, and if so, will link them and recursively check what their sources include.
You will only need to learn more make syntax, if you have more complex requirements. But if you do, there is no limit. Besides doing almost all that GNU make can, there are lots more useful things, and you can even extend your makefiles with some Perl programming.
Yes, the "MMD" flag will help you to generate ".d" file (dependency) files. If you include at end of your Makefile( -include *.d ) and then if you make any change in .h file, the respective .o file, will rebuild.
Take this as reference:
https://github.com/saanvijay/makefile-skeleton
There's a minor limitation to #Beta's answer which can be fixed pretty easily.
Say you have a file main.c which includes header.h. You build this, and your main.d file looks like this:
main.o: main.c header.h
Then you delete header.h and its corresponding include in main.c. The program is valid and should compile fine, but make fails due to the above rule, which has a dependency on a now-nonexistent file.
To fix this, you need main.o to depend on main.d, and a rule to create main.d.
main.d: main.c
$(CC) -MM -o main.d main.c
include main.d
This splits the generation of the .d file into a separate step, and make is smart enough to know that since it includes main.d, it should be rebuilt before it is included. This would fix the above issue. More info in the docs.
One issue with this approach is that make will rebuild the .d file when it's not needed, e.g. when running make clean. In these cases, you can just disable the include like in this answer. I'm interested to know if there is a smarter way to do this.
The complete Makefile would look something like this:
main.d: main.c
$(CC) -MM -o main.d main.c
main.o: main.c
$(CC) $(CFLAGS) $(CPPFLAGS) -o main.o main.c
main: main.o:
$(CC) main.o -o main $(LDLIBS)
.PHONY: clean
clean:
rm main.o main.d
ifneq ($(MAKECMDGOALS),clean)
include main.d
endif
I have the following makefile that I use to build a program (a kernel, actually) that I'm working on. Its from scratch and I'm learning about the process, so its not perfect, but I think its powerful enough at this point for my level of experience writing makefiles.
AS = nasm
CC = gcc
LD = ld
TARGET = core
BUILD = build
SOURCES = source
INCLUDE = include
ASM = assembly
VPATH = $(SOURCES)
CFLAGS = -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions \
-nostdinc -fno-builtin -I $(INCLUDE)
ASFLAGS = -f elf
#CFILES = core.c consoleio.c system.c
CFILES = $(foreach dir,$(SOURCES),$(notdir $(wildcard $(dir)/*.c)))
SFILES = assembly/start.asm
SOBJS = $(SFILES:.asm=.o)
COBJS = $(CFILES:.c=.o)
OBJS = $(SOBJS) $(COBJS)
build : $(TARGET).img
$(TARGET).img : $(TARGET).elf
c:/python26/python.exe concat.py stage1 stage2 pad.bin core.elf floppy.img
$(TARGET).elf : $(OBJS)
$(LD) -T link.ld -o $# $^
$(SOBJS) : $(SFILES)
$(AS) $(ASFLAGS) $< -o $#
%.o: %.c
#echo Compiling $<...
$(CC) $(CFLAGS) -c -o $# $<
#Clean Script - Should clear out all .o files everywhere and all that.
clean:
-del *.img
-del *.o
-del assembly\*.o
-del core.elf
My main issue with this makefile is that when I modify a header file that one or more C files include, the C files aren't rebuilt. I can fix this quite easily by having all of my header files be dependencies for all of my C files, but that would effectively cause a complete rebuild of the project any time I changed/added a header file, which would not be very graceful.
What I want is for only the C files that include the header file I change to be rebuilt, and for the entire project to be linked again. I can do the linking by causing all header files to be dependencies of the target, but I cannot figure out how to make the C files be invalidated when their included header files are newer.
I've heard that GCC has some commands to make this possible (so the makefile can somehow figure out which files need to be rebuilt) but I can't for the life of me find an actual implementation example to look at. Can someone post a solution that will enable this behavior in a makefile?
EDIT: I should clarify, I'm familiar with the concept of putting the individual targets in and having each target.o require the header files. That requires me to be editing the makefile every time I include a header file somewhere, which is a bit of a pain. I'm looking for a solution that can derive the header file dependencies on its own, which I'm fairly certain I've seen in other projects.
As already pointed out elsewhere on this site, see this page:
Auto-Dependency Generation
In short, gcc can automatically create .d dependency files for you, which are mini makefile fragments containing the dependencies of the .c file you compiled.
Every time you change the .c file and compile it, the .d file will be updated.
Besides adding the -M flag to gcc, you'll need to include the .d files in the makefile (like Chris wrote above).
There are some more complicated issues in the page which are solved using sed, but you can ignore them and do a "make clean" to clear away the .d files whenever make complains about not being able to build a header file that no longer exists.
You could add a 'make depend' command as others have stated but why not get gcc to create dependencies and compile at the same time:
DEPS := $(COBJS:.o=.d)
-include $(DEPS)
%.o: %.c
$(CC) -c $(CFLAGS) -MM -MF $(patsubst %.o,%.d,$#) -o $# $<
The '-MF' parameter specifies a file to store the dependencies in.
The dash at the start of '-include' tells Make to continue when the .d file doesn't exist (e.g. on first compilation).
Note there seems to be a bug in gcc regarding the -o option. If you set the object filename to say obj/_file__c.o then the generated _file_.d will still contain _file_.o, not obj/_file_c.o.
This is equivalent to Chris Dodd's answer, but uses a different naming convention (and coincidentally doesn't require the sed magic. Copied from a later duplicate.
If you are using a GNU compiler, the compiler can assemble a list of dependencies for you. Makefile fragment:
depend: .depend
.depend: $(SOURCES)
rm -f ./.depend
$(CC) $(CFLAGS) -MM $^>>./.depend;
include .depend
There is also the tool makedepend, but I never liked it as much as gcc -MM
You'll have to make individual targets for each C file, and then list the header file as a dependency. You can still use your generic targets, and just place the .h dependencies afterwards, like so:
%.o: %.c
#echo Compiling $<...
$(CC) $(CFLAGS) -c -o $# $<
foo.c: bar.h
# And so on...
Basically, you need to dynamically create the makefile rules to rebuild the object files when the header files change. If you use gcc and gnumake, this is fairly easy; just put something like:
$(OBJDIR)/%.d: %.c
$(CC) -MM -MG $(CPPFLAGS) $< | sed -e 's,^\([^:]*\)\.o[ ]*:,$(#D)/\1.o $(#D)/\1.d:,' >$#
ifneq ($(MAKECMDGOALS),clean)
include $(SRCS:%.c=$(OBJDIR)/%.d)
endif
in your makefile.
Over and above what #mipadi said, you can also explore the use of the '-M' option to generate a record of the dependencies. You might even generate those into a separate file (perhaps 'depend.mk') which you then include in the makefile. Or you can find a 'make depend' rule which edits the makefile with the correct dependencies (Google terms: "do not remove this line" and depend).
Simpler solution: Just use the Makefile to have the .c to .o compilation rule be dependent on the header file(s) and whatever else is relevant in your project as a dependency.
E.g., in the Makefile somewhere:
DEPENDENCIES=mydefs.h yourdefs.h Makefile GameOfThrones.S07E01.mkv
::: (your other Makefile statements like rules
::: for constructing executables or libraries)
# Compile any .c to the corresponding .o file:
%.o: %.c $(DEPENDENCIES)
$(CC) $(CFLAGS) -c -o $# $<
None of the answers worked for me. E.g. Martin Fido's answer suggests gcc can create dependency file, but when I tried that it was generating empty (zero bytes) object files for me without any warnings or errors. It might be a gcc bug. I am on
$ gcc --version gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
So here's my complete Makefile that works for me; it's a combination of solutions + something that wasn't mentioned by anyone else (e.g. "suffix replacement rule" specified as .cc.o:):
CC = g++
CFLAGS = -Wall -g -std=c++0x
INCLUDES = -I./includes/
# LFLAGS = -L../lib
# LIBS = -lmylib -lm
# List of all source files
SRCS = main.cc cache.cc
# Object files defined from source files
OBJS = $(SRCS:.cc=.o)
# # define the executable file
MAIN = cache_test
#List of non-file based targets:
.PHONY: depend clean all
## .DEFAULT_GOAL := all
# List of dependencies defined from list of object files
DEPS := $(OBJS:.o=.d)
all: $(MAIN)
-include $(DEPS)
$(MAIN): $(OBJS)
$(CC) $(CFLAGS) $(INCLUDES) -o $(MAIN) $(OBJS) $(LFLAGS) $(LIBS)
#suffix replacement rule for building .o's from .cc's
#build dependency files first, second line actually compiles into .o
.cc.o:
$(CC) $(CFLAGS) $(INCLUDES) -c -MM -MF $(patsubst %.o,%.d,$#) $<
$(CC) $(CFLAGS) $(INCLUDES) -c -o $# $<
clean:
$(RM) *.o *~ $(MAIN) *.d
Notice I used .cc .. The above Makefile is easy to adjust for .c files.
Also notice importance of these two lines :
$(CC) $(CFLAGS) $(INCLUDES) -c -MM -MF $(patsubst %.o,%.d,$#) $<
$(CC) $(CFLAGS) $(INCLUDES) -c -o $# $<
so gcc is called once to build a dependency file first, and then actually compiles a .cc file. And so on for each source file.
I believe the mkdep command is what you want. It actually scans .c files for #include lines and creates a dependency tree for them. I believe Automake/Autoconf projects use this by default.