I am reading Managing Projects with GNU Make and in chapter 2, the author writes the following way to do automatic dependency generation
depend: count_words.c lexer.c counter.c
$(CC) -M $(CPPFLAGS) $^ > $#
include depend
However, I usually do
makedepend -I $(CC) -f $(MAKEFILE) count_words.c lexer.c counter.c
I am not sure which way is better or they are serving equivalent purpose and can be use interchangeably?
Thanks much!
I can't think of any difference between the two, although I suppose the first means you need one less tool for your build so it's marginally better.
That said, an even better method is to add -MMD -MP to your preprocessor flags when compiling, which generates the dependency files as a side effect of compilation itself.
SRCS := count_words.c lexer.c counter.c
OBJS := $(SRC:.c=.o)
DEPS := $(SRC:.c=.d)
CPPFLAGS := -MMD -MP
$(OBJS):
-include $(DEPS)
Related
Based on this famous link and adapted from this gist, and supposing that all of your source files are .cpp, you get easily a solution like this to have auto-dependence generation:
SRCS := $(wildcard *.cpp */*.cpp)
DEPDIR := .d
DEPS := $(SRCS:%.cpp=$(DEPDIR)/%.d)
# Temporary .Td dependence file... ?
DEPFLAGS = -MT $# -MD -MP -MF $(DEPDIR)/$*.Td
# Isn't it better a order-only prerequisite?
$(shell mkdir -p $(dir $(DEPS)) >/dev/null)
%.o: %.cpp # Removal of implicit rules... ? Twice?
%.o: %.cpp $(DEPDIR)/%.d # Dependency on .d... ?
g++ -o $# $(DEPFLAGS) $(CXXFLAGS) $(CPPFLAGS) -c $<
# Avoid some bugs?
mv -f $(DEPDIR)/$*.Td $(DEPDIR)/$*.d && touch $#
# If the `%.d` dependency is removed, is this still necessary?
.PRECIOUS = $(DEPDIR)/%.d
$(DEPDIR)/%.d: ;
-include $(DEPS)
To don't make this question too long discussing deeply why I think that all of the lines commented in the snippet above are innecesary, I will ask it in a short form; is there any difference in behaviour if I just change this snippet to:
SRCS := $(wildcard *.cpp */*.cpp)
DEPDIR := .d
DEPS := $(SRCS:%.cpp=$(DEPDIR)/%.d)
DEPFLAGS := -MT $# -MD -MP -MF $(DEPDIR)/$*.d
$(DEPDIR)/%:
mkdir -p $#
%.o: %.cpp | $(DEPDIR)/$(dir %)
g++ -o $(DEPFLAGS) $(CXXFLAGS) $(CPPFLAGS) -c $<
# touch $(DEPDIR)/$.d # ??
-include $(DEPS)
I have still two extra doubts; in the first link above, it says that
it’s been reported that some versions of GCC may leave the object file
older than the dependency file, which causes unnecessary rebuilds.
However, in the gist (second link above), the touch command is removed, and since the dependence file is no longer prerequisite of any rule, is there any reason to keep including it? Does this "gcc bug" still apply in any form?
The second doubt is related to the directory creation, moved to an order-only rule; do I need to make the "order-only" $(DEPDIR)/% rule .PRECIOUS? I don't know if make will attempt to remove the directory if the %.o recipe fails, because I don't know the concrete features of order-only rules.
You cannot remove the %.d prerequisite. The reason this is needed is explained in the page you linked.
I don't know what you mean by your comment Removal of implicit rules... ? Twice?. The removal of the implicit rule is needed, to ensure that our new implicit rule is used, and we only remove it once.
The temporary file .Td is used in case someone uses ^C or similar to kill their make job right in the middle of creating this file. By writing to a temporary file then only atomically replacing the real file once we know it's complete, we never have to worry about partial files which could cause the next invocation of make to generate an error or, worse, not recompile source files that should be recompiled.
Regarding the comment about object files older than the dependency file, first the gist you link to uses clang not GCC and perhaps Clang doesn't have this issue (or perhaps it does but people don't realize it). Second that update to the blog post is relatively recent as people have reported this issue to me with GCC. I've not seen it myself (I only use GCC) so maybe it's only an issue with some versions of GCC.
Regarding .PRECIOUS, make never (currently) recursively deletes directories so it won't delete any non-empty directory regardless of that setting.
I have a need to generate a few object files with different load addresses. I also have a make target for invoking objdump to show me the respective disassembly.
code0:
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE0_RELOC) $#.S -o $#.o
objcopy -O binary $#.o --only-section=.text $#.bin
code1:
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE1_RELOC) $#.S -o $#.o
objcopy -O binary $#.o --only-section=.text $#.bin
dump: dump_code0 dump_code1
dump_code0: code0.bin
objdump $< -D > $<.decomp
dump_code1: code1.bin
objdump $< -D > $<.decomp
This will fail because there is no target for code.bin, but target dump_code depends on code.bin being present. To prevent it from being confusing (i.e. using dependency code0 such that code0.bin is then produced) I thought it would be easier to create a %.bin target that will create the respective binary file associated with each object file (since there is also duplication of this code across all code* targets anyway. This target will then be a dependency for each code* target. However, this necessitates (or at least I think it does) the need for dynamically assigning dependencies. I tried something like this that did not work:
%.bin: $(basename $#)
objcopy -O binary $(basename $#).o --only-section=.text $#.bin
code0: code0.bin
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE0_RELOC) $#.S -o $#.o
With the logic being that a dependency such as code0.bin would cause this target to execute with the single effective dependency of code0 to first build the object file.
I may be misunderstanding how the % character may be used to wildcard make targets. The first usesof basename was evaluating as empty (the second worked), so the target had no dependencies. It does make sense that the dependency tree be evaluated once at invocation, but I was hoping for a more dynamic capability.
Can this be done in Make?
What you are looking for is probably Static Pattern Rules with which you could rework your first attempt:
code0:
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE0_RELOC) $#.S -o $#.o
objcopy -O binary $#.o --only-section=.text $#.bin
as:
code0.o code1.o: code%.o: code%.S
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE$*_RELOC) $< -o $#
code0.bin code1.bin: %.bin: %.o
objcopy -O binary $< --only-section=.text $#
See the principle? $* in the recipe of the first rule is substituted by the matching stem of the static pattern rule (the digit in our case). This works but pattern-specific variables would probably be easier to understand and maintain:
# Default code reloc option
CODE_RELOC := default_code_reloc_option
# code1-specific code reloc option, if different from default
code1.o: CODE_RELOC := code1_code_reloc_option
code0.o code1.o: %.o: %.S
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE_RELOC) $< -o $#
The last part of your first makefile could also use static pattern rules, plus phony targets. Something like:
.PHONY: dump dump_code0 dump_code1
dump: dump_code0 dump_code1
dump_code0 dump_code1: dump_%: %.bin.decomp
code0.bin.decomp code1.bin.decomp: %.bin.decomp: %.bin
objdump $< -D > $#
Finally, we can use some more make tricks (discovery of source files, pattern substitutions) to automate a bit further:
CODES := $(wildcard *.S)
OBJS := $(patsubst %.S,%.o,$(CODES))
BINS := $(patsubst %.S,%.bin,$(CODES))
DECS := $(patsubst %.S,%.bin.decomp,$(CODES))
DUMPS := $(patsubst %.S,dump_%,$(CODES))
# Default code reloc option
CODE_RELOC := default_code_reloc_option
# code1-specific code reloc option, if different from default
code1.o: CODE_RELOC := code1_code_reloc_option
.PHONY: dump $(DUMPS)
dump: $(DUMPS)
$(DUMPS): dump_%: %.bin.decomp
$(DECS): %.bin.decomp: %.bin
objdump $< -D > $#
$(BINS): %.bin: %.o
objcopy -O binary $< --only-section=.text $#
$(OBJS): %.o: %.S
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE_RELOC) $< -o $#
.PHONY: clean
clean:
rm -f $(OBJS) $(BINS) $(DECS)
The clean target is a bonus gift.
All this has a significant advantage over what you were trying to do with targets that were not real files. Here, files depend on other files, except the phony targets that are just a kind of shorthand for real target files. And this is 100% in line with make's philosophy: express inter-files dependences such that make compares files timestamps and decides what is up-to-date and what must be rebuilt. Significant savings when a lot of targets are already up-to-date.
A second advantage is that expressing all inter-files dependences is parallel safe. If you run make on this makefile with:
make -j 8
(assuming you have about 8 cores on your computer), you can expect a speed-up factor of 8. Not much if you have only two source files, but quite interesting if you have hundreds of them...
I'm trying to set up a Makefile to handle two different targets from one set of sources, and I'm a little out of my element. The vast majority of it works fine, but my dependency structure is hosed and thus I'm forced to do a full recompile each time. A pared down sample is as follows:
first: OBJDIR = obj
second: OBJDIR = obj-2
SRCS = $(wildcard src/*.cc)
OBJECTS = $(patsubst %.cc,$(OBJDIR)/%.o,$(SRCS))
first: CFLAGS = -g -Wall -Wextra -std=c++11 -MMD
second: CFLAGS = -g -Wall -Wextra -std=c++11 -MMD -DCOMPILE_FLAG
$(OBJDIR)/%.o: %.cc
#mkdir -p $(OBJDIR)/src
clang++ -c $(CFLAGS) -o $(OBJDIR)$# $<
#DEPENDENCIES AREN'T WORKING PROPERLY
-include $(OBJECTS:.o=.d)
first: $(OBJECTS)
clang++ -o gen/first $(OBJECTS)
second: $(OBJECTS)
clang++ -o gen/second $(OBJECTS)
If I #echo $(OBJECTS:.o=.d) under my first: executable generation (or as it's used in the compilation step), it properly expands to /obj/src/filename.d. However, when it's in the include line, it shows up simply as /src/filename.d. Obviously the .d files don't exist in that location, so it doesn't find them and does a full recompile.
Makefiles are not something I'm heavily experienced with, so if there are better ways to do anything up above, I'm all ears. The key point, though, is being able to have two sets of object files and two sets of dependencies from the same sources.
--
To clarify on the eventual goals, what I have is a set of source files that are used to build two separate executables, with the differences handled via #ifdefs.
What I want to get out of the makefile structure is a single makefile with two targets. Each target generates its own .o/.d files, so that when a change is made to the source, I can run make first and make second to generate the two new executables without having to recompile everything from scratch. I've handled this previously by having two separate makefiles, but that just seems wrong.
You've missed a critical sentence in the GNU make manual related to target-specific variables:
As with automatic variables, these values are only available within the context of a target's recipe
This means that you can't use target-specific variables in either the target or prerequisite lists: any variables used there will ALWAYS have the global value and never the target-specific value. Similarly, include lines are parsed as the makefile is read in: there's no target context at all here either so the global value is used.
Further, any time you see ANY rule in a makefile that is creating a file which is not exactly $# but is instead some modification of it, that's a red flag and you know you have a problem: -o $(OBJDIR)$# is wrong.
In general, there's a lot wrong with this makefile and it's not entirely clear exactly what you're trying to do here. Maybe if you stepped back and described the goal you want to achieve we can give you some pointers.
I believe the answer is simply to rethink the way I was doing it. I've rewritten the makefile to be as follows (trimming out anything unrelated), after reading a lot of Makefile documentation and taking into consideration some comments from MadScientist:
CC = clang++
SRCS = $(wildcard src/*.cc)
OBJECTS = $(patsubst %.cc,$(OBJDIR)/%.o,$(SRCS))
CFLAGS = -g -Wall -Wextra -std=c++11 -MMD
.PHONY: all clean run
all: $(EXECUTABLE)
-include $(OBJECTS:.o=.d)
$(EXECUTABLE): $(OBJECTS)
#mkdir -p gen
$(CC) -o gen/$(EXECUTABLE) $(OBJECTS)
$(OBJDIR)/%.o: %.cc
#mkdir -p $(#D)
$(CC) -c $(CFLAGS) $(CMDFLAGS) -o $# $<
clean:
rm -rf obj obj-2 gen
run:
cd gen && ./$(EXECUTABLE)
From there, I made a couple aliases in my .bash_profile:
alias mfirst="make OBJDIR=obj EXECUTABLE=first"
alias msecond="make OBJDIR=obj-2 CMDFLAGS=-DCOMPILE_FLAG EXECUTABLE=second"
Because the variables are now set outside of any target specification, everything plays nicely. It keeps the object files and the dependencies separate, and the aliases still allow quick usage (including mfirst run, for example).
Feel free to point out any flaws here, but I'm fairly happy with the result.
I want to use the include directive only for a specific target. I do not want to run the other makefiles when the target is not needed because it means the makefiles are generated needlessly.
So is there a way to conditionally use the include directive, which is conditional on a target? Or somehow to make the include directive a prerequisite of a target.
Here's what I have so far:
# Flags
INCDIR = $(CURDIR)/include
CFLAGS = -Wall -Wno-overflow -Wno-uninitialized -pedantic -std=c99 -I$(INCDIR) -O3
LFLAGS = -flat_namespace -dynamiclib -undefined dynamic_lookup
# Directory names
# Set vpath search paths
vpath %.h include
vpath %.c src
vpath %.o build
vpath %.d build
# Get files for the core library
CORE_FILES = $(wildcard src/*.c)
CORE_OBJS = $(patsubst src/%.c, build/%.o, $(CORE_FILES))
CORE_DEPS = $(CORE_OBJS:.o=.d)
# Core library target linking
core : $(CORE_OBJS) | bin
$(CC) $(LFLAGS) -o bin/libcbitcoin.2.0.dylib $(CORE_OBJS)
# Include header prerequisites (How to do only for "core" target?)
include $(CORE_DEPS)
# Makefiles for header dependencies.
$(CORE_DEPS): build/%.d: src/%.c | build
rm -f $#; \
$(CC) -I$(INCDIR) -MM $< -MT '$(#:.d=.o) $#' > $#
# Objects depend on directory
$(CORE_OBS) : | build
# Create build directory
build:
mkdir build
# Create bin directory
bin:
mkdir bin
# Core Compilation
$(CORE_OBJS): build/%.o: src/%.c
$(CC) -c $(CFLAGS) $< -o $#
# Depencies require include/CBDependencies.h as a prerequisite
build/CBOpenSSLCrypto.o: include/CBDependencies.h
# Crypto library target linking
crypto : build/CBOpenSSLCrypto.o -lcrypto -lssl | bin
$(CC) $(LFLAGS) -o bin/libcbitcoin-crypto.2.0.dylib build/CBOpenSSLCrypto.o -lcrypto -lssl
# Crypto library compile
build/CBOpenSSLCrypto.o: dependencies/crypto/CBOpenSSLCrypto.c
$(CC) -c $(CFLAGS) $< -o $#
#Clean
clean:
rm -f $(CORE_OBJS) $(CORE_DEPS) build/CBOpenSSLCrypto.o
As you should be able to tell I do not need to include the ".d" files for "crypto" but I do for "core" (default goal).
Thank you for any help.
Make is not a procedural language, so treating it as one goes against the grain; your makefiles will be difficult to scale, and it can lead to subtle bugs.
There's a better way by Tom Tromey that's clean, efficient and scalable. The trick is to realize that you can build the dependency file in the same step as the object file. The dependencies simply tell Make when the object is due to be rebuilt; you don't need them when you first build the object, because Make knows that the object must be built. And if the dependencies change, that can only be because something in the source or the old dependencies has changed, so again Make knows that the object must be rebuilt. (This is not obvious, so it may take a little cogitation.)
$(CORE_OBJS): build/%.o: src/%.c
$(CC) -c $(CFLAGS) $< -o $#
$(CC) -MM -MF build/$*.d $<
-include build/*.d
There's one more hitch: if you alter the code so as to remove a dependency -- and also remove that file -- you won't be able to rebuild, because the old dependency list will still demand a file which can no longer be found. The sophisticated solution is to process the dependency file so as to make each prerequisite (e.g. header) a target in its own right, with no commands, so that it can be assumed to be rebuilt when needed:
$(CORE_OBJS): build/%.o: src/%.c
$(CC) -c $(CFLAGS) $< -o $#
$(CC) -MM -MF build/$*.d $<
#cp build/$*.d build/$*.P
#sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < build/$*.P >> build/$*.d;
#rm build/$*.P
A cruder method, but almost as foolproof, is to put in catch-all rules for headers and sources:
$(CORE_OBJS): build/%.o: src/%.c
$(CC) -c $(CFLAGS) $< -o $#
$(CC) -MM -MF build/$*.d $<
%.cc %.h:
EDIT:
To break down the new commands:
The -MM option tells gcc to produce a make rule for the object file, instead of preprocessing or compiling. The default is to send the rule to wherever it would send preprocessed output, which will usually be stdout.
The -MF option, used with -MM, specifies the output file. So -MM -MF build/$*.d will put the rule where we want it.
So the following two commands are (almost always) equivalent:
$(CC) -MM -MF build/$*.d $<
$(CC) -MM $< > build/$*.d
(I've left out the -I$(...) and the possibility of using the -MMD option, because both get a little complicated and are not really the point of the question.)
You can use MAKECMDGOALS.
ifeq (core,$(MAKECMDGOALS))
include $(CORE_DEPS)
endif
You could of course, use ifneq (,$(findstring core,$(MAKECMDGOALS))) if there was a possibility of more than one target.
Note: this is a 'quick and dirty' solution -- I agree with Beta that you shouldn't make this a common practice (this could get messy if you did it in a lot of makefiles...).
John
I can't help breaking the guidelines for what is a good answer.
My answer to the original question is in my opinion, no you cannot include rules that are dependant on the target -all rules are processed before targets are considered. This is a limitation of make (I guess). Then again, good point, there is MAKECMDGOALS, but is this not just a hack in the make utility itself?
The answer from Beta is reasonable and orthodox enough, but you can't call it clean even if it is the best that can be done. It won't work if make has not processed the particular target before and the appropriate build/*.d dependency file is not sitting there.
The Makefiles that I have dealt with, for the most part, are complex and hide a lot of relationships. I have never written one myself, and was wondering if anybody had some tips on writing a Makefile that is easy to read and reusable?
I usually use something like this, in this example the source files are main.c file2.c file3.c file4.c, to add more you simply add to the OBJECTS var.
They all depend on Makefile, so for a full recompile a simple touch Makefile would suffice.
PROGNAME = hi2u
LIBS = -ljpeg -ldirectfb -pthread
INCLUDES = -I/usr/local/include/directfb
LDFLAGS = -Llibs/
OBJECTS = main.o file2.o \
file3.o file4.o
CFLAGS = -W -Wall -O2 -ggdb
all: $(PROGNAME)
$(PROGNAME): $(OBJECTS)
gcc -o $(PROGNAME) $(OBJECTS) $(LIBS) $(INCLUDES) $(LDFLAGS)
$(OBJECTS): Makefile
.c.o:
gcc -c $(CFLAGS) $(INCLUDES) -o $# $<
clean:
rm *.o $(PROGNAME)
In all honesty, the complexity of a makefile relies on the complexity of the program. If you have a lot of folders and files and different compiling processes, you're makefile is probably going to be a little long and complicated. If you have a helloworld program, there's no reason for it to be longer than a few lines.
Here's some tips on makefiles : http://mrbook.org/tutorials/make/
Here's a very reusable makefile that's not too complicated:
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
A tutorial that I've found helpful for understanding Makefiles is http://www.jfranken.de/homepages/johannes/vortraege/make_inhalt.en.html
Another tip is to make generous use of regular expressions for source files and dependencies
For me, the read that got me thinking about these issues, is the classic "Recursive Make Considered Harmful".
When I get the chance to create makefiles from scratch, I try to use implicit rules as much as possible, and also define rules in a separate file, which I can include from the "real" makefile.
The challenges with using make can be divided in two major groups:
issues inherent with make itself, its rich semantics and syntax and somewhat archaic appearance
issues which are not makes "fault", but come from when make is used to call another make process. Suddenly we have another task at hand - communicating between two or more make processes. It is very easy to get lost with environment variables or other ways to pass information. Platform differences which make itself is designed to hide, may become visible.