My simple little makefile is exhibiting behavior which I'm not able to understand. If I touch any source file except Dictionary.cpp then no targets are built, and if I touch Dictionary.cpp then it compiles but doesn't link. Source files are in src/ object (.o) and dependencies (.d) are in obj/ and binary goes into bin/
If I rm obj/* then everything builds OK but the timestamps don't seem to be being picked up. Can anyone tell me where I'm going wrong?
The .d files seem to be being created correctly, here's Dictionary.d:
obj/Dictionary.o: src/Dictionary.cpp src/pch.h src/Types.h src/Util.h \
src/Refcount.h src/Dictionary.h
src/Dictionary.cpp:
src/pch.h:
src/Types.h:
src/Util.h:
src/Refcount.h:
src/Dictionary.h:
Which looks correct to me. Here's the makefile:
sources = Dictionary.cpp \
Util.cpp \
Tile.cpp \
Board.cpp \
Vec2.cpp \
Letter.cpp \
Random.cpp \
Server.cpp \
main.cpp
objects = $(patsubst %.cpp,obj/%.o,$(sources))
depends = $(patsubst %.cpp,obj/%.d,$(sources))
CXX = g++
CPPFLAGS = -Isrc -std=c++0x
CXXFLAGS = -c
-include $(depends)
bin/dictionary: $(objects)
#echo Link...
$(CXX) $(CPPFLAGS) $(objects) -o bin/dictionary -lrt
obj/%.o: src/%.cpp
#echo [$*]
#$(CXX) $(CPPFLAGS) $(CXXFLAGS) src/$*.cpp -o obj/$*.o
#$(CXX) $(CPPFLAGS) -MM src/$*.cpp -MF obj/$*.d
#mv -f obj/$*.d obj/$*.d.tmp
#sed -e 's|.*:|obj/$*.o:|' < obj/$*.d.tmp > obj/$*.d
#sed -e 's/.*://' -e 's/\\$$//' < obj/$*.d.tmp | fmt -1 | sed -e 's/^ *//' -e ' s/$$/:/' >> obj/$*.d
#rm -f obj/$*.d.tmp
You have to move the include to the end, or put the bin/dictionary rule before it, or add an all: bin/dictionary rule before the include, or something.
Or, resign yourself to always running make bin/dictionary, which will work as well.
Remember make, by default, tries to build the first target in the makefile. Because you have the include line before any other target, the first target defined by an included file will be considered the default goal, and that happens to be obj/Dictionary.o.
Related
My currently working makefile uses gcc to compile and link in one step. It is 600 lines long so I have cut it down to just show you the 'compile and link' and hex stages (very cut down code here!)
$(PROGRAM_ELF): \
$(BSP_DIR)/install/lib/$(CONFIGURATION)/libmetal.a \
$(BSP_DIR)/install/lib/$(CONFIGURATION)/libmetal-gloss.a \
$(BSP_DIR)/metal.$(LINK_TARGET).lds
mkdir -p $(dir $#)
$(MAKE) -C $(SRC_DIR) $(basename $(notdir $#)) \
PORT_DIR=$(PORT_DIR) \
AR=$(RISCV_AR) \
CC=$(RISCV_GCC) \
CXX=$(RISCV_GXX) \
ASFLAGS="$(RISCV_ASFLAGS)" \
CCASFLAGS="$(RISCV_CCASFLAGS)" \
CFLAGS="$(RISCV_CFLAGS)" \
CXXFLAGS="$(RISCV_CXXFLAGS)" \
XCFLAGS="$(RISCV_XCFLAGS)" \
LDFLAGS="$(RISCV_LDFLAGS)" \
LDLIBS="$(RISCV_LDLIBS)" \
PROJ_SRC="$(PROJ_SRC)"
$(PROGRAM_HEX): \
$(PROGRAM_ELF)
$(RISCV_OBJCOPY) -O ihex $(PROGRAM_ELF) $#
mv $(PROGRAM_HEX) $(PROGRAM_TMP)
$(RISCV_OBJCOPY) -O verilog $(PROGRAM_ELF) $#
cp $(PROGRAM_HEX) $(PROGRAM_MEM)
mv $(PROGRAM_TMP) $(PROGRAM_HEX)
However, I need the 'compile and link stage' to be in 2 steps now as I'll be using a different compiler which has separate compile and link exes. How would I do this ? So the above would need to be split into 2. Examples online are a bit vague.
at its simplest, you use the gcc (or g++) -c option to compile the source without linking. This will generate an object file - which you can use in the linker state. Here is a simple example:
SOURCE = test.cpp
OBJECT = test.o
OUTPUT = run
all:
#g++ $(SOURCE) -c -o $(OBJECT) <------ Compile test.cpp, produces test.o
#g++ $(OBJECT) -o $(OUTPUT) <------ Links test.o into executable `run`
clean:
#$(RM) -rf $(OBJECT) $(OUTPUT)
That's it...
I did this which seemed to work:
#list of all files
PROJ_SRC = $(SRC_DIR)/plsi2c_riscv.c \
$(SRC_DIR)/SimSpi.c \
etc
#assume BUILD_FOLDER defined already
OBJ_FILES = $(addprefix $(BUILD_FOLDER)/,$(PROJ_SRC:.c=.o))
$(BUILD_FOLDER)/%.o: %.c
#Create the folder structure for the output file
#mkdir -p $(dir $#)
$(MY_CC) $(CCFLAGS) $< -o $#
$(BUILD_FOLDER)/example: $(OBJ_FILES)
mkdir -p $(dir $#)
#echo Linking $(notdir $#)
$(MY_LINK) $(LFLAGS) $^ $(LDLIBS) -o $#
I have a simple C program with two source files, and the Makefile generates dependencies automatically, as documented in section 4.14 of the GNU Make Manual:
all: main
%.d: %.c
#set -e; rm -f $#; \
$(CC) -MM $(CPPFLAGS) $< > $#.$$$$; \
sed 's,\($*\)\.o[ :]*,\1.o $# : ,g' < $#.$$$$ > $#; \
rm -f $#.$$$$
include main.d
include hello.d
main: main.o hello.o
This creates files like main.d that look like so:
main.o main.d : main.c hello.h
The problem comes if I make a change to the source code (and Makefile) to remove hello.c and hello.h. Upon the next incremental rebuild, make fails:
make: *** No rule to make target `hello.h', needed by `main.d'. Stop.
The main.d file is out of date, but make cannot rebuild it since (according to the stale main.d) it still depends on the no-longer-existent hello.h.
In this situation, a clean build will succeed. How can I get the incremental build to succeed as well?
I modified the %.d recipe so that the dependencies are only considered if the file still exists. The new Makefile rule:
%.d: %.c
#set -e; rm -f $#; \
$(CC) -MM $(CPPFLAGS) $< > $#.$$$$; \
sed --in-place 's,\($*\)\.o[ :]*,\1.o $# : ,g' $#.$$$$ ; \
sed 's^: \(.*\)^: $$(foreach t,\1,$$(if $$(wildcard $$t),$$t,))^g' < $#.$$$$ > $#; \
rm -f $#.$$$$
The second sed modifies the main.d to look like:
main.o main.d : $(foreach t,main.c hello.h,$(if $(wildcard $t),$t,))
Thus if any of the dependent files should disappear, make will not complain.
One drawback to this approach is that an incremental build may succeed in situations where a clean build would fail. For example, if hello.h was deleted without properly removing the #include from main.c, then the incremental build would succeed (since it won't try to rebuild main.o) while a full build would fail.
Are there any other drawbacks to this approach? For instance, are there any cases where an incremental build would be incomplete?
I have the following directory structure:
root-----Makefile
|-----src #all source files here.
|-----obj #all object files here.
|-----bin #the final target.
The contents of Makefile is given below:
TARGET = exec
CC = gcc
CFLAGS = -g -I.
LINKER = gcc -o
LFLAGS = -I. -lm -lpthread
BINDIR = bin
OBJDIR = obj
SRCDIR = src
INTERFACE = interface
STD = -std=c99
PROGRAMSOURCES := $(wildcard $(SRCDIR)/*.c)
PROGRAMINTERFACE:= $(wildcard $(INTERFACE)/*.h)
OBJECTS := $(PROGRAMSOURCES:$(SRCDIR)/%.c=$(OBJDIR)/%.o)
$(BINDIR)/$(TARGET) : $(OBJECTS)
$(LINKER) $# $(LFLAGS) $(OBJECTS) $(STD)
#pull the dependencies to the .o files
-include $(OBJECTS:.o=.d)
$(OBJECTS) : $(OBJDIR)/%.o :$(SRCDIR)/%.c
$(CC) $(CFLAGS) -c $< -o $# $(STD)
$(CC) $(CFLAGS) -MM $< > $*.d
#mv -f $*.d $*.d.tmp
#sed -e 's|.*:|$(OBJDIR)/$*.o:|' < $*.d.tmp > $*.d
#sed -e 's/.*://' -e 's/\\$$//' < $*.d.tmp | fmt -1 | \
sed -e 's/^ *//' -e 's/$$/:/' >> $*.d`
#rm -f $*.d.tmp`
.PHONY : run
run :`
./$(BINDIR)/$(TARGET) ${TYPE} ${INP_FILE}
I have used the tutorial here have been modified to suit the corresponding directory structure. But something has got wrong in the modification and I cannot understand what. The dependency list generated in the .d files is not taken into account i.e. if I change a .h the rules are not compiling.
You appear to be producing the dependency files in root/, but looking for them in obj/.
Try changing this:
-include $(OBJECTS:.o=.d)
to this:
-include *.d
There are other improvements you can make, once the makefile is working.
EDIT: further improvements
1) The choice of where to put the dependency files is mainly arbitrary, but if we put them in obj/, we can simplify the $(OBJECTS) rule quite a lot:
-include $(OBJDIR)/*.d
$(OBJECTS): $(OBJDIR)/%.o :$(SRCDIR)/%.c
$(CC) $(CFLAGS) -MMD -MP -c $< -o $# $(STD)
2) PROGRAMINTERFACE and INTERFACE aren't used, so we can remove them.
3) Putting -o in LINKER like that is dangerous. And don't forget the automatic variable $^:
LINKER = gcc
$(BINDIR)/$(TARGET) : $(OBJECTS)
$(LINKER) $# $(LFLAGS) $^ $(STD)
4) It would be wise to give run a prerequisite:
run: $(BINDIR)/$(TARGET)
./$< ${TYPE} ${INP_FILE}
So I followed the Advanced Auto-Dependency Generation paper --
Makefile:
SRCS := main.c foo.c
main: main.o foo.o
%.o: %.c
$(CC) -MMD -MG -MT '$# $*.d' -c $< -o $#
cp $*.d $*.tmp
sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$$;;' \
-e '/^$$/d' -e 's;$$; :;' < $*.tmp >> $*.d
rm $*.tmp
clean::
-rm *.o *.d main
-include $(SRCS:.c=.d)
main.c:
#include "foo.h"
int main(int argc, char** argv) {
foo() ;
return 0 ;
}
foo.h:
#ifndef __FOO_H__
#define __FOO_H__
void foo() ;
#endif
-- and it works like a charm.
But when foo.h becomes a generated file --
Makefile:
...
HDRS := foo.h
$(HDRS):
mk_header.sh $*
clean::
-rm $(HDRS)
...
mk_header.sh:
#!/bin/bash
UP=$(tr "[:lower:]" "[:upper:]" <<< $1)
cat <<EOF > $1.h
#ifndef __${UP}_H__
#define __${UP}_H__
void $1() ;
#endif
EOF
The 1st time I run make, main.d is not yet generated, and thus foo.h is not considered a prerequisite, and thus isn't been generated:
$ ls
foo.c main.c Makefile mk_header.sh*
$ make
cc -MMD -MG -MT 'main.o main.d' -c main.c -o main.o
cp main.d main.tmp
sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$;;' \
-e '/^$/d' -e 's;$; :;' < main.tmp >> main.d
rm main.tmp
cc -MMD -MG -MT 'foo.o foo.d' -c foo.c -o foo.o
cp foo.d foo.tmp
sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$;;' \
-e '/^$/d' -e 's;$; :;' < foo.tmp >> foo.d
rm foo.tmp
cc main.o foo.o -o main
$ ls
foo.c foo.d foo.o
main* main.c main.d main.o
Makefile mk_header.sh*
Only in the 2nd invocation of make, the foo.h is generated, and as a result another build cascades.
$ make
./mk_header.sh foo
cc -MMD -MG -MT 'main.o main.d' -c main.c -o main.o
cp main.d main.tmp
sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$;;' \
-e '/^$/d' -e 's;$; :;' < main.tmp >> main.d
rm main.tmp
cc main.o foo.o -o main
$ ls
foo.c foo.d foo.h foo.o
main* main.c main.d main.o
Makefile mk_header.sh*
And only after that make realizes that:
$ make
make: `main' is up to date.
So my question is: Is there a way to extend the recipe suggested by the paper above, to allow for generated header files, without the elimination of the performance gain realized by not having to re-evaluate the entire make tree when including the *.d fragments?
The problem is that the *.d Makefile-fragments generation must be performed after all the header generation is complete. Putting it this way, one can use the make dependencies to force the right order:
SRCS := main.c foo.c
HDRS := foo.h
main: main.o foo.o
%.o: %.c | generated_headers
$(CC) -MMD -MG -MT '$# $*.d' -c $< -o $#
cp $*.d $*.tmp
sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$$;;' \
-e '/^$$/d' -e 's;$$; :;' < $*.tmp >> $*.d
rm $*.tmp
-include $(SRCS:.c=.d)
$(HDRS):
mk_header.sh $*
generated_headers: $(HDRS)
clean:
-rm $(HDRS) *.o *.d main
.PHONY: clean generated_headers
Notes:
I use an order-only dependency.
This solution is fairly scalable: Each generate-header rule, needs only to be a prerequisite of the generated_headers .PHONY target. Assuming that the header generation rule is written properly, once it has been generated correctly, satisfying the generated_headers target should be a no-op.
One can't compile a single object, even if that object does not require any generated headers, without generating all the generated headers of the project first. While this is technically sound, your developers will complain.
So you should think about having a FAST_AND_LOOSE flag, that will turn this feature off:
%.o: %.c | $(if $(FAST_AND_LOOSE),,generated_headers)
...
Thus a developer may issue:
make FAST_AND_LOOSE=1 main.o
The makefile in the original question doesn't work for me with gcc 4.8.2:
cc -MMD -MG -MT main.d -c main.c -o main.o
cc1: error: -MG may only be used with -M or -MM
I guess gcc changed the behaviour of -MG at some point in the last 4 years.
It seems that if you want to support generated header files, there is no longer
any way to generate the ".d" file and the ".o" file at the same time, without
invoking the C preprocessor twice.
So I've updated the recipe to:
%.o: %.c
$(CC) -MM -MG -MP -MT $*.o -MF $*.d $<
$(CC) -c $< -o $#
(Note also that gcc now has -MP to generate phony targets for each header,
so you no longer need to run sed on gcc's output.)
We still have the same problem as the original question -- running make the
first time fails to generate foo.h:
$ make
cc -MM -MG -MP -MT main.o -MF main.d main.c
cc -c main.c -o main.o
main.c:1:17: fatal error: foo.h: No such file or directory
#include "foo.h"
^
compilation terminated.
Makefile:7: recipe for target 'main.o' failed
make: *** [main.o] Error 1
Running it again works:
$ make
./mk_header.sh foo
cc -MM -MG -MP -MT main.o -MF main.d main.c
cc -c main.c -o main.o
cc main.o -o main
Since we have to run the C preprocessor twice anyway, let's generate the .d
file in a separate rule:
%.d: %.c
$(CC) -MM -MG -MP -MT $*.o -MF $# $<
%.o: %.c
$(CC) -c $< -o $#
Now it generates the header file correctly:
$ make clean
rm -f *.o *.d main foo.h
$ make
cc -MM -MG -MP -MT main.o -MF main.d main.c
./mk_header.sh foo
cc -c main.c -o main.o
cc main.o -o main
Does this suffer from the performance issue that the original question was
trying to avoid? This is essentially the "Basic Auto-Dependencies" solution
described in the Advanced Auto-Dependency
Generation paper.
That paper claims 3 problems with this solution:
We re-exec make if anything changes.
Ugly but harmless warning: "main.d: No such file or directory"
Fatal error "no rule to make targe foo.h" if foo.h file is removed, even
if mention of it is removed from the .c file.
Problem 2 is solved by using -include instead of include. As far as I can
tell, this is orthogonal to the paper's technique for avoiding re-exec of
make. At least I haven't been able to cause any problems by using -include
instead of include.
Problem 3 is solved by GCC's -MP (or the equivalent sed script) -- this is
also orthogonal to the technique for avoiding re-exec of make.
Problem 1 can be perhaps ameliorated somewhat by something like this:
%.d: %.c
$(CC) -MM -MG -MP -MT $*.o -MF $#.new $<
cmp $#.new $# 2>/dev/null || mv $#.new $#; rm -f $#.new
Before that change:
$ make clean
rm -f *.o *.d main foo.h
$ make -d 2>&1 | grep Re-executing
Re-executing[1]: make -d
$ make -d 2>&1 | grep Re-executing
$ touch main.c; make -d 2>&1 | grep Re-executing
Re-executing[1]: make -d
After that change:
$ make clean
rm -f *.o *.d main foo.h
$ make -d 2>&1 | grep Re-executing
Re-executing[1]: make -d
$ make -d 2>&1 | grep Re-executing
$ touch main.c; make -d 2>&1 | grep Re-executing
Slightly better. Of course if a new dependency is introduced, make will still
need to re-execute. Maybe there's nothing that can be done to improve this; it seems to be a tradeoff between correctness and speed.
All of the above was tested with make 3.81.
Short answer: no. The recipe described in the paper is very clever, one of my favorites, but it's a sophisticated use of a crude tool. It takes advantage of the usual scheme in which all needed headers exist; what it tries to solve is the problem of determining which headers, if recently modified, require the given object file to be rebuilt. In particular, if the object file doesn't exist then it must be rebuilt-- and in that case there's no reason to worry about the header files because the compiler will surely find them.
Now header files are generated. So foo.h may not exist, so somebody will have to run the script to generate it, and only Make can do that. But Make can't know that foo.h is necessary without performing some analysis of main.c. But that really can't happen until Make starts to execute main-related rules (e.g main.o or main.o.d), which it cannot execute until after it has decided which targets it is going to build.
So we will have to use... recursive make! [Dun-dun-dunnnn!]
We can't achieve the paper's goal of avoiding reinvocation of Make, but we can at least avoid (some) unnecessary rebuilding. You could do something like the "Basic Auto-Dependencies" described in the paper; the paper describes the problems of that approach. Or you could use a command like the one in the "Advanced" recipe to generate a list of headers, then pass that to $(MAKE); this approach is tidy, but might call Make many times on the same header, depending on what your code tree looks like.
You could create an explicit dependency rule for your generated header:
main.o: foo.h
If the generated header is directly included in a small number of files, this may be a workable approach.
I want to generate a single dependency file which consists of all the dependencies of source files using gcc -M flags through Makefile. I googled for this solution but, all the solutions mentioned are for generating multiple deps files for multiple objects.
DEPS = make.dep
$(OBJS): $(SOURCES)
#$(CC) -MM $(SOURCEs) > $(DEPS)
#mv -f $(DEPS) $(DEPS).tmp
#sed -e 's|.$#:|$#:|' < $(DEPS).tmp > $(DEPS)
#sed -e 's/.*://' -e 's/\\$$//' < $(DEPS).tmp | fmt -1 | \
sed -e 's/^ *//' -e 's/$$/:/' >> $(DEPS)
#rm -f $(DEPS).tmp
But it is not working properly. Please tell me where i'm making the mistake.
Something along these lines is what I use to get all my dependencies in a single file:
program_H_SRCS := $(wildcard *.h)
program_C_SRCS := $(wildcard *.c)
DEPS = make.deps
make.deps: $(program_C_SRCS) $(program_H_SRCS)
$(CC) $(CPPFLAGS) -MM $(program_C_SRCS) > make.deps
include $(DEPS)
This basically causes all the user ( as opposed to system ) dependencies to be rebuilt into a single file whenever any C or H file in the project is modified.
+++++++++ EDIT +++++++++++
I've since found a better way of doing things. I generate a separate dep file for each source file. Here is the basic makefile:
program_NAME := myprogram
program_SRCS := $(wildcard *.c)
program_OBJS := ${program_SRCS:.c=.o}
clean_list += $(program_OBJS) $(program_NAME)
# C Preprocessor Flags
CPPFLAGS +=
# compiler flags
CFLAGS += -ansi -Wall -Wextra -pedantic-errors
.PHONY: all clean distclean
all: $(program_NAME)
clean:
#- $(RM) $(clean_list)
distclean: clean
# Generate dependencies for all files in project
%.d: $(program_SRCS)
# $(CC) $(CPPFLAGS) -MM $*.c | sed -e 's#^\(.*\)\.o:#\1.d \1.o:#' > $#
clean_list += ${program_SRCS:.c=.d}
$(program_NAME): $(program_OBJS)
indent -linux -brf $(program_SRCS)
splint $(program_SRCS)
$(LINK.c) $(program_OBJS) -o $(program_NAME)
ifneq "$(MAKECMDGOALS)" "clean"
# Include the list of dependancies generated for each object file
-include ${program_SRCS:.c=.d}
endif
This does two things:
If any of the files that foo.c depend on change then foo.o is rebuilt without having to rebuild other files in the project.
The dep file itself has the same dependencies as the object file, so that if any of the deps are modified the dep file itself is also regenerated, before the object file deps are checked.
I think is is expected behaviour for gcc -M, where typically you'd do something like this:
FOO_SOURCES= \
src/foo.c \
src/bar.c
FOO_OBJECTS = $(FOO_SOURCES:.c=.o)
FOO_DEPS = $(FOO_OBJECTS:.o=.d)
(... lots of targets ...)
-include $(FOO_DEPS)
Note, -include not include as the dependencies will obviously not exist until at least one build has been run. Regardless, dependencies are generated on a per module basis.
Also note that gcc -M does not always work as you would expect it to work, broadly depending on what version of gcc you happen to be using.
I think what you want is something called makedep, which does what you want without sed hackery in the makefile.