Related
Currently I am trying to get a rather big project of mine to work with a Makefile. I used Make before but in a rather crude way and not really "dynamic", this means I am pretty new to good Makefiles.
My Makefile looks like this:
INCLUDE_DIR = /inc
SOURCE_DIR = /src
BUILD_DIR = /build
BUILD_NAME = build
CC = arm-none-eabi-gcc
CFLAGS = -I$(INCLUDE_DIR)
_INCLUDES = main.h pfc.h
INCLUDES = $(patsubst %, $(INCLUDE_DIR)/%, $(_INCLUDES))
_OBJ = main.o pfc.o
OBJ = $(patsubst %, $(BUILD_DIR)/%, $(_OBJ))
$(BUILD_DIR)/%.o: $(SOURCE_DIR)/%.c $(INCLUDES)
$(CC) -c -o $# $< $(CFLAGS)
$(BUILD_NAME): $(OBJ)
$(CC) -o $# $^ $(CFLAGS)
all: $(BUILD_NAME)
.PHONY: clean
clean:
rm -rf $(BUILD_DIR)/*
When I make the file I get this:
make: *** No rule to make target '/build/main.o', needed by 'build'. Stop.
I guess it is an error in this recipe:
$(BUILD_DIR)/%.o: $(SOURCE_DIR)/%.c $(INCLUDES)
$(CC) -c -o $# $< $(CFLAGS)
Sadly I am not able to get this done on my own.
All files a available in the correct folders.
I really appreciate all the help!
Tristan
You have a few issues here that I can see. But first off - just check (just incase) that all your rules are only indented with tabs and not spaces..... this can be a real "silent" killer of makefiles as they give crap error messages.
Ok so - lets assume you have:
INCLUDE_DIR = inc
SOURCE_DIR = src
BUILD_DIR = build
instead of /src etc.. as mentioned in the comments.
Do you really have inc/main.h and inc/pfc.h?
I copied and pasted your makefile added your src and inc folders (but I used gcc instead of arm-none-eabi-gcc. It did the compile lines correctly, but failed at the linker stage because you are trying to build an output file called build when there is already a folder called build (not allowed in linux - maybe ok for windows but I don't recommend).
I made an answer for another question - but it might be a better start point then you have here in the case where you have nested src/inc directories and you want to be able to clean your output folders - ill put it here for convenience:
# Get your source list (use wildcard or what ever, but just for clarity you should end up with a list of files with full paths to start with):
# Output folders/targets
SRC_DIR = src
OBJ_DIR = obj
BIN_DIR = bin
OUTPUT_FILE = output
# Generate list of source files - this is a linux command - but you can do this in pure make using wildcard and such).
SOURCES := $(shell find $(SOURCEDIR) -name '*.c')
# Create your objects list in the obj directory
OBJECTS = $(addprefix $(OBJ_DIR)/,$(addsuffix .o,$(basename $(SOURCES))))
# Create list of unique folders to create
DIRS = $(sort $(dir $(OBJECTS))) $(BIN_DIR)
# Create list of include paths
INCS = $(addprefix -I,$(sort $(dir $(SOURCES))))
# Main target rule
$(BIN_DIR)/$(OUTPUT_FILE): $(OBJECTS) | $(DIRS)
#echo linker: gcc $(OBJECTS) -o $#
#touch $#
# Rule to build your object file - ensure that the folders are created first (also create a dummy obj file) - note this works for parallel builds too (make -j
$(OBJ_DIR)/%.o: %.c | $(DIRS)
#echo compile: gcc $(INCS) -c $? -o $#
#touch $#
# Create your directories here
$(DIRS):
#echo Creating dir: $#
#mkdir -p $#
# Clean if needed
.PHONY: clean
clean:
rm -rf $(OBJ_DIR) $(BIN_DIR)
Note this is just a template, you still need to fill in the gcc/makefile flags - but its a decent start point...
Debugging
$(info ...) is your friend - for example you could do:
$(info OBJ = $(OBJ))
$(info objrule = $(BUILD_DIR)/%.o: $(SOURCE_DIR)/%.c $(INCLUDES))
To print our what make has expanded these variables / lines to be - this can yield useful debug.
Here is another version of your makefile with automatic dependency generation:
INCLUDE_DIR := inc
SOURCE_DIR := src
BUILD_DIR := build
CC := arm-none-eabi-gcc
CPPFLAGS := -I$(INCLUDE_DIR)
exes := build
build.obj := main.o pfc.o
all : ${exes:%=${BUILD_DIR}/%}
.SECONDEXPANSION:
${BUILD_DIR}:
mkdir -p $#
# Rule to link all exes.
${exes:%=${BUILD_DIR}/%} : ${BUILD_DIR}/% : $$(addprefix ${BUILD_DIR}/,$${$$*.obj}) | $${#D}
${CC} -o $# ${LDFLAGS} $^ ${LDLIBS}
# Rule to compile C sources. And generate header dependencies.
${BUILD_DIR}/%.o : ${SOURCE_DIR}/%.c | $${#D}
${CC} -o $# -c ${CPPFLAGS} ${CFLAGS} -MD -MP $<
# Include automatically generated header dependencies.
ifneq ($(MAKECMDGOALS),clean)
-include $(foreach exe,${exes},$(patsubst %.o,${BUILD_DIR}/%.d,${${exe}.obj}))
endif
clean:
rm -rf $(BUILD_DIR)
.PHONY: all clean
To add another executable target do:
exes += another
another.obj := another_main.o another_pfc.o
SEE UPDATES BELOW
Research Done: I'm finding learning how to evolve Makefiles from one situation to another is difficult. There are a ton of questions and answers out there but few of them actually show how a Makefile can evolve as your project changes. They also all seem to use various different techniques and idioms of Makefiles so translating between one question and another can be tricky when you are learning Makefiles for the first time, as I am.
Problem: My problem is that I have a project that started at as a flat directory structure but then is migrating to a structure with sub-directories. What I can't do is get my Makefile to along for the ride.
First I'll show what I created that works and then I show how I want it to evolve and how that doesn't work.
Flat Directory Structure, Working Makefile
I have project directory that has all my C files and one header file plus my Makefile:
project
Makefile
c8_asm.c
c8_dasm.c
c8_terp.c
chip8.h
Here is my Makefile (which works just fine):
CC = gcc
CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L
LDLIBS += -lm
# Targets
all: c8_dasm c8_asm c8_terp
c8_dasm: c8_dasm.o
$(CC) $(LDLIBS) c8_dasm.o -o $#
c8_asm: c8_asm.o
$(CC) $(LDLIBS) c8_asm.o -o $#
c8_terp: c8_terp.o
$(CC) $(LDLIBS) c8_terp.o -o $#
# Using implicit rules for updating an '.o' file from a correspondingly
# named '.c' file.
c8_dasm.o: chip8.h
c8_asm.o: chip8.h
c8_terp.o: chip8.h
.PHONY: clean
clean:
rm c8_dasm c8_asm c8_terp c8_dasm.o c8_asm.o c8_terp.o
I get all my .o files and my executables are created in the project directory.
Evolving The Project
But what I wanted to do is have my sources files (all .c and .h) in a src directory. I wanted to build into an obj directory and have the executables go in a bin directory. So my project would look like this:
project
src
c8_asm.c
c8_dasm.c
c8_terp.c
chip8.h
Makefile
Sub-Directory Structure, Makefile NOT Working
To accommodate the above, I changed my Makefile accordingly:
CC = gcc
CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L
LDLIBS += -lm
SRC_DIR = src
OBJ_DIR = obj
BIN_DIR = bin
SOURCES := $(wildcard $(SRC_DIR)/*.c)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.o)
MKDIR_P ?= mkdir -p
# Targets
all: $(BIN_DIR)/c8_dasm $(BIN_DIR)/c8_asm $(BIN_DIR)/c8_terp
$(BIN_DIR)/c8_dasm: $(OBJ_DIR)/c8_dasm.o
$(CC) $(LDLIBS) $(OBJ_DIR)/c8_dasm.o -o $#
$(BIN_DIR)/c8_asm: $(OBJ_DIR)/c8_asm.o
$(CC) $(LDLIBS) $(OBJ_DIR)/c8_asm.o -o $#
$(BIN_DIR)/c8_terp: $(OBJ_DIR)/c8_terp.o
$(MKDIR_P) $(dir $#)
$(CC) $(LDLIBS) $(OBJ_DIR)/c8_terp.o -o $#
$(OBJECTS): $(OBJ_DIR)/%.o : $(SRC_DIR)/%.c
$(MKDIR_P) $(dir $#)
$(CC) $< -o $(OBJ_DIR)/$#
# Using implicit rules for updating an '.o' file from a correspondingly
# named '.c' file.
$(OBJ_DIR)/c8_dasm.o: $(SRC_DIR)/chip8.h
$(OBJ_DIR)/c8_asm.o: $(SRC_DIR)/chip8.h
$(OBJ_DIR)/c8_terp.o: $(SRC_DIR)/chip8.h
.PHONY: clean
clean:
rm -r $(BUILD_DIR)
rm $(OBJECTS)
Upon running this I get the following:
mkdir -p obj/obj/
gcc src/c8_dasm.c -o obj/c8_dasm.o
gcc -lm obj/c8_dasm.o -o bin/c8_dasm
ld: can't link with a main executable file 'obj/c8_dasm.o' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bin/c8_dasm] Error 1
I wanted to stop here and get some assistance because I fear I'm making this Makefile for complicated than it need be and I'm trying to avoid getting into bad habits.
I'm hoping to hear opinions about what I'm not conceptualizing correctly here.
FIRST UPDATE
I managed to take it bit by bit and get it mostly working. Here is what I ended up with:
CC = gcc
CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L
LDLIBS += -lm
# Directories.
SRC_DIR = src
BIN_DIR = bin
$(shell mkdir -p $(BIN_DIR))
# Patterns for files.
SOURCES := $(wildcard $(SRC_DIR)/*.c)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.c=$(SRC_DIR)/%.o)
EXECUTABLES := c8_dasm c8_asm c8_terp
# Targets
all: $(EXECUTABLES)
c8_dasm: $(SRC_DIR)/c8_dasm.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Disassembler Built"
c8_asm: $(SRC_DIR)/c8_asm.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Assembler Built"
c8_terp: $(SRC_DIR)/c8_terp.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Interpreter Built"
# Using implicit rules for updating an '.o' file from a correspondingly
# named '.c' file.
c8_dasm.o: $(SRC_DIR)/chip8.h
c8_asm.o: $(SRC_DIR)/chip8.h
c8_terp.o: $(SRC_DIR)/chip8.h
.PHONY: clean
clean:
rm $(OBJECTS)
rm -r $(BIN_DIR)
Of course, as I'm finding with Make this leads to other obscure problems. For example doing this:
make
make clean
works fine. Meaning all files are generated and the files are cleaned, including the bin directory.
However, if I do this:
make c8_dasm
make clean
This builds fine. But the clean fails to delete the bin directory (although it does delete the object files). This happens regardless of what individual executable I try to build.
No amount of searching is helping me find out why that is.
SECOND UPDATE
I found that problem was solved as well. It just required using the "-f" for the rm statements in the clean target.
THIRD UPDATE
To get the object file directory part working, I tried (from this: path include and src directory makefile) to construct my Makefile as follows:
CC = gcc
CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L
LDLIBS += -lm
SRC_DIR = src
OBJ_DIR = obj
BIN_DIR = bin
$(shell mkdir -p $(BIN_DIR))
$(shell mkdir -p $(OBJ_DIR))
SOURCES := $(wildcard $(SRC_DIR)/*.c)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.o)
EXECUTABLES := c8_dasm c8_asm c8_terp
all: $(EXECUTABLES)
c8_dasm: $(SRC_DIR)/c8_dasm.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Disassembler Built"
c8_asm: $(SRC_DIR)/c8_asm.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Assembler Built"
c8_terp: $(SRC_DIR)/c8_terp.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Interpreter Built"
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.c
$(CC) $(CFLAGS) -c $< -o $(BIN_DIR)/$#
.PHONY: clean
clean:
rm -rf $(BIN_DIR)
rm -f $(OBJECTS)
I was able to condense the original three lines using chip8.h into one target but I have no way to know if that's correct. It compiles at least. I also changed the OBJECTS line to reflect the new OBJ_DIR I created.
However, this doesn't put the object files in the right place. It still puts them in the src directory rather than the obj directory.
This is why it makes sense to not do anything complicated with Makefiles. Just put the actual directory names in your commands. Never rely on wildcards.
People using C and C++ and using Makefiles spend too much time trying to get those to work rather than just actually getting things done. That's why you see so many of the questions that you see and why the answers vary so much.
In your specific case, your targets don't always have to contain the directory and that's part of the problem. The rules getting generated don't have an actual target in your file because of the directories you are prepending to everything. You have to think in terms of what is getting generated by each target: meaning, the output. So if c8_dasm is getting output, that's your target. The directory has nothing to do with that. So you need to remove all of your directory substitutions where they aren't needed.
But before doing that, ask yourself this: if your first solution was working, why change it? It's better to not even do directories when you're using Make. Just have everything in the same directory as you started off with. You can even see that this allows your Makefile to be much cleaner.
I believe I may have figured this out. Below is my Makefile. It seems to do what I want. It does the following:
Compiles all object files into the obj directory.
Compiles and links so that executables are generated in the bin directory.
Recognizes if any .c files are changed and recompiles accordingly.
Recognizes if the .h file is changed and recompiles all C files that reference it.
This seems to satisfy all the criteria but I can't tell if I've painted myself into some corner that I can't see yet.
CC = gcc
CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L
LDLIBS += -lm
SRC_DIR = src
OBJ_DIR = obj
BIN_DIR = bin
$(shell mkdir -p $(BIN_DIR))
$(shell mkdir -p $(OBJ_DIR))
SOURCES := $(wildcard $(SRC_DIR)/*.c)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.o)
EXECUTABLES := c8_dasm c8_asm c8_terp
all: $(EXECUTABLES)
c8_dasm: $(OBJ_DIR)/c8_dasm.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Disassembler Built"
c8_asm: $(OBJ_DIR)/c8_asm.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Assembler Built"
c8_terp: $(OBJ_DIR)/c8_terp.o
$(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$#
#echo "C8 Interpreter Built"
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.c $(SRC_DIR)/chip8.h
$(CC) $(CFLAGS) -c $< -o $#
.PHONY: clean
clean:
rm -rf $(BIN_DIR)
rm -rf $(OBJ_DIR)
Stackoverflow is whining about too many comments, so I'll make this another "answer." After our back-and-forth to my original comment, your last comment is correct. That's what I wanted you to see.
Understand that you can't use Make to do what you want to do exactly.
So here's really the answer: You can't create multiple executables AND with only some of the object files applying to each one AND while using a directory structure. Make is in no way capable of handling that.
Right now you're trying to use Make in a way that it wasn't intended for which is why you're running into so many problems. If you keep playing around you're going to run into is a series of errors that say "duplicate symbol" because you will be compiling each of your files multiple times for each executable, assuming you follow most of the advice you'll find.
Check out this How can I create a Makefile for C projects with SRC, OBJ, and BIN subdirectories? to see what I mean. That one works because all object files are being used to create a single executable. But as you've stated, that's not going to be the case for you. And that's what Make can't handle. That's why you're not finding an answer to that.
And while your chip8.h file is now not going to cause problems in terms of allowing you to compile, your Makefile with that third update would not recognize when the chip8.h file itself has changed. You would have to change a .c file to force a recompile so that changes to your .h were recognized. So you either have to stick with your second update or use something other than Make.
This question is different from the one at makefiles - compile all c files at once in the sense that I have one extra requirement: I want to redirect all the object files in a separate directory.
Here is the setup:
I have multiple sources in a directory say src/mylib.
I want the objects files to end up in build/mylib.
Please note also that under mylib there are subdirectories.
The first attempt was as follows:
sources = $(shell find src/ -name ".c")
objects_dirs = $(subst src/, build/, $(dir $(sources)) # This variable is used by the build rule to create directories for objects files prior to compilation
objects = $(subst src/, build/, $(patsubst %.c, %.o, $(sources))) # This variable has the paths to the objects files that will be generated in the build directory
# This is where things aren't working as expected
$(objects): build $(sources)
$(cc) $(cflags) -o $# $(word 2, $^))
build:
$(foreach dir, $(objects_dirs), $(shell mkdir -p $(dir)))
For the makefile above, only one object file was being generated. I guessed this might have something to do with GCC only being able to generate one object file at a time. Regardless of that, checking the values of $# and $(word 2, $^) in the $(objects) target shows that only one file is being considered even though I have multiple files.
So I changed my makefile to the following:
sources = $(shell find src/ -name ".c")
objects = $(subst src/, build/, $(patsubst %.c, %.o, $(sources))) # This variable has the paths to the objects files that will be generated in the build directory
# This works as expected but it appears to me like make is generating all the objects files even though source files did not change. This can be seen by checking the timestamps on new object files after running make again.
$(objects): build $(sources)
$(foreach source, $(sources), $(shell $(cc) $(cflags) -o $(subst src/,build/, $(patsubst %.o,%.c,$(source))) $(source)))
build:
$(foreach dir, $(objects_dirs), $(shell mkdir -p $(dir)))
The second makefile works as expected but objects files are being rebuilt again which defeats another purpose of using make: only recompile those source files that changed from the last compilation.
Hence my question: how does one generate all object files in a separate directory at once (by this I mean perform the compilation of all sources files in one rule) while making sure that if a source file didn't change the associated object file should not be regenerated.
I am not after speeding up compilation. What I seek is one rule that will generate all objects files such that only updated source files should be recompiled.
The last makefile does the job but there is a recompiling of all source files which defeats another purpose of using make: only changed source files should be recompiled.
EDIT
After reading comments, it appears I have not phrased my question properly. As the details of what I have are already present, I leave the question as it is with additional details below.
The second makefile in the source code above does work. But it does only half the job. The build directory effectively mirrors the src directory.
So if I have say a file as src/mylib/point/point.c, I get build/mylib/point/point.o generated. This is the first part.
The second part is that if point.c does not changes, point.o in the build/mylib/point/ directory must not be regenerated. But after checking timestamps on the object file, I can tell that a new object file replaced the old one after running make again. This is not good because for large projects, compilation time remains O(n) with n being the number of source files to compile.
So this question is about how to preserve the second makefile without make regenerating object files.
From what I can gather from comments, I am asking too much from make. But if anyone knows how to make this happen, I leave the question open.
Makefile:
all:
clean:
src_root := src
src_subdirs := foo foo/bar foo/bar/buz
build_root := build
o_suffix := .o
# Build list of sources. Iterate every subfolder from $(src_subdirs) list
# and fetch all existing files with suffixes matching the list.
source_suffixes := .c .cpp .cxx
sources := $(foreach d,$(addprefix $(src_root)/,$(src_subdirs)),$(wildcard $(addprefix $d/*,$(source_suffixes))))
# If src_subdirs make variable is unset, use 'find' command to build list of sources.
# Note that we use the same list of suffixes but tweak them for use with 'find'
ifeq ($(src_subdirs),)
sources := $(shell find $(src_root) -type f $(foreach s,$(source_suffixes),$(if $(findstring $s,$(firstword $(source_suffixes))),,-o) -name '*$s'))
endif
$(info sources=$(sources))
# Build source -> object file mapping.
# We want map $(src_root) -> $(build_root) and copy directory structure
# of source tree but populated with object files.
objects := $(addsuffix $(o_suffix),$(basename $(patsubst $(src_root)%,$(build_root)%,$(sources))))
$(info objects=$(objects))
# Generate rules for every .o file to depend exactly on corresponding source file.
$(foreach s,$(sources),$(foreach o,$(filter %$(basename $(notdir $s)).o,$(objects)),$(info New rule: $o: $s)$(eval $o: $s)))
# This is how we compile sources:
# First check if directory for the target file exists.
# If it doesn't run 'mkdir' command.
$(objects): ; $(if $(wildcard $(#D)),,mkdir -p $(#D) &&) g++ -c $< -o $#
# Compile all sources.
all: $(objects)
clean: ; rm -rf $(build_root)
.PHONY: clean all
Environment:
$ find
.
./src
./src/foo
./src/foo/bar
./src/foo/bar/bar.cxx
./src/foo/bar/buz
./src/foo/bar/buz/buz.c
./src/foo/bar/foo.c
./src/foo/foo.cpp
Run makefile:
$ make -f /cygdrive/c/stackoverflow/Makefile.sample -j
sources=src/foo/bar/bar.cxx src/foo/bar/buz/buz.c src/foo/bar/foo.c src/foo/foo.cpp
objects=build/foo/bar/bar.o build/foo/bar/buz/buz.o build/foo/bar/foo.o build/foo/foo.o
New rule: build/foo/bar/bar.o: src/foo/bar/bar.cxx
New rule: build/foo/bar/buz/buz.o: src/foo/bar/buz/buz.c
New rule: build/foo/bar/foo.o: src/foo/bar/foo.c
New rule: build/foo/foo.o: src/foo/bar/foo.c
New rule: build/foo/bar/foo.o: src/foo/foo.cpp
New rule: build/foo/foo.o: src/foo/foo.cpp
mkdir -p build/foo/bar && g++ -c src/foo/bar/bar.cxx -o build/foo/bar/bar.o
mkdir -p build/foo/bar/buz && g++ -c src/foo/bar/buz/buz.c -o build/foo/bar/buz/buz.o
mkdir -p build/foo/bar && g++ -c src/foo/bar/foo.c -o build/foo/bar/foo.o
mkdir -p build/foo && g++ -c src/foo/bar/foo.c -o build/foo/foo.o
Environment again:
$ find
.
./build
./build/foo
./build/foo/bar
./build/foo/bar/bar.o
./build/foo/bar/buz
./build/foo/bar/buz/buz.o
./build/foo/bar/foo.o
./build/foo/foo.o
./src
./src/foo
./src/foo/bar
./src/foo/bar/bar.cxx
./src/foo/bar/buz
./src/foo/bar/buz/buz.c
./src/foo/bar/foo.c
./src/foo/foo.cpp
Try running this Makefile with 'src_subdirs=' to exercise another approach to locate sources. Output should be the same.
I finally had some time to experiment with this, so here is what I came up with:
BUILD_DIR = build
SRC_DIR = src
SOURCES = $(shell find $(SRC_DIR)/ -name "*.c")
TARGET = program
OBJECTS = $(SOURCES:$(SRC_DIR)/%.c=$(BUILD_DIR)/%.o)
default: $(TARGET)
.SECONDEXPANSION:
$(OBJECTS) : $$(patsubst $(BUILD_DIR)/%.o,$(SRC_DIR)/%.c,$$#)
mkdir -p $(#D)
$(CC) -c -o $# $(CFLAGS) $<
$(TARGET): $(OBJECTS)
$(CC) -o $# $(CFLAGS) $^
.PHONY: default
Points of interest:
I had to change the sources find pattern from ".c" to "*.c", I'm not sure if it depends on the exact shell used, but if you want to stay portable, be sure to use a widely accepted pattern.
The .SECONDEXPANSION: is needed to enable the $$ rules for GNU Make. It is needed to allow target based substitution rules in the prerequisites for the $(OBJECTS).
The prerequisite $$(patsubst $(BUILD_DIR)/%.o,$(SRC_DIR)/%.c,$$#) is saying, that the current target depends on a specific source file with the same folder structure and name.
The command mkdir -p $(#D) is ensuring, that the path of the current target is created if it's missing.
If all you want is a single rule to handle all object files, without necessarily needing to "compile all at once" then you could have something like this:
BUILD_DIR = build
SOURCES = ...
TARGET = ...
OBJECTS = $(SOURCES:%.c=$(BUILD_DIR)/%.o)
default: target
target: $(TARGET)
$(TARGET): $(OBJECTS)
$(LD) -o $# $(LDFLAGS) $^ $(LIBS)
$(BUILD_DIR)/%.o: %.c
$(CC) -c -o $# $< $(CFLAGS)
$(BUILD_DIR):
-mkdir $#
[Note: This is written from memory and without testing.]
After reading the GNU make manual again, here is a solution that solves the second problem.
The first attempt was the correct path. And the second attempt has the $(sources) in the prerequisites but does not use it in the commands and this is silly.
So the working makefile follows. It puts object files in a separate directory and it only compiles files that have changed.
sources = $(shell find src/ -name ".c")
$objects_dirs = $(subst src/, build/, $(dir $(sources)) # This variable is used by the build rule to create directories for objects files prior to compilation
objects = $(subst src/, build/, $(patsubst %.c, %.o, $(sources))) # This variable has the paths to the objects files that will be generated in the build directory
# This should now work as expected: object files go into their designated directories under "build/" and only updated files will be recompiled.
$(objects): build $(sources)
# After running say "make clean", make will figure out the need to run the first prerequisite.
# If we are doing a clean build, the number of prerequisites will equal the number of new prerequisites.
ifeq ($(words $?), $(words $^))
# Note the use of "$?" instead of "$^". $? is used since it holds prerequisites that are newer than the target while $^ will holds all prerequisites whether they are new or not.
$(foreach source, $(wordlist 2, $(words $?), $?), $(shell $(cc) $(cflags) -o $(subst src/,build, $(patsubst %.c,%.o, $(source))) $(source)))
else
# If we have a few new targets, no need to exclude "build" from prerequisites because the first prerequisite will be a file that changed.
$(foreach source, $?, $(shell $(cc) $(cflags) -o $(subst src/,build, $(patsubst %.c,%.o, $(source))) $(source)))
endif
.PHONY: build
build:
$(foreach dir, $(objects_dirs), $(shell mkdir -p $(dir)))
.PHONY: clean
clean:
#rm -rf build/
The makefile is heavily commented with changes that made it work. The most important changes were:
Use of $(foreach) to compile each file individually as required by GCC
Use of $? to work only with prerequisites that are newer than the target
Use of conditional to detected whether the first prerequisite has changed depending on circumstances. If we have a clean build (running make for the first time or after running make clean), the number of updated prerequisites will be the same as the number of newer prerequisites compared to the target. In other words $(words $?) == $(words $^) will be true. So we use this fact to exclude the firs prerequisite listed (build in our case) from the list of files to pass to GCC.
Also, when building the executable from the objects files, make sure to use $^ and not $? when selecting prerequisites else you will end up with only newer files in the executable and it will not run.
target = bin/mylib.a
.PHONY: all
all: $(target)
$(target): $(objects)
ar -cvq $# $^ # Notice that we're not using $? else only updated object files will end up in the archive.
[edit]- The answer is I have incorrectly defined the path to the include directory. Embarrasing mistake.
I am writing a makefile that builds objects from .c files in a src directory with headers in an include directory and stores them in an obj directory, and then builds and exe from those objects. A toy version is below.
# Makefile
IDIR = ../include
CC = gcc
CFLAGS = -I$(IDIR) -Wall
SDIR = src
ODIR = obj
_SOURCES = main.c aux1.c aux2.c aux3.c
SOURCES = $(patsubst %,$(SDIR)/%,$(_SOURCES))
_OBJECTS = $(_SOURCES:.c=.o)
OBJECTS = $(patsubst %,$(ODIR)/%,$(_OBJECTS))
_DEPS = aux1.h aux2.h aux3.h
DEPS = $(patsubst %,$(IDIR)/%,$(_DEPS))
EXE = exefile
$(ODIR)/%.o: $(SDIR)/%.c $(DEPS)
$(CC) -c -o $# $< $(CFLAGS)
$(EXE): $(OBJECTS)
$(CC) -o $# $^ $(CFLAGS)
.PHONY: clean
clean:
rm *o $(EXE)
The directory the makefile is in looks like this (when you ls)
include Makefile obj src trash
When I type make, I get the following error
make: *** No rule to make target `obj/main.o', needed by `exefile'. Stop.
I suspect the problem is with this line
$(ODIR)/%.o: $(SDIR)/%.c $(DEPS)
I am attempting to build each object file in obj/ by referencing the corresponding source file in src/ but the line seems do to something else. Is this the case? Is there an easy way to reference .c files in the source directory without "entering" the directory?
Thanks in advance
I believe your problem is the $(DEPS) dependency.
You set IDIR to ../include but from the toplevel directory (where make is run from) that path is incorrect.
I'm having trouble with trying to use make to place object files in a separate subdirectory, probably a very basic technique. I have tried to use the information in this page:
http://www.gnu.org/software/hello/manual/make/Prerequisite-Types.html#Prerequisite-Types
I get the following output from make:
make: *** No rule to make target `ku.h', needed by `obj/kumain.o'. Stop.
However ku.h is a dependency not a target (although it's obviously #included within the c source files). When I don't try to use a subdirectory for object files (i.e. miss out the OBJDIR parts) it works fine. Why does make think ku.h is a target?
my makefile is this: (the style is after reading various sources of information)
.SUFFIXES:
.SUFFIXES: .c .o
CC=gcc
CPPFLAGS=-Wall
LDLIBS=-lhpdf
VPATH=%.c src
VPATH=%.h src
VPATH=%.o obj
OBJDIR=obj
objects= $(addprefix $(OBJDIR)/, kumain.o kudlx.o kusolvesk.o kugetpuz.o kuutils.o \
kurand.o kuASCboard.o kuPDFs.o kupuzstrings.o kugensud.o \
kushapes.o )
ku : $(objects)
$(CC) $(CPPFLAGS) -o ku $(objects) $(LDLIBS)
$(objects) : ku.h kudefines.h kuglobals.h kufns.h | $(OBJDIR)
$(OBJDIR):
mkdir $(OBJDIR)
.PHONY: clean
clean :
rm $(objects)
Edit:
I applied the change to use the vpath directive. My version was a bad mixture of VPATH=xxx and vpath %.c xxx. However I now get another problem (which was the original problem before I added the wrong vpath). This is now the output:
gcc -o ku -lhpdf obj/kumain.o obj/kudlx.o obj/kusolvesk.o ..etc
gcc: obj/kumain.o: No such file or directory
gcc: obj/kudlx.o: No such file or directory
gcc: obj/kusolvesk.o: No such file or directory
gcc: obj/kugetpuz.o: No such file or directory
gcc: obj/kuutils.o: No such file or directory
gcc: obj/kurand.o: No such file or directory
gcc: obj/kuASCboard.o: No such file or directory
gcc: obj/kuPDFs.o: No such file or directory
gcc: obj/kupuzstrings.o: No such file or directory
gcc: obj/kugensud.o: No such file or directory
gcc: obj/kushapes.o: No such file or directory
make: *** [ku] Error 1
It appears that make is not applying the implicit rule for an object file although the manual says
"Implicit rules tell make how to use customary techniques so that you do not have to specify them in detail when you want to use them. For example, there is an implicit rule for C compilation. File names determine which implicit rules are run. For example, C compilation typically takes a .c file and makes a .o file. So make applies the implicit rule for C compilation when it sees this combination of file name endings." and also "The search through the directories specified in VPATH or with vpath also happens during consideration of implicit rules (see Using Implicit Rules)."
Again here "For example, when a file foo.o has no explicit rule, make considers implicit rules, such as the built-in rule to compile foo.c if that file exists. If such a file is lacking in the current directory, the appropriate directories are searched for it. If foo.c exists (or is mentioned in the makefile) in any of the directories, the implicit rule for C compilation is applied."
Any assistance in getting implicit rules to work for my makefile would be greatly appreciated.
Edit no 2:
Thanks to Jack Kelly I have made an explicit rule to compile the .c files since I couldn't get anywhere trying to use implicit rules. Also thanks to al_miro for the vpath info.
Here is the working makfile:
.SUFFIXES:
.SUFFIXES: .c .o
CC=gcc
CPPFLAGS=-Wall
LDLIBS=-lhpdf
OBJDIR=obj
vpath %.c src
vpath %.h src
objects = $(addprefix $(OBJDIR)/, kumain.o kudlx.o kusolvesk.o kugetpuz.o kuutils.o \
kurand.o kuASCboard.o kuPDFs.o kupuzstrings.o kugensud.o \
kushapes.o )
ku : $(objects)
$(CC) $(CPPFLAGS) -o ku $(objects) $(LDLIBS)
$(OBJDIR) obj/%.o : %.c ku.h kudefines.h kuglobals.h kufns.h
$(CC) -c $(CPPFLAGS) $< -o $#
.PHONY : clean
clean :
rm $(objects)
Since you're using GNUmake, use a pattern rule for compiling object files:
$(OBJDIR)/%.o: %.c
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $# $<
This is the makefile that I use for most of my projects,
It permits putting source files, headers and inline files in subfolders, and subfolders of subfolders and so-forth, and will automatically generate a dependency file for each object This means that modification of headers and inline files will trigger recompilation of files which are dependent.
Source files are detected via shell find command, so there is no need to explicitly specify, just keep coding to your hearts content.
It will also copy all files from a 'resources' folder, into the bin folder when the project is compiled, which I find handy most of the time.
To provide credit where it is due, the auto-dependencies feature was based largely off Scott McPeak's page that can be found HERE, with some additional modifications / tweaks for my needs.
Example Makefile
#Compiler and Linker
CC := g++-mp-4.7
#The Target Binary Program
TARGET := program
#The Directories, Source, Includes, Objects, Binary and Resources
SRCDIR := src
INCDIR := inc
BUILDDIR := obj
TARGETDIR := bin
RESDIR := res
SRCEXT := cpp
DEPEXT := d
OBJEXT := o
#Flags, Libraries and Includes
CFLAGS := -fopenmp -Wall -O3 -g
LIB := -fopenmp -lm -larmadillo
INC := -I$(INCDIR) -I/usr/local/include
INCDEP := -I$(INCDIR)
#---------------------------------------------------------------------------------
#DO NOT EDIT BELOW THIS LINE
#---------------------------------------------------------------------------------
SOURCES := $(shell find $(SRCDIR) -type f -name *.$(SRCEXT))
OBJECTS := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%,$(SOURCES:.$(SRCEXT)=.$(OBJEXT)))
#Defauilt Make
all: resources $(TARGET)
#Remake
remake: cleaner all
#Copy Resources from Resources Directory to Target Directory
resources: directories
#cp $(RESDIR)/* $(TARGETDIR)/
#Make the Directories
directories:
#mkdir -p $(TARGETDIR)
#mkdir -p $(BUILDDIR)
#Clean only Objecst
clean:
#$(RM) -rf $(BUILDDIR)
#Full Clean, Objects and Binaries
cleaner: clean
#$(RM) -rf $(TARGETDIR)
#Pull in dependency info for *existing* .o files
-include $(OBJECTS:.$(OBJEXT)=.$(DEPEXT))
#Link
$(TARGET): $(OBJECTS)
$(CC) -o $(TARGETDIR)/$(TARGET) $^ $(LIB)
#Compile
$(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(SRCEXT)
#mkdir -p $(dir $#)
$(CC) $(CFLAGS) $(INC) -c -o $# $<
#$(CC) $(CFLAGS) $(INCDEP) -MM $(SRCDIR)/$*.$(SRCEXT) > $(BUILDDIR)/$*.$(DEPEXT)
#cp -f $(BUILDDIR)/$*.$(DEPEXT) $(BUILDDIR)/$*.$(DEPEXT).tmp
#sed -e 's|.*:|$(BUILDDIR)/$*.$(OBJEXT):|' < $(BUILDDIR)/$*.$(DEPEXT).tmp > $(BUILDDIR)/$*.$(DEPEXT)
#sed -e 's/.*://' -e 's/\\$$//' < $(BUILDDIR)/$*.$(DEPEXT).tmp | fmt -1 | sed -e 's/^ *//' -e 's/$$/:/' >> $(BUILDDIR)/$*.$(DEPEXT)
#rm -f $(BUILDDIR)/$*.$(DEPEXT).tmp
#Non-File Targets
.PHONY: all remake clean cleaner resources
The VPATH lines are wrong, they should be
vpath %.c src
vpath %.h src
i.e. not capital and without the = . As it is now, it doesn't find the .h file and thinks it is a target to be made.
In general, you either have to specify $(OBJDIR) on the left hand side of all the rules that place files in $(OBJDIR), or you can run make from $(OBJDIR).
VPATH is for sources, not for objects.
Take a look at these two links for more explanation, and a "clever" workaround.
http://mad-scientist.net/make/vpath.html
http://mad-scientist.net/make/multi-arch.html
Build from the output directory
Instead of building from the top-level directory, build from the output directory. You can access the source directories by setting the vpath. This option has the advantage that the built-in rules can be used.
build.sh
#!/bin/bash
mkdir -p obj
cp Makefile.template obj/Makefile
cd obj
make "$*"
Makefile
.SUFFIXES:
.SUFFIXES: .c .o
CC=gcc
CPPFLAGS=-Wall
LDLIBS=-lhpdf
VPATH=%.c ../src
VPATH=%.h ../src
objects=kumain.o kudlx.o kusolvesk.o kugetpuz.o kuutils.o \
kurand.o kuASCboard.o kuPDFs.o kupuzstrings.o kugensud.o \
kushapes.o
ku : $(objects)
$(objects) : ku.h kudefines.h kuglobals.h kufns.h
.PHONY: clean
clean :
rm $(objects)
The disadvantage is that error messages do not match the CWD. This can be solved by skipping build.sh and directly building from the obj directory.
Another advantage of this approach is that it's somewhat popular. cmake works in a similar fashion.
Create Rule based on output option
The following solution isn't nice in my opinion, as I really love the built-in rules. However, GNU make doesn't support something like vpath for output directories. And the built-in rules cannot match, as the % in %.o would match obj/foo of obj/foo.o, leaving make with a search in vpath %.c src/ for stuff like src/obj/foo.c, but not src/foo.c.
But this is as close to the built-in rules as you can get, and therefore to my best knowledge the nicest solution that's available.
$(OBJDIR)/%.o: %.c
$(COMPILE.c) $(OUTPUT_OPTION) $<
Explanation: $(COMPILE.c) $(OUTPUT_OPTION) $< actually is how .c.o is implemented, see http://git.savannah.gnu.org/cgit/make.git/tree/default.c (and it's even mentioned in the manual)
Besides, if $(OBJDIR) would only ever contain auto-gererated files, you could create it on-the-fly with an order-only prerequisite, making the clean rule slightly simpler:
$(OBJDIR):
mkdir -p $(OBJDIR)
$(OBJDIR)/%.o: %.c | $(OBJDIR)
$(COMPILE.c) $(OUTPUT_OPTION) $<
.PHONY: clean
clean:
$(RM) -r $(OBJDIR)
This requires that the feature order-only is available, which you can check using $(filter order-only, $(.FETAURES)). I've checked on Kubuntu 14.04 GNU make 3.81 and OpenSUSE 13.1 GNU make 3.82. Both were built with order-only enabled, and am now left puzzled why Kubuntu 14.04 comes with an older version of GNU make than OpenSUSE 13.1. Anyways, gonna download make 4.1 now :)
For anyone that is working with a directory style like this:
project
> src
> pkgA
> pkgB
...
> bin
> pkgA
> pkgB
...
The following worked very well for me. I made this myself, using
the GNU make manual as my main reference; this, in particular, was extremely helpful for my last rule, which ended up being the most important one for me.
My Makefile:
PROG := sim
CC := g++
ODIR := bin
SDIR := src
MAIN_OBJ := main.o
MAIN := main.cpp
PKG_DIRS := $(shell ls $(SDIR))
CXXFLAGS = -std=c++11 -Wall $(addprefix -I$(SDIR)/,$(PKG_DIRS)) -I$(BOOST_ROOT)
FIND_SRC_FILES = $(wildcard $(SDIR)/$(pkg)/*.cpp)
SRC_FILES = $(foreach pkg,$(PKG_DIRS),$(FIND_SRC_FILES))
OBJ_FILES = $(patsubst $(SDIR)/%,$(ODIR)/%,\
$(patsubst %.cpp,%.o,$(filter-out $(SDIR)/main/$(MAIN),$(SRC_FILES))))
vpath %.h $(addprefix $(SDIR)/,$(PKG_DIRS))
vpath %.cpp $(addprefix $(SDIR)/,$(PKG_DIRS))
vpath $(MAIN) $(addprefix $(SDIR)/,main)
# main target
#$(PROG) : all
$(PROG) : $(MAIN) $(OBJ_FILES)
$(CC) $(CXXFLAGS) -o $(PROG) $(SDIR)/main/$(MAIN)
# debugging
all : ; $(info $$PKG_DIRS is [${PKG_DIRS}])#echo Hello world
%.o : %.cpp
$(CC) $(CXXFLAGS) -c $< -o $#
# This one right here, folks. This is the one.
$(OBJ_FILES) : $(ODIR)/%.o : $(SDIR)/%.h
$(CC) $(CXXFLAGS) -c $< -o $#
# for whatever reason, clean is not being called...
# any ideas why???
.PHONY: clean
clean :
#echo Build done! Cleaning object files...
#rm -r $(ODIR)/*/*.o
By using $(SDIR)/%.h as a prerequisite for $(ODIR)/%.o, this forced make to look in source-package directories for source code instead of looking in the same folder as the object file.
I hope this helps some people. Let me know if you see anything wrong with what I've provided.
BTW: As you may see from my last comment, clean is not being called and I am not sure why. Any ideas?
For all those working with implicit rules (and GNU MAKE). Here is a simple makefile which supports different directories:
#Start of the makefile
VPATH = ./src:./header:./objects
OUTPUT_OPTION = -o objects/$#
CXXFLAGS += -Wall -g -I./header
Target = $(notdir $(CURDIR)).exe
Objects := $(notdir $(patsubst %.cpp,%.o,$(wildcard src/*.cpp)))
all: $(Target)
$(Target): $(Objects)
$(CXX) $(CXXFLAGS) -o $(Target) $(addprefix objects/,$(Objects))
#Beware of -f. It skips any confirmation/errors (e.g. file does not exist)
.PHONY: clean
clean:
rm -f $(addprefix objects/,$(Objects)) $(Target)
Lets have a closer look (I will refer to the current Directory with curdir):
This line is used to get a list of the used .o files which are in curdir/src.
Objects := $(notdir $(patsubst %.cpp,%.o,$(wildcard src/*.cpp)))
#expands to "foo.o myfoo.o otherfoo.o"
Via variable the output is set to a different directory (curdir/objects).
OUTPUT_OPTION = -o objects/$#
#OUTPUT_OPTION will insert the -o flag into the implicit rules
To make sure the compiler finds the objects in the new objects folder, the path is added to the filename.
$(Target): $(Objects)
$(CXX) $(CXXFLAGS) -o $(Target) $(addprefix objects/,$(Objects))
# ^^^^^^^^^^^^^^^^^^^^
This is meant as an example and there is definitly room for improvement.
For additional Information consult:
Make documetation. See chapter 10.2
Or:
Oracle: Programming Utilities Guide
You can specify the -o $# option to your compile command to force the output of the compile command to take on the name of the target. For example, if you have:
sources: cpp/class.cpp and cpp/driver.cpp
headers: headers/class.h
...and you want to place the object files in:
objects: obj/class.o obj/driver.o
...then you can compile cpp/class.cpp and cpp/driver.cpp separately into obj/class.o and obj/driver.o, and then link, with the following Makefile:
CC=c++
FLAGS=-std=gnu++11
INCS=-I./headers
SRC=./cpp
OBJ=./obj
EXE=./exe
${OBJ}/class.o: ${SRC}/class.cpp
${CC} ${FLAGS} ${INCS} -c $< -o $#
${OBJ}/driver.o: ${SRC}/driver.cpp ${SRC}/class.cpp
${CC} ${FLAGS} ${INCS} -c $< -o $#
driver: ${OBJ}/driver.o ${OBJ}/class.o
${CC} ${FLAGS} ${OBJ}/driver.o ${OBJ}/class.o -o ${EXE}/driver
None of these answers seemed simple enough - the crux of the problem is not having to rebuild:
makefile
OBJDIR=out
VPATH=$(OBJDIR)
# make will look in VPATH to see if the target needs to be rebuilt
test: moo
touch $(OBJDIR)/$#
example use
touch moo
# creates out/test
make test
# doesn't update out/test
make test
# will now update test
touch moo
make test