What is the difference between the two options below? Wouldn't $< expand to only one requisite (by definition "the first one") in both cases?
Option 1:
libcounter.a(lexer.o): lexer.o
$(AR) $(ARFLAGS) $# $<
libcounter.a(counter.o): counter.o
$(AR) $(ARFLAGS) $# $<
Option 2:
libcounter.a: lexer.o
$(AR) $(ARFLAGS) $# $<
libcounter.a: counter.o
$(AR) $(ARFLAGS) $# $<
ARFLAGS holds rv in both cases.
As a reference, the examples are taken from the book Managing Projects with GNU Make.
In the first option, make knows enough to look inside libcounter.a at the lexer.o object to see if it needs to update the library with the new object. In the second option, the timestamp on the library itself is compared with the object, which may lead to missed updates of the library - e.g. imagine 1) lexer.o gets built, then 2) counter.o gets built, then 3) libcounter.a gets new lexer.o added; now at this point, the timestamp on libcounter.a is newer than counter.o, so make wrongly concludes it does not need to add the new counter.o to the library.
Related
Based on this famous link and adapted from this gist, and supposing that all of your source files are .cpp, you get easily a solution like this to have auto-dependence generation:
SRCS := $(wildcard *.cpp */*.cpp)
DEPDIR := .d
DEPS := $(SRCS:%.cpp=$(DEPDIR)/%.d)
# Temporary .Td dependence file... ?
DEPFLAGS = -MT $# -MD -MP -MF $(DEPDIR)/$*.Td
# Isn't it better a order-only prerequisite?
$(shell mkdir -p $(dir $(DEPS)) >/dev/null)
%.o: %.cpp # Removal of implicit rules... ? Twice?
%.o: %.cpp $(DEPDIR)/%.d # Dependency on .d... ?
g++ -o $# $(DEPFLAGS) $(CXXFLAGS) $(CPPFLAGS) -c $<
# Avoid some bugs?
mv -f $(DEPDIR)/$*.Td $(DEPDIR)/$*.d && touch $#
# If the `%.d` dependency is removed, is this still necessary?
.PRECIOUS = $(DEPDIR)/%.d
$(DEPDIR)/%.d: ;
-include $(DEPS)
To don't make this question too long discussing deeply why I think that all of the lines commented in the snippet above are innecesary, I will ask it in a short form; is there any difference in behaviour if I just change this snippet to:
SRCS := $(wildcard *.cpp */*.cpp)
DEPDIR := .d
DEPS := $(SRCS:%.cpp=$(DEPDIR)/%.d)
DEPFLAGS := -MT $# -MD -MP -MF $(DEPDIR)/$*.d
$(DEPDIR)/%:
mkdir -p $#
%.o: %.cpp | $(DEPDIR)/$(dir %)
g++ -o $(DEPFLAGS) $(CXXFLAGS) $(CPPFLAGS) -c $<
# touch $(DEPDIR)/$.d # ??
-include $(DEPS)
I have still two extra doubts; in the first link above, it says that
it’s been reported that some versions of GCC may leave the object file
older than the dependency file, which causes unnecessary rebuilds.
However, in the gist (second link above), the touch command is removed, and since the dependence file is no longer prerequisite of any rule, is there any reason to keep including it? Does this "gcc bug" still apply in any form?
The second doubt is related to the directory creation, moved to an order-only rule; do I need to make the "order-only" $(DEPDIR)/% rule .PRECIOUS? I don't know if make will attempt to remove the directory if the %.o recipe fails, because I don't know the concrete features of order-only rules.
You cannot remove the %.d prerequisite. The reason this is needed is explained in the page you linked.
I don't know what you mean by your comment Removal of implicit rules... ? Twice?. The removal of the implicit rule is needed, to ensure that our new implicit rule is used, and we only remove it once.
The temporary file .Td is used in case someone uses ^C or similar to kill their make job right in the middle of creating this file. By writing to a temporary file then only atomically replacing the real file once we know it's complete, we never have to worry about partial files which could cause the next invocation of make to generate an error or, worse, not recompile source files that should be recompiled.
Regarding the comment about object files older than the dependency file, first the gist you link to uses clang not GCC and perhaps Clang doesn't have this issue (or perhaps it does but people don't realize it). Second that update to the blog post is relatively recent as people have reported this issue to me with GCC. I've not seen it myself (I only use GCC) so maybe it's only an issue with some versions of GCC.
Regarding .PRECIOUS, make never (currently) recursively deletes directories so it won't delete any non-empty directory regardless of that setting.
I have a small project for testing and I want to link different implementations of a solution to test.
So, I create makefile that looks like this
CC=g++
FLAGS=-Wall -O2
CFLAGS=-c $(FLAGS)
I_PATH=implementation1.cpp
all: instance
instance: instance.cpp .implementation.o
$(CC) $(FLAGS) instance.cpp .implementation.o -o $#
.implementation.o: $(I_PATH)
$(CC) $(CFLAGS) $(I_PATH) -o $#
clean:
-rm .implementation*
-rm instance
Here, I_PATH is a path to solution implementation. And I want to test different solution passing different implementation via command line arguments: make I_PATH=implementation2.cpp.
But, because of all my implementations compiled to the same object file .implementation.o, make can't understand that something changes and doesn't rebuild project.
Of course, I can call make clean before run make for a specific implementation. But this increase build time (I can run tests for one implementation many times) and not very comfortable.
I can fix this makefile to something like this:
CC=g++
FLAGS=-Wall -O2
CFLAGS=-c $(FLAGS)
I_PATH=implementation1.cpp
C_PATH := $(shell echo -n $(I_PATH) | md5sum | awk '{print ".implementation_" substr($$1, 0, 10) ".o";}')
all: force instance
force_relink:
touch -c $(C_PATH)
instance: instance.cpp $(C_PATH)
$(CC) $(FLAGS) instance.cpp $(C_PATH) -o $#
$(C_PATH): $(I_PATH)
$(CC) $(CFLAGS) $< -o $#
clean:
-rm .implementation*
-rm instance
Here I create I_PATH dependent object file(take hash of path to implementation) and in addition force re-linking instance.cpp with object file every time make runs.
But maybe there is some mechanism in make to fix this behavior? Or I can achieve the same goal with different approaches?
Wouldn't it make more sense to give each compiled .o file a distinct name, and simply link to the compiled .o file you want?
instance: instance.cpp $(IMPL_O)
$(CC) $(FLAGS) $^ -o $# # propably no need to override default rule
Use make IMPL_O=implementation2.o to generate the example in your question.
This way, the name of each file truly reveals its identity, and you don't have to keep track of anything explicitly.
(Obviously, you could refactor the .o extension into the Makefile itself so you can just say IMPL=implementation2 or whatever.)
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
What do the $# and $< do exactly?
$# is the name of the target being generated, and $< the first prerequisite (usually a source file). You can find a list of all these special variables in the GNU Make manual.
For example, consider the following declaration:
all: library.cpp main.cpp
In this case:
$# evaluates to all
$< evaluates to library.cpp
$^ evaluates to library.cpp main.cpp
From Managing Projects with GNU Make, 3rd Edition, p. 16 (it's under GNU Free Documentation License):
Automatic variables are set by make after a rule is matched. They
provide access to elements from the target and prerequisite lists so
you don’t have to explicitly specify any filenames. They are very
useful for avoiding code duplication, but are critical when defining
more general pattern rules.
There are seven “core” automatic variables:
$#: The filename representing the target.
$%: The filename element of an archive member specification.
$<: The filename of the first prerequisite.
$?: The names of all prerequisites that are newer than the target,
separated by spaces.
$^: The filenames of all the prerequisites, separated by spaces. This
list has duplicate filenames removed since for most uses, such as
compiling, copying, etc., duplicates are not wanted.
$+: Similar to $^, this is the names of all the prerequisites separated
by spaces, except that $+ includes duplicates. This variable was
created for specific situations such as arguments to linkers where
duplicate values have meaning.
$*: The stem of the target filename. A stem is typically a filename
without its suffix. Its use outside of pattern rules is
discouraged.
In addition, each of the above variables has two variants for
compatibility with other makes. One variant returns only the directory
portion of the value. This is indicated by appending a “D” to the
symbol, $(#D), $(<D), etc. The other variant returns only the file
portion of the value. This is indicated by appending an “F” to the
symbol, $(#F), $(<F), etc. Note that these variant names are more than
one character long and so must be enclosed in parentheses. GNU make
provides a more readable alternative with the dir and notdir
functions.
The $# and $< are called automatic variables. The variable $# represents the name of the target and $< represents the first prerequisite required to create the output file.
For example:
hello.o: hello.c hello.h
gcc -c $< -o $#
Here, hello.o is the output file. This is what $# expands to. The first dependency is hello.c. That's what $< expands to.
The -c flag generates the .o file; see man gcc for a more detailed explanation. The -o specifies the output file to create.
For further details, you can read this article on linoxide about Linux Makefiles.
Also, you can check the GNU make manuals. It will make it easier to make Makefiles and to debug them.
If you run this command, it will output the makefile database:
make -p
The $# and $< are special macros.
Where:
$# is the file name of the target.
$< is the name of the first dependency.
The Makefile builds the hello executable if any one of main.cpp, hello.cpp, factorial.cpp changed. The smallest possible Makefile to achieve that specification could have been:
hello: main.cpp hello.cpp factorial.cpp
g++ -o hello main.cpp hello.cpp factorial.cpp
pro: very easy to read
con: maintenance nightmare, duplication of the C++ dependencies
con: efficiency problem, we recompile all C++ even if only one was changed
To improve on the above, we only compile those C++ files that were edited. Then, we just link the resultant object files together.
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
main.o: main.cpp
g++ -c main.cpp
hello.o: hello.cpp
g++ -c hello.cpp
factorial.o: factorial.cpp
g++ -c factorial.cpp
pro: fixes efficiency issue
con: new maintenance nightmare, potential typo on object files rules
To improve on this, we can replace all object file rules with a single .cpp.o rule:
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
.cpp.o:
g++ -c $< -o $#
pro: back to having a short makefile, somewhat easy to read
Here the .cpp.o rule defines how to build anyfile.o from anyfile.cpp.
$< matches to first dependency, in this case, anyfile.cpp
$# matches the target, in this case, anyfile.o.
The other changes present in the Makefile are:
Making it easier to changes compilers from g++ to any C++ compiler.
Making it easier to change the compiler options.
Making it easier to change the linker options.
Making it easier to change the C++ source files and output.
Added a default rule 'all' which acts as a quick check to ensure all your source files are present before an attempt to build your application is made.
in exemple if you want to compile sources but have objects in an different directory :
You need to do :
gcc -c -o <obj/1.o> <srcs/1.c> <obj/2.o> <srcs/2.c> ...
but with most of macros the result will be all objects followed by all sources, like :
gcc -c -o <all OBJ path> <all SRC path>
so this will not compile anything ^^ and you will not be able to put your objects files in a different dir :(
the solution is to use these special macros
$# $<
this will generate a .o file (obj/file.o) for each .c file in SRC (src/file.c)
$(OBJ):$(SRC)
gcc -c -o $# $< $(HEADERS) $(FLAGS)
it means :
$# = $(OBJ)
$< = $(SRC)
but lines by lines INSTEAD of all lines of OBJ followed by all lines of SRC
Hey I'm trying to build some files at the same time with different suffixes. Somehow it seems imposible to do this in one line. My makefile looks as follows:
ARCH=ar
ARCHFLAGS=r
F90=gfortran
F90FLAGS=-O2 -Wall
LDFLAGS=-llapack -lblas
SRCF=/Users/pm/bin/src
OBJF=/Users/pm/bin/objs
MODF=/Users/pm/bin/mods
LIBF=/Users/pm/bin/include
SOURCES=a.f b.f90 c.f90
OBJECTS=$(addprefix $(OBJF)/,$(addsuffix .o,$(basename $(SOURCES))))
MODULES=$(addprefix $(MODF)/,*.mod)
TARGET=lib_pm_math_lib.a
$(LIBF)/$(TARGET): $(OBJECTS)
$(ARCH) $(ARCHFLAGS) $# $(OBJECTS) $(MODULES)
obmod.clean :
rm $(OBJECTS) $(MODULES)
clean :
rm $(OBJECTS) $(MODULES) $(LIBF)/$(TARGET)
$(OBJECTS): $(OBJF)/%.o : $(addprefix $(SRCF)/,$(join %.,$(suffix $(SOURCES))))
$(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o$# -J$(MODF)
#$(OBJECTS): $(OBJF)/%.o : $(subst .x, ,$(addprefix $(SRCF)/,$(addsuffix .x$(suffix $(SOURCES)),%)))
# $(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o$# -J$(MODF)
#$(OBJECTS): $(OBJF)/%.o : $(SRCF)/%.f90
# $(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o $# -J$(MODF)
As you can see, I already managed to define my OBJECTS. But I'm not able to creat a construct that does the same for the building part of the object. Of course my first try was to use the join without the extra dot, but this results in only the suffix, for whatever reasons. Substituting the two dots with one dot does this as well. So I'm lost. The lines that I commanded out are another interesting try, and a working version for only .f90 suffixes. Actually I was hoping for something like the following to be working:
$(OBJECTS): $(OBJF)/%.o : $(SRCF)/%.*
$(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o $# -J$(MODF)
I hope it's not too messy. I posted the whole file since I bet you guy's see other problems which I didn't even think of so far. Thanks in advance!
If I'm understanding you correctly, you're wanting a suffix-based wildcard rule that you can run on multiple file suffixes. You can only have one wildcard per recipe, so there's no way to do it directly. You'll need separate rules for each suffix.
The easy solution is to copy-paste one rule and change the suffix. This can become unmanageable when you start to have a lot of suffixes. Another option is to create a rule template and use that to dynamically generate your rules for you:
# Template for build rules
# Pass a file extension for an argument
define build_rule
$(OBJF)/%.o: $(SRCF)/%.$(1)
$(F90) $(F90FLAGS) $(LDFLAGS) -c $$< -o$$# -J$(MODF)
endef
# Generate rules for each selected file extension
FILE_EXTS = f f90
$(foreach ext,$(FILE_EXTS),$(eval $(call build_rule,$(ext))))
This will dynamically generate a rule that differs only by the file extension used on the input file. To support a new file extension, simply add it to the FILE_EXTS list.
Note that when make initially parses the recipe template (inside call), it will expand variables. You have to double-up the $ in the template for anything that you don't want make to expand until the recipe is actually executed (like $# or $<).
You shouldn't need to do anything special to ensure that only the objects in the OBJECTS list are compiled. Since your default make target only lists $(OBJECTS) as a dependency, the files in $(OBJECTS) will be the only ones that get built.
In this case I'd probably just use two rules:
$(OBJF)/%.o: $(SRCF)/%.f
$(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o$# -J$(MODF)
$(OBJF)/%.o: $(SRCF)/%.f90
$(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o$# -J$(MODF)
You could combine them into one, but it's not really worth the effort.
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
What do the $# and $< do exactly?
$# is the name of the target being generated, and $< the first prerequisite (usually a source file). You can find a list of all these special variables in the GNU Make manual.
For example, consider the following declaration:
all: library.cpp main.cpp
In this case:
$# evaluates to all
$< evaluates to library.cpp
$^ evaluates to library.cpp main.cpp
From Managing Projects with GNU Make, 3rd Edition, p. 16 (it's under GNU Free Documentation License):
Automatic variables are set by make after a rule is matched. They
provide access to elements from the target and prerequisite lists so
you don’t have to explicitly specify any filenames. They are very
useful for avoiding code duplication, but are critical when defining
more general pattern rules.
There are seven “core” automatic variables:
$#: The filename representing the target.
$%: The filename element of an archive member specification.
$<: The filename of the first prerequisite.
$?: The names of all prerequisites that are newer than the target,
separated by spaces.
$^: The filenames of all the prerequisites, separated by spaces. This
list has duplicate filenames removed since for most uses, such as
compiling, copying, etc., duplicates are not wanted.
$+: Similar to $^, this is the names of all the prerequisites separated
by spaces, except that $+ includes duplicates. This variable was
created for specific situations such as arguments to linkers where
duplicate values have meaning.
$*: The stem of the target filename. A stem is typically a filename
without its suffix. Its use outside of pattern rules is
discouraged.
In addition, each of the above variables has two variants for
compatibility with other makes. One variant returns only the directory
portion of the value. This is indicated by appending a “D” to the
symbol, $(#D), $(<D), etc. The other variant returns only the file
portion of the value. This is indicated by appending an “F” to the
symbol, $(#F), $(<F), etc. Note that these variant names are more than
one character long and so must be enclosed in parentheses. GNU make
provides a more readable alternative with the dir and notdir
functions.
The $# and $< are called automatic variables. The variable $# represents the name of the target and $< represents the first prerequisite required to create the output file.
For example:
hello.o: hello.c hello.h
gcc -c $< -o $#
Here, hello.o is the output file. This is what $# expands to. The first dependency is hello.c. That's what $< expands to.
The -c flag generates the .o file; see man gcc for a more detailed explanation. The -o specifies the output file to create.
For further details, you can read this article on linoxide about Linux Makefiles.
Also, you can check the GNU make manuals. It will make it easier to make Makefiles and to debug them.
If you run this command, it will output the makefile database:
make -p
The $# and $< are special macros.
Where:
$# is the file name of the target.
$< is the name of the first dependency.
The Makefile builds the hello executable if any one of main.cpp, hello.cpp, factorial.cpp changed. The smallest possible Makefile to achieve that specification could have been:
hello: main.cpp hello.cpp factorial.cpp
g++ -o hello main.cpp hello.cpp factorial.cpp
pro: very easy to read
con: maintenance nightmare, duplication of the C++ dependencies
con: efficiency problem, we recompile all C++ even if only one was changed
To improve on the above, we only compile those C++ files that were edited. Then, we just link the resultant object files together.
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
main.o: main.cpp
g++ -c main.cpp
hello.o: hello.cpp
g++ -c hello.cpp
factorial.o: factorial.cpp
g++ -c factorial.cpp
pro: fixes efficiency issue
con: new maintenance nightmare, potential typo on object files rules
To improve on this, we can replace all object file rules with a single .cpp.o rule:
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
.cpp.o:
g++ -c $< -o $#
pro: back to having a short makefile, somewhat easy to read
Here the .cpp.o rule defines how to build anyfile.o from anyfile.cpp.
$< matches to first dependency, in this case, anyfile.cpp
$# matches the target, in this case, anyfile.o.
The other changes present in the Makefile are:
Making it easier to changes compilers from g++ to any C++ compiler.
Making it easier to change the compiler options.
Making it easier to change the linker options.
Making it easier to change the C++ source files and output.
Added a default rule 'all' which acts as a quick check to ensure all your source files are present before an attempt to build your application is made.
in exemple if you want to compile sources but have objects in an different directory :
You need to do :
gcc -c -o <obj/1.o> <srcs/1.c> <obj/2.o> <srcs/2.c> ...
but with most of macros the result will be all objects followed by all sources, like :
gcc -c -o <all OBJ path> <all SRC path>
so this will not compile anything ^^ and you will not be able to put your objects files in a different dir :(
the solution is to use these special macros
$# $<
this will generate a .o file (obj/file.o) for each .c file in SRC (src/file.c)
$(OBJ):$(SRC)
gcc -c -o $# $< $(HEADERS) $(FLAGS)
it means :
$# = $(OBJ)
$< = $(SRC)
but lines by lines INSTEAD of all lines of OBJ followed by all lines of SRC