I am new to linux development.
I wrote a project using MPI and cuda. When
it gets bigger and bigger, I realize that I
need a Makefile now. So I learned how to write
one. The Makefile works, but will only compile
cpp files even if I have both of the following
lines in my Makefile:
.cpp.o:
$(CC) $(CCFLAGS) $<
.cu.o:
$(NVCC) $(CCFLAGS) $<
Any idea why this is happening? Thanks.
UNDERSTANDING MAKE
Make is all about generating missing files.
If you have TWO rules that generate the SAME file upon existence of a source then the first one in make's list that actually has a source file present will get invoked. So for instance if you have the rules:
.c.o:
$(CC) -o $# -c $<
.cpp.o:
$(CXX) -o $# -c $<
and you have two files, foo.c and bar.cpp then you can type:
$ make foo.o
it will use the first rule... and when you type
$ make bar.o
it will use the second rule.
Now suppose you have TWO files foo.c and foo.cpp
Here make has to make a choice as to which takes precedence. Make uses suffixes of files intimately for its build rules. What is considered a suffix is controlled by the .SUFFIXES directive.
The .SUFFIXES directive has a default built-in value that defines common suffixes such as .c .cpp .cc .o etc. in a particular order. If we want to change the order of precedence we clear that out with a blank line in Makefile i.e.:
.SUFFIXES:
and then follow it with our definition:
.SUFFIXES: .cpp .c .o
if you don't blank the line out, then make just appends the listed suffixes to its current list, that way multiple makefiles can simply add new suffixes without worrying about breaking each other.
Now since the .cpp is before .c the .cpp.o rule will take precedence (in case foo.cpp and foo.c are both present)
NOTE: Yes there is a "." before the words SUFFIXES and yes it is all capital letters.
Try to play with this Makefile to see the effects:
.SUFFIXES:
.SUFFIXES: .cpp .c .o
.c.o:
echo Compiling C
.cpp.o:
echo Compiling CPP
Make is very very powerful, and quite well documented so well worth the read. GNU make, which is probably the strongest implementation with amazing extensions has made me a lot of money in the past :-) enjoy the experience.
Your rule is wrong, you want something like this:
%.o : %.cu
$(NVCC) $(CCFLAGS) $< -o $#
That's assuming the command line you need to execute is something like
nvcc foo.cu -o foo.o
Otherwise, edit to suit.
Related
I'm trying to understand how to handle header file dependencies in Make rules. Let me give you a specific example.
I'm building application called myap using GNU Make. It consists of various *.h and *.c files.
Directory inc/ contains defs.h and util.h header files.
Directory src/ contains main.c, cmd.c and win.c files.
Directory obj/ contains all generated object files.
I have multiple applications that need different build options. So I don't want to rely on any implicit rules and would like to specify my own rules for all object files, etc.
I would like to specify the following rules:
Object files depend on specific *.h and *.c files. If any of them change, all object files must be regenerated. However, even though *.h files are part of the prerequisites list, I don't want to pass them to the compiler. I only want to compile *.c files.
Executable myapp depends on specific *.o files. If any of them change, executable file must be regenerated.
So far, the following Makefile with a static pattern rule seems to work correctly:
myapp_inc := inc/defs.h inc/util.h
myapp_src := src/main.c src/cmd.c src/win.c
myapp_obj := $(patsubst src/%.c,obj/%.o,$(myapp_src))
myapp_bin := obj/myapp
.PHONY: all
all:
# Create obj/main.o obj/cmd.o and obj/win.o from various *.c files
# If any *.h files in $(myapp_inc) list change, all objects are regenerated.
# If any *.c files in $(myapp_src) list change, all objects are regenerated.
$(myapp_obj): obj/%.o: src/%.c $(myapp_inc) $(myapp_src)
gcc -c -o $# $<
# Create obj/myapp from various *.o files
# If any *.o files in $(myapp_obj) list change, executable is regenerated.
$(myapp_bin): $(myapp_obj)
gcc -o $# $^
all: $(myapp_bin)
.PHONY: clean
clean:
rm -f obj/*
I don't quite understand how Make rules should be written correctly in order to handle such use case. Is the above static pattern rule, the only way that works correctly?
Specifically, I have tried the following combinations, as given in various simple examples on the Internet, and they all failed for various reasons.
This rule causes $< to always pass the name of the first prerequisite, which doesn't work with multiple *.c files:
$(myapp_obj): $(myapp_src) $(myapp_inc)
gcc -c -o $# $<
$ make
gcc -c -o obj/main.o src/main.c
gcc -c -o obj/cmd.o src/main.c
gcc -c -o obj/win.o src/main.c
gcc -o obj/myapp obj/main.o obj/cmd.o obj/win.o
/bin/ld: obj/cmd.o: in function `main':
main.c:(.text+0x0): multiple definition of `main'; obj/main.o:main.c:(.text+0x0): first defined here
/bin/ld: obj/win.o: in function `main':
main.c:(.text+0x0): multiple definition of `main'; obj/main.o:main.c:(.text+0x0): first defined here
collect2: error: ld returned 1 exit status
make: *** [Makefile:18: obj/myapp] Error 1
This rule causes $^ to always pass the names of all prerequisites, which fails:
$(myapp_obj): $(myapp_src) $(myapp_inc)
gcc -c -o $# $^
$ make
gcc -c -o obj/main.o src/main.c src/cmd.c src/win.c inc/defs.h inc/util.h
gcc: fatal error: cannot specify ‘-o’ with ‘-c’, ‘-S’ or ‘-E’ with multiple files
compilation terminated.
make: *** [Makefile:13: obj/main.o] Error 1
Now I understand the difference between $< and $^ variables, but a lot of documentation is not clear on how they should be used when dealing with a list of multiple *.c and *.h files as prerequisites.
What are the recommended usage pattern for this?
Why is it that when using $< only *.c files get passed to the recipe, but not *.h files? Is Make doing some internal filtering? Is this documented anywhere? Is it possible to modify this behavior for custom suffixes?
Is the above static pattern rule, the only way to make objects depend on *.h and *.c files, but exclude *.h files during compilation?
I don't understand the goal of trying to avoid implicit rules. But in any event, it doesn't matter to the recipe you write whether the rule was implicit or explicit: the same automatic variables are set either way. The $< automatic variable is always the first prerequisite, so if you write your rules such that the first prerequisite is the appropriate .c file then you can always use $< in your recipe to mean the .c file and no other files. All the following will work:
%.o : %.c $(headers)
gcc -c -o $# $<
foo.o: foo.c $(headers)
gcc -c -o $# $<
foo.o : %.o : %.c $(headers)
gcc -c -o $# $<
%.o : %.c
gcc -c -o $# $<
$(srcs) : $(headers)
and others.
Does this mean that all of the prerequisites apply, but only those that match the pattern get passed to the recipe?
I don't understand the question, really. The value of variables and the expansion of the recipe happens only AFTER make has decided to run the rule and is not really related (except for some special automatic variables like $?). Once make has decided that the target is out of date and the recipe needs to be run, it will assign the appropriate automatic variables, expand the recipe, then pass the recipe to the shell to be run.
The automatic variables are assigned as described in the manual: $# is the target, $< is the first prerequisite, $^ is all the prerequisites, etc.
ETA
You still haven't really explained why you don't want to use static pattern rules. They are a perfectly fine and reasonable way to do things.
If you explain what you don't like about static pattern rules, or what you wish you could do differently, then we can probably suggest alternatives that meet those requirements.
Specifically, I have tried the following combinations, as given in various simple examples on the Internet,
$(myapp_obj): $(myapp_src) $(myapp_inc)
Wherever you found this as a recommended example on the Internet, you should immediately delete from any bookmarks as that site doesn't know anything about make.
We see this paradigm at least once a week on SO. I've never really understand why people think it will work: I guess they think make is much more "magical" than it is. Consider, what does the above expand to? Suppose myapp_obj contained foo.o bar.o biz.o and myapp_src contained foo.c bar.c biz.c and myapp_inc contained foo.h bar.h, then make sees:
foo.o bar.o biz.o: foo.c bar.c biz.c foo.h bar.h
I suppose some people think make will intuit that the ".o" files should somehow match up with the ".c" files and will generate a bunch of rules that make that true. That's not what make does. The above line is exactly identical to writing this:
foo.o: foo.c bar.c biz.c foo.h bar.h
bar.o: foo.c bar.c biz.c foo.h bar.h
biz.o: foo.c bar.c biz.c foo.h bar.h
That is, if you have multiple targets make creates one copy of the rule for each target, with the same prerequisites and recipe.
This is obviously not what you want, and that's why none of the examples that try to do things this way can ever work properly.
Why is it that when using $< only *.c files get passed to the recipe, but not *.h files? Is Make doing some internal filtering? Is this documented anywhere? Is it possible to modify this behavior for custom suffixes?
None of that is the case. As I described above, the $< expands to the first prerequisite. That's all. It doesn't matter whether the first prerequisite is a .c file, a .h file, or some other file; whatever it is, $< will be that value. If you write your rule as:
foo.o : foo.c foo.h ; $(CC) -c -o $# $<
then your compiler will be invoked with foo.c. If you write your rule as:
foo.o : foo.h foo.c ; $(CC) -c -o $# $<
then your compiler will be invoked with foo.h. There's no magic here.
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
What do the $# and $< do exactly?
$# is the name of the target being generated, and $< the first prerequisite (usually a source file). You can find a list of all these special variables in the GNU Make manual.
For example, consider the following declaration:
all: library.cpp main.cpp
In this case:
$# evaluates to all
$< evaluates to library.cpp
$^ evaluates to library.cpp main.cpp
From Managing Projects with GNU Make, 3rd Edition, p. 16 (it's under GNU Free Documentation License):
Automatic variables are set by make after a rule is matched. They
provide access to elements from the target and prerequisite lists so
you don’t have to explicitly specify any filenames. They are very
useful for avoiding code duplication, but are critical when defining
more general pattern rules.
There are seven “core” automatic variables:
$#: The filename representing the target.
$%: The filename element of an archive member specification.
$<: The filename of the first prerequisite.
$?: The names of all prerequisites that are newer than the target,
separated by spaces.
$^: The filenames of all the prerequisites, separated by spaces. This
list has duplicate filenames removed since for most uses, such as
compiling, copying, etc., duplicates are not wanted.
$+: Similar to $^, this is the names of all the prerequisites separated
by spaces, except that $+ includes duplicates. This variable was
created for specific situations such as arguments to linkers where
duplicate values have meaning.
$*: The stem of the target filename. A stem is typically a filename
without its suffix. Its use outside of pattern rules is
discouraged.
In addition, each of the above variables has two variants for
compatibility with other makes. One variant returns only the directory
portion of the value. This is indicated by appending a “D” to the
symbol, $(#D), $(<D), etc. The other variant returns only the file
portion of the value. This is indicated by appending an “F” to the
symbol, $(#F), $(<F), etc. Note that these variant names are more than
one character long and so must be enclosed in parentheses. GNU make
provides a more readable alternative with the dir and notdir
functions.
The $# and $< are called automatic variables. The variable $# represents the name of the target and $< represents the first prerequisite required to create the output file.
For example:
hello.o: hello.c hello.h
gcc -c $< -o $#
Here, hello.o is the output file. This is what $# expands to. The first dependency is hello.c. That's what $< expands to.
The -c flag generates the .o file; see man gcc for a more detailed explanation. The -o specifies the output file to create.
For further details, you can read this article on linoxide about Linux Makefiles.
Also, you can check the GNU make manuals. It will make it easier to make Makefiles and to debug them.
If you run this command, it will output the makefile database:
make -p
The $# and $< are special macros.
Where:
$# is the file name of the target.
$< is the name of the first dependency.
The Makefile builds the hello executable if any one of main.cpp, hello.cpp, factorial.cpp changed. The smallest possible Makefile to achieve that specification could have been:
hello: main.cpp hello.cpp factorial.cpp
g++ -o hello main.cpp hello.cpp factorial.cpp
pro: very easy to read
con: maintenance nightmare, duplication of the C++ dependencies
con: efficiency problem, we recompile all C++ even if only one was changed
To improve on the above, we only compile those C++ files that were edited. Then, we just link the resultant object files together.
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
main.o: main.cpp
g++ -c main.cpp
hello.o: hello.cpp
g++ -c hello.cpp
factorial.o: factorial.cpp
g++ -c factorial.cpp
pro: fixes efficiency issue
con: new maintenance nightmare, potential typo on object files rules
To improve on this, we can replace all object file rules with a single .cpp.o rule:
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
.cpp.o:
g++ -c $< -o $#
pro: back to having a short makefile, somewhat easy to read
Here the .cpp.o rule defines how to build anyfile.o from anyfile.cpp.
$< matches to first dependency, in this case, anyfile.cpp
$# matches the target, in this case, anyfile.o.
The other changes present in the Makefile are:
Making it easier to changes compilers from g++ to any C++ compiler.
Making it easier to change the compiler options.
Making it easier to change the linker options.
Making it easier to change the C++ source files and output.
Added a default rule 'all' which acts as a quick check to ensure all your source files are present before an attempt to build your application is made.
in exemple if you want to compile sources but have objects in an different directory :
You need to do :
gcc -c -o <obj/1.o> <srcs/1.c> <obj/2.o> <srcs/2.c> ...
but with most of macros the result will be all objects followed by all sources, like :
gcc -c -o <all OBJ path> <all SRC path>
so this will not compile anything ^^ and you will not be able to put your objects files in a different dir :(
the solution is to use these special macros
$# $<
this will generate a .o file (obj/file.o) for each .c file in SRC (src/file.c)
$(OBJ):$(SRC)
gcc -c -o $# $< $(HEADERS) $(FLAGS)
it means :
$# = $(OBJ)
$< = $(SRC)
but lines by lines INSTEAD of all lines of OBJ followed by all lines of SRC
I have a c++ project with various extensions for the source files (.cpp, .c, .cc) and various extensions for the header files (.hpp, .h, .hh). The source files are located in a directory called SRC, and the header files are predictably in a directory called INC.
I would like to compile the source with a rule like
vpath %.c $(SRC)
%.o: %.c
$(COMPILER) $(FLAGS) $< $(INCFLAG)$(INC)
This of course works if I know the source file will be of the form %.c, but in the case of multiple possible file extensions, I would need to build a similar rule for %.cpp and %.cc as well. Of course three rules isn't a big deal to write, but it would be nice to be able to use this makefile as a drag and drop for any project, even in a different language, without having to re-write the rules.
So how can I write a rule (or some other construct that accomplishes the same goal) that works like:
SRC_EXT = cpp c cc
vpath %.$(SRC_EXT) $(SRC)
%.o: %.$(SRC_EXT)
$(COMPILER) $(FLAGS) $< $(INCFLAG)$(INC)
Thanks for your help.
You can't in standard POSIX make. However since you mention vpath I'll assume you're using GNU make. If you have a sufficiently new version (3.81 or newer), you can do this easily enough with call and eval:
SRC_EXT = cpp c cc
define compile_rule
%.o : %.$1
$$(COMPILER) $$(FLAGS) $$< $$(INCFLAG)$$(INC)
endef
$(foreach EXT,$(SRC_EXT),$(eval $(call compile_rule,$(EXT))))
If you don't have sufficiently new GNU make, or would prefer an alternate solution, you can do the same thing with generated makefiles.
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
What do the $# and $< do exactly?
$# is the name of the target being generated, and $< the first prerequisite (usually a source file). You can find a list of all these special variables in the GNU Make manual.
For example, consider the following declaration:
all: library.cpp main.cpp
In this case:
$# evaluates to all
$< evaluates to library.cpp
$^ evaluates to library.cpp main.cpp
From Managing Projects with GNU Make, 3rd Edition, p. 16 (it's under GNU Free Documentation License):
Automatic variables are set by make after a rule is matched. They
provide access to elements from the target and prerequisite lists so
you don’t have to explicitly specify any filenames. They are very
useful for avoiding code duplication, but are critical when defining
more general pattern rules.
There are seven “core” automatic variables:
$#: The filename representing the target.
$%: The filename element of an archive member specification.
$<: The filename of the first prerequisite.
$?: The names of all prerequisites that are newer than the target,
separated by spaces.
$^: The filenames of all the prerequisites, separated by spaces. This
list has duplicate filenames removed since for most uses, such as
compiling, copying, etc., duplicates are not wanted.
$+: Similar to $^, this is the names of all the prerequisites separated
by spaces, except that $+ includes duplicates. This variable was
created for specific situations such as arguments to linkers where
duplicate values have meaning.
$*: The stem of the target filename. A stem is typically a filename
without its suffix. Its use outside of pattern rules is
discouraged.
In addition, each of the above variables has two variants for
compatibility with other makes. One variant returns only the directory
portion of the value. This is indicated by appending a “D” to the
symbol, $(#D), $(<D), etc. The other variant returns only the file
portion of the value. This is indicated by appending an “F” to the
symbol, $(#F), $(<F), etc. Note that these variant names are more than
one character long and so must be enclosed in parentheses. GNU make
provides a more readable alternative with the dir and notdir
functions.
The $# and $< are called automatic variables. The variable $# represents the name of the target and $< represents the first prerequisite required to create the output file.
For example:
hello.o: hello.c hello.h
gcc -c $< -o $#
Here, hello.o is the output file. This is what $# expands to. The first dependency is hello.c. That's what $< expands to.
The -c flag generates the .o file; see man gcc for a more detailed explanation. The -o specifies the output file to create.
For further details, you can read this article on linoxide about Linux Makefiles.
Also, you can check the GNU make manuals. It will make it easier to make Makefiles and to debug them.
If you run this command, it will output the makefile database:
make -p
The $# and $< are special macros.
Where:
$# is the file name of the target.
$< is the name of the first dependency.
The Makefile builds the hello executable if any one of main.cpp, hello.cpp, factorial.cpp changed. The smallest possible Makefile to achieve that specification could have been:
hello: main.cpp hello.cpp factorial.cpp
g++ -o hello main.cpp hello.cpp factorial.cpp
pro: very easy to read
con: maintenance nightmare, duplication of the C++ dependencies
con: efficiency problem, we recompile all C++ even if only one was changed
To improve on the above, we only compile those C++ files that were edited. Then, we just link the resultant object files together.
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
main.o: main.cpp
g++ -c main.cpp
hello.o: hello.cpp
g++ -c hello.cpp
factorial.o: factorial.cpp
g++ -c factorial.cpp
pro: fixes efficiency issue
con: new maintenance nightmare, potential typo on object files rules
To improve on this, we can replace all object file rules with a single .cpp.o rule:
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
.cpp.o:
g++ -c $< -o $#
pro: back to having a short makefile, somewhat easy to read
Here the .cpp.o rule defines how to build anyfile.o from anyfile.cpp.
$< matches to first dependency, in this case, anyfile.cpp
$# matches the target, in this case, anyfile.o.
The other changes present in the Makefile are:
Making it easier to changes compilers from g++ to any C++ compiler.
Making it easier to change the compiler options.
Making it easier to change the linker options.
Making it easier to change the C++ source files and output.
Added a default rule 'all' which acts as a quick check to ensure all your source files are present before an attempt to build your application is made.
in exemple if you want to compile sources but have objects in an different directory :
You need to do :
gcc -c -o <obj/1.o> <srcs/1.c> <obj/2.o> <srcs/2.c> ...
but with most of macros the result will be all objects followed by all sources, like :
gcc -c -o <all OBJ path> <all SRC path>
so this will not compile anything ^^ and you will not be able to put your objects files in a different dir :(
the solution is to use these special macros
$# $<
this will generate a .o file (obj/file.o) for each .c file in SRC (src/file.c)
$(OBJ):$(SRC)
gcc -c -o $# $< $(HEADERS) $(FLAGS)
it means :
$# = $(OBJ)
$< = $(SRC)
but lines by lines INSTEAD of all lines of OBJ followed by all lines of SRC
I have a directory in which I keep adding different C++ source files, and generic Makefile to compile them. This is the content of the Makefile:
.PHONY: all clean
CXXFLAGS = -pipe -Wall -Wextra -Weffc++ -pedantic -ggdb
SRCS = $(wildcard *.cxx)
OBJS = $(patsubst %.cxx,%.out,$(SRCS))
all: $(OBJS)
clean:
rm -fv $(OBJS)
%.out: %.cxx
$(CXX) $(CXXFLAGS) $^ -o $#
NOTE: As is obvious from above, I am using *.out for executable file extensions (and not for object file).
Also, there are some files which are compiled together:
g++ file_main.cxx file.cxx -o file_main.out
To compile such files, until now I have been adding explicit rules in the Makefile:
file_main.out: file_main.cxx file.cxx
file.out: file_main.out
#echo "Skipping $#"
But now my Makefile has a lot of explicit rules, and I would like to replace them with a simpler implicit rule.
Any idea how to do it?
First, this method of compiling several source files directly into an executable is not a terribly good idea. The more common compile-then-link approach will save a lot of unnecessary compilation.
That said, the way to replace many explicit rules with a simpler rule depends on what the explicit rules have in common. You already have a pattern rule:
%.out: %.cxx
$(CXX) $(CXXFLAGS) $^ -o $#
and if all you want to do is add another source file to a particular target, you don't have to do this:
g++ file_main.cxx file.cxx -o file_main.out
you can get the effect just by adding a prerequisite (in a line by itself):
file_main.out: file.cxx
If you have several targets with that pattern, you can use a pattern rule:
file_main.out another_main.out a_third_main.out: %_main.out : %.cxx
If you have many such targets, you can use a variable:
MAIN_THINGS = file another a_third a_fourth and_yet_another
MAIN_TARGETS = $(addsuffix _main.out, $(MAIN_THINGS))
$(MAIN_TARGETS): %_main.out : %.cxx
And you can add other patterns for other target sets, even overlapping sets. Does that cover your situation?
It seems that you are putting the source code for multiple different programs in the same folder, and this is really the source of your problems. If you separate the source code for your libraries and programs into separate folders (or, better yet, separate projects), then you can skirt this issue by depending on all source files in the given folder. When you have everything intermixed, it is necessary to be explicit.
That said, if your dependencies have consistent, predictable names, then it is possible to eliminate this redundancy by using the eval function. For example, based on the example above:
#
# I'm going to use standard file extensions here,
# slightly deviating from your conventions. I am also
# assuming that there is a variable named PROGNAMES,
# which gives a list of all the programs to be built.
#
define ADD_EXECUTABLE
$(1): $(1).o $(1)_main.o
$(LINK.cc) $(1).o $(1)_main.o -o $(1)
endef
$(foreach progname,$(PROGNAMES),$(eval $(call ADD_EXECUTABLE,$(progname))))
Also, just a few suggestions... you should append to CXXFLAGS rather than overwrite it and you would be better off using standard file extensions (".cpp" for C++ source files, ".o" for object files, no extension for executables). See my Makefile tutorial for tips on making things easier with Make (no pun intended).