I have set up my makefile like below, to minimize code duplication
The recipes are a set of blocks that set a variable, and then run the sleeper_agent recipe. They work great when called individually as make xlsx_sleeper for example.
But when I call all_sleepers, only the first one (xlsx_sleeper) gets compiled.
I have tried declaring them as phony (.PHONY: all_sleepers xlsx_sleeper docx_sleeper pptx_sleeper pdf_sleeper png_sleeper), which changes nothing
and adding a .FORCE rule to the sleeper_agent rule, which results in no such file or directory:
.FORCE:
sleeper_agent: .FORCE [...]
Here is my makefile:
all_sleepers: xlsx_sleeper docx_sleeper pptx_sleeper png_sleeper pdf_sleeper
sleeper_agent: $(OBJ)/sleeper_agent.o $(OBJ)/identities.o
windres icons/$(ext)/resource.rc -O coff -o obj/$(ext).res
$(CC) -o $(BIN)/sleeper_$(ext).exe $^ $(OBJ)/$(ext).res $(CFLAGS) $(LIBS)
xlsx_sleeper: ext=xlsx
xlsx_sleeper: sleeper_agent
docx_sleeper: ext=docx
docx_sleeper: sleeper_agent
pptx_sleeper: ext=pptx
pptx_sleeper: sleeper_agent
png_sleeper: ext=png
png_sleeper: sleeper_agent
pdf_sleeper: ext=pdf
pdf_sleeper: sleeper_agent
Your problem is that make does not see any reason why it should rebuild the sleeper_agent target several times. You should probably stick to the make philosophy:
Try to have real files as targets ($(BIN)/sleeper_xlsx.exe).
Use phony (non-file) targets only:
To give symbolic names to other targets or groups of targets (all_sleepers, xlsx_sleeper, ...)
For rules that don't produce files (clean, help...)
Declare phony targets as such (.PHONY: ...)
Example using static pattern rules, automatic variables and the patsubst make function:
SLEEPER := xlsx docx pptx png pdf
EXE := $(patsubst %,$(BIN)/sleeper_%.exe,$(SLEEPER))
SHORT := $(patsubst %,%_sleeper,$(SLEEPER))
.PHONY: all_sleepers $(SHORT) clean_sleepers
all_sleepers: $(EXE)
$(SHORT): %_sleeper: $(BIN)/sleeper_%.exe
$(EXE): $(BIN)/sleeper_%.exe: $(OBJ)/sleeper_agent.o $(OBJ)/identities.o
windres icons/$*/resource.rc -O coff -o obj/$*.res
$(CC) -o $# $^ $(OBJ)/$*.res $(CFLAGS) $(LIBS)
clean_sleepers:
rm -f $(EXE)
And then you should be able to run:
make all_sleepers
to build them all or:
make xlsx_sleeper
to build only one of them. EXE is the list of real executable files and a static pattern rule explains how to build them. In its recipe the $* automatic variable expands as the string matching the % wildcard. SHORT is the list of xxxx_sleeper shortcuts and another static pattern rule explains for each of them to which real executable it corresponds. all_sleepers and the xxxx_sleeper shortcuts (plus the clean_sleepers I added as example) are properly declared as phony because there are no such real files.
Related
I'm trying to understand how to handle header file dependencies in Make rules. Let me give you a specific example.
I'm building application called myap using GNU Make. It consists of various *.h and *.c files.
Directory inc/ contains defs.h and util.h header files.
Directory src/ contains main.c, cmd.c and win.c files.
Directory obj/ contains all generated object files.
I have multiple applications that need different build options. So I don't want to rely on any implicit rules and would like to specify my own rules for all object files, etc.
I would like to specify the following rules:
Object files depend on specific *.h and *.c files. If any of them change, all object files must be regenerated. However, even though *.h files are part of the prerequisites list, I don't want to pass them to the compiler. I only want to compile *.c files.
Executable myapp depends on specific *.o files. If any of them change, executable file must be regenerated.
So far, the following Makefile with a static pattern rule seems to work correctly:
myapp_inc := inc/defs.h inc/util.h
myapp_src := src/main.c src/cmd.c src/win.c
myapp_obj := $(patsubst src/%.c,obj/%.o,$(myapp_src))
myapp_bin := obj/myapp
.PHONY: all
all:
# Create obj/main.o obj/cmd.o and obj/win.o from various *.c files
# If any *.h files in $(myapp_inc) list change, all objects are regenerated.
# If any *.c files in $(myapp_src) list change, all objects are regenerated.
$(myapp_obj): obj/%.o: src/%.c $(myapp_inc) $(myapp_src)
gcc -c -o $# $<
# Create obj/myapp from various *.o files
# If any *.o files in $(myapp_obj) list change, executable is regenerated.
$(myapp_bin): $(myapp_obj)
gcc -o $# $^
all: $(myapp_bin)
.PHONY: clean
clean:
rm -f obj/*
I don't quite understand how Make rules should be written correctly in order to handle such use case. Is the above static pattern rule, the only way that works correctly?
Specifically, I have tried the following combinations, as given in various simple examples on the Internet, and they all failed for various reasons.
This rule causes $< to always pass the name of the first prerequisite, which doesn't work with multiple *.c files:
$(myapp_obj): $(myapp_src) $(myapp_inc)
gcc -c -o $# $<
$ make
gcc -c -o obj/main.o src/main.c
gcc -c -o obj/cmd.o src/main.c
gcc -c -o obj/win.o src/main.c
gcc -o obj/myapp obj/main.o obj/cmd.o obj/win.o
/bin/ld: obj/cmd.o: in function `main':
main.c:(.text+0x0): multiple definition of `main'; obj/main.o:main.c:(.text+0x0): first defined here
/bin/ld: obj/win.o: in function `main':
main.c:(.text+0x0): multiple definition of `main'; obj/main.o:main.c:(.text+0x0): first defined here
collect2: error: ld returned 1 exit status
make: *** [Makefile:18: obj/myapp] Error 1
This rule causes $^ to always pass the names of all prerequisites, which fails:
$(myapp_obj): $(myapp_src) $(myapp_inc)
gcc -c -o $# $^
$ make
gcc -c -o obj/main.o src/main.c src/cmd.c src/win.c inc/defs.h inc/util.h
gcc: fatal error: cannot specify ‘-o’ with ‘-c’, ‘-S’ or ‘-E’ with multiple files
compilation terminated.
make: *** [Makefile:13: obj/main.o] Error 1
Now I understand the difference between $< and $^ variables, but a lot of documentation is not clear on how they should be used when dealing with a list of multiple *.c and *.h files as prerequisites.
What are the recommended usage pattern for this?
Why is it that when using $< only *.c files get passed to the recipe, but not *.h files? Is Make doing some internal filtering? Is this documented anywhere? Is it possible to modify this behavior for custom suffixes?
Is the above static pattern rule, the only way to make objects depend on *.h and *.c files, but exclude *.h files during compilation?
I don't understand the goal of trying to avoid implicit rules. But in any event, it doesn't matter to the recipe you write whether the rule was implicit or explicit: the same automatic variables are set either way. The $< automatic variable is always the first prerequisite, so if you write your rules such that the first prerequisite is the appropriate .c file then you can always use $< in your recipe to mean the .c file and no other files. All the following will work:
%.o : %.c $(headers)
gcc -c -o $# $<
foo.o: foo.c $(headers)
gcc -c -o $# $<
foo.o : %.o : %.c $(headers)
gcc -c -o $# $<
%.o : %.c
gcc -c -o $# $<
$(srcs) : $(headers)
and others.
Does this mean that all of the prerequisites apply, but only those that match the pattern get passed to the recipe?
I don't understand the question, really. The value of variables and the expansion of the recipe happens only AFTER make has decided to run the rule and is not really related (except for some special automatic variables like $?). Once make has decided that the target is out of date and the recipe needs to be run, it will assign the appropriate automatic variables, expand the recipe, then pass the recipe to the shell to be run.
The automatic variables are assigned as described in the manual: $# is the target, $< is the first prerequisite, $^ is all the prerequisites, etc.
ETA
You still haven't really explained why you don't want to use static pattern rules. They are a perfectly fine and reasonable way to do things.
If you explain what you don't like about static pattern rules, or what you wish you could do differently, then we can probably suggest alternatives that meet those requirements.
Specifically, I have tried the following combinations, as given in various simple examples on the Internet,
$(myapp_obj): $(myapp_src) $(myapp_inc)
Wherever you found this as a recommended example on the Internet, you should immediately delete from any bookmarks as that site doesn't know anything about make.
We see this paradigm at least once a week on SO. I've never really understand why people think it will work: I guess they think make is much more "magical" than it is. Consider, what does the above expand to? Suppose myapp_obj contained foo.o bar.o biz.o and myapp_src contained foo.c bar.c biz.c and myapp_inc contained foo.h bar.h, then make sees:
foo.o bar.o biz.o: foo.c bar.c biz.c foo.h bar.h
I suppose some people think make will intuit that the ".o" files should somehow match up with the ".c" files and will generate a bunch of rules that make that true. That's not what make does. The above line is exactly identical to writing this:
foo.o: foo.c bar.c biz.c foo.h bar.h
bar.o: foo.c bar.c biz.c foo.h bar.h
biz.o: foo.c bar.c biz.c foo.h bar.h
That is, if you have multiple targets make creates one copy of the rule for each target, with the same prerequisites and recipe.
This is obviously not what you want, and that's why none of the examples that try to do things this way can ever work properly.
Why is it that when using $< only *.c files get passed to the recipe, but not *.h files? Is Make doing some internal filtering? Is this documented anywhere? Is it possible to modify this behavior for custom suffixes?
None of that is the case. As I described above, the $< expands to the first prerequisite. That's all. It doesn't matter whether the first prerequisite is a .c file, a .h file, or some other file; whatever it is, $< will be that value. If you write your rule as:
foo.o : foo.c foo.h ; $(CC) -c -o $# $<
then your compiler will be invoked with foo.c. If you write your rule as:
foo.o : foo.h foo.c ; $(CC) -c -o $# $<
then your compiler will be invoked with foo.h. There's no magic here.
In the post How can I configure my makefile for debug and release builds?, I find the answer https://stackoverflow.com/a/20830354/5922947 perfect!
But it seems work with only one final target.
I've tried unsuccessfully to extend this makefile to my projects where there are several executables as targets.
What changes should I make to this example so that I can have multiple final targets and with
make
or
make all
I can produce all targets (in debug or release modes), or
make test
for a specific target?
EDIT:
One of the attempts I made but which did not produce any results, but serves to better illustrate what is intended:
(...)
OBJS = $(SRCS:.c=.o)
EXE=$($(TARGET)_EXE)
sample1_EXE=sample1
sample2_EXE=sample2
sample1: TARGET=sample1
sample1: debug
sample2: TARGET=sample2
sample2: debug
all: prep debug sample1 sample2
The problem has two parts: specifying the mode, and coordinating the build.
(This will be tricky, so we will take it in stages.)
First the mode. Setting the value of mode in the command, and using debug as the default, is easy. We put this in the makefile:
mode = debug
And a command like make sailfish mode=release will override it.
Now to use the mode:
mode = debug
ifeq ($(mode),debug)
BUILDDIR := debug
CFLAGS += -g -O0 -DDEBUG
else
ifeq ($(mode),release)
BUILDDIR := release
CFLAGS += -O3 -DNDEBUG
else
$(error unknown mode: $(mode))
endif
endif
Note that I added the error statement to catch mistakes like make mode=test and make mode=releas. Such errors are much easier to catch and correct if they cause Make to abort on the spot. (Also note that the leading whitespace is not necessary, it has no effect on execution, but it makes the makefile easier to read.)
Now for the build. Suppose we execute make sailfish, so that the mode is debug. First notice that although we give sailfish as the target, we are not actually building sailfish, we are building debug/sailfish. This is important, so we make sailfish a PHONY target that requires the file we actually want:
.PHONY: sailfish
sailfish: $(BUILDDIR)/sailfish
The relevant objects are sailfish.o and seaThing.o. (You must compose this list yourself, it is almost impossible for Make to deduce it.) We could put the objects in the makefile as a distinct variable:
sailfish_OBJS := sailfish.o seaThing.o
but it will make things simpler later if we make this a target-specific variable:
$(BUILDDIR)/sailfish: $(addprefix $(BUILDDIR)/,sailfish.o seaThing.o)
$(CC) $(CFLAGS) -o $# $^
We still need a rule to build the object files, and we notice that the two pattern rules in the linked answer collapse into one:
$(BUILDDIR)/%.o: %.c
$(CC) -c $(CFLAGS) -o $# $<
This is enough to build debug/sailfish and release/sailfish. To add the executable catamaran, we add:
.PHONY: catamaran
catamaran: $(BUILDDIR)/catamaran
$(BUILDDIR)/catamaran: $(addprefix $(BUILDDIR)/,catamaran.o boatThing.o seaThing.o)
$(CC) $(CFLAGS) -o $# $^
But we notice some redundancy here, so we combine the PHONY declarations:
.PHONY: sailfish catamaran
Then we notice that if we put the names of the executables in a variable:
EXECUTABLES := sailfish catamaran
we can use it in the PHONY declaration:
.PHONY: $(EXECUTABLES)
and we can also combine the two PHONY rules into a static pattern rule:
$(EXECUTABLES): %: $(BUILDDIR)/%
and we can separate the prerequisite lines from the recipes and use one recipe for both:
$(BUILDDIR)/sailfish: $(addprefix $(BUILDDIR)/,sailfish.o seaThing.o)
$(BUILDDIR)/catamaran: $(addprefix $(BUILDDIR)/,catamaran.o boatThing.o seaThing.o)
$(addprefix $(BUILDDIR)/,$(EXECUTABLES)):
$(CC) $(CFLAGS) -o $# $^
Now, to add another executable like seagull, we must only add seagull to the EXECUTABLES := ... line, and write another line:
$(BUILDDIR)/seagull: $(addprefix $(BUILDDIR)/,seagull.o birdThing.o seaThing.o)
A few more refinements are possible, but this should be enough for now.
This is my current makefile
CFLAGS = -Iheaders/
CC = g++
PROGRAM_NAME = sportsmanager
rwildcard = $(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))
SOURCES = $(call rwildcard,sources/,*.cpp)
OFILES = $(call rwildcard,obj-tmp/,*.o)
OBJDIR = obj-tmp/
compileAndRun:
make -s compile && make -s $(PROGRAM_NAME)
./$(PROGRAM_NAME)
compile: $(SOURCES)
mkdir -p $(OBJDIR)
$(CC) $(CFLAGS) -c $(SOURCES) && mv *.o $(OBJDIR)
$(PROGRAM_NAME): $(OFILES)
$(CC) $(CFLAGS) $(OFILES) -o $(PROGRAM_NAME)
Whenever I run $ make, target compile is triggered which compiles all .cpp files in directory sources/ to .o files which are then moved to obj-tmp/. Then the target $(PROGRAM_NAME) is triggered, which links all the .o files and outputs the executable file.
The problem is that all files are compiled each time I run make. What ideally should happen if I run 'make' twice in succession is that make should know that the program is up to date the second time. If I modify only one file, only that file should be compiled.
Heads up: I know that there exists similar questions regarding this, but I've yet to see a solution which works in conjunction with the above makefile.
Any input is greatly appreciated.
The whole point of make is to compile only those files which have been modified since the last build. The problem in your makefile is that your compile recipe has the $(SOURCES) variable as a dependency. As in, all the source files.
I would use vpath to organize the project folder like so:
vpath %.cpp src
vpath %.h include
This will tell make to look for c++ files in ./src and header files in ./include. Then, you can simplify your recipe for individual files like this:
%.o: %.cpp
$(CC) $(CFLAGS) -c -o $# $<
Having done this, you can now define an $(OBJECTS) variable with a wildcard that matches .o files and continue from there. As an aside, moving your object files into a separate folder is considered bad practice and I agree; it really adds nothing substantial of value but complicates recipes.
Remember that object files represent a dependency for the $(PROGRAM) recipe. So naturally, make looks for the necessary object files to see if they need to be rebuilt. If they've been moved, one of two things happens. Either make will determine that they don't exist and will rebuild all the object files again from scratch, thereby invalidating the very reason we use make in the first place, or you'll have to define a folder where the object files will live, and every time you handle wildcards, searches, etc., literally anything that has to do with the object files, you'll have to take this added complexity into account.
I agree that having a ton of object files in the project folder can be a little annoying, but it definitely beats waiting forever for the project to compile. Just remember to add *.o to your .gitignore or whatever source control platform you use and they'll be nothing more than an eyesore, while make will be that much easier to use.
To answer your question on handling subdirectories in the source folder, the answer is a little more complicated.
Rather than using the specific vpath <pattern> <folder> directive as above, you could just outright use the VPATH variable like this:
VPATH = include src src/sub
This would handle the job, but the first method is usually preferred because when using VPATH, make searches every directory every time when looking for a file, rather than being location-constrained by file extension.
It is possible to use make to conveniently manage large projects though, and it involves calling make itself recursively, writing makefiles for each module in the build process. This process is obviously much more complicated, and I would strongly recommend considering whether the project genuinely necessitates this, as any potential gains in build-process modularization may not be recuperated due to the complexity involved in implementation.
I'd like to point you to this and this, both of which are phenomenal resources on makefiles.
Change the dependency of compile to be the object files.
Add a pattern rule for the object files.
compile: $(OFILES)
$(OBJDIR)/%.o: sources/%.cpp
mkdir -p $(OBJDIR)
$(CC) $(CFLAGS) -c $< -o $#
Ok, a lot of good input in this thread! Here's a follow up. I've now updated the script to the following:
CC = g++
CFLAGS = -Iheaders/
PROGRAM_NAME = sportsmanager
OFILES = $(patsubst %.cpp,%.o,$(wildcard sources/*.cpp))
vpath %.cpp sources
compileAndRun:
#make -s $(PROGRAM_NAME)
#./$(PROGRAM_NAME)
$(PROGRAM_NAME): $(OFILES)
$(CC) $(CFLAGS) -o $(PROGRAM_NAME) $(OFILES)
%.o: %.cpp
$(CC) $(CFLAGS) -c -o $# $<
clean:
rm -rf $(PROGRAM_NAME) $(OFILES)
Any suggestions for further improvements are very welcome!
I have a project structure like this
-Project
--Common
---types.f90
---global.f90
---common_routines.f90
--Program 1
---program1.f90
---module1.f90
---module2.f90
---etc...
--Program 2
--etc...
Where, Common is folder that contains some modules that are shared across all programs. How do I include this modules on my makefile?
I tried this:
FC = gfortran
FCFLAGS = -fbounds-check -O3
FCFLAGS += -I ../Common
all: program1
program1: module1.o module2.o module3.o
program1.o: module1.o module2.o module3.o
module2.o: module1.o
module3.o: module2.o module1.o
%: %.o
$(FC) $(FCFLAGS) -o $# $^
%.o: %.f90
$(FC) $(FCFLAGS) -c $<
clean:
rm -rf *.o *.mod
but I get an undefined reference error to the common modules variables.
I tried FCFLAGS += -I../Common types.o global.o common_routines.o
This will not work because -I is an option to the GNU Fortran preprocessor
to specify a path that the preprocessor shall search for files to be INCLUDE-ed
prior to compilation. You cannot use it to specify a path where object files (*.o)
will be searched for, after compilation, by the linker. It means nothing to
the linker and is not passed to the linker.
For simplicity let's assume that the object files you need to to link for
program1 are just program1/program1.o plus the pre-existing common/types.o,
common/global.o and common/common_routines.o
Then the following Makefile, placed in directory program1, will build it:
OBJS = program1.o ../common/types.o ../common/global.o ../common/common_routines.o
.phony: all
all: program1
program1: program1.o
$(FC) -o $# $(FCFLAGS) $(OBJS)
clean:
rm -f *.o program1
Just list all the required object files to the linker, in this case via $(OBJS)
You might wish to take the precaution of making sure that the common modules
are up to date before you build program1, and you now might think that you can do that
simply be replacing:
program1: program1.o
with:
program1: $(OBJS)
thus prompting make to recompile any of the four object files that is out
of date with respect to the corresponding source file, as a prerequisite of
building program1
make will certainly endeavour to do that, but take care. That way, it will
recompile, say, ../common/types.o from ../common/types.f90 just by its
implicit default recipe for making an .o from an .f90, since this makefile is
not telling it to do any different. But that may not be the way in which
../common/types.f90 is meant to be compiled, if you also have is a makefile in common
that stipulates how to do it in some non-default manner.
In that case, the common object files should always be compiled as per the
makefile in common. Better leave the prerequisites of program1 alone but change the recipe to:
program1: program1.o
$(MAKE) -C ../common
$(FC) -o $# $(FCFLAGS) $(OBJS)
Now, any time program1 needs to be rebuilt, the recipe will preemptively run make in ../common
before it links the four object files. (It's a small inelegance that this $(MAKE) -C ../common
will be invoked even if there's nothing for it to do: this is avoidable by more advanced make usage).
Lastly you might also find a need (if not in this case, then in another) to distinguish
between flags passed to preprocessing and/or flags passed to compilation and/or flags passed to linkage.
Conventionally, these are assigned to distinct make variables, e.g. FPPFLAGS (preprocessor),
FCFLAGS (compiler), LDFLAGS (linker).
The GNU syntax to define additional include directory is -Idir not -I dir (extra space)
Also make sure that common modules are already compiled and include search path points to the directory where you have compiled modules, not source files:
This path is also used to search for .mod files when previously compiled modules are required by a USE statement.
I have a directory in which I keep adding different C++ source files, and generic Makefile to compile them. This is the content of the Makefile:
.PHONY: all clean
CXXFLAGS = -pipe -Wall -Wextra -Weffc++ -pedantic -ggdb
SRCS = $(wildcard *.cxx)
OBJS = $(patsubst %.cxx,%.out,$(SRCS))
all: $(OBJS)
clean:
rm -fv $(OBJS)
%.out: %.cxx
$(CXX) $(CXXFLAGS) $^ -o $#
NOTE: As is obvious from above, I am using *.out for executable file extensions (and not for object file).
Also, there are some files which are compiled together:
g++ file_main.cxx file.cxx -o file_main.out
To compile such files, until now I have been adding explicit rules in the Makefile:
file_main.out: file_main.cxx file.cxx
file.out: file_main.out
#echo "Skipping $#"
But now my Makefile has a lot of explicit rules, and I would like to replace them with a simpler implicit rule.
Any idea how to do it?
First, this method of compiling several source files directly into an executable is not a terribly good idea. The more common compile-then-link approach will save a lot of unnecessary compilation.
That said, the way to replace many explicit rules with a simpler rule depends on what the explicit rules have in common. You already have a pattern rule:
%.out: %.cxx
$(CXX) $(CXXFLAGS) $^ -o $#
and if all you want to do is add another source file to a particular target, you don't have to do this:
g++ file_main.cxx file.cxx -o file_main.out
you can get the effect just by adding a prerequisite (in a line by itself):
file_main.out: file.cxx
If you have several targets with that pattern, you can use a pattern rule:
file_main.out another_main.out a_third_main.out: %_main.out : %.cxx
If you have many such targets, you can use a variable:
MAIN_THINGS = file another a_third a_fourth and_yet_another
MAIN_TARGETS = $(addsuffix _main.out, $(MAIN_THINGS))
$(MAIN_TARGETS): %_main.out : %.cxx
And you can add other patterns for other target sets, even overlapping sets. Does that cover your situation?
It seems that you are putting the source code for multiple different programs in the same folder, and this is really the source of your problems. If you separate the source code for your libraries and programs into separate folders (or, better yet, separate projects), then you can skirt this issue by depending on all source files in the given folder. When you have everything intermixed, it is necessary to be explicit.
That said, if your dependencies have consistent, predictable names, then it is possible to eliminate this redundancy by using the eval function. For example, based on the example above:
#
# I'm going to use standard file extensions here,
# slightly deviating from your conventions. I am also
# assuming that there is a variable named PROGNAMES,
# which gives a list of all the programs to be built.
#
define ADD_EXECUTABLE
$(1): $(1).o $(1)_main.o
$(LINK.cc) $(1).o $(1)_main.o -o $(1)
endef
$(foreach progname,$(PROGNAMES),$(eval $(call ADD_EXECUTABLE,$(progname))))
Also, just a few suggestions... you should append to CXXFLAGS rather than overwrite it and you would be better off using standard file extensions (".cpp" for C++ source files, ".o" for object files, no extension for executables). See my Makefile tutorial for tips on making things easier with Make (no pun intended).