Introduction
Many programs call a tree of subroutines, functions, and/or elements from modules. Suppose all such elements are contained in separate files, and some call others. I'm wondering about dependencies between these elements/files during the compilation process in the context of Makefiles. I'll use Fortran for example, but this question is at least somewhat language-agnostic.
Example
Suppose the program has the following dependencies:
MyProgram calls subroutineA and subroutineB.
subroutineA calls subroutineC.
subroutineB calls functionD and uses the parameter pi defined in module constants..
subroutineC uses the parameter pi defined in module constants.
functionD calls nothing.
(The constants module could be the one given here.) Each program, subroutine, function or module is defined in a file with a matching name.
The corresponding Makefile could be written in at least three ways, as illustrated below, and I'm wondering which is best. In the example Makefiles, I'm assuming the usual Makefile variables have been set earlier in the Makefile:
F90 is set to the appropriate Fortran compiler (e.g. FC=ifort)
FFLAGS is the set of compiler flags (e.g. FFLAGS=-O2)
Trivial Makefile
Everything gets compiled at once:
MyProgram: constants.f90 myprogram.f90 subroutineA.f90 subroutineB.f90 subroutineC.f90 functionD.f90
$(FC) $^ $(FFLAGS) -o $#
More "Standard" Makefile
Precompiles the subroutines and functions via a Makefile pattern rule but precompiles the module separately. At the end, it compiles/links them all together at the end.
%.o: %.f90
$(FC) $^ -c $(FFLAGS) -o $#
%.o %.mod: %.f90
$(FC) $^ -c $(FFLAGS)
MyProgram: constants.o constants.mod myprogram.o subroutineA.o subroutineB.o functionC.o
$(FC) $(filter %.o,$^) $(FFLAGS) -o $#
Explicit Makefile
Expresses each dependency explicitly. This gives make has a complete, directed acyclic graph of dependencies:
MyProgram: constants.o myprogram.o subroutineA.o subroutineB.o subroutineC.o functionD.o constants.mod
$(FC) $(filter %.o,$^ ) -o $#
myprogram.o: myprogram.f90 subroutineA.o subroutineB.o
$(FC) -c $< -o $#
subroutineA.o: subroutineA.f90 subroutineC.o
$(FC) -c $< -o $#
subroutineB.o: subroutineB.f90 functionD.o constants.o constants.mod
$(FC) -c $< -o $#
subroutineC.o: subroutineC.f90 constants.o constants.mod
$(FC) -c $< -o $#
functionD.o: functionD.f90
$(FC) -c $< -o $#
constants.o constants.mod: constants.f90
$(FC) -c $<
My Question
For my question, let's assume the Makefile has enough files and/or is used by enough different developers to warrant using a Makefile over manual compilation, but not so many files that writing out dependencies explicitly is unreasonable or impossible.
Is it important, good practice, or necessary to write out explicit dependencies for these things in a Makefile? Under what circumstances might it be best to use the "Explicit Makefile" approach as opposed to the "More Standard Makefile"?
Also, does the answer to that question depend on whether or not modules are involved?
Related
I have a need to generate a few object files with different load addresses. I also have a make target for invoking objdump to show me the respective disassembly.
code0:
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE0_RELOC) $#.S -o $#.o
objcopy -O binary $#.o --only-section=.text $#.bin
code1:
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE1_RELOC) $#.S -o $#.o
objcopy -O binary $#.o --only-section=.text $#.bin
dump: dump_code0 dump_code1
dump_code0: code0.bin
objdump $< -D > $<.decomp
dump_code1: code1.bin
objdump $< -D > $<.decomp
This will fail because there is no target for code.bin, but target dump_code depends on code.bin being present. To prevent it from being confusing (i.e. using dependency code0 such that code0.bin is then produced) I thought it would be easier to create a %.bin target that will create the respective binary file associated with each object file (since there is also duplication of this code across all code* targets anyway. This target will then be a dependency for each code* target. However, this necessitates (or at least I think it does) the need for dynamically assigning dependencies. I tried something like this that did not work:
%.bin: $(basename $#)
objcopy -O binary $(basename $#).o --only-section=.text $#.bin
code0: code0.bin
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE0_RELOC) $#.S -o $#.o
With the logic being that a dependency such as code0.bin would cause this target to execute with the single effective dependency of code0 to first build the object file.
I may be misunderstanding how the % character may be used to wildcard make targets. The first usesof basename was evaluating as empty (the second worked), so the target had no dependencies. It does make sense that the dependency tree be evaluated once at invocation, but I was hoping for a more dynamic capability.
Can this be done in Make?
What you are looking for is probably Static Pattern Rules with which you could rework your first attempt:
code0:
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE0_RELOC) $#.S -o $#.o
objcopy -O binary $#.o --only-section=.text $#.bin
as:
code0.o code1.o: code%.o: code%.S
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE$*_RELOC) $< -o $#
code0.bin code1.bin: %.bin: %.o
objcopy -O binary $< --only-section=.text $#
See the principle? $* in the recipe of the first rule is substituted by the matching stem of the static pattern rule (the digit in our case). This works but pattern-specific variables would probably be easier to understand and maintain:
# Default code reloc option
CODE_RELOC := default_code_reloc_option
# code1-specific code reloc option, if different from default
code1.o: CODE_RELOC := code1_code_reloc_option
code0.o code1.o: %.o: %.S
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE_RELOC) $< -o $#
The last part of your first makefile could also use static pattern rules, plus phony targets. Something like:
.PHONY: dump dump_code0 dump_code1
dump: dump_code0 dump_code1
dump_code0 dump_code1: dump_%: %.bin.decomp
code0.bin.decomp code1.bin.decomp: %.bin.decomp: %.bin
objdump $< -D > $#
Finally, we can use some more make tricks (discovery of source files, pattern substitutions) to automate a bit further:
CODES := $(wildcard *.S)
OBJS := $(patsubst %.S,%.o,$(CODES))
BINS := $(patsubst %.S,%.bin,$(CODES))
DECS := $(patsubst %.S,%.bin.decomp,$(CODES))
DUMPS := $(patsubst %.S,dump_%,$(CODES))
# Default code reloc option
CODE_RELOC := default_code_reloc_option
# code1-specific code reloc option, if different from default
code1.o: CODE_RELOC := code1_code_reloc_option
.PHONY: dump $(DUMPS)
dump: $(DUMPS)
$(DUMPS): dump_%: %.bin.decomp
$(DECS): %.bin.decomp: %.bin
objdump $< -D > $#
$(BINS): %.bin: %.o
objcopy -O binary $< --only-section=.text $#
$(OBJS): %.o: %.S
gcc -m32 $(CFLAGS) -Wl,$(LDFLAGS),-Ttext=$(CODE_RELOC) $< -o $#
.PHONY: clean
clean:
rm -f $(OBJS) $(BINS) $(DECS)
The clean target is a bonus gift.
All this has a significant advantage over what you were trying to do with targets that were not real files. Here, files depend on other files, except the phony targets that are just a kind of shorthand for real target files. And this is 100% in line with make's philosophy: express inter-files dependences such that make compares files timestamps and decides what is up-to-date and what must be rebuilt. Significant savings when a lot of targets are already up-to-date.
A second advantage is that expressing all inter-files dependences is parallel safe. If you run make on this makefile with:
make -j 8
(assuming you have about 8 cores on your computer), you can expect a speed-up factor of 8. Not much if you have only two source files, but quite interesting if you have hundreds of them...
With the following makefile snippet:
main: main.o f1.o f2.o
$(CC) $(CFLAGS) -o program main.o f1.o f2.o
main.o: main.cc
$(CC) $(CFLAGS) -c main.cc
f1.o: f1.cc
$(CC) $(CFLAGS) -c f1.cc
f2.o: f2.cc
$(CC) $(CFLAGS) -c f2.cc
If I just change one file, only that file get recompiled when I rerun make, as desired. However, I'm having a hard time generalizing this without having to list each file individually. When I try something like:
$(OBJECTS): $(SOURCES)
$(CC) $(CFLAGS) -o $# -c $(patsubst %.o,%.cc,$#)
It builds each object file individually, but each object file depends on ALL my sources, so a change in any one file causing a full recompile. What's a good way to accomplish this?
Basically,
you do have to list each .o file's dependencies individually.
For example, each .o is likely to depend on a different bunch of headers.
Taking your f1.o, you need something like:
f1.o: include/i.h
f1.o: another.h dir/and-another.h
f1.o: f1.cc
$(CC) $(CFLAGS) -c f1.cc
(you can have as many dependency lines for a target as you like).
Maintaining that list is a nightmare.
Broken dependency lists render your Makefile worse than useless—you might as well use a batch file.
All is not lost!
If you are tidy,
you can get the compiler to do it automatically,
and pretty much for free.
Makes your Makefile tidier to boot.
Win Win.
As Ismail Badawi commented, pattern rules provide a nice solution. They are a type of implicit rule for make. Basically, implicit rules are automatic recipes based off the file extension. For example, make knows how to convert .c files into .o files implicitly. By default make will run the following recipe for .c files (see the rule catalogue):
$(CC) $(CPPFLAGS) $(CFLAGS) -c
You can modify the process either by setting the variables CC, CPPFLAGS, and CFLAGS, or by defining a pattern rule:
%.o: %.c
$(CC) $(CFLAGS) -c $<
The "$<" above matches the name of the first prerequisite, which will be the .c file in this example. See Beta's comment and automatic variables.
Condition 0:
Say, I have several source codes, a.c, b.c, ..., z.c, and I want a rule to have each of them compiled. Here is a solution:
%.o: %.c
$(CC) -c -o $# $(CFLAGS) $<
Condition 1:
Then I introduce a header c.h used in c.c, and another header e.h used in c.c and e.c, and things become complex:
%.o: %.c
$(CC) -c -o $# $(CFLAGS) $<
c.o: c.c c.h e.h
$(CC) -c -o $# $(CFLAGS) $<
e.o: e.c e.h
$(CC) -c -o $# $(CFLAGS) $<
My question:
Based on the solution of condition 1, is there something like add_dependency in make to simplify the solution and obtain something like the following one?
%.o: %.c
$(CC) -c -o $# $(CFLAGS) $<
add_dependency(c.o, c.h e.h)
add_dependency(e.o, e.h)
Or, what do you think is a better solution to condition 1?
EDITED:
Thanks for the kind notice #ctheo :)
Yes I did have a look at autotools and understood that shall satisfy all my needs. However what I'm dealing with is an existing project and its Makefile contains other directives dealing with codes in C++, and I think for now I'd better just modify a few lines instead of port the whole Makefile to autotools, unless I couldn't find a satisfying solution without introducing autotools. :)
At first I did not expected to exist a solution for this. It seemed to me that it was covered by autotools. However, after some search, I found this section of GNU/make manual.
It states that :
One file can be the target of several rules. All the prerequisites mentioned in all the rules are merged into one list of prerequisites for the target.
So there is a solution for your query
c.o: c.h e.h
e.o: e.h
%.o: %.c
$(CC) -c -o $# $(CFLAGS) $<
Thanks for insisting. I learned something today :)
In addition, the .o files in your example all depend on a .h file with the same stem, so you can generalise that part of your rules too:
c.o: e.h
%.o: %.c %.h
$(CC) -c -o $# $(CFLAGS) $<
This way, your “normal” situations are covered entirely by the rule that triggers compilation and your “unusual” situations stand out because those are the only additional rules.
Hey I'm trying to build some files at the same time with different suffixes. Somehow it seems imposible to do this in one line. My makefile looks as follows:
ARCH=ar
ARCHFLAGS=r
F90=gfortran
F90FLAGS=-O2 -Wall
LDFLAGS=-llapack -lblas
SRCF=/Users/pm/bin/src
OBJF=/Users/pm/bin/objs
MODF=/Users/pm/bin/mods
LIBF=/Users/pm/bin/include
SOURCES=a.f b.f90 c.f90
OBJECTS=$(addprefix $(OBJF)/,$(addsuffix .o,$(basename $(SOURCES))))
MODULES=$(addprefix $(MODF)/,*.mod)
TARGET=lib_pm_math_lib.a
$(LIBF)/$(TARGET): $(OBJECTS)
$(ARCH) $(ARCHFLAGS) $# $(OBJECTS) $(MODULES)
obmod.clean :
rm $(OBJECTS) $(MODULES)
clean :
rm $(OBJECTS) $(MODULES) $(LIBF)/$(TARGET)
$(OBJECTS): $(OBJF)/%.o : $(addprefix $(SRCF)/,$(join %.,$(suffix $(SOURCES))))
$(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o$# -J$(MODF)
#$(OBJECTS): $(OBJF)/%.o : $(subst .x, ,$(addprefix $(SRCF)/,$(addsuffix .x$(suffix $(SOURCES)),%)))
# $(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o$# -J$(MODF)
#$(OBJECTS): $(OBJF)/%.o : $(SRCF)/%.f90
# $(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o $# -J$(MODF)
As you can see, I already managed to define my OBJECTS. But I'm not able to creat a construct that does the same for the building part of the object. Of course my first try was to use the join without the extra dot, but this results in only the suffix, for whatever reasons. Substituting the two dots with one dot does this as well. So I'm lost. The lines that I commanded out are another interesting try, and a working version for only .f90 suffixes. Actually I was hoping for something like the following to be working:
$(OBJECTS): $(OBJF)/%.o : $(SRCF)/%.*
$(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o $# -J$(MODF)
I hope it's not too messy. I posted the whole file since I bet you guy's see other problems which I didn't even think of so far. Thanks in advance!
If I'm understanding you correctly, you're wanting a suffix-based wildcard rule that you can run on multiple file suffixes. You can only have one wildcard per recipe, so there's no way to do it directly. You'll need separate rules for each suffix.
The easy solution is to copy-paste one rule and change the suffix. This can become unmanageable when you start to have a lot of suffixes. Another option is to create a rule template and use that to dynamically generate your rules for you:
# Template for build rules
# Pass a file extension for an argument
define build_rule
$(OBJF)/%.o: $(SRCF)/%.$(1)
$(F90) $(F90FLAGS) $(LDFLAGS) -c $$< -o$$# -J$(MODF)
endef
# Generate rules for each selected file extension
FILE_EXTS = f f90
$(foreach ext,$(FILE_EXTS),$(eval $(call build_rule,$(ext))))
This will dynamically generate a rule that differs only by the file extension used on the input file. To support a new file extension, simply add it to the FILE_EXTS list.
Note that when make initially parses the recipe template (inside call), it will expand variables. You have to double-up the $ in the template for anything that you don't want make to expand until the recipe is actually executed (like $# or $<).
You shouldn't need to do anything special to ensure that only the objects in the OBJECTS list are compiled. Since your default make target only lists $(OBJECTS) as a dependency, the files in $(OBJECTS) will be the only ones that get built.
In this case I'd probably just use two rules:
$(OBJF)/%.o: $(SRCF)/%.f
$(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o$# -J$(MODF)
$(OBJF)/%.o: $(SRCF)/%.f90
$(F90) $(F90FLAGS) $(LDFLAGS) -c $< -o$# -J$(MODF)
You could combine them into one, but it's not really worth the effort.
Here's a shortened version of the Makefile I am fighting with on a Linux system:
VPATH=altsrc:src:obj
OBJECTS=\
nondepcode1.o \
nondepcode2.o \
nondepcode3.o \
depcode1.o \
depcode2.o \
depcode3.o
DEP_OBJS= depcode1.o depcode2.o depcode3.o
# Targets
execute: $(OBJECTS)
gfortran -o $# $^ $(LFLAGS)
$(DEP_OBJS): npts.h
obj/%.o: %.f
$(FORTRAN) $(FFLAGS) $< -o $#
obj/%.o: %.f90
$(FORTRAN) $(FFLAGS) $< -o $#
# Rules
.f.o:
$(FORTRAN) $(FFLAGS) -o obj/$# $<
%.o: %.f90
$(FORTRAN) $(FFLAGS) -o obj/$# $<
I was expecting that the three objects that depend on "npts.h" would be automatically updated on a run of make (or gmake) if npts.h was more recent than any of the objects. This just does not happen. Make (and gmake) thinks the objects are up to date. AFAICT, I am doing things the way they are described in the GNU make manual. Anyone have any idea why make/gmake is not doing what I expected? Thanks. BTW, there are tabs at the beginning of all the recipe lines in the actual Makefile. They went away here.
Major egg on my face. As I said, this was a shortened version of the Makefile. I found the bug in the real Makefile. Had some misnamed macros. Not too experienced with the finer points of this stuff. Mea culpa. Very sorry. Thanks for checking.