I have a make file with a number of phony targets, they all compile the same code just with different compilation flags.
EXECUTABLE=ecis
#debug build
.PHONY: debug
debug: FLAGS=-g
debug: $(EXECUTABLE)
#No optimization
.PHONY: opt0
opt0: FLAGS=
opt0: $(EXECUTABLE)
#level 1 optimization
.PHONY: opt1
opt1: FLAGS=-O1
opt1: $(EXECUTABLE)
#level 2 optimization
.PHONY: opt2
opt2: FLAGS=-O2
opt2: $(EXECUTABLE)
...
$(EXECUTABLE):$(FORTRAN_OBJECTS) $(CPP_OBJECTS)
$(CPP) $(FLAGS) $(FORTRAN_OBJECTS) $(CPP_OBJECTS) -lgfortran -o $#
...
When I first run the makefile with one build make opt2 option it runs just fine.
If I subsequently want to run make with another build option say make debug it claims that the target is up to date. I understand why it is doing this, make doesn't realize that the flags changed and so as far as make is concerned nothing has changed if the files haven't changed.
That said, is there an easy way around it other than calling make cleanall (which deletes the .o files and the executable)? Is there some way for make to recognize the different flags as changing the compilation? Are there any other paths to have it "do the right thing"?
Here's an example that should work:
.compile_flags: Makefile
[ "`cat $#`" = '$(FLAGS)' ] || echo '$(FLAGS)' > $#
$(FORTRAN_OBJECTS) $(CPP_OBJECTS) $(EXECUTABLE): .compile_flags
An alternative is to generate the object files and executable into a separate subdirectory for each different base target. Then they won't overlap. This has the added benefit that you don't have to recompile the world for each type of build (only since the last time you did that build). But it uses more disk space and may cause other issues if you have other parts of the system expecting things to live where they do now.
To put things in other directories can't be done with target-specific variables. The simplest way to do it is to use one instance of recursive make instead, like this:
EXECUTABLE=ecis
#debug build
.PHONY: debug
debug: FLAGS=-g
#No optimization
.PHONY: opt0
opt0: FLAGS=
#level 1 optimization
.PHONY: opt1
opt1: FLAGS=-O1
#level 2 optimization
.PHONY: opt2
opt2: FLAGS=-O2
debug opt0 opt1 opt2:
$(MAKE) OUTDIR=obj_$# FLAGS=$(FLAGS) obj_$#/$(EXECUTABLE)
...
FORTRAN_OBJECTS := $(addprefix $(OUTDIR)/,$(FORTRAN_OBJECTS))
CPP_OBJECTS := $(addprefix $(OUTDIR)/,$(CPP_OBJECTS))
$(OUTDIR)/$(EXECUTABLE):$(FORTRAN_OBJECTS) $(CPP_OBJECTS)
$(CPP) $(FLAGS) $(FORTRAN_OBJECTS) $(CPP_OBJECTS) -lgfortran -o $#
...
And, you'll have to create pattern rules for your object files like:
$(OUTDIR)/%.o : %.cpp
...
and ditto for FORTRAN.
If you're limited to GNU make itself, you'll have trouble achieving your goal. As #MadScientist suggested, you can basically get what you want with some shenanigans. John Graham-Cumming wrote up a good explanation in his old "Ask Mr. Make" column: Rebuilding when CPPFLAGS Changes.
If you can use other make implementations, you might check out Electric Make, a reimplementation of GNU make designed for performance and reliability. It includes a feature called "ledger" which provides precisely this functionality.
Disclaimer: I'm the architect and lead developer of Electric Make
Related
The automatic variables created by GNU-Make are quite handy for some scenarios:
.SECONDEXPANSION:
%-processed.md: $$(wildcard $$*.m4) $(wildcard macros/*.m4) %.md
m4 $^ > $#
However this idilic world blows up when somebody tries to add on a dependency somewhere else in the makefile:
.PHONY: force
force: ;
mybook-processed.md: force
Now all of a sudden m4 complains that "force" isn't a valid input file. Which of course it's not, but its showing up in the $^ variable because it got appended to the list of prerequisites.
Is there a way to access only the prerequisites defined in the immediate recipe definition while ignoring other ones?
No it's not possible. That information is not even available in make's internal structures.
Your example solution is not one I'd use though. If your command only accepts certain types of files, I would use filter to ensure that it sees only those types of files:
.SECONDEXPANSION:
%-processed.md: $$(wildcard $$*.m4) $(wildcard macros/*.m4) %.md
m4 $(filter %.m4 %.md,$^) > $#
You could adopt a naming convention for pseudo-targets like force, say MAGIC/force. Then $(filter-out MAGIC/%,$^) -- still a little repetitive, but not that bad.
So far the only way I've found to do this is stop using automatic variables and compose my own. I'm really hoping somebody posites a better answer, but here's my current solution:
.SECONDEXPANSION:
process_prereqs = $(wildcard $1.m4) $(wildcarad macros/*.m4)
%-processed.md: %.md $$(call process_prereqs,$$*)
m4 $(call process_prereqs,$$*) $< > $#
This way only the more stable $< automatic variable is used and the other is more predicable, but the code is somewhat repetitive.
I have some ancillary targets in a makefile that I want to restrict for internal or "private" use (only) inside the makefile. That is, I want to be able to specify these targets as dependencies from within the makefile, but I want to prevent the target from being specified as a build goal from the command line. Somewhat analogous to a private function from OOP: the target is harmful (or simply doesn't make sense) to build separately.
I wish there were a special-target .HIDDEN or .PRIVATE or something that did this, akin to what .PHONY does for non-file targets, but I don't think this exists. The private keyword is only for variables.
What is a good/general/elegant way to protect a target for internal/private use only?
The best workaround that I could come up with is to check $(MAKECMDGOALS) for "unacceptable" targets, then error-out if specified; this seems inelegant. I'm sure the makefile could be rewritten to avoid this situation -- perhaps a superior solution -- but that's not practical here.
Below the cut-line... here's a contrived example for illustration.
Though I'm looking for a general solution, one example of targets that are harmful as individual/primary goal is with inheriting of target-specific variable values:
override CFLAGS += -Wall
all : debug
%.o : %.c
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $# $<
debug : CFLAGS += -g3 -O0
release : CFLAGS += -O3
debug : CPPFLAGS += -DDEBUG
release : CPPFLAGS += -DRELEASE
debug release : foo.o bar.o main.o
$(CC) -o $# $^ $(LDFLAGS) $(LDLIBS)
clean:
-rm -f *.o debug release
.PHONY: all clean
Implicit rule duplicated (unnecessary) for illustration. With the goal of debug or release, foo.o and others will inherit respective CFLAGS and CPPFLAGS -- If one does make clean debug all objects will be consistent. But for example if someone builds foo.o separately, it will fail to inherit the appropriate flags; e.g., make clean foo.o debug you'll get foo.o built with default CFLAGS; then it doesn't need to be updated when building debug, so it will be linked with other objects with different optimizations or different macro settings. It will probably work in this case, but it's not what was intended. Marking foo.o, etc. as illegal goals would prevent this.
EDIT:
It's very clear that my example (above) was not a good choice for my more-general question: hiding targets was not the best way to fix an issue with my example. Here's a modified example that illustrates the modified question "How to enforce target-specific values?" -- it builds on commentary from #Michael, #Beta, #Ross below, and allows posing and answering this more limited scenario.
As described in previous responses below, it's a much better idea in this case to create objects that have different build flags in separate locations. e.g.,
bin_debug/%.o : %.c
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $# $<
bin_release/%.o : %.c
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $# $<
OBJS = foo.o bar.o main.o # or $(SRCS:.o=.c)
DEBUG_OBJS = $(addprefix bin_debug/,$OBJS)
RELEASE_OBJS = $(addprefix bin_release/,$OBJS)
debug : $(DEBUG_OBJS)
release : $(RELEASE_OBJS)
debug release :
$(CC) -o $# $^ $(LDFLAGS) $(LDLIBS)
Pattern rule duplicated because I think it has to be (multiple "pattern targets" (%) convince make all targets are built at once with one recipe; see SO questions this and this).
So now, add in target-specific flags:
debug : CPPFLAGS += -DDEBUG
release : CPPFLAGS += -DRELEASE
But this still suffers:
make bin_debug/foo.o
will not get the CPPFLAGS from debug. I've accepted #Michael's answer below as it got me thinking about the problem in a more helpful way, but also answered some of my own rhetorical questions below.
You kind of can define private targets by starting their name with two hyphens.
--private-target:
#echo private
public-target: --private-target
#echo public
You can call make public-target but make --private-target will complain about an unknown option:
$ make public-target
private
public
$ make --private-target
/Library/Developer/CommandLineTools/usr/bin/make: unrecognized option `--private-target'
This is not a feature of make, but takes advantage of the fact that command line options are passed with two hyphens and as a result make will complain about an unknown option. This also can be easily bypassed by signaling the end of options:
$ make -- --private-target
private
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin11.3.0
The problem you are trying to solve is legitimate but you are heading on the worse possible path to solve it.
Declaring private targets does not make any sense
When we write a Makefile, we are describing a compilation job in terms of targets, sources and recipes. The advancement of this job is described by the set of targets which are already built. Now you are accurately observing that the sequence
make clean
make foo.o
make debug
will produce objects whose format is inconsistent with foo.o thus leaving your build directory in an inconsistent state. But it is very wrong to deduce that the user should not be able to construct foo.o explicitly. Consider the following sequence:
make clean
# Wait for foo.o being compiles and
# interrupt the build job with a signal
make debug
Since make sees that foo.o it will resume its task where it was at and left foo.o untouched while compiling subsequent units with different flags, leaving the build directory the same inconsistent state as in the first scenario.
Hence, if we could implement private targets in Makefiles, this would be ineffective and could convey a false sense of security, which is even worse than insecurity by itself. Also the solution you imagined annihilates one of the most important advantages of using Makefiles over shell scripts: Make makes it easy to continue an interrupted task where it was at.
I documented some other aspects of using Makefiles in relation to the set of targets already built in my answer to the question “What is the purpose of linking object files separately in a Makefile?”.
Another solution to your problem
To address the issue of compilation flags inconsistency, we can arrange to store built targets into a special directory, depending on the compilation flags used. Implementing this would fix the issue without forcing us to resign upon the ease of resuming an interrupted compilation job.
Here is an implementation roadmap:
Identify build profiles, here you have release and build.
Choose which compilation to use for each build profile.
Choose in which directory to store built targets for each build profile.
Write your Makefile so that built targets are stored in the directories you choosed. Please refer Gnu make - how to get object files in separate subdirectory.
Note. In my opinion, the BSD variant of make has a much nicer support for writing targets in a special directory, see my answer to the question “How to write a Makefile using different directories for targets and sources”. Generally I prefer the BSD variant of make because its documentation is short and to the point and it enjoys a lot of useful advanced examples, since operating system build and ports build in the BSD world are orchestrated by this program.
One solution to the problem is to migrate the CPPFLAGS to the pattern rules (e.g., bin_debug/%.o: CPPFLAGS...) instead of the regular rule (debug: CPPFLAGS...), final result:
bin_debug/%.o : CPPFLAGS += -DDEBUG
bin_debug/%.o : %.c
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $# $<
bin_release/%.o : CPPFLAGS += -DRELEASE
bin_release/%.o : %.c
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $# $<
OBJS = foo.o bar.o main.o # or $(SRCS:.o=.c)
DEBUG_OBJS = $(addprefix bin_debug/,$OBJS)
RELEASE_OBJS = $(addprefix bin_release/,$OBJS)
debug : $(DEBUG_OBJS)
release : $(RELEASE_OBJS)
debug release :
$(CC) -o $# $^ $(LDFLAGS) $(LDLIBS)
so make bin_debug/foo.o will get CPPFLAGS including -DDEBUG.
Now, lets say you have >>2 rules: debug, release, config01, config02, config03, ... each with their own CPPFLAGS.
One way might be to continue reduplicating all of the pattern rules, but that gets annoying if anything has to change. Furthermore it's not really possible to use in a foreach. This seems handy:
debug : CPPFLAGS+=-DDEBUG
release : CPPFLAGS+=-DRELEASE
config01 : CPPFLAGS+=-DSOMETHING
config02 : CPPFLAGS+=-DSOMETHINGELSE
TARGETS = debug release config01 config02
OBJS = foo.o bar.o main.o # or $(SRCS:.o=.c)
define TARGET_template
bin_$(1)/%.o : %.c
$$(CC) $$(CFLAGS) $$(CPPFLAGS) -c -o $# $<
$(1): $(addprefix bin_$(1)/,$(OBJS))
# other TARGET-specific stuff here
endef
$(foreach tgt,$(TARGETS),$(eval $(call TARGET_template,$(tgt))))
But still doesn't fix the situation of make bin_debug/foo.o -- still doesn't get CPPFLAGS.
So, instead of making target-specific variable-value like debug: CPPFLAGS+=... you could have a variable that is specific to the target, like CPPFLAGS_debug, then add to each rule:
CPPFLAGS_debug = -DDEBUG
CPPFLAGS_release = -DRELEASE
CPPFLAGS_config01 = -DSOMETHING
CPPFLAGS_config02 = -DSOMETHINGELSE
TARGETS = debug release config01 config02
OBJS = foo.o bar.o main.o # or $(SRCS:.o=.c)
define TARGET_template
bin_$(1)/%.o : CPPFLAGS+=$$(CPPFLAGS_$(1))
bin_$(1)/%.o : %.c
$$(CC) $$(CFLAGS) $$(CPPFLAGS) -c -o $$# $$<
$(1): $(addprefix bin_$(1)/,$(OBJS))
# other TARGET-specific stuff here
endef
$(foreach tgt,$(TARGETS),$(eval $(call TARGET_template,$(tgt))))
Beware; above may need more $$(...)s, untested.
Problems? Better way?
Thinking about this and tried the following:
TEST := $(shell echo $$RANDOM)
test : $(TEST)
$(TEST):
<tab>#echo tada $(TEST)
then doing a make test on command line seems to work and I think it would be difficult to get the result without using the test target. Maybe this path can help?
I don't think there's any "elegant" way to have targets somehow made private. I think the only solution that could be called elegant would be to rewrite your makefile so that it doesn't matter what target users invoke, as Beta suggests. It would also have the advantage of making your makefile more maintainable and easier to understand.
A not so elegant but fairly simple way to make targets "private" would be to rename the makefile to something other than one of the default names. Then put a new makefile in it's place that invokes the "private" makefile to do it's work. Something like:
.SUFFIXES:
PUBLIC_TARGETS = all debug release clean
REAL_MAKEFILE = private.mak
define invoke_make
$(1): $(REAL_MAKEFILE)
$(MAKE) -f $(REAL_MAKEFILE) $(1)
endef
$(foreach target, $(PUBLIC_TARGETS), $(eval $(call invoke_make,$(target))))
.PHONY: $(PUBLIC_TARGETS)
Obviously this doesn't prevent a determined user from invoking "private" targets, but hopefully it makes it clear that they shouldn't be doing this. That's all making things private in object-oriented languages does anyways. It's always possible for a sufficiently determined user to bypass it.
Even if previous speakers called this a bad idea, I was very interested in the concept of having a custom special target like .PRIVATE to more or less protect some targets from beeing called straigt.
And for everyone interested in it... this is what I came up with:
ifeq ($(strip $(filter .PRIVATE,$(MAKECMDGOALS))),)
__PRIVATEGOALS = $(shell make -f $(firstword $(MAKEFILE_LIST)) -n .PRIVATE | tail -n 1)
$(foreach __privgoal,$(__PRIVATEGOALS),$(eval __PRIVATECMDGOALS += $(filter $(__privgoal),$(MAKECMDGOALS))))
endif
ifneq ($(strip $(__PRIVATECMDGOALS)),)
$(error tried to call *private* goal(s) $(strip $(__PRIVATECMDGOALS)))
endif
.PHONY: .PRIVATE
.SILENT: .PRIVATE
.PRIVATE:
##
$^
Put it at the top of your makefile, or at least in front of the first target declared as private. You could as well put it into a separate file, like private.mk and include it in your main makefile.
You should be able to use the .PRIVATE target in the same way as you use the .SILENT or the .PHONY targets. An error is triggered in case a "private" target is called and make stops.
I have found the following code from http://mrbook.org/tutorials/make/:
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
My question is: Do we really need the line
all: $(SOURCES) $(EXECUTABLE)
?
What will be wrong if I do not keep the line or does it have some other purpose?
From the perspective of the behavior of make
The line:
all: $(SOURCES) $(EXECUTABLE)
is special in this case as it is the first target defined in the Makefile. Since there is no .DEFAULT_GOAL defined that makes it the default goal and so what make will aim to build if no target is specified on the command line.
In this specific case all its presence is serving it to allow you to use the command make all.
It is also ensuring that make all and make will not work as expected if there is a file called all present as make will consider that when evaluating the rule.
The normal practice is to define such targets as phony so that they will not be effected by the presence of a file with their name.
For example: .PHONY : all.
The reasons it does not serve any other purpose in this example are:
$(SOURCES) is a redundant dependency as $(EXECUTABLE) depends on it already through $(OBJECTS) which depends on $(SOURCES) through the suffix rule .cpp.o:
The other dependence $(EXECUTABLE) is the next target defined, so removing the all target will make $(EXECUTABLE) the default goal.
From the perspective of convention
The all target is a Standard Target and "should be the default target".
Some users and tools may assume the target all is present.
For example the IDE eclipse by default expects Makefiles to have the target all which it will use for incremental builds. Without the all target extra configuration is required for eclipse be able to build the project.
When you run make without specifying a target, it will run the first target in the file. It's a convention that this target is responsible for creating a deployable file (package or executable) for installation on your platform. The name all is simply a convention (at least in the GNU project), and could be anything you like (for example $(name), compile or deployable).
I don't know why the $(SOURCES) variable appears as a prerequisite of the all target. That's pretty useless.
In your case, since you have only one executable, it's not necessary to have all. But many makefiles built multiple "final targets" which do not depend on each other. In that case it's traditional to have an all target which depends on them all so you can build them all with one command.
And since it's traditional, people put it into makefiles with only one target as well, for consistency.
Your small C/C++ project has reached a point where it's no longer practical to have all your code in one file. You want to split out a few components. So you make a src/ directory, and then... you have to write a real Makefile. Something more than hello: hello.o. Uh-oh... was it $# or $< or $^? Crap. You don't remember (I never do).
Do you have a 'one-size fits all' simple Makefile that can deal with straightforward source trees? If so, what's in it and why? I'm looking for the smallest, simplest Makefile that can compile a directory full of C files nicely without me having to edit the Makefile every time I add a file. Here's what I have so far:
CXX = clang++
CXXFLAGS = ...
LDFLAGS = ...
EXENAME = main
SRCS = $(wildcard src/*.cc)
OBJS = $(patsubst src%.cc,build%.o, $(SRCS))
all: $(EXENAME)
build/%.o: src/%.cc
#mkdir -p $(dir $#)
$(CXX) -c -o $# $^ $(CXXFLAGS)
$(EXENAME): $(OBJS)
$(CXX) -o $# $^ $(LDFLAGS)
clean:
rm -rf $(EXENAME) build/
This Makefile builds all the .cc files in the src/ directory into .o files in the build/ directory, then links them up into the parent directory.
What would you do differently?
I would reconsider you decision not to have an explicit list of sources-- I think it may cause you trouble in the long run. But if that's your decision, this makefile is pretty good.
In the %.o rule I would use $< instead of $^, so that later you can add dependencies like
build/foo.o: bar.h
And when you're ready, you can take a look at Advanced Auto-Dependency Generation.
I've never used CMake, so I really can't say anything about that. The best that I can offer is a program that we have at school called 'makemake', which automatically makes Makefiles - http://www.cs.rit.edu/~swm/makemake/ It's not a very advanced program, but it gets the job done. On the plus side, it's incredibly easy to use - simply do 'makemake > Makefile' in the directory and you have a Makefile which will build and link all the source files in that directory(C and C++). On the bright side, if you ever add more files, you just run makemake again and you have a new makefile. On the downside, there's no way to keep any custom targets that you've done from one generated makefile to the next.
As for 'one size fits all' makefiles, while you could definitely do that, it takes away from the purpose of the 'make' command in the first place - which is to keep track of the files last modified time, and thus only re-compile the files that have recently changed, or depend on header files that have just changed(although to generate the correct you can use 'makedepend' - http://www.x.org/archive/X11R7.5/doc/man/man1/makedepend.1.html ). You could use what you currently have plus makedepend in order to make a self-updating makefile.
Use automake tools. Its easy to make changes and less burden to the developer. Its as simple as specifying the SOURCES, LDLIBS, LDFLAGS as variables. At first it may seem like a bit weird. But it becomes your favorite as you do more on it.
Let's say you have a Makefile with two pseudo-targets, 'all' and 'debug'. The 'debug' target is meant to build the same project as 'all', except with some different compile switches (like -ggdb, for example). Since the targets use different compile switches, you obviously need to rebuild the entire project if you switch between the two. But GNUmake doesn't naturally recognize this.
So if you type make all you'll get
Building ...
...
Then if you type make debug, you get
make: Nothing to be done for `debug'.
So my question is: how do you implement a clean solution in the Makefile to notice that the last build used a different pseudo-target, or different compile switches, than the one you want currently? If they are different, the Makefile would rebuild everything.
Put the build products into different directory trees (whilst keeping one copy of the source of course). That way you are always just a short compile from an up-to-date build, be it debug or release (or even others). No possibility of confusion either.
EDIT
Sketch of the above.
src := 1.c 2.c 3.c
bare-objs := ${src:%.c=%.o}
release-objs := ${bare-objs:%=Release/%}
debug-objs := ${bare-objs:%=Debug/%}
Release/prog: ${release-objs}
Debug/prog: ${debug-objs}
${release-objs}: Release/%.o: %.c # You gotta lurve static pattern rules
gcc -c $< -o $#
${debug-objs}: Debug/%.o: %.c
gcc -c $< -o $#
Release/prog Debug/prog:
gcc $^ -o $#
.PHONY: all
all: Release/prog ; echo $# Success
.PHONY: debug
debug: Debug/prog ; echo $# Success
(Disclaimer: not tested, nor even run through make.)
There you go. It's even -j safe so you can do make -j5 all debug. There is a lot of obvious boiler plate just crying out for tidying up.
Keeping variant sets of object files (as in bobbogo's solution) is probably the best way, but if for some reason you don't want to do that, you can use empty files as markers, to indicate which way you last built the executable:
%-marker:
#rm -f $(OBJECTS) *-marker
#touch $#
debug: GCCFLAGS += -ggdb
debug: SOMEOTHERFLAG = WHATEVER
all debug: % : %-marker
#echo making $#
#$(MAKE) -S GCCFLAGS='$(GCCFLAGS)' SOMEOTHERFLAG='$(SOMEOTHERFLAG)' main
There are other variants on this idea; you could have a small file containing the flag settings, which the makefile would build and include. That would be clever, but not really any cleaner than this.
The only clean solution is to incorporate the difference into the target names.
E.g. you can define a variable $(DEBUG) and consistently use it in all targets that depend on the compile step.