I currently have this in my makefile:
ifneq ($(MAKECMDGOALS),clean)
-include $(DEPS_debug)
-include $(DEPS_unittest)
-include $(DEPS_release)
endif
The DEPS_ collections are all quite large since they are generated by gcc, so I'd like to only include those dependencies when absolutely necessary.
As it is now, any time I use this makefile to do anything other than cleanup, it incurs a two-second wait due to the loading of all the dependency files. Yes, it is probably unnecessarily checking a myriad of library headers for changes but I'll be damned if it doesn't work so well already that I don't want to screw with it.
I'd like to -include only $(DEPS_debug) when my make target is debug, etc.
Is there a better variable than MAKECMDGOALS I can make use of? An issue I forsee is that if I use MAKECMDGOALS to predicate the inclusion of my dependencies, if I later add an entry to the makefile foo: debug release and run make foo it won't load any of them!
How do you generate your dependencies?
gcc -M ... source.c
vs
gcc -MM ... source.c
The former lists the headers from /usr/include; the latter doesn't.
Related
In a simple embedded project, i have two files main.rs and module.rs. To build the project, I use something similar to this:
all: main.o
$(CC) main.o $(LDFLAGS)
%.o: %.rs
$(RUSTC) $(RUSTFLAGS) -o ${#} ${<}
If only module.rs is changed, make all won't recompile my Rust code. How can I fix this?
I'm posting a suboptimal self-answer as a first step, but would love to see better ways.
The best way to use Make is to encode every single dependency into the Makefile. That's what gives Make the power to know what to rebuild in order to reach a goal state.
To do this for a C project, you'll often use something like the GCC command line option -M. This brings the compiler into the mix as it's the best tool to parse C source code and understand the dependencies between the
files.
There is actually an equivalent switch for rustc, the Rust compiler: --emit=dep-info. When you run this on a source file, it will output a file with the extension .d, which contains an almost-Makefile-compatible list of dependencies. If you had a main.rs that referenced the module foo.rs, it would output something like:
main.d: main.rs foo.rs
With a bit of sed tweaking you can get this to play nicely. You can then include this in your Makefile:
main.o:
rustc -o $# $<
main.d: main.rs
rustc --emit=dep-info $<
# Add the object file as a rule
gsed 's/:/ $(#:.d=.o):/' -i $#
-include main.d
Here, I've specified main in a few parts, but I believe that you can easily modify them into pattern rules.
The pragmatic solution is to just use Cargo, the Rust build tool and package manager. Let it deal with dependencies (both local modules and other crates).
libbar.dylib: target/debug/libbar.dylib
cp $< $#
.PHONY: target/debug/libbar.dylib
target/debug/libbar.dylib:
cargo build --verbose
Here, I've marked the rule as PHONY, which says "always run this rule". I've added --verbose to have Cargo print out what it is doing so you can verify when things are rebuilt.
I'd recommend dropping off the cp step if you can and instead just use the nested path, but the copy might be needed if other things rely on the current location.
The pattern
%.o: %.rs
is familiar from building C projects, but that's not the only way a target can be written. Specific to the setup above, this would fix the situation:
main.o: main.rs module.rs
$(RUSTC) $(RUSTFLAGS) -o main.o main.rs
A noteworthy difference to the original code is that the names of the inputs is not really what matters for the command. We can generalize this as follows:
main.o: $(wildcard *.rs)
$(RUSTC) $(RUSTFLAGS) -o ${#} ${#:.o=.rs}
This is a start, but it still has some downsides I couldn't get rid of:
The main.o: part is hardcoded. If there are multiple top-level modules to compile, there would be code duplication
All Rust files will be considered for all top-level modules, due to the wildcard. In other words, changing any Rust file will require a full recompilation.
Is there a way how to ask gmake to never run two targets from a set in parallel?
I don't want to use .NOTPARALLEL, because it forces the whole Makefile to be run sequentially, not just the required part.
I could also add dependencies so that one depends on another, but then (apart from being ugly) I'd need to build all of them in order to build the last one, which isn't necessary.
The reason why I need this is that (only a) part of my Makefile invokes ghc --make, which takes care of its dependencies itself. And it's not possible to run it in parallel on two different targets, because if the two targets share some dependency, they can rewrite each other's .o file. (But ghc is fine with being called sequentially.)
Update: To give a specific example. Let's say I need to compile two programs in my Makefile:
prog1 depends on prog1.hs and mylib.hs;
prog2 depends on prog2.hs and mylib.hs.
Now if I invoke ghc --make prog1.hs, it checks its dependencies, compiles both prog1.hs and mylib.hs into their respective object and interface files, and links prog1. The same happens when I call ghc --make prog2.hs. So if they the two commands get to run in parallel, one will overwrite mylib.o of the other one, causing it to fail badly.
However, I need that neither prog1 depends on prog2 nor vice versa, because they should be compilable separately. (In reality they're very large with a lot of modules and requiring to compile them all slows development considerably.)
Hmmm, could do with a bit more information, so this is just a stab in the dark.
Make doesn't really support this, but you can sequential-ise two targets in a couple of ways. First off, a real use for recursive make:
targ1: ; recipe1...
targ2: ; recipe2...
both-targets:
${MAKE} targ1
${MAKE} targ2
So here you can just make -j both-targets and all is fine. Fragile though, because make -j targ1 targ2 still runs in parallel. You can use dependencies instead:
targ1: ; recipe1...
targ2: | targ1 ; recipe2...
Now make -j targ1 targ2 does what you want. Disadvantage? make targ2 will always try to build targ1 first (sequentially). This may (or may not) be a show-stopper for you.
EDIT
Another unsatisfactory strategy is to explicitly look at $MAKECMDGOALS, which lists the targets you specified on the command-line. Still a fragile solution as it is broken when someone uses dependencies inside the Makefile to get things built (a not unreasonable action).
Let's say your makefile contains two independent targets targ1 and targ2. Basically they remain independent until someone specifies on the command-line that they must both be built. In this particular case you break this independence. Consider this snippet:
$(and $(filter targ1,${MAKECMDGOALS)),$(filter targ2,${MAKECMDGOALS}),$(eval targ1: | targ2))
Urk! What's going on here?
Make evaluates the $(and)
It first has to expand $(filter targ1,${MAKECMDGOALS})
Iff targ1 was specified, it goes on to expand $(filter targ2,${MAKECMDGOALS})
Iff targ2 was also specified, it goes on to expand the $(eval), forcing the serialization of targ1 and targ2.
Note that the $(eval) expands to nothing (all its work was done as a side-effect), so that the original $(and) always expands to nothing at all, causing no syntax error.
Ugh!
[Now that I've typed that out, the considerably simpler prog2: | $(filter prog1,${MAKECMDGOALS})
occurs to me. Oh well.]
YMMV and all that.
I'm not familiar with ghc, but the correct solution would be to get the two runs of ghc to use different build folders, then they can happily run in parallel.
Since I got stuck at the same problem, here is another pointer in the direction that make does not provide the functionality you describe:
From the GNU Make Manual:
It is important to be careful when using parallel execution (the -j switch; see Parallel Execution) and archives. If multiple ar commands run at the same time on the same archive file, they will not know about each other and can corrupt the file.
Possibly a future version of make will provide a mechanism to circumvent this problem by serializing all recipes that operate on the same archive file. But for the time being, you must either write your makefiles to avoid this problem in some other way, or not use -j.
What you are attempting, and what I was attempting (using make to insert data in a SQLite3 database) suffers from the exact same problem.
I needed to separate the compilation from other steps (cleaning, building dirs and linking), as I wanted to run the compilation with more core processes and the -j flag.
I managed to solve this, with different makefiles including and calling each other. Only the "compile" make file is running in parallel with all the cores, the rest of the process is syncronous.
I divided my makefile in 3 separate scripts:
settings.mk: contains all the variables and flag definitions
makefile: has all the targets except the compilation one (It has .NOTPARALLEL directive). It calls compile.mk with -j flag
compile.mk: contains only the compile operation (without .NOTPARALLEL)
In settings.mk I have:
CC = g++
DB = gdb
RM = rm
MD = mkdir
CP = cp
MAKE = mingw32-make
BUILD = Debug
DEBUG = true
[... all other variables and flags needed, directories etc ...]
In makefile I have Link and compilation target as these:
include .makefiles/settings.mk
[... OTHER TARGETS (clean, directories etc)]
compilation:
#echo Compilation
#$(MAKE) -f .makefiles/compile.mk --silent -j 8 -Oline
#Link
$(TARGET): compilation
#echo -e Linking $(TARGET)
#$(CC) $(LNKFLAGS) -o $(TARGETDIR)/$(TARGET) $(OBJECTS) $(LIBDIRS) $(LIB)
#Non-File Targets
.PHONY: all prebuild release rebuild clean resources directories run debug
.NOTPARALLEL: all
# include dependency files (*.d) if available
-include $(DEPENDS)
And this is my compile.mk:
include .makefiles/settings.mk
#Defauilt
all: $(OBJECTS)
#Compile
$(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(SRCEXT)
#echo -e Compiling: $<
#$(MD) -p $(dir $#)
#$(CC) $(COMFLAGS) $(INCDIRS) -c $< -o $#
#Non-File Targets
.PHONY: all
# include dependency files (*.d) if available
-include $(DEPENDS)
Until now, it's working.
Note that I'm calling compile.mk with -j flag AND -Oline so that parallel processing doesn't mess up with the output.
Any syntax color can be setted in the makefile main script, since the -O flag invalidates escape color codes.
I hope it can help.
I had a similar problem so ended up solving it on the command line, like so:
make target1; make target2
to force it to do the targets sequentially.
Note: using MinGW's make (should be GNU make)
i have a couple of -include statements in my makefile to import dependencies which were generated using g++ -MM. However I would like to only do this when necessary. I have several different build targets and I don't want all of their respective dependency files to be included since this takes a while (suppose I'm running make clean: no need to include them in this case)
Here's the format of my makefile.
DEPS_debug = $(patsubst %.cpp,build_debug/%.d,$(SRC))
OBJ_debug = $(patsubst %.cpp,build_debug/%.o,$(SRC))
all: program_debug
-include $(DEPS_debug) #make: include: Command not found
program_debug: $(OBJ_debug)
$(CC) $(CFLAGS) $(OBJ_debug) -o $#
If you really don't want to include those files needlessly, you have a couple of options:
You can put in a conditional as Diego Sevilla suggests (but I would recommend using MAKECMDGOALS so that you can write a more flexible version, specific to targets, e.g. you'll include foo.d if and only if you're making foo.o).
You can use make recursively (heresy!), invoking $(MAKE) for each target object, using a makefile that includes that target's dependencies.
But actually including the file takes negligible time, it's the rebuilding of the file (automatic for any included file that's out of date) that takes time.
If needless rebuilding is what you want to avoid, you can use a very clever trick. When must foo.d be rebuilt? Only when something about foo has changed. But in that case foo.o must also be rebuilt. So don't have a seperate rule for foo.d, just rebuild it as a side effect of making foo.o. That way you can include all dependency files and not waste time rebuilding them if they aren't needed.
EDIT:
I'm astounded that merely including these files can add 2-3 seconds to make clean. My last paragraph is off the mark, so let me expand on the first two options.
If all is the only target for which these files should be included, and you make all from the command line (and not e.g. make all tests tarball install kitchenSink), then this will do it:
ifeq ($(MAKECMDGOALS),all)
-include $(DEPS_debug)
endif
Note that this will not include foo.d if you make foo.o. You can write a more sophisticated conditional, something like
$(foreach targ,$(MAKECMDGOALS),$(eval $(call include_deps $(targ)))...
but that's pretty advanced, so let's get a simple version working first.
If you'd rather avoid the conditional and use recursive Make, the simplest way is to split the makefile in two:
makefile:
all:
$(MAKE) -f makefile.all
clean:
rm whatever
...other rules
makefile.all:
DEPS_debug = $(patsubst %.cpp,build_debug/%.d,$(SRC))
OBJ_debug = $(patsubst %.cpp,build_debug/%.o,$(SRC))
-include $(DEPS_debug)
all: program_debug
program_debug: $(OBJ_debug)
$(CC) $(CFLAGS) $(OBJ_debug) -o $#
Indenting a line by a TAB makes make think it's a command to be passed to the shell (as you found out). It doesn't work that way.
The - in front of include suppresses errors that might result from DEPS_debug not existing (e.g. when running clean or release without having had a dependency-file-generating call first). Since DEPS_debug is not a dependency of those rules (clean / release), your dependency files do not get generated when you call them, and everything is fine. I don't really see the problem you're having - you don't have to make the include conditional.
Perhaps you'd like to change your approach, though. Instead of having a seperate *.d target, with a seperate -M preprocessor pass, you might want to try something like -MMD -MP which generates the dependency files inline during code generation, in your standard *.c -> *.o pass.
(I know this sounds completely wrong at first, but when you think about it, it makes sense. Makefile logic is a bit backwards that way, unless you're familiar with functional programming.)
includes are independent of the rules, as they are makefile indications, not compilation indications. You can, however, use makefile conditionals based on special makefile variables such as MAKECMDGOALS, that is set to the default goal:
ifeq ($(MAKECMDGOALS),all)
-include whatever
endif
This is included when no default goal is specified. You can change the condition to specify the exact goal you want to check to include other sub-makefiles.
I have a BSDmakefile and GNUmakefile that are pretty much identical except for dependency management.
The GNUmakefile:
ifneq ($(MAKECMDGOALS), "clean")
-include $(dependencies)
endif
The BSDmakefile:
.for i in $(dependencies)
.sinclude "${i}"
.endfor
Is there a way to make it so that I can detect if I am running under gmake or bsdmake and then execute the appropriate include statements based off of that? I remember seeing someone take advantage of a quirk in both makefile processors so that they could achieve a similar effect.
Alternatively, if there is a better approach than this, I would like to know! (switching to SCons or CMake is not appropriate!)
Thanks!
You could put your GNU-specific stuff in GNUmakefile, your BSD-specific stuff in BSDmakefile, and your common stuff in a file named Makefile.common or similar. Then include Makefile.common at the very beginning of each of the other two. Downside is, now you have 3 makefiles instead of 2. Upside, you'll only be editing 1.
I am new to Automake and I am attempting to compile without linking. My goal is to generate a simple Makefile as shown below using Automake.
CFLAG = -Wall
build: Thread.o
Thread.o: Thread.cc Thread.h
g++ $(CFLAG) -c Thread.cc
clean:
rm -f *.o
My attempt so far has brought me to the following Makefile.ac.
noinst_PROGRAMS = thread
thread_SOURCES = Thread.cc
EXTRA_DIST= Thread.h
How can I simulate my original Makefile?
One way is to do this is to fool Automake by providing link command that does not link:
thread_LINK = true
Other than that, I wouldn't be suprised if Automake did not have such feature.
For your example, you can just ask Automake to build your .o file directly, e.g.:
$ make Thread.o
I believe this is an implicit rule, so you won't see it in the output Makefile.
In general, Automake generates variables containing all the objects required for each executable or library target. It's pretty straightforward to use them in your Makefile, since it just generates their names by appending _OBJECTS to the target name. You could make your own target in Makefile.am like this:
build-thread: $(thread_OBJECTS)
Then you could build just Thread.o (and any other objects needed for thread) like this:
$ make build-thread
Or if you had multiple targets foo, bar, and baz, you could make your compile-only target in Makefile.am like this:
build: $(foo_OBJECTS) $(bar_OBJECTS) $(baz_OBJECTS)
The only pain here is that you'll need to maintain this list yourself based on the targets in your Makefile.am. You can invoke it at the command line like this:
$ make build
Automake is not designed to produce object. It will build either programs or libraries.
It's hard to answer your question without knowing why you'd want to compile a single object file and not something else. Maybe there is a cleaner answer to your "real" problem.
A Makefile.am you could write is
noinst_LIBRARIES = libThread.a
libThread_a_SOURCES = Thread.cc Thread.h # No need to put headers in EXTRA_DIST
The resulting Makefile would build a library libThread.a containing only libThread.o, ans because *.a libraries are just a collection of object files there is no linking involved.
The above Makefile.am also causes the emitted Makefile to contain rules to compile libThread.o, so you can add a build: rule if you like.
If you really want Automake to emit this compile rule, but not build the library, you could go with
EXTRA_LIBRARIES = libThread.a # EXTRA here means "output build rules but don't
# build unless something depends on it".
libThread_a_SOURCES = Thread.cc Thread.h
build: Thread.$(OBJEXT)
Now you are explicitely requiring the file Thread.$(OBJEXT) to be built only when you type make build, as in your original Makefile.
(Automake uses .$(OBJEXT) rather than .o to support extensions like .obj in DOS variants.)
First off, automake is a tool to auto make making Makefiles; make in and of itself is a whole different beast (and I'm pretty sure that what you were looking for was a make solution).
Here's the easiest GNU based Makefile to accomplish what you want:
all: Thread.o
This fills in something (by default) like the following (please change 4-space whitespace to hard tabs):
all: Thread.o
Thread.o: Thread.cc
$(COMPILE.cpp) $(OUTPUT_OPTION) $<
The COMPILE.cpp and OUTPUT_OPTION macros of course expand by default to GNU make specified values and aren't portable; $< is AT&T Make standard syntax though according to pmake(1)'s manpage though.
GNU make has a concept of implicit vs explicit rules, patterns, suffixes, etc that you could use, but that's not portable to all versions of make, and hence that's why all of the Makefile is plainly spelled out in terms of targets and variables as POSIX doesn't describe many of the desired scenarios for how one should write a Makefile.
Run gmake -p for more details and take a look at the texinfo manual for gmake in the topic of implicit, explicit rules, patterns, suffixes, etc.