I've got a makefile for Advent of Code that I've decided to do in chicken scheme.
However, it's starting to grow, and I'd like to remove the repetition.
Here's a section of what I've got so far, but how do I make it so it can automatically find all days instead of 2 entries per day and an ever increasing main target?
all: advent2015
# advent years - this is ever increasing with the *.so targets
advent2015: $(ADVENTS_SRC)2015/advent2015.scm \
aoc2015day01.so aoc2015day02.so
$(CSC) $(INCLUDE_FLAGS) $< -d3 -O2 -compile-syntax -o $#
# individual days - how do i write a single rule that works for all days?
aoc2015day01.so: $(ADVENTS_SRC)2015/aoc2015day01.scm aoc-files.so
$(CSC) $(INCLUDE_FLAGS) $(LIBFLAGS) $< -j aoc2015day01 -emit-types-file aoc2015day01.types -o $#
$(CSC) $(INCLUDE_FLAGS) $(IMPORTFLAGS) aoc2015day01.import.scm
aoc2015day01.import.so: aoc2015day01.so
$(CSC) $(INCLUDE_FLAGS) $(IMPORTFLAGS) aoc2015day01.import.scm
# day02...
aoc2015day02.so: $(ADVENTS_SRC)2015/aoc2015day02.scm aoc-files.so
$(CSC) $(INCLUDE_FLAGS) $(LIBFLAGS) $< -j aoc2015day02 -emit-types-file aoc2015day02.types -o $#
$(CSC) $(INCLUDE_FLAGS) $(IMPORTFLAGS) aoc2015day02.import.scm
aoc2015day02.import.so: aoc2015day02.so
$(CSC) $(INCLUDE_FLAGS) $(IMPORTFLAGS) aoc2015day02.import.scm
I tried a small change to the advent2015 target by specifying a wildcard in the so files it is using, but make completely skipped generating the targets.
No problem:
DAYS := 01 02 03 04 <...> 24 25
all: advent2015
advent2015: $(ADVENTS_SRC)2015/advent2015.scm \
$(foreach D,$(DAYS),aoc2015day$(D).so)
$(CSC) $(INCLUDE_FLAGS) $< -d3 -O2 -compile-syntax -o $#
%.so: $(ADVENTS_SRC)2015/%.scm aoc-files.so
$(CSC) $(INCLUDE_FLAGS) $(LIBFLAGS) $< -j $* -emit-types-file $*.types -o $#
$(CSC) $(INCLUDE_FLAGS) $(IMPORTFLAGS) $*.import.scm
%.import.so: %.so
$(CSC) $(INCLUDE_FLAGS) $(IMPORTFLAGS) $*.import.scm
I'm not sure these commands are actually correct but they're what you provided above, so you can fix them if not.
You may find the csm egg useful. I know it means getting rid of your makefile, but it should do the trick much more easily.
Alternatively, beaker provides ways of building based on .egg files, like those used by chicken-install.
Related
I want all my build files stored in a build directory inside the project root folder. Though, I don't want to have to rebuild all of the files if I add a debug flag. Thus, I have two directories .build_release and .build_debug. Then I make a symbolic link from build to the proper directory.
I want all of this to be handled by make. Here is my makefile:
## setup
ifdef DEBUG
BUILDDIR=.build_debug
else
BUILDDIR=.build_release
endif
BLACKLIST:=bayesP obsDataStats test3 test bayesPsamplesBR test2 tuneSp \
toyFeatures2Multi getCov getDist isConnected mergeClusters sample \
toyFeatures0 toyFeatures1 toyFeatures2 toyFeatures3 toyFeatures4 \
toyFeatures6 toyFeatures7 wnsFeatures0 wnsFeatures1 wnsFeatures2
## make code
PROGS:=$(shell find ./src/ -maxdepth 1 -name "*.cpp" -exec grep -l "int main" {} \;)
PROGS:=$(notdir $(basename $(PROGS)))
PROGS:=$(filter-out $(BLACKLIST),$(PROGS))
CPP_SRC:=$(wildcard src/*.cpp)
CPP_SRC:=$(notdir $(basename $(CPP_SRC)))
CPP_SRC:=$(filter-out $(PROGS) $(BLACKLIST),$(CPP_SRC))
PROGS:=$(PROGS:=.bin)
PROGS:=$(PROGS:%=$(BUILDDIR)/%)
CPP_SRC:=$(CPP_SRC:%=src/%.cpp)
CPP_OBJ:=$(CPP_SRC:src/%.cpp=$(BUILDDIR)/%.o)
LIB=$(BUILDDIR)/libspatialDecisionMaking.so
## test code
CPP_SRC_TEST:=$(wildcard src/test/*.cpp)
CPP_SRC_TEST:=$(notdir $(basename $(CPP_SRC_TEST)))
CPP_SRC_TEST:=$(filter-out $(BLACKLIST),$(CPP_SRC_TEST))
PROGS_TEST:=$(CPP_SRC_TEST:%=$(BUILDDIR)/test/%.bin)
CPP_OBJ_TEST:=$(CPP_SRC_TEST:%=$(BUILDDIR)/test/%.o)
CPP_SRC_TEST:=$(CPP_SRC_TEST:%=src/test/%)
## options
CC=g++-4.9
ifdef DEBUG
CPP_FLAGS=-std=c++11 -ggdb
else
CPP_FLAGS=-std=c++11 -O3
endif
LD_FLAGS=-Isrc -L$(BUILDDIR) -lgsl -larmadillo -fPIC -fopenmp
## rules
all: | $(BUILDDIR) $(LIB) $(PROGS) build
test: | $(BUILDDIR)/test $(LIB) $(PROGS_TEST) build
build: $(BUILDDIR)
ln -rfs $(BUILDDIR) build
$(BUILDDIR)/test: $(BUILDDIR)
mkdir $(BUILDDIR)/test
$(BUILDDIR):
mkdir $(BUILDDIR)
$(BUILDDIR)/%.bin: src/%.cpp $(LIB)
$(CC) $(CPP_FLAGS) -o $# $< $(LD_FLAGS) -l$(LIB:$(BUILDDIR)/lib%.so=%)
ln -rfs $# $(#:%.bin=%)
$(LIB): $(CPP_OBJ)
$(CC) $(CPP_FLAGS) -o $# $^ $(LD_FLAGS) -shared
$(BUILDDIR)/%.o: src/%.cpp $(BUILDDIR)/%.d
$(CC) $(CPP_FLAGS) -c $< -o $# $(LD_FLAGS)
$(BUILDDIR)/%.d: src/%.cpp
$(CC) $(CPP_FLAGS) -MM $< -MT $(#:%.d=%.o) > $# $(LD_FLAGS)
%.cpp:
%.hpp:
# include dependencies
-include $(CPP_OBJ:%.o=%.d)
clean:
rm -rf $(BUILDDIR)
Though, it seems to skip making $(BUILDDIR). I delete the directories before I run make every time and it goes directly to building the dependency makefiles based on the rules for target $(BUILDDIR)/%.d. However, it naturally complains when trying to build the dependencies because $(BUILDDIR) doesn't exist.
Any ideas why it would be skipping the recipe for making $(BUILDDIR)?
As I have one more question on SO than answers, so I had to find a question to answer :) , and you seemed to not like the answer that was already there, so OK, even though your question was not "minimal", I spent an hour to work on it.
Your Makefile is not bad generally, but it does not follow a number of "good practices". Once I tidied everything up, all the problems disappeared. I hope it helps you to learn from this example - how I changed your original makefile to follow good practices.
The only little problem left, is that build is always relinked every time. This is because normally Make does not "depend" on variable values (such as DEBUG), only on files. It is possible to fix that (in this small case it does not matter much, but maybe later you will need this solution), by creating "dependable variables". See my answer at
How do I force a target to be rebuilt if a variable is set?
Below is the complete working makefile, I put comments on changes outside the code.
## setup
use := when you can
ifdef DEBUG
BUILDDIR:=.build_debug
else
BUILDDIR:=.build_release
endif
do not use find to list files, better declare files explicitly
PROGS:=\
prog0 \
prog1 \
CPP_SRC:=\
spam \
eggs \
CPP_SRC_TEST:=\
spam_test \
eggs_test \
split off link targets, so only the target is created in a rule:
PROG_LINKS:=$(addprefix $(BUILDDIR)/, $(PROGS))
PROGS:=$(PROGS:=.bin)
PROGS:=$(PROGS:%=$(BUILDDIR)/%)
CPP_SRC:=$(CPP_SRC:%=src/%.cpp)
CPP_OBJ:=$(CPP_SRC:src/%.cpp=$(BUILDDIR)/%.o)
LIB:=$(BUILDDIR)/libspatialDecisionMaking.so
PROGS_TEST:=$(CPP_SRC_TEST:%=$(BUILDDIR)/test/%.bin)
PROGS_TEST_LINKS:=$(addprefix $(BUILDDIR)/test, $(CPP_SRC_TEST))
## options
CC=g++-4.9
you are confusing CPP_FLAGS and LD_FLAGS, I put correct flags in each
also, your method of finding your shared library, is too complicated, I made it simple
CPP_FLAGS:= -std=c++11 -Isrc -fPIC -fopenmp
ifdef DEBUG
CPP_FLAGS+= -ggdb
else
CPP_FLAGS+= -O3
endif
LD_FLAGS:= -L$(BUILDDIR) -lgsl -larmadillo
## rules
you have too many dependencies - list only those that are conceptually needed for the target at hand, and recurse
all: | $(PROGS) $(PROG_LINKS) build
test: | $(PROGS_TEST) $(PROGS_TEST_LINKS) build
link file does not depend on the link target in any way
in your case, it depends on the value of DEBUG really, but like I said above, it is not super-easy to implement that, so I skipped it here and have a phony instead, which relinks all the time
.PHONY: build
build:
ln -srf $(BUILDDIR) $#
this is the best way to handle directory creation
%/.:
mkdir -p $(#D)
unfortunately, this is needed, because mkdir -p is not re-entrant and subject to race conditions
$(BUILDDIR)/test/.: | $(BUILDDIR)/.
.SECONDEXPANSION:
$(PROGS_TEST_LINKS) $(PROG_LINKS): %: | %.bin
ln -sr $| $#
all non-trivial recipes should depend on this makefile, change Makefile to whatever is correct (there is a more complicated way, to handle this automatically)
$(BUILDDIR)/%.bin: src/%.cpp $(LIB) Makefile | $$(#D)/.
$(CC) $(CPP_FLAGS) -o $# $< $(LD_FLAGS) $(LIB)
$(LIB): $(CPP_OBJ) Makefile | $$(#D)/.
$(CC) $(CPP_FLAGS) -o $# $(CPP_OBJ) $(LD_FLAGS) -shared
this is the most efficient way to handle "automatic" dependency generation - it invokes the preprocessor only once, not twice as in your original makefile
I put this in quotes, because the whole method of automatic dependencies, is subtly flawed and cannot work in all cases - but in your simple case it is very unlikely you will run into that subtle flaw
Yes I am violating the good practice I mentioned above - only target created in rule. If one understands what a good practice is for, and still thinks better to violate it, then OK.
$(BUILDDIR)/%.o: src/%.cpp Makefile | $$(#D)/.
$(CC) $(CPP_FLAGS) -MMD -MP -c $< -o $# $(LD_FLAGS)
# include dependencies
-include $(CPP_OBJ:%.o=%.d)
clean:
rm -rf $(BUILDDIR)
The .d, .o, and .bin files logically depend on $(BUILDDIR), so tell make that's the case
$(BUILDDIR)/%.d: src/%.cpp | $(BUILDDIR)
I have the following Makefile, and I am trying to simplify it. Any ideas?
PROG_NAME=a.out
all: $(PROG_NAME)
"\nBuilt all\n"
$(PROG_NAME): build/file1.o build/file2.o build/file3.o build/file4.0
gcc -o $# $^
#build file1.c and file2.c which are both in ../src directory
build/file%.o: ../src/file%.c
gcc -I ../inc -o $# $<
#build file3.c and file4.c which are both in ../src2 directory
build/file%.o: ../src2/file%.c
gcc -I ../inc -o $# $<
I tried this and it does not work:
PROG_NAME=a.out
all: $(PROG_NAME)
"\nBuilt all\n"
$(PROG_NAME): build/file1.o build/file2.o build/file3.o build/file4.0
gcc -o $# $^
#build ../src/file1.c, ../src/file2.c, ../src2/file3.c, and ../src2/file4.c
build/file%.o: ../src/file%.c ../src2/file%.c
gcc -I ../inc -o $# $<
You cannot simplify it. The only way would be to use makefile metaprogramming to generate the rules via eval or something.
Your attempt will clearly not work, because this:
build/file%.o: ../src/file%.c ../src2/file%.c
gcc -I ../inc -o $# $<
says "for any target build/fileXXX.o that you want to build, you can create it by running this gcc command on the file ../src/fileXXX.c if either of the files ../src/fileXXX.c or ../src2/fileXXX.c are older than the .o". That's obviously not what you want to do.
I was able to simplify it using the make variable VPATH.
http://www.gnu.org/software/make/manual/make.html#General-Search
PROG_NAME=a.out
VPATH=../src:../src2
all: $(PROG_NAME)
"\nBuilt all\n"
$(PROG_NAME): build/file1.o build/file2.o build/file3.o build/file4.0
gcc -o $# $^
build/file%.o: file%.c
gcc -I ../inc -o $# $<
In my Makefile I have a rule to compile Fortran source files, like so
.SUFFIXES:
%.o: %.[fF]
$(FC) $(FFLAGS) -c $< -o $#
This has worked fine on several machines. When I tried it on another machine (incidentally with a newer (GNU) make, 3.82 instead of 3.81), it did not work, and I had to replace it with two separate rules for .f and .F.
The wildcard works without the %, i.e. this works:
.SUFFIXES:
test.o: test.[fF]
$(FC) $(FFLAGS) -c $< -o $#
What is going on?
I can fully reproduce the behavior here. It is a regression in make 3.82. The discussion attached to the bug report suggests using .SECONDEXPANSION to work around the problem. I've tried it and got it to work in a simple test setup:
.SUFFIXES:
.SECONDEXPANSION:
%.o: $$(wildcard %.[fF])
$(FC) $(FFLAGS) -c $< -o $#
Using the file above and two test files named test.f and test2.F, here's a couple commands I issue and the output produced by the Makefile:
$ make -n test2.o
f77 -c test2.F -o test2.o
$ make -n test.o
f77 -c test.f -o test.o
The workaround works both with 3.81 and 3.82.
Here's a shortened version of the Makefile I am fighting with on a Linux system:
VPATH=altsrc:src:obj
OBJECTS=\
nondepcode1.o \
nondepcode2.o \
nondepcode3.o \
depcode1.o \
depcode2.o \
depcode3.o
DEP_OBJS= depcode1.o depcode2.o depcode3.o
# Targets
execute: $(OBJECTS)
gfortran -o $# $^ $(LFLAGS)
$(DEP_OBJS): npts.h
obj/%.o: %.f
$(FORTRAN) $(FFLAGS) $< -o $#
obj/%.o: %.f90
$(FORTRAN) $(FFLAGS) $< -o $#
# Rules
.f.o:
$(FORTRAN) $(FFLAGS) -o obj/$# $<
%.o: %.f90
$(FORTRAN) $(FFLAGS) -o obj/$# $<
I was expecting that the three objects that depend on "npts.h" would be automatically updated on a run of make (or gmake) if npts.h was more recent than any of the objects. This just does not happen. Make (and gmake) thinks the objects are up to date. AFAICT, I am doing things the way they are described in the GNU make manual. Anyone have any idea why make/gmake is not doing what I expected? Thanks. BTW, there are tabs at the beginning of all the recipe lines in the actual Makefile. They went away here.
Major egg on my face. As I said, this was a shortened version of the Makefile. I found the bug in the real Makefile. Had some misnamed macros. Not too experienced with the finer points of this stuff. Mea culpa. Very sorry. Thanks for checking.
I'm reading Managing Projects with GNU Make, and found this example in Chapter 2.7 - Automatic Dependency Generation. The Author says their from the GNU manual:
%.d: %c
$(CC) -M $(CPPFLAGS $< > $#.$$$$; \
sed s',\($*\)\.o[ :]*,\1.o $# : ,g' < $#.$$$$ > $#; \
rm -f $#.$$$$
However, I was able to do the same thing with this (note the sed):
-include $(subst .c,.d,$(SOURCES))
%.d: %.c
#$(CC) -M $(CPPFLAGS) $< | sed 's|:| $*.d : |' > $#;
All these lines do is generate the dependencies, then add in the *.d name. They had to change the first line from:
foo.o: bar.h foo.h fubar.h
to
foo.o foo.d : bar.h foo.h fubar.h
Mine is simpler and seems to work quite well, but I assume that the GNU folks had a reason for their sed command. Also:
Why do a redirect of the file into sed? Why not simply take it as a commond line parameter
Why not skip the intermediary file completely?
I know the guys at GNU could have thought of these too, but for some reason, went with the more complex setup. I just want to understand their reasoning, so I can do these on the fly.
Actually even the rule itself is not necessary. There is a great overview of different approaches of generating Make-style dependencies in Advanced Auto-Dependency Generation article written by Paul D. Smith.
After all, the following rule should be enough (in case of using GCC):
%.o: %.c
$(CC) $(CPPFLAGS) $(CFLAGS) -MMD -MP -o $# -c $<
-include $(SOURCES:.c=.d)
UPD.
I have also answered a similar question a bit earlier. It contains an explanation (quotation of GCC manual) of -MMD -MP options.
Addressing the question: Why do a redirect of the file into sed? If you do:
#$(CC) -M $(CPPFLAGS) $< | sed 's|:| $*.d : |' > $#;
and the compilation fails (errors out and generates no output), you will create an empty target file. When make is run again, it will see the newly created empty file and not regenerate it, leading to build errors. Using intermediate files is a common defensive strategy to avoid accidentally created an empty target.
An even simpler solution is to get rid of the sed call completely, as gcc can do everything you need directly:
%.d: %.c
$(CC) -M $(CPPFLAGS) -MF $# $< -MT "$*.o $#"