makefile skips making the build directory - makefile

Question

I want all my build files stored in a build directory inside the project root folder. Though, I don't want to have to rebuild all of the files if I add a debug flag. Thus, I have two directories .build_release and .build_debug. Then I make a symbolic link from build to the proper directory.
I want all of this to be handled by make. Here is my makefile:
## setup
ifdef DEBUG
BUILDDIR=.build_debug
else
BUILDDIR=.build_release
endif
BLACKLIST:=bayesP obsDataStats test3 test bayesPsamplesBR test2 tuneSp \
toyFeatures2Multi getCov getDist isConnected mergeClusters sample \
toyFeatures0 toyFeatures1 toyFeatures2 toyFeatures3 toyFeatures4 \
toyFeatures6 toyFeatures7 wnsFeatures0 wnsFeatures1 wnsFeatures2
## make code
PROGS:=$(shell find ./src/ -maxdepth 1 -name "*.cpp" -exec grep -l "int main" {} \;)
PROGS:=$(notdir $(basename $(PROGS)))
PROGS:=$(filter-out $(BLACKLIST),$(PROGS))
CPP_SRC:=$(wildcard src/*.cpp)
CPP_SRC:=$(notdir $(basename $(CPP_SRC)))
CPP_SRC:=$(filter-out $(PROGS) $(BLACKLIST),$(CPP_SRC))
PROGS:=$(PROGS:=.bin)
PROGS:=$(PROGS:%=$(BUILDDIR)/%)
CPP_SRC:=$(CPP_SRC:%=src/%.cpp)
CPP_OBJ:=$(CPP_SRC:src/%.cpp=$(BUILDDIR)/%.o)
LIB=$(BUILDDIR)/libspatialDecisionMaking.so
## test code
CPP_SRC_TEST:=$(wildcard src/test/*.cpp)
CPP_SRC_TEST:=$(notdir $(basename $(CPP_SRC_TEST)))
CPP_SRC_TEST:=$(filter-out $(BLACKLIST),$(CPP_SRC_TEST))
PROGS_TEST:=$(CPP_SRC_TEST:%=$(BUILDDIR)/test/%.bin)
CPP_OBJ_TEST:=$(CPP_SRC_TEST:%=$(BUILDDIR)/test/%.o)
CPP_SRC_TEST:=$(CPP_SRC_TEST:%=src/test/%)
## options
CC=g++-4.9
ifdef DEBUG
CPP_FLAGS=-std=c++11 -ggdb
else
CPP_FLAGS=-std=c++11 -O3
endif
LD_FLAGS=-Isrc -L$(BUILDDIR) -lgsl -larmadillo -fPIC -fopenmp
## rules
all: | $(BUILDDIR) $(LIB) $(PROGS) build
test: | $(BUILDDIR)/test $(LIB) $(PROGS_TEST) build
build: $(BUILDDIR)
ln -rfs $(BUILDDIR) build
$(BUILDDIR)/test: $(BUILDDIR)
mkdir $(BUILDDIR)/test
$(BUILDDIR):
mkdir $(BUILDDIR)
$(BUILDDIR)/%.bin: src/%.cpp $(LIB)
$(CC) $(CPP_FLAGS) -o $# $< $(LD_FLAGS) -l$(LIB:$(BUILDDIR)/lib%.so=%)
ln -rfs $# $(#:%.bin=%)
$(LIB): $(CPP_OBJ)
$(CC) $(CPP_FLAGS) -o $# $^ $(LD_FLAGS) -shared
$(BUILDDIR)/%.o: src/%.cpp $(BUILDDIR)/%.d
$(CC) $(CPP_FLAGS) -c $< -o $# $(LD_FLAGS)
$(BUILDDIR)/%.d: src/%.cpp
$(CC) $(CPP_FLAGS) -MM $< -MT $(#:%.d=%.o) > $# $(LD_FLAGS)
%.cpp:
%.hpp:
# include dependencies
-include $(CPP_OBJ:%.o=%.d)
clean:
rm -rf $(BUILDDIR)
Though, it seems to skip making $(BUILDDIR). I delete the directories before I run make every time and it goes directly to building the dependency makefiles based on the rules for target $(BUILDDIR)/%.d. However, it naturally complains when trying to build the dependencies because $(BUILDDIR) doesn't exist.
Any ideas why it would be skipping the recipe for making $(BUILDDIR)?

Answer 1

As I have one more question on SO than answers, so I had to find a question to answer :) , and you seemed to not like the answer that was already there, so OK, even though your question was not "minimal", I spent an hour to work on it.
Your Makefile is not bad generally, but it does not follow a number of "good practices". Once I tidied everything up, all the problems disappeared. I hope it helps you to learn from this example - how I changed your original makefile to follow good practices.
The only little problem left, is that build is always relinked every time. This is because normally Make does not "depend" on variable values (such as DEBUG), only on files. It is possible to fix that (in this small case it does not matter much, but maybe later you will need this solution), by creating "dependable variables". See my answer at
How do I force a target to be rebuilt if a variable is set?
Below is the complete working makefile, I put comments on changes outside the code.
## setup
use := when you can
ifdef DEBUG
BUILDDIR:=.build_debug
else
BUILDDIR:=.build_release
endif
do not use find to list files, better declare files explicitly
PROGS:=\
prog0 \
prog1 \
CPP_SRC:=\
spam \
eggs \
CPP_SRC_TEST:=\
spam_test \
eggs_test \
split off link targets, so only the target is created in a rule:
PROG_LINKS:=$(addprefix $(BUILDDIR)/, $(PROGS))
PROGS:=$(PROGS:=.bin)
PROGS:=$(PROGS:%=$(BUILDDIR)/%)
CPP_SRC:=$(CPP_SRC:%=src/%.cpp)
CPP_OBJ:=$(CPP_SRC:src/%.cpp=$(BUILDDIR)/%.o)
LIB:=$(BUILDDIR)/libspatialDecisionMaking.so
PROGS_TEST:=$(CPP_SRC_TEST:%=$(BUILDDIR)/test/%.bin)
PROGS_TEST_LINKS:=$(addprefix $(BUILDDIR)/test, $(CPP_SRC_TEST))
## options
CC=g++-4.9
you are confusing CPP_FLAGS and LD_FLAGS, I put correct flags in each
also, your method of finding your shared library, is too complicated, I made it simple
CPP_FLAGS:= -std=c++11 -Isrc -fPIC -fopenmp
ifdef DEBUG
CPP_FLAGS+= -ggdb
else
CPP_FLAGS+= -O3
endif
LD_FLAGS:= -L$(BUILDDIR) -lgsl -larmadillo
## rules
you have too many dependencies - list only those that are conceptually needed for the target at hand, and recurse
all: | $(PROGS) $(PROG_LINKS) build
test: | $(PROGS_TEST) $(PROGS_TEST_LINKS) build
link file does not depend on the link target in any way
in your case, it depends on the value of DEBUG really, but like I said above, it is not super-easy to implement that, so I skipped it here and have a phony instead, which relinks all the time
.PHONY: build
build:
ln -srf $(BUILDDIR) $#
this is the best way to handle directory creation
%/.:
mkdir -p $(#D)
unfortunately, this is needed, because mkdir -p is not re-entrant and subject to race conditions
$(BUILDDIR)/test/.: | $(BUILDDIR)/.
.SECONDEXPANSION:
$(PROGS_TEST_LINKS) $(PROG_LINKS): %: | %.bin
ln -sr $| $#
all non-trivial recipes should depend on this makefile, change Makefile to whatever is correct (there is a more complicated way, to handle this automatically)
$(BUILDDIR)/%.bin: src/%.cpp $(LIB) Makefile | $$(#D)/.
$(CC) $(CPP_FLAGS) -o $# $< $(LD_FLAGS) $(LIB)
$(LIB): $(CPP_OBJ) Makefile | $$(#D)/.
$(CC) $(CPP_FLAGS) -o $# $(CPP_OBJ) $(LD_FLAGS) -shared
this is the most efficient way to handle "automatic" dependency generation - it invokes the preprocessor only once, not twice as in your original makefile
I put this in quotes, because the whole method of automatic dependencies, is subtly flawed and cannot work in all cases - but in your simple case it is very unlikely you will run into that subtle flaw
Yes I am violating the good practice I mentioned above - only target created in rule. If one understands what a good practice is for, and still thinks better to violate it, then OK.
$(BUILDDIR)/%.o: src/%.cpp Makefile | $$(#D)/.
$(CC) $(CPP_FLAGS) -MMD -MP -c $< -o $# $(LD_FLAGS)
# include dependencies
-include $(CPP_OBJ:%.o=%.d)
clean:
rm -rf $(BUILDDIR)

Answer 2

The .d, .o, and .bin files logically depend on $(BUILDDIR), so tell make that's the case
$(BUILDDIR)/%.d: src/%.cpp | $(BUILDDIR)