How to avoid running all targets every time a makefile is run? - makefile

Question

How can I avoid rebuilding of targets every time even though there are no changes to them? I have looked at these answers: answer1, answer2, answer3, but I still could not solve my problem. I think I understand conceptually why this problem occurs. It is just that I was unable to apply the solutions to my case.
Here is my makefile:
.PHONY: all dircreate dircreate_sub
# Create shortcuts to directories ##############################################
DAT4 = data/4-Year/
RES4 = data/results/4-Year/
FIG4 = figures/4-Year/
DAT2 = data/2-Year/
RES2 = data/results/2-Year/
FIG2 = figures/2-Year/
DEPVARS = \
ret.1st.2nd.term.left \
ret.1st.2nd.year.left \
ret.1st.7th.year.grad \
ret.1st.5th.year.grad
# Create directories ###########################################################
dircreate:
mkdir -p \
data/ \
data/4-Year/ \
data/2-Year/ \
data/results/ \
data/results/4-Year \
data/results/2-Year \
figures/ \
figures/4-Year/ \
figures/2-Year/
dircreate_sub:
for d in $(DEPVARS); do \
mkdir -p data/4-Year/$$d ; \
mkdir -p data/2-Year/$$d ; \
mkdir -p data/results/4-Year/$$d ; \
mkdir -p data/results/2-Year/$$d ; \
mkdir -p figures/4-Year/$$d ; \
mkdir -p figures/2-Year/$$d ; \
done;
#TARGETS_DATAPREP := \
#$(foreach dat, $(DAT4) $(DAT2), $\
# $(foreach filename, \
# train_index_outer.RDS \
# train_outer.RDS \
# train_inter_outer.RDS, $\
# $(foreach depvar, $(DEPVARS),$(dat)$(depvar)/$(filename))))
# Data prep:####################################################################
TARGETS_DATAPREP := \
$(foreach filename, $\
train_index_outer.RDS \
train_outer.RDS \
train_inter_outer.RDS \
entire_data.RDS \
entire_inter_data.RDS, $\
$(foreach depvar, $(DEPVARS),$(DAT4)$(depvar)/$(filename)))
$(TARGETS_DATAPREP): \
dataprep.R \
funcs.R \
../core/data/analysis.data.RDS
Rscript $<
# benchmark:####################################################################
DEPENDENCIES_BENCHMARK := \
$(foreach filename, $\
train_index_outer.RDS \
train_outer.RDS \
train_inter_outer.RDS \
entire_data.RDS \
entire_inter_data.RDS, $\
$(foreach depvar, $(DEPVARS),$(DAT4)$(depvar)/$(filename)))
TARGETS_BENCHMARK := \
$(foreach filename, $\
logreg_inner.RDS \
l1logreg_inner.RDS \
l1logreg.int_inner.RDS \
rf_inner.RDS \
xgb_inner.RDS \
logreg_outer.RDS \
l1logreg_outer.RDS \
l1logreg.int_outer.RDS \
rf_outer.RDS \
xgb_outer.RDS, $\
$(foreach depvar, $(DEPVARS),$(RES4)$(depvar)/$(filename)))
$(TARGETS_BENCHMARK): \
benchmark.R \
funcs.R \
$(DEPENDENCIES_BENCHMARK)
Rscript $<
# Process:######################################################################
TARGETS_PROCESS := \
$(foreach filename, $\
processed_inner.RDS \
processed_inner_outer.RDS, $\
$(foreach depvar, $(DEPVARS),$(RES4)$(depvar)/$(filename)))
$(TARGETS_PROCESS): \
process.R \
funcs.R \
$(TARGETS_BENCHMARK)
Rscript $<
# Graphs:#######################################################################
TARGETS_GRAPHS := \
$(foreach filename, $\
ave_auc_inner.png \
ave_ppv10_inner.png \
dist_auc_inner.png \
dist_ppv10_inner.png \
roc_inner.png \
ave_auc_outer.png \
ave_ppv10_outer.png \
dist_auc_outer.png \
dist_ppv10_outer.png \
roc_outer.png \
ave_auc_ppv10_inner_outer.png \
roc_inner_outer.png \
thresh_inner_outer.png, $\
$(foreach depvar, $(DEPVARS),$(FIG4)$(depvar)/$(filename)))
$(TARGETS_GRAPHS): \
graphs.R \
funcs.R \
$(TARGETS_PROCESS)
Rscript $<
# Make all
all: $(TARGETS_PROCESS)
So, the make -nd tells me dircreate needs to be remade every time. That's why I also tried adding a variable called, say, OUTDIRS and also created a rule for it. So, instead of dircreate, I added this bit:
OUTDIRS: $(DAT4) $(RES4) $(FIG4) $(DAT2) $(RES2) $(FIG2)
OUTDIRS := \
for d in $(DEPVARS); do \
$(DAT4)$$d ; \
$(DAT2)$$d ; \
$(RES4)$$d ; \
$(RES2)$$d ; \
$(FIG4)$$d ; \
$(FIG2)$$d ; \
done;
$(OUTDIRS):
mkdir -p $#
But, this time I get a missing separator error at $(OUTDIRS): mkdir -p $#, which, I know, is unrelated to the main question, but I am wondering if the way I am going about is at least on the right track?

Answer 1

Suppose you have
DIRS := 2-Year 4-Year
DEPVARS := term year
and you want to construct:
2-Year/term 2-Year/year 4-Year/term 4-Year/year
You can do it this way:
OUTDIRS := $(foreach d,$(DIRS),$(addprefix $(d)/,$(DEPVARS)))
And once you have that working you can abstract it into a function:
Multiply= $(foreach d,$(1),$(addprefix $(d)/,$(2)))
OUTDIRS:=$(call Multiply, $(DIRS), $(DEPVARS))
And one you have that working you can use it to construct those trees of yours:
DIRS := data data/results figures
DIRS := $(DIRS) $(call Multiply, $(DIRS), 4-Year 2-Year)
DEPVARS := $(addprefix ret.1st., 2nd.term.left 2nd.year.left 7th.year.grad 5th.year.grad)
OUTDIRS:=$(call Multiply, $(DIRS), $(DEPVARS))
$(OUTDIRS):
mkdir -p $#
Note that if you want Make to create whichever of those directories don't exist, you must add another rule before the $(OUTDIRS) rule:
.PHONY: all-dirs
all-dirs: $(OUTDIRS)