I have the following question on writing a reasonable Makefile for compiling a lot of LaTeX exercise sheets for my lectures. The exercises are organised in files file1.tex, file2.tex, ... fileXXX.tex where XXX is the total number (varies from course to course). Now the catch is that I use crossrefs from later files to earlier ones: the dependence of fileY.tex is on all the aux-files of the files fileZ.tex with Z < Y: so those have to be generated before.
So instead of hardcoding all the XXX files with their dependencies into a Makefile I'm looking for a more efficient way to do that. It would be somehow nice to specify just the total number XXX of files.
Like this?
N := 100 # can override on the command line, etc.
SRCS := $(foreach n,$(shell seq $(N)),file$(n).tex)
define dep
$(call dep-rec,$1,,)
endef
# $(call dep-rec,.tex files,accumulated aux files)
define dep-rec
$(if $1,$(eval $(word 1,$1): $2)$(call dep-rec,$(wordlist 2,$(words $1),$1),$2 $(call aux-files-of,$(word 1,$1))))
endef
# return the aux files of $1 (which is a .tex file) -- define as appropriate
define aux-files-of
$(basename $1).aux
endef
$(call dep,$(SRCS))
Related
I have a bunch of different source files in my static HTML blog. The outermost extensions explain the format to be processed next.
Example: Source file article.html.md.gz (with target article.html) should be processed by gunzip, then by my markdown processor.
Further details:
The order of the extensions may vary
Sometimes an extension is not used (article.html.gz)
I know how to process all different extensions
I know that the final form is always article.html
Ideally I would have liked to just write rules as follows:
...
all-articles: $(ALL_HTML_FILES)
%: %.gz
gunzip ...
%: %.md
markdown ...
%: %.zip
unzip ...
And let make figure out the path to take based on the sequence of extensions.
From the documentation however, I understand that there are constraints on match-all rules, and the above is not possible.
What's the best way forward? Can make handle this situation at all?
Extensions are made up examples. My actual source files make more sense :-)
I'm on holiday so I'll bite.
I'm not a fan of pattern rules, they are too restricted and yet too arbitrary at the same time for my tastes. You can achieve what you want quite nicely in pure make:
.DELETE_ON_ERROR:
all: # Default target
files := a.html.md.gz b.html.gz
cmds<.gz> = gzip -d <$< >$#
cmds<.md> = mdtool $< -o $#
define rule-text # 1:suffix 2:basename
$(if $(filter undefined,$(flavor cmds<$1>)),$(error Cannot handle $1 files: [$2$1]))
$2: $2$1 ; $(value cmds<$1>)
all: $2
endef
emit-rule = $(eval $(call rule-text,$1,$2))# 1:suffix 2:basename
emit-hierachy = $(if $(suffix $2),$(call emit-rule,$1,$2)$(call emit-hierachy,$(suffix $2),$(basename $2)))# 1:suffix 2:basename
emit-rules = $(foreach _,$1,$(call emit-hierachy,$(suffix $_),$(basename $_)))# 1:list of source files
$(call emit-rules,${files})
.PHONY: all
all: ; : $# Success
The key here is to set $files to your list of files.
This list is then passed to emit-rules.
emit-rules passes each file one-at-a-time to emit-hierachy.
emit-hierachy strips off each extension in turn,
generates the appropriate make syntax, which it passes to $(eval …).
emit-hierachy carries on until the file has only one extension left.
Thus a.html.md.gz becomes this make syntax:
a.html.md: a.html.md.gz ; gunzip <$< >$#
a.html: a.html.md ; mdtool $< -o $#
all: a.html
Similarly, b.html.gz becomes:
b.html: b.html.gz ; gunzip <$< >$#
all: b.html
Neato, or what?
If you give emit-rules a file with an unrecognised extension (c.html.pp say),
you get a compile-time error:
1:20: *** Cannot handle .pp files: [c.html.pp]. Stop.
Compile-time? Yeah, before any shell commands are run.
You can tell make how to handle .pp files by defining cmds<.pp> :-)
For extra points it's also parallel safe. So you can use -j9 on your 8 CPU laptop, and -j33 on your 32 CPU workstation. Modern life eh?
In a makefile is there any way to specifiy the same element of a list (I think this is the correct term) more than once? For example, having a a list with 3 files A.txt B.txt C.txt, I'd like to create targets based on these elements that would be contained within a directory where the pattern would have to be repeated twice: A/A-rambo B/B-rambo C/C-rambo.
I tried:
TXT = A.txt B.txt C.txt
DIR := $(patsubst %.txt,%,$(TXT))
OUT := $(patsubst %,%/%-rambo,$(DIR))
$(info $(OUT))
but this prints A/%-rambo B/%-rambo C/%-rambo, where the second %is not being replaced.
And so does OUT := $(patsubst %,$(addsuffix /%-rambo,%),$(DIR))
Make cannot handle a pattern rule with two wildcards in the target name, even if they're required to have the same value.
There's more than one way to get the effect you want, none ideal. I'd suggest generating a rule for each txt file:
define template
$(1)/$(1)-rambo: $(1).txt
#echo building $$# from $$^
endef
STEMS := A B C
$(foreach x,$(STEMS),$(eval $(call template,$(x))))
Try:
OUT := $(foreach T,$(TXT),$(T:.txt=)/$(T:.txt=-rambo))
I have a source file in a declarative language (twolc, actually) that I need to write many variations on: a normative version and many non-normative versions, each with one or more variations from the norm. For example, say the normative file has three rules:
Rule A:
Do something A-ish
Rule B:
Do something B-ish
Rule C:
Do something C-ish
Then one variation might have the exact same rules as the norm for A and C, but a different rule for B, which I will call B-1:
Rule A:
Do something A-ish
Rule B-1:
Do something B-ish, but with a flourish
Rule C:
Do something C-ish
Imagine that you have many different subtle variations on many different rules, and you have my situation. The problem I am worried about is code maintainability. If, later on, I decide that Rule A needs to be refactored somehow, then I will have 50+ files that need to have the exact same rule edited by hand.
My idea is to have separate files for each rule and concatenate them into variations using cat: cat A.twolc B.twolc C.twolc > norm.twolc, cat A.twolc B-1.twolc C.twolc > not-norm.twolc, etc.
Are there any tools designed to manage this kind of problem? Is there a better approach than the one I have in mind? Does my proposed solution have weaknesses I should watch out for?
As you added the makefile tag, here is a GNU-make-based (and Gnu make only) solution:
# Edit this
RULES := A B B-1 C
VARIATIONS := norm not-norm
norm-rules := A B C
not-norm-rules := A B-1 C
# Do not edit below this line
VARIATIONSTWOLC := $(patsubst %,%.twolc,$(VARIATIONS))
all: $(VARIATIONSTWOLC)
define GEN_rules
$(1).twolc: $$(patsubst %,%.twolc,$$($(1)-rules))
cat $$^ > $$#
endef
$(foreach v,$(VARIATIONS),$(eval $(call GEN_rules,$(v))))
clean:
rm -f $(VARIATIONSTWOLC)
patsubst is straightforward. The foreach-eval-call is a bit more tricky. Long story short: it loops over all variations (foreach). For each variation v, it expands (call) GEN_rules by replacing $(1) by $(v) (the current variation) and $$ by $. Each expansion result is then instantiated (eval) as a normal make rule. Example: for v=norm, the GEN_rules expansion produces:
norm.twolc: $(patsubst %,%.twolc,$(norm-rules))
cat $^ > $#
which is in turn expanded as (step-by-step):
step1:
norm.twolc: $(patsubst %,%.twolc,A B C)
cat $^ > $#
step2:
norm.twolc: A.twolc B.twolc C.twolc
cat $^ > $#
step3:
norm.twolc: A.twolc B.twolc C.twolc
cat A.twolc B.twolc C.twolc > norm.twolc
which does what you want: if norm.twolc does not exist or if any of A.twolc, B.twolc, C.twolc is more recent than norm.twolc, the recipe is executed.
I have the following project structure:
+-Makefile
+-src/
+-a/
| +-foo.py
+-b/
| +-foo.py
+-c/
| +-foo.py
Each foo.py file is a different file with exactly the same name (ie they have different inodes but are all literally called 'foo.py' - although of course in reality the name is not foo.py, which is just an example).
I wish to create a GNU Makefile rule which, when run, creates the following structure:
+-Makefile
+-src/
+-a/
| +-foo.py
| +-a.zip
+-b/
| +-foo.py
| +-b.zip
+-c/
| +-foo.py
| +-c.zip
This is the closest I have been able to figure out, although of course it fails due to using a target variable in the prereqs which doesn't seem to be allowed:
SRCDIR = src/
PRJ_DIRS = a b c
SRC_FILE = foo.py
# This next rather nasty line turns PRJ_DIRS in to, e.g., src/a/a.zip etc.
ZIP_FILES = $(addprefix $(SRCDIR),$(join $(PRJ_DIRS),$(PRJ_DIRS:%=/%.zip)))'
build: $(ZIP_FILES)
# Next line crashes because we can't use $# in the prereq
$(ZIP_FILES): $(addprefix $(dir $#), $(SRC_FILE))
touch $#
So one way to put the question:
How can I write a rule the uses each corresponding foo.py file as a prereq to build the appropriate .zip file?
Or, a more direct question that might actually be looking at this from the wrong angle:
How can I refer to the specific target that's being built in the prereqs?
Okay, since you consider secondary expansion a kludge, it's $(foreach ...) and $(eval ...) to the rescue again (a very powerful combination).
Replace your rule with the following and you should get what you want.
define rule
$(SRCDIR)$(1)/$(1).zip: $(SRCDIR)$(1)/$(SRC_FILE)
touch $$#
endef
$(foreach dir, $(PRJ_DIRS), $(eval $(call rule,$(dir))))
I want to write something like regex:
SRC:="a.dat.1 a.dat.2"
$(SRC): %.dat.%: (\\1).rlt.(\\2)
dat2rlt $^ $#
so that a.dat.1 and a.dat.2 will give a.rlt.1 and a.rlt.2.
In GNU Make info page, it says "the % can be used only once".
Is there some trick to achieve this in GNU Make?
I'm afraid what you are trying to do is not possible the way you suggest to do it, since - as you already mention - (GNU) make only allows a single stem '%', see http://www.gnu.org/software/make/manual/make.html#Pattern-Rules:
A pattern rule looks like an ordinary rule, except that its target
contains the character ‘%’ (exactly one of them).
Without it, creating such 'multi-dimensional' targets is cumbersome.
One way around this is by rebuilding the name of the dependency in the command (rather than in the dependency list):
SRC := a.dat.1 a.dat.2
all : $(SRC:%=%.dat2rlt)
%.dat2rlt :
dat2rtl $(word 1,$(subst ., ,$*)).rlt.$(word 2,$(subst ., ,$*)) $*
Of course, however, this way you would lose the dependency, it will not rebuild once the rlt has been updated.
The only way I can see to address that is by generating the rules explicitly:
SRC := a.dat.1 a.dat.2
all : $(SRC)
define GEN_RULE
$1.dat.$2 : $1.rlt.$2
dat2rtl $$< $$#
endef
$(foreach src,$(SRC),$(eval $(call GEN_RULE,$(word 1,$(subst ., ,$(src))),$(word 3,$(subst ., ,$(src))))))
Using named variables, we can write more readable code (based on answer of Paljas):
letters:=a b c
numbers:=1 2 3 4
define GEN_RULE
$(letter).dat.$(number) : $(letter).rlt.$(number)
./rlt2dat $$< $$#
endef
$(foreach number,$(numbers), \
$(foreach letter,$(letters), \
$(eval $(GEN_RULE)) \
) \
)
We can generate SRC in a similar way. Note that using that method SRC will contain all the combinations. That may or may not be beneficial.
Building on the answer of Erzsébet Frigó, you might additionally choose to:
in the inner loop, eval not the macro itself but the result of calling it
name the macro after program you're calling, dat2rtl
in combination, allowing you to
refer to the program name using make's ${0}
define a target, ${0}s (expanding to dat2rts - note the pluralization) with preconditions of all combinations of letters and numbers on which dat2r2 was called
Like this:
letters:=a b c
numbers:=1 2 3 4
define rlt2dat
${0}s::$(letter).dat.$(number)
$(letter).dat.$(number): $(letter).rlt.$(number)
./${0} $$< $$#
endef
$(foreach number,$(numbers), \
$(foreach letter,$(letters), \
$(eval $(call rlt2dat))))
allowing you to build all rlt2dat targets as:
make rlt2dats
For the limited example you gave, you can use a pattern with one %.
SRC := a.dat.1 a.dat.2
${SRC}: a.dat.%: a.rlt.%
dat2rlt $^ $#
$* in the recipe will expand to whatever the % matched.
Note that the "s around your original macro are definitely wrong.
Have a look at .SECONDEXPANSION in the manual for more complicated stuff (or over here).