Here is a simple Makefile:
all: actual.txt test1.txt test2.txt
actual.txt: header.txt actual_part1.txt actual_part2.txt footer.txt
cat $^ > $#
test1.txt: header.txt actual_part1.txt test1_part2.txt footer.txt
cat $^ > $#
test2.txt: header.txt test2_huge_part.txt footer.txt
cat $^ > $#
As you can see, all three targets are built with the same command, repeated three times. Is it possible to somehow reduce the repetition?
Given the makefile as you've provided it, there are a few ways.
One is to put the recipe into a variable:
BLDTXT = cat $^ $#
all: actual.txt test1.txt test2.txt
actual.txt: header.txt actual_part1.txt actual_part2.txt footer.txt
$(BLDTXT)
test1.txt: header.txt actual_part1.txt test1_part2.txt footer.txt
$(BLDTXT)
test2.txt: header.txt test2_huge_part.txt footer.txt
$(BLDTXT)
That doesn't reduce the number of lines to write, but it does allow you to make changes to the recipe in one place.
Another option is to create a single rule and move the prerequisite definitions to other lines, like this:
BLDTXT = cat $^ $#
ALLTXT = actual.txt test1.txt test2.txt
all: $(ALLTXT)
$(ALLTXT):
$(BLDTXT)
actual.txt: header.txt actual_part1.txt actual_part2.txt footer.txt
test1.txt: header.txt actual_part1.txt test1_part2.txt footer.txt
test2.txt: header.txt test2_huge_part.txt footer.txt
Related
There are a couple of kind of similar issues but I could not fit any of the proposed concepts to my case.
Just to give a little bit of context: I have a set of Julia files which create plots as PDFs which are part of a make procedure to create scientific papers, something like:
plots = $(shell find $(PLOT_PATH)/*.jl | sed 's/\.jl/\.pdf/g')
$(PLOT_PATH)/%.pdf: $(PLOT_PATH)/%.jl $(JULIA_SYSIMAGE)
$(JL) --project $< -o $(PLOT_PATH)
$(DOCUMENT_FILENAME).pdf: FORCE $(plots) $(figures)
latexmk $(DOCUMENT_FILENAME).tex
In the current setup, each XYZ.jl file is creating a XYZ.pdf file and it works absolutely fine.
Now I am dealing with cases where it would be much easier to create multiple plots from single Julia files, so a script like this:
#!/usr/bin/env julia
using PGFPlotsX
...
...
pgfsave("whatever.pdf")
pgfsave("another.pdf")
pgfsave("yetanother.pdf")
so that one could do a grep pgfsave SCRIPT | awk... to figure out the targets. However, I could not figure out how to generate dynamic targets (plots) based on the contents of the dependency file (Julia script).
An MWE for my problem is the following: I have a couple of files (dependencies) which are generating a bunch of targets, which are defined inside those files (and can be access via awk/grep/sed/whatever). For now, let's say that these are simply *.txt files and each line is a target.
file: a.txt
foo
bar
baz
file: b.txt
naarf
fjoord
A very basic (non-working) manual Makefile to demonstrate the goal would be something like this (it does not work as it cannot figure out how to make foo etc. but it shows the pattern for *.txt which needs to be repeated):
file: Makefile
all_products := $(shell find *.txt | xargs cat)
final_product: $(all_products)
echo $< > $#
(foo bar baz): a.txt
touch $(shell cat $<)
(narf fjoord): b.txt
touch $(shell cat $<)
so in principle, I need something to "process" the dependency (*.txt) to create a list of the targets, like
$(shell cat $%): %.txt
echo $< > $#
but I cannot manage to get a reference to the dependency on the target side ($% does not work).
Any ideas? Maybe the whole approach is just a bad idea ;)
A combination of GNU make foreach, eval and call functions is probably what you need. With your example:
TXT := $(wildcard *.txt)
.PHONY: all
.DEFAULT_GOAL := all
define MY_MACRO
$(1)-targets := $$(shell cat $(1))
$$($(1)-targets): $(1)
echo $$< > $$#
all: $$($(1)-targets)
endef
$(foreach t,$(TXT),$(eval $(call MY_MACRO,$(t))))
(pay attention to the $$ in the macro definition, they are needed). And then:
$ make
make
echo a.txt > foo
echo a.txt > bar
echo a.txt > baz
echo b.txt > naarf
echo b.txt > fjoord
If you want the recipe to build all targets at once you'll need a recent enough GNU make version (4.3 or later) and its new rule with grouped targets (x y z&: w):
TXT := $(wildcard *.txt)
.PHONY: all
.DEFAULT_GOAL := all
define MY_MACRO
$(1)-targets := $$(shell cat $(1))
$$($(1)-targets)&: $(1)
touch $$($(1)-targets)
all: $$($(1)-targets)
endef
$(foreach t,$(TXT),$(eval $(call MY_MACRO,$(t))))
And then:
$ make
touch foo bar baz
touch naarf fjoord
Note that in this case we could also use a simpler and less GNU make-dependent solution. Just use empty dummy files as time stamps, for instance .a.txt.tag for a.txt, and a static pattern rule:
TXT := $(wildcard *.txt)
TAG := $(patsubst %,.%.tag,$(TXT))
.PHONY: all
all: $(TAG)
$(TAG): .%.tag: %
touch `cat $<` $#
All
I just want to using $# at prerequisite in a Makefile like this
test.v: $(#:%.v=%.tv)
echo $< $#
It suppose output
test.tv test.v
but it only output
test.v
Anything I miss?
In order to reference target variable from dependency list you first need to enable secondary expansion, then you can delay evaluation of targets and functions with $$ instead of $, i.e.:
$ cat Makefile
.SECONDEXPANSION:
test.v: $$(patsubst %.v,%.tv,$$#)
echo $# $<
test.tv:
$ make
echo test.v test.tv
test.v test.tv
Note that you cannot use shortened notation of $$(#:%.v=%.tv) due to some parser limitations around static pattern rules.
Anyway, what you are trying to do is doable with simple pattern rules:
$ cat Makefile2
%.v: %.tv
echo $# $<
%.tv: ;
$ make -f Makefile2 test.v
echo test.v test.tv
test.v test.tv
This would make a generic rule for creating .v files out of .tv files, but will require explicit call of a target (as a pattern rule does not create a specific file to build on its own).
If you wish to limit allowed file naming to follow the rule, you may go with a static pattern rule:
$ cat Makefile3
test.v: %.v: %.tv
echo $# $<
test.tv:
$ make -f Makefile3
echo test.v test.tv
test.v test.tv
I'm trying to take some source files, create some customised versions of those sources, then process those customised sources down to output files I can use. I'm using this make file. Note that this file is not fully complete, currently it only does the CSS* make, once that's working I will add the PHP* make which is similar:
# root sources
CSSSOURCES = $(wildcard *.scss)
PHPSOURCES = $(wildcard *.phtml)
# partials, creates a configed source
CSSMSOURCES = $(addprefix m.,$(CSSSOURCES:.scss=.m))
CSSDSOURCES = $(addprefix d.,$(CSSSOURCES:.scss=.d))
PHPMSOURCES = $(addprefix m.,$(PHPSOURCES:.phtml=.m))
PHPDSOURCES = $(addprefix d.,$(PHPSOURCES:.phtml=.d))
# targets
CSSMTARGETS = $(CSSMSOURCES:.m=.css)
CSSDTARGETS = $(CSSDSOURCES:.d=.css)
PHPMTARGETS = $(PHPMSOURCES:.m=.php)
PHPDTARGETS = $(PHPDSOURCES:.d=.php)
# ensure no clash with built in rules
.SUFFIXES: .m .d .scss .css .phtml .php
all: $(CSSMTARGETS)
%.m: %.scss
echo "%define MOBILE" | cat - $< >tmp
mv tmp $#
%.d: %.scss
echo "%define DESKTOP" | cat - $< >tmp
mv tmp $#
%.css: %.m %d
cat $< | mym1.pl >$#
rm $<
.PHONY: test
test:
#echo "sources - $(CSSSOURCES)"
#echo "msources - $(CSSMSOURCES)"
#echo "targets - $(CSSMTARGETS)"
Instead of creating the CSS targets I get this error:
make: *** No rule to make target 'm.page.css', needed by 'all'. Stop.
make operates on file names; if there is no file named m.page.m and no file named m.page.d - which is what m.page.css depends on in accordance with your declared dependencies - then Make will conclude that it needs to create these. If it does not have any rules (built-in or by way of a recipe) to create those, the error message you get tells you pretty much exactly that.
I'm guessing what you actually want is something like
m.%: %
echo "%define MOBILE" | cat - $< >$#
d.%: %
echo "%define DESKTOP" | cat - $< >$#
This tells make how to create m.whatever and d.whatever from whatever; so it now knows how to create m.x.phtml from x.phtml and d.y.scss from y.scss etc.
(Notice also how this avoids the separate mv of a static temporary file name, as discussed in comments.)
%.css: %.scss
mym1.pl <$< >$#
%.php: %.phtml
mym1.pl <$< >$#
This tells make how to create m.z.css from m.z.scss, d.w.php from d.w.phtml, etc. I'm guessing here that mym1.pl can handle both cases.
(Notice also the refactoring to avoid useless use of cat.)
This does away with the somewhat mysterious .m and .d suffixes so you probably have to refactor the variables at the top of your Makefile.
Instead of creating the CSS targets I get this error:
make: *** No rule to make target 'm.page.css', needed by 'all'. Stop.
Your makefile provides exactly one rule by which m.page.css could be built or updated:
%.css: %.m %d
cat $< | mym1.pl >$#
rm $<
In order for that rule to apply, however, both of the prerequisites, expressed as %.m and %d [sic] need to exist or be able to be built. There are problems with both.
The former in this case represents a file m.page.m. This apparently does not already exist, but you have a rule by which it could be built from m.page.scss. Except you don't have an m.page.scss or any rule to build one. The corresponding root source file is actually page.sccs.
The latter prerequisite, %d, would have an analogous problem, but it doesn't even get that far because you have omitted a period from the prerequisite name (should be %.d, not %d).
The rule quoted above is moreover bogus because it designates two prerequisites but only uses one. That's not inherently wrong, but it does not do anything useful to serve you here.
Replacing your current pattern rules with these should help:
m.%.m: %.scss
echo "%define MOBILE" | cat - $< >tmp
mv tmp $#
d.%.d: %.scss
echo "%define DESKTOP" | cat - $< >tmp
mv tmp $#
%.css: %.m
cat $< | mym1.pl >$#
rm $<
%.css: %.d
cat $< | mym1.pl >$#
rm $<
Pattern rules work:
$ ls
Makefile hello.txt world.txt
$ cat Makefile
all: hello.out world.out
%.out: %.txt
cp $< $#
$ make
cp hello.txt hello.out
cp world.txt world.out
However, when I try to replace them with what I think is the exact equivalent suffix rules, they don't work:
$ ls
Makefile hello.txt world.txt
$ cat Makefile
.POSIX:
.SUFFIXES:
.SUFFIXES:.txt.out
all: hello.out world.out
.txt.out:
cp $< $#
$ make
make: *** No rule to make target 'hello.out', needed by 'all'. Stop.
I don't understand why.
This line is the problem:
.SUFFIXES: .txt.out
It declares a single suffix, .txt.out, not two of them. You can change it to this:
.SUFFIXES: .txt .out
I have write a makefile that prepare some files. I create ORIGINAL directory and then I use the file inside the folder for start the others rules
RDIR=.
RFILES:=$(wildcard $(RDIR)/*.vcf)
OUTDIR=ORIGINAL
OUTFILES=$(patsubst %.vcf,$(OUTDIR)/%.gz,$(RFILES))
BCFTOOLS=bcftools
OUTSOMATIC=SOMATIC
OUTVARDICT=$(patsubst
$(OUTDIR)/%vardict.gz,$(OUTSOMATIC)/%.somatic.vcf,$(wildcard
$(OUTDIR)/*vardict.gz))
OUTMUTEC2=$(patsubst
$(OUTDIR)/%mutect2_all.gz,$(OUTSOMATIC)/%mutect2.somatic.vcf,$(wildcard
$(OUTDIR)/*mutect2_all.gz))
OUTVARSCAN2=$(patsubst
$(OUTDIR)/%varscan.gz,$(OUTSOMATIC)/%varscan2.somatic.vcf,$(wildcard
$(OUTDIR)/*varscan.gz))
.PHONY: all
all: $(OUTDIR) $(OUTFILES) $(OUTSOMATIC) $(OUTVARDICT) $(OUTMUTEC2)
$(OUTVARSCAN2)
$(OUTDIR)/%.gz: %.vcf
bgzip -c $< > $#
$(OUTDIR):
test -d $# || mkdir $#
$(OUTSOMATIC):
test -d $# || mkdir $#
$(OUTSOMATIC)/%.somatic.vcf: $(OUTDIR)/%vardict.gz
$(BCFTOOLS) view -f PASS -i 'INFO/STATUS ~ ".*Somatic"' $< > $#
$(OUTSOMATIC)/%mutect2.somatic.vcf: $(OUTDIR)/%mutect2_all.gz
$(BCFTOOLS) view -f PASS $< > $#
$(OUTSOMATIC)/%varscan2.somatic.vcf: $(OUTDIR)/%varscan.gz
$(BCFTOOLS) view -f PASS -i 'SS="2"' $< > $#
clean:
rm -rf $(OUTDIR)
rm -rf $(OUTSOMATIC)
I need to launch 3 time make -f Makefile for execute all the rules. How
can improve that script?
What is the right way?
thanks for any help
If I understand you correctly, your makefile is zip vcf files in one directory into gz files in a second directory, then use those gz files to build vcf files in a third directory (building the directories as needed), and that those final vcf files are the real goal.
You can do it in one pass, if you modify the variable assignments to derive target names from the planned gz files, not the gz files that already exist:
OUTVARDICT=$(patsubst $(OUTDIR)/%vardict.gz,$(OUTSOMATIC)/%.somatic.vcf,$(filter $(OUTDIR)/%vardict.gz, $(OUTFILES)))
OUTMUTEC2= $(patsubst $(OUTDIR)/%mutect2_all.gz, $(OUTSOMATIC)/%mutect2.somatic.vcf, $(filter $(OUTDIR)/%mutect2_all.gz, $(OUTFILES)))
OUTVARSCAN2 = $(patsubst $(OUTDIR)/%varscan.gz,$(OUTSOMATIC)/%varscan2.somatic.vcf, $(filter $(OUTDIR)/%varscan.gz, $(OUTFILES)))
and modify the rules to allow Make to determine which intermediates to build:
all: $(OUTVARDICT) $(OUTMUTEC2) $(OUTVARSCAN2)
$(OUTDIR)/%.gz: %.vcf $(OUTDIR)
bgzip -c $< > $#
$(OUTDIR):
test -d $# || mkdir $#
$(OUTSOMATIC):
test -d $# || mkdir $#
$(OUTSOMATIC)/%.somatic.vcf: $(OUTDIR)/%vardict.gz $(OUTSOMATIC)
$(BCFTOOLS) view -f PASS -i 'INFO/STATUS ~ ".*Somatic"' $< > $#
$(OUTSOMATIC)/%mutect2.somatic.vcf: $(OUTDIR)/%mutect2_all.gz $(OUTSOMATIC)
$(BCFTOOLS) view -f PASS $< > $#
$(OUTSOMATIC)/%varscan2.somatic.vcf: $(OUTDIR)/%varscan.gz $(OUTSOMATIC)
$(BCFTOOLS) view -f PASS -i 'SS="2"' $< > $#