I have 6 targets and 1 dependency in my make rule. The target files have the following names when produced:
mm10.1.ebwt
mm10.2.ebwt
mm10.3.ebwt
mm10.4.ebwt
mm10.rev.1.ebwt
mm10.rev.2.ebwt
Is there anyway I can forgo listing all of these separately in the rule? For instance is there anyway I can specify two general files like so:
mm10.[1-4].ebwt mm10.rev.[1-2].ebwt: mm10.fa
# build genome index
bowtie-build $^
Edit for kebs:
On the command line running the following command:
bowtie build mm10.fa
will result in six files being produced with prefix mm10:
mm10.1.ebwt, mm10.2.ebwt, mm10.3.ebwt, mm10.4.ebwt, mm10.rev.1.ebwt, mm10.rev.2.ebwt
I would like to abstract the specific names to create a make rule for bowtie build, but listing each file produced results in a long line, I was wondering if there was a way to make it neater by defining a range of filenames produced.
Related
In my project, I have a set of sub-directories that contain package.yaml files, for e.g.:
A/package.yaml
B/package.yaml
C/package.yaml
If I run hpack A/package.yaml, the file A/A.cabal is (re-)generated. The list of such directories can change over time, so I want to use GNU make to find all immediate sub-directories containing package.yaml files and generate the corresponding .cabal files using hpack.
I tried this based on another question, but it didn't work:
HPACK_FILES := $(wildcard */package.yaml)
PKG_DIRS := $(subst /,,$(dir $(HPACK_FILES)))
CABAL_FILES := $(addsuffix .cabal,$(join $(dir $(HPACK_FILES)),$(PKG_DIRS)))
test:
#echo $(CABAL_FILES)
update-cabal: $(CABAL_FILES)
%.cabal: package.yaml
hpack $<
However, make update-cabal says there's nothing to be done. make test however does output the right cabal files. How can I fix this?
Cheers!
The problem is this:
%.cabal: package.yaml
There is no file package.yaml. The files are named things like A/package.yaml. That is not the same thing.
Because the prerequisite doesn't exist, make decides that this pattern rule cannot match and so it goes looking for another rule that might be able to build the target. It doesn't find any rule that can build the target, so make says there's nothing to do because all the output files already exist.
Unfortunately what you want to do is not at all easy with make, because make is most comfortable with input and output files that are tied together by the filename with extensions, or similar. And in particular, it has a really hard time with relationships where the variable part is repeated more than once (as in, A/A.cabal where the A is repeated). There's no easy way to do that in make.
You'll have to use an advanced feature such as eval to do this. Something like:
# How to build a cabal file
%.cabal:
hpack $<
# Declare the prerequisites
$(foreach D,$(dir $(HPACK_FILES)),$(eval $D/$D.cabal: $D/package.yml))
Here is the make file that I am running,
.PHONY: build
build: pre_build_script $(OUTPUTDIR)/%.cpp
$(OUTPUTDIR)/%.cpp: $(INTXTDIR)/%.txt
python.exe $(SOMEDIR)/somepythonscript.py $(INTXTDIR) $(OUTPUTDIR)
.PHONY: pre_build_script
pre_build_script:
pythonscript.exe $(PREBUILDDIR)
This is the output that I get:
$ make build
pythonscript.exe $(SAMPLEDIR)
make: *** No rule to make target '../obj/CPP/%.cpp', needed by 'build'. Stop.
Looks like I'm missing on some sytanx as I get this error inspite of declaring the target dependency. Any suggestions?
This means make cannot find a file named $(OUTPUTDIR)/%.cpp, a prerequisite for the first rule.
You cannot use % as a wildcard anywhere in a rules like this:
build: pre_build_script $(OUTPUTDIR)/%.cpp
it needs to be a part of pattern rule or a static pattern rule.
You can use $(wildcard $(OUTPUTDIR)/*.cpp) to get a complete list of files, but it's an anti-pattern (pun intended). You are supposed to either exactly know what files are used in what rules, or (know it even better and) create a generic pattern rule.
The second pattern rule (one using somepythonscript.py) is supposed to work on a single source-target file pair, $(INTXTDIR)/%.txt -> $(OUTPUTDIR)/%.cpp. The command seems to process all the files in the directory, which is not incremental: it will redo all the work even if only one file was updated.
I have this version of makefile
[sbsuser#compute-00-01 415]$ make --version GNU Make 3.81
I have directory SOMATIC where I have 3 file . I want to produce a only one output. This is what I wrote.
`
OUTSOMATIC=SOMATIC
FINAL=FINAL
INPUT=$(wildcard $(OUTSOMATIC)/*.vcf)
OUTSORT2= $(patsubst $(OUTSOMATIC)/%.vcf,$(FINAL)/%somatic.ensemble.gz,$(INPUT))
$(info lista $(OUTSORT2))
$(info lista $(INPUT))
.PHONY: all
all: $(INPUT) $(OUTSOMATIC) $(OUTSORT2) $(FINAL)
$(FINAL)/%somatic.ensemble.gz: $(OUTSOMATIC)/%.vcf $(INPUT)
~/jdk1.8.0_121/bin/java -XX:+UseSerialGC -Xms1g -Xmx10g -jar /illumina/software/PROG2/bcbio-variation-recall-0.1.7 ensemble -n 1 $(FINAL)/somatic_ensemble.gz /illumina/software/database/database_2016/hg19_primary.fa $^
`
With this script make 3 time the same files. I don't understand how to create only one output from list of input to use in the same time.
What is the best way to do this?
If I change $(FINAL)/%somatic.ensemble.gz: in $(FINAL)/somatic.ensemble.gz I have this error:
make: *** No rule to make target FINAL/415_merge_mutect2.somaticsomatic.ensemble.gz', needed byall'. Stop`
You probably should review the GNU make manual introductory sections where they describe how make works.
Let's look at your makefile; first you define some variables. Let's assume that you have the files SOMATIC/foo.vcf, SOMATIC/bar.vcf, and SOMATIC/baz.vcf. Then the variables you created will have these values, after they are expanded:
OUTSOMATIC = SOMATIC
FINAL = FINAL
INPUT = SOMATIC/foo.vcf SOMATIC/bar.vcf SOMATIC/baz.vcf
Now your patsubst finds all words in INPUT that match the pattern SOMATIC/%.vcf and replace that with FINAL/%somatic.ensemble.gz, where the part that matches the % in the input is substituted into the output:
OUTSORT2 = FINAL/foosomatic.ensemble.gz FINAL/barsomatic.ensemble.gz FINAL/bazsomatic.ensemble.gz
Now, make sees that you've defined an all target. Since it's the first target in the makefile this is the target that will be run by default. After expansion, it will look like this:
all: SOMATIC/foo.vcf SOMATIC/bar.vcf SOMATIC/baz.vcf SOMATIC FINAL/foosomatic.ensemble.gz FINAL/barsomatic.ensemble.gz FINAL/bazsomatic.ensemble.gz FINAL
So, make will try to build every prerequisite of the all target to be sure it's up to date. First it tries to build the SOMATIC/*.vcf files. Those files already exist and make doesn't have any rules about how to rebuild them, so it assumes they're up to date.
Next it tries to build the SOMATIC file. This is a directory and it also has no rule to be built, so make assumes that's up to date as well.
Next make tries to build the target FINAL/foosomatic.ensemble.gz. Make does have a rule that can build it, you've created one:
$(FINAL)/%somatic.ensemble.gz: $(OUTSOMATIC)/%.vcf $(INPUT)
~/jdk1.8.0_121/bin/java ...
This matches the target you want to build, with a % value of foo, so then make substitutes the % in the prerequisite for foo and finds that SOMATIC/foo.vcf exists and doesn't need to be rebuilt, so it runs your recipe. However your recipe doesn't actually create the target FINAL/foosomatic.ensemble.gz; it creates the target FINAL/somatic_ensemble.gz. So this rule is broken because it tells make it will do one thing, but it does something else.
You should always ensure all your recipes build the file represented by the automatic variable $#; that will ensure that you and make agree on the meaning of your rule. If you want your recipe to build some other file, then your rule is written incorrectly.
Next make does the same thing with the next prerequisite of all: FINAL/barsomatic.ensemble.gz. Since that file doesn't exist, make tries to build it using the pattern rule, but again that creates the same output file.
And again for the third .gz file FINAL/bazsomatic.ensemble.gz. That's why things are run three times.
If you change the pattern rule to an explicit rule building FINAL/somatic.ensemble.gz, which is what you want, then make can't find any way to build the prerequisites of the all target so it gives this error.
Your problem is the creation of OUTSORT2. You want to create only one output file, but you've set OUTSORT2 to contain three different files, so make tries to create all three files. You want this:
OUTSOMATIC = SOMATIC
FINAL = FINAL
INPUT = $(wildcard $(OUTSOMATIC)/*.vcf)
OUTSORT2 = $(FINAL)/somatic.ensemble.gz
.PHONY: all
all: $(OUTSORT2)
$(OUTSORT2): $(INPUT)
~/jdk1.8.0_121/bin/java -XX:+UseSerialGC -Xms1g -Xmx10g -jar /illumina/software/PROG2/bcbio-variation-recall-0.1.7 ensemble -n 1 $# /illumina/software/database/database_2016/hg19_primary.fa $^
My project consists of some configuration files and they are generated from the corresponding templates. e.g. foo.tmpl => foo. Now I wish to write a Makefile that can generate the corresponding file from the template.
I couldn't have something like this:
% : %.tmpl
generate-from-tmpl $<
Since it apply the target to all possible files, however, I only want to restrict its targets to those with a .tmpl correspondent. Now I have acquired the list of all templated files:
TEMPLATED_FILES=$(shell find -type f -name "*.tmpl")
GENERATED_FILES=$(TEMPLATED_FILES:.tmpl=)
I wish to have something that looks like:
$(foreach GENERATED_FILES) : $#.tmpl
generate-from-tmpl $<
How can I achieve that? thanks.
%: %.tmpl will attempt to match the target % against any target name but if no matching %.tmpl file exists then make will skip that pattern for that target entirely.
I believe that's what you want exactly. Do you have some reason not to think so?
That being said if you want to be more specific what you want is a Static Pattern Rule:
$(GENERATED_FILES) : % : %.tmpl
generate-from-tmpl $<
which will only apply to the files in $(GENERATED_FILES).
Also, if your template files are only one directory deep you can use:
TEMPLATED_FILES=$(wildcard *.tmpl)
instead of the shell and find (there are also recursive make wildcard defines available but find is reasonable if you need that).
Hi I have a build script called "buildMyJava" that builds a bunch of Java source code. Assuming those source code are in differnet directories such as "folder1" and "folder2", the output goes to some folder called "classes". How do I create a makefile so it KNOWS to build only when the source code meaning the *.java in those two directories have changed as well as the output classes is missing?
I have something like the following but it ALWAYS builds, dependencies are not working.
all: task
task: folder1/*.java folder2/*.java classes/
buildMyJava
First of all, the build script produces the .java files, thus the .java files should be targets, not prerequisites. So you should have something like this:
folder1/%.java folder2/%.java:
buildMyJava
The only problem with this is that if you do a make -j2, buildMyJava will run multiple times (once for folder1, and once for folder2). In fact, this is a limitation to makefiles -- you cannot have multiple targets invoke the same recipe only once. There is a good discussion on this here: http://www.cmcrossroads.com/article/rules-multiple-outputs-gnu-make
Notice though that a 'pattern' target counts as a single target though -- which means, if you can get a pattern to match all targets, you can invoke the recipe only once. A small caveat to that -- the % symbol cannot represent /'s. Thus you cannot do folder%.java, as that would not match folder1/file1.java... If you can split your script to output only to one directory at a time though, you may be able to do the following:
folder1/%.java:
buildMyJava folder1
folder2/%.java:
buildMyJava folder2
John