I'm trying to write a rule that will call my checking script on the output of each stage of my program, generating that output if it does not already exist using my %.output : %.input rule.
I tried check : $(wildcard stage[1234].output) but this causes the rule to require only those matching output files that already exist.
I could just define a variable like TARGETS = stage1.output stage2.output ..., but is there a way to generate all possible matches of a pattern and then require them?
for the %.output : %.input rule to apply, you need both
a goal that requires an intermediate that matches the pattern %.output
a corresponding %.input file - either preexisting or a rule to build it
If your stage*.input files already exist, you can use:
INPUTS=$(wildcard stage[1234].input)
TARGETS=$(INPUTS:%.input=%.output)
check: $(TARGETS)
If your stage*.input files don't exist but are expected to be built from similar pattern rules, reapply the same principle.
If your stage*.input are produced from more complicated means, but assuming that their name can be generated by the application of a substitution pattern simply apply that pattern. In your example it would be something like:
L:= 1 2 3 4
TARGETS=$(L:%=stage%.output)
If you want analogue of seq Unix utility in Make, here it is:
seq = $(if $(word $1,$2),$2,$(call seq,$1,$2 $(words $2 1)))
$(info seq(10)=$(call seq,10))
stage_sount:=7
stages:=$(patsubst %,stage%.output,$(call seq,$(stage_sount)))
$(info stages=$(stages))
all:
Output:
$ make
seq(10)= 1 2 3 4 5 6 7 8 9 10
stages=stage1.output stage2.output stage3.output stage4.output stage5.output stage6.output stage7.output
make: Nothing to be done for 'all'.
Related
Wondering how to apply something such as pattern rules to accomplish the following in the Makefile:
FILES := a b c d
$(FILES).c:
run build $(FILE).c
# use $(FILE) without .c extension, etc.
Without having to name all of the values in FILES like a.c, b.c, etc. because I am going to use the names elsewhere.
If I try the above with make a.c, I get:
No rule to make target `a.c'. Stop.
The addsuffix built-in function, which already operates on lists, could be useful here:
targets := $(addsuffix .c,$(FILES))
However, you could simply use pattern rules instead. The automatic variable $* would correspond to the stem (i.e., the % part):
%.c:
run build $#
# use $* for filename without .c extension, etc.
I have this version of makefile
[sbsuser#compute-00-01 415]$ make --version GNU Make 3.81
I have directory SOMATIC where I have 3 file . I want to produce a only one output. This is what I wrote.
`
OUTSOMATIC=SOMATIC
FINAL=FINAL
INPUT=$(wildcard $(OUTSOMATIC)/*.vcf)
OUTSORT2= $(patsubst $(OUTSOMATIC)/%.vcf,$(FINAL)/%somatic.ensemble.gz,$(INPUT))
$(info lista $(OUTSORT2))
$(info lista $(INPUT))
.PHONY: all
all: $(INPUT) $(OUTSOMATIC) $(OUTSORT2) $(FINAL)
$(FINAL)/%somatic.ensemble.gz: $(OUTSOMATIC)/%.vcf $(INPUT)
~/jdk1.8.0_121/bin/java -XX:+UseSerialGC -Xms1g -Xmx10g -jar /illumina/software/PROG2/bcbio-variation-recall-0.1.7 ensemble -n 1 $(FINAL)/somatic_ensemble.gz /illumina/software/database/database_2016/hg19_primary.fa $^
`
With this script make 3 time the same files. I don't understand how to create only one output from list of input to use in the same time.
What is the best way to do this?
If I change $(FINAL)/%somatic.ensemble.gz: in $(FINAL)/somatic.ensemble.gz I have this error:
make: *** No rule to make target FINAL/415_merge_mutect2.somaticsomatic.ensemble.gz', needed byall'. Stop`
You probably should review the GNU make manual introductory sections where they describe how make works.
Let's look at your makefile; first you define some variables. Let's assume that you have the files SOMATIC/foo.vcf, SOMATIC/bar.vcf, and SOMATIC/baz.vcf. Then the variables you created will have these values, after they are expanded:
OUTSOMATIC = SOMATIC
FINAL = FINAL
INPUT = SOMATIC/foo.vcf SOMATIC/bar.vcf SOMATIC/baz.vcf
Now your patsubst finds all words in INPUT that match the pattern SOMATIC/%.vcf and replace that with FINAL/%somatic.ensemble.gz, where the part that matches the % in the input is substituted into the output:
OUTSORT2 = FINAL/foosomatic.ensemble.gz FINAL/barsomatic.ensemble.gz FINAL/bazsomatic.ensemble.gz
Now, make sees that you've defined an all target. Since it's the first target in the makefile this is the target that will be run by default. After expansion, it will look like this:
all: SOMATIC/foo.vcf SOMATIC/bar.vcf SOMATIC/baz.vcf SOMATIC FINAL/foosomatic.ensemble.gz FINAL/barsomatic.ensemble.gz FINAL/bazsomatic.ensemble.gz FINAL
So, make will try to build every prerequisite of the all target to be sure it's up to date. First it tries to build the SOMATIC/*.vcf files. Those files already exist and make doesn't have any rules about how to rebuild them, so it assumes they're up to date.
Next it tries to build the SOMATIC file. This is a directory and it also has no rule to be built, so make assumes that's up to date as well.
Next make tries to build the target FINAL/foosomatic.ensemble.gz. Make does have a rule that can build it, you've created one:
$(FINAL)/%somatic.ensemble.gz: $(OUTSOMATIC)/%.vcf $(INPUT)
~/jdk1.8.0_121/bin/java ...
This matches the target you want to build, with a % value of foo, so then make substitutes the % in the prerequisite for foo and finds that SOMATIC/foo.vcf exists and doesn't need to be rebuilt, so it runs your recipe. However your recipe doesn't actually create the target FINAL/foosomatic.ensemble.gz; it creates the target FINAL/somatic_ensemble.gz. So this rule is broken because it tells make it will do one thing, but it does something else.
You should always ensure all your recipes build the file represented by the automatic variable $#; that will ensure that you and make agree on the meaning of your rule. If you want your recipe to build some other file, then your rule is written incorrectly.
Next make does the same thing with the next prerequisite of all: FINAL/barsomatic.ensemble.gz. Since that file doesn't exist, make tries to build it using the pattern rule, but again that creates the same output file.
And again for the third .gz file FINAL/bazsomatic.ensemble.gz. That's why things are run three times.
If you change the pattern rule to an explicit rule building FINAL/somatic.ensemble.gz, which is what you want, then make can't find any way to build the prerequisites of the all target so it gives this error.
Your problem is the creation of OUTSORT2. You want to create only one output file, but you've set OUTSORT2 to contain three different files, so make tries to create all three files. You want this:
OUTSOMATIC = SOMATIC
FINAL = FINAL
INPUT = $(wildcard $(OUTSOMATIC)/*.vcf)
OUTSORT2 = $(FINAL)/somatic.ensemble.gz
.PHONY: all
all: $(OUTSORT2)
$(OUTSORT2): $(INPUT)
~/jdk1.8.0_121/bin/java -XX:+UseSerialGC -Xms1g -Xmx10g -jar /illumina/software/PROG2/bcbio-variation-recall-0.1.7 ensemble -n 1 $# /illumina/software/database/database_2016/hg19_primary.fa $^
In my directory, I have many files named
A.xxx A01.xxx A02.xxx A03.xxx
B.xxx B01.xxx
Z......
I would like to create rules like
A.yyy : A.xxx
command type 1
A01.yyy : A01.xxx A.xxx
command type 2
B.yyy : B.xxx
command type 1
B01.yyy : B01.xxx B.xxx
command type 2
Z01.yyy: Z01.xxx
command type 3
Can someone give me an hint ?
I tried something like this (it seems ok for command 1 an 3, but in command 2 , I don't see how to put the second perequisit)
list:= A B
source=$(wildcard *.xxx)
compi:=$(patsubst %.xxx, %.yyy, $(source))
list_pattern=$(addsuffix %, $(list))
list_yyy=$(addsuffix .yyy, $(list))
list_nb_yyy=$(filter $(list_pattern), $(compi))
%.yyy:%xxx
#echo $# - $+ - command 3
all: $(compi)
.SECONDEXPANSION:
$(list_yyy): $$(patsubst %.yyy, %.xxx, $$#)
#echo $# - $+ - command 1
$(list_nb_yyy): $$(patsubst %.yyy, %.xxx, $$#)
#echp $# - $+ - command 2
Two important parts of the documentation that you should read include:
8.9 The eval Function
The eval function is very special: it allows you to define new makefile constructs that are not constant; which are the result of evaluating other variables and functions. The argument to the eval function is expanded, then the results of that expansion are parsed as makefile syntax. The expanded results can define new make variables, targets, implicit or explicit rules, etc.
10.5.4 How Patterns Match, particularly the shortest stem rule:
It is possible that more than one pattern rule will meet these criteria. In that case, make will choose the rule with the shortest stem (that is, the pattern that matches most specifically). If more than one pattern rule has the shortest stem, make will choose the first one found in the makefile.
The shortest stem rule will make it difficult to have 3 different pattern rules for A.yyy, A01.yyy and Z01.yyy respectively. However, with the eval function you can use the information available in the list variable to generate non-pattern rules for A.yyy and B.yyy, and the correct pattern rules for the other targets (assuming that the decision is made on the prefix of the name):
list:= A B
source=$(wildcard *.xxx)
compi:=$(patsubst %.xxx, %.yyy, $(source))
all: $(compi)
define LIST_RULE
$(1).yyy: $(1).xxx
#echo $$# - $$+ - command 1
$(1)%.yyy: $(1)%.xxx $(1).xxx
#echo $$# - $$+ - command 2
endef
$(foreach l,$(list),$(eval $(call LIST_RULE,$(l))))
%.yyy: %.xxx
#echo $# - $+ - command 3
Note that the fallback solution (command 3) won't be selected for A01.yyy and B01.yyy because the pattern rule in the LIST_RULE template leads to a shorter stem (01 instead of A01 or B01)
I have another make-like tool that produces an XML as an artifact after parsing my makefile which I'll then further process with Python.
It'd simplify things for me - a lot - if I could have make consider every single prerequisite to be an actual target because then this other tool
will classify each and every file as a "job".
This is a fragment of my makefile:
.obj/eventlookupmodel.o: C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h \
...
I'd want for make to think I have a dummy rule for each prerequisite such as below:
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h:
#echo target pre= $#
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h:
#echo target pre=$#
C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp :
#echo target pre=$#
C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h:
#echo target pre=$#
I don't care about the exact form of the rule just that each file is considered an actual target.
My method of passing in this rule would be by setting the MAKEFILES variable like so
make all MAKEFILES=Dummy.mk
with Dummy.mk containing this rule so that I do not modify the makefiles.
I've tried the following so far.
Dummy.mk:
%.h:
#echo header xyz = $#
%:
#echo other xyz= $#
This partially works.
I run make all --trace --print-data-base MAKEFILES=Dummy.mk and I can see that
make does "bind" the %.h: rule to the header files. In the --print-data-base section, I see that rule being assigned to the header files.
C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef.h:
# Implicit rule search has been done.
# Implicit/static pattern stem: 'C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef'
# Last modified 2016-05-27 12:39:16
# File has been updated.
# Successfully updated.
# recipe to execute (from '#$(QMAKE) top_builddir=C:/Users/User1/Desktop/A/HMI_FORGF/src/../lib/armle-v7/release/ top_srcdir=C:/Users/User1/Desktop/A/HMI_FORGF/ -Wall CONFIG+=release CONFIG+=qnx_build_release_with_symbols CONFIG+=rtc_build -o Makefile C:/Users/User1/Desktop/A/HMI_FORGF/src/HmiLogging/HmiLogging.pro
', line 2):
#echo header xyz = $#
However, I do NOT see the "echo header xyz $#"-rule being executed.
Regarding the %: rule, it is neither executed for the .cpp files nor "bound" to them in the --print-data-base section.
However, it is bound and executed for existing targets which have no suffix i.e.
all: library binary
binary: | library
ifs: | library
For the %: rule, the reason for this behavior is because of 10.5.5 Match-Anything Pattern Rules: If you do not mark the match-anything rule as terminal, then it is non-terminal. A non-terminal match-anything rule cannot apply to a file name that indicates a specific type of data. A file name indicates a specific type of data if some non-match-anything implicit rule target matches it.
If I make it non-terminal - no double colon - then the rule doesn't apply to built-in types like .cppunless I un-define the built-in rules that negate my intended %: rule.
If I make it terminal, "it does not apply unless its prerequisites actually exist". But a .h or .cpp doesn't technically have prerequisites; can I just create a dummy file and have that as its prerequisite?
NOTE: This has NOTHING to do with gcc -M generation. Yes the -M option would help in the specific case of header and source files but this question is for more generic targets and prerequisites that already exist in the makefile when make is launched.
This may take a few iterations. Try:
%.h: null
#echo header xyz = $#
%: null
#echo other xyz= $#
null:
#:
Try generating static pattern rules for the header files. See one of the answers to Make ignoring Prerequisite that doesn't exist.
Static pattern rules only apply to an explicit list of target files like this:
$(OBJECTS): %.o: %.c
*recipe here*
where the variable OBJECTS is defined earlier in the makefile to be a list of target files (separated by spaces), for example:
OBJECTS := src/fileA.c src/fileB.c src/fileC.c
Note that you can use the various make utility functions to build that list of target files. For example, $(wildcard pattern), $(addsuffix), etc.
You should also ensure that the recipe "touches" the header file to change the timestamp.
I've found that using static pattern rules instead of pattern rules fixes problems where make doesn’t build prerequisites that don’t exist, or deletes files that you want.
Here is an example of using wildcard to copy files from one directory to another.
# Copy images to build/images
img_files := $(wildcard src/images/*.png src/images/*.gif src/images/*.jpg \
src/images/*.mp3)
build_images := $(subst src/,$(BUILD_DIR)/,$(img_files))
$(build_images): $(BUILD_DIR)/images/% : src/images/%
mkdir -p $(dir $#)
cp -v -a $< $#
There are other make functions like addprefix that could be used to generate a more complex file specification.
In make, how do I use the wildcard % to describe multiple files in both target and prerequisites?
%.1: %.md
for f in $?; do ...; done
I thought the above was the way to trigger the rule to create *.1 files for any *.md that were modified since the last run, but shows this instead:
make: *** No targets. Stop.
Your rule is saying "if you need to make a .1, here is how to do so if you have a .md", you haven't asked make to do any .1. You probably need also something like:
SOURCES=$(wildcard *.md)
all: $(SOURCES:%.md=%.1)
%.1: %.md
echo $#
Note that * and % are similar (they stands for unspecified characters) but appears in different contexts (* to look for existing files, % when examining strings -- the strings are often part of file name, but that's not something mandatory and you could for instance use it to match arguments of utilities)