In my directory, I have many files named
A.xxx A01.xxx A02.xxx A03.xxx
B.xxx B01.xxx
Z......
I would like to create rules like
A.yyy : A.xxx
command type 1
A01.yyy : A01.xxx A.xxx
command type 2
B.yyy : B.xxx
command type 1
B01.yyy : B01.xxx B.xxx
command type 2
Z01.yyy: Z01.xxx
command type 3
Can someone give me an hint ?
I tried something like this (it seems ok for command 1 an 3, but in command 2 , I don't see how to put the second perequisit)
list:= A B
source=$(wildcard *.xxx)
compi:=$(patsubst %.xxx, %.yyy, $(source))
list_pattern=$(addsuffix %, $(list))
list_yyy=$(addsuffix .yyy, $(list))
list_nb_yyy=$(filter $(list_pattern), $(compi))
%.yyy:%xxx
#echo $# - $+ - command 3
all: $(compi)
.SECONDEXPANSION:
$(list_yyy): $$(patsubst %.yyy, %.xxx, $$#)
#echo $# - $+ - command 1
$(list_nb_yyy): $$(patsubst %.yyy, %.xxx, $$#)
#echp $# - $+ - command 2
Two important parts of the documentation that you should read include:
8.9 The eval Function
The eval function is very special: it allows you to define new makefile constructs that are not constant; which are the result of evaluating other variables and functions. The argument to the eval function is expanded, then the results of that expansion are parsed as makefile syntax. The expanded results can define new make variables, targets, implicit or explicit rules, etc.
10.5.4 How Patterns Match, particularly the shortest stem rule:
It is possible that more than one pattern rule will meet these criteria. In that case, make will choose the rule with the shortest stem (that is, the pattern that matches most specifically). If more than one pattern rule has the shortest stem, make will choose the first one found in the makefile.
The shortest stem rule will make it difficult to have 3 different pattern rules for A.yyy, A01.yyy and Z01.yyy respectively. However, with the eval function you can use the information available in the list variable to generate non-pattern rules for A.yyy and B.yyy, and the correct pattern rules for the other targets (assuming that the decision is made on the prefix of the name):
list:= A B
source=$(wildcard *.xxx)
compi:=$(patsubst %.xxx, %.yyy, $(source))
all: $(compi)
define LIST_RULE
$(1).yyy: $(1).xxx
#echo $$# - $$+ - command 1
$(1)%.yyy: $(1)%.xxx $(1).xxx
#echo $$# - $$+ - command 2
endef
$(foreach l,$(list),$(eval $(call LIST_RULE,$(l))))
%.yyy: %.xxx
#echo $# - $+ - command 3
Note that the fallback solution (command 3) won't be selected for A01.yyy and B01.yyy because the pattern rule in the LIST_RULE template leads to a shorter stem (01 instead of A01 or B01)
Related
I want to use the match pattern of a Makefile rule in a text function in the perquisites list.
Let's say I want to build a file, and that files perquisites are any other files with the same name in a list.
list_of_files=a/x b/x a/y b/y c/z
%:
echo 0 $#
touch $#
build/%: $(filter %/$*,$(list_of_files))
echo 1 target $#
echo 1 prereq $?
touch $#
.PHONY:
all: build/x
echo
In the above case, I want for a/x and b/x rules to also be triggered as they are filtered out of the list, however they are not.
The docs are pretty clear that what you're trying won't work as-is:
It’s very important that you recognize the limited scope in which automatic variable values are available: they only have values within the recipe. In particular, you cannot use them anywhere within the target list of a rule; they have no value there and will expand to the empty string. Also, they cannot be accessed directly within the prerequisite list of a rule.
One way to deal with this (as mentioned in the above documentation) is to use the secondary expansion capability:
.SECONDEXPANSION:
getprereqs = $(filter %/$*,$(list_of_files))
build/%: $$(getprereqs)
I would like to use an implicit rule (of the %.final: %.intermediate type) in a makefile, but only for certain files (only a.final and b.final, but not x.final)
I've tried to use
targets=a b c
${targets} %.intermediate: %.original
process1 $< > $#
${targets}: %.final: %.intermediate
process2 $< > $#
all: ${targets}
but I'm just not making make happy. I'm not sure exactly what these implicit rules are doing, and if targets needs to include '.final', or how to really make it come together.
If I do
a.intermediate: a.original
process1 a.original > a.intermediate
a.final: a.intermediate
process2 a.intermediate > a.final
#and continue for b, c, etc.
Then things are just fine and dandy.
Here's the general idea. It's not in good form because I'm not used to writing makefiles with pattern substitution and so forth.
#Reference data
CHIP_WCE_ACCESSION=SRR713343
CHIP_BLANK_ACCESSION=SRR713344
#Experiment data
CHIP_NANOG_ACCESSION=SRR713342
CHIP_SOX2_ACCESSION=SRR713341
CHIP_OCT4_ACCESSION=SRR713340
CHIP_TARGETS=${CHIP_NANOG_ACCESSION} ${CHIP_SOX2_ACCESSION} ${CHIP_OCT4_ACCESSION}
#Note that CHIP_WCE_ACCESSION is *not* processed with these.
CHIP_REFERENCE=${CHIP_BLANK_ACCESSION} ${CHIP_WCE_ACCESSION}
BAM_COVERAGE=bamCoverage --numberOfProcessors 16 --binSize 10
#This is a specific bioinformatics program.
${CHIP_REFERENCE}: %.sort.bam
echo hello > $# #Not the same way the CHIP_TARGETS are created.
${CHIP_TARGETS}: %.sort.bam
touch $# #There's actually a long chain of processing, but touch works for the minimal example.
${CHIP_TARGETS}: %.bw: %.sort.bam ${CHIP_BLANK_ACCESSION}.sort.bam
${BAM_COVERAGE} --use-reference ${CHIP_BLANK_ACCESSION}.sort.bam --bam $< -o $#
chip: ${CHIP_NANOG_ACCESSION}.bw ${CHIP_SOX2_ACCESSION}.bw ${CHIP_OCT4_ACCESSION}.bw
Expected result: ${BAM_COVERAGE} is executed on all of the CHIP_TARGETS, with appropriate execution of CHIP_REFERENCE rules for the $CHIP_BLANK_ACCESSION.
Current results: I don't know how to write this rule.
This:
${targets} %.intermediate: %.original
is not valid because all targets in a pattern rule must contain the pattern.
This:
${targets}: %.final: %.intermediate
is a static pattern rule, which is what you want, BUT every target in the static pattern rule must match the target. So you want this:
$(targets:%=%.final) : %.final : %.intermediate
I'm trying to write a rule that will call my checking script on the output of each stage of my program, generating that output if it does not already exist using my %.output : %.input rule.
I tried check : $(wildcard stage[1234].output) but this causes the rule to require only those matching output files that already exist.
I could just define a variable like TARGETS = stage1.output stage2.output ..., but is there a way to generate all possible matches of a pattern and then require them?
for the %.output : %.input rule to apply, you need both
a goal that requires an intermediate that matches the pattern %.output
a corresponding %.input file - either preexisting or a rule to build it
If your stage*.input files already exist, you can use:
INPUTS=$(wildcard stage[1234].input)
TARGETS=$(INPUTS:%.input=%.output)
check: $(TARGETS)
If your stage*.input files don't exist but are expected to be built from similar pattern rules, reapply the same principle.
If your stage*.input are produced from more complicated means, but assuming that their name can be generated by the application of a substitution pattern simply apply that pattern. In your example it would be something like:
L:= 1 2 3 4
TARGETS=$(L:%=stage%.output)
If you want analogue of seq Unix utility in Make, here it is:
seq = $(if $(word $1,$2),$2,$(call seq,$1,$2 $(words $2 1)))
$(info seq(10)=$(call seq,10))
stage_sount:=7
stages:=$(patsubst %,stage%.output,$(call seq,$(stage_sount)))
$(info stages=$(stages))
all:
Output:
$ make
seq(10)= 1 2 3 4 5 6 7 8 9 10
stages=stage1.output stage2.output stage3.output stage4.output stage5.output stage6.output stage7.output
make: Nothing to be done for 'all'.
I have another make-like tool that produces an XML as an artifact after parsing my makefile which I'll then further process with Python.
It'd simplify things for me - a lot - if I could have make consider every single prerequisite to be an actual target because then this other tool
will classify each and every file as a "job".
This is a fragment of my makefile:
.obj/eventlookupmodel.o: C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h \
...
I'd want for make to think I have a dummy rule for each prerequisite such as below:
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h:
#echo target pre= $#
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h:
#echo target pre=$#
C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp :
#echo target pre=$#
C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h:
#echo target pre=$#
I don't care about the exact form of the rule just that each file is considered an actual target.
My method of passing in this rule would be by setting the MAKEFILES variable like so
make all MAKEFILES=Dummy.mk
with Dummy.mk containing this rule so that I do not modify the makefiles.
I've tried the following so far.
Dummy.mk:
%.h:
#echo header xyz = $#
%:
#echo other xyz= $#
This partially works.
I run make all --trace --print-data-base MAKEFILES=Dummy.mk and I can see that
make does "bind" the %.h: rule to the header files. In the --print-data-base section, I see that rule being assigned to the header files.
C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef.h:
# Implicit rule search has been done.
# Implicit/static pattern stem: 'C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef'
# Last modified 2016-05-27 12:39:16
# File has been updated.
# Successfully updated.
# recipe to execute (from '#$(QMAKE) top_builddir=C:/Users/User1/Desktop/A/HMI_FORGF/src/../lib/armle-v7/release/ top_srcdir=C:/Users/User1/Desktop/A/HMI_FORGF/ -Wall CONFIG+=release CONFIG+=qnx_build_release_with_symbols CONFIG+=rtc_build -o Makefile C:/Users/User1/Desktop/A/HMI_FORGF/src/HmiLogging/HmiLogging.pro
', line 2):
#echo header xyz = $#
However, I do NOT see the "echo header xyz $#"-rule being executed.
Regarding the %: rule, it is neither executed for the .cpp files nor "bound" to them in the --print-data-base section.
However, it is bound and executed for existing targets which have no suffix i.e.
all: library binary
binary: | library
ifs: | library
For the %: rule, the reason for this behavior is because of 10.5.5 Match-Anything Pattern Rules: If you do not mark the match-anything rule as terminal, then it is non-terminal. A non-terminal match-anything rule cannot apply to a file name that indicates a specific type of data. A file name indicates a specific type of data if some non-match-anything implicit rule target matches it.
If I make it non-terminal - no double colon - then the rule doesn't apply to built-in types like .cppunless I un-define the built-in rules that negate my intended %: rule.
If I make it terminal, "it does not apply unless its prerequisites actually exist". But a .h or .cpp doesn't technically have prerequisites; can I just create a dummy file and have that as its prerequisite?
NOTE: This has NOTHING to do with gcc -M generation. Yes the -M option would help in the specific case of header and source files but this question is for more generic targets and prerequisites that already exist in the makefile when make is launched.
This may take a few iterations. Try:
%.h: null
#echo header xyz = $#
%: null
#echo other xyz= $#
null:
#:
Try generating static pattern rules for the header files. See one of the answers to Make ignoring Prerequisite that doesn't exist.
Static pattern rules only apply to an explicit list of target files like this:
$(OBJECTS): %.o: %.c
*recipe here*
where the variable OBJECTS is defined earlier in the makefile to be a list of target files (separated by spaces), for example:
OBJECTS := src/fileA.c src/fileB.c src/fileC.c
Note that you can use the various make utility functions to build that list of target files. For example, $(wildcard pattern), $(addsuffix), etc.
You should also ensure that the recipe "touches" the header file to change the timestamp.
I've found that using static pattern rules instead of pattern rules fixes problems where make doesn’t build prerequisites that don’t exist, or deletes files that you want.
Here is an example of using wildcard to copy files from one directory to another.
# Copy images to build/images
img_files := $(wildcard src/images/*.png src/images/*.gif src/images/*.jpg \
src/images/*.mp3)
build_images := $(subst src/,$(BUILD_DIR)/,$(img_files))
$(build_images): $(BUILD_DIR)/images/% : src/images/%
mkdir -p $(dir $#)
cp -v -a $< $#
There are other make functions like addprefix that could be used to generate a more complex file specification.
Given this bit of Makefile:
# for pattern matching
$(OBJDIR) := build
# just to see if a level of indirection will work
my_dir = $(dir $(1))
$(OBJECTS) : $(OBJDIR)/% : $(HEADERS) $(SRCDIR)/% | % $(dir %) $(call my_dir,%)
#echo output-only = $|
This is a static pattern rule with order-only prerequisites.
Consider the target "build/utility/debug.js". The output of the above rule will be this:
output-only = utility/debug.js ./
The first component, "utility/debug.js", is properly copied from the stem (%).
The second component, "./", is the output of calling the dir function in the prerequisites list.
The third component, an empty string, is the output of calling my my_dir function in the prerequisites list.
If I change my_dir to this:
my_dir = $(1)
The output remains the same. If I change it to this:
my_dir = "foo"
Then make complains there is no rule to make "foo" (which is expected). It appears, then, that $(1) is not getting bound in the call to my_dir.
What's going on? Why can't I pass the stem to a function? I have a workaround that uses secondary expansion, but I want to know why I can't write my rule this way.
EDIT: I'm new to stackoverflow, forgive me if this is not the way things are done here.
I want $(1) because I am passing the stem as an argument to my_dir, as Alex pointed out.
I don't know why it was suggested I want "$". I don't believe $ by itself expands to anything in any context.
I know that automatic variables are only available in the recipe. I am not using an automatic variable in the prerequisites - I am using the stem:
Each target is matched against the target-pattern to extract a part of the target name, called the stem. This stem is substituted into each of the prereq-patterns to make the prerequisite names (one from each prereq-pattern). - the manual
The fact that the stem is available is demonstrated by the example: the stem expands to the correct value when used alone, but not when passed to a function.
As can be seen in this section of the GNU make manual, variable and function references in the list of prerequisites are immediately expanded, during the read in phase. This means, before any pattern matching is done, so the % has no special meaning yet; it is interpreted as a literal character in the two function references, $(dir %) and $(call my_dir,%), both having ./ as a result, which get merged in the reference to $| in the recipe.
I don't know of any other workaround than the one you already found, i.e. secondary expansion.
Note $1 is not a special variable that expands to anything interesting related to pattern rules (or static pattern rules). The $1 variable only has unique behavior within the context of a user-defined macro invoked by the $(call ...) function.
You wanted to use $*, not $1; $* is an automatic variable which expands to the stem of the target of the rule.
However, in all versions of make (including the POSIX standard definition of make), automatic variables (including $*, $<, $#, $^, etc.) are only available in the context of the recipe. They are not available in the context of the target or prerequisite lists. See the GNU make manual section on Automatic Variables for more details.
As you suggest, there is a GNU make-specific feature enabled by the .SECONDEXPANSION pseudo target which provides a way to avoid this limitation.