I want to use the match pattern of a Makefile rule in a text function in the perquisites list.
Let's say I want to build a file, and that files perquisites are any other files with the same name in a list.
list_of_files=a/x b/x a/y b/y c/z
%:
echo 0 $#
touch $#
build/%: $(filter %/$*,$(list_of_files))
echo 1 target $#
echo 1 prereq $?
touch $#
.PHONY:
all: build/x
echo
In the above case, I want for a/x and b/x rules to also be triggered as they are filtered out of the list, however they are not.
The docs are pretty clear that what you're trying won't work as-is:
It’s very important that you recognize the limited scope in which automatic variable values are available: they only have values within the recipe. In particular, you cannot use them anywhere within the target list of a rule; they have no value there and will expand to the empty string. Also, they cannot be accessed directly within the prerequisite list of a rule.
One way to deal with this (as mentioned in the above documentation) is to use the secondary expansion capability:
.SECONDEXPANSION:
getprereqs = $(filter %/$*,$(list_of_files))
build/%: $$(getprereqs)
Related
I would like to use an implicit rule (of the %.final: %.intermediate type) in a makefile, but only for certain files (only a.final and b.final, but not x.final)
I've tried to use
targets=a b c
${targets} %.intermediate: %.original
process1 $< > $#
${targets}: %.final: %.intermediate
process2 $< > $#
all: ${targets}
but I'm just not making make happy. I'm not sure exactly what these implicit rules are doing, and if targets needs to include '.final', or how to really make it come together.
If I do
a.intermediate: a.original
process1 a.original > a.intermediate
a.final: a.intermediate
process2 a.intermediate > a.final
#and continue for b, c, etc.
Then things are just fine and dandy.
Here's the general idea. It's not in good form because I'm not used to writing makefiles with pattern substitution and so forth.
#Reference data
CHIP_WCE_ACCESSION=SRR713343
CHIP_BLANK_ACCESSION=SRR713344
#Experiment data
CHIP_NANOG_ACCESSION=SRR713342
CHIP_SOX2_ACCESSION=SRR713341
CHIP_OCT4_ACCESSION=SRR713340
CHIP_TARGETS=${CHIP_NANOG_ACCESSION} ${CHIP_SOX2_ACCESSION} ${CHIP_OCT4_ACCESSION}
#Note that CHIP_WCE_ACCESSION is *not* processed with these.
CHIP_REFERENCE=${CHIP_BLANK_ACCESSION} ${CHIP_WCE_ACCESSION}
BAM_COVERAGE=bamCoverage --numberOfProcessors 16 --binSize 10
#This is a specific bioinformatics program.
${CHIP_REFERENCE}: %.sort.bam
echo hello > $# #Not the same way the CHIP_TARGETS are created.
${CHIP_TARGETS}: %.sort.bam
touch $# #There's actually a long chain of processing, but touch works for the minimal example.
${CHIP_TARGETS}: %.bw: %.sort.bam ${CHIP_BLANK_ACCESSION}.sort.bam
${BAM_COVERAGE} --use-reference ${CHIP_BLANK_ACCESSION}.sort.bam --bam $< -o $#
chip: ${CHIP_NANOG_ACCESSION}.bw ${CHIP_SOX2_ACCESSION}.bw ${CHIP_OCT4_ACCESSION}.bw
Expected result: ${BAM_COVERAGE} is executed on all of the CHIP_TARGETS, with appropriate execution of CHIP_REFERENCE rules for the $CHIP_BLANK_ACCESSION.
Current results: I don't know how to write this rule.
This:
${targets} %.intermediate: %.original
is not valid because all targets in a pattern rule must contain the pattern.
This:
${targets}: %.final: %.intermediate
is a static pattern rule, which is what you want, BUT every target in the static pattern rule must match the target. So you want this:
$(targets:%=%.final) : %.final : %.intermediate
I have another make-like tool that produces an XML as an artifact after parsing my makefile which I'll then further process with Python.
It'd simplify things for me - a lot - if I could have make consider every single prerequisite to be an actual target because then this other tool
will classify each and every file as a "job".
This is a fragment of my makefile:
.obj/eventlookupmodel.o: C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h \
...
I'd want for make to think I have a dummy rule for each prerequisite such as below:
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h:
#echo target pre= $#
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h:
#echo target pre=$#
C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp :
#echo target pre=$#
C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h:
#echo target pre=$#
I don't care about the exact form of the rule just that each file is considered an actual target.
My method of passing in this rule would be by setting the MAKEFILES variable like so
make all MAKEFILES=Dummy.mk
with Dummy.mk containing this rule so that I do not modify the makefiles.
I've tried the following so far.
Dummy.mk:
%.h:
#echo header xyz = $#
%:
#echo other xyz= $#
This partially works.
I run make all --trace --print-data-base MAKEFILES=Dummy.mk and I can see that
make does "bind" the %.h: rule to the header files. In the --print-data-base section, I see that rule being assigned to the header files.
C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef.h:
# Implicit rule search has been done.
# Implicit/static pattern stem: 'C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef'
# Last modified 2016-05-27 12:39:16
# File has been updated.
# Successfully updated.
# recipe to execute (from '#$(QMAKE) top_builddir=C:/Users/User1/Desktop/A/HMI_FORGF/src/../lib/armle-v7/release/ top_srcdir=C:/Users/User1/Desktop/A/HMI_FORGF/ -Wall CONFIG+=release CONFIG+=qnx_build_release_with_symbols CONFIG+=rtc_build -o Makefile C:/Users/User1/Desktop/A/HMI_FORGF/src/HmiLogging/HmiLogging.pro
', line 2):
#echo header xyz = $#
However, I do NOT see the "echo header xyz $#"-rule being executed.
Regarding the %: rule, it is neither executed for the .cpp files nor "bound" to them in the --print-data-base section.
However, it is bound and executed for existing targets which have no suffix i.e.
all: library binary
binary: | library
ifs: | library
For the %: rule, the reason for this behavior is because of 10.5.5 Match-Anything Pattern Rules: If you do not mark the match-anything rule as terminal, then it is non-terminal. A non-terminal match-anything rule cannot apply to a file name that indicates a specific type of data. A file name indicates a specific type of data if some non-match-anything implicit rule target matches it.
If I make it non-terminal - no double colon - then the rule doesn't apply to built-in types like .cppunless I un-define the built-in rules that negate my intended %: rule.
If I make it terminal, "it does not apply unless its prerequisites actually exist". But a .h or .cpp doesn't technically have prerequisites; can I just create a dummy file and have that as its prerequisite?
NOTE: This has NOTHING to do with gcc -M generation. Yes the -M option would help in the specific case of header and source files but this question is for more generic targets and prerequisites that already exist in the makefile when make is launched.
This may take a few iterations. Try:
%.h: null
#echo header xyz = $#
%: null
#echo other xyz= $#
null:
#:
Try generating static pattern rules for the header files. See one of the answers to Make ignoring Prerequisite that doesn't exist.
Static pattern rules only apply to an explicit list of target files like this:
$(OBJECTS): %.o: %.c
*recipe here*
where the variable OBJECTS is defined earlier in the makefile to be a list of target files (separated by spaces), for example:
OBJECTS := src/fileA.c src/fileB.c src/fileC.c
Note that you can use the various make utility functions to build that list of target files. For example, $(wildcard pattern), $(addsuffix), etc.
You should also ensure that the recipe "touches" the header file to change the timestamp.
I've found that using static pattern rules instead of pattern rules fixes problems where make doesn’t build prerequisites that don’t exist, or deletes files that you want.
Here is an example of using wildcard to copy files from one directory to another.
# Copy images to build/images
img_files := $(wildcard src/images/*.png src/images/*.gif src/images/*.jpg \
src/images/*.mp3)
build_images := $(subst src/,$(BUILD_DIR)/,$(img_files))
$(build_images): $(BUILD_DIR)/images/% : src/images/%
mkdir -p $(dir $#)
cp -v -a $< $#
There are other make functions like addprefix that could be used to generate a more complex file specification.
In make, how do I use the wildcard % to describe multiple files in both target and prerequisites?
%.1: %.md
for f in $?; do ...; done
I thought the above was the way to trigger the rule to create *.1 files for any *.md that were modified since the last run, but shows this instead:
make: *** No targets. Stop.
Your rule is saying "if you need to make a .1, here is how to do so if you have a .md", you haven't asked make to do any .1. You probably need also something like:
SOURCES=$(wildcard *.md)
all: $(SOURCES:%.md=%.1)
%.1: %.md
echo $#
Note that * and % are similar (they stands for unspecified characters) but appears in different contexts (* to look for existing files, % when examining strings -- the strings are often part of file name, but that's not something mandatory and you could for instance use it to match arguments of utilities)
I would like to make a complete text document from several sources (since one of the file source change, I want the doc to change).
I have to pass it through a translator I develop. I would like to pass the language as argument, to make it cleaner.
Yesterday, late at night, I dreamed of a makefile like this...
#makefile
# ...
my_complete_doc.%.html: my_trans_exe header.%.html $(wildcard source/*.%.html)
$< --language $(variable_for_%) > $#
(?) Does it replace % by all the languages which have their own header.language.html files. And does the file my_completed_doc.language.html get changed as soon as one of the source/*.language.html get changed?
(?) How to get the % replaced in several prerequisites, possibly into the wildcard and necessarily in the recipe?
First, the easy problem: you wish to use the '%' variable in the recipe. The answer is to use the '$*' automatic variable:
my_complete_doc.%.html: my_trans_exe ...
$< --language $* > $#
Then the easy question: yes, the header.%.html prerequisite is correct. When you try to build my_complete_doc.dutch.html then Make will evaluate it as header.dutch.html, when you try to build my_complete_doc.french.html, Make will evaluate it as header.french.html.
Now the tricky problem: the prerequisite $(wildcard source/*.%.html). Ordinarily, Make expands $(wildcard ...) statements before executing any rule, or deciding which targets to build. So it searches for any files such as source/foo.%.html or source/bar.%.html (that is, files whose names contain the character '%'), finds none, and evaluates the statement as an empty string. But Make will defer this evaluation until it has chosen the rule, if you use SECONDEXPANSION:
.SECONDEXPANSION:
my_complete_doc.%.html: my_trans_exe header.%.html $$(wildcard source/*.%.html)
$< --language $* > $#
(Note the '$$'. In the first -- ordinary -- expansion, Make reduces "$$(...)" to "$(...)", and in the second -- when '%' has a value -- it expands "$(...)".) Now if you modify any file such as source/foo.german.html, Make will consider the file my_complete_doc.german.html out of date and in need of rebuilding.
Given this bit of Makefile:
# for pattern matching
$(OBJDIR) := build
# just to see if a level of indirection will work
my_dir = $(dir $(1))
$(OBJECTS) : $(OBJDIR)/% : $(HEADERS) $(SRCDIR)/% | % $(dir %) $(call my_dir,%)
#echo output-only = $|
This is a static pattern rule with order-only prerequisites.
Consider the target "build/utility/debug.js". The output of the above rule will be this:
output-only = utility/debug.js ./
The first component, "utility/debug.js", is properly copied from the stem (%).
The second component, "./", is the output of calling the dir function in the prerequisites list.
The third component, an empty string, is the output of calling my my_dir function in the prerequisites list.
If I change my_dir to this:
my_dir = $(1)
The output remains the same. If I change it to this:
my_dir = "foo"
Then make complains there is no rule to make "foo" (which is expected). It appears, then, that $(1) is not getting bound in the call to my_dir.
What's going on? Why can't I pass the stem to a function? I have a workaround that uses secondary expansion, but I want to know why I can't write my rule this way.
EDIT: I'm new to stackoverflow, forgive me if this is not the way things are done here.
I want $(1) because I am passing the stem as an argument to my_dir, as Alex pointed out.
I don't know why it was suggested I want "$". I don't believe $ by itself expands to anything in any context.
I know that automatic variables are only available in the recipe. I am not using an automatic variable in the prerequisites - I am using the stem:
Each target is matched against the target-pattern to extract a part of the target name, called the stem. This stem is substituted into each of the prereq-patterns to make the prerequisite names (one from each prereq-pattern). - the manual
The fact that the stem is available is demonstrated by the example: the stem expands to the correct value when used alone, but not when passed to a function.
As can be seen in this section of the GNU make manual, variable and function references in the list of prerequisites are immediately expanded, during the read in phase. This means, before any pattern matching is done, so the % has no special meaning yet; it is interpreted as a literal character in the two function references, $(dir %) and $(call my_dir,%), both having ./ as a result, which get merged in the reference to $| in the recipe.
I don't know of any other workaround than the one you already found, i.e. secondary expansion.
Note $1 is not a special variable that expands to anything interesting related to pattern rules (or static pattern rules). The $1 variable only has unique behavior within the context of a user-defined macro invoked by the $(call ...) function.
You wanted to use $*, not $1; $* is an automatic variable which expands to the stem of the target of the rule.
However, in all versions of make (including the POSIX standard definition of make), automatic variables (including $*, $<, $#, $^, etc.) are only available in the context of the recipe. They are not available in the context of the target or prerequisite lists. See the GNU make manual section on Automatic Variables for more details.
As you suggest, there is a GNU make-specific feature enabled by the .SECONDEXPANSION pseudo target which provides a way to avoid this limitation.