I am writing a makefile that is supposed to create multiple executables from the code for different architectures. The idea is that the suffix of each executable indicates the architecture for which it was built. This is a simplified excerpt:
TARGET_NAME = foo
TARGET_AMD64 = $(TARGET_NAME)_amd64
TARGET_ARMHF = $(TARGET_NAME)_armhf
.PHONY:
all: $(TARGET_AMD64) $(TARGET_ARMHF) ;
$(TARGET_NAME)_%: lib/mylib.a
#echo "$# built."
lib/mylib.a: bin/mylib/src/mac/foo_ct.o
#echo "$# built."
When typing make I receive a circular dependency:
make: Circular bin/mylib/src/mac/foo_ct.o <- lib/mylib.a dependency dropped.
bin/mylib/src/mac/foo_ct.o built.
lib/mylib.a built.
foo_amd64 built.
foo_armhf built.
Apparently, make is looking for a rule that matches bin/mylib/src/mac/foo_ct.o. For reasons beyond my understanding, make takes $(TARGET_NAME)_% (the second rule).
Why is that the case? Only the filename foo_ct.o would match the pattern rule $(TARGET_NAME)_%, but why is the preceding path bin/mylib/src/mac/ ignored?
This is actually the defined behaviour:
When the target pattern does not contain a slash (and it usually does not), directory names in the file names are removed from the file name before it is compared with the target prefix and suffix. After the comparison of the file name to the target pattern, the directory names, along with the slash that ends them, are added on to the prerequisite file names generated from the pattern rule’s prerequisite patterns and the file name. The directories are ignored [...]
You will therefore have to specify your target with its absolute path:
TARGET_NAME = $(CURDIR)/foo
Note however, that $(CURDIR) is a GNU make extension.
Alternatively you could also specify a more specific rule for `bin/mylib/src/mac/foo_ct.o` to break the cycle:
TARGET_NAME = foo
TARGET_AMD64 = $(TARGET_NAME)_amd64
TARGET_ARMHF = $(TARGET_NAME)_armhf
.PHONY:
all: $(TARGET_AMD64) $(TARGET_ARMHF;
$(TARGET_NAME)_%: lib/mylib.a
#echo "$# built."
lib/mylib.a: bin/mylib/src/mac/foo_ct.o
#echo "$# built."
bin/mylib/src/mac/foo_ct.o:
true
Related
I have this version of makefile
[sbsuser#compute-00-01 415]$ make --version GNU Make 3.81
I have directory SOMATIC where I have 3 file . I want to produce a only one output. This is what I wrote.
`
OUTSOMATIC=SOMATIC
FINAL=FINAL
INPUT=$(wildcard $(OUTSOMATIC)/*.vcf)
OUTSORT2= $(patsubst $(OUTSOMATIC)/%.vcf,$(FINAL)/%somatic.ensemble.gz,$(INPUT))
$(info lista $(OUTSORT2))
$(info lista $(INPUT))
.PHONY: all
all: $(INPUT) $(OUTSOMATIC) $(OUTSORT2) $(FINAL)
$(FINAL)/%somatic.ensemble.gz: $(OUTSOMATIC)/%.vcf $(INPUT)
~/jdk1.8.0_121/bin/java -XX:+UseSerialGC -Xms1g -Xmx10g -jar /illumina/software/PROG2/bcbio-variation-recall-0.1.7 ensemble -n 1 $(FINAL)/somatic_ensemble.gz /illumina/software/database/database_2016/hg19_primary.fa $^
`
With this script make 3 time the same files. I don't understand how to create only one output from list of input to use in the same time.
What is the best way to do this?
If I change $(FINAL)/%somatic.ensemble.gz: in $(FINAL)/somatic.ensemble.gz I have this error:
make: *** No rule to make target FINAL/415_merge_mutect2.somaticsomatic.ensemble.gz', needed byall'. Stop`
You probably should review the GNU make manual introductory sections where they describe how make works.
Let's look at your makefile; first you define some variables. Let's assume that you have the files SOMATIC/foo.vcf, SOMATIC/bar.vcf, and SOMATIC/baz.vcf. Then the variables you created will have these values, after they are expanded:
OUTSOMATIC = SOMATIC
FINAL = FINAL
INPUT = SOMATIC/foo.vcf SOMATIC/bar.vcf SOMATIC/baz.vcf
Now your patsubst finds all words in INPUT that match the pattern SOMATIC/%.vcf and replace that with FINAL/%somatic.ensemble.gz, where the part that matches the % in the input is substituted into the output:
OUTSORT2 = FINAL/foosomatic.ensemble.gz FINAL/barsomatic.ensemble.gz FINAL/bazsomatic.ensemble.gz
Now, make sees that you've defined an all target. Since it's the first target in the makefile this is the target that will be run by default. After expansion, it will look like this:
all: SOMATIC/foo.vcf SOMATIC/bar.vcf SOMATIC/baz.vcf SOMATIC FINAL/foosomatic.ensemble.gz FINAL/barsomatic.ensemble.gz FINAL/bazsomatic.ensemble.gz FINAL
So, make will try to build every prerequisite of the all target to be sure it's up to date. First it tries to build the SOMATIC/*.vcf files. Those files already exist and make doesn't have any rules about how to rebuild them, so it assumes they're up to date.
Next it tries to build the SOMATIC file. This is a directory and it also has no rule to be built, so make assumes that's up to date as well.
Next make tries to build the target FINAL/foosomatic.ensemble.gz. Make does have a rule that can build it, you've created one:
$(FINAL)/%somatic.ensemble.gz: $(OUTSOMATIC)/%.vcf $(INPUT)
~/jdk1.8.0_121/bin/java ...
This matches the target you want to build, with a % value of foo, so then make substitutes the % in the prerequisite for foo and finds that SOMATIC/foo.vcf exists and doesn't need to be rebuilt, so it runs your recipe. However your recipe doesn't actually create the target FINAL/foosomatic.ensemble.gz; it creates the target FINAL/somatic_ensemble.gz. So this rule is broken because it tells make it will do one thing, but it does something else.
You should always ensure all your recipes build the file represented by the automatic variable $#; that will ensure that you and make agree on the meaning of your rule. If you want your recipe to build some other file, then your rule is written incorrectly.
Next make does the same thing with the next prerequisite of all: FINAL/barsomatic.ensemble.gz. Since that file doesn't exist, make tries to build it using the pattern rule, but again that creates the same output file.
And again for the third .gz file FINAL/bazsomatic.ensemble.gz. That's why things are run three times.
If you change the pattern rule to an explicit rule building FINAL/somatic.ensemble.gz, which is what you want, then make can't find any way to build the prerequisites of the all target so it gives this error.
Your problem is the creation of OUTSORT2. You want to create only one output file, but you've set OUTSORT2 to contain three different files, so make tries to create all three files. You want this:
OUTSOMATIC = SOMATIC
FINAL = FINAL
INPUT = $(wildcard $(OUTSOMATIC)/*.vcf)
OUTSORT2 = $(FINAL)/somatic.ensemble.gz
.PHONY: all
all: $(OUTSORT2)
$(OUTSORT2): $(INPUT)
~/jdk1.8.0_121/bin/java -XX:+UseSerialGC -Xms1g -Xmx10g -jar /illumina/software/PROG2/bcbio-variation-recall-0.1.7 ensemble -n 1 $# /illumina/software/database/database_2016/hg19_primary.fa $^
I'm using Automake.
I have a few source files listed in dist_man1_MANS like this:
dist_man1_MANS = some-file.1 some-other-file.1
Now, Automake + configure eventually generate this in Makefile:
dist_man1_MANS = some-file.1 some-other-file.1
# ...
install-man1: $(dist_man1_MANS)
# generated recipe here
Since I'm not prefixing the .1 files with $(srcdir), I assume that, since I run make from the build directory (its current working directory), it should find them in the build directory.
So, I'm doing an out-of-tree build, for example, in /tmp/build:
/path/to/src/configure --prefix=$(pwd)/install
make
make install
and the build succeeds, that is, make finds the man pages and installs them. They are not in the build directory, however. I add this to the generated Makefile:
install-man1: $(dist_man1_MANS)
#echo ">>> $(^)"
#echo "::: $(dist_man1_MANS)"
# generated recipe here
Now, I would assume that both echos print the same thing, because $^ means the names of all the prerequisites, with spaces between them. To my surprise, the output is:
>>> /path/to/src/some-file.1 /path/to/src/some-other-file.1
::: some-file.1 some-other-file.1
So:
How did make find the /path/to/src/ prefix exactly? Where does it come from in this very context?
Why do $^ and $(dist_man1_MANS) differ?
I found the answer.
Automake sets the VPATH variable in its generated Makefile, a special variable for make, to something like:
VPATH = /path/to/src
From the previous links:
4.5.1 VPATH: Search Path for All Prerequisites
The value of the make variable VPATH specifies a list of directories that make should search. Most often, the directories are expected to contain prerequisite files that are not in the current directory; however, make uses VPATH as a search list for both prerequisites and targets of rules.
So make searches for the some-file.1 and some-other-file.1 prerequisites in the current working directory first, then for /path/to/src/some-file.1 and /path/to/src/some-other-file.1 if it cannot find the first ones. In this case I understand why $^ is different from $(dist_man1_MANS): $^ is the list of effective (resolved) prerequisites.
I have a very bizzare problem with GNU make. I have the following files:
a/x.html
b/Makefile
b/c/Makefile
The contents of a/x.html are irrelevant. The contents of b/Makefile are as follows:
SRC=../a
all: x.html
%.html: ${SRC}/%.html
rsync $< $#
The contents of b/c/Makefile are the same, except for the definition of SRC:
SRC=../../a
If I run make in b/c/ the result is as expected:
rsync ../../a/x.html x.html
and x.html gets copied from a/ to b/c/.
However, if I run make in b/ the output I get is several lines of:
make: stat: ../a/../a/.. (repeated many times) ../a/x.html: File name too long
It seems that make is applying the rule for %.html recursively, but why? Is there something obvious I am missing?
To build a target that matches the pattern %.html (i.e. any target name that ends in .html), make applies the rule if it can build the dependency (target built from the original target with ../a/ prepended).
You ask to build x.html. This matches the pattern %.html, so the rule applies: make sees if it can build ../a/x.html.
../a/x.html matches the pattern %.html, so the rule applies: make sees if it can build ../a/../a/x.html.
../../a/x.html matches the pattern %.html, so the rule applies, etc.
The stem character can match any part of a path, including directory separators.
You can see what make is trying by running make -r -d (-d to show debugging output, -r to turn off built-in rules which would cause a huge amount of noise).
When you're in b/c, this stops at step 2 because ../../a/x.html exists but ../../../../a/x.html doesn't.
One way to fix this is to list the files on which you want to act. You can build that list from the list of files that already exist in ../a:
$(notdir $(wildcard ${SRC}/*.html)): %.html: ${SRC}/%.html
rsync $< $#
This has the downside that if the HTML files in ../a are themselves built by a rule in b/Makefile, then running make in b won't built them in a pristine source directory. This shouldn't be a problem though: it would be unusual to have a makefile in b build things outside b.
Another approach which doesn't have this defect is to use an absolute path.
%.html: $(abspath ${SRC})/%.html
rsync $< $#
I have another make-like tool that produces an XML as an artifact after parsing my makefile which I'll then further process with Python.
It'd simplify things for me - a lot - if I could have make consider every single prerequisite to be an actual target because then this other tool
will classify each and every file as a "job".
This is a fragment of my makefile:
.obj/eventlookupmodel.o: C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h \
...
I'd want for make to think I have a dummy rule for each prerequisite such as below:
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h:
#echo target pre= $#
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h:
#echo target pre=$#
C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp :
#echo target pre=$#
C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h:
#echo target pre=$#
I don't care about the exact form of the rule just that each file is considered an actual target.
My method of passing in this rule would be by setting the MAKEFILES variable like so
make all MAKEFILES=Dummy.mk
with Dummy.mk containing this rule so that I do not modify the makefiles.
I've tried the following so far.
Dummy.mk:
%.h:
#echo header xyz = $#
%:
#echo other xyz= $#
This partially works.
I run make all --trace --print-data-base MAKEFILES=Dummy.mk and I can see that
make does "bind" the %.h: rule to the header files. In the --print-data-base section, I see that rule being assigned to the header files.
C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef.h:
# Implicit rule search has been done.
# Implicit/static pattern stem: 'C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef'
# Last modified 2016-05-27 12:39:16
# File has been updated.
# Successfully updated.
# recipe to execute (from '#$(QMAKE) top_builddir=C:/Users/User1/Desktop/A/HMI_FORGF/src/../lib/armle-v7/release/ top_srcdir=C:/Users/User1/Desktop/A/HMI_FORGF/ -Wall CONFIG+=release CONFIG+=qnx_build_release_with_symbols CONFIG+=rtc_build -o Makefile C:/Users/User1/Desktop/A/HMI_FORGF/src/HmiLogging/HmiLogging.pro
', line 2):
#echo header xyz = $#
However, I do NOT see the "echo header xyz $#"-rule being executed.
Regarding the %: rule, it is neither executed for the .cpp files nor "bound" to them in the --print-data-base section.
However, it is bound and executed for existing targets which have no suffix i.e.
all: library binary
binary: | library
ifs: | library
For the %: rule, the reason for this behavior is because of 10.5.5 Match-Anything Pattern Rules: If you do not mark the match-anything rule as terminal, then it is non-terminal. A non-terminal match-anything rule cannot apply to a file name that indicates a specific type of data. A file name indicates a specific type of data if some non-match-anything implicit rule target matches it.
If I make it non-terminal - no double colon - then the rule doesn't apply to built-in types like .cppunless I un-define the built-in rules that negate my intended %: rule.
If I make it terminal, "it does not apply unless its prerequisites actually exist". But a .h or .cpp doesn't technically have prerequisites; can I just create a dummy file and have that as its prerequisite?
NOTE: This has NOTHING to do with gcc -M generation. Yes the -M option would help in the specific case of header and source files but this question is for more generic targets and prerequisites that already exist in the makefile when make is launched.
This may take a few iterations. Try:
%.h: null
#echo header xyz = $#
%: null
#echo other xyz= $#
null:
#:
Try generating static pattern rules for the header files. See one of the answers to Make ignoring Prerequisite that doesn't exist.
Static pattern rules only apply to an explicit list of target files like this:
$(OBJECTS): %.o: %.c
*recipe here*
where the variable OBJECTS is defined earlier in the makefile to be a list of target files (separated by spaces), for example:
OBJECTS := src/fileA.c src/fileB.c src/fileC.c
Note that you can use the various make utility functions to build that list of target files. For example, $(wildcard pattern), $(addsuffix), etc.
You should also ensure that the recipe "touches" the header file to change the timestamp.
I've found that using static pattern rules instead of pattern rules fixes problems where make doesn’t build prerequisites that don’t exist, or deletes files that you want.
Here is an example of using wildcard to copy files from one directory to another.
# Copy images to build/images
img_files := $(wildcard src/images/*.png src/images/*.gif src/images/*.jpg \
src/images/*.mp3)
build_images := $(subst src/,$(BUILD_DIR)/,$(img_files))
$(build_images): $(BUILD_DIR)/images/% : src/images/%
mkdir -p $(dir $#)
cp -v -a $< $#
There are other make functions like addprefix that could be used to generate a more complex file specification.
I want to run pocketlint on all **/*.js files.
.PHONY: lint_js2
LINT_JS = $(wildcard static/js/*.js static/js/**/*.js)
LINT_JS_TARGETS = $(addprefix lint__,$(LINT_JS))
#.PHONY: $(LINT_JS_TARGETS)
lint_js2: $(LINT_JS_TARGETS)
echo $<
lint__%: %
pocketlint $<
However, I get this error:
make: *** No rule to make target `lint__static/js/ad_list.js', needed by `lint_js2'. Stop.
Why lint__static/js/ad_list.js is not captured by lint__%?
If I uncomment second .PHONY, it echoes lint__static/js/ad_list.js, but does not invoke pocketlint static/js/ad_list.js. Why?
If my approach is wrong, what would be the right approach? Since tasks are independent, I would appreciate if make -j50 would do what I am expecting.
Thanks!
It's not clear what is intended: does pocketlint write a file named lint__static/js/ad_list.js, or is that really a phony filename? Anyway...
Reread the second paragraph of How Patterns Match:
When the target pattern does not contain a slash (and it usually does not), directory names in the file names are removed from the file name before it is compared with the target prefix and suffix. After the comparison of the file name to the target pattern, the directory names, along with the slash that ends them, are added on to the prerequisite file names generated from the pattern rule's prerequisite patterns and the file name.
In short, % generally matches just a filename, not a pathname with slashes in it. So lint__static/js/ad_list.js is not captured because actually it is only ad_list.js that is being matched against lint__%.
If you can arrange it so that the output files from pocketlint are static/js/lint__ad_list.js etc, then this could be made to work:
LINT_JS_TARGETS = $(foreach f,$(LINT_JS),$(dir $f)lint__$(notdir $f))
lint__%: %
pocketlint $<
Alternatively you can make % match pathnames by having the target pattern be a pathname (containing a slash):
LINT_JS_TARGETS = $(addprefix linted/,$(LINT_JS))
linted/%: %
pocketlint $<
This time % = static/js/ad_list.js does match the pattern rule.
In either case, you're going to have to have pocketlint produce output (if indeed it produces output) named differently than lint__static/*.
Implicit rule search is suppressed for phony targets (see Phony Targets, paragraph 5). So the rule involving pocketlint is never considered when lint__static/js/ad_list.js is phony.
It's not obvious why the result is Nothing to be done for (phony) lint__static/js/ad_list.js rather than No rule to make target lint__static/js/ad_list.js, but I wouldn't lose sleep over it!