I'm trying to build several executables in one make instance, as suggested by
Recursive Make Considered Harmful.
The arguments of that paper apply to my project because some of my source files are generated, and go into multiple executables.
The object files for each of these executables go into separate directories, because they are compiled with different preprocessor flags.
How can I specify separate vpath settings for each of these executables, so that source files with duplicate filenames in separate directories go into the executables where I want them to go?
Setting vpath before the rules to build the object files for one executable, and erasing it afterwards (by not giving any directories) doesn't have the desired effect. Apparently, the last setting is used for all the rules.
Solutions I see currently:
Rename source filenames to use unique names so that I can use a global vpath setting
Instead of vpath, use separate rules for each source directory, with the source directory in the prerequisite pattern (*)
Recursive make, with separate vpath settings in each make instance, and somehow deal with the resulting trouble.
Use something different than make.
Is there a better solution, or which one of the above would you prefer?
(*) The solution with separate rules looks like this:
build/$(PROGRAM)/%.o: %.c
$(COMPILE_RECIPE)
build/$(PROGRAM)/%.o: $($(PROGRAM)_SOURCE_DIR)/%.c
$(COMPILE_RECIPE)
Ok for my current project, but would become ugly fast, if there where more than one directory different in the source paths of the executables
Edit: Test for suggestion of #Etan, which shows that $^ is empty, when the prerequisites appear only in pattern rules (y/y.c) - this only works if each dependency is given directly, as for x/x.c.
.PHONY: all
all: build/x.o build/y.o
build/x.o: x/x.c
# the following rule is ignored:
build/%.o: y/%.c
# because there is a matching pattern rule with a recipe:
build/%.o:
#echo \"$#\" from \"$^\"
#touch $#
The files x/x.c, y/y.c and the directory build exist.
output:
"build/x.o" from "x/x.c"
"build/y.o" from ""
Tested with GNU Make 3.82.90
First, it's never correct to use VPATH or vpath to find derived files (files that are built by make)... that includes "source files that are generated" (if they are generated by make, and not before make is invoked somehow). VPATH/vpath can only be used to find source files (files that are not built by make). See How not to use VPATH for more information.
Second, vpath is a global setting, not a per-target setting. Whenever make wants to find a prerequisite and that prerequisite doesn't exist "normally", make will use the vpath settings to find it. There's no facility in vpath to say "for this target use these vpath settings, for that target use those". In an ideal world, you would be able to set VPATH as a target-specific variable but this doesn't work.
There are various ways to generate prerequisites based on a target: you might find some of the suggestions on metaprogramming in make helpful.
Lastly, the makefile example you give regarding $^ is not correct:
# the following rule is ignored:
build/%.o: y/%.c
# because there is a matching pattern rule with a recipe:
build/%.o:
#echo \"$#\" from \"$^\"
#touch $#
You cannot specify a pattern rule without any recipe and have it "add" prerequisite patterns to some other existing pattern rule. It works for explicit rules, to add more prerequisites to a target, but specifying a pattern rule with no recipe simply deletes the pattern rule.
So, $^ is empty for build/y.o because there are no prerequisites defined in the pattern rule and you've not defined any explicit prerequisites for build/y.o like you did for build/x.o.
ETA: The reference to metaprogramming was to suggest that you can use it to make your second option, which is the one I would use, be less typing and easier maintenance. Pick the method you prefer and auto-generate the various pattern rules. This might be termed a "fifth option".
Other than that any of the four you suggest are fine, and there isn't another option you're missing that I'm aware of.
Related
Using GNU make, I am trying to solve a problem similar to make recipe execute twice — that is, to have a Makefile recipe run twice. In my case, however, the recipe is run under the .SECONDEXPANSION target, and the two different runs will be called with different parameters to generate different versions of the output file from the same input file. That is, with input file foo, this example Makefile should be callable via make foo.pdf or make foo.expanded.pdf to build one .pdf file, or make all to build both .pdf files:
.PHONY: all
all: foo.pdf foo.expanded.pdf
.SECONDEXPANSION:
%.expanded.pdf %.pdf: %
#echo building $(basename $#)
Of the two solutions given in that answer, the first is unsuitable because it always runs the rule twice; I want it run twice when the user asks for it.
The second solution posted there is conceptually what I am looking for and have implemented in the above example Makefile, with only the small problem that it doesn't work: although the all target lists both .pdf files as dependencies, only one is built when make all is run.
Is there a way to tell GNU make to build two different files using the same rule under a .SECONDEXPANSION?
EDIT: Clarified in problem description that the same input file is used to build both versions of the output file, and modified sample Makefile to include this dependency.
EDIT: I would like a solution as scalable as possible; that is, it should work if the input filename contains dots, specifying additional output file foo.reduced.pdf should require only adjusting the targets and recipe as appropriate, etc. This limits performing string surgery that relies on the filenames appearing exactly as given in this narrow example (e.g., changing the rule to %.pdf: $$(firstword $$(subst ., ,$$*)) fails if the input file could be either foo or foo.bar).
You are probably looking for Pattern-specific Variable Values. Let's assume your recipe depends on a make variable named BUILDFLAGS that takes value normal by default and special for the "expanded" targets. Then this Makefile:
BUILDER := builder
BUILDFLAGS := normal
.PHONY: all
all: foo.pdf foo.expanded.pdf
%.expanded.pdf: BUILDFLAGS := special
%.pdf:
$(BUILDER) $(BUILDFLAGS) $#
should do about what you want with the same rule for all targets, plus one pattern-specific variable value declaration. Replace builder, normal and special with what makes sense in your case. Demo:
$ make foo.pdf
builder normal foo.pdf
$ make foo.expanded.pdf
builder special foo.expanded.pdf
$ make
builder normal foo.pdf
builder special foo.expanded.pdf
Your problem has nothing to do with .SECONDEXPANSION. You can just drop that and the problem will be the same.
Your problem is that you are using a pattern rule with multiple target patterns, and expecting that it works similar to an explicit rule with multiple targets. But it does not (and in fact you cannot have a rule with both pattern and explicit targets).
For a pattern rule with multiple target patterns, Make matches the same pattern to all the %, including multiple times in the targets, and then assumes that it just has to execute the recipe with that pattern once, and it will make all the matched targets.
In your case the best way is to use multiple rules (I changed your recipe because using echo as a Make recipe is a bad idea):
.PHONY: all
all: foo.expanded.pdf foo.pdf
RECIPE = touch $#
%.expanded.pdf:
$(RECIPE)
%.pdf:
$(RECIPE)
I'm using Automake.
I have a few source files listed in dist_man1_MANS like this:
dist_man1_MANS = some-file.1 some-other-file.1
Now, Automake + configure eventually generate this in Makefile:
dist_man1_MANS = some-file.1 some-other-file.1
# ...
install-man1: $(dist_man1_MANS)
# generated recipe here
Since I'm not prefixing the .1 files with $(srcdir), I assume that, since I run make from the build directory (its current working directory), it should find them in the build directory.
So, I'm doing an out-of-tree build, for example, in /tmp/build:
/path/to/src/configure --prefix=$(pwd)/install
make
make install
and the build succeeds, that is, make finds the man pages and installs them. They are not in the build directory, however. I add this to the generated Makefile:
install-man1: $(dist_man1_MANS)
#echo ">>> $(^)"
#echo "::: $(dist_man1_MANS)"
# generated recipe here
Now, I would assume that both echos print the same thing, because $^ means the names of all the prerequisites, with spaces between them. To my surprise, the output is:
>>> /path/to/src/some-file.1 /path/to/src/some-other-file.1
::: some-file.1 some-other-file.1
So:
How did make find the /path/to/src/ prefix exactly? Where does it come from in this very context?
Why do $^ and $(dist_man1_MANS) differ?
I found the answer.
Automake sets the VPATH variable in its generated Makefile, a special variable for make, to something like:
VPATH = /path/to/src
From the previous links:
4.5.1 VPATH: Search Path for All Prerequisites
The value of the make variable VPATH specifies a list of directories that make should search. Most often, the directories are expected to contain prerequisite files that are not in the current directory; however, make uses VPATH as a search list for both prerequisites and targets of rules.
So make searches for the some-file.1 and some-other-file.1 prerequisites in the current working directory first, then for /path/to/src/some-file.1 and /path/to/src/some-other-file.1 if it cannot find the first ones. In this case I understand why $^ is different from $(dist_man1_MANS): $^ is the list of effective (resolved) prerequisites.
I have this implicit rule:
%.so: %.so.5
qnx_ln $< $#
I realized that for another target, I have to make all .so files the prerequisite for that target.
I tried this:
makegen: $(TEAM_ROOT)HMI_FORGF/src/src.pro module_dirs %.so
...
But I got the output
*** No rule to make target '%.so', needed by 'makegen'. Stop.
% prerequisite patterns can only be used in static and implicit pattern rules, where they match the respective % part of the target; when used in a regular rule % is a literal character.
You'll need to specify the dependencies literally, unless there is some correspondence between certain source filenames and the .so filenames that you can leverage, presumably you're already doing either of these to link the .so files in the first place.
As pointed out previously, no you can't do that because this is not how prerequisite patterns work. Maybe you gave the following a thought and rejected it but I suspect you might find the following a close-enough fit:
%.so.target: %.so.5
echo $< >> $(BUILD)/so.targets
SO_TARGETS=$(basename $(shell cat $(BUILD)/so.targets))
makegen: $(TEAM_ROOT)HMI_FORGF/src/src.pro module_dirs $(SO_TARGETS)
Maybe you are looking for a rule to match on every existing *.so file?
makegen: $(TEAM_ROOT)HMI_FORGF/src/src.pro module_dirs $(wildcard *.so)
...
However, if there are patterns which could generate *.so files which have not yet generated those files, they will (obviously) not be matched by the wildcard, which simply examines existing files. If that's what you actually want to accomplish, you'll probably want to enumerate the actual files, one way or another.
In a Makefile with
%.o: %.c
#echo Compiling $< ...
I'm getting the warning '%'-style pattern rules are a GNU make extension when I run autoreconf --install (of autoconf version 2.69). The warning is not very clear, but makes me think that there's something to be added to configure.ac.
I conducted searches on google.com, duckduckgo.com and yahoo.com, but they all don't seem to be able to differentiate between the large number of build logs they index (why should they...) which makes the search painful. I figured that:
I can silence the warning by adding AM_INIT_AUTOMAKE([-Wno-portability]) to configure.ac (found in a post of the openais mailing list) which seems not great because simply silencing a warning is generally not a good idea in a technical environment - please tell me if GNU autotools is an exception.
Replace
%.o: %.c
with
.c.o:
That's a suffix rule doing the same thing, and it's more portable. If your rule involves a suffix that is not known to make, list it in the prerequisites for the special .SUFFIXES target:
.SUFFIXES: .hack .win
.hack.win:
# build a .win file from a .hack file.
More on how this works in detail here. They recommend to use pattern rules instead of suffix rules because they're clearer and more general, which is true, but as autoconf notes, they are indeed less portable. So if that is a worry (if you want to build on BSD/Mac OS and not install GNU make, basically), fall back on the "old-fashioned suffix rules."
If you have a pattern rule that cannot be replaced by a suffix rule, another possible replacement that automake doesn't complain about is a static pattern rule. This is similar to a pattern rule but requires a list of targets it applies to. Instead of saying
%.o: %.c
You would have to say
OBJS = foo.o bar.o baz.o # list all .o files here
$(OBJS): %.o: %.c
Or more generally,
target-pattern: prerequisite-pattern
is replaced by
target-list: target-pattern: prerequisite-pattern
Winteermute covered almost all aspects of the issue. I will add one of my own frustrations here. In my make file, I have been using a pattern rule to install shell scripts without their .sh suffix. Automake does not like pattern rules and give you warning. You can simply ignore the warning for now. My configure.ac file:
AM_INIT_AUTOMAKE([-Wall -Wportability])
In my Makefile.am
dist_bin_SCRIPTS = foobar
# pattern rule
% : %.sh
cp $< $#
chmod +x $#
In my script directory I have the foobar.sh file.
I have not been able to figure out a better way to specify a suffix rule to cover the case where the target does not have an extension although I really tried hard to search on the internet and read the manuals backward a few times.
The reason I am distributing the *.sh file is that the user does not have to know the implementation details. They just need to remember the name of the executable. Furthermore, asking the user to type the extra .sh is really a waste of life.
Assume I have a make rule:
.PHONY:gen
gen: auto.template
generate-sources auto.template
that creates a bunch of files, for example auto1.src, auto2.src, auto3.src and so on.
If I now have rules to build targets from *.src files, like this:
$(patsubst %.src,%.target,$(wildcard *.src)): %.target: %.src
build $< > $#
How can I tell make to first execute the gen rule and then expand the preconditions for the second rule template? GNU extensions are welcome.
Note: I would like to keep it in one make invocation; A trivial solution to this would be to put the second rule in a secondary Makefile.secondrun and call $(MAKE) -f Makefile.secondrun after gen was processed. But I was wondering if there is a better option.
Building off Beta's answer, here's how you can do it using makefile remaking in GNU make, which is not the same thing as recursive make. Rather, it updates an included makefile using a rule in the main makefile, then restarts the original make instance. This is how *.d dependency files are typically generated and used.
# Get the list of auto-generated sources. If this file doesn't exist, or if it is older
# than auto.template, it will get built using the rule defined below, according to the
# standard behavior of GNU make. If autosrcs.mk is rebuilt, GNU make will automatically
# restart itself after autosrcs.mk is updated.
include autosrcs.mk
# Once we have the list of auto-generated sources, getting the list of targets to build
# from them is a simple pattern substitution.
TARGETS=$(patsubst %.src,%.target,$(AUTO_SRCS))
all: $(TARGETS)
# Rule describing how to build autosrcs.mk. This generates the sources, then computes
# the list of autogenerated sources and writes that to autosrcs.mk in the form of a
# make variable. Note that we use *shell* constructs to get the list of sources, not
# make constructs like $(wildcard), which could be expanded at the wrong time relative
# to when the source files are actually created.
autosrcs.mk: auto.template
./generate-sources auto.template
echo "AUTO_SRCS=`echo *.src`" > autosrcs.mk
# How to build *.target files from *.src files.
%.target: %.src
#echo 'build $< > $#'
Short answer: you can't. Make determines all of the rules it will have to execute before it executes any rule.
Longer answer: maybe you can. As you say, you can use recursive Make explicitly, or surreptitiously by, say, building a file which your makefile will include (I'm looking at you, Jack Kelly). Or if you could somehow obtain a list of the files which gen will build, you could write a rule around that. Or you could take a leap of faith like this:
%.target: %.src
build $< > $#
%.src: gen;