This question already has answers here:
GNU Makefile rule generating a few targets from a single source file
(9 answers)
Closed 1 year ago.
How do I write a rule to generate set of files using a single action.
Example:
Files x, y, z are generated as a result of single execution of script t.sh which takes file a as input.
x y z: a
t.sh $#
GNU make tries to execute t.sh 3 times.
You could implement one of the solutions specified in the automake manual.
Because you've tagged this gnumake, I should also point out that using a GNU make pattern rule (the ones with %) with multiple targets WILL consider both generated from one execution of the rule: see the GNU make manual.
Related
This question already has answers here:
What do #, - and + do as prefixes to recipe lines in Make?
(2 answers)
Closed 4 years ago.
In a few makefiles, I came across #rm syntax, vs -rm (ignore errors thrown by rm) and pure rm.
I'd like to know what's the difference.
# tells make to not echo the recipe to output when it's run.
- tells make to ignore the return value of the recipe (assume it passes)
you can use a combination of both of these and +. See the Echoing and Errors
in the gnu make manual for details
I encountered such pattern in makefile
CXXOBJ = f1.o f2.o f3.o
$(CXXOBJ): %.o: %.cpp
g++ -c $< -o $#
f1.o: f1.cpp f1.hpp f2.hpp
f2.o: f2.cpp f2.hpp f3.hpp macros.h
f3.o: f3.cpp f3.hpp
It works (at least with GNU make 4.0).
It uses generic recipe from 4th line,
but in addition uses dependencies defined at the bottom.
Questions
Is it standard make behavior? (or is it specific to GNU-make?)
Is it standard way to write make file? (i.e. are people usualy doing it this way or is it something 'exotic'?)
How exactly does it work?
How does make combine 2 distinct rules for same file? (just append dependency list or something more?)
(I was browsing through GNU-make manual, but could not find relevant part)
This is called static pattern rules (https://www.gnu.org/software/make/manual/html_node/Static-Usage.html). It is specific to GNU make. It might be useful when different targets require different recipes to build, but match the same pattern.
As for third question, there are no distinct rules for the same file. Everything is quite well defined, each target have corresponding .cpp file.
GNU Make manual:
One file can be the target of several rules. All the dependencies
mentioned in all the rules are merged into one list of dependencies
for the target....
There can only be one set of commands to be executed for a file. If
more than one rule gives commands for the same file, make uses the
last set given and prints an error message...
This question already has answers here:
Prioritizing pattern rules in Make
(2 answers)
Closed 2 years ago.
The situation below is simplified heavily, but it does reflect the problem I'm having.
I have a Makefile that looks like this:
prefix-%.zip: prefix-%
zip -r $# $<
prefix-%: base
cp -r base $#
This is placed in a directory together with another directory base containing some files. If I execute the command
make prefix-1.zip
I would like the directory prefix-1 to be created using the second rule, and then the zip file prefix-1.zip to be created based on that directory using the first rule.
However, it seems like it finds the first rule with the stem '1', but it doesn't pick that rule, because the directory doesn't exist. It then seems to favour the second rule with the stem '1.zip'. Although this is a longer stem, that rule gets picked. I assume that this is due to the missing prerequisite for the first rule. Is there some way I can guide Make to the right rule, without manually first making the directory?
Btw, first making the directory and then the zip file works
make prefix-1
make prefix-1.zip
This is a kludge, but it works:
prefix-%.zip: prefix-%
zip -r $# $<
prefix-%: base | dummy%
cp -r base $#
dummy%:
#:
According to the manual, "a rule whose prerequisites actually exist or are mentioned always takes priority over a rule with prerequisites that must be made by chaining other implicit rules." So we make the second rule less attractive by giving it a dummy prerequisite that must be chained. Crude but effective. And we use the pipe (|) to make it an "order-only" prerequisite, so as not to force the directory to be rebuilt needlessly.
Here is my solution.
prefix-%.zip: base
cp -r base $(subst .zip,,$#)
zip -r $# $(subst .zip,,$#)
I use the subst function to strip out the .zip extension by replacing .zip with the empty string.
I am currently working on a project using templates quite extensively, and running into memory constraints during instantiation. I have split up the instantiation into a number of very simple files, which are all three-liners consisting of includes only.
I let these be generated by a rule in Makefile.am. Now I have a quite long list of files that should be generated in my Makefile, and would like to refactor this list with a foreach-like expression. In more specific terms: I have a line like
libfoo_la_SOURCES = a_0.cpp a_1.cpp ... b_0.cpp b_1.cpp ... c_0.cpp c_1.cpp ...
which could be more concisely expressed as
libfoo_la_SOURCES = $(foreach i,a b ...,$(foreach j,0 1 ...,$i_$j.cpp))
However, the second construct is not only warned against by automake, but also does not work: The files given in this manner are neither compiled nor cleaned.
My current workaround is generating this file list by a shell script.
Any ideas how to implement this iteration?
I would forget about making loops: the GNU extension is not standard, and not understood by Automake. One standard (and portable) make construction you can use here is the macro expansion with substitution: $(var:subst1=subst2) will expand to the value of $(var) after replacing any suffix subst1 of a word by subst2. Automake understands this.
If subst1 is empty, as in $(var:=subst2), you are appending subst2 to all files in $(var). You can use this to construct your list of files as follows:
f = a b c d e f
g = $(f:=_0) $(f:=_1) $(f:=_2) $(f:=_3)
all_files = $(g:=.cpp)
echo:
#echo $(all_files)
Running make echo with the above Makefile will display all files from a_0.cpp to f_3.cpp.
Like you, I discovered that the GNU make foreach function will not work like this because
the sources need to be there at the time the Makefile is generated. So, I use GNU Autogen (also here) to generate a makefile fragment which is subsequently included in Makefile.am. So it's probably not that different than your shell script approach.
Because GNU make allows variables to be as large as memory allows, it has no problem building massive dependency lists. However, if you want to actually use these lists of files in a recipe (sequence of shell commands for building a target), you run into a problem: the command might exceed the shell's command line length limit, producing an error such as "Argument list too long".
For example, suppose I want to concatenate several files contained in the list $(INPUTS) to produce a file combined.txt. Ordinarily, I could use:
combined.txt: $(INPUTS)
cat $^ > $#
But if $(INPUTS) contains many thousands of files, as it does in my case, the call to cat is too long and fails. Is there a way to get around this problem in general? It's safe to assume that there exists some sequence of commands that have identical behaviour to the one enormous command -- in this case, a series of cat commands, one per input file, that use >> to append to combined.txt would work. But how can make be persuaded to generate those commands?
In looking for the answer, about the best suggestion I could find was to break up the list into a series of smaller lists and process them using shell for loops. But you can't always do that, and even when you can it's a messy hack: for example, it's not obvious how to get the usual make behaviour of stopping as soon as a command fails. Luckily, after much searching and experimentation, it turns out that a general solution does exist.
Subshells and newlines
make recipes invoke a separate subshell for each line in the recipe. This behaviour can be annoying and counterintuitive: for example, a cd command on one line will not affect subsequent commands because they are run in separate subshells. Nevertheless it's actually what we need to get make to perform actions on very long lists of files.
Ordinarily, if you build a "multiline" list of files with a regular variable assignment that uses backslashes to break the statement over multiple lines, make removes all newlines:
# The following two statements are equivalent
FILES := a b c
FILES := \
a \
b \
c
However, using the define directive, it's possible to build variable values that contain newlines. What's more, if you substitute such a variable into a recipe, each line will indeed be run using a separate subshell, so that for example running make test from /home/jbloggs with the makefile below (and assuming no file called test exists) will produce the output /home/jbloggs, because the effect of the cd .. command is lost when its subshell ends:
define CMDS
cd ..
pwd
endef
test:
$(CMDS)
If we create a variable that contains newlines using define, it can be concatenated with other text as usual, and processed using all the usual make functions. This, combined with the $(foreach) function, allows us to get what we want:
# Just a single newline! Note 2 blank lines are needed.
define NL
endef
combined.txt: $(INPUTS)
rm $#
$(foreach f,$(INPUTS),cat $(f) >> $#$(NL))
We ask $(foreach) to convert each filename into a newline-terminated command, which will be executed in its own subshell. For more complicated needs, you could instead write out the list of filenames to a file with a series of echo commands and then use xargs.
Notes
The define directive is described as optionally taking a =, := or += token on the end of the first line to determine which variable flavour is to be created -- but note that that only works on versions of GNU make 3.82 and up! You may well be running the popular version 3.81, as I was, which silently assigns nothing to the variable if you add one of these tokens, leading to much frustration. See here for more.
All recipe lines must begin with a literal tab character, not the 8 spaces I have used here.