I encountered such pattern in makefile
CXXOBJ = f1.o f2.o f3.o
$(CXXOBJ): %.o: %.cpp
g++ -c $< -o $#
f1.o: f1.cpp f1.hpp f2.hpp
f2.o: f2.cpp f2.hpp f3.hpp macros.h
f3.o: f3.cpp f3.hpp
It works (at least with GNU make 4.0).
It uses generic recipe from 4th line,
but in addition uses dependencies defined at the bottom.
Questions
Is it standard make behavior? (or is it specific to GNU-make?)
Is it standard way to write make file? (i.e. are people usualy doing it this way or is it something 'exotic'?)
How exactly does it work?
How does make combine 2 distinct rules for same file? (just append dependency list or something more?)
(I was browsing through GNU-make manual, but could not find relevant part)
This is called static pattern rules (https://www.gnu.org/software/make/manual/html_node/Static-Usage.html). It is specific to GNU make. It might be useful when different targets require different recipes to build, but match the same pattern.
As for third question, there are no distinct rules for the same file. Everything is quite well defined, each target have corresponding .cpp file.
GNU Make manual:
One file can be the target of several rules. All the dependencies
mentioned in all the rules are merged into one list of dependencies
for the target....
There can only be one set of commands to be executed for a file. If
more than one rule gives commands for the same file, make uses the
last set given and prints an error message...
Related
I'm working on a rather large project in pure C, and for various reasons we are not using CMake.
We have a good system which uses various shell commands to require very little maintanence. It will automatically find new headers and C files and add the headers to the dependencies and will compile the C files and include them in the output. However, we've gotten to the point where any changes we make to the header files becomes a problem because it requires a recompilation of the entire project, rather than just the C files that include it directly or indirectly.
I've worked on a system to ensure that the only header files which are added as dependencies are ones which are required by the C file itself (recursively up the tree).
I had thought I would be able to solve this myself, but it seems that make has rather inconsistent rules for expansion of variables.
The solution I came up with was to try to use ag and get all the includes out of the file. This works as intended, and I pipe it into tr so that I can ensure that no newlines are added, thereby messing up make. The issue that I'm having though is in determining which file to search in. This is the recipe that it's using for the section in question:
$(GAMEOBJDIR)/%.o : $(GAMEDIR)/%.c $(shell ag -o '(?<=(^#include "))(.*?)(?=("$$))' $(GAMEDIR)/%.c | tr '\n' ' ')
$(CC) $(CFLAGS) $(if $(RELEASE),$(RELFLAGS),$(DBFLAGS)) -c -o $# $<
The problem is that $(GAMEDIR)/%.c is expanding to src/game/%.c and not src/game/<filename>.c. I'm unsure how to fix this issue based on the expansion rules of make.
Once I can figure this out I'll be able to make sure this walks up the chain of header files too, but until I can get this figured out I have no reason to work on that.
Make's rules for expansion are completely consistent... but sometimes they aren't what people want. That's not the same thing :)
Rules for expansion are explained in the manual. In your case you're working with a prerequisite list which means the variables are expanded as the makefile is parsed. That includes the shell command and all other variables. Because you are working with a pattern rule, it means that at the time the makefile is parsed it doesn't match any particular file, it's just defining the pattern rule itself. So, % can't be expanded here. It will be expanded later, when make starts walking the dependency graph and trying to locate ways to build targets.
In general your method cannot work efficiently. You could defer the expansion of these parts of the prerequisites list using Secondary Expansion. However, that means every time make wants to TRY to use this pattern it will run these commands--that will just be slow.
Have you considered using a more standard way to manage detecting prerequisites? Try reading this description for example to see if it would work for you.
If you read the Catalog of Rules section of the GNU Make manual, it turns out that one of the suffixes that it recognizes is .out and there is a built-in rule:
%.out: %
# commands to execute (built-in):
#rm -f $#
cp $< $#
This means that if you have a file xyz in a directory, you can create xyz.out by simply typing make xyz.out.
My question is (two variants of the same question):
Who benefits from this rule?
In what circumstances is this used by people?
Obviously, I'm asking because I managed to run foul of the rule. I had some rules like:
test.01: ${PROGRAM} ${DRIVER} test.01.tst test.01.out ${DATA.01}
${DRIVER} ${D_FLAGS} $#
where the name test.01 is a phony target, but one of the dependencies is test.01.out. When actively run (not using make -n; that works fine), this gives me lots of errors like:
make[1]: Circular test.01 <- test.01.out dependency dropped.
I also tried dropping the .out suffix with:
.SUFFIXES:
.SUFFIXES: .sh
and that didn't seem to neuter the .out rule like I expected. Is that an expected feature of GNU Make?
I guess I'm going to have to work around this bug feature of GNU Make by changing my suffix to .req or something similar, but it is a nuisance and I'm left puzzled about why the .out rule is part of the standard GNU Make rule set.
I don't know the answer to your questions about the use of this rule. All I can say is that this rule already existed when GNU make was first checked into source control, in Jan 1992. It's not mentioned in any ChangeLog so probably it dates back to the very earliest versions.
The actual rule is defined as a pattern rule, so changing .SUFFIXES won't help. To get rid of it you can use:
%.out : %
(no recipe) which will delete the pattern rule.
In our makefile, we have one recipe that links together all our object and library files to make an executable (an .elf file). As a side effect, this step also produces a map-file and an Intel .hex file:
$(ELF_FILE) : <list of dependencies here>
<linker command line>
Until now, since we never actually had a $(MAP_FILE) or a $(HEX_FILE) target, when ever another target depended on one of the $(ELF_FILE)'s side products, we simply declared it to be dependent on $(ELF_FILE), even if the recipe of that target didn't want to access the $(ELF_FILE) itself at all. For instance:
# Target that needs map-file, which is a side product of the $(ELF_FILE) target.
$(TARGET_THAT_NEEDS_MAP_FILE) : $(ELF_FILE)
<build-recipe>
# Target that needs hex-file, which is also a side product of the $(ELF_FILE) target.
$(TARGET_THAT_NEEDS_HEX_FILE) : $(ELF_FILE)
<build-recipe>
We have recently found out that a recipe can be used for more than one target, like so:
$(MAP_FILE) $(HEX_FILE) $(ELF_FILE) : <list of dependencies here>
<linker command line>
With this new-found knowledge, we figured we could get rid of the above "hack" and just directly state each target's direct dependencies:
$(TARGET_THAT_NEEDS_MAP_FILE) : $(MAP_FILE)
<build-recipe>
$(TARGET_THAT_NEEDS_HEX_FILE) : $(HEX_FILE)
<build-recipe>
Having implemented these changes, we now observe an odd effect that makes us suspect that we've either misunderstood this multiple-targets-one-recipe feature of make, or we're not using it correctly. The odd effect is that the recipe that produces the .elf, .map and .hex files now appears to run twice. This doesn't seem to have caused any immediate problems, but it does seem to indicate that something is fishy here. So my question, can our new approach work at all, or should we stick to the hack I described above?
EDIT: We're running our make in a multi-threaded manner (i.e. with -j).
It might be that when make is trying to update a target (whether it is $(MAP_FILE), $(HEX_FILE) or $(ELF_FILE), it does not know that its recipe will also update another target, therefore it starts a recipe for that one too, even if it's the same.
Of course, that would only happen when using the -j option. (Did you had the possibility to try without ?)
To illustrate :
$(TARGET): $(ELF_FILE) $(MAP_FILE)
<update target>
Here make will try to update $(ELF_FILE) and $(MAP_FILE) and fire the recipe twice. (That should also applies if the dependencies are on different target, as long as the targets are updated by a one execution of make and that there is no bottlenecks between them.
I'm not completely sure about that, though, make might be able to know that this is the same recipe.
======
This answer might be of use to you.
Specifically :
However, if your output files and your input file share a common base,
you CAN write a pattern rule like this:
%.foo %.bar %.baz : %.boz ; $(BUILDIT)
Strangely, for implicit rules with multiple targets GNU make assumes
that a single invocation of the recipe WILL build all the targets, and it will behave exactly as you want.
MadScientist
It refers to that part of the make manual :
Pattern rules may have more than one target. Unlike normal rules, this
does not act as many different rules with the same prerequisites and
recipe. If a pattern rule has multiple targets, make knows that the
rule’s recipe is responsible for making all of the targets. The recipe
is executed only once to make all the targets. When searching for a
pattern rule to match a target, the target patterns of a rule other
than the one that matches the target in need of a rule are incidental:
make worries only about giving a recipe and prerequisites to the file
presently in question. However, when this file’s recipe is run, the
other targets are marked as having been updated themselves.
EDIT:
Gnu Make has now gained a feature that would support this usecase (in version 4.3) : grouped explicit targets. It allows make to be aware that one recipe generate several targets, and it used like this (from the gnu make manual) :
foo bar biz &: baz boz
echo $^ > foo
echo $^ > bar
echo $^ > biz
foo, bar, and biz are generated by this rule (note the use of &: instead of :.
Full documentation : https://www.gnu.org/software/make/manual/html_node/Multiple-Targets.html ("Rules with Grouped Targets")
I'm using GNUmake to generate a deployment version of some web content from a source directory. I want to be rules that will run some files through compression tools (e.g. YUI compressor), and then for anything that doesn't have a rule, just copy it.
So for example:
# Rule with all the $(WWW_OUT_DIR)/*.* files as a prerequisite
all: $(WWW_OUT_FILES)
# Generic rule to perform compression on Javascript files.
$(WWW_OUT_DIR)/%.js: $(WWW_SRC_DIR)/%.js
$(YUI_COMP) $(YUI_COMP_OPTS) $< > $#
# Generic rule to perform compression on CSS files.
$(WWW_OUT_DIR)/%.css: $(WWW_SRC_DIR)/%.css
$(YUI_COMP) $(YUI_COMP_OPTS) $< > $#
# TDB Rule to magically handle everything else? (This doesn't work)
$(WWW_OUT_DIR)/%.%: $(WWW_SRC_DIR)/%.%
cp $< $#
How do I accomplish what that last rule is trying to do? I.e. For everything in $(WWW_OUT_FILES) that's not a .js or .css, just copy it? If possible, I want to retain the dependency on the corresponding input file.
You are almost right, the only thing to fix is the last pattern rule, just remove redundant percent symbol:
$(WWW_OUT_DIR)/%: $(WWW_SRC_DIR)/%
cp $< $#
Also keep in mind that as of GNU Make 3.82 pattern search algorithm has been modified a bit (from Changelog):
The pattern-specific variables and pattern rules are now applied in the
shortest stem first order instead of the definition order (variables
and rules with the same stem length are still applied in the definition
order). This produces the usually-desired behavior where more specific
patterns are preferred.
It is exactly what you want in case if you use the most recent version of Make. In order to get your Makefile compatible with other versions of GNU Make (including versions earlier than 3.82), the rule must be defined after other ones (as in the original question).
UPD.
A good example from here:
Prior to version 3.82, when gmake finds multiple matches during a pattern search, it prefers patterns declared earlier in the makefile over patterns declared later. As of 3.82, gmake instead prefers the pattern that results in the shortest stem. That sounds a bit confusing thanks to the jargon, but I think this will actually cause gmake to better adhere to the principle of least astonishment. Here’s an example:
all: sub/foo.x
%.x:
#echo "Prefer first match (stem is $*)."
sub/%.x:
#echo "Prefer most specific match (stem is $*)."
Compare the output from gmake 3.81 and 3.82:
gmake 3.81
Prefer first match (stem is sub/foo).
gmake 3.82
Prefer most specific match (stem is foo).
gmake 3.82 prefers the second pattern because it is a more specific match than the first. Note that this is a significant backwards-incompatibility compared with previous versions of gmake!
Is there a mechanism in make to allow for default global implicit rules that are available anywhere, similar to the built-in rules?
Make provides some built-inimplicit rules for compiling C/C++/Fortran files, without even requiring a Makefile for simple cases. However, when compiling other languages (e.g. Go programming language files), a Makefile is always required. I would like to extend my Makeenvironment to have implicit rules available by default.
This is not normally desirable, as it would cause your Makefile to be less portable; it wouldn't work on somebody else's machine if they didn't have it set up that way.
However, if you want to do this, create a "global" Makefile somewhere with your default rules for Go files, then add its path to the MAKEFILES environment variable. This global Makefile will be processed before any Makefile when you run "make", just as if you had included its source at the top of the file.
I'm assuming you're referring to the fact that you can do
make hello.o
and make will automatically know how to make the .o from a .c file (or indeed from a .f or .p, if one exists) - but you want to do this for custom file types (say, building a .bar from a .foo.
The most portable way of doing this is as follows (in your Makefile):
.SUFFIXES: .foo .bar
.foo.bar:
foo2bar -in $> -out $#
The first line (.SUFFIXES) warns make that you'll be treating these as special suffixes; the second line says "here's a recipe for making a .bar from a .foo. The third line gives the command for doing this - $> and $# get changed by make to the input and output filenames.
NOTE: The indent for the third line MUST be a tab character.
A much more flexible method, that only works with GNU make, is to use its support for implicit rules. If you can guarantee you'll be using GNU make then this is probably to be recommended.
While I agree with dmazzoni, I just though I'd add my make recipe for a Go Makefile:
# Include default Golang Make magic
include $(GOROOT)/src/Make.$(GOARCH)
# Hack the following line
your_program: your_program.$O
$(LD) -o $# $^
# Compiles .go-files into architecture-specific binaries
%.$O: %.go
$(GC) -o $# $^
clean:
rm your_program *.$O
(Note: the $O is DOLLAR + UPPERCASE-o - not zero!)
While I haven't tested it on all the machines I have available, i believe it should port fairly well.