I have a directory structure that looks like this:
$ tree
.
|-- dir
| `-- subdir
| `-- data
`-- makefile
where data is a file. My makefile looks like this:
all: dir/analysis
.SECONDEXPANSION:
%/analysis: %/subdir
%/analysis: $(addsuffix /data, $$^)
#echo TARGET: $# DEPS: $^
touch $#
When I run make, I would expect the result to look like this:
TARGET: dir/analysis DEPS: dir/subdir/data dir/subdir
touch dir/analysis
Instead, it just reports
make: *** No rule to make target `dir/analysis', needed by `all'. Stop.
If I change the first rule to dir/analysis: dir/subdir then it works as I expected. So I suspect that make ignores the first rule and skips straight to the second when the first rule is %/analysis: %/subdir. It also works as expected when both rules have dir/analysis as their target instead of just the first rule. MadScientists's answer to a different question here seemed to apply to my problem. I tried adding rules like
dummy_target: dir/subdir
mkdir -p dir/subdir
and
dir/subdir:
mkdir -p dir/subdir
to the end of makefile to try to make the dependency on the first rule an explicit target but this didn't change anything. I'm pretty new to make, so I'm probably missing something pretty stupid, but I can't for the life of me figure it out. How do I get the first and second rules to execute in the order they're written? I'm using Make version 3.81 in case that matters.
--EDIT--
If I actually add a command after the first rule, like #echo RULE 1 then that command executes and not the second one.
I think that you understand the makefile:
# Original
all: dir/analysis
.SECONDEXPANSION:
%/analysis: %/subdir
%/analysis: $(addsuffix /data, $$^)
#echo TARGET: $# DEPS: $^
touch $#
like this:
By default we make dir/analysis
We request secondary expansion
We say that whatever/analysis depends on whatever/subdir. So
that makes dir/subdir a prerequisite of dir/analysis
We say also that whatever/analysis depends on the thing we get by
suffixing /data to the prior prerequisites of whatever/analysis. (Secondary
expansion of $$^ gives us the prior prerequisites of the target). And
since the prior prerequisites of dir/analysis are dir/subdir, that now
makes dir/subdir dir/subdir/data the new prerequisites of dir/analysis.
Hence the output of the recipe ought to be:
TARGET: dir/analysis DEPS: dir/subdir/data dir/subdir
touch dir/analysis
This understanding is badly wrong. You're confusing and conflating the operation
of pattern rules with the operation or ordinary rules.
If a makefile says, e.g:
# A
xx: xa
xx: xb
xx: xc
touch $#
or even:
# B
xx: xa
xx: xb
%x: %c
touch $#
then the prerequisites of all the empty rules for the target xx are combined
with those of the (one) non-empty rule when xx is considered as a target. So
# A is equivalent to:
xx: xa xb xc
touch $#
and # B is equivalent to:
xx: xa xb
%x: %c
touch $#
and in both cases the prerequisites of xx are xa xb xc.
However:
%x: %a
%x: %b
touch $#
is not equivalent to:
%x: %a %b
touch $#
If you ever write an empty pattern rule, its effect is simply to delete
any prior pattern rule with the same target and prerequisite patterns. And if you
ever write a non-empty pattern rule, you simply replace
any prior pattern rule with the same target and prerequisite patterns. See
10.5.6 Canceling Implicit Rules
So:
.SECONDEXPANSION:
%/analysis: %/subdir
%/analysis: $(addsuffix /data, $$^)
#echo TARGET: $# DEPS: $^
touch $#
is equivalent to:
.SECONDEXPANSION:
%/analysis: $(addsuffix /data, $$^)
#echo TARGET: $# DEPS: $^
touch $#
Then when this pattern rule is considered with respect to dir/analysis,
there are no prior prerequisites: $^ will expand to the empty string, and
in secondary expansion, so will $$^. So finally the recipe is equivalent to:
# Residual
%/analysis: /data
#echo TARGET: $# DEPS: $^
touch $#
You can satisfy yourself of this by running both of the # Original and
# Residual makefiles in debug mode (-d) and comparing the outputs.
This explains why the outputs in both cases conclude:
make: *** No rule to make target 'dir/analysis', needed by 'all'. Stop.
A pattern rule will only be instantiated to make a target if doing so offers a way for the target to be made. Why else select it? Since the prerequisite /data does not exist, the pattern rule
is not viable and is discarded. Just as if you tried to make foo.o with the makefile:
%.o: %c
touch $#
when there is no foo.c in the directory.
If you really want to see the output you were expecting, then you can get it from:
.PHONY: all clean
all: dir/analysis
%/analysis: %/subdir %/subdir/data
#echo TARGET: $# DEPS: $^
touch $#
clean:
rm -f dir/analysis
But it is hard to believe that is what you really want to see. This makefile
makes the analysis dependent on the data and also on the directory where
the data resides. What can be the point of that?
%/analysis: %/subdir/data
presumably expresses the desired relationship between the data and the analysis:-
If the data doesn't exist then the analysis is out of date and cannot be made (till you get some data).
If the data exists and is older than the analysis then the analysis does not
need to be made.
If the data exists and is newer than the analysis then the analysis can and will be
made.
By making dir/subdir an independent prerequisite all that you achieve is to introduce
a requirement to remake the analysis if it becomes older than the directory in
which the data resides - regardless of the data. So if, e.g. I run touch dir/subdir,
that will require make to re-run the analysis, even though the data hasn't changed.
I can make sense of your motivation only by supposing that in
reality you also want make to make the data, and the directory where it resides, if
they happen not exist when an analysis is required. If that is the situation then
you want a makefile on the lines of:
.PHONY: all clean
all: dir/analysis
.SECONDARY:
%/subdir:
mkdir -p $#
%/subdir/data: | %/subdir
touch $#
%/analysis: %/subdir/data
#echo TARGET: $# DEPS: $^
touch $#
clean:
rm -f dir/analysis dir/subdir/data
Here, the pattern rule:
%/subdir/data: | %/subdir
makes %/subdir an order-only prerequisite
of %/subdir/data. It means that /dir/subdir is not considered when determining whether /dir/subdir/data needs
to be made, but if it is determined that /dir/subdir/data is to be made, then /dir/subdir will be made first. This
way, /dir/subdir will be created, if it does not exist, when you need it to put the data there, but will have no influence
on whether you need to make the data or the analysis.
The special target .SECONDARY is a technicality. It directs make not to automatically delete any subsequent targets
that it deduces to be intermediate artefacts that emerge from chained pattern rules. Without it, in this example make
would deduce that dir/subdir/data and dir/subdir/ are just waste-products of making dir/analysis and auto-delete them
at the end.
If you run this makefile initially with no dir/subdir/data and no dir/subdir/ you get:
$ make
mkdir -p dir/subdir
touch dir/subdir/data
TARGET: dir/analysis DEPS: dir/subdir/data
touch dir/analysis
And subsequently:
$ make clean
rm -f dir/analysis dir/subdir/data
$ make
touch dir/subdir/data
TARGET: dir/analysis DEPS: dir/subdir/data
touch dir/analysis
And then:
$ make
make: Nothing to be done for 'all'.
$ touch dir/subdir
$ make
make: Nothing to be done for 'all'.
Related
I'm using a makefile to build some documents from markdown. Since I have various possible outputs, I'm using a pattern rule. And since the input documents may or may not have associated tables, they have a wildcard dependency for csv files with the same stem filename:
tables = $(wildcard $*_*.csv)
%.docx: %.md $(tables)
#echo building $#
#touch $#
This works fine the first time:
$ touch thing.md
$ touch thing_table.csv
$ make thing.docx
building thing.docx
$ ls thing.docx
thing.docx
However, if the table file is updated, make still thinks everything's up to date:
$ touch thing_table.csv
$ make thing.docx
make: `thing.docx' is up to date.
I think this must have something to do with the order in which make evaluates things, but I don't understand it well enough to figure out how to make this work.
I can do sort of what I want to do using a more "literal" pattern rule:
%.docx: %.md %_*.csv
#echo building $#
#touch $#
But of course this fails in the absence of appropriately-named csv files:
$ rm thing.docx thing_table.csv
$ make thing.docx
make: *** No rule to make target `thing.docx'. Stop.
Is there a way to specify a target dependency such that if a target is foo.docx the dependencies would be foo_*.csv, but have those dependencies be optional (i.e. the target still works if the files don't exist) AND have it appropriately update if those optional dependencies change?
Ok, it looks like secondary expansion will work here:
.SECONDEXPANSION:
%.docx: %.md $$(wildcard $$*_*.csv)
#echo building $#
#touch $#
I have a makefile like the following:
.PHONY: all
all: foo_1.txt foo_2.txt foo_xxx.txt
.PHONY: clean
clean:
rm -f foo_* bar_*
foo_%.txt: bar_%.txt
cp $< $#
#.PRECIOUS: bar_%.txt
bar_%.txt:
touch $#
bar_2.txt:
touch $#
The output of "make all" is folowing
touch bar_1.txt
cp bar_1.txt foo_1.txt
touch bar_2.txt
cp bar_2.txt foo_2.txt
touch bar_xxx.txt
cp bar_xxx.txt foo_xxx.txt
rm bar_xxx.txt bar_1.txt
The intermediate files created by the rule using pattern (bar_xxx.txt, bar_1.txt) are removed on the end. I have found that this behaviour can be supressed by .PRECIOUS (in the code is the line intentionally commented out).
Why are intermediate files created by rule with pattern removed by default and files created by rule without pattern are not?
By the definition of "intermediate files", you can't have an intermediate file created by an explicit rule (a rule "without a pattern").
See the section on Chains of Implicit Rules to understand this feature. If you have specific questions then please update your question.
I have a chain of pattern dependencies in a makefile, and in the end they should come together in one file, e.g.: *.x -> *.y -> onefile.z
So I made the files like this:
$ touch a.x b.x
and the rules:
%.y: %.x some-other-script
touch $#
onefile.z: %.y second-other-script
touch $#
This rule does not work:
$ make onefile.z
make: *** No rule to make target '%.y', needed by 'onefile.z'. Stop.
Using a wildcard:
%.y: %.x some-other-script
touch $#
z: $(wildcard *.y) second-other-script
touch $#
This does not work either: it sees there are no *.y files and proceeds making onefile.z skipping the first rule.
$ make onefile.z
touch onefile.z
$ ls
a.x b.x onefile.z Makefile
I probably could merge two rules into one, but in the real application, there are more steps, and some are making requests over HTTP and take much time, and in fact should not be repeated without a reason.
Is there a way to make such a dependency?
onefile.z: %.y second-other-script is a regular rule, not a pattern rule or a static pattern rule, so the % is interpreted literally. Even if it were a pattern rule, how is make supposed to infer what the stem is supposed to match?
$(wildcard *.y) tells make to find all the files that match *.y, but of course there are none yet so it returns an empty string.
The following should work, if I've understood your question correctly:
xfiles := $(wildcard *.x)
yfiles := $(xfiles:.x=.y)
%.y: %.x some-other-script
touch $#
onefile.z: $(yfiles) second-other-script
touch $#
See the Gnu Make documentation on
Wildcard functions $(wildcard ...)
Substitution functions $(foo:from=to)
I'm trying to generically add some behaviour to every target in a Makefile, without modifying the targets.
My current attempt is thus:
%: $*
#echo 'Logging $* target'
.PHONY: test
test:
#echo 'Inside explicit test target'
When I run make test, I'd like to match the % pattern rule, which would execute test as a prerequisite ($* expanding to the pattern stem), and then log the target that was run.
$ make test
Inside explicit test target
Logging test target
Instead, what happens is that make test matches the explicit test target (presumably since it's a closer match):
$ make test
Inside explicit test target
How can I get this to work, without changing the explicit test target?
EDIT:
Another attempt...
.SECONDEXPANSION:
%: $$*
#echo 'Logging $* target'
results in
$ make test
make: Circular Makefile <- Makefile dependency dropped.
inside actual test target
I appears from your own answer, which has beaten me to the punch, that
you're concerned only to trigger a preliminary action for targets that are
mentioned on the commandline - $(MAKECMDGOALS). From the posting I took
it that you wanted such an action for "every target in a Makefile", which
would include all targets that are prerequisite to the commandline targets or,
if there are no commandline targets, to the default target.
Anyhow, you may still be interested in a solution to the more general problem.
You want a preliminary action to be executed before the recipe for every target.
Your question is: how to match a patten rule before explicit rule?
This is an XY way of posing the problem, because make will consult pattern
rules to find a way of making a target only if you don't give it an explicit
recipe. You know, for example, that make has a pre-defined pattern rule for
making an .o file from a .c file. Even so, if my makefile is:
test.o:
#echo $#
then make prints test.o, without any attempt to find test.c and compile it.
And if my make file is:
test.o: test.c
#echo $#
test.c:
#echo $#
then make prints:
test.c
test.o
needing no resort to the pattern rule. But if my makefile is:
test.o: test.c
Then make says:
make: *** No rule to make target 'test.c', needed by 'test.o'. Stop
So you can't do what you're after in the way your question supposes,
because the preliminary action you want to provoke from the pattern
rule could be provoked only if there were no other action for the target.
In that case the reasons for the failures of your two posted attempts are fairly academic,
and you may wish to scroll to The Chase.
In your first attempt, with:
%: $*
#echo 'Logging $* target'
The pattern rule - which is unemployed by make test - is equivalent to:
%:
#echo 'Logging $* target'
because $* only assumes a value in the recipe, not in the pattern rule. You
can make this pattern rule be employed by making any target for which the
makefile does not provide a recipe, e.g. make nonsuch will print Logging nonsuch target;
but that is of no use.
The second attempt, with:
.SECONDEXPANSION:
%: $$*
#echo 'Logging $* target'
does the right thing to create the rule you intend to create. But the
meaning of that rule is:
<target>: <target>
#echo 'Logging <target> target'
making every target to which this rule is applied a prerequisite of itself.
Inevitably this will result in a circular dependency error for all such targets.
As you saw, this circularity does not affect the your test target because
it has an explicit recipe and does not employ the rule. But it does provoke
the surprising error:
make: Circular Makefile <- Makefile dependency dropped.
That happens because the first target that make automatically considers is
the makefile itself. Unlike the test target, you have no recipe for
the makefile; so the pattern rule applies to it, making the makefile dependent
on itself.
The Chase
You can achieve what you want by a different approach. In a actual project
it is more than likely that in any makefile you can compute a list of
all possible targets. From this you can generate a corresponding list of
auxiliary targets, say, target => target.prelim, where the
sole purpose of target.prelim is to provoke, when it should and not
otherwise, the required preliminary action for target; and you can get make
to generate a list of order-only rules, target: | target.prelim,
for each target, such that target.prelim will not be considered in determining whether target
must be made, but will be made before target whenever target needs to be made.
Here is an illustration:
SRCS := main.c foo.c
OBJS := $(SRCS:.c=.o)
TARGETS := all prog $(OBJS)
PRELIMS := $(patsubst %,%.prelim,$(TARGETS))
define prelim_rule =
$(1): | $(1).prelim
endef
$(foreach target,$(TARGETS),$(eval $(call prelim_rule,$(target))))
.PHONY: all
all: prog
prog: $(OBJS)
$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) -o $# $(OBJS) $(LIBS)
clean:
rm -f $(OBJS) $(PRELIMS) prog
%.prelim:
#echo "Logging target $(#:%.prelim=%)"
#touch $#
And a sample session:
$ make
Logging target all
Logging target main.o
cc -c -o main.o main.c
Logging target foo.o
cc -c -o foo.o foo.c
Logging target prog
cc -o prog main.o foo.o
$ make
make: Nothing to be done for 'all'.
$ make clean
rm -f main.o foo.o all.prelim prog.prelim main.o.prelim foo.o.prelim prog
$ make main.o
Logging target main.o
cc -c -o main.o main.c
$ make main.o
make: 'main.o' is up to date.
$ # A prelim can't out-date its target...
$ touch main.o.prelim
$ make main.o
make: 'main.o' is up to date.
I realise that this isn't answering my question as asked, but it has the effect I want - executing a shell command as late in the Makefile processing as possible.
MYVAR?=foo
.PHONY: test
test:
#echo 'Inside test target'
LOG=$(shell echo 'Logging $(MAKECMDGOALS), myvar=$(MYVAR)' > log)
.SECONDEXPANSION:
force: $$(LOG)
LOG is a deferred variable, so is not expanded until Make evaluates the prerequisite list of the force target.
In a single Makefile, the .SECONDEXPANSION: part is not needed, since the force target is evaluated after MYVAR is set.
However, if I move the LOG variable and force variable into a sub-makefile, it would be easy to include subMakefile before the MYVAR?= line - which would not work.
By specifying .SECONDEXPANSION for force, the reliance on ordering is removed.
BUILT_DIR = /tmp/obj
SRC = /source/dir
/tmp/obj/%/builtin.o : $(SRC)/%/*.c
gcc $^ -o $#
But you know :
In order for the pattern rule to apply, its target pattern must match the file name under consideration and all of its prerequisites (after pattern substitution) must name files that exist or can be made.
If i execute make /tmp/obj/hfa/builtin.o,make will complain :
make: *** No rule to make target/tmp/obj/hfa/builtin.o'. Stop.`
How can i modify the Makefile to satisfy my requirement？
You can use Secondary Expansion
BUILT_DIR = /tmp/obj
SRC = /source/dir
.SECONDEXPANSION:
/tmp/obj/%/builtin.o : $$(wildcard $(SRC)/%/*.c)
gcc $^ -o $#
The error you see indicates that there are no .c files which match the pattern $(SRC)/hfa/*.c since the % translates into hfa. So make cannot use the rule you've defined.
Make then starts to use the implicit rules for building and it would not match either.
Finally make gives up.
I just confirmed that the same Makefile and I get the same error only when there are no *.c files in the $(SRC)/hfa directory.
Otherwise, I see the gcc command getting executed.
And from your question, it is not quite clear what your requirement is.
The rule configuration that worked for me:
bar-%-foo:
#touch $#
.SECONDEXPANSION:
foo-%-bar: bar-$$*-foo
#echo "#: $#"
#echo "<: $<"
#touch $#
foo-bar: foo-biz-bar foo-baz-bar
.PHONY: foo-bar
And demo:
$:make foo-bar
#: foo-biz-bar
<: bar-biz-foo
#: foo-baz-bar
<: bar-baz-foo
rm bar-biz-foo bar-baz-foo
$:ls | grep foo
foo-baz-bar
foo-biz-bar