Makefile pattern rules cannot expand into directory with wildcards - makefile

Question

I am trying to use make to build a set of markdown documents from specific and shared files.
PKGS := foo bar baz app
DEFAULT := DEPENDENCIES.md RULES.md
BUILD := .build
VPATH := .:packages
$(BUILD)/%.history.md: $(DEFAULT) $(wildcard %/RULES.*.md) $(foreach f,$(DEFAULT),%/$(f))
echo "!! BUILDING NOW $#"
echo $^
echo
with the following directory structure:
.
├── ABSTRACT.md
├── DEPENDENCIES.md
├── IMPLEMENTATION.md
├── Makefile
├── packages
│   ├── foo
│   │   ├── ABSTRACT.md
│   │   ├── DEPENDENCIES.md
│   │   ├── IMPLEMENTATION.md
│   │   ├── RULES.0.md -> ../COMMON.md
│   │   └── RULES.md
│   ├── COMMON.md
│   ├── bar
│   │   ├── ABSTRACT.md
│   │   ├── DEPENDENCIES.md
│   │   ├── IMPLEMENTATION.md
│   │   └── RULES.md
│   ├── baz
│   │   ├── ABSTRACT.md
│   │   ├── DEPENDENCIES.md
│   │   ├── IMPLEMENTATION.md
│   │   └── RULES.md
│   └── app
│   ├── ABSTRACT.md
│   ├── DEPENDENCIES.md
│   ├── IMPLEMENTATION.md
│   ├── RULES.0.md -> ../COMMON.md
│   └── RULES.md
└── RULES.md
However, $(wildcard %/RULES.*.md) always seems to expand into an empty list.
When executing make .build/foo.history.md I expected it to evaluate the prerequisites as DEPENDENCIES.md RULES.md packages/foo/RULES.0.md packages/foo/DEPENDENCIES.md packages/foo/RULES.md but instead I see this:
make .build/foo.history.md
echo "!! BUILDING NOW .build/foo.history.md"
!! BUILDING NOW .build/foo.history.md
echo DEPENDENCIES.md RULES.md packages/foo/DEPENDENCIES.md packages/foo/RULES.md
DEPENDENCIES.md RULES.md packages/foo/DEPENDENCIES.md packages/foo/RULES.md
echo
I've also tried adding some extra targets to better understand the process:
# Pattern rule with `%` as a directory clearly works.
%.static: %/RULES.0.md
echo $# -> $^
%.semi: %/RULES.*.md
echo $# -> $^
%.x: $(wildcard packages/%/RULES.*.md)
echo $# -> $^
# Wildcards using * also clear work.
app.y: $(wildcard packages/app/RULES.*.md)
echo $# -> $^
# Combination of both fails.
# I expect that if the target is `foo.z` for the prerequisites to expand into `foo/RULES.*.md`,
# searching within the VPATH.
%.z: $(wildcard %/RULES.*.md)
echo $# -> $^
Which results in this:
$ make app.static --just-print
echo app.static -> packages/app/RULES.0.md
$ make app.semi --just-print
make: *** No rule to make target 'app.semi'. Stop.
$ make app.x --just-print
echo app.x ->
$ make app.y --just-print
echo app.y -> packages/app/RULES.0.md
$ make app.z --just-print
echo app.z ->
I've also tried using .SECONDEXPANSION, but I couldn't get it to work with that either and I keep seeing other people recommending against it.
Additionally, it does work for $(foreach f,$(DEFAULT),%/$(f)) expanding correctly into packages/app/DEPENDENCIES.md packages/app/RULES.md which adds confusion as to why it doesn't work in $(wildcard).

Answer 1

The high-level problem is that you're trying to use too many unfamiliar tools at once. Let's take VPATH out of the picture for now.
The wildcard function will work, with secondary expansion:
.SECONDEXPANSION:
$(BUILD)/%.history.md: $$(wildcard packages/%/RULES.*.md)
echo "!! BUILDING NOW $#"
echo $^
echo
Notice that we must escape the $ in the prerequisite list with another $. I don't know why people advised against secondary expansion, but I see no easier way to get the effect you want.
Now about that VPATH. There isn't much need for it, since you want to search only one directory (packages/). Also, it doesn't work well with wildcard because VPATH tells Make where to look for prerequisites; it doesn't tell wildcard where to look for matches. And Make doesn't know what prerequisite to look for until after wildcard has done its work, which it can't because it doesn't know where to look. I advise you to do without it.