I have a pattern rule that converts a type definition in a typescript file to a JSON schema file. The program that does this conversion requires two parameters:
The name of the source file
The typename to be extracted from that file
I have decided to encode the required typename as the name of the target file.
<--------- Typename ---------> <------- source file ------->
BlockLanguageGeneratorDocument.json : block-language.description.ts
And I defined this pattern rule to do the conversion:
%.json : %.ts
# $^ is the name of the input file
# $(notdir $(basename $#)) is the filename of the target file (without the .json suffix)
$(TYPESCRIPT_JSON_SCHEMA_BIN) --path $^ --type $(notdir $(basename $#)) > "$#.json"
Sadly the <typename>.json: <sourcefile> rule I setup as a dependency is a more specific rule compared to the pattern rule and therefore the pattern rule is never executed. So I decided to wrap the conversion in a define CONVERT_COMMAND and simply use this in every single of the above definitions:
BlockLanguageGeneratorDocument.json : block-language.description.ts
$(CONVERT_COMMAND)
While this does work, the repetition strikes me as ugly. Is there a way to declare a dependency from one file to another while still preferring the pattern rule?
Minimal repro: Run this with make BlockLanguageGeneratorDocument.json Unrelated.json and observe that the echo is never executed.
block-language.description.ts :
touch $#
another.description.ts :
touch $#
%.json : %.ts
echo "generic target"
BlockLanguageGeneratorDocument.json : block-language.description.ts
Unrelated.json : another.description.ts
If this helps: The debug output is as follows.
❯❯❯ make --debug=verbose BlockLanguageGeneratorDocument.json
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Reading makefile 'Makefile'...
Reading makefile 'Makefile.json' (search path) (no ~ expansion)...
Reading makefile '../../Makefile.common' (search path) (no ~ expansion)...
Updating makefiles....
Updating goal targets....
Considering target file 'BlockLanguageGeneratorDocument.json'.
Considering target file 'block-language.description.ts'.
Finished prerequisites of target file 'block-language.description.ts'.
No need to remake target 'block-language.description.ts'.
Finished prerequisites of target file 'BlockLanguageGeneratorDocument.json'.
Prerequisite 'block-language.description.ts' is newer than target 'BlockLanguageGeneratorDocument.json'.
Must remake target 'BlockLanguageGeneratorDocument.json'.
Successfully remade target file 'BlockLanguageGeneratorDocument.json'.
make: 'BlockLanguageGeneratorDocument.json' is up to date.
It's difficult to answer this without seeing a minimal complete example, but I think I see the problem. This pattern rule:
%.json : %.ts
...whatever...
uses foo.ts to build foo.json. Make will not attempt to use it to build BlockLanguageGeneratorDocument.json, because there is no BlockLanguageGeneratorDocument.ts.
I think what you're looking for is this:
%.json :
#echo do various things with $# and $^
BlockLanguageGeneratorDocument.json : block-language.description.ts
Note that the second rule has no recipe, it simply provides the prerequisite.
Assuming the mapping from dependency to target name is well defined you could script that part, send the output into a makefile -- say deps.mk -- and include that in your main makefile.
It appears the name mapping is something like...
<text-with-dashes>.description.ts --> <Capitalized text without dashes>.GeneratorDocument.json
The following perl script reads each .ts file name from its arg list and generates a suitable dependency...
#!/usr/bin/perl -w
# -*- perl -*-
use strict;
while (defined(my $file = shift(#ARGV))) {
my($d) = ($file =~ m,^(.*)\.description\.ts,);
my #d = split("-",$d);
s/(.)/\u\L$1/ for #d;
$d = join("",#d);
print "\nall: $d"."GeneratorDocument.json\n$d"."GeneratorDocument.json: $file\n\t\$(CONVERT_COMMAND)\n";
}
If we name the above script mapname.pl then...
./mapname.pl block-language.description.ts
should give the output...
all: BlockLanguageGeneratorDocument.json
BlockLanguageGeneratorDocument.json: block-language.description.ts
$(CONVERT_COMMAND)
Now your main makefile would be...
ALL_TS_FILES := $(wildcard *.description.ts)
MAPNAME := ./mapname.pl
all:
include deps.mk
deps.mk: $(ALL_TS_FILES) $(MAPNAME)
$(MAPNAME) $(ALL_TS_FILES) > $#.tmp
diff -q $# $#.tmp || mv $#.tmp $#
Related
Using make for generic purposes (not compiling)
Suppose I have a set of files with full path names, and I would like to do something with that file.
something --file a/b/c/X > $< # (for example)
And I have made a rule:
something-%:
something --file $*
Which matches fine against, say, "something-foo" but does not catch "something-a/b/c/foo". Is there a way to write a wildcard rule for this latter case?
The manual describes how patterns match:
When the target pattern does not contain a slash (and it usually does not), directory names in the file names are removed from the file name before it is compared with the target prefix and suffix.
In your case when calling as make something-a/b/c/foo, something-a/b/c/ is treated as directory and removed, so the rest does not match your rule. This can be easily checked:
$ cat Makefile
something-%:
echo something --file $<
f%o:
echo $*
Output:
$ make something-OtherDirectory/src/foo -dr
GNU Make 4.2.1
...
Considering target file 'something-OtherDirectory/src/foo'.
File 'something-OtherDirectory/src/foo' does not exist.
Looking for an implicit rule for 'something-OtherDirectory/src/foo'.
Trying pattern rule with stem 'o'.
Found an implicit rule for 'something-OtherDirectory/src/foo'.
Finished prerequisites of target file 'something-OtherDirectory/src/foo'.
Must remake target 'something-OtherDirectory/src/foo'.
echo something-OtherDirectory/src/o
...
Note that it matched the other pattern rule with the stem of o.
You can make it work your way if your pattern does include a slash. For sake of completeness I would also define a prerequisite if your rule is based on a file and declare target as phony if it does not generate a real output file:
$ cat Makefile
.PHONY: something/%
something/%: %
echo something --file $<
Output:
$ make something/OtherDirectory/src/foo.c
echo something --file OtherDirectory/src/foo.c
something --file OtherDirectory/src/foo.c
When I execute Make, I'd like to know what shell commands / recipes are executed (and perhaps which line in which Makefile invoked these). Is there a way of doing this (print onto the stdout or write to a file) without modifying the Makefiles?
The commands will be echoed to stdout by default, unless you disabled that by prefixing them with #.
A few options you can pass to make:
-n: echos commands that would be run, but without running them. This will also echo commands that are prefixed with #.
-d: print debug output. This will output the process that make goes through in attempting to find rules that match the targets that it's trying to make. There will be a lot of useless information in here, as it will try many default build rules (like how to build .o files from .c).
There are a few ways to cut down on the noise:
-r: disable implicit rules. If you don't depend on any of the default rules , this can be used in conjunction with -d to just print the parts you (mostly) care about
--debug=b: basic debug mode. prints what it's trying to make, but not any information about implicit rules.
None of these print the line numbers of the commands, but make -n --debug=b will print both the targets being built and the commands being run, so it's almost as good. Example below.
$ cat makefile:
c: a b
cat $^ > $#
a:
echo 'foo' > $#
b: a
cat $^ > $#
echo 'bar' >> $#
$ make -n --debug=b:
Reading makefiles...
Updating goal targets....
File 'a' does not exist.
Must remake target 'a'.
echo 'foo' > a
Successfully remade target file 'a'.
File 'b' does not exist.
Must remake target 'b'.
cat a > b
echo 'bar' >> b
Successfully remade target file 'b'.
Prerequisite 'a' is newer than target 'c'.
Prerequisite 'b' is newer than target 'c'.
Must remake target 'c'.
cat a b > c
Successfully remade target file 'c'.
I have a bunch of different source files in my static HTML blog. The outermost extensions explain the format to be processed next.
Example: Source file article.html.md.gz (with target article.html) should be processed by gunzip, then by my markdown processor.
Further details:
The order of the extensions may vary
Sometimes an extension is not used (article.html.gz)
I know how to process all different extensions
I know that the final form is always article.html
Ideally I would have liked to just write rules as follows:
...
all-articles: $(ALL_HTML_FILES)
%: %.gz
gunzip ...
%: %.md
markdown ...
%: %.zip
unzip ...
And let make figure out the path to take based on the sequence of extensions.
From the documentation however, I understand that there are constraints on match-all rules, and the above is not possible.
What's the best way forward? Can make handle this situation at all?
Extensions are made up examples. My actual source files make more sense :-)
I'm on holiday so I'll bite.
I'm not a fan of pattern rules, they are too restricted and yet too arbitrary at the same time for my tastes. You can achieve what you want quite nicely in pure make:
.DELETE_ON_ERROR:
all: # Default target
files := a.html.md.gz b.html.gz
cmds<.gz> = gzip -d <$< >$#
cmds<.md> = mdtool $< -o $#
define rule-text # 1:suffix 2:basename
$(if $(filter undefined,$(flavor cmds<$1>)),$(error Cannot handle $1 files: [$2$1]))
$2: $2$1 ; $(value cmds<$1>)
all: $2
endef
emit-rule = $(eval $(call rule-text,$1,$2))# 1:suffix 2:basename
emit-hierachy = $(if $(suffix $2),$(call emit-rule,$1,$2)$(call emit-hierachy,$(suffix $2),$(basename $2)))# 1:suffix 2:basename
emit-rules = $(foreach _,$1,$(call emit-hierachy,$(suffix $_),$(basename $_)))# 1:list of source files
$(call emit-rules,${files})
.PHONY: all
all: ; : $# Success
The key here is to set $files to your list of files.
This list is then passed to emit-rules.
emit-rules passes each file one-at-a-time to emit-hierachy.
emit-hierachy strips off each extension in turn,
generates the appropriate make syntax, which it passes to $(eval …).
emit-hierachy carries on until the file has only one extension left.
Thus a.html.md.gz becomes this make syntax:
a.html.md: a.html.md.gz ; gunzip <$< >$#
a.html: a.html.md ; mdtool $< -o $#
all: a.html
Similarly, b.html.gz becomes:
b.html: b.html.gz ; gunzip <$< >$#
all: b.html
Neato, or what?
If you give emit-rules a file with an unrecognised extension (c.html.pp say),
you get a compile-time error:
1:20: *** Cannot handle .pp files: [c.html.pp]. Stop.
Compile-time? Yeah, before any shell commands are run.
You can tell make how to handle .pp files by defining cmds<.pp> :-)
For extra points it's also parallel safe. So you can use -j9 on your 8 CPU laptop, and -j33 on your 32 CPU workstation. Modern life eh?
As per the gnu make documentation, a pattern rule's "...recipe is executed only once to make all the targets." However, I have the following Makefile
.PHONY: entrypoint
entrypoint: test_1.cpp test_2.cpp
test_%.cpp:
echo $#
And running make produces:
echo test_1.cpp
test_1.cpp
echo test_2.cpp
test_2.cpp
I'm new to make, and I'm probably misunderstanding something, but the documentation seems misleading if clear.
$ make -v
GNU Make 4.0
...
You're misreading the documentation. It means, the recipe is run only one time assuming that all the target patterns in that rule will be created.
Since you have only one target pattern in your rule (test_%.cpp`) make knows that each time it runs that recipe it will create one output file matching that pattern. To create different targets that match that pattern it will run multiple instances of the recipe.
If you had a rule like this:
%.x %.y %.z :
dothings
then make would expect that a single invocation of the recipe dothings would create all the targets matching this pattern (e.g., foo.x, foo.y, and foo.z).
Contrast this with an explicit rule like this:
foo.x foo.y foo.z :
dothings
Make here treats this exactly as if you'd written this:
foo.x :
dothings
foo.y :
dothings
foo.z :
dothings
That is, to build all three of these targets it would run the recipe three times.
There's no way to tell make "please run this recipe one time and it will produce every single target that could possibly match the pattern foo_%.cpp".
The following functions construct a dynamic list of dependencies of your multiple-target where the non-existent files are named last. This is more or less the method named "Another attempt" in the link you gave, except that it doesn't trip over missing files and is able to make a missing file by giving it as target on the command line. What it does not: execute the multitarget recipe if one of the multitargets is out of date relative to the others, but I think this is more of wanted side effect than a problem. The only drawback is the syntactic ugliness as you have to write it into an eval expression which forces you to quote all variables in the recipe which shall be evaluated at execution time.
define newline :=
endef
list2rules = $(firstword $1) $(if $(word 2,$1),: $(word 2,$1)$(newline)$(call list2rules,$(wordlist 2,1000,$1)))
multitarget = $(call list2rules,$(wildcard $1) $(filter-out $(wildcard $1),$1))
.PHONY: all
targets = test1 test2 footest3
#$(info $(call multitarget,$(targets)))
all: somefile
somefile: $(targets)
touch somefile
# here we generate the dependency list on the spot. Only one recipe to update all targets.
$(eval $(call multitarget,\
$(targets)) : ; \
touch $(targets) \
)
I'm using Gnu Make 3.81, and getting an error trying to match a pattern rule that also has a variable in it.
Here's the smallest example I could come up with:
YYMMDD:=$(shell date +%y%m%d)
TMP_DIR:=/tmp/$(YYMMDD)
# create a temporary directory, and put a "source" file in it
$(TMP_DIR):
mkdir $(TMP_DIR)
echo something > $(TMP_DIR)/somefile.orig
# to build an "object" file in the temp dir, process the "source" file
$(TMP_DIR)/%.new: $(TMP_DIR)/%.orig
wc -l $< > $#
atarget: $(TMP_DIR) $(TMP_DIR)/somefile.new
Then when I run make atarget, I get:
mkdir /tmp/141021
echo something > /tmp/141021/somefile.orig
make: *** No rule to make target `/tmp/141021/somefile.new', needed by `atarget'. Stop.
Shouldn't this work? it seems like the pattern rule should match this just fine.
It's because make doesn't know that the .orig file exists: you have a rule that builds $(TMP_DIR) but make doesn't know that this rule also builds $(TMP_DIR)/somefile.orig. So when make is trying to match the pattern rule it will see that the .orig file doesn't exist and it doesn't have any way that it knows how to make that file, so the pattern doesn't match, and after that there's no way to build the .new file.
You should write:
$(TMP_DIR)/%.orig:
mkdir -p $(TMP_DIR)
echo $* > $#
then it will work.