Consider the following makefile:
.SECONDEXPANSION:
%.result: jobs/% $$(call get-job,$$<)
echo $^
define get-job
$(shell head -n 1 $(1))
$(shell tail -n +2 $(1))
endef
The idea is that each file under jobs/ contains a list of filenames, which should be appended to the prerequisite list.
However, if I want to create xyz.result from an existing file jobs/xyz, I get the following error message:
$ make -n xyz.result
head: cannot open 'xyz.result' for reading: No such file or directory
tail: cannot open 'xyz.result' for reading: No such file or directory
head: cannot open 'xyz.result' for reading: No such file or directory
tail: cannot open 'xyz.result' for reading: No such file or directory
make: *** No rule to make target 'xyz.result'. Stop.
I am aware that $$< isn't set to what I want, as it reflects the prerequisite list of any previous rule.
What I don't understand is the following:
In my understanding, $$< should evaluate to the empty string (as shown in the example in the official doc, under the second sub-heading). However, it seems to be expanded to the value of the target here (xyz.result). Why is that?
It seems that the get-job function is called twice (head and tail both bark twice). I understand that the prerequisite list is expanded twice. But in the first run, the call is still escaped, so this isn't what I expect.
(Maybe the whole approach is flawed, and I shouldn't be (ab)using Makefile for this kind of task in the first place.)
I don't know why $$< expands that way.
You can make your configuration work by using $$* instead, though:
%.result: jobs/% $$(strip $$(call get-job,jobs/$$*))
echo $^
You need to add the strip call so that the newlines embedded in the results of the call will be turned into spaces.
Related
In a Makefile I'm writing I had an interest in cleaning up some of the CC prints and centralizing some of the build preparations (like creating directories in the build tree). I figured macros would be a good fit for this task. This is effectively what im trying to do, used all over various Makefiles:
define func
#mkdir -p $$(dir $(1))
#printf "%-5s $(2)\n" $(3)
endef
test:
#echo Run
$(eval $(call func,a,b,c))
My thought was that after first expansion I'd get something like (less any tabs maybe, I'm not exactly sure how the expansion works within eval):
test:
#echo Run
$(eval #mkdir -p $(dir a)\n#printf "%-5s b\n" c
and of course finally the commands would be executed. However, what I get is this:
# make
Makefile:7: *** recipe commences before first target. Stop.
I changed eval to info and got this:
#mkdir -p $(dir a)
#printf "%-5s b\n" c
Run
So I thought maybe my explicit tabs in the macro definition were causing trouble, so I removed them and tried again:
# make
Makefile:7: *** missing separator. Stop.
So it still does not quite work. If it is indeed possible at all, it seems some function of indentions in the macro, or maybe I'm defining the macros incorrectly. I thought perhaps the two commands in the macro was causing trouble (since the complaint is regarding a separator), but reducing the macro to a single line did not help either.
You don't want eval here. Eval is used to evaluate makefile syntax. That is, the thing you're evaluation has to be a valid, complete makefile. You can see that what info prints is not a valid makefile. If you put that into a file and ran make -f <file>, you'd get a syntax error.
You are just trying to expand a variable for shell syntax. Just remove the eval.
Using make for generic purposes (not compiling)
Suppose I have a set of files with full path names, and I would like to do something with that file.
something --file a/b/c/X > $< # (for example)
And I have made a rule:
something-%:
something --file $*
Which matches fine against, say, "something-foo" but does not catch "something-a/b/c/foo". Is there a way to write a wildcard rule for this latter case?
The manual describes how patterns match:
When the target pattern does not contain a slash (and it usually does not), directory names in the file names are removed from the file name before it is compared with the target prefix and suffix.
In your case when calling as make something-a/b/c/foo, something-a/b/c/ is treated as directory and removed, so the rest does not match your rule. This can be easily checked:
$ cat Makefile
something-%:
echo something --file $<
f%o:
echo $*
Output:
$ make something-OtherDirectory/src/foo -dr
GNU Make 4.2.1
...
Considering target file 'something-OtherDirectory/src/foo'.
File 'something-OtherDirectory/src/foo' does not exist.
Looking for an implicit rule for 'something-OtherDirectory/src/foo'.
Trying pattern rule with stem 'o'.
Found an implicit rule for 'something-OtherDirectory/src/foo'.
Finished prerequisites of target file 'something-OtherDirectory/src/foo'.
Must remake target 'something-OtherDirectory/src/foo'.
echo something-OtherDirectory/src/o
...
Note that it matched the other pattern rule with the stem of o.
You can make it work your way if your pattern does include a slash. For sake of completeness I would also define a prerequisite if your rule is based on a file and declare target as phony if it does not generate a real output file:
$ cat Makefile
.PHONY: something/%
something/%: %
echo something --file $<
Output:
$ make something/OtherDirectory/src/foo.c
echo something --file OtherDirectory/src/foo.c
something --file OtherDirectory/src/foo.c
I want to create a makefile to be included in others, kind of as a "library", let's call it library.mak:
TARGET_FILES = a.txt b.txt c.txt d.txt
SRC_FOLDER = somewhere/else
.PHONY: all
all:
for target in ${TARGET_FILES} ; do \
echo -e "\nMaking $${target}:" ;\
${MAKE} ${MFLAGS} --no-print-directory $${target} ;\
done
.PHONY: ${TARGET_FILES}
${TARGET_FILES}:
cp ${SRC_FOLDER}/$# $#
The two rules are there to make all, as well as one specific target, respectively; using a default rule (which is the purpose of library.mak).
In my "user makefile" called Makefile, I want to then do this:
include library.mak
# special handling of c.txt
c.txt:
grep -v 'all except this' ${SRC_FOLDER}/$# > $#
As you can see, the user wants to be able to override the behaviour for some special cases. While this works, it always greets the user with the dreaded warning: overriding recipe for target and warning: ignoring old commands for target messages, even though the behaviour is as intended.
So here's the question: Can this be done in a different way that avoids these warnings, or is there a means to suppress them?
The warning says you overwrite a recipe for an explicit rule. And this is really wrong. It makes sense to use a pattern rule instead, like: %.txt: ${SRC_FOLDER}/%.txt. Then it's ok to have some explicit rule (c.txt: ${SRC_FOLDER}/c.txt) overwriting a recipe. Of course, it's not 100% the same, but, I think that should not be a problem. Anyway, putting an explicit rule into a reusable file is a crime.
Next, you constantly use "phonies" and rules w/o prerequisites even when you definitely should have them. This is bad. You're trying to make "make" work as a non-branching shell script. That's not only inefficient but also is a misuse. Basically, make can be viewed as "shell extension" capable of "branching on file timestamps in a sophisticated way". If you don't need it, don't use make at all.
Next, I see absolutely no reason to go into recursive make. Whatever, you're going to do, I believe, you can do without it. And even if you really need recursion, then write simply $(MAKE). $(MFLAGS) shoud not be used anymore (read this).
P.S. There's yet another (more flexible) option to define parameterized variables (macros). For example:
define nl :=
endef
define myrule.text
$1: $2/$1
#cp $$< $$#
endef
myrule = $(eval $(foreach foo$0,$1,$(call myrule.text,$(foo$0),$2)$(nl)))
...
# user makefile
# for everything except c.txt call predefined macro
$(call myrule,$(filter-out c.txt,$(TARGET_FILES)),$(SRC_FOLDER))
# for c.txt only
c.txt: $(SRC_FOLDER)/c.txt
grep -v 'all except this' $< >$#
But as you see, it's a bit of a "cryptic" stuff, and I don't feel like recommending it for a beginner.
I have a bunch of different source files in my static HTML blog. The outermost extensions explain the format to be processed next.
Example: Source file article.html.md.gz (with target article.html) should be processed by gunzip, then by my markdown processor.
Further details:
The order of the extensions may vary
Sometimes an extension is not used (article.html.gz)
I know how to process all different extensions
I know that the final form is always article.html
Ideally I would have liked to just write rules as follows:
...
all-articles: $(ALL_HTML_FILES)
%: %.gz
gunzip ...
%: %.md
markdown ...
%: %.zip
unzip ...
And let make figure out the path to take based on the sequence of extensions.
From the documentation however, I understand that there are constraints on match-all rules, and the above is not possible.
What's the best way forward? Can make handle this situation at all?
Extensions are made up examples. My actual source files make more sense :-)
I'm on holiday so I'll bite.
I'm not a fan of pattern rules, they are too restricted and yet too arbitrary at the same time for my tastes. You can achieve what you want quite nicely in pure make:
.DELETE_ON_ERROR:
all: # Default target
files := a.html.md.gz b.html.gz
cmds<.gz> = gzip -d <$< >$#
cmds<.md> = mdtool $< -o $#
define rule-text # 1:suffix 2:basename
$(if $(filter undefined,$(flavor cmds<$1>)),$(error Cannot handle $1 files: [$2$1]))
$2: $2$1 ; $(value cmds<$1>)
all: $2
endef
emit-rule = $(eval $(call rule-text,$1,$2))# 1:suffix 2:basename
emit-hierachy = $(if $(suffix $2),$(call emit-rule,$1,$2)$(call emit-hierachy,$(suffix $2),$(basename $2)))# 1:suffix 2:basename
emit-rules = $(foreach _,$1,$(call emit-hierachy,$(suffix $_),$(basename $_)))# 1:list of source files
$(call emit-rules,${files})
.PHONY: all
all: ; : $# Success
The key here is to set $files to your list of files.
This list is then passed to emit-rules.
emit-rules passes each file one-at-a-time to emit-hierachy.
emit-hierachy strips off each extension in turn,
generates the appropriate make syntax, which it passes to $(eval …).
emit-hierachy carries on until the file has only one extension left.
Thus a.html.md.gz becomes this make syntax:
a.html.md: a.html.md.gz ; gunzip <$< >$#
a.html: a.html.md ; mdtool $< -o $#
all: a.html
Similarly, b.html.gz becomes:
b.html: b.html.gz ; gunzip <$< >$#
all: b.html
Neato, or what?
If you give emit-rules a file with an unrecognised extension (c.html.pp say),
you get a compile-time error:
1:20: *** Cannot handle .pp files: [c.html.pp]. Stop.
Compile-time? Yeah, before any shell commands are run.
You can tell make how to handle .pp files by defining cmds<.pp> :-)
For extra points it's also parallel safe. So you can use -j9 on your 8 CPU laptop, and -j33 on your 32 CPU workstation. Modern life eh?
I've been learning make from the software carpentry tutorial (make patterns) and it says that we can use $^ to get the dependency list for our rule but have extra dependencies for our target by writing extra empty rules. For example
all:
touch f1.txt f2.txt a.txt
result: a.txt
result: f*.txt
#echo $^
I thought that this would print f1.txt f2.txt but instead I see f1.txt f2.txt a.txt. Am I missing something or is the tutorial wrong?
Clearly, I should have been explicit, in the example I run make to create the files for the test, then I run make result to get the result shown.
Not sure I understand your problem and what you would like to do but if you type just make it is like if you were typing make all because all is the first target and thus the default goal. It should create the 3 files (or update their last modification date if they already exist), echo the recipe and you should see:
touch f1.txt f2.txt a.txt
If you then type make result you should see the result of #echo $^, that is the list of all dependencies of result:
f1.txt f2.txt a.txt
It is absolutely normal as you declared them all as dependencies of result.
Finally, if you type make result before make or make all and if a.txt does not exist or if there is no file matching f*.txt, you will get an error because make needs a.txt and at least one file matching f*.txt to make result and it does not know how to make them.
I cannot imagine a use case where the behaviour you expected ($^ being expanded as dependencies of the current rule only) would be useful, but if really you need this feature you can use the rather obscure Double-Colon Rules (DCR):
all:
touch f1.txt f2.txt a.txt
result:: a.txt
#echo $^
result:: f*.txt
#echo $^
Note that if there is a DCR for a target, all rules of this target must also be DCR. Note also that all DCR must have a recipe. A DCR recipe is applied if one of the dependencies of this rule is newer than the target. If several DCR apply, they are executed in order of appearance in the Makefile. And finally, what you were interested in: the $^ automatic variable logically expands as the dependencies of the particular DCR.
No, $^ refers to the prerequisites for the current target, not the current rule.
Also, the results you are reporting are not correct; for make result I get make: *** No rule to make target `f*.txt', needed by `result'. Stop. like I very much expected.