My question is to understand better what i missed in make process and .SECONDARY purpose vs .PRECIOUS, not to get my script working, since it does work already.
I am using make to either open a emacs editor on a file ( java but irrelevant for purpose of this question ) or to create it with a template if not existing.
If it works well with existing files, when using generated file it is removed at the end.
I added prerequisite in .SECONDARY but didn't help, i had to add it in .PRECIOUS.
This is question why wasn't it working in .SECONDARY ? .
From what i found on SO
.SECONDARY does not work with patterns ( % ), but even with knowing that i wonder if it is by design or if it is a bug in make. ( .SECONDARY for a pattern rule with GNU Make and Makefile pattern rule either ignores phony rule or spontaneously deletes output file )
Here a stripped down content of my Makefile to reproduce my problem ( please create a com/stackoverflow/question directory to test it ).
PACKAGE=com.stackoverflow.question
PACKAGE_DIR=$(subst .,/,$(PACKAGE))
OUT=out
clean:
find $(OUT) -name "*.class" -type f -print0|xargs -0 rm
# does not work : deleted at end due to intermediate file removal.
$(PACKAGE_DIR)/%.java:
#echo "package com.stackoverflow.question;\npublic class $(subst .java,,$(subst $(PACKAGE_DIR)/,,$#))\n{\n /** TODO */ \n}" >$#
work/%: $(PACKAGE_DIR)/$(subst work/,,%).java
emacs $<
.PHONY: clean work/%
# tried to avoid intermediate file removal : does not work
.SECONDARY: $(PACKAGE_DIR)/%.java
# if not commented this does work : once precious intermediate file is not removed.
#.PRECIOUS: $(PACKAGE_DIR)/%.java
try
make work/SoTest
I understand this is flagged intermediate.
then looking in SO i tried to set it in .SECONDARY: target list : does not work either.
looking at make source code i spotted that make intermediate files removal is done within this context :
if (f->intermediate && (f->dontcare || !f->precious)
&& !f->secondary && !f->cmd_target)
so i set my file in .PRECIOUS: and now it works.
it displays to console :
com/stackoverflow/question/SoTest.java
it run emacs with right template in it so creation is OK
here i exit emacs
and it removes the file at the end
rm com/stackoverflow/question/SoTest.java
Removal at end is due to intermediate file, this can be seen with -d option on make
LANG=C make -d work/SoTest
...
Must remake target 'work/SoTest'.
emacs com/stackoverflow/question/SoTest.java
Putting child 0xc3b580 (work/SoTest) PID 20681 on the chain.
Live child 0xc3b580 (work/SoTest) PID 20681
Reaping winning child 0xc3b580 PID 20681
Removing child 0xc3b580 PID 20681 from chain.
Successfully remade target file 'work/SoTest'.
Removing intermediate files...
rm com/stackoverflow/question/SoTest.java
To have it working i need to uncomment the .PRECIOUS paragraph.
make --version
GNU Make 4.0
Construit pour x86_64-pc-linux-gnu
Copyright (C) 1988-2013 Free Software Foundation, Inc.
Licence GPLv3+ : GNU GPL version 3 ou ultérieure <http://gnu.org/licenses/gpl.html>
Ceci est un logiciel libre : vous êtes autorisé à le modifier et à la redistribuer.
Il ne comporte AUCUNE GARANTIE, dans la mesure de ce que permet la loi.
Thanks to Alex (see answer) i went further in my search.
What i found is that it is recorded in TODO.private of make project for 15 years ....
Using git://git.savannah.gnu.org/make.git you can see history of TODO.private content :
6) Right now the .PRECIOUS, .INTERMEDIATE, and .SECONDARY
pseudo-targets have different capabilities. For example, .PRECIOUS
can take a "%", the others can't. Etc. These should all work the
same, insofar as that makes sense.
These should all work the same, insofar as that makes sense. but was not coded.
The answer to "Why .SECONDARY does not work with patterns (%) while .PRECIOUS does?" is here: the document says
You can also list the target pattern of an implicit rule (such as ‘%.o’) as a prerequisite file of the special target .PRECIOUS
but does not say this about .SECONDARY. But for the few explicit exceptions, none of the special targets accept patterns.
Related
According to the gnu make documentation, if a rule generates multiple targets by a single invocation (for instance with a recipe executing a tool with multiple output files), you can use the '&:' rule syntax to tell make.
I'm getting warnings when using this syntax, about target '&', however, when having multiple (but unique) targets in multiple rules. As if make mistakes the ampersand for a target name instead of being part of the target-prerequisite separator.
In my original project I've got two rules having multiple targets and a recipe generating those targets from a single statement/tool. The targets are unique for each of the two rules. I've created the following simple example to demonstrate the warning generated by make:
all: file_abbc
.PHONY: all clean
clean:
del /Q file_*
file_abbc: file_ab file_bc
copy file_ab+file_bc file_abbc
file_ab file_bc &: file_a file_b file_c
copy file_a+file_b file_ab
copy file_b+file_c file_bc
file_a file_b file_c &: content
copy content file_a
copy content file_b
copy content file_c
Warnings from running make on Windows on the above:
Makefile:17: warning: overriding recipe for target '&'
Makefile:13: warning: ignoring old recipe for target '&'
Why is make complaining about target '&' ?
You're using an old version of GNU make that doesn't understand &: rules, so just treats & as a file name. Use make --version to see which version you are running and upgrade if it is older that 4.3
If you read the Catalog of Rules section of the GNU Make manual, it turns out that one of the suffixes that it recognizes is .out and there is a built-in rule:
%.out: %
# commands to execute (built-in):
#rm -f $#
cp $< $#
This means that if you have a file xyz in a directory, you can create xyz.out by simply typing make xyz.out.
My question is (two variants of the same question):
Who benefits from this rule?
In what circumstances is this used by people?
Obviously, I'm asking because I managed to run foul of the rule. I had some rules like:
test.01: ${PROGRAM} ${DRIVER} test.01.tst test.01.out ${DATA.01}
${DRIVER} ${D_FLAGS} $#
where the name test.01 is a phony target, but one of the dependencies is test.01.out. When actively run (not using make -n; that works fine), this gives me lots of errors like:
make[1]: Circular test.01 <- test.01.out dependency dropped.
I also tried dropping the .out suffix with:
.SUFFIXES:
.SUFFIXES: .sh
and that didn't seem to neuter the .out rule like I expected. Is that an expected feature of GNU Make?
I guess I'm going to have to work around this bug feature of GNU Make by changing my suffix to .req or something similar, but it is a nuisance and I'm left puzzled about why the .out rule is part of the standard GNU Make rule set.
I don't know the answer to your questions about the use of this rule. All I can say is that this rule already existed when GNU make was first checked into source control, in Jan 1992. It's not mentioned in any ChangeLog so probably it dates back to the very earliest versions.
The actual rule is defined as a pattern rule, so changing .SUFFIXES won't help. To get rid of it you can use:
%.out : %
(no recipe) which will delete the pattern rule.
The makefile wildcard system doesn't seem to match targets if they contain the equal sign. Is there a way to work around this deficiency? Some flag or setting or rule to escape the equal sign? I know I can just not use the equal sign but I'd prefer to fix this idiosyncrasy of make if possible.
Here's an example of what I mean
$ cat Makefile
all:
echo Dummy target
b_%:
echo $#
$ make b_c=1
echo Dummy Target
$ make b_c1
echo b_c1
The first make command does not match b_% even though it should. I also wasn't able to find documentation for exactly what is supposed to be matched by the % wildcard. Any pointers? My make version is
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This program built for i386-apple-darwin10.0
The problem here is not with the % syntax, but with the fact that any command-line argument with an equals sign in it is interpreted as a variable assignment.
You should find that if you add the dependency all: b_c=1, then make all will generate the file just fine.
There are restrictions on what file names you can use with make -- they can't contain spaces or newlines, and e.g. backslashes are problematic, too (though not completely impossible to accommodate for simple use cases).
If you absolutely have to have a file named like this, my suggested workaround would be to use a different name internally, and then symlink it to the external name as the last step of the make recipe.
It looks to me like Makefile rules can be roughly classified into "positive" and "negative" ones: "positive" rules create missing or update outdated files, while "negative" ones remove files.
Writing prerequisites for "positive" rules is quite easy: if the target and the prerequisite are file names, make by default runs the recipe if the target is missing or outdated (a missing file in this context may be viewed as an infinitely old file).
However, consider a "negative" rule, for example for target clean. A usual way to write it seems to be something like this:
clean:
rm -f *.log *.synctex.gz *.aux *.out *.toc
This is clearly not the best way to do:
rm is executed even when there is nothing to do,
its error messages and exit status need to be suppressed with -f flag, which has other (possibly undesirable) effects, and
the fact that there were nothing to do for target clean is not reported to the user, unlike what is normal for "positive" targets.
My question is: how to write a Makefile rule that shall be processed by make only if certain files are present? (Like what would be useful for make clean.)
how to write a Makefile rule that shall be processed by make only if certain files are present? (Like what would be useful for make clean.)
You can do it like so:
filenames := a b c
files := $(strip $(foreach f,$(filenames),$(wildcard $(f))))
all: $(filenames)
$(filenames):
touch $#
clean:
ifneq ($(files),)
rm -f $(files)
endif
Example session:
$ make
touch a
touch b
touch c
$ make clean
rm -f a b c
$ make clean
make: Nothing to be done for 'clean'.
Useful perhaps for some purposes, but it strikes me as a strained refinement for make clean.
This can be easily remedied:
clean:
for file in *.log *.synctex.gz *.aux *.out *.toc; do \
if [ -e "$file" ]; then \
rm "$$file" || exit 1; \
else \
printf 'No such file: %s\n' "$file" \
fi \
done
The if statement is necessary unless your shell supports and has enabled nullglob or something similar.
If your printf supports %q you should use that instead of %s to avoid possible corruptions of your terminal when printing weird filenames.
A meta-answer is: are you sure you want to do this?
The other answers suggest to me that the cure is worse than the disease, since one involves an extension to POSIX make (ifneq), and the other uses a compound command which spreads over seven lines. Both of these are sometimes necessary expedients – I'm not criticising either answer – but both are things I avoid in a Makefile if I can. If I found myself wanting to do this in a clean rule, perhaps for the reason you mention in your comment to #MikeKinghans' answer, I'd try quite hard to change the rest of the Makefile to avoid needing this.
Reflecting on your three original points in turn:
rm is executed even when there is nothing to do: so what? The alternatives still need to, for example, expand the *.log *.synctex.gz ... so there's only miniscule efficiency gain to avoiding the rm. Make is a high-level tool which generally does not concern itself with efficiency.
its error messages and exit status need to be suppressed with -f flag: the -f flag doesn't generally suppress errors and the exit status, it merely indicates to rm that a non-existing or non-permissioned file is not to be regarded as an error.
the fact that there were nothing to do for target clean is not reported to the user: should the user really care?
The last point is the most interesting. People asking about make, on Stackoverflow and elsewhere, sometimes make things hard for themselves by trying to use it as a procedural language – make is not Python, or Fortran. Instead, it's a goal programming language (if we want to get fancy about it): you write snippets of rules to achieve sub-goals, so that the user (you, later) doesn't have to care about the details or the directory's current state, but can simply indicate a goal, and the program does whatever's necessary to get there. So whether there's is or isn't anything to do, the user ‘shouldn't’ care.
I think the short version of this answer is: it's idiomatic to keep make rules as simple (and thus as readable and robust) as possible, even at the expense of a little crudity or repetition.
Assume I have a make rule:
.PHONY:gen
gen: auto.template
generate-sources auto.template
that creates a bunch of files, for example auto1.src, auto2.src, auto3.src and so on.
If I now have rules to build targets from *.src files, like this:
$(patsubst %.src,%.target,$(wildcard *.src)): %.target: %.src
build $< > $#
How can I tell make to first execute the gen rule and then expand the preconditions for the second rule template? GNU extensions are welcome.
Note: I would like to keep it in one make invocation; A trivial solution to this would be to put the second rule in a secondary Makefile.secondrun and call $(MAKE) -f Makefile.secondrun after gen was processed. But I was wondering if there is a better option.
Building off Beta's answer, here's how you can do it using makefile remaking in GNU make, which is not the same thing as recursive make. Rather, it updates an included makefile using a rule in the main makefile, then restarts the original make instance. This is how *.d dependency files are typically generated and used.
# Get the list of auto-generated sources. If this file doesn't exist, or if it is older
# than auto.template, it will get built using the rule defined below, according to the
# standard behavior of GNU make. If autosrcs.mk is rebuilt, GNU make will automatically
# restart itself after autosrcs.mk is updated.
include autosrcs.mk
# Once we have the list of auto-generated sources, getting the list of targets to build
# from them is a simple pattern substitution.
TARGETS=$(patsubst %.src,%.target,$(AUTO_SRCS))
all: $(TARGETS)
# Rule describing how to build autosrcs.mk. This generates the sources, then computes
# the list of autogenerated sources and writes that to autosrcs.mk in the form of a
# make variable. Note that we use *shell* constructs to get the list of sources, not
# make constructs like $(wildcard), which could be expanded at the wrong time relative
# to when the source files are actually created.
autosrcs.mk: auto.template
./generate-sources auto.template
echo "AUTO_SRCS=`echo *.src`" > autosrcs.mk
# How to build *.target files from *.src files.
%.target: %.src
#echo 'build $< > $#'
Short answer: you can't. Make determines all of the rules it will have to execute before it executes any rule.
Longer answer: maybe you can. As you say, you can use recursive Make explicitly, or surreptitiously by, say, building a file which your makefile will include (I'm looking at you, Jack Kelly). Or if you could somehow obtain a list of the files which gen will build, you could write a rule around that. Or you could take a leap of faith like this:
%.target: %.src
build $< > $#
%.src: gen;