I often find myself trying to reference the dependency of a target (Target1) inside another one (Target2).
Consider the following Makefile. How nice would be a $(deps target-name) function !
Rule1 : file1 file2 file3
echo $^ # Careful, these are whitespaces, not a tab !
Rule2 : file4 file5 file6
echo $^ # Careful, these are whitespaces, not a tab !
clean-Rule-1-2 :
rm $(deps Rule1) $(deps Rule2) # Careful, these are whitespaces, not a tab !
I found this link mentioning that one could build himself his own dependency list, but it's looks rather tedious.
Does any one have a better solution (assuming none are natively implemented in the Makefile) and/or workflow tips referring to this issue ?
Why do you want to list prerequisites of a clean-type rule? That just forces make to build those dependencies if they're out of date, only to delete them again.
There is no way to do what you want because it is not possible to be consistent about it. For example, your rules could be written like this:
Rule1 : file1 file2 file3
echo $^ # Careful, these are whitespaces, not a tab !
clean-Rule-1-2 : $(deps Rule1)
rm $^ # Careful, these are whitespaces, not a tab !
Rule1 : file10 file11 file12
Oops! When $(deps ...) is expanded make doesn't know about the extra three prerequisites and so they won't be listed. And that's not even considering implicit rules, where make doesn't know what the full prerequisite list is when it parses the makefile; it only computes them when it's trying to build the target.
You don't give a real example of what you want to do, but generally the way makefiles are written is that you put the prerequisites into variables, then you can use the variables in multiple places.
Related
I have a makefile with a rule that produces multiple outputs. To work around the issue of this rule often being run multiple times when run in parallel, I've used a dummy "timestamp file". I also have a rule that depends on the one of the outputs of this "multi-output" rule.
When all this is run from a clean state, it all works fine. However, if the source of the multi-output rule is updated, the other rule is not run, until Make is run again.
I've looked at the debug output, but haven't been able to make much headway. It almost seems like Make might be caching the old timestamp of the previous version of the multi-output file?
Hopefully the below demonstrates the problem adequately.
$ cat Makefile
all: data.txt
multioutput.stamp: sourcefile.txt
touch multioutput1.txt
touch multioutput2.txt
touch $#
FILES=multioutput1.txt multioutput2.txt
$(FILES): multioutput.stamp
data.txt: multioutput1.txt
touch data.txt
$ touch sourcefile.txt
$ make
touch multioutput1.txt
touch multioutput2.txt
touch multioutput.stamp
touch data.txt
$ touch sourcefile.txt # update
$ make # data.txt is not updated!!
touch multioutput1.txt
touch multioutput2.txt
touch multioutput.stamp
$ make # except when it's run again??
touch data.txt
What am I doing wrong here, and what should I be doing instead?
You are lying to make. Don't do that.
Once you have run the recipe of a rule, make checks to see if a file has actually been updated by the recipe. If it has not changed, you don't have to re-make any target that lists the file as a dependency.
Here you have given no recipe for multioutput1.txt, just a dependency line:
multioutput1.txt: multioutput.stamp
Make knows there is no way to update multioutput1.txt.
Cheap fix
Force make to check the dependency by supplying an explicit recipe for multioutput1.txt.
Even an empty one will do:
${FILES}: multioutput.stamp ;
Yep, that's what the ; signifies — the first line of the recipe follows on the same line.
Better fix
The only way of saying to make "this recipe creates two files" is with a pattern rule.
Then there is no need for a multioutput.stamp.
.PHONY: all
all: data.txt
%1.txt %2.txt:
touch $*1.txt
touch $*2.txt
data.txt: multioutput1.txt multioutput2.txt
touch data.txt
Here $* in the recipe expands to whatever the % matched in the dependency line.
Why have I made data.txt depend on both multioutput files?
Here I took the view that if either of multioutput1 or multioutput2 were missing, we should probably run the recipe to create both.
YMMV.
Best Fix
YMMV but I don't like pattern rules.
They are too arbitrary for my tastes.
We observe that one of multioutput1.txt and multioutput2.txt will always be younger than the other.
They will never have the same timestamp assuming a modern filesystem.
.PHONY: all
all: data.txt
multioutput2.txt: start.stamp
touch $#
touch multioutput2.txt
multioutput1.txt: multioutput2.txt ;
data.txt: multioutput1.txt
touch data.txt
I want to create a makefile to be included in others, kind of as a "library", let's call it library.mak:
TARGET_FILES = a.txt b.txt c.txt d.txt
SRC_FOLDER = somewhere/else
.PHONY: all
all:
for target in ${TARGET_FILES} ; do \
echo -e "\nMaking $${target}:" ;\
${MAKE} ${MFLAGS} --no-print-directory $${target} ;\
done
.PHONY: ${TARGET_FILES}
${TARGET_FILES}:
cp ${SRC_FOLDER}/$# $#
The two rules are there to make all, as well as one specific target, respectively; using a default rule (which is the purpose of library.mak).
In my "user makefile" called Makefile, I want to then do this:
include library.mak
# special handling of c.txt
c.txt:
grep -v 'all except this' ${SRC_FOLDER}/$# > $#
As you can see, the user wants to be able to override the behaviour for some special cases. While this works, it always greets the user with the dreaded warning: overriding recipe for target and warning: ignoring old commands for target messages, even though the behaviour is as intended.
So here's the question: Can this be done in a different way that avoids these warnings, or is there a means to suppress them?
The warning says you overwrite a recipe for an explicit rule. And this is really wrong. It makes sense to use a pattern rule instead, like: %.txt: ${SRC_FOLDER}/%.txt. Then it's ok to have some explicit rule (c.txt: ${SRC_FOLDER}/c.txt) overwriting a recipe. Of course, it's not 100% the same, but, I think that should not be a problem. Anyway, putting an explicit rule into a reusable file is a crime.
Next, you constantly use "phonies" and rules w/o prerequisites even when you definitely should have them. This is bad. You're trying to make "make" work as a non-branching shell script. That's not only inefficient but also is a misuse. Basically, make can be viewed as "shell extension" capable of "branching on file timestamps in a sophisticated way". If you don't need it, don't use make at all.
Next, I see absolutely no reason to go into recursive make. Whatever, you're going to do, I believe, you can do without it. And even if you really need recursion, then write simply $(MAKE). $(MFLAGS) shoud not be used anymore (read this).
P.S. There's yet another (more flexible) option to define parameterized variables (macros). For example:
define nl :=
endef
define myrule.text
$1: $2/$1
#cp $$< $$#
endef
myrule = $(eval $(foreach foo$0,$1,$(call myrule.text,$(foo$0),$2)$(nl)))
...
# user makefile
# for everything except c.txt call predefined macro
$(call myrule,$(filter-out c.txt,$(TARGET_FILES)),$(SRC_FOLDER))
# for c.txt only
c.txt: $(SRC_FOLDER)/c.txt
grep -v 'all except this' $< >$#
But as you see, it's a bit of a "cryptic" stuff, and I don't feel like recommending it for a beginner.
Suppose I have a Makefile like this
B1.txt: A1.txt
python big_long_program.py A1.txt > $#
correct1.txt: B1.txt reference.txt
diff -q B1.txt reference.txt
touch $#
Then the output when I make correct1.txt is pretty well what I would expect:
python big_long_program.py A1.txt > B1.txt
diff -q B1.txt reference.txt
touch correct1.txt
Now if I have lots of files, B1.txt, B2.txt, B3.txt etc, so create an implicit rule:
B%.txt: A%.txt
python big_long_program.py A$*.txt > $#
correct%.txt: B%.txt reference.txt
diff -q B$*.txt reference.txt
touch $#
Instead this happens when I make correct1.txt:
python big_long_program.py A1.txt > B1.txt
diff -q B1.txt reference.txt
touch correct1.txt
rm B1.txt
i.e. there difference is that now the file B1.txt has been deleted, which in many cases is really bad.
So why are implicit rules different? Or am I doing something wrong?
You are not doing anything wrong. The behavior you observe and analyze is documented in 10.4 Chains of Implicit Rules. It states that intermediate files are indeed treated differently.
The second difference is that if make does create b in order to update
something else, it deletes b later on after it is no longer needed.
Therefore, an intermediate file which did not exist before make also
does not exist after make. make reports the deletion to you by
printing a rm -f command showing which file it is deleting.
The documentation does not explicitly explain why it behaves like this. Looking in the file ChangeLog.1, there is a reference to the remove_intermediates function as far back as 1988. At that time, disk space was expensive and at a premium.
If you do not want this behavior, mention the targets you want to keep somewhere in the makefile as an explicit prerequisite or target or use the .PRECIOUS or the .SECONDARY special built-in targets for that.
With thanks to MadScientist for the additional comments, see below.
I've been learning make from the software carpentry tutorial (make patterns) and it says that we can use $^ to get the dependency list for our rule but have extra dependencies for our target by writing extra empty rules. For example
all:
touch f1.txt f2.txt a.txt
result: a.txt
result: f*.txt
#echo $^
I thought that this would print f1.txt f2.txt but instead I see f1.txt f2.txt a.txt. Am I missing something or is the tutorial wrong?
Clearly, I should have been explicit, in the example I run make to create the files for the test, then I run make result to get the result shown.
Not sure I understand your problem and what you would like to do but if you type just make it is like if you were typing make all because all is the first target and thus the default goal. It should create the 3 files (or update their last modification date if they already exist), echo the recipe and you should see:
touch f1.txt f2.txt a.txt
If you then type make result you should see the result of #echo $^, that is the list of all dependencies of result:
f1.txt f2.txt a.txt
It is absolutely normal as you declared them all as dependencies of result.
Finally, if you type make result before make or make all and if a.txt does not exist or if there is no file matching f*.txt, you will get an error because make needs a.txt and at least one file matching f*.txt to make result and it does not know how to make them.
I cannot imagine a use case where the behaviour you expected ($^ being expanded as dependencies of the current rule only) would be useful, but if really you need this feature you can use the rather obscure Double-Colon Rules (DCR):
all:
touch f1.txt f2.txt a.txt
result:: a.txt
#echo $^
result:: f*.txt
#echo $^
Note that if there is a DCR for a target, all rules of this target must also be DCR. Note also that all DCR must have a recipe. A DCR recipe is applied if one of the dependencies of this rule is newer than the target. If several DCR apply, they are executed in order of appearance in the Makefile. And finally, what you were interested in: the $^ automatic variable logically expands as the dependencies of the particular DCR.
No, $^ refers to the prerequisites for the current target, not the current rule.
Also, the results you are reporting are not correct; for make result I get make: *** No rule to make target `f*.txt', needed by `result'. Stop. like I very much expected.
I thought that having multiple targets in a rule was the same as defining multiple rules with the same recipe for each of the targets; however, I seem to have a problem when the target/dependency have patterns.
Here's a simplified sample makefile with two different methods for building files.
all: $(foreach n, 1 2 3, $(foreach l, a b, out.$(l).$(n).txt))
1.txt 2.txt 3.txt:
touch $#
#Method One
out.a.%.txt out.b.%.txt: %.txt
touch $#
#Method Two
out.a.%.txt: %.txt
touch $#
out.b.%.txt: %.txt
touch $#
Using method one with a "make -n" I get
touch 1.txt
touch out.a.1.txt
touch 2.txt
touch out.a.2.txt
touch 3.txt
touch out.a.3.txt
but with method two I get the desired output
touch 1.txt
touch out.a.1.txt
touch out.b.1.txt
touch 2.txt
touch out.a.2.txt
touch out.b.2.txt
touch 3.txt
touch out.a.3.txt
touch out.b.3.txt
I didn't see any exceptions to the multiple target in a rule in the on-line docs. Did i miss a part of the documentation that explains why this doesn't work? Is there an easy way to make it work? (In reality, I have many more combinations of files to build than in this simple example).
(Using GNU Make 3.81)
Implicit (pattern) rules and explicit rules work in the opposite way from each other. For explicit rules, multiple targets define multiple separate rules, one for each target. For pattern rules, multiple targets means that a single invocation of the pattern rule recipe builds all the targets.
See the documentation on pattern rules.
Oh and BTW, no, there is no alternative to writing the pattern rule multiple times. You can use various methods to automatically generate the pattern rules, but that's about it.