#!make
.PRECIOUS: a/b/c/%_pqr
a/b/c/%_pqr:
echo $#
touch $#
.PRECIOUS: a/b/c/%_abc
a/b/c/%_abc: a/b/c/pqr_pqr
echo $#
touch $#
When I run first time, it honors its dependency:
make a/b/c/pqr_abc -f 1.mk
echo a/b/c/pqr_pqr
a/b/c/pqr_pqr
touch a/b/c/pqr_pqr
echo a/b/c/pqr_abc
a/b/c/pqr_abc
touch a/b/c/pqr_abc
But if I delete "a/b/c/pqr_pqr" file and then run this target again, it doesn't do the needful:
rm -f a/b/c/pqr_pqr; make a/b/c/pqr_abc -f 1.mk
make: `a/b/c/pqr_abc' is up to date.
As per my understanding, it should run both targets again.
Please somebody help me.
Both your targets are implicit and therefore intermediate. As per documentation:
The first difference is what happens if the intermediate file does not exist. [...] But if b is an intermediate file, then make can leave well enough alone. It won’t bother updating b, or the ultimate target, unless some prerequisite of b is newer than that target or there is some other reason to update that target.
Therefore since pqr_abc already exists and pqr_pqr is an intermediate file and has no prerequisites (so no prerequisites have changed), it will not be regenerated.
As per make's understanding pqr_pqr would need to be created only to generate pqr_abc from it (this is the definition of intermediate file), and since pqr_abc already exists, there is no need to regenerate it.
If you want the file to be regenerated every time, it has to be mentioned explicitly, so that it is no longer considered an intermediate file, i.e.:
$ cat Makefile
a/b/c/%_pqr:
touch $#
a/b/c/%_abc: a/b/c/pqr_pqr
touch $#
a/b/c/pqr_pqr:
The last line defines explicit target, but it does not define recipe, which will still be invoked from implicit rule. Note that it will no longer be considered intermediate and will not be automatically removed, so .PRECIOUS directives are also not needed.
Output:
$ make a/b/c/pqr_abc
touch a/b/c/pqr_pqr
touch a/b/c/pqr_abc
$ make a/b/c/pqr_abc
make: 'a/b/c/pqr_abc' is up to date.
$ rm a/b/c/pqr_pqr
$ make a/b/c/pqr_abc
touch a/b/c/pqr_pqr
touch a/b/c/pqr_abc
Related
I have a simple Makefile that will produce a file
all: build/foo.bin
build/foo.bin: foo.c
gcc $< -o $#
Works great and produces build/foo.bin as expected. If I then do a another make it will say make: Nothing to be done for 'all'. That's expected.
I then do rm build/foo.bin && make and it rebuilds the file. But if I do a echo "Modified" > build/foo.bin make doesn't think that anything has changed make: Nothing to be done for 'all'.
How can I write the rules of the Makefile to re-create the build/foo.bin if the binary ever gets modified outside of the Makefile?
Make compares timestamps between two files. If the target file exists and its timestamp is newer than all of the prerequisites' timestamps, then make decides the target is up to date and it doesn't need to do anything.
Make doesn't maintain some kind of database of timestamps on its own: it relies on the filesystem for that. So make cannot detect when a file changes from what it previously contained. It can only detect when some other file changed after the target file was last updated.
In short, make cannot do what you want it to do, using its standard methods.
If you want to do that you'll have to get complicated and create a way to turn the behavior you want to detect into a file with a timestamp, that make can compare.
One way to do this would be to keep the md5sum of the file in another file, then compare it and update the file only if it's changed. You can try this (I didn't test it):
build/foo.bin: foo.c checksum.out
gcc $< -o $#
md5sum $# > checksum.out
touch $#
checksum.out: FORCE
md5sum build/foo.bin > checksum.tmp; cmp $# checksum.tmp || cp checksum.tmp $#
FORCE: ;
Basically, the FORCE is there to require the md5sum check to always run, but then if the checksum doesn't actually change it doesn't update the output file which means that build/foo.bin won't be rebuilt (at least not because checksum.out is updated).
The working example below always runs touch fileA with each run of make. How can I stop this?
all: analysis.txt
analysis.txt: countries
touch $#
countries: usa.txt mexico.txt
$(countries): %.txt
%.txt:
echo "bash ./cityGenerator $*" > $#
fileA depends on workflowB. So as long as workflowB is out of date, fileA will be out of date and the touch will be run.
workflowB, as shown here, is marked .PHONY so it will ALWAYS be considered out of date by make, so fileA will ALWAYS be rebuilt.
Even if it weren't .PHONY, there is no rule to create a file named workflowB so it would still always be out of date.
As for how you can stop it, that depends entirely on what these things do which you haven't told us, and why you structured workflowB that way which you also haven't told us.
One way to solve it would be to remove the .PHONY and give workflowB a recipe that touches a file, then it would be up to date.:
workflowB: workflowB_files
touch $#
But of course, that might defeat the purpose of this (which, again, you haven't told us).
Your analysis.txt depends on countries which is considered as a file, but such file is never created, hence the rule is always run. You can find out by the debug output:
$ make -dr
...
Finished prerequisites of target file 'analysis.txt'.
Prerequisite 'countries' of target 'analysis.txt' does not exist.
Must remake target 'analysis.txt'.
touch analysis.txt
...
It seems like you are treating countries as a variable, so maybe this could be reworked:
$ cat Makefile
all: analysis.txt
countries := usa.txt mexico.txt
analysis.txt: $(countries)
touch $#
%.txt:
echo "bash ./cityGenerator $*" > $#
Now there is no countries file considered and make behaves as expected:
$ make
echo "bash ./cityGenerator usa" > usa.txt
echo "bash ./cityGenerator mexico" > mexico.txt
touch analysis.txt
$ make
make: Nothing to be done for 'all'.
I have a directory with test inputs and outputs. I wanted make to automatically test my program against this directory after build, for convenience. Thus I needed to somehow force the test target of Makefile to depend on the entire testing directory (it's called good, because it contains valid inputs and outputs for the program)
I read this question and the accepted answer and the comments about deleted files under this answer: Makefile rule that depends on all files under a directory (including within subdirectories) And, incorporating advice from this answer & comments, I came out with this:
my#comp:~/wtfdir$ cat Makefile
test : test.sh $(shell find good)
./test.sh
my#comp:~/wtfdir$
For the sake of MCVE, test.sh is very rudimentary:
my#comp:~/wtfdir$ cat test.sh
echo "blah"
my#comp:~/wtfdir$
However, I noticed, this behaves in a rather unexpected way:
my#comp:~/wtfdir$ ls good
test1 test1.out
my#comp:~/wtfdir$ make
./test.sh
blah
my#comp:~/wtfdir$ touch good/test1
my#comp:~/wtfdir$ make
cp good/test1 good/test1.out
./test.sh
blah
my#comp:~/wtfdir$
Why (expletive redacted) does modifying test1 cause make to overwrite test1.out with test1??? I'm not a big fan of data losses, you know.
What's going on here?
Your Make appears to be GNU Make. Here's why this happens. Your recipe:
test : test.sh $(shell find good)
./test.sh
adds to the prerequisites of test every file and directory that is listed
by find good in the current directory, which happen to be:
good
good/test1
good/test1.out
So to make target test, Make begins by determining if any of the specified
or built-in recipes require it to rebuild any of the prerequsities:
test.sh good good/test1 good/test1.out
Among its built-in recipes it finds:
%.out: %
# recipe to execute (built-in):
#rm -f $#
cp $< $#
as you can verify by running:
$ make --print-data-base | grep -A4 '%.out'
The rule for this recipe is matched by:
good/test1.out: good/test1
and by doing:
$ touch good/test1
you have made good/test1.out out of date with respect to good/test1.
So make executes the recipe:
#rm -f good/test1.out
cp good/test1 good/test1.out
the visible output of which is what you observed:
cp good/test1 good/test1.out
Then it proceeds with the recipe for test:
./test.sh
blah
There is always a risk of such booby-traps if you write a makefile that blindly
generates at runtime some set of preqrequisites or targets you don't know beforehand.
You could avoid this one in particular by explicitly deleting the offending
implicit pattern rule in your makefile by writing:
%.out: %
with no recipe. And you can avoid all possible booby-traps of this sort by disabling all
built-in recipes, with:
$ make --no-builtin-rules ...
but that will require you to write for yourself any builtin-recipes that your
makefile relies on.
The best solution for you is probably to amend your makefile as follows:
PREREQS := $(shell find good)
test : test.sh $(PREREQS)
./test.sh
$(PREREQS): ;
Then the last line explicitly specifies an empty recipe
for each of the $(PREREQS), and Make will not consult any pattern rules for targets
that have explicit recipes.
You should additionally make test a phony target:
.PHONY: test
for the avoidance of the booby-trap where something creates a file called test in the build directory.
So here's a Makefile to install foo.conf, based on a master copy called foo.conf.master. It installs it to the current directory rather than /etc, just for testing purposes:
all: foo.conf.copied
foo.conf.copied: foo.conf.master foo.conf
cp foo.conf.master foo.conf
touch $#
# Recipe to tell make that it is okay for foo.conf not to exist beforehand.
foo.conf:
So then create foo.conf.master:
$ touch foo.conf.master
$
and you're ready to test:
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$
The point is that if I (with my "trusted" sysadmin hat on) modify foo.conf.master then make (possibly called by cron) will roll out the update:
$ touch foo.conf.master
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$
But equally important: if I (with my "rogue" sysadmin hat on) modify the installed version then make will back out the update:
$ touch foo.conf
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$
Woohoo.
Okay, so now the problem: obviously foo.conf isn't the only file I want do this for, so I need to change my static rules to pattern rules. Okay, that's easy: substitute foo.conf for % in targets and dependencies, substitute foo.conf for $* in the commands, and make a minor modification to the last recipe (which would otherwise become only '%:') so that it doesn't look like I'm trying to cancel a builtin pattern rule.
So clean up and create this Makefile:
all: foo.conf.copied
%.copied: %.master %
cp $*.master $*
touch $#
# Recipe to tell make that it is okay for foo.conf not to exist beforehand.
# Nop tells make that I'm not *cancelling* a pattern rule here
# (see http://stackoverflow.com/questions/34315150/make-implicit-rules-dont-work-without-command).
%: ;
But this doesn't work:
$ make
make: *** No rule to make target `foo.conf.copied', needed by `all'. Stop.
$
The error message is misleading; it is really foo.conf that it doesn't know how to make, which can be demonstrated by adding the following at the bottom of the Makefile:
foo.conf:
touch $#
But then that's a static rule again, which I don't want.
There are a couple more requirements I would also like to satisfy, which the above example doesn't demonstrate. These are:
foo.conf should be installable anywhere in the filesystem (e.g. /etc/foo/server/foo.conf)
foo.conf.master should be in a central directory, or subdirectly thereof, for all master versions, preferably without the '.master' extension (e.g. ~/poor-mans-puppet/master-files/etc/foo/foo.conf)
foo.conf.copied should be in a central directory, not in the same directory as foo.conf (e.g. ~/poor-mans-puppet/timestamp-files/etc/foo/foo.conf)
After much googling, hair pulling, I'm asking here! Any ideas please? (PS: if copying Makefiles from here, remember to change indentation back to tabs.)
Mad Scientist below suggested an elegant static rule, but I really need it to be a pattern rule. The reason is that I need to hook extra dependencies in using rules:
all: <new-dependency>
rather than hooking them in using variables:
STUFF_ALL_SHOULD_DEPEND_ON += <new-dependency>
The reason for this requirement is for consistency with how other (non-%.copied) targets are handled in my very large Makefile.
However, based on Mad Scientist's idea, I tried the following, which didn't work, but perhaps helps somebody to help me:
all: foo.conf.copied
%.copied: %.master %
$(eval FILES_FOR_WHICH_AN_EMPTY_RECIPE_ARE_NEEDED += $$*)
cp $*.master $*
touch $#
define GENERATE_STATIC_EMPTY_RULE
$(1):
endef
$(foreach X,$(FILES_FOR_WHICH_AN_EMPTY_RECIPE_ARE_NEEDED),$(eval $(call GENERATE_STATIC_EMPTY_RULE,$(X))))
Can you explain why you're using this extra ".copied" file? Why don't you just use:
%: %.master ; cp $< $#
?
Anyway, you're running afoul of make's special rules related to match-anything rules (pattern rules like % that can build everything). If you change your pattern so it's not match-anything, like %.conf: ; then it will work. However you probably don't want to assume that all files end in .conf.
Alternatively you can use static pattern rules, like this:
FILES_TO_COPY = foo.conf bar.conf biz.baz
all: $(FILES_TO_COPY:%=%.copied)
$(FILES_TO_COPY:%=%.copied): %.copied : %.master %
cp $*.master $*
touch $#
and you don't need the extra pattern rule.
In the end, I dynamically generated static rules. The following pseudo-code hopefully makes the actual Makefile easier to understand:
if flag not set # flag won't be set on first call
prepare static rules
set flag # prevent running this clause again
recurse! # make invokes make
else
include static rules
do the normal thing
endif
Here's the real Makefile:
ifeq ($(MAKELEVEL),0)
all:
for X in $(patsubst %.copied,%,$^); do \
echo "$$X.copied: $$X.master $$X"; \
echo " cp $$X.master $$X"; \
echo " touch \$$#"; \
echo "$$X: ;"; \
done > Makefile.include
$(MAKE)
# The real dependencies on all are defined below, but while creating
# Makefile.include, we don't want make to digress and start making
# the dependencies; this pattern rule will stop it from doing that.
%.copied: ;
else
include Makefile.include
endif
all: foo.conf.copied
The outer make can be silenced by use of .SILENT and the --no-print-directory option (not shown above for clarity).
Here's the output:
$ touch foo.conf.master
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$ touch foo.conf
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$
I'm trying to create a Makefile that will download and process file a file to generate targets, this is a simplified version:
default: all
.PHONY: all clean filelist.d
clean:
#rm -fv *.date *.d
#The actual list comes from a FTP file, but let's simplify things a bit
filelist.d:
#echo "Getting updated filelist..."
#echo "LIST=$(shell date +\%M)1.date $(shell date +\%M)2.date" > $#
#echo 'all: $$(LIST)' >> $#
%.date:
touch $#
-include filelist.d
Unfortunately the target all doesn't get updated properly on the first run, it needs to be run again to get the files. This is the output I get from it:
$ make
Getting updated filelist...
make: Nothing to be done for `default'.
$ make
Getting updated filelist...
touch 141.date
touch 142.date
touch 143.date
I'm using GNU Make 3.81 whose documentation states that it reloads the whole thing if the included files get changed. What is going wrong?
You have specified filelist.d as a .PHONY target, so make believes making that target doesn't actually update the specified file. However, it does, and the new contents are used on the next run. For the first run, the missing file isn't an error because include is prefixed with the dash.
Remove filelist.d from .PHONY. However, remember it won't be regenerated again until you delete it (as it doesn't depend on anything).
By the same token, you should include "default" in .PHONY.
I wrote a shell script rather than lump all this in the makefile:
#!/bin/bash
# Check whether file $1 is less than $2 days old.
[ $# -eq 2 ] || {
echo "Usage: $0 FILE DAYS" >&2
exit 2
}
FILE="$1"
DAYS="$2"
[ -f "$FILE" ] || exit 1 # doesn't exist or not a file
TODAY=$(date +%s)
TARGET=$(($TODAY - ($DAYS * 24 * 60 * 60)))
MODIFIED=$(date -r "$FILE" +%s)
(($TARGET < $MODIFIED))
Replace X with the max number of days that can pass before filelist.d is downloaded again:
filelist.d: force-make
./less-than-days $# X || command-to-update
.PHONY: force-make
force-make:
Now filelist.d depends on a .PHONY target, without being a phony itself. This means filelist.d is always out of date (phony targets are always "new"), but its recipe only updates the file periodically.
Unfortunately, this requires you to write the update command as a single command, and space may be a problem if it is long. In that case, I would put it in a separate script as well.