makefile target is newer than prerequisites but gets re-done - makefile

Running my target twice in a row it gets executed twice. This is wrong, since configure file is newer.
$ cat makefile
.ONESHELL:
configure: configure.www
#REMOTE_FILE=$$(cat $<)
#wget $${REMOTE_FILE}
#tar -xf $$(basename $${REMOTE_FILE})
#mv pspp-1.4.1/configure .
$ cat configure.www
https://ftp.gnu.org/gnu/pspp/pspp-1.4.1.tar.gz
The $$(cat <$) happens before the cp that creates the configure file.

Posting #Renaud Pacalet answer from the comments for future users:
Indeed, the mv instruction does not change the modification date, so the makefile really compares between PSPP's last modification date of configure and my own configure.www modification date. Since the latter is newer (and will remain newer) the target will keep getting executed.
$ stat configure | grep "Modify"
Modify: 2020-09-06 01:20:01.000000000 +0300
$ stat configure.www | grep "Modify"
Modify: 2021-05-05 10:11:43.891400412 +0300 // Newer !
I solved it by adding a final touch (no pun intended :))
.ONESHELL:
configure: configure.www
#REMOTE_FILE=$$(cat $<)
#wget $${REMOTE_FILE}
#tar -xf $$(basename $${REMOTE_FILE})
#mv pspp-1.4.1/configure .
#touch $# <----------------------

Related

make keeps running a rule

The working example below always runs touch fileA with each run of make. How can I stop this?
all: analysis.txt
analysis.txt: countries
touch $#
countries: usa.txt mexico.txt
$(countries): %.txt
%.txt:
echo "bash ./cityGenerator $*" > $#
fileA depends on workflowB. So as long as workflowB is out of date, fileA will be out of date and the touch will be run.
workflowB, as shown here, is marked .PHONY so it will ALWAYS be considered out of date by make, so fileA will ALWAYS be rebuilt.
Even if it weren't .PHONY, there is no rule to create a file named workflowB so it would still always be out of date.
As for how you can stop it, that depends entirely on what these things do which you haven't told us, and why you structured workflowB that way which you also haven't told us.
One way to solve it would be to remove the .PHONY and give workflowB a recipe that touches a file, then it would be up to date.:
workflowB: workflowB_files
touch $#
But of course, that might defeat the purpose of this (which, again, you haven't told us).
Your analysis.txt depends on countries which is considered as a file, but such file is never created, hence the rule is always run. You can find out by the debug output:
$ make -dr
...
Finished prerequisites of target file 'analysis.txt'.
Prerequisite 'countries' of target 'analysis.txt' does not exist.
Must remake target 'analysis.txt'.
touch analysis.txt
...
It seems like you are treating countries as a variable, so maybe this could be reworked:
$ cat Makefile
all: analysis.txt
countries := usa.txt mexico.txt
analysis.txt: $(countries)
touch $#
%.txt:
echo "bash ./cityGenerator $*" > $#
Now there is no countries file considered and make behaves as expected:
$ make
echo "bash ./cityGenerator usa" > usa.txt
echo "bash ./cityGenerator mexico" > mexico.txt
touch analysis.txt
$ make
make: Nothing to be done for 'all'.

Makefile with dependency graph not known in advance

I'm trying to create a makefile (GNU make) that does the following:
A script generates a bunch of files--filenames not known in advance.
Each one of these files is converted to a different file.
After all are converted, all of these files are combined into a single output file.
How do I create a makefile with a "bellcurve"-patterned dependency graph, where the intermediate source and target files are not known in advance?
Conceptually I'm doing the following:
combined.pdf: $(filter combined.pdf, $(wildcard *.pdf))
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
$(wildcard *.svg):
# recipe is for simple example
# actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
.PHONY: clean
clean:
${RM} *.svg *.pdf *.d
Of course this doesn't work: Make evaluates the targets and sources before it runs the target that actually creates the svg. Also, there's no way to make sure all svgs are converted before they are combined.
I realized I could create dependencies and include them into the makefile, but I had trouble getting this to work too:
.PHONY: clean
include deps.d
combined.pdf: deps.d
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
deps.d:
## recipe is for simple example
## actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
## we know what files exist now, so we can establish dependencies
## "a.pdf : a.svg"
echo *.svg : > deps.d
## combined.pdf: a.pdf b.pdf c.pdf
ls *.svg \
| awk '{targetfn=$$0; sub(/\.svg$$/, ".pdf", targetfn); print targetfn, ":", $$0;}' \
>> deps.d
## combined.pdf: a.pdf b.pdf c.pdf
echo combined.pdf : $$(echo *.svg | sed -e 's/\.svg/\.pdf/g') >> deps.d
clean:
${RM} *.pdf *.svg *.d
However this still isn't connecting the dependency graph properly. When I run this, make quits as follows:
Makefile:3: deps.d: No such file or directory
echo a > a.svg
echo b > b.svg
echo c > c.svg
echo *.svg : > deps.d
ls *.svg \
| awk '{targetfn=$0; sub(/\.svg$/, ".pdf", targetfn); print targetfn, ":", $0;}' \
>> deps.d
echo combined.pdf : $(echo *.svg | sed -e 's/\.svg/\.pdf/g') >> deps.d
make: Nothing to be done for `a.svg'.
I still seem to have the problem that the make doesn't know about the rules in deps.d.
Also, this still doesn't solve the problem of building all the dependencies. I thought of using a marker file like this:
%.pdf: %.svg
cp $^ $#
## if all svgs are converted, touch a target allpdfs
if [ $(ls -1 *.svg | wc -l) -eq $(ls -1 *.pdf | grep -v combined\.pdf | wc -l) ]; touch allpdfs; fi
But there's no way to inform make that "allpdfs" may be created by this rule.
I'm surprised that moving the include directive makes a difference (what version of Make are you using?), but there is a simpler way. Your use of deps.d is in effect a recursive use of Make -- Make is arranging to execute itself a second time -- so we might as well make it official:
combined.pdf: ALL_SVGS
$(MAKE) ALL_PDFS
rm -f $# # just in case it exists already
cat *.pdf > $#
.PHONY: ALL_SVGS
ALL_SVGS:
# recipe is for simple example
# actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
# These variables will be empty in the first execution of Make
SVGS = $(wildcard *.svg)
PDFS = $(patsubst %.svg,%.pdf,$(SVGS))
.PHONY: ALL_PDFS
ALL_PDFS: $(PDFS))
%.pdf: %.svg
cp $^ $#
This isn't an answer exactly, because I don't know why this works, but I discovered that if I move the include directive after the target that creates the included file, everything works.
I.e. do this:
deps.d:
....
include deps.d
Because my deps.d includes enough dependency information, there's no need to have an intermediate target allpdfs file. Everything Just Works, even with make -j.
However, I don't know why this works. The include documentation isn't enlightening me.
UPDATE
I noticed the following note at the very bottom of the make manual discussing Automatic Prerequisites:
Note that the ‘.d’ files contain target definitions; you should be sure to place the include directive after the first, default goal in your makefiles or run the risk of having a random object file become the default goal. See How Make Works.
So what happened is that the first rule inside the generated deps.d became the default target, causing the mysterious premature completion of the build. So the solution is just to make sure include directives are not before your intended default target.
I was just working on this exact problem in a slightly different setting. Here is a clean solution - no need for recursion and such (and you can tweak the sed if you like):
include deps.d
combined.pdf:
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
deps.d:
echo a > a.svg
echo b > b.svg
echo c > c.svg
echo 'combined.pdf:' *.svg | sed 's/\.svg/\.pdf/g' > deps.d
Enjoy!

Suppress "Clock skew" warning for future-times in Makefile

I have a Makefile that does performs a task if it hasn't happened in the last hour. It does so like this:
HOUR_FROM_NOW = $(shell perl -e '($$s,$$m,$$h,$$d,$$M)=localtime(time()+3600); printf("%02d%02d%02d%02d\n",$$M+1,$$d,$$h,$$m);')
NOW_FILE = $(shell mkdir -p .make; touch .make/now; echo .make/now )
.PHONY: externals
externals: $(PROJECTS:%=.make/proj_%)
.make/proj_%: $(NOW_FILE)
$(MAKE) -s $(*F)
touch -t $(HOUR_FROM_NOW) $#
.PHONY: $(PROJECTS)
$(PROJECTS):
# do stuff, specifically, clone git-repo if not exists, else pull latest
That part works great, except that I now get warnings:
make: Warning: File `.make/proj' has modification time 3.5e+03 s in the future
make: Nothing to be done for `externals'.
make: warning: Clock skew detected. Your build may be incomplete.
Anyone know how to suppress those warnings? (Or to do a periodic task in a makefile)
Most versions of touch I have come across can do some date time maths which allows for setting the timestamp of a file directly via the --date option.
That and the fact that variables assigned with := are only "evaluated once" makes this a bit easier to read.
HOUR_AGO := .make/hour_ago
__UGLY := $(shell mkdir -p .make && touch --date='1hour ago' $(HOUR_AGO))
# The preceding line will be executed once
.make/proj_%: .make/hour_ago | .make
$(MAKE) -s $(*F)
#touch $#
.make:
mkdir -p $#
I'm using something very similar to this to periodically refresh login tokens.
Never would have thought of it if it wasn't for Dave's answer though.
The directory is created by specifying it as a order-only-prerequisite
I suspect that the + 3600 is at fault. What happens if you remove it?
I thought and thought, and then the stupid-obvious solution hit me ...
Instead of setting timestamps in the future with HOUR_FROM_NOW, I use the real time and compare with HOUR_AGO_FILE ...
HOUR_AGO = $(shell perl -e '($$s,$$m,$$h,$$d,$$M)=localtime(time()-3600); printf("%02d%02d%02d%02d\n",$$M+1,$$d,$$h,$$m);')
HOUR_AGO_FILE = $(shell mkdir -p .make; touch -t $(HOUR_AGO) .make/hour_ago; echo .make/hour_ago )
.PHONY: externals
externals: $(PROJECTS:%=.make/proj_%)
.make/proj_%: $(HOUR_AGO_FILE)
$(MAKE) -s $(*F)
#touch $#

gnu make reloads includes but doesn't update the targets

I'm trying to create a Makefile that will download and process file a file to generate targets, this is a simplified version:
default: all
.PHONY: all clean filelist.d
clean:
#rm -fv *.date *.d
#The actual list comes from a FTP file, but let's simplify things a bit
filelist.d:
#echo "Getting updated filelist..."
#echo "LIST=$(shell date +\%M)1.date $(shell date +\%M)2.date" > $#
#echo 'all: $$(LIST)' >> $#
%.date:
touch $#
-include filelist.d
Unfortunately the target all doesn't get updated properly on the first run, it needs to be run again to get the files. This is the output I get from it:
$ make
Getting updated filelist...
make: Nothing to be done for `default'.
$ make
Getting updated filelist...
touch 141.date
touch 142.date
touch 143.date
I'm using GNU Make 3.81 whose documentation states that it reloads the whole thing if the included files get changed. What is going wrong?
You have specified filelist.d as a .PHONY target, so make believes making that target doesn't actually update the specified file. However, it does, and the new contents are used on the next run. For the first run, the missing file isn't an error because include is prefixed with the dash.
Remove filelist.d from .PHONY. However, remember it won't be regenerated again until you delete it (as it doesn't depend on anything).
By the same token, you should include "default" in .PHONY.
I wrote a shell script rather than lump all this in the makefile:
#!/bin/bash
# Check whether file $1 is less than $2 days old.
[ $# -eq 2 ] || {
echo "Usage: $0 FILE DAYS" >&2
exit 2
}
FILE="$1"
DAYS="$2"
[ -f "$FILE" ] || exit 1 # doesn't exist or not a file
TODAY=$(date +%s)
TARGET=$(($TODAY - ($DAYS * 24 * 60 * 60)))
MODIFIED=$(date -r "$FILE" +%s)
(($TARGET < $MODIFIED))
Replace X with the max number of days that can pass before filelist.d is downloaded again:
filelist.d: force-make
./less-than-days $# X || command-to-update
.PHONY: force-make
force-make:
Now filelist.d depends on a .PHONY target, without being a phony itself. This means filelist.d is always out of date (phony targets are always "new"), but its recipe only updates the file periodically.
Unfortunately, this requires you to write the update command as a single command, and space may be a problem if it is long. In that case, I would put it in a separate script as well.

Suppress messages in make clean (Makefile silent remove)

I'm wondering how I can avoid some echo in a Makefile :
clean:
rm -fr *.o
this rule will print:
$>make clean
rm -fr *.o
$>
How can I avoid that?
To start with: the actual command must be on the next line (or at least that is the case with GNU Make, it might be different with other Make's - I'm not sure of that)
clean:
rm -rf *.o
(note, you need a TAB before rm -rf *.o as in every rule)
Making it silent can be done by prefixing a #:
so your makefile becomes
clean:
#rm -rf *.o
If there are no *.o files to delete, you might still end up with an error message. To suppress these, add the following
clean:
-#rm -rf *.o 2>/dev/null || true
2>/dev/null pipes any error message to /dev/null - so you won't see any errors
the - in front of the command makes sure that make ignores a non-zero return code
In fact I was looking for something else, adding this line to the Makefile :
.SILENT:clean
while execute every step of the "clean" target silently.
Until someone point some drawback to this, I use this as my favourite solution!
I'm responding to this ancient topic because it comes up high in search and the answers are confusing. To do just what the user wants,all that is needed is:
clean:
#rm -f *.o
The # means that make will not echo that command.
The -f argument to rm tells rm to ignore any errors, like there being no *.o files, and to return success always.
I removed the -r from the OPs example, because it means recursive and here we are just rming .o files, nothing to recurse.
There's no need for the 2>&1 >/dev/null because with the -f there will be no errors printed.
.SILENT: clean
works in place of the #, but it isn't at the same place in the Makefile as the command that it affects, so someone maintaining the project later might be confused. That's why # is preferred. It is better locality of reference.
If you put an # in front of the command, it doesn't echo onto the shell. Try changing rm to #rm. (Reference)
From the manual: .SILENT is essentially obsolete since # is more flexible.
Much worse is that make prints far too much information. Warning/error/private messages are buried in the output. On the other hand -s (.SILENT) suppresses just anything. Especially the "nothing to be done" and "up to date" messages can be a pain. There is no option to suppress them. You have to filter them out actively or use something like colormake. Here is a solution for grep:
make | egrep -hiv 'nothing to be done|up to date'
But the output will have line numbers. The Perl solution is therefore better, because it suppresses line numbers and flushes stdout immediately:
make | perl -ne '$|=1; print unless /nothing to be done|up to date/i'
Make's a flawed tool. "What’s Wrong With GNU make?" explains this better than I can.
There's a great article on using .SILENT that explains how to conditionally activate it.
I have used that information to put this in my Makefile:
# Use `make V=1` to print commands.
$(V).SILENT:
# Example rule, only the #echo needs to be added to existing rules
*.o: %.c
#echo " [CC] $<"
gcc ...
What this does is if you run make normally, normal output is silenced and instead the echo commands work:
$ make
[CC] test.c
[CC] test2.c
But it allows you to debug problems by passing the V=1 parameter, which still shows the [CC] messages as it helps break up the output, but the traditional Makefile output is also visible:
$ make V=1
[CC] test.c
gcc ...
[CC] test2.c
gcc ...

Resources