I wonder if there is a way to tell GNU make to remake a file older than, say, 24 hours.
I.e., something like force, but only if the target has been last updated over 24 hours ago.
Let's say I have a program which collects sensor data. I don't want to ping the sensors more often than every 24 hours.
Keep a sentinel file around. Date it 24 hours in the past and use normal make dependencies. Recursive make. (All round yuk!)
.PHONY: poll
poll:
touch -d yesterday sentinel
${MAKE} target
target: sentinel
blah blah
touch $#
Assumes gnu touch.
Related
write:
#echo `date`
file_1: write file_2
#echo "file_1 begin"
$(shell sleep 3)
#echo "file_1 end"
file_3: write file_4
#echo "file_3 begin"
$(shell sleep 3)
#echo "file_3 end"
all: file_1 file_3
.PHONY: write
When I run make, it outputs:
Sat Oct 9 15:22:45 CST 2021
file_1 begin
file_3 begin
file_1 end
file_3 end
target write run only once.
My ultimate goal is to calculate the execution time of file_1 and file_3 by writing the start time of target write. In the process of testing whether it will be covered, the above problems were found. If there is a better way to calculate the time, let me know.
First, it's virtually always an error to use the $(shell ...) function inside a recipe. A recipe is already running in the shell, so it's not needed, and it has unexpected behaviors. Why not just use #sleep 3 here instead?
There is no way to make a single target get built more than one time in a single invocation of make. That would be very bad: consider if you have multiple programs all depending on one .o file for example: would you want to recompile that .o file many times?
Anyway, this won't really do what you want:
file_1: write file_2
This doesn't measure the execution time of file_1 because the start date is written, then file_2 is built, then file_1 is built. So you're including the execution time of file_2 as well.
And, of course, if you ever enable parallel builds all bets are off.
If you want to measure something, put the start/stop operations inside the recipe, not using prerequisites. You can hide it in a variable to make it less onerous to type.
First time posting on stackoverflow!
I'm just getting started on using GNU make. I'm using a simple example Makefile described here (and printed below). There are two steps to be completed by the Makefile, both of which I can complete without error without using make.
Here's the Makefile (truncated from original because I don't have topojson installed):
counties.zip:
curl -o counties.zip 'http://www2.census.gov/geo/tiger/GENZ2010/gz_2010_us_050_00_20m.zip'
gz_2010_us_050_00_20m.shp: counties.zip
unzip counties.zip
touch gz_2010_us_050_00_20m.shp
When I attempt to use make make -f Makefile or just make, I get the following:
curl -o counties.zip 'http://www2.census.gov/geo/tiger/GENZ2010/gz_2010_us_050_00_20m.zip'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1277k 100 1277k 0 0 955k 0 0:00:01 0:00:01 --:- -:-- 956k
The file successfully downloads, but is never unzipped in the next step. If I type make again, it prints:
make: `counties.zip' is up to date.
I've tried similar simple multi-step test cases and the same thing is happening - the first step is completed, but no subsequent steps are. I know I've installed programs using make in the past on this machine (Ubuntu 14.04.3 LTS).
Any help would be greatly appreciated. Apologies if I've forgotten any relevant details or this is a redundant question.
You need to add the rule at the beginning of the Makefile.
all: counties.zip gz_2010_us_050_00_20m.shp
Make sure all are the same, there is a good tutorial here, it is basic.
I have a make file that is always re-running a rule despite the prerequisite being up-to-date:
data.tar.bz2: data.tar
bzip2 --keep data.tar
# The following rule always runs the rule for its prerequisite
# even if it is up-to-date. Note that it doesn't matter whether
# the bunzip2 directory exists or not. I suppose is has something
# to do with the dir/file naming of the rule but I haven't been
# able to decipher why.
bunzip2/data2.tar: data.tar.bz2
mkdir bunzip2 && cd bunzip2 && bzip2 -ckd ../data.tar.bz2 > data2.tar && cd ..
.PHONY: clean
clean:
rm -f data.tar.bz2
rm -Rf bunzip2
Any ideas appreciated.
The standard POSIX system calls that set timestamps on files do not support sub-second accuracy. In order for these tools to set a specific time on a file (which they try to do so that the compressed file has the same timestamp as the original file) and preserve the original accuracy they need to use different system calls; apparently they don't do so.
What you can do to work around this is change your rule like this:
data.tar.bz2: data.tar
bzip2 --keep data.tar && touch $#
so that the timestamp of the target is set to "now".
ETA The traditional system call for setting modification times on files is utime() which accepts timestamps only in increments of one second. Newer versions of the POSIX spec introduced utimensat() which allows nanosecond timestamp settings. There is also utimes() which allows microsecond accuracy but which has been considered "legacy" for a long time now.
If this behavior exists in the current latest release of bzip2, I would consider it a bug worthy of a bug report to them.
I think I figured out what was going on here.
Using #Thomas Dickey advice to try make -d, it was saying that the file data.tar.bz2 was newer than data.tar itself. That seems silly because the former is created from the later. But using ls --full-time shows bzip2 archive seems use the timestamp of the source file but it also truncates it after the second place. In my case, the timestamp 2015-03-12 09:32:02.452091888 was getting truncated to 2015-03-12 09:32:02.000000000 which makes it appear to be newer than the source. (By truncate I mean that 2015-03-12 13:10:29.681793152 goes to 2015-03-12 13:10:29.000000000... it does not round up even in that case) Seems like bzip needs to bump the accuracy of their timestamps.
It appears that lunzip also truncates the timestamp and unxz preserves the original timestamp while gzip uses the current time. In other words, be careful with your makefiles when using compression utilities because they handle things differently.
Something is missing from the example - but the process for analysis is the same, regardless: you can use the debug feature of make to see what rules it uses, e.g.,
make -d
This concerns an external project, where one OneLibrary.c and SomeLibraries.am where modified;
It would be nice to
./configure --please-configure-only-this-subdirectory=./SomeLibrariesSubdirectory
something similar to make -C ./SomeLibrariesSubdirectory
Because the whole "./configure" takes 5 minutes when only two lines have been modified ;)
(this is similar to GNU make: Execute target but take dependency from file but slightly different).
When I try to force to build a target, make automatically thinks that this target is out of date and forces a run of all targets which depend on it.
In my case, the target is a recursive make call which does a lot of work and might just return with "nothing to be done":
.PHONY: intermediate.dat
intermediate.dat:
$(MAKE) expensive_chain_which_finally_creates_intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
In this case, "make all" runs for 40 seconds although intermediate.dat might not have changed (recursive make returned "nothing to be done"). However, if the recursive make updated intermediate.dat, the target shall be out of date.
Is there really no way to do this?
Make intermediate.dat depend on a phony target instead of being phony itself.
.PHONY : always-remake
intermediate.dat : always-remake
IIRC, the last time I solved the problem, the .PHONY didn't work as intended and I used:
always-remake :
#true
instead. However, I can't recall why, so try the .PHONY first.
The problem with making intermediate.dat itself phony is that make never checks the existence/date of a phony file, which is behaviour that you want. You only need to trigger the rebuild rule, which is done by a prerequisite that is out of date; a phony prerequisite is always out of date, so it does the job.
Another trick is to use an order-only prerequisite, plus a recursive call to make. This can be useful when you want to implement some additional logic around when the intermediate.dat file needs to be created, or when you want to decouple a target from the usual exists && newer than check that make does.
For example:
Let's say that the file contains some date-sensitive material that subsequent tasks depend on, and that it expires after 12 hours. We only want to recreate the file when any of these is true:
The file is older than 12 hours
The file does not exist
To implement the make target & recipes, we'll create a .PHONY check-intermediate.dat target, which runs the time check, conditionally removes the file if it's expired, and then re-runs make for that file.
Note that the check-intermediate.dat target will be defined via the order-only prerequisite syntax (e.g. intermediate.dat: | check-intermediate.dat) which causes it to run always before the target (e.g. intermediate.dat). The time check is inexpensive, and we always want it to run and manage the file for us, but not invalidate the original make check on intermediate.dat if it already exists like a normal prerequisite .PHONY target would.
.PHONY: check-intermediate.dat
check-intermediate.dat:
if [ -e intermediate.dat ]; then find intermediate.dat -mmin +720 -exec bash -c 'rm -f "{}"; $(MAKE) intermediate.dat' \; ; fi
intermediate.dat: | check-intermediate.dat
echo run-expensive-tasks-here-to-create-intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
Now, anything that depends on the intermediate.dat target will first run the check, remove the file if it's expired, and re-run make intermediate.dat before running the rest of the dependent targets. If the intermediate.dat file already exists, it will run the check, and if the check passes, it will continue without running the time expensive recreation task.
The problem with depending on a .PHONY target is that it will always run the tasks which depend on it, because of the way GNU Make handles .PHONY targets. This can cause the make run to always perform the time expensive operations even when it does not need to:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file. As long as a phony target is never a prerequisite of a real target, the phony target recipe will be executed only when the phony target is a specified goal (see Arguments to Specify the Goals).