I'm writing some Makefile. There is a file auto that can be processed in automatic way (build by make). There is recipe for it. So, this file needs to be edited manually and saved as manual. This manual is used for further automatic processing. How can I achieve the next:
If auto is not exists --- build it --- done by the recipe for it.
If manual is older than auto --- print a message and exit make.
If manual is newer --- build further --- done by the recipe for it.
How can the second step be done in some native way?
EDIT
The process description was not clear enough because I had no understanding how it should be. That was my mistake. Here is the process I assume.
There is source.
The auto can be built automatically from source.
The manual can be made manually from auto.
The result can be built automatically from manual.
Going 1->2 and 3->4 is what is make for.
The manual depends on source. But make itself can't build manual but only auto to help the user. So, if the manual is up-to-date (newer than source), then just build result from it.
But if manual is have to be rebuilt (it is older than source), rebuild auto (if needed), print a message and stop.
So, I think that Makefile should looks like:
SOURCE = source
result : manual ; cp manual result
manual : auto $(SOURCE) ; #echo "CREATE $#"
auto : $(SOURCE) ; cp source auto
But making result I have to stop after manual and let user to create it and run make again. The manual will be newer than source and auto and will not have to be rebuilt. But how can I do this?
The easiest is probably to describe your dependencies as they really are, even if you cannot really build manual with a make recipe. Then, for manual, instead of writing a building recipe, just print your message and exit with a non-zero status. If manualdoes not exist or is out of date, make will print the message and stop before trying to (re)build result:
# Makefile
result: manual
cp -f "$<" "$#"
manual: auto
#echo "CREATE $#"
#exit 1
auto: source
cp -f "$<" "$#"
Demo:
$ rm -f auto manual result source
$ touch source
$ make
cp -f "source" "auto"
CREATE manual
make: *** [Makefile:6: manual] Error 1
$ touch manual
$ make
cp -f "manual" "result"
$ make
make: 'result' is up to date.
$ touch source
$ make
cp -f "source" "auto"
CREATE manual
make: *** [Makefile:6: manual] Error 1
If you don't like the error situation you can defer the stop to the result recipe:
# Makefile
result: manual
[ -f "$#" ] && cp -f "$<" "$#" || true
manual: auto
#echo "CREATE $#"
auto: source
cp -f "$<" "$#"
Demo:
$ rm -f auto manual result source
$ touch source
$ make
cp -f "source" "auto"
CREATE manual
[ -f "manual" ] && cp -f "manual" "result" || true
$ touch manual
$ make
[ -f "manual" ] && cp -f "manual" "result" || true
$ make
make: 'result' is up to date.
$ touch source
$ make
cp -f "source" "auto"
CREATE manual
$ make
CREATE manual
$ touch manual
$ make
[ -f "manual" ] && cp -f "manual" "result" || true
Related
I'm trying to find a good way to add some RAII-like actions into a makefile I'm maintaining. Currently, I have something similar to this:
out: in
lockfile in.lock
echo in // Some action which can fail
rm -f in.lock
This code works fine when using multiple jobs, as it is mainly meant sanity instead of performance. At least, if my action is not failing. So if I like to add a fallback to this. So in short, it'll look something like:
out: in
lockfile in.lock
(echo in) || (rm -f in.lock; false)
rm -f in.lock
Yet again looking good, though I don't like having to write twice rm -f in.lock, nor does the (echo in) looks elegant if the actual content is several lines of bash-script.
This would look similar to:
out: in
lockfile in.lock
trap "rm -f in.lock" EXIT; \
(echo in)
However, this would make the actual rules look more complex if you have different rules which are really separate.
out: in
lockfile in.lock
trap "rm -f in.lock" EXIT; \
$(SHOW_DEPENDENCY_ON_DEBUG) && \
(echo in)
Where SHOW_DEPENDENCY_ON_DEBUG can be defined as echo $# <=== $^ in certain circumstances and # in others. So I'm not that sure if I can nicely chain all commands. Therefore I hope any of you know of some tricks that I've missed.
In short, I like to transform:
out: in
lockfile in.lock
echo in // Some action(s) which can fail
rm -f in.lock
In a way that always executes rm -f in.lock, without having to chain bash-commands or duplicating the action(s) that have to be executed to finalize the actions in the rules.
For the problem of ensuring the your lockfile (or any file that make makes) is deleted come what may,
make has a stock solution: make it an .INTERMEDIATE target.
Then, if make creates the file, it will auto-delete it at the end, come what may, e.g.
Makefile
.PHONY: all clean
all: out
in:
touch $# # Whatever
.INTERMEDIATE: in.lock
%.lock: %
touch $# # Whatever
out: in.lock
if [ "`shuf -i 0-1 -n 1`" = "0" ]; then echo Fail; false ; else echo Succeed; touch $#; fi
rm -f $<
clean:
rm -f in out
Here the command:
if [ "`shuf -i 0-1 -n 1`" = "0" ]; then echo Fail; false ; else echo Succeed; touch $#; fi
will fail or succeed on a pseudo-random coin-toss.
Some runs:
$ make
touch in # Whatever
touch in.lock # Whatever
if [ "`shuf -i 0-1 -n 1`" = "0" ]; then echo Fail; false ; else echo Succeed; touch out; fi
Succeed
rm -f in.lock
$ make clean
rm -f in out
$ make
touch in # Whatever
touch in.lock # Whatever
if [ "`shuf -i 0-1 -n 1`" = "0" ]; then echo Fail; false ; else echo Succeed; touch out; fi
Fail
Makefile:14: recipe for target 'out' failed
make: *** [out] Error 1
rm in.lock
But don't push this feature so far as removing the
rm -f $<
from the recipe. make will delete the intermediates at exit, which is fine if the recipe fails.
But if the recipe succeeds you presumably want the lockfile deleted right away rather than when
make finishes, which might be arbitrarily later.
Later
Any chance the .INTERMEDIATE can refer to wildcard, like %.lock?
No. You'd have to mean:
.INTERMEDIATE: %.lock
and there is no "wildcard" there. With no % in lefthand side,
it's not a pattern-rule and % in the righthand side only just means %.
But you don't need this. You must know the names of the prerequisites you
want to lock or at least be able to compute them with makefunctions.
Otherwise you can't write the makefile at all. So say they are ina inb inc.
Then you make all the locks intermediate like:
inputs := ina inb inc
locks := $(patsubst %,%.lock,$(inputs))
.INTERMEDIATE: $(locks)
GNU Make includes a special target called .DELETE_ON_ERROR. If this is included in your Makefile, Make will delete any target whose build sequence completes with a non-zero return status. This is helpful so that in subsequent invocations Make does not assume that the target has been properly built.
Here's a dummy example.
.DELETE_ON_ERROR:
out.dat: in.dat
touch out.dat
false
Because false gives a non-zero return value, the build is considered failed and Make deletes the out.dat target. This is the advertised and expected behavior. However, this behavior does not seem to be preserved when the target is a directory. Consider another dummy example.
.DELETE_ON_ERROR:
outdir/: in.dat
mkdir outdir/
false
In this case, the build fails again but Make does not remove the outdir directory. Is there any way I can instruct Make to do this?
As noted in the comments, it is hard to use timestamps on directory. Few options:
proxy target (%.dir)
Atomic update using temporary folder.
Using proxy target, Makefile can be modified to incude a '%.done' target, which will embed the cleanup logic.
.PHONY: %.dir
outdir.dir:
$(MAKE) outdir ; if [ $? -ne 0 ] ; then echo CLEANUP $# ; rm -rf dir ; false ; fi
outdir: ... # as before
And use the outdir.dir as a dependency. Not elegant, but will get the work done. May be possible to to convert into a rule (disclaimer: I did not test this approach).
.PHONY %.dir
%.dir:
$(MAKE) $* ; if [ $? -ne 0 ] ; then echo CLEANUP $* ; rmd -rf $* ; false ; fi
Another variation is to change the outdir to add a "done" indicator file (if completed successfully), and use the proxy target to validate
%.dir:
$(MAKE) $* ; if [ ! -f $*.done ] ; then rm -rf $* ; false ; fi
outdir:
... commands, any can fail.
touch $*.done
As last resort (or first option, depending on your situation), consider, 'atomic' build for outdir - creating a temporary folder, and renaming it to outdir on success
outdir:
rm -rf $#.new $#
mkdir $#.new
# Command to create outdir.new here
mv $#.new $#
I am very new to Makefiles, so I am probably not doing this the best way (your input is much appreciated, since I would like to learn how/why mine is bad). Anyway, here is my problem:
I have a Daemon that I wrote for a program of mine and I am trying to install it with the Makefile (target is "install"). What the "install" target is supposed to do is move the daemon binary to a location, then move the "service script" to either /etc/init.d/ or /etc/rc.d/ (since different distros have different folders...). Here is my makefile so far:
all:
#echo "Making Components"
#cd Daemon; make
#echo "Components Built"
install:
#echo "Installing Components"
#mkdir -p /usr/lib/
#cp Daemon/myprog_d /usr/lib/myprog_d
-#test -d /etc/init.d && cp Scripts/myprog /etc/init.d/
-#test -d /etc/rc.d && cp Scripts/myprog /etc/rc.d/
-#test ! -d /etc/init.d -a ! -d /etc/rc.d && echo " Warning: Couldn't install script. Manually install Scripts/myprog"
#mkdir -p /var/log/
#echo "Installed."
uninstall:
#echo "Uninstalling Components"
#./Scripts/myprog stop > /dev/null
#rm -f /usr/lib/myprog_d
#echo "Uninstall complete"
clean:
#echo "Cleaning Components"
#cd Daemon; make clean
#echo "Clean complete"
As you can see, the "install" target tests to see if those two directories exist and, if they do, copies the script into them (I haven't yet done it to "uninstall", don't worry).
My first question: Is this the right way to do this? The "all" and "clean" targets work (there is another makefile in "Daemon/", as you can deduce), but I want to know if there is a cleaner way of doing this.
Secondly, because the "test" function returns non-zero, I had to do "-" before it so the error would be ignored. Unfortunately, that results in the "make install" output being:
Installing Components
make: [install] Error 1 (ignored)
make: [install] Error 1 (ignored)
Installed.
Which is very ugly and probably not good practice. What can I do in this case? (I have tried both -# and #-, but # will not suppress the "ignored" output)
Sincerely,
Chris
I'd do it this way:
#if [ -d /etc/init.d ]; then cp Scripts/myprog /etc/init.d/ ; fi
#if [ -d /etc/rc.d ]; then cp Scripts/myprog /etc/rc.d/ ; fi
And I'm a little confused by your next line (-#test ! -d /etc/init.d -a !...) but you can probably do it the same way.
That takes care of the error messages, but if you wanted to keep the makefile as it is, you could suppress them by running make -s.
I'm trying to create a Makefile that will download and process file a file to generate targets, this is a simplified version:
default: all
.PHONY: all clean filelist.d
clean:
#rm -fv *.date *.d
#The actual list comes from a FTP file, but let's simplify things a bit
filelist.d:
#echo "Getting updated filelist..."
#echo "LIST=$(shell date +\%M)1.date $(shell date +\%M)2.date" > $#
#echo 'all: $$(LIST)' >> $#
%.date:
touch $#
-include filelist.d
Unfortunately the target all doesn't get updated properly on the first run, it needs to be run again to get the files. This is the output I get from it:
$ make
Getting updated filelist...
make: Nothing to be done for `default'.
$ make
Getting updated filelist...
touch 141.date
touch 142.date
touch 143.date
I'm using GNU Make 3.81 whose documentation states that it reloads the whole thing if the included files get changed. What is going wrong?
You have specified filelist.d as a .PHONY target, so make believes making that target doesn't actually update the specified file. However, it does, and the new contents are used on the next run. For the first run, the missing file isn't an error because include is prefixed with the dash.
Remove filelist.d from .PHONY. However, remember it won't be regenerated again until you delete it (as it doesn't depend on anything).
By the same token, you should include "default" in .PHONY.
I wrote a shell script rather than lump all this in the makefile:
#!/bin/bash
# Check whether file $1 is less than $2 days old.
[ $# -eq 2 ] || {
echo "Usage: $0 FILE DAYS" >&2
exit 2
}
FILE="$1"
DAYS="$2"
[ -f "$FILE" ] || exit 1 # doesn't exist or not a file
TODAY=$(date +%s)
TARGET=$(($TODAY - ($DAYS * 24 * 60 * 60)))
MODIFIED=$(date -r "$FILE" +%s)
(($TARGET < $MODIFIED))
Replace X with the max number of days that can pass before filelist.d is downloaded again:
filelist.d: force-make
./less-than-days $# X || command-to-update
.PHONY: force-make
force-make:
Now filelist.d depends on a .PHONY target, without being a phony itself. This means filelist.d is always out of date (phony targets are always "new"), but its recipe only updates the file periodically.
Unfortunately, this requires you to write the update command as a single command, and space may be a problem if it is long. In that case, I would put it in a separate script as well.
I'm making a Makefile that moves an output file (foo.o) to a different directory (baz).
The output file moves as desired to the directory. However since make won't recompile the output file if I type make again, mv gets an error when it tries to move the non-existent empty file to the directory baz.
So this is what I have defined in my rule make all after all compilation:
-test -e "foo.o" || mv -f foo.o ../baz
Unfortunately, I'm still getting errors.
Errors in Recipes (from TFM)
To ignore errors in a recipe line, write a - at the beginning of the
line's text (after the initial tab).
So the target would be something like:
moveit:
-mv foo.o ../baz
I notice nobody has actually answered the original question itself yet, specifically how to ignore errors (all the answers are currently concerned with only calling the command if it won't cause an error).
To actually ignore errors, you can simply do:
mv -f foo.o ../baz 2>/dev/null; true
This will redirect stderr output to null, and follow the command with true (which always returns 0, causing make to believe the command succeeded regardless of what actually happened), allowing program flow to continue.
+#[ -d $(dir $#) ] || mkdir -p $(dir $#)
is what I use to silently create a folder if it does not exist. For your problem something like this should work
-#[ -e "foo.o" ] && mv -f foo.o ../baz
-test -e "foo.o" || if [ -f foo.o ]; then mv -f foo.o ../baz; fi;
That should work
Something like
test -e "foo.o" && mv -f foo.o ../baz
should work: the operator should be && instead of ||.
You can experiment with this by trying these commands:
test -e testfile && echo "going to move the file"
test -e testfile || echo "going to move the file"
I faced the same problem and as I am generating files, they always have different time. Workaround is set the same time to the files: touch -d '1 June 2018 11:02' file. In that case, gzip generates the same output and same md5sum. In my scenario, I don't need the time for the files.