I have a simple Makefile that will produce a file
all: build/foo.bin
build/foo.bin: foo.c
gcc $< -o $#
Works great and produces build/foo.bin as expected. If I then do a another make it will say make: Nothing to be done for 'all'. That's expected.
I then do rm build/foo.bin && make and it rebuilds the file. But if I do a echo "Modified" > build/foo.bin make doesn't think that anything has changed make: Nothing to be done for 'all'.
How can I write the rules of the Makefile to re-create the build/foo.bin if the binary ever gets modified outside of the Makefile?
Make compares timestamps between two files. If the target file exists and its timestamp is newer than all of the prerequisites' timestamps, then make decides the target is up to date and it doesn't need to do anything.
Make doesn't maintain some kind of database of timestamps on its own: it relies on the filesystem for that. So make cannot detect when a file changes from what it previously contained. It can only detect when some other file changed after the target file was last updated.
In short, make cannot do what you want it to do, using its standard methods.
If you want to do that you'll have to get complicated and create a way to turn the behavior you want to detect into a file with a timestamp, that make can compare.
One way to do this would be to keep the md5sum of the file in another file, then compare it and update the file only if it's changed. You can try this (I didn't test it):
build/foo.bin: foo.c checksum.out
gcc $< -o $#
md5sum $# > checksum.out
touch $#
checksum.out: FORCE
md5sum build/foo.bin > checksum.tmp; cmp $# checksum.tmp || cp checksum.tmp $#
FORCE: ;
Basically, the FORCE is there to require the md5sum check to always run, but then if the checksum doesn't actually change it doesn't update the output file which means that build/foo.bin won't be rebuilt (at least not because checksum.out is updated).
Related
#!make
.PRECIOUS: a/b/c/%_pqr
a/b/c/%_pqr:
echo $#
touch $#
.PRECIOUS: a/b/c/%_abc
a/b/c/%_abc: a/b/c/pqr_pqr
echo $#
touch $#
When I run first time, it honors its dependency:
make a/b/c/pqr_abc -f 1.mk
echo a/b/c/pqr_pqr
a/b/c/pqr_pqr
touch a/b/c/pqr_pqr
echo a/b/c/pqr_abc
a/b/c/pqr_abc
touch a/b/c/pqr_abc
But if I delete "a/b/c/pqr_pqr" file and then run this target again, it doesn't do the needful:
rm -f a/b/c/pqr_pqr; make a/b/c/pqr_abc -f 1.mk
make: `a/b/c/pqr_abc' is up to date.
As per my understanding, it should run both targets again.
Please somebody help me.
Both your targets are implicit and therefore intermediate. As per documentation:
The first difference is what happens if the intermediate file does not exist. [...] But if b is an intermediate file, then make can leave well enough alone. It won’t bother updating b, or the ultimate target, unless some prerequisite of b is newer than that target or there is some other reason to update that target.
Therefore since pqr_abc already exists and pqr_pqr is an intermediate file and has no prerequisites (so no prerequisites have changed), it will not be regenerated.
As per make's understanding pqr_pqr would need to be created only to generate pqr_abc from it (this is the definition of intermediate file), and since pqr_abc already exists, there is no need to regenerate it.
If you want the file to be regenerated every time, it has to be mentioned explicitly, so that it is no longer considered an intermediate file, i.e.:
$ cat Makefile
a/b/c/%_pqr:
touch $#
a/b/c/%_abc: a/b/c/pqr_pqr
touch $#
a/b/c/pqr_pqr:
The last line defines explicit target, but it does not define recipe, which will still be invoked from implicit rule. Note that it will no longer be considered intermediate and will not be automatically removed, so .PRECIOUS directives are also not needed.
Output:
$ make a/b/c/pqr_abc
touch a/b/c/pqr_pqr
touch a/b/c/pqr_abc
$ make a/b/c/pqr_abc
make: 'a/b/c/pqr_abc' is up to date.
$ rm a/b/c/pqr_pqr
$ make a/b/c/pqr_abc
touch a/b/c/pqr_pqr
touch a/b/c/pqr_abc
I need to do a makefile for run some programs. Every time I run that script all the file are processed also if the file are not changed. I'm sure there is a problem on my code but I don't understand where I made the mistakes.
RDIR=RAW
OUTDIR=Fusion_res/kallisto
RFILES:=$(wildcard $(RDIR)/*_R1_001.fastq.gz)
DATABASE=/home/sbsuser/databases/Kallsto_hg38_87
OUTFILE=$(patsubst %_R1_001.fastq.gz,%_R2_001.fastq.gz,$(RFILES))
OUTKAL=$(patsubst $(RDIR)/%_R1_001.fastq.gz,$(OUTDIR)/%,$(RFILES))
.PHONY: clean all
all: $(OUTFILE) $(RFILES) $(OUTDIR) $(OUTKAL)
#$(OUTKAL) $(OUTFILE): $(RDIR)/%._R1_001.fastq.gz
# echo "kallisto quant -i" $(DATABASE)/transcripts.idx -b 100 -o $# --fusion $< $(OUTFILE)
$(OUTDIR)/%: $(RDIR)/%_R1_001.fastq.gz $(OUTFILE)
kallisto quant -i $(DATABASE)/transcripts.idx -b 100 --fusion --rf-stranded -o $# $(RDIR)/$*_R1_00
1.fastq.gz $(RDIR)/$*_R2_001.fastq.gz
$(OUTDIR):
mkdir -p $(OUTDIR)
clean::
$(RM) -rf $(OUTDIR)
I suppose if the found some change on the input file and on the output execute the command. I don't know why every time force re-run. In some case Is that I want but I wan to also if there is some new input execute only that.
Thanks so much
A couple of things:
1) $(OUTDIR)/% is dependent on $(OUTFILE) (which is a list of all outfiles). Therefore if you change any one of the OUTFILEs, you make everything in $(OUTDIR)/% obsolete. I believe what you want is this:
$(OUTDIR)/%_R1_001.fastq.gz: $(RDIR)/%_R2_001.fastq.gz
.... (rules to make out/R1 from raw/R2
$(RDIR)/%_R2_001.fastq.gz: $(RDIR)/%_R1_001.fastq.gz
.... (rules to make R2 from R1
This makes each file dependent only on the files that effect it.
2) you have the target all dependent on $(OUTDIR) which is a directory. If you use parallel make, it may generate the $(OUTDIR) after it generates the other dependencies of all: (some of which would depend on $(OUTDIR) being created). What you want there is to remove all's dependency on $(OUTDIR), and add the line:
$(OUTFILE) : | $(OUTDIR)
Notice the |, which means order only (don't consider $(OUTFILE) out of date if $(OUTDIR) is newer. This is important, as a directory's timestamp is updated each time a file in the directory is changed, and so it tends to be newer than its contents.
My Makefile downloads a number of third-party files if they are not locally available.
CLOSURE_VERSION=20161024
CLOSURE_BASE_URL="http://dl.google.com/closure-compiler"
build/bin/closure-compiler.jar: build/src/hashes/closure-compiler-${CLOSURE_VERSION}.tar.gz.sha256
download-if-sha-matches <$< >$#.tar.gz \
${CLOSURE_BASE_URL}/compiler-${CLOSURE_VERSION}.tar.gz
tar -zxf $#.tar.gz closure-compiler-v${CLOSURE_VERSION}.jar
mv closure-compiler-v${CLOSURE_VERSION}.jar $#
rm $#.tar.gz
Here, build/src/hashes/closure-compiler-${CLOSURE_VERSION}.tar.gz.sha256 is the saved hash of the version of the file which we already know is correct.
download-if-sha-matches <hash >outfile url downloads the url and compares its hash to stdin, failing if they don't match.
This recipe works except that it always runs, even if build/bin/closure-compiler.jar already exists. Naturally, its timestamp is later than that of $< so I would expect this to not execute the recipe the second time I run make.
What have I gotten wrong?
Looks like tar -x preserves the timestamps of the contained files.
Add this to the recipe.
touch $#
I have a metadata dotfile which I store alongside my code (in each dir of a large project). This metadata file has a list of files in each directory that satisfies a particular constraint (not being auto-generated). This metadata is sincluded in Makefile.
I want to dep other targets (the auto-generated files) on this metadata file. If ever the list of "real" code files in a dir changes, update this metadata file, which will in turn cause the auto-generated files (plural) to be re-made.
I have a rule for the metadata file, and when that rule fires make restarts correctly. But I can't quite figure how to describe what I want. I want the rule to run but only consider $# as having been changed iff I actually touch the file. I can't use the timestamp of the dir because the act of auto-generating file A causes the timestamp to change, which triggers the need to re-generate file B, which cause the timestamp to change ...
I feel like I am missing something obvious, but I can not put my finger on it...
all: prep my-bin
# For demonstration purposes.
prep: real-code.foobar
real-code.foobar:
#touch real-code.foobar
my-bin: meta real-code.foobar genfile-A.foobar genfile-B.foobar genfile-C.foobar
touch $#
meta: .
F=$$(ls *.foobar | grep -v genfile); \
echo "FILES := $$F" > $#
sinclude meta
genfile-A.foobar: meta
touch $#
genfile-B.foobar: meta
touch $#
genfile-C.foobar: meta
touch $#
clean:
rm -f *.foobar my-bin meta
You could update the meta in a shell script at the beginning of the Makefile:
$(shell ls *.foobar | grep -v genfile > meta.tmp; \
diff -q meta.tmp meta && mv meta.tmp meta || rm meta.tmp)
This way the timestamp of meta is only updated if it changes, and it is updated before make has decided which rules to run (meaning dependencies of meta will not automatically rerun).
John
I have a makefile that uses a source file from the internet. There are two locations where the file resides, neither of which I consider very dependable, so I also keep a local copy. So the relevant lines of my makefile look like:
src.c:
wget -nv http://location.com/$# || wget -nv http://otherplace.com/$# || cp local/$# .
src.o: src.c
$(CC) -o $# $<
Is this the "right way" to do this? What if there are multiple steps in each different way of creating the target - how do I tell make "Try A. If A fails, try B. If B fails, ..."?
The right thing to do is this:
.PHONY: phony
src.c: phony
if (wget -nv http://location.com/$# -O $#.temp) && ! diff $#.temp $# >/dev/null; then \
mv $#.temp $#; \
fi
I shortened your command to a single wget but you can put whatever you want there, including a sequence of ||s to achieve "try this, if not, try that etc". Just make sure it outputs to a temporary file (and does not hang indefinitely !) .
It is in fact important to use phony here, and not only .PHONY. Can you see why?
Also, with this method, there is no longer a need to keep another "local" copy and/or use cp. Your target src.c is your "local copy" - the latest one you were able to successfully get from the Internet.