Consider the following makefile:
-include target.d
build:
%.d: %.c
#echo ">>> %.d: %.c"
touch $#
target.d: dependency
# #echo ">>> target.d: dependency"
dependency:
#echo ">>> dependency"
.PHONY: build
If you place this in a directory with two (empty) files, target.d and target.c and run make, then on one VM I use it produces the following output:
$ make
>>> dependency
>>> %.d: %.c
touch target.d
>>> dependency
>>> %.d: %.c
touch target.d
make: Nothing to be done for `build'.
On another VM, it loops infinitely. Both VMs are running Centos7, and both of them are using the same version of GNU make (3.82).
(Note: if you uncomment the commented out line below the target.d target, then both produce the exact same output; that behavior makes sense to me, at least)
I am aware that by adding the recipe to the target.d: dependency makes it prioritise that recipe over the generic one, while simply adding a dependency. But what I don't understand is why one system causes an infinite loop to occur while another extremely similar system does not.
What is the reason for this strange behaviour?
(EDIT: I found out I can simplify the makefile and still see the same behaviour)
I think the issue here is with having a target called target.d that is the same as the include file target.d such that when you edit target.d by touching it this somehow invokes an "undefined" behaviour. It seems like in this case it is triggering a re-call of the makefile. I can distil your problem down to this:
-include target.d
target.d: dep
touch target.d
dep:
#echo dep
With the only files that exist being: makefile. target.d is created on the first run of make.
$ ls
makefile
$ make
dep
touch target.d
dep
touch target.d
dep
touch target.d
:
etc
:
$ ls
makefile target.d
I do not believe it is correct to have a target that is also an item you include in your makefile. Usually with .d (e.g. auto generated dep files with gcc) you would generate these files in a compile rule like %.o: %.c ... which creates the .o and .d files. Then you have your include %.d type line. But you should not have a target to generate .d files directly. I do not think this is valid - please someone correct me if I am wrong here.
E.g. makefile that I think you intend:
-include target.d
.PHONY: build
build: target.o
# Simulates a compile line that generates and object file (.o) and dependency file (.d)
target.o: target.c dependency
touch target.o
touch target.d
.PHONY: dependency
dependency:
#echo ">>> dependency"
output:
$ ls
makefile target.c
$ make
>>> dependency
touch target.o
touch target.d
$ ls
makefile target.c target.d target.o
So as #code_fodder's answer suggested, the entire makefile can be simplified heavily to
-include a
a: b
touch a
b:
#echo b
There is a valid question as to whether or not the inclusion of a makefile that you create is a good idea; As stated in the comments above, this is a distillation of part of a build system that I am working with but do not have control over, so c'est la vie.
So there remains the question as to why this behaves differently on two different systems?
The problem turned out to be the following: one VM was run on a remote server, and the other VM was being run locally. In particular, my host machine is a mac and I was sharing some of the file system between the mac and the VM. It appears that the filesystem that I am using on my Mac only keeps timestamps to second precision, while that on the VMs is kept to sub-millisecond precision. As a consequence the Mac doesn't pick up the "updated" file.
I was able to see this on the VM that shares files with the mac by adding sleep commands of various lengths:
-include a
a: b
sleep 0.8
touch a
b:
#echo b
which would cause it to run a finite, but quasi-random number of times. Or, more simply, running make; make which yields
$ make; make
b
touch a
b
touch a
make: `a' is up to date.
b
touch a
make: `a' is up to date.
i.e. running make twice in short succession lets the second run see the updated timestamp of the first, and so it doesn't trigger a second time.
Related
I'm using a makefile to build some documents from markdown. Since I have various possible outputs, I'm using a pattern rule. And since the input documents may or may not have associated tables, they have a wildcard dependency for csv files with the same stem filename:
tables = $(wildcard $*_*.csv)
%.docx: %.md $(tables)
#echo building $#
#touch $#
This works fine the first time:
$ touch thing.md
$ touch thing_table.csv
$ make thing.docx
building thing.docx
$ ls thing.docx
thing.docx
However, if the table file is updated, make still thinks everything's up to date:
$ touch thing_table.csv
$ make thing.docx
make: `thing.docx' is up to date.
I think this must have something to do with the order in which make evaluates things, but I don't understand it well enough to figure out how to make this work.
I can do sort of what I want to do using a more "literal" pattern rule:
%.docx: %.md %_*.csv
#echo building $#
#touch $#
But of course this fails in the absence of appropriately-named csv files:
$ rm thing.docx thing_table.csv
$ make thing.docx
make: *** No rule to make target `thing.docx'. Stop.
Is there a way to specify a target dependency such that if a target is foo.docx the dependencies would be foo_*.csv, but have those dependencies be optional (i.e. the target still works if the files don't exist) AND have it appropriately update if those optional dependencies change?
Ok, it looks like secondary expansion will work here:
.SECONDEXPANSION:
%.docx: %.md $$(wildcard $$*_*.csv)
#echo building $#
#touch $#
I have a target that depends on an external library:
output.txt: library/build.txt main.txt
cat library/build.txt main.txt >output.txt
(This is just an example, my actual makefile is much more complicated)
However, library/build.txt is generated by a separate makefile.
I need to run the library makefile (to potentially update build.txt) before make checks if library/build.txt is newer than output.txt
I could just have it run $(MAKE) -C library, every time, but I'm hoping to only call that when it is needed (meaning, when you try to make a target which depends on library/build.txt)
Is this possible? (maybe there's a way to get a list of dependencies and then run something if that list contains library/build.txt?)
You can try things like :
FORCE:
library/build.txt: FORCE
#echo make to build $#
output.txt: library/build.txt main.txt
cat $^ > $#
I have a make target that depends on a variable, which contains both PHONY and real targets.
This target needs to depend only on the real targets in the variable.
How can I test a variable to determine if it is a PHONY or not, so I can filter them out?
(I can test for a file's existence inside the recipe, but I don't want my target to be triggered by execution of any of the PHONY targets.)
Thanks!
There is a way to do it, but I would strongly recommend against it. First of, phony targets can be also file targets. And there is no way to tell a phony file target from a non-phony file target.
It looks like the question implies that the phony targets the author wants to ignore are all non-file targets. In this case see the example below.
.PHONY: phony_target .FORCE
.FORCE:
ALL_TARGETS = phony_target file_target undetermined_target
-include detect_phony.inc
all: final_target
# All done
final_target: $(REAL_TARGETS)
# create $# triggered by $?
#touch $#
ifeq (,$(MAKE_RESTARTS))
# Generate the list of real file targets in make include file
detect_phony.inc: .FORCE
#echo 'REAL_TARGETS = ' `ls $(ALL_TARGETS) 2>/dev/null` > $# |:
endif
file_target:
touch $#
undetermined_target phony_target:
# process $#
clean:
rm -f file_target final_target
Here are the test results:
$make clean
rm -f file_target final_target
$ make
# create final_target triggered by
# All done
$ touch file_target
$ make
# create final_target triggered by file_target
# All done
$ make
# All done
As you can see it only triggers the final target when the file target is updated.
Before you criticize - Here are the flaws of this implementation:
make is always called twice, updating the generated detect_phony.inc include file at every run
if detect_phony.inc gets corrupted somehow, make execution will be locked by syntax errors, until you manually delete it.
it can't handle phony file targets as I mentioned before
if another generated include is added in this makefile that requires another restart before detect_phony.inc this functionality will break.
So it this method is hacky and has several gotchas. I would not use it in production environment. I would insist on changing the top level Makefile first.
Hopefully this is a very simple question. I have a makefile pattern rule that looks like this:
%.so : %.f %.pyf
f2py -c -L${LAPACK_DIR} ${GRASPLIBS} -m $* $^ ${SOURCES} --opt='-02' --f77flags='-fcray-pointer' >> silent.txt
I want the makefile to build a number of .so files, so I tried to get it to build two files (radgrd_py.so and lodiso_py.so) by doing this:
radgrd_py.so lodiso_py.so:
%.so : %.f %.pyf
f2py -c -L${LAPACK_DIR} ${GRASPLIBS} -m $* $^ ${SOURCES} --opt='-02' --f77flags='-fcray-pointer' >> silent.txt
and then tried this:
radgrd_py.so:
lodiso_py.so:
%.so : %.f %.pyf
f2py -c -L${LAPACK_DIR} ${GRASPLIBS} -m $* $^ ${SOURCES} --opt='-02' --f77flags='-fcray-pointer' >> silent.txt
But in each case, it only builds the first target that I specify. If I run 'make radgrd_py.so' it works fine, I'm just not sure how to specify a list of files that need to be built so I can just run 'make'.
The usual trick is to add a 'dummy' target as the first that depends on all targets you want to build when running a plain make:
all: radgrd_py.so lodiso_py.so
It is a convention to call this target 'all' or 'default'. For extra correctness, let make know that this is not a real file by adding this line to your Makefile:
.PHONY: all
Best way is to add:
.PHONY: all
.DEFAULT: all
all: radgrd_py.so lodiso_py.so
Explanations:
make uses the first target appearing when no .DEFAULT is specified.
.PHONY informs make that the targets (a coma-separated list, in fact) don't create any file or folder.
all: as proposed by schot
I'm trying to write a makefile to produce several output files for each of several sources, using pattern rules.
I have the following Makefile (GNU Make 3.8.1):
all : foo.all bar.all
%.all : %.pdf %.svg
#echo Made $*
%.pdf :
touch $#
%.svg :
touch $#
.PHONY: foo.all bar.all
Since *.all do not represent real output files, I tried marking them as .PHONY. However, running make then doesn't work:
$ ls
Makefile
$ make
make: Nothing to be done for `all'.
According to make -d:
No implicit rule found for `all'.
Considering target file `foo.all'.
File `foo.all' does not exist.
Finished prerequisites of target file `foo.all'.
Must remake target `foo.all'.
Successfully remade target file `foo.all'.
Considering target file `bar.all'.
File `bar.all' does not exist.
Finished prerequisites of target file `bar.all'.
Must remake target `bar.all'.
Successfully remade target file `bar.all'.
Finished prerequisites of target file `all'.
Must remake target `all'.
Successfully remade target file `all'.
make: Nothing to be done for `all'.
which seems to be pretending to run the %.all rules, but skipping the bodies.
But with the .PHONY line commented out, Make runs the targets, but then spontaneously decides to delete the output files:
$ make
touch foo.pdf
touch foo.svg
Made foo
touch bar.pdf
touch bar.svg
Made bar
rm foo.pdf foo.svg bar.pdf bar.svg
According to make -d, it says:
Removing intermediate files...
Minimal example
A minimal example giving anomalous behavior:
%.all: %.out
#echo Made $*
%.out:
touch $#
I expect running make somefile.all to cause it to create the file somefile.out, but it gets deleted:
$ make somefile.all
touch somefile.out
Made somefile
rm somefile.out
Keeping make from deleting intermediary files
I recommend against using .PRECIOUS (see below as to why). Using .SECONDARY would preserve the .out files:
TARGETS=foo bar
all: $(TARGETS:=.all)
%.all: %.out
#echo Made $*
%.out:
touch $#
.SECONDARY: $(TARGETS:=.out)
$(TARGETS:=.all) just appends .all to all names in TARGETS. $(TARGETS:=.out) appends .out. We apparently cannot use %.out as a target of .SECONDARY. These just save having to relist all targets individually.
I prefer to not use .PRECIOUS for this because the documentation says
if make is killed or interrupted during the execution of their recipes, the target is not deleted.
This can leave corrupted files in the file system. Here's an example.
all: foo.all bar.all
%.all: %.out
#echo Made $*
%.out:
sh -e -c 'echo "{1, 2, 3" > $#; FAIL!; echo "}" >> $#'
.PRECIOUS: %.out
The FAIL! command simulates a tool that crashes in the middle of its work. Here's a shell session working with the Makefile above:
$ ls
Makefile
$ make
sh -e -c 'echo "{1, 2, 3" > foo.out; FAIL!; echo "}" >> foo.out'
sh: 1: FAIL!: not found
make: *** [foo.out] Error 127
$ cat foo.out
{1, 2, 3
Yikes... my foo.out file is incomplete. Let's try making again:
$ make
Made foo
sh -e -c 'echo "{1, 2, 3" > bar.out; FAIL!; echo "}" >> bar.out'
sh: 1: FAIL!: not found
make: *** [bar.out] Error 127
$ cat *.out
{1, 2, 3
{1, 2, 3
Make is none the wiser about files left around by earlier runs so when you run make again, it will take the corrupted files at face value. foo.out was not remade (despite the "Made foo" message) because it already exists and the Makefile went straight to trying to make bar.
.SECONDARY makes it so that:
The targets which .SECONDARY depends on are treated as intermediate files, except that they are never automatically deleted.
This means they are never automatically deleted just because they are intermediate files. The default make behavior of deleting targets that were being rebuilt if the tool rebuilding them crashed is not affected.
Using .PHONY with pattern rules
It seems though that .PHONY works only for targets that are explicit, not inferred. I've not found documentation confirming this. However, this works:
TARGETS:=foo bar
TARGETS_all:=$(TARGETS:=.all)
.PHONY: all
all: $(TARGETS_all)
.PHONY: $(TARGETS_all)
$(TARGETS_all): %.all: %.out
#echo Made $*
%.out:
touch $#
.SECONDARY: $(TARGETS:=.out)
In this rule $(TARGETS_all): %.all: %.out $(TARGETS_all): gives the list of targets to which the pattern can be applied. It makes foo.all and bar.all explicit targets. Without this, they would be inferred targets.
You can test that it works by creating file called foo.all in your directory and run make over and over. The foo.all file has no effect on make.
Your somefile.out files are considered intermediate by GNU make, which is why they are automatically deleted in your example. You can instruct GNU make to preserve these files by use the of .PRECIOUS special target, like this:
%.all: %.out
#echo Made $*
%.out:
touch $#
.PRECIOUS: %.out