I have a very simple Makefile that just shells out to another Makefile:
all:
cd src && make all
My directory structure (the Makefile is in the top-level directory):
[I] mqudsi#php ~/bbcp> tree -d
.
├── bin
│ └── FreeBSD
├── obj
│ └── FreeBSD
├── src
└── utils
This works just fine under Linux, but under FreeBSD, it gives an error about src not being found.
To debug, I updated the Makefile command to pwd; cd src && make all and I discovered that somehow when I run make in the top-level directory, it is being executed under ./obj instead, meaning it's looking for ./obj/src/ to cd into.
Aside from the fact that I have no clue why it's doing that, I presumed for sure that calling gmake instead of make under FreeBSD would take care of it, but that wasn't the case (and I'm relieved, because I can't believe there is that huge of a difference between BSD make and GNU make in terms of core operation).
The odd thing is, deleting obj makes everything work perfectly. So in the presence of an obj directory, make cds into ./obj first; otherwise it executes as you'd expect it to.
Answering my own question here.
From the FreeBSD make man page:
.OBJDIR A path to the directory where the targets are built. Its
value is determined by trying to chdir(2) to the follow-
ing directories in order and using the first match:
1. ${MAKEOBJDIRPREFIX}${.CURDIR}
(Only if `MAKEOBJDIRPREFIX' is set in the environ-
ment or on the command line.)
2. ${MAKEOBJDIR}
(Only if `MAKEOBJDIR' is set in the environment or
on the command line.)
3. ${.CURDIR}/obj.${MACHINE}
4. ${.CURDIR}/obj
5. /usr/obj/${.CURDIR}
6. ${.CURDIR}
Variable expansion is performed on the value before it's
used, so expressions such as
${.CURDIR:S,^/usr/src,/var/obj,}
may be used. This is especially useful with
`MAKEOBJDIR'.
`.OBJDIR' may be modified in the makefile via the special
target `.OBJDIR'. In all cases, make will chdir(2) to
the specified directory if it exists, and set `.OBJDIR'
and `PWD' to that directory before executing any targets.
The key part being
In all cases, make will chdir(2) to specified directory if it exists, and set .OBJDIR'PWD' to that directory before executing any targets.
By contrast, the GNU make manual page makes no such reference to any sort of automatic determination of OBJDIR, only that it will be used if it is set.
The solution was to override the OBJDIR variable via the pseudotarget .OBJDIR:
.OBJDIR: ./
all:
cd src && make
clean:
cd src && make clean
An alternative solution is to prefix the cd targets with ${CURDIR}, which isn't modified after the chdir into OBJDIR.
I don't get why gmake behaved the same way, however. That feels almost like a bug to me.
Related
Suppose you have the following project structure:
.
├── Makefile
└── src
└── 1.py
The program 1.py creates multiple (0, 1, or more) files in the directory build/1. This generalizes to arbitrary numbers, i.e. a program x.py where x is some natural number would create multiple files in the directory build/x. The project can consist of many python(3) files.
A makefile for the specific scenario above could look like this:
PYTHON_FILES := $(shell find src -name '*.py')
TXT_FILES := build/1/test.txt
.PHONY: clean all
all: $(TXT_FILES)
build/1/test.txt: src/1.py
mkdir -p build/1
touch build/1/test.txt # emulates: python3 src/1.py
echo "success!"
clean:
rm -rf build
Running make with the above project structure and makefile results in the following project structure:
.
├── Makefile
├── build
│ └── 1
│ └── test.txt
└── src
└── 1.py
How do I generalize the rule head build/1/test.txt: src/1.py to handle projects with any number of python programs (or, equivalently, build subdirectories) and any number of output files per python program?
You can generalized the existing rule to work on ANY python code in src. Use '%' in the pattern rule, use '$* to refer to the number in the action list.
The rule will re-run the test, whenever the python test is modified. It will record "success" only if the python test indicate completion without an error.
Update 2019-11-24: Generalized the test to handle N tests, each generating multiple files. With rebuild.
Note 1: Make need a way to know if the python test passed without ANY failure. The solution assume non-zero exit code from the python code, or that there is another way to tell if all tests have passed.
Note 2: The done file capture the list of test files generated in the folder (excluding the test.done itself). This can be used to verify that NO output file was removed, if needed, in a separate target to compensate the the lack of explicit files generated by the process
TASK_LIST=1 2 3 4
all: ${TASK_LIST:%=build/%/task.done}
build/%/task.done: src/%.py
mkdir -p build/$*
touch build/$*/test.txt # emulates: python3 src/1.py
# Run script src/%.py should return non-zero exit on failure.
ls build/$* | grep -xv "$(#F)" > $#
touch $# # Mark "success!"
GNU Make documentation: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html
I have a directory tree like this with some "shared targets" in the file rules.Makefile:
├── Makefile
├── rules.Makefile
└── my_subdir
└── Makefile
I would like to invoke these "shared targets" in both the Makefile(s) in the parent directory and the child directory.
Also the "custom targets" in the Makefile in the child directory should be callable from the Makefile in the parent directory.
For some reason I am able to call the targets in rules.Makefile only from the sibling Makefile (the one in the parent directory). When using relative paths in the Makefile in the child directory trying to access the rules.Makefile in the parent directory I get some errors.
The content of the Makefile in the parent directory:
RULES_MAKEFILE_PATH=$(PWD)/rules.Makefile
include $(RULES_MAKEFILE_PATH)
foo-parent:
#echo $(RULES_MAKEFILE_PATH)
The content of the Makefile in the child directory (please note that double dot ..):
RULES_MAKEFILE_PATH=$(PWD)/../rules.Makefile
include "$(RULES_MAKEFILE_PATH)"
foo-child:
#echo $(RULES_MAKEFILE_PATH)
When calling from the parent directory make foo-parent then I see the expected path.
When calling from the child directyr make foo-child then I see this error:
$ make foo-child
Makefile:9: "/<PARENT_PATH>/my_subdir/../rules.Makefile": No such file or directory
make: *** No rule to make target '"/<PARENT_PATH>/my_subdir/../rules.Makefile"'. Stop.
How can I make the relative paths work in the "child directory"?
Also how can I call the targets defined in the Makefile in child directory (e.g. foo-child) from the Makefile in the parent directory?
Well first, $(PWD) is not a special variable to make. It's just a normal variable, that's imported from your shell. So it will always have the same value everywhere in your makefile and in all included makefiles, it won't change just because you're including a makefile from a different directory.
Second, even for $(CURDIR) (which is a special variable and is set by make to be the current directory when make starts), it is never reset when you include a makefile from another directory.
And, all paths in include lines are evaluated based on the directory make was in when it started, not on a path relative to the currently-parsed makefile. So if Makefile includes foo/Makefile, then foo/Makefile has an include bar.mk, make will look for bar.mk not foo/bar.mk.
The point about $(PWD) above is true. However, if that is not a concern, you could still use the shell function to execute commands to get paths of another Makefile to include: $(shell pwd)
I did something similar for a project that benefited from having the same Makefile used in many places, using git rev-parse --show-toplevel.
MAKEFILE := $(shell git rev-parse --show-toplevel)/makefiles/example.def
include $(MAKEFILE)
I'm trying to understand the Makefile that is automatically produced by sphinx-quickstart. Here it is:
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = myproj
SOURCEDIR = source
BUILDDIR = build
.PHONY: help Makefile
%: Makefile
#$(SPHINXBUILD) -M $# "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
The lines that confuse me are:
.PHONY: help Makefile
%: Makefile
I think I understand:
The % target means capture anything (wildcard). E.g., if I typed make html, % would capture html.
.PHONY Makefile means that make shouldn't look for a file called Makefile in its directory, thus, shouldn't check the file's modified time to determine whether or not to run a rule.
I don't understand:
Why Makefile is listed as a prerequisite for the target %. The way I interpret this is:
The target rule captured by % should run when the Makefile is changed.
But that doesn't make any sense in the context. What I would expect is:
The target rule captured by % should run when the source files for the project documentation or the API source files have changed.
Directory structure
.
├── build
├── Makefile
├── source
└── utils
The .PHONY: foo has the effect that foo is never to be considered up-to-date. (but see https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html for the more detailed explanations: the main use is for targets which are not filenames)
If you then have bar: foo, the rules for bar target will always be executed on make bar because the target depends upon foo but foo is considered never up-to-date. This can also be achieved with declaring bar target to be PHONY itself.
The problem with the catch-all % target was in case the repertory where the Makefile is located contained a repertory or a file having same name as a Sphinx builder. Say for example there is an html or a man in repertory where Makefile is located: then make html will not do anything if % has no dependencies, because html is then a file or repertory with no dependencies, hence never to get updated.
Thus the % was made to depend on Makefile pseudo target, and Makefile itself declared PHONY so that it is considered never up-to-date.(*) Even if repertory contains a file html then make html will get executed (and html repertory in build dir modified; the html in Makefile repertory will not be modified).
(*) edit: I had forgotten the exact details: Makefile is always considered a target, see a surprising (?) behaviour of GNU Make when using ``%`` as target. For reasons explained here % was made to depend upon Makefile, and the Makefile was declared PHONY in fact to avoid make complaining about circular dependency...
The idea is that the Makefile should not contain the hard-coded list of all possible builders: else they could have been declared PHONY targets individually, but then Sphinx maintainers would have to worry about keeping the Makefile template up-to-date when adding a new builder. It would also cause problems when projects keep same Makefile but a new Sphinx release adds a new builder.
The Makefile now created by sphinx-quickstart does not have to be modified if a new builder is added to Sphinx. It is of course certain that never will Makefile be the name of a builder...
TL;DR: How can I use find in a Makefile in order to identify the relevant source files (e.g., all .c files)? I know how to use a wildcard but I'm not able to get find to work.
Longer version:
I'm putting together a Makefile as part of an exercise on shared libraries; I noticed that when I use the following lines to specify the source and object files (i.e., .c files) for my shared library, I get an error after running make (gcc fatal error: no input files):
SRC=$(find src/ -maxdepth 1 -type f -regex ".*\.c")
OBJ=$(patsubst %.c,%.o,$(SRC))
*rest-of-makefile*
However, it compiles correctly when I use wildcard instead of find:
SRC=$(wildcard src/*.c)
OBJ=$(patsubst %.c,%.o,$(SRC))
*rest-of-makefile*
(As reference, included below is confirmation that the find command does indeed return the intended file when run from the shell.)
What is the correct syntax for using the find command (in my Makefile) to search for my source files (if it's at all possible)?
(Why would I prefer to use find?: I like the fact that I can quickly double-check the results of a find statement by running the command from the shell; I can't do that with wildcard. Also, I'd like to rely on regexes if possible. )
As reference, below is the relevant tree structure. As you can see (from the second code-block below), running the find command as specified in the Makefile (i.e., from above) does indeed return the intended file (src/libex29.c). In other words, the issue described above isn't because of a syntax problem in the find options or the regex.
.
├── build
├── Makefile
├── src
│ ├── dbg.h
│ ├── libex29.c
│ └── minunit.h
└── tests
├── libex29_tests.c
└── runtests.sh
Results of running find from the . folder above:
~/lchw30$ find src/ -maxdepth 1 -type f -regex ".*\.c"
src/libex29.c
P.S. I know this post technically violates the rule that all posted code must compile - I just thought that including the entire code for the both the Makefile as well as the libex29.c source file would be overkill. Let me know if that's not the case - happy to post the files in their entirety, if folks prefer.
Make doesn't have a find function. You have to use the shell function to run find. Also you should always use := not = for shell (and wildcard, for that matter) for performance reasons. And you should put spaces around assignments in make, just for clarity:
SRC := $(shell find src/ -maxdepth 1 -type f -regex ".*\.c")
Also I don't see why you want to use find here. find is good if you want to search and entire subdirectory structure which contains more than one level, but wildcard is far more efficient for simple directory lookups.
I want to create a Makefile rule that runs whenever anything is changed inside a directory (which contains multiple source files in different languages, and at different subdirectory levels).
As an example, take this Makefile:
newest: src
touch newest
with a tree like:
src/
src/a
scr/subdir/
scr/subdir/c
First time I run make, newest is created all right. But if I now touch src/subdir/b, make does nothing.
Is it possible at all to create such a rule?
I think you would need to use something like FILES := $(shell find src -type f) and a rule of newest: $(FILES) to get the sort of behavior you want.