I am writing a simple Makefile for practicing .PHONY target
#!/bin/make
dir/%/compile:
#echo "The target '$# had been built'"
In my working directory, the directory layout is as below
├── dir
│ └── subdir
│ └── compile
└── Makefile
When I type make dir/subdir/compile, it claimed the target is up to date as below
make: `dir/subdir/compile' is up to date.
It's right.
But it still claimed the target is up to date even a .PHONY target is added as below
#/bin/make
.PHONY: dir/%/compile
dir/%/compile:
#echo "The target '$# had been built'"
As I know .PHONY can build the target unconditionally no matter the target exists or not
However, it seemed not works as it should be. Could anyone help me figure it out? Thanks
You cannot use patterns with .PHONY. Only true target names will work. You're just saying that the literal target named dir/%/compile is phony.
Related
I have the following structure:
├── configure.ac
├── dir1
│ └── Makefile.am
├── dir2
│ └── Makefile.am
├── Makefile.am
...
configure.ac:
AC_INIT([amhello], [1.0], [bug-report#address])
AM_INIT_AUTOMAKE([foreign -Wall -Werror])
AC_PROG_CC
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([
Makefile
dir1/Makefile
dir2/Makefile
])
# Libtool
AM_PROG_AR
LT_INIT
AM_EXTRA_RECURSIVE_TARGETS([foo])
AC_OUTPUT
The top-level Makefile.am
SUBDIRS = dir1 dir2
foo-start:
#echo "executing foo!"
make foo
Both dir1 and dir2 Makefile.am's look like this:
foo-local:
echo "I'm here! $$(basename $(CURDIR))"
Running this yields the output like this:
execute foo!
I'm here! dir1
I'm here! dir2
Is there a way to write the subdirectory's Makefiles so that I don't have to write the foo-local task twice?
Is there a way to write the subdirectory's Makefiles so that I don't have to write the foo-local task twice?
You can put the rule that you want to duplicate among your subdirectory Makefiles in its own file, and use Automake's include feature to incorporate it by reference into each of the subdirectory makefiles.
(source tree root)/subdir-common.am
foo-local:
echo "I'm here! $$(basename $(CURDIR))"
(source tree root)/dir*/Makefile.am
include $(top_srcdir)/subdir-common.am
That doesn't win you much in the example presented, but presumably that's a model of something more complicated that you want to do, where you may benefit from avoiding repetition.
Note that there is nothing special about the .am extension for the name of the included file. I use it for its value as a mnemonic, but it is not inherently meaningful to the Autotools.
I have a tree of files that looks like this:
.
├── Makefile
├── README.md
├── exercises
│ ├── 100-exercises.ipynb
│ ├── 200-exercises.ipynb
│ ├── 300-exercises.ipynb
│ └── 400-exercises.ipynb
├── notes
│ ├── 101-notes-pandas.ipynb
│ ├── 102-notes-matplotlib-1.ipynb
│ ├── 103-notes-numpy-scipy.ipynb
│ └── 104-notes-matplotlib-seaborn.ipynb
└── tasks
├── 101-tasks-pandas.ipynb
├── 102-tasks-matplotlib-1.ipynb
├── 103-tasks-numpy-scipy.ipynb
└── 104-tasks-matplotlib-seaborn.ipynb
I would like to add some targets that only operates on files according to patterns in their filename. For example:
make lecture-1
make lecture-1-notes
make lecture-1-exercises
make lecture-2
make notes
make exercises
...
etc.
where lecture-1 refers to the set of targets that have a filename beginning with a 1 e.g. tasks/101-tasks-pandas.ipynb - to be clear the patterns are:
notes -> ./notes/*
exercises -> ./exercises/*
tasks -> ./tasks/*
lecture-1 -> ./*/1[0-9][0-9]*.ipynb
lecture-2 -> ./*/2[0-9][0-9]*.ipynb
lecture-1-notes -> ./notes/1[0-9][0-9]*.ipynb
The long way would be to have a separate target for each but I feel like there must be some kind of pattern/regex matching that can be done to avoid this.
EDIT:
For more information on the operations done on each target, I have an executable command which basically converts the IPython notebook to HTML. This is stored as a variable in make called RENDER_HTML
e.g. At the moment to render everything in the notes folder I have the following sections in my Makefile:
RENDER_HTML=jupyter nbconvert --execute --to html
NOTES_TARGETS=$(wildcard ./notes/*.ipynb)
...
.PHONY: notes
notes: ${NOTES_TARGETS}
#mkdir -p $#/html/
${RENDER_HTML} $^
#mv $#/*.html $#/html/
It would be a lot easier for us to help you if you provided example of what kinds of rules you want and what they would do: maybe implementing rules for one of these by hand that can serve as an example.
Without having any idea what the targets and prerequisites are, what I suggest is that you use recursive make to compute a list of targets to build; something like this:
lecture-%:
$(MAKE) $(patsubst ???,???,$(wildcard */$(*)*))
I just used ??? here since you don't provide any information on how the source files are translated to targets: you'll have to do that part yourself :).
If you can define a rule that builds a single output file, like this:
notes/%:
#mkdir -p $(#D)/html/
${RENDER_HTML} $#.ipynb
#mv $(#D)/*.html $(#D)/html/
then you can do this:
lecture-%:
$(MAKE) $(patsubst %.ipynb,%,$(wildcard */$(*)*))
The mv command there confuses me somewhat (seems like there should be a better way to do that for sure) but it's what you have in your question so I guess it's right.
I'm suspicious that this won't work for you, depending on the answer to my question above. If the render command needs to see ALL the files (for example to build an index.html or something) then I don't quite understand how you want this to work, when it builds only some of the files. Basically, the problem is still under-specified to allow us to give a working solution. But maybe there's enough info here to get you started.
Suppose you have the following project structure:
.
├── Makefile
└── src
└── 1.py
The program 1.py creates multiple (0, 1, or more) files in the directory build/1. This generalizes to arbitrary numbers, i.e. a program x.py where x is some natural number would create multiple files in the directory build/x. The project can consist of many python(3) files.
A makefile for the specific scenario above could look like this:
PYTHON_FILES := $(shell find src -name '*.py')
TXT_FILES := build/1/test.txt
.PHONY: clean all
all: $(TXT_FILES)
build/1/test.txt: src/1.py
mkdir -p build/1
touch build/1/test.txt # emulates: python3 src/1.py
echo "success!"
clean:
rm -rf build
Running make with the above project structure and makefile results in the following project structure:
.
├── Makefile
├── build
│ └── 1
│ └── test.txt
└── src
└── 1.py
How do I generalize the rule head build/1/test.txt: src/1.py to handle projects with any number of python programs (or, equivalently, build subdirectories) and any number of output files per python program?
You can generalized the existing rule to work on ANY python code in src. Use '%' in the pattern rule, use '$* to refer to the number in the action list.
The rule will re-run the test, whenever the python test is modified. It will record "success" only if the python test indicate completion without an error.
Update 2019-11-24: Generalized the test to handle N tests, each generating multiple files. With rebuild.
Note 1: Make need a way to know if the python test passed without ANY failure. The solution assume non-zero exit code from the python code, or that there is another way to tell if all tests have passed.
Note 2: The done file capture the list of test files generated in the folder (excluding the test.done itself). This can be used to verify that NO output file was removed, if needed, in a separate target to compensate the the lack of explicit files generated by the process
TASK_LIST=1 2 3 4
all: ${TASK_LIST:%=build/%/task.done}
build/%/task.done: src/%.py
mkdir -p build/$*
touch build/$*/test.txt # emulates: python3 src/1.py
# Run script src/%.py should return non-zero exit on failure.
ls build/$* | grep -xv "$(#F)" > $#
touch $# # Mark "success!"
GNU Make documentation: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html
I have a very simple Makefile that just shells out to another Makefile:
all:
cd src && make all
My directory structure (the Makefile is in the top-level directory):
[I] mqudsi#php ~/bbcp> tree -d
.
├── bin
│ └── FreeBSD
├── obj
│ └── FreeBSD
├── src
└── utils
This works just fine under Linux, but under FreeBSD, it gives an error about src not being found.
To debug, I updated the Makefile command to pwd; cd src && make all and I discovered that somehow when I run make in the top-level directory, it is being executed under ./obj instead, meaning it's looking for ./obj/src/ to cd into.
Aside from the fact that I have no clue why it's doing that, I presumed for sure that calling gmake instead of make under FreeBSD would take care of it, but that wasn't the case (and I'm relieved, because I can't believe there is that huge of a difference between BSD make and GNU make in terms of core operation).
The odd thing is, deleting obj makes everything work perfectly. So in the presence of an obj directory, make cds into ./obj first; otherwise it executes as you'd expect it to.
Answering my own question here.
From the FreeBSD make man page:
.OBJDIR A path to the directory where the targets are built. Its
value is determined by trying to chdir(2) to the follow-
ing directories in order and using the first match:
1. ${MAKEOBJDIRPREFIX}${.CURDIR}
(Only if `MAKEOBJDIRPREFIX' is set in the environ-
ment or on the command line.)
2. ${MAKEOBJDIR}
(Only if `MAKEOBJDIR' is set in the environment or
on the command line.)
3. ${.CURDIR}/obj.${MACHINE}
4. ${.CURDIR}/obj
5. /usr/obj/${.CURDIR}
6. ${.CURDIR}
Variable expansion is performed on the value before it's
used, so expressions such as
${.CURDIR:S,^/usr/src,/var/obj,}
may be used. This is especially useful with
`MAKEOBJDIR'.
`.OBJDIR' may be modified in the makefile via the special
target `.OBJDIR'. In all cases, make will chdir(2) to
the specified directory if it exists, and set `.OBJDIR'
and `PWD' to that directory before executing any targets.
The key part being
In all cases, make will chdir(2) to specified directory if it exists, and set .OBJDIR'PWD' to that directory before executing any targets.
By contrast, the GNU make manual page makes no such reference to any sort of automatic determination of OBJDIR, only that it will be used if it is set.
The solution was to override the OBJDIR variable via the pseudotarget .OBJDIR:
.OBJDIR: ./
all:
cd src && make
clean:
cd src && make clean
An alternative solution is to prefix the cd targets with ${CURDIR}, which isn't modified after the chdir into OBJDIR.
I don't get why gmake behaved the same way, however. That feels almost like a bug to me.
I'm trying to understand the Makefile that is automatically produced by sphinx-quickstart. Here it is:
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = myproj
SOURCEDIR = source
BUILDDIR = build
.PHONY: help Makefile
%: Makefile
#$(SPHINXBUILD) -M $# "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
The lines that confuse me are:
.PHONY: help Makefile
%: Makefile
I think I understand:
The % target means capture anything (wildcard). E.g., if I typed make html, % would capture html.
.PHONY Makefile means that make shouldn't look for a file called Makefile in its directory, thus, shouldn't check the file's modified time to determine whether or not to run a rule.
I don't understand:
Why Makefile is listed as a prerequisite for the target %. The way I interpret this is:
The target rule captured by % should run when the Makefile is changed.
But that doesn't make any sense in the context. What I would expect is:
The target rule captured by % should run when the source files for the project documentation or the API source files have changed.
Directory structure
.
├── build
├── Makefile
├── source
└── utils
The .PHONY: foo has the effect that foo is never to be considered up-to-date. (but see https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html for the more detailed explanations: the main use is for targets which are not filenames)
If you then have bar: foo, the rules for bar target will always be executed on make bar because the target depends upon foo but foo is considered never up-to-date. This can also be achieved with declaring bar target to be PHONY itself.
The problem with the catch-all % target was in case the repertory where the Makefile is located contained a repertory or a file having same name as a Sphinx builder. Say for example there is an html or a man in repertory where Makefile is located: then make html will not do anything if % has no dependencies, because html is then a file or repertory with no dependencies, hence never to get updated.
Thus the % was made to depend on Makefile pseudo target, and Makefile itself declared PHONY so that it is considered never up-to-date.(*) Even if repertory contains a file html then make html will get executed (and html repertory in build dir modified; the html in Makefile repertory will not be modified).
(*) edit: I had forgotten the exact details: Makefile is always considered a target, see a surprising (?) behaviour of GNU Make when using ``%`` as target. For reasons explained here % was made to depend upon Makefile, and the Makefile was declared PHONY in fact to avoid make complaining about circular dependency...
The idea is that the Makefile should not contain the hard-coded list of all possible builders: else they could have been declared PHONY targets individually, but then Sphinx maintainers would have to worry about keeping the Makefile template up-to-date when adding a new builder. It would also cause problems when projects keep same Makefile but a new Sphinx release adds a new builder.
The Makefile now created by sphinx-quickstart does not have to be modified if a new builder is added to Sphinx. It is of course certain that never will Makefile be the name of a builder...