I'm using GNU Make to build three different editions of a static html document.
I use Less as a CSS preprocessor.
My directory structure looks like this:
Makefile
160x600/style.less
300x250/style.less
728x90/style.less
This is my Makefile:
LESSC=lessc -x # use -x for debugging
.PHONY: all clean
all: 160x600 300x250 728x90
%.css: %.less
$(LESSC) $< > $#
160x600: 160x600/style.css
300x250: 300x250/style.css
728x90: 728x90/style.css
clean:
rm -f 160x600/*.css
rm -f 300x250/*.css
rm -f 728x90/*.css
This way, I can use make 160x600 to build style.css from style.less.
But I don't want to explicitly list a target rule for each directory. Instead, I tried adding this rule instead of the three directory specific ones:
%: %/style.css
But that does not work. I assume it's clear from that example what my goal is. Is there a way to accept any directory as a target, so that I just have to list the directory names in the all: rule?
use static pattern rule:
res_dirs = 160x600 300x250 728x90
$(res_dirs): %: %/style.css
Related
My goal is the following: I have a directory src which contains markdown files (.md). I want to run a command on each of these files so that the comments are removed and the edited files are stored in a separate directory. For this I want to use make.
This is the Makefile I have:
.PHONY: clean all
BUILD_DIR := build
SRC_DIRS := src
SRCS := $(shell find $(SRC_DIRS) -name *.md)
DSTS := $(patsubst $(SRC_DIRS)/%.md,$(BUILD_DIR)/%.md,$(SRCS))
all: $(DSTS)
# The aim of this is to remove all my comments from the final documents
$(DSTS): $(SRCS)
pandoc --strip-comments -f markdown -i $< -t markdown -o $#
clean:
rm $(BUILD_DIR)/*.md
While this works in general, I noticed that the command is executed on all files, even though I changed only one single file.
Example: I have 3 Files src/a.md, src/b.md and src/c.md. Now I run make and all files are correctly generated in the build folder. Now I only edit c.md and run make again. I would expect that make only "compiles" src/c.md anew but instead all three files are compiled again. What am I doing wrong?
Your line
$(DSTS): $(SRCS)
is saying ‘All of the DSTS depend on all of the SRCS’, so whenever any one of the $(SRCS) is newer than any of the $(DSTS), this pandoc action will be run.
That's not what you want to express. What you want is something more like
$(BUILD_DIR)/%.md: $(SRC_DIRS)/%.md
pandoc --strip-comments -f markdown -i $< -t markdown -o $#
all: $(DSTS)
That says that all of the $(DSTS) should be up to date, and the pattern rule teaches Make what each one depends on, and how to build it, if it is out of date.
(As a general point, looking your original rule, it's rarely the right thing to do to have multiple targets in a rule, as you have with $(DSTS); also note that in your original, $< always refers only to the first of the dependencies in $(SRCS))
Appreciating that the title is not quite on "target", how can I make it so that when I call make at the top level, it will recursively call the makefiles in the sub-directories?
Having been intrigued by the Kconfig pattern, to learn it, I've applied it to a mark down to pdf generator.
The recursive Makefile resides in ./scripts/Makefile.boilerplate and is defined:
HEADER=$(wildcard section-header.md)
.PHONY:all clean $(md-sub-y)
all clean: $(md-sub-y)
all: $(TARGET)
clean:
# $(RM) $(TARGET)
$(TARGET): $(HEADER) $(md-y) | $(md-sub-y)
# cat /dev/null $^ > $#
$(md-sub-y):
# $(MAKE) -C $(#D) TOPDIR=$(TOPDIR) $(MAKECMDGOALS)
I'm likely using the order-only prerequisite for the $(TARGET) target inappropriately, but it solved a minor problem.
In each directory there is a unique KConfig file (not shown), which lists CONFIG_{OPTION} macros that evaluate to either y or n. Then each directory contains a Makefile that has the form:
include Kconfig
md-$(CONFIG_INTRODUCTION)+= Introduction.md
md-$(CONFIG_FW_UPDATE)+= FW-update.md
md-sub-$(CONFIG_CHAPTERS)+= Chapters/Chapters.md
md-$(CONFIG_CHAPTERS)+= Chapters/Chapters.md
md-$(CONFIG_EXAMPLES)+= Examples.md
md-$(CONFIG_APPENDIX_I)+= Appendix-I.md
md-$(CONFIG_APPENDIX_II)+= Appendix-II.md
md-$(CONFIG_APPENDIX_III)+= Appendix-III.md
include ${TOPDIR}/scripts/Makefile.boilerplate
And finally, the very top level makefile is (abbreviated):
.PHONY: all clean pdf embedded_html
all clean test: JsonAPI/JsonAPI.md
all: pdf embedded_html
pdf: $(MARKDOWN_FILES:.md=.pdf)
embedded_html: $(MARKDOWN_FILES:.md=.html)
MAKEFLAGS += --no-print-directory
clean:
# $(RM) *.pdf *.html
JsonAPI/JsonAPI.md:
# $(MAKE) -C $(#D) TOPDIR=${CURDIR} $(MAKECMDGOALS)
%.html:%.md
# pandoc -s --toc -c /var/www/css/bootstrap.css $< -f markdown -t html -s -o $#
%.pdf:%.md
# pandoc --read=markdown --table-of-contents --toc-depth=3 --preserve-tabs --standalone --template=template.latex $(PANDOC_ENGINE)=pdflatex --listings -V geometry:margin=1in --highlight-style=pygments -H listing-setup.tex -r markdown+simple_tables+table_captions+yaml_metadata_block $< -o $#
If I call make on an unbuilt directory tree, it works fine. But there are a few problems I'm not sure how to address:
How can I ensure that if an updated .md deeply nested in the directory tree will cause the top level PDF file to be updated? Or, How can I force the makefile's in the sub-directories to be called?
The clean target at the top level is problematic, in that it doesn't recurse through the sub-directories. What do I need to do to remedy that?
Is there a better way to include the Makefile.boilerplate makefile, without having to define the TOPDIR on the $(MAKE) command line as I've done?
For 1, and 2, I'm guessing that an empty target dependency (FORCE:) will be required. And for 3, I've tried using $(CURDIR) but it was always evaluating to the directory the Makefile resided in, not the parent directory where the original make command was invoked.
Changing the md-sub-$(CONFIG_EEEE) macro definition to be just the directory was the key, and to make those targets have an empty rule.
Essentially, the per directory Makefile from above becomes:
include Kconfig
md-$(CONFIG_INTRODUCTION)+= Introduction.md
md-$(CONFIG_FW_UPDATE)+= FW-update.md
md-sub-$(CONFIG_CHAPTERS)+= Chapters/Chapters.md
md-$(CONFIG_CHAPTERS)+= Chapters
md-$(CONFIG_EXAMPLES)+= Examples.md
md-$(CONFIG_APPENDIX_I)+= Appendix-I.md
md-$(CONFIG_APPENDIX_II)+= Appendix-II.md
md-$(CONFIG_APPENDIX_III)+= Appendix-III.md
include ${TOPDIR}/scripts/Makefile.boilerplate
and the default Makefile.boilerplate changes the $(md-sub-y) target too:
$(md-sub-y): FORCE
# $(MAKE) -C $# TOPDIR=$(TOPDIR) $(MAKECMDGOALS)
FORCE:
And the top level makefile no longer needs $(#D) on the command line for the JsonAPI recipe, just $#.
I have a Makefile in which a couple of targets need to be made the same way, but one of the targets is a file whose basename is different from its prerequisite. Here is a minimal example:
ABOUT.html: README.md
help.html: help.md
%.html: %.md
pandoc --standalone --quiet -f gfm -H $(github-css) -o tmp.html $<
inliner -n < tmp.html > $#
rm -f tmp.html
With this Makefile, help.html gets made but ABOUT.html is never made. The reason, I presume, is because the base file name for %.html and %.md don't match up in the case of ABOUT.html because that target depends on README.md.
Is there a way to make this work without having to make a separate recipe for ABOUT.html?
One option is to create ABOUT.md symlink, so that your pattern rule works, by adding the following rule:
ABOUT.md : README.md
ln -s ${<F} ${#F}
You may like to avoid using the same temporary file in the recipe because that breaks in parallel builds. A better way is to use a unique temporary file for the target based on the target name:
%.html: %.md
pandoc --standalone --quiet -f gfm -H $(github-css) -o $#~ $<
inliner -n < $#~ > $#
rm -f $#~
I am trying to statically build HTML files that requires a markdown file and a meta file called "whatlinkshere" for the HTML file to demonstrate its back links.
I believe it can be effeciently done by a Makefile, by first generating all the "whatlinkshere" files. I don't think this can be done in parallel, because the program that generates these files needs to append to the whatlinkshere files, and there could be race conditions that I am not quite sure how to solve.
Once the "whatlinkshere" files are generated then if a markdown file is edited, say foo.mdwn to point to bar.mdwn, only foo.mdwn needs to be analysed again for "whatlinkshere" changes. And finally only foo.html and bar.html need to be rebuilt.
I am struggling to accomplish this in my backlinks project.
INFILES = $(shell find . -name "*.mdwn")
OUTFILES = $(INFILES:.mdwn=.html)
LINKFILES = $(INFILES:.mdwn=.whatlinkshere)
all: $(OUTFILES)
# These need to be all made before the HTML is processed
$(LINKFILES): $(INFILES)
#echo Creating backlinks $#
#touch $#
#go run backlinks.go $<
%.html: %.mdwn %.whatlinkshere
#echo Deps $^
#cmark $^ > $#
Current problems here is that *.whatlinkshere** aren't being generated on first run. My workaround is for i in *.mdwn; do go run backlinks.go $i; done. Furthermore there are not rebuilding as I want after editing a file as described earlier. Something is horribly wrong. What am I missing?
I think I finally understood your problem. If I understood well:
You have a bunch of *.mdwn source files.
You generate *.whatlinkshere files from your *.mdwn source files using the backlinks.go utility. But this utility does not produce foo.whatlinkshere from foo.mdwn. It analyzes foo.mdwn, searches for links to other pages in it and, for each link to bar it finds, it appends a [foo](foo.html) reference to bar.whatlinkshere.
From each foo.mdwn source file you want to build a corresponding foo.html file with:
$ cmark foo.mdwn foo.whatlinkshere
Your rule:
$(LINKFILES): $(INFILES)
#echo Creating backlinks $#
#touch $#
#go run backlinks.go $<
contains one error and has several drawbacks. The error is the use of the $< automatic variable in the recipe. It expands as the first prerequisite, that is probably always pageA.mdwn in your case. Not what you want. $^ expands as all prerequisites but it is not the correct solution because:
your go utility takes only one source file name, but even if it was accepting several...
...make will run the recipe several times, one per link file, which is a waste, and...
...as your go utility appends to the link files it will even be worse than a waste: back links will be counted several times each, and...
...if make runs in parallel mode (note that you can prevent this with make -j1 or by adding the .NOTPARALLEL: special rule to your Makefile, but it is a pity) there is a risk of race conditions.
Important: the following works only with a flat organization where all source files and HTML files are in the same directory as the Makefile. Other organizations are possible, of course, but they would require some modifications.
First option using multi-targets pattern rules
One possibility is to use a special property of make pattern rules: when they have several targets make considers that one single execution of the recipe produces all targets. For instance:
pageA.w%e pageB.w%e pageC.w%e: pageA.mdwn pageB.mdwn pageC.mdwn
for m in $^; do go run backlinks.go $$m; done
tells make that pageA.whatlinkshere, pageB.whatlinkshere and pageC.whatlinkshere are all generated by one execution of:
for m in pageA.mdwn pageB.mdwn pageC.mdwn; do go run backlinks.go $m; done
(make expands $^ as all prerequisites and $$m as $m). Of course, we want to automate the computation of the pageA.w%e pageB.w%e pageC.w%e pattern targets list. This should make it:
INFILES := $(shell find . -name "*.mdwn")
OUTFILES := $(INFILES:.mdwn=.html)
LINKFILES := $(INFILES:.mdwn=.whatlinkshere)
LINKPATTERN := $(INFILES:.mdwn=.w%e)
.PHONY: all clean
.PRECIOUS: $(LINKFILES)
all: $(OUTFILES)
# These need to be all made before the HTML is processed
$(LINKPATTERN): $(INFILES)
#echo Creating backlinks
#rm -f $(LINKFILES)
#touch $(LINKFILES)
#for m in $^; do go run backlinks.go $$m; done
%.html: %.mdwn %.whatlinkshere
#echo Deps $^
#cmark $^ > $#
clean:
rm -f $(LINKFILES) $(OUTFILES)
Notes:
I declared all and clean as phony because... it is what they are.
I declared the whatlinkshere files as precious because (some of them) are considered by make as intermediates and without this declaration make would delete them after building the HTML files.
In the recipe for the whatlinkshere files I added rm -f $(LINKFILES) such that, if the recipe is executed, we restart from a clean state instead of concatenating new stuff to old (possibly outdated) references.
The pattern stem in the $(LINKPATTERN) can be anything but must match at least one character. I used w%e but whatlin%shere would work too. Use whatever is specific enough in your case. If you have a pageB.where file prefer whatlin%shere or what%here.
There is a drawback with this solution but it is due to your particular set-up: each time one single mdwn file changes it must be re-analyzed (which is normal) but any whatlinkshere file can be impacted. This is not predictable, it depends on the links that have been modified in this source file. But more problematic is the fact that the result of this analysis is appended to the impacted whatlinkshere files. They are not "edited" with the old content relative to this source file replaced by the new one. So, if you change just a comment in a source file, all its links will be appended again to the respective whatlinkshere files (while they are already there). This is probably not what you want.
This is why the solution above deletes all whatlinkshere files and re-analyzes all source files each time one single source file changes. And another negative consequence is that all HTML files must also be re-generated because all whatlinkshere files changed (even if their content did not really change, but make does not know this). If the analysis is super fast and you have a small number of mdwn files, it should be OK. Else it is sub-optimal but not easy to solve because of your particular set-up.
Second option using recursive make, separated back link files and marker files
There is a possibility, however, which consists in:
separating all back links references with one whatlinkshere file per from/to pair: foo.backlinks/bar.whatlinkshere contains all references to bar found in foo.mdwn,
using recursive make with one first invocation (when the STEP make variable is unset) to update all whatlinkshere files that need to be and a second invocation (STEP set to 2) to generate the HTML files that need to be,
using empty dummy files to mark that a foo.mdwn file has been analyzed: foo.backlinks/.done,
using the secondary expansion to be able to refer to the stem of a pattern rule in its list of prerequisites (and using $$ to escape the fist expansion).
But it is probably a bit more difficult to understand (and maintain).
INFILES := $(shell find . -name "*.mdwn")
OUTFILES := $(INFILES:.mdwn=.html)
DONEFILES := $(patsubst %.mdwn,%.backlinks/.done,$(INFILES))
.PHONY: all clean
ifeq ($(STEP),)
all $(OUTFILES): $(DONEFILES)
$(MAKE) STEP=2 $#
%.backlinks/.done: %.mdwn
rm -rf $(dir $#)
mkdir -p $(dir $#)
cp $< $(dir $#)
cd $(dir $#); go run ../backlinks.go $<; rm $<
touch $#
else
all: $(OUTFILES)
.SECONDEXPANSION:
%.html: %.mdwn $$(wildcard *.backlinks/$$*.whatlinkshere)
#echo Deps $^
#cmark $^ > $#
endif
clean:
rm -rf *.backlinks $(OUTFILES)
Even if it looks more complicated there are a few advantages with this version:
only outdated targets are rebuilt and only once each,
all whatlinkshere files are updated (if needed) before any HTML file is updated (if needed),
the whatlinkshere files can be built in parallel,
the HTML files can be built in parallel.
Third option using only recursive make and marker files
If you do not care about inaccurate results where back links persist in the results after they disappeared from the source files or where back links are uselessly replicated, we can reuse ideas from the previous solution but drop the separation in individual from/to whatlinkshere files.
INFILES := $(wildcard *.mdwn)
OUTFILES := $(patsubst %.mdwn,%.html,$(INFILES))
LINKFILES := $(patsubst %.mdwn,%.whatlinkshere,$(INFILES))
DONEFILES := $(patsubst %.mdwn,.%.done,$(INFILES))
.PHONY: all clean
.PRECIOUS: $(LINKFILES)
ifeq ($(STEP),)
.NOTPARALLEL:
all $(OUTFILES): $(DONEFILES)
$(MAKE) STEP=2 $#
.%.done: %.mdwn
go run backlinks.go $<
touch $#
else
all: $(OUTFILES)
%.html: %.mdwn %.whatlinkshere
#echo Deps $^
#cmark $^ > $#
%.whatlinkshere:
touch $#
endif
clean:
rm -f $(OUTFILES) $(LINKFILES) $(DONEFILES)
Notes:
As this works only for a flat organization I replaced the $(shell find...) by the make built-in $(wildcard ...).
I used patsubst instead of the old syntax but it's just a matter of taste.
The %.whatlinkshere: rule is a default rule to create the missing empty whatlinkshere files.
The NOTPARALLEL: special target prevents parallel execution when building the whatlinkshere files.
I have a large project I'm working on, in which I want to perform include to some .mak file, but only after I make change to this file content via a command inside the original makefile. Since it's a large project it will be hard to write code, so I will give this ridiculous example instead:
I have some small C project that all it's C and header files are in the same directory, and I need to write a makefile. I'm not allowed to use clean rule in the makefile I write, but I have a file named file.mak that I can include in my makefile. Content of file.mak:
.PHONY: clean
cleam:
$(RM) $(objs) test
The problem here is that the rule is cleam and not clean. I'm also not allowed to change manually file.mak , but I'm allowed to do this with a command inside the original makefile. This can be done easily by:
sed -i 's/cleam/clean/g' file.mak
So I thought of writing the makefile like this:
CC = gcc
srcs = $(wildcard ./*.c)
objs = $(srcs:.c=.o)
test: $(objs) change_file include_file
$(CC) $^ -o $#
%.o: %.c
$(CC) -c $< -o $#
change_file:
$(shell sed -i 's/cleam/clean/g' file.mak)
include_file: change_file
include file.mak
But I get the following error:
include: Command not found
So I understand that there is a problem of using include inside a rule, so is there a way to achieve what I want?
(GNU) make has a feature Remaking Makefiles that can be used for scenarios like this, but your approach is wrong. include is a directive and can't be used in a recipe.
Instead, when you include a file, make first checks for rules creating this exact file and executes them. As in your case, the file you want to include already exists, you have to make this rule .PHONY to force its execution. It would look like this:
.PHONY: file.mak
file.mak:
sed -i 's/cleam/clean/g' file.mak
include file.mak
As a more robust alternative (without the need for a phony rule), consider creating a fixed version (copy) and include this:
file_fixed.mak: file.mak
sed -e 's/cleam/clean/g' <file.mak >file_fixed.mak
include file_fixed.mak