I developed a script to convert all the markdowns files in my folder to ePub and place them in epubs.
Nevertheless I would like to place my files in the markdowns sub-folder and not in the main directory (the one where the makefile is).
.
├── epubs
│ └── blabla.epubs
├── makefile
└── markdowns
└── blabla.pd
How should I modify my code?
MARKDOWN=$(shell find . -iname "*.pd")
EPUBS=$(MARKDOWN:.pd=.epub)
.PHONY = all clean
all: $(EPUBS)
%.epub: %.pd
pandoc --from markdown --to epub --smart $< -o epubs/$#
clean:
rm epubs/*
One approach is to explicitly tell make in your pattern rule where the .pd prerequisites for the .epub targets reside:
MARKDOWN_DIR := markdowns
EPUB_DIR := epubs
MARKDOWNS := $(wildcard $(MARKDOWN_DIR)/*.pd)
EPUBS := $(subst $(MARKDOWN_DIR),$(EPUB_DIR),$(MARKDOWNS:.pd=.epub))
$(EPUB_DIR)/%.epub: $(MARKDOWN_DIR)/%.pd
pandoc --from markdown --to epub --smart $< -o $#
.PHONY: all clean
all: $(EPUBS)
Note two minor additional changes. I replaced your shell find command with make's $(wildcard) function because that is what it's for. Your pandoc command no longer needs epubs/ in front of $# because that directory is now part of the target. And I reordered a bit according to my liking.
Related
I need a Makefile that create for every <file.rst> a <file> folder to then execute
hovercraft on the <file.rst> which need a folder as second argument
$ tree
.
├── a.rst
├── b.rst
└── Makefile
With this Makefile
$ cat Makefile
.PHONY: html
HTML_TARGETS:= $(patsubst %.rst,%.html,$(wildcard *.rst))
html: $(HTML_TARGETS)
%.html: %.rst
#rm -fr $(basename $# .html)
#mkdir -p $(basename $# .html)
#hovercraft -Ns $< $(basename $# .html)
$
It kind of works
.
├── a
│ └── index.html
├── a.rst
├── b
│ └── index.html
├── b.rst
└── Makefile
I fell how baroquish this Makefile is, what could be a better way to write it ?
BTW I fail to add in the Makefile this echo:
#echo output done in $(basename $# .html)/index.html
I get:
output done in a /index.html
output done in b /index.html
^
└─ with an unwanted space
I whould like to print:
output done in a/index.html
output done in b/index.html
If I understand correctly that you want to make a directory "x", then execute hovercraft x.rst x/index.html for every file "x.rst", then this should be a succinct way to do so.
SOURCES := $(wildcard *.rst)
TARGETS := $(SOURCES:.rst=/index.html)
%/index.html: %.rst
mkdir -p $*
hovercraft $< $#
.PHONY: all
all: $(TARGETS)
I have a problem with GNU Make 4.2.1. It seems there is some interaction
between the syntax of static pattern rules and conditional
functions that I do not quite understand.
The context: I have a project documented by a set of Markdown files, and
I would like to render these files to HTML in order to check them
locally. The directory structure should end up looking like this:
some_project/
├── README.md # entry page of documentation
├── doc/ # extra docs
│ ├── foo.md
│ ├── bar.md
│ └── ... # and some more
└── doc_html/ # HTML rendering of the docs
├── Makefile # the Makefile I am trying to write
├── index.html # rendered from README.md
├── foo.html # ............. doc/foo.md
├── bar.html # ............. doc/bar.md
└── ... # etc.
Without the special case of index.html, I could write something like:
%.html: ../doc/%.md
some list of commands $< $#
The problem is that the prerequisite of index.html (namely ../README.md)
does not match the pattern. I would like to handle this special case
without having to repeat the whole list of commands. This is what I have
so far:
DOC_PAGES = $(wildcard ../doc/*.md)
TARGETS = index.html $(patsubst %.md,%.html,$(notdir $(DOC_PAGES)))
# Function to find the source for page $(1)
source = $(if $(findstring index.html,$(1)), \
../README.md, \
$(patsubst %.html,../doc/%.md,$(1)) \
)
all: $(TARGETS)
$(TARGETS): %.html: $(call source,%.html)
#echo some list of commands $< $#
# Check the TARGETS variable and the `source' function
test:
#echo TARGETS = $(TARGETS)
#echo "source(index.html)" = $(call source,index.html)
#echo "source(foo.html)" = $(call source,foo.html)
My source function seems to work:
$ make test
TARGETS = index.html bar.html foo.html
source(index.html) = ../README.md
source(foo.html) = ../doc/foo.md
However, it doesn't behave properly in the static rule
$ make
make: *** No rule to make target '../doc/index.md', needed by 'index.html'. Stop.
Note that the rule does work if I remove index.html from $(TARGETS).
An idea of what I am doing wrong?
This line:
$(TARGETS): %.html: $(call source,%.html)
cannot work because you're expanding the source macro with an argument of the literal string %.html. You can't use patterns or automatic variables in macros in prerequisite lists: macros are expanded first, before parsing or expanding patterns.
However, it seems to me that you're making this way more complicated than it needs to be. If most of your targets are built one way but a few are built a different way, then just create a pattern rule for the "most" and write explicit rules for the "some":
%.html: ../doc/%.md
some list of commands $< $#
index.html : ../README.md
commands to build index.html
If the set of commands is identical and you don't want to repeat them, put them in a variable:
create_html = some list of commands $< $#
%.html: ../doc/%.md
$(create_html)
index.html : ../README.md
$(create_html)
(be sure to create the variable with = not := if you want to include $< and $# in the script).
ETA You asked about why things seemed to work: when make expands this it will substitute the literal string %.html. You can prove this to yourself by adding an $(info...) call like this:
source = $(info 1=$(1)) $(if $(findstring index.html,$(1)), \
../README.md, \
$(patsubst %.html,../doc/%.md,$(1)) \
)
and you'll see that it will print (one time only because the rule is only expanded once) 1=%.html.
What happens next? Well this means that your macro expands to this:
$(if $(findstring index.html,%.html), ../README.md, $(patsubst %.html,../doc/%.md,%.html))
(again, using the literal string %.html). The findstring always returns empty because index.html can't be found in the string %.html, so you expand the else-clause:
$(patsubst %.html,../doc/%.md,%.html)
Clearly %.html matches %.html with a stem of %, so the substitution is made and returns ../doc/%.md. So after all that your rule looks like this:
$(TARGETS): %.html: ../doc/%.md
#echo some list of commands $< $#
This exactly the same thing you had before with your simple pattern rule.
This would seem to be a solution:
DOC_PAGES := $(wildcard ../doc/*.md)
TARGETS := index.html $(patsubst %.md,%.html,$(notdir $(DOC_PAGES)))
all: $(TARGETS)
.INTERMEDIATE: ../doc/index.md
../doc/index.md: ../README.md
cp $< $#
%.html: ../doc/%.md
#echo some list of commands $< $#
test:
#echo $(TARGETS)
With output:
$ make
cp ../README.md ../doc/index.md
some list of commands ../doc/index.md index.html
some list of commands ../doc/bar.md bar.html
some list of commands ../doc/foo.md foo.html
rm ../doc/index.md
index.html looks for ../doc/index.md through the pattern rule, leading to the recipe copying ../README.md.
Prerequisites to the .INTERMEDIATE special target are removed when make completes. Optional.
I want to use make and create a static pattern rule that has the target in a output directory, and the prerequisite files are in the preceeding directory, and it has to work recursively.
I have a minimal example here:
.
├── anotherdir
│ ├── output
│ │ ├── source3.md
│ │ └── source4.md
│ ├── source3.json
│ └── source4.json
├── output
│ ├── source1.md
│ └── source2.md
├── source1.json
└── source2.json
I want to generate the output directories if they do not exist, and I want to generate *.md files from the *.json using make if they do not exist, or *.json is updated.
So far, I have the following Makefile:
SOURCE_FILES := $(shell find ./ -name "*.json")
OUTPUT_FILES := $(join $(addsuffix output/,$(dir $(SOURCE_FILES))), $(addsuffix .md,$(basename $(notdir $(SOURCE_FILES)))))
.PHONY: all
all: $(OUTPUT_FILES)
$(OUTPUT_FILES): %.md: %.json
mkdir -p $(dir $#)
# Command to create MD file from json file into the output directory here
The actual command to create the MD file from the json file doesn't matter here, because I have a script that I will call that will do this for me. The problem here, is that when I try to even run this at all, I get the following output:
> make all
make: *** No rule to make target 'anotherdir/output/source4.json', needed by 'anotherdir/output/source4.md'. Stop.
Obviously, source4.json is not in anotherdir/output, but rather, it's in the preceeding directory, which is just anotherdir. I don't know how to make it so that the pattern $(OUTPUT_FILES): %.md: %.json will match it properly.
Or is a static pattern rule not good here? I'm not sure what to do to fit my scenario.
EDIT: I tried to do something like this:
$(OUTPUT_FILES): %.md: $(join $(subst output,,$(dir %)), $(addsuffix .json,$(basename $(notdir %))))
and this doesn't work, I still get:
> make all
make: *** No rule to make target 'anotherdir/output/source4.json', needed by 'anotherdir/output/source4.md'. Stop.
Edit 2: to clarify, i start with the following files
.
├── anotherdir
│ ├── source3.json
│ └── source4.json
├── source1.json
└── source2.json
And then when i run make, i want it to generate the output folders like this
.
├── anotherdir
│ ├── output
│ │ ├── source3.md
│ │ └── source4.md
│ ├── source3.json
│ └── source4.json
├── output
│ ├── source1.md
│ └── source2.md
├── source1.json
└── source2.json
I want to use some kind of smart makefile syntax to pick up these files names without me hard coding it in myself. Hence, i looked at the documentation and saw that static pattern rules might be the solution that i want, except that i can't get the right prerequisite pattern down.
I would do it this way:
First, find the source files just as you did (with a small change to prevent the unsightly double-slash):
SOURCE_FILES := $(shell find . -name "*.json")
A pattern file would be nice, if we could use two wildcards at once, but Make can't quite do that. So I recommend using a template:
define template
TDIR := $(dir $(1))output
TARG := $$(TDIR)/$(notdir $(basename $(1))).md
$$(TARG): $(1)
mkdir -p $$#
#echo building $$# from $$<
# Command to create MD file from json file into the output directory here
endef
$(foreach SOURCE,$(SOURCE_FILES),$(eval $(call template,$(SOURCE))))
If this works, all that's left is to construct a list of output files, and a default rule that has all of them as prerequisites:
define template
TDIR := $(dir $(1))output
TARG := $$(TDIR)/$(notdir $(basename $(1))).md
OUTPUT_FILES += $$(TARG)
$$(TARG): $(1)
mkdir -p $$#
#echo building $$# from $$<
# Command to create MD file from json file into the output directory here
endef
all:
$(foreach SOURCE,$(SOURCE_FILES),$(eval $(call template,$(SOURCE))))
all: $(OUTPUT_FILES)
It isn't pretty, but it seems to work.
If it had not been proposed already in another answer I would have suggested foreach-eval-call. For completeness here are different solutions for GNU make (they may work also with other versions of make but I did not check):
Creating the output directories beforehand
If the output directories exist already you can refer to ../%.json in your pattern rule:
SOURCE_FILES := $(shell find . -name "*.json")
OUTPUT_FILES := $(join $(dir $(SOURCE_FILES)),\
$(patsubst %.json,output/%.md,$(notdir $(SOURCE_FILES))))
$(shell mkdir -p $(dir $(OUTPUT_FILES)))
.PHONY: all
all: $(OUTPUT_FILES)
%.md: ../%.json
: json2md $< -o $#
This may look strange but if you read carefully the Pattern match section of the GNU make manual you should quickly understand. The only constraint for this to work is that the output directories exist before make searches pattern rules that match the targets. If one does not exist make will complain that there is no eligible rule to build the target. This is the reason for the:
$(shell mkdir -p $(dir $(OUTPUT_FILES)))
at the beginning of the Makefile. Demonstration:
$ make
: json2md output/../source2.json -o output/source2.md
: json2md output/../source1.json -o output/source1.md
: json2md anotherdir/output/../source4.json -o anotherdir/output/source4.md
: json2md anotherdir/output/../source3.json -o anotherdir/output/source3.md
Using the secondary expansion
Secondary expansion gives you the possibility to use automatic variables in the list of prerequisites. The $$ are needed to escape the first expansion by make.
SOURCE_FILES := $(shell find . -name "*.json")
OUTPUT_FILES := $(join $(dir $(SOURCE_FILES)),\
$(patsubst %.json,output/%.md,$(notdir $(SOURCE_FILES))))
.PHONY: all
all: $(OUTPUT_FILES)
$(sort $(dir $(OUTPUT_FILES))):
mkdir -p $#
.SECONDEXPANSION:
$(OUTPUT_FILES): $$(patsubst %output,%,$$(#D))$$(basename $$(#F)).json | $$(dir $$#)
: json2md $< -o $#
Demonstration:
$ make
mkdir -p output/
mkdir -p anotherdir/output/
: json2md source2.json -o output/source2.md
: json2md source1.json -o output/source1.md
: json2md anotherdir/source4.json -o anotherdir/output/source4.md
: json2md anotherdir/source3.json -o anotherdir/output/source3.md
Note: instead of creating the output directories in the json-to-md rule (which has the drawback of creating them several times), I added them as order-only prerequisites and added a specific rule to create them.
Note: the sort function also removes duplicates.
Using recursive make
Here we invoke make (with always the same Makefile) recursively in each sub-directory (except output, of course). Each invocation handles only the local json files, which makes the paths of prerequisites and targets much simpler.
MF := $(realpath $(lastword $(MAKEFILE_LIST)))
SUB_DIRS := $(filter-out . ./output,$(shell find . -maxdepth 1 -type d))
SOURCE_FILES := $(filter-out $(SUB_DIRS),$(wildcard *.json))
OUTPUT_FILES := $(patsubst %.json,output/%.md,$(SOURCE_FILES))
.PHONY: $(SUB_DIRS) all
all: $(SUB_DIRS) $(OUTPUT_FILES)
$(OUTPUT_FILES): output/%.md: %.json | output
: json2md $< -o $#
output:
mkdir -p $#
$(SUB_DIRS):
$(MAKE) -C $# -f $(MF)
Demonstration:
$ make
make -C anotherdir -f /home/doe/json2md/Makefile
make[1]: Entering directory '/home/doe/json2md/anotherdir'
mkdir -p output
: json2md source4.json -o output/source4.md
: json2md source3.json -o output/source3.md
make[1]: Leaving directory '/home/doe/json2md/anotherdir'
mkdir -p output
: json2md source2.json -o output/source2.md
: json2md source1.json -o output/source1.md
I have the current folder structure for my project
.
├── Makefile
└── S6
├── CD_CS304.md
├── CN_CS306.md
├── DAA_CS302.md
└── graphviz
└── cs304_compilerphases.dot
2 directories, 5 files
I am building separate pdfs for each and every markdown file, here is my Makefile
# Generate PDFs from the Markdown source files
#
# In order to use this makefile, you need some tools:
# - GNU make
# - Pandoc
# All markdown files are considered sources
MD_SOURCES := $(wildcard **/*.md)
OUTPUT_PDFS := $(MD_SOURCES:.md=.pdf)
DOT_SOURCES := $(wildcard **/*.dot)
OUTPUT_DOTPNGS := $(DOT_SOURCES:.dot=.png)
all: $(OUTPUT_DOTPNGS) $(OUTPUT_PDFS)
# Recipe for building png files from dot files
%.png: %.dot
dot \
-Tpng $< \
-o $#
# Recipe for converting a Markdown file into PDF using Pandoc
%.pdf: %.md
pandoc \
--variable fontsize=12pt \
--variable date:"\today" \
--variable geometry:a4paper \
--variable documentclass:book \
--table-of-contents \
--number-sections \
--filter pandoc-fignos \
-f markdown $< \
-o $#
.PHONY : clean
clean: $(OUTPUT_PDFS) $(OUTPUT_DOTPNGS)
$(RM) $^
I want to embed the output of the dot program into the latex pdf's but here the Makefile does not make the dot files into png and goes straight into compiling the pdf.
This makes the pdf compilation run into errors as the png files are not present.
If you want to ensure that one file is built before another file, add a dependency.
Change this:
%.pdf: %.md
to this:
%.pdf: %.md $(OUTPUT_DOTPNGS)
This dependency says, "Don't build this pdf file unless you've built every png file."
My directory structure is the following (of course the src/ subdirectories also contain files, but these are not important right now):
examples/
├───testprog1/
│ ├───bin/
│ ├───obj/
│ ├───src/
│ └───Makefile
├───testprog2/
│ ├───bin/
│ ├───obj/
│ ├───src/
│ └───Makefile
└───Makefile
The Makefile's in testprog1/ and testprog2/ look similar to this one:
OBJECTS_FILES=$(subst .cpp,.o,$(subst src/,obj/,$(wildcard src/*.cpp)))
bin/testprog1.exe: $(OBJECTS_FILES)
g++ -o $# $^
obj/%.o: src/%.cpp
g++ -c -o $# $^
clean:
rm -f obj/*.o
rm -f bin/meanTest.exe
They work perfectly fine alone, but if I want to build all examples at once, it would be better to use the Makefile in examples/ for that. What should it look like? Of course, it has to have the targets all and clean it will execute for all subdirectories. I also would like to avoid for-loops because I heard that they prevent make from using parallel processing.
You can make a recipe run in the background by appending an & to the end, which would allow you to build in parallel, but that's messy as your target could complete before the background tasks have finished running, which could cause dependency issues, so we'll steer clear of that.
A better solution would be to create a new rule for each subdir:
.PHONY: testprog1 testprog2
testprog1 testprog2:
$(MAKE) -C $# $(MAKECMDGOALS)
all clean: testprog1 testprog2
If you do a make clean, then $(MAKECMDGOALS) will be clean. It will build both testprog1 and testprog2 with this target. Notice that because testprog1 and testprog2 are directory names, you have to declare them as phonies, as their timestamp will change when you build them.
You could also specify the target as %: testprog1 testprog2 if you wanted any command (save testprog1 or testprog2) to be passed down to the submakes.