the makefile below processes files matching the patterncontent/%.md and outputs the targets in the html directory. Source files are named with the convention of putting a leading number in front of them, like content/01.index.md or content/O2.second-page.md and so on. I would like to remove the leading 0x. number sequence in the target file. For instance, content/01.index.html would generate html/index.html.
How can I do this?
Thanks
MD_FILES = $(shell find content/ -type f -name '*.md')
HTML_FILES = $(patsubst content/%.md, html/%.html, $(MD_FILES))
all: $(HTML_FILES) html/static
html/%.html : content/%.md
mkdir -p $(#D)
python generator/generate.py $< $#
.PHONY: html/static
html/static :
rsync -rupE generator/static html/
.PHONY: clean
clean:
rm -fr html
Replace:
html/%.html : content/%.md
mkdir -p $(#D)
python generator/generate.py $< $#
with:
html/%.html : content/%.md
mkdir -p $(#D)
file='$(#F)'; python generator/generate.py $< "$(#D)/${file#*.}"
Unfortunately, I can't think of a good way of doing that in make itself. I can think of one way but it isn't as simple as that escaping and it isn't safe for files with spaces (not that that matters much here since make already can't handle those).
IMHO, it is a bad idea to use find or wildcards to list files in makefiles. This is because developers have temporary or debugging files sometimes. It is best to list files explicitly. This way, it forces the developer to think about their intent.
If you agree to list files explicitly, then in this case it is best to list the target files, rather than source files, and here is your answer:
HTML_FILES := html/index.html html/second-page.html
.SECONDEXPANSION:
$(HTML_FILES): html/%.html : $$(wildcard content/*.$$*.md)
(put recipe here, using $# and $<)
Related
I have a series of directories organized like this:
foo/
foo.file1 foo.file2
bar/
bar.file1 bar.file2
baz/
baz.file1 baz.file2
Right now I'm processing these files using a script that does all the checking for file existence etc but I thought that perhaps I could use a Makefile for it (since said script is very fragile), to avoid reprocessing files that did not change.
The problem is that each directory is independent, and I'd need to do, for example:
foo.file1.processed: foo.file1
run_random_program foo.file1 -o foo.file1.processed
for each of the 71 directories that are in total in that path. This looks like being extremely tedious and I wonder if there's something that would prevent me from writing all of this by hand.
Is such a thing possible?
EDIT: Some examples that show what I have in mind, had I a single Makefile for each directory:
file1.cds.callable: file1.callable
long_script_name -i $< -o $#
file1.rds: file1.cds.callable
another_long_script_name $< additional_file_in_folder $#
file1.csv: file1.rds
yet_another_script $< $#
Seems like pattern rules are exactly what you need:
# These are the original source files (based on the example)
CALLABLE := $(wildcard */*.callable)
# These are the final targets
TARGETS := $(CALLABLE:%.callable=%.csv)
all: $(TARGETS)
%.csv : %.rds
yet_another_script $< $#
%.rds: %.cds.callable
another_long_script_name $< additional_file_in_folder $#
%.cds.callable: %.callable
long_script_name -i $< -o $#
Hi!
I started messing with makefiles a few days ago, so I gave myself a exercise to learn as well as making my life easier.
I basically want to read the directory the makefile is in (root) and use luamin to compress the file as much as possible before I deploy it to our server. But I would like to have it as flexible as possible, so depending on where said file is in the directory it should mirror it to the server.
So if it finds a file in a sub folder called home it should create a new folder with the same name with the compressed file within. I have gotten the compression of files in the root folder working as well as creation of the directories where the files should reside.
objs = $(wildcard *.lua)
dirs = $(wildcard */)
compress: $(objs)
mkdir -p .build
luamin -f $(objs) > .build/$(objs)
mkdir .build/$(dirs)
clean:
rm -rf ./.build
deploy: .build
cp ./.build/* ~
If you use GNU make, there are several features that really help to do what you want. Warning: this works if and only if your file names do not contain spaces:
srcfiles := $(shell find . -path .build -prune -o -type f -name '*.lua' -print)
dstfiles := $(addprefix .build/,$(srcfiles))
.PHONY: compress clean deploy
compress: $(dstfiles)
$(dstfiles): .build/%: %
mkdir -p $(dir $#)
luamin -f $< > $#
clean:
rm -rf ./.build
deploy: .build
cp ./.build/* ~
Explanation:
The shell make function is used to run the find command that searches all subdirectories, except .build, for *.lua files. The result is assigned to the srcfiles make variable.
The addprefix make function is used to add the .build/ prefix to all words of the srcfiles make variable and assign the result to the dstfiles make variable.
The compress target is the first (real) target in the Makefile. It is thus the default goal that gets run when invoking just make. It is the same as invoking make compress. The compress target is declared as phony. This tells make that it is not a real file, just like clean and deploy. The compress target depends on all destination files. If one is missing or older than its corresponding source file, it must be rebuilt.
The make static pattern rule $(dstfiles): .build/%: %... declares a generic rule where each destination file (.build/./foo/bar/baz.lua) depends on the corresponding source file (./foo/bar/baz.lua). The recipe creates the destination directory (./foo/bar/), computed thanks to the dir make function. Then, it applies the luamin command. The recipe makes use of the $# and $< automatic variables.
I've seen a few approaches to making output directories in Make.
These include making all directories ahead of time outside of any rule, and
making an object's destination directory as part of the object's rule.
Both of these approaches involve making directories that likely already exist.
Am I missing any gotchas or drawbacks that explain why I haven't seen the below approach?
.SECONDEXPANSION:
$(OBJDIR)%.o: %.c | $$(#D)/
# Compile command
.PRECIOUS: %/
%/:
# mkdir Command
make is very good at dealing with files. make is not very good at dealing with directories.
So treating directories as implementation detail internal to the target rule makes sense, because then make never has to consider the directory at all:
MKDIR_P = mkdir -p
$(objdir)%.o: %.c
#$(MKDIR_P) $(#D)
$(COMPILE.c) -o $# -c $<
Note that the processing and IO required for the mkdir -p can be neglected next to the processing and IO required for the compilation.
The problem with directories is that (contrary to any other target) you don't care for their timestamp, you only need them to exist. Many Makefiles get directories somehow wrong, and creating them over and over again is what you observe, so make will never detect "Nothing to be done for ...".
In fact, the only thing you need for correct handling of directories with GNU make is an "order only dependency", like shown in your example. The trailing slash normally isn't needed (you seem to use it in order to have a pattern rule, I'm not sure whether this works), and you don't need .PRECIOUS either. Your trick with .SECONDEXPANSION looks quite neat, I guess this will work, given the pattern rule indeed works that way (didn't try).
For an alternative, most Makefiles that handle directories correctly take a simpler approach by concatenating all needed output directories for a rule in a single variable and use this variable as a target for another rule, e.g. like in this simplified example:
MODULES:=src/main
OBJDIR?=obj
OBJS:=$(addprefix $(OBJDIR)/,$(addsuffix .c,$(MODULES)))
DIRS:=$(sort $(addprefix $(OBJDIR)/,$(dir $(OBJS))))
TARGET:= myprogram
all: $(TARGET)
myprogram: $(OBJS)
$(CC) -o$# $^
$(DIRS):
mkdir -p $(DIRS)
$(OBJDIR)/%.o: %.c Makefile | $(DIRS)
$(CC) -c -o$# $<
clean:
rm -fr $(OBJDIR)
.PHONY: all clean
I wanted to write a general makefile that fulfils most of my needs for various projects at once. But then I tumbled over the following:
%/:
mkdir -p $#
.SECONDEXPANSION:
$(objects): $$(dir $$#)
I expected this to run just fine and create a directory whenever a dir for a target in $(objects) does need one. But somehow it didn't work. I checked and found the following: Targets can't end with '/'.
Is there a way I can avoid this?
You just need to select an suffix that will can't be the name of a file in the folder. For example .. So the rules become the following:
%/.:
mkdir -p $#
.SECONDEXPANSION:
$(objects): $$(dir $$#).
As . refers to the directory itself this should not create problems what-so-ever :)
I have a makefile with multiple targets that are generated by copying a file from outside the working directory.
a.tex : $(wildcard /foo/work1/a.tex)
cp -p $< $#
b.tex : $(wildcard /foo/work2/b.tex)
cp -p $< $#
I use $(wildcard) because sometimes I run Make on systems that do not have access to /foo.
What is the best way to avoid repeating the cp -p $< $# commands for every rule? Some options:
Setting up a %.tex : %.tex rule. This works, but it also applies to targets that aren't specifically indicated so I get lots of warnings like make: Circular a.tex <- a.tex dependency dropped.
Defining a sequence of commands with define. This seems pointless since the command is only one line. So instead of copying cp $< $# to every rule, I'd define a cp-dep sequence and copy cp-dep to every rule.
Defining the command as a variable so that I could do a.tex : $(wildcard /foo/work1/a.tex); $(CP-DEP)
Duplicating the target names as an additional rule. a.tex b.tex : ; cp -p $< $#. Error-prone.
Just copying and pasting. Clunky but effective and easy to understand.
I haven't tested it, but can't you just use a pattern rule without prerequisites, and specify the prerequisite for each target on a separate line?
a.tex: $(wildcard /foo/work1/a.tex)
b.tex: $(wildcard /foo/work2/b.tex)
%.tex:
cp -p $< $#
Btw. doesn't the wildcard function return the empty string when it doesn't find a match, so that $< is empty as well? Wouldn't that give a problem with cp?
I think your copyrule is overkill (and inflexible). If your objection to #eriktous's solution is that it will apply the rule to targets for which you haven't explicitly defined dependencies, that's easy to fix with a static pattern rule:
a.tex: $(wildcard /foo/work1/a.tex)
b.tex: $(wildcard /foo/work2/b.tex)
blue.tex: $(wildcard /some/other/path/green.tex)
TEXES = a.tex b.tex
$(TEXES): %.tex:
cp -p $< $#
(If this solves your problem you should accept eriktous's answer-- this is just a variation on it.)
I ended up doing this:
COPYFILES = /foo/work1/a.tex /foo/work2/b.tex
define copyrule
$(notdir $(1)): $$(wildcard $(1))
cp -p $$< $$#
endef
$(foreach file,$(COPYFILES),$(eval $(call copyrule,$(file))))
The advantage of this method is that I can easily add new files with a minimum of boilerplate text and I can easily copy the rule part of this to a new Makefile. The disadvantages are that I can no longer change the destination filename, and the implementation is rather opaque for people with less makefile experience.