How do I set up a Makefile to process all files in one directory, redirecting output to another (one output file per input file)? I have:
INPUTS := $(wildcard ./input/*.txt)
OUTPUTS := $(patsubst %.out,%.txt,$(wildcard ./input/*.txt))
$(OUTPUTS): $(INPUTS)
python process.py $# > ./output/${#:%.txt=%.out}
... but it keeps regenerating files in ./output that are already there.
Write it like this:
INPUTS := $(wildcard ./input/*.txt)
# Create a list of all the output files you want to generate
OUTPUTS := $(patsubst ./input/%.txt,./output/%.out,$(INPUTS))
# The default is to build all the OUTPUTS files
all: $(OUTPUTS)
# Tell make how to build a single output file
./output/%.out : ./input/%.txt
python process.py $< > $#
Related
My goal is the following: I have a directory src which contains markdown files (.md). I want to run a command on each of these files so that the comments are removed and the edited files are stored in a separate directory. For this I want to use make.
This is the Makefile I have:
.PHONY: clean all
BUILD_DIR := build
SRC_DIRS := src
SRCS := $(shell find $(SRC_DIRS) -name *.md)
DSTS := $(patsubst $(SRC_DIRS)/%.md,$(BUILD_DIR)/%.md,$(SRCS))
all: $(DSTS)
# The aim of this is to remove all my comments from the final documents
$(DSTS): $(SRCS)
pandoc --strip-comments -f markdown -i $< -t markdown -o $#
clean:
rm $(BUILD_DIR)/*.md
While this works in general, I noticed that the command is executed on all files, even though I changed only one single file.
Example: I have 3 Files src/a.md, src/b.md and src/c.md. Now I run make and all files are correctly generated in the build folder. Now I only edit c.md and run make again. I would expect that make only "compiles" src/c.md anew but instead all three files are compiled again. What am I doing wrong?
Your line
$(DSTS): $(SRCS)
is saying ‘All of the DSTS depend on all of the SRCS’, so whenever any one of the $(SRCS) is newer than any of the $(DSTS), this pandoc action will be run.
That's not what you want to express. What you want is something more like
$(BUILD_DIR)/%.md: $(SRC_DIRS)/%.md
pandoc --strip-comments -f markdown -i $< -t markdown -o $#
all: $(DSTS)
That says that all of the $(DSTS) should be up to date, and the pattern rule teaches Make what each one depends on, and how to build it, if it is out of date.
(As a general point, looking your original rule, it's rarely the right thing to do to have multiple targets in a rule, as you have with $(DSTS); also note that in your original, $< always refers only to the first of the dependencies in $(SRCS))
I would like to copy pdf files from several directories into a build directory, then use pdfunite to compile them into one pdf. The following make recipe works, but I have to run it twice because the first time through, I get an error from pdfunite - no files are found in the build directory (the PDFS variable is empty) even though they were just copied in the previous lines. How can I fix this so it works in one pass? I have simplified the recipe for clarity; I am actually pulling from various folders and making some pdfs on the fly as well, so I can't easily concatenate a full list of files from various subfolders (folder1 and folder2 in the example) to pass to pdfunite.
notebook:
mkdir -p $(out)
mkdir -p $(build)/notebook
$(eval PR := $(sort $(wildcard $(data)/folder1/*.pdf)) )
cp $(PR) $(build)/notebook
$(eval SR := $(sort $(wildcard $(data)/folder2/*.pdf)) )
cp $(SR) $(build)/notebook
$(eval PDFS := $(sort $(wildcard $(build)/notebook/*.pdf)) )
pdfunite $(PDFS) $(out)/notebook.pdf
Your Makefile is not in line with make's philosophy. You are using make as another scripting language, while make is more than this. It compares targets and prerequisites dates, based on this decides which must be built or re-built, and passes the recipes to the shell. So, for your particular problem, you should rather try something like:
PR := $(wildcard $(data)/folder1/*.pdf)
SR := $(wildcard $(data)/folder2/*.pdf)
PDFS1 := $(patsubst $(data)/folder1/%.pdf,$(build)/notebook/%.pdf,$(PR))
PDFS2 := $(patsubst $(data)/folder2/%.pdf,$(build)/notebook/%.pdf,$(SR))
PDFS := $(sort $(PDFS1) $(PDFS2))
.PHONY: notebook
notebook: $(out)/notebook.pdf
$(PDFS1): $(build)/notebook/%.pdf: $(data)/folder1/%.pdf | $(build)/notebook
cp $< $#
$(PDFS2): $(build)/notebook/%.pdf: $(data)/folder2/%.pdf | $(build)/notebook
cp $< $#
$(build)/notebook $(out):
mkdir -p $#
$(out)/notebook.pdf: $(PDFS) | $(out)
pdfunite $(PDFS) $#
The variables definitions are reasonably straightforward: patsubst, as its name says, substitutes strings. The target: pattern: prerequisites is a static pattern rule. And the prerequisites after | are order-only prerequisites.
What this makefile says, basically, is that $(out)/notebook.pdf depends on a set of pdf files in $(build)/notebook/ and that these pdf files depend on source pdf files with the same basenames in $(data)/folder1/ and $(data)/folder2/. It also says that directories must be created before being populated. Thanks to all this only what needs to be done will be done, no more, no less. And it is more in line with make's philosophy.
If you have many source folders and do not want to replicate the copying rules, you can use more advanced features like:
FOLDERS := folder1 folder2
.PHONY: notebook
notebook: $(out)/notebook.pdf
define MY_rule
$(1)_SRCS := $$(wildcard $$(data)/$(1)/*.pdf)
$(1)_DSTS := $$(patsubst $$(data)/$(1)/%.pdf,$$(build)/notebook/%.pdf,$$($(1)_SRCS))
PDFS += $$($(1)_DSTS)
$(1)_DSTS: $$(build)/notebook/%.pdf: $$(data)/$(1)/%.pdf | $$(build)/notebook
cp $$< $$#
endef
$(foreach f,$(FOLDERS),$(eval $(call MY_rule,$(f))))
$(build)/notebook $(out):
mkdir -p $#
$(out)/notebook.pdf: $(PDFS) | $(out)
pdfunite $(PDFS) $#
I would do another way.
PR:=$(sort $(wildcard $(data)/folder1/*.pdf))
SR:=$(sort $(wildcard $(data)/folder2/*.pdf))
PDFS=$(sort $(wildcard $(build)/notebook/*.pdf))
all: copy
pdfunite $(PDFS) $(out)/notebook.pdf
copy:
mkdir -p $(out)
mkdir -p $(build)/notebook
cp $(PR) $(build)/notebook
cp $(SR) $(build)/notebook
.PHONY: all copy
Please check: PDFS= and not PDFS:=. If you use simple = the variable's value will calculate when it needed (not sooner!).
When you run make it want to build all. The all's requirement is copy - so make does some mkdir and cp. After it return the all: the value of PDFS is needed so will evalute now - we have many-many pdf in $(build)/notebook :)
I am looking for Makefile macro to get list of all files in a directory that were generated as rule1 processing and using this list for rule2 processing.
Here's what I am trying to achieve :
Rule 1: Generate source .c files (using xml files) and place them in $(MYDIR) directory.
Rule 2: Get the list of all files in $(MYDIR) and create object files and place them in $(OBJDIR).
Problem is, I want to update list of files in Rule2 after Rule 1 has been processed, else list of files in $(MYDIR) will be empty.
all : rule_1 rule_2
rule1 : $(MYDIR)/generated_source1.c $(MYDIR)/generated_source2.c
$(MYDIR)/generated_source1.c:
xsltproc generator1.xml style_generator.xsl -o $(MYDIR)/generated_source_1.c
$(MYDIR)/generated_source2.c:
xsltproc generator2.xml style_generator.xsl -o $(MYDIR)generated_source_2.c
#Get list of all $(MYDIR).*c , create corresponding $(OBJDIR)/*.o list.
SOURCES := $(wildcard $(MYDIR)/*.c)
OBJECTS := $(notdir ${SOURCES})
GENERATED_OBJS := $(patsubst %.c,$(OBJDIR)/%.o,$(OBJECTS))
#This rule is compiling of all .c generated in rule1.
rule2 : $(GENERATED_OBJS)
ld -r -o $(OBJDIR)/generated_lib.o $(GENERATED_OBJS)
$(OBJDIR)/%.o: $(MYDIR)/%.c
gcc $(CFLAGS) -c -o $# $<
$(SOURCES) is shown empty, but actually it should contain generated_source1.c and generated_source2.c
I am not sure how .SECONDEXPANSION rule will work for my case.
You can't really (and don't really want to) play around with getting make to re-evaluate file existence during the running of the make process.
What you want to do is track the files from start to finish in make and then you have all your lists.
You can start at either direction but starting with the initial source tends to be easier.
So start with
MYDIR:=dir
OBJDIR:=obj
XML_SOURCES := $(wildcard $(MYDIR)/*.xml)
then translate from there to the generated source files
SOURCES := $(subst generator,generated_source,$(XML_SOURCES:.xml=.c))
and from there to the generated object files
GENERATED_OBJS := $(patsubst $(MYDIR)/%.c,$(OBJDIR)/%.o,$(SOURCES))
At which point you can define the default target
all: $(OBJDIR)/generated_lib.o
and then define the rules for each step
$(MYDIR)/%.c:
cat $^ > $#
$(OBJDIR)/%.o: $(MYDIR)/%.c
cat $^ > $#
$(OBJDIR)/generated_lib.o: $(GENERATED_OBJS)
ld -r -o $# $^
The $(MYDIR)/%.c rule needs a bit of extra magic to actually work correctly. You need to define the specific input/output pairs so that they are used correctly by that rule.
$(foreach xml,$(XML_SOURCES),$(eval $(subst generator,generated_source,$(xml:.xml=.c)): $(xml)))
This .xml to .c step would be easier if the input and output files shared a basename as you could then just use this and be done.
%.c: %.xml
cat $^ > $#
I'm trying to run a makefile on a directory such that it runs lint on all the cpp files in that directory and save the output in multiple files with the same names as source files.
e.g. in other words I want to save lint out put for abc.cpp to abc.txt and def.cpp to def.txt and so on for all the unknown number of files to a sub-directory lintfiles.
some thing like the following:-
*lint .ALWAYS:*
--lint-nt $(PATHS) $(OPTIONS) *.cpp > ./lintfiles/%f.txt
make util is not understanding Lint's %f option. I also tried the following but it generates no output.
lintfiles/%.txt: %.cpp
-lint-nt $(PATHS) $(OPTIONS) $# $^
please suggest.
I'm not familiar with pc-lint, but if the syntax is something like
lint-nt -o abc.lint abc.cpp
then this makefile should do the job:
SRC_DIR := src # or whatever
SRCS := $(wildcard $(SRC_DIR)/*.cpp)
LINTS := $(patsubst $(SRC_DIR)/%.cpp, $(SRC_DIR)/lint/%.lint, $(SRCS))
all: $(LINTS)
$(LINTS): $(SRC_DIR)/lint/%.lint : $(SRC_DIR)/%.cpp
lint-nt $(PATHS) $(OPTIONS) -o $# $<
I have .haml files and want to convert them automatically into .html files and update the latter when .haml is changed.
The generic makefile rule is no problem:
%.html: %.haml
hamlpy $< $#
But now I need a rule or a command to do the following:
find all X.haml files in templates/
execute make X.html command, where X is the same filename (haml is replaced with html).
I can't find how to do this with GNU Make or Unix find.
If all of your *.haml files are well name (i.e. no spaces or other funny characters), you can do it with a call to find(1):
HAML_FILES = $(shell find templates/ -type f -name '*.haml')
HTML_FILES = $(HAML_FILES:.haml=.html)
all: $(HTML_FILES)
%.html : %.haml
hamlpy $< $#
You can use GNU make wildcard function to find files in a certain directory:
INDIR := templates
OUTDIR := ${CURDIR}
haml_files := $(wildcard ${INDIR}/*.haml)
html_files := $(subst ${INDIR}/,${OUTDIR}/,${haml_files:.haml=.html})
all : ${html_files}
clean :
rm -f ${html_files}
${OUTDIR}/%.html : ${INDIR}/%.haml
hamlpy $< $#
.PHONY : all clean
INDIR and OUTDIR can be customized on the command line, for example, to use the current directory for inputs and iutputs:
$ make INDIR=. OUTDIR=.