I am using a Makefile to handle building a LaTeX document. I have two directories ./tikz and ./feyn that contain .tex files and Makefiles to compile them. They produce figures which I then include in the main document. I am using the following Makefile to compile the main document:
.PHONY : tikz feyn
main.pdf : main.tex feyn tikz
pdflatex main.tex
tikz :
$(MAKE) -C tikz
feyn :
$(MAKE) -C feyn
The problem is that even if main.pdf is up-to-date, running make will execute its rule anyways since phony targets are never up-tp-date. However, if I do not declare tikz and feyn as phony, their rules will not get executed. How can I fix this?
It depends, what do you want to do in the directories feyn and tikz. I guess that in tikz you want to generate some PDFs. Simply use an actual file as dependency. E.g.,
main.pdf : main.tex feyn tikz
pdflatex main.tex
tikz : tikz/somefile.pdf
tikz/somefile.pdf:
$(MAKE) -C tikz
# etc.
An alternative is to decouple main.pdf from your phony targets by using order targets:
.PHONY : tikz feyn
main.pdf : main.tex | feyn tikz
pdflatex main.tex
tikz :
$(MAKE) -C tikz
feyn :
$(MAKE) -C feyn
all: feyn tikz main.pdf
Don't run pdflatex in the subdirectories. Instead run latexmk in those diectories. The call to latexmk always happens (since the targets are phony), but latexmk will bail if nothing needs to be done for a subdirectory.
And then also probably use latexmk for the top-level target too.
Related
I have a makefile that for various reasons relies on a supporting python script to run every time and grab files from several external locations, copy into working directory, and run through a separate preprocessor before compiling.
This makefile must be able to be run in parallel (-j8) so the order of processing cannot be guaranteed.
In trying to explicitly specify prerequisites, I have created a situation where make skips all object files, goes straight to linking, and fails because the necessary objects do not exist. On a second run, all the objects already exist (the preprocess step skips the files that already exist) and all the files are compiled and linked properly.
When run without -j# everything works fine, but the moment I add -j2, the skipping begins.
Following is an example make file:
GEN_FILES := file1.cpp file2.cpp file3.cpp
CXX_FILES := bin_main.cpp $(GEN_FILES)
OBJ_FILES := $(patsubst %.cpp,%.o,$(CXX_FILES))
.PHONY : all clean prepare
all : bin_file
prepare :
# Copy and preprocess all source files
[ -f file1.cpp ] || cp d1/file1.cpp .
[ -f file2.cpp ] || cp d2/file2.cpp .
[ -f file3.cpp ] || cp d3/file3.cpp .
$(OBJ_FILES) : prepare
bin_file : $(OBJ_FILES)
[ -f file1.o ] && [ -f file2.o ] && [ -f file3.o ] && touch bin_file
%.o : %.cpp
#echo "Compiling $<..."
[ -f $< ] && touch $#
clean :
$(RM) *.o
$(RM) file*
$(RM) bin_file
How can I get this to build in one go, first running prepare to collect all files and then compiling and linking as necessary?
As code_fodder mentions the issue is the creation of the source files.
Basically what happens is, you have not told make how to create these source files, so as far as make knows they don't exist and there's no way to create them. So when make wants to build, for example, file1.o it looks at your pattern rule and finds it could build file1.o from file1.cpp. So then it looks for how to build file1.cpp. No file1.cpp exists, and there is no rule that make knows of that will build it, so make ignores that pattern rule as not matching.
Then make sees the target:
$(OBJ_FILES) : prepare
so it thinks there's no recipe needed to create the object files, and just runs the link line. The next time through, make sees the prepared source files (from the previous build) and then it can use your pattern rule.
If you change your pattern rule to a static pattern rule, where you explicitly tell make exactly what rule to use instead of providing it with a possible rule to use that it can ignore if it doesn't match (which is what a pattern rule is), you'll see the error:
$(OBJ_FILES): %.o : %.cpp
#echo "Compiling $<..."
sleep 1
[ -f $< ] && touch $#
will tell you:
make: *** No rule to make target 'file1.cpp', needed by 'file1.o'. Stop.
Remember, make is looking for a matching pattern rule BEFORE it actually builds anything: it doesn't want to build every possible prerequisite of every possible matching pattern rule, to decide whether or not at the end of it the rule can be used. The rule is matched based on the current state of the filesystem plus rules you have given make about changes it could make. Make has no idea that if it were to invoke the prepare target the source files it was looking for would magically come into existence.
Your basic problem is that this statement is the wrong dependency relationship:
$(OBJ_FILES) : prepare
It's not really true that the object files depend on prepare; what's true is that the PREPARED SOURCE FILES depend on prepare. The object files depend only the "prepared" source files, as your pattern rules shows. This rule should be written, instead:
$(GEN_FILES): prepare
If you do this with -j everything will wait as you want.
Yeah, this gets messy / difficult. The problem you have is that you can specify prerequisite lists - that can work in order, but as soon as you start to use -j then make can start processing prerequisites in any old order. So bin_file requires $(OBJ_FILES) which require prepare. Then %.o requires the same named %.cpp file - which it can do for main.o, but not the filex.o since they don't exist yet - but it tries anyway and fails - in the mean time make (in parallel) is potentially starting to generate the .cpp files, but by this time its too late...etc...
My Prerequisites Build Pattern
I use a very specific prerequisites pattern of my own design - some might frown upon - but I have carefully considered this over the years and found it to be optimal for me.
I create a rule called build or something - which requires build_prerequisites target and then calls make to do the actual build once this is complete:
.PHONY: build
build: build_prerequisites
build:
#echo "start_build"
#$(MAKE) bin_file
This means that build_prerequisites is always run first before the recipe runs. You cant seem to achieve the same forcing of order (at least not easily) using just dependencies. I.e. a list of dependencies can be run in any order with -j, but the rule recipe is always run last.
Now we have this pattern we can fill in the rest. First the build_prerequisites target which does your file generation - I am using echo in my example because I don't have your python script:
.PHONY: build_prerequisites
build_prerequisites:
#echo "build_prerequisites"
echo "create file1" > file1.cpp
echo "create file2" > file2.cpp
echo "create file3" > file3.cpp
Finally add in the c++ compile and link stages - these will be run with the single recursive make call from build - i.e. $(MAKE) bin_file (again I am using echo to create the files in my example):
%.o : %.cpp
#echo "compiling: $<"
##echo "$(CXX) $(SRC_INCLUDES) $(LIB_INCLUDES) $(CXXFLAGS) -c $< -o $#"
#echo "touch" > $#
bin_file : $(OBJ_FILES)
#echo "linking: $<"
#echo $(CXX) $(SRC_INCLUDES) $^ $(LIB_INCLUDES) $(LDFLAGS) -o $#
#echo "touch" > $#
Output
Here is the output from my test program (using echo) and main.cpp already exists usingn -j10:
make -j10
build_prerequisites
echo "create file1" > file1.cpp
echo "create file2" > file2.cpp
echo "create file3" > file3.cpp
start_build
make[1]: Entering directory '/mnt/d/software/ubuntu/make'
compile: bin_main.cpp
compile: file1.cpp
compile: file2.cpp
compile: file3.cpp
link: bin_main.o
g++ bin_main.o file1.o file2.o file3.o -o bin_file
make[1]: Leaving directory '/mnt/d/software/ubuntu/make'
Note: if I put a sleep 1 in the "compile" rule - this still takes only 1 second for all 4 files to compile.
Put it all together
GEN_FILES := file1.cpp file2.cpp file3.cpp
CXX_FILES := bin_main.cpp $(GEN_FILES)
OBJ_FILES := $(patsubst %.cpp,%.o,$(CXX_FILES))
###### STAGE 1
.PHONY: build
build: build_prerequisites
build:
#echo "start_build"
#$(MAKE) bin_file
.PHONY: build_prerequisites
build_prerequisites:
#echo "build_prerequisites"
copy_and_pp_files.py $(CXX_FILES) $(SEARCH_DIRS) .
copy_and_pp_files.py $(CFG_FILES) $(SEARCH_DIRS) .
###### STAGE 2
%.o : %.cpp
#echo "compiling: $<"
#$(CXX) $(SRC_INCLUDES) $(LIB_INCLUDES) $(CXXFLAGS) -c $< -o $#
bin_file : $(OBJ_FILES)
#echo "linking: $<"
#$(CXX) $(SRC_INCLUDES) $^ $(LIB_INCLUDES) $(LDFLAGS) -o $#
###### OTHER RULES
.PHONY: clean
clean :
#$(RM) *.o
#$(RM) file*
I have attempted to use your actual code, but I have no way to test this so there may be a bug in there. I split it up into 2 "stages" for clarity. Stage 1 is done in your makeor make build call, then state 2 is done in the recursive make call in the build recipe.
makefile noob here, my makefile always executes every recipe even though the files are up to date. Here is my code:
vpath *.pdf ../../../Figures/Arrowshape/ChemicalNoise
.PHONY : all clean
all : Fig_VP-CN-Revols_MeanfromDist_Dac0.0_F0.0-4.0_0to2.pdf\
Fig_VP-CN-Revols_MeanfromDist_Dac0.0_F0.0-4.0_2to4.pdf\
Fig_VP-CN-Revols_MeanfromDistImshow_Dac0.0_F0.0-4.0.pdf
Fig_%.pdf : %.py
$(warning Building $# )
python $<
Fig_%_2to4.pdf : %.py
$(warning Building $# )
python $<
Fig_%_0to2.pdf : %.py
$(warning Building $# )
python $<
clean:
rm all
I checked that the pdf files are put in the correct folder and the names match. What is wrong with my syntax?
Also, I know that my clean does not work, how would I make it work though?
When you say "put in the correct folder", which folder is that?
It's clearly not the local directory, because if it were your makefile would work.
The first thing wrong is you have the wrong syntax for vpath. See the manual; vpath takes a makefile pattern (that is a string with zero or one % character); it doesn't support shell globbing like *.h. This should be written:
vpath %.pdf ../../../Figures/Arrowshape/ChemicalNoise
However, even with that fix your makefile won't work as you hope, because vpath is not intended to find targets. It's intended to find source files (that is, files that are not created by make).
If you want to understand this deeply you can read http://make.mad-scientist.net/papers/how-not-to-use-vpath/
To get your makefile to work as you want you'll have to add in paths, like this:
OUTDIR = ../../../Figures/Arrowshape/ChemicalNoise
all : $(OUTDIR)/Fig_VP-CN-Revols_MeanfromDist_Dac0.0_F0.0-4.0_0to2.pdf\
$(OUTDIR)/Fig_VP-CN-Revols_MeanfromDist_Dac0.0_F0.0-4.0_2to4.pdf\
$(OUTDIR)/Fig_VP-CN-Revols_MeanfromDistImshow_Dac0.0_F0.0-4.0.pdf
$(OUTDIR)/Fig_%.pdf : %.py
$(warning Building $# )
python $<
$(OUTDIR)/Fig_%_2to4.pdf : %.py
$(warning Building $# )
python $<
$(OUTDIR)/Fig_%_0to2.pdf : %.py
$(warning Building $# )
python $<
I have created a Makefile to generate pdf and html for a md file mentioned while invoking make like make a.md should generate a.pdf and a.html, and should not convert other md files present in the directory.
My make file
But I am, getting error Nothing to be done for a.md
Could you please suggest changes?
Because a.md is already in place, make indeed does have nothing further to do. You can either just use for instance a as the specified target (which can depend on pdf and html files as prerequisites). Or pass the desired source through a variable and determine your desired target from that.
One (the latter) option would be this:
EXPORTED= $(SOURCE:%.md=%.html) $(SOURCE:%.md=%.pdf)
%.html : %.md
pandoc -o $# $<
%.pdf : %.md
pandoc -o $# $<
all: $(EXPORTED)
Which you can call with make SOURCE=a.md.
The other option (former):
%.html : %.md
pandoc -o $# $<
%.pdf : %.md
pandoc -o $# $<
.PHONY: $(MAKECMDGOALS)
$(MAKECMDGOALS): $(MAKECMDGOALS:%=%.html) $(MAKECMDGOALS:%=%.pdf)
Allows you to call make a.
But to reiterate. You cannot use (existing) source as a target, because make would (as it did) conclude it's done making that target.
Except (but I personally do not really like this as it is IMO confusing), if you insisted, you could do a variation on the previous take... but work with *.md targets... which you all declare as .PHONY. I.e. not real files and always to be considered/remade:
%.html : %.md
echo pandoc -o $# $<
%.pdf : %.md
echo pandoc -o $# $<
.PHONY: $(MAKECMDGOALS)
$(MAKECMDGOALS): $(MAKECMDGOALS:%.md=%.html) $(MAKECMDGOALS:%.md=%.pdf)
Now you could indeed call make a.md. I'd still prefer one of the two above.
I tried to make a makefile with patterns.
I want to have two variants:
when i write make it should just compile the list of files.
when i write make run it should run the respective files.
this is my current makefile:
files = test test1 test2
all: $(files)
$(files): % : %.scala
scalac $<
run: $(files)
$(files): % : %.scala
scala $<
now, regardless of whether i do make or make run it always executes the scala command and never scalac
You have a duplicate target:
$(files): % : %.scala
remove one of the targets (probably the second line was added by you)
I have a problem with make and I cannot figure out how to solve it. I would like to transform each file in a sub-directory using a shell script. This is what I wrote:
OBJ=$(wildcard src/*.in)
OUT=$(patsubst %.in, %.out, $(notdir $(OBJ)))
%.out : src/%.in
./myscript.sh $< > $#
$(OUT) : $(OBJ)
The makefile works but it runs the script only for the first .in file and then it stops. Sorry for the dumb question but I cannot fix this!
Because you are not telling the Makefile to build all $(OUT) files, you are just saying how it should be built.
Try instead this
OBJ=$(wildcard src/*.in)
OUT=$(patsubst %.in, %.out, $(notdir $(OBJ)))
all: $(OUT)
%.out : src/%.in
./myscript.sh $< > $#
.PHONY: all