I am trying to create a makefile to compile LaTeX files in subdirectories. The desired directory structure should look like this, except instead of simply a, b, and c, there can be any number of subdirectories under the latex folder.
├── latex
│ ├── a
│ │ ├── output
│ │ │ └── stuff.pdf
│ │ └── stuff.tex
│ ├── b
│ │ ├── blah.tex
│ │ └── output
│ │ └── blah.pdf
│ └── c
│ ├── asdf.tex
│ └── output
│ └── asdf.pdf
└── makefile
I want to do this with only one makefile in the latex directory that will automatically compile the tex files in every subdirectory.
My current makefile looks like this:
TEX_COMMAND = pdflatex
TEX_FILES = $(wildcard **/*.tex)
OUTPUT_DIRECTORIES = $(addsuffix output/,$(wildcard */))
PDF_FILES = $(join $(dir $(TEX_FILES)),$(addprefix output/,$(notdir $(TEX_FILES:tex=pdf))))
all: mkdir $(PDF_FILES)
mkdir:
#mkdir -p $(OUTPUT_DIRECTORIES)
$(PDF_FILES): $(TEX_FILES)
#$(TEX_COMMAND) -file-line-error -halt-on-error -output-directory $(dir $#) -aux_directory=$(dir $#) $(subst output/,$(notdir $(#:pdf=tex)),$(dir $#))
#$(TEX_COMMAND) -file-line-error -halt-on-error -output-directory $(dir $#) -aux_directory=$(dir $#) $(subst output/,$(notdir $(#:pdf=tex)),$(dir $#))
clean:
#rm -rf $(OUTPUT_DIRECTORIES)
This will correctly generate the proper pdf, aux, log, toc, etc. files in the output directory in each subdirectory. However, if I change one tex file, then make will cause everything to be recompiled.
I've already looked at many other similar questions. For other questions, the number of subdirectories is known, so you can hardcode them into the makefile. For this situation, the number of subdirectories in the latex folder is constantly changing and being added to, etc, which is why I'm using the wildcard to grab all the tex files. I would prefer not having to create a makefile for each subdirectory and using recursive make.
One of the fundamental shortcomings of Make is its crude handling of wildcards.
In this case, you can use secondary expansion to write a pattern rule that will do what you want:
all: mkdir $(PDF_FILES)
.SECONDEXPANSION:
%.pdf: $$(subst /output/,/, $$(subst .pdf,.tex, $$#))
#echo buiding $# from $<
Related
I have a project structure that looks something like this:
.
└── src
├── Module1
│ ├── source1.cc
│ ├── source2.cc
│ └── source3.cc
├── Module2
│ ├── source1.cc
│ ├── source2.cc
│ └── source3.cc
└── Module3
├── source1.cc
├── source2.cc
└── source3.cc
I have an implicit rule that will create an object file for each source file, and maintain the directory structure (e.g. src/Module1/source2.cc would compile to obj/Module1/source2.o).
However, I would now like to have an implicit rule to create archive files for each module. For example, each object file compiled from src/Module2 would be added to obj/Module2.a. My first idea looked quite similar to my implicit rule for object files:
obj/%.a: $(wildcard obj/%/*.o)
#mkdir -p $(#D);
ar -crs "$#" $^;
This would pass off the work of compiling the object files to the other implicit rule. However, the issue here is that the % character is not expanded.
Is there a way to access the % from the implicit rule within the wildcard function call?
You can do it with Secondary Expansion:
.SECONDEXPANSION:
obj/%.a: $$(wildcard obj/%/*.o)
#mkdir -p $(#D);
ar -crs "$#" $^;
I have a project structure that looks something like this:
.
└── src
├── Module1
│ ├── source1.cc
│ ├── source2.cc
│ └── source3.cc
├── Module2
│ ├── source1.cc
│ ├── source2.cc
│ └── source3.cc
└── Module3
├── source1.cc
├── source2.cc
└── source3.cc
I have an implicit rule that will create an object file for each source file, and maintain the directory structure (e.g. src/Module1/source2.cc would compile to obj/Module1/source2.o).
However, I would now like to have an implicit rule to create archive files for each module. For example, each object file compiled from src/Module2 would be added to obj/Module2.a. My first idea looked quite similar to my implicit rule for object files:
obj/%.a: $(wildcard obj/%/*.o)
#mkdir -p $(#D);
ar -crs "$#" $^;
This would pass off the work of compiling the object files to the other implicit rule. However, the issue here is that the % character is not expanded.
Is there a way to access the % from the implicit rule within the wildcard function call?
You can do it with Secondary Expansion:
.SECONDEXPANSION:
obj/%.a: $$(wildcard obj/%/*.o)
#mkdir -p $(#D);
ar -crs "$#" $^;
I'm using GNU Make on Arch Linux to generate PDFs from LilyPond source files. I have a directory structure as follows:
scores/
├── makefile
├── out
│ ├── others-songs
│ │ ├── ...
│ │ ├── ...
│ │ └── 失恋阵线联盟
│ │ ├── 失恋阵线联盟.edition.log
│ │ ├── 失恋阵线联盟.oll.log
│ │ └── 失恋阵线联盟.pdf
│ └── ...
├── src
│ ├── others-songs
│ │ ├── ...
│ │ ├── ...
│ │ └── 失恋阵线联盟
│ │ ├── chorus.ily
│ │ ├── verse.ily
│ │ ├── words.ily
│ │ └── 失恋阵线联盟.ly
│ └── ...
The PDFs in the out directory depend on the .ily and .ly files in the corresponding directory in src. The following implicit pattern rule works if the .ly file is modified, but not if any of the other files are:
LY = $(shell find src -iname '*.ly')
PDF = $(subst src,out,$(LY:.ly=.pdf))
pdf: $(PDF)
out/%.pdf: src/%.ly
#mkdir -p $(dir $#)
#lilypond --include=$(lib) \
-dpoint-and-click=\#f \
-o $(basename $#) $<
I tried doing several different things, like appending $(<D)*.ily to the prerequisites, but it wasn't successful. I've looked at the GNU make manual online for help, but I didn't come up with anything that that I was able to apply to my particular situation.
How can I write a pattern rule that makes each PDF depend on all the files in the appropriate corresponding source directory?
EDIT
I may not have been clear enough with my first question. The behavior I want is if any of the files in the source directories are changed, the corresponding PDF is updated. For example, if chorus.ily is changed, then 失恋阵线联盟.ly is made.
A working example:
LY := $(shell find src -iname '*.ly')
PDF := ${LY:src/%.ly=out/%.pdf}
pdf: $(PDF)
define pdf_deps
$(1:src/%.ly=out/%.pdf) : $(wildcard $(dir ${1})*)
endef
# Make each pdf depend on all the files in its src directory.
$(foreach ly,${LY},$(eval $(call pdf_deps,${ly})))
out/%.pdf:
#echo "making $# from $^"
Usage:
$ ls -1R
.:
Makefile
out
src
./out:
./src:
A
B
./src/A:
a.ly
./src/B:
b.1
b.ly
$ make
making out/B/b.pdf from src/B/b.ly src/B/b.1
mkdir -p out/B
touch out/B/b.pdf
making out/A/a.pdf from src/A/a.ly
mkdir -p out/A
touch out/A/a.pdf
$ make
make: Nothing to be done for 'pdf'.
$ touch src/B/b.ly
$ make
making out/B/b.pdf from src/B/b.ly src/B/b.1
mkdir -p out/B
touch out/B/b.pdf
$ make
make: Nothing to be done for 'pdf'.
$ touch src/B/b.1
$ make
making out/B/b.pdf from src/B/b.ly src/B/b.1
mkdir -p out/B
touch out/B/b.pdf
$ make
make: Nothing to be done for 'pdf'.
In short: I want to compile sources from different directories, and put object files into current directory.
For example, I have files:
test.c
../../lib1/boot.c
../../lib2/startup.c
../common/utils.c
(also few files .s (assembly) and .cpp, but I hope this is not important).
All of their object-files I want to be in the current directory:
test.o
boot.o
startup.o
utils.o
And I can't figure out how to write such rule in my makefile.
For example,
%o.: %.c
does not work now because make can't find a rule to build boot.o from ../../lib1/boot.c, it can only find rule to build ../../lib1/boot.o from ../../lib1/boot.c.
I tried to use this:
%o.: %.c
(my compilation line, for example "gcc -c $^ -o $#")
%o.: ../../lib1/%.c
(my compilation line)
%o.: ../../lib2/%.c
(my compilation line)
%o.: ../common/%.c
(my compilation line)
and it works. But obviously this is not generic enough, and in addition, some user came to me today and said that his application has also some ../../some_other_lib/common_things.c, hence my makefile failed. I looked through our project, and found many such cases with a lot of different directories involved. With my approach, I'll have to write a separate rule for each such directory, with identical compilation line. This does not seem good to me.
So my question is: how to make some generic compilation rule that puts (and checks) object files in current directory, while operating with sources in different directories?
Thank you.
The directories can be extracted from the CSRC variable with $(dir ...) and this list can then be used in the vpath directive.
vpath %.c $(sort $(dir $(CSRC)))
vpath %.s $(sort $(dir $(SSRC)))
vpath %.cpp $(sort $(dir $(CPPSRC)))
(I've thrown in the sort function to remove duplicates, but that's not absolutely necessary.)
Now the rules can be kept simple and make will search the source files in the list of directories.
$(COBJ) := $(notdir $(CSRC))
$(SOBJ) := $(notdir $(SSRC))
$(CPPOBJ) := $(notdir $(CPPSRC))
.PHONY: all
all: $(EXECUTABLE)
$(EXECUTABLE): $(COBJ) $(SOBJ) $(CPPOBJ)
....
$(COBJ): %.o: %.c
...
$(SOBJ): %.o: %.s
...
$(CPPOBJ): %.o: %.cpp
...
Try to use makefile function notdir as this:
%.o: %.c
gcc -c $< -o $(notdir $#)
$# must be equal to the full path ex: ../../lib2/startup.o ad notdir will trunk it to: startup.o.
With this rule you will be able to compile all your source in the current directory.
Actually, your example is like that:
.
└── common
├── lib1
│ └── boot.c
├── lib2
│ └── startup.c
├── test
│ ├── Makefile
│ └── test.c
└── utils.c
I think i will be better like that:
.
├── common
│ ├── lib1
│ │ ├── Makefile
│ │ ├── obj
│ │ └── src
│ │ └── boot.c
│ ├── lib2
│ │ ├── Makefile
│ │ ├── obj
│ │ └── src
│ │ └── startup.c
│ ├── Makefile
│ ├── obj
│ ├── src
│ │ └── utils.c
│ └── test
│ ├── Makefile
│ ├── obj
│ └── src
│ └── test.c
└── Makefile
For that you need all your Makefiles to call the subdirs Makefiles.
and the src/obj dirs is a separation between your source and objects.
SRC := utils.c
OBJ := $(SRC:%.c=%.o)
NAME := project
SRC_D := src
OBJ_D := obj
SUBDIRS := lib1/ \
lib2/ \
test/
all: $(NAME) $(SUBDIRS)
#for dir in $(SUBDIRS); \
do \
$(MAKE) -C $$dir; \
done
$(NAME): $(OBJ:%.o=$(OBJ_D)/%.o)
$(OBJ_D)/%.o : $(SRC_D)/%.c
gcc -c $< -o $#
OK, took me some time, but finally I found the solution (using some threads on this site by the way):
# Defining compilation rules in a way that object files will be produced in current directory, and not in the directory of source files:
all: <List of my targets>
define my_c_rule
$(subst .c,.o,$(notdir $(1))): $(1)
$(CC) $(CFLAGS) $(CDEFINES) $$^ -o $$#
endef
$(foreach f, $(CSRC), $(eval $(call my_c_rule, $(f))))
$(CSRC) contains list of source files with their paths.
Just need to take into account that if earlier I had something like this:
.c.o:
$(CC) $(CFLAGS) $(CDEFINES) $^ -o $#
all: <List of my targets>
...now I have to put all sentence above the rules which I described in my_c_rule procedure. If I don't do this, make stops after compiling first source file. This is because old "wildcard" rules like .c.o or %.o: %.c do not replace all as a default target (even being written earlier), but non-wildcard rules like boot.o: ../../lib1/boot.c (result of the above macros) do replace the default target in case they are written earlier.
I have a directory tree like this:
├── dir_a
│ └── file_1.txt
├── dir_b
│ └── dir_c
│ ├── file_2.txt
| └── file_3.txt
└── file_4.txt
I want to mirror this directory structure to hold the results of a command that processes each text file. I.e., the output would look like this:
├── build
│ ├── dir_a
│ │ └── processed_file_1.txt
│ ├── dir_b
│ │ └── dir_c
│ │ ├── processed_file_2.txt
│ | └── processed_file_3.txt
│ └── processed_file_4.txt
├── dir_a
│ └── file_1.txt
├── dir_b
│ └── dir_c
│ ├── file_2.txt
| └── file_3.txt
└── file_4.txt
I'm not very adept with Makefiles, so my question is: how can I get a Makefile to recreate the directory structure and recursively process all text files to place them into the right place inside the build directory? I'll be running this repeatedly as the input files change, so a Makefile that doesn't process unchanged files seems like the right way to go.
Update:
I should also mention that new input files will be added frequently, so I don't want the Makefile to name them explicitly.
It would be easier if you used stems with different suffixes rather than inserting that "processed_" string, but here's an example that works for me here:
OUTPUTS := build/dir_a/processed_file_1.txt \
build/dir_b/dir_c/processed_file_2.txt \
build/dir_b/dir_c/processed_file_3.txt \
build/processed_file_4.txt
all: $(OUTPUTS)
.SECONDEXPANSION:
$(OUTPUTS): build/% : $$(subst processed_file_,file_,%)
mkdir -p $(dir $#)
cp $< $#
clean:
rm -rf build
You could remove the complication of .SECONDEXPANSION by changing the end of the filename instead of the beginning:
OUTPUTS := build/dir_a/file_1.out \
build/dir_b/dir_c/file_2.out \
build/dir_b/dir_c/file_3.out \
build/file_4.out
all: $(OUTPUTS)
$(OUTPUTS) : build/%.out : %.txt
mkdir -p $(dir $#)
cp $< $#
clean:
rm -rf build
As Carl suggested, you could use secondary expansion, but in conjunction with order-only prerequisites.
BUILD_DIR = build
IN_FILES := dir_a/file_1.out \
dir_b/dir_c/file_2.out \
dir_b/dir_c/file_3.out \
file_4.out
OUT_FILES := $(IN_FILES:%=$(BUILD_DIR)/%)
all: $(OUT_FILES)
.SECONDEXPANSION:
$(OUT_FILES) : $(BUILD_DIR)/%.out : %.txt | $$(#D)/.
# your text processing rule here...
%/. :
mkdir -p $*
| $$(#D) means:
during the secondary expansion calculate the value of $(#D) automatic variable (which is the directory part of the target), and
add the order-only dependency on it, that is ensure that the directory exists, but don't consider remaking the target if it is older than the directory (which is an often case)