This is my first ever post to stackoverflow. It ought to be straightforward, but I have this problem whenever I try to write a makefile and I've never been able to figure out a satisfactory solution. Apologies if there is already a solution somewhere on the site. I couldn't find one.
What I'm trying to do is as follows:
Search my src directory for matching source files.
Compile the src code into a sandbox.
Here's my directory structure:
makefile
src1
file1.src
file2.src
src2
file3.src
subfolder
src3
file4.src
file5.src
And here's my makefile:
BUILDDIR := ./sandbox
SRC_DIRS := ./
SRCS := $(shell find $(SRC_DIRS) -name *.src)
OBJS := $(addprefix $(BUILDDIR)/o., $(notdir $(SRCS) ) )
# make print-X prints value of variable X
print-%: ; #echo $* = $($*)
.PHONY: help
help:
#echo "make <all|clean>"
.PHONY: all
all: $(OBJS)
#echo "compilation done"
$(OBJS) : $(SRCS) $(BUILDDIR)/.create
#echo "\"compiling\" $< to produce $#"
cp $< $#
$(BUILDDIR)/.create:
#echo "creating sandbox"
mkdir -p $(BUILDDIR) && cd $(BUILDDIR)
#touch $#
.PHONY: clean
clean:
#echo "deleting sandbox"
#rm -rf $(BUILDDIR)
If I type make all, the file works as expected. However, if I type make all again, instead of saying everything is up to data, I end up with the following contents in the sandbox:
o.file1.src o.file2.src o.file3.src o.file4.src o.file5.src o.o.file1.src o.o.file2.src o.o.file3.src o.o.file4.src o.o.file5.src
And the process of creating objects of objects continues recursively as many times as I type make.
Any help would be appreciated.
Incidentally, please don't post solutions that rely on the build in compile functions of make. I'm looking for a general solution that can be used for any task. For example, in this instance, I'm trying to read the source files into a tool using its command line interface.
Well, first, by having your sandbox as a subdirectory of your source directory, then using find on the source directory, every time you run it you're going to find all the files in both the source directory and all its subdirectories, including the sandbox. If the built files in the sandbox have the same names as the files in the source directories, the find will find them all.
Maybe instead of:
SRCS := $(shell find $(SRC_DIRS) -name *.src)
you want something like:
SRCS := $(shell find $(SRC_DIRS) -name $(notdir $(BUILDDIR)) -prune -o -name *.src -print)
Or, alternatively, don't make your sandbox a subdirectory of your source directory. Or, make sure that whatever name you give to the files in the sandbox directory won't match the *.src pattern you give to find.
But beyond that this is wrong:
$(OBJS) : $(SRCS) $(BUILDDIR)/.create
Suppose SRCS is foo.src bar.src, which means OBJS is sandbox/o.foo.src sandbox/o.bar.src. Then the above expands to this:
sandbox/o.foo.src sandbox/o.bar.src : foo.src bar.src sandbox/.create
This is a common mistake; people seem to think that make will go through the targets and prerequisites and match them up, so the first target depends on the first prerequisite and the second target depends on the second one etc. but of course this cannot work correctly and that's not how make works. Make treats the above as if you'd written one rule for each target, with the same prerequisites; like this:
sandbox/o.foo.src : foo.src bar.src sandbox/.create
sandbox/o.bar.src : foo.src bar.src sandbox/.create
You can see this won't do what you want at all, since the $< will always be foo.src which is clearly wrong.
You need to write a single pattern rule that will build ONE target. Then make sure the pattern applies to all the targets.
You have made things hard for yourself by trying to "flatten" a directory structure of multiple source subdirectories, into a single level of target directory (by using the $(notdir $(SRCS))). Because of this, there's no pattern that will match the same target and directory, unless you write a separate rule for every subdirectory.
Luckily there is a solution for this: VPATH. This should work for you:
VPATH := $(sort $(dir $(SRCS))
$(BUILDDIR)/o.%.src : %.src $(BUILDDIR)/.create
#echo "\"compiling\" $< to produce $#"
cp $< $#
The VPATH tells make to go look in all the directories that it found any sources in, whenever it can't find one to build.
The basic problem is that your SRCS is all files in all subdirectories that match the pattern *.src (when you run make). That means that all your object files ($(OBJS)) also match, so they copied as well.
The solution is to change your SRCS pattern so it does not match the "object" files in the build directory. Possibilities:
SRCS := $(wildcard *.src)
or
SRCS := $(shell find $(SRC_DIRS) -name $(notdir $(BUILDDIR)) -prune -false -o -name *.src)
or change the names of your "object" files so they don't end in .src
If I type make all, the file works as expected.
By which I take you to mean that directory ./sandbox exists and contains these files:
o.file1.src
o.file2.src
o.file3.src
o.file4.src
o.file5.src
However, if I type make all again, instead of saying everything is up to data, I end up with the following contents in the sandbox:
o.file1.src o.file2.src o.file3.src o.file4.src o.file5.src o.o.file1.src o.o.file2.src o.o.file3.src o.o.file4.src o.o.file5.src
Of course you do, because everything is not up to date at that point, according to the target list you create. This line ...
SRCS := $(shell find $(SRC_DIRS) -name *.src)
... defines SRCS as a list of all the files under path ./ (which is the value of SRC_DIRS) that ends in .src. That includes any such files in ./sandbox, which include all the files placed there by the first make run. You generate a corresponding target file to build for each source, and those targets corresponding to sources built by the previous make run will not, in general, exist yet. So make builds them, just as you instructed it.
The best solution, short of abandoning that automatic source identification altogether, would probably be to change your naming scheme so that outputs cannot be mistaken so easily for sources. For example, if you want the output names to have the form foo.src, then have the corresponding input named something like foo.src.in. In that particular case, you could convert from source names to target names with
OBJS := $(addprefix $(BUILDDIR)/o., $(basename $(notdir $(SRCS) ) ) )
Alternatively, you could modify the find command to skip the sandbox directory, maybe by modifying SRC_DIRS:
SRC_DIRS = src1 src2 subfolder/src3
(These specific alternatives are not mutually exclusive.)
Related
I'm using GNU make, which I thought would make this easier.
I'm trying to build in either build/Debug or build/Release, depending on whether the target is 'debug' or 'release'.
Here's the relevant excerpt of my makefile:
# Project files
SRC_DIRS = ../source ..source/[...], etc.
SRCS := $(shell find $(SRC_DIRS) -maxdepth 1 -name *.cpp -or -name *.cxx -or -name *.c -or -name *.s)
SRCS := $(sort $(SRCS))
vpath %.cpp $(SRC_DIRS)
vpath %.cxx $(SRC_DIRS)
vpath %.c $(SRC_DIRS)
# Object files
OBJS := $(addsuffix .o, $(notdir $(basename $(SRCS))))
# Dependencies
DEPS := $(OBJS:.o=.d)
debug: DEPS_DIR = build/Debug
release: DEPS_DIR = build/Release
DEPS := $(addprefix $(DEPS_DIR)/, $(DEPS))
include $(DEPS)
The problem is that I can't get $(DEPS) to evaluate properly. It seems like the DEPS addprefix is being evaluated before the target-specific DEPS_DIR assignments, so that it's resulting in "/filename.d", since it thinks $DEPS_DIR is blank. However, if I go down further and under the actual targets add a echo "DEPS_DIR = $(DEPS_DIR)", it expands correctly.
There's probably something obvious I'm doing wrong. How can I point include at the DEPS list in either of build/Debug or build/Release, depending on target?
You've missed the part of the documentation on target-specific variables where it specifies that they are in effect only when expanding the recipes for rules.
Make works by reading in the entire makefile all at once and building its internal graph of targets and prerequisites, and only after that is complete does it look to see which targets should be built based on the command line, etc. That's why order of rule definition, etc. mostly doesn't matter: make is not "running" the makefile as it is read in, as if it were a script. So there's no way for a target-specific variable to impact what files are included by the include statement.
There are a few ways to do what you want. The simplest and most portable one is to use a recursive invocation of make, like this:
debug:
$(MAKE) DEPS_DIR=build/Debug realtarget
release:
$(MAKE) DEPS_DIR=build/Release realtarget
Alternatively the MAKECMDGOALS make variable contains whatever goals were specified on the command line, so you can check this in a make ifeq statement or similar and set DEPS_DIR that way.
Hi!
I started messing with makefiles a few days ago, so I gave myself a exercise to learn as well as making my life easier.
I basically want to read the directory the makefile is in (root) and use luamin to compress the file as much as possible before I deploy it to our server. But I would like to have it as flexible as possible, so depending on where said file is in the directory it should mirror it to the server.
So if it finds a file in a sub folder called home it should create a new folder with the same name with the compressed file within. I have gotten the compression of files in the root folder working as well as creation of the directories where the files should reside.
objs = $(wildcard *.lua)
dirs = $(wildcard */)
compress: $(objs)
mkdir -p .build
luamin -f $(objs) > .build/$(objs)
mkdir .build/$(dirs)
clean:
rm -rf ./.build
deploy: .build
cp ./.build/* ~
If you use GNU make, there are several features that really help to do what you want. Warning: this works if and only if your file names do not contain spaces:
srcfiles := $(shell find . -path .build -prune -o -type f -name '*.lua' -print)
dstfiles := $(addprefix .build/,$(srcfiles))
.PHONY: compress clean deploy
compress: $(dstfiles)
$(dstfiles): .build/%: %
mkdir -p $(dir $#)
luamin -f $< > $#
clean:
rm -rf ./.build
deploy: .build
cp ./.build/* ~
Explanation:
The shell make function is used to run the find command that searches all subdirectories, except .build, for *.lua files. The result is assigned to the srcfiles make variable.
The addprefix make function is used to add the .build/ prefix to all words of the srcfiles make variable and assign the result to the dstfiles make variable.
The compress target is the first (real) target in the Makefile. It is thus the default goal that gets run when invoking just make. It is the same as invoking make compress. The compress target is declared as phony. This tells make that it is not a real file, just like clean and deploy. The compress target depends on all destination files. If one is missing or older than its corresponding source file, it must be rebuilt.
The make static pattern rule $(dstfiles): .build/%: %... declares a generic rule where each destination file (.build/./foo/bar/baz.lua) depends on the corresponding source file (./foo/bar/baz.lua). The recipe creates the destination directory (./foo/bar/), computed thanks to the dir make function. Then, it applies the luamin command. The recipe makes use of the $# and $< automatic variables.
I'm writing a pattern for compiling all .c file in the test directory.
Details of the directory is as follows:
./prj/makefile
./prj/test
./prj/test/test1/a.c
./prj/test/test1/b.c
./prj/test/test2/c.c
./prj/test/test2/d.c
./prj/test/e.c
...
Just a demo. This is my content of makefile:
# Find all files
rwildcard := $(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))
# All .c files
SRC_FILES := $(call rwildcard,test,*.c)
# All .o files
OBJ_FILES := $(SRC_FILES:.o=.c)
all : $(OBJ_FILES)
echo $(OBJ_FILES)
%.o : %.c
echo $# $<
Make prints No rule to make target '...'. I think make need know path of .o files and .c files. But I don't know how to setting the path, Since there is so many .c files in my prj.
Because OBJ_FILES has includes all .o files. Then I guess the pattern should be like this:
$(output_dir)/%.o : $(input_dir)/%.c
echo $# $
Since here may have many directories in ./prj/test, I cann't hardcoded it in makefile
Thanks for another friend, the above approach is right. since % can match many Multi-level directories。
We can't really solve your problem because you still have not specified where the object files should go. All in a specific directory? Always in the parent directory of the source file? Somewhere else?
Regardless of how you resolve that, you can add all your source directories to VPATH and have Make resolve the precise location while figuring out the dependencies.
VPATH=test:test/test1:test/test2
experiment: a.c d.c
echo $^
will echo test/test1/a.c test/test2/d.c with full paths.
So with this you can remove the hard-coded directory names from your %.o rule and simply specify which .o files you want built for the all target.
You can use this to get all c files in subdirectories:
$(wildcard */*.c)
Or also this to get all c files in subdirectories at any depth:
$(wildcard **/*.c)
I'd like to learn how to re-use the % sign in my makefile target's prequisite, supposing that the target is X.pdf, and the prerequesite is in X/X.tex.
To elaborate, I currently have a makefile like so:
all: foo.pdf
%.pdf: %.tex
pdflatex $*.tex
I additionally have a file foo.tex, and when I type make it will make foo.pdf by running pdflatex foo.tex.
Now for various reasons I can't control, my directory structure has changed:
my_dir
|- Makefile
|- foo
|- foo.tex
I'd like to modify my Makefile such that when it tries to make X.pdf, it looks for the file X/X.tex.
I tried the following (I tried to put '%/%.tex' to tell it to look for foo/foo.tex):
all: foo.pdf
%.pdf: %/%.tex
pdflatex $*/$*.tex
However, this yields:
No rule to make target `foo.pdf', needed by `all'. Stop.
So does %.pdf: $*/$*.tex.
If I change the %/%.tex to foo/%.tex it works as expected, but I don't want to hard-code the foo in there, because in the future I'll do all: foo.pdf bar.pdf and it should look for foo/foo.tex and bar/bar.tex.
I'm fairly new to Makefiles (experience limited to modifying someone else's to my needs), and have never done a more-than-absolutely basic one, so if anyone could give me a pointer that would help (I don't really know what words to search for in the Makefile documentation - the only ones that looked promising were % and $*, which I couldn't get to work).
you can use VPATH to specify a list of directories that make should search.
exemplary makefile:
# Find all tex files
tex := $(shell find -iname '*.tex')
# Make targets out of them
PDFS := $(notdir $(tex:%.tex=%.pdf))
# specify search folder
VPATH := $(dir $(tex))
all : $(PDFS)
%.pdf : %.tex
pdflatex $<
or even better would be to use vpath (lowercase):
vpath %.tex $(dir $(tex))
it will look only for .tex files in those directories.
I have a "lib" directory in my applications main directory, which contains an arbitrary number of subdirectories, each having its own Makefile.
I would like to have a single Makefile in the main directory, that calls each subdirectory's Makefile. I know this is possible if I manually list the subdirs, but I would like to have it done automatically.
I was thinking of something like the following, but it obviously does not work. Note that I also have clean, test, etc. targets, so % is probably not a good idea at all.
LIBS=lib/*
all: $(LIBS)
%:
(cd $#; $(MAKE))
Any help is appreciated!
The following will work with GNU make:
LIBS=$(wildcard lib/*)
all: $(LIBS)
.PHONY: force
$(LIBS): force
cd $# && pwd
If there might be something other than directories in lib, you could alternatively use:
LIBS=$(shell find lib -type d)
To address the multiple targets issue, you can build special targets for each directory, then strip off the prefix for the sub-build:
LIBS=$(wildcard lib/*)
clean_LIBS=$(addprefix clean_,$(LIBS))
all: $(LIBS)
clean: $(clean_LIBS)
.PHONY: force
$(LIBS): force
echo make -C $#
$(clean_LIBS): force
echo make -C $(patsubst clean_%,%,$#) clean
There is also a way of listing sub-directories with gmake commands only, without using any shell commands:
test:
#echo $(filter %/, $(wildcard lib/*/))
This will list all sub-directories with trailing '/'. To remove it you can use the substitute pattern:
subdirs = $(filter %/, $(wildcard lib/*/))
test:
#echo $(subdirs:%/=%)
Then to actually create rules executing makefiles in each sub-directory you can use a small trick - a phony target in a non-existent directory. I think in this case an example will tell more than any explanation:
FULL_DIRS =$(filter %/, $(wildcard lib/*/))
LIB_DIRS =$(FULL_DIRS:%/=%)
DIRS_CMD =$(foreach subdir, $(LIB_DIRS), make-rule/$(subdir))
make-rule/%:
cd $* && $(MAKE)
all: DIRS_CMD
Basically, target 'all' lists all sub-directories as prerequisites. For example, if LIB_DIRS contained lib/folder1 lib/folder2 then the expansion would look like this:
all: make-rule/lib/folder1 make-rule/lib/folder2
Then 'make', in order to execute rule 'all', tries to match each prerequisite with an existing target. In this case the target is 'make-rule/%:', which uses '$*' to extract the string after 'make-rule/' and uses it as argument in the recipe. For example, the first prerequisite would be matched and expanded like this:
make-rule/lib/folder1:
cd lib/folder1 && $(MAKE)
What if you want to call different targets than all in an unknown number of subdirectories?
The following Makefile uses macros so create a forwarding dummy-target for a number of subdirectories to apply the given target from the command line to each of them:
# all direct directories of this dir. uses "-printf" to get rid of the "./"
DIRS=$(shell find . -maxdepth 1 -mindepth 1 -type d -not -name ".*" -printf '%P\n')
# "all" target is there by default, same logic as via the macro
all: $(DIRS)
$(DIRS):
$(MAKE) -C $#
.PHONY: $(DIRS)
# if explcit targets where given: use them in the macro down below. each target will be delivered to each subdirectory contained in $(DIRS).
EXTRA_TARGETS=$(MAKECMDGOALS)
define RECURSIVE_MAKE_WITH_TARGET
# create new variable, with the name of the target as prefix. it holds all
# subdirectories with the target as suffix
$(1)_DIRS=$$(addprefix $(1)_,$$(DIRS))
# create new target with the variable holding all the subdirectories+suffix as
# prerequisite
$(1): $$($1_DIRS)
# use list to create target to fullfill prerequisite. the rule is to call
# recursive make into the subdir with the target
$$($(1)_DIRS):
$$(MAKE) -C $$(patsubst $(1)_%,%,$$#) $(1)
# and make all targets .PHONY
.PHONY: $$($(1)_DIRS)
endef
# evaluate the macro for all given list of targets
$(foreach t,$(EXTRA_TARGETS),$(eval $(call RECURSIVE_MAKE_WITH_TARGET,$(t))))
Hope this helps. Really helpfull when dealing with paralelism: make -j12 clean all in a tree with makefiles having these targets... As always: playing with make is dangerous, different meta-levels of programming are too close together ,-)