How to define a pattern which is valid for all files in the directory including subdirectories? - makefile

I'm writing a pattern for compiling all .c file in the test directory.
Details of the directory is as follows:
./prj/makefile
./prj/test
./prj/test/test1/a.c
./prj/test/test1/b.c
./prj/test/test2/c.c
./prj/test/test2/d.c
./prj/test/e.c
...
Just a demo. This is my content of makefile:
# Find all files
rwildcard := $(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))
# All .c files
SRC_FILES := $(call rwildcard,test,*.c)
# All .o files
OBJ_FILES := $(SRC_FILES:.o=.c)
all : $(OBJ_FILES)
echo $(OBJ_FILES)
%.o : %.c
echo $# $<
Make prints No rule to make target '...'. I think make need know path of .o files and .c files. But I don't know how to setting the path, Since there is so many .c files in my prj.
Because OBJ_FILES has includes all .o files. Then I guess the pattern should be like this:
$(output_dir)/%.o : $(input_dir)/%.c
echo $# $
Since here may have many directories in ./prj/test, I cann't hardcoded it in makefile
Thanks for another friend, the above approach is right. since % can match many Multi-level directories。

We can't really solve your problem because you still have not specified where the object files should go. All in a specific directory? Always in the parent directory of the source file? Somewhere else?
Regardless of how you resolve that, you can add all your source directories to VPATH and have Make resolve the precise location while figuring out the dependencies.
VPATH=test:test/test1:test/test2
experiment: a.c d.c
echo $^
will echo test/test1/a.c test/test2/d.c with full paths.
So with this you can remove the hard-coded directory names from your %.o rule and simply specify which .o files you want built for the all target.

You can use this to get all c files in subdirectories:
$(wildcard */*.c)
Or also this to get all c files in subdirectories at any depth:
$(wildcard **/*.c)

Related

Make doesn't realize that it's done

This is my first ever post to stackoverflow. It ought to be straightforward, but I have this problem whenever I try to write a makefile and I've never been able to figure out a satisfactory solution. Apologies if there is already a solution somewhere on the site. I couldn't find one.
What I'm trying to do is as follows:
Search my src directory for matching source files.
Compile the src code into a sandbox.
Here's my directory structure:
makefile
src1
file1.src
file2.src
src2
file3.src
subfolder
src3
file4.src
file5.src
And here's my makefile:
BUILDDIR := ./sandbox
SRC_DIRS := ./
SRCS := $(shell find $(SRC_DIRS) -name *.src)
OBJS := $(addprefix $(BUILDDIR)/o., $(notdir $(SRCS) ) )
# make print-X prints value of variable X
print-%: ; #echo $* = $($*)
.PHONY: help
help:
#echo "make <all|clean>"
.PHONY: all
all: $(OBJS)
#echo "compilation done"
$(OBJS) : $(SRCS) $(BUILDDIR)/.create
#echo "\"compiling\" $< to produce $#"
cp $< $#
$(BUILDDIR)/.create:
#echo "creating sandbox"
mkdir -p $(BUILDDIR) && cd $(BUILDDIR)
#touch $#
.PHONY: clean
clean:
#echo "deleting sandbox"
#rm -rf $(BUILDDIR)
If I type make all, the file works as expected. However, if I type make all again, instead of saying everything is up to data, I end up with the following contents in the sandbox:
o.file1.src o.file2.src o.file3.src o.file4.src o.file5.src o.o.file1.src o.o.file2.src o.o.file3.src o.o.file4.src o.o.file5.src
And the process of creating objects of objects continues recursively as many times as I type make.
Any help would be appreciated.
Incidentally, please don't post solutions that rely on the build in compile functions of make. I'm looking for a general solution that can be used for any task. For example, in this instance, I'm trying to read the source files into a tool using its command line interface.
Well, first, by having your sandbox as a subdirectory of your source directory, then using find on the source directory, every time you run it you're going to find all the files in both the source directory and all its subdirectories, including the sandbox. If the built files in the sandbox have the same names as the files in the source directories, the find will find them all.
Maybe instead of:
SRCS := $(shell find $(SRC_DIRS) -name *.src)
you want something like:
SRCS := $(shell find $(SRC_DIRS) -name $(notdir $(BUILDDIR)) -prune -o -name *.src -print)
Or, alternatively, don't make your sandbox a subdirectory of your source directory. Or, make sure that whatever name you give to the files in the sandbox directory won't match the *.src pattern you give to find.
But beyond that this is wrong:
$(OBJS) : $(SRCS) $(BUILDDIR)/.create
Suppose SRCS is foo.src bar.src, which means OBJS is sandbox/o.foo.src sandbox/o.bar.src. Then the above expands to this:
sandbox/o.foo.src sandbox/o.bar.src : foo.src bar.src sandbox/.create
This is a common mistake; people seem to think that make will go through the targets and prerequisites and match them up, so the first target depends on the first prerequisite and the second target depends on the second one etc. but of course this cannot work correctly and that's not how make works. Make treats the above as if you'd written one rule for each target, with the same prerequisites; like this:
sandbox/o.foo.src : foo.src bar.src sandbox/.create
sandbox/o.bar.src : foo.src bar.src sandbox/.create
You can see this won't do what you want at all, since the $< will always be foo.src which is clearly wrong.
You need to write a single pattern rule that will build ONE target. Then make sure the pattern applies to all the targets.
You have made things hard for yourself by trying to "flatten" a directory structure of multiple source subdirectories, into a single level of target directory (by using the $(notdir $(SRCS))). Because of this, there's no pattern that will match the same target and directory, unless you write a separate rule for every subdirectory.
Luckily there is a solution for this: VPATH. This should work for you:
VPATH := $(sort $(dir $(SRCS))
$(BUILDDIR)/o.%.src : %.src $(BUILDDIR)/.create
#echo "\"compiling\" $< to produce $#"
cp $< $#
The VPATH tells make to go look in all the directories that it found any sources in, whenever it can't find one to build.
The basic problem is that your SRCS is all files in all subdirectories that match the pattern *.src (when you run make). That means that all your object files ($(OBJS)) also match, so they copied as well.
The solution is to change your SRCS pattern so it does not match the "object" files in the build directory. Possibilities:
SRCS := $(wildcard *.src)
or
SRCS := $(shell find $(SRC_DIRS) -name $(notdir $(BUILDDIR)) -prune -false -o -name *.src)
or change the names of your "object" files so they don't end in .src
If I type make all, the file works as expected.
By which I take you to mean that directory ./sandbox exists and contains these files:
o.file1.src
o.file2.src
o.file3.src
o.file4.src
o.file5.src
However, if I type make all again, instead of saying everything is up to data, I end up with the following contents in the sandbox:
o.file1.src o.file2.src o.file3.src o.file4.src o.file5.src o.o.file1.src o.o.file2.src o.o.file3.src o.o.file4.src o.o.file5.src
Of course you do, because everything is not up to date at that point, according to the target list you create. This line ...
SRCS := $(shell find $(SRC_DIRS) -name *.src)
... defines SRCS as a list of all the files under path ./ (which is the value of SRC_DIRS) that ends in .src. That includes any such files in ./sandbox, which include all the files placed there by the first make run. You generate a corresponding target file to build for each source, and those targets corresponding to sources built by the previous make run will not, in general, exist yet. So make builds them, just as you instructed it.
The best solution, short of abandoning that automatic source identification altogether, would probably be to change your naming scheme so that outputs cannot be mistaken so easily for sources. For example, if you want the output names to have the form foo.src, then have the corresponding input named something like foo.src.in. In that particular case, you could convert from source names to target names with
OBJS := $(addprefix $(BUILDDIR)/o., $(basename $(notdir $(SRCS) ) ) )
Alternatively, you could modify the find command to skip the sandbox directory, maybe by modifying SRC_DIRS:
SRC_DIRS = src1 src2 subfolder/src3
(These specific alternatives are not mutually exclusive.)

How to compile files that reside in different directory in Makefile?

I have seen this questions asked before but was not able to decipher those answers.
Lets say I reside in working directory, lets call it proj and this proj directory contains src folder which contains all the *.cpp files. I want to compile those file staying on the proj directory because in future I will be creating bin directory and placing the *.o and binary in bin.
So my proj directory currently contains : Makefile and src
What I have done so far is :
SOURCE = src
# This gives the path to the proj directory
CURRENT_DIR = $(shell pwd)
# This gives list of all the *.cpp files
SRC = $(shell cd $(SOURCE) && echo *.cpp)
# Here all the name of the files stored in SRC are converted from *.cpp to *.o
OBJS = $(SRC:.cpp=.o)
.PHONY: all
all: $(TARGE)
# use the content of SRC to compile
$(TARGET): $(OBJS)
$(info $(OBJS))
$(OBJS): $(SRC)
$(CC) $(FLAGS) -c $?
When I try to run the make command it says
make: *** No rule to make target 'xxx.cpp', needed by 'xxx.o'. Stop
Now I know what it is trying to say. It gives error because although it knows the name of the file, since the file is not in the current directory makefile does not know about src folder and hence have no clue about the *.cpp files.
So my question is: Is there any macros or trick to use in makefile to make sure makefile see the xxx.cpp in src folder while staying in the current directory( I don't want to specify the folder by hand here)?

.cpp to .o files in Makefile

How can I generate .o file corresponding to all the .cpp files in a directory using Makefile?
I have a directory that contains .cpp files. Now, I want to compile them in .o files. The name of the .o files should be same as corresponding .cpp files. What should I do?
Actually, I already had an implementation but I am not sure how it work
SRCS := $(wildcard $(SRCDIR)/*.cpp)
OBJS := $(SRCS:$(SRCDIR)/%.cpp=$(OBJDIR)/%.o)
$(OBJS): $(OBJDIR)/%.o : $(SRCDIR)/%.cpp
// Recipe //
Try this
SRCS=$(wildcard *.cpp)
OBJS=$(SRCS:.cpp=.o )
%.o: %.cpp
$(CC) -c $< -o $#
Let me add some more explaination to it.
SRCS := $(wildcard $(SRCDIR)/*.cpp) - It will list all the .cpp files under SRCDIR directory and will assign to SRCS
OBJS := $(SRCS:$(SRCDIR)/%.cpp=$(OBJDIR)/%.o) - It will replace all the files with .cpp listed from the above statement to .o ex: main.cpp would be changed to main.o and assigned to OBJS
$(OBJDIR)/%.o : $(SRCDIR)/%.cpp - Object files depends on respective .cpp files and in the rule you can write rule to create object files
The % character can be used for wildcard pattern-matching, to provide generic targets. For example:
%.o: %.c
[TAB] actions
When % appears in the dependency list, it is replaced with the same string that was used to perform substitution in the target.
If the above explanation is not clear just go through the Makefile Basics once and try writing without using special variables and then go to the complex rules.

Generic target/rule to build all source files from a list, outputting objects to one directory

I am trying to make one generic target in my makefile that will builds sources from mixed directories and output the object files to on single directory.
We have a source structure that is mixed in various directories (like said above, below is just an example)
SRCS = ../a/b/source1.c \
b/source2.c \
../c/source3.c
But I would like all of the object files to output to the directory ./objs (same directory level as 'b')
To do this I was trying the following
OBJS = $(addprefix objs/, $(notdir $(SRCS:.c=.o)))
$(OBJS): %.o : $(filter %/$(basename $(notdir %)).c, $(SRCS))
echo "dependencies: $^" # shows up empty
$(CC) $(filter %/$(basename $(notdir $#)).c, $(SRCS)) -o $# # This works and finds the proper source file
$(CC) $^, $(SRCS)) -o $# # I would like to use this, but as I said the dependencies show up blank
There is a weird problem with this however, and I don't understand where the problem is.
In the dependency it doesn't match anything, but in the recipe it does match properly.
Now the weird part (for me atleast). If I try and test out by hard coding one of the paths then it match for ALL files in that path
$(OBJS): %.o : $(filter ../a/b/$(basename $(notdir %)).c, $(SRCS)) # matches for all files in "../a/b" directory
But using SECONDEXPANSION and hardcoding the directory it works
.SECONDEXPANSION:
$(OBJS): %.o : $$(filter ../a/b/$$(basename $$(notdir %)).c, $(SRCS))
And also not using SECONDEXPANSION and hardcoding the source file name works
$(OBJS): %.o : $(filter %source1.c, $(SRCS)) # Hardcoding source1.c works for source1.c
But it seems like I can't combine to two do what I want for some reason. I have tried secondexpansion stuff (thoguht I'm not really sure why I would need it in this case) and could never get anything working that way either.
I am trying to avoid manually declaring targets for each file individually i.e.
objs/source1.o : ../a/b/source1.c
Because our real world example has tons of files and it would be nice to have less to maintain. I feel like I am very close to getting it.
I am using Cygwin with GNU Make 4.0.
After googling a few more times I finally came across the fix here:
http://lists.gnu.org/archive/html/help-make/2010-09/msg00062.html
I still don't know exactly why I needed to use the SECONDEXPANSION ($$-ness) at all but in practice it doesn't work without it. But basically I needed to create a variable for the '%' sign. Doing the following works for me.
SRCS = ../a/b/source1.c \
b/source2.c \
../c/source3.c
OBJS = $(addprefix objs/, $(notdir $(SRCS:.c=.o)))
.SECONDEXPANSION:
PERCENT = %
$(OBJS): %.o : $$(filter $$(PERCENT)/$$(notdir %).c, $(SRCS))
$(CC) $< -o $#
This now builds source1.c, source2.c, and source3.c and outputs the object files into the objs/ directory.
What I didn't mention in my question but I knew all along was that this will only work if you have unique file names for all source files. But we are okay with that limitation (obviously).

Reuse percent sign in Makefile prerequesite as subdirectory name

I'd like to learn how to re-use the % sign in my makefile target's prequisite, supposing that the target is X.pdf, and the prerequesite is in X/X.tex.
To elaborate, I currently have a makefile like so:
all: foo.pdf
%.pdf: %.tex
pdflatex $*.tex
I additionally have a file foo.tex, and when I type make it will make foo.pdf by running pdflatex foo.tex.
Now for various reasons I can't control, my directory structure has changed:
my_dir
|- Makefile
|- foo
|- foo.tex
I'd like to modify my Makefile such that when it tries to make X.pdf, it looks for the file X/X.tex.
I tried the following (I tried to put '%/%.tex' to tell it to look for foo/foo.tex):
all: foo.pdf
%.pdf: %/%.tex
pdflatex $*/$*.tex
However, this yields:
No rule to make target `foo.pdf', needed by `all'. Stop.
So does %.pdf: $*/$*.tex.
If I change the %/%.tex to foo/%.tex it works as expected, but I don't want to hard-code the foo in there, because in the future I'll do all: foo.pdf bar.pdf and it should look for foo/foo.tex and bar/bar.tex.
I'm fairly new to Makefiles (experience limited to modifying someone else's to my needs), and have never done a more-than-absolutely basic one, so if anyone could give me a pointer that would help (I don't really know what words to search for in the Makefile documentation - the only ones that looked promising were % and $*, which I couldn't get to work).
you can use VPATH to specify a list of directories that make should search.
exemplary makefile:
# Find all tex files
tex := $(shell find -iname '*.tex')
# Make targets out of them
PDFS := $(notdir $(tex:%.tex=%.pdf))
# specify search folder
VPATH := $(dir $(tex))
all : $(PDFS)
%.pdf : %.tex
pdflatex $<
or even better would be to use vpath (lowercase):
vpath %.tex $(dir $(tex))
it will look only for .tex files in those directories.

Resources