Using suffix rules in Makefile - makefile

Suppose there are a.c, b.c, c.c, tt/at.c, tt/bt.c, fff/af.c, fff/bf.c. So I make Makefile like this:
OBJS=a.o b.o c.o
SRCS=$(OBJS:.o=.cc)
OBJS_TT=at.o bt.o
SRCS_TT=tt/at.c tt/bt.c
OBJS_FFF=af.o bf.o
SRCS_FFF=fff/af.c fff/bf.c
TARGET=test
.cc.o:
gcc -c $<
$(OBJS_TT):
gcc -c $(SRCS_TT)
$(OBJS_FFF):
gcc -c $(SRCS_FFF)
all:
gcc -o $(TARGET) $(OBJS) $(OBJS_TT) $(OBJS_FFF)
If C files in tt directory are added, I have to add file name in SRCS_TT & OBJS_TT.
Is there a way to improve make file?
How do I do all c files (including files in directory) using suffix rules ?

One common solution when source files reside in different directories is to build an isomorphic directory structure in the build directory. This way you can have multiple sources with the same file name (e.g. util.cc) but in different directories and the object files won't collide because they get built into different directories.
A working example that creates an isomorphic build directory structure also with auto-generated header dependencies:
build_dir := build
all : ${build_dir}/test
.SECONDEXPANSION:
.SECONDARY:
test_srcs := a.cc b.cc x/c.cc y/d.cc
${build_dir}/test : ${test_srcs:%.cc=${build_dir}/%.o} | ${build_dir}/
g++ -o $# ${LDFLAGS} ${LDLIBS} $^
# Include the auto-generated dependencies.
-include ${test_srcs:%.cc=${build_dir}/%.d}
# Compile and generate dependency files.
${build_dir}/%.o : %.cc | $$(dir $$#)
g++ -o $# -c -MD -MP ${CPPFLAGS} ${CXXFLAGS} $<
# Directory build rules. while loop to handle races on mkdir during parallel builds.
${build_dir}/%/ : | ${build_dir}/
while ! mkdir -p $#; do true; done
# Build root directory rule.
${build_dir}/ :
mkdir -p $#
# Don't try to rebuild these, these are generated when compiling.
${build_dir}/%.d : ;
clean :
rm -rf ${build_dir}
.PHONY : all clean
Usage example:
$ mkdir x y
$ touch b.cc x/c.cc y/d.cc
$ echo "int main() {}" > a.cc
$ make
mkdir -p build/
g++ -o build/a.o -c -MD -MP a.cc
g++ -o build/b.o -c -MD -MP b.cc
while ! mkdir -p build/x/; do true; done
g++ -o build/x/c.o -c -MD -MP x/c.cc
while ! mkdir -p build/y/; do true; done
g++ -o build/y/d.o -c -MD -MP y/d.cc
g++ -o build/test build/a.o build/b.o build/x/c.o build/y/d.o
$ tree build/
build/
├── a.d
├── a.o
├── b.d
├── b.o
├── test
├── x
│   ├── c.d
│   └── c.o
└── y
├── d.d
└── d.o
2 directories, 9 files
$ make
make: Nothing to be done for 'all'.
$ make clean
rm -rf build

Related

MAKEFILE header file error when try to generate dependency file

Code structure
work
├── code
| |
| ├──inc/
| | └── main.h
| |
| └── main.c
├── _Build/
│ └── Makefile
Below is the Makefile code
WORKSPACE = ..
DIR_CODE = $(WORKSPACE)/code
DIR_BIN = $(WORKSPACE)/bin
DIR_OBJ = $(WORKSPACE)/obj
C_SRC_WITH_PATH = $(shell find $(DIR_CODE) -iname *.c)
C_SRC_ONLY_PATH = $(patsubst %/,%,$(sort $(dir $(C_SRC_WITH_PATH))))
TARGET = $(DIR_BIN)/application
.PHONY : all run clean
all : $(TARGET)
$(TARGET) : ../obj/main.o | $(DIR_BIN) ../deps
gcc $^ -o $#
../obj/main.o : ../code/main.c ../deps | $(DIR_OBJ)
gcc -c -o $# $< -MMD -MP -MF ../deps/main.d
$(DIR_BIN) $(DIR_OBJ) ../deps :
mkdir -p $#
run :
$(TARGET)
clean :
rm -rf $(DIR_BIN) $(DIR_OBJ) ../deps
-include ../deps/main.d
I am getting below error
gcc -c -o ../obj/main.o ../code/main.c -MMD -MF ../deps/main.d
../code/main.c:2:10: fatal error: main.h: No such file or directory
2 | #include "main.h"
| ^~~~~~~~
compilation terminated.
make: *** [Makefile:26: ../obj/main.o] Error 1
Although I am trying to generate the dependency file in the makefile, the file main.d is not generated and getting error for main.h file which is included in main.c file.
Any guidance is deeply appreciated.
This doesn't have anything to do with .d files etc. The compiler can't find your header file, because you haven't add the correct directory to the compile line. If your headers live in ../code/inc then you have to add this to your compile line, else how can the compiler find them?
Since you're not using any make variables to hold the compiler or flags, you'll have to add it directly to your recipe:
../obj/main.o : ../code/main.c ../deps | $(DIR_OBJ)
gcc -I../code/inc -c -o $# $< -MMD -MP -MF ../deps/main.d
There are a lot of weird things about this makefile but at least this will solve your compile error.

Getting Make to automatically detect changes in protocol buffers

I want Make to automatically compile protos when I update them, here is what I've got so far:
TARGET=main
BIN_DIR=bin
SRC_DIR=src
OBJ_DIR=obj
PROTO_DIR=protos/
PROTO_COMPILE_DIR=src/$(PROTO_DIR)
CC = g++
CFLAGS = -Wall -std=c++17 -ggdb -pipe -I.
LINKER = g++
LFLAGS = $(CFLAGS) -lprotobuf
SOURCES = $(wildcard src/*.cc) \
$(wildcard src/protos/*.cc) \
$(wildcard src/db_handler/*.cc)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.cc=$(OBJ_DIR)/%.o)
$(BIN_DIR)/$(TARGET): proto $(OBJECTS)
#mkdir -p $(BIN_DIR)/
$(LINKER) $(OBJECTS) $(LFLAGS) -o $#
$(OBJECTS): $(OBJ_DIR)/%.o : $(SRC_DIR)/%.cc
#mkdir -p obj/ obj/protos obj/db_handler
$(CC) $(CFLAGS) -c $< -o $#
.PHONY: proto
proto:
#printf "Compiling protos...\n"
#cd $(PROTO_DIR) && protoc * --grpc_out=../$(PROTO_COMPILE_DIR)\
--cpp_out=../$(PROTO_COMPILE_DIR)\
--plugin=protoc-gen-grpc="/usr/local/bin/grpc_cpp_plugin"\
&& cd ../
It successfully compiles protos; but, it does so every time, even if there are no changes in files. How can I prevent this and compile protos only if protos change?
Edit: Added project structure
├── LICENSE
├── makefile
├── protos
│   ├── client.proto
│   └── person.proto
├── README.md
└── src
├── db_handler
│   ├── db_handler.cc
│   └── db_handler.h
├── main.cc
└── protos
├── client.grpc.pb.cc
├── client.grpc.pb.h
├── client.pb.cc
├── client.pb.h
├── person.grpc.pb.cc
├── person.grpc.pb.h
├── person.pb.cc
└── person.pb.h
You want to add the source files of the protocol buffers to the right (the prerequisites) of the rule that cares about them. This is how Make understands and tracks their time stamps.
By adding the files to the prerequisites, Make will understand that this is the rule which cares about those source files.
PROTO_SOURCES := $(wildcard $(PROTO_DIR)/*.proto)
PROTOS := $(patsubst $(PROTO_DIR)/%.proto,$(PROTO_COMPILE_DIR)/%.cc,$(PROTO_SOURCES))
$(PROTOS): $(PROTO_SOURCES)
#printf "Compiling protos...\n"
#cd $(PROTO_DIR) && protoc * --grpc_out=../$(PROTO_COMPILE_DIR)\
--cpp_out=../$(PROTO_COMPILE_DIR)\
--plugin=protoc-gen-grpc="/usr/local/bin/grpc_cpp_plugin"
However, this $(PROTOS) : $(PROTO_SOURCES) is not good if you use parallel builds; because, Make will try to run the command once for each output file. So, make will run N instances of the command at the same time, which means they may clobber each other.
In order to know the fully correct solution you need to provide more information (for those not familiar with protoc). Is it a requirement that you invoke protoc once with all inputs? Or is it valid to run protoc individually on each input .proto file to get its output? Then you can write a pattern rule that generates one file at a time.
Note 1: As to your other attempt, if you use .PHONY to mark the rule then Make will rebuild the rule every time weather it needs to or not.
Note 2: You don't need the cd .. at the end of the second instruction, since it is run in a sub shell.
Thanks to #FiddlingBits I managed to figure out how to do it correctly
TARGET=main
BIN_DIR=bin
SRC_DIR=src
OBJ_DIR=obj
PROTO_DIR=protos/
PROTO_COMPILE_DIR=src/$(PROTO_DIR)
rm = rm -f
CC = g++
CFLAGS = -Wall -std=c++17 -ggdb -pipe -I.
LINKER = g++
LFLAGS = $(CFLAGS) -lprotobuf
SOURCES = $(wildcard src/*.cc) \
$(wildcard src/protos/*.cc) \
$(wildcard src/db_handler/*.cc)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.cc=$(OBJ_DIR)/%.o)
PROTOS := $($(PROTO_DIR)/%.proto=$(PROTO_COMPILE_DIR)/%.cc)
$(BIN_DIR)/$(TARGET): $(PROTOS) $(OBJECTS)
echo $(PROTOS)
#mkdir -p $(BIN_DIR)/
$(LINKER) $(OBJECTS) $(LFLAGS) -o $#
$(OBJECTS): $(OBJ_DIR)/%.o : $(SRC_DIR)/%.cc
#mkdir -p obj/ obj/protos obj/db_handler
$(CC) $(CFLAGS) -c $< -o $#
$(PROTOS):
#printf "Compiling protos...\n"
#cd $(PROTO_DIR) && protoc * --grpc_out=../$(PROTO_COMPILE_DIR)\
--cpp_out=../$(PROTO_COMPILE_DIR)\
--plugin=protoc-gen-grpc="/usr/local/bin/grpc_cpp_plugin"\
&& cd ../
.PHONY: clean
clean:
#$(rm) -r $(OBJ_DIR)/*
#$(rm) -r $(BIN_DIR)/*
#printf "Cleanup complete!\n"

*.c not rebult when dependency x.h changed, using auto-generated *.d file

I am facing one issue while handling dependency files during compilation. I am just giving you the scenario which I faced in my project.
I have two C source files called a.c, b.c which includes one header file called c.h. I ran makefile which has instructions to compile both files. I can successfully compile the a.c file, but I have seen some failures while comping b.c which requires to do some changes in c.h to fix that issue. After I made change in c.h and trigger build (incremental build) the a.c file should compile again, too, right? Because a.c also depends on the c.h file.
I followed all dependency mechanism (creating auto dependency files & including .d files, etc)
DEPSALL := $(wildcard $(patsubst %,%.d,$(basename $(TGTFILES)/*.c)))
-include $(DEPSALL)
$(TGTFILES)/%.o: $(TGTFILES)/%.c
mkdir -p $(#D)
$(CC64) -MT $# -MMD -MP -MF $(patsubst %,%.d,$(basename $#)) -o $(#) -c $(CFLAGS64) $<
...
...
Am I missing something here? I want to rebuild all the .c files which are including the particular header file which I changed.
Your question is incomplete: you do not really describe the problem you are facing (but we can probably guess that object files are not rebuilt while they should) and the part of the Makefile you show is not sufficient to understand what your goals are.
Anyway, first of all, this expression:
DEPSALL := $(wildcard $(patsubst %,%.d,$(basename $(TGTFILES)/*.c)))
is uselessly complex. It is equivalent to the much simpler and easier to understand:
DEPSALL := $(wildcard $(TGTFILES)/*.d))
Similarly, in your compilation recipe you can replace:
$(patsubst %,%.d,$(basename $#))
by:
$(TGTFILES)/$*.d
But let's go back to your main problem (at least what I guess is your main problem): when modifying your header file, some object files are not rebuilt while they should.
My guess is that you think:
DEPSALL := $(wildcard $(patsubst %,%.d,$(basename $(TGTFILES)/*.c)))
will assign to DEPSALL a list of dependency files, one per source file, like this other form would do:
DEPSALL := $(patsubst %.c,%.d,$(wildcard $(TGTFILES)/*.c))
If this is what you think, then you are wrong. Your version will assign to DEPSALL the list of dependency files that currently exist in $(TGTFILES) when you invoke make. If some (or all) are missing, some object files will not be rebuilt...
I suggest that you carefully read this excellent post about Auto-Dependency Generation. If you adapt it to your set-up, you should end up with something like:
TGTFILES := tgtfiles
SRCS := $(wildcard $(TGTFILES)/*.c)
OBJS := $(patsubst %.c,%.o,$(SRCS))
DEPS := $(patsubst %.c,%.d,$(SRCS))
INCLUDES := include
CFLAGS += -I$(INCLUDES)
.PHONY: objs clean
objs: $(OBJS)
%.o: %.c
%.o: %.c %.d
$(CC) -MT $# -MMD -MP -MF $*.Td $(CFLAGS) $(CPPFLAGS) -c -o $# $<
#mv -f $*.Td $*.d && touch $#
%.d: ;
.PRECIOUS: %.d
clean:
rm -f $(OBJS) $(DEPS)
include $(DEPS)
Several aspects may look strange, useless or even completely wrong. But if you carefully read the above mentioned post you will see that it perfectly makes sense. Demo:
$ tree
.
├── Makefile
├── include
│   └── c.h
└── tgtfiles
├── a.c
└── b.c
2 directories, 4 files
$ make
cc -MT tgtfiles/b.o -MMD -MP -MF tgtfiles/b.Td -Iinclude -c -o tgtfiles/b.o tgtfiles/b.c
cc -MT tgtfiles/a.o -MMD -MP -MF tgtfiles/a.Td -Iinclude -c -o tgtfiles/a.o tgtfiles/a.c
$ tree
.
├── Makefile
├── include
│   └── c.h
└── tgtfiles
├── a.c
├── a.d
├── a.o
├── b.c
├── b.d
└── b.o
2 directories, 8 files
$ cat tgtfiles/a.d
tgtfiles/a.o: tgtfiles/a.c include/c.h
include/c.h:
$ make
make: Nothing to be done for 'objs'.
$ touch include/c.h
$ make
cc -MT tgtfiles/b.o -MMD -MP -MF tgtfiles/b.Td -Iinclude -c -o tgtfiles/b.o tgtfiles/b.c
cc -MT tgtfiles/a.o -MMD -MP -MF tgtfiles/a.Td -Iinclude -c -o tgtfiles/a.o tgtfiles/a.c
wildcard function application doesn't look right. Should probably be:
DEPSALL := $(patsubst %,%.d,$(basename $(wildcard $(TGTFILES)/*.c)))

Makefile - move object files

After a bit of searching, I've managed to throw together the following Makefile:
CC = gcc
CFLAGS = -c -Wall
LDFLAGS =
SOURCEFILES = main.c
SOURCES = src/$(SOURCEFILES)
OBJECTS = $(SOURCES:.c=.o)
EXECUTABLE = netsim
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.c.o:
$(CC) $(CFLAGS) $< -o $#
clean:
rm -rf netsim $(OBJECTS)
I would like to be able to move my object files into another directory, but have been struggling with getting that to work. What am I missing?
The trick is to not move your objects.
You should build it and use it from where they are built.
For example you have the following directory structure:
$ tree .
├── Makefile
├── include
│   └── common_head.h
├── obj
└── src
├── foo.c
└── main.c
Manual execution:
$ gcc -o ./obj/foo.o -c ./src/foo.c -I ./include # Build Object #
$ gcc -o ./obj/main.o -c ./src/main.c -I ./include
$ gcc -o exe ./obj/foo.o ./obj/main.o # Build Executable #
Makefile to simulate the above
C_FLAGS := -g -Wall -Wextra
CC := gcc
RM := rm
LINKFLAGS := -lanylibrary
.PHONY: $(TARGET)
.PHONY: clean
VPATH:= ./src/ ./obj/ ./include/
# Path for .c , .h and .o Files
SRC_PATH := ./src/
OBJ_PATH := ./obj/
INC_PATH := -I ./include
# Executable Name
TARGET := exe
# Files to compile
OBJ1 := foo.o \
main.o
OBJ := $(patsubst %,$(OBJ_PATH)%,$(OBJ1))
# Build .o first
$(OBJ_PATH)%.o: $(SRC_PATH)%.c
#echo [CC] $<
#$(CC) $(C_FLAGS) -o $# -c $< $(INC_PATH)
# Build final Binary
$(TARGET): $(OBJ)
#echo [INFO] Creating Binary Executable [$(TARGET)]
#$(CC) -o $# $^ $(LINKFLAGS)
# Clean all the object files and the binary
clean:
#echo "[Cleaning]"
#$(RM) -rfv $(OBJ_PATH)*
#$(RM) -rfv $(TARGET)
So when you do a Make
$ make -B
[CC] src/foo.c
[CC] src/main.c
[INFO] Creating Binary Executable [exe]
To see a dry-run use make -n
$ make clean ; make -n
g++ -g -Wall -Wextra -o obj/foo.o -c src/foo.c -I ./include
g++ -g -Wall -Wextra -o obj/main.o -c src/main.c -I ./include
g++ -o exe obj/foo.o obj/main.o -lanylibrary
So after building your directory structure should look like this.
$ tree .
├── Makefile
├── exe
├── include
│   └── common_head.h
├── obj
│   ├── foo.o
│   └── main.o
└── src
├── foo.c
└── main.c
So from my previous answer.
You don't have to use any PHONY move and also no objects are recreated unnecessarily.
Something like this?
SOURCES = src/main.c
OBJECTS = obj/main.o
...
obj/%.o: src/%.c
$(CC) $(CFLAGS) $< -o $#
Once that's working, you can add further tricks, like this:
OBJECTS = $(patsubst src/%.c, obj/%.o, $(SOURCES)

How to make makefile find target in subdirectory makefile

How to make top level Makefile to call all targets in subdirectory Makefile ?
My folder structure is like this
/Makefile
/src/Makefile
I code all targets in /src/Makefile, but now I want to write another /Makefile to ease my work. How to write this top level Makefile ?
You could write in your TOP makefile use
all:
#$(MAKE) -C src
and if you do NOT use makefile in sub dir, for example, you use somename.mk, you could use
all:
#$(MAKE) -C src -f somename.mk
I show you my example, my DIR looks like:
TOPDIR-- Makefile
|
|-- debug
| |-- debug.c
| |-- debug.h
| |-- debug.mk
| |-- instrument.c
| `-- uart_print.c
|-- driver
| |-- driver.c
| |-- driver_ddi.c
| |-- driver_ddi.h
| |-- driver.h
| `-- driver.mk
|-- include
| `-- common.h
|-- Makefile
|-- mw
| |-- manager.c
| `-- mw.mk
|-- root
| |-- main.c
| `-- root.mk
and my TOP makefile looks like:
MAKE_DIR = $(PWD)
ROOT_DIR := $(MAKE_DIR)/root
DRV_DIR := $(MAKE_DIR)/driver
INCLUDE_DIR := $(MAKE_DIR)/include
DEBUG_DIR := $(MAKE_DIR)/debug
INC_SRCH_PATH :=
INC_SRCH_PATH += -I$(ROOT_DIR)
INC_SRCH_PATH += -I$(DRV_DIR)
INC_SRCH_PATH += -I$(INCLUDE_DIR)
INC_SRCH_PATH += -I$(DEBUG_DIR)
LIB_SRCH_PATH :=
LIB_SRCH_PATH += -L$(MAKE_DIR)/libs
COLOR_ON = color
COLOR_OFF =
CC = $(COLOR_ON)gcc
#CC = $(COLOR_OFF)gcc
LD = ld
LINT = splint
LIBS := -ldriver -ldebug -lmw -lm -lpthread
CFLAGS :=
CFLAGS += $(INC_SRCH_PATH) $(LIB_SRCH_PATH)
CFLAGS += -Wall -O -ggdb -Wstrict-prototypes -Wno-pointer-sign -finstrument-functions -fdump-rtl-expand
CFLAGS += -DDEBUG -D_REENTRANT
LDFLAGS :=
export MAKE_DIR CC LD CFLAGS LDFLAGS LIBS LINT INC_SRCH_PATH
all:
#$(MAKE) -C debug -f debug.mk
#$(MAKE) -C driver -f driver.mk
#$(MAKE) -C mw -f mw.mk
#$(MAKE) -C root -f root.mk
.PHONY: clean
clean:
#$(MAKE) -C debug -f debug.mk clean
#$(MAKE) -C driver -f driver.mk clean
#$(MAKE) -C mw -f mw.mk clean
#$(MAKE) -C root -f root.mk clean
.PHONY: lint
lint:
$(MAKE) -C debug -f debug.mk lint
it will call sub DIR *.mk during the compile. The sub DIR makefile, I just write a simple example for you reference:
LIB = $(MAKE_DIR)/libs/yourmodulename.a
SRCS = $(wildcard *.c)
OBJS = $(patsubst %.c, %.o, $(SRCS))
$(LIB): $(OBJS)
#mkdir -p ../libs
#$(AR) cr $# $^
#echo " Archive $(notdir $#)"
$(OBJS): $(SRCS)
#$(CC) $(CFLAGS) -c $^
#echo " CC $(OBJS)"
.PHONY: clean
clean:
#$(RM) -f $(LIB) $(OBJS)
#$(RM) -f *.expand
#echo " Remove Objects: $(OBJS)"
#echo " Remove Libraries: $(notdir $(LIB))"
.PHONY: lint
lint:
$(LINT) $(INC_SRCH_PATH) $(SRCS)
for the makefile, which generate the target file is little bit different, because I use the sub makefile to generate LIB file, and I use a root.mk to generate target:
PROG = ../prog/DEMO
SRCS = $(wildcard *.c)
OBJS = $(patsubst %.c, %.o, $(SRCS))
$(PROG): $(SRCS)
#mkdir -p ../prog
#$(CC) $^ $(CFLAGS) -Wl,-Map=$(PROG).map $(LIBS) -o $#
#echo " Generate Program $(notdir $(PROG)) from $^"
.PHONY: clean
clean:
#$(RM) -f $(OBJS) $(PROG)
#$(RM) -f *.expand
#$(RM) -rf ../prog ../libs
#echo " Remove Objects: $(OBJS)"
#echo " Remove Libraries: $(notdir $(PROG))"
Beware, note there are, at least, two ways of doing this.
GNU Changedir Option
One way is using a specific feature of GNU Make, the -C option allow to change your compile directory and reach another one:
all:
make -C dir
The make manual says:
-C dir, --directory=dir
Change to directory dir before reading the makefiles or doing anything
else. If multiple -C options are specified, each is interpreted relative
to the previous one: -C / -C etc is equivalent to -C /etc. This is
typically used with recursive invocations of make.
You can also combine this option by calling a specific target inside the targeted directory. For example, the following target will get into the src/ directory and call make with the clean target:
clean:
#rm -f *.o
make -C src/ clean
POSIX Way
The problem with the GNU way of doing is that it only works with GNU Make, and not the standard Make. If you may use another Make (for whatever reason), you better consider doing it in a more POSIX way.
In POSIX Make, you have to rely more on the cd command, like this:
all:
cd src/ && make
Note that, I used && and not ;. It is quite important to avoid infinite recursive calls to make. Indeed, cmd1 ; cmd2 will execute sequentially cmd1 and cmd2 whatever is the result of each command, where cmd1 && cmd2 will execute sequentially cmd1, and cmd2 will be executed only if cmd1 returned an EXIT_SUCCESS. In our case, imagine that the first cd failed because the directory has been removed. Then, the initial makefile will be executed again and again in an infinite recursive loop.
Anyway, this POSIX manner, is the more robust way of descending in subdirectories and execute other Makefile. I would recommend you to use it better than relying on an option linked to GNU Make.
And if you want to do multiple dirs easily:
SUBDIRS=dir1 dir2
all::
#echo make all
$(foreach var,$(SUBDIRS),echo $(var): ; cd $(var)/ && make $# && cd ..;)
clean:
#echo make clean
$(foreach var,$(SUBDIRS),echo $(var): ; cd $(var)/ && make $# && cd ..;)

Resources