GNU make path substitution (directory flattening) - makefile

I have a makefile that gives me the source files in a hierarchy.
SRCS := $(wildcard $(SRC_DIR)/*.c $(SRC_DIR)/*/*.c)
gives me
./src/main.c ./src/add/add.c ./src/sub/sub.c
I want to flatten the object files into a single "obj" directory.
Of course
OBJS := $(SRCS:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.obj)
gives me
./obj/main.obj ./obj/add/add.obj ./obj/sub/sub.obj
instead of desired
./obj/main.obj ./obj/add.obj ./obj/sub.obj
Question: How do I get rid of additional source directory levels?
Never had to use complex substitution so far. My intuitive try with additional "/%" in pattern
# won't work as expected:
OBJS := $(SRCS:$(SRC_DIR)/%/%.c=$(OBJ_DIR)/%.obj)
produces no meaningful result (${OBJS} becomes same as ${SRCS}).
All examples I found so far only have single occurance of "%" in match pattern.
MAK_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
SRC_DIR = $(MAK_DIR)./src
OBJ_DIR = $(MAK_DIR)./obj
# gives: ./src/main.c ./src/add/add.c ./src/sub/sub.c
SRCS := $(wildcard $(SRC_DIR)/*.c $(SRC_DIR)/*/*.c)
# gives: ./obj/main.obj ./obj/add/add.obj ./obj/sub/sub.obj
OBJS := $(SRCS:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.obj)
.PHONY : all
all :
#echo $(SRCS)
#echo $(OBJS)

Just use the notdir function, and also patsubst, more powerful (and that I find easier to understand than the shorthand):
OBJS := $(patsubst %.c,$(OBJ_DIR)/%.obj,$(notdir $(SRCS)))
But it is not the whole story because later on you will probably want to do something like:
$(OBJ_DIR)/%.obj: $(SRC_DIR)/%.c
which will not work any more. Not mentioning the fact that you could have several source files with the same name in different directories. Assuming you do not have such names conflicts you can generate all your dependencies using foreach-eval-call:
MAK_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
SRC_DIR := $(MAK_DIR)./src
OBJ_DIR := $(MAK_DIR)./obj
SRCS := $(shell find $(MAK_DIR) -type f -name '*.c')
OBJS := $(patsubst %.c,$(OBJ_DIR)/%.obj,$(notdir $(SRCS)))
.PHONY: objs
objs: $(OBJS)
# $(1): source file
define DEP_rule
$(1)-obj := $$(patsubst %.c,$$(OBJ_DIR)/%.obj,$$(notdir $(1)))
$(1)-dep := $$(patsubst %.c,$$(OBJ_DIR)/%.d,$$(notdir $(1)))
$$($(1)-obj): $(1) $$($(1)-dep)
endef
$(foreach src,$(SRCS),$(eval $(call DEP_rule,$(src))))
Just remember that the DEP_rule macro gets expanded twice, thus the $$.

Related

Capture two folders in prerequisite for Makefile's target

For my Makefile, my prerequisite is of the form
$(Root)a/b/c/d/File.source
and I'd like to target
a/c.zip
knowing that there are multiple folders for a and c, that b is fixed, that d can vary, but that the source file is always called File.source.
Taking inspiration from this answer, I've tried
As:= $(notdir $(shell find $(Root) -mindepth 1 -maxdepth 1 -type d))
define RULE
$1/%.zip: $1/b/%/*/File.source
echo "Test"
endef
$(foreach a, $(As), $(eval $(call RULE, $(a))))
But it does not seem to work.
I think this could be relatively simple, given the fact how patterns match. Having a rule
%.zip: b/%.foo
will cause make a/c.zip to search for a/b/c.foo file. With this in mind we can write a quite simple Makefile:
$ cat Makefile
# Extract directories that contain File.source files
sourcedirs := $(dir $(wildcard $(ROOT)*/b/*/*/File.source))
# Generate archive names: drop the /b/ part and add .zip extension
archives := $(patsubst %/,%.zip,$(subst /b/,/,$(dir $(sourcedirs:/=))))
%.zip: b/%/*/File.source
echo Making $# from $<
all: $(archives)
The output:
$ find . -name File.source
./a/b/c/d/File.source
./foo/b/bar/baz/File.source
./foo/b/cfg/d/File.source
$ make -s
Making foo/bar.zip from foo/b/bar/baz/File.source
Making foo/cfg.zip from foo/b/cfg/d/File.source
Making a/c.zip from a/b/c/d/File.source
Let's first find all File.source files that match the $(Root)*/b/*/*/File.source pattern:
FILES := $(shell find $(Root)*/b/* -maxdepth 2 -mindepth 2 -type f -name File.source)
Then, let's create a macro for each of them (I over-exaggerated the number of intermediate variables for easier understanding, you can simplify a bit, if you wish):
# $(1): File.source path
define MY_RULE
$(1)-dirs := $$(subst /, ,$(1))
$(1)-Roota := $$(word 1,$$($(1)-dirs))
$(1)-a := $$(patsubst $$(Root)%,%,$$($(1)-Roota))
$(1)-c := $$(word 3,$$($(1)-dirs))
$(1)-target := $$($(1)-a)/$$($(1)-c).zip
$$($(1)-target): $(1)
#echo "Test for $$< (target $$#)"
endef
And finally, let's call the macro on each File.source file:
$(foreach f,$(FILES),$(eval $(call MY_RULE,$(f))))
All in all, plus a test phony default goal for easier testing:
FILES := $(shell find $(Root)*/b/* -maxdepth 2 -mindepth 2 -type f -name File.source)
.PHONY: test
.DEFAULTGOAL := test
# $(1): File.source path
define MY_RULE
$(1)-dirs := $$(subst /, ,$(1))
$(1)-Roota := $$(word 1,$$($(1)-dirs))
$(1)-a := $$(patsubst $$(Root)%,%,$$($(1)-Roota))
$(1)-c := $$(word 3,$$($(1)-dirs))
$(1)-target := $$($(1)-a)/$$($(1)-c).zip
$$($(1)-target): $(1)
#echo "Test for $$< (target $$#)"
test: $$($(1)-target)
endef
$(foreach f,$(FILES),$(eval $(call MY_RULE,$(f))))
Note: there is a possibility that you have more than one File.source file with the same corresponding target, because of the d part of your pattern. If this happens GNU make will issue warnings and use only the last encountered rule for each such target. There are ways to raise an error, if you prefer, but as all this is already quite complicated, let's leave it as an exercise on make conditionals.

Variables in non-recursive Makefiles

I'm using GNU Make and attempting to design Makefiles using non-recursive approach. The issue I have - there doesn't seem to be a way to limit the scope of variables in different Makefiles.
For example, if I have two Makefiles for modules libA and libB
libA Makefile.inc:
src_dir := libA/src
inc_dir := libA/inc
libB Makefile.inc:
src_dir := libB/src
inc_dir := libB/inc
Then when I include the above Makefiles into a master Makefile
include libA/Makefile.inc
include libB/Makefile.inc
Values in variables src_dir and inc_dir are overwritten by the very last Makefile that was included. OK this is probably expected since variables are global in scope here, and this messes up build commands that use those variables, i.e. build command for libA finds variable values for libB.
A way around it is to create unique variables for each Makefile
libA Makefile.inc:
src_dir_libA := libA/src
inc_dir_libA := libA/inc
libB Makefile.inc:
src_dir_libB := libB/src
inc_dir_libB := libB/inc
This solves the issue, but it's a bit awkward to use, since each variable has to be renamed. Does anyone know if there is a better way to solve this, e.g. if GNU Make has some notion of scope or namespace? I've looked at documentation, but can't seem to find anything of that sort. There are target-specific variables, but they appear to have nasty side-effects.
OK, just to be very specific, below is an example Makefile
dep_all :=
all:
# Begin Makefile.inc for project1
# ------------------------------------------------------------------------------
$(foreach var,$(filter local_%,$(.VARIABLES)),$(eval $(var) := ))
local_src_dir := project1/src
local_obj_dir := project1/obj
local_src := $(local_src_dir)/fileA.c $(local_src_dir)/fileB.c
local_obj := $(patsubst $(local_src_dir)/%.c,$(local_obj_dir)/%.o,$(local_src))
dep_all += $(local_src)
dep_all += $(local_obj)
$(info local_obj="$(local_obj)")
$(local_obj): $(local_obj_dir)/%.o: $(local_src_dir)/%.c
gcc -L$(local_obj_dir) -c -o $# $<
# ------------------------------------------------------------------------------
# End Makefile.inc for project1
# Begin Makefile.inc for project2
# ------------------------------------------------------------------------------
$(foreach var,$(filter local_%,$(.VARIABLES)),$(eval $(var) := ))
local_src_dir := project2/src
local_obj_dir := project2/obj
local_src := $(local_src_dir)/fileX.c $(local_src_dir)/fileY.c
local_obj := $(patsubst $(local_src_dir)/%.c,$(local_obj_dir)/%.o,$(local_src))
dep_all += $(local_src)
dep_all += $(local_obj)
$(info local_obj="$(local_obj)")
$(local_obj): $(local_obj_dir)/%.o: $(local_src_dir)/%.c
gcc -L$(local_obj_dir) -c -o $# $<
# ------------------------------------------------------------------------------
# End Makefile.inc for project2
$(info dep_all="$(dep_all)")
.PHONY: all
all: $(dep_all)
.PHONY: clean
clean:
rm -f project1/obj/* project2/obj/*
If I run it then the -L<object_path> option passed to gcc contains value from the last included Makefile, i.e. when building project1, it runs gcc with -Lproject2/obj, which is not the right object path for this project. This is the problem I'm trying to solve.
mkdir -p project1/{src,obj} project2/{src,obj}
touch project1/src/{fileA.c,fileB.c} project2/src/{fileX.c,fileY.c}
$ make
local_obj="project1/obj/fileA.o project1/obj/fileB.o"
local_obj="project2/obj/fileX.o project2/obj/fileY.o"
dep_all=" project1/src/fileA.c project1/src/fileB.c project1/obj/fileA.o project1/obj/fileB.o project2/src/fileX.c project2/src/fileY.c project2/obj/fileX.o project2/obj/fileY.o"
gcc -Lproject2/obj -c -o project1/obj/fileA.o project1/src/fileA.c
gcc -Lproject2/obj -c -o project1/obj/fileB.o project1/src/fileB.c
gcc -Lproject2/obj -c -o project2/obj/fileX.o project2/src/fileX.c
gcc -Lproject2/obj -c -o project2/obj/fileY.o project2/src/fileY.c
The solution is to name all your variables local to a specific file with some prefix.
For example, I use this_ prefix.
Then, in the beginning of each submakefile those variales which have this_ prefix can be cleared as follows:
$(foreach var,$(filter this_%,$(.VARIABLES)),$(eval $(var) := ))
PS. I use this approach in my implementation of non-recursive makefiles, clearing the variables described in the WIKI here: https://github.com/cppfw/prorab
I think I've just figured out a solution. I was reading "The GNU Make Book" and it states a side effect of target specific variables
Target-specific variables apply not just to a target, but also to all
that target’s prerequisites, as well as all their prerequisites, and
so on. A target-specific variable’s scope is the entire tree of
targets, starting from the target for which the variable was defined.
This is not the behaviour I want, however starting with GNU Make 3.82 there is support for private target specific variables
A target-specific variable is normally defined for a target and all
its prerequisites. But if the target-specific variable is prefixed
with the keyword private, it is defined only for that target, not its
prerequisites.
So the following Makefile seems to work correctly for those private variables
dep_all :=
all:
# Begin Makefile.inc for project1
# ------------------------------------------------------------------------------
inc_dir := project1/inc
src_dir := project1/src
obj_dir := project1/obj
src := $(src_dir)/fileA.c $(src_dir)/fileB.c
obj := $(patsubst $(src_dir)/%.c,$(obj_dir)/%.o,$(src))
# These flags will be overwritten by another Makefile
CFLAGS := -I$(inc_dir)
# These flags will be private and not be overwritten by another Makefile
CFLAGS_priv := -I$(inc_dir) -L$(obj_dir)
dep_all += $(src)
dep_all += $(obj)
# Private target specific variables
$(obj): private CFLAGS_priv:=$(CFLAGS_priv)
$(obj): $(obj_dir)/%.o: $(src_dir)/%.c
gcc $(CFLAGS) $(CFLAGS_priv) -c -o $# $<
# ------------------------------------------------------------------------------
# End Makefile.inc for project1
# Begin Makefile.inc for project2
# ------------------------------------------------------------------------------
inc_dir := project2/inc
src_dir := project2/src
obj_dir := project2/obj
src := $(src_dir)/fileX.c $(src_dir)/fileY.c
obj := $(patsubst $(src_dir)/%.c,$(obj_dir)/%.o,$(src))
# These flags will be overwritten by another Makefile
CFLAGS := -I$(inc_dir)
# These flags will be private and not be overwritten by another Makefile
CFLAGS_priv := -I$(inc_dir) -L$(obj_dir)
dep_all += $(src)
dep_all += $(obj)
# Private target specific variables
$(obj): private CFLAGS_priv:=$(CFLAGS_priv)
$(obj): $(obj_dir)/%.o: $(src_dir)/%.c
gcc $(CFLAGS) $(CFLAGS_priv) -c -o $# $<
# ------------------------------------------------------------------------------
# End Makefile.inc for project2
.PHONY: all
all: $(dep_all)
.PHONY: clean
clean:
rm -f project1/obj/* project2/obj/*
The output is what I wanted, since now for each project I can define private CFLAGS_priv variable that sets -I<dir>, -L<dir> paths and it won't be overwritten by other Makefiles.
$ make
gcc -Iproject2/inc -Iproject1/inc -Lproject1/obj -c -o project1/obj/fileA.o project1/src/fileA.c
gcc -Iproject2/inc -Iproject1/inc -Lproject1/obj -c -o project1/obj/fileB.o project1/src/fileB.c
gcc -Iproject2/inc -Iproject2/inc -Lproject2/obj -c -o project2/obj/fileX.o project2/src/fileX.c
gcc -Iproject2/inc -Iproject2/inc -Lproject2/obj -c -o project2/obj/fileY.o project2/src/fileY.c
I'm hoping this will resolve all issues I had and I don't have to use recursive make with it's various associated pitfalls.

Makefile with subdirectories with sequential numbering

I have, in the main repository some subdirectories called assignment_1, assignment_2, ..., assignment_n.
I'm tring to write a Makefile that compiles the all TeX files inside those subdirectories.
This is what I have so far, but it doesn't work:
.PHONY: papers clean
PUBLISH_DIR := publish
TEX_DIR := .tex
SRC_DIR := assignment_$(wildcard *)
SRC_FILES := $(wildcard $(SRC_DIR)/*.tex)
CC := xelatex
FLAGS := -shell-escape -output-directory=$(TEX_DIR)
all: $(patsubst $(SRC_DIR)/%.tex, $(PUBLISH_DIR)/%.pdf, $(SRC_FILES))
$(PUBLISH_DIR)/%.pdf: $(SRC_DIR)/%.tex
mkdir -p $(TEX_DIR)
$(CC) $(FLAGS) $<
$(CC) $(FLAGS) $<
mkdir -p $(PUBLISH_DIR)
mv $(TEX_DIR)/*.pdf $(PUBLISH_DIR)/
clean:
rm -rf $(PUBLISH_DIR) $(TEX_DIR)
If I change this line
SRC_DIR := assignment_$(wildcard *)
with
SRC_DIR := assignment_1
it works beautifully but (obviously) only with the TeX file inside assignment_1.
Beside traversing the subdirectories, is there anything else I can improove in this Makefile?
I think you should modify your wildcard:
SRC_DIR := assignment_$(wildcard *)
to
SRC_DIR := $(wildcard assignment_*)
If $(wildcard *) expands to 1 2 3 then assignment_$(wildcard *) will expand to assignment_1 2 3 which is clearly not what you want.
Try this:
SRC_DIR := $(addprefix assignment_,$(wildcard *))
to add the assignment_ prefix to the start of each word.

Process list of files in makefile with $(notdir ...)

I would like to apply the 'notdir'-function to a list of files I obtain from a wildcard match. While '$(notdir $(wildcard dir/*.tst))' works, I do not manage to first store the list in a variable ('FILES' in the Makefile below) that then is processed by $(notdir ...). Using the variable directly ('$(notdir $(FILES))') results in the wildcard being returned, using value ('$(notdir $(value $(FILES)))') yields an empty result.
.PHONY: show
FILES := dir/*.tst
FILES2 := dir/a.tst dir/b.tst
#NAMES := $(notdir $(FILES))
NAMES1 := $(notdir $(value $(FILES)))
NAMES2 := $(notdir $(FILES2))
NAMES3 := $(notdir $(wildcard dir/*.tst))
show:
#echo "FILES: " $(FILES)
#echo "NAMES1: " $(NAMES1)
#echo "NAMES2: " $(NAMES2)
#echo "NAMES3: " $(NAMES3)
I also tried $(notdir $(eval $$(FILES))), but this results in a "missing separator" error.
What am I missing here? I'd have expected that value would do the job...
Try the following:
FILES := $(wildcard dir/*.tst)
NAMES := $(notdir ${FILES})

Need help in my makefile to compile a source code

I have 2 folders with their subfolders 04-Software Components / 03-Specific_sources and 08-Ext_Proj / SWC_ADC, SWC_PWM / 03-SRC
There are some .c and .h files in 03-Specific_sources and there is one .c file and their .h
files in both SWC_ADC / 03-SRC (which is ADC.c) and
SWC_PWM / 03-SRC (which is PWM.c)
And I want to compile them and link them all to get the output file.
so what I did is this:
EXEDIR_RELEASE := ../01-EXE\02-Release
OBJDIR_RELEASE := ../05-Obj\02-Release
SRCDIR := ../../../03-Specific_sources (This is the path from my make folder)
EXT_DIR := ../../08-Ext_Proj (This is the path from my make folder)
FIND_SRCS := $(wildcard $(EXT_DIR)/SWC_*/03-SRC/*.c)
INCLUDES := -I $(SRCDIR) $(foreach ext, $(wildcard $(EXT_DIR)/SWC*), -I $(ext)/03-SRC)
CSRCS := $(wildcard $(SRCDIR)/*.c) $(FIND_SRCS)
CSRCS := $(notdir $(CSRCS))
CROBJS := $(patsubst %.c, %.r34, $(CSRCS))
CROBJS := $(addprefix $(OBJDIR_RELEASE)/, $(CROBJS))
.PHONY: all
all: $(EXEDIR_RELEASE)/$(TARGET_RELEASE)
# Compiling
$(OBJDIR_RELEASE)/%.r34: $(FIND_SRCS)
#echo '$^ $#'
$(CC) $(CFLAGS) $(INCLUDES) -o $# $<
# Linking
$(EXEDIR_RELEASE)/Target.mot: $(CROBJS)
$(LD) $(MFLAGS) # These are the linker and its flags
The problem is that the compilation is not working. So when I do the echo I get this:
ADC_prg.c PWM_prg.c DDF_prg.r34
ADC_prg.c PWM_prg.c DFAN_prg.r34
ADC_prg.c PWM_prg.c DFN_prg.r34
ADC_prg.c PWM_prg.c DIO_prg.r34
ADC_prg.c PWM_prg.c DMIO_prg.r34
So I think that the mathing pattern %.r34 is not working and when I use
($(OBJDIR_RELEASE)/%.r34: $(FIND_SRCS)/%.c) it also not working because I can't use %
with a variable that has wildcard function performed on it.
I also used the foreach function but with no use.
So please I need your help guys, I have a delivery today... please respond asap.
Thanks in advance
SAM
This makefile has a lot of problems. Notably:
$(OBJDIR_RELEASE)/%.r34: $(FIND_SRCS)
#echo '$^ $#'
$(CC) $(CFLAGS) $(INCLUDES) -o $# $<
This pattern rule has no wildcard ("%") in the prerequisite list. Any r34 target depends on all sources in $(FIND_SRCS). If that isn't bad enough, look at where these variables come from:
SRCDIR := ../../../03-Specific_sources (This is the path from my make folder)
EXT_DIR := ../../08-Ext_Proj (This is the path from my make folder)
FIND_SRCS := $(wildcard $(EXT_DIR)/SWC_*/03-SRC/*.c)
...
CSRCS := $(wildcard $(SRCDIR)/*.c) $(FIND_SRCS)
CSRCS := $(notdir $(CSRCS))
CROBJS := $(patsubst %.c, %.r34, $(CSRCS))
CROBJS := $(addprefix $(OBJDIR_RELEASE)/, $(CROBJS))
In other words, you seem to expect that for every source file in one directory there will be a file of the same name in the other directory. According to your echo statements, there are no such matches.
There are other problems, but this is a good place to start. Try this:
CSRCS := $(wildcard $(SRCDIR)/*.c) $(EXT_DIR)/SWC_ADC/03-SRC/ADC.c $(EXT_DIR)/SWC_PWM/03-SRC/PWM.c
CSRCS := $(notdir $(CSRCS))
CROBJS := $(patsubst %.c, $(OBJDIR_RELEASE)/%.r34, $(CSRCS))
vpath %.c $(SRCDIR) $(EXT_DIR)/SWC_ADC/03-SRC $(EXT_DIR)/SWC_PWM/03-SRC
$(OBJDIR_RELEASE)/%.r34: %.c
$(CC) $(CFLAGS) $(INCLUDES) -o $# $<
Go down the list of object files you want to build, and try building each one:
make DDF_prg.r34
make DFAN_prg.r34
...
If that works, try linking them by hand, and if that works, take a look at the linking rule -- which looks completely wrong but which I can't really diagnose because I don't know what linker you're using.

Resources