Make: Can we Optimize make file targets? - makefile

We are supporting 32 bit and 64 bit build in our workflow.For that We have multiple rules in makefiles which are separate for 32-bit and 64-bit. Let me show pair of rules which are same except for the string “32” vs “64” .
Makefile Snippet:-
$(TGTDIR32)/logdir/set_user.c: $(CURDIR)/slv/set_user.c
$(file_transfer)
$(TGTDIR64)/logdir/set_user.c: $(CURDIR)/slv/set_user.c
$(file_transfer)
If you notice, We have targets with same except for the string “32” vs “64”, i want to replace them by single rule/definition. Because we have hundreds of rules like above in our infrastructure code.
Do we have any simplified way in GNUmake to do that?
Thanks in Advance!

Targets which have the same prerequisites and recipe can simply be combined, like so:
$(TGTDIR32)/logdir/set_user.c $(TGTDIR64)/logdir/set_user.c: $(CURDIR)/slv/set_user.c
$(file_transfer)
or more generally:
THESE_TARGETS := $(TGTDIR32)/logdir/set_user.c $(TGTDIR64)/logdir/set_user.c # More...?
...
$(THESE_TARGETS): $(CURDIR)/slv/set_user.c
$(file_transfer)
If Make decides that any member of $(THESE_TARGETS) is out-of-date with respect to the prerequisites, then it will run the recipe for that target.
This makefile:
.PHONY: all clean
all: a b c
a: d e
touch $#
b: d e
touch $#
c: d e
touch $#
d:
touch $#
e:
touch $#
clean:
$(RM) a b c d e
is equivalent to this one:
.PHONY: all clean
all: a b c
a b c: d e
touch $#
d e:
touch $#
clean:
$(RM) a b c d e
Later
There are some static pattern rules...
The same applies. This makefile with static pattern rules:
.PHONY: default clean
default: a.k b.k
a.k: %.k: %.j
cp -f $< $#
b.k: %.k: %.j
cp -f $< $#
a.j:
touch $#
b.j:
touch $#
clean:
$(RM) a.k b.k a.j b.j
is equivalent to this one:
.PHONY: default clean
JS := a.j b.j
KS := $(JS:.j=.k)
default: $(KS)
$(KS): %.k: %.j
cp -f $< $#
$(JS):
touch $#
clean:
$(RM) $(JS) $(KS)

In my opinion this is an appropriate place to use recursive make, at least for the top-level build.
In this situation you could do something like this:
TGTDIR64 = ...
TGTDIR32 = ...
.PHONY: all all32 all64 build
all: all32 all64
all32:
$(MAKE) TGTDIR=$(TGTDIR32) build
all64:
$(MAKE) TGTDIR=$(TGTDIR64) build
# Things below here should just use TGTDIR
build: $(TGTDIR)/b1 $(TGTDIR)/b2
$(TGTDIR)/logdir/set_user.c: $(CURDIR)/slv/set_user.c
$(file_transfer)
$(HEADERGEN_NOTSPLIT_H_COPY): $(TGTDIR)/%.h: %.h $(copy_file)
...

The two rules are semantically identical, they only use a different way to refer to a "parameterized" target. Why you don't just use one target for this
$(TGTDIR)/logdir/set_user.c: $(CURDIR)/slv/set_user.c
$(file_transfer)
and use a properly configured TGTDIR (I suspect this would be something like "xxxx_32" vs "xxxx_64")?
You could achieve this in several ways; one typical would be
ifdef choose32
TGTDIR=xxxx_32
else
TGTDIR=xxxx_64
endif

Related

Move multiple files for different folders using Make

Have two files, namely pyproject.toml and poetry.lock which is located in a folder called dump. I want to move those files to 2 directories for when running tests.
Today I do thise
PROJECT_DIR := $(realpath $(CURDIR))
BUILD_DUMP_DIR := $(PROJECT_DIR)/dump
DESTINATION_DIRS := unit system endtoend
PY_SOURCES = $(patsubst %,$(BUILD_DUMP_DIR)/%, pyproject.toml)
POETRY_SOURCES = $(patsubst %,$(BUILD_DUMP_DIR)/%, poetry.lock)
PY_PROJECT = $(foreach dir, $(DESTINATION_DIRS), $(patsubst %, $(BUILD_DUMP_DIR)/tests/$(dir)/%, pyproject.toml))
POETRY_PROJECT = $(foreach dir, $(DESTINATION_DIRS), $(patsubst %, $(BUILD_DUMP_DIR)/tests/$(dir)/%, poetry.lock))
$(PY_PROJECT): $(PY_SOURCES)
#echo "=> Moving $< to $#"
#cp $< $#
$(POETRY_PROJECT): $(POETRY_SOURCES)
#echo "=> Moving $< to $#"
#cp $< $#
copy-dump: $(PY_PROJECT) $(POETRY_PROJECT)
so running make copy-dump will move those files to the specified directory. Realize there must be nicer MakeFile command to do this. Thanks for all input
Not sure I understood all details but if you use GNU make and you want to copy (not move) your 2 files to 3 different locations each, the following is a bit simpler:
PROJECT_DIR := $(realpath $(CURDIR))
BUILD_DUMP_DIR := $(PROJECT_DIR)/dump
DESTINATION_DIRS := unit system endtoend
PY_SOURCES = $(BUILD_DUMP_DIR)/pyproject.toml
POETRY_SOURCES = $(BUILD_DUMP_DIR)/poetry.lock
PY_PROJECT = $(patsubst %,$(BUILD_DUMP_DIR)/tests/%/pyproject.toml,$(DESTINATION_DIRS))
POETRY_PROJECT = $(patsubst %,$(BUILD_DUMP_DIR)/tests/%/poetry.lock,$(DESTINATION_DIRS))
.PHONY: copy-dump
copy-dump: $(PY_PROJECT) $(POETRY_PROJECT)
.SECONDEXPANSION:
$(PY_PROJECT) $(POETRY_PROJECT): $(BUILD_DUMP_DIR)/$$(notdir $$#)
#echo "=> Moving $< to $#"
#cp $< $#
See the GNU make documentation for the details about secondary expansion.

Makefile wildcard for makefile variables, to define generic rules

Background, I suspect XY problem
I have simpler C modules in a directory. I want to write unit tests for these in a sub-directory test/. These unit tests are no more than C programs linking to the module under test, one directory above. I want a Makefile that defines several build targets and lets me build and run the test executables in one step, or separately.
My attempted solution
I've attempted the following:
CC = gcc
CFLAGS = -ggdb -Wall -Wextra -Werror -O3 -std=c99
PARAM_LIST_TARGET = parameter_list_test
PARAM_LIST_SOURCE_FILES = \
../parameter_list.c \
parameter_list_test.c
PARAM_LIST_OBJECT_FILES := $(addsuffix .o,$(basename $(PARAM_LIST_SOURCE_FILES)))
TARGETS = $(PARAM_LIST_TARGET)
all: $(TARGETS)
$(%_TARGET): $(%_OBJECT_FILES)
$(CC) $(CFLAGS) $^ -o $#
.c.o:
$(CC) -c $< -o $# $(CFLAGS)
clean:
$(RM) *.o $(TARGETS)
test: all
#for t in $(TARGETS) ; do ./$$t ; done
This doesn't work, and it's because of the $(%_TARGET): row. Not surprising, I didn't expect it to work, but I hope this illustrates what I'm trying to achieve.
I want to create more chunks of the form _TARGET, _SOURCE_FILES, and _OBJECT_FILES, to test other modules besides PARAM_LIST, for example:
PARAM_LIST_TARGET = parameter_list_test
PARAM_LIST_SOURCE_FILES = \
../parameter_list.c \
parameter_list_test.c
PARAM_LIST_OBJECT_FILES := $(addsuffix .o,$(basename $(PARAM_LIST_SOURCE_FILES)))
OTHER_MODULE_TARGET = other_module_test
OTHER_MODULE_SOURCE_FILES = \
../other_module.c \
other_module_test.c
OTHER_MODULE_OBJECT_FILES := $(addsuffix .o,$(basename $(OTHER_MODULE_SOURCE_FILES)))
I understand that % works on filenames, so attempting to use it on variables fails:
$(%_TARGET): $(%_OBJECT_FILES)
$(CC) $(CFLAGS) $^ -o $#
How can I write a rule that matches the Makefile variables _TARGET to their associated _OBJECT_FILES, without creating one per test target?
Or more importantly, how should I do it totally differently?
Edit: I've seen this, however it seems it's only working with a single source file per executable.
You can always access make variables by constructing their names:
MY_VAR := "my var"
HIS_VAR := "his var"
HER_VAR := "her var"
CATS_VAR := "cats var"
DOGS_VAR := "dogs var"
ALL_PERSONS := MY HIS HER CATS DOGS
ALL_VARS := $(foreach p,$(ALL_PERSONS),$($(p)_VAR))
$(info $(ALL_VARS))
Output:
$ make
"my var" "his var" "her var" "cats var" "dogs var"
Defining the dependencies separately seems to work, thanks to this answer:
TARGETS = $(PARAM_LIST_TARGET) $(OTHER_MODULE_TARGET)
all: $(TARGETS)
$(PARAM_LIST_TARGET): $(PARAM_LIST_OBJECT_FILES)
$(OTHER_MODULE_TARGET): $(OTHER_MODULE_OBJECT_FILES)
$(TARGETS):
$(CC) $(CFLAGS) $^ -o $#
This eliminates the need for a duplicate rule (one per target). Still, the definition of dependencies for each target looks like duplicates, a pattern match for these would be nice.
More than that, the OBJECT_FILES variable becomes unnecessary. This works:
PARAM_LIST_TARGET = parameter_list_test
PARAM_LIST_SOURCE_FILES = \
../parameter_list.c \
parameter_list_test.c
$(PARAM_LIST_TARGET): $(addsuffix .o,$(basename $(PARAM_LIST_SOURCE_FILES))) # The dependencies directly
It would still feel nice to have this last row as one rule for all targets. Something like "for all variables ending with TARGET, build a dependency to the content of the variable with the same name, but ending with SOURCE_FILES instead".

Make Target Name From Another

Have problem in dynamically "create" target name with .SECONDEXPANSION:
Small Makefile to reproduce problem:
CONFIGS = test1 test2 test3
.SECONDEXPANSION:
all: $(CONFIGS)
OBJECTS=$$(CFG_NAME)_OBJECTS
$(CONFIGS) : CFG_NAME=$#
$(CONFIGS) : $(OBJECTS)
#echo $(CFG_NAME) $# from $^
$(OBJECTS):
#echo OBJECTS $# from $^
#echo DO IT
It says: "No rule to make target 'test1_OBJECTS'.
How can I solve this problem?
EDIT: CHANGE OF THE ANSWER
Thank you much for the answer. It was the simple variant for my task.
So I try to answer in another way.
CONFIGS = test1 test2 test3
PLATFORMS = x86 ppc arm
#will be test1x86 test2x86 ... test1ppc ... test3arm,
#so it is long way to enumarate all variants
VARIANTS = $(foreach c, $(CONFIGS), $(foreach p, $(PLATFORMS), $(c)$(p)))
#C FILE LIST
CFILES:=$(shell /bin/find -name "*.c")
.SECONDEXPANSION:
all: $(VARIANTS)
#More Comlex Rule
#Want to corresponding objects be in bins/test1x86/
OBJECTS:=$(CFILES:%.c=bins/$$(CFGNAME)%.o)
$(CONFIGS) : CFG_NAME=$#
$(CONFIGS) : $(OBJECTS)
#echo $(CFG_NAME) $# from $^
#More complex prerequisites
#I understand that $$(CFGNAME) will be resolve incorrect.
#For each *.c file in subdir I would have object in corresponding path.
#For example, '1/2/3/test.c' will use for generate
#object file 'bins/test1x86/1/2/3/test.o',
#when I call 'make testx86' or 'make all' (it will build all VARIANTS),
#in 'bins/test1x86/1/2/3/'.
#So what have I do?
$(OBJECTS): bins/$$(CFGNAME)_OBJECTS/%o : %.c
#echo OBJECTS $# from $^
#echo DO IT
So, I would like to avoid recursive make calls. Can you help me?
Thank you.
You have a rule for $(OBJECTS), but that target expands to $(CFG_NAME)_OBJECTS, which is not expanded again (ever), so it can't match anything. Try this instead:
test1_OBJECTS test2_OBJECTS test3_OBJECTS:
#echo OBJECTS $# from $^
#echo DO IT
Or better:
OBJECT_SETS = $(addsuffix _OBJECTS, $(CONFIGS))
$(OBJECT_SETS):
#echo OBJECTS $# from $^
#echo DO IT
(And I'm sure you realize your example doesn't really need SECONDEXPANSION at all.)
EDIT:
That should be a separate question, but I'll try to answer it here. (And please use punctuation in the comments in your makefile; they are very difficult to understand.)
There is more than one solution to your problem. Here is one:
vpath %.c $(dir $(CFILES))
CFILES := $(notdir $(CFILES))
I've gotcha it.
CONFIGS = test1 test2 test3
PLATFORMS = p1 p2
#Will be testp1 test1p2 .. test3p2
VARIANTS = $(foreach c, $(CONFIGS), $(foreach p, $(PLATFORMS), $(c)$(p)))
.SECONDEXPANSION:
#.c files list in all subfolders
cfiles = $(shell /bin/find -name "*.c")
#objects for these .c files for custom VARIANT
objects = $(patsubst %.c,%.o,$(addprefix bins/$1/,$(cfiles)))
#Get .c source for object (e.g. bins/test1p1/tests/main_test.o => tests/main_test.c)
get_src=$(shell echo $1 | sed 's/[^\/]*\/[^\/]*\/\(.*\)/\1.c/')
#Build All Variants
all: $(VARIANTS)
#Build objects. Target list contains all objects for all variants.
#Prerequisites get .c sources from the pattern rule for targets
$(foreach v, $(VARIANTS), $(call objects,$(v))) : %.o : $$(call get_src,$$*)
#echo OBJECTS $# FROM $^
#Variants rule, depends on objects
$(VARIANTS): $(call objects,$$#)
#echo $# from $^
Thank you, Beta. You only have tried. :)
Maybe anyone have style or efficiency suggestions.

Need to make multiple times in order to work

I've this:
include makefile_vars.mk
# Target
TARGET := $(OBJ)/$(TARGETNAME)
# Move objects to ../Obj
OBJECTS := $(addsuffix .obj,$(SOURCES))
override OBJECTS := $(OBJECTS:$(SRC)/%.obj=$(OBJ)/%.obj)
# Objects o
OBJ_INC_DIRS := $(sort $(dir $(OBJECTS)))
# Target
all: $(TARGET)
$(TARGET): $(OBJECTS)
# Objects
$(OBJ)/%.c.obj: $(SRC)/%.c | $(OBJ_INC_DIRS)
#echo Compiling $(<F) ...
$(OBJ)/%.a66.obj: $(SRC)/%.a66 | $(OBJ_INC_DIRS)
#echo Compiling $(<F) ...
$(OBJ_INC_DIRS):
mkdir -p $#
The first time I launch it, it says me that there is no rule to make the target MyFile.c.obj, when running again (whitout modifying anything) the rule is found and the compilation goes on.
What am I doing wrong?
I'm not sure that this will work, but I would move $(OBJ_INC_DIRS) order-only prerequisite out from the pattern rules:
$(OBJECTS) : | $(OBJ_INC_DIRS)
$(OBJ)/%.c.obj: $(SRC)/%.c
#echo Compiling $(<F) ...
...

Makefile: using variable with computed name as prerequisite

In Make it's possible to compute variable names in runtime using double-evaluation, like $($(var)).
I'm interested if it's possible somehow to make this work:
one.js_DEPS=a b
two.js_DEPS=c b
all: one.js two.js
%.js: $(%.js_DEPS)
cat $^ > $#
I can make this work by declaring two rules with explicit dependencies, like this:
one.js: $(one.js_DEPS)
But this seems a bit anti-DRY. So I'm wondering if I miss something because make doesn't seem to understand me. I use GNU Make 3.81.
% and implicit rules are somewhat second-class citizens in make, since variable expansion seems to happen before implicit rule expansion. So, in your above Makefile, $(%.js_DEPS) is expanded to the empty string before % is substituted.
Alternative:
%.js: %.js_DEPS
cat $^ > $#
%.js_DEPS :
cat $^ > $#
.INTERMEDIATE: one.js_DEPS two.js_DEPS
or simply:
one.js : a b
two.js : c d
%.js :
cat $^ > $#
This could be done using Secondary Expansion.
JS := one.js two.js
all: $(JS)
.SECONDEXPANSION:
one.js_DEPS := a b
two.js_DEPS := c b
$(JS) : $$($$(#F)_DEPS)
cat $^ > $#
But in fact, the goal can be achieved much simply, without using any GNU Make extensions, as thiton has suggested in his answer:
all: one.js two.js
one.js : a b
two.js : c b
%.js :
cat $^ > $#

Resources