Capture two folders in prerequisite for Makefile's target - shell

For my Makefile, my prerequisite is of the form
$(Root)a/b/c/d/File.source
and I'd like to target
a/c.zip
knowing that there are multiple folders for a and c, that b is fixed, that d can vary, but that the source file is always called File.source.
Taking inspiration from this answer, I've tried
As:= $(notdir $(shell find $(Root) -mindepth 1 -maxdepth 1 -type d))
define RULE
$1/%.zip: $1/b/%/*/File.source
echo "Test"
endef
$(foreach a, $(As), $(eval $(call RULE, $(a))))
But it does not seem to work.

I think this could be relatively simple, given the fact how patterns match. Having a rule
%.zip: b/%.foo
will cause make a/c.zip to search for a/b/c.foo file. With this in mind we can write a quite simple Makefile:
$ cat Makefile
# Extract directories that contain File.source files
sourcedirs := $(dir $(wildcard $(ROOT)*/b/*/*/File.source))
# Generate archive names: drop the /b/ part and add .zip extension
archives := $(patsubst %/,%.zip,$(subst /b/,/,$(dir $(sourcedirs:/=))))
%.zip: b/%/*/File.source
echo Making $# from $<
all: $(archives)
The output:
$ find . -name File.source
./a/b/c/d/File.source
./foo/b/bar/baz/File.source
./foo/b/cfg/d/File.source
$ make -s
Making foo/bar.zip from foo/b/bar/baz/File.source
Making foo/cfg.zip from foo/b/cfg/d/File.source
Making a/c.zip from a/b/c/d/File.source

Let's first find all File.source files that match the $(Root)*/b/*/*/File.source pattern:
FILES := $(shell find $(Root)*/b/* -maxdepth 2 -mindepth 2 -type f -name File.source)
Then, let's create a macro for each of them (I over-exaggerated the number of intermediate variables for easier understanding, you can simplify a bit, if you wish):
# $(1): File.source path
define MY_RULE
$(1)-dirs := $$(subst /, ,$(1))
$(1)-Roota := $$(word 1,$$($(1)-dirs))
$(1)-a := $$(patsubst $$(Root)%,%,$$($(1)-Roota))
$(1)-c := $$(word 3,$$($(1)-dirs))
$(1)-target := $$($(1)-a)/$$($(1)-c).zip
$$($(1)-target): $(1)
#echo "Test for $$< (target $$#)"
endef
And finally, let's call the macro on each File.source file:
$(foreach f,$(FILES),$(eval $(call MY_RULE,$(f))))
All in all, plus a test phony default goal for easier testing:
FILES := $(shell find $(Root)*/b/* -maxdepth 2 -mindepth 2 -type f -name File.source)
.PHONY: test
.DEFAULTGOAL := test
# $(1): File.source path
define MY_RULE
$(1)-dirs := $$(subst /, ,$(1))
$(1)-Roota := $$(word 1,$$($(1)-dirs))
$(1)-a := $$(patsubst $$(Root)%,%,$$($(1)-Roota))
$(1)-c := $$(word 3,$$($(1)-dirs))
$(1)-target := $$($(1)-a)/$$($(1)-c).zip
$$($(1)-target): $(1)
#echo "Test for $$< (target $$#)"
test: $$($(1)-target)
endef
$(foreach f,$(FILES),$(eval $(call MY_RULE,$(f))))
Note: there is a possibility that you have more than one File.source file with the same corresponding target, because of the d part of your pattern. If this happens GNU make will issue warnings and use only the last encountered rule for each such target. There are ways to raise an error, if you prefer, but as all this is already quite complicated, let's leave it as an exercise on make conditionals.

Related

Move multiple files for different folders using Make

Have two files, namely pyproject.toml and poetry.lock which is located in a folder called dump. I want to move those files to 2 directories for when running tests.
Today I do thise
PROJECT_DIR := $(realpath $(CURDIR))
BUILD_DUMP_DIR := $(PROJECT_DIR)/dump
DESTINATION_DIRS := unit system endtoend
PY_SOURCES = $(patsubst %,$(BUILD_DUMP_DIR)/%, pyproject.toml)
POETRY_SOURCES = $(patsubst %,$(BUILD_DUMP_DIR)/%, poetry.lock)
PY_PROJECT = $(foreach dir, $(DESTINATION_DIRS), $(patsubst %, $(BUILD_DUMP_DIR)/tests/$(dir)/%, pyproject.toml))
POETRY_PROJECT = $(foreach dir, $(DESTINATION_DIRS), $(patsubst %, $(BUILD_DUMP_DIR)/tests/$(dir)/%, poetry.lock))
$(PY_PROJECT): $(PY_SOURCES)
#echo "=> Moving $< to $#"
#cp $< $#
$(POETRY_PROJECT): $(POETRY_SOURCES)
#echo "=> Moving $< to $#"
#cp $< $#
copy-dump: $(PY_PROJECT) $(POETRY_PROJECT)
so running make copy-dump will move those files to the specified directory. Realize there must be nicer MakeFile command to do this. Thanks for all input
Not sure I understood all details but if you use GNU make and you want to copy (not move) your 2 files to 3 different locations each, the following is a bit simpler:
PROJECT_DIR := $(realpath $(CURDIR))
BUILD_DUMP_DIR := $(PROJECT_DIR)/dump
DESTINATION_DIRS := unit system endtoend
PY_SOURCES = $(BUILD_DUMP_DIR)/pyproject.toml
POETRY_SOURCES = $(BUILD_DUMP_DIR)/poetry.lock
PY_PROJECT = $(patsubst %,$(BUILD_DUMP_DIR)/tests/%/pyproject.toml,$(DESTINATION_DIRS))
POETRY_PROJECT = $(patsubst %,$(BUILD_DUMP_DIR)/tests/%/poetry.lock,$(DESTINATION_DIRS))
.PHONY: copy-dump
copy-dump: $(PY_PROJECT) $(POETRY_PROJECT)
.SECONDEXPANSION:
$(PY_PROJECT) $(POETRY_PROJECT): $(BUILD_DUMP_DIR)/$$(notdir $$#)
#echo "=> Moving $< to $#"
#cp $< $#
See the GNU make documentation for the details about secondary expansion.

How can I split a string by a delimiter in Makefile?

Here's the code snippet from my Makefile:
%/foo: %/bar.yaml
$(BINARY) generate -g go \
--package-name {COOL_VALUE}
# COOL_VALUE should be the parent folder of a `foo`, e.g., `foo1/foo2/foo -> foo2`
the question is how can I split $# string by / to get the second last element:
E.g.,
make foo1/foo2/foo
> ./binary generate -g go \
--package-name foo2
make foo3/foo
> ./binary generate -g go \
--package-name foo3
My attempts:
I came up with
$(eval package_name := $(word 1,$(subst /, ,$#)))
% pick second last element somehow
If you are really talking about / as a delimiter, then your best bet is to use the filename functions like this:
PARENT = $(notdir $(patsubst %/,%,$(dir $#)))
$(eval package_folders := $(filter-out foo,$(subst /, ,$#)))
$(eval package_name := $(word $(words $(package_folders)), $(package_folders)))
#echo "$(package_name)"

GNU make path substitution (directory flattening)

I have a makefile that gives me the source files in a hierarchy.
SRCS := $(wildcard $(SRC_DIR)/*.c $(SRC_DIR)/*/*.c)
gives me
./src/main.c ./src/add/add.c ./src/sub/sub.c
I want to flatten the object files into a single "obj" directory.
Of course
OBJS := $(SRCS:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.obj)
gives me
./obj/main.obj ./obj/add/add.obj ./obj/sub/sub.obj
instead of desired
./obj/main.obj ./obj/add.obj ./obj/sub.obj
Question: How do I get rid of additional source directory levels?
Never had to use complex substitution so far. My intuitive try with additional "/%" in pattern
# won't work as expected:
OBJS := $(SRCS:$(SRC_DIR)/%/%.c=$(OBJ_DIR)/%.obj)
produces no meaningful result (${OBJS} becomes same as ${SRCS}).
All examples I found so far only have single occurance of "%" in match pattern.
MAK_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
SRC_DIR = $(MAK_DIR)./src
OBJ_DIR = $(MAK_DIR)./obj
# gives: ./src/main.c ./src/add/add.c ./src/sub/sub.c
SRCS := $(wildcard $(SRC_DIR)/*.c $(SRC_DIR)/*/*.c)
# gives: ./obj/main.obj ./obj/add/add.obj ./obj/sub/sub.obj
OBJS := $(SRCS:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.obj)
.PHONY : all
all :
#echo $(SRCS)
#echo $(OBJS)
Just use the notdir function, and also patsubst, more powerful (and that I find easier to understand than the shorthand):
OBJS := $(patsubst %.c,$(OBJ_DIR)/%.obj,$(notdir $(SRCS)))
But it is not the whole story because later on you will probably want to do something like:
$(OBJ_DIR)/%.obj: $(SRC_DIR)/%.c
which will not work any more. Not mentioning the fact that you could have several source files with the same name in different directories. Assuming you do not have such names conflicts you can generate all your dependencies using foreach-eval-call:
MAK_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
SRC_DIR := $(MAK_DIR)./src
OBJ_DIR := $(MAK_DIR)./obj
SRCS := $(shell find $(MAK_DIR) -type f -name '*.c')
OBJS := $(patsubst %.c,$(OBJ_DIR)/%.obj,$(notdir $(SRCS)))
.PHONY: objs
objs: $(OBJS)
# $(1): source file
define DEP_rule
$(1)-obj := $$(patsubst %.c,$$(OBJ_DIR)/%.obj,$$(notdir $(1)))
$(1)-dep := $$(patsubst %.c,$$(OBJ_DIR)/%.d,$$(notdir $(1)))
$$($(1)-obj): $(1) $$($(1)-dep)
endef
$(foreach src,$(SRCS),$(eval $(call DEP_rule,$(src))))
Just remember that the DEP_rule macro gets expanded twice, thus the $$.

Makefile expanding variable inside define

define func1
include $(shell pwd)/test/$(strip $1)/component.mk
$(info :::::::${NAME} ::::::::::::::: )
endef
INCLUDES := a b c
$(foreach dir, $(INCLUDES), $(eval $(call func1, $(dir)) ))
all : $(objs)
Contents of each makefile:
cat test/a/component.mk
NAME := AA
cat test/b/component.mk
NAME := BB
cat test/c/component.mk
NAME := CC
Output is
::::::: :::::::::::::::
:::::::AA :::::::::::::::
:::::::BB :::::::::::::::
It looks like first time NAME is empty.
Let's look at the expansion of $(foreach dir, ${INCLUDES}, $(eval $(call func1, ${dir}) )) in painful detail.
${INCLUDES} is expanded, giving $(foreach dir,a b c,$(eval $(call func1,${dir})))
Next dir is set to a
$(call func1,a) is expanded
1 is set to a
func1 is expanded:
include $(shell pwd)/test/$(strip $1)/component.mk
$(info :::::::${NAME} ::::::::::::::: )
$(shell pwd) becomes HERE, say (N.B. Use ${CURDIR} instead)
$(strip $1) becomes $(strip a) becomes a
${NAME} expands to nothing
$(info ::::::: ::::::::::::::: ) expands to nothing
As a side effect ::::::: ::::::::::::::: appears on stdout
$(eval $(call func1,a)) expands to $(eval include HERE/test/a/component.mk), expands to nothing
As a side effect, the include is processed by make
Presumably HERE/test/a/component.mk exists and contains valid make syntax,
and the variable NAME gets a value.
1 is set to b. Lather, rinse, repeat.
Tip
To get a hint of problems in code like this, always run make with --warn:
$ make --warn -Rr
Makefile:8: warning: undefined variable 'NAME'
::::::: :::::::::::::::
⋮
Fix
To get some insight, replace the $(eval stuff) with $(error [stuff])
$ make
::::::: :::::::::::::::
Makefile:8: *** [ include /cygdrive/c/Users/somewhere/a/component.mk
]. Stop.
Here we see the $(info …) has disappeared even before it has got to the eval.
The naive fix is pretty horrible.
define func1
include $(shell pwd)/test/$(strip $1)/component.mk
$$(info :::::::$${NAME} ::::::::::::::: )
endef
Running this with the $(error …) in place gives
$ make
Makefile:8: *** [ include /cygdrive/c/Users/somewhere/a/component.mk
$(info :::::::${NAME} ::::::::::::::: )]. Stop.
That stuff between the [ and ] is valid make syntax.
Tidied up it looks like:
include /cygdrive/c/Users/somewhere/a/component.mk
$(info :::::::${NAME} ::::::::::::::: )
Job done. There are cleaner ways, but you need to understand the pain first!

How to generate targets in a Makefile by iterating over a list?

CODE:
LIST=0 1 2 3 4 5
PREFIX=rambo
# some looping logic to interate over LIST
EXPECTED RESULT:
rambo0:
sh rambo_script0.sh
rambo1:
sh rambo_script1.sh
Since my LIST has 6 elements, 6 targets should be generated. In future, if I want to add more targets, I want to be able to just modify my LIST and not touch any other part of the code.
How should the looping logic be written?
If you're using GNU make, you can generate arbitrary targets at run-time:
LIST = 0 1 2 3 4 5
define make-rambo-target
rambo$1:
sh rambo_script$1.sh
all:: rambo$1
endef
$(foreach element,$(LIST),$(eval $(call make-rambo-target,$(element))))
Use text-transforming functions. With patsubst you can make quite general transformations. For constructing filenames, addsuffix and addprefix are both convenient.
For the rules, use pattern rules.
The overall result might look something like this:
LIST = 0 1 3 4 5
targets = $(addprefix rambo, $(LIST))
all: $(targets)
$(targets): rambo%: rambo%.sh
sh $<
Just my 2 cents to #Idelic answer, if you need to use some Make $cmd you must escape them using $$
e.g.
LIST = 0 1 2 3 4 5
define make-rambo-target
$(info create target: $(addprefix rambo_script, $(addsuffix .sh, $1)).)
rambo$1: $$(addprefix rambo_script, $$(addsuffix .sh, $1))
sh $$<
endef
all: $(addprefix rambo, $(LIST))
$(foreach element, $(LIST), $(eval $(call make-rambo-target,$(element))))
output:
$ make
create target: rambo_script0.sh.
create target: rambo_script1.sh.
create target: rambo_script2.sh.
create target: rambo_script3.sh.
create target: rambo_script4.sh.
create target: rambo_script5.sh.
sh rambo_script0.sh
sh rambo_script1.sh
sh rambo_script2.sh
sh rambo_script3.sh
sh rambo_script4.sh
sh rambo_script5.sh
note: here rules "are seen" by Make as
rambo0: $(addprefix rambo_script, $(addsuffix .sh, 0))
sh $<
But here we could have written without escaping i.e.
rambo$1: $(addprefix rambo_script, $(addsuffix .sh, $1))
sh $$<
So rule "are seen" as:
rambo0 : rambo_script0.sh
sh $<
when Make parse it.

Resources