I have a parameter space, say A={1,2}, B={u,v}, and I need to create a file for each pair (a,b) in A x B.
What I need generated is a set of Makefile targets such as:
file_1_u.csv:
./run_program --A 1 --B u > file_1_u.csv
file_1_v.csv:
./my_program --A 1 --B v > file_1_u.csv
file_2_u.csv:
./run_program --A 2 --B u > file_1_u.csv
file_2_v.csv:
./my_program --A 2 --B v > file_1_u.csv
Is it possible to generate the targets and use the pertaining parameters in each generated target?
PS.:
I know that targets can be created by using variables, but then $# only gives only the full target name (e.g. file_1_u.csv). Instead, I need a and b individually to pass them as arguments to my_program.
Generally speaking, handling more than 1 element in the pattern in a makefile require eval, $$, includes and other non-straightforward processing.
However, in your specific example there is an obvious substitution that could be used like this:
A:=1 2
B:=u v
l:=$(foreach a, $(A), $(foreach b, $(B), $(a)_$(b)))
all: $(l:%=file_%.csv)
file_%.csv:
./run_program --A $(subst _, --B ,$*) > $#
It looks like you need something like this:
define foo
test_$(1)_$(2) :
./my_program --A $(1) --B $(2) > file_$(1)_$(2).csv
endef
a=1 2
b=u v
$(foreach a_,$(a),$(foreach b_,$(b),$(eval $(call foo,$(a_),$(b_)))))
Related
I am having an set of event as INPUT in makefile and I would like to assign them with EVENT_1, EVENT_2...so on so forth. And I would also like to check the number of events in INPUT variable, is there a way I can do this?
I have tried the following:
INPUT=g1 g2
all:
for eventid in INPUT; do \
echo evenit; \
i+=1; \
done
echo i
seems that i only valid in the loop and I could not check the eventid and number of event outside the loop.
After this, I would like to make use of eventid and number to have targets like "generate_EVENT1". Is it possible?
Check the amount of elements in INPUT:
COUNT := $(words $(INPUT))
Generate a rule and create a folder for each element in INPUT:
INPUT := g1 g2
.PHONY: all
all: $(addprefix run_dir/generate_, $(INPUT))
run_dir/generate_%:
mkdir -p $#
You can also omit the INPUT variable declaration and call make like so:
make INPUT="g1 g2"
I have the following:
FILE_1_DEPS := a b c
FILE_2_DEPS := d e f
output_1:
mycommand $(FILE_1_DEPS) $#
output_2:
mycommand $(FILE_2_DEPS) $#
I would like to combine the multiple targets, which differ only by the name of the variable, into a single line (the FILE_%_DEPS being left as is). I am thinking something like the following, which does not work:
output_%:
mycommand $($(patsubst output_%, FILE_%_DEPS, $#)) $#
Easy enough, just look up Automatic Variables:
output_%:
mycommand $(FILE_$*_DEPS) $#
I need to Validate the ID with pattern (Abbbbb-yyy)
Example :
ID := A12345-789 B98765-123 C58730-417
VARIANT := test1 test2 test3
Build and post processing will generate files depends up on VARIANTS :
`sw_main_test1.hex ,sw_main_test1.hex and sw_main_test1.hex `
.PHONY : SW_TEST
SW_TEST :
if <ID is correct>
cp sw_main_test1.hex --> A12345-789.hex
cp sw_main_test2.hex --> B98765-123.hex
cp sw_main_test3.hex --> C58730-417.hex
I am facing issue in validating the ID with pattern
`Abbbbb-yyy.txt`
Where : A=[A-Z]; b=[0-9]; y=[0-9]
Please let me know how to verify ID is correct using regular expressions inside the Makefile using any tool or utility
In this script, I assume, you get your ID from a file (I called it here someidcontent.txt). Then you could write a script like this (assuming, you only working on Linux).
getID = $(shell cat someidcontent.txt)
all:
if [ "$(getID)" == "1234567890" ]; then \
cp -v output.txt ./delivery/$(getID).txt; \
fi
.PHONY: all
Edit
I made a mistake in my previous script. I did not check, if the ID is correct. Now my newer script does this: I read from a file the ID and check it for correctness. If ID is correct, then some file will be copied into target dir with ID number.
# get ID from a file
getID := $(shell cat someidcontent.txt)
# need a hack for successful checking
idToCheck := $(getID)
# check procedure
checkID := $(shell echo $(idToCheck) | grep "[A-Z][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9]$$")
all:
ifeq "$(checkID)" "$(idToCheck)"
echo found
cp -v output.txt ./delivery/$(idToCheck).txt;
endif
.PHONY: all
Edit 2
Ok, this was a little bit challenging, but I solved it somehow. Maybe there are also other ways to solve this better. In my solution, I assume that the file with IDs and source filenames look like this (in other words, this is the content of my someidcontent.txt):
A2345-678:output1.txt
B3456-123:output.txt
C0987-987:thirdfile.txt
And this is my makefile with comments for additional explanation. I hope, they are sufficient
# retrieve id and filename data from other file
listContent := $(shell cat someidcontent.txt)
# extract only IDs from other files
checkIDs = $(shell echo $(listContent) | grep -o "[A-Z][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9]")
all:
# iterate only over IDs
# first, give me the ID
# second retrieve the filename part for successful copy procedure
# and copy the file to the target dir with ID as filename
#$(foreach x,$(checkIDs), \
echo $(x); \
cp -v $(shell echo $(listContent) | grep -o "$(x):[A-Z0-9a-z\.]*" | sed "s/[-A-Z0-9]*://g") ./delivery/$(x).t$
)
.PHONY: all
You can check simple string patterns quite ok (don't want to say "nicely") from within make:
[A-F] := A B C D E F#
[a-f] := a b c d e f#
[A-Z] := $([A-F]) G H I J K L M N O P Q R S T U V W X Y Z#
[a-z] := $([a-f]) g h i j k l m n o p q r s t u v w x y z#
[0-9] := 0 1 2 3 4 5 6 7 8 9#
######################################################################
##### $(call explode,_stringlist_,_string_)
## Insert a blank after every occurrence of the strings from _stringlist_ in _string_.
## This function serves mainly to convert a string into a list.
## Example: `$(call explode,0 1 2 3 4 5 6 7 8 9,0xl337c0de)` --> `0 xl3 3 7 c0 de`
explode = $(if $1,$(subst $(firstword $1),$(firstword $1) ,$(call explode,$(wordlist 2,255,$1),$2)),$2)
ID := A12345-789 B98765-123 C58730-417 123456+328
############################################################
# $(call check-id,_id-string_)
# Return 'malformed' or the given id
check-id = $(if $(call check-id-1,$(call explode,- $([A-Z]) $([0-9]),$1)),malformed,$1)
check-id-1 = $(strip $(filter-out $([A-Z]),$(wordlist 1,1,$1)) $(filter-out $([0-9]),$(wordlist 2,6,$1)) $(filter-out -,$(word 7,$1)) $(filter-out $([0-9]),$(wordlist 8,10,$1)) )
$(info $(foreach w,$(ID),$(call check-id,$(w))))
I have a shell program that takes ages to complete. As written, executing make build takes 4 x 2 seconds to complete because $(value) is computed for each file.
A solution is to declare value a deferred variable by using := instead of =.
Unfortunately this is not a solution either because it slows down the execution of make clean and any other targets by 2 seconds because value is computed for nothing.
value = $(shell sleep 2 && echo 42)
in = a b c d
out = $(addsuffix .out,$(in))
build: $(out)
%.out: %
echo $(value) > $< || [ rm $# -a true ]
init:
touch $(in)
clean:
rm -vf $(out)
How can I set a variable what is assigned only if used, but only computed once ?
Said differently, I would like build to take 2 seconds to complete and clean to be immediate.
I am not interested to a solution that involves conditionals in order to bypass the assignment of value if the target is not build.
An alternative solution would be this. Unfortunately in this case I need to check whether or not the shelve file needs to be regenerated.
value = $(cat shelve)
shelve:
sleep 2 && echo 42 > $# || [ rm $# -a true ]
in = a b c d
out = $(addsuffix .out,$(in))
build: $(out)
%.out: %
echo $(value) > $< || [ rm $# -a true ]
init:
touch $(in)
clean:
rm -vf $(out)
Here's a trick you can play:
value = $(eval value := $(shell cat shelve))$(value)
How this works: value is first assigned using recursive assignment so the value on the RHS is not expanded.
The first time value is expanded the make parser will first run the $(eval ...) which starts up a "new parser" for makefiles. In that parser, the content value := $(cat shelve) is evaluated. Here, value is a simple variable assignment so the RHS is expanded immediately and the $(shell ...) is run and assigned to value.
Remember make doesn't really have a concept of variable scope, so this value is just the same global value variable that we are setting in the outer parser.
Then the eval completes and expands to the empty string, and make continues parsing things. Here it finds the value $(value) and expands that... value now has the result from the eval, not the eval text itself, so that's what will be expanded.
Maybe this will help:
value = $(eval value := $(shell cat shelve))$(value)
Here value contains the string $(eval value := $(shell cat shelve))$(value)
Now you expand it:
%.out: %
echo $(value) > $< ...
Make starts to expand this recipe. It gets to $(value) and sees it needs to expand the variable value: since it's recursive it expands the value:
$(eval value := $(shell cat shelve))$(value)
First it expands the eval, which parses this:
value := $(shell cat shelve)
That sets the value variable as a simply-expanded variable, so the RHS is expanded immediately. Say the results of cat shelve are "foo", so value is now set to foo (and it's marked simply expanded).
That's the end of the eval, so then make starts the next part which is $(value), so it looks up the variable value and discovers it's a simply-expanded variable with the value foo.
One solution would be to turn that value into a regular file target that gets updated only when its prerequisites change. If you insist on rebuilding that target for every build, mark it as phony.
When clean target does not depend on that file, then it won't be rebuilt when you invoke make clean.
In
%.out: %
echo $(value) > $< || [ rm $# -a true ]
echo $(value) > $< updates the prerequisite, whereas make expects it to update the target only. Updating a prerequisite must be done by a separate rule with that prerequisite being the target.
You can make the assignment depend on the target name in $(MAKECMDGOALS):
ifneq ($(MAKECMDGOALS),clean)
value := $(shell sleep 2 && echo 42)
endif
See also the docs for details.
How can I list the current value of all variables (also called macros) in a Makefile when running make?
E.g. if this is in the Makefile:
CUR-DIR := $(shell /bin/pwd)
LOG-DIR := $(CUR-DIR)/make-logs
Then I would like it to tell me:
CUR-DIR = /home/johv/src/test
LOG-DIR = /home/johv/src/test/make-logs
GNU make provides .VARIABLES
which holds all global variables' names.
However, this includes built-in variables(like MAKEFLAGS).
If you have to exclude built-in variables, some filtering like the following
might be needed.
The following makefile prints user-defined variables(CUR-DIR, LOG-DIR)
using info:
# Place this line at the top of your Makefile
VARS_OLD := $(.VARIABLES)
# Define your variables
CUR-DIR := $(shell pwd)
LOG-DIR := $(CUR-DIR)/make-logs
# Put this at the point where you want to see the variable values
$(foreach v, \
$(filter-out $(VARS_OLD) VARS_OLD,$(.VARIABLES)), \
$(info $(v) = $($(v))))
Thanks to #Ise Wisteria, condensed down, this shows all variables, useful for large projects with multiple makefiles (Buildroot).
$(foreach v, $(.VARIABLES), $(info $(v) = $($(v))))
output: BR2_GCC_TARGET_TUNE = "cortex-a8" ...
If you get an error like: insufficient number of arguments (1) to function 'addprefix' this project had some broken variables... I trimmed the list of variables to show, only with a prefix BR2_
$(foreach v, $(filter BR2_%,$(.VARIABLES)), $(info $(v) = $($(v))))
I ended up doing it like this:
gmake -pn | grep -A1 "^# makefile"| grep -v "^#\|^--" | sort | uniq > makevars.txt
which gives:
CUR-DIR := /home/johv/src/test
LOG-DIR := /home/johv/src/test/make-logs
MAKEFILE_LIST := Makefile
MAKEFLAGS = pn
SHELL = /bin/sh
VARS_OLD := [...]
gmake -pn is really verbose and looks kinda like this:
# environment
GNOME2_PATH = /usr/local:/opt/gnome:/usr:/usr/local:/opt/gnome:/usr
# automatic
#F = $(notdir $#)
# makefile
SHELL = /bin/sh
# default
RM = rm -f
It's also doable without saving all the .VARIABLES and filtering them out.
Moreover, if one of the original .VARIABLES was modified in your makefile, the two most voted answers won't catch it.
Check out $(origin) function. This target filters out and prints all the variables that were defined in a makefile:
print_file_vars:
$(foreach v, $(.VARIABLES), $(if $(filter file,$(origin $(v))), $(info $(v)=$($(v)))))
I get only a few excess variables this way: CURDIR SHELL MAKEFILE_LIST .DEFAULT_GOAL MAKEFLAGS.
One can replace file with environment or command line to print the respective kinds of variables.
There are a lot of good answers here, but you're going to have problems using $($(v)) if some of your variables are of the recursive flavor. This is why you should use $(value $(v)).
This variation cleans this up a little bit, sorts variables by name and makes the output a bit more readable.
dump:
$(foreach v, \
$(shell echo "$(filter-out .VARIABLES,$(.VARIABLES))" | tr ' ' '\n' | sort), \
$(info $(shell printf "%-20s" "$(v)")= $(value $(v))) \
)
Thanks to #kevinf for the great idea. I would suggest a minor change to prevent .VARIABLE itself from printing out in the variable list:
$(foreach v, $(filter-out .VARIABLES,$(.VARIABLES)), $(info $(v) = $($(v))))
Thanks to #kevinf for the foreach solution -- if one wants to export this list as a somewhat machine-readable file, one will have a hard time with uneven quotes or newlines when using echo or printf, since Make isn't able to quote the data correctly -- one needs to use the $(file ...) function to write the data to avoid sh/bash complaining about invalid syntax. For example, use this in your rule -- it prints variable name, definition and expanded value:
$(file > $(MAKEFILE_ENV_FILE),)
$(foreach v, $(.VARIABLES), \
$(file >> $(MAKEFILE_ENV_FILE),$(v)) \
$(file >> $(MAKEFILE_ENV_FILE), := $(value $(v))) \
$(file >> $(MAKEFILE_ENV_FILE), == $($(v))) \
$(file >> $(MAKEFILE_ENV_FILE),) \
)
(This will still not allow to always distinguish malicious variables with double newlines from two variables, for this one now add a sufficiently unique separator infront of each Makefile-generated newline just after each comma inside $(file >> NAME,TEXT))
Set MAKEFILE_ENV_FILE to some filename, e.g.:
MAKEFILE_ENV_FILE := $(abspath $(lastword $(MAKEFILE_LIST))).env