GNU Make for loop with two variables - makefile

I want to write something along these lines:
$(foreach (var1, var2), ($(LIST1), $(LIST2)), cp $(var1) $(var2);)
How do I go about doing this in a GNU makefile?

Beta's suggestion to use join is on the right track, but the problem is that it's not so easy to use it in a way that constructs a command line containing whitespace, such as the one you originally wanted:
$(foreach (var1, var2), ($(LIST1), $(LIST2)), cp $(var1) $(var2);)
because join joins words together: it was originally intended for constructing filenames. However you can play a trick; here's an example of a way to use join that gives you the output you are looking for:
$(subst ^, ,$(join $(addprefix cp^,$(LIST1)),$(patsubst %,^%;,$(LIST2))))
If you think your lists might contain ^ characters then choose something else. Let me know if you need this unpacked/explained.

LIST1 := a b c
LIST2 := 1 2 3
# outside a rule:
$(foreach var1, a b c, $(foreach var2, 1 2 3, $(info $(var1)_$(var2))))
# inside a rule: first line starts with a TAB, all the rest are spaces
all:
#for x in $(LIST1);\
do \
for y in $(LIST2);\
do\
echo $$x $$y; \
done \
done
(Please note that a nested loop that does cp doesn't make much sense.)
EDIT:
Well why didn't you say so?
LIST3 := $(join $(LIST1),$(LIST2))

This is a good candidate for gsml (GNU Make Standard Library). You can include it by putting the files __gmsl and gml in the current directory (or in /usr/gnu/include, /usr/local/include/ etc.) and adding the line include gsml in your Makefile. It includes the pairmap function, which does exactly what you want (i.e. zipWith).
include gmsl
cp2 = cp $1 $2;
zip = $1 : $2
$(LIST2):
#echo $(call pairmap, zip, $(LIST1), $(LIST2))
$(call pairmap, cp2, $(LIST1), $(LIST2))
Outputs
$ make
A : 1 B : 2 C : 3 D : 4
cp A 1; cp B 2; cp C 3; cp D 4;

A bit old post but I found this lets you do exactly what you need. You will need the seq utility which is a Unix utility and will be available on Linux and Mac.
I wrote the example with printing variables but it should work with copying.
# Two lists
LIST1 = a1 a2 a3 a4
LIST2 = b1 b2 b3 b4
# This will generate
# LISTSEQ=1 2 3 4
LISTSEQ=$(shell seq $(words $(LIST1)))
# Print variable for example
define printvar
$(info $(1))
$(info $(2))
endef
# First line loops through 1 2 3 4
# and next line prints the corresponding value from LIST1 and LIST2
$(foreach j, $(LISTSEQ), \
$(call printvar,$(word $(j), $(LIST1)) $(word $(j), $(LIST2))) \
)
Output:
a1 b1
a2 b2
a3 b3
a4 b4

Related

Bash: reshape a dataset of many rows to dataset of many columns

Suppose I have the following data:
# all the numbers are their own number. I want to reshape exactly as below
0 a
1 b
2 c
0 d
1 e
2 f
0 g
1 h
2 i
...
And I would like to reshape the data such that it is:
0 a d g ...
1 b e h ...
2 c f i ...
Without writing a complex composition. Is this possible using the unix/bash toolkit?
Yes, trivially I can do this inside a language. The idea is NOT TO "just" do that. So if some cat X.csv | rs [magic options] sort of solution (and rs, or the bash reshape command, would be great, except it isn't working here on debian stretch) exists, that is what I am looking for.
Otherwise, an equivalent answer that involves a composition of commands or script is out of scope: already got that, but would rather not have it.
Using GNU datamash:
$ datamash -s -W -g 1 collapse 2 < file
0 a,d,g
1 b,e,h
2 c,f,i
Options:
-s sort
-W use whitespace (spaces or tabs) as delimiters
-g 1 group on the first field
collapse 2 print comma-separated list of values of the second field
To convert the tabs and commas to space characters, pipe the output to tr:
$ datamash -s -W -g 1 collapse 2 < file | tr '\t,' ' '
0 a d g
1 b e h
2 c f i
bash version:
function reshape {
local index number key
declare -A result
while read index number; do
result[$index]+=" $number"
done
for key in "${!result[#]}"; do
echo "$key${result[$key]}"
done
}
reshape < input
We just need to make sure input is in unix format

bitwise left shift and | operations in makefile

I need to do left shift and OR operations in makefile. Something like below:
a = $(b) << 2 | 0x1
…where is b is a numeric value read using a $(shell ) command in a makefile.
I tried the following but it didn't help.
a = $(shell echo $(b) << 2 | bc)
I mean a as got value of b but not the shifted value after running the script.
The version of bc I have access to doesn't support bitwise operations, so it seems to be barking up the wrong tree. The default shell gnumake uses is /bin/sh but you can make it use bash instead and then access bash's bitwise operators directly:
SHELL=bash
a := $(( $(b) << 2 | 1 ))
Thanks to #EtanReisner for pointing out that this will cause that entire string to be stored in the variable a which would then be evaluated if you use it in a recipe. If you want the computed value stored directly in a you still need to get the shell to evaluate it:
a := $(shell echo "$$(( $(b) << 2 | 1 ))" )

Foreach with two arrays in a Makefile

I have to dynamically build rules in a Makefile from two arrays:
A file suffix
A generation code
I found the following syntax on SO, but unfortunately it doesn't work
$(foreach (a,b), ($(arrayA),$arrayB), $(eval $(call BUILD_RULES,$(a),$(b))))
Instead I found this solution which involve a third variable to iterate trough my arrays:
ITERATE = 1 2 3
EXT = _FOO _BAR _QUX
CODE = 34 33 36
define BUILD_RULES
dir/file_a$(word $1, $(EXT)).h:
genfile -a $(word $1, $(CODE)) > $$#
dir/file_b$(word $1, $(EXT)).h:
genfile -b $(word $1, $(CODE)) > $$#
endef
$(foreach i, $(ITERATE), $(eval $(call BUILD_RULES,$(i))))
Is there a better way to write this?
Well, I don't know if it's better or not, but you could alternatively do something like this:
# Initial values
EXT := _FOO _BAR _QUX
CODE := 34 33 36
# Getters
JOINED := $(join $(addsuffix :,$(EXT)),$(CODE))
GET_EXT = $(word 1,$(subst :, ,$1))
GET_CODE = $(word 2,$(subst :, ,$1))
define BUILD_RULES
dir/file_a$1.h:
genfile -a $1 > $$#
dir/file_b$1.h:
genfile -b $2 > $$#
endef
$(foreach j,$(JOINED),$(eval $(call BUILD_RULES,$(call GET_EXT,$j),$(call GET_CODE,$j))))

Makefile variable referencing within a nested loop

I am calling a script within a for loop and am running into an issue with variable expansion where the first of two variables is not being included in the output. (note: code adapted from here)
LIST1 := a b c
LIST2 := 1 2 3
all:
#for x in $(LIST1); do \
for y in $(LIST2); do\
echo $$x $$y; \
echo $$x_$$y.txt; \
done \
done
#This will output:
a 1
1.txt
a 2
2.txt ....
#Where I expect
a 1
a_1.txt
a 2
a_2.txt
Any ideas on how to get around this issue?
Thanks
zach cp
This is a shell issue, not a make issue. If you execute x=1; y=a; echo $x_$y.txt you'll see the same output. That's because _ is a valid shell variable name character, so $x_ is a valid shell variable, which is not set. The shell is printing the variable $x_ followed by $y followed by .txt.
Be sure to use braces around your shell variables, if the following character is a valid shell variable name character:
LIST1 := a b c
LIST2 := 1 2 3
all:
#for x in $(LIST1); do \
for y in $(LIST2); do\
echo $$x $$y; \
echo $${x}_$$y.txt; \
done \
done

Iterating over an empty item with GNU make's $(foreach)

I just discovered GNU make's $(foreach) function, and I'm following the foreach-eval-call pattern used in the documentation; for instance,
graphviz_progs := dot neato circo fdp
define LAYOUT_template
%-$(1).dot: %.dot
$(1) -Tdot $$? > $$#
endef
$(foreach p, $(graphviz_progs), \
$(eval $(call LAYOUT_template,$(p))) \
)
This works pretty well: $(foreach) function treats $(graphviz_progs) as a space-separated list of items and iterates over each of them.
Now my problem is that I frequently want to iterate over a list of items one of which is the empty string.
Is this possible in GNU make? (I can think of a workaround, but having the empty item in my list would be cleaner.)
I think the only way to get the behavior you want is by adding a level of indirection. Either crudely:
graphviz_progs := dot neato circo fdp
gplist := gp1 gp2 gp3 gp4 gp5
gp1 := dot
gp2 := neato
gp3 := circo
gp4 := fdp
gp5 :=
$(foreach p, $(gplist), \
$(eval $(call LAYOUT_template,$($(p)))))
or a little more neatly:
graphviz_progs := dot neato circo fdp
gplist := gp1 gp2 gp3 gp4 gp5
NUMBERS = 1 2 3 4 5
$(foreach n,$(NUMBERS), \
$(eval $(word $(n),$(gplist)) = $(word $(n),$(graphviz_progs))))
$(foreach p, $(gplist), \
$(eval $(call LAYOUT_template,$($(p)))))
There are a few more tricks, e.g. to do without NUMBERS or make it automatically, but they get kind of ugly.

Resources