Got stuck in dynamically allocating an associative array in bash - bash

I try to create an associative array in bash like this:
#! /bin/bash
declare -a arr
while read key
do
arr+=([$key]=1)
done < /dev/stdin
for i in ${!arr[#]}
do
echo "key:$i,value:${arr[$i]}"
done
I have inputed "leopard" and "longtuan" as a key,the output supposed to be like this in my mind:
key:leopard,value:1
key:longtuan,value:1
But I only got:
key:0,value:11
How can I get through of this problem, Thanks for any help.

You're creating a regular array not an associative array. Change this:
declare -a arr
into this:
declare -A arr
and the problem will disappear.

Associative arrays are declared with capital A:
declare -A arr
declare -a declares a regular array.

Related

Dynamically creating associative arrays in bash

I have a variable ($OUTPUT) that contains the following name / value pairs:
member_id=4611686018429783292
platform=Xbox
platform_id=1
character_id=2305843009264966985
period_dt=2020-11-25 20:31:14.923158 UTC
mode=all Crucible modes
mode_id=5
activities_entered=18
activities_won=10
activities_lost=8
assists=103
kills=233
average_kill_distance=15.729613
total_kill_distance=3665
seconds_played=8535
deaths=118
average_lifespan=71.72269
total_lifespan=8463.277
opponents_defeated=336
efficiency=2.8474576
kills_deaths_ratio=1.9745762
kills_deaths_assists=2.411017
suicides=1
precision_kills=76
best_single_game_kills=-1
Each line ends with \n.
I want to loop through them, and parse them into an associative array, and the access the values in the array by the variable names:
while read line
do
key=${line%%=*}
value=${line#*=}
echo $key=$value
data[$key]="$value"
done < <(echo "$OUTPUT")
#this always prints the last value
echo ${data['seconds_played']}
This seems to work, i.e. key/value print the right values, but when I try to pull any values from the array, it always returns the last value (in this case -1).
I feel like im missing something obvious, but have been banging my head against it for a couple of hours.
UPDATE: My particular issue is I'm running a version of bash (3.2.57 on OSX) that doesn't support associative arrays). I'll mark the correct answer below.
Without declare -A data, then data is a normal array. In normal arrays expressions in [here] first undergo expansions, then arithmetic expansion. Inside arithmetic expansion unset variables are expanded to 0. You are effectively only just setting data[0]=something, because data[$key] is data[seconds_played] -> variable seconds_played is not defined, so it expands to data[0]
Add declare -A data and it "should work". You could also just:
declare -A data
while IFS== read -r key value; do
data["$key"]="$value"
done <<<"$OUTPUT"
Try declaring data as an associative array before populating it, eg:
$ typeset -A data # declare as an associative array
$ while read line
do
key=${line%%=*}
value=${line#*=}
echo $key=$value
data[$key]="$value"
done <<< "${OUTPUT}"
$ typeset -p data
declare -A data=([mode]="all Crucible modes" [period_dt]="2020-11-25 20:31:14.923158 UTC" [deaths]="118" [best_single_game_kills]="-1" [efficiency]="2.8474576" [precision_kills]="76" [activities_entered]="18" [seconds_played]="8535" [total_lifespan]="8463.277" [average_lifespan]="71.72269" [character_id]="2305843009264966985" [kills]="233" [activities_won]="10" [average_kill_distance]="15.729613" [activities_lost]="8" [mode_id]="5" [assists]="103" [suicides]="1" [total_kill_distance]="3665" [platform]="Xbox" [kills_deaths_ratio]="1.9745762" [platform_id]="1" [kills_deaths_assists]="2.411017" [opponents_defeated]="336" [member_id]="4611686018429783292" )
$ echo "${data['seconds_played']}"
8535

Creating and populating dynamically named arrays in bash

I am creating dynamic arrays which all have different letters in their name. For the purpose of this question, my initial array of letters has been set at a fixed length. However, in my final implementation this letter array will be any length.
For each letter, I construct a string
I declare a new array with that string, making use of eval to evaluate the variable's value within the declare command.
I add some values to the array, again using eval to evaluate any variable values.
Here is the code:
declare -a LETTER_ARRAY=( "A" "B" "C" "D" "E" )
for i in "${LETTER_ARRAY[#]}"
do
name_string="apple${i}"
color0="red"
color1="green"
eval "declare -a ${name_string}_array"
eval "${name_string}_array[0]=$color0"
eval "${name_string}_array[1]=$color1"
done
So, how can I iterate through these dynamic arrays and echo what is in them? I have tried the following:
for i in "${LETTER_ARRAY[#]}"
do
eval "array_name='apple${i}_array'"
echo ${array_name[0]}
done
This has not worked for me. I can confirm that my dynamic arrays were successfully created and populated, as when I echo out a value manually, I get a result:
echo ${appleA_array[0]}
red
A perfect place to use a namereference:
letter_array=(A B C D E)
for i in "${letter_array[#]}"; do
declare -n var="apple${i}_array"
var[0]=red
var[1]=green
done
declare -p appleA_array
would output:
declare -a appleA_array=([0]="red" [1]="green")
how can I iterate through these dynamic arrays and echo what is in them?
With the above:
for i in "${letter_array[#]}"; do
declare -n var="apple${i}_array"
printf "%s\n" "${var[0]}"
done
Notes:
Do not use eval. Eval is evil.
Do not use upper case variables, by convention they are used for exported variables, like COLUMNS, PWD, UID, EUID, LINES. Use lower case variables in your scripts.
Check your scripts with http://shellcheck.net for most common mistakes
But if you are creating a 2d array, then an associative array might be better:
declare -A apple_arrays
letter_array=(A B C D E)
for i in "${letter_array[#]}"; do
apple_arrays[$i,0]=red
apple_arrays[$i,1]=green
done
for i in "${letter_array[#]}"; do
printf "one=%s two=%s\n" "${apple_arrays[$i,0]}" "${apple_arrays[$i,1]}"
done
how can I iterate through these dynamic arrays
echo ${array_name[0]} does not work because array_name is not the name of an array; $array_name is. Therefore, eval "echo \"\${${array_name}[0]}\"" would to the trick.
However, I'd recommend namerefs.
By The way: declare works without eval and is more reliable that way.
#! /usr/bin/env bash
letters=({A..E})
for i in "${letters[#]}"; do
declare -a "apple${i}_array=(red green)"
done
for i in "${letters[#]}"; do
declare -n array="apple${i}_array"
# now you can use `array` as if it was `appleA_array`, `appleB_array`, ...
echo "${array[0]}"
done
Your first line is not bash syntax. If I try the line
declare -a LETTER_ARRAY = [ "A" "B" "C" "D" "E" ]
I get:
bash: declare: `=': not a valid identifier
bash: declare: `[': not a valid identifier
bash: declare: `]': not a valid identifier
I think, you get similar error messages, but you ignored them
More errors:
Forgotten do
case mismatch: suffix on definition: _array, and for output: _ARRAY
Use always double quotes when using [#]
One correct syntax is:
declare -a LETTER_ARRAY=( "A" "B" "C" "D" "E" )
for i in "${LETTER_ARRAY[#]}"
do
name_string="apple${i}"
color0="red"
color1="green"
eval "declare -a ${name_string}_array"
echo "${name_string}_array[0]=$color0"
eval "${name_string}_array[0]=$color0"
eval "${name_string}_array[1]=$color1"
done
echo ${appleA_array[0]}
Your eval "array_name='AZ${i}_ARRAY'" makes array_name a scalar, not an array. Arrays in bash are usually created like this
arr=( your elements go here )
If you want to assign one array to another, you have to interpolate the elements between those parenthesis, for instance:
arr=( ${other_array[#]} )
Since you are using bash, this would perform word splitting the elements of other_array, if they contain spaces. Hence you would usually write it for the safe side as
arr=( "${other_array[#]}" )
Hence, for your case, you could do a
eval "array_name=( \${AZ${i}_ARRAY[#]} )"
This causes an array array_name to be created, with the elements of the respective AZi_ARRAY.
I omitted here for simplicity the prevention against word splitting, because in your example, the array elements contain single words only.

Adding to Bash associative arrays inside functions

I'm trying to use associative arrays as a work around for Bash's poor function parameter passing. I can declare a global associative array and read/write to that but I would like to have the variable name passed to the function since many times I want to use the same function with different parameter blocks.
Various Stack Overflow posts have approaches for reading a passed array within a function but not writing to it to allow return values. Pseudo Bash for what I'm trying to do is thus:
TestFunc() {
local __PARMBLOCK__=${1} # Tried ${!1} as well
# Do something with incoming array
__PARMBLOCK__[__rc__]+=1 # Error occured
__PARMBLOCK__[__error__]+="Error in TestFunc"
}
declare -A FUNCPARM
# Populate FUNCPARM
TestFunc FUNCPARM
if [[ ${FUNCPARM[__rc__]} -ne 0 ]]; then
echo "ERROR : ${FUNCPARM[__error__]}
fi
Is this kind of thing possible or do I really need to abandon Bash for something like Python?
EDIT: Found the duplicate. This is basically the same answer as this one.
You can use a reference variable for that, see help declare:
declare [-aAfFgilnrtux] [-p] [name[=value] ...]
[...]
-n make NAME a reference to the variable named by its value
[...]
When used in a function, declare makes NAMEs local, as with the local command.
f() {
declare -n paramblock="$1"
# example for reading (print all keys and entries)
paste <(printf %s\\n "${!paramblock[#]}") <(printf %s\\n "${paramblock[#]}")
# example for writing
paramblock["key 1"]="changed"
paramblock["new key"]="new output"
}
Example usage:
$ declare -A a=(["key 1"]="input 1" ["key 2"]="input 2")
$ f a
key 2 input 2
key 1 input 1
$ declare -p a
declare -A a=(["key 2"]="input 2" ["key 1"]="changed" ["new key"]="new output" )
This works very well. The only difference to an actual associative array I found so far is, that you cannot print the referenced array using declare -p as that will only show the reference.

How to iterate over list of dictionaries in bash

I am currently learning shell scripting and need your help!
> array = [{u'name': u'androidTest', u'arn': u'arn:XXX', u'created':
1459270137.749}, {u'name': u'android-provider2016-03-3015:23:30', u'arn':XXXXX', u'created': 1459365812.466}]
I have a list of dictionary and want to extract the arn value from the dictionary. In python it is pretty simple for example:
for project in array:
print project['arn']
How will I write the equivalent loop in bash? If I try something like this, it is not working:
for i in "$array"
do
echo $i['arn']
done
The suggested duplicate is for associative arrays, not a list of associative arrays.
Bash can't nest data structures, so the thing that would correspond to the list of dictionaries doesn't exist natively. Using a relatively recent Bash (4.3 or newer), we can achieve something similar with namerefs:
# Declare two associative arrays
declare -A arr1=(
[name]='androidTest'
[arn]='arn:XXX'
[created]='1459270137.749'
)
declare -A arr2=(
[name]='android-provider2016-03-3015:23:30'
[arn]='XXXXX'
[created]='1459365812.466'
)
# Declare array of names of associative arrays
names=("${!arr#}")
# Declare loop variable as nameref
declare -n arr_ref
# Loop over names
for arr_ref in "${names[#]}"; do
# Print value of key 'arn'
printf "%s\n" "${arr_ref[arn]}"
done
This returns
arn:XXX
XXXXX
for project in $array
do
echo $project{'arn'}
done
Or in one line:
for project in $array; do echo $project{'arn'}; done

Multidimensional associative arrays in Bash

I'm trying to create a multidimensional associative array but need some help. I have reviewed the page suggested in this SO answer but it confused me even more. So far here is what I have:
The script:
#!/bin/bash
declare -A PERSONS
declare -A PERSON
PERSON["FNAME"]='John'
PERSON["LNAME"]='Andrew'
PERSONS["1"]=${PERSON[#]}
PERSON["FNAME"]='Elen'
PERSON["LNAME"]='Murray'
PERSONS["2"]=${PERSON[#]}
for KEY in "${!PERSONS[#]}"; do
TMP="${PERSONS["$KEY"]}"
echo "$KEY - $TMP"
echo "${TMP["FNAME"]}"
echo "${TMP["LNAME"]}"
done
The output:
1 - John Andrew
John Andrew
John Andrew
2 - Elen Murray
Elen Murray
Elen Murray
As you can see trying to access a specific index of the $TMP array in the for loop returns the whole array.
[Q] What do I need to do in order to separately access the "FNAME" and "LNAME" indexes of the $TMP array inside the for loop?
Thanks.
You can't do what you're trying to do: bash arrays are one-dimensional
$ declare -A PERSONS
$ declare -A PERSON
$ PERSON["FNAME"]='John'
$ PERSON["LNAME"]='Andrew'
$ declare -p PERSON
declare -A PERSON='([FNAME]="John" [LNAME]="Andrew" )'
$ PERSONS[1]=([FNAME]="John" [LNAME]="Andrew" )
bash: PERSONS[1]: cannot assign list to array member
You can fake multidimensionality by composing a suitable array index string:
declare -A PERSONS
declare -A PERSON
PERSON["FNAME"]='John'
PERSON["LNAME"]='Andrew'
i=1
for key in "${!PERSON[#]}"; do
PERSONS[$i,$key]=${PERSON[$key]}
done
PERSON["FNAME"]='Elen'
PERSON["LNAME"]='Murray'
((i++))
for key in "${!PERSON[#]}"; do
PERSONS[$i,$key]=${PERSON[$key]}
done
declare -p PERSONS
# ==> declare -A PERSONS='([1,LNAME]="Andrew" [2,FNAME]="Elen" [1,FNAME]="John" [2,LNAME]="Murray" )'
I understand what you need. I also wanted the same for weeks.
I was confused whether to use Python or Bash.
Finally, exploring something else I found this
Bash: How to assign an associative array to another variable name (e.g. rename the variable)?
Here, I got to know how to assign some string and use it later as command.
Then with my creativity I found solution to your problem as below:-
#!/bin/bash
declare -A PERSONS
declare -A PERSON
PERSON["FNAME"]='John'
PERSON["LNAME"]='Andrew'
string=$(declare -p PERSON)
#printf "${string}\n"
PERSONS["1"]=${string}
#echo ${PERSONS["1"]}
PERSON["FNAME"]='Elen'
PERSON["LNAME"]='Murray'
string=$(declare -p PERSON)
#printf "${string}\n"
PERSONS["2"]=${string}
#echo ${PERSONS["2"]}
for KEY in "${!PERSONS[#]}"; do
printf "$KEY - ${PERSONS["$KEY"]}\n"
eval "${PERSONS["$KEY"]}"
printf "${PERSONS["$KEY"]}\n"
for KEY in "${!PERSON[#]}"; do
printf "INSIDE $KEY - ${PERSON["$KEY"]}\n"
done
done
OUTPUT:-
1 - declare -A PERSON='([FNAME]="John" [LNAME]="Andrew" )'
declare -A PERSON='([FNAME]="John" [LNAME]="Andrew" )'
INSIDE FNAME - John
INSIDE LNAME - Andrew
2 - declare -A PERSON='([FNAME]="Elen" [LNAME]="Murray" )'
declare -A PERSON='([FNAME]="Elen" [LNAME]="Murray" )'
INSIDE FNAME - Elen
INSIDE LNAME - Murray
The problem actually with multi dimensional arrays in bash and specifically in your approach is that you are assigning PERSON array values to the array element PERSONS[1] which is converted to a list and not an assoc array when you assigned it.
And so it no longer will take it as 2 elements of an array as you are not keeping any info about the array data structure in your value.
So, I found this hack to be sufficient with only 1 limitation that you will have to do this each time you want to do store/retrieve values. But it shall solve your purpose.

Resources