how to create a variable for evaluation in Bash [duplicate] - bash

This question already has answers here:
Dynamic variable names in Bash
(19 answers)
Lookup shell variables by name, indirectly [duplicate]
(5 answers)
Closed 12 months ago.
I have a CSV that says this:
Source,Target,Database,Table,Is_deletable,Action
DBServer1,DBServer2,DB,TBL1,true,Add
DBServer2,DBServer1,DB,TBL2,true,Add
I have shell script that does this:
while IFS=, read -r source target database table is_deletable action; do
echo "Building pipeline for ${database}.${table}";
total=$((total+1))
Shost="server1.myorganization.com"
Thost="server2.myorganization.com"
I need Shost to look like this:
Shost = ${'the value of the source column'}
Thost = ${'the value of the target column'}
How do I set this to evaluate dynamically the variable I need based on the value of the column data.
For example:
Shost=${DBServer1}
Thost=${DBServer2}
then on the next loop:
Shost=${DBServer2}
Thost=${DBServer1}
Thanks!

Something like this should work for you:
DBServer1='server1.myorganization.com'
DBServer2='server2.myorganization.com'
while IFS=, read -r source target database table is_deletable action; do
[[ $source = "Source" ]] && continue
((total++))
Shost="${!source}"
Thost="${!target}"
# check variables Shost Thost total
declare -p Shost Thost total
done < file.csv
declare -- Shost="server1.myorganization.com"
declare -- Thost="server2.myorganization.com"
declare -- total="1"
declare -- Shost="server2.myorganization.com"
declare -- Thost="server1.myorganization.com"
declare -- total="2"

We can also use an associative array for this problem:
declare -A servers=(
[DBServer1]='server1.myorganization.com'
[DBServer2]='server2.myorganization.com'
)
{
read header
while IFS=, read -r source target database table is_deletable action; do
Shost=${servers[$source]}
Thost=${servers[$target]}
...
done
} < file.csv

In Bash -- but not necessarily in other shells -- you can reference a variable indirectly, exactly as you seem to want to do:
$ DBServer1=db1.my.com
$ source=DBServer1
$ echo ${source}
DBServer1
$ echo ${!source}
db1.my.com
As the Bash manual puts it (ref):
If [in a parameter expansion of the
form ${parameter}] the first character of parameter is an exclamation point (!), and parameter is
not a nameref, it introduces a level of variable indirection. Bash
uses the value of the variable formed from the rest of parameter as
the name of the variable; this variable is then expanded and that
value is used in the rest of the substitution, rather than the
value of parameter itself.
Applying that to your sample code and data, we get
DBServer1=server1.myorganization.com
DBServer2=server2.myorganization.com
# ...
while IFS=, read -r source target database table is_deletable action; do
echo "Building pipeline for ${database}.${table}";
total=$((total+1))
Shost="${!source}"
Thost="${!target}"

Related

how to assign each of multiple lines in a file as different variable?

this is probably a very simple question. I looked at other answers but couldn't come up with a solution. I have a 365 line date file. file as below,
01-01-2000
02-01-2000
I need to read this file line by line and assign each day to a separate variable. like this,
d001=01-01-2000
d002=02-01-2000
I tried while read commands but couldn't get them to work.It takes a lot of time to shoot one by one. How can I do it quickly?
Trying to create named variable out of an associative array, is time waste and not supported de-facto. Better use this, using an associative array:
#!/bin/bash
declare -A array
while read -r line; do
printf -v key 'd%03d' $((++c))
array[$key]=$line
done < file
Output
for i in "${!array[#]}"; do echo "key=$i value=${array[$i]}"; done
key=d001 value=01-01-2000
key=d002 value=02-01-2000
Assumptions:
an array is acceptable
array index should start with 1
Sample input:
$ cat sample.dat
01-01-2000
02-01-2000
03-01-2000
04-01-2000
05-01-2000
One bash/mapfile option:
unset d # make sure variable is not currently in use
mapfile -t -O1 d < sample.dat # load each line from file into separate array location
This generates:
$ typeset -p d
declare -a d=([1]="01-01-2000" [2]="02-01-2000" [3]="03-01-2000" [4]="04-01-2000" [5]="05-01-2000")
$ for i in "${!d[#]}"; do echo "d[$i] = ${d[i]}"; done
d[1] = 01-01-2000
d[2] = 02-01-2000
d[3] = 03-01-2000
d[4] = 04-01-2000
d[5] = 05-01-2000
In OP's code, references to $d001 now become ${d[1]}.
A quick one-liner would be:
eval $(awk 'BEGIN{cnt=0}{printf "d%3.3d=\"%s\"\n",cnt,$0; cnt++}' your_file)
eval makes the shell variables known inside your script or shell. Use echo $d000 to show the first one of the newly defined variables. There should be no shell special characters (like * and $) inside your_file. Remove eval $() to see the result of the awk command. The \" quoted %s is to allow spaces in the variable values. If you don't have any spaces in your_file you can remove the \" before and after %s.

Dynamically creating associative arrays in bash

I have a variable ($OUTPUT) that contains the following name / value pairs:
member_id=4611686018429783292
platform=Xbox
platform_id=1
character_id=2305843009264966985
period_dt=2020-11-25 20:31:14.923158 UTC
mode=all Crucible modes
mode_id=5
activities_entered=18
activities_won=10
activities_lost=8
assists=103
kills=233
average_kill_distance=15.729613
total_kill_distance=3665
seconds_played=8535
deaths=118
average_lifespan=71.72269
total_lifespan=8463.277
opponents_defeated=336
efficiency=2.8474576
kills_deaths_ratio=1.9745762
kills_deaths_assists=2.411017
suicides=1
precision_kills=76
best_single_game_kills=-1
Each line ends with \n.
I want to loop through them, and parse them into an associative array, and the access the values in the array by the variable names:
while read line
do
key=${line%%=*}
value=${line#*=}
echo $key=$value
data[$key]="$value"
done < <(echo "$OUTPUT")
#this always prints the last value
echo ${data['seconds_played']}
This seems to work, i.e. key/value print the right values, but when I try to pull any values from the array, it always returns the last value (in this case -1).
I feel like im missing something obvious, but have been banging my head against it for a couple of hours.
UPDATE: My particular issue is I'm running a version of bash (3.2.57 on OSX) that doesn't support associative arrays). I'll mark the correct answer below.
Without declare -A data, then data is a normal array. In normal arrays expressions in [here] first undergo expansions, then arithmetic expansion. Inside arithmetic expansion unset variables are expanded to 0. You are effectively only just setting data[0]=something, because data[$key] is data[seconds_played] -> variable seconds_played is not defined, so it expands to data[0]
Add declare -A data and it "should work". You could also just:
declare -A data
while IFS== read -r key value; do
data["$key"]="$value"
done <<<"$OUTPUT"
Try declaring data as an associative array before populating it, eg:
$ typeset -A data # declare as an associative array
$ while read line
do
key=${line%%=*}
value=${line#*=}
echo $key=$value
data[$key]="$value"
done <<< "${OUTPUT}"
$ typeset -p data
declare -A data=([mode]="all Crucible modes" [period_dt]="2020-11-25 20:31:14.923158 UTC" [deaths]="118" [best_single_game_kills]="-1" [efficiency]="2.8474576" [precision_kills]="76" [activities_entered]="18" [seconds_played]="8535" [total_lifespan]="8463.277" [average_lifespan]="71.72269" [character_id]="2305843009264966985" [kills]="233" [activities_won]="10" [average_kill_distance]="15.729613" [activities_lost]="8" [mode_id]="5" [assists]="103" [suicides]="1" [total_kill_distance]="3665" [platform]="Xbox" [kills_deaths_ratio]="1.9745762" [platform_id]="1" [kills_deaths_assists]="2.411017" [opponents_defeated]="336" [member_id]="4611686018429783292" )
$ echo "${data['seconds_played']}"
8535

Adding to Bash associative arrays inside functions

I'm trying to use associative arrays as a work around for Bash's poor function parameter passing. I can declare a global associative array and read/write to that but I would like to have the variable name passed to the function since many times I want to use the same function with different parameter blocks.
Various Stack Overflow posts have approaches for reading a passed array within a function but not writing to it to allow return values. Pseudo Bash for what I'm trying to do is thus:
TestFunc() {
local __PARMBLOCK__=${1} # Tried ${!1} as well
# Do something with incoming array
__PARMBLOCK__[__rc__]+=1 # Error occured
__PARMBLOCK__[__error__]+="Error in TestFunc"
}
declare -A FUNCPARM
# Populate FUNCPARM
TestFunc FUNCPARM
if [[ ${FUNCPARM[__rc__]} -ne 0 ]]; then
echo "ERROR : ${FUNCPARM[__error__]}
fi
Is this kind of thing possible or do I really need to abandon Bash for something like Python?
EDIT: Found the duplicate. This is basically the same answer as this one.
You can use a reference variable for that, see help declare:
declare [-aAfFgilnrtux] [-p] [name[=value] ...]
[...]
-n make NAME a reference to the variable named by its value
[...]
When used in a function, declare makes NAMEs local, as with the local command.
f() {
declare -n paramblock="$1"
# example for reading (print all keys and entries)
paste <(printf %s\\n "${!paramblock[#]}") <(printf %s\\n "${paramblock[#]}")
# example for writing
paramblock["key 1"]="changed"
paramblock["new key"]="new output"
}
Example usage:
$ declare -A a=(["key 1"]="input 1" ["key 2"]="input 2")
$ f a
key 2 input 2
key 1 input 1
$ declare -p a
declare -A a=(["key 2"]="input 2" ["key 1"]="changed" ["new key"]="new output" )
This works very well. The only difference to an actual associative array I found so far is, that you cannot print the referenced array using declare -p as that will only show the reference.

How can I create an array whose name includes a variable?

I'm having some trouble writing a command that includes a String of a variable in Bash and wanted to know the correct way to do it.
I want to try and fill the Row arrays with the numbers 1-9 but I'm getting myself stuck when trying to pass a variable Row$Line[$i]=$i.
Row0=()
Row1=()
Row2=()
FillArrays() {
for Line in $(seq 0 2)
do
for i in $(seq 1 9)
do
Row$Line[$i]=$i
done
done
}
I can get the desired result if I echo the command but I assume that is just because it is a String.
I want the for loop to select each row and add the numbers 1-9 in each array.
FillArrays() {
for ((Line=0; Line<8; Line++)); do
declare -g -a "Row$Line" # Ensure that RowN exists as an array
declare -n currRow="Row$Line" # make currRow an alias for that array
for ((i=0; i<9; i++)); do # perform our inner loop...
currRow+=( "$i" ) # ...and populate the target array...
done
unset -n currRow # then clear the alias so it can be reassigned later.
done
}
References:
https://wiki.bash-hackers.org/syntax/ccmd/c_for describes the C-style for loop in bash
BashFAQ #6 discusses indirect reference and assignment in detail, including techniques that precede namevars.
Variable expansion happens too late for an assignment to understand it. You can delay the assignment by using the declare builtin. -g is needed in a function to make the variable global.
Also, you probably don't want to use $Line as the array index, but $i, otherwise you wouldn't populate each line array with numbers 1..9.
#! /bin/bash
Row0=()
Row1=()
Row2=()
FillArrays() {
for Line in $(seq 0 8)
do
for i in $(seq 1 9)
do
declare -g Row$Line[$i]=$i
done
done
}
FillArrays
echo "${Row1[#]}"
But note that using variables as parts of variable names is dangerous. For me, needing this always means I need to switch from the shell to a real programming language.

Open file with two columns and dynamically create variables

I'm wondering if anyone can help. I've not managed to find much in the way of examples and I'm not sure where to start coding wise either.
I have a file with the following contents...
VarA=/path/to/a
VarB=/path/to/b
VarC=/path/to/c
VarD=description of program
...
The columns are delimited by the '=' and some of the items in the 2nd column may contain gaps as they aren't just paths.
Ideally I'd love to open this in my script once and store the first column as the variable and the second as the value, for example...
echo $VarA
...
/path/to/a
echo $VarB
...
/path/to/a
Is this possible or am I living in a fairy land?
Thanks
You might be able to use the following loop:
while IFS== read -r name value; do
declare "$name=$value"
done < file.txt
Note, though, that a line like foo="3 5" would include the quotes in the value of the variable foo.
A minus sign or a special character isn't allowed in a variable name in Unix.
You may consider using BASH associative array for storing key and value together:
# declare an associative array
declare -A arr
# read file and populate the associative array
while IFS== read -r k v; do
arr["$k"]="$v"
done < file
# check output of our array
declare -p arr
declare -A arr='([VarA]="/path/to/a" [VarC]="/path/to/c" [VarB]="/path/to/b" [VarD]="description of program" )'
What about source my-file? It won't work with spaces though, but will work for what you've shared. This is an example:
reut#reut-home:~$ cat src
test=123
test2=abc/def
reut#reut-home:~$ echo $test $test2
reut#reut-home:~$ source src
reut#reut-home:~$ echo $test $test2
123 abc/def

Resources