I'm wondering if anyone can help. I've not managed to find much in the way of examples and I'm not sure where to start coding wise either.
I have a file with the following contents...
VarA=/path/to/a
VarB=/path/to/b
VarC=/path/to/c
VarD=description of program
...
The columns are delimited by the '=' and some of the items in the 2nd column may contain gaps as they aren't just paths.
Ideally I'd love to open this in my script once and store the first column as the variable and the second as the value, for example...
echo $VarA
...
/path/to/a
echo $VarB
...
/path/to/a
Is this possible or am I living in a fairy land?
Thanks
You might be able to use the following loop:
while IFS== read -r name value; do
declare "$name=$value"
done < file.txt
Note, though, that a line like foo="3 5" would include the quotes in the value of the variable foo.
A minus sign or a special character isn't allowed in a variable name in Unix.
You may consider using BASH associative array for storing key and value together:
# declare an associative array
declare -A arr
# read file and populate the associative array
while IFS== read -r k v; do
arr["$k"]="$v"
done < file
# check output of our array
declare -p arr
declare -A arr='([VarA]="/path/to/a" [VarC]="/path/to/c" [VarB]="/path/to/b" [VarD]="description of program" )'
What about source my-file? It won't work with spaces though, but will work for what you've shared. This is an example:
reut#reut-home:~$ cat src
test=123
test2=abc/def
reut#reut-home:~$ echo $test $test2
reut#reut-home:~$ source src
reut#reut-home:~$ echo $test $test2
123 abc/def
Related
this is probably a very simple question. I looked at other answers but couldn't come up with a solution. I have a 365 line date file. file as below,
01-01-2000
02-01-2000
I need to read this file line by line and assign each day to a separate variable. like this,
d001=01-01-2000
d002=02-01-2000
I tried while read commands but couldn't get them to work.It takes a lot of time to shoot one by one. How can I do it quickly?
Trying to create named variable out of an associative array, is time waste and not supported de-facto. Better use this, using an associative array:
#!/bin/bash
declare -A array
while read -r line; do
printf -v key 'd%03d' $((++c))
array[$key]=$line
done < file
Output
for i in "${!array[#]}"; do echo "key=$i value=${array[$i]}"; done
key=d001 value=01-01-2000
key=d002 value=02-01-2000
Assumptions:
an array is acceptable
array index should start with 1
Sample input:
$ cat sample.dat
01-01-2000
02-01-2000
03-01-2000
04-01-2000
05-01-2000
One bash/mapfile option:
unset d # make sure variable is not currently in use
mapfile -t -O1 d < sample.dat # load each line from file into separate array location
This generates:
$ typeset -p d
declare -a d=([1]="01-01-2000" [2]="02-01-2000" [3]="03-01-2000" [4]="04-01-2000" [5]="05-01-2000")
$ for i in "${!d[#]}"; do echo "d[$i] = ${d[i]}"; done
d[1] = 01-01-2000
d[2] = 02-01-2000
d[3] = 03-01-2000
d[4] = 04-01-2000
d[5] = 05-01-2000
In OP's code, references to $d001 now become ${d[1]}.
A quick one-liner would be:
eval $(awk 'BEGIN{cnt=0}{printf "d%3.3d=\"%s\"\n",cnt,$0; cnt++}' your_file)
eval makes the shell variables known inside your script or shell. Use echo $d000 to show the first one of the newly defined variables. There should be no shell special characters (like * and $) inside your_file. Remove eval $() to see the result of the awk command. The \" quoted %s is to allow spaces in the variable values. If you don't have any spaces in your_file you can remove the \" before and after %s.
Have a string as follows:
files="applications/dbt/Dockerfile applications/dbt/cloudbuild.yaml applications/dataform/Dockerfile applications/dataform/cloudbuild.yaml"
Want to extract the first two directories and save it as another string like this:
"applications/dbt applications/dbt applications/dataform pplications/dataform"
But while filling up the second string, its being saved as
applications/dbtapplications/dbtapplications/dataformapplications/dataform
What i tried:
files="applications/dbt/Dockerfile applications/dbt/cloudbuild.yaml applications/dataform/Dockerfile applications/dataform/cloudbuild.yaml"
arr=($files)
#extracting the first two directories and saving it to a new string
for i in ${arr[#]}; do files2+=$(echo "$i" | cut -d/ -f 1-2); done
echo $files2
files2 echoes the following
applications/dbtapplications/dbtapplications/dataformapplications/dataform
Reusing your code as much as possible:
(assuming to only remove the last right part):
arr=( applications/dbt/Dockerfile applications/dbt/cloudbuild.yaml applications/dataform/Dockerfile applications/dataform/cloudbuild.yaml )
#extracting the first two directories and saving it to a new string
for file in "${arr[#]}"; do
files2+="${file%/*} "
done
echo "$files2"
applications/dbt applications/dbt applications/dataform
You could use a for loop as requested
for dir in ${files};
do file2+=$(printf '%s ' "${dir%/*}")
done
which will give output
$ echo "$file2"
applications/dbt applications/dbt applications/dataform applications/dataform
However, it would be much easier with sed
$ sed -E 's~([^/]*/[^/]*)[^ ]*~\1~g' <<< $files
applications/dbt applications/dbt applications/dataform applications/dataform
Convert the string in an array first. Assuming there are no white/blank/newline space embedded in your strings/path name. Something like
#!/usr/bin/env bash
files="applications/dbt/Dockerfile applications/dbt/cloudbuild.yaml applications/dataform/Dockerfile applications/dataform/cloudbuild.yaml"
mapfile -t array <<< "${files// /$'\n'}"
Now check the value of the array
declare -p array
Output
declare -a array=([0]="applications/dbt/Dockerfile" [1]="applications/dbt/cloudbuild.yaml" [2]="applications/dataform/Dockerfile" [3]="applications/dataform/cloudbuild.yaml")
Remove all the last / from the path name in the array.
new_array=("${array[#]%/*}")
Check the new value
declare -p new_array
Output
declare -a new_array=([0]="applications/dbt" [1]="applications/dbt" [2]="applications/dataform" [3]="applications/dataform")
Now the value is an array, assign it to a variable or do what ever you like with it. Like what was mentioned in the comment section. Use an array from the start.
Assign the first 2 directories/path in a variable (weird requirement)
new_var="${new_array[#]::2}"
declare -p new_var
Output
declare -- new_var="applications/dbt applications/dbt"
I have a variable ($OUTPUT) that contains the following name / value pairs:
member_id=4611686018429783292
platform=Xbox
platform_id=1
character_id=2305843009264966985
period_dt=2020-11-25 20:31:14.923158 UTC
mode=all Crucible modes
mode_id=5
activities_entered=18
activities_won=10
activities_lost=8
assists=103
kills=233
average_kill_distance=15.729613
total_kill_distance=3665
seconds_played=8535
deaths=118
average_lifespan=71.72269
total_lifespan=8463.277
opponents_defeated=336
efficiency=2.8474576
kills_deaths_ratio=1.9745762
kills_deaths_assists=2.411017
suicides=1
precision_kills=76
best_single_game_kills=-1
Each line ends with \n.
I want to loop through them, and parse them into an associative array, and the access the values in the array by the variable names:
while read line
do
key=${line%%=*}
value=${line#*=}
echo $key=$value
data[$key]="$value"
done < <(echo "$OUTPUT")
#this always prints the last value
echo ${data['seconds_played']}
This seems to work, i.e. key/value print the right values, but when I try to pull any values from the array, it always returns the last value (in this case -1).
I feel like im missing something obvious, but have been banging my head against it for a couple of hours.
UPDATE: My particular issue is I'm running a version of bash (3.2.57 on OSX) that doesn't support associative arrays). I'll mark the correct answer below.
Without declare -A data, then data is a normal array. In normal arrays expressions in [here] first undergo expansions, then arithmetic expansion. Inside arithmetic expansion unset variables are expanded to 0. You are effectively only just setting data[0]=something, because data[$key] is data[seconds_played] -> variable seconds_played is not defined, so it expands to data[0]
Add declare -A data and it "should work". You could also just:
declare -A data
while IFS== read -r key value; do
data["$key"]="$value"
done <<<"$OUTPUT"
Try declaring data as an associative array before populating it, eg:
$ typeset -A data # declare as an associative array
$ while read line
do
key=${line%%=*}
value=${line#*=}
echo $key=$value
data[$key]="$value"
done <<< "${OUTPUT}"
$ typeset -p data
declare -A data=([mode]="all Crucible modes" [period_dt]="2020-11-25 20:31:14.923158 UTC" [deaths]="118" [best_single_game_kills]="-1" [efficiency]="2.8474576" [precision_kills]="76" [activities_entered]="18" [seconds_played]="8535" [total_lifespan]="8463.277" [average_lifespan]="71.72269" [character_id]="2305843009264966985" [kills]="233" [activities_won]="10" [average_kill_distance]="15.729613" [activities_lost]="8" [mode_id]="5" [assists]="103" [suicides]="1" [total_kill_distance]="3665" [platform]="Xbox" [kills_deaths_ratio]="1.9745762" [platform_id]="1" [kills_deaths_assists]="2.411017" [opponents_defeated]="336" [member_id]="4611686018429783292" )
$ echo "${data['seconds_played']}"
8535
I am creating dynamic arrays which all have different letters in their name. For the purpose of this question, my initial array of letters has been set at a fixed length. However, in my final implementation this letter array will be any length.
For each letter, I construct a string
I declare a new array with that string, making use of eval to evaluate the variable's value within the declare command.
I add some values to the array, again using eval to evaluate any variable values.
Here is the code:
declare -a LETTER_ARRAY=( "A" "B" "C" "D" "E" )
for i in "${LETTER_ARRAY[#]}"
do
name_string="apple${i}"
color0="red"
color1="green"
eval "declare -a ${name_string}_array"
eval "${name_string}_array[0]=$color0"
eval "${name_string}_array[1]=$color1"
done
So, how can I iterate through these dynamic arrays and echo what is in them? I have tried the following:
for i in "${LETTER_ARRAY[#]}"
do
eval "array_name='apple${i}_array'"
echo ${array_name[0]}
done
This has not worked for me. I can confirm that my dynamic arrays were successfully created and populated, as when I echo out a value manually, I get a result:
echo ${appleA_array[0]}
red
A perfect place to use a namereference:
letter_array=(A B C D E)
for i in "${letter_array[#]}"; do
declare -n var="apple${i}_array"
var[0]=red
var[1]=green
done
declare -p appleA_array
would output:
declare -a appleA_array=([0]="red" [1]="green")
how can I iterate through these dynamic arrays and echo what is in them?
With the above:
for i in "${letter_array[#]}"; do
declare -n var="apple${i}_array"
printf "%s\n" "${var[0]}"
done
Notes:
Do not use eval. Eval is evil.
Do not use upper case variables, by convention they are used for exported variables, like COLUMNS, PWD, UID, EUID, LINES. Use lower case variables in your scripts.
Check your scripts with http://shellcheck.net for most common mistakes
But if you are creating a 2d array, then an associative array might be better:
declare -A apple_arrays
letter_array=(A B C D E)
for i in "${letter_array[#]}"; do
apple_arrays[$i,0]=red
apple_arrays[$i,1]=green
done
for i in "${letter_array[#]}"; do
printf "one=%s two=%s\n" "${apple_arrays[$i,0]}" "${apple_arrays[$i,1]}"
done
how can I iterate through these dynamic arrays
echo ${array_name[0]} does not work because array_name is not the name of an array; $array_name is. Therefore, eval "echo \"\${${array_name}[0]}\"" would to the trick.
However, I'd recommend namerefs.
By The way: declare works without eval and is more reliable that way.
#! /usr/bin/env bash
letters=({A..E})
for i in "${letters[#]}"; do
declare -a "apple${i}_array=(red green)"
done
for i in "${letters[#]}"; do
declare -n array="apple${i}_array"
# now you can use `array` as if it was `appleA_array`, `appleB_array`, ...
echo "${array[0]}"
done
Your first line is not bash syntax. If I try the line
declare -a LETTER_ARRAY = [ "A" "B" "C" "D" "E" ]
I get:
bash: declare: `=': not a valid identifier
bash: declare: `[': not a valid identifier
bash: declare: `]': not a valid identifier
I think, you get similar error messages, but you ignored them
More errors:
Forgotten do
case mismatch: suffix on definition: _array, and for output: _ARRAY
Use always double quotes when using [#]
One correct syntax is:
declare -a LETTER_ARRAY=( "A" "B" "C" "D" "E" )
for i in "${LETTER_ARRAY[#]}"
do
name_string="apple${i}"
color0="red"
color1="green"
eval "declare -a ${name_string}_array"
echo "${name_string}_array[0]=$color0"
eval "${name_string}_array[0]=$color0"
eval "${name_string}_array[1]=$color1"
done
echo ${appleA_array[0]}
Your eval "array_name='AZ${i}_ARRAY'" makes array_name a scalar, not an array. Arrays in bash are usually created like this
arr=( your elements go here )
If you want to assign one array to another, you have to interpolate the elements between those parenthesis, for instance:
arr=( ${other_array[#]} )
Since you are using bash, this would perform word splitting the elements of other_array, if they contain spaces. Hence you would usually write it for the safe side as
arr=( "${other_array[#]}" )
Hence, for your case, you could do a
eval "array_name=( \${AZ${i}_ARRAY[#]} )"
This causes an array array_name to be created, with the elements of the respective AZi_ARRAY.
I omitted here for simplicity the prevention against word splitting, because in your example, the array elements contain single words only.
I want to create a dictionary in bash from a text file which looks like this:
H96400275|A
H96400276|B
H96400265|C
H96400286|D
Basically I want a dictionary like this from this file file.txt:
KEYS VALUES
H96400275 = A
H96400276 = B
H96400265 = C
H96400286 = D
I created following script:
#!/bin/bash
declare -a dictionary
while read line; do
key=$(echo $line | cut -d "|" -f1)
data=$(echo $line | cut -d "|" -f2)
dictionary[$key]="$data"
done < file.txt
echo ${dictionary[H96400275]}
However, this does not print A, rather it prints D. Can you please help ?
Associative arrays (dictionaries in your terms) are declared using -A, not -a. For references to indexed (ones declared with -a) arrays' elements, bash performs arithmetic expansion on the subscript ($key and H96400275 in this case); so you're basically overwriting dictionary[0] over and over, and then asking for its value; thus D is printed.
And to make this script more effective, you can use read in conjunction with a custom IFS to avoid cuts. E.g:
declare -A dict
while IFS='|' read -r key value; do
dict[$key]=$value
done < file
echo "${dict[H96400275]}"
See Bash Reference Manual ยง 6.7 Arrays.
the only problem is that you have to use -A instead of -a
-a Each name is an indexed array variable (see Arrays above).
-A Each name is an **associative** array variable (see Arrays above).
What you want to do is so named associative array. And to declare it you need to use command:
declare -A dictionary