This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 4 years ago.
This is my $a output:
[root#node1 ~]# echo "${a}"
/dev/vdc1 /gfs1
/dev/vdd1 /elastic
mfsmount /usr/local/flytxt
I gotta store these in an associative array fsmounts with first column as keys and second col as value.
This is my code for that:
declare -A fsmounts
echo "$a" | while read i ; do key=$(echo "$i" | awk '{print $1}'); value=$(echo "$i" | awk '{print $2}');fsmounts[$key]=$value; done;
But when I try to print outside the loop with
[root#node1 ~]# echo ${fsmounts[/dev/vdb1]}
Blank is the output. I think the associative array fsmounts is not actually storing values. Please help me.
But I can actually able to echo fsmounts[$key] inside the loop. See this:
echo "$a" | while read i ; do key=$(echo "$i" | awk '{print $1}'); value=$(echo "$i" | awk '{print $2}');fsmounts[$key]=$value; echo ${fsmounts[$key]}; done;
/gfs1
/elastic
/usr/local/flytxt
The imminent problem associated with your logic is I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?
You don't need awk or any third party tool at all for this. Just run it over a loop with a read and feed the string via standard input over the loop
declare -A fsmounts
while read -r key value; do
fsmounts["$key"]="$value"
done <<<"$a"
now loop over the array with key and values
for key in "${!fsmounts[#]}"; do
printf 'key=%s value=%s\n' "$key" "${fsmounts[$key]}"
done
More on how to use associative arrays - How can I use variable variables (indirect variables, pointers, references) or associative arrays?
Related
This question already has an answer here:
How to store state between two consecutive runs of a bash script
(1 answer)
Closed 1 year ago.
I wrote this two simple functions to backup and restore the content of a bash dictionary:
declare -A dikv
declare -A dict
backup_dikv()
{
FILE=$1
rm -f $FILE
for k in "${!dikv[#]}"
do
echo "$k,${dikv[$k]}" >> $FILE
done
}
restore_dict()
{
FILE=$1
for i in $(cat $FILE)
do
key=$(echo $i | cut -f 1 -d ",")
val=$(echo $i | cut -f 2 -d ",")
dict[$key]=$val
done
}
# Initial values
dikv=( ["k1"]="v1" ["k2"]="v2" ["k3"]="v3" ["k4"]="v4")
backup_dikv /tmp/backup
restore_dict /tmp/backup
echo "${!dict[#]}"
echo "${dict[#]}"
My questions:
As you can see, these two funcions are very limited as the name of the backuped (dikv) and restored (dict) dictionaries is hardcoded. I would like to pass the dictionary as an input ($2) argument, but I don't know how to pass dictionaries as funcion arguments in bash.
Is this method to write keys and values into a file, using a string format ("key","value") and parse that string format to restore the dictionary, the unique / most eficient way to do that? Do you know some better mechanism to backup and restore a dictionary?
Thanks!
Use declare -p to reliably serialize variables regardless of their type
#!/usr/bin/env bash
if [ -f saved_vars.sh ]; then
# Restore saved variables
. saved_vars.sh
else
# No saved variables, so lets populate them
declare -A dikv=([foo]="foo bar from dikv" [bar]="bar baz from dikv")
declare -A dict=([baz]="baz qux from dict" [qux]="qux corge from dict")
fi
# Serialise backup dikv dict into the saved_vars.sh file
declare -p dikv dict >'saved_vars.sh'
printf %s\\n "${!dict[#]}"
printf %s\\n "${dict[#]}"
printf %s\\n "${!dikv[#]}"
printf %s\\n "${dikv[#]}"
Found a way to pass arrays to functions, using local -n in this way:
declare -A dikv
declare -A dict
backup_dictionary()
{
local -n dict_ref=$1
FILE=/tmp/backup
for k in "${!dict_ref[#]}"
do
echo "$k,${dict_ref[$k]}" >> $FILE
done
}
restore_dictionary()
{
local -n dict_ref=$1
FILE=/tmp/backup
for i in $(cat $FILE)
do
key=$(echo $i | cut -f 1 -d ",")
val=$(echo $i | cut -f 2 -d ",")
dict_ref[$key]=$val
done
}
dikv=( ["k1"]="v1" ["k2"]="v2" ["k3"]="v3" ["k4"]="v4")
backup_dictionary dikv
restore_dictionary dict
echo "${!dict[#]}"
echo "${dict[#]}"
Still trying to find the most convenient way to backup and restore the content.
My shell script look like this
i="10 ID:794 A:TX-SPN S:0"
A=`echo $i | cut -d" " -f 3| cut -d":" -f2` # gives TX-SPN
ID=`echo $i | cut -d" " -f 2|cut -d":" -f2` # gives 794
sZeroCount=`echo $i | cut -d" " -f 1` # gives 10
by above commands,I am able to get the values for A,ID,sZeroCount variables, since the value for i contains only one entry, value of i not limited to 1 it may go upto 1000. Is there any better approach in which I can obtain those values.
With an array. Split string i with separator space and : to array a:
i="10 ID:794 A:TX-SPN S:0"
IFS=" :" a=($i)
echo "${a[4]}" # TX-SPN
echo "${a[2]}" # 794
echo "${a[0]}" # 10
With chepner's bugfix:
i="10 ID:794 A:TX-SPN S:0"
IFS=": " read -a a <<< "$i"
echo "${a[4]}" # TX-SPN
echo "${a[2]}" # 794
echo "${a[0]}" # 10
With this piece of code you can convert your line into a proper associative array:
declare -A dict
for token in START:$i # choose a value for START that is not a key
do
IFS=: read key value <<< "$token"
dict["$key"]=$value
done
You can dump the result using declare -p dict:
declare -A dict='([A]="TX-SPN" [S]="0" [ID]="794" [START]="10" )'
And you can access the contents e. g. using this: echo "${dict[A]}"
TX-SPN
The start value (the 10 in your example) can be accessed as "${dict[START]}". Choose a value for START that doesn't appear as key in your input.
If you want to iterate over a lot of lines like your $i, you can do it like this:
while read i
do
declare -A dict
# ... add code from above ...
done < input_file
The advantage of using associative arrays is that this way you can access your values in a much more understandable way, i. e. by using the keys instead of some arbitrary indexes which can easily be mixed up and which need constant maintenance when changing your code.
This question already has answers here:
Sort keys and Sum their values in bash
(4 answers)
sum of column in text file using shell script
(4 answers)
How can I sum values in column based on the value in another column?
(5 answers)
Closed 4 years ago.
I have a file with below records:
$ cat sample.txt
ABC,100
XYZ,50
ABC,150
QWE,100
ABC,50
XYZ,100
Expecting the output to be:
$ cat output.txt
ABC,300
XYZ,150
QWE,100
I tried the below script:
PREVVAL1=0
SUM1=0
cat sam.txt | sort > /tmp/Pos.part
while read line
do
VAL1=$(echo $line | awk -F, '{print $1}')
VAL2=$(echo $line | awk -F, '{print $2}')
if [ $VAL1 == $PREVVAL1 ]
then
SUM1=` expr $SUM + $VAL2`
PREVVAL1=$VAL1
echo $VAL1 $SUM1
else
SUM1=$VAL2
PREVVAL1=$VAL1
fi
done < /tmp/Pos.part
I want to get some one liner command to get the required output. Wanted to avoid the while loop concept. I want to just add the numbers where the first column is same and show it in a single line.
awk -F, '{a[$1]+=$2} END{for (i in a) print i FS a[i]}' sample.txt
Output
QWE,100
XYZ,150
ABC,300
The first part is executed for each line and creates an associative array. The END part prints this array.
It's an awk one-liner:
awk -F, -v OFS=, '{sum[$1]+=$2} END {for (key in sum) print key, sum[key]}' sample.txt > output.txt
sum[$1] += $2 creates an associative array whose keys are the first field and values are the corresponding sums.
This can also be done easily enough in native bash. The following uses no external tools, no subshells and no pipelines, and is thus far faster (I'd place money on 100x the throughput on a typical/reasonable system) than your original code:
declare -A sums=( )
while IFS=, read -r name val; do
sums[$name]=$(( ${sums[$name]:-0} + val ))
done
for key in "${!sums[#]}"; do
printf '%s,%s\n' "$key" "${sums[$key]}"
done
If you want to, you can make this a one-liner:
declare -A sums=( ); while IFS=, read -r name val; do sums[$name]=$(( ${sums[$name]:-0} + val )); done; for key in "${!sums[#]}"; do printf '%s,%s\n' "$key" "${sums[$key]}"; done
This question already has answers here:
How do I pipe a file line by line into multiple read variables?
(3 answers)
How to split one string into multiple variables in bash shell? [duplicate]
(5 answers)
Read tab-separated file line into array
(3 answers)
Closed 4 years ago.
Suppose I have a string with pipe separator:
str="1|2|3|4"
I want them to be assigned to specific variables.
var_a=1
var_b=2
var_c=3
var_d=4
I am doing it in this way:
var_a="`echo $str | cut -d'|' -f1`"
var_b="`echo $str | cut -d'|' -f2`"
var_c="`echo $str | cut -d'|' -f3`"
var_d="`echo $str | cut -d'|' -f4`"
Can this be done in an efficient way? Please suggest.
It is better to use an array to store individual delimited values:
str="1|2|3|4"
IFS='|' read -ra arr <<< "$str"
#examine array values
declare -p arr
declare -a arr='([0]="1" [1]="2" [2]="3" [3]="4")'
To loop through array, use:
for i in "${arr[#]}"; do echo "$i"; done
1
2
3
4
IFS='|' read -r var_a var_b var_c var_d rest <<<"$str"
rest is the variable that gets further columns after the first four, should any others exist. If you just want to discard them, the conventional name to use for a placeholder variable is _.
This is covered in detail in BashFAQ #1: How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
I have the following at the moment:
for file in *
do
list="$list""$file "`cat $file | wc -l | sort -k1`$'\n'
done
echo "$list"
This is printing:
fileA 10
fileB 20
fileC 30
I would then like to cycle through $list and cut column 2 and perform calculations.
When I do:
for line in "$list"
do
noOfLinesInFile=`echo "$line" | cut -d\ -f2`
echo "$noOfLinesInFile"
done
It prints:
10
20
30
BUT, the for loop is only being entered once. In this example, it should be entering the loop 3 times.
Can someone please tell me what I should do here to achieve this?
If you quote the variable
for line in "$list"
there is only one word, so the loop is executed just once.
Without quotes, $line would be populated with any word found in the $list, which is not what you want, either, as it would process the values one by one, not lines.
You can set the $IFS variable to newline to split $list on newlines:
IFS=$'\n'
for line in $list ; do
...
done
Don't forget to reset IFS to the original value - either put the whole part into a subshell (if no variables should survive the loop)
(
IFS=$'\n'
for ...
)
or backup the value:
IFS_=$IFS
IFS=$'\n'
for ...
IFS=$IFS_
...
done
This is because list in shell are just defined using space as a separator.
# list="a b c"
# for i in $list; do echo $i; done
a
b
c
# for i in "$list"; do echo $i; done
a b c
in your first loop, you actually are not building a list in shell sens.
You should setting other than default separators either for the loop, in the append, or in the cut...
Use arrays instead:
#!/bin/bash
files=()
linecounts=()
for file in *; do
files+=("$file")
linecounts+=("$(wc -l < "$file")")
done
for i in "${!files[#]}" ;do
echo "${linecounts[i]}"
printf '%s %s\n' "${files[i]}" "${linecounts[i]}" ## Another form.
done
Although it can be done simpler as printf '%s\n' "${linecounts[#]}".
wc -l will only output one value, so you don't need to sort it:
for file in *; do
list+="$file "$( wc -l < "$file" )$'\n'
done
echo "$list"
Then, you can use a while loop to read the list line-by-line:
while read file nlines; do
echo $nlines
done <<< "$list"
That while loop is fragile if any filename has spaces. This is a bit more robust:
while read -a words; do
echo ${words[-1]}
done <<< "$list"