How to split blocks of terminal output into array elements that can be looped over - bash

Just looking for the best practice method to split a multi line output such as you might get from a command or a curl request into an array of entries using Bash or Zsh
For example, if a command or printing a file gave an output of
"User A:
Name: Bob
Age: 36
User B:
Name: Jeff
Age: 42"
How would you create an array from that where each user were an entry in the array?
Or if you had an output of say devices similar to
"Computer A:
Name Bob's Computer
Serial 123456
Uptime 12hr
Computer B:
Name Jeff's Computer
Serial 789101
Uptime 8hr"
How would you split that into an array of computers, so that you could do things like see how many computers there were by the number of elements in the array, or pass them one by one to another command, etc?
I've been looking for ways to split strings and output, all the answers I find seem to target splitting a single line with a single character deliminator. I figure the way to do it is either to split by using "User" or "Computer" as the deliminator in the above examples, or to use those as a pattern to read from and to, but I'm not sure how to do that in Bash?
Thanks in advance

Assuming:
You want to split the lines into e.g. "Computer A" block and "Computer B" block then store the computer (or User) names into an array.
(May be optional) You want to parse the lines of attributes such as "Name", "Serial" ... and store them in array of arrays.
Then would you please try the following:
#!/bin/bash
str="Computer A:
Name Bob's Computer
Serial 123456
Uptime 12hr
Computer B:
Name Jeff's Computer
Serial 789101
Uptime 8hr"
keyword="Computer" # the keyword to split the lines into array
declare -A hash # create associative array
i=0 # index of the "real" array name
readarray -t a <<< "$str" # read the string splitting on newlines
for line in "${a[#]}"; do # loop over lines
if [[ $line =~ ^${keyword}[[:blank:]]+([^:]+): ]]; then
# if the line starts with the "keyword"
name="${BASH_REMATCH[1]}" # name is assigned to "A" or "B" ...
hash[$name]="array$i" # define "real" array name and assign the hash value to it
declare -A "array$i" # create a new associative array with the name above
declare -n ref="array$i" # "ref" is a reference to the newly created associative array
(( i++ )) # increment the index for new entry
else
read -r key val <<< "$line" # split on the 1st blank character
key=${key%:} # remove traiking ":"
key=${key## } # remove leading whitespace(s)
ref[$key]=$val # store the "key" and "val" pair
fi
done
print number of elements of the array
echo "The array has ${#hash[#]} elements"
# print the values of each array of the array
for i in "${!hash[#]}"; do # loop over "A" and "B"
echo "$i" # print it
declare -n ref=${hash[$i]} # assign ref to array name "array0" or "array1" ... then make it an indirect reference
for j in "${!ref[#]}"; do # loop over "${array0[#]}" or "${array1[#]}" ...
echo " $j => ${ref[$j]}"
done
done
Output:
The array has 2 elements
A
Name => Bob's Computer
Uptime => 12hr
Serial => 123456
B
Name => Jeff's Computer
Uptime => 8hr
Serial => 789101
Please note bash does not natively support array of arrays and we need to make use of declare -n statement to create a reference to a variable, which makes the code less readable. If Python is your option, please let me know. It will be much more suitable for this kind of task.

This will do the trick in Zsh:
split-blocks() {
local MATCH MBEGIN MEND
reply=(
${(0)1//(#m)$'\n'[^[:space:]]## [^[:space:]]##:[[:space:]]##/$'\0'$MATCH}
)
}
What this does:
Find each occurrence of <newline><text> <text>:<whitespace>.
Insert a null byte before each match.
Split the result on null bytes.
Assign the resulting elements to array $reply.

Related

How to concatenate string to comma-separated element in bash

I am new to Bash coding. I would like to concatenate a string to each element of a comma-separated strings "array".
This is an example of what I have in mind:
s=a,b,c
# Here a function to concatenate the string "_string" to each of them.
# Expected result:
a_string,b_string,c_string
One way:
$ s=a,b,c
$ echo ${s//,/_string,}_string
a_string,b_string,c_string
Using a proper array is generally a much more robust solution. It allows the values to contain literal commas, whitespace, etc.
s=(a b c)
printf '%s\n' "${s[#]/%/_string}"
As suggested by chepner, you can use IFS="," to merge the result with commas.
(IFS=","; echo "${s[#]/%/_string}")
(The subshell is useful to keep the scope of the IFS reassignment from leaking to the current shell.)
Simply, you could use a for loop
main() {
local input='a,b,c'
local append='_string'
# create an 'output' variable that is empty
local output=
# convert the input into an array called 'items' (without the commas)
IFS=',' read -ra items <<< "$input"
# loop over each item in the array, and append whatever string we want, in this case, '_string'
for item in "${items[#]}"; do
output+="${item}${append},"
done
# in the loop, the comma was re-added back. now, we must remove the so there are only commas _in between_ elements
output=${output%,}
echo "$output"
}
main
I've split it up in three steps:
Make it into an actual array.
Append _string to each element in the array using Parameter expansion.
Turn it back into a scalar (for which I've made a function called turn_array_into_scalar).
#!/bin/bash
function turn_array_into_scalar() {
local -n arr=$1 # -n makes `arr` a reference the array `s`
local IFS=$2 # set the field separator to ,
arr="${arr[*]}" # "join" over IFS and assign it back to `arr`
}
s=a,b,c
# make it into an array by turning , into newline and reading into `s`
readarray -t s < <(tr , '\n' <<< "$s")
# append _string to each string in the array by using parameter expansion
s=( "${s[#]/%/_string}" )
# use the function to make it into a scalar again and join over ,
turn_array_into_scalar s ,
echo "$s"

return array from perl to bash

I'm trying to get back an array from perl to bash.
My perl scrip has an array and then I use return(#arr)
from my bash script I use
VAR = `perl....
when I echo VAR
I get the aray as 1 long string with all the array vars connected with no spaces.
Thanks
In the shell (and in Perl), backticks (``) capture the output of a command. However, Perl's return is normally for returning variables from subroutines - it does not produce output, so you probably want print instead. Also, in bash, array variables are declared with parentheses. So this works for me:
$ ARRAY=(`perl -wMstrict -le 'my #array = qw/foo bar baz/; print "#array"'`); \
echo "<${ARRAY[*]}> 0=${ARRAY[0]} 1=${ARRAY[1]} 2=${ARRAY[2]}"
<foo bar baz> 0=foo 1=bar 2=baz
In Perl, interpolating an array into a string (like "#array") will join the array with the special variable $" in between elements; that variable defaults to a single space. If you simply print #array, then the array elements will be joined by the variable $,, which is undef by default, meaning no space between the elements. This probably explains the behavior you mentioned ("the array vars connected with no spaces").
Note that the above will not work the way you expect if the elements of the array contain whitespace, because bash will split them into separate array elements. If your array does contain whitespace, then please provide an MCVE with sample data so we can perhaps make an alternative suggestion of how to return that back to bash. For example:
( # subshell so IFS is only affected locally
IFS=$'\n'
ARRAY=(`perl -wMstrict -e 'my #array = ("foo","bar","quz baz"); print join "\n", #array'`)
echo "0=<${ARRAY[0]}> 1=<${ARRAY[1]}> 2=<${ARRAY[2]}>"
)
Outputs: 0=<foo> 1=<bar> 2=<quz baz>
Here is one way using Bash word splitting, it will split the string on white space into the new array array:
array_str=$(perl -E '#a = 1..5; say "#a"')
array=( $array_str )
for item in ${array[#]} ; do
echo ": $item"
done
Output:
: 1
: 2
: 3
: 4
: 5

Open file with two columns and dynamically create variables

I'm wondering if anyone can help. I've not managed to find much in the way of examples and I'm not sure where to start coding wise either.
I have a file with the following contents...
VarA=/path/to/a
VarB=/path/to/b
VarC=/path/to/c
VarD=description of program
...
The columns are delimited by the '=' and some of the items in the 2nd column may contain gaps as they aren't just paths.
Ideally I'd love to open this in my script once and store the first column as the variable and the second as the value, for example...
echo $VarA
...
/path/to/a
echo $VarB
...
/path/to/a
Is this possible or am I living in a fairy land?
Thanks
You might be able to use the following loop:
while IFS== read -r name value; do
declare "$name=$value"
done < file.txt
Note, though, that a line like foo="3 5" would include the quotes in the value of the variable foo.
A minus sign or a special character isn't allowed in a variable name in Unix.
You may consider using BASH associative array for storing key and value together:
# declare an associative array
declare -A arr
# read file and populate the associative array
while IFS== read -r k v; do
arr["$k"]="$v"
done < file
# check output of our array
declare -p arr
declare -A arr='([VarA]="/path/to/a" [VarC]="/path/to/c" [VarB]="/path/to/b" [VarD]="description of program" )'
What about source my-file? It won't work with spaces though, but will work for what you've shared. This is an example:
reut#reut-home:~$ cat src
test=123
test2=abc/def
reut#reut-home:~$ echo $test $test2
reut#reut-home:~$ source src
reut#reut-home:~$ echo $test $test2
123 abc/def

How to use the single dimensional array in shell script and display the array?

results=
results['startlogdate']="Start time"
results['endlogdate']="$finish_time"
echo "${results[*]}"
I am trying to initialise the array and adding the value to array and echo the array. The code above is my attempt.
In bash scripts, there are two kinds of arrays: numerically indexed and associatively indexed.
Depending on the version of your shell, associatively indexed arrays might not be supported.
Related to the example in your question, the correct syntax to obtain the values of the array, each as a separate word, is:
"${results[#]}"
To get the keys of the associative array, do:
"${!results[#]"
The script below demonstrates the use of an associative array. For more details, see the Arrays section in the bash manpage.
#!/bin/bash
# tst.sh
declare -A aa
aa[foo]=bar
aa[fee]=baz
aa[fie]=tar
for key in "${!aa[#]}" ; do
printf "key: '%s' val: '%s'\n" $key "${aa[$key]}"
done
echo "${aa[#]}"
exit
Here is the output:
$ bash tst.sh
key: 'foo' val: 'bar'
key: 'fee' val: 'baz'
key: 'fie' val: 'tar'
tar bar baz
Finally, I've made available my library of array functions (aka "lists"), which I've been using for many years to make managing data in arrays easy.
Check out https://github.com/aks/bash-lib/blob/master/list-utils.sh
Even if you choose not to make use of the library, you can learn a lot about arrays by reading the code there.
Good luck.
If want to use array in bash. you will be able to do in two ways.
Declare an array in bash.
declare -a Unix=('Debian' 'Red hat' 'Red hat' 'Suse' 'Fedora');
echo ${Unix[0]} # Prints the first element
echo ${Unix[*]} # prints all the elements of an array
Use directly (i.e) without declare.
Unix[0]='Debian';Unix[1]='Red hat'
finish_time=`date`
results[0]="Start time"
results[1]="$finish_time"
echo ${results[#]}
Output: Start time Wed Jan 8 12:25:14 IST 2014
Number of elements: echo ${#results[#]}
Arrays in bash are zero indexed, so ${results[0]} will be "Start time" and ${results[1]} will be "Wed Jan 8 12:25:14 IST 2014"

bash find keyword in an associative array

I have incoming messages from a chat server that need to be compared against a list of keywords. I was using regular arrays, but would like to switch to associative arrays to try to increase the speed of the processing.
The list of words would be in an array called aWords and the values would be a 'type' indicator, i.e. aWords[damn]="1", with 1 being swear word in a legend to inform the user.
The issue is that I need to compare every index value with the input $line looking for substrings. I'm trying to avoid a loop thru each index value if at all possible.
From http://tldp.org/LDP/abs/html/string-manipulation.html, I'm thinking of the Substring Removal section.
${string#substring}
Deletes shortest match of $substring from front of $string.
A comparison of the 'removed' string from the $line, may help, but will it match also words in the middle of other words? i.e. matching the keyword his inside of this.
Sorry for the long-winded post, but I tried to cover all of what I'm attempting to accomplish as best I could.
# create a colon-separated string of the array keys
# you can do this once, after the array is created.
keys=$(IFS=:; echo "${!aWords[*]}")
if [[ ":$keys:" == *:"$word":* ]]; then
# $word is a key in the array
case ${aWords[$word]} in
1) echo "Tsk tsk: $word is a swear word" ;;
# ...
esac
fi
This is the first time I heard of associative arrays in bash. It inspired me to also try to add something, with the chance ofcourse that I completely miss the point.
Here is a code snippet. I hope I understood how it works:
declare -A SWEAR #create associative array of swearwords (only once)
while read LINE
do
[ "$LINE"] && SWEAR["$LINE"]=X
done < "/path/to/swearword/file"
while :
do
OUTGOING="" #reset output "buffer"
read REST #read a sentence from stdin
while "$REST" #evaluate every word in the sentence
do
WORD=${REST%% *}
REST=${REST#* }
[ ${SWEAR[$WORD]} ] && WORD="XXXX"
OUTGOING="$OUTGOING $WORD"
done
echo "$OUTGOING" #output to stdout
done

Resources