Output of command to array not working - shell

Question

I'm attempting to store the output of a series of beeline HQL queries into an array, so that I can parse it to pull out the interesting bits. Here's the relevant code:
#!/usr/bin/env ksh
ext_output=()
while IFS= read -r line; do
ext_output+=( "$line" )
done < <( bee --hiveconf hive.auto.convert.join=false -f temp.hql)
bee is just an alias to the full beeline command with the JDBC url, etc. Temp.hql is multiple hql queries.
And here's a snippet of what the output of each query looks like:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| tableName:myTable |
| owner:foo |
| location:hdfs://<server>/<path>...
<big snip>
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
15 rows selected (0.187 seconds)
The problem is, my array is only getting the last line from each result (15 rows selected (0.187 seconds).
Am I doing something wrong here? The exact same approach is working in other instances, so I really don't understand.

Answer 1

Hmmmm, I'm not having any problems with the code you've posted.
I can reproduce what I think you may be seeing (ie, array contains a single value consisting of the last line of output) if I make the following change in your code:
# current/correct code - from your post
ext_output+=( "$line" )
# modified/wrong code
ext_output=+( "$line" )
Notice the placement of the plus sign (+):
when on the left side of the equal sign (+=) each $line is appended to the end of the array (see sample run - below)
when on the right side of the equal sign (=+) each $line is assigned to the first slot in the array (index=0); the plus sign (+) and parens (()) are treated as part of the data to be stored in the array (see sample run - at bottom of this post)
Could there be a typo between what you're running (with 'wrong' results) vs what you've posted here in this thread (and what you've mentioned generates the correct results in other instances)?
Here's what I get when I run your posted code (plus sign on the left of the equal sign : +=) ...
NOTE: I've replaced the bee/HCL call with an output file containing your sample lines plus a couple (bogus) data lines; also cut down the longer lines for readability:
$ cat temp.out
-----------------------------------------+--+
| tableName:myTable
| owner:foo
| location:hdfs://<server>/<path>...
abc def ghi
123 456 789
-----------------------------------------+--+
15 rows selected (0.187 seconds)
Then I ran your code against temp.out:
ext_output=()
while IFS= read -r line
do
ext_output+=( "$line" )
done < temp.out
Some stats on the array:
$ echo "array size : ${#ext_output[*]}"
array size : 10
$ echo "array indx : ${!ext_output[*]}"
array indx : 0 1 2 3 4 5 6 7 8 9
$ echo "array vals : ${ext_output[*]}"
array vals : -----------------------------------------+--+ | tableName:myTable | owner:foo | location:hdfs://<server>/<path>... abc def ghi 123 456 789 -----------------------------------------+--+ 15 rows selected (0.187 seconds)
And a dump of the array's contents:
$ for i in ${!ext_output[*]}
> do
> echo "${i} : ${ext_output[$i]}"
> done
0 : -----------------------------------------+--+
1 : | tableName:myTable
2 : | owner:foo
3 : | location:hdfs://<server>/<path>...
4 :
5 : abc def ghi
6 : 123 456 789
7 :
8 : -----------------------------------------+--+
9 : 15 rows selected (0.187 seconds)
If I modify your code to place the plus sign on the right side of the equal sign (=+) ...
ext_output=()
while IFS= read -r line
do
ext_output=+( "$line" )
done < temp.out
... the array stats:
$ echo "array size : ${#ext_output[*]}"
array size : 1
$ echo "array indx : ${!ext_output[*]}"
array indx : 0
$ echo "array vals : ${ext_output[*]}"
array vals : +( 15 rows selected (0.187 seconds) )
... and the contents of the array:
$ for i in ${!ext_output[*]}
> do
> echo "${i} : ${ext_output[$i]}"
> done
0 : +( 15 rows selected (0.187 seconds) )
!! Notice that the plus sign and parens are part of the string stored in ext_output[0]