change array into table bash scripting - bash

how to change array into tabular form?
eg:
array = 1 2 3 4 5 6 7 8 9 10 11 12
result
1 10 11
2 9 12
3 8
4 7
5 6
in this particular order up-down-down-up-up-down
the array is taken from a .txt file and its based on user input so the value varies
here is some of my code
declare -a myarray
# Load file into array.
readarray myarray < temp2.txt
s=$myarray
f or i in $(seq 0 $((${#s} - 1))); do
echo "s[$i] = \"${s:$i:1}\""
done

This script does what you want:
#!/bin/bash
a=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 )
rows=5
for (( j=0; j<rows; ++j )); do
for (( i=0; i<=$(( ${#a[#]} / rows )); ++i )); do
if (( i%2 )); then idx=$(( (i + 1) / 2 * 2 * rows - j - 1 ))
else idx=$(( (i / 2) * 2 * rows + j )); fi
printf "%-4s" "${a[idx]}"
done
printf "\n"
done
Output:
1 10 11
2 9 12
3 8 13 18
4 7 14 17
5 6 15 16
To make it work from left to right rather than from top to bottom, you can simply swap the i and j loops around (and change the name rows to cols so that it still makes sense):
#!/bin/bash
a=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 )
cols=5
for (( i=0; i<=$(( ${#a[#]} / cols )); ++i )); do
for (( j=0; j<cols; ++j )); do
if (( i%2 )); then idx=$(( (i + 1) / 2 * 2 * cols - j - 1 ))
else idx=$(( (i / 2) * 2 * cols + j )); fi
printf "%-4s" "${a[idx]}"
done
printf "\n"
done
Output:
1 2 3 4 5
10 9 8 7 6
11 12 13 14 15
18 17 16

declare -a s
s=(1 2 3 4 5 6 7 8 9 10 11 12)
ofstabled=(9 7 5 3 1)
ofstableu=(1 3 5 7 9)
for ((i=0; i<5;++i)); do
for ((j=$i; j<${#s[#]};)); do
printf "%d " ${s[$j]}
let j=$j+${ofstabled[$i]}
if [ $j -lt ${#s[#]} ]; then
printf "%d " ${s[$j]}
let j=$j+${ofstableu[$i]}
fi
done
printf "\n"
done
You can gussy it up if you want but it works.

Script:
#!/bin/bash
a=(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18)
rows=${1:-5}
col=()
for ((i = 0; i < ${#a[#]}; i++)); do
((ind=i % rows))
(( ((i / rows) % 2 == 0) && (ind = ((rows - 1) - ind)) ))
[ -n "${col[(rows - 1) - $ind]}" ] && col[(rows - 1) - $ind]+=" "
col[(rows - 1) - $ind]+=${a[$i]}
done
printf %s\\n "${col[#]}" | column -t
Output:
$ ./order.sh
1 10 11
2 9 12
3 8 13 18
4 7 14 17
5 6 15 16
$ ./order.sh 3
1 6 7 12 13 18
2 5 8 11 14 17
3 4 9 10 15 16
With credit to #TomFenech for the inspiration to make the row count controllable.

You can use awk to read from the file and print it the way you want:
$ awk '{
if(NR<=5) {
a[NR]=$0
next
} else {
row=NR%10
}
}
!(row in a) {
if(row == 0)
row++
else
row=5 - (NR % 5) + 1
}
{
a[row]=a[row] FS $0
}
END {
for(i in a)
print a[i]
}' file
1 10 11
2 9 12
3 8
4 7
5 6

Related

passing values to another file using shell script

I would like to create a shell script file such that it gets certain values from a .dat or excel .xls file then pass these values to another file as in the following example;
1-The .dat value has 32x5 matrix size;Example
1 3 4 5 6
4 5 7 9 8
:
:
1 1 1 2 4
2-.geo file has the following
x1= ;
x2= ;
x3= ;
x4= ;
x5= ;
I would like to create 32 geo file with each row of the .dat or .xls file e.g.
1.geo has;
x1 = 1;
x2 = 3;
x3 = 4;
x4 = 5;
x5 = 6;
2.geo has the second row etc.
In summary, shell script loop through the rows and pass them to the geo file and save it with different geo file name.
Any help would be appreciated.
Many thanks,
Your exercise is actually simply handled by reading each line into separate variable, e.g. $a, $b, $c, $d and $e. Once read, you can write them out in into separate $count.geo files (e.g. 1.geo, 2.geo, ...) in the format you desire with printf. A simple implementation would be:
#!/bin/sh
dat=${1:-geo.dat} ## .dat filename ('geo.dat' default)
count=1
test -r "$dat" || { ## validate input file is readable
printf "error: file not readable '%s'\n" "$dat"
exit 1
}
while read -r a b c d e; do ## read each value into a b c d e
printf "x1 = %d;\nx2 = %d;\nx3 = %d;\nx4 = %d;\nx5 = %d;\n" \
$a $b $c $d $e >$count.geo
((count++))
done <"$dat"
Using awk, you could use a one-liner similar to:
awk '{printf "x1 = %d;\nx2 = %d;\nx3 = %d;\nx4 = %d;\nx5 = %d;\n", \
$1, $2, $3, $4, $5 > FNR".geo"}' geo.dat
Example Input File geo.dat
$ cat geo.dat
5 9 4 4 2
8 4 3 8 7
5 8 4 3 7
1 3 9 8 2
5 8 2 2 1
7 1 8 3 4
3 7 4 7 2
5 6 6 9 5
3 4 4 8 6
2 9 9 1 7
2 6 5 4 8
5 1 9 4 3
4 6 1 7 5
2 4 1 7 3
6 2 6 1 9
3 3 9 3 4
6 5 9 2 8
8 7 8 8 2
1 2 9 1 1
4 3 5 5 1
8 2 2 5 3
2 7 5 1 1
9 6 5 9 8
4 9 6 2 8
8 3 1 7 4
2 1 6 7 7
7 5 9 9 9
2 3 7 8 3
9 8 1 5 7
9 9 7 6 1
6 4 5 7 2
8 9 3 6 6
Example Use
$ sh readgoedat.sh geo.dat
or
$ awk '{printf "x1 = %d;\nx2 = %d;\nx3 = %d;\nx4 = %d;\nx5 = %d;\n", \
$1, $2, $3, $4, $5 > FNR".geo"}' geo.dat
Example Output Files
$ cat 1.geo
x1 = 5;
x2 = 9;
x3 = 4;
x4 = 4;
x5 = 2;
$ cat 2.geo
x1 = 8;
x2 = 4;
x3 = 3;
x4 = 8;
x5 = 7;
$ cat 32.geo
x1 = 8;
x2 = 9;
x3 = 3;
x4 = 6;
x5 = 6;
Look things over and let me know if you have further questions.

Sorting Columns From File BASH

I have the following shell script that reads in data from a file inputted at the command line. The file is a matrix of numbers, and I need to separate the file by columns and then sort the columns. Right now I can read the file and output the individual columns but I am getting lost on how to sort. I have inputted a sort statement, but it only sorts the first column.
EDIT:
I have decided to take another route and actual transpose the matrix to turn the columns into rows. Since I have to later calculate the mean and median and have already successfully done this for the file row-wise earlier in the script - it was suggested to me to try and "spin" the matrix if you will to turn the columns into rows.
Here is my UPDATED code
declare -a col=( )
read -a line < "$1"
numCols=${#line[#]} # save number of columns
index=0
while read -a line ; do
for (( colCount=0; colCount<${#line[#]}; colCount++ )); do
col[$index]=${line[$colCount]}
((index++))
done
done < "$1"
for (( width = 0; width < numCols; width++ )); do
for (( colCount = width; colCount < ${#col[#]}; colCount += numCols ) ); do
printf "%s\t" ${col[$colCount]}
done
printf "\n"
done
This gives me the following output:
1 9 6 3 3 6
1 3 7 6 4 4
1 4 8 8 2 4
1 5 9 9 1 7
1 5 7 1 4 7
Though I'm now looking for:
1 3 3 6 6 9
1 3 4 4 6 7
1 2 4 4 8 8
1 1 5 7 9 9
1 1 4 5 7 7
To try and sort the data, I have tried the following:
sortCol=${col[$colCount]}
eval col[$colCount]='($(sort <<<"${'$sortCol'[*]}"))'
Also: (which is how I sorted the row after reading in from line)
sortCol=( $(printf '%s\t' "${col[$colCount]}" | sort -n) )
If you could provide any insight on this, it would be greatly appreciated!
Note, as mentioned in the comments, a pure bash solution isn't pretty. There are a number of ways to do it, but this is probably the most straight forward. The following requires reading all values per line into the array, and saving the matrix stride so it can be transposed to read all column values into a row matrix and sorted. All sorted columns are inserted into new row matrix a2. Transposing that row matrix yields your original matrix back in column sort order.
Note this will work for any rank of column matrix in your file.
#!/bin/bash
test -z "$1" && { ## validate number of input
printf "insufficient input. usage: %s <filename>\n" "${0//*\//}"
exit 1;
}
test -r "$1" || { ## validate file was readable
printf "error: file not readable '%s'. usage: %s <filename>\n" "$1" "${0//*\//}"
exit 1;
}
## function: my sort integer array - accepts array and returns sorted array
## Usage: array=( "$(msia ${array[#]})" )
msia() {
local a=( "$#" )
local sz=${#a[#]}
local _tmp
[[ $sz -lt 2 ]] && { echo "Warning: array not passed to fxn 'msia'"; return 1; }
for((i=0;i<$sz;i++)); do
for((j=$((sz-1));j>i;j--)); do
[[ ${a[$i]} -gt ${a[$j]} ]] && {
_tmp=${a[$i]}
a[$i]=${a[$j]}
a[$j]=$_tmp
}
done
done
echo ${a[#]}
unset _tmp
unset sz
return 0
}
declare -a a1 ## declare arrays and matrix variables
declare -a a2
declare -i cnt=0
declare -i stride=0
declare -i sz=0
while read line; do ## read all lines into array
a1+=( $line );
(( cnt == 0 )) && stride=${#a1[#]} ## calculate matrix stride
(( cnt++ ))
done < "$1"
sz=${#a1[#]} ## calculate matrix size
## print original array
printf "\noriginal array:\n\n"
for ((i = 0; i < sz; i += stride)); do
for ((j = 0; j < stride; j++)); do
printf " %s" ${a1[i+j]}
done
printf "\n"
done
## sort columns from stride array
for ((j = 0; j < stride; j++)); do
for ((i = 0; i < sz; i += stride)); do
arow+=( ${a1[i+j]} )
done
a2+=( $(msia ${arow[#]}) ) ## create sorted array
unset arow
done
## print the sorted array
printf "\nsorted array:\n\n"
for ((j = 0; j < cnt; j++)); do
for ((i = 0; i < sz; i += cnt)); do
printf " %s" ${a2[i+j]}
done
printf "\n"
done
exit 0
Output
$ bash sort_cols2.sh dat/matrix.txt
original array:
1 1 1 1 1
9 3 4 5 5
6 7 8 9 7
3 6 8 9 1
3 4 2 1 4
6 4 4 7 7
sorted array:
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7
Awk script
awk '
{for(i=1;i<=NF;i++)a[i]=a[i]" "$i} #Add to column array
END{
for(i=1;i<=NF;i++){
split(a[i],b) #Split column
x=asort(b) #sort column
for(j=1;j<=x;j++){ #loop through sort
d[j]=d[j](d[j]~/./?" ":"")b[j] #Recreate lines
}
}
for(i=1;i<=NR;i++)print d[i] #Print lines
}' file
Output
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7
Here's my entry in this little exercise. Should handle an arbitrary number of columns. I assume they're space-separated:
#!/bin/bash
linenumber=0
while read line; do
i=0
# Create an array for each column.
for number in $line; do
[ $linenumber == 0 ] && eval "array$i=()"
eval "array$i+=($number)"
(( i++ ))
done
(( linenumber++ ))
done <$1
IFS=$'\n'
# Sort each column
for j in $(seq 0 $i ); do
thisarray=array$j
eval array$j='($(sort <<<"${'$thisarray'[*]}"))'
done
# Print each array's 0'th entry, then 1, then 2, etc...
for k in $(seq 0 ${#array0[#]}); do
for j in $(seq 0 $i ); do
eval 'printf ${array'$j'['$k']}" "'
done
echo ""
done
Not bash but i think this python code worths a look showing how this task can be achieved using built-in functions.
From the interpreter:
$ cat matrix.txt
1 1 1 1 1
9 3 4 5 5
6 7 8 9 7
3 6 8 9 1
3 4 2 1 4
6 4 4 7 7
$ python
Python 2.7.3 (default, Jun 19 2012, 17:11:17)
[GCC 4.4.3] on hp-ux11
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> f = open('./matrix.txt')
>>> for row in zip(*[sorted(list(a))
for a in zip(*[a.split() for a in f.readlines()])]):
... print ' '.join(row)
...
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7

Bash array contents get printed in columns

$declare -a inputs=("(1 3 4 8 6 2 7 0 5)" "(2 8 1 0 4 3 7 6 5)"
$ for i in ${inputs[#]}; do echo $i; done;
gives
(1
3
4
8
6
2
7
0
5)
(2
8
1
0
4
3
7
6
5)
I want each array in a row.
Use quotes:
for i in "${inputs[#]}"; do echo "$i"; done;
(1 3 4 8 6 2 7 0 5)
(2 8 1 0 4 3 7 6 5)
You need to use quotes. Say:
for i in "${inputs[#]}"; do echo $i; done
This would return:
(1 3 4 8 6 2 7 0 5)
(2 8 1 0 4 3 7 6 5)
Moreover, remove the ; after done unless it's the last line in your script!

Combine multiple columns of different lengths into one column in BASH

I need to combine columns of different lengths into one column using BASH. Here is an example input file:
11 1 2 3 4 5 6 7 8
12 1 2 3 4 5 6 7 8
13 1 2 3 4 5 6 7 8
14 1 2 5 6 7 8
15 1 2 7 8
And my desired output:
1
1
1
1
1
3
3
3
5
5
5
5
7
7
7
7
7
The input data is pairs of columns as shown. Each pair is separated from another by a fixed number of spaces. Values within a pair of columns are separated by one space. Thanks in advance!
Using GNU awk for fixed width field handling:
$ cat file
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
1 2 5 6 7 8
1 2 7 8
$ cat tst.awk
BEGIN{ FIELDWIDTHS="1 1 1 3 1 1 1 3 1 1 1 3 1 1 1" }
{
for (i=1;i<=NF;i++) {
a[NR,i] = $i
}
}
END {
for (i=1;i<=NF;i+=4)
for (j=1;j<=NR;j++)
if ( a[j,i] != " " )
print a[j,i]
}
$ gawk -f tst.awk file
1
1
1
1
1
3
3
3
5
5
5
5
7
7
7
7
7
You may try the following:
awk -f ext.awk input.txt
where input.txt is your input data file and ext.awk is:
BEGIN {
ncols=4 # number of columns
nspc=3 # number of spaces that separates the columns
}
{
str=$0;
for (i=1; i<=ncols; i++) {
pos=match(str,/^([0-9]+) ([0-9]+)/,a)
if (pos>0) {
b[NR,i]=a[1]
if (NR==1) colw[i]=RLENGTH; #assume col width are given as in first row
}
str=substr(str,colw[i]+1+nspc);
}
}
END {
for (i=1;i<=ncols;i++)
for (j=1;j<=NR;j++) {
if (b[j,i]) print b[j,i];
}
}

how to subtract fields pairwise in bash?

I have a large dataset that looks like this:
5 6 5 6 3 5
2 5 3 7 1 6
4 8 1 8 6 9
1 5 2 9 4 5
For every line, I want to subtract the first field from the second, third from fourth and so on deepening on the number of fields (always even). Then, I want to report those lines for which difference from all the pairs exceeds a certain limit (say 2). I should also be able to report next best lines i.e., lines in which one pairwise comparison fails to meet the limit, but all other pairs meet the limit.
from the above example, if I set a limit to 2 then, my output file should contain
best lines:
2 5 3 7 1 6 # because (5-2), (7-3), (6-1) are all > 2
4 8 1 8 6 9 # because (8-4), (8-1), (9-6) are all > 2
next best line(s)
1 5 2 9 4 5 # because except (5-4), both (5-1) and (9-2) are > 2
My current approach is to read every line, save each field as a variable, do subtraction.
But I don't know how to proceed further.
Thanks,
Prints "best" lines to the file "best", and prints "next best" lines to the file "nextbest"
awk '
{
fail_count=0
for (i=1; i<NF; i+=2){
if ( ($(i+1) - $i) <= threshold )
fail_count++
}
if (fail_count == 0)
print $0 > "best"
else if (fail_count == 1)
print $0 > "nextbest"
}
' threshold=2 inputfile
Pretty straightforward stuff.
Loop through fields 2 at a time.
If (next field - current field) does not exceed threshold, increment fail_count
If that line's fail_count is zero, that means it belongs to "best" lines.
Else if that line's fail_count is one, it belongs to "next best" lines.
Here's a bash-way to do it:
#!/bin/bash
threshold=$1
shift
file="$#"
a=($(cat "$file"))
b=$(( ${#a[#]}/$(cat "$file" | wc -l) ))
for ((r=0; r<${#a[#]}/b; r++)); do
br=$((b*r))
for ((c=0; c<b; c+=2)); do
if [[ $(( ${a[br + c+1]} - ${a[br + c]} )) < $threshold ]]; then
break; fi
if [[ $((c+2)) == $b ]]; then
echo ${a[#]:$br:$b}; fi
done
done
Usage:
$ ./script.sh 2 yourFile.txt
2 5 3 7 1 6
4 8 1 8 6 9
This output can then easily be redirected:
$ ./script.sh 2 yourFile.txt > output.txt
NOTE: this does not work properly if you have those empty lines between each line...But I'm sure the above will get you well on your way.
I probably wouldn't do that in bash. Personally, I'd do it in Python, which is generally good for those small quick-and-dirty scripts.
If you have your data in a text file, you can read here about how to get that data into Python as a list of lines. Then you can use a for-loop to process each line:
threshold = 2
results = []
for line in content:
numbers = [int(n) for n in line.split()] # Split it into a list of numbers
pairs = zip(numbers[::2],numbers[1::2]) # Pair up the numbers two and two.
result = [abs(y - x) for (x,y) in pairs] # Subtract the first number in each pair from the second.
if sum(result) > threshold:
results.append(numbers)
Yet another bash version:
First a check function that return nothing but a result code:
function getLimit() {
local pairs=0 count=0 limit=$1 wantdiff=$2
shift 2
while [ "$1" ] ;do
[ $(( $2-$1 )) -ge $limit ] && : $((count++))
: $((pairs++))
shift 2
done
test $((pairs-count)) -eq $wantdiff
}
than now:
while read line ;do getLimit 2 0 $line && echo $line;done <file
2 5 3 7 1 6
4 8 1 8 6 9
and
while read line ;do getLimit 2 1 $line && echo $line;done <file
1 5 2 9 4 5
If you can use awk
$ cat del1
5 6 5 6 3 5
2 5 3 7 1 6
4 8 1 8 6 9
1 5 2 9 4 5
1 5 2 9 4 5 3 9
$ cat del1 | awk '{
> printf "%s _ ",$0;
> for(i=1; i<=NF; i+=2){
> printf "%d ",($(i+1)-$i)};
> print NF
> }' | awk '{
> upper=0;
> for(i=1; i<=($NF/2); i++){
> if($(NF-i)>threshold) upper++
> };
> printf "%d _ %s\n", upper, $0}' threshold=2 | sort -nr
3 _ 4 8 1 8 6 9 _ 4 7 3 6
3 _ 2 5 3 7 1 6 _ 3 4 5 6
3 _ 1 5 2 9 4 5 3 9 _ 4 7 1 6 8
2 _ 1 5 2 9 4 5 _ 4 7 1 6
0 _ 5 6 5 6 3 5 _ 1 1 2 6
You can process result further according to your needs. The result is sorted by ‘goodness’ order.

Resources