I would like to create a shell script file such that it gets certain values from a .dat or excel .xls file then pass these values to another file as in the following example;
1-The .dat value has 32x5 matrix size;Example
1 3 4 5 6
4 5 7 9 8
:
:
1 1 1 2 4
2-.geo file has the following
x1= ;
x2= ;
x3= ;
x4= ;
x5= ;
I would like to create 32 geo file with each row of the .dat or .xls file e.g.
1.geo has;
x1 = 1;
x2 = 3;
x3 = 4;
x4 = 5;
x5 = 6;
2.geo has the second row etc.
In summary, shell script loop through the rows and pass them to the geo file and save it with different geo file name.
Any help would be appreciated.
Many thanks,
Your exercise is actually simply handled by reading each line into separate variable, e.g. $a, $b, $c, $d and $e. Once read, you can write them out in into separate $count.geo files (e.g. 1.geo, 2.geo, ...) in the format you desire with printf. A simple implementation would be:
#!/bin/sh
dat=${1:-geo.dat} ## .dat filename ('geo.dat' default)
count=1
test -r "$dat" || { ## validate input file is readable
printf "error: file not readable '%s'\n" "$dat"
exit 1
}
while read -r a b c d e; do ## read each value into a b c d e
printf "x1 = %d;\nx2 = %d;\nx3 = %d;\nx4 = %d;\nx5 = %d;\n" \
$a $b $c $d $e >$count.geo
((count++))
done <"$dat"
Using awk, you could use a one-liner similar to:
awk '{printf "x1 = %d;\nx2 = %d;\nx3 = %d;\nx4 = %d;\nx5 = %d;\n", \
$1, $2, $3, $4, $5 > FNR".geo"}' geo.dat
Example Input File geo.dat
$ cat geo.dat
5 9 4 4 2
8 4 3 8 7
5 8 4 3 7
1 3 9 8 2
5 8 2 2 1
7 1 8 3 4
3 7 4 7 2
5 6 6 9 5
3 4 4 8 6
2 9 9 1 7
2 6 5 4 8
5 1 9 4 3
4 6 1 7 5
2 4 1 7 3
6 2 6 1 9
3 3 9 3 4
6 5 9 2 8
8 7 8 8 2
1 2 9 1 1
4 3 5 5 1
8 2 2 5 3
2 7 5 1 1
9 6 5 9 8
4 9 6 2 8
8 3 1 7 4
2 1 6 7 7
7 5 9 9 9
2 3 7 8 3
9 8 1 5 7
9 9 7 6 1
6 4 5 7 2
8 9 3 6 6
Example Use
$ sh readgoedat.sh geo.dat
or
$ awk '{printf "x1 = %d;\nx2 = %d;\nx3 = %d;\nx4 = %d;\nx5 = %d;\n", \
$1, $2, $3, $4, $5 > FNR".geo"}' geo.dat
Example Output Files
$ cat 1.geo
x1 = 5;
x2 = 9;
x3 = 4;
x4 = 4;
x5 = 2;
$ cat 2.geo
x1 = 8;
x2 = 4;
x3 = 3;
x4 = 8;
x5 = 7;
$ cat 32.geo
x1 = 8;
x2 = 9;
x3 = 3;
x4 = 6;
x5 = 6;
Look things over and let me know if you have further questions.
Related
how to change array into tabular form?
eg:
array = 1 2 3 4 5 6 7 8 9 10 11 12
result
1 10 11
2 9 12
3 8
4 7
5 6
in this particular order up-down-down-up-up-down
the array is taken from a .txt file and its based on user input so the value varies
here is some of my code
declare -a myarray
# Load file into array.
readarray myarray < temp2.txt
s=$myarray
f or i in $(seq 0 $((${#s} - 1))); do
echo "s[$i] = \"${s:$i:1}\""
done
This script does what you want:
#!/bin/bash
a=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 )
rows=5
for (( j=0; j<rows; ++j )); do
for (( i=0; i<=$(( ${#a[#]} / rows )); ++i )); do
if (( i%2 )); then idx=$(( (i + 1) / 2 * 2 * rows - j - 1 ))
else idx=$(( (i / 2) * 2 * rows + j )); fi
printf "%-4s" "${a[idx]}"
done
printf "\n"
done
Output:
1 10 11
2 9 12
3 8 13 18
4 7 14 17
5 6 15 16
To make it work from left to right rather than from top to bottom, you can simply swap the i and j loops around (and change the name rows to cols so that it still makes sense):
#!/bin/bash
a=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 )
cols=5
for (( i=0; i<=$(( ${#a[#]} / cols )); ++i )); do
for (( j=0; j<cols; ++j )); do
if (( i%2 )); then idx=$(( (i + 1) / 2 * 2 * cols - j - 1 ))
else idx=$(( (i / 2) * 2 * cols + j )); fi
printf "%-4s" "${a[idx]}"
done
printf "\n"
done
Output:
1 2 3 4 5
10 9 8 7 6
11 12 13 14 15
18 17 16
declare -a s
s=(1 2 3 4 5 6 7 8 9 10 11 12)
ofstabled=(9 7 5 3 1)
ofstableu=(1 3 5 7 9)
for ((i=0; i<5;++i)); do
for ((j=$i; j<${#s[#]};)); do
printf "%d " ${s[$j]}
let j=$j+${ofstabled[$i]}
if [ $j -lt ${#s[#]} ]; then
printf "%d " ${s[$j]}
let j=$j+${ofstableu[$i]}
fi
done
printf "\n"
done
You can gussy it up if you want but it works.
Script:
#!/bin/bash
a=(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18)
rows=${1:-5}
col=()
for ((i = 0; i < ${#a[#]}; i++)); do
((ind=i % rows))
(( ((i / rows) % 2 == 0) && (ind = ((rows - 1) - ind)) ))
[ -n "${col[(rows - 1) - $ind]}" ] && col[(rows - 1) - $ind]+=" "
col[(rows - 1) - $ind]+=${a[$i]}
done
printf %s\\n "${col[#]}" | column -t
Output:
$ ./order.sh
1 10 11
2 9 12
3 8 13 18
4 7 14 17
5 6 15 16
$ ./order.sh 3
1 6 7 12 13 18
2 5 8 11 14 17
3 4 9 10 15 16
With credit to #TomFenech for the inspiration to make the row count controllable.
You can use awk to read from the file and print it the way you want:
$ awk '{
if(NR<=5) {
a[NR]=$0
next
} else {
row=NR%10
}
}
!(row in a) {
if(row == 0)
row++
else
row=5 - (NR % 5) + 1
}
{
a[row]=a[row] FS $0
}
END {
for(i in a)
print a[i]
}' file
1 10 11
2 9 12
3 8
4 7
5 6
$declare -a inputs=("(1 3 4 8 6 2 7 0 5)" "(2 8 1 0 4 3 7 6 5)"
$ for i in ${inputs[#]}; do echo $i; done;
gives
(1
3
4
8
6
2
7
0
5)
(2
8
1
0
4
3
7
6
5)
I want each array in a row.
Use quotes:
for i in "${inputs[#]}"; do echo "$i"; done;
(1 3 4 8 6 2 7 0 5)
(2 8 1 0 4 3 7 6 5)
You need to use quotes. Say:
for i in "${inputs[#]}"; do echo $i; done
This would return:
(1 3 4 8 6 2 7 0 5)
(2 8 1 0 4 3 7 6 5)
Moreover, remove the ; after done unless it's the last line in your script!
I need to combine columns of different lengths into one column using BASH. Here is an example input file:
11 1 2 3 4 5 6 7 8
12 1 2 3 4 5 6 7 8
13 1 2 3 4 5 6 7 8
14 1 2 5 6 7 8
15 1 2 7 8
And my desired output:
1
1
1
1
1
3
3
3
5
5
5
5
7
7
7
7
7
The input data is pairs of columns as shown. Each pair is separated from another by a fixed number of spaces. Values within a pair of columns are separated by one space. Thanks in advance!
Using GNU awk for fixed width field handling:
$ cat file
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
1 2 5 6 7 8
1 2 7 8
$ cat tst.awk
BEGIN{ FIELDWIDTHS="1 1 1 3 1 1 1 3 1 1 1 3 1 1 1" }
{
for (i=1;i<=NF;i++) {
a[NR,i] = $i
}
}
END {
for (i=1;i<=NF;i+=4)
for (j=1;j<=NR;j++)
if ( a[j,i] != " " )
print a[j,i]
}
$ gawk -f tst.awk file
1
1
1
1
1
3
3
3
5
5
5
5
7
7
7
7
7
You may try the following:
awk -f ext.awk input.txt
where input.txt is your input data file and ext.awk is:
BEGIN {
ncols=4 # number of columns
nspc=3 # number of spaces that separates the columns
}
{
str=$0;
for (i=1; i<=ncols; i++) {
pos=match(str,/^([0-9]+) ([0-9]+)/,a)
if (pos>0) {
b[NR,i]=a[1]
if (NR==1) colw[i]=RLENGTH; #assume col width are given as in first row
}
str=substr(str,colw[i]+1+nspc);
}
}
END {
for (i=1;i<=ncols;i++)
for (j=1;j<=NR;j++) {
if (b[j,i]) print b[j,i];
}
}
Hello stackoverflow users!
Generally I would like to tune up script I am using, just to make it more insensitive to missing data.
My example data looks like this (tab delimited csv file with headers):
ColA ColB ColC
6 0 0
3 5.16551 12.1099
1 10.2288 19.4769
6 20.0249 30.6543
3 30.0499 40.382
1 59.9363 53.2281
2 74.9415 57.1477
2 89.9462 61.3308
6 119.855 64.0319
4 0 0
8 5.06819 46.8086
6 10.0511 60.1357
9 20.0363 71.679
6 30.0228 82.1852
6 59.8738 98.4446
3 74.871 100.648
1 89.9973 102.111
6 119.866 104.148
3 0 0
1 5.07248 51.9168
2 9.92203 77.3546
2 19.9233 93.0228
6 29.9373 98.7797
6 59.8709 100.518
6 74.7751 100.056
3 89.9363 99.5933
1 119.872 100
I use awk script found elsewhere, as follows:
awk 'BEGIN { fn=0 }
NR==1 { next }
NR==2 { delim=$2 }
$2 == delim {
f=sprintf("file_no%02d.txt",fn++);
print "Creating " f
}
{ print $0 > f }'
Which gives me output I want - omit 1st line, find 2nd column and set delimiter - in this example it will be '0':
file_no00.txt
6 0 0
3 5.16551 12.1099
1 10.2288 19.4769
6 20.0249 30.6543
3 30.0499 40.382
1 59.9363 53.2281
2 74.9415 57.1477
2 89.9462 61.3308
6 119.855 64.0319
file_no01.txt
4 0 0
8 5.06819 46.8086
6 10.0511 60.1357
9 20.0363 71.679
6 30.0228 82.1852
6 59.8738 98.4446
3 74.871 100.648
1 89.9973 102.111
6 119.866 104.148
file_no02.txt
3 0 0
1 5.07248 51.9168
2 9.92203 77.3546
2 19.9233 93.0228
6 29.9373 98.7797
6 59.8709 100.518
6 74.7751 100.056
3 89.9363 99.5933
1 119.872 100
To make the script more robust (imagine that rows with 0's are deleted) I would need to split file according to the subtracted value of rows 'n+1' and 'n' if this value is below 0 split file, so basically if (value_row_n+1)-value_row_n < 0 then split file. Of course I would need also to maintain the file naming. Preferred way is bash with awk use. Any advices? Thanks in advance!
Cheers!
Here is awk command that you can use:
cat file
ColA ColB ColC
3 5.16551 12.1099
1 10.2288 19.4769
6 20.0249 30.6543
3 30.0499 40.382
1 59.9363 53.2281
2 74.9415 57.1477
2 89.9462 61.3308
6 119.855 64.0319
8 5.06819 46.8086
6 10.0511 60.1357
9 20.0363 71.679
6 30.0228 82.1852
6 59.8738 98.4446
3 74.871 100.648
1 89.9973 102.111
6 119.866 104.148
1 5.07248 51.9168
2 9.92203 77.3546
2 19.9233 93.0228
6 29.9373 98.7797
6 59.8709 100.518
6 74.7751 100.056
3 89.9363 99.5933
1 119.872 100
awk 'NR == 1 {
next
}
!p || $2 < p {
f = sprintf("file_no%02d.txt",fn++);
print "Creating " f
}
{
p = $2;
print $0 > f
}' file
I suggest small modifications to your current script:
awk 'BEGIN { fn=0; f=sprintf("file_no%02d.txt",fn++); print "Creating " f }
NR==1 { next }
NR==2 { delim=$2 }
$2 - delim < 0 {
f=sprintf("file_no%02d.txt",fn++);
print "Creating " f
}
{ print $0 > f; delim = $2 }' infile
First, create the first file name just before starting the processing.
Second, in last condition save the value of current line to compare with the value of next line.
Third, instead the comparison with zero, do the substraction between previous value and current one to check if result is less than zero.
It yields:
==> file_no00.txt <==
6 0 0
3 5.16551 12.1099
1 10.2288 19.4769
6 20.0249 30.6543
3 30.0499 40.382
1 59.9363 53.2281
2 74.9415 57.1477
2 89.9462 61.3308
6 119.855 64.0319
==> file_no01.txt <==
4 0 0
8 5.06819 46.8086
6 10.0511 60.1357
9 20.0363 71.679
6 30.0228 82.1852
6 59.8738 98.4446
3 74.871 100.648
1 89.9973 102.111
6 119.866 104.148
==> file_no02.txt <==
3 0 0
1 5.07248 51.9168
2 9.92203 77.3546
2 19.9233 93.0228
6 29.9373 98.7797
6 59.8709 100.518
6 74.7751 100.056
3 89.9363 99.5933
1 119.872 100
I have a large dataset that looks like this:
5 6 5 6 3 5
2 5 3 7 1 6
4 8 1 8 6 9
1 5 2 9 4 5
For every line, I want to subtract the first field from the second, third from fourth and so on deepening on the number of fields (always even). Then, I want to report those lines for which difference from all the pairs exceeds a certain limit (say 2). I should also be able to report next best lines i.e., lines in which one pairwise comparison fails to meet the limit, but all other pairs meet the limit.
from the above example, if I set a limit to 2 then, my output file should contain
best lines:
2 5 3 7 1 6 # because (5-2), (7-3), (6-1) are all > 2
4 8 1 8 6 9 # because (8-4), (8-1), (9-6) are all > 2
next best line(s)
1 5 2 9 4 5 # because except (5-4), both (5-1) and (9-2) are > 2
My current approach is to read every line, save each field as a variable, do subtraction.
But I don't know how to proceed further.
Thanks,
Prints "best" lines to the file "best", and prints "next best" lines to the file "nextbest"
awk '
{
fail_count=0
for (i=1; i<NF; i+=2){
if ( ($(i+1) - $i) <= threshold )
fail_count++
}
if (fail_count == 0)
print $0 > "best"
else if (fail_count == 1)
print $0 > "nextbest"
}
' threshold=2 inputfile
Pretty straightforward stuff.
Loop through fields 2 at a time.
If (next field - current field) does not exceed threshold, increment fail_count
If that line's fail_count is zero, that means it belongs to "best" lines.
Else if that line's fail_count is one, it belongs to "next best" lines.
Here's a bash-way to do it:
#!/bin/bash
threshold=$1
shift
file="$#"
a=($(cat "$file"))
b=$(( ${#a[#]}/$(cat "$file" | wc -l) ))
for ((r=0; r<${#a[#]}/b; r++)); do
br=$((b*r))
for ((c=0; c<b; c+=2)); do
if [[ $(( ${a[br + c+1]} - ${a[br + c]} )) < $threshold ]]; then
break; fi
if [[ $((c+2)) == $b ]]; then
echo ${a[#]:$br:$b}; fi
done
done
Usage:
$ ./script.sh 2 yourFile.txt
2 5 3 7 1 6
4 8 1 8 6 9
This output can then easily be redirected:
$ ./script.sh 2 yourFile.txt > output.txt
NOTE: this does not work properly if you have those empty lines between each line...But I'm sure the above will get you well on your way.
I probably wouldn't do that in bash. Personally, I'd do it in Python, which is generally good for those small quick-and-dirty scripts.
If you have your data in a text file, you can read here about how to get that data into Python as a list of lines. Then you can use a for-loop to process each line:
threshold = 2
results = []
for line in content:
numbers = [int(n) for n in line.split()] # Split it into a list of numbers
pairs = zip(numbers[::2],numbers[1::2]) # Pair up the numbers two and two.
result = [abs(y - x) for (x,y) in pairs] # Subtract the first number in each pair from the second.
if sum(result) > threshold:
results.append(numbers)
Yet another bash version:
First a check function that return nothing but a result code:
function getLimit() {
local pairs=0 count=0 limit=$1 wantdiff=$2
shift 2
while [ "$1" ] ;do
[ $(( $2-$1 )) -ge $limit ] && : $((count++))
: $((pairs++))
shift 2
done
test $((pairs-count)) -eq $wantdiff
}
than now:
while read line ;do getLimit 2 0 $line && echo $line;done <file
2 5 3 7 1 6
4 8 1 8 6 9
and
while read line ;do getLimit 2 1 $line && echo $line;done <file
1 5 2 9 4 5
If you can use awk
$ cat del1
5 6 5 6 3 5
2 5 3 7 1 6
4 8 1 8 6 9
1 5 2 9 4 5
1 5 2 9 4 5 3 9
$ cat del1 | awk '{
> printf "%s _ ",$0;
> for(i=1; i<=NF; i+=2){
> printf "%d ",($(i+1)-$i)};
> print NF
> }' | awk '{
> upper=0;
> for(i=1; i<=($NF/2); i++){
> if($(NF-i)>threshold) upper++
> };
> printf "%d _ %s\n", upper, $0}' threshold=2 | sort -nr
3 _ 4 8 1 8 6 9 _ 4 7 3 6
3 _ 2 5 3 7 1 6 _ 3 4 5 6
3 _ 1 5 2 9 4 5 3 9 _ 4 7 1 6 8
2 _ 1 5 2 9 4 5 _ 4 7 1 6
0 _ 5 6 5 6 3 5 _ 1 1 2 6
You can process result further according to your needs. The result is sorted by ‘goodness’ order.