My need is to print a range of array elements, minus 20 and plus 20 elements from the point "string" is found using KSH93.
I have tried many iterations of code and read many links for example,
How do I iterate over a range of numbers defined by variables in Bash?
/usr/bin/ksh93 -c 'mdm=(`/usr/sbin/mdmprpt 2>/dev/null`);
for index in "${!mdm[#]}"; do
if [[ ${mdm[$index]} =~ Fault.? ]]; then
i=${mdm[$index]};
for x in {1..$i}; do
echo $x
done
fi
done
actual result is
{1..(Faulting}
when it should print 20 lines before and or after of index 52. Ideally both.
__Raw Data__from_sample_code
mdm[32] is 6400000000000000
mdm[33] is 0000000000000000
mdm[34] is 0000000000000000
mdm[35] is 0000000000000000
mdm[36] is 00000000
mdm[37] is Symptom
mdm[38] is Information:
mdm[39] is Crash
mdm[40] is Location:
mdm[41] is [000000000010D614]
mdm[42] is IPRA.$ha_critic+114
mdm[43] is Component:
mdm[44] is COMP
mdm[45] is Exception
mdm[46] is Type:
mdm[47] is 131
mdm[48] is Data
mdm[49] is From
mdm[50] is CPU
mdm[51] is #8
mdm[52] is (Faulting
mdm[53] is CPU)
mdm[54] is backup_files
mdm[55] is cfgbackups
mdm[56] is config
mdm[57] is install.log
mdm[58] is ioscli.log
mdm[59] is pager.trace
mdm[60] is rules
mdm[61] is smit.log
mdm[62] is smit.script
mdm[63] is smit.transaction
mdm[64] is snap.pax.Z
mdm[65] is MST
mdm[66] is State:
mdm[67] is R0:
mdm[68] is 0000000000050FB4
mdm[69] is R1:
mdm[70] is F00000002FF471D0
mdm[71] is R2:
mdm[72] is 00000000038B6110
When you find the matching string you also have its (numerical) index (${index}), so just +/-20 to ${index} to get the desired range.
We'll also need some additional logic to make sure our desired range of indexes falls within the range of available indexes. Keep in mind that for an array with 'n' records the available index range will be '0 to (n-1)'.
for index in "${!mdm[#]}"
do
if [[ ${mdm[$index]} =~ Fault.? ]]
then
start=$((index-20))
end=$((index+20))
# if 'start' is less than 0 then reset it to 0
[ ${start} -lt 0 ] && start=0
for x in $( seq ${start} ${end} )
do
# break if we run out of array elements
[ "${mdm[${x}]:-undefined}" = 'undefined' ] && break
# display our numeric index and contents of associated array item
echo "${x} : ${mdm[${x}]}"
done
break
fi
done
I created a data file with 32 initial lines of 'XXXXXX', the 41 lines of sample data from the question, and an additional dozen lines of 'XXXXXX' at the end of the file; I then ran the above code snippet against the file and generated:
32 : 6400000000000000
33 : 0000000000000000
34 : 0000000000000000
35 : 0000000000000000
36 : 00000000
37 : Symptom
38 : Information:
39 : Crash
40 : Location:
41 : [000000000010D614]
42 : IPRA.$ha_critic+114
43 : Component:
44 : COMP
45 : Exception
46 : Type:
47 : 131
48 : Data
49 : From
50 : CPU
51 : #8
52 : (Faulting
53 : CPU)
54 : backup_files
55 : cfgbackups
56 : config
57 : install.log
58 : ioscli.log
59 : pager.trace
60 : rules
61 : smit.log
62 : smit.script
63 : smit.transaction
64 : snap.pax.Z
65 : MST
66 : State:
67 : R0:
68 : 0000000000050FB4
69 : R1:
70 : F00000002FF471D0
71 : R2:
72 : 00000000038B6110
Well I could just use a bunch of prints, which does do what I want, but Id think there would be a range operator I have yet to get to work.
/usr/bin/ksh93 -c 'mdm=(`/usr/sbin/mdmprpt 2>/dev/null`);
for index in "${!mdm[#]}"; do
if [[ ${mdm[$index]} =~ Fault.? ]]; then
print ${mdm[$index-20]}
print ${mdm[$index-19]}
print ${mdm[$index-18]}
print ${mdm[$index-17]}
print ${mdm[$index-16]}
print ${mdm[$index-15]}
print ${mdm[$index-14]}
print ${mdm[$index-13]}
print ${mdm[$index-12]}
print ${mdm[$index-11]}
print ${mdm[$index-10]}
print ${mdm[$index-9]}
print ${mdm[$index-8]}
print ${mdm[$index-7]}
print ${mdm[$index-6]}
print ${mdm[$index-5]}
print ${mdm[$index-4]}
print ${mdm[$index-3]}
print ${mdm[$index-2]}
print ${mdm[$index-1]}
print ${mdm[$index]}
fi
done'
6400000000000000
0000000000000000
0000000000000000
0000000000000000
00000000
Symptom
Information:
Crash
Location:
[000000000010D614]
IPRA.$ha_critic+114
Component:
COMP
Exception
Type:
131
Data
From
CPU
8
(Faulting
Related
This question already has answers here:
Summing values of a column using awk command
(2 answers)
Closed 1 year ago.
so basically my code looks through data and greps whatever it begins with, and so I've been trying to figure out a way where I'm able to add the those values.
the sample input is
35 45 75 76
34 45 53 55
33 34 32 21
my code:
for id in $(awk '{ print $1 }' < $3); do echo $id; done
I'm printing it right now to see the values but basically whats outputted is
35
34
33
I'm trying to add them all together but I cant figure out how, some help would be appreciated.
my desired output would be
103
Lots of ways to do this, a few ideas ...
$ cat numbers.dat
35 45 75 76
34 45 53 55
33 34 32 21
Tweaking OP's current code:
$ sum=0
$ for id in $(awk '{ print $1 }' < numbers.dat); do ((sum+=id)); done
$ echo "${sum}"
102
Eliminating awk:
$ sum=0
$ while read -r id rest_of_line; do sum=$((sum+id)); done < numbers.dat
$ echo "${sum}"
102
Using just awk (looks like Aivean beat me to it):
$ awk '{sum+=$1} END {print sum}' numbers.dat
102
awk '{ sum += $1 } END { print sum }'
Test:
35 45 75 76
34 45 53 55
33 34 32 21
Result:
102
(sum(35, 34, 33) = 102, that's what you want, right?)
Here is the detailed explanation of how this works:
$1 is the first column of the input.
sum is the variable that holds the sum of all the values in the first column.
END { print sum } is the action to be performed after all the input has been processed.
So the awk program is basically summing up the first column of the input and printing the result.
This answer was partially generated by Davinci Codex model, supervised and verified by me.
I need to read all lines of the file separating at intervals. A function will execute a command with each batch of lines.
Lines range example:
1 - 20
21 - 50
51 - 70
...
I tried with the sed command in a forloop, but the range does not go to the end of the file. For example, a file with 125 lines reads up to 121, missing lines to reach the end.
I commented on the sed line because in this loop the range goes up to 121 and the COUNT is 125.
TEXT=`cat wordlist.txt`
COUNT=$( wc -l <<<$TEXT )
for i in $(seq 1 20 $COUNT);
do
echo "$i"
#sed -n "1","${i}p"<<<$TEXT
done
Output:
1
21
41
61
81
101
121
Thanks!
Quick fix - ensure the last line is processed by throwing $COUNT on the end of of values assigned to i:
for i in $(seq 1 20 $COUNT) $COUNT;
do
echo "$i"
done
1
21
41
61
81
101
121
125
If COUNT happens to be the same as the last value generated by seq then we'll need to add some logic to skip the second time around; for example, if COUNT=121 then we'll want to skip the second time around when i=121, eg:
# assume COUNT=121
lasti=0
for i in $(seq 1 20 $COUNT) $COUNT;
do
[ $lasti = $COUNT ] && break
echo "$i"
lasti=$i
done
1
21
41
61
81
101
121
How do we compare two arrays and display the result in a shell script?
Suppose we have two arrays as below:
list1=( 10 20 30 40 50 60 90 100 101 102 103 104)
list2=( 10 20 30 40 50 60 70 80 90 100 )
My requirement is to compare these two arrays in an order that it will only display the result as (101 102 103 104) from list1. It should not include the values 70 and 80 which are present in list2 but not in list1.
This does not help since it is including everything:
echo "${list1[#]}" "${list2[#]}" | tr ' ' '\n' | sort | uniq -u
I tried something like this below, but why is it not working?
list1=( 10 20 30 40 50 60 70 90 100 101 102 103 104)
list2=( 10 20 30 40 50 60 70 80 90 100 )
for (( i=0; i<${#list1[#]}; i++ )); do
for (( j=0; j<${#list2[#]}; j++ )); do
if [[ ${list1[#]} == ${list2[#] ]]; then
echo 0
break
if [[ ${#list2[#]} == ${#list1[#]-1} && ${list1[#]} != ${list2[#]} ]];then
echo ${list3[$i]}
fi
fi
done
done
You can use comm for this:
readarray -t unique < <( \
comm -23 \
<(printf '%s\n' "${list1[#]}" | sort) \
<(printf '%s\n' "${list2[#]}" | sort) \
)
resulting in
$ declare -p unique
declare -a unique=([0]="101" [1]="102" [2]="103" [3]="104")
or, to get your desired format,
$ printf '(%s)\n' "${unique[*]}"
(101 102 103 104)
comm -23 takes two sorted files (using sort here) and prints every line that is unique to the first one; process substitution is used to feed the lists into comm.
Then, readarray reads the output and puts each line into an element of the unique array. (Notice that this requires Bash.)
Your attempt failed, among other things, because you were trying to compare multiple elements in a single comparison:
[[ ${list1[#]} != ${list2[#]} ]]
expands to
[[ 10 20 30 40 50 60 90 100 101 102 103 104 != 10 20 30 40 50 60 70 80 90 100 ]]
and Bash complains about a binary operator expected instead of the second element, 20.
Could also use this kind of approach
#!/bin/ksh
list1=( 10 20 30 40 50 60 90 100 101 102 103 104 )
list2=( 10 20 30 40 50 60 70 80 90 100 )
# Creating a temp array with index being the same as the values in list1
for i in ${list1[*]}; do
list3[$i]=$i
done
# If value of list2 can be found in list3 forget this value
for j in ${list2[*]}; do
if [[ $j -eq ${list3[$j]} ]]; then
unset list3[$j]
fi
done
# Print the remaining values
print ${list3[*]}
Output is
101 102 103 104
Hope it could help
EDIT
In case of the 2 list are the same :
# Print the remaining values
if [[ ${#list3[*]} -eq 0 ]]; then
print "No differences between the list"
else
print ${list3[*]}
fi
ksh associative arrays are handy for this:
list1=( 10 20 30 40 50 60 90 100 101 102 103 104)
list2=( 10 20 30 40 50 60 70 80 90 100 )
typeset -a onlyList1
typeset -A inList2
for elem in "${list2[#]}"; do inList2["$elem"]=1; done
for elem in "${list1[#]}"; do [[ -v inList2["$elem"] ]] || onlyList1+=("$elem"); done
typeset -p onlyList1
typeset -a onlyList1=(101 102 103 104)
Or similarly, start with all of list1 and remove what's in list2:
typeset -A inList1
for elem in "${list1[#]}"; do inList1["$elem"]=1; done
for elem in "${list2[#]}"; do unset inList1["$elem"]; done
onlyList1=( "${!inList1[#]}" )
I have the following test file
Kmax Event File - Text Format
1 4 1000
65 4121 9426 12312
56 4118 8882 12307
1273 4188 8217 12309
1291 4204 8233 12308
1329 4170 8225 12303
1341 4135 8207 12306
63 4108 8904 12300
60 4106 8897 12307
731 4108 8192 12306
...
ÿÿÿÿÿÿÿÿ
In this file I want to delete the first two lines and apply some mathematical calculations. For instance each column i will be $i-(i-1)*number. A script that does this is the following
#!/bin/bash
if test $1 ; then
if [ -f $1.evnt ] ; then
rm -f $1.dat
sed -n '2p' $1.evnt | (read v1 v2 v3
for filename in $1*.evnt ; do
echo -e "Processing file $filename"
sed '$d' < $filename > $1_tmp
sed -i '/Kmax/d' $1_tmp
sed -i '/^'"$v1"' '"$v2"' /d' $1_tmp
cat $1_tmp >> $1.dat
done
v3=`wc -l $1.dat | awk '{print $1}' `
echo -e "$v1 $v2 $v3" > .$1.dat
rm -f $1_tmp)
else
echo -e "\a!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
echo -e " Event file $1.evnt doesn't exist !!!!!!"
echo -e "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
fi
else
echo -e "\a!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
echo -e "!!!!! Give name for event files !!!!!"
echo -e "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
fi
awk '{print $1, $2-4096, $3-(2*4096), $4-(3*4096)}' $1.dat >$1_Processed.dat
rm -f $1.dat
exit 0
The file won't always have 4 columns. Is there a way to read the number of columns, print this number and apply those calculations?
EDIT The idea is to have an input file (*.evnt), convert it to *.dat or any other ascii file(it doesn't matter really) which will only include the number in columns and then apply the calculation $i=$i-(i-1)*number. In addition it will keep the number of columns in a variable, that will be called in another program. For instance in the above file, number=4096 and a sample output file is the following
65 25 1234 24
56 22 690 19
1273 92 25 21
1291 108 41 20
1329 74 33 15
1341 39 15 18
63 12 712 12
60 10 705 19
731 12 0 18
while in the console I will get the message There are 4 detectors.
Finally a new file_processed.dat will be produced, where file is the initial name of awk's input file.
The way it should be executed is the following
./myscript <filename>
where <filename> is the name without the format. For instance, the files will have the format filename.evnt so it should be executed using
./myscript filename
Let's start with this to see if it's close to what you're trying to do:
$ numdet=$( awk -v num=4096 '
NR>2 && NF>1 {
out = FILENAME "_processed.dat"
for (i=1;i<=NF;i++) {
$i = $i-(i-1)*num
}
nf = NF
print > out
}
END {
printf "There are %d detectors\n", nf | "cat>&2"
print nf
}
' file )
There are 4 detectors
$ cat file_processed.dat
65 25 1234 24
56 22 690 19
1273 92 25 21
1291 108 41 20
1329 74 33 15
1341 39 15 18
63 12 712 12
60 10 705 19
731 12 0 18
$ echo "$numdet"
4
Is that it?
Using awk
awk 'NR<=2{next}{for (i=1;i<=NF;i++) $i=$i-(i-1)*4096}1' file
I have a shell script that contains a loop. This loop is calling another script. The output of each run of the loop is appended inside a file (outOfLoop.tr). when the loop is finished, awk command should calculate the average of specific columns and append the results to another file(fin.tr). At the end, the (fin.tr) is printed.
I managed to get the first part which is appending the results from the loop into (outOfLoop.tr) file. also, my awk commands seem to work... But I'm not getting the final expected output in terms of format. I think I'm missing something. Here is my try:
#!/bin/bash
rm outOfLoop.tr
rm fin.tr
x=1
lmax=4
while [ $x -le $lmax ]
do
calling another script >> outOfLoop.tr
x=$(( $x + 1 ))
done
cat outOfLoop.tr
#/////////////////
#//I'm getting the above part correctly and the output is :
27 194 119 59 178
27 180 100 30 187
27 175 120 59 130
27 189 125 80 145
#////////////////////
#back again to the script
echo "noRun\t A\t B\t C\t D\t E"
echo "----------------------\n"
#// print the total number of runs from the loop
echo "$lmax\t">>fin.tr
#// extract the first column from the output which is 27
awk '{print $1}' outOfLoop.tr >>fin.tr
echo "\t">>fin.tr
#Sum the column---calculate average
awk '{s+=$5;max+=0.5}END{print s/max}' outOfLoop.tr >>fin.tr
echo "\t">>fin.tr
awk '{s+=$4;max+=0.5}END{print s/max}' outOfLoop.tr >>fin.tr
echo "\t">>fin.tr
awk '{s+=$3;max+=0.5}END{print s/max}' outOfLoop.tr >>fin.tr
echo "\t">>fin.tr
awk '{s+=$2;max+=0.5}END{print s/max}' outOfLoop.tr >> fin.tr
echo "-------------------------------------------\n"
cat fin.tr
rm outOfLoop.tr
I want the format to be like :
noRun A B C D E
----------------------------------------------------------
4 27 average average average average
I have incremented max inside the awk command by 0.5 as there was new line between the out put of the results (output of outOfLoop file)
$ cat file
27 194 119 59 178
27 180 100 30 187
27 175 120 59 130
27 189 125 80 145
$ cat tst.awk
NF {
for (i=1;i<=NF;i++) {
sum[i] += $i
}
noRun++
}
END {
fmt="%-10s%-10s%-10s%-10s%-10s%-10s\n"
printf fmt,"noRun","A","B","C","D","E"
printf "----------------------------------------------------------\n"
printf fmt,noRun,$1,sum[2]/noRun,sum[3]/noRun,sum[4]/noRun,sum[5]/noRun
}
$ awk -f tst.awk file
noRun A B C D E
----------------------------------------------------------
4 27 184.5 116 57 160