improve shell script awk & sed - shell

I want to monitor Disk IO of a server and want to set different threshold for
the warning if %busy above 60% and
error if %busy above 80%.
From sar output need to find error disk with more than 80% busy and on 2nd line need disk more than 60 but less than 80% busy if disk not crossing theresholds then it should print 'ok' for respective category
I've created a script which will do this, but is there a better way to run sar just one time and use the same output for error & warning threshold? My system is AIX 6.1.
sar -d 2 2 |sed -n '/Average/,$p'|
sed -e 's/Average//g'|
awk 'NR> 1 {
if ($2 >80) {
print $1, $2; f=1
}
}
END { if (!f) print "ok" }'
sar -d 2 2 |
sed -n '/Average/,$p'|
sed -e 's/Average//g'|
awk 'NR> 1{ if ($2 >60 && $2 <81) { print $1, $2; f=1; } }
END { if (!f) print "ok" }'
Pls find my updated script below
bash-3.00$ cat diskio.sh
#!/bin/bash
a=`sar -d 2 2 |sed -n '/Average/,$ {s/Average//g;p;}'`
e=`$a|awk 'NR> 1{ if ($2 >40) { print $1, $2; f=1; } }
END { if (!f) print "ok" }'`
w=`$a|awk 'NR> 1{ if ($2 <40 && $2 >20) { print $1, $2; f=1; } }
END { if (!f) print "ok" }'`
echo $e
echo $w
bash-3.00$ sh -v diskio.sh
#!/bin/bash
a=`sar -d 2 2 |sed -n '/Average/,$ {s/Average//g;p;}'`
e=`$a|awk 'NR> 1{ if ($2 >40) { print $1, $2; f=1; } }
END { if (!f) print "ok" }'`
diskio.sh[3]: hdisk18: not found.
w=`$a|awk 'NR> 1{ if ($2 <40 && $2 >20) { print $1, $2; f=1; } }
END { if (!f) print "ok" }'`
diskio.sh[4]: hdisk18: not found.
echo $e
ok
echo $w
ok
bash-3.00$ sar -d 2 2 |
sed -n '/Average/,$ {s/Average//g;p;}'|
awk 'NR> 1{ if ($2 >40) { print $1, $2; f=1; } }
END { if (!f) print "ok" }'
hdisk79 135
hdisk101 85
bash-3.00$

sed -n '/Average/,$p'|sed -e 's/Average//g'
could be replaced by
sed -n '/Average/,$ {s/Average//g;p;}'
but sed does not make any evaluation of value (compare) easily so the awk for all is a better solution

This can be shorten some.
sar -d 2 2 | awk '/Average/ {t=1;next} t && NR>1 && $2>80 {print $1,$2;f=1} END { if (!f) print "ok" }'
This should do what you want
sar -d 2 2 | awk '/Average/ {t=1;next} t && NR>1 {if ($2>80) h=1;if ($2>60) l=1} END {if (h) {print "80+";exit};if (l) print "60+"}'
It will print 60+ if its above 60 and 80+ above 80
Or store it to a variable
status=$(sar -d 2 2 | awk '/Average/ {t=1;next} t && NR>1 {if ($2>80) h=1;if ($2>60) l=1} END {if (h) {print "H";exit};if (l) print "L"}')
Here it set status to L for 60+ and H for 80+
This should also print the status:
sar -d 2 2 | awk '/Average/ {t=1;next} t && NR>1 {if ($2>80) h=1;if ($2>60) {print $1,$2;l=1}} END {if (!l) print "ok"}'

Related

Testing grep output

The cmd:
STATUS=`grep word a.log | tail -1 | awk '{print $1,$2,$7,$8,$9}'`
echo "$STATUS"
The output:
2020-05-18 09:27:01 1 of 122
I need to display this $STATUS and need to do the test comparison as well.
How to compare number 122 in below? How to represent 122 in $X?
The number 122 can be any number, resulted from above cmd.
if [ "$X" -gt "300" ]
then
echo "$STATUS. This in HIGH queue ($X)"
else
echo "$STATUS. This is NORMAL ($X)"
fi
You could do it with one awk script:
awk '
/word/{ status=$1" "$2" "$7" "$8" "$9; x=$9 }
END{ printf status". This %s (%s)\n", (x>300 ? "in HIGH queue" : "is NORMAL"), x }
' a.log
I would suggest using lowercase for variables to reduce possible confusion for someone other than the original author reading the script in the future. Also using $() is typically preferable to using back-ticks -- makes quoting easier to get right.
status="$(grep word a.log | tail -1 | awk '{print $1,$2,$7,$8,$9}')"
x="$(printf '%s' "$status" | awk '{ print $NF }')"
if [ "$x" -gt 300 ]
then
echo "$status. This in HIGH queue ($x)"
else
echo "$status. This is NORMAL ($x)"
fi
Note -- we could refactor the status line a bit:
status="$(awk '/word/ { x = $1 OFS $2 OFS $7 OFS $8 OFS $9 } END { print x }' a.log)"

system command for awk

Asked this question earlier, trying to figure out where to place " in system command and one user had suggested to use escape characters as below, but still getting the syntax error. Any leads appreciated!
free -m | awk 'NR==2{
if (($3*100/$2)>=10.00){
printf system("ps aux --sort=-%mem | awk \'NR==2{print $2}\'");
}
else
{
printf "Memory Usage: %s/%sMB (%.2f%%)\n", $3,$2,$3*100/$2 ;
}
}'
As tripleee's comment too, I believe we could use a different mechanism (may be in shell for calculations) and then could write something, calling awk inside awk is not that cool to be honest. Could you please try following once.
free -m | awk -v s1="'" 'NR==2{
if (($3*100/$2)>=10.00){
printf system("ps aux --sort=-%mem | awk " s1 "NR==2{print $2}" s1 );
}
else
{
printf("Memory Usage: %s/%sMB %.2f%%\n", $3,$2,$3*100/$2)
}
}'
EDIT: In case one don't want to use variable for storing ' then use Octal representation of it as follows.
free -m | awk 'NR==2{
if (($3*100/$2)>=10.00){
printf system("ps aux --sort=-%mem | awk \047 NR==2{print $2} \047" );
}
else
{
printf("Memory Usage: %s/%sMB %.2f%%\n", $3,$2,$3*100/$2)
}
}'
Here is a different approach using a single awk with process substitution:
awk 'NR == FNR {
if (FNR == 2)
if ($3*100/$2 > 10)
p=1
else
printf "Memory Usage: %s/%sMB (%.2f%%)\n", $3,$2,$3*100/$2
next
}
p && FNR == 2 {
print $2
exit
}' <(free -m) <(ps aux --sort=-%mem)

How to print the remaining columns using awk?

Right now I have a command that prints my log file with a delimited | per column.
cat ambari-alerts.log | awk -F '[ ]' '{print $1 "|" $2 "|" $3 "|" $4 "|" $5 "|"}' |
grep "$(date +"%Y-%m-%d")"
Sample of the log file data is this:
2016-02-11 09:40:33,875 [OK] [MAPREDUCE2] [mapreduce_history_server_rpc_latency] (History Server RPC Latency) Average Queue Time:[0.0], Average Processing Time:[0.0]
The result of my command is this:
2016-02-11|09:40:33,875|[OK]|[MAPREDUCE2]|[mapreduce_history_server_rpc_latency]
I want to print the remaining columns. How can I do that? I tried this syntax adding $0, but unfortunately it just prints the whole line again.
awk -F '[ ]' '{print $1 "|" $2 "|" $3 "|" $4 "|" $5 "|" $0}'
Hope you can help me, newbie here in using awk.
This seems to be all you need:
$ awk '{for (i=1;i<=5;i++) sub(/ /,"|")} 1' file
2016-02-11|09:40:33,875|[OK]|[MAPREDUCE2]|[mapreduce_history_server_rpc_latency]|(History Server RPC Latency) Average Queue Time:[0.0], Average Processing Time:[0.0]
This is a bit of a hassle with awk
awk -F '[ ]' '{
printf "%s|%s|%s|%s|%s|", $1, $2, $3, $4, $5
for (i=6; i<=NF; i++) printf "%s ", $i
print ""
}'
or, replace the first 5 spaces:
awk -F '[ ]' '{
sub(/ /, "|");sub(/ /, "|");sub(/ /, "|");sub(/ /, "|");sub(/ /, "|")
print
}'
This is actually easier in bash
while IFS=" " read -r a b c d e rest; do
echo "$a|$b|$c|$d|$e|$rest"
done < file.log
Folding in your grep:
awk -F '[ ]' -v date="$(date +%Y-%m-%d)" '{
$0 ~ date {
printf "%s|%s|%s|%s|%s|", $1, $2, $3, $4, $5
for (i=6; i<=NF; i++) printf "%s ", $i
print ""
}
}'
Here is some awk that provides a somewhat more generalized approach than brute-forcing the first 5 columns:
awk '{
for (i = 1; i < 6; i++)
printf "%s|", $i
for (i = 6; i < NF; i++)
printf " %s ", $i
}' ambari-alerts.log | grep "$(date +"%Y-%m-%d")"

search pattern in file and print following lines containing other pattern

i have written awk to print specyfic lines from space-delimited log file.
if [ ! -f $userlog ]
then
awk -F' ' '$7$8 == "Loginitialized" {print $2" "$3" "$4" "$5" process start"}''$7$8$9 == "Applictionsuccessfullyended" {print $2" "$3" "$4" "$5" process end"}' $preplog > $userlog
fi
output sample (userlog)
2015-03-02 13:14:25 (PID 19594) process start
2015-03-02 22:42:29 (PID 30473) process start
2015-03-02 22:53:20 (PID 30473) process end
2015-03-03 07:16:55 (PID 31078) process start
2015-03-03 14:53:15 (PID 16591) process start
2015-03-03 14:54:10 (PID 18292) process start
I need to use same awk on $preplog but from last line i've printed with it.
i was trying with that but i failed:
if [ -f $userlog ]
then
lastpid=`awk -F' ' 'END{print $4}' $userlog` #pid ostatniego procesu
cat $preplog | awk 'BEGIN{ found=0} /PID /$lastpid/ Log initialized/{found=1} {if (found) {awk -F" " "$7$8 == "Loginitialized" {print $2" "$3" "$4" "$5" process start"}''$7$8$9 == "Applictionsuccessfullyended" {print $2" "$3" "$4" "$5" process end"}"} ;}' >> $userlog
fi
but my awk programing is not strong in me. I literaly have no idea how to bite it.
Let's start by cleaning up you current script:
awk -F' ' '$7$8 == "Loginitialized" {print $2" "$3" "$4" "$5" process start"}''$7$8$9 == "Applictionsuccessfullyended" {print $2" "$3" "$4" "$5" process end"}' $preplog > $userlog
Change that to:
awk '
$7$8 == "Loginitialized" { state="start" }
$7$8$9 == "Applictionsuccessfullyended" { state="end" }
state { print $2, $3, $4, $5, "process", state; state="" }
' "$preplog" > "$userlog"
for improved robustness and clarity, and to remove unnecessary constructs and redundancy.
For what you want to do next though - see if reading https://stackoverflow.com/a/18409469/1745001 helps and if not edit your question to show sample input and expected output plus a clearer, more detailed explanation of what you're trying to do.
Hang on, I think I just had a revelation after re-reading your attempted code segment - MAYBE what you want is to only analyze and operate on the last line of the input file instead of every line of it. If so that'd be:
awk '
{ var1=$7$8; var2=$7$8$9 }
END {
if (var1 == "Loginitialized") { state="start" }
if (var2 == "Applictionsuccessfullyended") { state="end" }
if (state) { print $2, $3, $4, $5, "process", state; state="" }
}
' "$preplog" > "$userlog"
If that's not what you wanted you'll need to provide more information.

Awk Bash If-Else Issue

I have a problem of being unable to printout a error message if 0 records are found.
this is what I have as of now.
function search_title
{
awk -F':' -v search="$Title" '$2 ~ search { i++;} END { printf "%d records found\n", i }' test.txt
awk -F':' -v search="$Title" '$2 ~ search { i++; printf "%s, %s,%s,%s,%s\n", $1, $2, $3, $4, $5 } END {}' test.txt
}
function search_author
{
awk -F':' -v search="$Author" '$2 ~ search { i++;} END { printf "%d records found\n", i }' test.txt
awk -F':' -v search="$Author" '$2 ~ search { i++; printf "%s, %s,%s,%s,%s\n", $1, $2, $3, $4, $5 } END {}' test.txt
}
function search_both
{
awk -F':' -v search="$Title" -v search1="$Author" '$1 ~ search && $2 ~ search1 { i++;} END { printf "%d records found\n", i }' test.txt
awk -F':' -v search="$Title" -v search1="$Author" '$1 ~ search && $2 ~ search1 { i++; printf "%s, %s,%s,%s,%s\n", $1, $2, $3, $4, $5 } END {}' test.txt
}
read -p $'Title: ' Title
read -p $'Author: ' Author
if [ "$Title" == "" ];
then
search_author
elif [ "$Author" == "" ];
then
search_title
else
search_both
fi
I need a if else statement to check if the counter is 0 in awk print out "Error! Book does not exist"
For example,
Title input as DAFT
Author input as Linken
(Both value not in test.txt)
"Error! Book does not exist"
instead of the printf now which is "0 Record Found"
You don't need 2 awk command in each function:
You can combine both awk in one command:
awk -F':' -v search="$Title" -v search1="$Author" '$1 ~ search && $2 ~ search1 {
i++;
printf "%s, %s,%s,%s,%s\n", $1, $2, $3, $4, $5;
}
END {
if (!i)
print "Error! Book does not exists!";
else
printf "%d records found\n", i;
}' test.txt

Resources