Prevent grep returning an error when input doesn't match. I would like it to keep running and not exit with exit code: 1
set -euo pipefail
numbr_match=$(find logs/log_proj | grep "$name" | wc -l);
How could I solve this?
In this individual case, you should probably use
find logs/log_proj -name "*$name*" | wc -l
More generally, you can run grep in a subshell and trap the error.
find logs/log_proj | ( grep "$name" || true) | wc -l
... though of course grep | wc -l is separately an antipattern;
find logs/log_proj | grep -c "$name" || true
I don't know why you are using -e and pipefail when you don't want to have this behaviour. If your goal is just to treat exit code 2 (by grep) as error, but exit code 1 as no-error, you could write a wrapper script around grep, which you
call instead of grep:
#!/bin/bash
# This script behaves exactly like grep, only
# that it returns exit code 0 if there are no
# matching lines
grep "$#"
rc=$?
exit $((rc == 1 ? 0 : rc))
The problem is that tail -f doesn't exits by itself and my script stalls.
I am using sleep 15 because after triggering the shutdown command string 'org.apache.coyote.AbstractProtocol' appears after 15 sec in logs. In this way tail -100 works and gives the desired result.
Below is the snippet of code which I have written and its working fine, but I want to check it dynamically by tail -f. Please guide me how can I do it by tail -f and exiting the tail by some way.
ALFRESCO_LOGS="$CATALINA_BASE/logs"
printf "Stopping alfresco instance.\n\n"
$CATALINA_HOME/bin/shutdown.sh &> /dev/null
sleep 15
CHECK_ALFRESCO_LOGS=$(tail -100 $ALFRESCO_LOGS/catalina.out | grep --line-buffered 'org.apache.coyote.AbstractProtocol' | awk 'NR==1{print $NF}')
echo "$CHECK_ALFRESCO_LOGS"
if [[ "$CHECK_ALFRESCO_LOGS" == "stop" ]];
then
echo "Alfresco instance has been stopped."
fi
If you're OK with tail -f receiving SIGPIPE, this is a straightforward way:
while read CHECK_ALFRESCO_LOGS; do
echo "$CHECK_ALFRESCO_LOGS"
if [[ "$CHECK_ALFRESCO_LOGS" == "stop" ]]; then
echo "Alfresco instance has been stopped."
break
fi
done < <( tail -f $ALFRESCO_LOGS/catalina.out | awk '/org.apache.coyote.AbstractProtocol/{print $NF}')
When the condition matches, the while loop ends. That closes the input redirection, and thus the tail process receives SIGPIPE, which closes it.
until commandThatProducesOutput | grep -m 1 "Done"
do
???
sleep 5
done
While this script is running, I'd like to pipe the output that commandThatProducesOutput produces to the screen but can't seem to get the correct syntax.
How about:
output=$(commandThatProducesOutput)
until echo "$output" | grep -m 1 "Done"
do
echo "$output"
output=$(commandThatProducesOutput)
done
I have a line working from this thread that tails a file until a matching pattern is found. It works well, but I can't find a way to suppress the output that occurs afterwards.
The line is:
sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$ ;}'
piping to /dev/null doesn't work as I don't get any output at all from the tail command that way. Also, I'm on OSX and various other sed and awk suggestions don't work due to the syntax.
It always finishes with the below, instead of nothing:
sh: line 10: 14285 Terminated: 15 sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$ ;}'
I'd also like not to display the matched text (EOF in the above example).
Any suggestions welcomed.
for a file (log for example)
sed -u "/pattern/ q" YourFile
for a pipe
ls -l | sed -u "/pattern/ q"
the -u of sed tell it to work as a stream input
It's the shell's job monitoring popping the message.
nomonitor() {
set +m
"$#"
set -m
}
nomonitor sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$; }'
You can actually discard the stderr like this:
sh -c 'tail -n +0 -f $logfile | { sed "/EOF/q" && p=$$ && kill $((p+1)) ; }'
First off, I'm new to this. I have some experience with windows scripting and apple script but not much with bash. What I'm trying to do is grab the PID and %CPU of a specific process. then compare the %CPU against a set number, and if it's higher, kill the process. I feel like I'm close, but now I'm getting the following error:
[[: 0.0: syntax error: invalid arithmetic operator (error token is ".0")
what am I doing wrong? here's my code so far:
#!/bin/bash
declare -i app_pid
declare -i app_cpu
declare -i cpu_limit
app_name="top"
cpu_limit="50"
app_pid=`ps aux | grep $app_name | grep -v grep | awk {'print $2'}`
app_cpu=`ps aux | grep $app_name | grep -v grep | awk {'print $3'}`
if [[ ! $app_cpu -gt $cpu_limit ]]; then
echo "crap"
else
echo "we're good"
fi
Obviously I'm going to replace the echos in the if/then statement but it's acting as if the statement is true regardless of what the cpu load actually is (I tested this by changing the -gt to -lt and it still echoed "crap"
Thank you for all the help. Oh, and this is on a OS X 10.7 if that is important.
I recommend taking a look at the facilities of ps to avoid multiple horrible things you do.
On my system (ps from procps on linux, GNU awk) I would do this:
ps -C "$app-name" -o pid=,pcpu= |
awk --assign maxcpu="$cpu_limit" '$2>maxcpu {print "crappy pid",$1}'
The problem is that bash can't handle decimals. You can just multiply them by 100 and work with plain integers instead:
#!/bin/bash
declare -i app_pid
declare -i app_cpu
declare -i cpu_limit
app_name="top"
cpu_limit="5000"
app_pid=`ps aux | grep $app_name | grep -v grep | awk {'print $2'}`
app_cpu=`ps aux | grep $app_name | grep -v grep | awk {'print $3*100'}`
if [[ $app_cpu -gt $cpu_limit ]]; then
echo "crap"
else
echo "we're good"
fi
Keep in mind that CPU percentage is a suboptimal measurement of application health. If you have two processes running infinite loops on a single core system, no other application of the same priority will ever go over 33%, even if they're trashing around.
#!/bin/sh
PROCESS="java"
PID=`pgrep $PROCESS | tail -n 1`
CPU=`top -b -p $PID -n 1 | tail -n 1 | awk '{print $9}'`
echo $CPU
I came up with this, using top and bc.
Use it by passing in ex: ./script apache2 50 # max 50%
If there are many PIDs matching your program argument, only one will be calculated, based on how top lists them. I could have extended the script by catching them all and avergaing the percentage or something, but this will have to do.
You can also pass in a number, ./script.sh 12345 50, which will force it to use an exact PID.
#!/bin/bash
# 1: ['command\ name' or PID number(,s)] 2: MAX_CPU_PERCENT
[[ $# -ne 2 ]] && exit 1
PID_NAMES=$1
# get all PIDS as nn,nn,nn
if [[ ! "$PID_NAMES" =~ ^[0-9,]+$ ]] ; then
PIDS=$(pgrep -d ',' -x $PID_NAMES)
else
PIDS=$PID_NAMES
fi
# echo "$PIDS $MAX_CPU"
MAX_CPU="$2"
MAX_CPU="$(echo "($MAX_CPU+0.5)/1" | bc)"
LOOP=1
while [[ $LOOP -eq 1 ]] ; do
sleep 0.3s
# Depending on your 'top' version and OS you might have
# to change head and tail line-numbers
LINE="$(top -b -d 0 -n 1 -p $PIDS | head -n 8 \
| tail -n 1 | sed -r 's/[ ]+/,/g' | \
sed -r 's/^\,|\,$//')"
# If multiple processes in $PIDS, $LINE will only match\
# the most active process
CURR_PID=$(echo "$LINE" | cut -d ',' -f 1)
# calculate cpu limits
CURR_CPU_FLOAT=$(echo "$LINE"| cut -d ',' -f 9)
CURR_CPU=$(echo "($CURR_CPU_FLOAT+0.5)/1" | bc)
echo "PID $CURR_PID: $CURR_CPU""%"
if [[ $CURR_CPU -ge $MAX_CPU ]] ; then
echo "PID $CURR_PID ($PID_NAMES) went over $MAX_CPU""%"
echo "[[ $CURR_CPU""% -ge $MAX_CPU""% ]]"
LOOP=0
break
fi
done
echo "Stopped"
Erik, I used a modified version of your code to create a new script that does something similar. Hope you don't mind it.
A bash script to get the CPU usage by process
usage:
nohup ./check_proc bwengine 70 &
bwegnine is the process name we want to monitor 70 is to log only when the process is using over 70% of the CPU.
Check the logs at: /var/log/check_procs.log
The output should be like:
DATE | TOTAL CPU | CPU USAGE | Process details
Example:
03/12/14 17:11 |20.99|98| ProdPROXY-ProdProxyPA.tra
03/12/14 17:11 |20.99|100| ProdPROXY-ProdProxyPA.tra
Link to the full blog:
http://felipeferreira.net/?p=1453
It is also useful to have app_user information available to test whether the current user has the rights to kill/modify the running process. This information can be obtained along with the needed app_pid and app_cpu by using read eliminating the need for awk or any other 3rd party parser:
read app_user app_pid tmp_cpu stuff <<< \
$( ps aux | grep "$app_name" | grep -v "grep\|defunct\|${0##*/}" )
You can then get your app_cpu * 100 with:
app_cpu=$((${tmp_cpu%.*} * 100))
Note: Including defunct and ${0##*/} in grep -v prevents against multiple processes matching $app_name.
I use top to check some details. It provides a few more details like CPU time.
On Linux this would be:
top -b -n 1 | grep $app_name
On Mac, with its BSD version of top:
top -l 1 | grep $app_name