I am trying to find the id's of jobs that end in error.
When I use
for i in *log
do tail -n 1 $i | grep error
echo $i
done
It seems to find error on the last line of each file, even for files that don't have errors on the last line, and returns all of the filenames with
STOP fatal_error
out1.log
STOP fatal_error
out2.log
STOP fatal_error
out3.log
....
even though
grep error out1.log
returns nothing
Alternatively, is there an easier way to get a list of the jobs that end in error? I tagged with qsub because I use qsub to submit the jobs
You need an if statement so that you only echo the filename when the grep succeeds:
for i in *.log
do
if tail -n 1 $i | grep error > /dev/null
then
echo $i
fi
done
Also, redirect the grep results to /dev/null so it doesn't appear in the output.
You want to say
do tail -n 1 $i | grep error
not
do tail -n 1 *.log | grep error
Otherwise, you are checking every log file at every iteration and will always get the same results.
Your logic is incorrect.
echo $i will have the listing of the file, not the grep output
Related
I have a shell script that outputs information about successes to stdout, and also does a grep looking for errors in logs
inner.sh:
# do some things
echo success
# do other things
echo success
grep 'error' logs/*
I have another shell script that calls this one, counts up the successes and compares them to an expected number of successes:
outer.sh:
bash ./inner.sh | grep success | wc -l # I compare this number to the expected number
What I can't figure out how to do is have the output of grep go to stderr, so its not counted by the wc -l in outer.sh, but rather makes it around the wc to the terminal for the operator to see.
So I want a command like stdin_to_stderr that I can pipe the grep to, that would make any results it finds leave inner.sh on its stderr.
Is there already such a thing? Or do I just need to write the tiny script that would do this? Or am I thinking about this wrong?
bash ./inner.sh >&2 output_log_file
grep -c success output_log_file -- Count of success
grep -v -c success output_log_file -- Count of not "success"
Example :
echo -e "success.\nerror.\nsuccess.\nError.\nsuccess.\nerror" | grep -c success
echo -e "success.\nerror.\nsuccess.\nError.\nsuccess." | grep -v -c success
output :
3
2
I'm currently monitoring a log file and my ultimate goal is to write a script that uses tail -n0 -f and execute a certain command once grep finds a correspondence. My current code:
tail -n 0 -f $logfile | grep -q $pattern && echo $warning > $anotherlogfile
This works but only once, since grep -q stops when it finds a match. The script must keep searching and running the command, so I can update a status log and run another script to automatically fix the problem. Can you give me a hint?
Thanks
use a while loop
tail -n 0 -f "$logfile" | while read LINE; do
echo "$LINE" | grep -q "$pattern" && echo "$warning" > "$anotherlogfile"
done
awk will let us continue to process lines and take actions when a pattern is found. Something like:
tail -n0 -f "$logfile" | awk -v pattern="$pattern" '$0 ~ pattern {print "WARN" >> "anotherLogFile"}'
If you need to pass in the warning message and path to anotherLogFile you can use more -v flags to awk. Also, you could have awk take the action you want instead. It can run commands via the system() function where you pass the shell command to run
I'm using watch because I need to detect new files created in a log folder and tail them. I can't seem to use tail ie. tail /dir/*.log and have it detect new files created in the folder. So at the moment I'm using
#!/bin/bash
while :
do
watch -n 1 "tail /tmp/tomcat-logs/*.log | grep --line-buffered \"ERROR\|INFO: Server startup in:\|Exception:\" | sed 's/ERROR/PROBLEMO/g' | tee /tmp/errchecker-log.txt"
echo "do some processing here when a token is found"
done
In this case, when a token is found "ERROR" I need to stop watching - then grep the output ( count lines etc.. ) then re-watch until the next error .. rinse repeat
Cheers
Use while read
#!/bin/bash
while read LINE; do
echo $LINE
done < <(for i in $(seq 10); do echo $i; sleep 1; done)
need some help
I was trying to enhance the pre-commit hook to see if I can ignore parsing the commit message if there is a certain key ("!ignore") in the commit message
The if conditions before this one , like to check if there are empty commit messages works. But this if condition somehow is not working.
Ok when I do a commit with message "SMARTCOMMITTEST" which does not contain my check key "!ignore", the commit succeeds which means the If condition below never executed or did not execute as expected. So trying to understand what is wrong wit it.
SMARTCOMMIT=1
$SVNLOOK log -t "$TXN" "$REPOS" | grep "!ignore" | wc -c || SMARTCOMMIT=0
if [ $SMARTCOMMIT = 0];
then
echo "Please use !ignore if you dont want to use smart commits in your commit message." 1>&2
exit 1
fi
Thanks a lot to Etan for some tips...
I changed the condition like the other if condition in the comments and then it worked
SMARTCOMMIT=$($SVNLOOK log -t "$TXN" "$REPOS" | grep "!ignore" | wc -c)
if [ "$SMARTCOMMIT" = "0" ]; then
echo "Please use !ignore if you dont want to use smart commits in your commit message." 1>&2
exit 1
fi
This one worked fine..
#David W .. I now have a situation to check multiple conditions in the same if
SMARTCOMMIT=$($SVNLOOK log -t "$TXN" "$REPOS" | grep '!ignore' | wc -c)
COMMITMESSAGENOREVIEW=$($SVNLOOK log -t "$TXN" "$REPOS" | grep '#comment' | wc -c)
COMMITMESSAGEWITHREVIEW=$($SVNLOOK log -t "$TXN" "$REPOS" | grep '+review' | wc -c)
if [ "$SMARTCOMMIT" = "0" -a "$COMMITMESSAGENOREVIEW" = "0" -a "COMMITMESSAGEWITHREVIEW" = "0" ]; then
echo "Please use #comment or +review to enable smart commits or !ignore to not use smart commits." 1>&2
exit 1
fi
I tried as given in the link here but still I dont see the if condition getting executed at all. Can you help me with this now?
The if statement can look at the exit output of a command, so you don't need to set an environment variable depending upon the output:
shopt extglob > /dev/null && extglob=1
if ! $SVNLOOK log -t "$TXN" "$REPOS" | grep -q '!ignore'
then
echo 'Please use "!ignore" if you dont want to use smart commits in your commit message.' 1>&2
exit 1
fi
This runs $SVNLOOK log -t "$TXN" "$REPOS" | grep -q '!ignore'
The -q is quiet mode. grep either exits zero if the string exists or non-zero if it doesn't exist. By putting this in the if !, the then clause executes only if !ignore was found.
You have to make sure that !ignore is surrounded by single quotes, or you put a \ in front of the !. Bash has a csh type history mechanism in there, and there's no way to turn it off. Anytime the shell sees !, it assumes it has to do with the process ID number.
I need to check if any of the strings "Added/Changed/Fixed/Deleted" are in a commit log message. I am writing an svn precommit hook, and the expected commit comment should have one of these 4 strings in the message.
The code I am using is as below
REPOS=$1
TXN=$2
SVN="/usr/bin/svn";
SVNLOOK="/usr/bin/svnlook";
$SVNLOOK log "$REPOS" -t "$TXN" | \
grep "[a-zA-Z0-9]" > /dev/null
GREP_STATUS=$?
if [ $GREP_STATUS -ne 0 ]
then
"${ECHO}" "No Log comments present" >> "${LOG}"
echo "Your commit has been blocked because you didn't give any log message" 1>&2
echo "Please write a log message describing the purpose of your changes and" 1>&2
echo "then try committing again. -- Thank you" 1>&2
exit 1
fi
In the above code,
$SVNLOOK log "$REPOS" -t "$TXN"
will give me the commit message that the user has entered. Now I have to check for the presence of any of the strings "Added, Changed, Fixed, Deleted" in the message.
That is,
if (any of the above 4 strings are not present),
exit 1
I tried with
$($SVNLOOK log -t "$TXN" "$REPOS" | grep -q "Added\|Changed\|Fixed\|Deleted"|)
but it doesnt seem to be working.
Use grep -e option (multiple times) like this:
grep -e Added -e Changed -e Fixed -e Deleted
otherwise go to the regex route:
grep --regexp=Added|Changed|Fixed|Deleted
To used alternation you need Extended Regexp:
grep -qE 'Added|Changed|Fixed|Deleted'
Or:
egrep -q 'Added|Changed|Fixed|Deleted'
Remove the backslashes and use egrep I also recommend -i for case insensitive matching:
egrep -q -i "added|changed|fixed|deleted"