grep last 10 minutes of a log with bad data format - bash

Question

my log has this date format at the beginning of each line:
2018 Sep 21 17:16:27:796
I need to grep the last 10 minutes of this log... any help?
my current experiments:
tenminutesago=$(date --date='10 minutes ago' +"%Y %b %e %H:%M:%S"):999
My idea was to convert the log format to a progressive number and then check everything greater than that number.
I see that the command: date +"%Y %b %e %H:%M:%S" gives a date in the same format of the log. The command: date +"%Y%m%e%H%M%S" gives a date in a progressive number (201810041204019)

Answer 1

You could do
for i in {10..0}; do
d=$(date -d "$i minutes ago" +'%Y %b %e %H:%M')
grep "$d" logfile
done
This just divides the problem in the 11 sequential subtasks of getting all lines from 10 minutes ago, all lines from 9 minutes ago, etc. until the current minute.
Edit:
Here's an alternate solution that prints all lines following the first one where a date stamp from the last 10 minutes was found, not only those that carry a date stamp, and also avoids reading the file over from start several times:
# build a regex pattern that matches any date in the given format from the last 10 minutes
pattern=$(date +'%Y %b %e %H:%M')
for i in {10..1}; do
pattern+=\|$(date -d "$i minutes ago" +'%Y %b %e %H:%M')
done
# print all lines starting from the first one that matches one of the dates in the pattern
awk "/$pattern/,0" logfile

Answer 2

Under the assumption that your loglines looks like
YYYY Bbb dd HH:MM:SS:sss Some random log message is here
You can do the following:
awk -v d=$(date -d "10 minutes ago" "+%Y %m %d %T") '
{ mm = sprintf("%0.2d",(index("JanFebMarAprMayJunJulAugSepOctNovDec",$2)+2)/3)
s = $1 " " mm " " $3 " "$4 }
(s >= d){print}' logfile
The idea is to convert your date format into a Sortable format (Note that "Jan" < "Mar" but "Feb" < "Jan"). This is done by converting your month into a number with two digits and then compare it stringwise against the correct date.

Answer 3

Try your current approach without the seconds and milliseconds.
tenminutesago=$(date --date='10 minutes ago' +"%Y %b %e %H:%M")
Is not exactly the last ten minutes to a second level, but I think it is enough for most of the cases. That will give you the first line in the log within the time window. Now you can get the total lines and subtract the line number of your previous grep, and then tail the file. The script could be like this:
LOGFILE="filename.log"
tenminutesago=$(date --date='9000 minutes ago' +"%Y %b %e %H:%M") # matching pattern
tlines=$(cat $LOGFILE | wc -l) # Total lines in file
let lines=$tlines-$(grep -n "$tenminutesago" $LOGFILE | grep -m 1 -oP "^[0-9]*" || echo $tlines) # lines after matching occurence
echo "$lines lines FOUND from the last X minutes"
tail -n $lines $LOGFILE # last lines in file
As suggested by #Gem Taylor, this could be reduced using +N option in tail.
LOGFILE="filename.log"
tenminutesago=$(date --date='9000 minutes ago' +"%Y %b %e %H:%M") # matching pattern
lines=$(grep -n "$tenminutesago" $LOGFILE | grep -m 1 -oP "^[0-9]*" || echo "0") # lines after matching occurence
echo "$lines lines FOUND from the last X minutes"
let lines -eq 0 && tail -n +$lines $LOGFILE # last lines in file if lines is not 0