I have the following logs (removed unnecessary info) :
Feb 18 11:38:54 Kingston dhcpd: DHCPACK
Feb 18 11:39:01 duxbury /USR/SBIN/CRON[27892]:
Feb 18 17:39:01 ruby /USR/SBIN/CRON[13080]:
How Can I grep for a server name (kingston, ruby or duxbury) while ensuring that date/time info is next to the server name? so for instance I could grep for kingston, and it would return "Feb 18 11:38:54 Kingston dhcpd: DHCPACK" but if only "some data Kingston" (no date/time info) was available, then nothing would be returned. Thanks for the help!
grep -E "^[a-zA-Z]+ [0-9]+ [0-9]+:[0-9]+:[0-9]+ Kingston"
Related
I need to add the timestamp of all remote servers as part of output and check & compare whether the timestamp is the same or not,
I am able to print the machine IP and date.
#!/bin/bash
all_ip=(192.168.1.121 192.168.1.122 192.168.1.123)
for ip_addr in "${all_ip[#]}"; do
aws_ip=$"ip route get 1 | sed -n 's/^.*src \([0-9.]*\) .*$/\1/p'"
date=date
sshpass -p "password" ssh root#$ip_addr "$aws_ip & $date"
echo "==================================================="
done
Getting Output as :
Wed 27 Jul 2022 05:48:15 AM PDT
192.168.1.121
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.122
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.123
===================================================
How to check whether the timestamp ( ignoring seconds ) of all machines is the same or not ,
eg: (Wed 27 Jul 2022 05:48:15 || Wed 27 Jul 2022 05:48:15 || Wed 27 Jul 2022 05:48:15)
Expected Output:
|| Time are in sync on all machines || # if in sync
|| Time are not in sync on all machines || # if not sync
Wed 27 Jul 2022 05:48:15 AM PDT
192.168.1.121
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.122
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.123
===================================================
How to check whether the time ( ignoring seconds )
tmpdir=$(mktemp -d)
trap 'rm -r "$tmpdir"' EXIT
for ip in "${allips[#]}"; do
# Do N connections, in paralllel, each one writes to a separate file.
sshpass -p "password" ssh root#"$ip" "date +%Y-%m-%d_%H:%M" > "$tmpdir/$ip.txt" &
done
wait
times=$(
for i in "$tmpdir"/*.txt; do
# print filename with file contents.
echo "$i $(<$i)"
done |
# Sort them on second column
sort -k2 |
# Uniq on second field
uniq -f 2
)
echo "$times"
timeslines=$(wc -l <<<"$times")
if ((timeslines == 1)); then
echo "YAY! minutes on all servers the same"
fi
First, you may adjust your "date" command as folow in order to exclude the seconds:
date +%Y-%m-%d_%H:%M
Then, simply grep your output and validate that all the timestamps are identical. You may dump in a temporary file or any other way.
Ex:
grep [aPatternSpecificToTheLinewithTheDate] [yourTemporaryFile] | sort | uniq | wc -l
If the result is 1, it means that all the timestamps are identical.
However you will have to deal with the corner case where the minute shift while you are fetching the time form all your servers.
I wrote a shell script and added it to my cron. It's supposed to run every minute and check for the average server load, past 1 minute, and if it's over 40 it should log the load, date and then restart Apache httpd. Here is my script:
#!/bin/bash
LOGFILE=/home/user/public_html/domain.com/cron/restart.log
function float_to_int() {
echo $1 | cut -d. -f1
}
check=$(uptime | awk -F' *,? *' '{print $12}')
now=$(date)
checkk=$(float_to_int $check)
if [[ $checkk > 40 ]]; then
echo $now $checkk >> $LOGFILE 2>&1
/usr/bin/systemctl restart httpd.service
fi
If I look at the log file I see the following:
Wed Jul 3 20:02:01 EDT 2019 70
Wed Jul 3 23:03:01 EDT 2019 43
Wed Jul 3 23:12:01 EDT 2019 9
Wed Jul 3 23:13:01 EDT 2019 7
Wed Jul 3 23:14:01 EDT 2019 6
Wed Jul 3 23:15:02 EDT 2019 5
Wed Jul 3 23:16:01 EDT 2019 5
Something is clearly wrong as it should only log and restart Apache if the load is over 40 but as you can see from the logs the load was 9, 7, 6, 5 and 5. Could someone point me in the right direction?
From man bash, section CONDITIONAL EXPRESSIONS (emphasis mine) :
string1 > string2
True if string1 sorts after string2 lexicographically.
You will either want to use [['s -gt operator, or use arithmetic evaluation instead of [[ :
if (( chekk > 40 )); then
Here's one in GNU awk (GNU awk due to strftime()):
awk '
$1 > 0.4 { # interval above 0.4
logfile="./log.txt" # my logpath, change it
print strftime("%c"), $1 >> logfile # date and load to log
cmd="/usr/bin/systemctl restart httpd.service" # command to use for restarting
if((ret=(cmd|getline res)) !=0 ) # store return value and result
print "failed: " ret # if failed
else
print "success"
}' /proc/loadavg # getting load avg from /proc
How can I get the number of logins of each day from the beginning of the wtmp file using AWK?
I thought about using an associative array but I don't know how to implement it in AWK..
myscript.sh
#!/bin/bash
awk 'BEGIN{numberoflogins=0}
#code goes here'
The output of the last command:
[fnorbert#localhost Documents]$ last
fnorbert tty2 /dev/tty2 Mon Apr 24 13:25 still logged in
reboot system boot 4.8.6-300.fc25.x Mon Apr 24 16:25 still running
reboot system boot 4.8.6-300.fc25.x Mon Apr 24 13:42 still running
fnorbert tty2 /dev/tty2 Fri Apr 21 16:14 - 21:56 (05:42)
reboot system boot 4.8.6-300.fc25.x Fri Apr 21 19:13 - 21:56 (02:43)
fnorbert tty2 /dev/tty2 Tue Apr 4 08:31 - 10:02 (01:30)
reboot system boot 4.8.6-300.fc25.x Tue Apr 4 10:30 - 10:02 (00:-27)
fnorbert tty2 /dev/tty2 Tue Apr 4 08:14 - 08:26 (00:11)
reboot system boot 4.8.6-300.fc25.x Tue Apr 4 10:13 - 08:26 (-1:-47)
wtmp begins Mon Mar 6 09:39:43 2017
The shell script's output should be:
Apr 4: 4
Apr 21: 2
Apr 24: 3
, using associative array if it's possible
In awk, arrays can be indexed by strings or numbers, so you can use it like an associative array.
However, what you're asking will be hard to do with awk reliably because the delimiters are whitespace, therefore empty fields will throw off the columns, and if you use FIELDWIDTHS you'll also get thrown off by columns longer than their assigned width.
If all you're looking for is just the number of logins per day you might want to use a combination of sed and awk (and sort):
last | \
sed -E 's/^.*(Mon|Tue|Wed|Thu|Fri|Sat|Sun) (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) ([ 0-9]{2}).*$/\2 \3/p;d' | \
awk '{arr[$0]++} END { for (a in arr) print a": " arr[a]}' | \
sort -M
The sed -E uses extended regular expressions, and the pattern just prints the date of each line that is emitted by last (This matches on the day of week, but only prints the Month and Date)
We could have used uniq -c to get the counts, but using awk we can do an associative array as you hinted.
Finally using sort -M we're sorting on the abbreviated date formats like Apr 24, Mar 16, etc.
Try the following awk script(assuming that the month is the same, points to current month):
myscript.awk:
#!/bin/awk -f
{
a[NR]=$0; # saving each line into an array indexed by line number
}
END {
for (i=NR-1;i>1;i--) { # iterating lines in reverse order(except the first/last line)
if (match(a[i],/[A-Z][a-z]{2} ([A-Z][a-z]{2}) *([0-9]{1,2}) [0-9]{2}:[0-9]{2}/, b))
m=b[1]; # saving month name
c[b[2]]++; # accumulating the number of occurrences
}
for (i in c) print m,i": "c[i]
}
Usage:
last | awk -f myscript.awk
The output:
Apr 4: 4
Apr 21: 2
Apr 24: 3
I'm trying to write a shell script that displays unique Names, user name and Date using finger command.
Right now when I enter finger, it displays..
Login Name Tty Idle Login Time Office
1xyz xyz pts/13 Dec 2 18:24 (76.126.34.32)
1xyz xyz pts/13 Dec 2 18:24 (76.126.34.32)
2xxxx xxxx pts/23 2 Dec 2 21:35 (108.252.136.12)
2zzzz zzzz pts/61 13 Dec 2 20:46 (24.4.205.223)
2yyyy yyyy pts/32 57 Dec 2 21:06 (205.154.255.145)
1zzz zzz pts/35 37 Dec 2 20:56 (71.198.36.189)
1zzz zzz pts/48 12 Dec 2 20:56 (71.198.36.189)
I would the script to eliminate the unique values of the username and display it like..
xyz (1xyz) Dec 2 18:24
xxxx (2xxxx) Dec 2 21:35
zzzz (2zzzz) Dec 2 20:46
yyyy (2yyyy) Dec 2 21:06
zzz (1zzz) Dec 2 20:56
the Name is in the first column and the user name is in () and Date is last column
Thanks in Advance!
Ugly but should work.
finger | sed 's/\t/ /' | sed 's/pts\/[0-9]* *[0-9]*//' | awk '{print $2"\t("$1")\t"$3" "$4" "$5}' | sort | uniq
Unique names with sort-u is the easy part.
When you only want to parse the data in your example, you can try matching all strings in one command.
finger | sed 's/^\([^ ]*\) *\([^ ]*\) *pts[^A-Z]*\([^(]*\).*/\2\t(\1)\t\3/'
However, this is hard work and waiting to fail. My finger returns
Login Name Tty Idle Login Time Where
notroot notroot *:0 - Nov 26 15:30 console
notroot notroot pts/0 7d Nov 26 15:30
notroot notroot *pts/1 - Nov 26 15:30
You can try to improve the sed command, good luck with that!
I think the only way is looking at the columns: Read the finger output one line a time and slice each line with ${line:start:len} into parts (and remove spaces afterwards). Have a nice count (and be aware for that_user_with_a_long_name).
I was trying to search for string 'Cannot proceed: the database is empty' in file out.log from bottom to top only (as log file is quite huge and everyday it appends the log at last only) during time-stamps yesterday 10:30 pm to today 00:30 am only.
Extract from out.log is as below:
[Thu Jun 5 07:56:17 2014]Local/data///47480280486528/Info(1019022)
Writing Database Mapping For [data]
[Thu Jun 5 07:56:18 2014]Local/data///47480280486528/Info(1250008)
Setting Outline Paging Cachesize To [8192KB]
[Thu Jun 5 07:56:18 2014]Local/data///47480280486528/Info(1013202)
Cannot proceed: the database is empty
[Thu Jun 5 07:56:20 2014]Local/data///47480280486528/Info(1013205)
Received Command [Load Database]
[Thu Jun 5 07:56:21 2014]Local/data///47480280486528/Info(1019018)
Writing Parameters For Database
I searched on google and SO and explored commands like sed and grep but unfortunately it seems like grep doesn't parse timestamps and sed prints all lines between two patterns.
Can anybody please let me know how I can achieve this ?
You can make use of date comparison in awk:
tac file | awk '/Cannot proceed: the database is empty/ {f=$0; next} f{if (($3==5 && $4>"22:30:00") || ($4==6 && $4<="00:30:00")) {print; print f} f=""}'
Test
For this given file:
$ cat a
[Thu Jun 5 07:56:17 2014]Local/data///47480280486528/Info(1019022)
Writing Database Mapping For [data]
[Thu Jun 5 07:56:18 2014]Local/data///47480280486528/Info(1250008)
Setting Outline Paging Cachesize To [8192KB]
[Thu Jun 5 07:56:18 2014]Local/data///47480280486528/Info(1013202)
Cannot proceed: the database is empty
[Thu Jun 5 07:56:20 2014]Local/data///47480280486528/Info(1013205)
Received Command [Load Database]
[Thu Jun 5 07:56:21 2014]Local/data///47480280486528/Info(1019018)
Writing Parameters For Database
[Thu Jun 5 23:56:20 2014]Local/data///47480280486528/Info(1013205)
Writing Parameters For Database
[Thu Jun 5 23:56:20 2014]Local/data///47480280486528/Info(1013205)
Cannot proceed: the database is empty
[Thu Jun 5 22:56:21 2014]Local/data///47480280486528/Info(1019018)
Cannot proceed: the database is empty
It returns:
$ tac a | awk '/Cannot proceed: the database is empty/ {f=$0; next} f{if (($3==5 && $4>"22:30:00") || ($4==6 && $4<="00:30:00")) {print; print f} f=""}'
[Thu Jun 5 22:56:21 2014]Local/data///47480280486528/Info(1019018)
Cannot proceed: the database is empty
[Thu Jun 5 23:56:20 2014]Local/data///47480280486528/Info(1013205)
Cannot proceed: the database is empty
You can get the line with this:
awk '/Cannot proceed: the database is empty/{ts = last; msg = $0; next}; {last = $0}; END{if (ts) printf "%s\n%s\n", ts, msg}' log
Output:
[Thu Jun 5 07:56:18 2014]Local/data///47480280486528/Info(1013202)
Cannot proceed: the database is empty
It should be easy to refine the code depending on which part is really needed.