I am trying to implement a solution for automatic mail sending once it finds a domain that expiration date has been exceeded. I am really new to this, therefore I managed to get as far as code below, that shows expiration dates and sends an email containg the output.
The kind of help I am looking for is at least a clue how to compare expiration date with the current date and get a result as number of days. I will really appreciate any kind of help.
#!/bin/bash
DOM="onet.pl wp.pl"
for d in $DOM
do
echo -n "$d - "
whois $d | egrep -i 'Expiration|Expires on' | head -1
whois $d | egrep -i 'Expiration|Expires on' | head -1 >> /tmp/domain.date
echo ""
done
#[ -f /tmp/domain.date ] && mail -s 'Domain renew / expiration date' myemail#gmail.com < /tmp/domain.date || :
Look no further than the date command, it has everything you need !
Here is a straightforward solution using date -d to parse the date :
# Get the expiration date
expdate="$(whois $d | egrep -i 'Expiration|Expires on' | head -1)"
# Turn it into seconds (easier to compute with)
expdate="$(date -d"$expdate" +%s)"
# Get the current date in seconds
curdate="$(date +%s)"
# Print the difference in days
printf "Number of days to expiration : %s\n" "$(((expdate-curdate)/86400))"
Good luck !
Related
I have this script (which works fine) that will write all the date/time per second, from a start date/time till an end date/time to a file
while read line; do
FIRST_TIMESTAMP="20230109-05:00:01" #this is normally a variable that changes with each $line
LAST_TIMESTAMP="20230112-07:00:00" #this is normally a variable that changes with each $line
date=$FIRST_TIMESTAMP
while [[ $date < $LAST_TIMESTAMP || $date == $LAST_TIMESTAMP ]]; do
date2=$(echo $date |sed 's/ /-/g' |sed "s/^/'/g" |sed "s/$/', /g")
echo "$date2" >> "OUTPUTFOLDER/output_LABELS_$line"
date=$(date -d "$date +1 sec" +"%Y%m%d %H:%M:%S")
done
done < external_file
However this sometimes needs to run 10 times, and the start date/time and end date/time sometimes lies days apart.
Which makes the script take a long time to write all that data.
Now I am wondering if there is a faster way to do this.
Avoid using a separate date call for each date. In the next example I added a safety parameter maxloop, avoiding loosing resources when the dates are wrong.
#!/bin/bash
awkdates() {
maxloop=1000000
awk \
-v startdate="${first_timestamp:0:4} ${first_timestamp:4:2} ${first_timestamp:6:2} ${first_timestamp:9:2} ${first_timestamp:12:2} ${first_timestamp:15:2}" \
-v enddate="${last_timestamp:0:4} ${last_timestamp:4:2} ${last_timestamp:6:2} ${last_timestamp:9:2} ${last_timestamp:12:2} ${last_timestamp:15:2}" \
-v maxloop="${maxloop}" \
'BEGIN {
T1=mktime(startdate);
T2=mktime(enddate);
linenr=1;
while (T1 <= T2) {
printf("%s\n", strftime("%Y%m%d %H:%M:%S",T1));
T1+=1;
if (linenr++ > maxloop) break;
}
}'
}
mkdir -p OUTPUTFOLDER
while IFS= read -r line; do
first_timestamp="20230109-05:00:01" #this is normally a variable that changes with each $line
last_timestamp="20230112-07:00:00" #this is normally a variable that changes with each $line
awkdates >> "OUTPUTFOLDER/output_LABELS_$line"
done < <(printf "%s\n" "line1" "line2")
Using epoch time (+%s and #) with GNU date and GNU seq to
produce datetimes in ISO 8601 date format:
begin=$(date -ud '2023-01-12T00:00:00' +%s)
end=$(date -ud '2023-01-12T00:00:12' +%s)
seq -f "#%.0f" "$begin" 1 "$end" |
date -uf - -Isec
2023-01-12T00:00:00+00:00
2023-01-12T00:00:01+00:00
2023-01-12T00:00:02+00:00
2023-01-12T00:00:03+00:00
2023-01-12T00:00:04+00:00
2023-01-12T00:00:05+00:00
2023-01-12T00:00:06+00:00
2023-01-12T00:00:07+00:00
2023-01-12T00:00:08+00:00
2023-01-12T00:00:09+00:00
2023-01-12T00:00:10+00:00
2023-01-12T00:00:11+00:00
2023-01-12T00:00:12+00:00
if you're using macOS/BSD's date utility instead of the gnu one, the equivalent command to parse would be :
(bsd)date -uj -f '%FT%T' '2023-01-12T23:34:45' +%s
1673566485
...and the reverse process is using -r flag instead of -d, sans "#" prefix :
(bsd)date -uj -r '1673566485' -Iseconds
2023-01-12T23:34:45+00:00
(gnu)date -u -d '#1673566485' -Iseconds
2023-01-12T23:34:45+00:00
I am trying to get the file time from the HTTP Header with the following command.
curl -sLI http://sgp-ping.vultr.com/vultr.com.100MB.bin | grep -i '^Last-Modified' | cut -c16- | date -f- '+%F %T'
Under normal circumstances, it can return query results.
If the query fails due to network problems, it will return null.
I want to return the current system time when the query fails. So I use the shell operator "||", which is used as follows.
curl -sLI http://sgp-ping.vultr.com/xxx.bin | grep -i '^Last-Modified' | cut -c16- | date -f- '+%F %T' || date '+%F %T'
But it doesn't work as expected, what's wrong with it?
Any help, thanks in advance!
If you are writing a script you can deal with it via $? just need to allow for it.
e.g.
#!/bin/bash
doTheCurl() {
curl -v -sLI -o /dev/null "$1" 2>&1 | grep -i '< Last-Modified:'
[ $? -ne 0 ] && date '+%F %T'
}
doTheCurl "http://sgp-ping.vultr.com/vultr.com.100MB.bin"
doTheCurl "http://non-sgp-ping.vultra.com/xxx.bin"
Gives this [I ignored extracting / formatting - but you can see the results are different because they came from different paths.].
< Last-Modified: Wed, 28 Oct 2020 18:06:00 GMT
2021-03-12 16:24:19
The first one fails because the grep doesn't match anything - so it then gives the date.
You can do whatever you like - pipe thru sed or whatever after.
Set Up
I currently have the below script working to download files with curl, using a ref file with multiple variables. When I created the script it suited my needs however as the ref file has gotten larger and the data I am requesting via curl is takes longer to generate, my script is now taking too much time to complete.
Objective
I want to be able to update this script so I have curl request and download multiple files as they are ready - as opposed to waiting for each file to be requested and downloaded sequentially.
I've had a look around and seen that I could use either xargs or parallel to achieve this however based on the past questions I've seen, youtube videos and other forum posts, I have haven't been able to find an example that explains if this is possible using more than one variable.
Can someone confirm if this is possible and which tool is better suited to achieve this? Is my current script in the right configuration or do I need to amend a lot of it to shoe horn these commands in?
I suspect this may be a questions that's been asked previously and I may have just not found the right one.
account-list.tsv
client1 account1 123 platform1 50
client2 account1 234 platform1 66
client3 account1 344 platform1 78
client3 account2 321 platform1 209
client3 account2 321 platform2 342
client4 account1 505 platform1 69
download.sh
#!/bin/bash
set -eu
user="user"
pwd="pwd"
D1=$(date "+%Y-%m-%d" -d "1 days ago")
D2=$(date "+%Y-%m-%d" -d "1 days ago")
curr=$D2
cheese=$(pwd)
curl -o /dev/null -s -S -L -f -c cookiejar 'https://url/auth/' -d name=$user -d passwd=$pwd
while true; do
while IFS=$' ' read -r client account accountid platform platformid
do
curl -o /dev/null -s -S -f -b cookiejar -c cookiejar 'https://url/auth/' -d account=$accountid
curl -sSfL -o "$client€$account#$platform£$curr.xlsx" -J -b cookiejar -c cookiejar "https://url/platform=$platformid&date=$curr"
done < account-list.tsv
[ "$curr" \< "$D1" ] || break
curr=$( date +%Y-%m-%d --date "$curr +1 day" ) ## used in instances where I need to grade data for past date ranges.
done
exit
Using GNU Parallel it looks something like this to fetch 100 entries in parallel:
#!/bin/bash
set -eu
user="user"
pwd="pwd"
D1=$(date "+%Y-%m-%d" -d "1 days ago")
D2=$(date "+%Y-%m-%d" -d "1 days ago")
curr=$D2
cheese=$(pwd)
curl -o /dev/null -s -S -L -f -c cookiejar 'https://url/auth/' -d name=$user -d passwd=$pwd
fetch_one() {
client="$1"
account="$2"
accountid="$3"
platform="$4"
platformid="$5"
curl -o /dev/null -s -S -f -b cookiejar -c cookiejar 'https://url/auth/' -d account=$accountid
curl -sSfL -o "$client€$account#$platform£$curr.xlsx" -J -b cookiejar -c cookiejar "https://url/platform=$platformid&date=$curr"
}
export -f fetch_one
while true; do
cat account-list.tsv | parallel -j100 --colsep '\t' fetch_one
[ "$curr" \< "$D1" ] || break
curr=$( date +%Y-%m-%d --date "$curr +1 day" ) ## used in instances where I need to grade data for past date ranges.
done
exit
One (relatively) easy way to run several processes in parallel is to wrap the guts of the call in a function and then call the function inside the while loop, making sure to put the function call in the background, eg:
# function definition
docurl () {
curl -o /dev/null -s -S -f -b cookiejar -c cookiejar 'https://url/auth/' -d account=$accountid
curl -sSfL -o "$client€$account#$platform£$curr.xlsx" -J -b cookiejar -c cookiejar "https://url/platform=$platformid&date=$curr"
}
# call the function within OP's inner while loop
while true; do
while IFS=$' ' read -r client account accountid platform platformid
do
docurl & # put the function call in the background so we can continue loop processing while the function call is running
done < account-list.tsv
wait # wait for all background calls to complete
[ "$curr" \< "$D1" ] || break
curr=$( date +%Y-%m-%d --date "$curr +1 day" ) ## used in instances where I need to grade data for past date ranges.
done
One issue with this approach is that for a large volume of curl calls it may be possible to bog down the underlying system and/or cause the remote system to reject 'too many' concurrent calls. In this case it'll be necessary to limit the number of concurrent curl calls.
One idea would be to keep a counter of the number of currently running (backgrounded) curl calls and when we hit a limit we wait for a background process to complete before spawning a new one, eg:
max=5 # limit of 5 concurrent/backgrounded calls
ctr=0
while true; do
while IFS=$' ' read -r client account accountid platform platformid
do
docurl &
ctr=$((ctr+1))
if [[ "${ctr}" -ge "${max}" ]]
then
wait -n # wait for a background process to complete
ctr=$((ctr-1))
fi
done < account-list.tsv
wait # wait for last ${ctr} background calls to complete
[ "$curr" \< "$D1" ] || break
curr=$( date +%Y-%m-%d --date "$curr +1 day" ) ## used in instances where I need to grade data for past date ranges.
done
I'm trying to find a pattern "INFO: Server startup in" for last 5 mins in a log file.
Here is the line from which I'm trying to find the pattern: "INFO | jvm 1 | main | 2018/07/09 00:11:29.077 | INFO: Server startup in 221008 ms"
The pattern is coming, but I need to shorten the code or create a loop for it.
I tried to create a loop, but it is not working. Here is my code without loops, which is working:
#!/bin/bash
#Written by Ashutosh
#We will declare variables with date and time of last 5 mins.
touch /tmp/a.txt;
ldt=$(date +"%Y%m%d");
cdt=$(date +"%Y/%m/%d %H:%M");
odtm5=$(date +"%Y/%m/%d %H:%M" --date "-5 min");
odtm4=$(date +"%Y/%m/%d %H:%M" --date "-4 min");
odtm3=$(date +"%Y/%m/%d %H:%M" --date "-3 min");
odtm2=$(date +"%Y/%m/%d %H:%M" --date "-2 min");
odtm1=$(date +"%Y/%m/%d %H:%M" --date "-1 min");
## Finding the pattern and storing it in a file
grep -e "$odtm1" -e "$cdt" -e "$odtm2" -e "$odtm3" -e "$odtm4" -e
"$odtm5" /some/log/path/console-$ldt.log
> /tmp/a.txt;
out=$(grep 'INFO: Server startup in' /tmp/a.txt);
echo "$out"
## remove the file that contains the pattern
rm /tmp/a.txt;
I have tried to use sed also, but date function is not working with it.
Can someone please give me the new changed script with loops?
Adopting your original logic:
time_re='('
for ((count=5; count>0; count--)); do
time_re+="$(date +'%Y/%m/%d %H:%M' --date "-$count min")|"
done
time_re+="$(date +'%Y/%m/%d %H:%M'))"
ldt=$(date +'%Y%m%d')
awk -v time_re="$time_re" '
$0 ~ time_re && /INFO: Server startup in/ { print $0 }
' "/some/log/path/console-$ldt.log"
Performance enhancements are certainly possible -- this could be made much faster by bisecting the log for the start time -- but the above addresses the explicit question (about using a loop to generate the time window). Note that it will get unwieldy -- you wouldn't want to use this to search for the last day, for example, as the regex would become utterly unreasonable.
Sounds like all you need is:
awk -v start="$(date +'%Y/%m/%d %H:%M' --date '-5 min')" -F'[[:space:]]*[|][[:space:]]*' '
($4>=start) && /INFO: Server startup in/
' file
No explicit loops or multiple calls to date required.
Here is a bash script that does the job (thanks to Charles for its improvement):
#!/bin/bash
limit=$(date -d '5 minutes ago' +%s)
today_logs="/some/log/path/console-$(date +'%Y%m%d').log"
yesterday_logs="/some/log/path/console-$(date +'%Y%m%d' -d yesterday).log"
tac "$today_logs" "$yesterday_logs" \
| while IFS='|' read -r prio jvm app date log; do
[ $(date -d "$date" +%s) -lt "$limit" ] && break
echo "|$jvm|$prio|$app|$date|$log"
done \
| grep -F 'INFO: Server startup in' \
| tac
It has the following advantages over your original script:
optimized: it parses log lines starting from the more recent ones and stops at the first line encountered that is more than 5 min old. At 23:59, no need to parse log lines from 0:00 to 23:53
arbitrary time window: you can replace "5 minutes" with "18 hours" and it will still work. A time window of more than one day needs adaptation since each day has it own log file
works correctly when day changes: at 0:00 the original script will never parse the log lines from 23:55:00 to 23:59:59
Mixing the above code with Ed Morton's answer, you get:
#!/bin/bash
limit=$(date -d '5 minutes ago' +'%Y/%m/%d %H:%M')
today_logs="/some/log/path/console-$(date +'%Y%m%d').log"
yesterday_logs="/some/log/path/console-$(date +'%Y%m%d' -d yesterday).log"
tac "$today_logs" "$yesterday_logs" \
| awk -v stop="$limit" -F'[[:space:]]*[|][[:space:]]*' '
($4 < stop) { exit }
/INFO: Server startup in/
' \
| tac
I need get te next saturday from starting the current date for create a file. For example, on linux:
date +%Y%m%d -d saturday
This return 20160430, the next saturday with current date 4/25/2016. That's I want.
I need the same result (or something like that) on Unix (SunOS 5.10). If I execute the same command, the return is the current date 20160425
There's another way to do that, for example using command cal? I tried but just return the first Saturday. The command:
cal | grep -v 2016 |grep -v Th | head -3 | head -1 | sed 's|^.*\([0-9]\)$|\1|g'
So, there's a way to get the next Saturday, starting from current date, using cal command?
Thanks for help!
You can obtain the day using following command:
cal -s | tail -n +2 | grep `date +%d` | cut -d' ' -f 7
Prints calendar of current month, starting weeks with Sunday, strips the year line, greps the line with today's date and gets its 7-th column, which is your Saturday month day number.
Now here is a little script that calculates the rest.
#!/bin/sh
TODAY=$(date +%d)
DAY=$(cal -s | tail -n +2 | grep ${TODAY} | cut -d' ' -f 7)
MONTH=$(date +%m)
YEAR=$(date +%Y)
if [[ $DAY -lt $TODAY ]]; then
# Saturday is next month
let MONTH=MONTH+1
if [[ $MONTH -eq 13 ]]; then
# Saturday is next year
let YEAR=YEAR+1
fi
fi
echo $YEAR$MONTH$DAY