I want to know the minute of the day in a bash shell script, and at the moment I can only think to do this by piping two date commands with the hour of day and minute of hour to bc in this way:
mod=$(echo 60*$(date +%H)+$(date +%M) | bc)
echo $mod
This works, but is very clunky and not very elegant, is there a nicer way? I didn't see an option of minute of day in the date command.
I'm using bash version
4.4.20(1)-release
As a more efficient approach (with bash 4.4 or later), albeit not necessarily a shorter one:
printf -v dateMath '%( (10#%H*60)+10#%M )T' -1
mod=$(( dateMath ))
...as a less-efficient one-liner (but still much faster than using date and bc), you could also write this as:
mod=$(( $(printf '%( (10#%H*60)+10#%M )T' -1) ))
Related
I have two dates in forms like: YYYYMMDDHH and want to calculate the differences (in hours) between these two dates. For example
start_date=1996010100
end_date=1996010122
which stands for two dates: 1996-01-01 00:00:00 and 1996-01-01 22:00:00. I want to use date to calculate the difference in hours, the result shall be 22 hours. I tried with
START=$(date -d "$start_date" +"%s")
END=$(date -d "$end_date" +"%s")
HOURS=$(bc -l <<< "($END - $START) / 3600")
but it failed...
So how can I do this? Thanks!
For performance reasons we want to limit the number of sub-process calls we need to invoke:
use bash substring functionality to convert inputs into usable date/time strings
use bash math to replace bc call
bash substring functionality to break the inputs into a usable date/time format, eg:
# convert to usable date/time format:
$ start_date=1996010100
$ echo "${start_date:0:4}-${start_date:4:2}-${start_date:6:2} ${start_date:8:2}:00:00"
1996-01-01 00:00:00
# convert to epoch/seconds:
$ start=$(date -d "${start_date:0:4}-${start_date:4:2}-${start_date:6:2} ${start_date:8:2}:00:00" +"%s")
$ echo $start
820476000
Applying to ${end_date} and using bash math:
$ end_date=1996010122
$ end=$(date -d "${end_date:0:4}-${end_date:4:2}-${end_date:6:2} ${end_date:8:2}:00:00" +"%s")
$ echo $end
820555200
$ hours=$(( (end - start) / 3600))
$ echo $hours
22
This leaves us with 2 sub-process calls ($(date ...)). While other languages/tools (awk, perl, etc) can likely speed this up a bit, if you need to store the result in a bash variable then you're looking at needing at least 1 sub-process call (ie, hours=$(awk/perl/??? ...)).
If performance is really important (eg, needing to perform this 1000's of times) take a look at this SO answer that uses a fifo, background date process and io redirection ... yeah, a bit more coding and a bit more convoluted but also a bit faster for large volumes of operations.
busybox date can do the trick
start_date=1996010100
end_date=1996010122
START=$(busybox date -D "%Y%m%d%H" -d "$start_date" +"%s")
END=$(busybox date -D "%Y%m%d%H" -d "$end_date" +"%s")
HOURS=$(bc -l <<< "scale=0;($END - $START) / 3600")
echo $HOURS
If it's possible for you to use a more fully-featured scripting language like Python, it'll provide a much more pleasant and understandable date parsing experience, and is probably installed by default (datetime is also a standard Python library)
Structured with shell vars
start_date=1996010100
end_date=1996010122
python -c "import datetime ; td = datetime.datetime.strptime('${end_date}', '%Y%m%d%H') - datetime.datetime.strptime('${start_date}', '%Y%m%d%H') ; print(int(td.total_seconds() / 3600))"
Structured to read dates and format code from stdin
echo '%Y%m%d%H' 1996010100 1996010122 | python -c "import datetime,sys ; fmt, date_start, date_end = sys.stdin.read().strip().split() ; td = datetime.datetime.strptime(date_end, fmt) - datetime.datetime.strptime(date_start, fmt) ; print(int(td.total_seconds() / 3600))"
Should work with both Python 3 and Python 2.7
format codes available here (1989 C standard)
https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
which stands for two dates: 1996-01-01 00:00:00
So convert it to that form if it stands for it.
start_date=1996010100
start_date=$(sed -E 's/(....)(..)(..)(..)/\1-\2-\3 \4:00:00/' <<<"$start_date")
start=$(date -d "$start_date" +"%s")
and the same with end.
the most simple way is to install "dateutils" using this command
sudo apt-get install dateutils
Run these commands to get the difference in seconds:
dateutils.ddiff -i '%Y%m%d%H%M%S' 20200817040001 20210817040101
output:
31536060s
next step: Simply divide by 86400 to get the number of days or similarly for hours and minutes :)
I need to do date calculations in a shell script. Unfortunately my operating system (SunOS) does not provide a very handy date function: it does not support the -d option which is exactly what I need.
Roaming on the web to find an alternate solution i found something that looks to be powerful enough with ksh93 printf builtin function. It supports syntaxes like that:
yesterday=${ printf "%(%Y%m%d)T" yesterday; }
friday=${ printf "%(%Y%m%d)T" "3 days ago"; }
In my case, i need to calculate 2 days before a calculated date, in my understanding it should be written like that (mydate is formatted as "YYYYMMDD"):
dayinthepast=${ printf "%(%Y%m%d)T" "3 days before $mydate"; }
or
dayinthepast=${ printf "%(%Y%m%d)T" "$mydate - 3 days"; }
But it does not work.
The more surprising is that the second syntax is recognized but it takes the minus as a plus and add 3 days to the given date.
I have read a useful blog covering the ksh93 printf builtin syntax but it does not cover my case. I'm giving the link here (thanks for the author): ksh93-date-manipulation
Any help would be appreciated.
Thanks
this worked for me on AIX 7.1:
$ ksh93
$ MY_DATE=20210601
$ printf "%(%Y%m%d)T" "${MY_DATE} 3 days ago"
20210529
got the hint from the link - ksh93-date-manipulation - there's a paragraph there that mentions "23 days ago"
I'm currently creating a shell script that will run a python code once an hour that collects, processes, and displays data from a radar for the previous hour.
The python code I am using requires a UTC begin time and end time in format "YYYYMMDDHHmm". So far, I have found using the unix command date -u +"%Y%m%d%H%M" will retrieve my current time in the correct format, but I have not been able to find a way to subtract 60 minutes from this first time and have it output the "start" time/
code I have tried:
date +"%Y%m%d%H%M-60" >> out: 201908201833-60
now= date -u +"%Y%m%d%H%M" >> out:201908201834
echo "$now - 60" >> out: - 60
I'm just starting to self teach/learn shell coding and I am more comfortable with python coding which is why my attempts are set up more like how you would write with python. I'm sure there is a way to store the variable and have it subtract 60 from the end time, but I have not been able to find a good online source for this (both on here and via Google).
You can use -d option in date:
date -u +"%Y%m%d%H%M" -d '-60 minutes'
or else subtract 1 hour instead of 60 minutes:
date -u +"%Y%m%d%H%M" -d '-1 hour'
To be able to capture this value in a variable, use command substitution:
now=$(date -u +"%Y%m%d%H%M" -d '-1 hour')
On OSX (BSD) use this date command as -d is not supported:
now=$(date -u -v-1H +"%Y%m%d%H%M")
Your current attempt has some simple shell script errors.
now= date -u +"%Y%m%d%H%M" >> out:201908201834
This assigns an empty string to the variable now and then runs the date command as previously. If the plan is to capture the output to the variable now, the syntax for that is
now=$(date -u +"%Y%m%d%H%M")
Next up, you try to
echo "$now - 60"
which of course will output the literal string
201908201834 - 60
rather than perform arithmetic evaluation. You can say
echo "$((now - 60))"
to subtract 60 from the value and echo that -- but of course, date arithmetic isn't that simple; subtracting 60 from 201908210012 will not produce 201908202312 like you would hope.
If you have GNU date (that's a big if if you really want to target any Unix) you could simply have done
date -u -d "60 minutes ago" +%F%H%M
but if you are doing this from Python anyway, performing the date extraction and manipulation in Python too will be a lot more efficient as well as more portable.
from datetime import datetime, timedelta
dt = datetime.strptime(when,'%Y%m%d%H%M')
print(dt - timedelta(minutes=60))
The shell command substitution $(command) and arithmetic evaluation $((expression)) syntaxes look vaguely similar, but are really unrelated. Both of them have been introduced after the fundamental shell syntax was already stable, so they had to find a way to introduce new syntax which didn't already have a well-established meaning in the original Bourne shell.
I am trying to get the time difference between two dates as given below in Bash script. However I am not successful
head_info: 05-31-2017:04:27:37
tail_info: 05-31-2017:04:30:57
the problem is that after Reformation above time and while trying to calculate in seconds due to space, it is ignoring time.
This is my script:
fm_head_info=(${head_info:6:4}"-"${head_info:0:2}"-"${head_info:3:2}" \
"${head_info:11:8})
fm_tail_info=(${tail_info:6:4}"-"${tail_info:0:2}"-"${tail_info:3:2}" \
"${tail_info:11:8})
$ fm_head_info
-bash: 2017-05-31: command not found
Thank you
Let's define your shell variables:
$ tail_info=05-31-2017:04:30:57
$ head_info=05-31-2017:04:27:37
Now, let's create a function to convert those dates to seconds-since-epoch:
$ date2sec() { date -d "$(sed 's|-|/|g; s|:| |' <<<"$*")" +%s; }
To find the time difference between those two date in seconds:
$ echo $(( $(date2sec "$tail_info") - $(date2sec "$head_info") ))
200
As written above, this requires bash (or similar advanced shell) and GNU date. In other words, this should work on any standard Linux. To make this work on OSX, some changes to the date command will likely be necessary.
How it works
Starting with the innermost command inside the function date2sec, we have:
sed 's|-|/|g; s|:| |' <<<"$*"
In the argumnet to the function, this replaces all - with / and it replaces the first : with a space. This converts the the dates from the format in your input to one that the GNU date function will understand. For example:
$ sed 's|-|/|g; s|:| |' <<<"05-31-2017:04:30:57"
05/31/2017 04:30:57
With this form, we can use date to find seconds-since-epoch:
$ date -d "05/31/2017 04:30:57" +%s
1496230257
And, for the head_info:
$ date -d "05/31/2017 04:27:37" +%s
1496230057
Now that we have that, all that is left is to subtract the times:
$ echo $(( 1496230257 - 1496230057 ))
200
Your immediate issue is the inclusion of erroneous (...) surrounding your string indexed assignment and your questionable quoting. It looks like you intended:
fm_head_info="${head_info:6:4}-${head_info:0:2}-${head_info:3:2} ${head_info:11:8}"
fm_tail_info="${tail_info:6:4}-${tail_info:0:2}-${tail_info:3:2} ${tail_info:11:8}"
Your use of string indexes is correct, e.g.
#!/bin/bash
head_info=05-31-2017:04:27:37
tail_info=05-31-2017:04:30:57
fm_head_info="${head_info:6:4}-${head_info:0:2}-${head_info:3:2} ${head_info:11:8}"
fm_tail_info="${tail_info:6:4}-${tail_info:0:2}-${tail_info:3:2} ${tail_info:11:8}"
echo "fm_head_info: $fm_head_info"
echo "fm_tail_info: $fm_tail_info"
Example Use/Output
$ bash headinfo.sh
fm_head_info: 2017-05-31 04:27:37
fm_tail_info: 2017-05-31 04:30:57
You can then do something similar with the differences in date -d "$var" +%s as John shows in his answer to arrive at the time difference. Note, string indexes are limited to bash, while a sed solution (absent the herestring) would be portable on all POSIX shells.
I have a list of times that I am looping through in the format HH:MM:SS to find the nearest but not past time. The code that I have is:
for i in ${times[#]}; do
hours=$(echo $i | sed 's/\([0-9]*\):.*/\1/g')
minutes=$(echo $i | sed 's/.*:\([0-9]*\):.*/\1/g')
currentHours=$(date +"%H")
currentMinutes=$(date +"%M")
if [[ hours -ge currentHours ]]; then
if [[ minutes -ge currentMinutes ]]; then
break
fi
fi
done
The variable times is an array of all the times that I am sorting through (its about 20-40 lines). I'd expect this to take less than 1 second however it is taking upwards of 5 seconds. Any suggestions for decreasing the time of the regular expression would be appreciated.
times=($(cat file.txt))
Here is a list of the times that are stored in a text file and are imported into the times variable using the above line of code.
6:05:00
6:35:00
7:05:00
7:36:00
8:08:00
8:40:00
9:10:00
9:40:00
10:11:00
10:41:00
11:11:00
11:41:00
12:11:00
12:41:00
13:11:00
13:41:00
14:11:00
14:41:00
15:11:00
15:41:00
15:56:00
16:11:00
16:26:00
16:41:00
16:58:00
17:11:00
17:26:00
17:41:00
18:11:00
18:41:00
19:10:00
19:40:00
20:10:00
20:40:00
21:15:00
21:45:00
One of the key things to understand in looking at bash scripts from a performance perspective is that while the bash interpreter is somewhat slow, the act of spawning an external process is extremely slow. Thus, while it can often speed up your scripts to use a single invocation of awk or sed to process a large stream of input, starting those invocations inside a tight loop will greatly outweigh the performance of those tools once they're running.
Any command substitution -- $() -- causes a second copy of the interpreter to be fork()ed off as a subshell. Invoking any command not built into bash -- date, sed, etc -- then causes a subprocess to be fork()ed off for that process, and then the executable associated with that process to be exec()'d -- something involves a great deal of OS-level overhead (the binary needs to be linked, loaded, etc).
This loop would be better written as:
IFS=: read -r currentHours currentMinutes < <(date +"%H:%M")
while IFS=: read -r hours minutes _; do
if (( hours >= currentHours )) && (( minutes >= currentMinutes )); then
break
fi
done <file.txt
In this form only one external command is run, date +"%H:%M", outside the loop. If you were only targeting bash 4.2 and newer (with built-in time formatting support), even this would be unnecessary:
printf -v currentHours '%(%H)T' -1
printf -v currentMinutes '%(%M)T' -1
...will directly place the current hour and minute into the variables currentHours and currentMinutes using only functionality built into modern bash releases.
See:
BashFAQ #1 - How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
BashFAQ #100 - How can I do native string manipulations in bash? (Subsection: "Splitting a string into fields")
To be honest I'm not sure why it's taking an extremely long time but there are certainly some things which could be made more efficient.
currentHours=$(date +"%H")
currentMinutes=$(date +"%M")
for time in "${times[#]}"; do
IFS=: read -r hours minutes seconds <<<"$time"
if [[ hours -ge currentHours && minutes -ge currentMinutes ]]; then
break
fi
done
This uses read, a built-in command, to split the text into variables, rather than calling external commands and creating subshells.
I assume that you want the script to run so quickly that it's safe to reuse currentHours and currentMinutes within the loop.
Note that you can also just use awk to do the whole thing:
awk -F: -v currentHours="$(date +"%H") -v currentMinutes="$(date +"%M")" '
$1 >= currentHours && $2 >= currentMinutes { print; exit }' file.txt
Just to make the program produce some output, I added a print, so that the last line is printed.
awk to the rescue!
awk -v time="12:12:00" '
function pad(x) {split(x,ax,":"); return (ax[1]<10)?"0"x:x}
BEGIN {time=pad(time)}
time>pad($0) {next}
{print; exit}' times
12:41:00
with 0 padding the hour you can do string only comparison.