I am translating a PowerShell script in Bash.
This is how the ticks for current datetime are obtained in PowerShell:
[System.DateTime]::Now.Ticks;
By following the definition of Ticks, this is how I am trying to approximate the same calculation using the date command in bash:
echo $(($(($(date -u '+%s') - $(date -d "0001-01-01T00:00:00.0000000 UTC" '+%s'))) * 10000000 ))
This is what I got the last time I tried:
$ echo $(($(($(date -u '+%s') - $(date -d "0001-01-01T00:00:00.0000000 UTC" '+%s'))) * 10000000 )) ; pwsh -c "[System.DateTime]::Now.Ticks;"
637707117310000000
637707189324310740
In particular, the first 7 digits are identical, but digits in position 8 and 9 are still too different between the two values.
I calculated that this means there is just a 2 hours difference between the 2 values. But why? It cannot be the timezone, since I specified UTC timezone in both date commands, right? What do you think?
Note: my suspects about the timezone are increasing, since I am currently based in UTC+2 (therefore 2 hours difference from UTC), but how is this possible since I explicitly specified UTC as timezone in the date commands?
Solved it! The problem wasn't in the date commands, it was in the PowerShell command, which was using the +2 Timezone (CEST time). To fix this, I am now using UtcNow instead of Now.
This is what I am getting now:
$ echo $(($(($(date -u '+%s') - $(date -d "0001-01-01T00:00:00.0000000 UTC" '+%s'))) * 10000000 )) ; pwsh -c "[System.DateTime]::UtcNow.Ticks;"
637707132410000000
637707132415874110
As you can see, now all the digits are identical, except for the last 7th digits, since I added zeros on purpose to convert from seconds to ticks, as I am not interested in fractions of seconds (for now) and I consider them negligible.
Alternative way
Another way to make the two values identical (still excluding fractions of seconds), is to remove the -u option in the first date command in order to use the current time zone, and replace UTC with +0200 in the second date command. If I do this, I can leave Now on the PowerShell command (instead of replacing it with UtcNow).
By doing this, I am getting:
$ echo $(($(($(date '+%s') - $(date -d "0001-01-01T00:00:00.0000000 +0200" '+%s'))) * 10000000)) ; pwsh -c "[System.DateTime]::Now.Ticks;"
637707218060000000
637707218067248090
If you also want fractions of seconds
I just understood that if you also need to consider fractions of seconds, then you just need to add the result of date '+%N' (nanoseconds) divided by 100 to the calculation, in any of the two approaches shown above.
Since the result of date '+%N' can have some leading zeros, Bash may think it's an octal value. To avoid this, just prepend 10# to explicitly say it is a decimal value.
For example, taking the second approach shown above (the "alternative way"), now I get:
$ echo $(($(($(date '+%s') - $(date -d "0001-01-01T00:00:00.0000000 +0200" '+%s'))) * 10000000 + $((10#$(date '+%N')/100)) ))
637707225953311420
Related
I have two dates in forms like: YYYYMMDDHH and want to calculate the differences (in hours) between these two dates. For example
start_date=1996010100
end_date=1996010122
which stands for two dates: 1996-01-01 00:00:00 and 1996-01-01 22:00:00. I want to use date to calculate the difference in hours, the result shall be 22 hours. I tried with
START=$(date -d "$start_date" +"%s")
END=$(date -d "$end_date" +"%s")
HOURS=$(bc -l <<< "($END - $START) / 3600")
but it failed...
So how can I do this? Thanks!
For performance reasons we want to limit the number of sub-process calls we need to invoke:
use bash substring functionality to convert inputs into usable date/time strings
use bash math to replace bc call
bash substring functionality to break the inputs into a usable date/time format, eg:
# convert to usable date/time format:
$ start_date=1996010100
$ echo "${start_date:0:4}-${start_date:4:2}-${start_date:6:2} ${start_date:8:2}:00:00"
1996-01-01 00:00:00
# convert to epoch/seconds:
$ start=$(date -d "${start_date:0:4}-${start_date:4:2}-${start_date:6:2} ${start_date:8:2}:00:00" +"%s")
$ echo $start
820476000
Applying to ${end_date} and using bash math:
$ end_date=1996010122
$ end=$(date -d "${end_date:0:4}-${end_date:4:2}-${end_date:6:2} ${end_date:8:2}:00:00" +"%s")
$ echo $end
820555200
$ hours=$(( (end - start) / 3600))
$ echo $hours
22
This leaves us with 2 sub-process calls ($(date ...)). While other languages/tools (awk, perl, etc) can likely speed this up a bit, if you need to store the result in a bash variable then you're looking at needing at least 1 sub-process call (ie, hours=$(awk/perl/??? ...)).
If performance is really important (eg, needing to perform this 1000's of times) take a look at this SO answer that uses a fifo, background date process and io redirection ... yeah, a bit more coding and a bit more convoluted but also a bit faster for large volumes of operations.
busybox date can do the trick
start_date=1996010100
end_date=1996010122
START=$(busybox date -D "%Y%m%d%H" -d "$start_date" +"%s")
END=$(busybox date -D "%Y%m%d%H" -d "$end_date" +"%s")
HOURS=$(bc -l <<< "scale=0;($END - $START) / 3600")
echo $HOURS
If it's possible for you to use a more fully-featured scripting language like Python, it'll provide a much more pleasant and understandable date parsing experience, and is probably installed by default (datetime is also a standard Python library)
Structured with shell vars
start_date=1996010100
end_date=1996010122
python -c "import datetime ; td = datetime.datetime.strptime('${end_date}', '%Y%m%d%H') - datetime.datetime.strptime('${start_date}', '%Y%m%d%H') ; print(int(td.total_seconds() / 3600))"
Structured to read dates and format code from stdin
echo '%Y%m%d%H' 1996010100 1996010122 | python -c "import datetime,sys ; fmt, date_start, date_end = sys.stdin.read().strip().split() ; td = datetime.datetime.strptime(date_end, fmt) - datetime.datetime.strptime(date_start, fmt) ; print(int(td.total_seconds() / 3600))"
Should work with both Python 3 and Python 2.7
format codes available here (1989 C standard)
https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
which stands for two dates: 1996-01-01 00:00:00
So convert it to that form if it stands for it.
start_date=1996010100
start_date=$(sed -E 's/(....)(..)(..)(..)/\1-\2-\3 \4:00:00/' <<<"$start_date")
start=$(date -d "$start_date" +"%s")
and the same with end.
the most simple way is to install "dateutils" using this command
sudo apt-get install dateutils
Run these commands to get the difference in seconds:
dateutils.ddiff -i '%Y%m%d%H%M%S' 20200817040001 20210817040101
output:
31536060s
next step: Simply divide by 86400 to get the number of days or similarly for hours and minutes :)
I want to round down the minutes to the nearest 15 min interval i.e. 00,15,30,45. I'm currently doing the below:
echo $(date +'%Y/%m/%d/%H/')$((($(($(date +'%M') / 15))-0)*15))
But at the start of the hour between 1-14 minutes, I get "/2021/11/03/21/0" instead of 00.
Also, I'm not sure if this is the best way to do this. Are there any alternatives?
Would you please try the following:
mod=$(( 10#$(date +%M) \% 15 ))
date -d "-${mod} minutes" +%Y/%m/%d/%H/%M
The variable mod holds the remainder of the minutes divided by 15.
Then round down to the nearest 15 minute interval by subtracting mod.
[Edit]
The manpage of crontab says:
Percent-signs (%) in the command, unless
escaped with backslash (), will be changed into newline
characters, and all data after the first % will be sent to
the command as standard input.
If you want to execute the command within crontab, please modify the command as:
mod=$(( 10#$(date +\%M) \% 15 ))
date -d "-${mod} minutes" +\%Y/\%m/\%d/\%H/\%M
[Edit2]
If you want to embed the code in crontab file, please add a line which look like:
0 12 * * * username bash -c 'mod=$(( 10#$(date +\%M) \% 15 )); DATEVAR=$(date -d "-${mod} minutes" +\%Y/\%m/\%d/\%H/\%M); write.sh "$DATEVAR"'
Please modify the execution time/date and the username accordingly.
The default shell to execute crontab command may be /bin/sh. Then you will need to explicitly switch it to /bin/bash to execute bash commands.
My apology that a backslash in front of % 15 (modulo operation) was missing in my previous post.
Another approach:
min=$(printf "%0.2d" $(( ($(date +'%M') / 15) * 15 )))
echo "$(date +'%Y/%m/%d/%H/')$min"
date -d "#$((($(date +%s) + 450) / 900 * 900))"
This uses the properties of integer division to “subtract a modulus” and adds half of the desired interval to (almost) mimic a rounding operation.
A bit of extra sub-second rounding precision (for no good reason) can be achieved by taking %N (nanoseconds) into account. But it will not matter, because the exact half of the rounding interval (450 seconds) is already aligned with the default epoch resolution (1 second). (If the number of seconds in the desired rounding interval was odd, then the following would increase the time rounding precision.)
date -d "#$((($(date +%s%N) + 45*10**10) / (9*10**11) * 900))"
Pure bash, bash version 4.3 or higher:
printf '%(%Y/%m/%d/%H/%M)T\n' "$(( $(printf '%(%s)T') /(15*60)*(15*60) ))"
Using GNU date (any bash version or POSIX shell):
date -d #$(( $(date +%s) /(15*60)*(15*60) )) +%Y/%m/%d/%H/%M
Truncates the current epoch date (seconds since 1970-01-01 00:00:00) to a 15 minute (900 second) interval, then converts to desired format.
Retrieves the current date/time once only.
If you build a date/time from two separate date/times, it can be wrong, when a unit ticks over in between.
The printf date-time format string was added in bash 4.2, and was changed in 4.3 to also print the current time, if no input date was given.
Note that bash arithmetic treats numbers that start with zero as octals, and numbers like 08 and 09 will cause an error (because they are not octal numbers).
I'm currently creating a shell script that will run a python code once an hour that collects, processes, and displays data from a radar for the previous hour.
The python code I am using requires a UTC begin time and end time in format "YYYYMMDDHHmm". So far, I have found using the unix command date -u +"%Y%m%d%H%M" will retrieve my current time in the correct format, but I have not been able to find a way to subtract 60 minutes from this first time and have it output the "start" time/
code I have tried:
date +"%Y%m%d%H%M-60" >> out: 201908201833-60
now= date -u +"%Y%m%d%H%M" >> out:201908201834
echo "$now - 60" >> out: - 60
I'm just starting to self teach/learn shell coding and I am more comfortable with python coding which is why my attempts are set up more like how you would write with python. I'm sure there is a way to store the variable and have it subtract 60 from the end time, but I have not been able to find a good online source for this (both on here and via Google).
You can use -d option in date:
date -u +"%Y%m%d%H%M" -d '-60 minutes'
or else subtract 1 hour instead of 60 minutes:
date -u +"%Y%m%d%H%M" -d '-1 hour'
To be able to capture this value in a variable, use command substitution:
now=$(date -u +"%Y%m%d%H%M" -d '-1 hour')
On OSX (BSD) use this date command as -d is not supported:
now=$(date -u -v-1H +"%Y%m%d%H%M")
Your current attempt has some simple shell script errors.
now= date -u +"%Y%m%d%H%M" >> out:201908201834
This assigns an empty string to the variable now and then runs the date command as previously. If the plan is to capture the output to the variable now, the syntax for that is
now=$(date -u +"%Y%m%d%H%M")
Next up, you try to
echo "$now - 60"
which of course will output the literal string
201908201834 - 60
rather than perform arithmetic evaluation. You can say
echo "$((now - 60))"
to subtract 60 from the value and echo that -- but of course, date arithmetic isn't that simple; subtracting 60 from 201908210012 will not produce 201908202312 like you would hope.
If you have GNU date (that's a big if if you really want to target any Unix) you could simply have done
date -u -d "60 minutes ago" +%F%H%M
but if you are doing this from Python anyway, performing the date extraction and manipulation in Python too will be a lot more efficient as well as more portable.
from datetime import datetime, timedelta
dt = datetime.strptime(when,'%Y%m%d%H%M')
print(dt - timedelta(minutes=60))
The shell command substitution $(command) and arithmetic evaluation $((expression)) syntaxes look vaguely similar, but are really unrelated. Both of them have been introduced after the fundamental shell syntax was already stable, so they had to find a way to introduce new syntax which didn't already have a well-established meaning in the original Bourne shell.
I have a script:
#!/bin/bash
date +%T &
Hours=`date +"%H"` &
Minutes=`date +"%M"` &
Seconds=`date +"%S"`
echo "$Hours:$Minutes:$Seconds"
The objective is to echo date two times and then take out hours, minutes, seconds and calculate how many seconds elapsed between those two commands. So my solution is to write hours, minutes, seconds into variables, then work with those variables.
Problem: echo only echoes seconds which means my interpretation of & is wrong.
How can I fix the & problem? I need those commands to run simultaneously so I can check.
date +%s apparently won't work on certain inputs like:
Wed Mar 4 10:34:59 2015
Wed Mar 4 10:35:08 2015
Will give result of 00:00:01 instead of 00:00:09 or:
Wed Mar 4 10:34:59 2015
Wed Mar 4 17:43:08 2015
will give the result of 12:13:14 instead of 07:08:09. Is it true? Or can I use date +%s and then decrease those two outputs?
You don't need to run the commands simultaneously. Run just one command:
read hours minutes seconds < <( date '+%H %M %S' )
But it can be even simpler: just use the +%s format to get number of seconds since the epoch. You'll get two numbers you can safely subtract.
#!/bin/bash
start=$(date +%s)
sleep 10
end=$(date +%s)
echo The command took $(( end - start )) seconds.
The easiest way is to use the shell variable $SECONDS.
Each time this parameter is referenced, the number of seconds since shell invocation is returned. If a value is assigned to SECONDS, the value returned upon
subsequent references is the number of seconds since the assignment plus the
value assigned. If SECONDS is unset, it loses its special properties, even if
it is subsequently reset.
the main problem is that in this case the command date will be executed tree times to get the values..
so you don't need to execute the commad date too many times
#!/bin/bash
#date +%T &
function myTime(){
now=`date +"%H:%M:%S" &`
}
myTime
s1=`echo $now | cut -d":" -f2`
myTime
s2=`echo $now | cut -d":" -f2`
echo "s1[$s1] - s2[$s2]"
Then you can apply your rules to verify the time elapsed
Regards
Claudio
I've got a series of files that are namedHHMMSSxxxxxxxxxxxxxxxx.mp3, where HH,MM, and SS are parts of a timestamp and the x's are unique per file.
The timestamp follows a 24 hour form (where 10am is 100000, 12pm is 120000, 6pm is 180000, 10pm is 220000, etc). I'd like to shift each down by 10 hours, so that 10am is 000000, 12pm is 020000, etc.
I know basic BASH commands for renaming and moving, etc, but I can't figure out how to do the modular arithmetic on the filenames.
Any help would be very much appreciated.
#!/bin/bash
for f in *.mp3
do
printf -v newhour '%02d' $(( ( 10#${f:0:2} + 14 ) % 24 ))
echo mv "$f" "$newhour${f:2}"
done
Remove the echo to make it functional.
Explanation:
printf -v newhour '%02d' - this is like sprintf(), the value is stored in the named variable
$(( ( 10#${f:0:2} + 14 ) % 24 )) - 10# forces the number to base 10 (e.g. 08 would otherwise be considered an invalid octal), ${f:0:2} extracts the first two characters (the hour), the rest does the math
"$newhour${f:2}" - prepend the new hour before the substring of the original name, starting at the third character
The easiest way is probably to extract the timestamp and use date to turn it into a number of seconds, do normal math on the result, then convert it back. date -d datestring +format lets you do these conversions.