So I'm writing some code trying to convert some oracle dates I got into a different time zone. The issue I'm having is that when the time portion is 00:00:00 I don't now how to determine if it is legitimately midnight or if the date was meant to not include a time.
Currently, I'm making the assumption that if the time is 00:00:00 then the value is just a time-free date, because unfortunately that is sometimes the case, but while statistically small, there is a chance that the date is legitimately midnight so I'm trying to find a better approach with no success.
I can't assume all 00:00:00 are midnight, because if the data was intended to only have a date then converting to most other US time zones would change the date.
Any suggestions?
Check for other values in the same column to see what the distribution of 00:00:00's is. If all entries are 00:00:00 then the column is definitely dates, if it's roughly 1 in (24*60*60) occurrences then it's definitely times, if it's somewhere in between then you've got a problem.
You could also look for a check constraint to see if times are constrained to be midnight.
You could look at the semantics of the column also -- what does the name tell you?
Related
This is the value which I have,
Sun Mar 29 2020 02:55:00 GMT+0530
and I want to get,for example
Asia/Calcutta
as ouput. Thanks in advance.
Offset does not indicate zone
get TimeZone value using time stamp
No.
You cannot determine a time zone from an offset.
Many time zones can share the same offset-from-UTC (the number of hours-minutes-seconds ahead or behind the prime meridian).
See the list of time zone names in Wikipedia. Click on the column header to sort by offset. Notice how often several zones share the same offset.
Specific to your example, notice how we currently have two zones that coincidentally share an offset of five and a half hours ahead of UTC:
Asia/Kolkata (India)
Asia/Colombo (Sri Lanka)
So, without further input, there is no way to know if the author of your input string intended India time or Sri Lanka time.
By the way, the name Asia/Calcutta has been changed to Asia/Kolkata. If your system has no such name, then your tzdata is several years out of date. Always keep all the copies of tzdata up-to-date in OSes, database servers such as Postgres, and runtimes such as Java.
Another complication: politicians frequently change the offset used in their jurisdictions.
So while all of India today uses the same offset of +05:30, that has not always been the case, nor is it likely to always be true in the future (based on the history of how often zones change around the world).
ISO 8601
The ISO 8601 standard defines many sensible formats for representing date-time values as text.
2020-01-23T12:34:56.123456789+05:30
The java.time framework built into Java 8 and later extends one of those format wisely by appending the name of the time zone in square brackets. I suggest using this format if feasible.
2020-01-23T12:34:56.123456789+05:30[Asia/Kolkata]
I'm reverse engineering a query from a kibana board and it has timestamp values in it like '1408884022624'.
In reading over the elastic search date mapping docs I don't see anything in there regarding what (appears to be) some sort of millisecond or tick format. Could someone tell me what the number above represents in my query? (I'm pretty sure we're not using a custom date format.)
It's the number of milliseconds since the beginning of Unix Epoch time, 00:00:00 UTC January 1 1970. Sometimes referred to as Java Epoch time. Technically it's not Unix Epoch time as that's tracked as the number of seconds since the above date, but many tools/converters handle both seconds and milliseconds.
Care should be taken, though, as it's quite easy to accidentally get the time in one format (let's say seconds) and pass it to a function or method expecting it in the other.
http://en.wikipedia.org/wiki/Unix_time
http://www.javaworld.com/article/2074293/core-java/groovy--java--and-the-unix-epoch-time.html
I am planning on starting a project that will need to record timestamps of incoming transactions. I appreciate that Unix Time is an integer value and I can use this type of functionality to my advantage. However, Unix Time only measures in seconds. As a minimal requirement I need to record transaction times at the millisecond level.
I know that there are ways that I could get around this issue, but I was wondering if there was another standardized way of representing time data that also represented milliseconds (or, some factor of sub-milliseconds) in the time value that is fully expressed as an integer value since epoch.
Does such a time format exist? FYI, so long as the date data-type is standardized, I don't care what system this is native in. I can code my own implementation, however, I would like to use an existing date/time format, rather than create my own.
One place where such a standard is used is ECMAScript / Javascript. Javascript date objects use milliseconds since January 1, 1970, midnight UTC for their numerical integer representation. This is detailed here.
You can test this using your browser's console:
var d = new Date();
console.log(d.getTime()); // yields integer milliseconds since epoch
So yes, there is prior art for such a use.
date +%s
outputs timestamp in seconds
date +%s%N
returns timestamp in nanoseconds
To get milliseconds divide the nanoseconds by 1 000 000
UNIX time is not appropriate for time stamping transactions because it does some weird stuff, inserting leap seconds on occasion, thus making it so that you won't be able to add and subtract time stamps reliably, nor sort transactions by timestamp.
A more appropriate standard for timestamps is TAI https://www.nist.gov/pml/time-and-frequency-division/nist-time-frequently-asked-questions-faq#tai . TAI is stored in the same way as UNIX time as a number of seconds and or microseconds and or nanoseconds since the UNIX epoch, however, it is the true number, no leap seconds are added or removed. This means that you can actually add and subtract TAI timestamps to get elapsed time and TAI timestamps are always sortable. Unfortunately, support for TAI timestamps is somewhat limited. For example, linux added support for TAI timestamps only recently in version 3.10 python added this support only in version 3.9/time.html?highlight=time#module-time
I'm currently dealing with a system which uses an unknown timestamp mechanism.
The system is running on a Windows machine, so my first thought was that it uses some kind of Windows epoch for its timestamps, but it appears it does not.
My goal is to convert these timestamps to Unix timestamps.
A few examples:
The following timestamp: 2111441659 converts to: 2013-10-01 11:59
2111441998 to 2013-10-01 17:14
2111443876 to 2013-10-02 14:36
2111444089 to 2013-10-02 17:57
(All dates are GMT+2)
I've tried to calculate the reference date using the data above, but somehow I get a different result with every single timestamp.
Could anybody shed some light on this rather odd problem?
Thanks in advance!
To me the number seems to small to be milliseconds. My first guess was then seconds but looking at the speed this number varies with i think minutes is a better guess. Doing some math on it 2111441659/60/24/365 = 4017.20254756 which suggests the epoch might be sometime in the year -2000?
Here is a list of common epochs in computing but the year -2000 is not really there :) How are you obtaining this timestamp?
P.S. are you sure the year is set to 2013 on this machine and not to 4013? :) This would then fit with the .NET epoch of January 1, Year 1
In order to distinguish your timestamp from Unix timestamp, let's call yours The Counter.
So we have four counter values with their corresponding DateTime value. The first thing to do is calculate the counter's unit correspondence to a real time unit, let's say a second.
In order to do that, we need (1) the difference d between two counter values and (2) the difference s between their corresponding DateTimes, in seconds.
Considering the first two values we have d1=2111441998-2111441659=339. The difference between 2013-10-01 11:59 and 2013-10-01 17:14 (in seconds) is s1=18900. Consequently, the counter's unit corresponds to u1=s1/d1=55.7522123894 seconds.
But if we do the same with pairs #2 and #3, we will find that u2=40.9584664536 seconds.
Similarily, pairs #3 and #4 give us u3=56.6197183114 seconds.
My conclusion therefore, is that there's no alignment between the counter values and the corresponding DateTimes provided. That's the reason why you get a different result with each sample.
Finally, after many hours of comparing the timestamps with the datetimes, trying to discover the logic between them, I've found the answer by reverse engineering the software which generates the timestamps.
It turns out that the integer timestamps are actually bitwise representations* of the datetimes.
In pseudocode:
year = TimeStamp >> 20;
month = (TimeStamp >> 16) & 15;
day = (TimeStamp >> 11) & 31;
hour = (TimeStamp >> 6) & 31;
minute = TimeStamp & 63;
*I'm not sure if this is the correct term for it, if not, please correct me.
I'm looking for a better than O(n) algorithm to determine if a date in the future will have daylight savings time applied (and how much). Given a year, month, day, hour, minute and time zone (and a copy of the Olsen Time Zone database) how does one efficiently determine if that date will be in DST? I'm looking for the algorithm, not a library function to call.
Thank you.
FURTHER EXPLANATION: The date library I'm using is very slow when you create an object with a date in the future and a time zone. It turns out its doing a linear calculation to calculate if the date is in daylight savings time. Not only that, its doing this at object creation time. Obviously it could wait until asked, but it should also be more efficient.
Sure, DST rules change and a date library can't predict the future, but the alternative is to put an arbitrary upper limit on localized dates.
Everybody's already commented on the problems with always-changing DSTs. But I can accept the premise that we just pretend the currently known rules will apply forever.
To get your DST information, the first thing to do is to calculate the year/month/day for your future date (if it isn't in that form already). Then you look up your time zone and pull out the variation against UTC, the DST on/off rule and offset. There could be several different rules depending on which year, you want to be sure to grab the right one for your "target" year. For reasons explained below, it may be handy to also be aware of the rules for the preceding year.
The on/off rules will have a funny spec like "Oct lastSun": That means the switch occurs in the night of the last Sunday in October.
What you need to do is gather up all of these tersely formatted "rules" and develop a bit of code for each to determine the last date indicated by that rule. It's currently December, so given a couple of rules like "Mar lastSun" and "Oct lastSun" for my time zone, those dates would be March 29, 2009 and October 25, 2009. Which of these dates is more recent? October. October is associated with an "off", so we must currently have NO DST.
You can calculate the DST on/off dates for the current (i.e. target) year regardless of whether the target date is before or after those dates; if the on/off date is in the future of your target date, then simply do the rule calculation again for the previous year. Note that the rules may have changed during the interval, so be sure to apply the correct one for the year you're looking at.
Worst case for this calculation is, you have to repeat your two rule calculations for the previous year. But there's no searching going on otherwise, so it's strictly O(1).
I found a Local/DST/Tz calculator here: http://home-4.tiscali.nl/~t876506/WhatDay.html and as it's a JavaScript applet you should be able to simply crib the code. It doesn't handle all rules, though, so you will need to add some code for the remaining rules.
Update: I just noticed you have an hour and minute in your time as well. That complicates matters just a little. If your date is not on a "switch" date then the instructions I gave above will do you fine. Otherwise, you need to consider the time. I guess the cleanest thing to do would be to include the time in your determination of "most recent". I.e. if your target time is 00:30 UTC and switch time for the given zone is 01:00, then the target year's switch time is still in the future and you have to use the previous year's switch time instead. For practical purposes, this will mean that the "other" switch time was the most recent, and its on/off status applies.
Your number one problem is daylight savings rules that are set by the local authorities. The latter can pass almost any law at any time and therefore change the rules in a way you can't possibly predict.
As far as I know DST changes that are known start and end on a fixed day each year (first weekend in april, last weekend in october, stuff like that). So you could ese the Doomsday Algorithm to find the days of the week for the given year and calculate the conversion dates from that. Then you can determine if DST is in effect in source and/or destination locale. The converion itself is simply a matter of adding and/or subtracting an hour to compensate for DST and then factor in the timezone difference.
Hmm, as I see the problematic point is to determine weekday for a given day, far in the future.
For that, I suggest something like that:
after every 400 years, the complete system turns around, so first divide the number of years with 400, take the integral part. In 400 years, there are 99 leap years and 301 simple ones. If an arbitrary day is Monday, then the same day 400 years later will be 301+2x99 = 499 (mod 7) ---> Monday+2 ---> Wednesday. So you have to say something like that:
wday = (ref_day + 2 * (int)((target_year - ref_year) / 400)) mod 7
then you can do further optimizations, but I guess you can go year-by-year, that will do it. At most you make 399 simple operations, if (leap_year) then ++ else +=2, mod 7.
After you have the weekday for Jan 1 that year, you can calculate DST switching dates, as Carl Smotricz has written.