In DB2, I have to store records with granular level timestamps [microsec]. I went for double datatype and stored unix epoch time.
However, I need some recommendations from performance a standpoint of CRUD operations.
Note: My app doesn't need dates earlier than 1970.
Please advise.
Related
TL;DR - am I correct in thinking there is no way to determine the timezone used for a given record of a Date field when the server's time zone was changed at some point in the past?
A factory control system was designed to use UTC server (not Oracle) time so all Date columns were UTC. At some point in the past they changed the server time to local. Now years later, they would like the apps to display the correct date. Without knowing the date the change occurred, is there a way to SELECT so all Date records would be correct? I think I can either make the "modern" records correct or the pre-change records correct but not both unless I know when that happened. Since it was a Windows OS change, I don't think even the alert log would help assuming they haven't trimmed it at some point, but it's not easy to get any info or server access from them anyway. It had been 10g and is now 12c.
Correct. Oracle's DATE datatype is timezone agnostic. If you want to store timezone info in the database, use TIMESTAMP WITH TIMEZONE datatype.
So, I am trying to build a query in RMAN Catalogue ( using RC_RMAN_BACKUP_JOB_DETAILS) to compare the most recent backup duration (TIME_TAKEN_DISPLAY) for each database (DB_NAME) with its historical average AVG backup duration (TIME_TAKEN_DISPLAY).
How do I convert TIME_TAKEN_DISPLAY(timestamp; HH:MM:SS), i.e. in VARCHAR2 Format to a minute format, i.e number only, so as to run the query against the entire RC_RMAN_BACKUP_JOB_DETAILS to compare AVG time taken in past with time takes for last backup for each DB.
One thing that may work is converting String(Time_taken_display)->To_TIME(Time_taken_display in Time format)->TO_NUM(Time_taken_display in minutes in number format), but this will be so highly inefficient.
The solution can be pretty simple and complex depending on the requirements:
One simple solution is:
select avg(substr(TIME_TAKEN_DISPLAY, 0,2)*60 + substr(TIME_TAKEN_DISPLAY, 4,2) + substr(TIME_TAKEN_DISPLAY, 7,2)/60) from RC_RMAN_BACKUP_JOB_DETAILS;
Using Type Casting Functions:
Cast TIME_TAKEN_DISPLAY into time format using TO_TIMESTAMP and then cast to TO_NUMBER, but I did not want to take this approach as I plan to run my scripts against all databases logged in the view, and multiple casting will leave the performance highly inefficient.
But as per #alex Poole comment, I will be using ENLAPSED_SECONDS field as it is readily available in seconds and number data type.
I am actually using the mongDB API via a tLibraryLoad component, as I find this easier to build complex multi-level documents using tJavaRow and tJava components, than using the MongoDB palette components.
I am reading in data from Oracle which are date values with a zero time stamp component: For example:
29-JUN-08 00.00.00
The import works via Talend, however the records in mongo shell appear to be the previous day. You can see the record is inserted as the 28-JUN-2008.
Extract from JSON document in mongoDB:
"status_date" : ISODate("2008-06-28T23:00:00Z")
It is almost as if mongoDB (or Talend?) sees a midnight date as the end of the previous day, rather than the start of the 29th June 2008.
In my Talend schema I have specified the Oracle columns as Date types with a DB Type of Date also.
Any advice appreciated.
---UPDATE 1------
As only some dates are affected, it seems this is a DST adjustment in mongoDB to adjust perhaps the display to my local timezone, as the dates which are impacted are in the back half of the year in daylight saving time.
Is it adjusting the date due to the location of the mongo shell?
It is adjusting the date due to the location of the mongo server so all people using the mongo shell would get the same answers to date queries?
Would different people running different mongo queries on dates get different results based on their location, their DST kick in dates...i.e. you could imagine dates from the 1st November 2015, being counted as contributing to October 31st figures (at 23:00)....
i
I feel your pain - this is derived from MongoDB itself.
At issue is that MongoDB stores dates in UTC format by default.
https://docs.mongodb.org/manual/tutorial/model-time-data/
You can use Mongo's suggestion above but in this case you are storing just the date and not the time. I've used two solutions:
Don't bother storing dates as DATEs. Convert all your dates to %Y%m%d format and store them as integers. You can easily compare dates using $gt and $lte just using integers - just be sure to bring in your date using the same format and convert them back in your program later.
... or ...
Since in your case your date seems to be off by an hour, add an hour to it before you make your insert. It all depends on how long the timezone offset is to your local machine.
On linux you can see what the utc value is using:
date -u
I suppose you could change your local timezone on your machine to be UTC time and see what happens.
Personally, I've never had an issue with using the first approach. It's fast and ensures that I have what I want in there.
I think it is because of the timezone of your server. Try to remove the timezone from your date object and then insert it to the mongodb.
With moment.js you can create date without the timezone in such way:
var date = moment.utc("29-JUN-08 00.00.00", 'DD-MM-YYYY h.mm.s').format()
I got a problem with Oracle Dates. It seems that predefined Dates in an Java Application are different after inserting into an OracleDB.
Insert via JPA entity:
entity.setDateOfCreation(new Date(System.currentTimeInMillis()));
// 1350565985000
After commit and retrieve:
entity.getDateOfCreation() // 1350565985047
Why is this different?
I assumed Oracle would just insert my specific Date Object with these exact Milliseconds into the Database. But obviously it doesn't. Because of the minimal delay it seems to "overwrite" the given Date with its own Date in milliseconds (and despite I do NOT use #GeneratedValue).
Does the table you working with have a trigger which populates that column? I would hope it does. I have experienced lots of problems in the last with time differences between the app server and the database. It is much better to have a single of time which ensures consistent timings across the state.
I am developing an API using Codeigniter and MongoDB.
I am not sure what date format that is the most "flexible" for the timestamp of
each document in the database.
Currently I am using: yyyy-mm-dd hh:mm:ss
Is it better (for mongodb searching and internationalization) to use another format?
My question is just this, what is the best format for a timestamp in a MongoDB document?
Over the years, I have been forced to a very strong personal commitment to always storing date/time as the 10-digit Linux/Unix timestamp, which gives the current (add: local) time as seconds since the Epoch. Just a few moments ago, the time was 1329126719. To me, this is the most flexible format possible. When it comes time to display a date/time, it's simple to convert the 10-digit timestamp to any string you care to show.
Edit: Perhaps a better choice for me would be milliseconds from the Epoch, since that seems to be increasingly favored as the art evolves.
MongoDB has a built-in datetime type which interacts with the datetime types in your application's language (in PHP they become MongoDate instances). MongoDB datetimes are always stored in UTC, and are internally stored as milliseconds since the UNIX epoch. This means that they are compact (they are always 8 bytes, as opposed to string formats which are longer depending on how much precision you choose to store). Additionally, the MongoDB tools all "understand" datetime objects -- you can manipulate them easily from javascript in a Map-Reduce, or using the new aggregation framework.