I am creating a java plugin for moving data from cassandra database to elastic search. I am getting all the data but the date which I am getting from the database is in human readable form ie Row[Fri Jul 25 11:36:10 IST 2014].I want this to be converted to epoch timestamp format like 1414386721.
I do not know Cassandra DB, but according to this doc your driver should be translating the date-time value in Cassandra to a java.util.Date object in Java.
You may be confused about how a java.util.Date object works. The j.u.Date class is confusing and difficult in many ways, one of which is that while a Date has no time zone its toString implementation on-the-fly applies the JVM’s current default time zone as it generates the string.
You may also be new to date-time work and therefore confusing a date-time object with its String representation. Consider that 1.4 is a number and should not be confused with its representation as a String in the format of a price €1.40. Likewise a date-time object is not a String but can be represented as a String generated any number of formats.
Lastly, if you are indeed getting a java.util.Date object, learn to convert that to either the Joda-Time library or the java.time library. The java.util.Date and .Calendar classes are notoriously troublesome.
Related
I have a conversation today with my engineering coworkers about application and database design in Fintech.
How do we handle time? We all know that we need to store the datetime information normalized in UTC. But our debate is among whether we should store in 1) unix epoch timestamp which is an integer in our database, for example, 1596507157. which is 08/04/2020 # 2:12am UTC or 2) store in ISO 8601 format in string 2020-08-04T02:12:37+00:00 or 2020-08-04T02:12:37.123456789Z
The downside of unix timestamp is obviously not immediate human-readable.
I am here looking for some design advices on whether we should adopt either approach.
In my opinion, you should store it as integer rather than string, one of many reasons is obvious, int just needs 4 byte while string as ISO 8601 takes a lot more.
What you see as downside, it doesn't impact the backend, it's just a make up (only needed on user view).
Most of database have datetime data type. I guess your 'epoch timestamp' means this datetime or timestamp data type. It's easier (and better performance) to handle datetime type for database engineer. and database also have datetime formatting function.
(Oracle/MS SQL Server - convert, MySQL - date_format, PostgreSQL - to_char ...)
In my opinion, store and handle with datetime type and use formatting function for human.
I wanna persistant an object with LocalDateTime fields with spring data and a couchbase behind in a spring boot app.
Here are the fieldmapping:
#Field
private LocalDateTime start;
#Field
private LocalDateTime end;
When I save the object then the dates are stored as numbers in couchbase.
Here are the stored data in couchbase:
"start": 1518818215508,
So the problem is, if I store an LocalDateTime e.g at 10.00, and then read it from db the result is 09:00 instead of 10:00 because of my local time +1:00.
In Postgres I would save the date in an column with mapping: columnDefinition= "TIMESTAMP WITH TIME ZONE"
How I can solve this problem in couchbase?
JSON (famously) has no date format (unlike relational databases which have a sprawling number of different formats). If you want to store timezone data in JSON, here are two options I can think of:
Using a string instead of a number, and then using something like ISO-8601.
Store the epoch time as you currently are as a UTC value, but also store a separate number field that represents a timezone offset from UTC
I would recommend going with the first approach.
I'm no Spring/Java expert, but a quick look at the documentation says you can do this by setting system property org.springframework.data.couchbase.useISOStringConverterForDate to true
Is there anyway to get a java.time (new in Java 8) compatible time class out of a ResultSet?
I am aware you can use ResultSet's getDate or getTimestamp but these method return java.sql.Date / java.sql.Timestamp objects which are now deprecated so it seems like bad practice to use them in order to create a ZonedDateTime or similar.
Most database vendors don't support JDBC 4.2 yet. This specification says that the new java.time-types like LocalDate will/should be supported using the existing methods setObject(...) and getObject(). No explicit conversion is required and offered (no API-change).
A workaround for the missing support can be manual conversion as described on the Derby-mailing list.
Something like:
LocalDate birthDate = resultSet.getDate("birth_date").toLocalDate();
As you can see, these conversions use the non-deprecated types java.sql.Date etc., see also the javadoc.
New Methods On Timestamp
Java 8 includes new methods on the java.sql.Timestamp class to convert to and from java.time objects. These convenience methods are a stop-gap measure until JDBC drivers can be updated for the new data types.
toInstant
toLocalDateTime
valueOf
from
Ditto For Date & Time
The java.sql.Date and java.sql.Time classes have similar java.time conversion methods added in Java 8 as well.
Today most of us are using JDBC 4.2 compliant drivers, which improves the situation quite a bit compared to the answers from 2015.
To get a LocalDate from your result set:
LocalDate dateFromDatabase = yourResultSet.getObject(yourColumnIndex, LocalDate.class);
or
LocalDate dateFromDatabase = yourResultSet.getObject("yourColumnLabel", LocalDate.class);
No new method has been added to ResultSet for this to work. The getObject method was there all the time. The new thing is that since JDBC 4.2 it accepts LocalDate.class as the second argument and returns a LocalDate. The above works when the query returns a column with SQL datatype date (really the JDBC type counts, but they tend to agree).
You can pass classes of other java.time types too. And get the corresponding type back. For example:
OffsetDateTime dateTimeFromDatabase
= yourResultSet.getObject(yourTimestampWithTimeZoneColumnIndex, OffsetDateTime.class);
The java.time types to use are:
SQL datatype | java.time type
------------------------+-----------------------------------------------------------
date | LocalDate
time | LocalTime
timestamp | LocalDateTime
timestamp with timezone | Officially OffsetDateTime; many drivers accept Instant too
time with timezone | OffsetTime
For passing the other way, from Java to your database (for use as query parameters or for storing) PreparedStatement.setObject now accepts objects of the above java.time types too. Since you are passing an object of a type, there is no need for a separate type parameter when going this way.
Does hibernate have any mapping for this oracle data type:(10G)
TIMESTAMP(6) WITH TIME ZONE
I am getting:
No Dialect mapping for JDBC type: -101
My manager does not want to do the: registerHibernateType(-101, Hibernate.getText().getname())
He thinks it is too much.:)
What alternative can I have?
The answer you provide to yourself is more like a workaround than a proper solution. For the sake of the visitors looking for an answer, I'll provide my view on this:
1) Database date-based fields should be always set to UTC, never with a specific timezone. Date calculation with timezone information is an unneeded complexity. Remember that timezones usually changes twice a year for a lot of countries in the world ("daylight saving time"). There's a reason why only a few RDMBS' supports this, and there's a reason why Hibernate developers refuse to support this data-type. The patch for Hibernate is simple enough (one line of code), the implications aren't.
2) Converting your "timestamp with timezone" to a String will only cause problems later. Once you retrieve it as String, you'll need to convert it again to a Date/Calendar object, an unneeded overhead. Not to mention the risks associated with this operation.
3) If you need to know in which timezone is some user, just store the String representing the timezone offset (like "Europe/Prague"). You can use this in Java to build a Calendar with date/time and timezone, as it'll take care of DST for you.
For now, I solved the problem by:
`select TO_CHAR(TRUNC(field)) from table` //field is the one having type= timestamp with timezone
This ensures that when the query returns, the field has datatype 'String'
I'm storing a simple java.util.date in an Oracle XE database via hibernate.
When testing with JUnit if I can retrieve the correct value, I get an error like this:
junit.framework.AssertionFailedError:
expected:<Sun Dec 28 11:20:27 CET 2008>
but was:<2008-12-28 11:20:27.0>
The value is stored in an Oracle Date column (which should have a second-precision) which looks okay to me. Also, I'm surprised that 11:20:27 is not equal to 11:20:27.0. Or does this have to do with timezones?
Any help is welcome.
Thorsten
Okay, worked some more on it ...
Oracle Date columns only store values with an accuracy of a second.
Java Dates do contain milliseconds, but they are typically not printed. So
expected:
was actually created by a date like 11:20:27,345, which is of course not equal to 11:20:27.0
Solution:
either only use full second dates to store and retrieve
or
get hibernate to create the correct Oracle Datatype (TIMESTAMP) - this is very dependent on the dialect specified in the hibernate config (OracleDialect and Oracle10gDialect create different types).
If you compare a java.util.Date to a java.sql.Date that both represent the same instant in time, equals(Object) will return false (it considers two objects of different classes to never be equal).
Your tests need to account for that. The easiest way to do this is to convert the dates to UNIX time (e.g. java.util.Date.getTime()) and compare those values.