I have joined in a Data competition for students. They gave me timestamps of users' interaction. Is it useful?
Some of them are:
1615983880510
1615767552552
1615767577100
1616036788631
What you are looking at is a linux timestamp. It's actually a pretty interesting time representation. In short, it's just a number. And it represents the total number of seconds that have passed since January 1st 1970. A date known as "Unix Epoch Time".
Now, if you want to convert that into a readable date there are many ways to do it in basically every programing language. For example in python you might do something like this:
from datetime import datetime
def printdate(unix):
print(datetime.utcfromtimestamp(unix).strftime('%Y-%m-%d %H:%M:%S'))
BUT! It seems like your dates might actually be in miliseconds. Meaning that you might actually want to divide your dates by 1000 before passing them trough the function. So...
def printdatems(unix):
return printdate(unix/1000)
And there you go!
printdatems(1615983880510) #2021-03-17 12:24:40
printdatems(1615767552552) #2021-03-15 00:19:12
printdatems(1615767577100) #2021-03-15 00:19:37
printdatems(1616036788631) #2021-03-18 03:06:28
That's the output for the example dates you provided.
Of course you can find much more information on Wikipedia:
https://en.wikipedia.org/wiki/Unix_time
It's an intresting read!
Related
There are several ways to get time in Tarantool:
Using the clock module
Using fiber.time function
Using os.date
But what is the correct way to work with dates?
For the first, there are several routines for Unix epoch:
os.time() — classic Lua time function. Kinda slow and not very efficient. Not recommended to use inside tarantool for getting current epoch, but of course will work. May be used for getting epoch of arbitrary date (within local timezone). ex:
os.time({ year = 2020, month = 6, day = 4 })
will produce 1591261200, which is 12:00:00 in my GMT+3 timezone
clock.time() (and clock.time64()) — High resolution timer, almost raw binding to clock_gettime. More information may be taken from doc
fiber.time() (and also fiber.time64()) — Cached version of clock.time. Updated every event loop iteration. Recommended for use if absolute precision of clock is not required.
For converting epoch into different formats and timezones there are variants:
os.date("<format>" [, epoch ]) — Convert epoch into local timezone.
os.date("!<format>" [, epoch ]) (note ! prefix) — Convert epoch into GMT timezone.
For getting components of a date as a table we may use os.date('*t') for local and os.date('!*t') for UTC
icu-date may be considered it you need to work with different timezones and/or formats.
For example, if you need UTC time, it's ok to use cached fiber.time with os.date:
local fiber = require 'fiber'
os.date("!%Y-%m-%dT%H:%M:%SZ", fiber.time())
will return something like 2020-06-04T11:48:54Z independently on timezone
It depends on your task.
If it's important for you to manipulate with timezones/formats etc.
I suggest to use icu-data library (https://github.com/tarantool/icu-date)
I have a dataset of around 5 million records. The dates are read in as strings. They are in the form MM/DD/YYYY HH:MM:SS. I am only interested in the date part of it so I read them in as (A10) format which effectively trims the time.
I then do ALTER TYPE DateVar (SDATE10). I do this as I thought sorting dates would be quicker but I can't find confirmation of this.
Is there a way to time SPSS commands to work out questions like this?
The quickest way I can think of is to use python for the timestamps, and normal SPSS syntax for the sorting - just to replicate real-life conditions
***Start timer, in python.
begin program.
import time
start = time.time()
end program.
***go out of python, into normal SPSS syntax, and do your stuff.
/*Put the syntax you want to test here
***get back to python, stop timer, and calculate time difference.
begin program.
end = time.time()
print("It took ",end - start, " seconds")
end program.
Check the output log, and it will show you the time.
Not very scientific, but quick and easy.
I recommend re-starting SPSS between tests - just to be sure one test is not affecting the other.
From my experience, alter type does something that affects code execution times. Not sure what, but everything seems slower after an alter type. So you might also consider saving and re-opening after using alter type.
You should keep the Date format, because:
Dates In spss are actually numbers (formatted in the display as dates but just numbers all the same). Sorting numbers is faster than sorting strings.
In any case, sorting by dates as strings will not order the file by dates (eg. "12-OCT-2017" > "11-NOV-2017").
See another good reason in #horace_vr's comment below.
What's the difference between strptime and strftime? I see that strptime is a method in the DateTime class, and strftime is a method in the Time class.
What's the difference between Time and DateTime, other than that they have different core methods? The explanation for the Time class in the Ruby docs is helpful, but the one for DateTime just says "datetime". There's also the Date class, which says it provides Date and DateTime. Help me make sense of this.
I see strptime and I want to pronounce it "strip time", but that doesn't make sense. Is there a good mnemonic-device for it?
What do strptime and strftime mean, anyway?
How do you remember which does what?
The difference between Time and DateTime has to do with implementation. A large amount of the DateTime functionality comes from the Rails world and is an arbitrary date with time of day. It's more of a calendar-based system. Time is measured as seconds since January 1, 1970 UTC and is time-zone agnostic. On some systems it is limited to values between 1901 and 2038, a limitation of how traditionally this value is stored as a signed 32-bit integer, but newer versions of Ruby can handle a much wider range, using a 64-bit value or BigNum as required.
In short, DateTime is what you get from a database in Rails where Time is what Ruby has traditionally used. If you're working with values where dates are important and you want to know things like the end of the month or what day it'll be six weeks ahead, use DateTime. If you're just measuring elapsed time and don't care about that, use Time. They're easy to convert between if necessary.
Date on the other hand is just a calendar date and doesn't have any associated times. You might want to use these where times are irrelevant.
strptime is short for "parse time" where strftime is for "formatting time". That is, strptime is the opposite of strftime though they use, conveniently, the same formatting specification. I've rarely seen strptime used since DateTime.parse is usually good at picking up on what's going on, but if you really need to spell it out, by all means use the legacy parser.
strptime means string parser, this will convert a string format to datetime.
Example:-
datetime.strptime('2019-08-09 01:01:01', "%Y-%m-%d %H:%M:%S")
datetime.datetime(2019, 8, 9, 1, 1, 1)//Result
strftime means string formatter, this will format a datetime object to string format.
Example:-
sample_date=datetime.strptime('2019-08-09 01:01:01', "%Y-%m-%d %H:%M:%S")
datetime.strftime(sample_date, "%Y-%d-%m %H:%M:%S")
'2019-09-08 01:01:01'//Result
I read the above answer and it is clear in its delineation of Time, DateTime and Date in Ruby.
Time is packaged with Ruby. It is measured as seconds since January 1, 1970 UTC and is time-zone agnostic. More specifically, the Time class stores integer numbers, which presents the seconds intervals since the Epoch. We can think of this as Unix Time. It has some limitations. I read somewhere if stored as a 64-bit signed integer, it can represent dates between 1823-11-12 to 2116-02-20, but on my system it can represent dates outside this range. If you do not specify the timezone to use in the enviroment variable ENV['TZ'], then it will default to your system time found in /etc/localtime on Unix-like systems. When to use Time? It is useful for measuring time elapse or interpolating a timestamp into a string value.
Rails actually extends the Time class. It accomplishes this through ActiveSupport::TimeWithZone. It provides support for configurable time zones. Note Rails will always convert time zone to UTC before it writes to or reads from the database, no matter what time zone you set in the configuration file. In other words, it is the default behaviour of Rails that all your time will get saved into database in UTC format.
# Get current time using the time zone of current local system or ENV['TZ'] if the latter is set.
Time.now
# Get current time using the time zone of UTC
Time.now.utc
# Get the unix timestamp of current time => 1524855779
Time.now.to_i
# Convert from unix timestamp back to time form
Time.at(1524855779)
# USE Rails implementation of Time! Notice we say Time.current rather than Time.now. This will allow you to use the timezone defined in Rails configuration and get access to all the timezone goodies provided by ActiveSupport::TimeWithZone.
Time.current
TimeWithZone provides a lot of very useful helper methods:
# Get the time of n day, week, month, year ago
1.day.ago
1.week.ago
3.months.ago
1.year.ago
# Get the beginning of or end of the day, week, month ...
Time.now.beginning_of_day
30.days.ago.end_of_day
1.week.ago.end_of_month
# Convert time to unix timestamp
1.week.ago.beginning_of_day.to_i
# Convert time instance to date instance
1.month.ago.to_date
For most cases, the Time with the time zone class from Rails’ ActiveSupport is sufficient. But sometimes you just need a date.
Just as with the Time class, Ruby is packaged with the Date class. Simply require the time library:
require "time"
Time.parse("Dec 8 2015 10:19")
#=> 2015-12-08 10:19:00 -0200
Date.parse("Dec 8 2015")
#=> #<Date: 2015-12-08>
Time.new(2015, 12, 8, 10, 19)
#=> 2015-12-08 10:19:00 -0200
Date.new(2015, 12, 8)
Since Date is part of Ruby, it by default uses the timezone defined in /etc/localtime on Unix-like systems, unless you modify the TZ environmental variable. Just as with the Time class, Rails extends the Date class. Use Date.current instead of Date.today to take advantage of ActiveSupport::TimeWithZone and use Rails-based timezone configurations.
Now there is one more class available with regards to dates and times. DateTime is a subclass of Date and can easily handles date, hour, minute, second and offset. It is both available in Ruby (via require 'time') and in Rails (via require 'date'). Rails extends it with TimeZone capabilities just like with the Time class.
require 'date'
DateTime.new(2001,2,3,4,5,6)
I personally do not see a need for using DateTime in your applications, for you can use Time itself to represent dates and times, and you can use Date to represent dates.
The second part of the question was regarding strptime and strftime. Time, Date and DateTime all have the strptime and strftime methods. strptime parses the given string representation and creates an object. Here is an example:
> result = Time.strptime "04/27/2018", "%m/%d/%Y"
=> 2018-04-27 00:00:00 -0400
> result.class
=> Time
This is useful if you have an application and a user submits a form and you are given a date and/or represented as a string. You will want to parse it into a Time or Date before you save it to the database.
strftime formats a date or time. So you call it on a Date or Time object:
> Date.current.strftime("%Y-%m-%d")
=> "2018-04-27"
And you can use them together to first parse user input and then format it in a certain way, perhaps to output into a csv file:
value = Date.strptime(val, '%m/%d/%Y').strftime('%Y-%m-%d')
Is there a way to represent dates like 12/25 without year information? I'm thinking of just using an array of [month, year] unless there is a better way.
You could use the Date class and hard set the year to a leap year (so that you could represent 2/29 if you wanted). This would be convenient if you needed to perform 'distance' calculations between two dates (assuming that you didn't need to wrap across year boundaries and that you didn't care about the off-by-one day answers you'd get when crossing 2/29 incorrectly for some years).
It might also be convenient because you could use #strftime to display the date as (for example) "Mar-3" if you wanted.
Depending on the usage, though, I think I would probably represent them explicitly, either in a paired array or something like YearlessDate = Struct.new(:month,:day). That way you're not tempted to make mistakes like those mentioned above.
However, I've never had a date that wasn't actually associated with a year. Assuming this is the case for you, then #SeanHill's answer is best: keep the year info but don't display it to the user when it's not appropriate.
You would use the strftime function from the Time class.
time = Time.now
time.strftime("%m/%d")
While #Phrogz answer makes perfect sense, it has a downside:
YearlessDate = Struct.new(:month,:day)
yearless_date = YearlessDate.new(5, 8)
This interface is prone to MM, DD versus DD, MM confusion.
You might want to use Date instead and consider the year 0 as "yearless date" (provided you're not a historian dealing with real dates around bc/ad of course).
The year 0 is a leap year and therefore accommodates every possible day/month duple:
Date.parse("0000-02-29").leap? #=> true
If you want to make this convention air tight, just define your own class around it, here's a minimalistic example:
class YearlessDate < Date
private :year
end
The most "correct" way to represent a date without a year is as a Fixnum between 001 and 365. You can do comparisons on them without having to turn it into a date, and can easily create a date for a given year as needed using Date.ordinal
How to get UTC timestamp in Ruby?
You could use: Time.now.to_i.
time = Time.now.getutc
Rationale: In my eyes a timestamp is exactly that: A point in time. This can be accurately represented with an object. If you need anything else, a scalar value, e.g. seconds since the Unix epoch, 100-ns intervals since 1601 or maybe a string for display purposes or storing the timestamp in a database, you can readily get that from the object. But that depends very much on your intended use.
Saying that »a true timestamp is the number of seconds since the Unix epoch« is a little missing the point, as that is one way of representing a point in time, but it also needs additional information to even know that you're dealing with a time and not a number. A Time object solves this problem nicely by representing a point in time and also being explicit about what it is.
The default formatting is not very useful, in my opinion. I prefer ISO8601 as it's sortable, relatively compact and widely recognized:
>> require 'time'
=> true
>> Time.now.utc.iso8601
=> "2011-07-28T23:14:04Z"
if you need a human-readable timestamp (like rails migration has) ex. "20190527141340"
Time.now.utc.to_formatted_s(:number) # using Rails
Time.now.utc.strftime("%Y%m%d%H%M%S") # using Ruby
Usually timestamp has no timezone.
% irb
> Time.now.to_i == Time.now.getutc.to_i
=> true
What good is a timestamp with its granularity given in seconds? I find it much more practical working with Time.now.to_f. Heck, you may even throw a to_s.sub('.','') to get rid of the decimal point, or perform a typecast like this: Integer(1e6*Time.now.to_f).
Time.utc(2010, 05, 17)
time = Time.zone.now()
It will work as
irb> Time.zone.now
=> 2017-12-02 12:06:41 UTC
The proper way is to do a Time.now.getutc.to_i to get the proper timestamp amount as simply displaying the integer need not always be same as the utc timestamp due to time zone differences.