I am uploading an XML in which one of the field is dailyTime. This dailyTime is an epoch time and i want to convert it into human readable time.
<globalView id="108" version="17" recordClassName="NormalizedEvent" retention="0" hourly="-1" hourlyTime="1284336038994" daily="-1" dailyTime="1284336038994" intervalMilliseconds="60000" writeUniqueCountersTime="0">
<criteria bop="AND">
<left>
<expr>
<interval serialization="custom">
<com.q1labs.ariel.Interval>
<short>5000</short>
<boolean>true</boolean>
<short>5000</short>
<boolean>true</boolean>
</com.q1labs.ariel.Interval>
</interval>
</expr>
<key class
My props.conf are
[XMLPARSING]
KV_MODE = xml
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = <globalView\s\w*=("\d\d\d")
MAX_EVENTS = 600
EXTRACT-dailyTime = ^(?:[^=\n]*=){8}"(\d+)
TIME_FORMAT=%s%3N
TIME_PREFIX=dailyTime=
Lookahead=13
TRUNCATE = 1000
category = Custom
disabled = false
pulldown_type = true
Typically, you'd convert from the timestamp (ie epoch time) to something human-readable in your search
Like this:
index=ndx sourcetype=srctp earliest=-4h
| stats max(_time) as rtime min(_time) as etime by fieldA
| sort 0 - rtime + fieldA
| eval rtime=strftime(rtime,"%c"), etime=strftime(etime,"%c")
| rename rtime as "Most Recent" etime as "Earliest"
Splunk strftime docs: https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/DateandTimeFunctions#strftime.28X.2CY.29
Further formatting info for strptime and strftime: https://strftime.org
Related
The hackerrank challenge is in the following url: https://www.hackerrank.com/challenges/python-time-delta/problem
I got testcase 0 correct, but the website is saying that I have wrong answers for testcase 1 and 2, but in my pycharm, I copied the website expected output and compared with my output and they were exactly the same.
Please have a look at my code.
#!/bin/pyth
# Complete the time_delta function below.
from datetime import datetime
def time_delta(tmp1, tmp2):
dicto = {'Jan':1, 'Feb':2, 'Mar':3,
'Apr':4, 'May':5, 'Jun':6,
'Jul':7, 'Aug':8, 'Sep':9,
'Oct':10, 'Nov':11, 'Dec':12}
# extracting t1 from first timestamp without -xxxx
t1 = datetime(int(tmp1[2]), dicto[tmp1[1]], int(tmp1[0]), int(tmp1[3][:2]),int(tmp1[3][3:5]), int(tmp1[3][6:]))
# extracting t1 from second timestamp without -xxxx
t2 = datetime(int(tmp2[2]), dicto[tmp2[1]], int(tmp2[0]), int(tmp2[3][:2]), int(tmp2[3][3:5]), int(tmp2[3][6:]))
# converting -xxxx of timestamp 1
t1_utc = int(tmp1[4][:3])*3600 + int(tmp1[4][3:])*60
# converting -xxxx of timestamp 2
t2_utc = int(tmp2[4][:3])*3600 + int(tmp2[4][3:])*60
# absolute difference
return abs(int((t1-t2).total_seconds()-(t1_utc-t2_utc)))
if __name__ == '__main__':
# fptr = open(os.environ['OUTPUT_PATH'], 'w')
t = int(input())
for t_itr in range(t):
tmp1 = list(input().split(' '))[1:]
tmp2 = list(input().split(' '))[1:]
delta = time_delta(tmp1, tmp2)
print(delta)
t1_utc = int(tmp1[4][:3])*3600 + int(tmp1[4][3:])*60
For a time zone like +0715, you correctly add “7 hours of seconds” and “15 minutes of seconds”
For a timezone like -0715, you are adding “-7 hours of seconds” and “+15 minutes of seconds”, resulting in -6h45m, instead of -7h15m.
You need to either use the same “sign” for both parts, or apply the sign afterwards.
Hi I would like to subtract time from a CSV array using Ruby
time[0] is 12:12:00AM
time[1] is 12:12:01AM
Here is my code
time_converted = DateTime.parse(time)
difference = time_converted[1].to_i - time_converted[0].to_i
p difference
However, I got 0
p time[0].to_i gives me 12
is there a way to fix this?
You can use Time#strptime to define the format of the parsed string.
In your case the string is %I:%M:%S%p.
%I = 12 hour time
%M = minutes
%S = seconds
%p = AM/PM indicator
So to parse your example:
require 'time'
time = %w(12:12:00AM 12:12:01AM)
parsed_time = time.map { |t| Time.strptime(t, '%I:%M:%S%p').to_i }
parsed_time.last - parsed_time.first
=> 1
Use the Ruby DateTime class and parse your dates into objects of that class.
I would like to know how to precisely get the number of hours between 2 datetimes in spark.
There is a function called datediff which I could use to get the number of days and then convert to hours however this is less precise than I'd like
example of what I want modeled after datediff:
>>> df = sqlContext.createDataFrame([('2016-04-18 21:18:18','2016-04-19 19:15:00')], ['d1', 'd2'])
>>> df.select(hourdiff(df.d2, df.d1).alias('diff')).collect()
[Row(diff=22)]
Try using UDF Here is the sample code, You can modify to UDF return what ever granularity as you want.
from pyspark.sql.functions import udf, col
from datetime import datetime, timedelta
from pyspark.sql.types import LongType
def timediff_x():
def _timediff_x(date1, date2):
date11 = datetime.strptime(date1, '%Y-%m-%d %H:%M:%S')
date22 = datetime.strptime(date2, '%Y-%m-%d %H:%M:%S')
return (date11 - date22).days
return udf(_timediff_x, LongType())
df = sqlContext.createDataFrame([('2016-04-18 21:18:18','2016-04-25 19:15:00')], ['d1', 'd2'])
df.select(timediff_x()(col("d2"), col("d1"))).show()
+----------------------------+
|PythonUDF#_timediff_x(d2,d1)|
+----------------------------+
| 6|
+----------------------------+
If your columns are of type TimestampType(), you can use the answer at the following question:
Spark Scala: DateDiff of two columns by hour or minute
However, if your columns are of type StringType(), you have an option that is easier than defining an UDF, using the built-in functions:
from pyspark.sql.functions import *
diffCol = unix_timestamp(col('d1'), 'yyyy-MM-dd HH:mm:ss') - unix_timestamp(col('d2'), 'yyyy-MM-dd HH:mm:ss')
df = sqlContext.createDataFrame([('2016-04-18 21:18:18','2016-04-19 19:15:00')], ['d1', 'd2'])
df2 = df.withColumn('diff_secs', diffCol)
I have three strings, pulled from a database:
"2015-03-18" (the date the event occurs)
"22:00" (the hour an event occurs)
"-05:00" (the UTC offset in the location the event occurs).
I want to combine these three strings to produce a Ruby Time object. I'm doing:
utc_offset = "-05:00"
airtime = "22:00"
airday = "2015-03-18"
year,month,day,hour,minutes = airday.split("-").map(&:to_i) + airtime.split(":").map(&:to_i)
Time.new(year,month,day,hour,minutes,0,utc_offset)
This works; I'm just wondering if it's the correct/standard/idiomatic/clearest way.
By using Time.parse
When ‘time’ is required, Time is extended with additional methods for
parsing and converting Times.
require 'time'
utc_offset = "-05:00"
airtime = "22:00"
airday = "2015-03-18"
time = Time.parse("#{airday} #{airtime} #{utc_offset}")
I think this is the way to do it.
Time.new(*airday.split("-"), *airtime.split(":"), 0, utc_offset)
How about this? http://ruby-doc.org/stdlib-2.1.5/libdoc/time/rdoc/Time.html#method-c-strptime
require 'time'
raw_time = '2015-03-18 22:00'
parsed_time = Time.strptime(raw_time, '%Y-%m-%d %H:%M') # 2015-03-18 22:00:00 +0100
I have converted date to my local time as below:
$this->date_string = "%Y/%m/%d %h:%i:%s";
$timestamp = now();
$timezone = 'UP45';
$daylight_saving = TRUE;
$time = gmt_to_local($timestamp, $timezone, $daylight_saving);
$this->updated_date = mdate($this->date_string,$time);
And I'm storing this field in to database.
Now at retrieval time I want format like this:
"11-04-2011 4:50:00 PM"
I have used this code:
$timestamp = strtotime($rs->updated_date);
$date1 = "%d-%m-%Y %h:%i:%s %a";
$updat1 = date($date1,$timestamp);
But this will give me only
"11-04-2011 4:50:00 AM"
But I have stored it like it was PM.
Might get voted down, but will have a go at it.
Is it because the MySQL stores it in 24 hour format? (assuming you are using the datetime field type)
Maybe this will help
Converting mysql TIME from 24 HR to AM/PM format
sorry if it doesn't.