Apply strftime on last column in jq - bash
I am processing a file and filtering out certain fields from it. Then, I am converting the whole result to a CSV file. However, the last column that I am filtering out is the epoch timestamp (upto millisecond).
Searching around, I found that I can use the strftime function in jq for conversion, however; when applying the | strftime inside the whole array, it applies the function to all the values.
TZ='Asia/Kolkata' grep $id ~/hhfh.log | grep -oE '\{.+' | jq -r '[.highPrice, .lowPrice, .openPrice, .closePrice, .volumeTradedToday, .totalBuyQuantity, .totalSellQuantity, .tickTimestamp | strftime("%B %d %Y %I:%M%p %Z")]' | jq -r 'map(tostring) | join(", ")'
Can I modify this so that strftime("%B %d %Y %I:%M%p %Z") only applies to the .tickTimestmap value? If not, Would I have to depend on awk?
Log lines from hhfh.log:
2018-03-06 03:30:04,938 DEBUG KiteTickerSourceLogic [ReadingThread] TickData: {"mode":"full","tradable":false,"instrumentToken":2997505,"lastTradedPrice":740.0,"highPrice":0.0,"lowPrice":0.0,"openPrice":731.5,"closePrice":739.9,"change":0.01351533991080183,"lastTradedQuantity":23.0,"averageTradePrice":0.0,"volumeTradedToday":12.0,"totalBuyQuantity":285.0,"totalSellQuantity":1469.0,"lastTradedTime":1520245282000,"oi":0.0,"tickTimestamp":1520307004000,"openInterestDayHigh":0.0,"openInterestDayLow":0.0,"marketDepth":{"buy":[{"quantity":1,"price":735.55,"orders":1},{"quantity":86,"price":731.5,"orders":1},{"quantity":168,"price":731.0,"orders":1},{"quantity":25,"price":730.1,"orders":1},{"quantity":0,"price":0.0,"orders":0}],"sell":[{"quantity":550,"price":743.6,"orders":1},{"quantity":550,"price":746.6,"orders":1},{"quantity":10,"price":750.0,"orders":1},{"quantity":25,"price":777.0,"orders":1},{"quantity":12,"price":-0.01,"orders":1}]}}
What I am currently generating:
January 01 1970 12:05:08AM UTC, January 01 1970 12:05:02AM UTC, January 01 1970 12:05:04AM UTC, January 01 1970 12:05:06AM UTC, January 19 1970 08:56:53AM UTC, January 01 1970 04:39:20AM UTC, January 01 1970 12:00:00AM UTC, August 31 50144 10:33:20AM UTC
What it should be like:
751.95, 734.1, 745.45, 742.8, 1659987, 6358, 0, <time in IST zone>
Simply use parentheses:
(.tickTimestamp | strftime("%B %d %Y %I:%M%p %Z"))
If you want the time relative to TZ, use strflocaltime, which however requires a more recent version of jq than 1.5.
Related
Subtract hours from date object in bash
I need to find a time X hours before the previous midnight, thus I would like to subtract X hours from a date object. Example # Finding the previous midnight date -d "yesterday 23:59:59" Mon Jul 11 00:00:00 CEST 2022 What I want I would like to find the date X hours before this midnight x=4 Mon Jul 10 20:00:00 CEST 2022
You can use this date command: date -d "today 0 -4 hours" Here: today 0: gets midnight date-time for today's date -4 hours: subtracts 4 hours from midnight time
It seems one can just write the following: date --date 'yesterday 23:59:59 CEST -4 hours'
shell date "-n hours" differs with "n hours ago" in some situation
in shell, when I print this date -d "2016-11-23 13:05 -1 hours " "+%Y-%m-%d %H:00:00" I get 2016-11-23 23:00:00.Strange! when I print this date -d "2016-11-23 13:05 1 hours ago" "+%Y-%m-%d %H:00:00" I get 2016-11-23 12:00:00. Why they are different? What I think is that they are both 2016-11-23 12:00:00.
This is because the negative number is treated as an offset to your timezone, not to the 13:05. In my timezone, MET (one hour east of GMT), this is what I get: $ date -d "2016-11-23 13:05 -1 hours " "+%Y-%m-%d %H:00:00" 2016-11-23 16:00:00 $ TZ=GMT date -d "2016-11-23 13:05 -1 hours " "+%Y-%m-%d %H:00:00" 2016-11-23 15:00:00 $ TZ=GMT-1 date -d "2016-11-23 13:05 -1 hours " "+%Y-%m-%d %H:00:00" 2016-11-23 16:00:00 $ TZ=GMT-1 date -d "2016-11-23 13:05 -2 hours " "+%Y-%m-%d %H:00:00" 2016-11-23 17:00:00 The timezone offset is usually specified as a four digit number, as in Sun, 29 Feb 2004 16:21:42 -0800 but apparently date(1) is happy with a -1 as well. From the man page: DATE STRING The --date=STRING is a mostly free format human readable date string such as "Sun, 29 Feb 2004 16:21:42 -0800" or "2004-02-29 16:21:42" or even "next Thursday". A date string may contain items indicating calendar date, time of day, time zone, day of week, relative time, relative date, and numbers. An empty string indicates the beginning of the day. The date string format is more complex than is easily documented here but is fully described in the info documentation.
How to find date n days in the future from past date?
I want to find a date that is 57 –working– days after an arbitrary date using date or bash. For instance, today is august 21st, reference date is july 15th, what days will be 57 working days after july 15th?
This should work to just get all days date -d '7/15/14 +57 days' To get number of work days (M..F) you can do something lazy like this #!/bin/bash days=0 for ((i=1;i>0;i++)) do future=$(date -d "7/15/14 +$i days" '+%w') ((future!=0 && future!=6)) && ((days++)) # ignore sunday (0) and saturday (6) ((days==57)) && date -d "7/15/14 +$i days" && break done e.g. > ./abovescript > Thu Oct 2 00:00:00 CDT 2014
Weird solution: day=21 mon=8 year=2014 days=4 curl -s "http://www.timeanddate.com/scripts/dateserver.php?mode=addweekdays&d1=$day&m1=$mon&y1=$year&type=add&ad=$days&atyp=0&ach=3" | sed -n 's|.*<h2>Result: \(.*\)</h2>.*|\1|p' prints Wednesday, August 27, 2014 the date after 4 working days from 2014.08.21 for the day=15 mon=7 year=2014 days=57 prints Friday, October 3, 2014
How to compute a relative date from a given date?
The GNU date command can output date computed from relative items such as -1 hour or 1 month ago. It can also output date from different input format (for instance calendar date, seconds since epoch, etc...). Is it possible to combine both a date format and a relative time to compute a new date from a given date using the GNU§ date utility? Something like: date_epoch=`date "+%s"` date --date="#$date_epoch -1 month" The second command gives an "incorrect date" error...
You can do it, sure: $ date -d"$(date -d#$date_epoch) -1 month" "+%Y-%m-%d" 2013-09-16 Which comes from: $ date_epoch=$(date "+%s") $ date -d#$date_epoch Wed Oct 16 23:46:50 CEST 2013 that you can use it "hardcoded": $ date -d"Wed Oct 16 23:46:50 CEST 2013 -1 month" Mon Sep 16 23:46:50 CEST 2013 Or using the variable: $ date -d"$(date -d#$date_epoch) -1 month" Mon Sep 16 23:46:50 CEST 2013
How to get epoch time in shell script (for ksh)?
How to get epoch time in shell script (for ksh)? I am interested in getting epoch time for the start of day (so e.g. now is July 28th, 2011 ~ 14:25:00 EST, I need time at midnight).
If you have GNU date, epoch=$( date -d 00:00 +%s ) Otherwise, if you have tclsh, epoch=$( echo 'puts [clock scan 00:00]' | tclsh ) Otherwise, epoch=$( perl -MTime::Local -le 'print timelocal(0,0,0,(localtime)[3..8])' )
ksh's printf '%(fmt)T' supports time calculating. For example: $ printf '%T\n' now Mon Mar 18 15:11:46 CST 2013 $ printf '%T\n' '2 days ago' Sat Mar 16 15:11:55 CST 2013 $ printf '%T\n' 'midnight today' Mon Mar 18 00:00:00 CST 2013 $