calculating some seasonal climate metrics with Iris - python-iris

I have a new project on, calculating some seasonal climate metrics. As part of this, I need to identify, eg the wettest quarter in a set of climatological monthly data:
print(pr_cube)
Precipitation / (mm) (time: 12; latitude: 125; longitude: 211)
Dimension coordinates:
time x - -
latitude - x -
longitude - - x
where time is every month, averaged across 30-years with coord('time) =
DimCoord([2030-01-01 00:00:00, 2030-02-01 00:00:00, 2030-03-01 00:00:00,
2030-04-01 00:00:00, 2030-05-01 00:00:00, 2030-06-01 00:00:00,
2030-07-01 00:00:00, 2030-08-01 00:00:00, 2030-09-01 00:00:00,
2030-10-01 00:00:00, 2030-11-01 00:00:00, 2030-12-01 00:00:00]
I was wondering if I could add a seasons coordinate for all sets of consecutive 3 months, including 'wrapping around', something like this:
iris.coord_categorisation.add_season(cube, coord, name='season',
seasons=(''jfm', 'fma', 'mam', 'amj', 'mjj', 'jja', 'jas', 'aso', 'son', 'ond', 'ndj', 'djf'))
or
season = ('jfm', 'fma', 'mam', 'amj', 'mjj', 'jja', 'jas', 'aso', 'son', 'ond', 'ndj', 'djf')
iris.coord_categorisation.add_season_membership(cube, coord, season, name='all_quarters')
Not tested this yet, just wondered if about suggestions or a recommendation?
And then, get the season with the max rainfall?
Qtr_max_rain = pr_cube.collapsed('season', iris.analysis.MAX)
Would that work correctly ?

There may be a way to achieve this using coord_categorisation, but I believe the simplest way is to instead use iris.cube.Cube.rolling_window(). There's no native way to wrap around in the way you need, so you can hack it by duplicating Jan and Feb on the end of the existing data.
I've tested the below and it seems to work as intended. Hopefully it works for you.
# Create extra cube based on Jan and Feb from pr_cube.
extra_months_cube = pr_cube[:2, ...]
# Replace time coordinate with another that is advanced by a year - ensures correct sorting.
# Adjust addition depending on the unit of the time coordinate.
extra_months_coord = extra_months_cube.coord("time") + (24 * 365)
extra_months_cube.remove_coord("time")
extra_months_cube.add_dim_coord(extra_months_coord, 0)
# Combine original cube with extra cube.
both_cubes = iris.cube.CubeList([pr_cube, extra_months_cube])
fourteen_month_cube = both_cubes.concatenate_cube()
# Generate cube of 3-month MAX aggregations.
rolling_cube = fourteen_month_cube.rolling_window("time", iris.analysis.MAX, 3)
Once done, you would of course be free to add your suggested three month labels using iris.cube.Cube.add_aux_coord().

Related

Dynamic number system in Qlik Sense

My data consists of large numbers, I have a column say - 'amount', while using it in charts(sum of amount in Y axis) it shows something like 1.4G, I want to show them as if is billion then e.g. - 2.8B, or in millions then 80M or if it's in thousands (14,000) then simply- 14k.
I have used - if(sum(amount)/1000000000 > 1, Num(sum(amount)/1000000000, '#,###B'), Num(sum(amount)/1000000, '#,###M')) but it does not show the M or B at the end of the figure and also How to include thousand in the same code.
EDIT: Updated to include the dual() function.
This worked for me:
=dual(
if(sum(amount) < 1, Num(sum(amount), '#,##0.00'),
if(sum(amount) < 1000, Num(sum(amount), '#,##0'),
if(sum(amount) < 1000000, Num(sum(amount)/1000, '#,##0k'),
if(sum(amount) < 1000000000, Num(sum(amount)/1000000, '#,##0M'),
Num(sum(amount)/1000000000, '#,##0B')
))))
, sum(amount)
)
Here are some example outputs using this script to format it:
=sum(amount)
Formatted
2,526,163,764
3B
79,342,364
79M
5,589,255
5M
947,470
947k
583
583
0.6434
0.64
To get more decimals for any of those, like 2.53B instead of 3B, you can format them like '#,##0.00B' by adding more zeroes at the end.
Also make sure that the Number Formatting property is set to Auto or Measure expression.

Tibco Spotfire - time in seconds & milliseconds in Real, convert to a time of day

I have a list of time in a decimal format of seconds, and I know what time the series started. I would like to convert it to a time of day with the offset of the start time applied. There must be a simple way to do this that I am really missing!
Sample source data:
\Name of source file : 260521-11_58
\Recording from 26.05.2021 11:58
\Channels : 1
\Scan rate : 101 ms = 0.101 sec
\Variable 1: n1(rpm)
\Internal identifier: 63
\Information1:
\Information2:
\Information3:
\Information4:
0.00000 3722.35645
0.10100 3751.06445
0.20200 1868.33350
0.30300 1868.36487
0.40400 3722.39355
0.50500 3722.51831
0.60600 3722.50464
0.70700 3722.32446
0.80800 3722.34277
0.90900 3722.47729
1.01000 3722.74048
1.11100 3722.66650
1.21200 3722.39355
1.31300 3751.02710
1.41400 1868.27539
1.51500 3722.49097
1.61600 3750.93286
1.71700 1868.30334
1.81800 3722.29224
The Start time & date is 26.05.2021 11:58, and the LH column is elapsed time in seconds with the column name [Time] . So I just want to convert the decimal / real to a time or timespan and add the start time to it.
I have tried lots of ways that are really hacky, and ultimately flawed - the below works, but just ignores the milliseconds.
TimeSpan(0,0,0,Integer(Floor([Time])),[Time] - Integer(Floor([Time])))
The last part works to just get milli / micro seconds on its own, but not as part of the above.
Your formula isn't really ignoring the milliseconds, you are using the decimal part of your time (in seconds) as milliseconds, so the value being returned is smaller than the format mask.
You need to convert the seconds to milliseconds, so something like this should work
TimeSpan(0,0,0,Integer(Floor([Time])),([Time] - Integer(Floor([Time]))) * 1000)
To add it to the time, this would work
DateAdd(Date("26-May-2021"),TimeSpan(0,0,0,Integer([Time]),([Time] - Integer([Time])) * 1000))
You will need to set the column format to
dd-MMM-yyyy HH:mm:ss:fff

Ruby time subtraction

There is the following task: I need to get minutes between one time and another one: for example, between "8:15" and "7:45". I have the following code:
(Time.parse("8:15") - Time.parse("7:45")).minute
But I get result as "108000.0 seconds".
How can I fix it?
The result you get back is a float of the number of seconds not a Time object. So to get the number of minutes and seconds between the two times:
require 'time'
t1 = Time.parse("8:15")
t2 = Time.parse("7:45")
total_seconds = (t1 - t2) # => 1800.0
minutes = (total_seconds / 60).floor # => 30
seconds = total_seconds.to_i % 60 # => 0
puts "difference is #{minutes} minute(s) and #{seconds} second(s)"
Using floor and modulus (%) allows you to split up the minutes and seconds so it's more human readable, rather than having '6.57 minutes'
You can avoid weird time parsing gotchas (Daylight Saving, running the code around midnight) by simply doing some math on the hours and minutes instead of parsing them into Time objects. Something along these lines (I'd verify the math with tests):
one = "8:15"
two = "7:45"
h1, m1 = one.split(":").map(&:to_i)
h2, m2 = two.split(":").map(&:to_i)
puts (h1 - h2) * 60 + m1 - m2
If you do want to take Daylight Saving into account (e.g. you sometimes want an extra hour added or subtracted depending on today's date) then you will need to involve Time, of course.
Time subtraction returns the value in seconds. So divide by 60 to get the answer in minutes:
=> (Time.parse("8:15") - Time.parse("7:45")) / 60
#> 30.0

Smaller variation between times of different days

I have working on a algorithm that select a set of date/time objects with a certain characteristic, but with no success.
The data to be used are in a list of lists of date/time objects,
e.g.:
lstDays[i][j], i <= day chooser, j <= time chooser
What is the problem? I need a set of nearest date/time objects. Each time of this set must come from different days.
For example: [2012-09-09 12:00,2012-09-10 12:00, 2012-09-11 12:00]
This example of a set of date/time objects is the best example because it minimize to zero.
Important
Trying to contextualize this: I want to observe if a phenomenon occurs at the same time in differents days. If not, I want to evaluate if distance between the hours is reasonable for my study.
I would like a generic algorithm to any number of days and time. This algorithm should return all set of datetime objects and its time distance:
[2012-09-09 12:00,2012-09-10 12:00, 2012-09-11 12:00], 0
[2012-09-09 13:00,2012-09-10 13:00, 2012-09-11 13:05], 5
and so on.
:: "0", because the diff between all times on the first line from datetime objects is zero seconds.
:: "5", because the diff between all times on the second line from datetime objects is five seconds.
Edit: Code here
for i in range(len(lstDays)):
for j in range(len(lstDays[i])):
print lstDays[i][j]
Output:
2013-07-18 11:16:00
2013-07-18 12:02:00
2013-07-18 12:39:00
2013-07-18 13:14:00
2013-07-18 13:50:00
2013-07-19 11:30:00
2013-07-19 12:00:00
2013-07-19 12:46:00
2013-07-19 13:19:00
2013-07-22 11:36:00
2013-07-22 12:21:00
2013-07-22 12:48:00
2013-07-22 13:26:00
2013-07-23 11:18:00
2013-07-23 11:48:00
2013-07-23 12:30:00
2013-07-23 13:12:00
2013-07-24 11:18:00
2013-07-24 11:42:00
2013-07-24 12:20:00
2013-07-24 12:52:00
2013-07-24 13:29:00
Note: lstDays[i][j] is a datetime object.
lstDays = [ [/*datetime objects from a day i*/], [/*datetime objects from a day i+1*/], [/*datetime objects from a day i+2/*], ... ]
And I am not worried with perfomance, a priori.
Hope that you can help me! (:
Generate a histogram:
hours = [0] * 24
for object in objects: # whatever your objects are
# assuming object.date_time looks like '2013-07-18 10:55:00'
hour = object.date_time[11:13] # assuming the hour is in positions 11-12
hours[int(hour)] += 1
for hour in xrange(24):
print '%02d: %d' % (hour, hours[hour])
You can always resort to calculating the times into a list, then estimate the differences, and group those objects that are below that limit. All packed into a dictionary with the difference as the value and the the timestamps as keys. If this is not exactly what you need, I'm pretty sure it should be easy to select whatever result you need from it.
import numpy
import datetime
times_list = [object1.time(), object2(), ..., objectN()]
limit = 5 # limit of five seconds
groups = {}
for time in times_list:
delta_times = numpy.asarray([(tt-time).total_seconds() for tt in times_list])
whr = numpy.where(abs(delta_times) < limit)[0]
similar = [str(times_list[ii]) for ii in whr]
if len(similar) > 1:
similar.sort()
max_time = numpy.max(delta_times[whr]) # max? median? mean?
groups[tuple(similar)] = max_time

Using the Ruby Date class for Astronomical data

~ Approximate Solar Noon
lw = 88.743 # my longitude
jdate = Date.ordinal_to_jd(Time.now.year, Time.now.yday)
n = (jdate - 2451545 - 0.0009 - lw / 360).round # lw is users longitude west of 0.
j_noon = 2451545 + 0.0009 + lw / 360 + n
puts j_noon
=> 2455616.24740833
As an update, part of the confusion would be that solar noon is where all
calculations started since January 1, 4713 BC Greenwich noon.
The correct use of Date.ordinal_to_jd has not compensated for this fact. So by
adding or subtracting 12 hours like this:
jdn = Date.ordinal_to_jd(Time.now.year, Time.now.yday) - 0.5
we should get less errors. Just which do we use though since our calculations
start with yesterdays noon?
The code is derived from the two equations from this page Sunrise_equation.
The first answer I got from a user here was that we don't understand the use of
0.0009 and lw / 360. lw / 360 would appear to be a fractional day of arc from the
prime meridian. As for the 0.0009, it must be a small amount of variance in
seconds since January 1, 4713 BC Greenwich noon. see IAU standards for more info
I calculate it to be 0.007776 seconds according to this page.
I have a little bit of info from Date class not including method details.
=begin
--------------------------------------------------------------------- Class: Date
Class representing a date.
See the documentation to the file date.rb for an overview.
Internally, the date is represented as an Astronomical Julian Day Number, ajd.
The Day of Calendar Reform, sg, is also stored, for conversions to other date formats.
(There is also an of field for a time zone offset,
but this is only for the use of the DateTime subclass.)
A new Date object is created using one of the object creation class methods named
after the corresponding date format, and the arguments appropriate to that date
format; for instance, Date::civil()
(aliased to Date::new()) with year, month, and day-of-month, or Date::ordinal() with
year and day-of-year.
All of these object creation class methods also take the Day of Calendar Reform as an
optional argument.
Date objects are immutable once created.
Once a Date has been created, date values can be retrieved for the different date
formats supported using instance methods. For instance, #mon() gives the Civil month,
#cwday() gives the Commercial day of the week, and #yday() gives the Ordinal day of
the year. Date values can be retrieved in any format, regardless of what format was
used to create the Date instance.
The Date class includes the Comparable module, allowing date objects to be compared
and sorted, ranges of dates to be created, and so forth.
---------------------------------------------------------------------------------
Includes:
Comparable(<, <=, ==, >, >=, between?)
Constants:
MONTHNAMES: [nil] + %w(January February March April May June July August
September October November December)
DAYNAMES: %w(Sunday Monday Tuesday Wednesday Thursday Friday Saturday)
ABBR_MONTHNAMES: [nil] + %w(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec)
ABBR_DAYNAMES: %w(Sun Mon Tue Wed Thu Fri Sat)
ITALY: 2299161
ENGLAND: 2361222
JULIAN: Infinity.new
GREGORIAN: -Infinity.new
Class methods:
_load, _parse, _strptime, ajd_to_amjd, ajd_to_jd, amjd_to_ajd, civil, civil_to_jd,
commercial, commercial_to_jd, day_fraction_to_time, gregorian?, gregorian_leap?, jd,
jd_to_ajd, jd_to_civil, jd_to_commercial, jd_to_ld, jd_to_mjd, jd_to_ordinal,
jd_to_wday, julian?, julian_leap?, ld_to_jd, mjd_to_jd, new, now, ordinal,
ordinal_to_jd, parse, s3e, strptime, time_to_day_fraction, today, valid_civil?,
valid_commercial?, valid_jd?, valid_ordinal?, valid_time?
Instance methods:
+, -, <<, <=>, ===, >>, _dump, ajd, amjd, asctime, civil, commercial, ctime, cwday,
cweek, cwyear, day, day_fraction, downto, england, eql?, gregorian, gregorian?, hash,
hour, inspect, italy, jd, julian, julian?, ld, leap?, mday, min, mjd, mon, month,
new_offset, new_start, next, next_day, offset, ordinal, sec, sec_fraction, start,
step, strftime, succ, time, to_s, to_yaml, upto, wday, weeknum0, weeknum1, wnum0,
wnum1, yday, year, zone
=end
As a side note, it's great that Ruby has a way to calculate the julian-date.
I'm looking into the Javascript code from NOAA.
Here is a class that I was inspired to write by the link.
class JulianDayNumber
def initialize(year = 2000, month = 1, day = 1) #defaults to Jan. 01, 2000
#year = year
#month = month
#day = day
end
def calcJDN
if (#month <= 2) then
#year -= 1
#month += 12
end
varA = (#year/100).floor
varB = 2 - varA + (varA/4).floor
jdn = (365.25*(#year + 4716)).floor \
+ (30.6001*(#month+1)).floor \
+ #day + varB - 1524.5
return jdn
end
end
jd = JulianDayNumber.new(2011, 3, 2)
julianday = jd.calcJDN
puts julianday
=> 2455622.5
Now this gets me there but I'm still researching for the way back for a number such
as the one calculated by the top most equation. Trying this we can see that we do
get a 0.5 in the JDN. Who is right? Ruby or NOAA?
NOAA uses the January 1st 2000 value of 2451545.0 that is subtracted from the jd to get time
in fractional century like this
def calcTimeJulianCent(j)
t = (j - 2451545.0)/36525.0
return t
end
Ruby has a number of ways of calculating Julian Day and you need to pick the right one. NOAA is calculating the JD since January 1, 4713 BC Greenwich noon as you know. It always ends in .5 because they are leaving out the fractional days.
Ruby's Julian Day is weird:
For scientific purposes, it is
convenient to refer to a date simply
as a day count, counting from an
arbitrary initial day. The date first
chosen for this was January 1, 4713
BCE. A count of days from this date is
the Julian Day Number or Julian Date,
which is abbreviated as jd in the Date
class. This is in local time, and
counts from midnight on the initial
day.
Which makes no sense for astronomical use. but wait..
The stricter usage is in UTC, and
counts from midday on the initial day.
This is referred to in the Date class
as the Astronomical Julian Day Number,
and abbreviated as ajd. In the Date
class, the Astronomical Julian Day
Number includes fractional days.
(rubydoc)
This is what you are looking for, ajd. Just get it without the fractional days:
julianday = Date.civil(#year, #month, #day).ajd
puts julianday
=> 2455622.5
No need to port 9 lines of JavaScript from NOAA. Ruby's got your back! ;)
Well thanks everybody, I guess that I can answer my own question now. I overlooked a simple method in the Date class. It is Date.day_fraction_to_time(day fractional). As I have a working program now I would like to share it with eveyone.
include Math
to_r = PI / 180.0
to_d = 180.0 / PI
latitude = 41.9478 # my latitude
longitude = 88.74277 # my longitude
lw = longitude / 360
jdate = Date.civil(Time.now.year, Time.now.month, Time.now.day).ajd
jdate = (jdate * 2).to_i/2 + 1
n = (jdate - 2451545 - 0.0009 - lw).round
j_noon = 2451545 + 0.0009 + lw + n
mean_anomaly = (357.52911 + 0.98560028 * (jdate - 2451545)) % 360
center = 1.9148 * sin(mean_anomaly * to_r) + 0.0200 * sin(2 * mean_anomaly * to_r) + \
0.0003 * sin(3 * mean_anomaly * to_r)
lambda = (mean_anomaly + 102.9372 + center + 180) % 360
j_transit = j_noon + (0.0053 * sin(mean_anomaly * to_r)) - (0.0069 * sin(2 * lambda * \
to_r))
delta = asin(0.397753054 * sin(lambda * to_r)) * to_d
omega = acos(sin(-0.83 * to_r)/cos(latitude * to_r) * cos(delta * to_r) \
- tan(latitude * to_r) * tan(delta * to_r)) * to_d
j_set = 2451545 + 0.0009 + ((omega + longitude)/360 + n + 0.0053 * sin(mean_anomaly * \
to_r)) - 0.0069 * sin(2 * lambda * to_r)
j_rise = j_transit - (j_set - j_transit)
rise = Date.day_fraction_to_time(j_rise - jdate)# + 0.25 for + 6 hours
risehour = rise[0].to_s
risemin = rise[1].to_s
risetime = "#{risehour}:#{risemin}"
puts "Sun rise = #{risetime} UTC"
transit = Date.day_fraction_to_time(j_transit - jdate)# + 0.25
transithour = transit[0].to_s
transitmin = transit[1].to_s
transittime = "#{transithour}:#{transitmin}"
puts "Solar noon = #{transittime} UTC"
set = Date.day_fraction_to_time(j_set - jdate)# + 0.25
sethour = set[0].to_s
setmin = set[1].to_s
settime = "#{sethour}:#{setmin} UTC"
puts "Sun set = #{settime}"
The method ordinal_to_jd converts the day with index 0 of the year 2011 (Gregorian calendar) to the corresponding day in the Julian calendar, then you are using the magical value of 0.0009 for which i dont know any reason, then you are adding the ratio of your longitude (east or west?) of the whole 360* circle and then adding todays day-of-year (54 if you evaluated it today). The combination of Julian calendar and longitudinal ratio makes not much sense, but hey its a nice number since you mixed a 0.0009 in.

Resources