I am trying to filter my data list using D3. What I am trying to do is filter my data based on date I specify and threshold value for precipitation.
Here is my code as
$(function() {
$("#datepicker").datepicker();
$("#datepicker").on("change",function(){
//var currentDate = $( "#datepicker" ).datepicker( "getDate" )/1000;
//console.log(currentDate)
});
});
function GenerateReport() {
d3.csv("/DataTest.csv", function(data) {
var startdate = $( "#datepicker" ).datepicker( "getDate" )/1000;
var enddate = startdate + 24*60*60
var data_Date = d3.values(data.filter(function(d) { return d["Date"] >=
startdate && d["Date"] <= enddate} ))
var x = document.getElementById("threshold").value
console.log(data_Date)
var data_Date_Threshold = data_Date.filter(function(d) {return
d.Precipitation > x});
My data set looks like
ID Date Prcip Flow Stage
1010 1522281000 0 0 0
1010 1522281600 0 0 0
1010 1522285200 10 0 0
1010 1522303200 12 200 1.2
1010 1522364400 6 300 2
1010 1522371600 4 400 2.5
1010 1522364400 6 500 2.8
1010 1522371600 4 600 3.5
2120 1522281000 0 0 0
2120 1522281600 0 0 0
2120 1522285200 10 100 1
2120 1522303200 12 1000 2
2120 1522364400 6 2000 3
2120 1522371600 4 2500 3.2
2290 1522281000 0 0 0
2290 1522281600 4 0 0
2290 1522285200 5 200 1
2290 1522303200 10 800 1.5
2290 1522364400 6 1500 3
2290 1522371600 0 1000 2
6440 1522281000 0 0 0
6440 1522281600 4 0 0
6440 1522285200 5 200 0.5
6440 1522303200 10 800 1
6440 1522364400 6 1500 2
6440 1522371600 0 100 1.4
When I use filter function, I have some problems.
What I have found is that when I use x = 2 to filter precipitation value, it does not catch precipitation = 10 or 12. However, when I use x=1, it works fine. I am guessing that it catches only the first number (e.g., if x=2, it regards precipitation = 10 or 12 is less than 2 since it looks only 1 in 10 and 12) Is there anyone who had the same issue what I have? Can anyone help me to solve this problem?
Thanks.
You are comparing strings. This comparison is therefore done lexicographically.
In order to accomplish what you want, you need to first convert these strings to numbers:
var x = Number(document.getElementById("threshold").value)
var data_Date_Threshold = data_Date.filter(function(d) {return Number(d.Precipitation) > x});
Alternatively, floats:
var x = parseFloat(document.getElementById("threshold").value)
var data_Date_Threshold = data_Date.filter(function(d) {return parseFloat(d.Precipitation) > x});
I am trying to run script for astronomical algorithm VSOP2013.
but when I running this script, it displays error in line 178
how to solve it?
What's wrong with unpack function?
FYI, the original script is python2, i am using python3
a = self.fmt.unpack(terms.encode())
error: unpack requires a bytes object of length 117
This is my full python3 script, adapted from the original python2 version from http://domenicomustara.blogspot.co.id
# -*- coding: utf-8 -*-
import gmpy2 as gmp
import struct
import ctypes
gmp.get_context().precision=200
def cal2jul(year, month, day, hour=0, minute =0, second=0):
month2 = month
year2 = year
if month2 <= 2:
year2 -= 1
month2 += 12
if (year*10000 + month*100 + day) > 15821015:
a = int(year2/100)
b = 2 - a + int(a/4)
else:
a = 0
b = 0
if year < 0:
c = int((365.25 * year2)-0.75)
else:
c = int(365.25 * year2)
d = int(30.6001 *(month2 + 1))
return b + c + d + day + hour / 24.0 + minute / 1440.0 + second / 86400.0 + 1720994.5
class VSOP2013():
def __init__(self, t, planet, precision=1e-7):
# calculate millennia from J2000
self.JD = t
self.t = gmp.div((t - cal2jul(2000,1,1,12)), 365250.0)
# predefine powers of self.t
self.power = []; self.power.append(gmp.mpfr(1.0)); self.power.append(self.t)
for i in range(2,21):
t = self.power[-1]
self.power.append(gmp.mul(self.t,t))
# choose planet file in a dict
self.planet = planet
self.planets = {'Mercury':'VSOP2013p1.dat',
'Venus' :'VSOP2013p2.dat',
'EMB' :'VSOP2013p3.dat',
'Mars' :'VSOP2013p4.dat',
'Jupiter':'VSOP2013p5.dat',
'Saturn' :'VSOP2013p6.dat',
'Uranus' :'VSOP2013p7.dat',
'Neptune':'VSOP2013p8.dat',
'Pluto' :'VSOP2013p9.dat'}
# VSOP2013 routines precision
self.precision = precision
# lambda coefficients
# l(1,13) : linear part of the mean longitudes of the planets (radian).
# l(14): argument derived from TOP2013 and used for Pluto (radian).
# l(15,17) : linear part of Delaunay lunar arguments D, F, l (radian).
self.l = (
(gmp.mpfr(4.402608631669), gmp.mpfr(26087.90314068555)),
(gmp.mpfr(3.176134461576), gmp.mpfr(10213.28554743445)),
(gmp.mpfr(1.753470369433), gmp.mpfr(6283.075850353215)),
(gmp.mpfr(6.203500014141), gmp.mpfr(3340.612434145457)),
(gmp.mpfr(4.091360003050), gmp.mpfr(1731.170452721855)),
(gmp.mpfr(1.713740719173), gmp.mpfr(1704.450855027201)),
(gmp.mpfr(5.598641292287), gmp.mpfr(1428.948917844273)),
(gmp.mpfr(2.805136360408), gmp.mpfr(1364.756513629990)),
(gmp.mpfr(2.326989734620), gmp.mpfr(1361.923207632842)),
(gmp.mpfr(0.599546107035), gmp.mpfr(529.6909615623250)),
(gmp.mpfr(0.874018510107), gmp.mpfr(213.2990861084880)),
(gmp.mpfr(5.481225395663), gmp.mpfr(74.78165903077800)),
(gmp.mpfr(5.311897933164), gmp.mpfr(38.13297222612500)),
(gmp.mpfr(0.000000000000), gmp.mpfr(0.3595362285049309)),
(gmp.mpfr(5.198466400630), gmp.mpfr(77713.7714481804)),
(gmp.mpfr(1.627905136020), gmp.mpfr(84334.6615717837)),
(gmp.mpfr(2.355555638750), gmp.mpfr(83286.9142477147)))
# planetary frequencies in longitude
self.freqpla = {'Mercury' : gmp.mpfr(0.2608790314068555e5),
'Venus' : gmp.mpfr(0.1021328554743445e5),
'EMB' : gmp.mpfr(0.6283075850353215e4),
'Mars' : gmp.mpfr(0.3340612434145457e4),
'Jupiter' : gmp.mpfr(0.5296909615623250e3),
'Saturn' : gmp.mpfr(0.2132990861084880e3),
'Uranus' : gmp.mpfr(0.7478165903077800e2),
'Neptune' : gmp.mpfr(0.3813297222612500e2),
'Pluto' : gmp.mpfr(0.2533566020437000e2)}
# target variables
self.ax = gmp.mpfr(0.0) # major semiaxis
self.ml = gmp.mpfr(0.0) # mean longitude
self.kp = gmp.mpfr(0.0) # e*cos(perielium longitude)
self.hp = gmp.mpfr(0.0) # e*sin(perielium longitude)
self.qa = gmp.mpfr(0.0) # sin(inclination/2)*cos(ascending node longitude)
self.pa = gmp.mpfr(0.0) # sin(inclination/2)*cos(ascending node longitude)
self.tg_var = {'A':self.ax, 'L':self.ml, 'K':self.kp,
'H':self.hp, 'Q':self.qa, 'P':self.pa }
# eps = (23.d0+26.d0/60.d0+21.41136d0/3600.d0)*dgrad
self.eps = gmp.mpfr((23.0+26.0/60.0+21.411360/3600.0)*gmp.const_pi()/180.0)
self.phi = gmp.mpfr(-0.051880 * gmp.const_pi() / 180.0 / 3600.0)
self.ceps = gmp.cos(self.eps)
self.seps = gmp.sin(self.eps)
self.cphi = gmp.cos(self.phi)
self.sphi = gmp.sin(self.phi)
# rotation of ecliptic -> equatorial rect coords
self.rot = [[self.cphi, -self.sphi*self.ceps, self.sphi*self.seps],
[self.sphi, self.cphi*self.ceps, -self.cphi*self.seps],
[0.0, self.seps, self.ceps ]]
self.fmt = struct.Struct("""6s 3s 3s 3s 3s x 3s 3s 3s 3s 3s x 4s 4s 4s 4s x
6s x 3s 3s 3s 20s x 3s 20s x 3s x""")
self.gmp_ = {
'Mercury' : gmp.mpfr(4.9125474514508118699e-11),
'Venus' : gmp.mpfr(7.2434524861627027000e-10),
'EMB' : gmp.mpfr(8.9970116036316091182e-10),
'Mars' : gmp.mpfr(9.5495351057792580598e-11),
'Jupiter' : gmp.mpfr(2.8253458420837780000e-07),
'Saturn' : gmp.mpfr(8.4597151856806587398e-08),
'Uranus' : gmp.mpfr(1.2920249167819693900e-08),
'Neptune' : gmp.mpfr(1.5243589007842762800e-08),
'Pluto' : gmp.mpfr(2.1886997654259696800e-12)}
self.gmsol = gmp.mpfr(2.9591220836841438269e-04)
self.rgm = gmp.sqrt(self.gmp_[self.planet]+self.gmsol)
# run calculus routine
self.calc()
def __str__(self):
vsop_out = "{:3.13} {:3.13} {:3.13} {:3.13} {:3.13} {:3.13}\n".format(
self.tg_var['A'],
self.tg_var['L'],
self.tg_var['K'],
self.tg_var['H'],
self.tg_var['Q'],
self.tg_var['P'])
vsop_out += "{:3.13} {:3.13} {:3.13} {:3.13} {:3.13} {:3.13}\n".format(
self.ecl[0],
self.ecl[1],
self.ecl[2],
self.ecl[3],
self.ecl[4],
self.ecl[5])
vsop_out += "{:3.13} {:3.13} {:3.13} {:3.13} {:3.13} {:3.13}\n".format(
self.equat[0],
self.equat[1],
self.equat[2],
self.equat[3],
self.equat[4],
self.equat[5])
return vsop_out
def calc(self):
with open(self.planets[self.planet]) as file_in:
terms = []
b = '*'
while b != '':
b = file_in.readline()
if b != '':
if b[:5] == ' VSOP':
header = b.split()
#print header[3], header[7], header[8], self.t**int(header[3])
no_terms = int(header[4])
for i in range(no_terms):
#6x,4i3,1x,5i3,1x,4i4,1x,i6,1x,3i3,2a24
terms = file_in.readline()
# print('terms',terms)
a = self.fmt.unpack(terms.encode())
S = gmp.mul(gmp.mpfr(a[18]),gmp.exp10(int(a[19])))
C = gmp.mul(gmp.mpfr(a[20]),gmp.exp10(int(a[21])))
if gmp.sqrt(S*S+C*C) < self.precision:
break
aa = 0.0; bb = 0.0;
for j in range(1,18):
aa += gmp.mul(gmp.mpfr(a[j]), self.l[j-1][0])
bb += gmp.mul(gmp.mpfr(a[j]), self.l[j-1][1])
arg = aa + bb * self.t
power = int(header[3])
comp = self.power[power] * (S * gmp.sin(arg) + C * gmp.cos(arg))
if header[7] == 'L' and power == 1 and int(a[0]) == 1:
pass
else:
self.tg_var[header[7]] += comp
self.tg_var['L'] = self.tg_var['L'] + self.t * self.freqpla[self.planet]
self.tg_var['L'] = self.tg_var['L'] % (2 * gmp.const_pi())
if self.tg_var['L'] < 0:
self.tg_var['L'] += 2*gmp.const_pi()
print ("Julian date {}".format(self.JD))
file_in.close()
##print self.tg_var
#### def ELLXYZ(self):
xa = self.tg_var['A']
xl = self.tg_var['L']
xk = self.tg_var['K']
xh = self.tg_var['H']
xq = self.tg_var['Q']
xp = self.tg_var['P']
# Computation
xfi = gmp.sqrt(1.0 -xk * xk - xh * xh)
xki = gmp.sqrt(1.0 -xq * xq - xp * xp)
u = 1.0/(1.0 + xfi)
z = complex(xk, xh)
ex = abs(z)
ex2 = ex * ex
ex3 = ex2 * ex
z1 = z.conjugate()
#
gl = xl % (2*gmp.const_pi())
gm = gl - gmp.atan2(xh, xk)
e = gl + (ex - 0.1250 * ex3) * gmp.sin(gm)
e += 0.50 * ex2 * gmp.sin(2.0 * gm)
e += 0.3750 * ex3 * gmp.sin(3.0 * gm)
#
while True:
z2 = complex(0.0, e)
zteta = gmp.exp(z2)
z3 = z1 * zteta
dl = gl - e + z3.imag
rsa = 1.0 - z3.real
e = e + dl / rsa
if abs(dl) < 1e-15:
break
#
z1 = u * z * z3.imag
z2 = gmp.mpc(z1.imag, -z1.real)
zto = (-z+zteta+z2)/rsa
xcw = zto.real
xsw = zto.imag
xm = xp * xcw - xq * xsw
xr = xa * rsa
#
self.ecl = []; self.equ = {}
self.ecl.append(xr * (xcw -2.0 *xp * xm))
self.ecl.append(xr * (xsw +2.0 *xq * xm))
self.ecl.append(-2.0 * xr * xki * xm)
#
xms = xa *(xh + xsw) / xfi
xmc = xa *(xk + xcw) / xfi
xn = self.rgm / xa ** (1.50)
#
self.ecl.append( xn *((2.0 * xp * xp - 1.0) * xms + 2.0 * xp * xq * xmc))
self.ecl.append( xn *((1.0 -2.0 * xq * xq) * xmc -2.0 * xp * xq * xms))
self.ecl.append( 2.0 * xn * xki * (xp * xms + xq * xmc))
# Equatorial rectangular coordinates and velocity
#
#
# --- Computation ------------------------------------------------------
#
self.equat = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
for i in range(3):
for j in range(3):
self.equat[i] = self.equat[i] + self.rot[i][j] * self.ecl[j]
self.equat[i+3] = self.equat[i+3] + self.rot[i][j] * self.ecl[j+3]
if __name__ == '__main__':
for planet in ('Mercury', 'Venus', 'EMB', 'Mars', 'Jupiter',
'Saturn', 'Uranus', 'Neptune', 'Pluto'):
print ("PLANETARY EPHEMERIS VSOP2013 "+ planet + "(TDB)\n"+"""
1/ Elliptic Elements: a (au), lambda (radian), k, h, q, p - Dynamical Frame J2000
2/ Ecliptic Heliocentric Coordinates: X,Y,Z (au) X',Y',Z' (au/d) - Dynamical Frame J2000
3/ Equatorial Heliocentric Coordinates: X,Y,Z (au) X',Y',Z' (au/d) - ICRS Frame J2000
""")
init_date = cal2jul(1890,6,26,12)
set_date = init_date
while set_date < init_date + 41000:
v = VSOP2013(set_date, planet)
print (v)
set_date += 4000
Respond comment from Mr.barrycarter,
this is a bit of the result when i print(terms.encode()):
b' 2 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0.6684459764580090 -07 0.3603178002233933 -06\n'
b' 3 0 2 -4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.2383757728520679 -07 0.9861749707454420 -07\n'
b' 4 0 4 -6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -0.2193692495097233 -07 -0.8959173003201546 -07\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 0.1017891898227051 -03\n'
b' 2 0 3 -5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.4236543085970792 -07 -0.8775084424897674 -08\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 0.4702795245810685 -04\n'
b' 2 0 3 -5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -0.5710471800210820 -09 -0.1800837750117577 -08\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 -0.5421827377115325 -06\n'
b' 2 0 3 -5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -0.7074507338012408 -10 0.1742474656298139 -10\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 -0.2508633795522544 -07\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 0.4575014479216901 -09\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 0.5208591612817609 -11\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 -0.1737141639583644 -12'
I have vector y_vec, How to convert the vector to a matrix of form Y_matrix
y_vec = [0; 1; 1; 2; 3; 4]
Y_matrix = [1 0 0 0 0
0 1 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1]
So far, I've tried using a for loop.
Y_mat = full(spzeros(length(y_vec), length(unique(y_vec))))
for (i,j) in enumerate(1:length(y_vec))
Y_mat[i, y_vec[j]+1] = 1
end
But, there seems to be a problem when y_vec is not continuous, say y_vec = [0; 1; 1; 2; 3; 4; 8], using for loop fails !!! How to get around this issue.
Is there a way to solve the above problem using sparse matrix in Julia.
you can use sparse matrix constructor sparse(I,J,V):
y_vec = [0; 1; 1; 2; 3; 4; 8]
I = collect(1:length(y_vec))
J = y_vec+1
V = ones(length(y_vec))
S = sparse(I,J,V)
full(S)
julia> full(S)
7x9 Array{Float64,2}:
1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
I have this data
y x1 x2 pre
1 16 1 1 14
2 15 1 1 13
3 14 1 2 14
4 13 1 2 13
5 12 2 1 12
6 11 2 1 12
7 11 2 2 13
8 13 2 2 13
9 10 3 1 10
10 11 3 1 11
11 11 3 2 11
12 9 3 2 10
And I fitted the following model
lm(y ~ x1 + x2 + x1*x2)
My design matrix is
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,] 1 14 1 0 1 1 0
[2,] 1 13 1 0 1 1 0
[3,] 1 14 1 0 0 0 0
[4,] 1 13 1 0 0 0 0
[5,] 1 12 0 1 1 0 1
[6,] 1 12 0 1 1 0 1
[7,] 1 13 0 1 0 0 0
[8,] 1 13 0 1 0 0 0
[9,] 1 10 0 0 1 0 0
[10,] 1 11 0 0 1 0 0
[11,] 1 11 0 0 0 0 0
[12,] 1 10 0 0 0 0 0
I'm trying to use this design to reproduce the following table:
Source DF Squares Mean Square F Value Pr > F
Model 6 44.79166667 7.46527778 12.98 0.0064
Error 5 2.87500000 0.57500000
Corrected Total 11 47.66666667
Source DF Type III SS Mean Square F Value Pr > F
pre 1 3.12500000 3.12500000 5.43 0.0671
x1 2 4.58064516 2.29032258 3.98 0.0923
x2 1 3.01785714 3.01785714 5.25 0.0706
x1*x2 2 1.25000000 0.62500000 1.09 0.4055
The first part is fine
XtX <- t(x) %*% x
XtXinv <- solve(XtX)
betahat <- XtXinv %*% t(x) %*% y
H <- x %*% XtXinv %*% t(x)
IH <- (diag(1,12) - H)
yhat <- H %*% y
e <- IH %*% y
ybar <- mean(y)
MSS <- t(betahat) %*% t(x) %*% y - length(y)*(ybar^2)
ESS <- t(e) %*% e
TSS <- MSS + ESS
dfM <- sum(diag(H)) - 1
dfE <- sum(diag(IH))
dfT <- dfM + dfE
MSM <- MSS/dfM
MSE <- ESS/dfE
Ftest <- MSM / MSE
pr <- 1 - pf(Ftest, dfM, dfE)
The contrast coefficient matrix for 'pre' seems correct.
L <- matrix(c(0,1,0,0,0,0,0), 1, 7, byrow=T)
Lb <- L %*% betahat
LXtXinvLt <- round(L %*% XtXinv %*% t(L), digits=4)
SSpre <- t(Lb) %*% solve(LXtXinvLt) %*% (Lb)
MSpre <- SSpre / 1
Fpre <- MSpre / MSE
PRpre <- 1 - pf(Fpre, 1, 12-7)
But I can't understand how to define the contrast coefficient matrix for x1, x2, and x1*x2. What's the problem with the rest of my code? Below an example for how I think I should calculate for x1
L <- matrix(c(0,0,1,1,0,0,0), 1, 7, byrow=T)
Lb <- L %*% betahat
LXtXinvLt <- round(L %*% XtXinv %*% t(L), digits=4)
SSX1 <- t(Lb) %*% solve(LXtXinvLt) %*% (Lb)
MSX1 <- SSX1 / 1
FX1 <- MSX1 / MSE
PRX1 <- 1 - pf(FX1, 1, 12-7)
Thanks!