Based on the equation for ARIMA model, it includes a constant.
But the ARIMA implementation in python statsmodels library assume no trend (i.e., no constant) when d is not zero. See specifications here. Why is that?
Related
A colleague and I cannot reach a consensus in what the GBrank model (after training) should look like.
Introduction
The method starts by performing the typical conversion from pairwise to a pointwise dataset, where the target variable z represents now a score which should be higher, zi > zj when i is preferred over j. The authors then suggest using Gradient Boosting Trees, and "punish" cases where that model is predicting zj > zj and i is preferred over j. The "punishment" is performed by switching the scores and also incrementing or decrementing by τ.
Disagreement
Where we disagree is whether Gradient Boosting Rank is itself an ensembler. That is, is the model we are training gk or is it hk?
Reference Material
The original article: http://www.cc.gatech.edu/~zha/papers/fp086-zheng.pdf
h is used in the whole paper to denote the hypothesis you are working with, g is just a domain specific regression model used to construct h, thus GBRank is hk. In particular, it is a boosting method, thus it has to be an ensemble, trained by building a strong learner from set of weak learners (from definition of boosting posed by Kearns and Valiant in late '80) - h is an ensemble (due to recurrent definition), g is not (as it is just a regressor trained on some transformed dataset).
I have a piece of code in Fortran 90 in which I have to solve both a non-linear (for which I have to invert the Jacobian matrix) and a linear system of equations. When I say small I mean n unknowns for both operations, with n<=4. Unfortunately, n is not known a priori. What do you think is the best option?
I thought of writing explicit formulas for cases with n=1,2 and using other methods for n=3,4 (e.g. some functions of the Intel MKL libraries), for the sake of performance. Is this sensible or should I write explicit formulas for the inverse matrix also for n=3,4?
My questions concerns the Mixed integer programming (MIP) in Scip:
I have the following code:
$\min trace(X)$
subject to
$$(A+D)^TX+X(A+D)=I\\
d_i \in \left\{0,1\right\} \mbox{ for } i=1,\ldots,n$$
where A is a n*n matrix and $D=diag(d_1,\ldots,d_n)$ is a diagonal matrix.
Since the matrix constraints are linear the equation can be transformed to a system of linear equations (via Kronecker product and vectorize operation), but this is limited to small n. Is it possible to solve the matrix equation directly with Scip? Is there a way to embed an external solver? Or do I have to write my own solver for the continuous lyapunov matrix equation?
You could try using the pip file format used for polynomial constraints and objective. See http://polip.zib.de/ and http://polip.zib.de/pipformat.php
You would have to do the matrix operations yourself or use ZIMPL
Matrix equations cannot be handled in SCIP. You would need to transform them into linear equations. Also, all the data has to be loaded into an LP solver at some time and needs to be formulated as usual constraints here as well. So even if SCIP itself would be able to handle matrix equations you are sooner or later to required to expand the problem.
I have been trying to implement a locally-weighted logistic regression algorithm in Ruby. As far as I know, no library currently exists for this algorithm, and there is very little information available, so it's been difficult.
My main resource has been the dissertation of Dr. Kan Deng, in which he described the algorithm in what I feel is pretty light detail. My work so far on the library is here.
I've run into trouble when trying to calculate B (beta). From what I understand, B is a (1+d x 1) vector that represents the local weighting for a particular point. After that, pi (the probability of a positive output) for that point is the sigmoid function based on the B for that point. To get B, use the Newton-Raphson algorithm recursively a certain number of times, probably no more than ten.
Equation 4-4 on page 66, the Newton-Raphson algorithm itself, doesn't make sense to me. Based on my understanding of what X and W are, (x.transpose * w * x).inverse * x.transpose * w should be a (1+d x N) matrix, which doesn't match up with B, which is (1+d x 1). The only way that would work, then, is if e were a (N x 1) vector.
At the top of page 67, under the picture, though, Dr. Deng just says that e is a ratio, which doesn't make sense to me. Is e Euler's Constant, and it just so happens that that ratio is always 2.718:1, or is it something else? Either way, the explanation doesn't seem to suggest, to me, that it's a vector, which leaves me confused.
The use of pi' is also confusing to me. Equation 4-5, the derivative of the sigmoid function w.r.t. B, gives a constant multiplied by a vector, or a vector. From my understanding, though, pi' is just supposed to be a number, to be multiplied by w and form the diagonal of the weight algorithm W.
So, my two main questions here are, what is e on page 67 and is that the 1xN matrix I need, and how does pi' in equation 4-5 end up a number?
I realize that this is a difficult question to answer, so if there is a good answer then I will come back in a few days and give it a fifty point bounty. I would send an e-mail to Dr. Deng, but I haven't been able to find out what happened to him after 1997.
If anyone has any experience with this algorithm or knows of any other resources, any help would be much appreciated!
As far as I can see, this is just a version of Logistic regression in which the terms in the log-likelihood function have a multiplicative weight depending on their distance from the point you are trying to classify. I would start by getting familiar with an explanation of logistic regression, such as http://czep.net/stat/mlelr.pdf. The "e" you mention seems to be totally unconnected with Euler's constant - I think he is using e for error.
If you can call Java from Ruby, you may be able to make use of the logistic classifier in Weka described at http://weka.sourceforge.net/doc.stable/weka/classifiers/functions/Logistic.html - this says "Although original Logistic Regression does not deal with instance weights, we modify the algorithm a little bit to handle the instance weights." If nothing else, you could download it and look at its source code. If you do this, note that it is a fairly sophisticated approach - for instance, they check beforehand to see if all the points actually lie pretty much in some subspace of the input space, and project down a few dimensions if they do.
I'm in search for a free package that do most matrix/vector operations. I can write some basic functions myself but for advanced ones like computing eigenvalues and eigenvectors I would prefer robust code and I would like to know if such packages are freely available. If I understand correctly Ada 2005 have more matrix operations facilities but it has a function to calculate the eigenvalues for a symmetric and hermitian matrices only. I'll need a more general packages which can handle any kind of matrix.
An Ada95 matrix package (54KB tar.gz file) from Drexel Fusion Laboratory had the link: http://dflwww.ece.drexel.edu/research/ada/ but the page for this ink is non-existent today.
Thanks a lot...
I think that the Ada95 package you mean is here -- but it's only 35k, and it seems to have less functionality than the Ada2005 standard library does.
Not sure how this Ada95 binding to BLAS came to be in my browser cache! I see that for general matrix solving you need LAPACK too, I wonder whether the bindings already in GNAT will help? Package System.Generic_Real_LAPACK in file s-gerela.ad[bs]. The comments say
-- LAPACK Computational Routines
-- gerfs Refines the solution of a system of linear equations with
-- a general matrix and estimates its error
-- getrf Computes LU factorization of a general m-by-n matrix
-- getri Computes inverse of an LU-factored general matrix
-- square matrix, with multiple right-hand sides
-- getrs Solves a system of linear equations with an LU-factored
-- square matrix, with multiple right-hand sides
-- orgtr Generates the Float orthogonal matrix Q determined by sytrd
-- steqr Computes all eigenvalues and eigenvectors of a symmetric or
-- Hermitian matrix reduced to tridiagonal form (QR algorithm)
-- sterf Computes all eigenvalues of a Float symmetric
-- tridiagonal matrix using QR algorithm
-- sytrd Reduces a Float symmetric matrix to tridiagonal form
which I suspect is a small subset of the full library. Still, could act as a useful springboard for more extensive bindings.
As suggested in John Barnes Rationale for Ada 2005, Ada's Annex G: Numerics is not intended "as a substitute for professional libraries such as the renowned BLAS," but nothing precludes an implementation from using BLAS internally. As a concrete example, the GNAT compiler implements both G.3.1 Real Vectors and Matrices and G.3.2 Complex Vectors and Matrices using BLAS and LAPACK. To see the details, you can examine the relevant package bodies:
$ export ADA_INC = /your/path/to/adinclude
$ view $ADA_INC/$(gnatkr Ada.Numerics.Generic_Real_Arrays.adb)
$ view $ADA_INC/$(gnatkr Ada.Numerics.Generic_Complex_Arrays.adb)
The site at which this package was previously available has been migrated and the old content is now available at:
http://dfl.ece.drexel.edu/content/ada95-matrix-package