I have a square matrix that is largely zeros but interspersed with values. Is there a way to 'solve' this matrix so that all the information in it is contained in its lower triangle only, with the upper triangle containing only zeros?
Not in general.
If symmetric and positive definite you can do a Cholesky Decomposition.
If non-symmetric you can do an LU decomposition.
The Q matrix in quadratic forms (x'Qx) can be made symmetric and then lower-triangular. This is sometimes used when setting up a Quadratic Programming (QP) model.
Sparse versions of decomposition approaches are a well-studied area (not trivial though). In large-scale LP solvers sparse LU (simplex) or sparse Cholesky (interior point) are widely used.
Related
Eigen provides methods for both sparse matrices in Eigen/Sparse and dense matrices in Eigen/Dennse.
My questions are:
What is the motivation for the Sparse module? My feeling is that saving a sparse matrix in a sparse form can save space and avoid useless computations for very large matrices. Are there more reasons?
Are the underlying algorithms the same for both dense and sparse matrices? For example, the algorithm to calculate a singular value decomposition of a matrix.
Does Eigen try to identify special types of a sparse matrix, for example band diagonal, and apply specialized algorithms?
Can a 2D transpose convolution operation be represented as a matrix multiplication with the Toeplitz matrix, as can be done for a normal convolution?
I want to generalise some ideas from a dense network to a convolutional network. For normal convolutions, this is not a problem, as they can be represented as matrix multiplications with the Toeplitz matrix. But I couldn't find a clear mathematical formulation of transposed convolution, so I am not sure about this case.
I was looking for a mathematical answer, so should have probably asked somewhere else, anyways I think my latex write-up is correct and answers the question:
formula transposed convolution
In most graphics libraries I've seen, there's some function that returns the determinant from 3x3 and 4x4 matrices, but I have no idea when you'd actually need to use the determinant in 3D computer graphics.
What are some examples of using a determinant in 3D graphics programming?
Off the top of my head...
If the determinant is 0 then the matrix cannot be inverted, which can be useful to know.
If the determinant is negative, then objects transformed by the matrix will reversed as if in a mirror (left handedness becomes right handedness and vice-versa)
For 3x3 matrices, the volume of an object will be multiplied by the determinant when it is transformed by the matrix. Knowing this could be useful for determining, for example, the level of detail / number of polygons to use when rendering an object.
In 3D vector graphics
there are used 4x4 homogenuous transform matrices and we need booth direct and inverse matrices which can be computed by (sub)determinants. But for orthogonal matrices there are faster and more accurate methods like
full pseudo inverse matrix.
Many intersection tests use determinants (or can be converted to use them) especially for quadratic equations (ellipsoids,...) for example:
ray and ellipsoid intersection accuracy improvement
as Matt Timmermans suggested you can decide if your matrix is invertible or left/right handed which is useful to detect errors in matrices (accuracy degradation) or porting skeletons in between formats or engines etc.
And I am sure there area lot of other uses for it in vector math (IIRC IGES use them for rotational surfaces, cross product is determinant,...)
The incircle test is a key primitive for computing Voronoi diagrams and Delaunay triangulations. It is given by the sign of a 4x4 determinant.
(picture from https://www.cs.cmu.edu/~quake/robust.html)
There are several methods available for computing SVD of a general matrix. I am interested to know about the best approach which could be used for computing SVD of an upper triangular matrix. Please suggest me an algorithm which could be optimized for this special case of matrices.
What are eigen values, vectors and expansions and as an algorithm designer how can I use them?
EDIT: I want to know how YOU have used it in your program so that I get an idea. Thanks.
they're used for a lot more than matrix algebra. examples include:
the asymptotic state distribution of a hidden markov model is given by the left eigenvector associated with the eigenvalue of unity from the state transition matrix.
one of the best & fastest methods of finding community structure in a network is to construct what's called the modularity matrix (which basically is how "surprising" is a connection between two nodes), and then the signs of the elements of the eigenvector associated with the largest eigenvalue tell you how to partition the network into two communities
in principle component analysis you essentially select the eigenvectors associated with the k largest eigenvalues from the n>=k dimensional covariance matrix of your data and project your data down to the k dimensional subspace. the use of the largest eigenvalues ensures that you're retaining the dimensions that are most significant to the data, since they are the ones that have the greatest variance.
many methods of image recognition (e.g. facial recognition) rely on building an eigenbasis from known data (a large set of faces) and seeing how difficult it is to reconstruct a target image using the eigenbasis -- if it's easy, then the target image is likely to be from the set the eigenbasis describes (i.e. eigenfaces easily reconstruct faces, but not cars).
if you're in to scientific computing, the eigenvectors of a quantum hamiltonian are those states that are stable, in that if a system is in an eigenstate at time t1, then at time t2>t1, if it hasn't been disturbed, it will still be in that eigenstate. also, the eigenvector associated with the smallest eigenvalue of a hamiltonian is the ground state of a system.
Eigen vectors and corresponding eigen values are mainly used to switch between different coordinate systems. This might simplify problems and computations enormously by moving the problem sphere to from one coordinate system to another.
This new coordinates system has the eigen vectors as its base vectors, i.e. they "span" this coordinate system. Since they can be normalized, the transformation matrix from the first coordinate system is "orthonormal", that is the eigen vectors have magnitude 1 and are perpendicular to each other.
In the transformed coordinate system, a linear operation A (matrix) is pure diagonal. See Spectral Theorem, and Eigendecomposition for more information.
A quick implication is for example that you can from a general quadratic curve:
ax^2 + 2bxy + cy^2 + 2dx + 2fy + g = 0
rewrite it as
AX^2 + BY^2 + C = 0
where X and Y are counted along the direction of the eigen vectors.
Cheers !
check out http://mathworld.wolfram.com/Eigenvalue.html
Using eigen values in algorithms will need you to be proficient with the math involved.
I'm absolutely the wrong person to be talking about math: I puke on it.
cheers, jrh.
Eigen values and vectors are used in matrix computation as finding of reverse matrix. So if you need to write math code, precomputing them can speed some operations.
In short, you need them if you do matrix algebra, linear algebra etc.
Using the notation favored by physicists, if we have an operator H, then |x> is an eigenstate of H if and only if
H|x> = h|x>
where we call h the eigenvalue associated with the eigenvector |x> under H.
(Here the state of the system can be represented by a matrix, making this math isomorphic with all the other expressions already linked.)
Which brings us to the uses of these things once they have been discovered:
The full set of eigenvectors of a system under a given operator form an orthagonal spanning set for they system. This set may be a basis if there is no degeneracy. This is very useful because it allows extremely compact expressions of arbitrary (non eigen-) states of the system.