Can a 2D transpose convolution operation be represented as a matrix multiplication with the Toeplitz matrix, as can be done for a normal convolution?
I want to generalise some ideas from a dense network to a convolutional network. For normal convolutions, this is not a problem, as they can be represented as matrix multiplications with the Toeplitz matrix. But I couldn't find a clear mathematical formulation of transposed convolution, so I am not sure about this case.
I was looking for a mathematical answer, so should have probably asked somewhere else, anyways I think my latex write-up is correct and answers the question:
formula transposed convolution
Related
Eigen provides methods for both sparse matrices in Eigen/Sparse and dense matrices in Eigen/Dennse.
My questions are:
What is the motivation for the Sparse module? My feeling is that saving a sparse matrix in a sparse form can save space and avoid useless computations for very large matrices. Are there more reasons?
Are the underlying algorithms the same for both dense and sparse matrices? For example, the algorithm to calculate a singular value decomposition of a matrix.
Does Eigen try to identify special types of a sparse matrix, for example band diagonal, and apply specialized algorithms?
I have a square matrix that is largely zeros but interspersed with values. Is there a way to 'solve' this matrix so that all the information in it is contained in its lower triangle only, with the upper triangle containing only zeros?
Not in general.
If symmetric and positive definite you can do a Cholesky Decomposition.
If non-symmetric you can do an LU decomposition.
The Q matrix in quadratic forms (x'Qx) can be made symmetric and then lower-triangular. This is sometimes used when setting up a Quadratic Programming (QP) model.
Sparse versions of decomposition approaches are a well-studied area (not trivial though). In large-scale LP solvers sparse LU (simplex) or sparse Cholesky (interior point) are widely used.
Discr. convolution can be represented as multiplication of input with matrix M.
Where M is presented a special case of Toeplitz matrices - circulant matrices.
The questions is: is 2d convolution can also be represented as matrix multiplication?
p.s. By dicr. convolution I mean dicr. convolution with indexing discrete samples in modulus fashion, i.e. the discrete signal is repeating
....X[n-1]x[0]x[1]...x[N-1]x[0]...
Yes, it can, but it will generally be a rather big matrix. If your data set is on a grid of size NxM, then the convolution is a matrix operating on a vector of length N*M; the convolution matrix has N2M2 elements.
If your convolution kernel is small, then the matrix will typically a band matrix where the width of the band is at least N or M.
There are several methods available for computing SVD of a general matrix. I am interested to know about the best approach which could be used for computing SVD of an upper triangular matrix. Please suggest me an algorithm which could be optimized for this special case of matrices.
Here,I have an matrix,e.g,A,where A=[1 0.9 0.5;0.9 1 0.9;0.5 0.9 1],how to caculate its closest positive semi-definite matrix ? Is there any comand or algorithm ?
The closest positive semi-definite matrix is obtained using the polar decomposition. The jury is still out if the computation of this decomposition using the SVD or direct iterative methods is faster.
closest in what sense?
Typically, the best way to think about this is in eigenspace. If you don't have any constraints on the eigenvalues, I am not sure there is any way to make sense of your question. Sure, there exist other matrices which are semi positive definite; but in what sense are they still related to your original matrix?
But in case you have all real eigenvalues, things become a little more tangible. You can translate the eigenvalues along the real axis by adding to the diagonal, for instance.
Also, in practice one often deals with matrices which are SPD up a scaling of rows/columns; finding that scaling shouldn't be too hard, if it exists; but that scaling should then typically be available from the surrounding code. (a mass matrix of sorts).