Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Lately I've been working with matrices and I started learning about sparse matrices but I don't understand why they even exist. What are some practical uses of sparse matrices? If all they mainly hold are 0's do they have any useful functions?
Sparse matrices are generally just matrices with a lot of zero-entries (typically at least > 50%).
They can be represented in a very concise way, (wiki) which can be used to do matrix operations (e.g. multiplication, transpose of a matrix, ...) in a fast and efficient way. Google Maps and other applications would be impossible without efficient sparse matrix algorithms.
If you want to dig deeper, I recommend this website & professor, who developed some of those algorithms. Apparently, it's an ongoing research topic of high interest.
I feel you're asking the question the other way around.
Sparse matrices are matrices with a high density of zeros.
Sparse matrices are very frequently encountered in many fields of scientific computing, and in veeery big sizes. It is not if they're useful, they just exist.
Now, the interesting and useful thing is how we represent them.
We tend to compress the sparse matrices to only take into account the non zero values and keep track of their location in the matrix.
This is useful ion a sense that it saves a lot of storage and the matrix operations are restricted to its non zero values.
A field where sparse matrices are very common is solving discrete partial differential equations.
In a nutshell: you have a grid of voxels and the discretized differential equation states a relation between the quantity you are solving only for neighbouring voxels. The resulting matrix equation you need to solve typically has a band structure. In 2D the structure is even simpler and you have only values on the diagonal and one off the diagonal, all the other coefficients are zero.
Consider a 100k X 100k matrix.
If you want to store all of that using double precision in a 2D array you'll need 80Gbytes not counting any memory management overhead. If many of the elements are zero you can save a lot of memory.
For instance, if you have a regular sparse matrix such as a tridiagonal this can be stored in a 3 X 100k array with 2.4Mbytes. Implicitly all elements off the tridiagonal are zero.
As another example, in electronic circuit analysis matrices are used to solve a system of PDEs using the Newton-Raphson method. Since most circuit elements are connected to very few other ones the resulting matrix is sparse. This can be represented with a 2D linked list of non-zero elements. Again anything that isn't in the linked list is implicitly zero.
Related
I am trying to parallelize gaussian elimination on sparse linear equations. I could not find data to test any where on the internet. If you could provide links to such data set that will be great.
Also could someone please explain how are sparse linear equations produced, that is practically, what problems produce such equations.
Thanks in advance.
First:
If indeed interested in linear problem sparse matrices, you may enjoy datasets from almost any linear Finite Element Method problem formulation.
Their problem-formulation may still use a vast amount of just linear equations.
Their problem-space is principally sparse in 3D-sense, so even if the original data were not represented as sparse-matrices, you can easily convert any of their dense-matrix representation into any sparse-matrix representation of your choice.
Next:
There are further sources of further interest for dense & sparse array I/O using the Matrix Market format
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
In usual circumstances, sorting arrays of ~1000s of simple items like integer or floats is sufficiently fast that the small differences between implementations just doesn't matter.
But what if you need to sort N modest sized arrays that have been generated by some similar process or simply have have some relatedness?
I leave the specifics of what of the mysterious array generator and relationships of the generated arrays intentionally vague. It is up to any applicable algorithms to specify a large as possible domain where they will work when they will be most useful.
EDIT: Let's narrow this by letting the arrays be independent samples. There exists an unchanging probability distribution of arrays that will be generated. Implicitly then there's a stable probability distribution of elements in the arrays but it's conditonal -- the elements within an array might not be independent. It seems like it'd be extremely hard to make use of relationships between elements within the arrays but I could be wrong. We can narrow further if needed by letting the elements in the arrays be independent. In that case the problem is to effectively learn and use the probability distribution of elements in the arrays.
Here is a paper on a self improving sorting algorithm. I am pretty strong with algorithms and machine learning, but this paper is definitely not an easy read for me.
The abstract says this
We investigate ways in which an algorithm can improve
its expected performance by fine-tuning itself automatically with respect to an arbitrary, unknown input distribution. We give such self-improving algorithms for
sorting and clustering. The highlights of this work:
a sorting algorithm with optimal expected limiting running time ...
In all cases, the algorithm begins with a learning phase
during which it adjusts itself to the input distribution
(typically in a logarithmic number of rounds), followed
by a stationary regime in which the algorithm settles to
its optimized incarnation.
I was asked about various storage of matrices; in particular, about band storage and other variations. I know it is something related to storage of sparse matrices in an efficient way. But, I have no clear idea about the details.
Simply put, a matrix is banded if all the non-zero elements are close to the diagonal. More formally, consider a matrix A whose elements are aij. The matrix is said to be banded with bandwith m if aij = 0 for all i, j such that abs(i-j)>=m.
A banded matrix is a sparse matrix of a very special form. Band structure is very easy to understand and operate on. Storage is efficient, and very efficient algorithms exist for banded matrices. More so than for more general sparse matrices.
Band storage takes advantage of the structure of the matrices by only storing the elements that may be non-zero.
Read more about this here: http://en.wikipedia.org/wiki/Band_matrix
I'm looking for a library that performs matrix operations on large sparse matrices w/o sacrificing numerical stability. Matrices will be 1000+ by 1000+ and values of the matrix will be between 0 and 1000. I will be performing the index calculus algorithm (en.wikipedia.org/wiki/Index_calculus_algorithm) so I will be generating (sparse) row vectors of the matrix serially. As I develop each row, I will need to test for linear independence. Once I fill my matrix with the desired number of linearly independent vectors, I will then need to transform the matrix into reduced row echelon form.
The problem now is that my implementation uses Gaussian elimination to determine linear independence (ensuring row echelon form once all my row vectors have been found). However, given the density and size of the matrix, this means the entries in each new row become exponentially larger over time, as the lcm of the leading entries must be found in order to perform cancellation. Finding the reduced form of the matrix further exacerbates the problem.
So my question is, is there an algorithm, or better yet an implementation, that can test linear independence and solve the reduced row echelon form while keeping the entries as small as possible? An efficient test for linear independence is especially important since in the index calculus algorithm it is performed by far the most.
Thanks in advance!
Usually if you are working with large matrices, people use LAPACK: this library contains all the basic matrix routines and support many different matrix types (sparse, ...). You can use this library to implement your algorithm, I think it will help you
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
For a class, a question that was posed by my teacher was the algorithmic cost of multiplying a matrix times its transpose. With the standard 3 loop matrix multiplication algorithm, the efficiency is O(N^3), and I wonder if there was a way to manipulate or take advantage of matrix * matrix transpose to get a faster algorithm. I understand that when you multiply a matrix by its transpose you have to calculate less of the matrix because its symmetrical, but I can't think of how to manipulate an algorithm that could take less than O(n^3).
i know there's algorithms like Coppensmith and Straussen that are faster general matrix multiplication algorithms but could anyone give any hints or insights on how to computationally take advantage of the transpose?
Thanks
As of right now there aren't any aymptotic barrier-breaking properties of this particular multiplication.
The obvious optimization is to take advantage of the symmetry of the product. That is to say, the [i][j]th entry is equal to the [j][i]th entry.
For implementation-specific optimizations, there is a significant amount of caching that you can do. A very significant amount of time in the multiplication of large matrices is spent transferring data to and from memory and CPU. So CPU designers implemented a smart caching system whereby recently used memory is stored in a small memory section called the cache. In addition to that, they also made it so that nearby memory is also cached. This is because a lot of the memory IO is due to reading/writing from/to arrays, which are stored sequentially.
Since the transpose of a matrix is simply the same matrix with the indices swapped, caching a value in the matrix can have over twice the impact.