I read this in Introduction to algorithms book, they didn't want to copy matrix entries because it would take Theta n^2 time and the trick they used was the index calculation and they said,
In fact, we can partition the matrices without copying entries. The trick is to use index calculations. We identify a submatrix by a range of row indices and a range of column indices of the original matrix. We end up representing a submatrix a little differently from how we represent the original matrix.
I want to know what index calculation is because it's the first step for Strassen's method
Related
We have M unique integers between 1 and N. In real life, N is a few millions, and M is between N/10 and N/3. I need to compute a distribution of pairwise distances between the M integers.
The brute-force complexity of the problem is M^2, but the output is just N numbers. So the natural question is whether there is a faster algorithm. Even an algorithm as fast as N * sqrt(M) should be sufficient for our purposes.
The problem appeared as a subset of the following problem. We have a large virtual square symmetric matrix, few million by few million elements. Some rows and columns of the matrix are masked out. We need to find how many masked-out elements are in each diagonal of the matrix. One can easily calculate how many masked-out bins intersect each diagonal. But often a masked-out row and column would intersect right on the diagonal, thus masking out only one bin. To avoid double-counting these, we need pairwise distribution of distances between masked-out columns.
You can do this in O(NlogN) using the Fourier transform.
The idea is that you first compute a histogram H(x) of your M integers where H(x) is the number of times the value x appears in your input (which will be either 0 or 1 if all M are distinct - but this is not essential).
Then what you want to compute is A(d), where A(d) defined as the number of pairs of integers that are exactly d apart.
This can be computed as A(d) = sum(H(x)*H(x+d) for all x)
This type of function is called a convolution and can be efficiently computed by taking the Fourier transform, multiplying the output by itself, and then computing the inverse transform. Care needs to be taken to pad appropriately, for example see this question.
If you use Python, this is particularly easy as you can call scipy.signal.fftconvolve to do this operation.
Is there a way to get both rows and columns
from a matrix (2D) array in O(1) time?
Obviously, to get one or the other, its trivial to just return the array.
Is there an easy way to get both in O(1) time?
I'm thinking I could have a second matrix, but that would double
my space requirements.
edit:
I could build "columns" but would take O(n) time assuming a n by n matrix. Because, by default, the matrix is an array of arrays. My question is, more concretely, is there a way to change my matrix data structure so that I can do both operations in O(1) time?
I have an integer vector. The size of the vector is around 2k, and each number in the vector is in the range of [0, 2M] and there is a high possibility to be 0.
Since it is a sparse vector, I'm wondering if there is a better algorithm than the regular ones to sort the vector? Which sorting algorithm would be the best for this scenario?
Thanks
This answer might be a bit too obvious...
Since most entries are zero why not do a preliminary exchange so that all the zeros are at one
end of the vector and the non-zero elements at the other.
Start from both ends of the
vector. From one end search for the first non-zero element, from the other end search for the
first zero element. Swap them and then continue until the two search positions meet. The vector is now partitioned into two parts at the meeting point. One part contains only zero elements and the other non-zero elements. Sort the vector from the meeting point over the non-zero elements. There should be very few items that acutally need sorting.
When sorting a few dozen elements or so the actual sorting algorithm used doesn't make much difference from a performance point of view (for a half dozen elements or so, bubble sort is hard to beat!).
If you have a vector of 2000 elements, do not worry too much on how to sort it... it is very small!
That said, if you have a vector with n integers, each of them between 0 and M, and M is small, you can sort it in O(n) time using Counting sort.
If the vector has n real numbers in some known range, and the numbers are uniformly distributed, you can use Bucket sort to sort them in O(n) expected time.
You're describing a regular dense vector that happens to have lots of 0 elements. A sparse vector only stores the nonzero elements, and if an element is not stored then it is assumed to be 0.
To sort a sparse vector just sort it normally. 2000 is already small, but if you genuinely use a sparse structure and "there is a high possibility [an element is] 0" then that number will be much smaller.
An example of a sparse structure is vector< pair<int, double> > where pair.first is the index and pair.second is the value.
The best which comes to my mind is Radix Sort, but thats harder to implement than 3-way quicksort. 3-way quicksort is optimal because it will skip a lot of the same elements, being O(n*log(n)) -> O(n), + i think there is an implementation in almost every programming language.
Say we have two square matrices of the same size n, named A and B.
A and B share the property that each entry in their main diagonal diagonals is the same value (i.e., A[0,0] = A[1,1] = A[2,2] ... = A[n,n] and B[0,0] = B[1,1] = B[2,2] ... = B[n,n]).
Is there a way to represent A and B so that they can be added to each other in O(n) time, rather than O(n^2)?
In general: No.
For an nxn matrix, there are n^2 output values to populate; that takes O(n^2) time.
In your case: No.
Even if O(n) of the input/output values are dependent, that leaves O(n^2) that are independent. So there is no representation that can reduce the overall runtime below O(n^2).
But...
In order to reduce the runtime, it is necessary (but not necessarily sufficient) to increase the number of dependent values to O(n^2). Obviously, whether or not this is possible is dictated by the particular scenario...
To complement Oli Cherlesworth answer, I'd like to point out that in the specific case of sparse matrices, you can often obtain a runtime of O(n).
For instance, if you happen to know that your matrices are diagonal, you also know that the resulting matrix will be diagonal, and hence you only need to compute n values.
Similarly, there are band matrices that can be added in O(n), as well as more "random" sparse matrices. In general, in a sparse matrix, the number of non-zero elements per row is more or less constant (you obtain these elements from a finite element computation for example, or from graph adjacency matrices etc.), and as such, using an appropriate representation such as "Compressed row storage" or "Compressed column storage", you will end up using O(n) operations to add your two matrices.
Also a special mention for sublinear randomized algorithms, that only propose you to know the final value that is "not-too-far" from the real solution, up to random errors.
I'm looking for a library that performs matrix operations on large sparse matrices w/o sacrificing numerical stability. Matrices will be 1000+ by 1000+ and values of the matrix will be between 0 and 1000. I will be performing the index calculus algorithm (en.wikipedia.org/wiki/Index_calculus_algorithm) so I will be generating (sparse) row vectors of the matrix serially. As I develop each row, I will need to test for linear independence. Once I fill my matrix with the desired number of linearly independent vectors, I will then need to transform the matrix into reduced row echelon form.
The problem now is that my implementation uses Gaussian elimination to determine linear independence (ensuring row echelon form once all my row vectors have been found). However, given the density and size of the matrix, this means the entries in each new row become exponentially larger over time, as the lcm of the leading entries must be found in order to perform cancellation. Finding the reduced form of the matrix further exacerbates the problem.
So my question is, is there an algorithm, or better yet an implementation, that can test linear independence and solve the reduced row echelon form while keeping the entries as small as possible? An efficient test for linear independence is especially important since in the index calculus algorithm it is performed by far the most.
Thanks in advance!
Usually if you are working with large matrices, people use LAPACK: this library contains all the basic matrix routines and support many different matrix types (sparse, ...). You can use this library to implement your algorithm, I think it will help you