For a research paper, I have been assigned to research the fastest algorithm for computing the determinant of a matrix.
I already know about LU decomposition and Bareiss algorithm which both run in O(n^3), but after doing some digging, it seems there are some algorithms that run somewhere between n^2 and n^3.
This source (see page 113-114) and this source (see page 198) say that an algorithm exists that runs in O(n^2.376) because it is based on the Coppersmith-Winograd's algorithm for multiplying matrices. However, I have not been able to find any details on such an algorithm.
My questions are:
What is the fastest created (non-theoretical) algorithm for computing the determinant of a matrix?
Where can I find information about this fastest algorithm?
Thanks so much.
I believe the fastest in practice (and commonly used) algorithm is the Strassen Algorithm. You can find explanation on Wikipedia along with sample C code.
Algorithms based on Coppersmith-Winograd's multiplication algorithms are too complex to be practical, though they have best asymptotic complexity as far.
I know this is not a direct answer for my question, but for the purposes of completing my research paper, it is enough.
I just ended up asking my professor and I will summarize what he said:
Summary:
The fastest matrix-multiplication algorithms (e.g., Coppersmith-Winograd and more recent improvements) can be used with O(n^~2.376) arithmetic operations, but use heavy mathematical tools and are often impractical.
LU Decomposition and Bareiss do use O(n^3) operations, but are more practical
In short, even though LU Decomposition and Bareiss are not as fast as the most efficient algorithms, they are more practical and I should focus my research paper on these two.
Thanks for all who commented and helped!
See the following Matlab test script, which computes determinants of arbitrary square matrices (comparisons to Matlab's built-in function is also included):
nMin = 2; % Start with 2-by-2 matrices
nMax = 50; % Quit with 50-by-50 matrices
nTests = 10000;
detsOfL = NaN*zeros(nTests, nMax - nMin + 1);
detsOfA = NaN*zeros(nTests, nMax - nMin + 1);
disp(' ');
for n = nMin:nMax
tStart = tic;
for j = 1:nTests
A = randn(n, n);
detA1 = det(A); % Matlab's built-in function
if n == 1
detsOfL(j, 1) = 1;
detsOfA(j, 1) = A;
continue; % Trivial case => Quick return
end
[L, U, P] = lu(A);
PtL = P'*L;
realEigenvaluesOfPtL = real(eig(PtL));
if min(prod(realEigenvaluesOfPtL)) < 0 % det(L) is always +1 or -1
detL = -1;
else
detL = 1;
end
detU = prod(diag(U));
detA2 = detL * detU; % Determinant of A using LU decomposition
if detA1 ~= detA2
error(['Determinant computation failed at n = ' num2str(n) ', j = ' num2str(j)]);
end
detsOfL(j, n - nMin + 1) = detL;
detsOfA(j, n - nMin + 1) = detA2;
end
tElapsed = toc(tStart);
disp(sprintf('Determinant computation was successful for n = %d!! Elapsed time was %.3f seconds', n, tElapsed));
end
disp(' ');
Related
I want to find out coefficients of the n degree polynomial with roots 0,1,2...n-1. Can anybody suggest a good algorithm? I tried using FFT but didn't work fast enough
The simple solution that I would use is to write a function like this:
def poly_with_root_sequence (start, end, gap):
if end < start + gap:
return Polynomial([1, -start])
else:
p1 = poly_with_root_sequence(start, end, gap*2)
p2 = poly_with_root_sequence(start+gap, end, gap*2)
return p1 * p2
answer = poly_with_root_sequence(1, n, 1)
With a naive algorithm this will take O(n^2) arithmetic operations. However some of the operations will involve very large numbers. (Note that n! has more than n digits for large n.) But we have arranged that very few of the operations will involve very large numbers.
There is still no chance of producing answers as quickly as you want unless you are using a polynomial implementation with a very fast multiplication algorithm.
https://gist.github.com/ksenobojca/dc492206f8a8c7e9c75b155b5bd7a099 advertises itself as an implementation of the FFT algorithm for multiplying polynomials in Python. I can't verify that. But it gives you a shot at going fairly fast.
As replied on Evaluating Polynomial coefficients, you can do this as simple way:
def poly(lst, x):
n, tmp = 0, 0
for a in lst:
tmp = tmp + (a * (x**n))
n += 1
return tmp
print poly([1,2,3], 2)
i have following expression and i need to calculate time complexity of this algorithm. Could anybody help to get correct time complexity of this algorithm.
% save a matrix-vector multiply
Atb = A'*b;
% cache the factorization (using cholesky factorization)
[L U] = factor(A, a);
for( k = 0; k < maxiter; k++)
{
x^k+1 = (A^TA + a* I)^-1 (A^Tb + a (z^k - u^k))^T
}
Where A = mxn matrix and n>>>m, b,u,z = nx1 vectors, I = identity matrix and a=0.001
The most computationally intensive operation here is matrix inversion, so it depends on how you implement this operation. If we assume that you implemented with a Gauss–Jordan algorithm which takes O(n^3) then overall complexity is O(maxiter * n^3). Here i take into account that n is bigger than m (A^T*A takes O(m*n^2)).
If you calculate (A^T*A + a*I)^-1 and A^Tb outside then you are left with
Inv * (Atb + a(z^k - u^k))^T
which is O(n^2) because you need to multiply nxn matrix by nx1 vector while addition and subtraction take O(n).
Still, you have some inconsistencies in sizes which i described in comments for the question.
The article on Wikipedia describes how to derive Durand-Kerner root-finding method from Newton method. The method is attractive because of its good convergence and simplicity, as proven in ACM Algorithm 283 by Immo Kerner himself ("translated to PL/I by R. A. Vowels"):
Prrs: procedure (A, X, n, epsin) options (reorder);
declare (A(*), X(*), epsin) float, n fixed binary;
declare (i, k, j, nits) fixed binary, (xx, P, Q, eps) float;
eps = epsin*epsin;
nits = 1;
W: Q = 0;
do i = 1 to n;
xx = A(n); P = A(n);
do k = 1 to n;
xx = xx * X(i) + A(n-k);
if k ^= i then P = P * (X(i) - X(k));
end;
X(i) = X(i) - xx/P;
Q = Q + (xx/P)*(xx/P);
end;
nits = nits + 1;
if Q > eps then go to W;
end Prrs;
Is it somehow possible to derive similar method for simultaneously finding all the approximations of roots of a system of n polynomial equations in n variables?
The line of thinking was: to find a root of a polynomial in one variable, it is possible to use Newton's method. It is simple and fast, but which exact root is going to be found depends on the initial guess and so it is difficult to find all the roots.
To approximate all the roots simultaneously, there are several generalizations of Newton's method which employ the so-called Weierstrass correction, for example the above mentioned Durand-Kerner's or Aberth's methods.
For systems of n multivariate polynomial equations in n variables there exists another generalization of Newton's method which allows one to find a root (i.e. a set of n values where the system becomes zero). It uses Jacobian matrix.
So my question is, would it be possible to use the Jacobian in the corrections, instead of Newton, kind of like combining Durand-Kerner and multivariate Newton? By sheer luck, does anyone know of an implementation of such algorithm?
I'm currently utilizing an online variance algorithm to calculate the variance for a given sequence. This works nicely, and also gives good numerical stability and overflow resistance, at the cost of some speed, which is fine. My question is, does an algorithm exist that will be faster than this if the sample mean is already known, while having similar stability and resistance to overflow (hence not something like a naive variance calculation).
The current online variance calculation is a single-pass algorithm with both divisions and multiplications in the main loop (which is what impacts speed). From wikipedia:
def online_variance(data):
n = 0
mean = 0
M2 = 0
for x in data:
n = n + 1
delta = x - mean
mean = mean + delta/n
M2 = M2 + delta*(x - mean)
variance = M2/(n - 1)
return variance
The thing that causes a naive variance calculation to go unstable is the fact that you separately sum the X (to get mean(x)) and the X^2 values and then take the difference
var = mean(x^2) - (mean(x))^2
But since the definition of variance is
var = mean((x - mean(x))^2)
You can just evaluate that and it will be as fast as it can be. When you don't know the mean, you have to compute it first for stability, or use the "naive" formulation that goes through the data only once at the expense of numerical stability.
EDIT
Now that you have given the "original" code, it's easy to be better (faster). As you correctly point out, the division in the inner loop is slowing you down. Try this one for comparison:
def newVariance(data, mean):
n = 0
M2 = 0
for x in data:
n = n + 1
delta = x - mean
M2 = M2 + delta * delta
variance = M2 / (n - 1)
return variance
Note - this looks a lot like the two_pass_variance algorithm from Wikipedia, except that you don't need the first pass to compute the mean since you say it is already known.
Hello
Assume I have the set of numbers I want a quick to calculate some measure of uniformity.
I know the variance is the most obvious answer but i am afraid the complexity of naive algorithm is too high
Anyone have any suggestions?
"Intuitive" algorithms for calculating variance usually suffer one or both of the following:
Use two loops (one for calculating the mean, the other for the variance)
Are not numerically stable
A good algorithm, with only one loop and numerically stable is due to D. Knuth (as always).
From Wikipedia:
n = 0
mean = 0
M2 = 0
def calculate_online_variance(x):
n = n + 1
delta = x - mean
mean = mean + delta/n
M2 = M2 + delta*(x - mean) # This expression uses the new value of mean
variance_n = M2/n
variance = M2/(n - 1) #note on the first pass with n=1 this will fail (should return Inf)
return variance
You should invoke calculate_online_variance(x) for each point, and it returns the variance calculated so far.