i have following expression and i need to calculate time complexity of this algorithm. Could anybody help to get correct time complexity of this algorithm.
% save a matrix-vector multiply
Atb = A'*b;
% cache the factorization (using cholesky factorization)
[L U] = factor(A, a);
for( k = 0; k < maxiter; k++)
{
x^k+1 = (A^TA + a* I)^-1 (A^Tb + a (z^k - u^k))^T
}
Where A = mxn matrix and n>>>m, b,u,z = nx1 vectors, I = identity matrix and a=0.001
The most computationally intensive operation here is matrix inversion, so it depends on how you implement this operation. If we assume that you implemented with a Gauss–Jordan algorithm which takes O(n^3) then overall complexity is O(maxiter * n^3). Here i take into account that n is bigger than m (A^T*A takes O(m*n^2)).
If you calculate (A^T*A + a*I)^-1 and A^Tb outside then you are left with
Inv * (Atb + a(z^k - u^k))^T
which is O(n^2) because you need to multiply nxn matrix by nx1 vector while addition and subtraction take O(n).
Still, you have some inconsistencies in sizes which i described in comments for the question.
Related
Matrix Chain Multiplication has a dynamic programming solution where a recursive definition is used which works like this :
Problem : multiply i to j
Sub-problem : multiply i to k + multiply k+1 to j + multiplication cost
and this looks straight forward to memoize, due the repeating (i,j) sub-problems. But the following recursive definition which is bit different, I am facing difficulty memoizing it :
Can someone help memoizing this algo for matrix chain multiplication :
P is sequence of orders of matrices.
For eg, A(2,3)*B(3,4)*C(4,5), then P = {2,3,4,5}, i.e. order of ith matrix is P[i-1]*P[i]
also assumed P is 0-indexed.
Here I am multiplying adjacent matrices and recursing
Pseudocode :
chain_mul(P, n) {
if(n = 1) return 0
min_cost = inf
for( i = 1 to n-1) {
cost = P[i-1]*P[i]*P[i+1] + chain_mul(P-{P[i]}, n-1);
if(cost < min_cost) min_cost = cost
}
return min_cost
}
Here repeating sub-problem is structure of P, like I have shown below :
This cannot be memoized efficiently, because the argument P, iterates over all the subsets of the initial set P, so the memory required would be O(2^n).
The algoritms that can be memoized call the function specifying sections of the matrix chain, each section is characterized by two numbers, start and end index. The number of segments will be something like (n * (n + 1) / 2), and it is easy to implement a data structure to store and retrieve the results indexed by two numbers (e.g a matrix).
I want to find out coefficients of the n degree polynomial with roots 0,1,2...n-1. Can anybody suggest a good algorithm? I tried using FFT but didn't work fast enough
The simple solution that I would use is to write a function like this:
def poly_with_root_sequence (start, end, gap):
if end < start + gap:
return Polynomial([1, -start])
else:
p1 = poly_with_root_sequence(start, end, gap*2)
p2 = poly_with_root_sequence(start+gap, end, gap*2)
return p1 * p2
answer = poly_with_root_sequence(1, n, 1)
With a naive algorithm this will take O(n^2) arithmetic operations. However some of the operations will involve very large numbers. (Note that n! has more than n digits for large n.) But we have arranged that very few of the operations will involve very large numbers.
There is still no chance of producing answers as quickly as you want unless you are using a polynomial implementation with a very fast multiplication algorithm.
https://gist.github.com/ksenobojca/dc492206f8a8c7e9c75b155b5bd7a099 advertises itself as an implementation of the FFT algorithm for multiplying polynomials in Python. I can't verify that. But it gives you a shot at going fairly fast.
As replied on Evaluating Polynomial coefficients, you can do this as simple way:
def poly(lst, x):
n, tmp = 0, 0
for a in lst:
tmp = tmp + (a * (x**n))
n += 1
return tmp
print poly([1,2,3], 2)
I need to compute 0^j, 1^j, ..., k^j for some very large k and j (both in the order of a few millions). I am using GMP to handle the big integer numbers (yes, I need integer numbers as I need full precision). Now, I wonder, once I have gone through the effort of computing n^j, isn't there a way to speed up the computation of (n + 1)^j, instead of starting from scratch?
Here is the algorithm I am currently using to compute the power:
mpz_class pow(unsigned long int b, unsigned long int e)
{
mpz_class res = 1;
mpz_class m = b;
while(e)
{
if(e & 1)
{
res *= m;
}
e >>= 1;
m *= m;
}
return res;
}
As you can see, every time I start from scratch, and it takes a lot of time.
To compute n^j, why not find at least one factor of n, say k perform n^j = k^j * (n/k)^j ? By the time n^j is being computed, both k^j and (n/k)^j should be known.
However the above takes potentially O(sqrt(n)) time for n. We have a computation of n^j independently in O(log(j)) time by Exponentiation by Squaring as you have mentioned in the code above.
So you could have a mix of the above depending on which is larger:
If n is much smaller than log(j), compute n^j by factorization.
Whenever n^j is known compute {(2*n)^j, (3*n)^j, ..., ((n-1)*n)^j, n * n^j} and keep it in a lookup table.
If n is larger than log(j) and a ready computation as above is not possible, use the logarithmic method and then compute the other related powers like above.
If n is a pure power of 2 (possible const time computation), compute the jth power by shifting and calculate the related sums.
If n is even (const time computation again), use the factorization method and compute associated products.
The above should make it quite fast. For example, identification of even numbers by itself should convert half of the power computations to multiplications. There could be many more thumb rules that could be found regarding factorization that could reduce the computation further (especially for divisibility by 3, 7 etc)
You may want to use the binomial expansion of (n+1)^j as n^j + jn^(j-1)+j(j-1)/2 * n^(j-2) +... + 1 and memoize lower powers already computed and reuse them to compute (n+1)^j in O(n) time by addition. If you compute the coefficients j, j*(j-1)/2,... incrementally while adding each term, that can be done in O(n) too.
It's my code:
image = imread ('imageFile.jpg');
[m,n]=size(image);
for i=1:m
mask = image(i,:);
Mean = mean(mask(:));
stdm = std(mask(:));
bwx(i,:) = ( mask > Mean*( 10 + k * ( stdm / r+10 )));
end
I should compare this algorithm with another algorithm aspect of time. How can I compute time complexity of this algorithm?
r and k are constant.
mean and std are Matlab Functions.
mean = mean of each row of image.
std = standard deviation of each row of image.
Here is the formula for mean,
<img src="http://latex.codecogs.com/gif.latex?<s>=\frac&space;1&space;n&space;\sum&space;^n&space;_{i=0}&space;s_i" title="<s>=\frac 1 n \sum ^n _{i=0} s_i" />
If you count how many basic mathematical operations there are, addition, multiplication, subtraction, there are n total additions. So the time complexity of mean is O(n). That is mean is linear in time. I'll leave it up to you to figure out the complexity of std dev and the for loop in your code.
For a research paper, I have been assigned to research the fastest algorithm for computing the determinant of a matrix.
I already know about LU decomposition and Bareiss algorithm which both run in O(n^3), but after doing some digging, it seems there are some algorithms that run somewhere between n^2 and n^3.
This source (see page 113-114) and this source (see page 198) say that an algorithm exists that runs in O(n^2.376) because it is based on the Coppersmith-Winograd's algorithm for multiplying matrices. However, I have not been able to find any details on such an algorithm.
My questions are:
What is the fastest created (non-theoretical) algorithm for computing the determinant of a matrix?
Where can I find information about this fastest algorithm?
Thanks so much.
I believe the fastest in practice (and commonly used) algorithm is the Strassen Algorithm. You can find explanation on Wikipedia along with sample C code.
Algorithms based on Coppersmith-Winograd's multiplication algorithms are too complex to be practical, though they have best asymptotic complexity as far.
I know this is not a direct answer for my question, but for the purposes of completing my research paper, it is enough.
I just ended up asking my professor and I will summarize what he said:
Summary:
The fastest matrix-multiplication algorithms (e.g., Coppersmith-Winograd and more recent improvements) can be used with O(n^~2.376) arithmetic operations, but use heavy mathematical tools and are often impractical.
LU Decomposition and Bareiss do use O(n^3) operations, but are more practical
In short, even though LU Decomposition and Bareiss are not as fast as the most efficient algorithms, they are more practical and I should focus my research paper on these two.
Thanks for all who commented and helped!
See the following Matlab test script, which computes determinants of arbitrary square matrices (comparisons to Matlab's built-in function is also included):
nMin = 2; % Start with 2-by-2 matrices
nMax = 50; % Quit with 50-by-50 matrices
nTests = 10000;
detsOfL = NaN*zeros(nTests, nMax - nMin + 1);
detsOfA = NaN*zeros(nTests, nMax - nMin + 1);
disp(' ');
for n = nMin:nMax
tStart = tic;
for j = 1:nTests
A = randn(n, n);
detA1 = det(A); % Matlab's built-in function
if n == 1
detsOfL(j, 1) = 1;
detsOfA(j, 1) = A;
continue; % Trivial case => Quick return
end
[L, U, P] = lu(A);
PtL = P'*L;
realEigenvaluesOfPtL = real(eig(PtL));
if min(prod(realEigenvaluesOfPtL)) < 0 % det(L) is always +1 or -1
detL = -1;
else
detL = 1;
end
detU = prod(diag(U));
detA2 = detL * detU; % Determinant of A using LU decomposition
if detA1 ~= detA2
error(['Determinant computation failed at n = ' num2str(n) ', j = ' num2str(j)]);
end
detsOfL(j, n - nMin + 1) = detL;
detsOfA(j, n - nMin + 1) = detA2;
end
tElapsed = toc(tStart);
disp(sprintf('Determinant computation was successful for n = %d!! Elapsed time was %.3f seconds', n, tElapsed));
end
disp(' ');