What would be the computational complexity of the following pseudocode?
integer recursive (integer n) {
if (n == 1)
return (1);
else
return (recursive (n-1) + recursive (n-1));
}
In the real world, the calls would get optimized and yield linear complexity, but with the RAM model on which big-Oh is calculated, what would be the complexity? 2^n?
The complexity of this algorithm in its current form is indeed O(2n), because on each level of call, there will be twice more number of calls.
The first call (recursive(n)) constitutes one call
The next level (recursive(n-1)) constitutes 2 calls
At the base case (recursive(1)) it constitutes 2n-1 calls.
So the total number of function calls is 1+2+…+2n-1 = 2n-1
So the complexity is O(2n).
Additional points:
As you said, this can be easily made O(n) (or perhaps O(log n) for this special case using fast exponentiation) by memoization, or dynamic programming.
Your complexity will be
Why is it so? Simply mathematical induction proof:
N=1: special case, count of steps = 1.
N=2, Obvious, = 2, so it's correct
Let it be correct for N=K, i.e. for N=K it will be
Assuming N=K+1. The function recursive will call itself recursively for N=K two times: recursive(K+1) = recursive(K) + recursive(K) as it follows from the code. That is: . So, for N=K+1 we got steps.
So we've proof that complexity for N will be in common case (from definition of mathematical induction).
Related
The function myAlgo has a complexity of O(A + B):
function myAlgo() {
doA(); // O(A);
doB(); // O(B);
}
And myAlgo2 has a complexity of O(N):
function myAlgo2(N) {
for i in N
for j in N
}
Why does myAlgo2 not have a complexity of O(N + N)?
For clarity: Stop taking the code too literally. It is pseudo-code (this should be obvious). I commented the complexity of the functions for this reason.
Saying Big-O is additive misses the point of Big-O which is to simplify runtime performance to an asymptotic equation.
Two linear searches (e.g. your loops) may be each O(N) for a total of O(2N), but that's still O(N).
Your myAlgo2 has complexity of O(N+N).
Wich you can write as O(2N).
Wich you can write as 2O(N) and since 2 is constant it can be ignored so Complexity of myAlgo2 is O(N).
Since myAlgo has complexity O(A+B) you can't do the same becuase A might not be B.
Note: even if complexity does not change in real case scenario having O(2N) or O(N) will still make a (sometimes huge) difference.
You do not know the complexity of doA() and doB() functions, which means you have to take into account both of them. Imagine one of them is O(n^2) and the other one is O(n^3), then myAlgo() would be O(n^2+n^3)=O(n^3), as you take max of both functions.
If you look at myAlgo2() function, you have 2 linear loops O(n), which means that myAlgo2() is O(2n)=O(n).
To be more precise, you should take a look at the definition of summing big O functions, which will let you apply it to every case you encounter:
f1 = O(g1) and f2 = O(g2) => f1 + f2 = O(max(g1, g2))
I have two pseudo-code algorithms:
RandomAlgorithm(modVec[0 to n − 1])
b = 0;
for i = 1 to n do
b = 2.b + modVec[n − i];
for i = 1 to b do
modVec[i mod n] = modVec[(i + 1) mod n];
return modVec;
Second:
AnotherRecursiveAlgo(multiplyVec[1 to n])
if n ≤ 2 do
return multiplyVec[1] × multiplyVec[1];
return
multiplyVec[1] × multiplyVec[n] +
AnotherRecursiveAlgo(multiplyVec[1 to n/3]) +
AnotherRecursiveAlgo(multiplyVec[2n/3 to n]);
I need to analyse the time complexity for these algorithms:
For the first algorithm i got the first loop is in O(n),the second loop has a best case and a worst case , best case is we have O(1) the loop runs once, the worst case is we have a big n on the first loop, but i don't know how to write this idea as a time complexity cause i usually get b=sum(from 1 to n-1) of 2^n-1 . modVec[n-1] and i get stuck here.
For the second loop i just don't get how to solve the time complexity of this one, we usually have it dependant on n , so we need the formula i think.
Thanks for the help.
The first problem is a little strange, all right.
If it helps, envision modVec as an array of 1's and 0's.
In this case, the first loop converts this array to a value.
This is O(n)
For instance, (1, 1, 0, 1, 1) will yield b = 27.
Your second loop runs b times. The dominating term for the value of b is 2^(n-1), a.k.a. O(2^n). The assignment you do inside the loop is O(1).
The second loop does depend on n. Your base case is a simple multiplication, O(1). The recursion step has three terms:
simple multiplication
recur on n/3 elements
recur on n/3 elements (from 2n/3 to the end is n/3 elements)
Just as your binary partitions result in log[2] complexities, this one will result in log[3]. The base doesn't matter; the coefficient (two recursive calls) doesn't' matter. 2*O(log3) is still O(log N).
Does that push you to a solution?
First Loop
To me this boils down to the O(First-For-Loop) + O(Second-For-Loop).
O(First-For-Loop) is simple = O(n).
O(Second-For-Loop) interestingly depends on n. Therefore, to me it's can be depicted as O(f(n)), where f(n) is some function of n. Not completely sure if I understand the f(n) based on the code presented.
The answer consequently becomes O(n) + O(f(n)). This could boil down to O(n) or O(f(n)) depending upon which one is larger and more dominant (since the lower order terms don't matter in the big-O notation.
Second Loop
In this case, I see that each call to the function invokes 3 additional calls...
The first call seems to be an O(1) call. So it won't matter.
The second and the third calls seems to recurses the function.
Therefore each function call is resulting in 2 additional recursions.
Consequently , the time complexity on this would be O(2^n).
I have a question on forming recurrence relations and calculating the time complexity.
If we have a recurrence relation T(n)=2T(n/2) + c then it means that the constant amount of work c is divided into 2 parts T(n/2) + T(n/2) when drawing recursion tree.
Now consider recurrence relation of factorial which is T(n)=n*T(n-1) + c . If we follow the above method then we should divide the work c into n instances each of T(n-1) and then evaluate time complexity. However if calculate in this way then answer will O(n^n) because we will have O(n^n) recursive calls which is wrong.
So my question is why can't we use the same approach of dividing the elements into sub parts as in first case.
Let a recurrence relation be T(n) = a * T(n/b) + O(n).
This recurrence implies that there is a recursive function which:
divides the original problem into a subproblems
the size of each subproblem will be n/b if the current problem size is n
when the subproblems are trivial (too easy to solve), no recursion is needed and they are solved directly (and this process will take O(n) time).
When we say that original problem is divided into a subproblems, we mean that there are a recursive calls in the function body.
So, for instance, if the function is:
int f(int n)
{
if(n <= 1)
return n;
return f(n-1) + f(n-2);
}
we say that the problem (of size n) is divided into 2 subproblems, of sizes n-1 and n-2. The recurrence relation would be T(n) = T(n-1) + T(n-2) + c. This is because there are 2 recursive calls, and with different arguments.
But, if the function is like:
int f(int n)
{
if(n <= 2)
return n;
return n * f(n-1);
}
we say that the problem (of size n) is divided into only 1 subproblem, which is of size n-1. This is because there is only 1 recursive call.
So, the recurrence relation would be T(n) = T(n-1) + c.
If we multiply the T(n-1) with n, as would seem normal, we are conveying that there were n recursive calls made.
Remember, our main motive for forming recurrence relations is to perform (asymptotic) complexity analysis of recursive functions. Even though it would seem like n cannot be discarded from the relation as it depends on the input size, it would not serve the same purpose as it does in the function itself.
But, if you are talking about the value returned by the function, it would be f(n) = n * f(n-1). Here, we are multiplying with n because it is an actual value, that will be used in the computation.
Now, coming to the c in T(n) = T(n-1) + c; it merely suggests that when we are solving a problem of size n, we need to solve a smaller problem of size n-1 and some other constant (constant time) work like comparison, multiplication and returning values are also performed.
We can never divide "constant amount of work c" into two parts T(n/2) and T(n/2), even using recursive tree method. What we are, in fact, dividing, is the problem into two halves. The same "c" amount of work will be needed in each recursive call in each level of the recursive tree.
If there were a recurrence relation like T(n) = 2T(n/2) + O(n), where the amount of work to be done depends on the input size, then the amount of work to be done at each level will be halved at the next level, just like you described.
But, if the recurrence relation were like T(n) = T(n-1) + O(n), we would not be dividing the amount of work into two halves in the next recursion level. We would just be reducing the amount of work by one at each successive level (n-sized problem becomes n-1 at next level).
To check how the amount of work will change with recursion, apply substitution method to your recurrence relation.
I hope I have answered your question.
What's the complexity of a recursive program to find factorial of a number n? My hunch is that it might be O(n).
If you take multiplication as O(1), then yes, O(N) is correct. However, note that multiplying two numbers of arbitrary length x is not O(1) on finite hardware -- as x tends to infinity, the time needed for multiplication grows (e.g. if you use Karatsuba multiplication, it's O(x ** 1.585)).
You can theoretically do better for sufficiently huge numbers with Schönhage-Strassen, but I confess I have no real world experience with that one. x, the "length" or "number of digits" (in whatever base, doesn't matter for big-O anyway of N, grows with O(log N), of course.
If you mean to limit your question to factorials of numbers short enough to be multiplied in O(1), then there's no way N can "tend to infinity" and therefore big-O notation is inappropriate.
Assuming you're talking about the most naive factorial algorithm ever:
factorial (n):
if (n = 0) then return 1
otherwise return n * factorial(n-1)
Yes, the algorithm is linear, running in O(n) time. This is the case because it executes once every time it decrements the value n, and it decrements the value n until it reaches 0, meaning the function is called recursively n times. This is assuming, of course, that both decrementation and multiplication are constant operations.
Of course, if you implement factorial some other way (for example, using addition recursively instead of multiplication), you can end up with a much more time-complex algorithm. I wouldn't advise using such an algorithm, though.
When you express the complexity of an algorithm, it is always as a function of the input size. It is only valid to assume that multiplication is an O(1) operation if the numbers that you are multiplying are of fixed size. For example, if you wanted to determine the complexity of an algorithm that computes matrix products, you might assume that the individual components of the matrices were of fixed size. Then it would be valid to assume that multiplication of two individual matrix components was O(1), and you would compute the complexity according to the number of entries in each matrix.
However, when you want to figure out the complexity of an algorithm to compute N! you have to assume that N can be arbitrarily large, so it is not valid to assume that multiplication is an O(1) operation.
If you want to multiply an n-bit number with an m-bit number the naive algorithm (the kind you do by hand) takes time O(mn), but there are faster algorithms.
If you want to analyze the complexity of the easy algorithm for computing N!
factorial(N)
f=1
for i = 2 to N
f=f*i
return f
then at the k-th step in the for loop, you are multiplying (k-1)! by k. The number of bits used to represent (k-1)! is O(k log k) and the number of bits used to represent k is O(log k). So the time required to multiply (k-1)! and k is O(k (log k)^2) (assuming you use the naive multiplication algorithm). Then the total amount of time taken by the algorithm is the sum of the time taken at each step:
sum k = 1 to N [k (log k)^2] <= (log N)^2 * (sum k = 1 to N [k]) =
O(N^2 (log N)^2)
You could improve this performance by using a faster multiplication algorithm, like Schönhage-Strassen which takes time O(n*log(n)*log(log(n))) for 2 n-bit numbers.
The other way to improve performance is to use a better algorithm to compute N!. The fastest one that I know of first computes the prime factorization of N! and then multiplies all the prime factors.
The time-complexity of recursive factorial would be:
factorial (n) {
if (n = 0)
return 1
else
return n * factorial(n-1)
}
So,
The time complexity for one recursive call would be:
T(n) = T(n-1) + 3 (3 is for As we have to do three constant operations like
multiplication,subtraction and checking the value of n in each recursive
call)
= T(n-2) + 6 (Second recursive call)
= T(n-3) + 9 (Third recursive call)
.
.
.
.
= T(n-k) + 3k
till, k = n
Then,
= T(n-n) + 3n
= T(0) + 3n
= 1 + 3n
To represent in Big-Oh notation,
T(N) is directly proportional to n,
Therefore,
The time complexity of recursive factorial is O(n).
As there is no extra space taken during the recursive calls,the space complexity is O(N).
Why is the recurrence relation of recursive factorial algorithm this?
T(n)=1 for n=0
T(n)=1+T(n-1) for n>0
Why is it not this?
T(n)=1 for n=0
T(n)=n*T(n-1) for n>0
Putting values of n i.e 1,2,3,4...... the second recurrence relation holds(The factorials are correctly calculated) not the first one.
we generally use recurrence relation to find the time complexity of algorithm.
Here, the function T(n) is not actually for calculating the value of an factorial but it is telling you about the time complexity of the factorial algorithm.
It means for finding a factorial of n it will take 1 more operation than factorial of n-1
(i.e. T(n)=T(n-1)+1) and so on.
so correct recurrence relation for a recursive factorial algorithm is
T(n)=1 for n=0
T(n)=1+T(n-1) for n>0
not that you mentioned later.
like recurrence for tower of hanoi is
T(n)=2T(n-1)+1 for n>0;
Update:
It does not have anything to do with implementation generally.
But recurrence can give an intuition of programming paradigm for eg if T(n)=2*T(n/2)+n (merge sort) this gives kind of intuition for divide and conquer because we are diving n into half.
Also, If you will solve the equation it will give you a bound on running time.eg big oh notation.
Looks like T(n) is the recurrence relation of the time complexity of the recursive factorial algorithm, assuming constant time multiplication. Perhaps you misread your source?
What he put was not the factorial recursion, but the time complexity of it.
Assuming this is the pseudocode for such a recurrence:
1. func factorial(n)
2. if (n == 0)
3. return 1
4. return n * (factorial - 1)
I am assuming that tail-recursion elimination is not involved.
Line 2 and 3 costs a constant time, c1 and c2.
Line 4 costs a constant time as well. However, it calls factorial(n-1) which will take some time T(n-1). Also, the time it takes to multiply factorial(n-1) by n can be ignored once T(n-1) is used.
Time for the whole function is just the sum: T(n) = c1 + c2 + T(n-1).
This, in big-o notation, is reduced to T(n) = 1 + T(n-1).
This is, as Diam has pointed out, is a flat recursion, therefore its running time should be O(n). Its space complexity will be enormous though.
I assume that you have bad information. The second recurrence relation you cite is the correct one, as you have observed. The first one just generates the natural numbers.
This question is very confusing... You first formula is not factorial. It is simply T(n) = n + 1, for all n. Factorial of n is the product of the first n positive integers: factorial(1) = 1. factorial(n) = n * factorial(n-1). Your second formula is essentially correct.
T(n) = T(n-1) + 1 is correct recurrence equation for factorial of n.
This equation gives you the time to compute factorial of n NOT value of the factorial of n.
Where did you find the first one ? It's completely wrong.
It's only going to add 1 each time whatever the value is .
First you have to find a basic operation and for this example it is multiplication. Multiplication happens once in every recursion. So
T(n) = T(n-1) +1
this +1 is basic operation (mutliplication for this example)
T(n-1) is next recursion call.
TL;DR: The answer to your question actually depends on what sequence your recurrence relation is defining. That is, whether the sequence Tn in your question represents the factorial function or the running-time cost of computing the factorial functionX.
The factorial function
The recursive defintion of the factorial of n, fn, is:
fn = n • fn-1 for n > 0 with f0 = 1.
As you can see, the equation above is actually a recurrence relation, since it is an equation that, together with the initial term (i.e., f0 = 1), recursively defines a sequence (i.e., the factorial function, fn).
Modelling the running-time cost of computing the factorial
Now, we are going to find a model for representing the running-time cost of computing the factorial of n. Let's call Tn the running-time cost of computing fn.
Looking at the definition above of the factorial function fn, its running-time cost Tn will consist of the running-time cost of computing fn-1 (i.e., this cost is Tn-1) plus the running-time cost of performing the multiplication between n and fn-1. The multiplication is achieved in constant time. Therefore we could say that Tn = Tn-1 + 1.
However, what is the value of T0? T0 represents the running-time cost of computing f0. Since the value of f0 is initially known by definition, the running-time cost for computing f0 is actually constant. Therefore, we could say that T0 = 1.
Finally, what we obtain is:
Tn = Tn-1 + 1 for n > 0 with T0 = 1.
This equation above is also a recurrence relation. However, what it defines (together with the initial term), is a sequence that models the running-time cost of computing the factorial function.
XTaking into account how the sequence in your recurrence relation is called (i.e., Tn), I think it very likely represents the latter, i.e., the running-time cost of computing the factorial function.