I am learning about time complexity and I trying to figure out a relationship. My lecture notes describe a recursive find functions as :
find(array A, item I)
if(arrayEmpty(A)) return BAD;
if(item == A[0]) return GOOD;
return find(allButFirst(A), I);
useful link : https://www.youtube.com/watch?v=_cG5KZSn1LE
My notes says that the starting relationship for time complexity is as a follow:
T(n) = 1 + T(n-1) // I understand this
T(1) = 1 // only one computation, understandable
then we unroll T(n)
T(N) = 1 + 1 + T(n-2) // every recursive step 1 comparison plus recursive call
T(N) = 1 + 1 + 1 + T(n-3)
T(N) = 1 + 1 + 1 + 1 + T(n-4)
...
T(N) = (n - 1) + T(n-(n-1)) // This point I am lost how they got this generalisation
If someone can explain how the above relation was generalised to T(N) = (n - 1) + T(n-(n-1)) and perhaps with a example would be better for clarity.
For example I want to try the above relationship with some values so let's say A {1, 2, 3} and I = 3
then here are the following computation
1 + T(n-1) // {2,3}
1 + 1 + T(n-2) // {3}
1 // {3} Found
So for above we had total 3 comparison and 2 recursive calls. So I would say the relationship is T(n) = n + t(n-(n-1)) = n + t(1) = n.
In each step of the unrolled pattern, if we are k steps into the recursion (for the first step, k=1, the last step, k=n), there are k 1's.
In the expansion you posted, the last line, T(N) = (n - 1) + T(n-(n-1)) is the second-to-last step, since T(n-(n-1)) expands to T(1), so for that line, k=n-1.
So there are n-1 1's in that line, hence the (n-1) term.
Likewise, the parameter passed to T at the kth step is n minus the current step, since it's the number of steps remaining. For the first row, that's n-k = n-1, hence T(n-1). For the second-to-last step, it's n-k = n-(n-1), hence T(n-(n-1)) = T(1).
Related
def maximum(array):
max = array[0]
counter = 0
for i in array:
size +=1
if i>max:
max=i
return max
I need to analyze that algorithm which find maximum number in array with n numbers in it. the only thing I want to know how to get Recursive and General formula for Average case of this algorithm.
Not sure what you mean by "Recursive and General formula for Average case of this algorithm". Your algorithm is not recursive. So, how can it be "recursive formula"?
Recursive way to find maximum in an array:
def findMax(Array, n):
if (n == 1):
return A[0]
return max(Array[n - 1], findMax(Array, n - 1))
I guess you want Recurrence relation.
Let T(n) be time taken to find the maximum of n elements. So, for above written code.
T(n) = T(n-1) + 1 .... Equation I
In case you are interested to solve the recurrence relation:
T(n-1) = T((n-1)-1) + 1 = T(n-2) + 1 .... Equation II
If you substitute value of T(n-1) from Equation II into Equation I, you get:
T(n) = (T(n-2) + 1) + 1 = T(n-2) + 2
Similarly,
T(n) = T(n-3) + 3
T(n) = T(n-4) + 4
and so on..
Continuing the above for k times,
T(n) = T(n-k) + k
If n-k = 0, means n = k. The equation then becomes
T(n) = T(0) + n = 1 + n
Therefore, the recursive algorithm we came up with has time complexity O(n).
Hope it helped.
How do I determine the running time (in terms of Big-Theta) for the algorithm of input size n that satisfies recurrence relation T(n) = T(n-1)+n where n >= 1 and with initial condition T(1) = 1?
Edit: I was practicing a past exam paper. Got stuck on this question. Need guidance
Look at it this way: T(n) = T(n-1) + n = T(n-2) + (n-1) + n = T(n-3) + (n-2) + (n-1) + n. Which means if n >= 1 then you will get something like T(n) = 1 + 2 + 3 + ... + n. If you work out the pattern of this series you will see that (n+1)n/2. Therefore, Ө(n^2)
What is the time complexity of the recursive solution to this in the code taken from: http://www.geeksforgeeks.org/dynamic-programming-set-32-word-break-problem/ :
// returns true if string can be segmented into space separated
// words, otherwise returns false
bool wordBreak(string str)
{
int size = str.size();
// Base case
if (size == 0) return true;
// Try all prefixes of lengths from 1 to size
for (int i=1; i<=size; i++)
{
// The parameter for dictionaryContains is str.substr(0, i)
// str.substr(0, i) which is prefix (of input string) of
// length 'i'. We first check whether current prefix is in
// dictionary. Then we recursively check for remaining string
// str.substr(i, size-i) which is suffix of length size-i
if (dictionaryContains( str.substr(0, i) ) &&
wordBreak( str.substr(i, size-i) ))
return true;
}
// If we have tried all prefixes and none of them worked
return false;
}
I'm thinking its n^2 because for n calls to the method, it worst case does (n-1) work (iterates over the rest of string recursively?). Or is it exponential/n!?
I'm having a tough time figuring out Big(O) of recursive functions such as these. Any help is much appreciated!
The answer is exponential, to be precise O(2^(n-2)). (2 power n-2)
In each call you are calling the recursive function with length 1,2....n-1(in worst case). To do the work of length n you are recursively doing the work of all the strings of length n-1, n-2, ..... 1. So T(n) is the time complexity of your current call, you are internally doing a work of sum of T(n-1),T(n-2)....T(1).
Mathematically :
T(n) = T(n-1) + T(n-2) +.....T(1);
T(1) = T(2) = 1
If you really don't know how to solve this, an easier way to solve the above recurrence is by just substitute values.
T(1) = T(2) = 1
T(3) = T(1) + T(2) = 1+1 =2; // 2^1
T(4) = T(1)+ T(2) + T(3) = 1+1+2 =4; //2^2
T(5) = T(1) + T(2) +T(3) +T(4) = 1+1+2+4 =8; //2^3
So if you substitute first few values, it will be clear that the Time complexity is 2^(n-2)
The short version:
The worst-case runtime of this function is Θ(2n), which is surprising because it ignores the quadratic amount of work done by each recursive call simply splitting the string into pieces and checking which prefixes are words.
The longer version: let's imagine we have an input string consisting of n copies of the letter a followed by the letter b. (we'll abbreviate this as aⁿb), and create a dictionary containing the words a, aa, aaa, ..., aⁿ.
Now, what will the recursion do?
First, notice that none of the recursive calls will return true, because there's no way to account for the b at the end of the string. This means that each recursive call will be to a string of the form aᵏb. Let's denote the amount of time required to process such a string as T(k). Each one of these calls will fire off k smaller calls, one to each suffix of aᵏb.
However, we also have to account for the other contributors to the runtime. In particular, calling string::substr to form a substring of length k takes time O(k). We also need to factor in the cost of checking if a prefix is a word. The code isn't shown here for how to do this, but assuming we use a trie or a hash table we can assume that the cost of checking if a string of length k is a word is O(k) as well. This means that, at each point where we make a recursive call, we will do O(n) work - some amount of work to check if the prefix is a word, and some amount of work to form the substring corresponding to the suffix.
Therefore, we get that
T(k) = T(0) + T(1) + T(2) + ... + T(k-1) + O(k2)
Here, the first part of the recurrence corresponds to each of the recursive calls, and the second part of the recurrence accounts for the cost of making each of the substrings. (There are n substrings, each of which takes time O(n) to process). Our goal is to solve this recurrence, and just for simplicity we'll assume that T(0) = 1.
To do this, we'll use the "expand and contract" technique. Let's write out the values of T(k) and T(k+1) next to each other:
T(k) = T(0) + T(1) + T(2) + ... + T(k-1) + O(k2)
T(k+1) = T(0) + T(1) + T(2) + ... + T(k-1) + T(k) + O(k2)
Subtracting this first expression from the second gives us that
T(k+1) - T(k) = T(k) + O(k),
or that
T(k+1) = 2T(k) + O(k).
How did we get O(k) out of the difference of two O(k2) terms? It's because (k + 1)2 - k2 = 2k + 1 = O(k).
This is an easier recurrence to work with, since each term just depends on the previous one. For simplicity, we're going to assume that the O(k) term is literally just k, giving the recurrence
T(k+1) = 2T(k) + k.
This recurrence solves to T(k) = 2k+1 - k - 1. To see this, we can use a quick inductive argument. Specifically:
T(0) = 1 = 2 - 1 = 20+1 - 0 - 1
T(k+1) = 2T(k) + k = 2(2k - k - 1) + k
= 2k+1 - 2k - 2 + k
= 2k+1 - k - 2
= 2k+1 - (k + 1) - 1
Therefore, we get that our runtime is Θ(2n), since we can ignore the lower-order n term.
I was very surprised to see this, because this means that the quadratic work done by each recursive call does not factor into the overall runtime! I would have initially guessed that the runtime would be something like Θ(n · 2n) before doing this analysis. :-)
I believe the answer should actually be O(2^(n-1)). You can see a proof as this as well as a worst-case example here:
https://leetcode.com/problems/word-break/discuss/169383/The-Time-Complexity-of-The-Brute-Force-Method-Should-Be-O(2n)-and-Prove-It-Below
One intuitive way I think about the complexity here is, how many ways are there to add spaces in the string here or break the word here ?
for a 4 letter word:
no of ways to break at index 0-1 * no of ways to break at index 1-2 * no of ways to break at index 2-3 = 2 * 2 * 2.
2 is there to signify the two options => you break it, you don't break it
O(2^(n-1)) is the recursive complexity of the wordbreak then ;)
I give you three short codes:
First code:
procedure Proc (n:integer)
begin
if n>0 then
begin
writeln('x')
Proc(n-2)
writeln('*');
Proc(n-2)
end
end
Second code:
procedure Proc (n:integer)
begin
if n>0 then
begin
writeln('*');
Proc(n-1)
end
end
Third code:
procedure Proc (n:integer)
begin
if n>0 then
begin
writeln('x')
Proc(n/2)
writeln('*');
Proc(n/2)
end
end
I would like to know how to determine the computational complexity of each code that I gave, cuz it will help me to better understand.. Can someone write an algorithm for determination of computational complexity of sample code step by step, and do it so that it was possible to apply this algorithm for another examples of codes?
First Question: Assume you know that for the value of n - 2, Proc is called T(n-1) times. Therefore, for the value of n, T(n) = 1 + 2T(n-2), as there would be one call to Proc(n) which would in turn call Proc(n-2) twice. T(n) = 1 + 2T(n-2) is a variant of Tower of Hanoi which is T(n) = 1+2T(n-1). There are proofs here http://en.wikipedia.org/wiki/Tower_of_Hanoi to Show that T(n) = 1+2T(n-1) = 2^n-1. Therefore T(n-1) = 1+2T((n-1)-1)= 1+2T(n-2) = 2^(n-1) -1. In your case T(n) = 1 + 2T(n-2) = 2^(n-1) -1. In other words, subtracting out every other term in the Tower of Hanoi problem saves about half the calls. 2^(n-1) - 1 = 2^n/2 - 1 which is O(2^n).
Second Question: This is easier. T(0) = 1 and T(n) = 1 + T(n-1). You can solve this many different ways, but one is via telescoping:
T(n) = 1 + T(n-1)
T(n-1) = 1 + (n-2)
...
T(1) = 1 + T(0)
Adding up both sides...
T(n) + T(n-1)+...+T(1) = 1 + T(n-1) + ... + 1 + T(0) = n + T(n-1)+...+T(0)
Subtract out like terms.
T(n) = n + T(0) = n + 1. So this is O(n).
Third Question: Similar to the first. T(0) = 1, say we know that for value of n-1, you can see that T(n) = 1 + 2 T(n/2). Note here that T(n) = 1 + 2T(n/2) < n + 2T(n/2).
So solve for 2T(n/2) + n with rolling out the recurrence:
T(n) = 2 T(n/2) + n
T(n/2) = 2 T(n/4) + n/2
So T(n) = 4T(n/4) + n + n
T(n/4) = 2T(n/8) + n/4
So T(n) = 8T(n/8) n + n + n
... It looks like T(n) = 2^kT(n/2^k)+kn for positive k.
Prove it by induction.
k = 1: T(n) = 2 T(n/2)+n which was given. This is our base case.
If true for k-1, show true for k:
T(n) = (2^(k-1))T(n/2^(k-1))+(k-1)n //Inductive hypothesis
T(n/2^(k-1)) = 2 T([n/2^(k-1))]/2)+n/2^(k-1)) //Given recurrence
= 2T(n/2^k)+n/2^(k-1)
=> T(n) = (2^k)T(n/2^k)+ n + (k-1)n = (2^k)T(n/2^k) + kn. So true for k.
T(n) = 2^kT(n/2^k)+kn, choose an appropriate positive k, such as k = ln(n).
T(n) = 2^ln(n) T(n/2^Ln(n)) + nln(n) = nT(1) +nln(n).
T(1) = 1 since Proc would just end. So n(T(1)) + nln(n) = nln(n) + n = O(nln(n)).
Unfortunately, there is not a one-size-fits all procedure for complexity. You have to take it on a case-by-case basis and figure out the problem.
I am trying to find complexity of Fibonacci series using a recursion tree and concluded height of tree = O(n) worst case, cost of each level = cn, hence complexity = n*n=n^2
How come it is O(2^n)?
The complexity of a naive recursive fibonacci is indeed 2ⁿ.
T(n) = T(n-1) + T(n-2) = T(n-2) + T(n-3) + T(n-3) + T(n-4) =
= T(n-3) + T(n-4) + T(n-4) + T(n-5) + T(n-4) + T(n-5) + T(n-5) + T(n-6) = ...
In each step you call T twice, thus will provide eventual asymptotic barrier of:
T(n) = 2⋅2⋅...⋅2 = 2ⁿ
bonus: The best theoretical implementation to fibonacci is actually a close formula, using the golden ratio:
Fib(n) = (φⁿ – (–φ)⁻ⁿ)/sqrt(5) [where φ is the golden ratio]
(However, it suffers from precision errors in real life due to floating point arithmetics, which are not exact)
The recursion tree for fib(n) would be something like :
n
/ \
n-1 n-2 --------- maximum 2^1 additions
/ \ / \
n-2 n-3 n-3 n-4 -------- maximum 2^2 additions
/ \
n-3 n-4 -------- maximum 2^3 additions
........
-------- maximum 2^(n-1) additions
Using n-1 in 2^(n-1) since for fib(5) we will eventually go down to fib(1)
Number of internal nodes = Number of leaves - 1 = 2^(n-1) - 1
Number of additions = Number of internal nodes + Number of leaves = (2^1 + 2^2 + 2^3 + ...) + 2^(n-1)
We can replace the number of internal nodes to 2^(n-1) - 1 because it will always be less than this value :
= 2^(n-1) - 1 + 2^(n-1)
~ 2^n
Look at it like this. Assume the complexity of calculating F(k), the kth Fibonacci number, by recursion is at most 2^k for k <= n. This is our induction hypothesis. Then the complexity of calculating F(n + 1) by recursion is
F(n + 1) = F(n) + F(n - 1)
which has complexity 2^n + 2^(n - 1). Note that
2^n + 2^(n - 1) = 2 * 2^n / 2 + 2^n / 2 = 3 * 2^n / 2 <= 2 * 2^n = 2^(n + 1).
We have shown by induction that the claim that calculating F(k) by recursion is at most 2^k is correct.
You are correct that the depth of the tree is O(n), but you are not doing O(n) work at each level. At each level, you do O(1) work per recursive call, but each recursive call then contributes two new recursive calls, one at the level below it and one at the level two below it. This means that as you get further and further down the recursion tree, the number of calls per level grows exponentially.
Interestingly, you can actually establish the exact number of calls necessary to compute F(n) as 2F(n + 1) - 1, where F(n) is the nth Fibonacci number. We can prove this inductively. As a base case, to compute F(0) or F(1), we need to make exactly one call to the function, which terminates without making any new calls. Let's say that L(n) is the number of calls necessary to compute F(n). Then we have that
L(0) = 1 = 2*1 - 1 = 2F(1) - 1 = 2F(0 + 1) - 1
L(1) = 1 = 2*1 - 1 = 2F(2) - 1 = 2F(1 + 1) - 1
Now, for the inductive step, assume that for all n' < n, with n ≥ 2, that L(n') = 2F(n + 1) - 1. Then to compute F(n), we need to make 1 call to the initial function that computes F(n), which in turn fires off calls to F(n-2) and F(n-1). By the inductive hypothesis we know that F(n-1) and F(n-2) can be computed in L(n-1) and L(n-2) calls. Thus the total runtime is
1 + L(n - 1) + L(n - 2)
= 1 + 2F((n - 1) + 1) - 1 + 2F((n - 2) + 1) - 1
= 2F(n) + 2F(n - 1) - 1
= 2(F(n) + F(n - 1)) - 1
= 2(F(n + 1)) - 1
= 2F(n + 1) - 1
Which completes the induction.
At this point, you can use Binet's formula to show that
L(n) = 2(1/√5)(((1 + √5) / 2)n - ((1 - √5) / 2)n) - 1
And thus L(n) = O(((1 + √5) / 2)n). If we use the convention that
φ = (1 + √5) / 2 ≈ 1.6
We have that
L(n) = Θ(φn)
And since φ < 2, this is o(2n) (using little-o notation).
Interestingly, I've chosen the name L(n) for this series because this series is called the Leonardo numbers. In addition to its use here, it arises in the analysis of the smoothsort algorithm.
Hope this helps!
t(n)=t(n-1)+t(n-2)
which can be solved through tree method:
t(n-1) + t(n-2) 2^1=2
| |
t(n-2)+t(n-3) t(n-3)+t(n-4) 2^2=4
. . 2^3=8
. . .
. . .
similarly for the last level . . 2^n
it will make total time complexity=>2+4+8+.....2^n
after solving the above gp we will get time complexity as O(2^n)
The complexity of Fibonacci series is O(F(k)), where F(k) is the kth Fibonacci number. This can be proved by induction. It is trivial for based case. And assume for all k<=n, the complexity of computing F(k) is c*F(k) + o(F(k)), then for k = n+1, the complexity of computing F(n+1) is c*F(n) + o(F(n)) + c*F(n-1) + o(F(n-1)) = c*(F(n) + F(n-1)) + o(F(n)) + o(F(n-1)) = O(F(n+1)).
The complexity of recursive Fibonacci series is 2^n:
This will be the Recurrence Relations for recursive Fibonacci
T(n)=T(n-1)+T(n-2) No of elements 2
Now on solving this relation using substitution method (substituting value of T(n-1) and T(n-2))
T(n)=T(n-2)+2*T(n-3)+T(n-4) No of elements 4=2^2
Again substituting values of above term we will get
T(n)=T(n-3)+3*T(n-4)+3*T(n-5)+T(n-6) No of elements 8=2^3
After solving it completely, we get
T(n)={T(n-k)+---------+---------}----------------------------->2^k eq(3)
This implies that maximum no of recursive calls at any level will be at most 2^n.
And for all the recursive calls in equation 3 is ϴ(1) so time complexity will be 2^n* ϴ(1)=2^n
The O(2^n) complexity of Fibonacci number calculation only applies to the recursion approach. With a few extra space, you can achieve a much better performance with O(n).
public static int fibonacci(int n) throws Exception {
if (n < 0)
throws new Exception("Can't be a negative integer")
if (n <= 1)
return n;
int s = 0, s1 = 0, s2 = 1;
for(int i= 2; i<=n; i++) {
s = s1 + s2;
s1 = s2;
s2 = s;
}
return s;
}
I cannot resist the temptation of connecting a linear time iterative algorithm for Fib to the exponential time recursive one: if one reads Jon Bentley's wonderful little book on "Writing Efficient Algorithms" I believe it is a simple case of "caching": whenever Fib(k) is calculated, store it in array FibCached[k]. Whenever Fib(j) is called, first check if it is cached in FibCached[j]; if yes, return the value; if not use recursion. (Look at the tree of calls now ...)