I know that with master theorem i'll have teta(n), but i'm trying to resolve the recurrence in other way like this:
T(n) = T(n/2) + n
T(n) = T(n/4) + 2n
T(n) = T(n/8) + 3n
.
.
.
T(n) = T(n/2^k) + kn
k=logn -> T(1) + **nlogn**
what's the problem?
There are two steps involved in solving a recurrence relation from scratch:
Somehow find the answer
Prove that it's correct
The "Somehow find the answer part" can be hard. Usual ways are to expand a few terms and extrapolate, like Rodojf did in the other answer, graph/table it out and eyeball it, or recognize a familiar pattern. All of those ways work pretty well for your example.
Let's put a few points in a table or graph (letting C = T(1)):
n | T(n)
===========
1 | C
2 | C + 2
4 | C + 6
8 | C + 14
16 | C + 30
Looks to me like T(n) = 2n - 2 + C
We can prove T(2x) = 2x+1 - 2 + C to be correct for all x>=0 by induction:
If x=0, then 2x+1 - 2 + C = C, so T(2x) = 2x+1 - 2 + C, from the table
For any x, T(2x) = 2x+1 - 2 + C ⇒ T(2x+1) = 2x+1 - 2 + C + 2x+1 = 2(x+1)+1 - 2 + C, by substitution
So the fact that the formula is correct for n=20 implies that it is correct for all n=2x
To cover all the other values of n, you should also note that there is always some 2x between n and 2n, and between n/2 and n, and that their values of T(2x) bound T(n).
This is a fair bit of work for one single case... which is why we remember the master theorem, which has been proven for all of the cases that match its patterns.
T(n)=T(n/2)+n=T(n/2^2)+n/2+n=T(n/2^3)+n/2^2+n/2+n= ... =
=T(n/2^k)+n/2^(k-1)+n/2^(n-2)+...+n=
=T(n/2^k)+n*(1-(1/2)^k)/(1-1/2)=
=T(n/2^k)+2*n*(1-(1/2)^k)<=
<=T(n/2^k)+2*n
If you know that T is a function defined and bounded in (0,0+e) for some small e>0 then you can conclude that T is O(n).
In the last chapters of Foundations of Algorithms - Richard E. Neapolitan.pdf you find several way to solve recurrence relation .
some way like :
1- recurrence tree
2- change variable and differential equations
and also you can see link
Related
So I'm trying to find the closed form solution for the Towers of Hanoi problem. I understand that the recurrence relation is T(n) = 2T(n-1) + 1, because it takes T(n-1) to move the top of the tower back and forth which is why there are two, and the "+ 1" is to move the base. However, I cannot understand why the closed solution is 2^n - 1.
When I am trying to solve for the answer and I use back substitution, I get as far as: T(n) = 8T(n-3) + 4 + 2 + 1, which is T(n) = 2^k (T(n - k)) + 2^k-1 + 2^k-2 + 2^k-3 where k is the step? I know the last part is also geometric series, which means it is 2^(n + 1) - 1/(2-1). But I just can't understand where the answer comes from.
edit:
is it because the geometric series part is not 2^k + 2^k-1 + ... + 2^k-k? which means that the geometric series is not 2^n + 1 - 1, but rather 2^n - 1. and we use H(0) as the base case --> so H(n - k), use k = n?
You can easily prove it by induction. Let's assume that T(n) = 2^n - 1 is true for a given n. Then:
T(n+1) = 2*T(n) + 1
= 2*(2^n-1) + 1
= 2^(n+1) - 2 + 1
= 2^(n+1) - 1
As we know that T(0) = 0 = 2^0 - 1 it proves that for any n the equality T(n) = 2^n - 1 is true.
A trick which sometimes works is to find another function for which the relation is simpler.
So we start from T(n)=2*T(n-1)+1.
It looks similar to T(n)=2*T(n-1) which has an obvious solution.
So we should transform the equation so that +1 is inside of 2*(...).
In this case it's T(n)+1=2*T(n-1)+2=2*(T(n-1)+1).
So T(n)+1=2^n*(T(0)+1) and T(n)=2^n*(T(0)+1)-1.
I have a snippet of algorithm and must find the worst-case recurrence and find its closed form. So far I have the worst-case recurrence:
T(n)= 2T(n/4) + C for n > 1.
I tried expanding it, and I have this form currently:
T(n) = 2kT(n/4k) + Ck
with k = log4(n) or k = (log2(n))/2.
I have T(1) = 1000.
I am at a loss on what to do next, or how to find its closed form exactly. I still cannot see a pattern in the algorithm or my expansion of T(n). Any insight would be great, thank you.
What you can get is a closed formula when n = 4^k:
T(4^k) = 2^k x 10^3 + C + 2C + ... + 2^(k-1)C
= 2^k x 10^3 + (2^k - 1)C
Where the last eqaulity comes from the geometric series formula.
For all other n, I think the best you can do is to apply the master theorem
Your equation falls in case 1 of the theorem (you have a = 2, b = 4, c = 0).
Therefore:
log_b(a) = 1 / 2
and
T(n) = O(sqrt(n))
I'm not sure if it admits an unique closed form.
Karatsuba Algorithm involves the recursion relation T(n) = 3T(n/2) + n.
By the recursion tree method, we can approximate the big O of T to be O(nlog23)
However, by the substitution method I am having trouble verifying the approximate result I found by the recursion tree method
I'll simply use lg 3 to mean log23.
Substitution method:
Hypothesis -> T(n) <= cnlg 3 where c is a positive constant
Proof -> T(n) <= 3c(n/2)lg 3 + n
= cnlg 3 + n
But step 2 of the proof shows that I cannot prove my hypothesis because of n term.
I modified step 2 of proof
T(n) <= cnlg 3 + nlg 3
= (c+1)nlg 3
And later realized I had made a mistake because the hypothesis is not proven.
T(n) <= cnlg 3 has to be proven, not T(n) <= (c+1)nlg 3
But the answer is T(n) is O(nlg 3)
When using the substitution method, you sometimes have to strengthen the inductive hypothesis and guess a more complex form of the expression that upper-bounds the recurrence.
Try making a guess of the form T(n) ≤ c0 nlg 3 - c1n. Now that you are subtracting some term of the form c1 n, you can probably make the recurrence work out by using some of the linear term to offset the n term added in later.
For example:
T(n) ≤ 3T(n / 2) + n
≤ 3(c0(n/2)lg 3 - c1(n/2)) + n
= 3(c0(n/2)lg 3) - 3c1n/2 + n
Now, choose c1 so that -3c1n/2 + n = -c1n. This solves to
-3c1n/2 + n = -c1n
-3c1/2 + 1 = -c1
-3c1 + 2 = -2c1
2 = c1
This choice of c1 will then let you cancel out the +n term successfully, letting the induction work successfully.
Hope this helps!
So, someone posted this question earlier, but essentially no effort was put into it, it was poorly tagged and then closed. Nonetheless, I think it could have been a good question. I'm posting because according to the OP, my answer (posted in a comment) did not agree with the solution. So, I'm trying to figure out what I'm doing incorrectly (assuming that the answer that he has in indeed correct):
We have:
T(N) = T(N-1) + T(N-2) + T(N-3)
where N > 3. He didn't have a base case listed, but since N > 3, I assumed that there are probably 3 base cases for T(3), T(2) and T(1) . To calculate T(K), we do the following:
T(K) = T(K-1) + T(K-2) + T(K-3)
Then we must calculate:
T(K-1) = T((K-1)-1) + T((K-1)-2) + T((K-1)-3)
T(K-2) = T((K-2)-1) + T((K-2)-2) + T((K-2)-3)
T(K-3) = T((K-3)-1) + T((K-3)-2) + T((K-3)-3)
and so on...
Here's a tree-representation:
L0 T(K)
/ | \
L1 T(K-1) T(K-2) T(K-3)
/ | \ / | \ / | \
L2 T((K-1)-1) T((K-1)-2) T((K-1)-3) T((K-2)-1) T((K-2)-2) T((K-2)-3) T((K-3)-1) T((K-3)-2) T((K-3)-3)
... ... ...
So we have 3 children, then 9 children, then 27 children,..., until we hit our base cases. Hence, the algorithm is O(3^(N-3)), the N-3 is there to account for the three base cases, ie after T(4), we can only have bases cases, no more branching.
The actual solution was never provided, but like I said, I'm told that this is incorrect. Any help would be appreciated.
This is a cool method that I learned, so I thought I'll share it with you.It's really simple to estimate the time complexity.
Looking at the recurrence we guess that the time complexity is exponential.
Lets say:
T(N)=x^n
The given recurrence is
T(N) = T(N-1) + T(N-2) + T(N-3)
Substituting
x^n = x^n-1 + x^n-2 + x^n-3
Dividing throughout by x^n-3
x^3 = x^2 + x^1 + 1
Rearranging
x^3 - x^2 - x - 1=0
You can find out it's cubic roots here.
This cubic equation has one real root( 1.8392867552141612) and two complex roots(of magnitude 0.7373527).
Thus asymptotically our algorithm's running time is bounded by T(N)=1.839^n.
The recurrence you have set up is the following:
T(n) = T(n - 1) + T(n - 2) + T(n - 3)
I assume the base cases are probably
T(0) = T(1) = T(2) = 1
If you start to expand out the terms of this recurrence, you get
T(0) = 1
T(1) = 1
T(2) = 1
T(3) = 3
T(4) = 5
T(5) = 9
T(6) = 17
T(7) = 31
...
There doesn't seem to be an obvious pattern here. Fortunately, we can go to the Online Encyclopedia of Integer Sequences and punch in the terms 1, 1, 1, 3, 5, 9, 17 and you'll find that this is the Tribonacci sequence whose first three terms are 1.
If you look at the information about the Tribonacci numbers, you'll see the following:
a(n)/a(n-1) tends to the tribonacci constant, 1.839286755...
(here, a(n) is the notation the site uses for my T(n)). Since the ratio of consecutive terms of the Tribonacci sequence tends to approximately 1.839286755, we know that the Tribonacci sequence must be exponentially growing, and it grows exponentially at a rate that is approximately Θ(1.839286755n). (Compare this to the Fibonacci sequence, which is known to grow at Θ(φn), where φ is the golden ratio). Doing some further reading on Wikipedia gives this formula for the Tribonacci constant:
and confirms the exponential growth rate.
Consequently, we can conclude that the runtime is Θ(1.839286755n).
So... how would you compute this on your own? The easiest way to do this (and I think the way that these values are known) is to use generating functions. You can try to derive a generating function for the recurrence you have written out here, then try to rewrite the generating function in a closed-form to get the exact value. This is one way to get the closed-form for Fibonacci numbers, and it should generalize here (though it might be a lot of slogging through unpleasant math.) Alternatively, as #tmyklebu points out, you could write out this matrix:
| 0 1 0 |
M = | 0 0 1 |
| 1 1 1 |
and compute its eigenvalues, the largest of which will come out to the Tribonacci constant. (Note that this matrix has the property that
| 0 1 0 | |a| | b |
| 0 0 1 | x |b| = | c |
| 1 1 1 | |c| |a + b + c|
Consequently, if you put three consecutive values from the recurrence into a column vector v and compute Mv, you get back a new column vector holding the latter two values from the recurrence, plus the next value in the recurrence. In this way, you can compute the kth value of the recurrence by computing Mkv and looking at the first component of the vector.)
Hope this helps!
As a few people have noticed, this recurrence is different than the original recurrence T(N) = T(N-1) + T(N-2) - T(N-3). I prefer the approach of assuming T(N)=x^N given by #Aravind. With this recurrence, you get the characteristic equation x^3-x^2-x+1=(x-1)^2(x+1). (This will be the characteristic equation for the matrix approach of #templatetypedef, and the denominator of the generating function if you took that approach.)
The repeated root causes all sorts of difficulties. The matrix isn't diagonalizable. The generating function has a repeated denominator when you factor it. When you assume T(N)=x^N, you only get two linearly independent solutions, and need a third.
In general, when you assume T(N)=x^N and get a double root r, that means the linearly independent solutions are r^N and N*r^N (a triple root would introduce N^2*r^N). So in our case, the three linearly independent solutions to the recurrence are (-1)^N, 1^N=1, and N*1^N=N. This means that a general solution is T(N)=A(-1)^N+B+C*N, and you use the initial conditions to determine A, B, and C. If C!=0, then T(N)=Θ(N), otherwise T(N)=Θ(1). Which is probably not that realistic for an algorithm.
I have this recurrence:
W(n)= 2W(floor(n/2)) + 3
W(2)=2
My try is as follow:
the tree is like this:
W(n) = 2W(floor(n/2)) + 3
W(n/2) = 2W(floor(n/4)) + 3
W(n/4) = 2W(floor(n/8)) + 3
...
the hight of the tree : I assume its lgn because the tree has 2 branches at every expanding process, not sure though :S
the cost of the last level : 2^lgn * W(2) = 2n
the cost of all levels until level h-1 : 3 * sigma from 0 to lgn-1 of (2^i), which is a geometric series = 3 (n-1)
So, T(n) = 5n - 3 which belong to Theta(n)
my question is: Is that right?
I don't think it's exactly 5n-3 except n is 2t, but your theta is right if you look at Master Theorem, there is no need to calculate it (but its good for startup):
assume you have:
T(n) = aT(n/b) + f(n), where a>=1, b>1 then:
if f(n) = nlogba-eps for any eps > 0 then T(n) = nlogba like your case, in which a=b=2, f(n) = O(1).
f(n) = Theta(nlogba * logkn) then T(n)=Theta(nlogba * logk+1n).
Otherwise is Theta(f(n)). (see detail of constraint in this case in CLRS or wiki, ...)
for detail see wiki.
Well, if you calculate W(4), you find W(4) = 2*W(2) + 3 = 2*2 + 3 = 7, but 5*4 - 3 = 17, so your result for T(n) is not correct. It is close, though, there's just a minor slip in your reasoning (or possibly in a certain other place).
Edit: To be specific, your calculation would work if W(1) was given, but it's W(2) in the question. Either the latter is a typo or you're off by one with the height. (and of course, what Saeed Amiri said.)