I am quite frustrated over this.
In CLRS 3rd edition, page 95 (chapter 4.5), it mentions that recurrences like
T(n) = 2T(n/2) + n lg n
cannot be solved with the Master Theorem because the difference
f(n)/n^(log_b(a)) = (n lg n)/n^1 = lg n
is not polynomial.
But then I come across pages like this this where, at the bottom of the page, it mentions the exact same recurrence and says that it CAN be solved with the Master Theorem because it falls into an "extended case 2" even though the difference is non-polynomial. It becomes n lg^2 n (incrementing the log factor on f(n) by one).
Then I come across pages like this where in example (e) seems like a clear application of Extended Case 2 (the recurrence is T(n) = 4T(n/2) + n^2 lg n), but then the solution is not n^2 log^2 n, but rather n^2 log n! Am I wrong or is the paper wrong?
Can anyone please clear up the contradictions and make it very clear exactly when Master Theorem can be used and when it cannot? When does the polynomial-difference check matter, and when does it not? Is the extended case 2 usable, or does it actually violate something?
EDIT:
I tried solving recurrence (e) directly from the second paper and I get:
T(n) = n^2 lg^2(n)/2 + n^2 lg(n)/2
Is this not big theta n^2 lg^2 n?
The book states that it cannot be solved using Case 3:
even though it appears to have the proper form: ... You might mistakenly think that case 3 should apply
However, this recurrence formula can be solved using master theorem, case 2.
T(n) = 2T(n/2) + nlgn:
We define:
a = 2, b = 2, f(n) = nlgn
Using Master theorem case 2:
c = log_2(2) = 1
k = 1
And f(n) is indeed in Theta(nlogn).
So, all conditions to master theorem case 2 apply, and we can deduce:
T(n) is in Theta(n^c * log(n)^(k+1)) = Theta(n*log(n)^2)
Long story short, Master theorem have 3 cases. Each case have it's prerequisites to be applied. Case 3 have more complicated prerequisites, because it also requires convergence.
Since the prerequisites for case 3 does not apply for this formula, you cannot use case 3. However, the prerequisites of case 2 - do apply, and you can use it.
Related
I know how to calculate the master theorem and I managed to calculate it for best and average case.
T(n) = 2T(n/2) + Theta(n)
The worst case equation is
T(n) = T(n-1) + Theta(n)
If I am correct a is 1, b is n/(n-1) and f(n) is n.
But how do I choose the right case of the master theorem and get a worst-case time complexity of Theta(n^2)?
Thanks!
As #DavidEisenstat pointed out in the comments, the Master Theorem doesn’t apply to the recurrence you’ve come up with here.
To give some context as to why this is - the Master Theorem is specifically designed for the case where
the number of subproblems is a constant, and
the sizes of the subproblems decays geometrically.
In this case, that second requirement doesn’t hold, since your recurrence has the problem size decay linearly rather than geometrically.
You are correct, however, that the recurrence solves to Θ(n2). To see why, note that if you unroll the recurrence, you get that the cost is
Θ(n + (n-1) + (n-2) + ... + 2 + 1)
= Θ(n(n+1)/2)
= Θ(n2).
Hope this helps!
I am following a course of Algorithms and Data Structures.
Today, my professor said the complexity of the following algorithm is 2n.
I waited till the lesson was over, approached him and told him I actually believed it was an O(n) algorithm, and I did the computation to prove it, and wanted to show them to it, but he continued to say it was not, without giving me any convincing explanation.
The algorithm is recursive, and it has this complexity:
{ 1 if n=1
T(n) = {
{ 2T(n/2) otherwise
I computed it down to be a O(n), this way:
Let's expand T(n)
T(n) = 2 [2 * T(n/(2^2))]
= 2^2 * T(n/(2^2))
= 2^2 * [2 * T(n/(2^3))]
= 2^3 * T(n/(2^3))
= ...
= 2^i * T(n/(2^i)).
We stop when the term inside the T is 1, that is:
n/(2i) = 1 ==> n = 2i ==> i = log n
After the substitution, we obtain
T(n) = 2^log n * T(1)
= n * 1
= O(n).
Since this algorithm jumped out of a lesson on Merge Sort, I noted how Merge Sort, which notoriously is O(n log n) has a complexity of 2T(n/2) + Θ(n) (obviously higher than 2T(n/2)), and I asked him why is it, that an algorithm with a lower complexity, gets a higher big-O. Because, at this point, it's counter intuitive for me. He replied, words for words, "If you think that is counter-intuitive, you have serious problem in your math."
My questions are:
Is there any fallacy in my demonstration?
Wouldn't the last situation be counter-intuitive?
Yes, this is also a vent.
Proof - 1
This recurrence falls in case - 3 of Master Theorem, with
a = 2;
b = 2; and,
c = -∞
and thus Logba = 1 which is bigger than -∞. Therefore the running time is Θ(n1) = Θ(n).
Proof - 2
Intuitively, you are breaking the problem of size n into 2 problems of size n/2 and the cost to join the result of two sub-problems is 0 (i.e. there is no constant component in the recurrence).
Hence at the bottom-most level you have n problems of cost 1 each, resulting in the running time of n * O(1) which is equal to O(n).
Edit: Just to complete this answer I will also add the answers to specific questions asked by you.
Is there any fallacy in my demonstration?
No. It is correct.
Wouldn't the last situation be counter-intuitive?
Definitely it is counter-intuitive. See the Proof-2 above.
You are correct in computing the time complexity of the given relation. If we are measuring the input size in n(which we should) then your professor is wrong in claiming that the time complexity is 2^n.
You should probably discuss it with him and clear any misunderstanding that you might have.
You are clearly correct that a function T(n) which satisfies that recurrence relation is O(n). It is essentially obvious since it says that the complexity of a given problem is twice that of a problem which is half the size. You can't get much more linear than that. For example -- the complexity of searching through a list of 1000 elements with a linear search is twice that of searching through a list with 500 elements.
If your professor is also correct then perhaps you are incorrect about the complexity satisfying that recurrence. Alternatively, sometimes there is some confusion about how the input size is being measured. For example, an integer n is exponential in the number of bits needed to specify it. For example -- brute force trial division of an integer n is O(sqrt(n)) which is much better than O(n). The reason that this doesn't contradict that fact that brute force factoring is essentially worthless for e.g. cracking RSA is because for say a 256 bit key the relevant n is around 2^256.
I am trying to apply the Master's Theorem to a recurrence of this type:
T(n) = T(n/2) + 2^n
However, f(n) = 2^n doesn't seem to fit any of the three cases described in the master's theorem, which all seem to have base n instead of base 2. How can I solve a recurrence of this type, could anyone please help ? Thanks.
If none of the cases of the theorem applies, then the theorem won't solve your recurrence. It can't solve every single recurrence out there.
To address your issue: what you get by repeatedly substituting the recursive case is T(n) = 2^n + 2^(n/2) + 2^(n/4) + ... + 2, and since there are log n many terms to add up, you end up with something below 2^(n+1), so in total you're in Θ(2^n).
We can take log on both sides and solve. It will fall in case 3 of Master's theorem.
How do you go about finding the asymptotic complexity based off a running time? For example:
If the run time of a recursive algorithm is given as
T(n) = 2 T(n/2) + O(n)
considering the Master Theorem, what is its asymptotic complexity?
Could someone explain the steps on how to figure this out?
For the Master Theorem, there are 3 different cases to solve.
First step is to understand which case applies.
For questions involving Master Theorem, our general formula is T(n) = aT(n/b) + f(n)
So first thing to do is compare n^(logb a) with f(n).
Simply whichever is bigger that's the complexity of it(these are Cases 1 and 3) and if they are equal then you multiply the result with lgn, such as if you have a case like T(n) = 16 T(n/4) + O(n^2) then
n^(logb a) would make n^2 as well as f(n) = n^2 so the answer would be Θ(n^(2)*lgn), which is by using the case #2 of the Master Theorem.
T (1) = c
T (n) = T (n/2) + dn
How would I determine BigO of this quickly?
Use repeated backsubstitution and find the pattern. An example here.
I'm not entirely sure what dn is, but assuming you mean a constant multiplied by n:
According to Wolfram Alpha, the recurrence equation solution for:
f(n) = f(n / 2) + cn
is:
f(n) = 2c(n - 1) + c1
which would make this O(n).
Well, the recurrence part of the relationship is the T(n/2) part, which is in effect halving the value of n each time.
Thus you will need approx. (log2 n) steps to get to the termination condition, hence the overall cost of the algorithm is O(log2 n). You can ignore the dn part as is it a constant-time operation for each step.
Note that as stated, the problem won't necessarily terminate since halving an arbitrary value of n repeatedly is unlikely to exactly hit 1. I suspect that the T(n/2) part should actually read T(floor (n / 2)) or something like that in order to ensure that this terminates.
use master's theorem
see http://en.wikipedia.org/wiki/Master_theorem
By the way, the asymptotic behaviour of your recurrence is O(n) assuming d is positive and sufficiently smaller than n (size of problem)