I'm solving some recurrences and I didn't quite understand when to put O or Theta on the result.
If I have a recurrence like this:
T(N)=2T(n/2)+n
and I solve this with the iterative method I know the result is an O (nlogn).
could I also use theta (nlogn)?
is there any verification I could do to establish this?
By definition,
If T(n) = O(g(n)) as well as T(n) = Ω(g(n)),
then we can say that T(n) = Θ(g(n))
In case of T(n) = 2T(n/2) + n you're dividing n inputs into two branches and then merging those inputs from bottom up. This method is also known as Divide and Conquer.
So for every branch you're doing n operations. How many number of times? That is the height of the tree. For binary tree here, the height is log_2(n). Thus the total operations you're doing is at most n * log_2(n).
Since the height of the tree is fixed and the number of operations performed is also fixed so the total operations you'll be doing is also at least n * log_2(n).
Thus for your above function, T(n) = Θ(n * log_2(n)).
Related
We know that for some algorithm with time complexity of lets say T(n) = n^2 + n + 1 we can drop the less significant terms and say that it has a worst case of O(n^2).
What about when we're in the middle of calculating time complexity of an algorithm such as T(n) = 2T(n/2) + n + log(n)? Can we just drop the less significant terms and just say T(n) = 2T(n/2) + n = O(n log(n))?
In this case, yes, you can safely discard the dominated (log n) term. In general, you can do this any time you only need the asymptotic behaviour rather than the exact formula.
When you apply the Master theorem to solve a recurrence relation like
T(n) = a T(n/b) + f(n)
asymptotically, then you don't need an exact formula for f(n), just the asymptotic behaviour, because that's how the Master theorem works.
In your example, a = 2, b = 2, so the critical exponent is c = 1. Then the Master theorem tells us that T(n) is in Θ(n log n) because f(n) = n + log n, which is in Θ(nc) = Θ(n).
We would have reached the same conclusion using f(n) = n, because that's also in Θ(n). Applying the theorem only requires knowing the asymptotic behaviour of f(n), so in this context it's safe to discard dominated terms which don't affect f(n)'s asymptotic behaviour.
First of all you need to understand that T(n) = n^2 + n + 1 is a closed form expression, in simple terms it means you can inject some value for n and you will get the value of this whole expression.
on the other hand T(n) = 2T(n/2) + n + log(n) is a recurrence relation, it means this expression is defined recursively, to get a closed form expression you will have to solve the recurrence relation.
Now to answer your question, in general we drop lower order terms and coefficients when we can clearly see the highest order term, in T(n) = n^2 + n + 1 its n^2. but in a recurrence relation there is no such highest order term, because its not a closed form expression.
but one thing to observe is that highest order term in the closed form expression of a recurrence relation would be result of depth of recurrence tree multiplied with the highest order term in recurrence relation, so in your case it would be depthOf(2T(n/2)) * n, this would result in something like logn*n, so you can say that in terms of big O notation its O(nlogn).
after analyzing the algorithm complexity I have a few questions:
For the best case complexity - the recurrence relation is T(n) = T(n/2) + dn which implies that the complexity is Θ(n).
So by the master theory I can clearly see why this is true , but when I draw the algorithm recursive calls as a tree I don't fully understand the final result. (It seems like I have one branch in height of log(n) which in each level I operate a partition O(n) - so it suppose to be nlog(n) .
(just for memorizing - this is very similar to the best case of mergeSort algorithem , but here we ignore the unwanted sub-array after partition).
Thanks!
It is as Yves Daoust wrote. Image it with real numbers, i.e. n=1024
T(n) = T(n/2) + dn
T(1024) = T(512) + 1024
T(512) = T(256) + 512
....
T(2) = T(1) + 2 -> this would be the last operation
Therefore you get 1024+512+256+...+1 <= 2048, which is 2n
You must think about that dn is as big as n, but in recurrence relation the n is not global variable, it is local variable based on method you call.
So there is log(n) calls but they do not take n-time everyone, they take less and less time.
The question is :
T(n) = √2*T(n/2) + log n
I'm not sure whether the master theorem works here, and kinda stuck.
This looks more like the Akra-Bazzi theorem: http://en.wikipedia.org/wiki/Akra%E2%80%93Bazzi_method#The_formula with k=1, h=0, g(n)=log n, a=(2)^{1/2}, b=1/2. In that case, p=1/2 and you need to evaluate the integral \int_1^x log(u)/u^{3/2} du. You can use integration by parts, or a symbolic integrator. Wolfram Alpha tells me the indefinite integral is -2(log u + 2)/u^{1/2} + C, so the definite integral is 4 - 2(log x + 2)/x^{1/2}. Adding 1 and multiplying by x^{1/2}, we get T(x) = \Theta(5x^{1/2} - 2 log x - 4).
Master theorem have only constrains on your a and b which holds for your case. The fact that a is irrational and you have log(n) as your f(n) has no relation to it.
So in your case your c = log2(sqrt(2)) = 1/2. Since n^c grows faster than your log(n), the complexity of the recursion is O(sqrt(n)).
P.S. solution of Danyal is wrong as the complexity is not nlogn and the solution of Edward Doolittle is correct, also it is an overkill in this simple case.
As per master theorem, f(n) should be polynomial but here
f(n) = logn
which is not a polynomial so it can not be solved by master theorem as per rules. I read somewhere about the fourth case as well. I must mention that as well.
It is also discussed here:
Master's theorem with f(n)=log n
However, there is a limited "fourth case" for the master theorem, which allows it to apply to polylogarithmic functions.
If
f(n) = O(nlogba logk n), then T(n) = O(nlogba log k+1 n).
In other words, suppose you have T(n) = 2T (n/2) + n log n. f(n) isn't a polynomial, but f(n)=n log n, and k = 1. Therefore, T(n) = O(n log2 n)
See this handout for more information: http://cse.unl.edu/~choueiry/S06-235/files/MasterTheorem-HandoutNoNotes.pdf
I know that we can apply the Master Theorem to find the running time of a divide and conquer algorithm, when the recurrence relation has the form of:
T(n) = a*T(n/b) + f(n)
We know the following :
a is the number of subproblems that the algorithm divides the original problem
b is the size of the sun-problem i.e n/b
and finally.. f(n) encompasses the cost of dividing the problem and combining the results of the subproblems.
Now we then find something (I will come back to the term "something")
and we have 3 cases to check.
The case that f(n) = O(n^log(b)a-ε) for some ε>0; Then T(n) is O(n*log(b)a)
The case that f(n) = O(n^log(b)a) ; Then T(n) is O(n^log(b)a * log n).
If n^log(b)a+ε = O(f(n)) for some constant ε > 0, and if a*f(n/b) =< cf(n) for some constant
c < 1 and almost all n, then T(n) = O(f(n)).
All fine, I am recalling the term something. How we can use general examples (i.e uses variables and not actual numbers) to decide which case the algorithm is in?
In instance. Consider the following:
T(n) = 8T(n/2) + n
So a = 8, b = 2 and f(n) = n
How I will proceed then? How can I decide which case is? While the function f(n) = some big-Oh notation how these two things are comparable?
The above is just an example to show you where I don't get it, so the question is in general.
Thanks
As CLRS suggests, the basic idea is comparing f(n) with n^log(b)a i.e. n to the power (log a to the base b). In your hypothetical example, we have:
f(n) = n
n^log(b)a = n^3, i.e. n-cubed as your recurrence yields 8 problems of half the size at every step.
Thus, in this case, n^log(b)a is larger because n^3 is always O(n) and the solution is: T(n) = θ(n^3).
Clearly, the number of subproblems vastly outpaces the work (linear, f(n) = n) you are doing for each subproblem. Thus, the intuition tells and master theorem verifies that it is the n^log(b)a that dominates the recurrence.
There is a subtle technicality where the master theorem says that f(n) should be not only smaller than n^log(b)a O-wise, it should be smaller polynomially.
The Master Method is a direct way to get the solution. The Master Method works only for following type of recurrences or for recurrences that can be transformed to following type.
T(n) = a T(n / b) + f(n) where a ≥ 1, b > 1, and f(n) = Θ(nc).
There are following three cases:
If c < logb(a) then T(n) = Θ(nlogb(a)).
If c = logb(a) then T(n) = Θ(nc log(n)).
If c > logb(a) then T(n) = Θ(f(n)).
In the Master Method, if f(n) contains some term of log(n), is it possible to solve this by the Master Method?
for example in
T(n)=4T(n/2)+n^2logn
Here master's theorem appplicable or not
It is not really possible to tell directly whether or not the Master Method works for some logarithmic function. This would depend on the specific recurrence you're trying to solve. It all depends on how f grows in comparison to nlogb a.
In the example given by JPC (where T(n) = 4T(n/2) + log(n)), it is indeed possible. However, also consider the example T(n) = 2T(n/5) + log(n). In this recurrence it is harder to determine whether nlog5 2 grows faster than log(n). If the logarithmic function f(n) gets more complex (e.g. log3(n/2)), it becomes even harder.
In short, it may be hard to determine for logarithmic functions how they grow when compared to an exponential function when the exponent is less than 1 (for exponents >= 1, log(n) is always faster). If it doesn't seem to work for you, you'll have to use other techniques to solve the recurrence.