I am trying to calculate T (n) = 2 T (n/2) + n (log n)^2.
Following the step I got:
=2^kT(n/2^k)+ nlog2 (n/2^(k-1))+ nlog2 (n/2^(k-2))+…+ n(log (n/2))^2 + n (log2 n)^2
when n=2^k I got:
But I have no idea about how to simplify the summation formula and get Θ() notation.
Any one can help? Thanks a lot
The summation you have doesn't look quite right to me. Let's re-derive it:
... after m iterations. Let's assume the stopping condition is n = 1 (without loss of generality):
... where we have employed two of the logarithm rules. As you can see the summation is in fact over "free indices" and not the logs themselves. Using the following integer power sums:
... we get:
To evaluate the Θ-notation, the highest order term is:
If you have read the Master theorem, you will realise that the question you have asked is actually the 2nd case of Master Theorem (Refer to the link above).
So, here a=2, b=2, and f(n) = 0[n^(c_crit)(log n)^k] where k=2 and c known as c_crit = log a to base b = 1.
So, by Master Theorem, T(n) = 0[(n^c_crit)(log k)^(k+1)] = 0[n(log n)^3]
Related
I know that the time complexity of a recursive function dividing its input by /2 is log n base 2,I have come across some interesting scenarios on
https://stackoverflow.com/a/42038565/8169857
Kindly help me to understand the logic behind the scenarios in the answer regarding the derivation of the formula
It's back to the recursion tree. Why for 1/2 is O(log2(n))? Because if n = 2^k, you should divide k times to reach to 1. Hence, the number of computation is k = log2(n) comparison at most. Now suppose it is (c-1)/c. Hence, if n = (c/(c-1))^k, we need log_{c/(c-1)}(n) operations to reach to 1.
Now as for any constant c > 1, limit log2(n)/log_{c/(c-1)}(n), n \to \infty is equal to a constant greater than zero, log_{c/(c-1)}(n) = \Theta(log2(n)). Indeed, you can say this for any constants a, b > 1, log_a(n) = \Theta(log_b(n)). Now, the proof is completed.
In Cracking the Coding Interview there's an example where the runtime for a recursive algorithm that counts the nodes in a binary search tree is O(2^(logN)). The book explains how we simplify to get O(N) like so...
2^p = Q
logQ = P
Let P = 2^(logN).
but I am lost at the step when they say Let P = 2^(logN). I don't understand how we know to set those two equal to one another, and I also don't understand this next step... (Although they tell me they do it by the definition of log base 2)
logP = logN
P = N
2^(logN) = N
Therefore the runtime of the code is O(N)
Assuming logN is log2N
This line:
Let P = 2^(logN).
Just assumes that P equals to 2^(logN). You do not know N yet, you just define how P and N relates to each other.
Later, you can apply log function to both sides of equation. And since log(2^(logN)) is logN, the next step is:
logP = logN
And, obviously, when logP = logN, then:
P = N
And previously you assumed that P = 2^(logN), then:
2^(logN) = N
Moreover, all of this could be simplified to 2^logN = N by definition of the log function.
The short answer is that the original question probably implicitly assumed that the logarithm was supposed to be in base 2, so that 2^(log_2(N)) is just N, by definition of log_2(x) as the inverse function of 2^y.
However, it's interesting to examine this a bit more carefully if the logarithm is to a different base. Standard results allow us to write the logarithm to base b as follows:
where ln(x) is the natural logarithm (using base e). Similarly, one can rewrite 2^x as follows:
We can then rewrite the original order-expression as follows:
which can be reduced to:
So, if the base b of our logarithm is 2, then this is clearly just N. However, if the base is different, then we get N raised to a power. For example, if b=10 we get N raised to the power 0.301, which is definitely a more slowly increasing function than O(N).
We can check this directly with the following Python script:
import numpy
import matplotlib.pyplot as plt
N = numpy.arange(1, 100)
plt.figure()
plt.plot(N, 2**(numpy.log2(N)))
plt.xlabel('N')
plt.ylabel(r'$2^{\log_2 N}$')
plt.figure()
plt.plot(N, 2**(numpy.log10(N)))
plt.xlabel('N')
plt.ylabel(r'$2^{\log_{10} N}$')
plt.show()
The graph this produces when we assume that the logarithm is to base two:
is very different from the graph when the logarithm is taken to base ten:
The definition of logarithm is “to what power does the base need to be raised to get this value” so if the base of the logarithm is 2, then raising 2 to that power brings us to the original value.
Example: N is 256. If we take the base 2 log of it we get 8. If we raise 2 to the power of 8 we get 256. So it is linear and we can make it to be just N.
If the log would be in a different base, for example 10, the conversion would just require dividing the exponent with a constant, making the more accurate form into N = 2^(log N / log 2), which can be changed into N / 2^(1 / log 2) = 2^log N. Here the divider for N on the left is a constant so we can forget it when discussing complexity and again come to N = 2^log N.
You can also test it by hand. Log2 of 256 is 8. Log2 of 128 is 7. 8/7 is about 1.14. Log10 of 256 is 2.4. Log10 of 128 is 2.1. 2.4/2.1 is about 1.14. So the base doesn’t matter, the value you get out isn’t the same but it is linear. So mathematically N doesn’t equal to 2^Log10 N, but in complexity terms it does.
So I've been studying sorting algorithms.
I am stuck on finding the complexity of merge sort.
Can someone please explain to me how h=1+lg(n)
If you keep dividing n by 2, you'll eventually get to 1.
Namely, it takes log2(n) divisions by 2 to make this happen, by definition of the logarithm.
Every time we divide by 2, we add a new level to the recursion tree.
Add that to the root level (which didn't require any divisions), and we have log2(n) + 1 levels total.
Here's a cooler proof. Notice that, rearranging, we have T(2n) - 2 T(n) = 2 c n.
If n = 2k, then we have T(2k + 1) - 2 T(2k) = 2 c 2k.
Let's simplify the mess. Let's define U(k) = T(2k) / (2 c).
Then we have U(k + 1) - 2 U(k) = 2k, or, if we define U'(k) = U(k + 1) - U(k):
U'(k) - U(k) = 2k
k is discrete here, but we can let it be continuous, and if we do, then U' is the derivative of U.
At that point the solution is obvious: if you've ever taken derivatives, then you know that if the difference of a function and its derivative is exponential, then the function itself has to be exponential (since only in that case will the derivative be some multiple of itself).
At that point you know U(k) is exponential, so you can just plug in an exponential for the unknown coefficients in the exponential, and plug it back in to solve for T.
For the algorithms below I need help with the following.
Algorithm Sum(m, n)
//Input: A positive integer n and another positive integer m ≤ n
//Output: ?
sum = 0
for i=m to n do
for j=1 to i do
sum = sum + 1
end for j
end for i
return sum
I need help figuring out what it computes? And what is the formula of the total number of additions sum=(sum+1).
I have The algorithm computes all of the positive integers between m and n including m and n.
The formula for the number of additions is.
m+m+1+…..+n
I don't get your questions...It seems you ask something but you also provide the answers by yourself already...anyway here's my answer to the questions...
For Q1, it seems you are asking the output and the number of total number of iteration (which is summation(m..n) = (n+m)(n-m+1)/2)
For Q2, it seems you are also asking how many times of the comparison has been performed, which is n-1 times.
To solve the recurrence T(n) = aT(n-1) + c where a,c is a constant,
by repeat substitution of n-2, n-3 ... until 1, you can find that T(n) = O(n)
PS: If it is a homework, maybe you did as you seem to have your own answer already, I strongly advice you to try go through some specific cases For Q1. For Q2 you should try to understand several methods to work out the recurrence relation, substitution method can be used to solve this kind of easy relation, many others may need to use master theorem.
Also you should be make yourself able to understand why Q2's complexity is actually the same as a normal naive for loop iterative method.
I need to Find the solution of the recurrence for n, a power of two if T(n)=3T(n/2)+n for n>1 and T(n)=1 otherwise.
using substitution of n=2^m,S(m)=T(2^(m-1)) I can get down to:
S(m)=2^m+3*2^(m-1)+3^2*2^(m-2)+⋯+3^(m-1) 2^1+3^m
But I have no idea how to simply that.
These types of recurrences are most easily solved by Master Theorem for analysis of algorithms which is explained as follows:
Let a be an integer greater than or equal to 1, b be a real number greater than 1, and c be a positive real number. Given a recurrence of the form -
T (n) = a * T(n/b) + nc where n > 1, then for n a power of b, if
Logba < c, T (n) = Θ(nc);
Logba = c, T (n) = Θ(nc * Log n);
Logba > c, T (n) = Θ(nlogba).
English translation of your recurrence
The most critical thing to understand in Master Theorem is the constants a, b, and c mentioned in the recurrence. Let's take your own recurrence - T(n) = 3T(n/2) + n - for example.
This recurrence is actually saying that the algorithm represented by it is such that,
(Time to solve a problem of size n) = (Time taken to solve 3 problems of size n/2) + n
The n at the end is the cost of merging the results of those 3 n/2 sized problems.
Now, intuitively you can understand that:
if the cost of "solving 3 problems of size n/2" has more weight than "n" then the first item will determine the overall complexity;
if the cost "n" has more weight than "solving 3 problems of size n/2" then the second item will determine the overall complexity; and,
if both parts are of same weight then solving the sub-problems and merging their results will have an overall compounded weight.
From the above three intuitive understanding, only the three cases of Master Theorem arise.
In your example, a = 3, b = 2 and c = 1. So it falls in case-3 as Logba = Log23 which is greater than 1 (the value of c).
The complexity therefore is straightforward - Θ(nlogba) = Θ(nlog23).
You can solve this using Masters theorem, but also by opening the recursion tree in the following way:
At the root of the recursion tree, you will have a work of n.
In the second stage, the tree splits into three parts, and in each part, the work will be n / 2.
Keep going until you reach the leaves. The entire work leaf will be: O (1) = O (n / 2 ^ k) when: n = 2 ^ k.
Note that at each step m have 3 ^ m splits.
Now we'll combine all the steps together, using the geometric progression and logarithms rules. In the end, you will get:
T(n) = 3T(n/2)+n = 2n^(log3)-2n
the calculation
Have a look here at page 60 http://www.cs.columbia.edu/~cs4205/files/CM2.pdf.
And maybe you should have asked here https://math.stackexchange.com/
The problems like this can be solved using Masters theorem.
In your case a = 3, b = 2 and f(n) = n.
So c = log_b(a) = log_2(3), which is bigger than 1, and therefore you fall into the first case. So your complexity is:
O(n^{log_2(3)}) = O(n^{1.58})