I need to Find the solution of the recurrence for n, a power of two if T(n)=3T(n/2)+n for n>1 and T(n)=1 otherwise.
using substitution of n=2^m,S(m)=T(2^(m-1)) I can get down to:
S(m)=2^m+3*2^(m-1)+3^2*2^(m-2)+⋯+3^(m-1) 2^1+3^m
But I have no idea how to simply that.
These types of recurrences are most easily solved by Master Theorem for analysis of algorithms which is explained as follows:
Let a be an integer greater than or equal to 1, b be a real number greater than 1, and c be a positive real number. Given a recurrence of the form -
T (n) = a * T(n/b) + nc where n > 1, then for n a power of b, if
Logba < c, T (n) = Θ(nc);
Logba = c, T (n) = Θ(nc * Log n);
Logba > c, T (n) = Θ(nlogba).
English translation of your recurrence
The most critical thing to understand in Master Theorem is the constants a, b, and c mentioned in the recurrence. Let's take your own recurrence - T(n) = 3T(n/2) + n - for example.
This recurrence is actually saying that the algorithm represented by it is such that,
(Time to solve a problem of size n) = (Time taken to solve 3 problems of size n/2) + n
The n at the end is the cost of merging the results of those 3 n/2 sized problems.
Now, intuitively you can understand that:
if the cost of "solving 3 problems of size n/2" has more weight than "n" then the first item will determine the overall complexity;
if the cost "n" has more weight than "solving 3 problems of size n/2" then the second item will determine the overall complexity; and,
if both parts are of same weight then solving the sub-problems and merging their results will have an overall compounded weight.
From the above three intuitive understanding, only the three cases of Master Theorem arise.
In your example, a = 3, b = 2 and c = 1. So it falls in case-3 as Logba = Log23 which is greater than 1 (the value of c).
The complexity therefore is straightforward - Θ(nlogba) = Θ(nlog23).
You can solve this using Masters theorem, but also by opening the recursion tree in the following way:
At the root of the recursion tree, you will have a work of n.
In the second stage, the tree splits into three parts, and in each part, the work will be n / 2.
Keep going until you reach the leaves. The entire work leaf will be: O (1) = O (n / 2 ^ k) when: n = 2 ^ k.
Note that at each step m have 3 ^ m splits.
Now we'll combine all the steps together, using the geometric progression and logarithms rules. In the end, you will get:
T(n) = 3T(n/2)+n = 2n^(log3)-2n
the calculation
Have a look here at page 60 http://www.cs.columbia.edu/~cs4205/files/CM2.pdf.
And maybe you should have asked here https://math.stackexchange.com/
The problems like this can be solved using Masters theorem.
In your case a = 3, b = 2 and f(n) = n.
So c = log_b(a) = log_2(3), which is bigger than 1, and therefore you fall into the first case. So your complexity is:
O(n^{log_2(3)}) = O(n^{1.58})
Related
I know that the time complexity of a recursive function dividing its input by /2 is log n base 2,I have come across some interesting scenarios on
https://stackoverflow.com/a/42038565/8169857
Kindly help me to understand the logic behind the scenarios in the answer regarding the derivation of the formula
It's back to the recursion tree. Why for 1/2 is O(log2(n))? Because if n = 2^k, you should divide k times to reach to 1. Hence, the number of computation is k = log2(n) comparison at most. Now suppose it is (c-1)/c. Hence, if n = (c/(c-1))^k, we need log_{c/(c-1)}(n) operations to reach to 1.
Now as for any constant c > 1, limit log2(n)/log_{c/(c-1)}(n), n \to \infty is equal to a constant greater than zero, log_{c/(c-1)}(n) = \Theta(log2(n)). Indeed, you can say this for any constants a, b > 1, log_a(n) = \Theta(log_b(n)). Now, the proof is completed.
I am trying to calculate T (n) = 2 T (n/2) + n (log n)^2.
Following the step I got:
=2^kT(n/2^k)+ nlog2 (n/2^(k-1))+ nlog2 (n/2^(k-2))+…+ n(log (n/2))^2 + n (log2 n)^2
when n=2^k I got:
But I have no idea about how to simplify the summation formula and get Θ() notation.
Any one can help? Thanks a lot
The summation you have doesn't look quite right to me. Let's re-derive it:
... after m iterations. Let's assume the stopping condition is n = 1 (without loss of generality):
... where we have employed two of the logarithm rules. As you can see the summation is in fact over "free indices" and not the logs themselves. Using the following integer power sums:
... we get:
To evaluate the Θ-notation, the highest order term is:
If you have read the Master theorem, you will realise that the question you have asked is actually the 2nd case of Master Theorem (Refer to the link above).
So, here a=2, b=2, and f(n) = 0[n^(c_crit)(log n)^k] where k=2 and c known as c_crit = log a to base b = 1.
So, by Master Theorem, T(n) = 0[(n^c_crit)(log k)^(k+1)] = 0[n(log n)^3]
Let us say we are given a number n.
We need to find the number of values S ^ (S+n) lying in the range [L, R].
(Where S is any non-negative integer and ^ is the bitwise xor operator).
I can easily do this if n is power of two (they have a very useful pattern)
I am not sure how to solve this for any general n.
Any suggestions?
EDIT:
n is also a non-negative integer.
n, L, R are all less than 10^18.
This was a programming question in some practice test which i gave sometime back, i just remembered this seeing a similar question in StackOverflow today.
EDIT 2:
Explaining with an example,
say n = 1.
Then we know that S ^ (S + 1) will always have a binary representation of all ones. eg: 1,3,7,...
So solving this is easy we just have to count the number of such numbers within the Range [L,R] it is quite simple.
For n = any power of 2 similar methods work. But i have no idea what to do if n is not a power of 2.
Let C(n) be the (infinite) set of numbers that can be written as S ^ (S + n) for some S.
We have the following recurrence relations on the sets C(n):
If n = 2k is even, then C(n) = {2x : x in C(k)};
If n = 2k + 1 is odd, then C(n) = {2x + 1 : x in C(k)} union {2x + 1 : x in C(k + 1)}.
An algorithm can be deduced from these relations. More precisely, a pair (C(n), C(n + 1)) can be deduced from (C(n / 2), C(n / 2 + 1)). Note that the union above is really a disjoint union, because every element in C(n) has the same parity as n, hence C(k) and C(k + 1) do not intersect.
Proof of the recurrence relations:
Simply look at the last binary digits of n and S.
So I've been studying sorting algorithms.
I am stuck on finding the complexity of merge sort.
Can someone please explain to me how h=1+lg(n)
If you keep dividing n by 2, you'll eventually get to 1.
Namely, it takes log2(n) divisions by 2 to make this happen, by definition of the logarithm.
Every time we divide by 2, we add a new level to the recursion tree.
Add that to the root level (which didn't require any divisions), and we have log2(n) + 1 levels total.
Here's a cooler proof. Notice that, rearranging, we have T(2n) - 2 T(n) = 2 c n.
If n = 2k, then we have T(2k + 1) - 2 T(2k) = 2 c 2k.
Let's simplify the mess. Let's define U(k) = T(2k) / (2 c).
Then we have U(k + 1) - 2 U(k) = 2k, or, if we define U'(k) = U(k + 1) - U(k):
U'(k) - U(k) = 2k
k is discrete here, but we can let it be continuous, and if we do, then U' is the derivative of U.
At that point the solution is obvious: if you've ever taken derivatives, then you know that if the difference of a function and its derivative is exponential, then the function itself has to be exponential (since only in that case will the derivative be some multiple of itself).
At that point you know U(k) is exponential, so you can just plug in an exponential for the unknown coefficients in the exponential, and plug it back in to solve for T.
This is a question from Introduction to Algorithms By Cormen. But this isn't a homework problem instead self-study.
There is an array of length n. Consider a modification to merge sort in which n/k sublists each of length k are sorted using insertion sort and then merged using merging mechanism, where k is a value to be determined.
The relationship between n and k isn't known. The length of array is n. k sublists of n/k means n * (n/k) equals n elements of the array. Hence k is simply a limit at which the splitting of array for use with merge-sort is stopped and instead insertion-sort is used because of its smaller constant factor.
I was able to do the mathematical proof that the modified algorithm works in Θ(n*k + n*lg(n/k)) worst-case time. Now the book went on to say to
find the largest value of k as a function of n for which this modified algorithm has the same running time as standard merge sort, in terms of Θ notation. How should we choose k in practice?
Now this got me thinking for a lot of time but I couldn't come up with anything. I tried to solve
n*k + n*lg(n/k) = n*lg(n) for a relationship. I thought that finding an equality for the 2 running times would give me the limit and greater can be checked using simple hit-and-trial.
I solved it like this
n k + n lg(n/k) = n lg(n)
k + lg(n/k) = lg(n)
lg(2^k) + lg(n/k) = lg(n)
(2^k * n)/k = n
2^k = k
But it gave me 2 ^ k = k which doesn't show any relationship. What is the relationship? I think I might have taken the wrong equation for finding the relationship.
I can implement the algorithm and I suppose adding an if (length_Array < k) statement in the merge_sort function here(Github link of merge sort implementation) for calling insertion sort would be good enough. But how do I choose k in real life?
Well, this is a mathematical minimization problem, and to solve it, we need some basic calculus.
We need to find the value of k for which d[n*k + n*lg(n/k)] / dk == 0.
We should also check for the edge cases, which are k == n, and k == 1.
The candidate for the value of k that will give the minimal result for n*k + n*lg(n/k) is the minimum in the required range, and is thus the optimal value of k.
Attachment, solving the derivitives equation:
d[n*k + n*lg(n/k)] / dk = d[n*k + nlg(n) - nlg(k)] / dk
= n + 0 - n*1/k = n - n/k
=>
n - n/k = 0 => n = n/k => 1/k = 1 => k = 1
Now, we have the candidates: k=n, k=1. For k=n we get O(n^2), thus we conclude optimal k is k == 1.
Note that we found the derivitives on the function from the big Theta, and not on the exact complexity function that uses the needed constants.
Doing this on the exact complexity function, with all the constants might yield a bit different end result - but the way to solve it is pretty much the same, only take derivitives from a different function.
maybe k should be lg(n)
theta(nk + nlog(n/k)) have two terms, we have the assumption that k>=1, so the second term is less than nlog(n).
only when k=lg(n), the whole result is theta(nlog(n))