I'm trying to do time complexity analysis on the bottom up heap analysis and I'm stuck. I've done the mathematical evaluation that shows it is O(n) and i completely understand why. The part I'm stuck understanding is how in the "code" it achieves this. I know the outer for executes floor(n/2) times, and I believe the while executes log times, but I don't know how to get from floor(n/2)log to O(n).
Pseudo code: Time analysis:
for i = n/2-1; i <=0; i-- n/2+1
k=i n/2
while(2*k-1 <= n) n/2(????)+1 <-- this is where I'm stuck. Should run log n times?
j = k*2-1 ...
if(j<n && H[j] < H[j+1]) ...
j++ ...
if(H[k] < h[j]) ...
break ...
swap(H[k],H[j]) ...
k=j ...
So I can see that the while probably runs log n times, but I can't see how to get from there (n/2)log n to O(n). I'm only looking for worst case since I know best case is n/2 + 1 since it breaks when the subtree is a heap. Any help or direction to reading material is welcome.
The best advice I have to offer about working out the big-O cost of different loops is this one:
"When in doubt, work inside out!"
In other words, rather than starting with the outermost loop and working inward, start with the innermost loop and work outward.
In this case, we have this code:
for i = n/2-1; i >= 0; i--
k=i
while (2*k-1 <= n)
j = k*2-1
if(j<n && H[j] < H[j+1])
j++
if(H[k] < h[j])
break
swap(H[k],H[j])
k=j
Since we're working inside out, let's focus first on this loop:
Let's start by analyzing the innermost loop:
while (2*k-1 <= n)
j = k*2-1
if(j<n && H[j] < H[j+1])
j++
if(H[k] < h[j])
break
swap(H[k],H[j])
k=j
I'm going to assume this is a worst-case analysis and that we never trigger the inner break statement. In that case, this means that the loop progresses by having k move to either 2k - 1 or 2k after each step of the loop. This means that k is roughly doubling with each iteration of the loop. The loop ends when k exceeds n, so the number of iterations of the loop is equal to the number of times we have to double k before k exceeds n. That works out to O(log(n / k)) total loop iterations. Note that this isn't a constant; as k gets smaller, we end up doing more and more work per iteration.
We can replace the inner loop with the simpler "do O(log(n / k)) work" to get this:
for i = n/2-1; i >= 0; i--
k=i
do O(log (n / k)) work;
And, since k = i, we can rewrite this as
for i = n/2-1; i >= 0; i--
do O(log (n / i)) work;
Now, how much total work is being done here? Adding up the work done per iteration across all iterations, we get that the work done is
log (n / (n/2)) + log (n / (n/2 - 1)) + log (n / (n/2 - 2)) + ... + log(n / 2) + log(n / 1).
Now, "all" we have to do is simplify this sum. :-)
Using properties of logarithms, we can rewrite this as
(log n - log (n/2)) + (log n - log(n/2 - 1)) + (log n - log(n/2 - 2)) + ... + (log n - log 1)
= (log n + log n + ... + log n) - (log(n/2) + (log(n/2 - 1) + ... + log 1)
= (n/2)(log n) - log((n/2)(n/2 - 1)(n/2 - 2) ... 1)
= (n/2)(log n) - log((n/2)!)
Now, we can use Stirling's approximation to rewrite
log((n/2)!) = (n/2)log(n/2) - n log e + O(log n)
And, therefore, to get this:
(n/2)(log n) - log((n/2)!)
= (n/2)(log n) - (n/2)log(n/2) + n log e - O(log n)
= (n/2)(log (2n / 2)) - (n/2) log (n/2) + O(n)
= (n/2)(log 2 + log(n/2)) - (n/2) log (n/2) + O(n)
= (n/2)(1 + log(n/2)) - (n/2) log (n/2) + O(n)
= n/2 + O(n)
= O(n).
So this whole sum works out to O(n).
As you can see, this is a decidedly nontrivial big-O to calculate! Indeed, it's a lot trickier than just counting up the work done per iteration and multiplying by the number of iterations, because the way in which the work per iteration changes across iterations makes that a lot harder to do. Rather, instead we have to do a more nuanced analysis of how much work is done by each loop, then convert things into a summation and pull out some nontrivial (though not completely unexpected) tricks (Stirling's approximation and properties of logarithms) to get everything to work out as expected.
I would categorize this particular set of loops as a fairly tricky one to work through and not particularly representative of what you'd "normally" see when doing a loop analysis. But hopefully the techniques here give you a sense of how to work through trickier loop analyses and a glimpse of some of the beautiful math that goes into them.
Hope this helps!
Related
Can someone help me with calculating the time complexity of the inner loop? As far as I understand, the outer one will be O(n). But I have no idea how to calculate what happens inside the second one.
for (int i = 2; i < n; i++) {
for (int j = 2; i * j < n; j++) {
}
For every iteration of "outer loop", inner loop runs n/i times
So, total complexity of this will be given by:
n/2 + n/3 + n/4 + ...
= n * (1/2 + 1/3 + 1/4 ...)
For the right term above, upper bound is ln(n)
Hence, complexity of this code is O(n log n).
The inner loop runs from 2 up to but not including n/i times. You can express it as n/i - 2.
If we run the inner loop n - 2 times (since that's the number of times the outer loop runs), we get the following summation:
(n/2 - 2) + (n/3 - 2) + ... + (3 - 2)
I have a hunch but can't remember 100% that this series sums up to log_e(n) * n or similar. So in terms of time complexity, this becomes O(log n * n).
The loop exits as soon as i * j ≥ n, i.e. when j = ceiling(n / i) ~ n / i. As it starts from j=2, the number of iterations is ceiling(n / i) - 1.
I was asked if what time complexity if this:
What is the time complexity (with respect of n) of this algorithm:
k=0
for(i = n / 2 ; i < n ; i++ ) {
for( j=0 ; j < i ; j++)
k = k + n / 2
}
choices was : a. O(n) b. O(n/2) c. O(n log(n) and d. O(n^2)
can have a multiple answers.
i know the algorithm above is d. O(n^2) but i came with with a. O(n) since it is looking for complexity of n only?.
if you are to have this question. how would you answer it.?? im so curious about the answer.
The answer is O(n²).
This is easy to understand. I will try to make you understand it.
See, the outer for loop block is executed n - n/2 = n/2 times.
Of course it depends whether the number n is even or odd. If it's even then the outer loop is executed n/2 times. If it's odd then it's executed for (n-1)/2 times.
But for time complexity, we don't consider this. We just assume that the outer for loop is executed n/2 times where i starts from n/2 and ends at n - 1 (because the terminating condition is i < n and not i <= n).
For each iteration of the outer loop, the inner loop executes i times.
For example, for every iteration, inner loop starts with j = 0 to j = i - 1. This means that it executes i times (not i - 1 times because j starts from 0 and not from 1).
Therefore, for 1st iteration the inner loop is executed i = n / 2 times. i = n / 2 + 1 for 2nd iteration and so on upto i = n - 1 times.
Now, the total no. of times the inner loop executes is n/2 + (n/2 + 1) + (n/2 + 2) + ... + (n - 2) + (n - 1). It's simple math that this sums up to (3n² - n)/2 times.
So, the time complexity becomes O((3n² - n)/2).
But we ignore the n term because n² > n and the constant terms because for every n they will remain the same.
Therefore, the final time complexity is O(n²).
Hope this helps you understand.
I have the following code. What time complexity does it have?
I have tried to write a recurrence relation for it but I can't understand when will the algorithm add 1 to n or divide n by 4.
void T(int n) {
for (i = 0; i < n; i++);
if (n == 1 || n == 0)
return;
else if (n%2 == 1)
T(n + 1);
else if (n%2 == 0)
T(n / 4);
}
You can view it like this: you always divide by four only if you have odd you add 1 to n before division. So, you should count how many times 1 was added. If there no increments then you have log4n recursive calls. Let's assume that you always have to add 1 before division. Then can rewrite it like this:
void T(int n) {
for (i = 0; i < n; i++);
if (n == 1 || n == 0)
return;
else if (n%2 == 0)
T(n / 4 + 1);
}
But n/4 + 1 < n/2, and in case of recursive call T(n/2), running time is O(log(n,4)), but base of logarithm doesn't impact running time in big-O notation because it's just like constant factor. So running time is O(log(n)).
EDIT:
As ALB pointed in a comment, there is cycle of length n. So, with accordance with master theorem running time is Theta(n). You can see it in another way as sum of n * (1 + 1/2 + 1/4 + 1/8 + ...) = 2 * n.
Interesting question. Be aware that even though your for loop is doing nothing, since it is not an optimized solution (see Dukeling's comment), it will be considered in your time complexity as if computing time was taken to iterate through it.
First part
The first section is definitely O(n).
Second part
Let's suppose for the sake of simplicity that half the time n will be odd and the other half time it will be even. Hence, the recursion is looping (n+1) half the time and (n/4) the other half.
Conclusion
For each time T(n) is called, the recursion will implicitly loop n times. Hence, The first half of the time, we will have a complexity of n * (n+1) = n^2 + n. The other half of the time, we will deal with a n * (n/4) = (1/4)n^2.
For Big O notation, we care more about the upper bound than its precise behavior. Hence, your algorithm would be bound by O(n^2).
I have the following code and have to determine the big theta complexity
for i =1 to n do
for j = 1 to n do
k=j
while k<= n do
k = k*3
end while
end for
end for
It's easy to see that the first two for-loops run n times each, but the while loop is throwing my off. The first time it runs log3(n) times, but after that i can't really tell.
Anyone who can help?
Let T be the run time. It is clear T is Ω(n2). We can use Stirling's Approximation to expand ln n! to get
T = ∑i ∑j ⌈lg3(n/j)⌉ = n * O(∑j ln n - ln j + 1) = n * O(n ln n - ln n! + n) = n * O(n ln n - (n ln n - n + O(ln n)) + n) = O(n2)
Thus T = Θ(n2)
Solution without using heavy-weight math:
Turn the problem on its head: instead of thinking about the first time the inner loop runs, think about the last time: it runs only once. In fact, the innermost loop runs only once for most values of j.
It runs once when j > n/3, that is, for 2n/3 values of j
It runs twice when n/9 < j <= n/3, that is, for 2n/9 values of j
It runs 3 times when n/27 < j <= n/9, that is, for 2n/27 values of j
It runs 4 times when n/81 < j <= n/27, that is, for 2n/81 values of j
...
The total number of times the innermost loop runs is going to be
1 * 2n/3 + 2 * 2n/9 + 3 * 2n/27 + 4 * 2n/81 + ...
= 2n(1/3 + 2/9 + 3/27 + ... )
< 2n Sum[k/3^k, for k=1 to infinity]
It's easy to see that the series Sum[k/3^k] converges (ratio test). Therefore the j-loop runs in O(n) time, and the entire thing in O(n²) time.
for(i=0;i< m; i++)
{
for(j=i+1; j < m; j++)
{
for(k=0; k < n;k++)
{
for(l=0;l< n;l++)
{if(condition) do something}
}
}
}
In details:
The two first loops will result in (m-1) + (m-2) + (m-3) + ... + 1 repetitions, which is equal to m*(m-1)/2. As for the second two loops, they basically run from 0 to n-1 so they need n^2 iterations.
As you have no clue whether the condition will be fulfilled or not, then you take the worst case, which is it being always fulfilled.
Then the number of iterations is:
m*(m+1)/2*n^2*NumberOfIterations(Something)
In O notation, the constants and lower degrees are not necessary, so the complexity is:
O(m^2*n^2)*O(Something)
for(i=0;i< m; i++)
{
for(j=i+1; j < m; j++)
{
The inner loop will run ((m-1) + (m-2) + ... 1) times
= 1 + 2 + 3 + ...m-1
= m * (m - 1) / 2
for(k=0; k < n;k++)
{
for(l=0;l< n;l++)
{
In this case, the inner loop clearly runs n * n times
So clearly, the number of iterations is exactly
(m * (m - 1) / 2) * (n * n)
= O(m^2 * n^2)
Obviously, this assumes that
if(condition) do something
runs in constant time.
Looks like O(m^2 n^2) to me, assuming the "something" is constant-time.
Although the j loop starts from a different point with each step, the combined effect of the i and j loops is still an m^2 factor.
Evaluating the unstated condition itself would normally be (at least) a constant time operation, so certainly the loop cannot be faster than O(m^2 n^2) - unless, of course, the "something" includes a break, goto, exception throw or whatever that exits out of one or more of the loops early.
All bets are off if, for any reason, either n or m isn't constant throughout.
I assume the time complexity of "do something" is O(S).
Let's start with the most inner loop: It's time complexity is O(n*S) because it does something n times. The loop which wraps the most inner loop has a time complexity of O(n)O(nS)=O(n^2*S) because it does the inner loop n times.
The loop whcih wraps the second most inner loop has a time complexity of O(m-i)*O(n^2*S) because it does an O(n^2*S) operation m-i times.
Now for the harder part: for each i in the range 0...m-1 we do an (m-i)*O(n^2*S) operation. How long does it take? (1 + 2 + 3 + ... + m)*O(n^2*S).
But (1 + 2 + ... + m) is the sum of an arithmetic sequence. Therefore the sum equals to m*(m-1)/2=O(m^2).
Conclusion: We do an O(n^2*S) operation about m^2 times. The time complexity of the whole thing is O(m^2*n^2*S)
O(m^2*n^2*(compexity of something)). If condition and something are executed in constant time then O(m^2*n^2).
O(m²*n²) *complexity of "something"