What would be the order of growth of the code below. My guess was, each loop's growth is linear but the if statement is confusing me. How do I include that with the whole thing. I would very much appreciate an explanatory answer so I can understand the process involved.
int count = 0;
for (int i = 0; i < N; i++)
for (int j = i+1; j < N; j++)
for (int k = j+1; k < N; k++)
if(a[i] + a[j] + a[k] == 0)
count++;
There are two things that can be confusing when trying to determine the code's complexity.
The fact that not all loops start from 0. The second loop starts from i + 1 and the third from j + 1. Does this affect the complexity? It does not. Let's consider only the first two loops. For i = 0, the second runs N - 1 times, for i = 1 it runs N - 2 times, ..., for i = N - 1 it runs 0 times. Add all these up:
0 + 1 + ... + N - 1 = N(N - 1) / 2 = O(N^2).
So not starting from 0 does not affect the complexity (remember that big-oh ignores lower-order terms and constants). Therefore, even under this setting, the entire thing is O(N^3).
The if statement. The if statement is clearly irrelevant here, because it's only part of the last loop and contains no break statement or other code that would affect the loops. It only affects the incrementation of a count, not the execution of any of the loops, so we can safely ignore it. Even if the count isn't incremented (an O(1) operation), the if condition is checked (also an O(1) operation), so the same rough number of operations is performed with and without the if.
Therefore, even with the if statement, the algorithm is still O(N^3).
Order of growth of the code would be O(N^3).
In general k nested loops of length N contribute growth of O(N^k).
Here are two was to find that the time complexity is Theta(N^3) without much calculation.
First, you select i<j<k from the range 0 through N-1. The number of ways to choose 3 objects out of N is the binomial coefficient N choose 3 = N*(N-1)*(N-2)/(3*2*1) ~ (N^3)/6 = O(N^3), and more precisely Theta(N^3).
Second, an upper bound is that you choose i, j, and k from N possibilities, so there are at most N*N*N = N^3 choices. This is O(N^3). You can also find a lower bound of the same type since you can choose i from 0 through N/3-1, j from N/3 through 2N/3-1, and k from 2N/3 through N-1. This gives you at least floor(N/3)^3 choices, which is about N^3/27. Since you have an upper bound and lower bound of the same form, the time complexity is Theta(N^3).
Related
What is the big-O for the following code :
y=1;
x=3;
for(int i =1 ; i < =n ; i*=2)
for(int j =1; j<= i * i; j++)
if (i % j == 0)
for(int k = 1; k<=j; k++)
y=y*x;
My Thoughts :
Looking at another similar questions I think the inner most loop is O(n) and the first loop is O(log (n))..as for the middle its O(n^2)
so the overall result would be O(log(n)*n^3)
Is my answer and way of thinking right ? I'm new to this so i hope i can get some help explaning how this loops work.
the most inner loop will run j time if i % j == 0. As the middle loop will run i^2 times, only when j < i it will be possible to satisfy the specified condition. Hence, among i^2 iteration of the middle loop, at least i^2 - i times, the condition will not be satisfied.
Suppose we denote the number of divisors of i with tau(i), among j < i only tau(i) times the condition will satisfy that means the total complexity of the most inner loop is equal to the sum of divisions of i which is at most 77/16 i (see this post for the proof).
Hence, the total complexity of the middle loop with the inner loop is at most (i^2 - i) + (i - tau(i)) + 77/16 i = i^2 + 77/16 i - tau(i).
We also know that the tau(i) is in O(i^(1/loglog(i))) (see the proof here). Now, to find the complexity of the whole loop, we need to sum the last expression for i = 1, 2, 4, ..., n. As we desire to find the asymptotic complexity and we have a sum here, we can ignore the lower powers of i. Therefore, the time complexity of the whole loop is 1 + 2^2 + (2^2)^2 + ... + (2^2)^log(n) = ((2^2)^(log(n)+1)-1)/(2^2-1) = Theta(n^2) (a geometric sum with factor of 2^2 and log(n) items).
In sum, the higher time complexity analysis for the specified code is Theta(n^2) which is also in O(n^2) as well.
for(i=0; i<n; i++) // time complexity n+1
{
k=1; // time complexity n
while(k<=n) // time complexity n*(n+1)
{
for(j=0; j<k; j++) // time complexity ??
printf("the sum of %d and %d is: %d\n",j,k,j+k); time complexity ??
k++;
}
What is the time complexity of the above code? I stuck in the second (for) and i don't know how to find the time complexity because j is less than k and not less than n.
I always having problems related to time complexity, do you guys got some good article on it?
especially about the step count and loops.
From the question :
because j is less than k and not less than n.
This is just plain wrong, and I guess that's the assumption that got you stuck. We know what values k can take. In your code, it ranges from 1 to n (included). Thus, if j is less than k, it is also less than n.
From the comments :
i know the the only input is n but in the second for depends on k an not in n .
If a variable depends on anything, it's on the input. j depends on k that itself depends on n, which means j depends on n.
However, this is not enough to deduce the complexity. In the end, what you need to know is how many times printf is called.
The outer for loop is executed n times no matter what. We can factor this out.
The number of executions of the inner for loop depends on k, which is modified within the while loop. We know k takes every value from 1 to n exactly once. That means the inner for loop will first be executed once, then twice, then three times and so on, up until n times.
Thus, discarding the outer for loop, printf is called 1+2+3+...+n times. That sum is very well known and easy to calculate : 1+2+3+...+n = n*(n+1)/2 = (n^2 + n)/2.
Finally, the total number of calls to printf is n * (n^2 + n)/2 = n^3/2 + n^2/2 = O(n^3). That's your time complexity.
A final note about this kind of codes. Once you see the same patterns a few times, you quickly start to recognize the kind of complexity involved. Then, when you see that kind of nested loops with dependent variables, you immediately know that the complexity for each loop is linear.
For instance, in the following, f is called n*(n+1)*(n+2)/6 = O(n^3) times.
for (i = 1; i <= n; ++i) {
for (j = 1; j <= i; ++j) {
for (k = 1; k <= j; ++k) {
f();
}
}
}
First, simplify the code to show the main loops. So, we have a structure of:
for(int i = 0; i < n; i++) {
for(int k = 1; k <= n; k++) {
for(int j = 0; j < k; j++) {
}
}
}
The outer-loops run n * n times but there's not much you can do with this information because the complexity of the inner-loop changes based on which iteration of the outer-loop you're on, so it's not as simple as calculating the number of times the outer loops run and multiplying by some other value.
Instead, I would find it easier to start with the inner-loop, and then add the outer-loops from the inner-most to outer-most.
The complexity of the inner-most loop is k.
With the middle loop, it's the sum of k (the complexity above) where k = 1 to n. So 1 + 2 + ... + n = (n^2 + n) / 2.
With the outer loop, it's done n times so another multiplication by n. So n * (n^2 + n) / 2.
After simplifying, we get a total of O(n^3)
The time complexity for the above code is : n x n x n = n^3 + 1+ 1 = n^3 + 2 for the 3 loops plus the two constants. Since n^3 carries the heaviest growing rate the constant values can be ignored, so the Time complexity would be n^3.
Note: Take each loop as (n) and to obtained the total time, multiple the (n) values in each loop.
Hope this will help !
I currently have the following pseudo code, and I am trying to figure out why the answer to the question is O(n).
sum = 0;
for (i = 0; i < n; i++) do
for (j = n/3;j < 2*n; j+= n/3) do
sum++;
I thought the answer would be O(n^2) since the first for loop would run 'n' times and the second for loop has += n/3, giving it another (n divided by something times), which would just simplify to O(n^2). Could somebody explain why it is O(n)?
This is because the second loop runs in constant amount of operations (does not depend on n). From n/3 to 2n with a step n/3 which is similar to from 1/3 to 2 with a step 1/3.
This will run 5-6 times for reasonable n (not 0) (the number is not important and depends on how do you calculate /)
The inner loop increments by a multiple of n, not by 1, so its runtime is bounded by a constant (6?). So the total number of steps is bounded by a constant multiple of n (namely 6n).
This question is based off of this resource http://algs4.cs.princeton.edu/14analysis.
Can someone break down why Exercise 6 letter b is linear? The outer loop seems to be increasing i by a factor of 2 each time, so I would assume it was logarithmic...
From the link:
int sum = 0;
for (int n = N; n > 0; n /= 2)
for (int i = 0; i < n; i++)
sum++;
This is a geometric series.
The inner loops runs i iterations per iteration of the outer loop, and the outer loop decreases by half each time.
So, summing it up gives you:
n + n/2 + n/4 + ... + 1
This is geometric series, with r=1/2 and a=n - that converges to a/(1-r)=n/(1/2)=2n, so:
T(n) <= 2n
And since 2n is in O(n) - the algorithm runs in linear time.
This is a perfect example to see that complexity is NOT achieved by multiplying the complexity of each nested loop (that would have got you O(nlogn)), but by actually analyzing how many iterations are needed.
Yes its simple
See the value of n is decreasing by half each time and I runs n times.
So for the first time i goes from 1 to n
next time 0 to n/2
and hence 0 to n/k on kth term.
Now total time inner loop would run = Log(n)
So its a GP the number of times i is running.
with terms
n,n/2,n/4,n/8....0
so we can find the sum of the GP
2^(long(n) +1)-1 / (2-1)
2^(long(n)+1) = n
hence n-1/(1) = >O(n)
for i = 0 to n do
for j = n to 0 do
for k = 1 to j-i do
print (k)
I'm wondering about the lower-bound runtime of the above code. In the notes I am reading it explains the lower bound runtime to be
with the explanation;
To find the lower bound on the running time, consider the values of i, such that 0 <= i <= n/4 and values of j, such that 3n/4 <= j <= n. Note that for each of the n^2/16 different combinations of i and j, the innermost loop executes at least n/2 times.
Can someone explain where these numbers came from? They seem to be arbitrary to me.
There are n iterations of the first loop and for each of them n iterations of the second loop. In total these are n^2 iterations of the second loop.
Now if you only consider the lower quarter of possible values for i, then you have n^2/4 iterations of the inner loop left. If you also only consider the upper quarter of values for j then you have n^2/16 iterations of the inner loop left.
For each iteration of these constrained cases you have j-i >= 3n/4-n/4 = n/2 and therefore the most inner loop is iterated at least n/2 times for each of these n^2/16 iterations of the outer loops. Therefore the full number of iterations of the most inner loop is at least n^2/16*n/2.
Because we considered only specific iterations, the actual number of iterations is higher and this result is a lower bound. Therefore the algorithm is in Omega(n^3).
The values are insofar arbitrary that you could use many others. But these are some simple ones which make the argument j-i >= 3n/4-n/4 = n/2 possible. For example if you took only the lower half of the i iterations and the upper half of the j iterations, then you would have j-i >= n/2-n/2 = 0, leading to Omega(0), which is not interesting. If you took something like lower tenth and upper tenth it would still work, but the numbers wouldn't be as nice.
I can't really explain the ranges from your book, but if you attempt to proceed via the methodology below, I hope it would become more clear to find what is it that you are looking for.
The ideal way for the outer loop (index i) and the inner loop (index j) is as follows, since j - i >= 1 should be sustained, so that the innermost loop would execute (at least once in every case).
The performed decomposition was made because the range of j from i to 0 is ignored by the innermost loop.
for ( i = 0 ; i < n ; i ++ ) {
for ( j = n; j > i ; j -- ) {
for ( k = 1; k <= j - i ; k ++ ) {
printf(k);
}
}
}
This algorithm's order of growth complexity T(n) is:
Hence, your algorithm does iterate more than the algorithm above (j goes from n to 0).
Using Sigma Notation, you can do this:
Where c represent the execution cost of print(k), and c' is the execution cost of the occurring iterations that don't involve the innermost loop.