I have to analyze the Big O complexity for the below code fragments:
a)
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i; j < n; j++)
sum++;
b)
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i + 1; j > i; j--)
// loop 3
for(int k = n; k > j; k--)
sum++;
I'm not sure how to do so any help provided will be greatly appreciated. Thanks.
To analize Big-Oh complexity you have to try to count how many basic operations are made by your code.
In your first loop:
for(int i = 0; i < n; i++)
for(int j = i; j < n; j++)
sum++;
How many times is sum++ called?
The first loop happens n times, and in each one of these, the second loop happens around n times.
This gives you around n * n operations, which is equivalent to a complexity of O(n^2).
I'll let you work out the second one.
The first is straight forward (using the tools of the 2nd code snap, which is a bit trickier) - I'll focus on the 2nd code snap.
Big O notation is giving asymptotic upper bound to the number of ops the algorithm do.
Let's assume each inner iteration do 1 op, and let's neglect the counters and overhead of looping.
Denote T(n) total number of ops done in the program.
It is clear that the program has NO MORE ops then:
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i+1; j > i; j--) //note a single op in here, see (1) for details
// loop 3
for(int k = n; k > 0; k--) //we change k > j to j > 0 - for details see (2)
sum++;
(1) Since j is initialized as i+1, and is decreased each iteration, after the first iteration of loop2, you will get j == i, and the condition will yield false - thus - a single iteration is done
(2) The original loop iterates NO MORE then n times (since j >= 0) - thus the "new program" is "not better" then the old one (in terms of upper bounds).
Complexity of the simplified program
The total complexity of the above program is O(n^2), since loop1 and loop3 repeat n times each, and loop2 repeats exactly once.
If we assume single command is done each inner loop - the total number of commands which are done is then n^2.
Conclusion:
Since the new program is doing n^2 "ops" (according to the assumptions) and the original is "not worse then the new" - it is doing T(n) <= n^2 steps.
From definition of big O notation (with c=1, and for every N) - you can conclude the program is O(n^2)
Related
I want to find the time complexity for this below code. Here's my understanding-
The outer for loop will loop 2n times and in the worst case when i==n, we will enter the if block where the nested for loops have complexity of O(n^2), counting the outer for loop, the time complexity for the code block will be O(n^3).
In best case when i!=n, else has complexity of O(n) and the outer for loop is O(n) which makes the complexity, in best case as O(n^2).
Am I correct or am I missing something here?
for (int i = 0; i < 2*n; i++)
{
if (i == n)
{
for (int j = 0; j < i; j++)
for (int k = 0; k < i; k++)
O(1)
}
else
{
for (int j = 0; j < i; j++)
O(1)
}
}
No.
The question "what is T(n)?".
What you are saying is "if i=n, then O(n^3), else O(n^2)".
But there is no i in the question, only n.
Think of a similar question:
"During a week, Pete works 10 hours on Wednesday, and 1 hour on every other day, what is the total time Pete works in a week?".
You don't really answer "if the week is Wednesday, then X, otherwise Y".
Your answer has to include the work time on Wednesday and on every other day as well.
Back in your original question, Wednesday is the case when i=n, and all other days are the case when i!=n.
We have to sum them all up to find the answer.
This is a question of how many times O(1) is executed per loop. The time complexity is a function of n, not i. That is, "How many times is O(1) executed at n?"
There is one run of a O(n^2) loop when i == n.
There are (2n - 2) instances of the O(n) loop in all other cases.
Therefore, the time complexity is O((2n - 2) * n + 1 * n^2) = O(3n^2 - 2*n) = O(n^2).
I've written a C program to spit out the first few values of n^2, the actual value, and n^3 to illustrate the difference:
#include <stdio.h>
int count(int n){
int ctr = 0;
for (int i = 0; i < 2*n; i++){
if (i == n)
for (int j = 0; j < i; j++)
for (int k = 0; k < i; k++)
ctr++;
else
for (int j = 0; j < i; j++)
ctr++;
}
return ctr;
}
int main(){
for (int i = 1; i <= 20; i++){
printf(
"%d\t%d\t%d\t%d\n",
i*i, count(i), 3*i*i - 2*i, i*i*i
);
}
}
Try it online!
(You can paste it into Excel to plot the values.)
The First loop is repeated 2*n times:
for (int i = 0; i < 2*n; i++)
{
// some code
}
This part Just occur once, when i == n and time complexity is : O(n^2):
if (i == n)
{
for (int j = 0; j < i; j++)
for (int k = 0; k < i; k++)
O(1)
}
And this part is depends on i.
else
{
for (int j = 0; j < i; j++)
O(1)
}
Consider i when:
i = 0 the loop is repeated 0 times
i = 1 the loop is repeated 1 times
i = 2 the loop is repeated 2 times
.
.
i = n the loop is repeated n times. (n here is 2*n)
So the loop repeated (n*(n+1)) / 2 times But when i == n else part is not working so (n*(n+1)) / 2 - n and time complexity is O(n^2).
Now we sum all of these parts: O(n^2) (first part) + O(n^2) (second part) because the first part occurs once so it's not O(n^3). Time complaxity is: O(n^2).
Based on #Gassa answer lets sum up all:
O(n^3) + O((2n)^2) = O(n^3) + O(4n^2) = O(n^3) + 4*O(n^2) = O(n^3)
Big O notation allows us throw out 4*O(n^2) because O(n^3) "eats" it
I have some trouble finding the time complexity of the code below. I figured that the if statement will run for approximately n times; however, I could not manage to describe it mathematically. Thanks in advance.
int sum = 0;
for (int i = 1; i < n; i++) {
for (int j = 1 ; j < i*i; j++) {
if (j % i == 0) {
for (int k = 0; k < j; k++) {
sum++;
}
}
}
}
Outer loop
Well, it's clear that it's O(n) because i is bounded by n.
Inner loops
If we take a look at the second loop alone, then it looks as follows:
...
for (int j = 1 ; j < i*i; j++){
...
j is bounded by i*i or simply n^2.
However, the innermost loop won't be executed for every j, but only for js that are divisible by i because that's what the constraint j % i == 0 means. Since j ~ i*i, there will be only i cases, when the innermost loop is executed. So, the number of iterations in the inner loops is bounded by i^3 or simply n^3.
Result
Hence, the overall time complexity is O(n4).
This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 8 years ago.
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i; j++)
sum++;
My guess is this is simply a O(N^2), but the j < i is giving me doubts.
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i*i; j++)
sum++;
Seems like an O(N^3)
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i*i; j++)
if( j % i == 0 )
for( k = 0; k < j; k++ )
sum++
O(N^5)?
So for each loop j has a different value. If it was j < n*n, it would've been more straight forward, but this one is a tricky one, so please help. Thanks.
In the first case sum++ executes 0 + 1 + ... + n-1 times. If you apply arithmetic progression formula, you'll get n (n-1) / 2, which is O(n^2).
In the second case we'll have 0 + 1 + 4 + 9 + ... + (n-1)^2, which is sum of squares of first n-1 numbers, and there's a formula for it: (n-1) n (2n-1)
The last one is interesting. You can see, actually, that the most nested for loop is called only when j is a multiplicand of i, so you can rewrite the program as follows:
int n = 500;
for(int i = 0; i < n; i++) {
for(int m = 0; m < i; m++) {
int j = m * i;
for( k = 0; k < j; k++)
sum++
}
}
It's easier to work with math notation:
The formula is derived from the code by analysis: we can see that sum++ gets called j times in the innermost loop, which is called i times, which is called n times. In the end, the problem boils down to a sum of cubes of first n numbers plus lower-order terms (which do not affect the asymptotics)
One final note: it looks obvious, but I'd like to show that in general sum of first N natural numbers in dth power is Ω(N^(d+1)) (see Wikipedia for Big-Omega notation), that is it grows no slower than that function. You can apply the same reasoning to prove that a stronger condition is satisfied, namely, it belongs to Θ(N^(d+1)), which combines both Ω and O.
You are right for all but the last one, which has a tighter bound of O(n^4): note that the last for loop is only executed if j is a multiple of i. There are x / i multiples of i lower than or equal to x, and i * i / i = i. So the last loop is only executed for i values out of the i * i.
Note that big-oh gives an upper bound, so i*i vs n*n makes little difference. Strictly speaking, saying they are all O(n^2015) is also correct (because that is a valid upper bound), but it's hardly helpful, so in practice a tight bound is usually used.
IVlad already gave the correct answer.
I think what confuses you is the "Big Oh" definition.
N^2 has O(N^2)
1/2N^2 has O(N^2)
1/2N^2 + c*N + b also has
O(N^2) - by given c and b are constants independent from N
Check Big Oh definition from here
I recently started playing with algorithms from this princeton course and I observed the following pattern
O(N)
double max = a[0];
for (int i = 1; i < N; i++)
if (a[i] > max) max = a[i];
O(N^2)
for (int i = 0; i < N; i++)
for (int j = i+1; j < N; j++)
if (a[i] + a[j] == 0)
cnt++;
O(N^3)
for (int i = 0; i < N; i++)
for (int j = i+1; j < N; j++)
for (int k = j+1; k < N; k++)
if (a[i] + a[j] + a[k] == 0)
cnt++;
The common pattern here is that as the nesting in the loop grows the exponent also increases.
Is it safe to assume that if I have 20-for loops my complexity would be 0(N^20)?
PS: Note that 20 is just a random number I picked, and yes if you nest 20 for loops in your code there is clearly something wrong with you.
It depends on what the loops do. For example, if I change the end of the 2nd loop to just do 3 iterations like this:
for (int i = 0; i < N; i++)
for (int j = i; j < i+3; j++)
if (a[i] + a[j] == 0)
cnt++;
we get back to O(N)
The key is whether the number of iterations in the loop is related to N and increases linearly as N does.
Here is another example where the 2nd loop goes to N ^ 2:
for (int i = 0; i < N; i++)
for (int j = i; j < N*N; j++)
if (a[i] + a[j] == 0)
cnt++;
This would be o(N^3)
Yes, if the length of the loop is proportional to N and the loops are nested within each other like you described.
In your specific pattern, yes. But it is not safe to assume that in general. You need to check whether the number of iterations in each loop is O(n) regardless of the state of all the enclosing loops. Only after you have verified that this is the case can you conclude that the complexity is O(nloop-nesting-level).
Yes. Even though you decrease the interval of iteration, Big-o notation works with N increasing towards infinity and as all your loops' lengths grow proportional to N, it is true that such an algorithm would have time complexity O(N^20)
I strongly recommend that you understand why a doubly nested loop with each loop running from 0 to N is O(N^2).Use summations to evaluate the number of steps involved in the for loops, and then dropping constants and lower order terms, you will get the Big-Oh of that algorithm.
This would be part # 2 of my question about analysis of for loop running time
http://faculty.simpson.edu/lydia.sinapova/www/cmsc250/LN250_Weiss/L03-BigOhSolutions.htm#PR4 contains solutions, and I have question about two particular "for" loops
Could someone explain to me how to figure out running time for both of them. Thanks !
1.
sum = 0;
for( i = 0; i < n; i++)
for( j = 0; j < i*i; j++)
for( k = 0; k < j; k++)
sum++;
2.
sum = 0;
for( i = 0; i < n; i++)
for( j = 0; j < i*i; j++)
if (j % i ==0)
for( k = 0; k < j; k++)
sum++;
The first snippet is O(n^5).
Top Loop = 0 - O(n) = O(n) iterations
Middle Loop = 0 - O(n^2) = O(n^2) iterations
Inner Loop = 0 - O(n^2) = O(n^2) iterations
Total = O(n^5)
Here's the closed-form solution of the first snippet: (computed via Mathematica)
sum = -(1/10)*n + (1/4)*n^2 - (1/4)*n^4 + (1/10)*n^5
This is a 5th order polynomial, therefore it is: O(n^5)
The second snippet appears to be O(n^4).
Top Loop = 0 - O(n) = O(n) iterations
Middle Loop = 0 - O(n^2) = O(n^2) iterations
If statement enters: O(1 / n) times
Inner Loop = 0 - O(n^2) = O(n^2) iterations
Total = O(n^4)
Here's the closed-form solution of the second snippet: (computed via Mathematica)
sum = -(1/12)*n + (3/8)*n^2 - (5/12)*n^3 + (1/8)*n^4
This is a 4th order polynomial, therefore it is: O(n^4)
Further explanation of the effect of the if-statement:
The middle loop iterates from 0 to i*i. The if-statement checks if j is divisible by i. But that is only possible when j is a multiple of i.
How many times is j a multiple of i if 0 <= j < i*i? Exactly i times. Therefore only 1/i of the iterations of the middle loop will fall through to the inner-most loop.
The relationship of 'n' as well as the other variables in the second for loop statement ( ..., x<=n, ...) would really define how fast it would be. Try to visualize a for loop as a racem and the second statement says how many laps you would make. So for example, variable 'n' = 1000, then you would have to run the same lap for 1000 times, truly time wasting. Hope that got you a better view on things.