I understand that simple statements like:
int x = 5; // is 1 or O(1)
And a while loop such as:
while(i<); // is n+1 or O(n)
And same with a for a single for loop (depending).
With nested while or for loop such as:
for(int i = 0; i<n; i++){ // this is n + 1
for(int j = 0; j<n; j++){ // this is (n+1)*n, total = O(n^2)
}
Also anytime we have a doubling effect it's log_2(n), tripling effect log_3(n) and so on. And if the control varible is being halved or quarted that's also either log_2(n) or log_4(n).
But I am dealing with much more complicated examples. How would one figure these examples out. I have the answers I just don't know how to work them out on paper come an examination.
Example1:
for (i = 1; i < (n*n+3*n+17) / 4 ; i += 1)
System.out.println("Sunshine");
Example2:
for (i = 0; i < n; i++)
if ( i % 2 == 0) // very confused by what mod would do to runtime
for (j = 0; j < n; j++)
System.out.print("Bacon");
else
for (j = 0; j < n * n; j++)
System.out.println("Ocean");
Example3:
for (i = 1; i <= 10000 * n: i *= 2)
x += 1;
Thank you
Example 1 is bounded by the term (n*n+3*n+17) and therefore should be O(n^2). The reason this is O(n^2) is because the largest, and therefore dominant, term in the expression is n^2.
The second example is a bit more tricky. The outer loop in i will iterate n times, but what executes on the inside depends on whether that value of i be odd or even. When even, another loop over n happens, but when odd a loop in n^2 happens. The odd case will dominate the running time eventually, so example 2 should be O(n^3).
The third example iterates until hitting 10000*n, but does so by doubling the loop counter i at each step. This will have an O(lgn) performance, where lg means the log base 2. To see why, imagine we wanted to reach n=32, starting at i=1 and doubling each time. Well we would have 2, 4, 8, 16, and 32, i.e. 6 steps, which grows as lg(32).
Related
This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 8 years ago.
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i; j++)
sum++;
My guess is this is simply a O(N^2), but the j < i is giving me doubts.
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i*i; j++)
sum++;
Seems like an O(N^3)
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i*i; j++)
if( j % i == 0 )
for( k = 0; k < j; k++ )
sum++
O(N^5)?
So for each loop j has a different value. If it was j < n*n, it would've been more straight forward, but this one is a tricky one, so please help. Thanks.
In the first case sum++ executes 0 + 1 + ... + n-1 times. If you apply arithmetic progression formula, you'll get n (n-1) / 2, which is O(n^2).
In the second case we'll have 0 + 1 + 4 + 9 + ... + (n-1)^2, which is sum of squares of first n-1 numbers, and there's a formula for it: (n-1) n (2n-1)
The last one is interesting. You can see, actually, that the most nested for loop is called only when j is a multiplicand of i, so you can rewrite the program as follows:
int n = 500;
for(int i = 0; i < n; i++) {
for(int m = 0; m < i; m++) {
int j = m * i;
for( k = 0; k < j; k++)
sum++
}
}
It's easier to work with math notation:
The formula is derived from the code by analysis: we can see that sum++ gets called j times in the innermost loop, which is called i times, which is called n times. In the end, the problem boils down to a sum of cubes of first n numbers plus lower-order terms (which do not affect the asymptotics)
One final note: it looks obvious, but I'd like to show that in general sum of first N natural numbers in dth power is Ω(N^(d+1)) (see Wikipedia for Big-Omega notation), that is it grows no slower than that function. You can apply the same reasoning to prove that a stronger condition is satisfied, namely, it belongs to Θ(N^(d+1)), which combines both Ω and O.
You are right for all but the last one, which has a tighter bound of O(n^4): note that the last for loop is only executed if j is a multiple of i. There are x / i multiples of i lower than or equal to x, and i * i / i = i. So the last loop is only executed for i values out of the i * i.
Note that big-oh gives an upper bound, so i*i vs n*n makes little difference. Strictly speaking, saying they are all O(n^2015) is also correct (because that is a valid upper bound), but it's hardly helpful, so in practice a tight bound is usually used.
IVlad already gave the correct answer.
I think what confuses you is the "Big Oh" definition.
N^2 has O(N^2)
1/2N^2 has O(N^2)
1/2N^2 + c*N + b also has
O(N^2) - by given c and b are constants independent from N
Check Big Oh definition from here
I'm asked to give a big-O estimates for some pieces of code but I'm having a little bit of trouble
int sum = 0;
for (int i = 0; i < n; i = i + 2) {
for (int j = 0; j < 10; j + +)
sum = sum + i + j;
I'm thinking that the worst case would be O(n/2) because the outer for loop is from i to array length n. However, I'm not sure if the inner loop affects the Big O.
int sum = 0;
for (int i = n; i > n/2; i − −) {
for (int j = 0; j < n; j + +)
sum = sum + i + j;
For this one, I'm thinking it would be O(n^2/2) because the inner loop is from j to n and the outer loop is from n to n/2 which gives me n*(n/2)
int sum = 0;
for (int i = n; i > n − 2; i − −) {
for (int j = 0; j < n; j+ = 5)
sum = sum + i + j;
I'm pretty lost on this one. My guess is O(n^2-2/5)
Your running times for the first two examples are correct.
For the first example, the inner loop of course always executes 10 times. So we can say the total running time is O(10n/2).
For the last example, the outer loop only executes twice, and the inner loop n/5 times, so the total running time is O(2n/5).
Note that, because of the way big-O complexity is defined, constant factors and asymptotically smaller terms are negligible, so your complexities can / should be simplified to:
O(n)
O(n2)
O(n)
If you were to take into account constant factors (using something other than big-O notation of course - perhaps ~-notation), you may have to be explicit about what constitutes a unit of work - perhaps sum = sum + i + j constitutes 2 units of work instead of just 1, since there are 2 addition operations.
You're NOT running nested loops:
for (int i = 0; i < n; i = i + 2);
^----
That semicolon is TERMINATING the loop definition, so the i loop is just counting from 0 -> n, in steps of 2, without doing anything. The j loop is completely independent of the i loop - both are simply dependent on n for their execution time.
For the above algorithms worst case/best case are the same.
In case of Big O notation, lower order terms and coefficient of highest order term can be ignored as Big O is used for describing asymptotic upper bound.
int sum = 0;
for (int i = 0; i < n; i = i + 2) {
for (int j = 0; j < 10; j + +)
sum = sum + i + j;
Total number of outer loop iteration =n/2.for each iteration of outer loop, number of inner loop iterations=10.so total number of inner loop iterations=10*n/2=5n. so clearly it is O(n).
Now think about rest two programs and determine time complexities on your own.
I have to analyze the Big O complexity for the below code fragments:
a)
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i; j < n; j++)
sum++;
b)
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i + 1; j > i; j--)
// loop 3
for(int k = n; k > j; k--)
sum++;
I'm not sure how to do so any help provided will be greatly appreciated. Thanks.
To analize Big-Oh complexity you have to try to count how many basic operations are made by your code.
In your first loop:
for(int i = 0; i < n; i++)
for(int j = i; j < n; j++)
sum++;
How many times is sum++ called?
The first loop happens n times, and in each one of these, the second loop happens around n times.
This gives you around n * n operations, which is equivalent to a complexity of O(n^2).
I'll let you work out the second one.
The first is straight forward (using the tools of the 2nd code snap, which is a bit trickier) - I'll focus on the 2nd code snap.
Big O notation is giving asymptotic upper bound to the number of ops the algorithm do.
Let's assume each inner iteration do 1 op, and let's neglect the counters and overhead of looping.
Denote T(n) total number of ops done in the program.
It is clear that the program has NO MORE ops then:
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i+1; j > i; j--) //note a single op in here, see (1) for details
// loop 3
for(int k = n; k > 0; k--) //we change k > j to j > 0 - for details see (2)
sum++;
(1) Since j is initialized as i+1, and is decreased each iteration, after the first iteration of loop2, you will get j == i, and the condition will yield false - thus - a single iteration is done
(2) The original loop iterates NO MORE then n times (since j >= 0) - thus the "new program" is "not better" then the old one (in terms of upper bounds).
Complexity of the simplified program
The total complexity of the above program is O(n^2), since loop1 and loop3 repeat n times each, and loop2 repeats exactly once.
If we assume single command is done each inner loop - the total number of commands which are done is then n^2.
Conclusion:
Since the new program is doing n^2 "ops" (according to the assumptions) and the original is "not worse then the new" - it is doing T(n) <= n^2 steps.
From definition of big O notation (with c=1, and for every N) - you can conclude the program is O(n^2)
I seem to understand the basic concepts of easier loops like so...the first loop runs in O(n), as does the inner loop. Because they're both nested, you multiply to get a total running time of O(n^2).
sum = 0;
for ( i = 0; i < n; i++ )
for j = 0; j < n; j++ )
++sum;
Though when things start getting switched around, I get completely lost as to how to figure it out. Could someone explain to me how to figure out running time for both of the following? Also, any links to easy to understand references that could further help me improve is also appreciated. Thanks!
sum = 0;
for( i = 0; i < n; i += 2 )
for( j = 0; j < n; j++ )
++sum;
The only thing I can gather from this is that the inner loop runs in O(n). The i+=2 really throws me off in the outer loop.
sum = 0;
for( i = 1; i < n; i *= 2 )
for( j = 0; j < n; j++ )
++sum;
From my attempt...outer loop is O(log(n)), inner is O(n), so total is O(n log(n))?
A good way of thinking about Big-O performance is to pretend each element of the code is a mathematical function that takes in n items and returns the number of computations performed on those items.
For example, a single for loop like for ( i = 0; i < n; i++ ) would be equivalent to a function i(), where i(n) = n, indicating that one computation is performed for each input n.
If you have two nested loops, then the functional equivalent for
for ( i = 0; i < n; i++ )
for j = 0; j < n; j++ )
would look like these two functions:
i(n) = n * j(n)
j(n) = n
Working these two functions out produces an end result of n*n = n^2, since j(n) can be substituted for n.
What this means is that as long as you can solve for the Big-O of any single loop, you can then apply those solutions to a group of nested loops.
For example, let's look at your second problem:
for( i = 0; i < n; i += 2 )
for( j = 0; j < n; j++ )
i+=2 means that for an input set of n items (n0, n1, n2, n3, n4) you're only touching every other element of that set. Assuming you initialize so that i=0, that means you're only touching the set of (n0,n2,n4). This means you're halving the size of the data set that you're using for processing, and means the functional equivalents work out like:
i(n) = (n/2) * j(n)
j(n) = n
Solving these gets you (n/2) * n = (n^2)*(1/2). Since this is Big-O work, we remove the constants to produce a Big-O value of (n^2).
The two key points to remember here:
Big-O math starts with a set of n data elements. If you're trying to determine the Big-O of a for loop that iterates through that set of n elements, your first step is to look at how the incrementor changes the number of data elements that the for routine actually touches.
Big-O math is math. If you can solve for each for expression individually, you can use those solutions to build up into your final answer, just like you can solve for a set of equations with common definitions.
This would be part # 2 of my question about analysis of for loop running time
http://faculty.simpson.edu/lydia.sinapova/www/cmsc250/LN250_Weiss/L03-BigOhSolutions.htm#PR4 contains solutions, and I have question about two particular "for" loops
Could someone explain to me how to figure out running time for both of them. Thanks !
1.
sum = 0;
for( i = 0; i < n; i++)
for( j = 0; j < i*i; j++)
for( k = 0; k < j; k++)
sum++;
2.
sum = 0;
for( i = 0; i < n; i++)
for( j = 0; j < i*i; j++)
if (j % i ==0)
for( k = 0; k < j; k++)
sum++;
The first snippet is O(n^5).
Top Loop = 0 - O(n) = O(n) iterations
Middle Loop = 0 - O(n^2) = O(n^2) iterations
Inner Loop = 0 - O(n^2) = O(n^2) iterations
Total = O(n^5)
Here's the closed-form solution of the first snippet: (computed via Mathematica)
sum = -(1/10)*n + (1/4)*n^2 - (1/4)*n^4 + (1/10)*n^5
This is a 5th order polynomial, therefore it is: O(n^5)
The second snippet appears to be O(n^4).
Top Loop = 0 - O(n) = O(n) iterations
Middle Loop = 0 - O(n^2) = O(n^2) iterations
If statement enters: O(1 / n) times
Inner Loop = 0 - O(n^2) = O(n^2) iterations
Total = O(n^4)
Here's the closed-form solution of the second snippet: (computed via Mathematica)
sum = -(1/12)*n + (3/8)*n^2 - (5/12)*n^3 + (1/8)*n^4
This is a 4th order polynomial, therefore it is: O(n^4)
Further explanation of the effect of the if-statement:
The middle loop iterates from 0 to i*i. The if-statement checks if j is divisible by i. But that is only possible when j is a multiple of i.
How many times is j a multiple of i if 0 <= j < i*i? Exactly i times. Therefore only 1/i of the iterations of the middle loop will fall through to the inner-most loop.
The relationship of 'n' as well as the other variables in the second for loop statement ( ..., x<=n, ...) would really define how fast it would be. Try to visualize a for loop as a racem and the second statement says how many laps you would make. So for example, variable 'n' = 1000, then you would have to run the same lap for 1000 times, truly time wasting. Hope that got you a better view on things.