I have the following algorithm:
I analyzed this algoritm as follow:
Since the outer for loop goes from i to n it iterates at most n times,
and the loop on j iterates again from i to n which we can say at most n times,
if we do the same with the whole algorithm we have 4 nested for loop so the running time would be O(n^4).
But when I run this code for different input size I get the following result:
As you can see the result is much closer to n^3? can anyone explain why does this happen or what is wrong with my analysis that I get a loose bound?
Formally, you may proceed like the following, using Sigma Notation, to obtain the order of growth complexity of your algorithm:
Moreover, the equation obtained tells the exact number of iterations executed inside the innermost loop:
int sum = 0;
for( i=0 ; i<n ; i++ )
for( j=i ; j<n ; j++ )
for( k=0 ; k<j ; k++ )
for( h=0 ; h<i ; h++ )
sum ++;
printf("\nsum = %d", sum);
When T(10) = 1155, sum = 1155 also.
I'm sure there's a conceptual way to see why, but you can prove by induction the above has (n + 2) * (n + 1) * n * (n - 1) / 24 loops. Proof left to the reader.
In other words, it is indeed O(n^4).
Edit: You're count increases too frequently. Simply try this code to count number of loops:
for (int n = 0; n < 30; n++) {
int sum = 0;
for (int i = 0; i < n; i++) {
for (int j = i; j < n; j++) {
for(int k = 0; k < j; k++) {
for (int h = k; h < i; h++) {
sum++;
}
}
}
}
System.out.println(n + ": " + sum + " = " + (n + 2) * (n + 1) * n * (n - 1) / 24);
}
You are having a rather complex algorithm. The number of operations is clearly less than n^4, but it isn't at all obvious how much less than n^4, and whether it is O (n^3) or not.
Checking the values n = 1 to 9 and making a guess based on the results is rather pointless.
To get a slightly better idea, assume that the number of steps is either c * n^3 or d * n^4, and make a table of the values c and d for 1 <= n <= 1,000. That might give you a better idea. It's not a foolproof method; there are algorithms changing their behaviour dramatically much later than at n = 1,000.
Best method is of course a proof. Just remember that O (n^4) doesn't mean "approximately n^4 operations", it means "at most c * n^4 operations, for some c". Sometimes c is small.
Related
I have seen that in some cases the complexity of nested loops is O(n^2), but I was wondering in which cases we can have the following complexities of nested loops:
O(n)
O(log n) I have seen somewhere a case like this, but I do not recall the exact example.
I mean is there any kind of formulae or trick to calculate the complexity of nested loops? Sometimes when I apply summation formulas I do not get the right answer.
Some examples would be great, thanks.
Here is an example for you where the time complexity is O(n), but you have a double loop:
int cnt = 0;
for (int i = N; i > 0; i /= 2) {
for (int j = 0; j < i; j++) {
cnt += 1;
}
}
You can prove the complexity in the following way:
The first iteration, the j loop runs N times. The second iteration, the j loop runs N / 2 times. i-th iteration, the j loop runs N / 2^i times.
So in total: N * ( 1 + 1/2 + 1/4 + 1/8 + … ) < 2 * N = O(N)
It would be tempting to say that something like this runs in O(log(n)):
int cnt = 0;
for (int i = 1; i < N; i *= 2) {
for (int j = 1; j < i; j*= 2) {
cnt += 1;
}
}
But I believe that this runs in O(log^2(N)) which is polylogarithmic
I'm trying to study for an upcoming quiz about Big-O notation. I've got a few examples here but they're giving me trouble. They seem a little too advanced for a lot of the basic examples you find online to help. Here are the problems I'm stuck on.
1. `for (i = 1; i <= n/2; i = i * 2) {
sum = sum + product;
for (j= 1; j < i*i*i; j = j + 2) {
sum++;
product += sum;
}
}`
For this one, the i = i * 2 in the outer loop implies O(log(n)), and I don't think the i <= n/2 condition changes anything because of how we ignore constants. So the outer loop stays O(log(n)). The inner loops condition j < i*i*i confuses me because its in terms of 'i' and not 'n'. Would the Big-O of this inner loop then be O(i^3)? And thus the Big-O for the entire problem
be O( (i^3) * log(n) )?
2. `for (i = n; i >= 1; i = i /2) {
sum = sum + product
for (j = 1; j < i*i; j = j + 2) {
sum ++;
for (k = 1 ; k < i*i*j; k++)
product *= i * j;
}
}`
For this one, the outermost loop implies O(log(n)). The middle loop implies, again unsure, O(i^2)? And the innermost loop implies O(i^2*j)? I've never seen examples like this before so I'm almost guessing at this point. Would the Big-O notation for this problem be O(i^4 * n * j)?
3. `for (i = 1; i < n*n; i = i*2) {
for (j = 0; j < i*i; j++) {
sum ++;
for (k = i*j; k > 0; k = k - 2)
product *= i * j;
}
}`
The outermost loop for this one has an n^2 condition, but also a logarithmic increment, so I think that cancels out to be just regular O(n). The middle loop is O(i^2), and the innermost loop is I think just O(n) and trying to trick you. So for this problem the Big-O notation would be O(n^2 * i^2)?
4. `int i = 1, j = 2;
while (i <= n) {
sum += 1;
i = i * j;
j = j * 2;
}`
For this one I did a few iterations to better see what was happening:
i = 1, j = 2
i = 2, j = 4
i = 8, j = 8
i = 64, j = 16
i = 1024, j = 32
So clearly, 'i' grows very quickly, and thus the condition is met very quickly. However I'm not sure just what kind of Big-O notation this is.
Any pointers or hints you can give are greatly appreciated, thanks guys.
You can't add i or j to O-notation, it must be converted to n.
For the first one:
Let k be log 2 i.
Then inner loop is done 2^(k*3)/2=2^(3k-1) times for each iteration of outer loop.
k goes from 1 to log2n.
So total number of iterations is
sum of 2^(3k-1) for k from 1 to log 2 n which is 4/7(n^3-1) according to Wolfram Alpha, which is O(n^3).
For the last one, i=j1*j2*j3*...jk, and jm=2^m
i=2^1*2^2*...2^k=2^(1+2+...k)
So 1+2+3+...+k=log 2 n
(k+1)k/2 = log 2 n
Which is O(sqrt(log n))
BTW, log n^2 is not n.
This question is better to ask at computer science than here.
This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 8 years ago.
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i; j++)
sum++;
My guess is this is simply a O(N^2), but the j < i is giving me doubts.
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i*i; j++)
sum++;
Seems like an O(N^3)
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i*i; j++)
if( j % i == 0 )
for( k = 0; k < j; k++ )
sum++
O(N^5)?
So for each loop j has a different value. If it was j < n*n, it would've been more straight forward, but this one is a tricky one, so please help. Thanks.
In the first case sum++ executes 0 + 1 + ... + n-1 times. If you apply arithmetic progression formula, you'll get n (n-1) / 2, which is O(n^2).
In the second case we'll have 0 + 1 + 4 + 9 + ... + (n-1)^2, which is sum of squares of first n-1 numbers, and there's a formula for it: (n-1) n (2n-1)
The last one is interesting. You can see, actually, that the most nested for loop is called only when j is a multiplicand of i, so you can rewrite the program as follows:
int n = 500;
for(int i = 0; i < n; i++) {
for(int m = 0; m < i; m++) {
int j = m * i;
for( k = 0; k < j; k++)
sum++
}
}
It's easier to work with math notation:
The formula is derived from the code by analysis: we can see that sum++ gets called j times in the innermost loop, which is called i times, which is called n times. In the end, the problem boils down to a sum of cubes of first n numbers plus lower-order terms (which do not affect the asymptotics)
One final note: it looks obvious, but I'd like to show that in general sum of first N natural numbers in dth power is Ω(N^(d+1)) (see Wikipedia for Big-Omega notation), that is it grows no slower than that function. You can apply the same reasoning to prove that a stronger condition is satisfied, namely, it belongs to Θ(N^(d+1)), which combines both Ω and O.
You are right for all but the last one, which has a tighter bound of O(n^4): note that the last for loop is only executed if j is a multiple of i. There are x / i multiples of i lower than or equal to x, and i * i / i = i. So the last loop is only executed for i values out of the i * i.
Note that big-oh gives an upper bound, so i*i vs n*n makes little difference. Strictly speaking, saying they are all O(n^2015) is also correct (because that is a valid upper bound), but it's hardly helpful, so in practice a tight bound is usually used.
IVlad already gave the correct answer.
I think what confuses you is the "Big Oh" definition.
N^2 has O(N^2)
1/2N^2 has O(N^2)
1/2N^2 + c*N + b also has
O(N^2) - by given c and b are constants independent from N
Check Big Oh definition from here
for ( int i = n, i>0; i / = 2) {
for ( int j = 1, j<n; j * = 2) {
for ( int k = 0, k<n; k += 2) {
} // not nested
}
}
Answer: O(n(log n)^ 2), (2 is to the square root by the way)
The two outer loops are all log n, because it's having, and the inner one is N because it is halving right?
For this code, the correct answer is O(n) ^ 2, I understand the outer loop is n, and the middle loop is log n, and the inner loop should be n too. so why is the answer not N * N * log n?
for( int i = n; i > 0; i - -) {
for( int j = 1; j < n; j *= 2 ) {
for( int k = 0; k < j; k++ ) {
// constant number C of operations
}
}
}
Finally, how do I know when to add or multiply loops? if two loops are nested I just multiply them right? and when do I take the greatest N value over the other loops?
Here it is #2 formatted for readability:
for( int i = n; i > 0; i --)
for( int j = 1; j < n; j *= 2 )
for( int k = 0; k < j; k++ )
action
Forget the i-loop; we know it multiplies the inner bits by N.
The number of times action gets done by the nested j-, k-loops is then
1 + 2 + 4 + 8 + ... N. (If N is not a power of 2, replace it with the next lower power of 2.)
Put this in binary and sum it. For my example, let's let N be 16, but you can easily generalize.
00001
00010
00100
01000
10000
which sums to
11111
which is 2*N-1, or O(N).
Multiplying that by the i-loop range of N gives us O(N^2).
Interesting problem!
the outer and middle loops execute the same number of times, each sqrt(n). the inner loop executes n/2 times. Since they're nested, yes you multiply them together, for a total of sqrt(n)*sqrt(n)*n/2 critical step executions. This equals n^2/2, for which the limit as n approaches infinity is n^2, so it's in the O(n^2) classification of functions. You don't really just take the greatest N value, you take the asymptotic limit, when asked which family the function belongs to.
I'm asked to give a big-O estimates for some pieces of code but I'm having a little bit of trouble
int sum = 0;
for (int i = 0; i < n; i = i + 2) {
for (int j = 0; j < 10; j + +)
sum = sum + i + j;
I'm thinking that the worst case would be O(n/2) because the outer for loop is from i to array length n. However, I'm not sure if the inner loop affects the Big O.
int sum = 0;
for (int i = n; i > n/2; i − −) {
for (int j = 0; j < n; j + +)
sum = sum + i + j;
For this one, I'm thinking it would be O(n^2/2) because the inner loop is from j to n and the outer loop is from n to n/2 which gives me n*(n/2)
int sum = 0;
for (int i = n; i > n − 2; i − −) {
for (int j = 0; j < n; j+ = 5)
sum = sum + i + j;
I'm pretty lost on this one. My guess is O(n^2-2/5)
Your running times for the first two examples are correct.
For the first example, the inner loop of course always executes 10 times. So we can say the total running time is O(10n/2).
For the last example, the outer loop only executes twice, and the inner loop n/5 times, so the total running time is O(2n/5).
Note that, because of the way big-O complexity is defined, constant factors and asymptotically smaller terms are negligible, so your complexities can / should be simplified to:
O(n)
O(n2)
O(n)
If you were to take into account constant factors (using something other than big-O notation of course - perhaps ~-notation), you may have to be explicit about what constitutes a unit of work - perhaps sum = sum + i + j constitutes 2 units of work instead of just 1, since there are 2 addition operations.
You're NOT running nested loops:
for (int i = 0; i < n; i = i + 2);
^----
That semicolon is TERMINATING the loop definition, so the i loop is just counting from 0 -> n, in steps of 2, without doing anything. The j loop is completely independent of the i loop - both are simply dependent on n for their execution time.
For the above algorithms worst case/best case are the same.
In case of Big O notation, lower order terms and coefficient of highest order term can be ignored as Big O is used for describing asymptotic upper bound.
int sum = 0;
for (int i = 0; i < n; i = i + 2) {
for (int j = 0; j < 10; j + +)
sum = sum + i + j;
Total number of outer loop iteration =n/2.for each iteration of outer loop, number of inner loop iterations=10.so total number of inner loop iterations=10*n/2=5n. so clearly it is O(n).
Now think about rest two programs and determine time complexities on your own.