I tried to solve the following exercise :
What is the order of growth of the worst case running time of the following code fragment
as a function of N?
int sum = 0;
for (int i = 1; i <= N; i++)
for (int j = 1; j <= i*i; j++)
for (int k = 1; k <= j*j; k++)
sum++;
and I found that the complexity is O(n4), however the correct answer is :
The answer is : N7
For a given value of i, the body of the innermost loop is executed 12 + 22 + 32 + ... + (i2)2 ~ 1/3 i6 times. Summing up over all values of i yields ~ 1/21 N7.
I would like some help to understand this answer and the correct way to calculate complexity in this case.
EDIT : In particular, I don't understand this statement :
12 + 22 + 32 + ... + (i2)2 ~ 1/3 i6
Because for me, 2 + 22 + 32 + ... + (i2)2 ~ i4
EDIT:
I'll add a bit of explanation to clear up your confusion about the quote in your question. Let's consider a fixed value of i and focus on the innermost two loops:
for (int j = 1; j <= i*i; j++)
for (int k = 1; k <= j*j; k++)
sum++;
How many times is the j-loop iterated? The answer is i^2 times. On each of those iterations, the k-loop is iterated j^2 times, which is different for each outer iteration because the value of j increases from 1 all the way to i^2.
When j = 1, the k-loop iterates 1^2 times. When j = 2, the k-loop iterates 2^2 times. When j=3, 3^2 times. Tallying up the total number of the iterations of the k-loop over all values of j, you have 1^2 + 2^2 + 3^2 + ... + (i^2)^2, since j runs between 1 and i^2. Hopefully that clarifies how you arrive at the following statement:
For a given value of i, the body of the innermost loop is executed 12 + 22 + 32 + ... + (i2)2 ~ 1/3 i6 times.
The total number of iterations can be expressed in sum form. The innermost loop has exactly j^2 iterations for each (varying) value of j, the middle loop has i^2 iterations for each value of i, and the outermost loop has N iterations. More neatly, the exact number of iterations is:
Multiplying through, you'll find this is a 7th order polynomial in N, so it is apparent why this is O(N^7).
In case you doubt the answer above is correct, simply run your own code and compare the value of sum you get with the formula provided above:
var sum = 0;
var N = 10;
for (var i = 1; i <= N; i++)
for (var j = 1; j <= i*i; j++)
for (var k = 1; k <= j*j; k++)
sum++;
function T(N) { return (1/420)*N*(1+N)*(1+2*N)*(8+11*N+21*N*N+20*N*N*N+10*N*N*N*N); }
console.log(sum===T(N));
Here's a demo: http://jsfiddle.net/wby9deax/. No matter what value of N you put in the answer will be correct (note: be careful with large values for N, it will probably freeze up your browser, since the number of iterations grows very rapidly).
int sum = 0;
for (int i = 1; i <= N; i++) -- O(N^1)
for (int j = 1; j <= i*i; j++) -- O(N^2)
for (int k = 1; k <= j*j; k++) -- O(N^4)
sum++;
Since they're nested (and since they're all linear) you get O(N1 × N2 × N4) = O(N1+2+4) = O(N7)
EDIT : In particular, I don't understand this statement :
12 + 22 + 32 + ... + (i2)2 ~ 1/3 i6
Keep in mind that you may have N terms hiding in the "…" part.
because will be
N^1 loops - first for
N^2 loops - second for
N^4 loops - third for
and N^1 * N^2 * N^4 = N^7
I think it is good idea to substitute variables (i,j and k) by N values.
for (int i = 1; i <= N; i++) //<- i = N
for (int j = 1; j <= i*i; j++) //<- i*i = N*N
for (int k = 1; k <= j*j; k++) //<- j*j = (i*i)*(i*i) = N*N*N*N
In the first loop number of iterations will be N, that's simple part.
In second loop number of iterations will be N*N. In the third - N*N*N*N.
So finally, number of iterations will be N (first loop), times N*N (second), times N*N*N*N (third), so N*(N*N)*(N*N*N*N) = N^7
Related
int sum = 0;
for(int i = 1; i < n; i++) {
for(int j = 1; j < i * i; j++) {
if(j % i == 0) {
for(int k = 0; k < j; k++) {
sum++;
}
}
}
}
I don't understand how when j = i, 2i, 3i... the last for loop runs n times. I guess I just don't understand how we came to that conclusion based on the if statement.
Edit: I know how to compute the complexity for all the loops except for why the last loop executes i times based on the mod operator... I just don't see how it's i. Basically, why can't j % i go up to i * i rather than i?
Let's label the loops A, B and C:
int sum = 0;
// loop A
for(int i = 1; i < n; i++) {
// loop B
for(int j = 1; j < i * i; j++) {
if(j % i == 0) {
// loop C
for(int k = 0; k < j; k++) {
sum++;
}
}
}
}
Loop A iterates O(n) times.
Loop B iterates O(i2) times per iteration of A. For each of these iterations:
j % i == 0 is evaluated, which takes O(1) time.
On 1/i of these iterations, loop C iterates j times, doing O(1) work per iteration. Since j is O(i2) on average, and this is only done for 1/i iterations of loop B, the average cost is O(i2 / i) = O(i).
Multiplying all of this together, we get O(n × i2 × (1 + i)) = O(n × i3). Since i is on average O(n), this is O(n4).
The tricky part of this is saying that the if condition is only true 1/i of the time:
Basically, why can't j % i go up to i * i rather than i?
In fact, j does go up to j < i * i, not just up to j < i. But the condition j % i == 0 is true if and only if j is a multiple of i.
The multiples of i within the range are i, 2*i, 3*i, ..., (i-1) * i. There are i - 1 of these, so loop C is reached i - 1 times despite loop B iterating i * i - 1 times.
The first loop consumes n iterations.
The second loop consumes n*n iterations. Imagine the case when i=n, then j=n*n.
The third loop consumes n iterations because it's executed only i times, where i is bounded to n in the worst case.
Thus, the code complexity is O(n×n×n×n).
I hope this helps you understand.
All the other answers are correct, I just want to amend the following.
I wanted to see, if the reduction of executions of the inner k-loop was sufficient to reduce the actual complexity below O(n⁴). So I wrote the following:
for (int n = 1; n < 363; ++n) {
int sum = 0;
for(int i = 1; i < n; ++i) {
for(int j = 1; j < i * i; ++j) {
if(j % i == 0) {
for(int k = 0; k < j; ++k) {
sum++;
}
}
}
}
long cubic = (long) Math.pow(n, 3);
long hypCubic = (long) Math.pow(n, 4);
double relative = (double) (sum / (double) hypCubic);
System.out.println("n = " + n + ": iterations = " + sum +
", n³ = " + cubic + ", n⁴ = " + hypCubic + ", rel = " + relative);
}
After executing this, it becomes obvious, that the complexity is in fact n⁴. The last lines of output look like this:
n = 356: iterations = 1989000035, n³ = 45118016, n⁴ = 16062013696, rel = 0.12383254507467704
n = 357: iterations = 2011495675, n³ = 45499293, n⁴ = 16243247601, rel = 0.12383580700180696
n = 358: iterations = 2034181597, n³ = 45882712, n⁴ = 16426010896, rel = 0.12383905075183874
n = 359: iterations = 2057058871, n³ = 46268279, n⁴ = 16610312161, rel = 0.12384227647628734
n = 360: iterations = 2080128570, n³ = 46656000, n⁴ = 16796160000, rel = 0.12384548432498857
n = 361: iterations = 2103391770, n³ = 47045881, n⁴ = 16983563041, rel = 0.12384867444612208
n = 362: iterations = 2126849550, n³ = 47437928, n⁴ = 17172529936, rel = 0.1238518469862343
What this shows is, that the actual relative difference between actual n⁴ and the complexity of this code segment is a factor asymptotic towards a value around 0.124... (actually 0.125). While it does not give us the exact value, we can deduce, the following:
Time complexity is n⁴/8 ~ f(n) where f is your function/method.
The wikipedia-page on Big O notation states in the tables of 'Family of Bachmann–Landau notations' that the ~ defines the limit of the two operand sides is equal. Or:
f is equal to g asymptotically
(I chose 363 as excluded upper bound, because n = 362 is the last value for which we get a sensible result. After that, we exceed the long-space and the relative value becomes negative.)
User kaya3 figured out the following:
The asymptotic constant is exactly 1/8 = 0.125, by the way; here's the exact formula via Wolfram Alpha.
Remove if and modulo without changing the complexity
Here's the original method:
public static long f(int n) {
int sum = 0;
for (int i = 1; i < n; i++) {
for (int j = 1; j < i * i; j++) {
if (j % i == 0) {
for (int k = 0; k < j; k++) {
sum++;
}
}
}
}
return sum;
}
If you're confused by the if and modulo, you can just refactor them away, with j jumping directly from i to 2*i to 3*i ... :
public static long f2(int n) {
int sum = 0;
for (int i = 1; i < n; i++) {
for (int j = i; j < i * i; j = j + i) {
for (int k = 0; k < j; k++) {
sum++;
}
}
}
return sum;
}
To make it even easier to calculate the complexity, you can introduce an intermediary j2 variable, so that every loop variable is incremented by 1 at each iteration:
public static long f3(int n) {
int sum = 0;
for (int i = 1; i < n; i++) {
for (int j2 = 1; j2 < i; j2++) {
int j = j2 * i;
for (int k = 0; k < j; k++) {
sum++;
}
}
}
return sum;
}
You can use debugging or old-school System.out.println in order to check that i, j, k triplet is always the same in each method.
Closed form expression
As mentioned by others, you can use the fact that the sum of the first n integers is equal to n * (n+1) / 2 (see triangular numbers). If you use this simplification for every loop, you get :
public static long f4(int n) {
return (n - 1) * n * (n - 2) * (3 * n - 1) / 24;
}
It is obviously not the same complexity as the original code but it does return the same values.
If you google the first terms, you can notice that 0 0 0 2 11 35 85 175 322 546 870 1320 1925 2717 3731 appear in "Stirling numbers of the first kind: s(n+2, n).", with two 0s added at the beginning. It means that sum is the Stirling number of the first kind s(n, n-2).
Let's have a look at the first two loops.
The first one is simple, it's looping from 1 to n. The second one is more interesting. It goes from 1 to i squared. Let's see some examples:
e.g. n = 4
i = 1
j loops from 1 to 1^2
i = 2
j loops from 1 to 2^2
i = 3
j loops from 1 to 3^2
In total, the i and j loops combined have 1^2 + 2^2 + 3^2.
There is a formula for the sum of first n squares, n * (n+1) * (2n + 1) / 6, which is roughly O(n^3).
You have one last k loop which loops from 0 to j if and only if j % i == 0. Since j goes from 1 to i^2, j % i == 0 is true for i times. Since the i loop iterates over n, you have one extra O(n).
So you have O(n^3) from i and j loops and another O(n) from k loop for a grand total of O(n^4)
Can I get some help in understanding how to solve this tutorial question! I still do not understand my professors explanation. I am unsure of how to count the big 0 for the third/most inner loop. She explains that the answer for this algorithm is O(n^2) and that the 2nd and third loop has to be seen as one loop with the big 0 of O(n). Can someone please explain to me the big O notation for the 2nd / third loop in basic layman terms
Assuming n = 2^m
for ( int i = n; i > 0; i --) {
for (int j =1; j < n; j *= 2){
for (int k =0; k < j; k++){
}
}
}
As far as I understand, the first loop has a big O notation of O(n)
Second loop = log(n)
Third loop = log (n) (since the number of times it will be looped has been reduced by logn) * 2^(2^m-1)( to represent the increase in j? )
lets add print statement to the innermost loop.
for (int j =1; j < n; j *= 2){
for (int k =0; k < j; k++){
print(1)
}
}
output for
j = 1, 1 1
j = 2, 1 1 1
j = 4, 1 1 1 1 1
...
j = n, 1 1 1 1 1 ... n+1 times.
The question boils down to how many 1s will this print.
That number is
(2^0 + 1) + (2^1 + 1) + (2^2 + 1) + ... + (n + 1)
= (2^0 + 1) + (2^1 + 1) + (2^2 + 1) + ... + (n + 1)
= log n + (1 + 2 + 4 + ... + n)
= O(log n + n)
= O(n).
assuming you know why (1 + 2 + 4 + ... + n) = O(n)
O-notation is an upperbound. You can say it has O(n^2). For least upperbound, I believe it should be O(n*log(n)*log(n)) which belongs to O(n^2).
It’s because of the logarithm. If you have log(16) raised to the power 2 is 16. So log(n) raised to the power of 2 is n. That is why your teacher says to view the second and third loop as O(n) together.
If the max iterations for the second loop are O(log(n)) then the second and third loops will be: O(1 + 2 + 3 + ... + log(n)) = O(log(n)(log(n) + 1)/2) = O((log(n)^2 + log(n))/2) = O(n)
for ( int i = n; i > 0; i --) { // This runs n times
for (int j =1; j < n; j *= 2){ // This runs atmost log(n) times, i.e m times.
for (int k =0; k < j; k++){ // This will run atmost m times, when the value of j is m.
}
}
}
Hence, the overall complexity will be the product of all three, as mentioned in the comments under the question.
Upper bound can be loose or tight.
You can say that it is loosely bound under O(n^2) or tightly bound under O(n * m^2).
I'm working on a data structures course and I'm not sure how to proceed w/ this Big O analysis:
sum = 0;
for(i = 1; i < n; i++)
for(j = 1; j < i*i; j++)
if(j % i == 0)
for(k = 0; k < j; k++)
sum++;
My initial idea is that this is O(n^3) after reduction, because the innermost loop will only run when j/i has no remainder and the multiplication rule is inapplicable. Is my reasoning correct here?
Let's ignore the outer loop for a second here, and let's analyze it in terms of i.
The mid loop runs i^2 times, and is invoking the inner loop whenever j%i == 0, that means you run it on i, 2i, 3i, ...,i^2, and at each time you run until the relevant j, this means that the inner loop summation of running time is:
i + 2i + 3i + ... + (i-1)*i = i(1 + 2 + ... + i-1) = i* [i*(i-1)/2]
The last equality comes from sum of arithmetic progression.
The above is in O(i^3).
repeat this to the outer loop which runs from 1 to n and you will get running time of O(n^4), since you actually have:
C*1^3 + C*2^3 + ... + C*(n-1)^3 = C*(1^3 + 2^3 + ... + (n-1)^3) =
= C/4 * (n^4 - 2n^3 + n^2)
The last equation comes from sum of cubes
And the above is in O(n^4), which is your complexity.
This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 8 years ago.
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i; j++)
sum++;
My guess is this is simply a O(N^2), but the j < i is giving me doubts.
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i*i; j++)
sum++;
Seems like an O(N^3)
int n = 500;
for(int i = 0; i < n; i++)
for(int j = 0; j < i*i; j++)
if( j % i == 0 )
for( k = 0; k < j; k++ )
sum++
O(N^5)?
So for each loop j has a different value. If it was j < n*n, it would've been more straight forward, but this one is a tricky one, so please help. Thanks.
In the first case sum++ executes 0 + 1 + ... + n-1 times. If you apply arithmetic progression formula, you'll get n (n-1) / 2, which is O(n^2).
In the second case we'll have 0 + 1 + 4 + 9 + ... + (n-1)^2, which is sum of squares of first n-1 numbers, and there's a formula for it: (n-1) n (2n-1)
The last one is interesting. You can see, actually, that the most nested for loop is called only when j is a multiplicand of i, so you can rewrite the program as follows:
int n = 500;
for(int i = 0; i < n; i++) {
for(int m = 0; m < i; m++) {
int j = m * i;
for( k = 0; k < j; k++)
sum++
}
}
It's easier to work with math notation:
The formula is derived from the code by analysis: we can see that sum++ gets called j times in the innermost loop, which is called i times, which is called n times. In the end, the problem boils down to a sum of cubes of first n numbers plus lower-order terms (which do not affect the asymptotics)
One final note: it looks obvious, but I'd like to show that in general sum of first N natural numbers in dth power is Ω(N^(d+1)) (see Wikipedia for Big-Omega notation), that is it grows no slower than that function. You can apply the same reasoning to prove that a stronger condition is satisfied, namely, it belongs to Θ(N^(d+1)), which combines both Ω and O.
You are right for all but the last one, which has a tighter bound of O(n^4): note that the last for loop is only executed if j is a multiple of i. There are x / i multiples of i lower than or equal to x, and i * i / i = i. So the last loop is only executed for i values out of the i * i.
Note that big-oh gives an upper bound, so i*i vs n*n makes little difference. Strictly speaking, saying they are all O(n^2015) is also correct (because that is a valid upper bound), but it's hardly helpful, so in practice a tight bound is usually used.
IVlad already gave the correct answer.
I think what confuses you is the "Big Oh" definition.
N^2 has O(N^2)
1/2N^2 has O(N^2)
1/2N^2 + c*N + b also has
O(N^2) - by given c and b are constants independent from N
Check Big Oh definition from here
It's from homework, but I'm asking for a general method.
Calculate the following code's worst case running time.
int sum = 0;
for (int i = 0; i*i < N; i++)
for (int j = 0; j < i*i; j++)
sum++;
the answer is N^3/2, could anyone help me through this?
Is there a general way to calculate this?
This is what I thought:
when i = 0, sum++ will be called 0 time
when i = 1, sum++ will be called 1 time
when i = 2, sum++ will be called 4 times
...
when i = i, sum++ will be called i^2 times
so the worst time will be
0 + 1 + 4 + 9 + 16 + ... + i^2
but what next?? I'm lost here...
You want to count how many times the innermost cycle will run.
The outer one will run from i = 0, to i = sqrt(N) (since i*i < N).
For each iteration of the outer one the inner one will run i^2 times.
Thus the total number of times the inner one will run is:
1^2 + 2^2 + 3^2 + ... + sqrt(N)^2
There is a formula:
1^2 + 2^2 + ... + k^2 = k(k+1)(2k+1) / 6 = O(k^3).
In your case k = sqrt(N).
This the total complexity is O(sqrt(N)^3) = O(N^(3/2)).
Your algorithm can be converted to the following shape:
int sum = 0;
for (int i = 0; i < Math.sqrt(N); i++)
for (int j = 0; j < i*i; j++)
sum++;
Therefore, we may straightforwardly and formally do the following:
then just calculate this sum
(i^2)/2 * N^(1/2) = N/2 * N^(1/2) = N^(3/2)
You are approaching this problem in the wrong way. To count the worst time, you need to find the maximum number of operations that will be performed. Because you have only a single operation in a double loop, it is enough to find out how many times the inner loop will execute.
You can do this by examining the limits of your loops. For the outer loop it is:
i^2 < N => i < sqrt(N)
The limit for your inner loop is
j < i^2
You can substitute in the second equasion to get j < N.
Because these are nested loops you multiply their limits to get the final result:
sqrt(N)*N = N^3/2