sum = 0;
for (i = 0; i < m; i++)
for (j = 0; j < m*m; j++)
sum++;
Been trying to figure out the time complexity of this algorithm with Big-O Notation.
I've only found out that 1- first loop counts as O(m) and 2- Second loop counts as O(m^2). But I have a single problem. Does this nested loop counts as O(m^3) or do I have to take the fastest growing function as a basis and say that this loop counts as O(m^2) ?
You have to count how many times sum++ will effectively be performed, and not speculate about the growth of the functions.
Every time the inner loop is called, it executes sum++ exactly m² times.
And the inner loop is executed exactly m times by the outer loop, hence in total m³ sum++.
Needless to say, O(m³).
What helped me in university, just think of an example - e.g. 2.
for(int i = 0; i<2; i++)
for(int j = 0; j<4; j++)
So the loop runs 8 times, which is 2 * 4 times, which is 8, which is 2*2*2, or 2³.
Related
For the loop below,
int sum = 0;
for (int i = 1; i < N; i *= 2)
for (int j = 0; j < N; j++)
sum++;
what is the time complexity and how should I think? My guess is that the outer loop runs a total of log(N). The inner loop runs N times. Therefore, the time complexity should be Nlog(N).
Am I correct?
Thanks in advance.
For the first loop, the number of iterations is equal to log2(N), as i is doubled every iteration.
For each iteration of the first loop, the second loop runs for N times.
Therefore overall time complexity = (log2(N) * N), where the function log2(x) = log(x) to the base 2.
Suppose we have the following block of code:
sum = 0;
for(i = 0; i < n; i++)
for(j = 0; j < n; j++)
if(i == j)
for(k = 0; k < n; k++)
sum++;
In the book I'm reading it's stated that the complexity is O(n^2), thetha (n^2) to be precise, but I'm not sure why. When trying to calculate it myself I get O(n^3):
Outer loop complexity is O(n)
By the second loop the complexity gets to O(nˆ2) by following the nested loops rule
The if statement would be true n times, so by the third loop the complexity would be O(n*nˆ2)
The sum++ statement takes constant time, so complexity stays at O(nˆ3)
That's my logic, but it's flawed it seems. What is the reasoning for the O(n^2) complexity here?
The second loop runs n steps. Each step takes only constant time (evaluating the if statement) except for one step. The i-th step takes O(n) because the third loop is executed. So the second loop takes 2n = O(n) in total. So in total you get O(n^2).
for (int i = 0; i*i < N; i++)
for (int j = 0; j*j < N*N*N; j++)
sum++;
It is one of the exercises in the course Algorithms, Part I, on Coursera.
The outer for loop is the square root of N, and the inner loop is somehow the square root of N (j*j) and also N^3 (N * N * N). How does that become N^2?
There are two parts to answering this question. First, let's discuss how to determine the running time of programs will multiple loops. What we care about is how many times the sum++ statement is called - that's what we are measuring. Consider this program:
for (int i = 0; i < N; i++)
for (int j = 0; j < N; j++)
sum++;
First we look at the outer loop, and we see that it will be run N times. Now, we look at the inner loop, and we want to know how many times it will be run for each iteration of the outer loop. Once we know that, we can simply multiply the two values and will obtain the total number of times the sum++ has been called. In this case, it is easy to see that each time the outer loop is run, the inner loop will be run N times. Since the outer loop is also run N times, this has a running time of N^2.
As you noted, the outer loop will indeed be run sqrt(N) times. Now, we have to see how many times the inner loop will run for each iteration of the outer loop. We can do this by simplifying the counter in each loop. By doing this, we can see that the inner loop will be run N^(3/2) times each time the outer loop is called.
Why is this true? Well, observe that what you wrote is equivalent to
for (int i = 0; i < N^(1/2); i++)
for (int j = 0; j < (N*N*N)^(1/2); j++)
sum++;
Since for each iteration of the outer loop we run the inner loop N^(3/2) times, we can simply multiply these to get a total running time of O(N^2).
I am attending a course online and stuck at the following question. What is the order of growth of the worst case running time of the following code fragment as a function of N?
int sum = 0;
for (int i = 1; i <= N*N; i++)
for (int j = 1; j <= i; j++)
for (int k = 1; k <= j; k++)
sum++;
I thought that it is of the order of N^4 but it seems this answer is wrong. can you please explain?
It is of order O(N^6). You should note that it is not true that every loop simply add an order N to the complexity. Considering the following example:
int sum = 0;
for (int i = 1; i <= M; i++)
for (int j = 1; j <= i; j++)
for (int k = 1; k <= j; k++)
sum++;
You should be able to figure out it is of order O(M^3) easily, so if you replace M=N^2, then you will get the answer. The key point is that every inner loop are of order O(N^2) in this case, but not O(N).
Let's denote n = N^2. Then, the loop will execute each time that k <= i <= j. This will be approximately n^3/6 times. Thus, the runtime is O(n^3)= O(N^6)
Explanation: Ignoring for a moment the cases where k==i or j==i or j==k,
we take 1 out of 6 distinct triples :
(a1,a2,a3)
(a1,a3,a2)
(a2,a1,a3)
(a2,a3,a1)
(a3,a2,a1)
(a3,a1,a2)
Overall, there are n^3 triples. Only one out of 6 triples obeys the order.
One run of the inner loop increments sum exactly j times.
One run of the middle loop invokes the inner loop exactly i times, with values of j between 1 and i (inclusive). So it increments sum exactly 1+2+3+...i times, which is i.(i+1)/2 by the well-known "triangular numbers" formula.
The outer loop invokes the middle loop exactly N^2 times (let us denote it as M), with values of i between 1 and M (inclusive). So it increments sum exactly 1+3+6+...M.(M+1)/2 times. Similarly, this is M.(M+1).(M+2)/6, by the not-so-well-known "tetrahedral numbers" formula (http://en.wikipedia.org/wiki/Tetrahedral_number).
All in all, the final value of sum is N^2.(N^2+1).(N^2+2)/6.
Thinking in asymptotic terms, the inner loop is O(j), the middle one O(i^2) (by summation) and the outer one O(M^3) (by summation), i.e. O(N^6).
Also see Faulhaber's formula, which shows that the sum of n^k is O(N^(k+1)) (http://en.wikipedia.org/wiki/Faulhaber%27s_formula).
Any given run of the innermost (k) loop has a time proportional to j, but we've got to do one of those for each of j = 1 through j = i, and that sum 1 + 2 + … + i grows like i^2. So for any given i we've got an O(i^2) running time, but of course we've got to deal with i = 1 through i = N^2. The sum of i^2 for i = 1 through N^2 happens to grow like N^6.
Given the following code:
for (int i = 0; i < n-1; ++i)
{
for (int j = i+1; j < n; ++j)
{
// Do work.
}
}
What is the Big-Oh value for it (over n)? I'm thinking it's O(N^2) but I'm not sure.
I did find a similar question here: complexity for nested loops
but it's not quite the same I think.
Yes, that's O(N^2). Pair up the iterations of the inner loop in the beginning and at the end of the outer loop, like this:
The inner loop will execute...
N-1 times on the first iteration of the outer loop, and 1 time on the last iteration
N-2 times on the second iteration of the outer loop, and 2 times on the second to last iteration
N-3 times on the third iteration of the outer loop, and 3 times on the third to last iteration
... and so on; you will have N/2 pairs like that; when N is odd, the last pair is incomplete.
You can see that each of the pairs executes a total of N times, and you have N/2 such pairs, for a total of N*(N-1)/2 times.
The way the formula is derived comes from the derivation of the formula for the sum of arithmetic progression with the common difference of 1.
It should be possible to check it this way.
int z = 0, n = 10; // try 20 etc
for (int i = 0; i < n-1; ++i)
{
for (int j = i+1; j < n; ++j)
{
z++;
}
}
Now, check the value of z.
With n = 10; z becomes 45
With n = 20; z becomes 190
With n = 40; z becomes 780
A doubling in n caused z to become ~4 times its value. Hence, it is approximately O(n^2).
Methodically, using Sigma notation (empirically verified), you can obtain the exact number of iterations plus the order of growth complexity: