My teacher told me that basically whenever we had a loop and a nested loop the number of operation was as follows n(n+1)/2.
However, I looked at some programs and I realized that it's unlikely to be the case.
for(i=0, i<n, n++)
for(j=i, j<n, j++)
{x=i+j}
in this case it would be n(n+1)/2, ignoring i=0, j=0, n++, j++ and x=i+j, but here:
for(i=0, i<n, n++)
for(j=0, j<n, j++)
{x=i+j}
it would be n^n unless i am mistaken.
Can someone tell me exactly when two loops have n(n+1)/2 number of operations? I am kinda confused right now.
In your first example, the operation would be done n times, then n-1 times, then n-2 times. If I remember correctly, this is n(n-1)/2, but you could be right and it's n(n+1)/2. Either way, it's a very small difference.
In your second example, it would be done n times, then n times, then n times... until you've done it n times n times -- in other words, n^2.
You probably misunderstood the teacher (or the teacher made a mistake). n(n-1)/2 is just one example of a common loop runtime. It can be anything as you've observed. Your second example has n^2 operations though, another common pattern. 'n^n' is much rarer.
for(i=0, i<n, n++)
for(j=0, j<n, j++)
{x=i+j}
First loop runs n times and for each iteration you run second loop n times, thus you have n*n = n^2.
for(i=0, i<n, n++)
for(j=i, j<n, j++)
{x=i+j}
First loop runs n times and for each iteration you run the second loop (n-i) times....thus number of times x=i+j is getting executed is 1 + 2 + 3 ..... + n times, this sequence is sum of first n integers, which is same as n(n+1)/2
Related
int cnt = 0;
for (int i = 1; i < n; i++) {
for (int j = i; j < n; j++) {
for (int k = j * j; k < n; k++) {
++cnt;
}
}
}
I have no idea of it.
How to analyze the time complexity of it?
It's easy to see that the code is Omega(n²) (that is, is at least quadratic) - the two outer loops execute around n²/2 times.
The inner k loop executes zero times unless j is less than sqrt(n). Even though it executes zero times, it takes some computation to compute the conditions for the loop, so it's O(1) work in these cases.
When j is less than sqrt(n), i must also be less than sqrt(n), since by the construction of the loops, j is always greater than or equal to i. In these cases, the k loop does n-j² iterations. We can construct a bound for the total amount of work in this inner loop in these cases: both i and j are less than sqrt(n), and there's at worst O(n) work done in the k loop, so there's at most O(n²) (ie: sqrt(n) * sqrt(n) * n) total work done in the inner loop.
There's also at most O(n²) total work done for the cases where the inner loop is trivial (ie: when j>sqrt(n)).
This gives a proof that the runtime complexity of the code is θ(n²).
Methods involving looking at nested loops individually and constructing big-O bounds for them do not in general give tight bounds, and this is an example question where such a method fails.
The first approach would be to look at the loops separately, meaning that we have three O(.) that are connected by a product. Hence,
Complexity of the Algorithm = O(OuterLoop)*O(MiddleLoop)*O(InnerLoop)
Now look at each loop separately:
Outerloop: This is the most simple one. Incrementing from 1 to n, resulting in O(n)
Middleloop: This is non-obvious, the terminate condition of the loop is still n, but the starting iterator value is i, meaning that the larger i gets, less time it will take to finish the loop. But this factor is quadratical-asymptotically only a constant, meaning that it is still O(n), hence O(n^2) "until" the second loop.
Inner loop: We see, that the iterator increases quadratically. But we also see that the quadratic-increasing depends on the second loop, which we said to be O(n). Since, we again only look at the complexity asymptomatically, means that we can assume that j rises linearly, and since k rises quadratically until n, it will take \square(n) iterations until n is reached. Meaning that the inner most loop has a running time of O(\square(n)).
Putting all these results together,
O(n * n* square(n))=O(n^2*square(n))
Looking at the code below:
Algorithm sort
Declare A(1 to n)
n = length(A)
for i = 1 to n
for j = 1 to n-1 inclusive do
if A[i-1] > A[i] then
swap( A[i-1], A[i] )
end if
next j
next i
I would say that there are:
2 loops, both n, n*n = n^2 (n-1 truncated to n)
1 comparison, in the j loop, that will execute n^2 times
A swap that will execute n^2 times
There are also 2n additions with the loops, executing n^2 times, so 2n^2
The answers given in a mark scheme:
Evaluation of algorithm
Comparisons
The only comparison appears in the j loop.
Since this loop will iterate a total of n^2
times, it will execute
exactly n^2
Data swaps
There may be a swap operation carried out in the j loop.
Swap( A[i-1], A[i] ) Each of these will happen n^2 times.
Therefore there are 2n^2 operation carried out within the j loop
The i loop has one addition operation incrementing i which happens n
times
Adding these up we the number of addition operations which is 2n^2 +
n
As n gets very big then n^2 will dominate therefore it is O(n^2)
NOTE: Calculations might include assignment operations but these will not affect overall time so ignore
Marking overview:
1 mark for identifying i loop will execute n times.
1 mark for identifying j loop will execute 2n^2 times Isn't this meant to be n*n = n^2? For i and j
1 mark for correct number of calculations 2n^2 + n Why is this not
+2n?
1 mark for determining that the order will be dominated by n^2 as n
gets very big giving O(n^2) for the algorithm
Edit: As can be seen from the mark scheme, I am expected to count:
Loop numbers, but n-1 can be truncated to n
Comparisons e.g. if statements
Data swaps (counted as one statement, i.e. arr[i] = arr[i+1], temp = arr[i], etc. are considered one swap)
Calculations
Space - just n for array, etc.
Could someone kindly explain how these answers are derived?
Thank you!
Here's my take on the marking scheme, explicitly marking the operations they're counting. It seems they're counting assignments (but conveniently forgetting that it takes 2 or 3 assignments to do a swap). That explains why they count increment but not the [i-1] indexing.
Counting swaps
i loop runs n times
j loop runs n-1 times (~n^2-n)
swap (happens n^2 times) n^2
Counting additions (+=)
i loop runs n times
j loop runs n-1 times (~n^2)
increment j (happens n^2 times) n^2
increment i (happens n times) n
sum: 2n^2 + n
I currently have the following pseudo code, and I am trying to figure out why the answer to the question is O(n).
sum = 0;
for (i = 0; i < n; i++) do
for (j = n/3;j < 2*n; j+= n/3) do
sum++;
I thought the answer would be O(n^2) since the first for loop would run 'n' times and the second for loop has += n/3, giving it another (n divided by something times), which would just simplify to O(n^2). Could somebody explain why it is O(n)?
This is because the second loop runs in constant amount of operations (does not depend on n). From n/3 to 2n with a step n/3 which is similar to from 1/3 to 2 with a step 1/3.
This will run 5-6 times for reasonable n (not 0) (the number is not important and depends on how do you calculate /)
The inner loop increments by a multiple of n, not by 1, so its runtime is bounded by a constant (6?). So the total number of steps is bounded by a constant multiple of n (namely 6n).
What would be the order of growth of the code below. My guess was, each loop's growth is linear but the if statement is confusing me. How do I include that with the whole thing. I would very much appreciate an explanatory answer so I can understand the process involved.
int count = 0;
for (int i = 0; i < N; i++)
for (int j = i+1; j < N; j++)
for (int k = j+1; k < N; k++)
if(a[i] + a[j] + a[k] == 0)
count++;
There are two things that can be confusing when trying to determine the code's complexity.
The fact that not all loops start from 0. The second loop starts from i + 1 and the third from j + 1. Does this affect the complexity? It does not. Let's consider only the first two loops. For i = 0, the second runs N - 1 times, for i = 1 it runs N - 2 times, ..., for i = N - 1 it runs 0 times. Add all these up:
0 + 1 + ... + N - 1 = N(N - 1) / 2 = O(N^2).
So not starting from 0 does not affect the complexity (remember that big-oh ignores lower-order terms and constants). Therefore, even under this setting, the entire thing is O(N^3).
The if statement. The if statement is clearly irrelevant here, because it's only part of the last loop and contains no break statement or other code that would affect the loops. It only affects the incrementation of a count, not the execution of any of the loops, so we can safely ignore it. Even if the count isn't incremented (an O(1) operation), the if condition is checked (also an O(1) operation), so the same rough number of operations is performed with and without the if.
Therefore, even with the if statement, the algorithm is still O(N^3).
Order of growth of the code would be O(N^3).
In general k nested loops of length N contribute growth of O(N^k).
Here are two was to find that the time complexity is Theta(N^3) without much calculation.
First, you select i<j<k from the range 0 through N-1. The number of ways to choose 3 objects out of N is the binomial coefficient N choose 3 = N*(N-1)*(N-2)/(3*2*1) ~ (N^3)/6 = O(N^3), and more precisely Theta(N^3).
Second, an upper bound is that you choose i, j, and k from N possibilities, so there are at most N*N*N = N^3 choices. This is O(N^3). You can also find a lower bound of the same type since you can choose i from 0 through N/3-1, j from N/3 through 2N/3-1, and k from 2N/3 through N-1. This gives you at least floor(N/3)^3 choices, which is about N^3/27. Since you have an upper bound and lower bound of the same form, the time complexity is Theta(N^3).
How do you calculate the big O run-time efficiency for this code? My gut tells me it is O(n^3) but I'm not sure, as I'm not really sure if the loops are independent or dependent.
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
for (k=1; k<=n; k++)
print ("%d %d %d/n", i, j, k);
Your gut feeling is right. You have three nested for loops iterating over n, so for each of the first n loops you make another n loops, each of which in turn make n more loops. Thus O(n^3).
Edit: think about how this will play out-
i is first 1, j 1 as well, and then k loops 1 through n. Only after k has undergone that whole loop, will j increment to 2, then k undergoes the loop once again and so on.
Yes, this function would be O(N^3) because inside each loop you are running N iterations.
The loops are not independent, because they are nested.
N * N * N = N^3