Trying to understand Big O and nested loops i've been going through the notes and can't understand how the nested loop part of this question works...I have an answer of 6 + 1.5n + nlogn wrote down from lectures but don't understand how to get the n log n part
Simple Statement;
Simple Statement;
Simple Statement;
Simple Statement;
for ( int i = 0; i < ( n / 2 ); i++ ) {
Simple Statement;
Simple Statement;
Simple Statement;
}
Simple Statement;
Simple Statement;
for ( int i = 0; i < 2 * n; i++ ) {
for ( int j = 0; j < n; j = 2 * j ) {
Simple Statement;
Simple Statement;
}
}
My understanding is the 6 is from the six statements not inside a loop and the 1.5n comes from 3(n-1 + n-2 +....1)/2 so if someone could help with the last part or correct me if im wrong it would be greatly appreciated
Part im stuck on:
for ( int i = 0; i < 2 * n; i++ ) {
for ( int j = 0; j < n; j = 2 * j ) {
Simple Statement;
Simple Statement;
}
}
Well, I guess, there´s a typo in the question, the inner loop should be
// notice "j = 1", not "j = 0",
// otherwise you have an infinite loop, since 0 * 2 == 0
for (int j = 1; j < n; j = 2 * j )
in that case, the outer loop
for (int i = 0; i < 2 * n; i++ )
brings 2 * n, while the inner one (notice j = 2 * j)
for (int j = 1; j < n; j = 2 * j )
results in just log(n); finally (since loops are nested we should multiplicate complexities) we have
O(n * log(n))
Iterating from 0 to 2*n will result in a complexity of O(N). Iterating from 0 to n having the steps power of 2 will result in a complexity of O(log(N)). Multiplying this 2 complexities will result in a final complexity of O(N * log(N)).
Related
Consider:
bar (a) {
for (int i = 0; i < n; ++i)
for (int j = 0; j < i; ++j)
a = a * (i + j);
return a;
}
Find the time complexity for the above function.
The time complexity is O(n^2).
Since the inner for loop runs from 0 to i and outer for loop runs from 0 to n it follows the pattern 1 + 2 + ... + n and this sums up to n*(n-1)/2, which in turn is O(n^2)
Please refer to the code fragment below:
sum = 0;
for( i = 0; i < n; i++ )
for( j = 0; j < i * i; j++ )
for( k = 0; k < j; k++ )
sum++;
If one was to analyze the run-time of this fragment in Big-Oh Notation, what would be the most appropriate value? Would it be O(N^4)?
I have to say O(n^5). I maybe wrong. n is the length of the array in this question I am assuming. If you substitute it, i will stop when i < n j will stop when j < n*n and k will stop when k < n * n. Since these are nested for-loops, the run time is O(n^5)
I think it's O(n^5)
We know that
1 + 2 + 3 + ... + n = n(n+1) / 2 = O(n^2)
1^2 + 2^2 + ... + n^2 = n(n+1)(2n+1) / 6 = O(n^3)
I think that similary statement is true for sum of bigger powers.
Let's think about these two lines
for( j = 0; j < i * i; j++ )
for( k = 0; k < j; k++ )
This loop's time complexity for fixed i is
1 + 2 + ... + (i-1)^2 = (i-1)^2(1 + (i-1)^2) / 2 = O(i^4)
But in our code i changes from 0 to n-1, so we like add fourth powers, and as I said before
1^k + 2^k + ... + n^k = O(n^(k+1))
That's why time complexity is O(n^5)
Given this code, consider each loop separately.
sum = 0;
for( i = 0; i < n; i++ ) //L1
for( j = 0; j < i * i; j++ ) //L2
for( k = 0; k < j; k++ ) //L3
sum++; //
Consider the outermost loop (L1) alone. Clearly this loop is O(n).
for( i = 0; i < n; i++ ) //L1
L2(i);
Consider the next loop (L2). This one is harder, but the loop is O(n^2).
See this SO answer for why the loop is O(n^2): Big O, how do you calculate/approximate it?
for( j = 0; j < i * i; j++ ) //L2
L3(j);
Consider the next loop (L3). Since you know that the L2 loop is O(N^2), what is this loop? (leaving reader to complete their homework).
for( k = 0; k < j; k++ ) //L3
sum++; //
I have seen that in some cases the complexity of nested loops is O(n^2), but I was wondering in which cases we can have the following complexities of nested loops:
O(n)
O(log n) I have seen somewhere a case like this, but I do not recall the exact example.
I mean is there any kind of formulae or trick to calculate the complexity of nested loops? Sometimes when I apply summation formulas I do not get the right answer.
Some examples would be great, thanks.
Here is an example for you where the time complexity is O(n), but you have a double loop:
int cnt = 0;
for (int i = N; i > 0; i /= 2) {
for (int j = 0; j < i; j++) {
cnt += 1;
}
}
You can prove the complexity in the following way:
The first iteration, the j loop runs N times. The second iteration, the j loop runs N / 2 times. i-th iteration, the j loop runs N / 2^i times.
So in total: N * ( 1 + 1/2 + 1/4 + 1/8 + … ) < 2 * N = O(N)
It would be tempting to say that something like this runs in O(log(n)):
int cnt = 0;
for (int i = 1; i < N; i *= 2) {
for (int j = 1; j < i; j*= 2) {
cnt += 1;
}
}
But I believe that this runs in O(log^2(N)) which is polylogarithmic
I've got to analyze this loop, among others, and determine its running time using Big-O notation.
for ( int i = 0; i < n; i += 4 )
for ( int j = 0; j < n; j++ )
for ( int k = 1; k < j*j; k *= 2 )`
Here's what I have so far:
for ( int i = 0; i < n; i += 4 ) = n
for ( int j = 0; j < n; j++ ) = n
for ( int k = 1; k < j*j; k *= 2 ) = log^2 n
Now the problem I'm coming to is the final running time of the loop. My best guess is O(n^2), however I am uncertain if this correct. Can anyone help?
Edit: sorry about the Oh -> O thing. My textbook uses "Big-Oh"
First note that the outer loop is independent from the remaining two - it simply adds a (n/4)* multiplier. We will consider that later.
Now let's consider the complexity of
for ( int j = 0; j < n; j++ )
for ( int k = 1; k < j*j; k *= 2 )
We have the following sum:
0 + log2(1) + log2(2 * 2) + ... + log2(n*n)
It is good to note that log2(n^2) = 2 * log2(n). Thus we re-factor the sum to:
2 * (0 + log2(1) + log2(2) + ... + log2(n))
It is not very easy to analyze this sum but take a look at this post. Using Sterling's approximation one can that it is belongs to O(n*log(n)). Thus the overall complexity is O((n/4)*2*n*log(n))= O(n^2*log(n))
In terms of j, the inner loop is O(log_2(j^2)) time, but sine
log_2(j^2)=2log(j), it is actually O(log(j)).
For each iteration of middle loop, it takes O(log(j)) time (to do the
inner loop), so we need to sum:
sum { log(j) | j=1,..., n-1 } log(1) + log(2) + ... + log(n-1) = log((n-1)!)
And since log((n-1)!) is in O((n-1)log(n-1)) = O(nlogn), we can conclude middle middle loop takes O(nlogn) operations .
Note that both middle and inner loop are independent of i, so to
get the total complexity, we can just multiply n/4 (number of
repeats of outer loop) with complexity of middle loop, and get:
O(n/4 * nlogn) = O(n^2logn)
So, total complexity of this code is O(n^2 * log(n))
Time Complexity of a loop is considered as O(n) if the loop variables is incremented / decremented by a constant amount (which is c in examples below):
for (int i = 1; i <= n; i += c) {
// some O(1) expressions
}
for (int i = n; i > 0; i -= c) {
// some O(1) expressions
}
Time complexity of nested loops is equal to the number of times the innermost statement is executed. For example the following sample loops have O(n²) time complexity:
for (int i = 1; i <=n; i += c) {
for (int j = 1; j <=n; j += c) {
// some O(1) expressions
}
}
for (int i = n; i > 0; i += c) {
for (int j = i+1; j <=n; j += c) {
// some O(1) expressions
}
Time Complexity of a loop is considered as O(logn) if the loop variables is divided / multiplied by a constant amount:
for (int i = 1; i <=n; i *= c) {
// some O(1) expressions
}
for (int i = n; i > 0; i /= c) {
// some O(1) expressions
}
Now we have:
for ( int i = 0; i < n; i += 4 ) <----- runs n times
for ( int j = 0; j < n; j++ ) <----- for every i again runs n times
for ( int k = 1; k < j*j; k *= 2 )` <--- now for every j it runs logarithmic times.
So complexity is O(n²logm) where m is n² which can be simplified to O(n²logn) because n²logm = n²logn² = n² * 2logn ~ n²logn.
Evaluate Big-Oh of the following code fragment:
sum = 0
for( i = 1; i < n; ++i )
for( j = 1; j < i * i; ++j )
if( j % i == 0 )
for( k = 0; k < j; ++k )
++sum
This is a homework problem in a textbook for my algorithms class. The answer as stated in the textbook is O(n^4). I've tried doing the problem many ways, but I am always getting O(n^5).
I'm using the summation method and mathematically evaluating from the innermost nested loop outward. The summations are not shown here because I don't know how to express my math in this space, but please follow my work below.
Here is my logic for the innermost loop:
for( k = 0; k < j; ++k )
My thinking is that the inner loop makes j+1 iterations, which can be as big as i*i, which itself can be as big as n, so this loop as an upper bound of O(n^2).
Here is my logic for the middle loop:
for( j = 1; j < i * i; ++j )
j iterates as high as i^2 times, which itself can go as high as n, so this loop has an upper bound of O(n^2).
Here is my logic for the outer loop:
for( i = 1; i < n; ++i )
i iterates as high as n times, so the loop has an upper-bound of O(n).
O(n * n^2 * n^2) = O(n^5)
Again, the answer is O(n^4). Please help me, using mathematical loops to aid your answer. Please use simple language. I am still new to algorithm analysis.
The trick is in this line:
if( j % i == 0 )
What this does is ensures the inner loop only executes when j is an exact multiple of i; otherwise no work is done.
So one shortcut you could think about is saying that this is O(n * n^2 / n * n^2) = O(n^4).
Another way you could think about it is that this is equivalent to writing:
sum = 0
for( i = 1; i < n; ++i )
for( j = 1; j < i * i; j += i )
for( k = 0; k < j; ++k )
++sum
which is O(N^4) by inspection.