Confusion about complexity of two dependent loops? - algorithm

for(i=1; i < n; i++){
for(j=1; j <= i; j++){
statement1;
}
}
outer loop = O(N)
Inner loop = N(N-1)/2
Total = N*N(N-1)/2 = N^3
it seems n^3 is complexity of these nested loops. but accordings to books, its complexity is n^2 from N(N-1)/2 .

The only interesting thing to count is how often statement1 will be executed.
Therefore, note that something like
for (int i = 0; i < 2; i++)
for (int j = 0; j < 3; j++)
statement1;
triggers 2 * 3 = 6 executions. So you count how often the inner loop gets executed per outer loop iteration.
However, in your example you did a mistake and multiplied the iterations of the outer loop with the total iterations of the inner loop, not the number of iterations per outer loop iteration.
In the example above that would be like 2 * 6 = 12 instead of only 2 * 3 = 6.
Let's take a closer look at what happens in your specific example. The outer loop triggers n iterations of the inner loop. The inner loop first yields 1 iteration. In the next iteration of the outer loop it will yield 2 iterations, then 3 and so on.
In total you will thus receive 1 + 2 + ... + n = (n^2 - n)/2 iterations of the inner loop. Again, note the 'in total'. So statement1 will in total be executed (n^2 - n)/2 times. The outer loops iterations are already taken into account for the computation of the inner loops total runs, no additional multiplication.
(n^2 - n)/2 is obviously in O(n^2) due to its asymptotic complexity. Intuitively only the biggest factor plays a role, we can drop other stuff by estimating with <=.
(n^2 - n)/2
<= n^2 - n
<= n^2 in O(n^2)

for(i=1; i < n; i++){
for(j=1; j <= i; j++){
statement1;
}
}
In order to simplify the problem, let's assume that n is 5 here.
So line 1 will execute 5 times since it will check and increment i value 5 times.
line 2 will execute (5-1)=4 times because for i=5, it will not execute but line 1 will execute for i=5.
line 3 will execute for 1 time, 2 times 3 times and so on, each time i is incremented.
Take complexity of 3rd line into context and you'll find that it is executing 1+2+3+4=10 times. It's simply the sum of numbers from 1 to 4 or you can say, n(n+1)/2 where n=4.
We can ignore the complexity of line 1 and line 2 since they are constant and in asymptotic notation , the complexity will be O(n^2).

You can think about the 2 nested loops as checking all the cells on the diagonal and below the diagonal on a N x N matrix.
So you'll always do a number of operations close to 1/2 of N^2. So the total number of operations of your code will be N^2 * constant. By the definition of Big-O notation, that means that your code runtime complexity is O(n^2).
Here is a simple code to help you understand my explanation.
#include <vector>
#include <iostream>
using std::vector;
using std::cout;
using std::endl;
// These function count the number of times that your code will execute statement1
int count(int N){
int total = 0;
for(int l = 0; l < N; ++l){
for(int r = 0; r <= l; ++r){
total++;
}
}
return total;
}
// this code will show the cells of the matrix that you are "executing"
int showMatrix(int N){
vector<vector<bool> > mat(N, vector<bool>(N, false) );
for(int l = 0; l < N; ++l){
for(int r = 0; r <= l; ++r){
mat[l][r] = true;
}
}
for(int line = 0; line < N; ++line){
for(int column = 0; column < N; ++column){
cout << (mat[line][column] == true ? "1 " : "0 ");
}
cout << endl;
}
}
int main(){
showMatrix(10);
cout << count(10) << endl;
return 0;
}

Related

Time Complexity for for loop with if-else block

I want to find the time complexity for this below code. Here's my understanding-
The outer for loop will loop 2n times and in the worst case when i==n, we will enter the if block where the nested for loops have complexity of O(n^2), counting the outer for loop, the time complexity for the code block will be O(n^3).
In best case when i!=n, else has complexity of O(n) and the outer for loop is O(n) which makes the complexity, in best case as O(n^2).
Am I correct or am I missing something here?
for (int i = 0; i < 2*n; i++)
{
if (i == n)
{
for (int j = 0; j < i; j++)
for (int k = 0; k < i; k++)
O(1)
}
else
{
for (int j = 0; j < i; j++)
O(1)
}
}
No.
The question "what is T(n)?".
What you are saying is "if i=n, then O(n^3), else O(n^2)".
But there is no i in the question, only n.
Think of a similar question:
"During a week, Pete works 10 hours on Wednesday, and 1 hour on every other day, what is the total time Pete works in a week?".
You don't really answer "if the week is Wednesday, then X, otherwise Y".
Your answer has to include the work time on Wednesday and on every other day as well.
Back in your original question, Wednesday is the case when i=n, and all other days are the case when i!=n.
We have to sum them all up to find the answer.
This is a question of how many times O(1) is executed per loop. The time complexity is a function of n, not i. That is, "How many times is O(1) executed at n?"
There is one run of a O(n^2) loop when i == n.
There are (2n - 2) instances of the O(n) loop in all other cases.
Therefore, the time complexity is O((2n - 2) * n + 1 * n^2) = O(3n^2 - 2*n) = O(n^2).
I've written a C program to spit out the first few values of n^2, the actual value, and n^3 to illustrate the difference:
#include <stdio.h>
int count(int n){
int ctr = 0;
for (int i = 0; i < 2*n; i++){
if (i == n)
for (int j = 0; j < i; j++)
for (int k = 0; k < i; k++)
ctr++;
else
for (int j = 0; j < i; j++)
ctr++;
}
return ctr;
}
int main(){
for (int i = 1; i <= 20; i++){
printf(
"%d\t%d\t%d\t%d\n",
i*i, count(i), 3*i*i - 2*i, i*i*i
);
}
}
Try it online!
(You can paste it into Excel to plot the values.)
The First loop is repeated 2*n times:
for (int i = 0; i < 2*n; i++)
{
// some code
}
This part Just occur once, when i == n and time complexity is : O(n^2):
if (i == n)
{
for (int j = 0; j < i; j++)
for (int k = 0; k < i; k++)
O(1)
}
And this part is depends on i.
else
{
for (int j = 0; j < i; j++)
O(1)
}
Consider i when:
i = 0 the loop is repeated 0 times
i = 1 the loop is repeated 1 times
i = 2 the loop is repeated 2 times
.
.
i = n the loop is repeated n times. (n here is 2*n)
So the loop repeated (n*(n+1)) / 2 times But when i == n else part is not working so (n*(n+1)) / 2 - n and time complexity is O(n^2).
Now we sum all of these parts: O(n^2) (first part) + O(n^2) (second part) because the first part occurs once so it's not O(n^3). Time complaxity is: O(n^2).
Based on #Gassa answer lets sum up all:
O(n^3) + O((2n)^2) = O(n^3) + O(4n^2) = O(n^3) + 4*O(n^2) = O(n^3)
Big O notation allows us throw out 4*O(n^2) because O(n^3) "eats" it

some examples of algorithm complexity of nested loops?

I have seen that in some cases the complexity of nested loops is O(n^2), but I was wondering in which cases we can have the following complexities of nested loops:
O(n)
O(log n) I have seen somewhere a case like this, but I do not recall the exact example.
I mean is there any kind of formulae or trick to calculate the complexity of nested loops? Sometimes when I apply summation formulas I do not get the right answer.
Some examples would be great, thanks.
Here is an example for you where the time complexity is O(n), but you have a double loop:
int cnt = 0;
for (int i = N; i > 0; i /= 2) {
for (int j = 0; j < i; j++) {
cnt += 1;
}
}
You can prove the complexity in the following way:
The first iteration, the j loop runs N times. The second iteration, the j loop runs N / 2 times. i-th iteration, the j loop runs N / 2^i times.
So in total: N * ( 1 + 1/2 + 1/4 + 1/8 + … ) < 2 * N = O(N)
It would be tempting to say that something like this runs in O(log(n)):
int cnt = 0;
for (int i = 1; i < N; i *= 2) {
for (int j = 1; j < i; j*= 2) {
cnt += 1;
}
}
But I believe that this runs in O(log^2(N)) which is polylogarithmic

Complexity of double-nested loop

What is the order of growth of the worst case running time of the following code fragment as a function of N?
int cnt = 0;
for (int i = 1; i <= N; i = i*4)
for (int j = 0; j < i; j++)
{ cnt++; }
I now for example that first loop execute ~log(4, N) times and the second loop execute ~N times. But how to combine this knowlege to find the answer?
What is the general way to find that kind of complexity?
Maybe, we need to know how much time the body of the inner loop is executed?
For example 1 + 4 + 16 + 64 + ... + N
Geometric progression = (x^n - 1)/(x-1) where n=Log(4,N), so the result is
(x^log(x, N) - 1)/ (x-1) = (4N - 1)/3
Let's N belong to the interval [4^k; 4^(k+1)), then we have got sum:
sum 4^i, i=0..k = (4^(k+1)-1)/3 = O(n)
I was late some minutes and minus...

Troubles with Big O Estimate

I'm asked to give a big-O estimates for some pieces of code but I'm having a little bit of trouble
int sum = 0;
for (int i = 0; i < n; i = i + 2) {
for (int j = 0; j < 10; j + +)
sum = sum + i + j;
I'm thinking that the worst case would be O(n/2) because the outer for loop is from i to array length n. However, I'm not sure if the inner loop affects the Big O.
int sum = 0;
for (int i = n; i > n/2; i − −) {
for (int j = 0; j < n; j + +)
sum = sum + i + j;
For this one, I'm thinking it would be O(n^2/2) because the inner loop is from j to n and the outer loop is from n to n/2 which gives me n*(n/2)
int sum = 0;
for (int i = n; i > n − 2; i − −) {
for (int j = 0; j < n; j+ = 5)
sum = sum + i + j;
I'm pretty lost on this one. My guess is O(n^2-2/5)
Your running times for the first two examples are correct.
For the first example, the inner loop of course always executes 10 times. So we can say the total running time is O(10n/2).
For the last example, the outer loop only executes twice, and the inner loop n/5 times, so the total running time is O(2n/5).
Note that, because of the way big-O complexity is defined, constant factors and asymptotically smaller terms are negligible, so your complexities can / should be simplified to:
O(n)
O(n2)
O(n)
If you were to take into account constant factors (using something other than big-O notation of course - perhaps ~-notation), you may have to be explicit about what constitutes a unit of work - perhaps sum = sum + i + j constitutes 2 units of work instead of just 1, since there are 2 addition operations.
You're NOT running nested loops:
for (int i = 0; i < n; i = i + 2);
^----
That semicolon is TERMINATING the loop definition, so the i loop is just counting from 0 -> n, in steps of 2, without doing anything. The j loop is completely independent of the i loop - both are simply dependent on n for their execution time.
For the above algorithms worst case/best case are the same.
In case of Big O notation, lower order terms and coefficient of highest order term can be ignored as Big O is used for describing asymptotic upper bound.
int sum = 0;
for (int i = 0; i < n; i = i + 2) {
for (int j = 0; j < 10; j + +)
sum = sum + i + j;
Total number of outer loop iteration =n/2.for each iteration of outer loop, number of inner loop iterations=10.so total number of inner loop iterations=10*n/2=5n. so clearly it is O(n).
Now think about rest two programs and determine time complexities on your own.

Running time of for loop - part #2

This would be part # 2 of my question about analysis of for loop running time
http://faculty.simpson.edu/lydia.sinapova/www/cmsc250/LN250_Weiss/L03-BigOhSolutions.htm#PR4 contains solutions, and I have question about two particular "for" loops
Could someone explain to me how to figure out running time for both of them. Thanks !
1.
sum = 0;
for( i = 0; i < n; i++)
for( j = 0; j < i*i; j++)
for( k = 0; k < j; k++)
sum++;
2.
sum = 0;
for( i = 0; i < n; i++)
for( j = 0; j < i*i; j++)
if (j % i ==0)
for( k = 0; k < j; k++)
sum++;
The first snippet is O(n^5).
Top Loop = 0 - O(n) = O(n) iterations
Middle Loop = 0 - O(n^2) = O(n^2) iterations
Inner Loop = 0 - O(n^2) = O(n^2) iterations
Total = O(n^5)
Here's the closed-form solution of the first snippet: (computed via Mathematica)
sum = -(1/10)*n + (1/4)*n^2 - (1/4)*n^4 + (1/10)*n^5
This is a 5th order polynomial, therefore it is: O(n^5)
The second snippet appears to be O(n^4).
Top Loop = 0 - O(n) = O(n) iterations
Middle Loop = 0 - O(n^2) = O(n^2) iterations
If statement enters: O(1 / n) times
Inner Loop = 0 - O(n^2) = O(n^2) iterations
Total = O(n^4)
Here's the closed-form solution of the second snippet: (computed via Mathematica)
sum = -(1/12)*n + (3/8)*n^2 - (5/12)*n^3 + (1/8)*n^4
This is a 4th order polynomial, therefore it is: O(n^4)
Further explanation of the effect of the if-statement:
The middle loop iterates from 0 to i*i. The if-statement checks if j is divisible by i. But that is only possible when j is a multiple of i.
How many times is j a multiple of i if 0 <= j < i*i? Exactly i times. Therefore only 1/i of the iterations of the middle loop will fall through to the inner-most loop.
The relationship of 'n' as well as the other variables in the second for loop statement ( ..., x<=n, ...) would really define how fast it would be. Try to visualize a for loop as a racem and the second statement says how many laps you would make. So for example, variable 'n' = 1000, then you would have to run the same lap for 1000 times, truly time wasting. Hope that got you a better view on things.

Resources