I recently started playing with algorithms from this princeton course and I observed the following pattern
O(N)
double max = a[0];
for (int i = 1; i < N; i++)
if (a[i] > max) max = a[i];
O(N^2)
for (int i = 0; i < N; i++)
for (int j = i+1; j < N; j++)
if (a[i] + a[j] == 0)
cnt++;
O(N^3)
for (int i = 0; i < N; i++)
for (int j = i+1; j < N; j++)
for (int k = j+1; k < N; k++)
if (a[i] + a[j] + a[k] == 0)
cnt++;
The common pattern here is that as the nesting in the loop grows the exponent also increases.
Is it safe to assume that if I have 20-for loops my complexity would be 0(N^20)?
PS: Note that 20 is just a random number I picked, and yes if you nest 20 for loops in your code there is clearly something wrong with you.
It depends on what the loops do. For example, if I change the end of the 2nd loop to just do 3 iterations like this:
for (int i = 0; i < N; i++)
for (int j = i; j < i+3; j++)
if (a[i] + a[j] == 0)
cnt++;
we get back to O(N)
The key is whether the number of iterations in the loop is related to N and increases linearly as N does.
Here is another example where the 2nd loop goes to N ^ 2:
for (int i = 0; i < N; i++)
for (int j = i; j < N*N; j++)
if (a[i] + a[j] == 0)
cnt++;
This would be o(N^3)
Yes, if the length of the loop is proportional to N and the loops are nested within each other like you described.
In your specific pattern, yes. But it is not safe to assume that in general. You need to check whether the number of iterations in each loop is O(n) regardless of the state of all the enclosing loops. Only after you have verified that this is the case can you conclude that the complexity is O(nloop-nesting-level).
Yes. Even though you decrease the interval of iteration, Big-o notation works with N increasing towards infinity and as all your loops' lengths grow proportional to N, it is true that such an algorithm would have time complexity O(N^20)
I strongly recommend that you understand why a doubly nested loop with each loop running from 0 to N is O(N^2).Use summations to evaluate the number of steps involved in the for loops, and then dropping constants and lower order terms, you will get the Big-Oh of that algorithm.
Related
I have some trouble finding the time complexity of the code below. I figured that the if statement will run for approximately n times; however, I could not manage to describe it mathematically. Thanks in advance.
int sum = 0;
for (int i = 1; i < n; i++) {
for (int j = 1 ; j < i*i; j++) {
if (j % i == 0) {
for (int k = 0; k < j; k++) {
sum++;
}
}
}
}
Outer loop
Well, it's clear that it's O(n) because i is bounded by n.
Inner loops
If we take a look at the second loop alone, then it looks as follows:
...
for (int j = 1 ; j < i*i; j++){
...
j is bounded by i*i or simply n^2.
However, the innermost loop won't be executed for every j, but only for js that are divisible by i because that's what the constraint j % i == 0 means. Since j ~ i*i, there will be only i cases, when the innermost loop is executed. So, the number of iterations in the inner loops is bounded by i^3 or simply n^3.
Result
Hence, the overall time complexity is O(n4).
I was asked to find the big-O and Big-Omega if I know that function f has O(log(n)), Ω(1) and function g has O(n), Ω((log(n))^2)
for (int i = n; i >= 0; i/=2)
if (f(i) <= g(i))
for (int j = f(i)*f(i); j < n; j++)
f(j);
The big problem that I have is that I don't know how to incorporate the complexity of the funstions in the calculation. I mean I know how to calculate the complexity of loops that looks like this:
for(int i =0 ; i< n*2; i++) {
....
}
or like this
for(int i = 0; i < n; i++) {
for(int j = 0; j < n; j++) {
}
}
Thank you in advance.
This is what I've tried:
for (int i = n; i >= 0; i/=2)// this is aproximatly O(log(n))
if (f(i) <= g(i))// because O(n) < O(log(n)) this line is O(n)
for (int j = f(i)*f(i); j < n; j++)// O(n*log(n))
f(j);// O(log(n))
So by my calculation I get O(log(n)*n *n *log(n)*log(n))=O(n^2*log^3(n))
This is tricky question, because the loop execution depends on values, returned by functions f and g. However remember, that you need to estimate the worst case - so you need to assume two things:
f(i) <= g(i) is always true, so the internal loop always executes
the internal loop starts from 0, because it's a minimal value, which you can get as a result of squaring the f(i) value
So, your piece of code becomes much simpler:
for (int i = n; i >= 0; i/=2)
{
f(i);
g(i);
f(i);
f(i);
for (int j = 0; j < n; j++)
f(j);
}
I think you can take over from here.
I am working out a function:
total = 0;
for (i = 0; i < N; i++){
for (j = 0; j < i*i; j++){
if (j % i == 0){
for (k=0; k < j; k++){
total++;
I the Big O number for this N^4 or N^5 when you break it down. I am not sure how to handle the % sign and the run time of that inner loop.
A roughly equivalent code would be
total=0;
for (i=1; i<=N; i++)
for(j=1; j <= i*i; j+= i)
for (k=1; k <= j; k++)
total++;
by restricting j to those values that are actually divisible by i. Shifting the range of each variable by one avoids the issue of having i = 0.
Rewriting again gives
total=0;
for (i=1; i<=N; i++)
for(j=1; j <= i; j+= 1)
for (k=1; k <= j*j; k++)
total++;
The j loop iterates the same number of times, but instead ranging over the square numbers directly, we simply iterate over the simple integers and shift the multiplication into the k loop. From this, it should be a little easier to prove that total is increment O(N^4) times: the inner k loop runs O(N^2) times, and itself iterates over O(N^2) values.
I have to analyze the Big O complexity for the below code fragments:
a)
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i; j < n; j++)
sum++;
b)
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i + 1; j > i; j--)
// loop 3
for(int k = n; k > j; k--)
sum++;
I'm not sure how to do so any help provided will be greatly appreciated. Thanks.
To analize Big-Oh complexity you have to try to count how many basic operations are made by your code.
In your first loop:
for(int i = 0; i < n; i++)
for(int j = i; j < n; j++)
sum++;
How many times is sum++ called?
The first loop happens n times, and in each one of these, the second loop happens around n times.
This gives you around n * n operations, which is equivalent to a complexity of O(n^2).
I'll let you work out the second one.
The first is straight forward (using the tools of the 2nd code snap, which is a bit trickier) - I'll focus on the 2nd code snap.
Big O notation is giving asymptotic upper bound to the number of ops the algorithm do.
Let's assume each inner iteration do 1 op, and let's neglect the counters and overhead of looping.
Denote T(n) total number of ops done in the program.
It is clear that the program has NO MORE ops then:
// loop 1
for(int i = 0; i < n; i++)
// loop 2
for(int j = i+1; j > i; j--) //note a single op in here, see (1) for details
// loop 3
for(int k = n; k > 0; k--) //we change k > j to j > 0 - for details see (2)
sum++;
(1) Since j is initialized as i+1, and is decreased each iteration, after the first iteration of loop2, you will get j == i, and the condition will yield false - thus - a single iteration is done
(2) The original loop iterates NO MORE then n times (since j >= 0) - thus the "new program" is "not better" then the old one (in terms of upper bounds).
Complexity of the simplified program
The total complexity of the above program is O(n^2), since loop1 and loop3 repeat n times each, and loop2 repeats exactly once.
If we assume single command is done each inner loop - the total number of commands which are done is then n^2.
Conclusion:
Since the new program is doing n^2 "ops" (according to the assumptions) and the original is "not worse then the new" - it is doing T(n) <= n^2 steps.
From definition of big O notation (with c=1, and for every N) - you can conclude the program is O(n^2)
Okay, so I have a midterm later today and one of the items I am reviewing is big-O. Now, I did the homework way back in the day and got 100%....but I can't find it now and I am unsure of what I am doing. Sooo could someone give me an explanation as to what I am doing wrong...and if I am doing it right...well maybe you know why I am doubting myself?
Thanks!
Also, I remember before with my homework I was using summations, and I would work from the inside out. And when I finished each summation I used some "forumla" to calculate the highest n, and then keep that value and move on to the next summation, and so on and so forth until the summations were all completed.
Problem 1.
sum = 0;
for (i = 0; i < n; i++)
sum++;
So, since I forgot the whole summation aspect of this, my gut instinct tell me this is O(N), because the maximum runtimes is N times...since it is just one for loop.
Problem 2.
sum = 0;
for (i = 0; i < n; i++)
for (j = 0; j < n; j++)
sum++;
For this one, I "think" it is O(N^2) for the highest run time, since both loops are dependent on n, and it could maximize at N * N per if loop.
Problem 3.
sum = 0;
for (i = 0; i < n; i++)
for (j = 0; j < n * n; j++)
sum++;
This is where I get stuck...I feel like I actually need to use the summation layout along with the formula for adding them up. The inner most loop can maximize at n*n, so n^2. On top of which, it can maximize at N again for the outermost loop...so I would guess 0(N^3).
Problem 4.
sum = 0;
for (i = 0; i < n; i++)
for (j = 0; j < i; j++)
sum++;
Again, I am more lost on this one. The inner loop can maximize i times...which is dependent on i however, which is dependent on N....So...I see three maximized variables, and I am literally unsure of how to compare them to find a maximized runtime. (I really need to remember that summation setup and formula).
Same goes for the next problems, no clue where to start, and I'd rather not try to because I don't want to get the wrong thinking in my head. I am positive once I see the formula again it will instantly click, because I got it before...I just lost it somehow.
Any help appreciated!
Problem 5:
sum = 0;
for (i = 0; i < n; i++)
for (j = 0; j < i * i; j++)
for (k = 0; k < j; k++)
sum++;
Problem 6:
sum = 0;
for (i = 1; i < n; i++)
for (j = 1; j < i * i; j++)
if (j % i == 0)
for (k = 0; k < j; k++)
sum++;
for problems 4 to 6, I would assume i j and k are all integers, unlike n which is a variable. how I would approach the problems would be:
e.g. problem 4
inner loop - iterations from 0 to (i-1), which gives us i number of iterations.
outer loop - n summations
combined - O(i * n) = O(n) since i is an integer.