I am studying algorithms complexity, and I have a question about the difference between the following two algorithms:
Algorithm #1:
sum = 0
i = 1
while (i < n) {
for j = 1 to i {
sum = sum + 1
}
i = i*2;
}
return sum
Algorithm #2:
sum = 0
i = 1
while (i < n) {
for j = 1 to n {
sum = sum + 1
}
i = i*2;
}
return sum
The only difference is the 'for' loop, but what is the difference between the time complexity of these algorithms ? When do I have to multiply or add the complexity of nested loops?
Let say for simplisity that n is a power of 2 i.e. 2^k. Then it is obvious that outer loop will proceed k times and on each step inner loop will proceed:
1 to 1 i.e. 2^0
1 to 2 i.e. 2^1
1 to 4 i.e. 2^2
...
1 to 2^k
So we just need to find the sum 2^0 + 2^1 + ... + 2^k
It is knows as 2^(k+1) - 1 = 2^k * 2 - 1 = 2*n + 1
So omitting constants we get O(n)
Second one is simple. Outer loop is log(n), inner n so you get O(n*log(n)).
Related
I feel that in worst case also, condition is true only two times when j=i or j=i^2 then loop runs for an extra i + i^2 times.
In worst case, if we take sum of inner 2 loops it will be theta(i^2) + i + i^2 , which is equal to theta(i^2) itself;
Summation of theta(i^2) on outer loop gives theta(n^3).
So, is the answer theta(n^3) ?
I would say that the overall performance is theta(n^4). Here is your pseudo-code, given in text format:
for (i = 1 to n) do
for (j = 1 to i^2) do
if (j % i == 0) then
for (k = 1 to j) do
sum = sum + 1
Appreciate first that the j % i == 0 condition will only be true when j is multiples of n. This would occur in fact only n times, so the final inner for loop would only be hit n times coming from the for loop in j. The final for loop would require n^2 steps for the case where j is near the end of the range. On the other hand, it would only take roughly n steps for the start of the range. So, the overall performance here should be somewhere between O(n^3) and O(n^4), but theta(n^4) should be valid.
For fixed i, the i integers 1 ≤ j ≤ i2 such that j % i = 0 are {i,2i,...,i2}. It follows that the inner loop is executed i times with arguments i * m for 1 ≤ m ≤ i and the guard executed i2 times. Thus, the complexity function T(n) ∈ Θ(n4) is given by:
T(n) = ∑[i=1,n] (∑[j=1,i2] 1 + ∑[m=1,i] ∑[k=1,i*m] 1)
= ∑[i=1,n] ∑[j=1,i2] 1 + ∑[i=1,n] ∑[m=1,i] ∑[k=1,i*m] 1
= n3/3 + n2/2 + n/6 + ∑[i=1,n] ∑[m=1,i] ∑[k=1,i*m] 1
= n3/3 + n2/2 + n/6 + n4/8 + 5n3/12 + 3n2/8 + n/12
= n4/8 + 3n3/4 + 7n2/8 + n/4
For the following block of code, select the most appropriate run-time formula in terms of primitive operations needed for input of size n:
When resolving from inside out, i get:
inner loop = 3n+1
main loop + inner loop = 3 + (3n +1) + logn = 4 + 3n + logn
extra steps + all loops = 4 + n(4 + 3n + logn) = 4 + 4n + 3n2 + logn
This is the code to analyze:
def rate(n):
total= 0
i = 1
while i < n:
j = 0
while j < n:
total= i * j + total
j = j + 1
i = i * 2
return total
and the answer is supposed to be --> f(n) = 4 + 4log(n) + log(n)*(3n)
I am actually coming up with O(NlgN) here for the overall running time. Appreciate that the inner loop in j is not dependent on the outer loop in i. The following should be true:
The outer loop in i is O(lgN), because i is doubling at each iteration, which is exponential behavior.
The inner loop in j is O(N), because j cycles from 0 to N at each iteration, regardless of the value of i.
We may therefore multiply together these complexities to get the overall complexity.
Note that for N of arbitrarily large size, your expression:
4 + 4log(n) + log(n)*(3n)
reduces to NlgN.
def rate(n):
total= 0
i = 1
while i < n: //This outer loop runs O(log(n)) times
j = 0
while j < n: //This inner loop runs O(n) times for each iteration of outer loop
total= i * j + total
j = j + 1
i = i * 2
return total
Hence, the total runtime complexity for your implementation in big-O is = O(log(n)) * O(n) = O(nlog(n)).
This question already has answers here:
Time complexity for dependant nested for loop?
(3 answers)
Closed 5 years ago.
Suppose I have a code with two for-loops:
int sum = 0;
for (int i = 1; i < N; i *= 2)
for(int j = 0; j < i; j++)
sum++;
How would I find the worst-case time complexity of this code? I've looked at many tutorials on finding time complexity and I understood them. But this one just seems a bit different from those in the tutorials.
Lets do simple maths.
Value of i in each iteration goes as 1,2,4,8... (log N + 1) terms. In each iteration, the inner loop goes i times. Adding up the values of i
T(N) = 1 + 2 + ... + (log N + 1) terms i.e. GP with a = 1 and r = 2 and n = (log N + 1)
T(N) = a[(rn-1)/(r-1)]
= 1[(2(logN+1) - 1)/(2-1)]
= 2N - 1
= O(N)
So the complexity is O(N) in all scenarios.
First consider a simpler case when N=2^k+1 for some integer k. In this case the outer loop has k iterations and the overall number of operations is
T(N) = 1 + 2 + 4 + ... + 2^k = 2^(k+1) - 1 = 2N - 3.
So the worst case complexity is at least O(2N - 3) = O(N).
Now suppose 2^k + 1 < N <= 2^(k+1) for some k. Then
T(N) = 1 + 2 + 4 + ... + 2^k = 2^(k+1) - 1 < 2N = O(N)
Therefore T(N) = O(N).
// n > 0
i ← 0
while (i < n)
j ← 0
while (j < power(2,i))
j ← j + 1
done
i ← i + 1
done
Is the overall complexity O(n(log(n)) because the inner while loop has a conditional where 2^i so 2^0 2^1 2^2 ... = 1 2 8 16 32 64 128... etc. So for 2^i < n --> log(n) > i?
And the outer loop looks to be simply O(n).
Multiple both loop complexities for O(n(log(n)), confirm please? Thanks in advance.
It's O(2^n)
For the outer loop, the number of iterations is n, so the inner loop executes for every value of i from 0 to n-1.
The number of iterations of the inner loop each time is 2^i, so the total number of iterations for the entire program is:
2^0 + 2^1 + 2^2 + 2^3 + 2^4 + 2^5 + ... +2^(n-1)
This sum is equal to 2^n - 1. Because 2^n is so large compared to 1, we can drop the 1 in the big-O notation, giving us O(2^n).
Using a formal methodology through Sigma notation:
looks like O(2^n), inner loop is a sum of (2^i) from i=0 to i=n-1, which sums to 2^(i)-1 operations
i ← 0
while (i < n)
j ← 0
while (j < power(2,i))
j ← j + 1
done
i ← i + 1
done
Time Complexity
Time = 2^0 + (2^0 + 2^1) + .. + (2^0 + .. + 2^(n-1))
Time = n * 2^0 + (n-1) * 2^1 + .. + (n-(n-1)) * 2^(n-1)
Time = SUM { (n-k) * 2^k | k = 0..(n-1) }
Time = 2^(n+1) - n - 2
Time ~ O(2^(n+1))
Here's the pseudocode:
Baz(A) {
big = −∞
for i = 1 to length(A)
for j = 1 to length(A) - i + 1
sum = 0
for k = j to j + i - 1
sum = sum + A(k)
if sum > big
big = sum
return big
So line 3 will be O(n) (n being the length of the array, A)
I'm not sure what line 4 would be...I know it decreases by 1 each time it is run, because i will increase.
and I can't get line 6 without getting line 4...
All help is appreciated, thanks in advance.
Let us first understand how first two for loops work
for i = 1 to length(A)
for j = 1 to length(A) - i + 1
First for loop will run from 1 to n(length of Array A) and the second for loop will depend on value of i. SO when i = 1 second for loop will run for n times..When i increments to 2 your second for loop will run for (n-1) time ..so it will go on till 1.
So your second for loop will run as follows:
n + (n - 1) + (n - 2) + (n - 3) + .... + 1 times...
You can use following formula: sum(1 to n) = N * (N + 1) / 2 which gives (N^2 + N)/2 So we have Big oh for these two loops as
O(n^2) (Big Oh of n square )
Now let us consider third loop also...
Your third for loop looks like this
for k = j to j + i - 1
But this actually means,
for k = 0 to i - 1 (you are just shifting the range of values by adding/subtracting j but number of times the loop should run will not change, as difference remains same)
So your third loop will run from 0 to 1(value of i) for first n iterations of second loop then it will run from 0 to 2(value of i) for first (n - 1) iterations of second loop and so on..
So you get:
n + 2(n-1) + 3(n-2) + 4(n-3).....
= n + 2n - 2 + 3n - 6 + 4n - 12 + ....
= n(1 + 2 + 3 + 4....) - (addition of some numbers but this can not be greater than n^2)
= `N(N(N+1)/2)`
= O(N^3)
So your time complexity will be N^3 (Big Oh of n cube)
Hope this helps!
Methodically, you can follow the steps using Sigma Notation:
Baz(A):
big = −∞
for i = 1 to length(A)
for j = 1 to length(A) - i + 1
sum = 0
for k = j to j + i - 1
sum = sum + A(k)
if sum > big
big = sum
return big
For Big-O, you need to look for the worst scenario
Also the easiest way to find the Big-O is to look into most important parts of the algorithm, it can be loops or recursion
So we have this part of the algorithm consisting of loops
for i = 1 to length(A)
for j = 1 to length(A) - i + 1
for k = j to j + i - 1
sum = sum + A(k)
We have,
SUM { SUM { i } for j = 1 to n-i+1 } for i = 1 to n
= 1/6 n (n+1) (n+2)
= (1/6 n^2 + 1/6 n) (n + 2)
= 1/6 n^3 + 2/6 2 n^2 + 1/6 n^2 + 2/6 n
= 1/6 n^3 + 3/6 2 n^2 + 2/6 n
= 1/6 n^3 + 1/2 2 n^2 + 1/3 n
T(n) ~ O(n^3)