Given:
for (int i = 1; i <= n; i *= 2) {
for (int j = 0; j < i; j++) {
// Statement(s) that take(s) constant time
}
}
Running Time Complexity = O(n)
Explanation was:
I understand that the outer loop is log(n) and the inner loop is O(n).
But why time complexity is not O(n log n)? why it's O(n + log n)?
The partial sums calculation is correct. However, the total execution time explanation is unclear, although the result is indeed O(n).
The actual time is consumed by executing the inner statements and the number of these statements executions is defined by the loops. The method of partial sums already takes into consideration both loops (1 time unit for i=1, 2 time units for i=2, 4 time units for i=4, etc.) and in total the internal statements are executed O(n) times as shown by the partial sums calculation. Therefore, total time is O(n)*O(constant for the internal statements)=O(n). I don't see any reason to include the O(logn) in calculations here.
The outer loop runs log(n) times. But, in each time, the inner loop does not run in O(n). As shown in the explanation, if i = 1, the inner loop runs 1 time. If i = 2, the inner loop runs 2 times. If i = 4, the inner loop runs 4 times. Hence, the total number of running time is T(n) = 1 + 2 + 4 + ... + 2^log(n).
Now, simplify the T(n):
T(n) = 2^(log(n)+1) - 1 = 2^log(n) * 2 - 1 = 2n - 1 = Theta(n)
Related
This question already has answers here:
How can I find the time complexity of an algorithm?
(10 answers)
Closed 5 years ago.
int a = 0;
for (i = 0; i < N; i++) {
for (j = N; j > i; j--) {
a = a + i + j;
}
}
Why does this code has time complexity of O(NN) ? Are all nested loops with n,m,q,.... bounds have a complexity of O(nm*q....) even though few iterations of the loops will not happen?
The reason for this is that constant factors are ignored in Big-O notation.
Your outer loop runs N times, while the inner loop runs on average N/2 times for each of the outer iterations.
This gives O(N * 1/2 * N) executions of the statements within the inner loop. Since we ignore constant factors, we end up with O(N * N) which is O(N^2).
The reason for omitting constants is simple: Big-O notation is about what happens when N is big. If you look at it this way - O((N^2)/2) - you see that increasing N has much more influence in the term, than whether or not we omit the division by two.
I wouldn't say all nested loops have that complexity, but this one certainly does. The number of iterations through the inner loop is something like this (there might be an off-by-one error):
i = 0: N times
i = 1: N - 1 times
i = 2: N - 2 times
. . .
i = N-1: 1 times
i = N: 0 times
Let's count the total number of times the innermost code is called. In this case, you can figure this out arithmetically. The innermost code is executed about N * (N + 1) / 2 times (this is the sum of elements from 1 to N). This simplifies to 0.5 * N^2 + 0.5 * N and that has a complexity of N^2.
Need help figuring this hw example out guys.
the example states 0(n) running time.
I see that the outter loop is O(logn)
I cant figure out how to describe the inner loop in relation to n.
Help is much appreciated.
for (int i = 1; i <= N; i = i*2) // log n
for (int j = 0; j < i; j++) // less than n i don't know how to describe the growth
sum++;
Answer:: 0(n)
It might be easiest to analyze the runtime of the piece of code in the aggregate. On the first iteration of the outer loop, the inner loop will run 1 time. On the second iteration of the outer loop, the inner loop will run 2 times. On the third iteration of the outer loop, the inner loop will run 4 times. More generally, on the kth iteration of the outer loop, the inner loop will run 2k times. The outer loop stops as soon as i becomes greater than N, which happens after log2 N iterations.
If we sum up the total work done, we'll see that it's
1 + 2 + 4 + 8 + ... + 2log2 N = 2log2 N + 1 - 1 = 2n - 1
(This uses the fact that 1 + 2 + 4 + 8 + ... + 2k = 2k+1 - 1). Therefore, the total work done for the entire piece of code (that is, including both loops) is O(n).
The inner loop is O(n) time because it will go through each element once. If i is 1000, 1000 times, if i is 10000, 10000 times, etc.
This question is based off of this resource http://algs4.cs.princeton.edu/14analysis.
Can someone break down why Exercise 6 letter b is linear? The outer loop seems to be increasing i by a factor of 2 each time, so I would assume it was logarithmic...
From the link:
int sum = 0;
for (int n = N; n > 0; n /= 2)
for (int i = 0; i < n; i++)
sum++;
This is a geometric series.
The inner loops runs i iterations per iteration of the outer loop, and the outer loop decreases by half each time.
So, summing it up gives you:
n + n/2 + n/4 + ... + 1
This is geometric series, with r=1/2 and a=n - that converges to a/(1-r)=n/(1/2)=2n, so:
T(n) <= 2n
And since 2n is in O(n) - the algorithm runs in linear time.
This is a perfect example to see that complexity is NOT achieved by multiplying the complexity of each nested loop (that would have got you O(nlogn)), but by actually analyzing how many iterations are needed.
Yes its simple
See the value of n is decreasing by half each time and I runs n times.
So for the first time i goes from 1 to n
next time 0 to n/2
and hence 0 to n/k on kth term.
Now total time inner loop would run = Log(n)
So its a GP the number of times i is running.
with terms
n,n/2,n/4,n/8....0
so we can find the sum of the GP
2^(long(n) +1)-1 / (2-1)
2^(long(n)+1) = n
hence n-1/(1) = >O(n)
I have a contradiction by analyzing the running time of a program. For example, consider the following piece of code:
for(int i=0;i<n;i++)
{
for(int j=0;j<n;j++)
{
.....
}
}
Here, the computational complexity of 1st for loop is O(n2), and for the second loop it is O(n). However, the second loop is executed n2 times whereas first loop is executed n times. For example, if we put a cout statement inside the inner loop, it outputs n2 times, but if we put a cout somewhere inside the 1st loop but outside the inner loop, it outputs n times. So why do we say the complexity of inner loop is O(n), but for outer loop it is O(n2). We say the complexity of outer loop is O(n2) but it executes n times, why is this the case? Am i doing something wrong? Thanks.
The inner loop executes n times, which takes O(n). The outer loop executes the inner loop n times, but you have to account for the cost of the inner loop for each of those n outer loop executions. This makes it O(n * O(n)) = O(n^2).
Outer loop will run for n times and inner loop will run for n times for every iteranion of outer loop making inner loop to run for n^2 times. Thus statements in inner loop will get executed for n^2 times.
Let say, n = 7 ; then
for(int i=0 ; i< 7 ; i++) //time complexity is : (1) + (7+1) + (7)
{
for(int j=0;j<7;j++) // time complexity is: (1) + [(7+1)+(7+1)+(7+1)+(7+1)+(7+1)+(7+1)+(7+1)] + (7)
{
.....
}
}
Total time complexity = [2+2(7)] + [1 + (7+1)^2 + 7]
Now replace 7 = n ; We will get
Total time complexity = [2+2n] + [1 + (n+1)^2 + n] = 2 + 2n + 1 + (n^2 + 2n + 1) + n = n^2 + 5n + 4
Here the dominant term is n^2. So the worst case will be O(n^2)
What is the time complexity of this algorithm:
sum = 0
i = 1
while (i < n) {
for j = 1 to i {
sum = sum + 1
}
i = i*2;
}
return sum
I know that the while loop is O(logn), but what is the complexity of the for loop? Is it O(n) or O(logn)?
One way to analyze this would be to count up the number of iterations of the inner loop. On the first iteration, the loop runs one time. On the second iteration, it runs two times. It runs four times on the third iteration, eight times on the fourth iteration, and more generally 2k times on the kth iteration. This means that the number of iterations of the inner loop is given by
1 + 2 + 4 + 8 + ... + 2r = 2r + 1 - 1
Where r is the number of times that the inner loop runs. As you noted, r is roughly log n, meaning that this summation works out to (approximately)
2log n + 1 - 1 = 2(2log n) - 1 = 2n - 1
Consequently, the total work done by the inner loop across all iterations in O(n). Since the program does a total of O(log n) work running the outer loop, the total runtime of this algorithm is O(n + log n) = O(n). Note that we don't multiply these terms together, since the O(log n) term is the total amount of work done purely in the maintenance of the outer loops and the O(n) term is total amount of work done purely by the inner loop.
Hope this helps!