Consider a nested loop. The outer loop starts at i=0 and iterates N times, and the inner loop starts at j=i+1 and iterates up to j=N. So the inner loop will roughly do n/2 steps. At the end, however, the runtime is considered O(N2) Why is the inner loop considered O(N) and not O(N/2), since we have other codes that have O(log n) runtimes?
It seems that you're mixing two different cases (division in the final formula - N**2/C - where C can be ignored: O(N**2/C) == O(N**2); and division in the loop: for (int j = N; j >= 1; j /= C) where C leads to logarithm):
for (int i = 1; i <= N; ++i)
for (int j = i + 1; j <= N; ++j)
SomeOperation(i, j);
Let's count the number of SomeOperation(i, j) to be performed:
i j
-------------------
1 N - 1
2 N - 2
3 N - 3
..
N 0
So we have
(N - 1) + (N - 2) + ... + 2 + 1 + 0 ==
N * (N - 1) / 2 ==
N**2 / 2 - N / 2 ==
O(N**2 / 2 - N / 2) == O(N**2 / 2) == O(N**2)
On the contrary (please, notice j /= 2 instead of ++j) which means far fewer inner loops
for (int i = 1; i <= N; ++i)
for (int j = N; j >= 1; j /= 2)
SomeOperation(i, j);
i j
-------------------
1 log(N)
2 log(N)
3 log(N)
..
N log(N)
And here we have
log(N) + log(N) + ... + log(N) ==
N * log(N) ==
O(N * log(N))
Big-O notation represents the complexity of time it takes a segment of code to execute, in proportion to some metric. Usually, the symbols used in braces represent quantities like input size, container size, etc.
In an intuitive sense, O(N) refers to the number of times a code runs in proportion to the symbols included inside the braces, as opposed to the exact number of times it runs. It may run K = N/2 times in reality, but the point Big-O notation tries to underscore is the fact that, the value of K is estimated by how large N is, and it is directly proportional to K.
To further clarify, notice that for a large enough N, the division by 2 does not really matter, as it is simply a constant factor. The notion that for a large enough N, constants are negligible is critical to understand to get a good grasp of various complexity notations, including Big-O.
Related
for (i = 1; i <= n; i++)
{
for (j = n; j >= i; j--)
}
I'm struggling with this algorithm. I can't even know what time complexity of this algorithm is? I've checked using online software it shows me only o(n).
First of all, the algorithm should be something like this:
for (i = 1; i <= n; i++)
for (j = n; j >= i; j--)
DoSomeWork(i, j); // <- Payload which is O(1)
To find out the time complexity, let's count how many times DoSomeWork will be executed:
i : j : executed, times
----------------------------
1 : 1..n : n
2 : 2..n : n - 1
3 : 3..n : n - 2
.. : ... ...
n : n..n : 1
So far so good, DoSomeWork will be executed
n + n - 1 + n - 2 + ... + 2 + 1 = (n + 1) * n / 2
times; time complexity for your case is
O((n + 1) * n / 2) = O((n + 1) * n) = O(n * n) + O(n) = O(n * n)
Nested loops are not necessary have quadratic time complexity, e.g.
for (i = 1; i <= n; i++)
for (j = n; j >= i; j /= 2) // note j /= 2, instead of j--
DoSomeWork(i, j);
has O(n * log(n)) time complexity
Think about this a little. How many times do we enter into the outer loop? It's n times, as you surely already know, since we have step1, ..., stepn, in total n steps.
So, we have
n * average(inner)
In the first step, the inner loop has n steps. Then it has n - 1 steps and so on, on the final step we have 1 step in the inner loop.
so, the inner loop has:
n, n-1, ..., 1 steps in the respective iterations.
Since addition is both commutative and associative, we have:
(n + 1) + (n - 1 + 2) + ...
that is (n+1)/2 on average.
Since we worry about the scenario when n -> infinity, adding 1 to it or dividing it by 2 is less of a concern, so we roughly have n * n steps, hence it has a complexity of O(n * n), don't listen to the tool that says otherwise, unless you have some extra information about your algorithm to share with us.
How do you calculate the time complexity or big-O of this algorithm, where it is not clear how many iterations the loop iterates?
cin >> n;
int i = 0;
for (int j=1; i <= n; j++) {
i += j;
}
Appreciate that the series in j looks like this:
1 + 2 + 3 + 4 + ... + n (not the n in your question)
The sum of this series is given by the Gaussian formula:
n * (n + 1) / 2
This means that the sum varies as n^2, where n is the number of terms or steps in the loop. Therefore, the loop should vary as:
O(sqrt(n))
Where now n here is the n from your loop code, i.e. the upper bound of the loop.
I have a nested for loop that I am trying to analyze the efficiency of. The loop looks like this:
int n = 1000;
for (int i = 0; i < n; i++) {
for (int j = 0; j < i; j++) {
System.out.print("*");
}
}
I don't believe that this algorithm is O(n^2) because the inner loop does not run n times, it only runs i times. However, it certainly is not O(n). So I hypothesize that it must be between the two efficiencies, which gives O(nlog(n)). Is this accurate or is it really a O(n^2) algorithm and I'm misunderstanding the effect the inner loop has on the efficiency?
Your algorithm will run a triangular number of times:
n * (n + 1) / 2
In the above case, n = 999 because the first j loop doesn't run:
(999 * 1000) / 2 = 499500
It is lower than n**2, but it still is O(n**2), because n * (n + 1) / 2 is n**2 / 2 + n / 2. When n is large, you can ignore n / 2 compared to n**2 / 2, and you can also ignore the constant 1 / 2 factor.
I kind of get your doubts, but try to think in this way: what value will i have in the worst case scenario? Answer is n-1, right? So, as the complexity is evaluated by considering the worst case scenario it turns out that it is O(n^2) as n * (n-1) ~ n^2.
The number of iterations is sum from i=0 to n-1 (sum from j=0 to i-1 (1)). The inner sum is obviously equal to i. sum from i=0 to n-1 (i) = n * (n-1) / 2 = O(n^2) is well known.
How can I analyze this code fragment to conclude that it is O(N)?
int sum = 0;
for (int i = 1; i < N; i *= 2)
for (int j = 0; j < i; j++)
sum++;
The value of i in the outer loop increases exponentially, you can think of it as increasing a binary digit each time. The number of digits it takes to represent N is log(N). So the outer loop will execute log(N) times The inner loop will execute
2^0 + 2^1 + 2^2 + ... + 2^log(N)
The formula for this geometric series is (updated from Niklas B's comment)
1(1 - 2^log(N))/(1 - 2)
= 2^(log(N) + 1) - 1
~= 2N
Over all the algorithm will be O(2N + log(N))
but for big-O notation, the 2N component will overwhelm log(N) so overall the complexity is O(N)
In the book Programming Interviews Exposed it says that the complexity of the program below is O(N), but I don't understand how this is possible. Can someone explain why this is?
int var = 2;
for (int i = 0; i < N; i++) {
for (int j = i+1; j < N; j *= 2) {
var += var;
}
}
You need a bit of math to see that. The inner loop iterates Θ(1 + log [N/(i+1)]) times (the 1 + is necessary since for i >= N/2, [N/(i+1)] = 1 and the logarithm is 0, yet the loop iterates once). j takes the values (i+1)*2^k until it is at least as large as N, and
(i+1)*2^k >= N <=> 2^k >= N/(i+1) <=> k >= log_2 (N/(i+1))
using mathematical division. So the update j *= 2 is called ceiling(log_2 (N/(i+1))) times and the condition is checked 1 + ceiling(log_2 (N/(i+1))) times. Thus we can write the total work
N-1 N
∑ (1 + log (N/(i+1)) = N + N*log N - ∑ log j
i=0 j=1
= N + N*log N - log N!
Now, Stirling's formula tells us
log N! = N*log N - N + O(log N)
so we find the total work done is indeed O(N).
Outer loop runs n times. Now it all depends on the inner loop.
The inner loop now is the tricky one.
Lets follow:
i=0 --> j=1 ---> log(n) iterations
...
...
i=(n/2)-1 --> j=n/2 ---> 1 iteration.
i=(n/2) --> j=(n/2)+1 --->1 iteration.
i > (n/2) ---> 1 iteration
(n/2)-1 >= i > (n/4) ---> 2 iterations
(n/4) >= i > (n/8) ---> 3 iterations
(n/8) >= i > (n/16) ---> 4 iterations
(n/16) >= i > (n/32) ---> 5 iterations
(n/2)*1 + (n/4)*2 + (n/8)*3 + (n/16)*4 + ... + [n/(2^i)]*i
N-1
n*∑ [i/(2^i)] =< 2*n
i=0
--> O(n)
#Daniel Fischer's answer is correct.
I would like to add the fact that this algorithm's exact running time is as follows:
Which means: