If you had code that looks like this, what would the big O be? I'm uncertain as to how if statements affect big O.
n = some arbitrary number
for(i = 0; i < n; i++)
for(j = 0; j < n; j++)
if(i <= j)
for(k = i; k <= j; k++)
//do some simple operation
y = x+1
else
//do some simple operation
y = y+1
I'm not considering compiler optimizations. I know this is somewhere between O(n^2) and O(n^3) but am not sure as the if statement does not always execute the inner most loop.
O(N * N * N) which we can just say is O(N^3)
First Loop happens N times.
Second Loop happens N times.
Those multiply together to get O(N^2)
Out of all the possible N^2 loops, the third loop would run about half the time, which is O(N/2) which is equivalent to O(N).
And that's how you get O(N * N * N) or O(N^3)
In fact you can count (almost exactly) how many operations you do:
i: 0 to n-1 = N operations
x
j: 0 to n-1 = N operations
x
only when i<=j, from i to j, or another task O(1)
The other task gives you NxN operations, then O(NxN)
that is, if you invert
for every j (0 to n-1) : N operations
then for every i from 0 to j, you do an operation from i to j, that is j-i+1
exactly the same as every 0 to j operations. Then you have (j+1)x(j+2)/2 operations.
then finally, you get Sum (j+1)x(j+2)/2 from 0 to N which is
1/2 ((N+1)x(N+2)/2 + (N+1)^3/3+(N+1)^2/2+(N+1)/6) operations, so O(N^3)
Perhaps, I forgot some +/-1
You can analyse your algorithm using Sigma notation:
From this, it's obvious the time complexity will depend on cubic n terms, and hence your algorithm is in O(n^3).
This is O(N^3).
Proof: http://www.wolframalpha.com/input/?i=sum+sum+%28j+-+i+%2B+1%29%2C+j+%3D+i+to+n+-+1%2C+i+%3D+0+to+n+-+1
Last cycle runs (j - i + 1) times.
How to find this sum manually?
This is not a rocket math.
Try to read about https://en.wikipedia.org/wiki/Telescoping_series
Just to save time it's easier to use wolframalpha for that purpose.
Related
I found this example problem on the internet that I just cannot understand how the author came to their conclusion.
sum1 = 0;
for(k=1; k<=n; k*=2) // Do log n times
for (j=1; j<=n; j++) // Do n times
sum1++;`
sum2 = 0;
for (k=1; k<=n; k*=2) // Do log n times
for (j=1; j<=k; j++) // Do k times
sum2++;
I understand that the running time for the first loop is O(n) = nlog(n), but the author claims that for the second loop, the running time is O(n) = n.
Why is that?
The closest I can get to an answer is:
O(n) = k * log(n)
k = 2^i
O(n) = 2^i * log(n) ----> this is where I get stuck
I'm guessing some property of logarithms is used, but I can't figure out which one. Can someone point me in the right direction?
Thanks.
In the second example, the complexity is sum_j 2^j, i.e. the total number of operations in the inner loop.
As 2^j <= n, there are logn terms.
This sum is equal to 2^{jmax+1} - 1, with 2^jmax roughly (<=) equal to n.
Then, effectively, a complexity O(2n) = O(n).
sum2++ is executed 1+2+4+8+...+K times, where K is the largest power of 2 less than or equal to n. That sum is equal to 2K-1.
Since n/2 < K <= n (because K is the largest power of 2 less than or equal to n), the number of iterations is between n-1 and 2n-1. That's Theta(n) if you want to express it in asymptotic notation.
for(i=0; i<n; i++) // time complexity n+1
{
k=1; // time complexity n
while(k<=n) // time complexity n*(n+1)
{
for(j=0; j<k; j++) // time complexity ??
printf("the sum of %d and %d is: %d\n",j,k,j+k); time complexity ??
k++;
}
What is the time complexity of the above code? I stuck in the second (for) and i don't know how to find the time complexity because j is less than k and not less than n.
I always having problems related to time complexity, do you guys got some good article on it?
especially about the step count and loops.
From the question :
because j is less than k and not less than n.
This is just plain wrong, and I guess that's the assumption that got you stuck. We know what values k can take. In your code, it ranges from 1 to n (included). Thus, if j is less than k, it is also less than n.
From the comments :
i know the the only input is n but in the second for depends on k an not in n .
If a variable depends on anything, it's on the input. j depends on k that itself depends on n, which means j depends on n.
However, this is not enough to deduce the complexity. In the end, what you need to know is how many times printf is called.
The outer for loop is executed n times no matter what. We can factor this out.
The number of executions of the inner for loop depends on k, which is modified within the while loop. We know k takes every value from 1 to n exactly once. That means the inner for loop will first be executed once, then twice, then three times and so on, up until n times.
Thus, discarding the outer for loop, printf is called 1+2+3+...+n times. That sum is very well known and easy to calculate : 1+2+3+...+n = n*(n+1)/2 = (n^2 + n)/2.
Finally, the total number of calls to printf is n * (n^2 + n)/2 = n^3/2 + n^2/2 = O(n^3). That's your time complexity.
A final note about this kind of codes. Once you see the same patterns a few times, you quickly start to recognize the kind of complexity involved. Then, when you see that kind of nested loops with dependent variables, you immediately know that the complexity for each loop is linear.
For instance, in the following, f is called n*(n+1)*(n+2)/6 = O(n^3) times.
for (i = 1; i <= n; ++i) {
for (j = 1; j <= i; ++j) {
for (k = 1; k <= j; ++k) {
f();
}
}
}
First, simplify the code to show the main loops. So, we have a structure of:
for(int i = 0; i < n; i++) {
for(int k = 1; k <= n; k++) {
for(int j = 0; j < k; j++) {
}
}
}
The outer-loops run n * n times but there's not much you can do with this information because the complexity of the inner-loop changes based on which iteration of the outer-loop you're on, so it's not as simple as calculating the number of times the outer loops run and multiplying by some other value.
Instead, I would find it easier to start with the inner-loop, and then add the outer-loops from the inner-most to outer-most.
The complexity of the inner-most loop is k.
With the middle loop, it's the sum of k (the complexity above) where k = 1 to n. So 1 + 2 + ... + n = (n^2 + n) / 2.
With the outer loop, it's done n times so another multiplication by n. So n * (n^2 + n) / 2.
After simplifying, we get a total of O(n^3)
The time complexity for the above code is : n x n x n = n^3 + 1+ 1 = n^3 + 2 for the 3 loops plus the two constants. Since n^3 carries the heaviest growing rate the constant values can be ignored, so the Time complexity would be n^3.
Note: Take each loop as (n) and to obtained the total time, multiple the (n) values in each loop.
Hope this will help !
I have an array of lists(i.e. each cell in the array contains a list). The length of the array is n and the sum of all the lengths of all the lists is k
I want to iterate over all the list elements(in the whole array):
for(int i = 0; i < n; ++i) {
for(int j = 0; j < array[i].list.Length(); ++j) {
//do something in O(1)
}
}
NOTE the inner loop runs less than k times per an iteration of the outer loop, but the total iterations it does for all the i is k
QuestionDoes the time complexity of the code is O(n + k)? Or would it be O(n*k)?
Question Does the time complexity of the code is O(n + k)? Or would it be O(n*k)?
Neither.
The complexity is O(n + k). In the case where n <= k, this would equal O(k), but this is not necessarily the case.
n <= k (original answer)
If the sum of all lengths is k, then, if you don't do anything else in the outer loop, the running time would be O(k). n is irrelevant in this case, since there is nothing interesting you're doing n times. Your data just happens to be split up in n chunks.
On average, each list's size would be k/n. That makes the time complexity of the algorithm O(n * k/n) which results in O(k).
n > k
In the case that n is larger than k, n becomes relevant since work has to be done each time, even if it's only checking the Length() of array[i]. Because of that, in this case the complexity is O(n + k).
Update
As Jordi Vermeulen correctly points out in the comments, my original answer that only took into consideration the case where n <= k is incomplete incorrect. The answer has been edited accordingly.
This is O(n + k), which is O(k) when n is O(k). This is, however, not necessarily the case (as suggested in the answer by Bart van Nierop). Consider, for instance, the case where n = k2. The loop is still running k2 times, so you can't say the complexity is O(k), even though in many iterations no work will be done other than increasing the counter.
For every i of the external loop, the inner loop is run array[i].list.Length() which you say is k:
k times + -+
k times + |
... |
... +--- n times
... |
k times -+
So the resulting time is O(n * k)
You should use n*k.
Foreach cols, process each lines.
You've got to do a loop (for or foreach) over each columns (n).
And then inside the n loop, you do another loop (for or foreach) that process each rows (k).
for (int i = 0; i < n; i++) {
for (int j = 0; j < array[i].list.length(); j++) {
// do something with array[i][j]
}
}
O(k).do something part will occur k times.
n is irrelevant in this case.
What would be the order of growth of the code below. My guess was, each loop's growth is linear but the if statement is confusing me. How do I include that with the whole thing. I would very much appreciate an explanatory answer so I can understand the process involved.
int count = 0;
for (int i = 0; i < N; i++)
for (int j = i+1; j < N; j++)
for (int k = j+1; k < N; k++)
if(a[i] + a[j] + a[k] == 0)
count++;
There are two things that can be confusing when trying to determine the code's complexity.
The fact that not all loops start from 0. The second loop starts from i + 1 and the third from j + 1. Does this affect the complexity? It does not. Let's consider only the first two loops. For i = 0, the second runs N - 1 times, for i = 1 it runs N - 2 times, ..., for i = N - 1 it runs 0 times. Add all these up:
0 + 1 + ... + N - 1 = N(N - 1) / 2 = O(N^2).
So not starting from 0 does not affect the complexity (remember that big-oh ignores lower-order terms and constants). Therefore, even under this setting, the entire thing is O(N^3).
The if statement. The if statement is clearly irrelevant here, because it's only part of the last loop and contains no break statement or other code that would affect the loops. It only affects the incrementation of a count, not the execution of any of the loops, so we can safely ignore it. Even if the count isn't incremented (an O(1) operation), the if condition is checked (also an O(1) operation), so the same rough number of operations is performed with and without the if.
Therefore, even with the if statement, the algorithm is still O(N^3).
Order of growth of the code would be O(N^3).
In general k nested loops of length N contribute growth of O(N^k).
Here are two was to find that the time complexity is Theta(N^3) without much calculation.
First, you select i<j<k from the range 0 through N-1. The number of ways to choose 3 objects out of N is the binomial coefficient N choose 3 = N*(N-1)*(N-2)/(3*2*1) ~ (N^3)/6 = O(N^3), and more precisely Theta(N^3).
Second, an upper bound is that you choose i, j, and k from N possibilities, so there are at most N*N*N = N^3 choices. This is O(N^3). You can also find a lower bound of the same type since you can choose i from 0 through N/3-1, j from N/3 through 2N/3-1, and k from 2N/3 through N-1. This gives you at least floor(N/3)^3 choices, which is about N^3/27. Since you have an upper bound and lower bound of the same form, the time complexity is Theta(N^3).
It's diffcult for me to understand logarithmic complexity of algorithm.
For example
for(int j=1; j<=n; j*=2){
...
}
Its complexity is O(log2N)
So what if it is j*=3? The complexity will then be O(log3N)?
You could say yes as long as the loop body is O(1).
However, note that log3N = log2N / log23, so it is also O(log2N), since the constant factor does not matter.
Also note it is apparent from this argument, for any fixed constant k, O(logkN) is also O(log2N), since you could substitute 3 with k.
Basicly, yes.
Let's assume that your for loop looks like this:
for (int j = 1; j < n; j *= a) {...}
where a is some const.
If the for loop executes k times, then in the last iteration, j will be equal to ak. And since N = O(j) and j = O(ak), N = O(ak). It follows that k = O(logaN). Once again, for loop executes k times, so time complexity of this algorithm is O(k) = O(logaN).