Big O Notation for two code fragments - algorithm

I've have two fragments of code and an explanation of what Big O category they fall into. However, try as I might, I can't tally the explanation with what I can come up either by looking at it or doing sample runs.
The first:
long count = 0;
long n = 1000;
long i, j, k;
for(i = 0; i < n; i++)
for (j = 0; j < i * i; j++)
for (k = 0; k < j; k++)
count++;
Sample runs of this consistently give me N^4, but the answer I've been given is "j can be as large as i^2, which could be as large as N^2. k can be as large as j, which is N^2. The running time is thus proportional to N^N^2^N^2, which is O(N^5)"
Second snippet:
long i, j, k;
long n = 1000;
long count = 0;
for (i = 1; i < n; i++)
for (j = 1; j < i * i; j++)
if (j % i == 0)
for (k = 0; k < j; k++)
count++;
For this the notes say "The if statement is executed at most N3 times, by previous arguments, but it is true only O(N^2) times (because it is true exactly i times for each i). Thus the innermost loop is only executed O(N^2) times. Each time through, it takes O(j^2) = O(N^2) time, for a total of O(N^4)"
For this the notes seem to be accurate enough for the N^4 (although I keep getting a result of N^4 / 10). I don't follow the modulo calculation only being true i times for each i however, it seems to enter that loop a lot less.
So the question is can anyone clarify what I'm not understanding?

For the first one:
sum from i = 0 to n-1 of
sum from j = 0 to i*i-1 of
sum from k = 0 to j-1 of
1
We know the sum of 1 m times is equal to m, so we can reduce this to
sum from i = 0 to n-1 of
sum from j = 0 to i*i-1 of
j
We know the sum 1 + 2 + ... + m = m * (m + 1) / 2, so we can reduce further:
sum from i = 1 to n-1 of
(i * i - 1) * i * i / 2 = (1/2) * (i * i * i * i - i * i)
We can make this easier by taking the (1/2) outside the summation and then splitting up the i * i * i * i and i * i terms; however, the summations are still harder and less well-known than for i alone. It does turn out to be Theta(n^5) hence O(n^5); to at least get an intuitive feeling for why this turns out, recognize that the difference f(n+1) - f(n) = (1/2)(n^4-n^2) which is on the order of n^4, so if f were a continuous function and this difference were the derivative, then the order of f would be one higher.
For the second case:
sum from i = 0 to n-1 of
sum from j = 0 to i-1 of
sum from k = 0 to i*j-1
1
Note that j now assumes only i different values for the purposes of the innermost loop: 0, i, 2i, ..., (i-1)i. The inner loop runs for i times as many iterations as the counter value for j. We do this multiplication shifting to avoid introducing a "step" notation so we can use our usual mathematical results.
sum from i = 0 to n-1 of
sum from j = 0 to i-1 of
i*j
sum from i = 0 to n-1 of
i * (1/2) * i * (i - 1) = (1/2)(i * i * i - i)
Again, we can cheat or do the math or we can use our intuition again to (correctly) surmise this turns out to be Theta(n^4).

Related

What is the time complexity of this code that has 3 nested loops but j stops when it's equal to i * 2?

for (int i = 1 to n) {
for (int j = i to n) {
for (int k = j to n) {
sum += a[i] * b[j] * c[k]; //O(1)
}
if (j == 2 * i) {
j = n;
}
}
}
I've spent so many hours tracing the code and tried different n values. I realize that j doesn't run more than [n/2 + 1] times. I made a table that looks like this for n = 8, 9, 12.
(Apologies for the picture, I am new to Stack Overflow)
i and j are values of i and j, whereas the k is the number of times that innermost loop runs
So it turns into something like:
[n+(n-1)] + [(n-1)+(n-2)+(n-3)] + [(n-2)+(n-3)+(n-4)+(n-5)] + ...+ [3+2+1] + [2+1] + 1
I'm not sure how to put this into a arithmetic progression or summation. Please help.
There's n-j work in the inner loop.
For any particular value of j, there's approximately j/2 + 1 possible values of i. For example j can take the value 10 when i is 5,6,7,8,9 or 10.
So the total time complexity is sum((j/2+1) * (n-j), j=1..n), which is n^3/12+n^2/2-7/12 = O(n^3)
I've been sloppy with rounding here, but it doesn't affect the complexity.

is the time complexity of nested for-loops always of O(n^2)?

for (i = 1; i <= n; i++)
{
for (j = n; j >= i; j--)
}
I'm struggling with this algorithm. I can't even know what time complexity of this algorithm is? I've checked using online software it shows me only o(n).
First of all, the algorithm should be something like this:
for (i = 1; i <= n; i++)
for (j = n; j >= i; j--)
DoSomeWork(i, j); // <- Payload which is O(1)
To find out the time complexity, let's count how many times DoSomeWork will be executed:
i : j : executed, times
----------------------------
1 : 1..n : n
2 : 2..n : n - 1
3 : 3..n : n - 2
.. : ... ...
n : n..n : 1
So far so good, DoSomeWork will be executed
n + n - 1 + n - 2 + ... + 2 + 1 = (n + 1) * n / 2
times; time complexity for your case is
O((n + 1) * n / 2) = O((n + 1) * n) = O(n * n) + O(n) = O(n * n)
Nested loops are not necessary have quadratic time complexity, e.g.
for (i = 1; i <= n; i++)
for (j = n; j >= i; j /= 2) // note j /= 2, instead of j--
DoSomeWork(i, j);
has O(n * log(n)) time complexity
Think about this a little. How many times do we enter into the outer loop? It's n times, as you surely already know, since we have step1, ..., stepn, in total n steps.
So, we have
n * average(inner)
In the first step, the inner loop has n steps. Then it has n - 1 steps and so on, on the final step we have 1 step in the inner loop.
so, the inner loop has:
n, n-1, ..., 1 steps in the respective iterations.
Since addition is both commutative and associative, we have:
(n + 1) + (n - 1 + 2) + ...
that is (n+1)/2 on average.
Since we worry about the scenario when n -> infinity, adding 1 to it or dividing it by 2 is less of a concern, so we roughly have n * n steps, hence it has a complexity of O(n * n), don't listen to the tool that says otherwise, unless you have some extra information about your algorithm to share with us.

Why codes that does N/2 steps are considered O(N)?

Consider a nested loop. The outer loop starts at i=0 and iterates N times, and the inner loop starts at j=i+1 and iterates up to j=N. So the inner loop will roughly do n/2 steps. At the end, however, the runtime is considered O(N2) Why is the inner loop considered O(N) and not O(N/2), since we have other codes that have O(log n) runtimes?
It seems that you're mixing two different cases (division in the final formula - N**2/C - where C can be ignored: O(N**2/C) == O(N**2); and division in the loop: for (int j = N; j >= 1; j /= C) where C leads to logarithm):
for (int i = 1; i <= N; ++i)
for (int j = i + 1; j <= N; ++j)
SomeOperation(i, j);
Let's count the number of SomeOperation(i, j) to be performed:
i j
-------------------
1 N - 1
2 N - 2
3 N - 3
..
N 0
So we have
(N - 1) + (N - 2) + ... + 2 + 1 + 0 ==
N * (N - 1) / 2 ==
N**2 / 2 - N / 2 ==
O(N**2 / 2 - N / 2) == O(N**2 / 2) == O(N**2)
On the contrary (please, notice j /= 2 instead of ++j) which means far fewer inner loops
for (int i = 1; i <= N; ++i)
for (int j = N; j >= 1; j /= 2)
SomeOperation(i, j);
i j
-------------------
1 log(N)
2 log(N)
3 log(N)
..
N log(N)
And here we have
log(N) + log(N) + ... + log(N) ==
N * log(N) ==
O(N * log(N))
Big-O notation represents the complexity of time it takes a segment of code to execute, in proportion to some metric. Usually, the symbols used in braces represent quantities like input size, container size, etc.
In an intuitive sense, O(N) refers to the number of times a code runs in proportion to the symbols included inside the braces, as opposed to the exact number of times it runs. It may run K = N/2 times in reality, but the point Big-O notation tries to underscore is the fact that, the value of K is estimated by how large N is, and it is directly proportional to K.
To further clarify, notice that for a large enough N, the division by 2 does not really matter, as it is simply a constant factor. The notion that for a large enough N, constants are negligible is critical to understand to get a good grasp of various complexity notations, including Big-O.

What is the time complexity for this Algorithm?

for i = 1 to n do
j = 2
while j < i do
j = j * j
I think it's time complexity is : log(n!) = n * log(n).
but solution said that it is : n * loglog(n) and I didn't understand why?
In the explanation below, I assume that all arithmetic and comparison operations are O(1).
for i = 1 to n do
The below is repeated N times, which makes the n * part in the solution.
j = 2
while j < i do
j = j * j
The above calculates the first number of the following sequence that's >= i :
2 = 2^(2^0)
4 = 2^(2^1)
16 = 2^(2^2)
256 = 2^(2^3)
65536 = 2^(2^4)
...
So the only thing you need to do is to find the relation between i and 2^(2^i). And log(log(2^(2^i))) = log(2^i) = i.
Let's break it down and work from the inside out.
Imagine the following:
j = 2
while j < n do
j = j * 2
j goes 2, 4, 8, 16..., so if n doubles in size, it only takes roughly one more iteration for j to surpass it. That's basically the definition of logarithmic.
The inner loop in your case is a bit different:
j = 2
while j < n do
j = j * j
Now j goes 2, 4, 16, 256, 65536... and surpasses n even more easily. In the first case, j was growing exponentially per iteration, now it's growing doubly exponentially. But we're interested in the inverse--j surpasses n in log(log(n)) steps.
Then the outer loop just means you do that n times.

Asymptotic analysis of three interdependent nested for loops

The code fragment I am to analyse is below:
int sum = 0;
for (int i = 0; i < n; i++)
for (int j = 0; j < i * i; j++)
for (int k = 0; k < j; k++)
sum++;
I know that the first loop is O(n) but that's about as far as I've gotten. I think that the second loop may be O(n^2) but the more I think about it the less sense it makes. Any guidance would be much appreciated.
The first loop executes n times. Each time, the value i grows. For each such i, the second loop executes i*i times. That means the second loop executes 1*1 + 2*2 + 3*3 + ... + n*n times.
This is a summation of squares, and the formula for this is well-known. Hence we have the second loop executing (n(1 + n)(1 + 2 n))/6 times.
Thus, we know that in total there will be (n(1 + n)(1 + 2 n))/6 values of j, and that for each of these the third loop will execute 1 + 2 + ... + j = j(j+1)/2 times. Actually calculating how many times the third loop executes in total would be very difficult. Luckily, all you really need is a least upper bound for the order of the function.
You know that for each of the (n(1 + n)(1 + 2 n))/6 values of j, the third loop will execute less than n(n+1)/2 times. Therefore you can say that the operation sum++ will execute less than [(n(1 + n)(1 + 2 n))/6] * [n(n+1)/2] times. After some quick mental math, that amounts to a polynomial of maximal degree 5, therefore your program is O(n^5).
int sum = 0;
for (int i = 0; i < n; i++) // Let's call this N
for (int j = 0; j < i * i; j++) // Let's call this M
for (int k = 0; k < j; k++) // Let's call this K
sum++;
N is the number of steps of the entire program, M is the number of steps the two inner loops do and lastly K is the number of steps the last loop does.
It is easy to see that K = j, it takes j steps to do.
Then M = Sum(j=0,i^2,K) = Sum(j=0, i^2, j)
(First param is the iterator, second is the upper bound and last param is what we are adding)
This is actually now a sum of n numbers to i*i. Which means we can apply the formula ((n+1)*n)/2
M = Sum(j=0,i^2,j) = ((i^2+1)*(i^2))/2 = (i^4 + i^2)/2
N = Sum(i=0, n, M) = 1/2 * ( Sum(i=0, n, (i^4)) + Sum(i=0, n, (i^2)) )
These are both well known formulas and after a little playing you get:
N = (n^5)/10 + (n^4)/4 + (n^3)/3 + (n^2)/4 + n/15
This should be the exact number of steps the loop takes, but if you are interested in the O notation you can note that n^5 is the strongest growing part so the solution is O(n^5)
If you proceed methodically using Sigma Notation, you'll end up with the following result:
Try to count how many times the inner loop is executed:
The middle loop runs
0*0 times when i == 0
1*1 times when i == 1
2*2 times when i == 2
...
n*n = n^2 times when i == n.
So it is O(n^2).

Resources