I just started my data structures course and I'm having troubles trying to figure out the time complexity in the following code:
{
int j, y;
for(j = n; j >= 1; j--)
{
y = 1;
while (y < j)
y *= 2;
while (y > 2)
y = sqrt(y);
}
The outer 'for' loop is running n times in every run of the code, and the first 'while' loop runs about log2(j) if I'm not mistaken.
I'm not sure about the second 'while' loop and how to determine the overall time complexity of the code.
My initial thoughts were to determine which 'while' loop would "cost" more in each iteration of the 'for' loop, consider only the higher of the two and sum it up but obviously it didn't lead me to an answer.
Would appreciate any help, especially in what is the the process and overall approach in trying to compute the complexity in codes such as this one.
You are right that the first while loop has time complexity O(log(j)). The second while loop repeatedly executes square root on y until it's less than 2.
Since y is approximately j (it's between j and 2j), the question is: how often can you perform a square root on j until you get a number less than or equal to 2? But equivalently you could ask: how often can you square 2 until you get a number larger than or equal to j? Or as an equation:
(((2^2)^...)^2 >= j // k repeated squares
<=> 2^(2^k) >= j
<=> k >= log(log(j))
So the second while loop has time complexity O(log(log(j)). That's negligible compared to O(log(j)). Now since j <= n and the loop is itereated n times, we get the overall complexity O(n log(n)).
Related
What's the big O of this?
for (int i = 1; i < n; i++) {
for (int j = 1; j < (i*i); j++) {
if (j % i == 0) {
for (int k = 0; k < j; k++) {
// Simple computation
}
}
}
}
Can't really figure it out. Inclined to say O(n^4 log(n)) but feel like i'm wrong here.
This is quite a confusing analysis, so let's break it down bit by bit to make sense of the calculations:
The outermost loop runs for n-1 iterations (since 1 ≤ i < n).
The next loop inside it makes (i² - 1) iterations for each index i of the outer loop (since 1 ≤ j < i²).
In total, this means the number of iterations for these two loops is equal to calculating the sum of (i²-1) for each 1 ≤ i < n. This is similar to computing the sum of the first n squares, and is order of magnitude of O(n³).
Note the modulo operator % takes constant time (O(1)) to compute, therefore checking the condition if (j % i == 0) for all iterations of these two loops will not affect the O(n³) runtime.
Now let's talk about the inner loop inside the conditional.
We are interested in seeing how many times (and for which values of j) this if condition evaluates to true, since this would dictate how many iterations the innermost loop will run.
Practically speaking, (j % i) will never equal 0 if j < i, so the second loop could actually be shortened to start from i rather than from 1, however this will not impact the Big-O upper bound of the algorithm.
Notice that for a given number i, (j % i == 0) if and only if i is a divisor of j. Since our range is (1 ≤ j < i²), there will be a total of (i-1) values of j for which this will be true, for any given i. If this is confusing, consider this example:
Let's assume i = 4. Then our index j would iterate through all values 1,..,15=i²,
and (j%i == 0) would be true for j = 4, 8, 12 - exactly (i - 1) values.
The innermost loop would therefore make a total of (12 + 8 + 4 = 24) iterations. Thus for a general index i, we would look for the sum: i + 2i + 3i + ... + (i-1)i to indicate the number of iterations the innermost loop would make.
And this could be generalized by calculating the sum of this arithmetic progression. The first value is i and the last value is (i-1)i, which results in a sum of (i³ - i²)/2 iterations of the k loop for every value of i. In turn, the sum of this for all values of i could be computed by calculating the sum of cubes and the sum of squares - for a total runtime of O(n⁴) iterations of the innermost loop (the k loop) for all values of i.
Thus in total, the runtime of this algorithm would be the total of both runtimes we calculated above. We checked the if statement O(n³) times and the innermost loop ran for O(n⁴), so assuming // Simple computation runs in constant time, our total runtime would come down to:
O(n³) + O(n⁴)*O(1) = O(n⁴)
Let us assume that i = 2.Then j can be [1,2,3].The "k" loop will run for j = 2 only.
Similarly for i=3,j can be[1,2,3,4,5,6,7,8].hence, k can run for j = 3,6. You can see a pattern here that for any value of i, the 'k' loop will run (i-1) times.The length of loops will be [i,2*i,3*i,....i*i].
Hence the time complexity of k loop is
=i+(2*i)+(3*i)+ ..... +(i*i)
=(i^2)(i+1)/2
Hence the final complexity will be
= (n^3)(n+3)/2
foo(int n)
{
int s=0;
for(int i=1;i<=n;i++)
for(int j=1;j<=i*i;j++)
if(j%i==0)
for(k=1;k<=j;k++)
s++;
}
What is the time complexity of the above code?
I am getting it as O(n^5) but it is not correct.
The complexity is O(n^4).
Innermost loop will be executed i times for each i. (i multiples of i within 0..i*i)
It will be like the inner loop will run for
j = 0 1 2...i i+1 ...2*i ....3*i .... 4*i .... 5*i... i*i
x x x x x x
\------/\--------/\-------/ \------/
These x denotes the execution of the innermost for loop with complexity j. Rest of the time this is not touched and just the test is done and it fails.
So now check the thing, these \-----/ has i*j (j = 1,2,3...i) looping and i checks.
And now we do i times precisely.
So total work = i*(1+1+1+...1) + i*(1+2+3+..i)
= i*i+ i*i*(i+1)/2 ~ i^3
With the outer loop it will be n^4.
Now what is the meaning of it. The whole work can be divided in like this
O(i*j+i)
^^^ ^
| The other cases when it simply skips
The innermost loop executed
Now if we iterate over j then it will have complexity O(n^3).
With added external loop it will be O(n^4).
Your function computes 4-dimensional pyramidal numbers (A001296). The number of increments to s can be computed using this formula:
a(n) = n*(1+n)*(2+n)*(1+3*n)/24
Therefore, the complexity of your function is O(n4).
The reason it is not O(n5) is that if (j%i == 0) proceeds with the "payload" loop only for multiples of i, of which we have exactly i among all js in the range from 1 to i2, inclusive.
Hence, we add one for the outermost loop, one for the loop in the middle, and two for the innermost loop, because it iterates up to i2, for the total of 4.
Why only one for middle (j) ? It runs up to i2 right?
Perhaps it would be easier to see if we rewrite the code to exclude the condition:
int s=0;
for(int i=1;i<=n;i++)
for(int j=1;j<=i;j++)
for(int k=1;k<=i*j;k++)
s++;
return s;
This code produces the same number of "payload loop" iterations, but rather than "filtering out" the iterations that skip the inner loop it removes them from consideration by computing the terminal value of k in the innermost loop as i*j.
1) i=s=1;
while(s<=n)
{
i++;
s=s+i;
}
2) for(int i=1;i<=n;i++)
for(int j=1;j<=n;j+=i)
cout<<"*";
3) j=1;
for(int i=1;i<=n;i++)
for(j=j*i;j<=n;j=j+i)
cout<<"*";
can someone explain me the time complexity of these three codes?
I know the answers but I can't understand how it came
1) To figure this out, we need to figure out how large s is on the x'th iteration of the loop. Then we'll know how many iterations occur until the condition s > n is reached.
On the x'th iteration, the variable i has value x + 1
And the variable s has value equal to the sum of i for all previous values. So, on that iteration, s has value equal to
sum_{y = 1 .. x} (y+1) = O(x^2)
This means that we have s = n on the x = O(\sqrt{n}) iteration. So that's the running time of the loop.
If you aren't sure about why the sum is O(x^2), I gave an answer to another question like this once here and the same technique applies. In this particular case you could also use an identity
sum_{y = 1 .. x} y = y choose 2 = (y+1)(y) / 2
This identity can be easily proved by induction on y.
2) Try to analyze how long the inner loop runs, as a function of i and n. Since we start at one, end at n, and count up by i, it runs n/i times. So the total time for the outer loop is
sum_{i = 1 .. n} n/i = n * sum_{i = 1 .. n} 1 / i = O(n log n)
The series sum_{i = 1 .. n} 1 / i is called the harmonic series. It is well-known that it converges to O(log n). I can't enclose here a simple proof. It can be proved using calculus though. This is a series you just have to know. If you want to see a simple proof, you can look on on wikipedia at the "comparison test". The proof there only shows the series is >= log n, but the same technique can be used to show it is <= O(log n) also.
3.) This looks like kind of a trick question. The inner loop is going to run once, but once it exits with j = n + 1, we can never reenter this loop, because no later line that runs will make j <= n again. We will run j = j * i many times, where i is a positive number. So j is going to end up at least as large as n!. For any significant value of n, this is going to cause an overflow. Ignoring that possibility, the code is going to perform O(n) operations in total.
I recently learned about formal Big-O analysis of algorithms; however, I don't see why these 2 algorithms, which do virtually the same thing, would have drastically different running times. The algorithms both print numbers 0 up to n. I will write them in pseudocode:
Algorithm 1:
def countUp(int n){
for(int i = 0; i <= n; i++){
print(n);
}
}
Algorithm 2:
def countUp2(int n){
for(int i = 0; i < 10; i++){
for(int j = 0; j < 10; j++){
... (continued so that this can print out all values 0 - Integer.MAX_VALUE)
for(int z = 0; z < 10; z++){
print("" + i + j + ... + k);
if(("" + i + j + k).stringToInt() == n){
quit();
}
}
}
}
}
So, the first algorithm runs in O(n) time, whereas the second algorithm (depending on the programming language) runs in something close to O(n^10). Is there anything with the code that causes this to happen, or is it simply the absurdity of my example that "breaks" the math?
In countUp, the loop hits all numbers in the range [0,n] once, thus resulting in a runtime of O(n).
In countUp2, you do somewhat the exact same thing, a bunch of times. The bounds on all your loops is 10.
Say you have 3 loop running with a bound of 10. So, outer loop does 10, inner does 10x10, innermost does 10x10x10. So, worst case your innermost loop will run 1000 times, which is essentially constant time. So, for n loops with bounds [0, 10), your runtime is 10^n which, again, can be called constant time, O(1), since it is not dependent on n for worst case analysis.
Assuming you can write enough loops and that the size of n is not a factor, then you would need a loop for every single digit of n. Number of digits in n is int(math.floor(math.log10(n))) + 1; lets call this dig. So, a more strict upper bound on the number of iterations would be 10^dig (which can be kinda reduced to O(n); proof is left to the reader as an exercise).
When analyzing the runtime of an algorithm, one key thing to look for is the loops. In algorithm 1, you have code that executes n times, making the runtime O(n). In algorithm 2, you have nested loops that each run 10 times, so you have a runtime of O(10^3). This is because your code runs the innermost loop 10 times for each run of the middle loop, which in turn runs 10 times for each run of the outermost loop. So the code runs 10x10x10 times. (This is purely an upper bound however, because your if-statement may end the algorithm before the looping is complete, depending on the value of n).
To count up to n in countUp2, then you need the same number of loops as the number of digits in n: so log(n) loops. Each loop can run 10 times, so the total number of iterations is 10^log(n) which is O(n).
The first runs in O(n log n) time, since print(n) outputs O(log n) digits.
The second program assumes an upper limit for n, so is trivially O(1). When we do complexity analysis, we assume a more abstract version of the programming language where (usually) integers are unbounded but arithmetic operations still perform in O(1). In your example you're mixing up the actual programming language (which has bounded integers) with this more abstract model (which doesn't). If you rewrite the program[*] so that is has a dynamically adjustable number of loops depending on n (so if your number n has k digits, then there's k+1 nested loops), then it does one iteration of the innermost code for each number from 0 up to the next power of 10 after n. The inner loop does O(log n) work[**] as it constructs the string, so overall this program too is O(n log n).
[*] you can't use for loops and variables to do this; you'd have to use recursion or something similar, and an array instead of the variables i, j, k, ..., z.
[**] that's assuming your programming language optimizes the addition of k length-1 strings so that it runs in O(k) time. The obvious string concatenation implementation would be O(k^2) time, meaning your second program would run in O(n(log n)^2) time.
So I am learning how to calculate time complexities of algorithms from Introduction to Algorithms by Cormen. The example given in the book is insertion sort:
1. for i from 2 to length[A]
2. value := A[i]
3. j := i-1
4. while j > 0 and A[j] > value
5. A[j+1] := A[j]
6. j := j-1
7. A[j+1] = value
Line 1. executes n times.
Line 4., according to the book, executes times.
So, as a general rule, is all inner loops' execution time represented by summation?
Anecdotally, most loops can be represented by summation. In general this isn't true, for example I could create the loop
for(int i = n; i > 1; i = ((i mod 2 == 0) ? i / 2 : 3 * i + 1)))
i.e. initialize i to n, on each iteration if i is even then divide it by two, else multiply i by three and add one, halt when i <= 1. What's the big-oh for this? Nobody knows.
Obviously I've never seen a real-life program using such a weird loop, but I have seen while loops that either increase and decrease the counter depending on some arbitrary condition that might change from iteration to iteration.
When the inner loop has a condition, the times the program will iterate through it(ti) will differ in each iteration of the outer loop. That's why it's equal to the summation of all the (ti)s.
However, if the inner loop was a for loop, the times the program will iterate through it is constant. So you just multiply the times of the outer loop by the times of the inner loop.
like this:
for i=1 to n
for j=1 to m
s++
the complexity here is n*m