Consider this fragment of code
int sum = 0;
for( int i = 1; i <= n*n; i = i*2 ){
sum++ ;
}
How to do a quick proper analysis for it to get order of growth of the worst case running time?
How does changing the increment statement to i = i * 3 instead of i = i * 2 changes the worst case running time?
And is our analysis affected by changing comparison operator to < instead of <= ?
int sum = 0;
for( int i = 0; i <= n*n; i = i*2 ){
sum++ ;
}
As it stands, this is an infinite loop which will never stop, since i is never changing.
As complexity is defined for only Algorithms, which by definition should terminate in finite amount of time, it is undefined for this snippet.
However, if you change the code to the following :
int sum = 0;
for( int i = 1; i <= n*n; i = i*2 ){
sum++ ;
}
We can analyze the complexity as follows:
Let the loop run k - 1 times, and terminate at kth updation of i.
Since it's better to be redundant than to be unclear, here is what is happening:
Init(1) -> test(1) -> Loop(1) [i = 1]->
Update(2) -> test(2) -> Loop(2) [i = 2]->
...
Update(k - 1) -> test(k - 1) -> Loop(k - 1) [i = 2 ^ (k - 2)] ->
Update(k) -> test(k)->STOP [Test fails as i becomes 2 ^ (k - 1)]
Where Update(k) means kth update (i = i * 2).
Since, the increments in i are such that in the pth loop (or equivalently, after pth updation), the value of i will be 2 ^ (p - 1), we can say that at termination:
2 ^ (k - 1) > (n * n)
In verbose, we have terminated at the kth updation. Whatever the value of i was, it would've been greater than (n * n) or we would have gone for the kth loop. Taking log base 2 on both sides:
k ~ 2 * log(n)
Which implies that k is O(log(n)).
Equivalently, the number of times the loop runs is O(log(n)).
You can easily extend this idea to any limit (say n*n*n) and any increments (i*3, i*4 etc.)
The Big O complexity will be unaffected by using < instead of <=
Actualy this loop is infinte loop.
i=0
i=i*2 //0*2=0
So this loop will never end. Make i=1 to get the count of powers of 2 till n^2 not sum.
for any loop, to analys it. u have to see 2 things. the condition that will make it exit, and the iteration applied to the variable used in the condition..
for your code. u can notice that the loop stops when i goes from 0 to n*n (n^2). and the variable i is increasing like i = i*2. as i is increasing i in this manner, the loop would run for log (n^2) times. this you can see by taking an example value of n^2, like 128, and then iterate it manually one by one.
Related
What's the big O of this?
for (int i = 1; i < n; i++) {
for (int j = 1; j < (i*i); j++) {
if (j % i == 0) {
for (int k = 0; k < j; k++) {
// Simple computation
}
}
}
}
Can't really figure it out. Inclined to say O(n^4 log(n)) but feel like i'm wrong here.
This is quite a confusing analysis, so let's break it down bit by bit to make sense of the calculations:
The outermost loop runs for n-1 iterations (since 1 ≤ i < n).
The next loop inside it makes (i² - 1) iterations for each index i of the outer loop (since 1 ≤ j < i²).
In total, this means the number of iterations for these two loops is equal to calculating the sum of (i²-1) for each 1 ≤ i < n. This is similar to computing the sum of the first n squares, and is order of magnitude of O(n³).
Note the modulo operator % takes constant time (O(1)) to compute, therefore checking the condition if (j % i == 0) for all iterations of these two loops will not affect the O(n³) runtime.
Now let's talk about the inner loop inside the conditional.
We are interested in seeing how many times (and for which values of j) this if condition evaluates to true, since this would dictate how many iterations the innermost loop will run.
Practically speaking, (j % i) will never equal 0 if j < i, so the second loop could actually be shortened to start from i rather than from 1, however this will not impact the Big-O upper bound of the algorithm.
Notice that for a given number i, (j % i == 0) if and only if i is a divisor of j. Since our range is (1 ≤ j < i²), there will be a total of (i-1) values of j for which this will be true, for any given i. If this is confusing, consider this example:
Let's assume i = 4. Then our index j would iterate through all values 1,..,15=i²,
and (j%i == 0) would be true for j = 4, 8, 12 - exactly (i - 1) values.
The innermost loop would therefore make a total of (12 + 8 + 4 = 24) iterations. Thus for a general index i, we would look for the sum: i + 2i + 3i + ... + (i-1)i to indicate the number of iterations the innermost loop would make.
And this could be generalized by calculating the sum of this arithmetic progression. The first value is i and the last value is (i-1)i, which results in a sum of (i³ - i²)/2 iterations of the k loop for every value of i. In turn, the sum of this for all values of i could be computed by calculating the sum of cubes and the sum of squares - for a total runtime of O(n⁴) iterations of the innermost loop (the k loop) for all values of i.
Thus in total, the runtime of this algorithm would be the total of both runtimes we calculated above. We checked the if statement O(n³) times and the innermost loop ran for O(n⁴), so assuming // Simple computation runs in constant time, our total runtime would come down to:
O(n³) + O(n⁴)*O(1) = O(n⁴)
Let us assume that i = 2.Then j can be [1,2,3].The "k" loop will run for j = 2 only.
Similarly for i=3,j can be[1,2,3,4,5,6,7,8].hence, k can run for j = 3,6. You can see a pattern here that for any value of i, the 'k' loop will run (i-1) times.The length of loops will be [i,2*i,3*i,....i*i].
Hence the time complexity of k loop is
=i+(2*i)+(3*i)+ ..... +(i*i)
=(i^2)(i+1)/2
Hence the final complexity will be
= (n^3)(n+3)/2
Given this algorithm :
m = 1
while(a>m*b){
m = m*2
}
while(a>=b){
while(a>=m*b){
a = a-m*b
}
m=m/2
}
My question : What is the time complexity of this algorithm ?
What I have done : I have to find the number of instructions. So I found out that, for the first while, there is m=log_2(a/b) iterations approximately. Now for the inner while of the second part of this algorithm, I found this pattern : a_i = a - i*m where i is the number of iterations. So there is a/bm iterations for the inner while.
But I don't know how to calculate the outer now because the condition depends on what the inner while have done to a.
Let's begin by "normalizing" the function in the same way as in your previous question, noting that once again all changes in a and stopping conditions are proportional to b:
n = a/b
// 1)
m = 1
while(n>m){
m = m*2
}
// 2)
while(n>=1){
while(n>=m){
n = n-m
}
m=m/2
}
Unfortunately, this is where the similarity ends...
Snippet 1)
Note that m can be written as an integer power of 2, since it doubles every loop:
i = 0
while (n > pow(2, i)) {
i++
}
// m = pow(2, i)
From the stopping condition:
Snippet 2)
Here m decreases in the exact opposite way to 1), so it can again be written as a power of 2:
// using i from the end of 1)
while (n>=1) {
k = pow(2, i)
while (n >= k) {
n = n - k
}
i--
}
The inner loop is simpler than the inner loop from your previous question, because m does not change inside it. It is easy to deduce the number of times c it executes, and the value of n at the end:
This is the exact definition of the Modulus operator % in the "C-family" of languages:
while (n>=1) {
k = pow(2, i)
n = n % k // time complexity O(n / k) here instead of O(1)
i--
}
Note that, because consecutive values of k only differ by a factor of 2, at no point will the value of n be greater than or equal to 2k; this means that the inner loop executes at most once per outer loop. Therefore the outer loop executes at most i times.
Both the first and second loops are O(log n), which means the total time complexity is O(log n) = O(log [a/b]).
Update: numerical tests in Javascript as before.
function T(n)
{
let t = 0;
let m = 1;
while (n > m) {
m *= 2; t++;
}
while (n >= 1) {
while (n >= m) {
n -= m; t++;
}
m/=2;
}
return t;
}
Plotting T(n) against log(n) shows a nice straight line:
Edit: a more thorough explanation of snippet 2).
At the end of snippet 1), the value of i = ceil(log2(n)) represents the number of significant bits in the binary representation of the integer ceil(n).
Computing the modulus of an integer with a positive power-of-2 2^i is equivalent to discarding all but the first i bits. For example:
n = ...00011111111 (binary)
m = ...00000100000 (= 2^5)
n % m = ...00000011111
----- (5 least significant bits)
The operation of snippet 2) is therefore equivalent to removing the most significant bit of n, one at a time, until only zero is left. For example:
outer loop no | n
----------------------------
1 | ...110101101
| ^
2 | ...010101101
| ^
3 | ...000101101
| ^
4 | ...000001101
| ^
: | :
: | :
i (=9) | ...000000001
| ^
----------------------------
final | 000000000
When the current most significant bit (pointed to by ^) is:
0: the inner loop does not execute because the value of n is already smaller than k = 2^i (equal to the bit position value of ^).
1: the inner loop executes once because n is greater than k, but less than 2k (which corresponds the bit above the current position ^).
Hence the "worst" case occurs when all significant bits of n are 1, in which case the inner loop to always executes once.
Regardless, the outer loop executes ceil(log2(n)) times for any value of n.
1) i=s=1;
while(s<=n)
{
i++;
s=s+i;
}
2) for(int i=1;i<=n;i++)
for(int j=1;j<=n;j+=i)
cout<<"*";
3) j=1;
for(int i=1;i<=n;i++)
for(j=j*i;j<=n;j=j+i)
cout<<"*";
can someone explain me the time complexity of these three codes?
I know the answers but I can't understand how it came
1) To figure this out, we need to figure out how large s is on the x'th iteration of the loop. Then we'll know how many iterations occur until the condition s > n is reached.
On the x'th iteration, the variable i has value x + 1
And the variable s has value equal to the sum of i for all previous values. So, on that iteration, s has value equal to
sum_{y = 1 .. x} (y+1) = O(x^2)
This means that we have s = n on the x = O(\sqrt{n}) iteration. So that's the running time of the loop.
If you aren't sure about why the sum is O(x^2), I gave an answer to another question like this once here and the same technique applies. In this particular case you could also use an identity
sum_{y = 1 .. x} y = y choose 2 = (y+1)(y) / 2
This identity can be easily proved by induction on y.
2) Try to analyze how long the inner loop runs, as a function of i and n. Since we start at one, end at n, and count up by i, it runs n/i times. So the total time for the outer loop is
sum_{i = 1 .. n} n/i = n * sum_{i = 1 .. n} 1 / i = O(n log n)
The series sum_{i = 1 .. n} 1 / i is called the harmonic series. It is well-known that it converges to O(log n). I can't enclose here a simple proof. It can be proved using calculus though. This is a series you just have to know. If you want to see a simple proof, you can look on on wikipedia at the "comparison test". The proof there only shows the series is >= log n, but the same technique can be used to show it is <= O(log n) also.
3.) This looks like kind of a trick question. The inner loop is going to run once, but once it exits with j = n + 1, we can never reenter this loop, because no later line that runs will make j <= n again. We will run j = j * i many times, where i is a positive number. So j is going to end up at least as large as n!. For any significant value of n, this is going to cause an overflow. Ignoring that possibility, the code is going to perform O(n) operations in total.
I'm trying to find the worse case complexity function of this algorithm considering comparisons statement as the most relevant operation. That's when the if and else if are both always executed under the loop, so the function is 2*number of loop executions.
Since the variable i is beeing increased by a bigger number each time the complexity is probably O(log n) but how do i find the exac number of executions? Thanks.
int find ( int a[], int n, int x ) {
int i = 0, mid;
while ( i < n ) {
mid = ( n + i ) / 2;
if ( a[mid] < x )
n = mid;
else if ( a[mid] > x )
i = mid + 1;
else return mid;
}
return 0;
}
Qualitative Understanding
Well let's try to look at the loop invariant to figure out how long this function is going to run.
We can see that the function will continue to execute code until this while(i < n){ ... } condition is met.
Let's also note that within this while loop, i or n is always being mutated to some variation of mid:
if ( a[mid] < x ) # Condition 1:
n = mid; # we set n to mid
else if ( a[mid] > x ) # Condition 2:
i = mid + 1; # we set i to mid+1
else return mid; # Condition 3: we exit loop (let's not worry about this)
So now let's focus on mid since our while condition always seems to be getting cut down depending on this value (since the while condition is dependent on i and n, one of which will be set to the value of mid after each loop iteration):
mid = ( n + i ) / 2; # mid = average of n and i
So effectively we can see what's going on here after looking at these pieces of the function:
The function will execute code while i < n, and after each loop iteration the value of i or n is set to the average value, effectively cutting down the space between i and n by half each time the loop iterates.
This algorithm is known as a binary search, and the idea behind it is we keep cutting the array boundaries in half each time we iterate in the loop.
So you can think about it as we keep cutting n in half until we can't cut in half anymore.
Quantitative Analysis
A mathematical way to look at this is to see that we're effectively dividing n by 2 each iteration, until i and n are equal to each other (or n < i).
So let's think about it as how many times can we divide our n by 2 until it equals 1? We want our n to equal 1 in this case because that's when we are unable to split the list any further.
So we're left with an equation, where x is the amount of time we need to execute the while loop:
n/2^x = 1
n = 2^x
lg(n) = lg(2^x)
lg(n) = x lg(2)
lg(n) = x
As you can see, x = lg(n) so we can conclude that your algorithm runs in O(lgn)
I have been practicing analyzing algorithms lately. I feel like I have a pretty good understanding of analyzing non-recursive algorithms but I am unsure, and have just begun to embark on a full understanding of recursive algorithm as well. Although, I have not had a formal check on my methods and if what I have been doing is indeed correct
Would it be too much to ask if someone could check a few algorithms that I have implemented and analyzed and see if my understanding is along the right lines or if I am completely off.
here:
1)
sum = 0;
for (i = 0; i < n; i++){
for (j = 0; j < i*i; j++){
if (j % i == 0) {
for (k = 0; k < j; k++){
sum++;
}
}
}
}
My analysis of this one was O(n^5) due to:
Sum(i = 0 to n)[Sum(j = 0 to i^2)[Sum(k = 0 to j) of 1]]
which evaluated to:
(1/2)(n^5/5 + n^4/2 + n^3/3 - n/30) + (1/2)(n^3/3 + n^2/2 + n/6) + (1/2)(n^3/3 + n^2/2 + n/6) + n + 1.
Hence it is O(n^5)
Is this correct as an evaluation of the summations of the loops?
a triple summation. I have assumed that the if statement will always pass for worse case complexity. Is this a correct assumption for worst case?
2)
int tonyblair (int n, int a) {
if (a < 12) {
for (int i = 0; i < n; i++){
System.out.println("*");
}
tonyblair(n-1, a);
} else {
for (int k = 0; k < 3000; k++){
for (int j = 0; j < nk; j++){
System.out.println("#");
}
}
}
}
My analysis of this algorithm is O(infinity) due to the infinite recursion in the if statement if it is assumed to be true, which would be the worst case. Although, for pure analysis, I analyzed if this were not true and the if statement would not run. I then got a complexity of O(nk) due to:
Sum(k = 0 to 3000)[Sum(j = 0 to nk) of 1]
which then evaluated to nk(3001) + 3001. Hence is O(nk), where k is not discarded due to it controlling the number of iterations of the loop.
Number 1
I can't tell how you've derived your formula. Usually adding terms happens when there are multiple steps in an algorithm, such as precomputing data and then looking up values from the data. Instead, nested for loops implies multiplication. Also, the worst case is the best case for this snippet of code, because given a value of n, sum will be the same at the end.
To find the complexity, we want to find the number of times that the inner loop is evaluated. Summations are often easy to solve if they go from 1 to n, so I'm going to drop the 0s from them later on. If i is 0, the middle loop won't run, and if j is 0, the inner loop won't run. We can rewrite the code equivalently as:
sum = 0;
for (i = 1; i < n; i++)
{
for (j = 1; j < i*i; j++)
{
if (j % i == 0)
{
for (k = 0; k < j; k++)
{
sum++;
}
}
}
}
I could make my life harder by forcing the outer loop to start at 2, but I'm not going to. The outer loop now runs from 1 to n-1. The middle loop runs based on the current value of i, so we need to do a summation:
The middle for loop always goes to (i^2 - 1), and j will only be divisible by i for a total of (i - 1) times (i, i*2, i*3, ..., i*(i-2), i*(i-1)). With this, we get:
The middle loop then executes j times. The j in our summation is not the same as the j in the code though. The j in the summation represents each time the middle loop executes. Each time the middle loop executes, the j in the code will be (i * (number of executions so far)) = i * (the j in the summation). Therefore, we have:
We can move the i to in-between the two summations, as it is a constant for the inner summation. Then, the formula for the sum of 1 to n is well known: n*(n+1)/2. Because we are going to n - 1, we must subtract n out. This gives:
The summations for the sum of squares and the sum of cubes are also well known. Keeping in mind that we are only summing to n-1 in both cases, we must remember to subtract n^3 and n^2, respectively, and we get:
This is obviously n^4. If we solve it all the way, we get:
Number 2
For the last one, it is in fact O(infinity) if a < 12 because of the if statement. Well, technically everything is O(infinity), because Big-O only provides an upper bound on runtime. If a < 12, it is also omega(infinity) and theta(infinity). If only the else runs, then we have the summation from 1 to 2999 of i*n:
It's very important to notice that the summation from 1 to 2999 is a constant (it's 4498500). No matter how large a constant is, it's still a constant, and not dependent on n. We will end up throwing it out of the runtime calculations. Sometimes, when a theoretically fast algorithm has a large constant, it is practically slower than other algorithms that are theoretically slow. One example I can think of is Chazelle's linear time triangulation algorithm. No one has ever implemented it. In any case, we have 4498500 * n. This is theta(n):