I'm trying to find the worse case complexity function of this algorithm considering comparisons statement as the most relevant operation. That's when the if and else if are both always executed under the loop, so the function is 2*number of loop executions.
Since the variable i is beeing increased by a bigger number each time the complexity is probably O(log n) but how do i find the exac number of executions? Thanks.
int find ( int a[], int n, int x ) {
int i = 0, mid;
while ( i < n ) {
mid = ( n + i ) / 2;
if ( a[mid] < x )
n = mid;
else if ( a[mid] > x )
i = mid + 1;
else return mid;
}
return 0;
}
Qualitative Understanding
Well let's try to look at the loop invariant to figure out how long this function is going to run.
We can see that the function will continue to execute code until this while(i < n){ ... } condition is met.
Let's also note that within this while loop, i or n is always being mutated to some variation of mid:
if ( a[mid] < x ) # Condition 1:
n = mid; # we set n to mid
else if ( a[mid] > x ) # Condition 2:
i = mid + 1; # we set i to mid+1
else return mid; # Condition 3: we exit loop (let's not worry about this)
So now let's focus on mid since our while condition always seems to be getting cut down depending on this value (since the while condition is dependent on i and n, one of which will be set to the value of mid after each loop iteration):
mid = ( n + i ) / 2; # mid = average of n and i
So effectively we can see what's going on here after looking at these pieces of the function:
The function will execute code while i < n, and after each loop iteration the value of i or n is set to the average value, effectively cutting down the space between i and n by half each time the loop iterates.
This algorithm is known as a binary search, and the idea behind it is we keep cutting the array boundaries in half each time we iterate in the loop.
So you can think about it as we keep cutting n in half until we can't cut in half anymore.
Quantitative Analysis
A mathematical way to look at this is to see that we're effectively dividing n by 2 each iteration, until i and n are equal to each other (or n < i).
So let's think about it as how many times can we divide our n by 2 until it equals 1? We want our n to equal 1 in this case because that's when we are unable to split the list any further.
So we're left with an equation, where x is the amount of time we need to execute the while loop:
n/2^x = 1
n = 2^x
lg(n) = lg(2^x)
lg(n) = x lg(2)
lg(n) = x
As you can see, x = lg(n) so we can conclude that your algorithm runs in O(lgn)
Related
Given this algorithm :
m = 1
while(a>m*b){
m = m*2
}
while(a>=b){
while(a>=m*b){
a = a-m*b
}
m=m/2
}
My question : What is the time complexity of this algorithm ?
What I have done : I have to find the number of instructions. So I found out that, for the first while, there is m=log_2(a/b) iterations approximately. Now for the inner while of the second part of this algorithm, I found this pattern : a_i = a - i*m where i is the number of iterations. So there is a/bm iterations for the inner while.
But I don't know how to calculate the outer now because the condition depends on what the inner while have done to a.
Let's begin by "normalizing" the function in the same way as in your previous question, noting that once again all changes in a and stopping conditions are proportional to b:
n = a/b
// 1)
m = 1
while(n>m){
m = m*2
}
// 2)
while(n>=1){
while(n>=m){
n = n-m
}
m=m/2
}
Unfortunately, this is where the similarity ends...
Snippet 1)
Note that m can be written as an integer power of 2, since it doubles every loop:
i = 0
while (n > pow(2, i)) {
i++
}
// m = pow(2, i)
From the stopping condition:
Snippet 2)
Here m decreases in the exact opposite way to 1), so it can again be written as a power of 2:
// using i from the end of 1)
while (n>=1) {
k = pow(2, i)
while (n >= k) {
n = n - k
}
i--
}
The inner loop is simpler than the inner loop from your previous question, because m does not change inside it. It is easy to deduce the number of times c it executes, and the value of n at the end:
This is the exact definition of the Modulus operator % in the "C-family" of languages:
while (n>=1) {
k = pow(2, i)
n = n % k // time complexity O(n / k) here instead of O(1)
i--
}
Note that, because consecutive values of k only differ by a factor of 2, at no point will the value of n be greater than or equal to 2k; this means that the inner loop executes at most once per outer loop. Therefore the outer loop executes at most i times.
Both the first and second loops are O(log n), which means the total time complexity is O(log n) = O(log [a/b]).
Update: numerical tests in Javascript as before.
function T(n)
{
let t = 0;
let m = 1;
while (n > m) {
m *= 2; t++;
}
while (n >= 1) {
while (n >= m) {
n -= m; t++;
}
m/=2;
}
return t;
}
Plotting T(n) against log(n) shows a nice straight line:
Edit: a more thorough explanation of snippet 2).
At the end of snippet 1), the value of i = ceil(log2(n)) represents the number of significant bits in the binary representation of the integer ceil(n).
Computing the modulus of an integer with a positive power-of-2 2^i is equivalent to discarding all but the first i bits. For example:
n = ...00011111111 (binary)
m = ...00000100000 (= 2^5)
n % m = ...00000011111
----- (5 least significant bits)
The operation of snippet 2) is therefore equivalent to removing the most significant bit of n, one at a time, until only zero is left. For example:
outer loop no | n
----------------------------
1 | ...110101101
| ^
2 | ...010101101
| ^
3 | ...000101101
| ^
4 | ...000001101
| ^
: | :
: | :
i (=9) | ...000000001
| ^
----------------------------
final | 000000000
When the current most significant bit (pointed to by ^) is:
0: the inner loop does not execute because the value of n is already smaller than k = 2^i (equal to the bit position value of ^).
1: the inner loop executes once because n is greater than k, but less than 2k (which corresponds the bit above the current position ^).
Hence the "worst" case occurs when all significant bits of n are 1, in which case the inner loop to always executes once.
Regardless, the outer loop executes ceil(log2(n)) times for any value of n.
I've recently come across a method which returns the smallest factor of a given number:
public static int findFactor(int n)
{
int i = 1;
int j = n - 1;
int p = j; // invariant: p = i * j
while(p != n && i < j)
{
i++;
p += j;
while(p > n)
{
j--;
p -= i;
}
}
return p == n ? i : n;
}
After examining the method, I've been able to (most likely incorrectly) determine the quantities which some of is variables respectively represent:
n = the int that is subject to factorization for
the purposes of determining its smallest factor
i = the next potential factor of n to be tested
j = the smallest integer which i can be multiplied by to yield a value >= n
The problem is I don't know what quantity p represents. The inner loop seems to treat (p+=j) - n as a
potential multiple of i, but given what I believe j represents, I don't understand how that can be true
for all i, or how the outer loop accounts for the "extra" iteration of the inner loop that is carried out
before the latter terminates as a result of p < n
Assuming I've correctly determined what n, i, and j represent, what quantity does p represent?
If any of my determinations are incorrect, what do each of the quantities represent?
p stands for “product”. The invariant, as stated, is p == i*j; and the algorithm tries different combinations of i and j until the product (p) equals n. If it never does (the while loop falls through), you get p != n, and hence n is returned (n is prime).
At the end of the outer while loop's body, j is the largest integer which i can be multiplied by to yield a value ≤ n.
The algorithm avoids explicit division, and tries to limit the number of j values inspected for each i. At the beginning of the outer loop, p==i*j is just less than n. As i is gradually increased, j needs to gradually shrink. In each outer loop, i is increased (and p is corrected to match the invariant). The inner loop then decreases j (and corrects p) until p is ≤ n again. Since i*j is only just less than n at the beginning of the next outer loop, increasing i makes the product greater than n again, and the process repeats.
The algorithm tries all divisors between 1 and n / i (continuing past n / i is of no use as the corresponding quotients have already been tried).
So the outer loop actually performs
i= 1
while i * (n / i) != n && i < n / i)
{
i++;
}
It does it in a clever way, by avoiding divisions. As the annotation says, the invariant p = i * j is maintained; more precisely, p is the largest multiple of i that doesn't exceed n, and this actually establishes j = n / i.
There is a little adjustment to perform when i is incremented: i becoming i + 1 makes p = i * j become (i + 1) * j = p + j, and p may become too large. This is fixed by decrementing j as many times as necessary (j--, p-= i) to compensate.
public int foo ( int x , int k ) {
if ( x <= k )
return 1;
else
return foo ( x / k , k ) + 1;
}
From my understanding, the runtime of this should equal the runtime of the conditional + the runtime of the larger if/else statement.
However, I'm having trouble determining the correct runtime of the statement "return foo(x/k, k) + 1". Would this be constant? The increase of x and k doesn't seem to have an effect on the runtime, as it is the ratio between them that is important for the outcome.
Any clarity would be much appreciated. Thanks!
The recursive call divides x by k each time, so the complexity for k >= 2 is O(logk(x)) - logarithm of x in base k.
For k < 2 it might not terminate (in reality, it will run out of memory or divide by zero).
I'm having troubles understanding how to estimate the Big-O. We've had two lectures on this topic and the only thing I undestand is to take the leading coefficient from the largest polynomial in the function and replace it with an O so it would look like O(...)
During the first lecture this was shown
int i = length;
while (i>0) {
i--;
int j = i -1;
while (j >= 0) {
if (a[i] == a[j]) {
return 1;
}
j--;
}
}
return 0;
Followed by this on the following slide
int i = length; // Counts as 1
while (i>0) { // Counts as N+1
i--; // Counts as N
int j = i -1; // Coutns as N
while (j >= 0) { // Counts as i+1
if (a[i] == a[j]) { // Counts as i
return 1;
}
j--; // Counts as i
}
}
return 0; // Counts as 1
From this, I'm wondering why
return 1;
isn't counted as a step.
Following that slide, it tells us that the
Outer Loop count is 3N+1
Inner Loop count is 3i+1 ; for all possible i from 0 to N-1
I understand that the second [while] loop will occur N times and following that, the [if] will occur i times where i is equal to N-1 since if j < 0, the second while loop will still be read but nothing else will happen after it.
The slide shows that the Total from the Inner loop is equal to
3N^2 - 1/2N
and that the Grand Total is equal to 3/2N^2 + 5/2N +3.
Wondering if anyone has time to walk me through how to acquire the functions used in Big-O estimations like in the example above; I have no idea how 3i+1 translated into 3N^2 - 1/2N as well has how the Grand Total is calculated.
I will try to explain the calculation of the complexity of your example.
First we notice, that every operation requries only constant time, written as O(1), which means, that the run time does not depend on the input.
int i = length; // O(1), executed only one time
while (i > 0) { // outer loop, condition needs O(1)
i--; // O(1)
int j = i - 1; // O(1)
while (j >= 0) { // inner loop, condition needs O(1)
if (a[i] == a[j]) { // O(1)
return 1; // first return statement
}
j--; // O(1)
}
}
return 0; // second return statement, executed only one time
The number of operations in every loop is constant, so we only have to count how often they are executed.
The outer loop runs from i = n to i = 1. For each i the inner loop is executed once and executes i constant time operations itself. In total we get
3 + Σi=0,...,n-13 + 3i + 1 = 3 + 4n + 3/2⋅(n-1)⋅n = 3/2⋅n² + 5/2⋅n + 3
(1) (2) (3) (4) (5)
Explanations:
The 3 contains the first line, the last line and the an additional execution of the outer loop condition. The condition evaluates n times to true and one time to false.
This 3 contains one evaluation of the condition of the outer loop and the first two lines in the outer loop.
The factor 3 in front of the i contains one evaluation of the inner loop condition, the evaluation of the if statement and the last line of the inner loop.
The 1 is for the additional evaluation where the inner loop condition evaluates to false.
The sum of consecutive integers 1 to n evaluates to 1/2⋅n⋅(n+1). Notice the sum here is from 0 to n-1, so it evaluates to 1/2⋅(n-1)⋅n.
The frist (inner) return statement is not counted, because if executed the algorithm terminates. But we want to calculate the maximum number of steps, the so called worst case. This case is when the algorithm terminates as late as possible.
Notice: The calculation of the steps is very precise. This is not necessary to get the big-O complexity. It would be enough to say, that each loop runs in O(n) and since they are nested the complexity has to be multiplied so you get O(n)⋅O(n) = O(n²).
Consider this fragment of code
int sum = 0;
for( int i = 1; i <= n*n; i = i*2 ){
sum++ ;
}
How to do a quick proper analysis for it to get order of growth of the worst case running time?
How does changing the increment statement to i = i * 3 instead of i = i * 2 changes the worst case running time?
And is our analysis affected by changing comparison operator to < instead of <= ?
int sum = 0;
for( int i = 0; i <= n*n; i = i*2 ){
sum++ ;
}
As it stands, this is an infinite loop which will never stop, since i is never changing.
As complexity is defined for only Algorithms, which by definition should terminate in finite amount of time, it is undefined for this snippet.
However, if you change the code to the following :
int sum = 0;
for( int i = 1; i <= n*n; i = i*2 ){
sum++ ;
}
We can analyze the complexity as follows:
Let the loop run k - 1 times, and terminate at kth updation of i.
Since it's better to be redundant than to be unclear, here is what is happening:
Init(1) -> test(1) -> Loop(1) [i = 1]->
Update(2) -> test(2) -> Loop(2) [i = 2]->
...
Update(k - 1) -> test(k - 1) -> Loop(k - 1) [i = 2 ^ (k - 2)] ->
Update(k) -> test(k)->STOP [Test fails as i becomes 2 ^ (k - 1)]
Where Update(k) means kth update (i = i * 2).
Since, the increments in i are such that in the pth loop (or equivalently, after pth updation), the value of i will be 2 ^ (p - 1), we can say that at termination:
2 ^ (k - 1) > (n * n)
In verbose, we have terminated at the kth updation. Whatever the value of i was, it would've been greater than (n * n) or we would have gone for the kth loop. Taking log base 2 on both sides:
k ~ 2 * log(n)
Which implies that k is O(log(n)).
Equivalently, the number of times the loop runs is O(log(n)).
You can easily extend this idea to any limit (say n*n*n) and any increments (i*3, i*4 etc.)
The Big O complexity will be unaffected by using < instead of <=
Actualy this loop is infinte loop.
i=0
i=i*2 //0*2=0
So this loop will never end. Make i=1 to get the count of powers of 2 till n^2 not sum.
for any loop, to analys it. u have to see 2 things. the condition that will make it exit, and the iteration applied to the variable used in the condition..
for your code. u can notice that the loop stops when i goes from 0 to n*n (n^2). and the variable i is increasing like i = i*2. as i is increasing i in this manner, the loop would run for log (n^2) times. this you can see by taking an example value of n^2, like 128, and then iterate it manually one by one.