What will be the time complexity(big O notation) of the following algorithm
for(int i=3 ; i<n/3; i+=3){
for(int j=2 ; j<n/2; j+=2){
//O(1) operation
}
}
by approximation:
for(int i=3 ; i<n/3; i+=3){ // ( n/3 - 3 ) / 3 = n/9 - 1 times
for(int j=2 ; j<n/2; j+=2){ // ( n/2 - 2 ) / 2 = n/4 - 1 times
//O(1) operation
}
}
so the final answer is: (n/9 - 1)*(n/4 - 1)*O(1) = (n^2 / 36 - ...)*O(1) ≈ O(n^2)
Related
Can I get some help in understanding how to solve this tutorial question! I still do not understand my professors explanation. I am unsure of how to count the big 0 for the third/most inner loop. She explains that the answer for this algorithm is O(n^2) and that the 2nd and third loop has to be seen as one loop with the big 0 of O(n). Can someone please explain to me the big O notation for the 2nd / third loop in basic layman terms
Assuming n = 2^m
for ( int i = n; i > 0; i --) {
for (int j =1; j < n; j *= 2){
for (int k =0; k < j; k++){
}
}
}
As far as I understand, the first loop has a big O notation of O(n)
Second loop = log(n)
Third loop = log (n) (since the number of times it will be looped has been reduced by logn) * 2^(2^m-1)( to represent the increase in j? )
lets add print statement to the innermost loop.
for (int j =1; j < n; j *= 2){
for (int k =0; k < j; k++){
print(1)
}
}
output for
j = 1, 1 1
j = 2, 1 1 1
j = 4, 1 1 1 1 1
...
j = n, 1 1 1 1 1 ... n+1 times.
The question boils down to how many 1s will this print.
That number is
(2^0 + 1) + (2^1 + 1) + (2^2 + 1) + ... + (n + 1)
= (2^0 + 1) + (2^1 + 1) + (2^2 + 1) + ... + (n + 1)
= log n + (1 + 2 + 4 + ... + n)
= O(log n + n)
= O(n).
assuming you know why (1 + 2 + 4 + ... + n) = O(n)
O-notation is an upperbound. You can say it has O(n^2). For least upperbound, I believe it should be O(n*log(n)*log(n)) which belongs to O(n^2).
It’s because of the logarithm. If you have log(16) raised to the power 2 is 16. So log(n) raised to the power of 2 is n. That is why your teacher says to view the second and third loop as O(n) together.
If the max iterations for the second loop are O(log(n)) then the second and third loops will be: O(1 + 2 + 3 + ... + log(n)) = O(log(n)(log(n) + 1)/2) = O((log(n)^2 + log(n))/2) = O(n)
for ( int i = n; i > 0; i --) { // This runs n times
for (int j =1; j < n; j *= 2){ // This runs atmost log(n) times, i.e m times.
for (int k =0; k < j; k++){ // This will run atmost m times, when the value of j is m.
}
}
}
Hence, the overall complexity will be the product of all three, as mentioned in the comments under the question.
Upper bound can be loose or tight.
You can say that it is loosely bound under O(n^2) or tightly bound under O(n * m^2).
int something = 0 ;
int t = n ;
while ( t > 1 ) {
for (int i=0 ; i < t ; i++) {
something++ ;
}
t = t / 2 ;
}
//number 2
int sum = 0;
int i = 1;
while (sum <= n) {
sum = sum + i;
i++;
}
How do I find the tightest upper bound in big O notation. I think they would both be log n but I am not sure if the for loop in the first segment of code affects its run time significantly.
The complexity of the above code is O(n).
Let's calculate the complexity of first part of the program.
int t = n;
while ( t > 1 ) {
for (int i=0 ; i < t ; i++) {
something++ ;
}
t = t / 2 ;
}
This will execute: t + t/2 + t/4 + ... + 1
and the sum of this series is 2t - 1. (Refer this).
Therefore, the time complexity of the first part if O(2t - 1) = O(t) or O(n).
For second part:
int sum = 0;
int i = 1;
while (sum <= n) {
sum = sum + i;
i++;
}
Sum will be 1+2+3+....i which is i*(i+1)/2.
So sum =i*(i+1)/2 ~ i2
Also sum <= n
So i2 <= n
or i ~
Therfore the complexity of second part is O()
So the complexity of program is:
T(n): O(n) + O() = O(n)
Overall complexity is O(n)
so I was wondering how would I work out the time complexity (T(n)) of a piece of code, for example, the one below, in terms of the number of operations.
for( int i = n; i > 0; i /= 2 ) {
for( int j = 1; j < n; j *= 2 ) {
for( int k = 0; k < n; k += 2 ) {
... // constant number of operations
}
}
}
I'm sure its simple but this concept wasn't taught very well by my lecturer and I really want to know how to work out the time complexity!
Thank you in advance!
To approach this, one method is to breakdown the complexity of your three loops individually.
A key observation we can make is that:
(P): The number of steps in each of the loop does not depend on the value of the "index" of its parent loop.
Let's call
f(n) the number of operations aggregated in the outer loop (1)
g(n) in the intermediate inner loop (2)
h(n) in the most inner loop (3).
for( int i = n; i > 0; i /= 2 ) { // (1): f(n)
for( int j = 1; j < n; j *= 2 ) { // (2): g(n)
for( int k = 0; k < n; k += 2 ) { // (3): h(n)
// constant number of operations // => (P)
}
}
}
Loop (1)
Number of steps
i gets the values n, n/2, n/4, ... etc. until it reaches n/2^k where 2^k is greater than n (2^k > n), such that n/2^k = 0, at which point you exit the loop.
Another way to say it is that you have step 1 (i = n), step 2 (i = n/2), step 3 (i = n/4), ... step k - 1 (i = n/2^(k-1)), then you exit the loop. These are k steps.
Now what is the value of k? Observe that n - 1 <= 2^k < n <=> log2(n - 1) <= k < log2(n) <= INT(log2(n - 1)) <= k <= INT(log2(n)). This makes k = INT(log2(n)) or loosely speaking k = log2(n).
Cost of each step
Now how many operations do you have for each individual step?
At step i, it is g(i) = g(n) according to the notations we chose and the property (P).
Loop (2)
Number of steps
You have step (1) (j = 1), step (2) (j = 2), step (3) (j = 4), etc. until you reach step (p) (j = 2^p) where p is defined as the smallest integer such that 2^p > n, or loosely speaking log2(n).
Cost of each step
The cost of step j is h(j) = h(n) according to the notations we chose and the property (P).
Loop (3)
Number of steps
Again, let's count the steps: (1):k = 0, (1):k = 2, (2):k = 4, ..., k = n - 1 or k = n - 2. This amounts to n / 2 steps.
Cost of each step
Because of (P), it is constant. Let's call this constant K.
All loops altogether
The number of aggregated operations is
T(n) = f(n) = sum(i = 0, i < log2(n), g(i))
= sum(i = 0, i < log2(n), g(n))
= log2(n).g(n)
= log2(n).sum(j = 0, j < log2(n), h(j))
= log2(n).log2(n).h(n)
= log2(n).log2(n).(n/2).K
So T(n) = (K/2).(log2(n))^2.n
Write a method, add a counter, return the result:
int nIterator (int n) {
int counter = 0;
for( int i = n; i > 0; i /= 2 ) {
for( int j = 1; j < n; j *= 2 ) {
for( int k = 0; k < n; k += 2 ) {
++counter;
}
}
}
return counter;
}
Protocol for fast increasing N and document in a readable manner the results:
int old = 0;
for (int i = 0, j=1; i < 18; ++i, j*=2) {
int res = nIterator (j);
double quote = (old == 0) ? 0.0 : (res*1.0)/old;
System.out.printf ("%6d %10d %3f\n", j, res, quote);
old=res;
}
Result:
1 0 0,000000
2 2 0,000000
4 12 6,000000
8 48 4,000000
16 160 3,333333
32 480 3,000000
64 1344 2,800000
128 3584 2,666667
256 9216 2,571429
512 23040 2,500000
1024 56320 2,444444
2048 135168 2,400000
4096 319488 2,363636
8192 745472 2,333333
16384 1720320 2,307692
32768 3932160 2,285714
65536 8912896 2,266667
131072 20054016 2,250000
So n is increasing by factor 2, the counter increases in the beginning with more than 2², but then decreases rapidly towards something, not much higher than 2. This should help you find the way.
I would like to know the time complexity of this algorithm and how it is calculated.
for (i = 1; i < 2n; i++) {
for (j = 1; j < i; j++) {
for (k = 1; k < j; k++) {
// do something
}
}
}
Assume the inner statement takes constant time.
The inner loop runs (j-1) times, hence its run time is
t_inner(j) = Sum {k from 1 to j-1} 1
= j-1
The middle loop runs i-1 times. Its run time is:
t_middle(i) = Sum { j from 1 to i-1 } t_inner(j)
= Sum { j from 1 to i-1 } j-1
= 1/2 * (2 - 3 * i + i^2)
The outer loop runs 2n-1 times. Its run time is:
t_outer(n) = Sum { i from 1 to 2n-1 } t_middle(i)
= Sum { i from 1 to 2n-1 } 1/2 * (2 - 3 * i + i^2)
= 1/3 (-3 + 11 n - 12 n^2 + 4 n^3)
From the last formula, we see that the time complexity is O(n^3).
Where n is the input to the function can be any integer.
i = n, total = 0;
while (i > 0) {
for (j=0; j<i; j++)
for (k=0; k<i; k++)
total++;
i = i/4;
}
What is the time complexity of this function?
One way to think about this is to look at the loops independently.
This inner loop nest:
for (j=0; j<i; j++)
for (k=0; k<i; k++)
total++;
will execute a total of Θ(i2) operations, since each loop independently runs i times.
Now, let's look at the outer loop:
while (i > 0) {
/* do Theta(i^2) work */
i = i/4;
}
This loop will run a total of at most 1 + log4 i times, since on each iteration i is cut by a factor of 1/4, and this can only happen 1 + log4 i times before i drops to zero. The question, then, is how much work will be done.
One way to solve this is to write a simple recurrence relation for the total work done. We can think of the loop as a tail-recursive function, where each iteration does Θ(i2) work and then makes a recursive call on a subproblem of size 4. This gives this recurrence:
T(n) = T(n / 4) + Θ(n2).
Applying the Master Theorem, we see that a = 1, b = 4, and c = 2. Since logb a = log4 1 = 0 and 0 < c, the Master Theorem says (by Case 3) that the runtime solves to Θ(n2). Therefore, the total work done is Θ(n2).
Here's another way to think about this. The first iteration of the loop does n2 work. The next does (n / 4)2 = n2 / 16 work. The next does (n / 64)2 = n2 / 256 work. In fact, iteration x of the loop will do n2 / 16x work. Therefore, the total work done is given by
n2(1 + 1 / 16 + 1 / 162 + 1 / 163 + ...)
≤ n2(1 / (1 - 1/16))
= Θ(n2)
(This uses the formula for the sum of an infinite geometric series).
Hope this helps!
The running time is O(n^2), and the number of times that total is incremented is asymptotic to n^2/(1-1/16) which is about 1.067 n^2.
The recurrence is going to be
T(n) = n^2 + T(n/4)
= n^2 + n^2/16 + T(n/16)
= n^2 (1 + 1/16 + 1/16^2 + ...)
= n^2 / (1 - 1/16)
This code fragment:
i = n, total = 0;
while (i > 0) {
for (j=0; j<i; j++)
for (k=0; k<i; k++)
total++;
i = i/4;
}
is equivalent to this one:
for ( i = n ; i > 0 ; i = i / 4 )
for ( j = 0 ; j < i ; j ++)
for ( k = 0 ; k < i ; k ++)
total ++;
Therefore, methodically (empirically verified), you may obtain the following using Sigma Notation:*
With many thanks to WolframAlpha.