I need to analyze the O-Notation complexity of the following code. I think I have some idea, but need some help. Do my O-Notations for each loop correct?
int sum = 0;
for (int i = 1; i <= n*2; i++ ) //N operations, ignore the multiplication by 2
sum++;
O(N)
int sum = 0;
for ( int i = 1; i <= n; i++) //N operations
for ( int j = n; j > 0; j /= 2) //N operations, ignore division by constant?
sum++;
O(N^2)
int sum = 0;
for ( int i = 1; i <= n; i++) //N ops
for ( int j = i; j <= n; j += 2) // N ops
sum++;
O(N^2)
int sum = 0;
for ( int i = 1; i <= n * n; i++) // N * N ops
for ( int j = 1; j < i; j++ ) // N opts
sum++;
O(N^3)
You iterate through 2*N elements, so its O(2N) = O(N)
You iterate through N elements and in each iteration, you go through logN elements, so its O(N*logN) - this time its different from 1., because you divide the counter j, not the limiting variable N, so , that means, if N is 20, you get this:
first iteration: j = 20
second iteration: j = 10
third iteration: j = 5
fourth iteration j = 2
fifth iteration j = 1
so its log(20) = 4.something, and that means 5 iterations...
So, it depends where is calculation perform, either in limiting factor or iteration calculation...
This is O(N^2), because you have for loop with n iterations, and in each iteration you have n/2 iterations, so its O(N * N / 2) = O(N^2)
In first loop you have N*N iterations. In each iteration you have to go to i, which depends on n, so you have:
i=1 => no iteration in inner for loop
i=2 => 1 iteration
i=3 => 2 iterations
i=4 => 3 iterations
...
i=n*n => n*n - 1 iterations
Sum of it is:
0+1+2+3+...+n*n-1 = 2+3+...n*n = 1+2+3+...+m-1(where m is n*n) = m*(m+1)/2 - m
So its: O(M^2) = O(N^4)
Related
I understand that the innermost for loop is Θ(logn)
and the two outermost for loops is Θ(n^2) because it's an arithmetic sum. The if-statement is my main problem. Does anyone know how to solve this?
int tally=0;
for (int i = 1; i < n; i ++)
{
for (int j = i; j < n; j ++)
{
if (j % i == 0)
{
for (int k = 1; k < n; k *= 2)
{
tally++;
}
}
}
}
Edit:
Now I noticed loop order: i before j.
In this case for given i value j varies from i to n and there are (n/i) successful if-conditions.
So program will call then most inner loop
n/1 +n/2+n/3+..+n/n
times. This is sum of harmonic series, it converges to n*ln(n)
So inner loop will be executed n*log^2(n) times.
As you wrote, two outermost loops provide O(n^2) complexity, so overall complexity is O(n^2 + n*log^2(n)), the first term overrides the second one, loop, and finally overall complexity is quadratic.
int tally=0;
for (int i = 1; i < n; i ++)
{
// N TIMES
for (int j = i; j < n; j ++)
{
//N*N/2 TIMES
if (j % i == 0)
{
//NlogN TIMES
for (int k = 1; k < n; k *= 2)
{
//N*logN*logN
tally++;
}
}
}
}
Old answer (wrong)
This complexity is linked with sum of sigma0(n) function (number of divisors) and represented as sequence A006218 (Dirichlet Divisor problem)
We can see that approximation for sum of divisors for values up to n is
n * ( log(n) + 2*gamma - 1 ) + O(sqrt(n))
so average number of successful if-conditions for loop counter j is ~log(j)
so I was wondering how would I work out the time complexity (T(n)) of a piece of code, for example, the one below, in terms of the number of operations.
for( int i = n; i > 0; i /= 2 ) {
for( int j = 1; j < n; j *= 2 ) {
for( int k = 0; k < n; k += 2 ) {
... // constant number of operations
}
}
}
I'm sure its simple but this concept wasn't taught very well by my lecturer and I really want to know how to work out the time complexity!
Thank you in advance!
To approach this, one method is to breakdown the complexity of your three loops individually.
A key observation we can make is that:
(P): The number of steps in each of the loop does not depend on the value of the "index" of its parent loop.
Let's call
f(n) the number of operations aggregated in the outer loop (1)
g(n) in the intermediate inner loop (2)
h(n) in the most inner loop (3).
for( int i = n; i > 0; i /= 2 ) { // (1): f(n)
for( int j = 1; j < n; j *= 2 ) { // (2): g(n)
for( int k = 0; k < n; k += 2 ) { // (3): h(n)
// constant number of operations // => (P)
}
}
}
Loop (1)
Number of steps
i gets the values n, n/2, n/4, ... etc. until it reaches n/2^k where 2^k is greater than n (2^k > n), such that n/2^k = 0, at which point you exit the loop.
Another way to say it is that you have step 1 (i = n), step 2 (i = n/2), step 3 (i = n/4), ... step k - 1 (i = n/2^(k-1)), then you exit the loop. These are k steps.
Now what is the value of k? Observe that n - 1 <= 2^k < n <=> log2(n - 1) <= k < log2(n) <= INT(log2(n - 1)) <= k <= INT(log2(n)). This makes k = INT(log2(n)) or loosely speaking k = log2(n).
Cost of each step
Now how many operations do you have for each individual step?
At step i, it is g(i) = g(n) according to the notations we chose and the property (P).
Loop (2)
Number of steps
You have step (1) (j = 1), step (2) (j = 2), step (3) (j = 4), etc. until you reach step (p) (j = 2^p) where p is defined as the smallest integer such that 2^p > n, or loosely speaking log2(n).
Cost of each step
The cost of step j is h(j) = h(n) according to the notations we chose and the property (P).
Loop (3)
Number of steps
Again, let's count the steps: (1):k = 0, (1):k = 2, (2):k = 4, ..., k = n - 1 or k = n - 2. This amounts to n / 2 steps.
Cost of each step
Because of (P), it is constant. Let's call this constant K.
All loops altogether
The number of aggregated operations is
T(n) = f(n) = sum(i = 0, i < log2(n), g(i))
= sum(i = 0, i < log2(n), g(n))
= log2(n).g(n)
= log2(n).sum(j = 0, j < log2(n), h(j))
= log2(n).log2(n).h(n)
= log2(n).log2(n).(n/2).K
So T(n) = (K/2).(log2(n))^2.n
Write a method, add a counter, return the result:
int nIterator (int n) {
int counter = 0;
for( int i = n; i > 0; i /= 2 ) {
for( int j = 1; j < n; j *= 2 ) {
for( int k = 0; k < n; k += 2 ) {
++counter;
}
}
}
return counter;
}
Protocol for fast increasing N and document in a readable manner the results:
int old = 0;
for (int i = 0, j=1; i < 18; ++i, j*=2) {
int res = nIterator (j);
double quote = (old == 0) ? 0.0 : (res*1.0)/old;
System.out.printf ("%6d %10d %3f\n", j, res, quote);
old=res;
}
Result:
1 0 0,000000
2 2 0,000000
4 12 6,000000
8 48 4,000000
16 160 3,333333
32 480 3,000000
64 1344 2,800000
128 3584 2,666667
256 9216 2,571429
512 23040 2,500000
1024 56320 2,444444
2048 135168 2,400000
4096 319488 2,363636
8192 745472 2,333333
16384 1720320 2,307692
32768 3932160 2,285714
65536 8912896 2,266667
131072 20054016 2,250000
So n is increasing by factor 2, the counter increases in the beginning with more than 2², but then decreases rapidly towards something, not much higher than 2. This should help you find the way.
I am trying to determine the Big-O runtimes for these loops. I believe the answers I have are correct, but I would like to check with the community.
int sum = 0;
for (int i = 1; i <= n*2; i++ )
sum++;
My answer is O(n)
This is because the loop iterates n x 2 times. We drop the 2 and are left with n. Therefore O(n).
int sum = 0;
for ( int i = 1; i <= n; i++)
for ( int j = n; j > 0; j /= 2)
sum++;
My answer is O(n lgn)
The outer loop iterates n times. The inner loop iterates from n down to 0, but only through half of the items. This works out to be Log base 2 of n. We drop the 2 and keep log n. The inner loop (log n) times the outer loop (n), gives us O(n lgn).
int sum = 0;
for ( int i = 1; i <= n; i++)
for ( int j = i; j <= n; j += 2)
sum++;
My answer is O(n^2)
This one is easy. The inner loop and outer loop each iterate n times. n x n = n^2. Therefore O(n^2).
int sum = 0;
for ( int i = 1; i <= n * n; i++)
for ( int j = 1; j < i; j++ )
sum++;
My answer is O(n^3)
Are my answers correct? If not, how can I correct them?
Only the last one is wrong. It should be O(n⁴).
You can see it this way: substitute n * n with x. The number of operations will be the usual O(x*(x+1)/2) = O(x²). Now substitute n * n back into x.
Extra challenge for you.
You correctly said that:
int sum = 0;
for ( int i = 1; i <= n; i++)
for ( int j = n; j > 0; j /= 2)
sum++;
Is O(n log n). But what about this one:
int sum = 0;
for ( int i = 1; i <= n; i++)
for ( int j = i; j > 0; j /= 2)
sum++;
I only changed j = n to j = i.
I've got to analyze this loop, among others, and determine its running time using Big-O notation.
for ( int i = 0; i < n; i += 4 )
for ( int j = 0; j < n; j++ )
for ( int k = 1; k < j*j; k *= 2 )`
Here's what I have so far:
for ( int i = 0; i < n; i += 4 ) = n
for ( int j = 0; j < n; j++ ) = n
for ( int k = 1; k < j*j; k *= 2 ) = log^2 n
Now the problem I'm coming to is the final running time of the loop. My best guess is O(n^2), however I am uncertain if this correct. Can anyone help?
Edit: sorry about the Oh -> O thing. My textbook uses "Big-Oh"
First note that the outer loop is independent from the remaining two - it simply adds a (n/4)* multiplier. We will consider that later.
Now let's consider the complexity of
for ( int j = 0; j < n; j++ )
for ( int k = 1; k < j*j; k *= 2 )
We have the following sum:
0 + log2(1) + log2(2 * 2) + ... + log2(n*n)
It is good to note that log2(n^2) = 2 * log2(n). Thus we re-factor the sum to:
2 * (0 + log2(1) + log2(2) + ... + log2(n))
It is not very easy to analyze this sum but take a look at this post. Using Sterling's approximation one can that it is belongs to O(n*log(n)). Thus the overall complexity is O((n/4)*2*n*log(n))= O(n^2*log(n))
In terms of j, the inner loop is O(log_2(j^2)) time, but sine
log_2(j^2)=2log(j), it is actually O(log(j)).
For each iteration of middle loop, it takes O(log(j)) time (to do the
inner loop), so we need to sum:
sum { log(j) | j=1,..., n-1 } log(1) + log(2) + ... + log(n-1) = log((n-1)!)
And since log((n-1)!) is in O((n-1)log(n-1)) = O(nlogn), we can conclude middle middle loop takes O(nlogn) operations .
Note that both middle and inner loop are independent of i, so to
get the total complexity, we can just multiply n/4 (number of
repeats of outer loop) with complexity of middle loop, and get:
O(n/4 * nlogn) = O(n^2logn)
So, total complexity of this code is O(n^2 * log(n))
Time Complexity of a loop is considered as O(n) if the loop variables is incremented / decremented by a constant amount (which is c in examples below):
for (int i = 1; i <= n; i += c) {
// some O(1) expressions
}
for (int i = n; i > 0; i -= c) {
// some O(1) expressions
}
Time complexity of nested loops is equal to the number of times the innermost statement is executed. For example the following sample loops have O(n²) time complexity:
for (int i = 1; i <=n; i += c) {
for (int j = 1; j <=n; j += c) {
// some O(1) expressions
}
}
for (int i = n; i > 0; i += c) {
for (int j = i+1; j <=n; j += c) {
// some O(1) expressions
}
Time Complexity of a loop is considered as O(logn) if the loop variables is divided / multiplied by a constant amount:
for (int i = 1; i <=n; i *= c) {
// some O(1) expressions
}
for (int i = n; i > 0; i /= c) {
// some O(1) expressions
}
Now we have:
for ( int i = 0; i < n; i += 4 ) <----- runs n times
for ( int j = 0; j < n; j++ ) <----- for every i again runs n times
for ( int k = 1; k < j*j; k *= 2 )` <--- now for every j it runs logarithmic times.
So complexity is O(n²logm) where m is n² which can be simplified to O(n²logn) because n²logm = n²logn² = n² * 2logn ~ n²logn.
i, j, N, sum is all integer type. N is input.
( Code1 )
i = N;
while(i > 1)
{
i = i / 2;
for (j = 0; j < 1000000; j++)
{
sum = sum + j;
}
}
( Code2 )
sum = 0;
d = 1;
d = d << (N-1);
for (i = 0; i < d; i++)
{
for (j = 0; j < 1000000; j++)
{
sum = sum + i;
}
}
How to calculate step count and time complexity for a Code1, Code2?
to calculate the time complexity try to understand what takes how much time, and by what n are you calculating.
if we say the addition ("+") takes O(1) steps then we can check how many time it is done in means of N.
the first code is dividing i in 2 each step, meaning it is doing log(N) steps. so the time complexity is
O(log(N) * 1000000)= O(log(N))
the second code is going form 0 to 2 in the power of n-1 so the complexity is:
O(s^(N-1) * 1000000)= O(2^(N-1))
but this is just a theory, because d has a max of 2^32/2^64 or other number, so it might not be O(2^(N-1)) in practice