O(n log log n) time complexity - algorithm

I have a short program here:
Given any n:
i = 0;
while (i < n) {
k = 2;
while (k < n) {
sum += a[j] * b[k]
k = k * k;
}
i++;
}
The asymptotic running time of this is O(n log log n). Why is this the case? I get that the entire program will at least run n times. But I'm not sure how to find log log n. The inner loop is depending on k * k, so it's obviously going to be less than n. And it would just be n log n if it was k / 2 each time. But how would you figure out the answer to be log log n?

For mathematical proof, inner loop can be written as:
T(n) = T(sqrt(n)) + 1
w.l.o.g assume 2 ^ 2 ^ (t-1)<= n <= 2 ^ (2 ^ t)=>
we know 2^2^t = 2^2^(t-1) * 2^2^(t-1)
T(2^2^t) = T(2^2^(t-1)) + 1=T(2^2^(t-2)) + 2 =....= T(2^2^0) + t =>
T(2^2^(t-1)) <= T(n) <= T(2^2^t) = T(2^2^0) + log log 2^2^t = O(1) + loglogn
==> O(1) + (loglogn) - 1 <= T(n) <= O(1) + loglog(n) => T(n) = Teta(loglogn).
and then total time is O(n loglogn).
Why inner loop is T(n)=T(sqrt(n)) +1?
first see when inner loop breaks, when k>n, means before that k was at least sqrt(n), or in two level before it was at most sqrt(n), so running time will be T(sqrt(n)) + 2 ≥ T(n) ≥ T(sqrt(n)) + 1.

Time Complexity of a loop is O(log log n) if the loop variables is reduced / increased exponentially by a constant amount. If the loop variable is divided / multiplied by a constant amount then complexity is O(Logn).
Eg: in your case value of k is as follow. Let i in parenthesis denote the number of times the loop has been executed.
2 (0) , 2^2 (1), 2^4 (2), 2^8 (3), 2^16(4), 2^32 (5) , 2^ 64 (6) ...... till n (k) is reached.
The value of k here will be O(log log n) which is the number of times the loop has executed.
For the sake of assumption lets assume that n is 2^64. Now log (2^64) = 64 and log 64 = log (2^6) = 6. Hence your program ran 6 times when n is 2^64.

I think if the codes are like this, it should be n*log n;
i = 0;
while (i < n) {
k = 2;
while (k < n) {
sum += a[j] * b[k]
k *= c;// c is a constant bigger than 1 and less than k;
}
i++;
}

Okay, So let's break this down first -
Given any n:
i = 0;
while (i < n) {
k = 2;
while (k < n) {
sum += a[j] * b[k]
k = k * k;
}
i++;
}
while( i<n ) will run for n+1 times but we'll round it off to n times.
now here comes the fun part, k<n will not run for n times instead it will run for log log n times because here instead of incrementing k by 1,in each loop we are incrementing it by squaring it. now this means it'll take only log log n time for the loop. you'll understand this when you learn design and analysis of algorithm
Now we combine all the time complexity and we get n.log log n time here I hope you get it now.

Related

is the time complexity of nested for-loops always of O(n^2)?

for (i = 1; i <= n; i++)
{
for (j = n; j >= i; j--)
}
I'm struggling with this algorithm. I can't even know what time complexity of this algorithm is? I've checked using online software it shows me only o(n).
First of all, the algorithm should be something like this:
for (i = 1; i <= n; i++)
for (j = n; j >= i; j--)
DoSomeWork(i, j); // <- Payload which is O(1)
To find out the time complexity, let's count how many times DoSomeWork will be executed:
i : j : executed, times
----------------------------
1 : 1..n : n
2 : 2..n : n - 1
3 : 3..n : n - 2
.. : ... ...
n : n..n : 1
So far so good, DoSomeWork will be executed
n + n - 1 + n - 2 + ... + 2 + 1 = (n + 1) * n / 2
times; time complexity for your case is
O((n + 1) * n / 2) = O((n + 1) * n) = O(n * n) + O(n) = O(n * n)
Nested loops are not necessary have quadratic time complexity, e.g.
for (i = 1; i <= n; i++)
for (j = n; j >= i; j /= 2) // note j /= 2, instead of j--
DoSomeWork(i, j);
has O(n * log(n)) time complexity
Think about this a little. How many times do we enter into the outer loop? It's n times, as you surely already know, since we have step1, ..., stepn, in total n steps.
So, we have
n * average(inner)
In the first step, the inner loop has n steps. Then it has n - 1 steps and so on, on the final step we have 1 step in the inner loop.
so, the inner loop has:
n, n-1, ..., 1 steps in the respective iterations.
Since addition is both commutative and associative, we have:
(n + 1) + (n - 1 + 2) + ...
that is (n+1)/2 on average.
Since we worry about the scenario when n -> infinity, adding 1 to it or dividing it by 2 is less of a concern, so we roughly have n * n steps, hence it has a complexity of O(n * n), don't listen to the tool that says otherwise, unless you have some extra information about your algorithm to share with us.

What's the complexity of sum i=0 -> n (n_i*i))

This is a test I failed because I thought this complexity would be O(n), but it appears i'm wrong and it's O(n^2). Why not O(n)?
First, notice that the question does not ask what is the time complexity of a function calculating f(n), but rather the complexity of the function f(n) itself. you can think about f(n) as being the time complexity of some other algorithm if you are more comfortable talking about time complexity.
This is indeed O(n^2), when the sequence a_i is bounded by a constant and each a_i is at least 1.
By the assumption, for all i, a_i <= c for some constant c.
Hence, a_1*1+...+a_n*n <= c * (1 + 2 + ... + n). Now we need to show that 1 + 2 +... + n = O(n^2) to complete the proof.
1 + 2 + ... + n <= n + n + ... + n = n * n = n ^ 2
and
1 + 2 + ... + n >= n / 2 + (n / 2 + 1) + ... + n >= (n / 2) * (n / 2) = n^2/4
So the complexity is actually Theta(n^2).
Note that if a_i was not constant, e.g., a_i = i then the result is not correct.
in that case, f(n) = 1^2 + 2^2 + ... + n^2 and you can show easily (using the same method as before) that f(n) = Omega(n^3), which means it's not O(n^2).
Preface, not super great with complexity-theory but I'll take a stab.
I think what is confusing is that its not a time complexity problem, but rather the functions complexity.
So for easy part i just goes up to n ie. 1,2,3 ...n , then for ai all entries must be above 0 meaning that a could be something like this 2,5,1... for n times. If you multiply them together n*n = O(n2).
The best case would be if a is 1,1,1 which drop the complexity down to O(n) but the average case will be n so you get squared.
Unless it's mentioned that a[i] is O(n), it's definitely O(n)
Here an another try to achieve O(n*n) if sum should be returned as result.
int sum = 0;
for(int i = 0; i<=n; i++){
for(int j = 0; j<=n; j++){
if(i == j){
sum += A[i] * j;
}
}
return sum;

Time complexity of the code segment

I'm trying to calculate the time complexity of the following code snippet
sum = 0;
for(i = 0; i <= n; i++) {
for(j = 1; j <= i; j++) {
if(i % j == 0) {
for(k = 0; k <= n; k++) {
sum = sum + k;
}
}
}
}
What i think , that out of N iterations of First loop, only 1 value which is 0 allowed to enter K loop and from i = 1.....N, K loop never runs.
So, only 1 value of I runs j loop N times and k loop N times and for other values of I only J loop runs N times
So, is the TC = O(N^2) ?
Here let d(n) is the number of divisors of n.
I see your program doing O(n) work(innermost loop) for O( d(n) ) number of divisors of each i (i looping from 0 to n in outermost loop: O(n) ).
Its complexity is O( n * d(n) * n ).
Reference
for large n, d() ~ O( exp( log(n)/log(log n) ) ).
So the overall complexity is O( n^(2 + 1/log(log n) ) ).
I've got another answer. Lets replace the inner loop with an abstract func():
for(i=0;i<=n;i++) {
for(j=1;j<=i;j++) {
if(i%j==0) {
func();
}
}
}
Firstly, forgetting the calls to func(), the complexity M to calculate all (i % j) is O(n^2).
Now, we can ask ourselves how many times the func() is called. It's called once for each divisor of i. That is it is called d(i) times for each i. This is a Divisor summatory function D(n). D(n) ~ n log n for large n.
So func() is called n log n times. At the same time the func() itself has complexity of O(n). So it gives the complexity P = O(n * n log n).
So total complexity is M + P = O(n^2) + O(n^2 log n) = O(n^2 log n)
Edit
Vow, thanks for downvote! I guess I need to prove it using python.
This code prints out n, how many times the inner loop is called for n, and outputs ratio of the latter and Divisor summatory function
import math
n = 100000
i = 0
z = 0
gg = 2 * 0.5772156649 - 1
while i < n:
j = 1
while j <= i:
if i % j == 0:
#ignoring the most inner loop just calculate the number of times it is called
z+=1
j+=1
if i > 0 and i % 1000 == 0:
#Exact divisor summatory function, to make z/Di converge to 1.0 quicker
Di = (i * math.log(i) + i * gg)
#prints n Di z/Di
print str(i) + ": " + str(z) + ": " + str(z/Di)
i+=1
Output sample:
24000: 245792: 1.00010672544
25000: 257036: 1.00003672445
26000: 268353: 1.00009554815
So the most inner loop is called n * log n times, and total complexity is n^2 * log n

What is the complexity of this code whose nested for loop repeatedly doubles its counter?

In the book Programming Interviews Exposed it says that the complexity of the program below is O(N), but I don't understand how this is possible. Can someone explain why this is?
int var = 2;
for (int i = 0; i < N; i++) {
for (int j = i+1; j < N; j *= 2) {
var += var;
}
}
You need a bit of math to see that. The inner loop iterates Θ(1 + log [N/(i+1)]) times (the 1 + is necessary since for i >= N/2, [N/(i+1)] = 1 and the logarithm is 0, yet the loop iterates once). j takes the values (i+1)*2^k until it is at least as large as N, and
(i+1)*2^k >= N <=> 2^k >= N/(i+1) <=> k >= log_2 (N/(i+1))
using mathematical division. So the update j *= 2 is called ceiling(log_2 (N/(i+1))) times and the condition is checked 1 + ceiling(log_2 (N/(i+1))) times. Thus we can write the total work
N-1 N
∑ (1 + log (N/(i+1)) = N + N*log N - ∑ log j
i=0 j=1
= N + N*log N - log N!
Now, Stirling's formula tells us
log N! = N*log N - N + O(log N)
so we find the total work done is indeed O(N).
Outer loop runs n times. Now it all depends on the inner loop.
The inner loop now is the tricky one.
Lets follow:
i=0 --> j=1 ---> log(n) iterations
...
...
i=(n/2)-1 --> j=n/2 ---> 1 iteration.
i=(n/2) --> j=(n/2)+1 --->1 iteration.
i > (n/2) ---> 1 iteration
(n/2)-1 >= i > (n/4) ---> 2 iterations
(n/4) >= i > (n/8) ---> 3 iterations
(n/8) >= i > (n/16) ---> 4 iterations
(n/16) >= i > (n/32) ---> 5 iterations
(n/2)*1 + (n/4)*2 + (n/8)*3 + (n/16)*4 + ... + [n/(2^i)]*i
N-1
n*∑ [i/(2^i)] =< 2*n
i=0
--> O(n)
#Daniel Fischer's answer is correct.
I would like to add the fact that this algorithm's exact running time is as follows:
Which means:

Tricky Big-O complexity

public void foo(int n, int m) {
int i = m;
while (i > 100) {
i = i / 3;
}
for (int k = i ; k >= 0; k--) {
for (int j = 1; j < n; j *= 2) {
System.out.print(k + "\t" + j);
}
System.out.println();
}
}
I figured the complexity would be O(logn).
That is as a product of the inner loop, the outer loop -- will never be executed more than 100 times, so it can be omitted.
What I'm not sure about is the while clause, should it be incorporated into the Big-O complexity? For very large i values it could make an impact, or arithmetic operations, doesn't matter on what scale, count as basic operations and can be omitted?
The while loop is O(log m) because you keep dividing m by 3 until it is below or equal to 100.
Since 100 is a constant in your case, it can be ignored, yes.
The inner loop is O(log n) as you said, because you multiply j by 2 until it exceeds n.
Therefore the total complexity is O(log n + log m).
or arithmetic operations, doesn't matter on what scale, count as basic operations and can be omitted?
Arithmetic operations can usually be omitted, yes. However, it also depends on the language. This looks like Java and it looks like you're using primitive types. In this case it's ok to consider arithmetic operations O(1), yes. But if you use big integers for example, that's not really ok anymore, as addition and multiplication are no longer O(1).
The complexity is O(log m + log n).
The while loop executes log3(m) times - a constant (log3(100)). The outer for loop executes a constant number of times (around 100), and the inner loop executes log2(n) times.
The while loop divides the value of m by a factor of 3, therefore the number of such operations will be log(base 3) m
For the for loops you could think of the number of operations as 2 summations -
summation (k = 0 to i) [ summation (j = 0 to lg n) (1)]
summation (k = 0 to i) [lg n + 1]
(lg n + 1) ( i + 1) will be total number of operations, of which the log term dominates.
That's why the complexity is O(log (base3) m + lg n)
Here the lg refers to log to base 2

Resources