What is the time complexity of the problem below.
int j=1;
while(j<n){
j+=log(j+5);
}
By expanding the first three terms of the sum:
You can see that it's just a sum of iterations of log(log(j))'s.
Since O(j) >> O(log(j)), it follows that O(log(j)) >> O(log(log(j)); the first term therefore overshadows all of the other terms.
The sum is therefore O(log(j)), which means the time complexity is
.
Numerical tests show that this is actually O(n^0.82...).
With the edited code (+= instead of =), I'll conjecture that the asymptotic time complexity of this code (assuming that log() and other elementary operations take constant time) is Θ(n / log n).
It's easy to show that the loop takes at least n / log(n+5) = n / (log n × log 5) iterations to complete, since it's counting up to n, and each iteration increments the counter by an amount strictly less than log(n+5). Thus, the asymptotic time complexity is at least Ω(n / log n).
Showing that it's also O(n / log n), and thus Θ(n / log n), seems a bit trickier. Basically, my argument is that, for sufficiently large n, incrementing the counter j from exp(k) up to exp(k+1) takes on the order of C × exp(k) / k iterations (for a constant C ≈ (1 − exp(−1)) / 5, if I'm not mistaken). Letting h = ceil(log(n)), incrementing the counter from 1 to n thus takes at most
T = C × ( exp(h) / h + exp(h−1) / (h−1) + exp(h−2) / (h−2) + … + exp(1) / 1 )
iterations. For large n (and thus h), the leading exp(h) / h term should dominate the rest, such that T ≤ C2 × exp(h) / h for some constant C2, and thus the loop should run in O(exp(h) / h) = O(n / log n) time.
That said, I admit that this is just a proof sketch, and there might be gaps or errors in my argument. In particular, I have not actually determined an upper bound for the constant(?) C2. (C2 = C × h would obviously satisfy the inequality, but would only yield an O(n) upper bound on the runtime; C2 = C / (1 − exp(−1)) would give the desired bound, but is obviously too low.) Thus, I cannot completely rule out the possibility that the actual time complexity might be (very slightly) higher.
Related
Given this question where increment over the iterator i happens by incrementing its value by its own log value. What will be the big O time complexity for this fragment?
i = 10
while(i<n) {
i=i+log(i);
}
Interesting question! Here's an initial step toward working out the runtime.
Let's imagine that we have reached a point in the loop where the value of i has reached 2k for some natural number k. Then for i to increase to 2k+1, we'll need approximately 2k / k iterations to elapse. Why? That's because
The amount i needs to increase is 2k+1 - 2k = 2k(2 - 1) = 2k, and
At each step, increasing i by log i will increase i by roughly log 2k = k.
We can therefore break the algorithm apart into "stages." In the first stage, we grow from size 23 to size 24, requiring (roughly) 23/3 steps. In the second stage, we grow from size 24 to size 25, requiring (roughly) 24 / 4 steps. After repeating this process many times, we eventually grow from size n/2 = 2log n - 1 to size n = 2log n, requiring (roughly) 2log n / log n steps. The total amount of work done is therefore given by
23 / 3 + 24/4 + 25 / 5 + ... + 2log n / log n.
The goal now is to find some bounds on this expression. We can see that the sum is at least equal to its last term, which is 2log n / log n = n / log n, so the work done is Ω(n / log n). We can also see that the work done is less than
23 + 24 + 25 + ... + 2log n
≤ 2log n + 1 (sum of a geometric series)
= 2n,
so the work done is O(n). That sandwiches the runtime between Ω(n / log n) and O(n).
It turns out that Θ(n / log n) is indeed a tight bound here, which can be proved by doing a more nuanced analysis of the summation.
Let us look a bit about the definition of g(n)=O(f(n)): saying the function g is of order O(f(n)) means there exists a number n0 and a constant c such that for all n>n0 it is g(n)<=cf(n).
Looking at the worst case scenario, the while will run for maximum n times which means we can say your code is of order O(n).
Now let's assume that the code inside the while loop is
(*) while(i<n) {
i = i + i ;
}
which obviously should skip more iteration that the original one. So we can use this code to estimate a lower bound. Examining (*) we see that in each iteration the counter will be double, if we think a little bit about it we see that for n iteration each time half of the input will be thrown out. so the code in (*) will have worst case asymptotic runtime O(log n).
Now we estimate that the original code should be between both so we can say its asymptotic lower bound is Ω(log n) and its asymptotic upper bound is O(n).
My professor just taught us that any operation that halves the length of the input has an O(log(n)) complexity as a thumb rule. Why is it not O(sqrt(n)), aren't both of them equivalent?
They are not equivalent: sqrt(N) will increase a lot more quickly than log2(N). There is no constant C so that you would have sqrt(N) < C.log(N) for all values of N greater than some minimum value.
An easy way to grasp this, is that log2(N) will be a value close to the number of (binary) digits of N, while sqrt(N) will be a number that has itself half the number of digits that N has. Or, to state that with an equality:
log2(N) = 2log2(sqrt(N))
So you need to take the logarithm(!) of sqrt(N) to bring it down to the same order of complexity as log2(N).
For example, for a binary number with 11 digits, 0b10000000000 (=210), the square root is 0b100000, but the logarithm is only 10.
Assuming natural logarithms (otherwise just multiply by a constant), we have
lim {n->inf} log n / sqrt(n) = (inf / inf)
= lim {n->inf} 1/n / 1/(2*sqrt(n)) (by L'Hospital)
= lim {n->inf} 2*sqrt(n)/n
= lim {n->inf} 2/sqrt(n)
= 0 < inf
Refer to https://en.wikipedia.org/wiki/Big_O_notation for alternative defination of O(.) and thereby from above we can say log n = O(sqrt(n)),
Also compare the growth of the functions below, log n is always upper bounded by sqrt(n) for all n > 0.
Just compare the two functions:
sqrt(n) ---------- log(n)
n^(1/2) ---------- log(n)
Plug in Log
log( n^(1/2) ) --- log( log(n) )
(1/2) log(n) ----- log( log(n) )
It is clear that: const . log(n) > log(log(n))
No, It's not equivalent.
#trincot gave one excellent explanation with example in his answer. I'm adding one more point. Your professor taught you that
any operation that halves the length of the input has an O(log(n)) complexity
It's also true that,
any operation that reduces the length of the input by 2/3rd, has a O(log3(n)) complexity
any operation that reduces the length of the input by 3/4th, has a O(log4(n)) complexity
any operation that reduces the length of the input by 4/5th, has a O(log5(n)) complexity
So on ...
It's even true for all reduction of lengths of the input by (B-1)/Bth. It then has a complexity of O(logB(n))
N:B: O(logB(n)) means B based logarithm of n
One way to approach the problem can be to compare the rate of growth of O()
and O( )
As n increases we see that (2) is less than (1). When n = 10,000 eq--1 equals 0.005 while eq--2 equals 0.0001
Hence is better as n increases.
No, they are not equivalent; you can even prove that
O(n**k) > O(log(n, base))
for any k > 0 and base > 1 (k = 1/2 in case of sqrt).
When talking on O(f(n)) we want to investigate the behaviour for large n,
limits is good means for that. Suppose that both big O are equivalent:
O(n**k) = O(log(n, base))
which means there's a some finite constant C such that
O(n**k) <= C * O(log(n, base))
starting from some large enough n; put it in other terms (log(n, base) is not 0 starting from large n, both functions are continuously differentiable):
lim(n**k/log(n, base)) = C
n->+inf
To find out the limit's value we can use L'Hospital's Rule, i.e. take derivatives for numerator and denominator and divide them:
lim(n**k/log(n)) =
lim([k*n**(k-1)]/[ln(base)/n]) =
ln(base) * k * lim(n**k) = +infinity
so we can conclude that there's no constant C such that O(n**k) < C*log(n, base) or in other words
O(n**k) > O(log(n, base))
No, it isn't.
When we are dealing with time complexity, we think of input as a very large number. So let's take n = 2^18. Now for sqrt(n) number of operation will be 2^9 and for log(n) it will be equal to 18 (we consider log with base 2 here). Clearly 2^9 much much greater than 18.
So, we can say that O(log n) is smaller than O(sqrt n).
To prove that sqrt(n) grows faster than lgn(base2) you can take the limit of the 2nd over the 1st and proves it approaches 0 as n approaches infinity.
lim(n—>inf) of (lgn/sqrt(n))
Applying L’Hopitals Rule:
= lim(n—>inf) of (2/(sqrt(n)*ln2))
Since sqrt(n) and ln2 will increase infinitely as n increases, and 2 is a constant, this proves
lim(n—>inf) of (2/(sqrt(n)*ln2)) = 0
i came across this piece of code to perform merge sort on a link list..
the author claims that it runs in time O(nlog n)..
here is the link for it...
http://www.geeksforgeeks.org/merge-sort-for-linked-list/
my claim is that it takes atleast O(n^2) time...and here is my argument...
look, you divide the list(be it array or linked list), log n times(refer to recursion tree), during each partition, given a list of size i=n, n/2, ..., n/2^k, we would take O(i) time to partition the original/already divided list..since sigma O(i)= O(n),we can say , we take O(n) time to partition for any given call of partition(sloppily), so given the time taken to perform a single partition, the question now arises as to how many partitions are going to happen all in all, we observe that the number of partitions at each level i is equal to 2^i , so summing 2^0+2^1+....+2^(lg n ) gives us [2(lg n)-1] as the sum which is nothing but (n-1) on simplification , implying that we call partition n-1, (let's approximate it to n), times so , the complexity is atleast big omega of n^2..
if i am wrong, please let me know where...thanks:)
and then after some retrospection , i applied master method to the recurrence relation where i replaced theta of 1 which is there for the conventional merge sort on arrays with theta of n for this type of merge sort (because the divide and combine operations take theta of n time each), the running time turned out to be theta of (n lg n)...
also i noticed that the cost at each level is n (because 2 power i * (n/(2pow i)))...is the time taken for each level...so its theta of n at each level* lg n levels..implying that its theta of (n lg n).....did i just solve my own question??pls help i am kinda confused myself..
The recursive complexity definition for an input list of size n is
T(n) = O(n) + 2 * T(n / 2)
Expanding this we get:
T(n) = O(n) + 2 * (O(n / 2) + 2 * T(n / 4))
= O(n) + O(n) + 4 * T(n / 4)
Expanding again we get:
T(n) = O(n) + O(n) + O(n) + 8 * T(n / 8)
Clearly there is a pattern here. Since we can repeat this expansion exactly O(log n) times, we have
T(n) = O(n) + O(n) + ... + O(n) (O(log n) terms)
= O(n log n)
You are performing a sum twice for some weird reason.
To split and merge a linked list of size n, it takes O(n) time. The depth of recursion is O(log n)
Your argument was that a splitting step takes O(i) time and sum of split steps become O(n) And then you call it the time taken to perform only one split.
Instead, lets consider this, a problem of size n forms two n/2 problems, four n/4 problems eight n/8 and so on until 2^log n n/2^logn subproblems are formed. Sum these up you get O(nlogn) to perform splits.
Another O(nlogn) to combine sub problems.
I got this question wrong on an exam : Name a function that is neither O(n) nor Omega(n).
After attempting to learn this stuff on my own through youtube, I'm thinking this may be a correct answer:
(n3 (1 + sin n)) is neither O(n) nor Omega(n).
Would that be accurate?
Name a function that is neither O(n) nor Omega(n)
Saying f ∈ O(g) means the quotient
f(x)/g(x)
is bounded from above for all sufficiently large x.
f ∈ Ω(g) on the other hand means the quotient
f(x)/g(x)
is bounded below away from zero for all sufficiently large x.
So to find a function that is neither O(n) nor Ω(n) means finding a function f such that the quotient
f(x)/x
becomes arbitrarily large, and arbitrarily close to zero on every interval [y, ∞).
I'm thinking this may be a correct answer: (n^3 (1 + sin n)) is neither O(n) nor Omega(n).
Let's plug it in our quotient:
(n^3*(1 + sin n))/n = n^2*(1 + sin n)
The n^2 grows to infinity, and the factor 1 + sin n is larger than 1 for roughly three out of every six n. So one every interval [y, ∞) the quotient becomes arbitrarily large. Given an arbitrary K > 0, let N_0 = y + K + 1 and N_1 the smallest of N_0 + i, i = 0, 1, ..., 4 such that sin (N_0+i) > 0. Then f(N_1)/N_1 > (y + K + 1)² > K² + K > K.
For the Ω(n) part, it's not so easy to prove, although I believe it is satisfied.
But, we can modify the function a bit, retaining the idea of multiplying a growing function with an oscillating one in such a way that the proof becomes simple.
Instead of sin n, let us choose cos (π*n), and, to offset the zeros, add a fast decreasing function to it.
f'(n) = n^3*(1 + cos (π*n) + 1/n^4)
now,
/ n^3*(2 + 1/n^4), if n is even
f'(n) = <
\ 1/n , if n is odd
and it is obvious that f' is neither bounded from above, nor from below by any positive constant multiple of n.
I would consider something like a binary search. This is both O(log N) and Ω(log N). Since Omega is defined as a lower bound, it's not allowed to exceed the function itself -- so O(log N) definitely is not Ω(N).
I think some of the comments on the deleted answer deserve some...clarification -- perhaps even outright correction. To quote from CLRS, "Ω-notation gives a lower bound for a function to within a constant factor."
Since N2 differs from N by more than a constant factor, N2 is not Ω(N).
Given the following algorithm on a dataset of size N:
Separate the data into M=(N/lg N)
blocks in O(N) time.
Partition the blocks in O(M lg M) time. *
What is the big-O? How do I evaluate (N/lg N) * lg (N/lg N) ?
If it is not O(N), is there an M small enough that the whole thing does become O(N)?
* The partition algorithm is the STL's stable_partition which, in this example, will do M tests and at most M lg M swaps. But, the items being swapped are blocks of size lg N. Does this push the practical time of step 2 back up to O(N lg N) if they must be swapped in place?
Not homework, just a working engineer poking around in comp-sci stuff.
You evaluate by doing a bit of math.
log(x/y) = log(x) - log(y)
->
log(N / log(N)) = log(N) - log(log(N))
So, plugging this back in and combining into a single fraction.
N(log(N) - log(log(N))) / log(N)
=
N - N(log(log(N)) / log(N))
<=, since log(log(N)) <= log(N) as N -> inf., it's like multiplying by <= 1
N
So, the whole thing is O(N).
You can pretty easily guess that it is O(N log N) by noticing that M = N / log N is, itself, O(N). I don't know of a quick way to figure out that it's O(N) without a bit of doubt on my part due to having to multiply in the log M.
It is O(N):
N / lgN * lg(N / lgN)=N / lgN * (lgN-lglgN)=N*(1-lglgN / lgN)<=N