This question already has answers here:
Big-O Notation regarding logarithms
(4 answers)
Finding Big O of the Harmonic Series
(3 answers)
Asymptotic complexity of T(n)=T(n-1)+1/n [closed]
(2 answers)
Closed 5 years ago.
Am seeking clarification on how the bound of a particular function is being determined.
E.g 1: A(n) = log(2^n) + n^(1/3) + 1000
Would I be right to say that the last 2 terms can be "ignored" as they are insignificant as compared to the first? And thus the bound is O(2^n)?
E.g 2: B(n) = n + (1/2)*n + (1/3)*n + (1/4)*n + ... + 1
I am more uncertain about this one, but I would give a guess that it would be O(n)? 1 is ignored (as per reasoning for 1000 in E.g. 1), that's what I'm sure.
Was also thinking if the fractions in E.g. 2 are modified, such that the denominators run in different patterns instead (e.g. (1/2)*n + (1/4)*n) + (1/8)*n...), would the order of growth be faster/slower than E.g. 2?
Appreciate any guidance available! Thank you!
E.g 1: A(n) = log(2^n) + n^(1/3) + 1000
Here log(2^n) = n which is bigger than n^(1/3) so by the property of Order function A(n0 = O(n)
E.g 2: B(n) = n + (1/2)*n + (1/3)*n + (1/4)*n + ... 1
= n*(1 + 1/2 + 1/3 + 1/4 ....+ 1/n)
Now (1 + 1/2 + 1/3 + 1/4 ....) you can approximate by thinking it is integration of dx/x from 1 to n which comes to be log(n) making the resulting Order = O(nlgn)
E.g 2 Modified = n + (1/2)*n + (1/4)*n + (1/8)*n +.....
= n( 1+ 1/2 + 1/4 +1/8...) [GP series]
= n / (1/(1-1/2))
= 2n
So it becomes O(n)
E.g 1: A(n) = log(2^n) + n^(1/3) + 1000 Would I be right to say that the last 2 terms can be "ignored" as they are insignificant as compared to the first? And thus the bound is O(2^n)?
If you simplify the expression, you get A(n) = n*log(2) + n^(1/3) + 1000. The last two terms grow more slowly than the first, n*log(2), which is simply O(n). Hence A(n) is O(n).
E.g 2: B(n) = n + (1/2)*n + (1/3)*n + (1/4)*n + ... + 1 I am more uncertain about this one, but I would give a guess that it would be O(n)? 1 is ignored (as per reasoning for 1000 in E.g. 1), that's what I'm sure.
This one is tricky because it involves an infinite series. If you only had n + (1/2)*n + (1/3)*n + (1/4)*n, then it would be equivalent to a*n with some fractional a, and that is O(n). However, since the expression is a divergent infinite series known as the harmonic series, you cannot conclude that B(n) is O(n). In fact, B(n) can be simplified as S_k(1/i)*n + 1 as k tends to infinity, with S_k(1/i) the sum of 1/i with i from 1 to k. And because S_k diverges as k tends to infinity, B(n) also tends to infinity, assuming n>0.
In the end, B(n) in not bounded. It doesn't have a proper order of growth.
Edit: if B(n) does not contain an infinite series but instead stops at (1/n)*n, which is the last 1 in your expression, then the answer is different.
The partial sums of the harmonic series have logarithmic growth. That means that B(n)/n, which is exactly the partial sum, up to n, of the harmonic series, is O(log n). In the end, B(n) is simply O(n log n).
Related
Say there is an algorithm with input of size n. On the first iteration, it performs n computations, then is left with a problem instance of size floor(n/2) - for the worst case. Now it performs floor(n/2) computations. So, for example, an input of n=25 would see it perform 25+12+6+3+1 computations until an answer is reached, which is 47 total computations. How do you put this into Big O form to find worst case complexity?
You just need to write the corresponding recurrence in a formal manner:
T(n) = T(n/2) + n = n + n/2 + n/4 + ... + 1 =
n(1 + 1/2 + 1/4 + ... + 1/n) < 2 n
=> T(n) = O(n)
so I got this algorithm I need to calculate its time complexity
which goes like
for i=1 to n do
k=i
while (k<=n) do
FLIP(A[k])
k = k + i
where A is an array of booleans, and FLIP is as it is, flipping the current value. therefore it's O(1).
Now I understand that the inner while loop should be called
n/1+n/2+n/3+...+n/n
If I'm correct, but is there a formula out there for such calculation?
pretty confused here
The more exact computation is T(n) \sum((n-i)/i) for i = 1 to n (because k is started from i). Hence, the final sum is n + n/2 + ... + n/n - n = n(1 + 1/2 + ... + 1/n) - n, approximately. We knew 1 + 1/2 + ... + 1/n = H(n) and H(n) = \Theta(\log(n)). Hence, T(n) = \Theta(n\log(n)). The -n has not any effect on the asymptotic computaional cost, as n = o(n\log(n)).
Lets say we want to calculate sum of this equation
n + n / 2 + n / 3 + ... + n / n
=> n ( 1 + 1 / 2 + 1 / 3 + ..... + 1 / n )
Then in bracket ( 1 + 1 / 2 + 1 / 3 + ... + 1 / n ) this is a well known Harmonic series and i am afraid there is no proven formula to calculate Harmonic series.
The given problem boils down to calculate below sum -Sum of harmonic series
Although this sum can't be calculated accurately, however you can still find asymptotic upper bound for this sum, which is approximately O(log(n)).
Hence answer to above problem will be - O(nlog(n))
Somehow, I find that it is much harder to derive Big O complexities for recursive algorithms compared to iterative algorithms. Do provide some insight about how I should go about solving these 2 questions.
*assume that submethod has linear complexity
def myMethod(n)
if (n>0)
submethod(n)
myMethod(n/2)
end
end
def myMethod(k,n)
if(n>0)
submethod(k)
myMethod(k,n/2)
end
end
For your first problem, the recurrence will be:
T(n) = n + T(n/2)
T(n/2) = n/2 + T(n/4)
...
...
...
T(2) = 2 + T(1)
T(1) = 1 + T(0) // assuming 1/2 equals 0(integer division)
adding up we get:
T(n) = n + n/2 + n/4 + n/8 + ..... 1 + T(0)
= n(1 + 1/2 + 1/4 + 1/8 .....) + k // assuming k = T(0)
= n*1/(1 - 1/2) ( sum of geometric series a/(1-r) when n tends to infinity)
= 2n + k
Therefore, T(n) = O(n). Remember i have assumed n tends to infinity ,cause this is what we do in Asymptotic analysis.
For your second problem its easy to see that, we perform k primitive operations everytime till n becomes 0. This happens log(n) times. Therefore, T(n) = O(k*log(n))
All you need to do is count how many times a basic operation is executed. This is true for analysing any kind of algorithm. In your case, we will count the number of times submethod is called.
You could break-down the running time of call myMethod(n) to be 1 + myMethod(n / 2). Which you can further break down to 1 + (1 + myMethod(n / 4)). At some point you will reach the base case, in log(n)th step. That gives you an algorithm of log(n).
The second one is no different, since k is constant all the time, it will again take log(n) time, assuming submethod takes constant time regardless of its input.
Func(n)
{
i = n
while(i>=1)
g(i);
i = i/3;
}
what is the complexity of this algorithm?
(while the complexity of g(n) is theta(n²))
I assumed for bigger n's you say that the complexity is
n² + (n/3)² + (n/3²)² + (n/3³)²..... to infinity.
And the answer is theta(n²).
Is that true?
As you have observed, the loop runs as follows.
Iteration 1: n^2 = n^2/3^0
Iteration 2: (n/3)^2 = n^2/3^2
Iteration 3: (n/3^2)^2 = n^2/3^4
Iteration 4: (n/3^3)^2 = n^2/3^6
...
Iteration k: (n/3^(k-1))^2 = n^2/3^(2*(k-1))
Using the formula for Summation of a geometric progression, we get that total time taken is
T(iteration1) + T(iteration2) + ... + T(Iterationk)
term 1 = n^2
ratio = 1/9
sum = 9 * n^2 / 8
When K is a large number that can be assumed to be infinite.
Since Big O notation ignores constants,
O( 9* n^2 /8) = O(n^2)
Lets look at the series that we have got in our hands.
=> n² + (n²/3) + (n/3)² + (n/3²)² + (n/3³)².....
taking n² common
=> n² * [ 1 + (1/3) + (1/3)² + (1/3²)² + (1/3³)²..... ]
As [ 1 + (1/3) + (1/3)² + (1/3²)² + (1/3³)²..... ] is a decreasing series, it is equal to 1.
Thus the answer is O(n²)
EDIT 1:
Prove for the sum of series [ 1 + (1/3) + (1/3)² + (1/3²)² + (1/3³)²..... ], is below.
Rigorously speaking, i is an integer and it soon becomes exactly 0 (after floor(log3(n)) iterations), so there is no reason to go to infinity.
Anyway, considering i as a rational results in an approximation of the true formula that doesn't change the asymptotic behavior, still O(n²). The approximation appears in two ways
i/3 may differ from floor(i/3);
one can add to infinity; the terms smaller than 1 just add to 4/3 at worse, which is completely neglectible.
I am to show that log(n!) = Θ(n·log(n)).
A hint was given that I should show the upper bound with nn and show the lower bound with (n/2)(n/2). This does not seem all that intuitive to me. Why would that be the case? I can definitely see how to convert nn to n·log(n) (i.e. log both sides of an equation), but that's kind of working backwards.
What would be the correct approach to tackle this problem? Should I draw the recursion tree? There is nothing recursive about this, so that doesn't seem like a likely approach..
Remember that
log(n!) = log(1) + log(2) + ... + log(n-1) + log(n)
You can get the upper bound by
log(1) + log(2) + ... + log(n) <= log(n) + log(n) + ... + log(n)
= n*log(n)
And you can get the lower bound by doing a similar thing after throwing away the first half of the sum:
log(1) + ... + log(n/2) + ... + log(n) >= log(n/2) + ... + log(n)
= log(n/2) + log(n/2+1) + ... + log(n-1) + log(n)
>= log(n/2) + ... + log(n/2)
= n/2 * log(n/2)
I realize this is a very old question with an accepted answer, but none of these answers actually use the approach suggested by the hint.
It is a pretty simple argument:
n! (= 1*2*3*...*n) is a product of n numbers each less than or equal to n. Therefore it is less than the product of n numbers all equal to n; i.e., n^n.
Half of the numbers -- i.e. n/2 of them -- in the n! product are greater than or equal to n/2. Therefore their product is greater than the product of n/2 numbers all equal to n/2; i.e. (n/2)^(n/2).
Take logs throughout to establish the result.
Sorry, I don't know how to use LaTeX syntax on stackoverflow..
See Stirling's Approximation:
ln(n!) = n*ln(n) - n + O(ln(n))
where the last 2 terms are less significant than the first one.
For lower bound,
lg(n!) = lg(n)+lg(n-1)+...+lg(n/2)+...+lg2+lg1
>= lg(n/2)+lg(n/2)+...+lg(n/2)+ ((n-1)/2) lg 2 (leave last term lg1(=0); replace first n/2 terms as lg(n/2); replace last (n-1)/2 terms as lg2 which will make cancellation easier later)
= n/2 lg(n/2) + (n/2) lg 2 - 1/2 lg 2
= n/2 lg n - (n/2)(lg 2) + n/2 - 1/2
= n/2 lg n - 1/2
lg(n!) >= (1/2) (n lg n - 1)
Combining both bounds :
1/2 (n lg n - 1) <= lg(n!) <= n lg n
By choosing lower bound constant greater than (1/2) we can compensate for -1 inside the bracket.
Thus lg(n!) = Theta(n lg n)
Helping you further, where Mick Sharpe left you:
It's deriveration is quite simple:
see http://en.wikipedia.org/wiki/Logarithm -> Group Theory
log(n!) = log(n * (n-1) * (n-2) * ... * 2 * 1) = log(n) + log(n-1) + ... + log(2) + log(1)
Think of n as infinitly big. What is infinite minus one? or minus two? etc.
log(inf) + log(inf) + log(inf) + ... = inf * log(inf)
And then think of inf as n.
Thanks, I found your answers convincing but in my case, I must use the Θ properties:
log(n!) = Θ(n·log n) => log(n!) = O(n log n) and log(n!) = Ω(n log n)
to verify the problem I found this web, where you have all the process explained: http://www.mcs.sdsmt.edu/ecorwin/cs372/handouts/theta_n_factorial.htm
http://en.wikipedia.org/wiki/Stirling%27s_approximation
Stirling approximation might help you. It is really helpful in dealing with problems on factorials related to huge numbers of the order of 10^10 and above.
This might help:
eln(x) = x
and
(lm)n = lm*n
If you reframe the problem, you can solve this with calculus! This method was originally shown to me via Arthur Breitman https://twitter.com/ArthurB/status/1436023017725964290.
First, you take the integral of log(x) from 1 to n it is n*log(n) -n +1. This proves a tight upper bound since log is monotonic and for every point n, the integral from n to n+1 of log(n) > log(n) * 1. You can similarly craft the lower bound using log(x-1), as for every point n, 1*log(n) > the integral from x=n-1 to n of log(x). The integral of log(x) from 0 to n-1 is (n-1)*(log(n-1) -1), or n log(n-1) -n -log(n-1)+1.
These are very tight bounds!