Could someone please explain what the correct answer should be:
Question = What is the complexity of 1^n + n^4 + 4n + 4?
Options = Exponential, Polynomial, Linear, Constant
My interpretation:
Answer should be exponential, because law of addition for O() is that the answer is the dominant term, which imo would be 1^n aka c^n (c = constant) aka exponential. But the answer is wrong and I don't get it :(
Well 1^n is always 1. Therefore, 1^n + n^4 + 4n + 4 can be simplified to 1 + n^4 + 4n + 4. Consequently, O(n^4).
So the correct answer is Polynomial.
Related
I am wondering what is the time complexity of this recurrence relation.
Sorry if I'm writing with images, but I needed to include some math:
T(n) = T(n-1) + f(n)
Means
T(n) = T(0) + Sum_from_i=1_to_n_of( f(i) )
In your case, that's:
T(n) = T(0) + 02 + 12 + 22 ... (n-1)2
If you don't know immediately from discrete calculus that the sum comes out to O(n3), you can notice that there are n terms, with the largest being (n-1)2, and there more than (n/3) terms that are >= (n/2)2.
I think the complexity of this recursion is O(n^2/3)` by change variable and induction. but I'm not sure. Is this solution correct?
This is a fascinating recurrence and it does not solve to Θ(n). Rather, it appears to solve to Θ(n2/3).
To give an intuition for why this isn't likely to be Θ(n), let's imagine that we're dealing with a really, really large value of n. Then since
T(n) = (nT(√n) + n)1/2
under the assumption that T(√n) ≈ √n, we'd get that
T(n) = (n√n + n)1/2
= (n3/2 + n)1/2
≈ n3/4.
In other words, assuming that T(n) = Θ(n) would give us a different value of T(n) as n gets large.
On the other hand, let's assume that T(n) = Θ(n2/3). Then the same calculation gives us that
T(n) = (nT(n) + n)1/2
= (n · n2/3 + n)1/2
≈ (n4/3)1/2
= n2/3,
which is consistent with itself.
To validate this, I wrote a short program that printed out different values of T(n) given different inputs and plotted the results. Here's the version of T(n) that I wrote up:
double T(double n) {
if (n <= 2) return n;
return sqrt(n * T(sqrt(n)) + n);
}
I decided to use 2 as a base case, since repeatedly taking square roots will never let n drop to one. I also decided to use real-valued arguments rather than discrete integer values just to make the math easier.
If you plot the values of T(n), you get this curve:
.
This doesn't look like what I'd expect from a linear plot. To figure out what this was, I plotted it on a log/log plot, which has the nice property that all polynomial functions get converted to straight lines whose slope is equal to the exponent. Here's the result:
I consulted my Handy Neighborhood Regression Software and asked it to determine the slope of this line. Here's what it gave back:
Slope: 0.653170918815869
R2: 0.999942627574643
That's a very good fit, and the slope of 0.653 is pretty close to 2/3. So that's more empirical evidence supporting that the recurrence solves to Θ(n2/3).
All that's left to do now is to work out the math. We'll solve this recurrence using a series of substitutions.
First, I'm generally not that comfortable working with exponents in the way that this recurrence uses them, so let's take the log of both sides. (Throughout this exposition, I'll use lg n to mean log2 n).
lg T(n) = lg (nT(√n) + n)1/2
= (1/2) lg (nT(√n) + n)
= (1/2) lg(T(√n) + 1) + (1/2)lg n
≈ (1/2) lg T(√n) + (1/2) lg n
Now, let's define S(n) = lg T(n). Then we have
S(n) = lg T(n)
≈ (1/2) lg T(√ n) + (1/2) lg n
= (1/2) S(√ n) + (1/2) lg n
That's a lot easier to work with, though we still have the problem of the recurrence shrinking by powers each time. To address this, let's do one more substitution, which is a fairly common one when working with these sorts of expressions. Let's define R(n) = S(2n). Then we have that
R(n) = S(2n)
≈ (1/2)S(√2n) + (1/2) lg 2n
= (1/2)S(2n/2) + (1/2) n
= (1/2) R(n / 2) + (1/2) n
Great! All that's left to do now is to solve R(n).
Now, there is a slight catch here. We could immediately use the Master Theorem to conclude that R(n) = Θ(n). The problem with this is that just knowing that R(n) = Θ(n) won't allow us to determine what T(n) is. Specifically, let's suppose that we just know R(n) = Θ(n). Then we could say that
S(n) = S(2lg n) = R(lg n) = Θ(log n)
to get that S(n) = Θ(log n). However, we get stuck when trying to solve for T(n) in terms of S(n). Specifically, we know that
T(n) = 2S(n) = 2Θ(log n),
but we cannot go from this to saying that T(n) = Θ(n). The reason is that the hidden coefficient in the Θ(log n) is significant here. Specifically, if S(n) = k lg n, then we have that
2k lg n = 2lg nk = nk,
so the leading coefficient of the logarithm will end up determining the exponent on the polynomial. As a result, when solving R, we need to determine the exact coefficient of the linear term, which translates into the exact coefficient of the logarithmic term for S.
So let's jump back to R(n), which we know is
R(n) ≈ (1/2) R(n/2) + (1/2)n.
If we iterate this a few times, we see this pattern:
R(n) ≈ (1/2) R(n/2) + (1/2)n
≈ (1/2)((1/2) R(n/4) + (1/4)n) + (1/2)n
≈ (1/4)R(n/4) + (1/8)n + (1/2)n
≈ (1/4)((1/2)R(n/8) + n/8) + (1/8)n + (1/2)n
≈ (1/8)R(n/8) + (1/32)n + (1/8)n + (1/2)n.
The pattern appears to be that, after k iterations, we get that
R(n) ≈ (1/2k)R(n/2k) + n(1/2 + 1/8 + 1/32 + 1/128 + ... + 1/22k+1).
This means we should look at the sum
(1/2) + (1/8) + (1/32) + (1/128) + ...
This is
(1/2)(1 + 1/4 + 1/16 + 1/64 + ... )
which, as the sum of a geometric series, solves to
(1/2)(4/3)
= 2/3.
Hey, look! It's the 2/3 we were talking about earlier. This means that R(n) works out to approximately (2/3)n + c for some constant c that depends on the base case of the recurrence. Therefore, we see that
T(n) = 2S(n)
= 2S(2lg n)
= 2R(lg n)
≈ 2(2/3)lg n + c
= 2lg n2/3 + c
= 2c 2lg n2/3
= 2c n2/3
= Θ(n2/3)
Which matches the theoretically predicted and empirically observed values from earlier.
This was a very fun problem to work through and I'll admit I'm surprised by the answer! I am a bit nervous, though, that I may have missed something when going from
lg T(n) = (1/2) lg (T(√n) + 1) + (1/2) lg n
to
lg T(n) ≈ (1/2) lg T(√ n) + (1/2) lg n.
It's possible that this +1 term actually introduces some other term into the recurrence that I didn't recognize. For example, is there an O(log log n) term that arises as a result? That wouldn't surprise me, given that we have a recurrence that shrinks by a square root. However, I've done some simple data explorations and I'm not seeing any terms in there that look like there's a double log involved.
Hope this helps!
We know that:
T(n) = sqrt(n) * sqrt(T(sqrt(n)) + 1)
Hence:
T(n) < sqrt(n) * sqrt(T(sqrt(n)) + T(sqrt(n)))
1 is replaced by T(sqrt(n)). So,
T(n) < sqrt(2) * sqrt(n) * sqrt(T(sqrt(n))
Now, to find an upper bound we need to solve the following recurrent relation:
G(n) = sqrt(2n) * sqrt(G(sqrt(n))
To solve this, we need to expand it (suppose n = 2^{2^k} and T(1) = 1):
G(n) = (2n)^{1/2} * (2n)^{1/8} * (2n)^{1/32} * ... * (2n)^(1/2^k) =>
G(n) = (2n)^{1/2 + 1/8 + 1/32 + ... + 1/2^k} =
If we take a factor 1/2 from 1/2 + 1/8 + 1/32 + ... + 1/2^k we will have 1/2 * (1 + 1/4 + 1/8 + ... + 1/2^{k-1}).
As we know that 1 + 1/4 + 1/8 + ... + 1/2^{k-1} is a geometric series with a ratio 1/4, it is equal to 4/3 at infinity. Therefore G(n) = Theta(n^{2/3}) and T(n) = O(n^{2/3}).
Notice that as sqrt(n) * sqrt(T(sqrt(n)) < T(n), we can show similar to the previous case that T(n) = Omega(n^{2/3}). It means T(n) = Theta(n^{2/3}).
Is T (n) = T (n/2) + 2^n possible on master method?
How to measure the time complexity of this recurrence.Thanks
Well, do a few expansion:
T(n) = 2^n + T(n/2)
= 2^n + 2^(n/2) + T(n/4)
= 2^n + 2^(n/2) + 2^(n/4) + T(n/8)
...etc.
Obviously, the complexity is at least Ω(2^n), since there's a 2^n in the expansion.
Furthermore, the terms added in each expansion get small really fast. You probably already know that:
2^n + 2^(n-1) + 2^(n-2)... = O(2^n)
You have:
2^n + 2^(n/2) + 2^(n/4)...
and those terms, except for the first, are all smaller until n<2 (around a base case), so your sequence is in O(2^n) as well, i.e., the lower bound is tight and the recurrence is in Θ(2^n)
I am to show that log(n!) = Θ(n·log(n)).
A hint was given that I should show the upper bound with nn and show the lower bound with (n/2)(n/2). This does not seem all that intuitive to me. Why would that be the case? I can definitely see how to convert nn to n·log(n) (i.e. log both sides of an equation), but that's kind of working backwards.
What would be the correct approach to tackle this problem? Should I draw the recursion tree? There is nothing recursive about this, so that doesn't seem like a likely approach..
Remember that
log(n!) = log(1) + log(2) + ... + log(n-1) + log(n)
You can get the upper bound by
log(1) + log(2) + ... + log(n) <= log(n) + log(n) + ... + log(n)
= n*log(n)
And you can get the lower bound by doing a similar thing after throwing away the first half of the sum:
log(1) + ... + log(n/2) + ... + log(n) >= log(n/2) + ... + log(n)
= log(n/2) + log(n/2+1) + ... + log(n-1) + log(n)
>= log(n/2) + ... + log(n/2)
= n/2 * log(n/2)
I realize this is a very old question with an accepted answer, but none of these answers actually use the approach suggested by the hint.
It is a pretty simple argument:
n! (= 1*2*3*...*n) is a product of n numbers each less than or equal to n. Therefore it is less than the product of n numbers all equal to n; i.e., n^n.
Half of the numbers -- i.e. n/2 of them -- in the n! product are greater than or equal to n/2. Therefore their product is greater than the product of n/2 numbers all equal to n/2; i.e. (n/2)^(n/2).
Take logs throughout to establish the result.
Sorry, I don't know how to use LaTeX syntax on stackoverflow..
See Stirling's Approximation:
ln(n!) = n*ln(n) - n + O(ln(n))
where the last 2 terms are less significant than the first one.
For lower bound,
lg(n!) = lg(n)+lg(n-1)+...+lg(n/2)+...+lg2+lg1
>= lg(n/2)+lg(n/2)+...+lg(n/2)+ ((n-1)/2) lg 2 (leave last term lg1(=0); replace first n/2 terms as lg(n/2); replace last (n-1)/2 terms as lg2 which will make cancellation easier later)
= n/2 lg(n/2) + (n/2) lg 2 - 1/2 lg 2
= n/2 lg n - (n/2)(lg 2) + n/2 - 1/2
= n/2 lg n - 1/2
lg(n!) >= (1/2) (n lg n - 1)
Combining both bounds :
1/2 (n lg n - 1) <= lg(n!) <= n lg n
By choosing lower bound constant greater than (1/2) we can compensate for -1 inside the bracket.
Thus lg(n!) = Theta(n lg n)
Helping you further, where Mick Sharpe left you:
It's deriveration is quite simple:
see http://en.wikipedia.org/wiki/Logarithm -> Group Theory
log(n!) = log(n * (n-1) * (n-2) * ... * 2 * 1) = log(n) + log(n-1) + ... + log(2) + log(1)
Think of n as infinitly big. What is infinite minus one? or minus two? etc.
log(inf) + log(inf) + log(inf) + ... = inf * log(inf)
And then think of inf as n.
Thanks, I found your answers convincing but in my case, I must use the Θ properties:
log(n!) = Θ(n·log n) => log(n!) = O(n log n) and log(n!) = Ω(n log n)
to verify the problem I found this web, where you have all the process explained: http://www.mcs.sdsmt.edu/ecorwin/cs372/handouts/theta_n_factorial.htm
http://en.wikipedia.org/wiki/Stirling%27s_approximation
Stirling approximation might help you. It is really helpful in dealing with problems on factorials related to huge numbers of the order of 10^10 and above.
This might help:
eln(x) = x
and
(lm)n = lm*n
If you reframe the problem, you can solve this with calculus! This method was originally shown to me via Arthur Breitman https://twitter.com/ArthurB/status/1436023017725964290.
First, you take the integral of log(x) from 1 to n it is n*log(n) -n +1. This proves a tight upper bound since log is monotonic and for every point n, the integral from n to n+1 of log(n) > log(n) * 1. You can similarly craft the lower bound using log(x-1), as for every point n, 1*log(n) > the integral from x=n-1 to n of log(x). The integral of log(x) from 0 to n-1 is (n-1)*(log(n-1) -1), or n log(n-1) -n -log(n-1)+1.
These are very tight bounds!
2^n + 6n^2 + 3n
I guess it's just O(2^n), using the highest order term, but what is the formal approach to go about proving this?
You can prove that 2^n + n^2 + n = O(2^n) by using limits at infinity. Specifically, f(n) is O(g(n)) if lim (n->inf.) f(n)/g(n) is finite.
lim (n->inf.) ((2^n + n^2 + n) / 2^n)
Since you have inf/inf, an indeterminate form, you can use L'Hopital's rule and differentiate the numerator and the denominator until you get something you can work with:
lim (n->inf.) ((ln(2)*2^n + 2n + 1) / (ln(2)*2^n))
lim (n->inf.) ((ln(2)*ln(2)*2^n + 2) / (ln(2)*ln(2)*2^n))
lim (n->inf.) ((ln(2)*ln(2)*ln(2)*2^n) / (ln(2)*ln(2)*ln(2)*2^n))
The limit is 1, so 2^n + n^2 + n is indeed O(2^n).
Please see:
Big O Notation Homework--Code Fragment Algorithm Analysis?
Big O, how do you calculate/approximate it?
Can someone please explain the difference between Big-O and Little-O Notation?
Big-O for Eight Year Olds? (not meant to be insulting)