I'm currently in the progress of getting familiar with asymptotic notations for Big Theta, and I was wondering if I could have someone double check my practice.
Problem: f(n)=2(3n^3+2n+1)^3, find big theta notation, assume conditions have been met for big O and omega.
My solution: 2 * (3n^(3+3) + 2n^3 + 1^3), I add the exponent 3 to the power inside
6n^6 + 2n^3 + 2, I multiply the constant 2 to the values in parenthesis
n^6, f(n)= theta(n^6), I take the highest power and drop the constants
Can anyone please let me know if I did this correctly? Thanks.
It is quiet simple.
We say that f(n) is O(g(n)) if exists a constant C that the limit of C * g(n) / f(n) is constant.
We say that f(n) is Big Omega(g(n)) (sorry, no LaTeX here) if if exists a constant C that the limit of C * f(n) / g(n) is a constant (kind of "the opposite" of the big O notation)
We say that a function is theta(g(n)) if both (1) and (2) co-exist.
Now, we need to find a function that behaves almost as f(n), that will suppress it from above with a constant C, and will be suppressed by it with another constant D.
It is easy to prove that when talking about f(n) that is a polynomial function, it is always theta(n^k) where k is the degree ("the highest power") of that polynomial.
However, you need to know how to prove it!
Anyhow, you were wrong when expanding f(n). The real expansion is:
f(n) = 54 n^9 + 108 n^7 + 54 n^6 + 72 n^5 + 72 n^4 + 34 n^3 + 24 n^2 + 12 n + 2
with C = 55 and D = 53 you get 53 * n^9 <= f(n) <= 55 * n^9 when n is large enough, (or, the two limits (1) and (2) are constants)
Related
Example-3 Find upper bound for f(n) = n^4 + 100n^2 + 50
Solution: n^4 + 100n^2 + 50 ≤ 2n^4, for all n ≥ 11
∴ n^4 + 100n^2 + 50 = O(n^4 ) with c = 2 and n0 = 11
In the above question the solution says n>11 and n-nought is 11.
can anybody explain why is it 11?
for reference - this is a problem from the Data Structures and Algorithms Made Easy by Narasimha Karumanchi
f(n) = n^4 + 100n^2 + 50
Intuitively, n^4 grows very fast; n^2 grows less fast than n^4; and 50 doesn't grow at all.
However, for small values of n, n^4 < 50; additionally, the n^2 term has a factor 100 in front of it. Because of this factor, for small values of n, n^4 < 100 n^2.
But because we have the intuition that n^4 grows much faster than n^2, we expect that, for n big enough, 100 n^2 + 50 < n^4.
In order to assert and prove this claim, we need to be more precise on what "for n big enough" means. Your textbook found an exact value; and they claimed: for n ≥ 11, 100 n^2 + 50 < n^4.
How did they find that? Maybe they solved the inequality for n. Or maybe they just intuited it by noticing that:
100 n^2 = 10 * 10 * n * n`
n^4 = n * n * n * n
Thus n^4 is going to be the bigger of the two as soon as n is bigger than 10.
In conclusion: as soon as n ≥ 11, f(n) < 2 n^4. Thus, f(n) satisfies the textbook definition for f(n) = O(n^4).
It doesn't say that n>11 it says that n4 + 100n2 + 50 ≤ 2n4, for all n ≥ 11.
Is it true? You can substitute n for 11 in the formula and check it yourself.
How was 11 obtained? By solving the inequality.
It is not finding an upper bound for a function. It is an asymptotic analysis of a function with big-O notation. Hence, the constant c = 11 does not matter for the analysis, and if you can show the inequality is valid for all n greater than any constant, for instance c = 100, that will be accepted. By the way, you can show that it is true for all n > 11 by the mathematical induction.
f(n) = 4 * 2n + 4n + 20n5
So, g(n) = 4n
Now our f(n) = O(g(n))
4 * 2n + 4n + 20n5 ≤ c*4n
How do we do this? I know how to do it for simple cases, but this one is far more complex. Would it go along the lines of removing the constant 4 and 20n5 to then have 2n + 4n ≤ c*4n?
Or would it be for any c > 4*2n + 20n5. It feels like a lame answer, so i'm going to assume i'm wrong. Would prefer if someone hinted at the idea of how to solve these problems rather than give me the answer, thank you.
Hint / preparations
In the context of asymptotic analysis and, in this case, Big-O notation specifically; generally when wanting to prove that inequalities such as
4 * 2^n + 4^n + 20n^5 ≤ c*4^n, (+)
for some constant c > 0,
for n larger than some constant n0; n > n0
holds, we approach the left hand side expression term by term. Since we're free to choose any constants c and n0 to show that (+) holds, we can always express the lower order terms as less or equal to (≤) the higher order term by making n sufficiently large, e.g., choosing the value of n0 as we see fit.
Solution (spoilers ahead!)
Below follows one way to show that (+) holds for some set of positive constants c and n0. Since you only asked for hints, I suggest you start with the section above, and return to this section in case you get stuck or want to verify the derivation you ended up using.
Term by term analysis (in terms of 4^n) of the left hand side expression of(+)` follows.
Term 4 * 2^n:
4 * 2^n = 4^n <=> (2*2)*2^n = (2^2)^n <=> 2^(n+2) = 2^(2n)
<=> n+2 = 2n => n = 2
=> 4 * 2^n ≤ 4^n for n ≥ 2 (i)
Term 4^n: Trivial
Term 20n^5:
for which n is 20 * n^5 = 4^n?
Graphical solution:
=> 20 * n^5 ≤ 4^n for n ≥~ 10.7 (choose 11) (ii)
Inserting inequalities (i) and (ii) in the lhs of (+) yields:
4 * 2^n + 4^n + 20n^5 ≤ 4^n + 4^n + 4^n = 3*4^n
^
for n>max(2,11)=11 <-- choice of n0 |
choice of c
Hence, we have showed that (+) holds for constants n0 = 11 and c=3. Naturally, the choice of these constants is not unique (in fact, if such constants exists, an infinite amount of them exists). Subsequently, the lhs of (+) is in O(4^n).
Now, I note that your title mentions Big-Θ (whereas your question covers only Big-O). For deriving that lhs of (+) is Θ(4^n), we need to find also a lower asymptotic bound on the lhs of (+) in terms of 4^n. Since n > 0, this is, in this case, quite trivial:
4 * 2^n + 4^n + 20n^5 ≥ c2*4^n ? for n > n0 ? (++)
=> 4 * 2^n + 4^n + 20n^5 ≥ 4^n, for n > 0
I.e., in addition to showing that (+) holds (which implies O(4^n)), we've shown that (++) holds for e.g. c2 = 1 and (re-use) n0 = 11, which implies that lhs of (+) is Θ(4^n).
One way to approach an asymptotic analysis of a function such as the left hand side of (+) would be to make use of the somewhat rigorous term-by-term analysis shown in this solution. In practice, however, we know that 4^n will quickly dominate the lower order terms, so we could've just chosen a somewhat large n0 (say 100) and tested, term by term, if the lower order terms could be replaced by the higher order term with less or equal to (≤) relation, given n>n0. Or, given in what context we need to make use of our asymptotic bounds, we could just glance at the function and, without rigour, directly state that the asymptotic behaviour of the function is naturally O(4^n), due to this being the dominant term. This latter method should, imo, only be used after one has grasped how to formally analyse the asymptotic behaviour of functions and algorithms in the context of Big-O/-Omega and -Theta notation.
The formal definition is
O(g(n)) = {f(n) | ∃c, n₀ : 0 ≤ f(n) ≤ c g(n), ∀ n ≥ n₀}
But when you want to check if f(x) ∈ O(g(n)), you can use the simpler
f(n)
lim sup ────── ≤ c
n → ∞ g(n)
In this case,
4*2ⁿ + 4ⁿ + 20n⁵
lim sup ────────────────── = 1
n → ∞ 4ⁿ
So yes, we can choose for example c = 1.
How do you work this out? do you get c first which is the ratio of the two functions then with the ratio find the range of n ? how can you tell ? please explain i'm really lost, Thanks.
Example 1: Prove that running time T(n) = n^3 + 20n + 1 is O(n^3)
Proof: by the Big-Oh definition,
T(n) is O(n^3) if T(n) ≤ c·n^3 for some n ≥ n0 .
Let us check this condition:
if n^3 + 20n + 1 ≤ c·n^3 then 1 + 20/n^2 + 1/n^3 <=c .
Therefore,
the Big-Oh condition holds for n ≥ n0 = 1 and c ≥ 22 (= 1 + 20 + 1). Larger
values of n0 result in smaller factors c (e.g., for n0 = 10 c ≥ 1.201 and so on) but in
any case the above statement is valid.
I think the trick you're seeing is that you aren't thinking of LARGE numbers. Hence, let's take a counter example:
T(n) = n^4 + n
and let's assume that we think it's O(N^3) instead of O(N^4). What you could see is
c = n + 1/n^2
which means that c, a constant, is actually c(n), a function dependent upon n. Taking N to a really big number shows that no matter what, c == c(n), a function of n, so it can't be O(N^3).
What you want is in the limit as N goes to infinity, everything but a constant remains:
c = 1 + 1/n^3
Now you can easily say, it is still c(n)! As N gets really, really big 1/n^3 goes to zero. Hence, with very large N in the case of declaring T(n) in O(N^4) time, c == 1 or it is a constant!
Does that help?
I learned that,using Big O notation
O(f(n)) + O(g(n)) -> O(max(f(n),g(n))
O( f(n) )* O( g(n)) -> O( f(n) g(n))
but now, I have this equation for running time T for input size N
T(N) = O(N^2) // O of N square
I need to find the ratio T(2N) / T(N)
I tried this
T(2N) / T(N) --> O((2N)^2) /O( N^2) --> 4
Is this correct? Or is the above division invalid?
I would also say this is incorrect. My intuition was that if T(N) is O(N^2), then T(2N)/T(N) is O(1), consistent with your suggestion that T(2N)/T(N) = 4. I think however that the intuition is wrong.
Consider a counter-example.
Let T(N) be 1 if N is odd, and N^2 if N is even, as below.
This is clearly O(N^2), because we can choose a constant p=1, such that T(N) ≤ pN^2 for sufficiently large N.
What is T(2N)? This is (2N)^2 = 4N^2, as below, because 2N is always even.
Hence T(2N)/T(N) is 4N^2/1 = 4N^2 when N is odd, and 4N^2/N^2=4 when N is even, as below.
Clearly T(2N)/T(N) is not 4. It is, however, O(N^2), because we can choose a constant q=4 such that T(2N)/T(N) ≤ qN^2 for sufficiently large N.
R code for the plots below
x=1:50
t1=ifelse(x%%2, 1, x^2)
plot(t1~x,type="l")
x2=2*x
t2=ifelse(x2%%2, 1, x2^2)
plot(t2~x,type="l")
ratio=t2/t1
plot(ratio~x,type="l")
This problem is an interesting one and strikes me as belonging in the realm of pure mathematics, i.e. limits of sequences and the like. I am not trained in pure mathematics and I would be nervous of claiming that T(2N)/T(N) is always O(N^2), as it might be possible to invent some rather tortuous counter-examples.
Even if T(N) = Θ(N²) (big-theta) this doesn't work. (I'm not even going to talk about big-O.)
c1 * N² <= T(N) <= c2 * N²
c1 * 4 * N² <= T(2N) <= c2 * 4 * N²
T(N) = c_a * N² + f(N)
T(2N) = c_b * 4 * N² + g(N)
For c_a and c_b somewhere between c1 and c2, and f(N) and g(N) small-o of N². (Thanks G. Bach!) And there is nothing to guarantee that the quotient will be equal to 4 since both c_a, c_b, f(N) and g(N) can be all sorts of things. For example, take c_a = 1, c_b = 2 and f(N) = g(N) = 0. Divide them and you get
T(2N)/T(N) = (2 * 4 * N²)/N² = 8
Good question!
This is incorrect.
Running time is not the same as time complexity(here Big O).Time complexity says like it can't have a running time worse that a constant times N^2.Running time can be quite different, maybe very low, maybe close to the asymptotic limit.That purely depends on the hidden constants.If someone asked you this question, it's a trick question.
Hidden constants refers to the actual number of primitive instructions carried out.So in this case the total number of operations could be:
5*N^2
or
1000*N^2.
or maybe
100*N^2+90N
or maybe just
100*N (Recall this is also O(N^2))
The factor depends on the implementation and the actual instructions carried out.This is found out from the algorithm.
So whichever the case we simply say the Big-O is O(N^2).
I'm starting to learn about Big-Oh notation.
What is an easy way for finding C and N0 for a given function?
Say, for example:
(n+1)5, or n5+5n4+10n2+5n+1
I know the formal definition for Big-Oh is:
Let f(n) and g(n) be functions mapping
nonnegative integers to real numbers.
We say that f(n) is O(g(n)) if there
is a real constant c > 0 and an
integer constant N0 >= 1
such that f(n) <= cg(n) for every integer N > N0.
My question is, what is a good, sure-fire method for picking values for c and N0?
For the given polynomial above (n+1)5, I have to show that it is O(n5). So, how should I pick my c and N0 so that I can make the above definition true without guessing?
You can pick a constant c by adding the coefficients of each term in your polynomial. Since
| n5 + 5n4 + 0n3 + 10n2 + 5n1 + 1n0 | <= | n5 + 5n5 + 0n5 + 10n5 + 5n5 + 1n5 |
and you can simplify both sides to get
| n5 + 5n4 + 10n2 + 5n + 1 | <= | 22n5 |
So c = 22, and this will always hold true for any n >= 1.
It's almost always possible to find a lower c by raising N0, but this method works, and you can do it in your head.
(The absolute value operations around the polynomials are to account for negative coefficients.)
Usually the proof is done without picking concrete C and N0. Instead of proving f(n) < C * g(n) you prove that f(n) / g(n) < C.
For example, to prove n3 + n is O(n3) you do the following:
(n3 + n) / n3 = 1 + (n / n3) = 1 + (1 / n2) < 2 for any n >= 1. Here you can pick any C >= 2 with N0 = 1.
You can check what the lim abs(f(n)/g(n)) is when n->+infitity and that would give you the constant (g(n) is n^5 in your example, f(n) is (n+1)^5).
Note that the meaning of Big-O for x->+infinity is that if f(x) = O(g(x)), then f(x) "grows no faster than g(x)", so you just need to prove that lim abs(f(x)/g(x)) exists and is less than +infinity.
It's going to depend greatly on the function you are considering. However, for a given class of functions, you may be able to come up with an algorithm.
For instance, polynomials: if you set C to any value greater than the leading coefficient of the polynomial, then you can solve for N0.
After you understand the magic there, you should also get that big-O is a notation. It means that you do not have to look for these coefficients in every problem you solve, once you made sure you understood what's going on behind these letters. You should just operate the symbols according to the notaion, according to its rules.
There's no easy generic rule to determine actual values of N and c. You should recall your calculus knowledge to solve it.
The definition to big-O is entangled with definition of the limit. It makes c satisfy:
c > lim |f(n)/g(n)|, given n approaches +infinity.
If the sequence is upper-bounded, it always has a limit. If it's not, well, then f is not O(g). After you have picked concrete c, you will have no problem finding appropriate N.