f(n) = 4 * 2n + 4n + 20n5
So, g(n) = 4n
Now our f(n) = O(g(n))
4 * 2n + 4n + 20n5 ≤ c*4n
How do we do this? I know how to do it for simple cases, but this one is far more complex. Would it go along the lines of removing the constant 4 and 20n5 to then have 2n + 4n ≤ c*4n?
Or would it be for any c > 4*2n + 20n5. It feels like a lame answer, so i'm going to assume i'm wrong. Would prefer if someone hinted at the idea of how to solve these problems rather than give me the answer, thank you.
Hint / preparations
In the context of asymptotic analysis and, in this case, Big-O notation specifically; generally when wanting to prove that inequalities such as
4 * 2^n + 4^n + 20n^5 ≤ c*4^n, (+)
for some constant c > 0,
for n larger than some constant n0; n > n0
holds, we approach the left hand side expression term by term. Since we're free to choose any constants c and n0 to show that (+) holds, we can always express the lower order terms as less or equal to (≤) the higher order term by making n sufficiently large, e.g., choosing the value of n0 as we see fit.
Solution (spoilers ahead!)
Below follows one way to show that (+) holds for some set of positive constants c and n0. Since you only asked for hints, I suggest you start with the section above, and return to this section in case you get stuck or want to verify the derivation you ended up using.
Term by term analysis (in terms of 4^n) of the left hand side expression of(+)` follows.
Term 4 * 2^n:
4 * 2^n = 4^n <=> (2*2)*2^n = (2^2)^n <=> 2^(n+2) = 2^(2n)
<=> n+2 = 2n => n = 2
=> 4 * 2^n ≤ 4^n for n ≥ 2 (i)
Term 4^n: Trivial
Term 20n^5:
for which n is 20 * n^5 = 4^n?
Graphical solution:
=> 20 * n^5 ≤ 4^n for n ≥~ 10.7 (choose 11) (ii)
Inserting inequalities (i) and (ii) in the lhs of (+) yields:
4 * 2^n + 4^n + 20n^5 ≤ 4^n + 4^n + 4^n = 3*4^n
^
for n>max(2,11)=11 <-- choice of n0 |
choice of c
Hence, we have showed that (+) holds for constants n0 = 11 and c=3. Naturally, the choice of these constants is not unique (in fact, if such constants exists, an infinite amount of them exists). Subsequently, the lhs of (+) is in O(4^n).
Now, I note that your title mentions Big-Θ (whereas your question covers only Big-O). For deriving that lhs of (+) is Θ(4^n), we need to find also a lower asymptotic bound on the lhs of (+) in terms of 4^n. Since n > 0, this is, in this case, quite trivial:
4 * 2^n + 4^n + 20n^5 ≥ c2*4^n ? for n > n0 ? (++)
=> 4 * 2^n + 4^n + 20n^5 ≥ 4^n, for n > 0
I.e., in addition to showing that (+) holds (which implies O(4^n)), we've shown that (++) holds for e.g. c2 = 1 and (re-use) n0 = 11, which implies that lhs of (+) is Θ(4^n).
One way to approach an asymptotic analysis of a function such as the left hand side of (+) would be to make use of the somewhat rigorous term-by-term analysis shown in this solution. In practice, however, we know that 4^n will quickly dominate the lower order terms, so we could've just chosen a somewhat large n0 (say 100) and tested, term by term, if the lower order terms could be replaced by the higher order term with less or equal to (≤) relation, given n>n0. Or, given in what context we need to make use of our asymptotic bounds, we could just glance at the function and, without rigour, directly state that the asymptotic behaviour of the function is naturally O(4^n), due to this being the dominant term. This latter method should, imo, only be used after one has grasped how to formally analyse the asymptotic behaviour of functions and algorithms in the context of Big-O/-Omega and -Theta notation.
The formal definition is
O(g(n)) = {f(n) | ∃c, n₀ : 0 ≤ f(n) ≤ c g(n), ∀ n ≥ n₀}
But when you want to check if f(x) ∈ O(g(n)), you can use the simpler
f(n)
lim sup ────── ≤ c
n → ∞ g(n)
In this case,
4*2ⁿ + 4ⁿ + 20n⁵
lim sup ────────────────── = 1
n → ∞ 4ⁿ
So yes, we can choose for example c = 1.
Related
According to this book, big O means:
f(n) = O(g(n)) means c · g(n) is an upper bound on f(n). Thus there exists some constant c such that f(n) is always ≤ c · g(n), for large enough n (i.e. , n ≥ n0 for some constant n0).
I have trubble understanding the following big O equation
3n2 − 100n + 6 = O(n2), because I choose c = 3 and 3n2 > 3n2 − 100n + 6;
How can 3 be a factor? In 3n2 − 100n + 6, if we drop the low order terms -100n and 6, aren't 3n2 and 3.n2 the same? How to solve this equation?
I'll take the liberty to slightly paraphrase the question to:
Why do and have the same asymptotic complexity.
For that to be true, the definition should be in effect both directions.
First:
let
Then for the inequality is always satisfied.
The other way around:
let
We have a parabola opened upwards, therefore there is again some after which the inequality is always satisfied.
Let's look at the definition you posted for f(n) in O(g(n)):
f(n) = O(g(n)) means c · g(n) is an upper bound on f(n). Thus there
exists some constant c such that f(n) is always ≤ c · g(n), for
large enough n (i.e. , n ≥ n0 for some constant n0).
So, we only need to find one set of constants (c, n0) that fulfils
f(n) < c · g(n), for all n > n0, (+)
but this set is not unique. I.e., the problem of finding the constants (c, n0) such that (+) holds is degenerate. In fact, if any such pair of constants exists, there will exist an infinite amount of different such pairs.
Note that here I've switched to strict inequalities, which is really only a matter of taste, but I prefer this latter convention. Now, we can re-state the Big-O definition in possibly more easy-to-understand terms:
... we can say that f(n) is O(g(n)) if we can find a constant c such
that f(n) is less than c·g(n) or all n larger than n0, i.e., for all
n>n0.
Now, let's look at your function f(n)
f(n) = 3n^2 - 100n + 6 (*)
Let's describe your functions as a sum of it's highest term and another functions
f(n) = 3n^2 + h(n) (**)
h(n) = 6 - 100n (***)
We now study the behaviour of h(n) and f(n), respectively:
h(n) = 6 - 100n
what can we say about this expression?
=> if n > 6/100, then h(n) < 0, since 6 - 100*(6/100) = 0
=> h(n) < 0, given n > 6/100 (i)
f(n) = 3n^2 + h(n)
what can we say about this expression, given (i)?
=> if n > 6/100, the f(n) = 3n^2 + h(n) < 3n^2
=> f(n) < c*n^2, with c=3, given n > 6/100 (ii)
Ok!
From (ii) we can choose constant c=3, given that we choose the other constant n0 as larger than 6/100. Lets choose the first integer that fulfils this: n0=1.
Hence, we've shown that (+) golds for constant set **(c,n0) = (3,1), and subsequently, f(n) is in O(n^2).
For a reference on asymptotic behaviour, see e.g.
https://www.khanacademy.org/computing/computer-science/algorithms/asymptotic-notation/a/big-o-notation
y=3n^2 (top graph) vs y=3n^2 - 100n + 6
Consider the sketch above. By your definition, 3n^2 only needs to be bigger than 3n^2 - 100n + 6 for large enough n (i.e. , n ≥ n0 for some constant n0). Let that n0 = 5 in this case (it could be something a little smaller, but it's clear which graph is bigger by n=5 so we'll just go with that).
Clearly from the graph, 3n^2 >= 3n^2 - 100n + 6 in the range we've plotted. The only way for 3n^2 - 100n + 6 to get bigger than 3n^2 then is for it to grow more steeply.
But the gradients of 3n^2 and 3n^2 - 100n + 6 are 6n and 6n - 100 respectively, so 3n^2 - 100n + 6 can't grow more steeply, therefore must always be underneath.
So your definition holds - 3n^2 - 100n + 6 <= 3n^2 for all n>=5
I am not an expert, but this looks a lot similar to what we just had in our real analysis course.
Basically if you have something like f(n) = 3n^2 − 100n + 6, the "fastest growing" term "wins" the other terms, when you have really really big n.
So in this case 3n^2 surpasses what ever 100n is, when the n is really big.
Another example would be something like f(n) = n/n^2 or f(n) = n! * n^2.
The first one gets smaller, as n simply cannot "keep up" with n^2. In the second example n! clearly grows faster than n^2, so I guess the answer for that should be f(n) = n! then, because the n^2 technically stops mattering with big n.
And terms like +6, which have no n affecting them are constants and matter even less as they cannot grow even if n grows.
It is all about what happends when n is really big. If your n is 34934854385754385463543856, then n^2 is hell of a bigger than 100n, because n^2 = n * n = 34934854385754385463543856 * 34934854385754385463543856.
The great people at MyCodeSchool.com have this introductory video on YouTube, covering the basics of Big-O, Theta, and Omega notation.
The following definition of Big-O notation is provided:
O(g(n) ) := { f(n) : f(n) ≤ cg(n) }, for all n ≥ n0
My casual understanding of the equation is as follows:
Given a function f(), which takes as its input n, there exists another function g(), whose output is always greater than or equal to the output of f()--given two conditions:
g() is multiplied by some constant c
n is greater than some lower bound n0
Is my understanding correct?
Furthermore, the following specific example was provided to illustrate Big-O:
Given:
f(n) = 5n2 + 2n + 1
Because all of the following are true:
5n2 > 2n2, for all n
5n2 > 1n2, for all n
It follows that:
c = 5 + 2 + 1 = 8
Therefore, the video concludes, f(n) ≤ 8n2 for all n ≥ 1, and g(n) = 8n2
I think maybe the video concluded that n0 must be 1, because 1 is the only positive root of the equality 8n2 = 5n2 + 2n + 1 ( Negative one-third is also a root, but n is limited to whole numbers. So, no dice there. )
Is this the standard way of computing n0 for Big-O notation?
Take the largest powered factor in your polynomial
Multiply it by the sum of the coefficients in your time function
Set their product equal to your time function
Solve for zero
Reject all roots that are not in the set of whole numbers
Any help would be greatly appreciated. Thanks in advance.
Your understanding is mostly correct, but from your wording - "I think maybe the video concluded that n0 must be 1", I have to point out that it is also valid to take n0 to be 2, or 3 etc. In fact, any number greater than 1 will satisfy the required condition, there are actually infinitely many choices for the pair (c, n0)!
The important point to note is that the values of the constant c and n0 does not really matter, all we care is the existence a pair of constants (c, n0).
The Basics
Big-O notation describes the asymptotic behavior of a given function f, it essential describes the upper bound of f when the its input value is sufficiently large.
Formally, we say that f is big-O of another function g, i.e. f(x) = O(g(x)), if there exists a positive constant c and a constant n0 such that the following inequality holds:
f(n) ≤ c g(n), for all n ≥ n0
Note that the inequality captures the idea of upper bound: f is upper-bounded by a positive multiple of g. Moreover, the "for all" condition satisfies that the upper bound holds when the input n is sufficiently large (e.g. larger than n0).
How to Pick (c, n0)?
In order to prove f(x) = O(g(x)) for given functions f, g, all we need is to pick any pair of (c, n0) such that the inequality holds, and then we are done!
There is no standard way of finding (c, n0), just use whatever mathematical tools you find helpful. For example, you may fix n0, and then find c by using Calculus to compute the maximum value of f(x) / g(x) in the interval [n0, +∞).
In your case, it appears that you are trying to prove that a polynomial of degree d is big-O of xd, the proof of the following lemma gives a way to pick (c, n0):
Lemma
If f is a polynomial of degree d, then f(x) = O(xd).
Proof: We have f(x) = ad xd + ad-1 xd-1 + ... + a1 x + a0, for each coefficient ai, we have ai ≤ |ai| (absolute value of ai).
Take c = (|ad| + |ad-1| + ... + |a1| + |a0|) , and n0 = 1, then we have:
f(x) = ad xd + ad-1 xd-1 + ... + a1 x + a0
≤ |ad| xd + |ad-1| xd-1 + ... + |a1| x + |a0|
≤ (|ad| + |ad-1| + ... + |a1| + |a0|) xd
= c xd, for all x ≥ 1
Therefore we have f(x) = O(xd)
This question already has answers here:
What is a plain English explanation of "Big O" notation?
(43 answers)
Closed 9 years ago.
The definition in a book said
The "Big-Oh" Notation
Let f(n) and g(n) be functions mapping nonnegative integers to real numbers. We say that f(n) is O(g(n)) if there is a real constant c > 0 and an real constant n0 ≥ 1 such that
f(n) ≤cg(n), for n ≥ n0.
I couldn't able to understand terminologies used in formula and definition can somebody explain in plain English.
Basically, f(n) is O(g(n)) then g(n) is proportional to the worst-case scenario of f(x).
For example, binary search is O(log n) (or O(ln n), which is equivalent). Why?
(Binary search works like this: take the middle element, and compare to the target. If it's the one, you're done. If it's bigger than the target, throw out the second half of the list and repeat on the first half; if it's smaller than the target, throw out the first half and repeat the search on the second half.)
Because you need 1 operation to find something in a list that is 3 elements long; 2 operations when it's 7 elements long; 3 if it is 15 elements long. Thus, when number of elements n is (2^x - 1) for any x, the number of operations is x; turn it around, and you'd say for number of elements n, number of operations is log_2 n. And say that each operation lasts 2 seconds (say you're comparing stuff by hand), and the worst time to search is log_2 n * 2 seconds. log_2 n can be rewritten as ln n / ln 2, so the formula becomes:
worst search time(n) = (ln n / ln 2) * 2 seconds
= (2 seconds / ln 2) * ln n
Now, 2 seconds / ln 2 is a constant; let's call it c. Let's call "search time for n elements" f(n). And let's call ln n as g(n).
We said before, if n = 3, g(3) <= c * ln 3 (because c * ln 3 is worst search time, real search time is always less or equal to that; but we could always find it on our first try). If n = 7, g(7) <= c * ln 7; etc.
The bit about n0 is just a guard that says the complexity we calculate for the small n might be a deviation, an anomaly, an exception from the rule, and if we go with big enough data (i.e. n >= n0), the rule becomes obvious and inviolate. In this case, the rule works pretty much from the start, but some algorithms might have extra costs that throw off the calculation on small numbers.
Translation to "plain English": Imagine that f(n) are g(n) function that take a positive number or zero as input, and give a real number as output (no imaginary numbers).
Big-Oh allows us to compare two functions to see if one is bounded by the other. For example, an exponential function f(n) would not be bounded by a linear function g(n), so f(n) would not be O(g(n)).
We can say that f(n) is O(g(n)) if the following is possible: f(n) ≤ c * g(n) for n ≥ n0. If there is some way to solve the equation by plugging in for c and n0, then f(n) is O(g(n)).
For example (same as above), let f(n) = 2^n, g(n) = n. Is the following solvable: 2^n ≤ c * n for n ≥ n0? The answer is no. No matter what value is plugged into c, the left side will always be bigger than the right side as n approaches infinity. There is no way to make the left side smaller than the right side for all values n ≥ n0.
On the other hand, if f(n) = 2n, g(n) = n, then the condition is 2n ≤ c * n for n ≥ n0. This is solvable: c = 2, n0 = 0.
Let f(n) and g(n) be functions mapping nonnegative integers to real numbers.
Let f(n) and g(n) be functions where the values of n i.e. domain is 0 or positive integers, the values of f(n) and g(n) for those values of n may be real numbers.
We say that f(n) is O(g(n)) if there is a real constant c > 0 and an real constant n0 ≥ 1 such that:
f(n) ≤cg(n), for n ≥ n0.
f(n) = O(g(n)) if there exist positive constants c and n0 such that
0 <= f(n) <= cg(n) for all n >= n0. Actually , it means that f(n) is asymptotically less than or equal to g(n).
For example, consider f(n) = 3 * n^2 + 5. We can show that f(n) is O(n^2) by choosing c = 4 and n0 = 2. This is because for all values of n greater than 2:
3 * n^2 + 5 <= 4 * n^2
f(n) is not O(n), because whatever constant c and value n0 you choose, I can always find a value of n greater than n0 so that 3 * n^2 + 5 is greater than c * n.
I came across two asymptotic function proofs.
f(n) = O(g(n)) implies 2^f(n) = O(2^g(n))
Given: f(n) ≤ C1 g(n)
So, 2^f(n) ≤ 2^C1 g(n) --(i)
Now, 2^f(n) = O(2^g(n)) → 2^f(n) ≤ C2 2^g(n) --(ii)
From,(i) we find that (ii) will be true.
Hence 2^f(n) = O(2^g(n)) is TRUE.
Can you tell me if this proof is right? Is there any other way to solve this?
2.f(n) = O((f(n))^2)
How to prove the second example? Here I consider two cases one is if f(n)<1 and other is f(n)>1.
Note: None of them are homework questions.
The attempted-proof for example 1 looks well-intentioned but is flawed. First, “2^f(n) ≤ 2^C1 g(n)” means 2^f(n) ≤ (2^C1)*g(n), which in general is false. It should have been written 2^f(n) ≤ 2^(C1*g(n)). In the line beginning with “Now”, you should explicitly say C2 = 2^C1. The claim “(ii) will be true” is vacuous (there is no (ii)).
A function like f(n) = 1/n disproves the claim in example 2 because there are no constants N and C such that for all n > N, f(n) < C*(f(n))². Proof: Let some N and C be given. Choose n>N, n>C. f(n) = 1/n = n*(1/n²) > C*(1/n²) = C*(f(n))². Because N and C were arbitrarily chosen, this shows that there are no fixed values of N and C such that for all n > N, f(n) < C*(f(n))², QED.
Saying that “f(n) ≥ 1” is not enough to allow proving the second claim; but if you write “f(n) ≥ 1 for all n” or “f() ≥ 1” it is provable. For example, if f(n) = 1/n for odd n and 1+n for even n, we have f(n) > 1 for even n > 0, and less than 1 for odd n. To prove that f(n) = O((f(n))²) is false, use the same proof as in the previous paragraph but with the additional provision that n is even.
Actually, “f(n) ≥ 1 for all n” is stronger than necessary to ensure f(n) = O((f(n))²). Let ε be any fixed positive value. No matter how small ε is, “f(n) ≥ ε for all n > N'” ensures f(n) = O((f(n))²). To prove this, take C = max(1, 1/ε) and N=N'.
I'm starting to learn about Big-Oh notation.
What is an easy way for finding C and N0 for a given function?
Say, for example:
(n+1)5, or n5+5n4+10n2+5n+1
I know the formal definition for Big-Oh is:
Let f(n) and g(n) be functions mapping
nonnegative integers to real numbers.
We say that f(n) is O(g(n)) if there
is a real constant c > 0 and an
integer constant N0 >= 1
such that f(n) <= cg(n) for every integer N > N0.
My question is, what is a good, sure-fire method for picking values for c and N0?
For the given polynomial above (n+1)5, I have to show that it is O(n5). So, how should I pick my c and N0 so that I can make the above definition true without guessing?
You can pick a constant c by adding the coefficients of each term in your polynomial. Since
| n5 + 5n4 + 0n3 + 10n2 + 5n1 + 1n0 | <= | n5 + 5n5 + 0n5 + 10n5 + 5n5 + 1n5 |
and you can simplify both sides to get
| n5 + 5n4 + 10n2 + 5n + 1 | <= | 22n5 |
So c = 22, and this will always hold true for any n >= 1.
It's almost always possible to find a lower c by raising N0, but this method works, and you can do it in your head.
(The absolute value operations around the polynomials are to account for negative coefficients.)
Usually the proof is done without picking concrete C and N0. Instead of proving f(n) < C * g(n) you prove that f(n) / g(n) < C.
For example, to prove n3 + n is O(n3) you do the following:
(n3 + n) / n3 = 1 + (n / n3) = 1 + (1 / n2) < 2 for any n >= 1. Here you can pick any C >= 2 with N0 = 1.
You can check what the lim abs(f(n)/g(n)) is when n->+infitity and that would give you the constant (g(n) is n^5 in your example, f(n) is (n+1)^5).
Note that the meaning of Big-O for x->+infinity is that if f(x) = O(g(x)), then f(x) "grows no faster than g(x)", so you just need to prove that lim abs(f(x)/g(x)) exists and is less than +infinity.
It's going to depend greatly on the function you are considering. However, for a given class of functions, you may be able to come up with an algorithm.
For instance, polynomials: if you set C to any value greater than the leading coefficient of the polynomial, then you can solve for N0.
After you understand the magic there, you should also get that big-O is a notation. It means that you do not have to look for these coefficients in every problem you solve, once you made sure you understood what's going on behind these letters. You should just operate the symbols according to the notaion, according to its rules.
There's no easy generic rule to determine actual values of N and c. You should recall your calculus knowledge to solve it.
The definition to big-O is entangled with definition of the limit. It makes c satisfy:
c > lim |f(n)/g(n)|, given n approaches +infinity.
If the sequence is upper-bounded, it always has a limit. If it's not, well, then f is not O(g). After you have picked concrete c, you will have no problem finding appropriate N.