I am reading through Skiena's "Algorithm Design Manual".
The first chapter states a formal definition for Big O notation:
f(n) = O(g(n)) means that c * g(n) is an upper bound on f(n).
i.e. there exists some constant c such that f(n) is always less than or equal to c * g(n) for large enough n. (i.e. n >= n0 for some constant n0)
So that's fine and makes sense.
But then the author goes on to describe the Big O of a particular function: 3n^2 - 100n + 6
He says that O(3n^2 - 100n - 6) is NOT equal to O(n). And his reason is that for any c that I choose, c * n is always < 3n^2 when n>c. Which is true, but what about the (-100n + 6) part?
Let's say I choose c = 1 and n = 2.
3n^2 - 100n + 6 = 12 - 200 + 6 = -182
and c * n is 1*2 which is 2. -182 is definitely less than 2, so why does Skiena ignore those terms?
Note the n >= n0 in the definition.
If you pick some c and n0, it has to be true for each n >= n0, not just n0.
So if you have c = 1 and n0 = 2, it also has to be true for n = 1000 for example, which it isn't.
3n^2 - 100n + 6
=> 3(1000)^2 - 100(1000) + 6 = 3 000 000 - 100 000 + 6 = 2 900 006
c.n
=> 1(1000) = 1 000
It's simplification. 3n^2 is greater than any 100n-6 for every n >= (SQRT(2482)+50)/3 ~= 33.2732249428 - please check - it's simple equation. Thus O(3n^2) > O(100n-6). That's why it's not worth considering that part - it does not add any value.
Please note that according to definition you have to find (at least one) c for which every c*n is always < 3n^2 - 100n + 6 for every n greater or equal than some n0 (at least one). Just pick c = 1000 and n0=1000 and you will see that it is always true for those c and n0. Because I have found such c and n0 than statement O(n) < O(3n^2 - 100n - 6) holds true.
But I agree that this simplification might be misleading.
Related
6n^4 ā3n^2 +3 is Ī©(n4)
Hello, I need to determine whether this statement is true or false.
Any help is appreciated.
Thank you
I am leaning towards true due to the n^4, however the omega complexity is making me doubt this.
I believe if it was big O it would be a true statement.
f is Omega(g) if there exist constants c and n0 such that for all n > n0, f(n) >= c * g(n). For us, we need to evaluate whether there are constants n0 and c such that 6n^4 - 3n^2 + 3 > cn^4 for all n > n0. If we choose n = 5 we get...
6n^4 - 3n^2 + 3 > 5n^4
n^4 - 3n^2 + 3 > 0
Using the quadratic formula we can find values for n^2 where the LHS equals zero:
n^2 = [-b +- sqrt(b^2 - 4ac)] / 2a
= [3 +- sqrt(9 - 12] / 2
But the discriminant is negative, which means there are no real values of n^2 where the LHS equals 0. This means that the LHS has no roots and never crosses the X-axis; it is either always positive or always negative. We can see which is the case easily by plugging in 0:
0^4 - 30^2 + 3 = 3 > 0
So, with the choice of c=5, our inequality is true for all n; we are free to choose any n0, e.g., n0 = 1 works.
Because there exists a pair c=5 and n0=1 which gives us f(n) = 6n^4 - 3n^2 + 3 > 5n^4 = cg(n) for all n > n0, we can say that f is Omega(g).
I was looking at this question:
Prove that 100š+5 ā š(šĀ²) (Which is 100š+5 is upper bounded by šĀ²)
š(š) ā¤ šš(š) for all š ā„ š0
so it becomes 100š+5 ā¤ ššĀ²
The answer was:
š0 ā 25.05 (the number where the šĀ² algorithm intercepts the š algorithm) and š = 4 so that when š increases above 25.05 no matter what it will
still prove that 100š+5āššĀ² is true
My question is: how do you derive that š0 = 25.05 and š = 4? Is it a guess and trial method, or is there a proper way to get that particular answer? Or you just gotta start from 1 and work your way up to see if it works?
A good approach to tackle such kind of problems is to first fix the c
let's take 4 in this example
and then all you have to do is figure out n0 using a simple equality
100n + 5 = 4n^2 <=> 4n^2 - 100n - 5 = 0 <=> n = 25.05 or n = -0.05 and here you can remark that they intersect twice in -0.08 and 25.05 and as you want n0 such that after which 100n +5 is always below 4n^2 -0.05 is not the one as 25.05 > -0.05 and in 25.05 they intersect so n0 = 25.05 .
Before fixing c and trying to figure out n0 you could try big numbers for n0 to have an idea whether it's an upper bound or not.
There are infinitely many choices for n0 and c that can be used to prove this bound holds. We need to find n0 and c such that for n >= n0, f(n) <= c * g(n). In your case, we need 100n + 5 <= cn^2. We can rearrange this as follows using basic algebra:
cn^2 - 100n - 5 >= 0
We can use the quadratic formula to find the roots:
n1, n2 = [100 +- sqrt(10000 + 20c)]/2c
Because c is positive we know the sqrt term will be greater than 100 once evaluated and since we are only interested in n > 0 we can discard the smaller of these solutions and focus on this:
n0 = [100 + sqrt(10000 + 20c)]/2c
We can simplify this a bit:
n0 = [100 + sqrt(10000 + 20c)]/2c
= [100 + 2*sqrt(2500 + 5c)]/2c
= [50 + sqrt(2500 + 5c)]/c
At this point, we can choose either a value for c or a value for n0, and solve for the other one. Your example chooses c = 4 and gets the approximate answer n0 ~ 25.05. If we'd prefer to choose n0 directly (say we want n0 = 10) then we calculate as follows:
10 = [50 + sqrt(2500 + 5c)]/c
10c = 50 + sqrt(2500 + 5c)
(10c - 50) = sqrt(2500 + 5c)
(100c^2 - 1000c + 2500) = (2500 + 5c)
100c^2 - 1005c = 0
c(100c - 1005) = 0
c = 0 or c = 1005/100 ~ 10.05
Because the solution c=0 is obviously no good, the solution c ~ 10.05 appears to work for our choice of n0 = 10. You can choose other n0 or c and find the corresponding constant in this way.
I'm trying to explain to my friend why 7n - 2 = O(N). I want to do so based on the definition of big O.
Based on the definition of big O, f(n) = O(g(n)) if:
We can find a real value C and integer value n0 >= 1 such that:
f(n)<= C . g(n) for all values of n >= n0.
In this case, is the following explanation correct?
7n - 2 <= C . n
-2 <= C . n - 7n
-2 <= n (C - 7)
-2 / (C - 7) <= n
if we consider C = 7, mathematically, -2 / (C - 7) is equal to negative infinity, so
n >= (negative infinity)
It means that for all values of n >= (negative infinity) the following holds:
7n - 2 <= 7n
Now we have to pick n0 such that for all n >= n0 and n0 >= 1 the following holds:
7n - 2 <= 7n
Since for all values of n >= (negative infinity) the inequality holds, we can simply take n0 = 1.
You're on the right track here. Fundamentally, though, the logic you're using doesn't work. If you are trying to prove that there exist an n0 and c such that f(n) ā¤ cg(n) for all n ā„ n0, then you can't start off by assuming that f(n) ā¤ cg(n) because that's ultimately what you're trying to prove!
Instead, see if you can start with the initial expression (7n - 2) and massage it into something upper-bounded by cn. Here's one way to do this: since 7n - 2 ā¤ 7n, we can (by inspection) just pick n0 = 0 and c = 7 to see that 7n - 2 ā¤ cn for all n ā„ n0.
For a more interesting case, let's try this with 7n + 2:
7n + 2
ā¤ 7n + 2n (for all n ā„ 1)
= 9n
So by inspection we can pick c = 9 and n0 = 1 and we have that 7n + 2 ā¤ cn for all n ā„ n0, so 7n + 2 = O(n).
Notice that at no point in this math did we assume the ultimate inequality, which means we never had to risk a divide-by-zero error.
How they are calculating Big O notation?
Big O notation examples:
Question:
Find upper bound for f(n) = nā“ + 100nĀ² + 50
Solution:
nā“ + 100nĀ² + 50 <= 2nā“, for all n >= 11
nā“ + 100nĀ² + 50 = O(nā“) with c = 2 and n0 = 11
I am unable to understand - how they are calculating Big O notation?
How come c = 2 and n0 = 11
Question:
Find upper bound for f(n) = 2nĀ³ - 2nĀ²
Solution:
2nĀ³ - 2nĀ² <= 2nĀ³ , for all n >= 1
2nĀ³ - 2nĀ² = O(2nĀ³) with c = 2 and n0 =1
How come n >= 1?
Question:
f(n) = n
Solution :
n <= n , for all n >= 1
N = O(n) with c = 1 and n0 = 1
Question 1:
Definition of big O notation is (note that the wikipedia definition uses x0 and M, there is no real difference, just different signs for the same things).
f(n) is said to be in O(g(n)) if there are constants n0 and c such that for all n>n0 - f(n) <= c*g(n).
In your example, since n^4+100n^2+50 <= 2n^4 for all n>11, it is basically same as saying the above with c=2,n0=11 - thus, by definition of big O notation, you could say n^4+100n^2+50 is in O(n^4)
Question 2:
In here, n>=1 because the claim is true for all positive values of n - note that 2n^3-2n^2 will always be smaller than 2n^3 - for all positive values of n.
I'm reading a textbook right now for my Java III class. We're reading about Big-Oh and I'm a little confused by its formal definition.
Formal Definition: "A function f(n) is of order at most g(n) - that is, f(n) = O(g(n)) - if a positive real number c and positive integer N exist such that f(n) <= c g(n) for all n >= N. That is, c g(n) is an upper bound on f(n) when n is sufficiently large."
Ok, that makes sense. But hold on, keep reading...the book gave me this example:
"In segment 9.14, we said that an
algorithm that uses 5n + 3 operations
is O(n). We now can show that 5n + 3 =
O(n) by using the formal definition of
Big Oh.
When n >= 3, 5n + 3 <= 5n + n = 6n.
Thus, if we let f(n) = 5n + 3, g(n) =
n, c = 6, N = 3, we have shown that
f(n) <= 6 g(n) for n >= 3, or 5n + 3 =
O(n). That is, if an algorithm
requires time directly proportional to
5n + 3, it is O(n)."
Ok, this kind of makes sense to me. They're saying that if n = 3 or greater, 5n + 3 takes less time than if n was less than 3 - thus 5n + n = 6n - right? Makes sense, since if n was 2, 5n + 3 = 13 while 6n = 12 but when n is 3 or greater 5n + 3 will always be less than or equal to 6n.
Here's where I get confused. They give me another example:
Example 2: "Let's show that 4n^2 + 50n
- 10 = O(n^2). It is easy to see that: 4n^2 + 50n - 10 <= 4n^2 + 50n
for any n. Since 50n <= 50n^2 for n
= 50, 4n^2 + 50n - 10 <= 4n^2 + 50n^2 = 54n^2 for n >= 50. Thus, with c = 54 and N = 50, we have shown that 4n^2
+ 50n - 10 = O(n^2)."
This statement doesn't make sense: 50n <= 50n^2 for n >= 50.
Isn't any n going to make the 50n less than 50n^2? Not just greater than or equal to 50? Why did they even mention that 50n <= 50n^2? What does that have to do with the problem?
Also, 4n^2 + 50n - 10 <= 4n^2 + 50n^2 = 54n^2 for n >= 50 is going to be true no matter what n is.
And how in the world does picking numbers show that f(n) = O(g(n))?
Keep in mind that you're looking for "an upper bound on f(n) when n is sufficiently large." Thus, if you can show that f(n) is less than or equal to some cg(n) for values of n greater than N, this means cg(n) is an upper bound for f(n) and f(n)'s complexity is therefore O(g(n)).
The examples given are intended to show that the given function f(n) can never grow beyond c*g(n) for any n > N. By manipulating an initial upper bound so it can be expressed more simply (if 4n^2 + 50n is an upper bound on f(n) then so is 4n^2 + 50n^2, which is equal to 54n^2, which becomes your 54*g(n) where c = 54 and g(n) = n^2), the authors can show that f(n) is bounded by c*g(n), which has complexity O(g(n)) and therefore so does f(n).
The whole thing about picking numbers is just this: To make it easier. Because you're allowed to pick any numbers you like for N and c, the author just picks something, where it's most easy to see. And that's what you can also do (when writing an exam etc).
So while it would often be possible to use a smaller N, the reasoning would become a little bit harder (often requiring some previous knowledge about analysis - we've all learnt years before, that x doesn't grow as fast as x^2... But do you want to write down the analysis proof?)
Keep it simple, is the message :-) It's just a bit strange to get used to this at first.
50n <= 50n^2 for n >= 50
because if n is 50, then 50n is the same as n^2, because 50*50 equals 50^2.
Substituting n^2 for 50n we get
n^2 <= 50n^2 for n >= 50
which is obvious.
Probably the reason that they said 50n<=50n^2 for n>=50 is that if n is less than 1, than n^2 < n. Of course, if n is a positive integer, then yes 50n<=50n^2. In this case, it seems that n is assumed to be a positive integer, although the formal definition they give doesn't state that explicitly.
I can see why saying 50n<=50n^2 for n>=50 may seem a little silly. But it's still true. The book doesn't say 50n<=50n^2 holds ONLY for n>=50; that would be false.
As an analogy, if I say "all of my siblings speak English", that would be true, even though there are a lot of people who speak English who are not my siblings.
Regarding the proof, we might split it into different statements.
(1): 4n^2 + 50n - 10 <= 4n^2 + 50n (for all n)
(2): 4n^2 + 50n <= 4n^2 + 50n^2 (for all n>=50)
(3): 4n^2 + 50n^2 = 54 n^2 (for all n, including all n>=50)
(4): Therefore, 4n^2 + 50n - 10 <= 54n^2 for all n>=50
(5): Therefore, for f(n)=4n^2 + 50n - 10, g(n)=n^2, N=50, and c=54,
the statement f(n) <= c g(n) for all n >= N is true
(6): Therefore, by definition 4n^2 + 50n - 10=O(n^2).
It should be clear that each of these statements is true, either on its own (1,2,3), or as a result of the previous statements.
Formal definition:
f(n) = O(g(n)) means there exist c > 0 and n0 such that for any n >= n0 f(n) <= c*g(n)
f(n) = o(g(n)) means for any c > 0 there exist n0 such that for any n >= n0 f(n) <= c*g(n)
As you can note there are slightly different :)