According to this book, big O means:
f(n) = O(g(n)) means c · g(n) is an upper bound on f(n). Thus there exists some constant c such that f(n) is always ≤ c · g(n), for large enough n (i.e. , n ≥ n0 for some constant n0).
I have trubble understanding the following big O equation
3n2 − 100n + 6 = O(n2), because I choose c = 3 and 3n2 > 3n2 − 100n + 6;
How can 3 be a factor? In 3n2 − 100n + 6, if we drop the low order terms -100n and 6, aren't 3n2 and 3.n2 the same? How to solve this equation?
I'll take the liberty to slightly paraphrase the question to:
Why do and have the same asymptotic complexity.
For that to be true, the definition should be in effect both directions.
First:
let
Then for the inequality is always satisfied.
The other way around:
let
We have a parabola opened upwards, therefore there is again some after which the inequality is always satisfied.
Let's look at the definition you posted for f(n) in O(g(n)):
f(n) = O(g(n)) means c · g(n) is an upper bound on f(n). Thus there
exists some constant c such that f(n) is always ≤ c · g(n), for
large enough n (i.e. , n ≥ n0 for some constant n0).
So, we only need to find one set of constants (c, n0) that fulfils
f(n) < c · g(n), for all n > n0, (+)
but this set is not unique. I.e., the problem of finding the constants (c, n0) such that (+) holds is degenerate. In fact, if any such pair of constants exists, there will exist an infinite amount of different such pairs.
Note that here I've switched to strict inequalities, which is really only a matter of taste, but I prefer this latter convention. Now, we can re-state the Big-O definition in possibly more easy-to-understand terms:
... we can say that f(n) is O(g(n)) if we can find a constant c such
that f(n) is less than c·g(n) or all n larger than n0, i.e., for all
n>n0.
Now, let's look at your function f(n)
f(n) = 3n^2 - 100n + 6 (*)
Let's describe your functions as a sum of it's highest term and another functions
f(n) = 3n^2 + h(n) (**)
h(n) = 6 - 100n (***)
We now study the behaviour of h(n) and f(n), respectively:
h(n) = 6 - 100n
what can we say about this expression?
=> if n > 6/100, then h(n) < 0, since 6 - 100*(6/100) = 0
=> h(n) < 0, given n > 6/100 (i)
f(n) = 3n^2 + h(n)
what can we say about this expression, given (i)?
=> if n > 6/100, the f(n) = 3n^2 + h(n) < 3n^2
=> f(n) < c*n^2, with c=3, given n > 6/100 (ii)
Ok!
From (ii) we can choose constant c=3, given that we choose the other constant n0 as larger than 6/100. Lets choose the first integer that fulfils this: n0=1.
Hence, we've shown that (+) golds for constant set **(c,n0) = (3,1), and subsequently, f(n) is in O(n^2).
For a reference on asymptotic behaviour, see e.g.
https://www.khanacademy.org/computing/computer-science/algorithms/asymptotic-notation/a/big-o-notation
y=3n^2 (top graph) vs y=3n^2 - 100n + 6
Consider the sketch above. By your definition, 3n^2 only needs to be bigger than 3n^2 - 100n + 6 for large enough n (i.e. , n ≥ n0 for some constant n0). Let that n0 = 5 in this case (it could be something a little smaller, but it's clear which graph is bigger by n=5 so we'll just go with that).
Clearly from the graph, 3n^2 >= 3n^2 - 100n + 6 in the range we've plotted. The only way for 3n^2 - 100n + 6 to get bigger than 3n^2 then is for it to grow more steeply.
But the gradients of 3n^2 and 3n^2 - 100n + 6 are 6n and 6n - 100 respectively, so 3n^2 - 100n + 6 can't grow more steeply, therefore must always be underneath.
So your definition holds - 3n^2 - 100n + 6 <= 3n^2 for all n>=5
I am not an expert, but this looks a lot similar to what we just had in our real analysis course.
Basically if you have something like f(n) = 3n^2 − 100n + 6, the "fastest growing" term "wins" the other terms, when you have really really big n.
So in this case 3n^2 surpasses what ever 100n is, when the n is really big.
Another example would be something like f(n) = n/n^2 or f(n) = n! * n^2.
The first one gets smaller, as n simply cannot "keep up" with n^2. In the second example n! clearly grows faster than n^2, so I guess the answer for that should be f(n) = n! then, because the n^2 technically stops mattering with big n.
And terms like +6, which have no n affecting them are constants and matter even less as they cannot grow even if n grows.
It is all about what happends when n is really big. If your n is 34934854385754385463543856, then n^2 is hell of a bigger than 100n, because n^2 = n * n = 34934854385754385463543856 * 34934854385754385463543856.
Related
I am going through the Asymptotic notations from here. I am reading this f(n) ≤ c g(n)
For example, if f(n) = 2n + 2, We can satisfy it in any way as f(n) is O (c.g(n)) by adjusting the value of n and c. Or is there any specific rule or formula for selecting the value of c and n. Will no always be 1?
There is no formula per se. You can find the formal definition here:
f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n. (big-O notation).
What I understood from your question is, you are not getting the essence of big-O notation. If your complexity is, for example, O(n^2), then you can guarantee that there is some value of n (greater than k) after which f(n) in no case will exceed c g(n).
Let's try to prove f(n) = 2n + 2 is O(n):
As it seems from the function itself, you cannot set the value of c equal to 2 as you want to find f(n) ≤ c g(n). If you plug in c = 2, you have to find k such that f(n) ≤ c g(n) for n ≥ k. Clearly, there is no n for which 2n ≥ 2n + 2. So, we move on to c = 3.
Now, let's find the value of k. So, we solve the equation 3n ≥ 2n + 2. Solving it:
3n ≥ 2n + 2
=> 3n - 2n ≥ 2
=> n ≥ 2
Therefore, for c = 3, we found value of k = 2 (n ≥ k).
You must also understand, your function isn't just O(n). It is also O(n^2), O(n^3), O(n^4) and so on. All because corresponding values of c and k exist for g(n) = n^2, g(n) = n^3 and g(n) = n^4.
Hope it helps.
I am learning about complexity theory and have a question asking to show truth/falsehood of a number of Big-O statements.
I've done the first few e.g. showing 2^(n+1) is in O(2^n) by finding a constant and N value. But now they are asking more abstract things, for example:
If f(n) is O(g(n)), is log f(n) in O(log g(n))?
Is 2^(f(n)) in O(2^(g(n)))
These both seem like they would be true but I don't know how to express them formally with a constant and a N value. If you can give an example of how I could show these I can go do the rest of the problems.
The comments are both accurate. Here are some notes along the lines you are probably looking for.
Assume f(n) is O(g(n)). Then there exist n0 and c such that f(n) <= cg(n) for n >= n0. Take the logarithm of both sides. log(f(n)) <= log(cg(n)). We can use the laws of logarithms to rewrite this as log(f(n)) <= log(c) + log(g(n)). If g(n) is greater than 1, then log(c) + log(g(n)) <= (1+log(c))*log(g(n)), so we can choose c' = 1 + log(c) and get the desired result. Otherwise, note that for g(n) = 1 we're still good since any choice for c' works.
The second one is not true. Choose f(n) = 2n and g(n) = n. We see f(n) is O(g(n)) by choosing c = 3. However, 2^(2n) = 4^n is not O(2^n). To see that, assume we had n0 and c. Then 4^n <= c*2^n. Dividing by 2^n gives 2^n <= c. But this can't be correct since n can increase indefinitely whereas c is fixed.
My textbook describes the relationship as follows:
There is a very nice mathematical intuition which describes these classes too. Suppose we have an algorithm which has running time N0 when given an input of size n, and a running time of N1 on an input of size 2n. We can characterize the rates of growth in terms of the relationship between N0 and N1:
Big-Oh Relationship
O(log n) N1 ≈ N0 + c
O(n) N1 ≈ 2N0
O(n²) N1 ≈ 4N0
O(2ⁿ) N1 ≈ (N0)²
Why is this?
That is because if f(n) is in O(g(n)) then it can be thought of as acting like k * g(n) for some k.
So for example if f(n) = O(log(n)) then it acts like k log(n), and now f(2n) ≈ k log(2n) = k (log(2) + log(n)) = k log(2) + k log(n) ≈ k log(2) + f(n) and that is your desired equation with c = k log(2).
Note that this is a rough intuition only. An example of where it breaks down is that f(n) = (2 + sin(n)) log(n) = O(log(n)). The oscillating 2 + sin(n) bit means that f(2n)-f(n) can be basically anything.
I personally find this kind of rough intuition to be misleading and therefore worse than useless. Others find it very helpful. Decide for yourself how much weight you give it.
Basically what they are trying to show is just basic algebra after substituting 2n for n in the functions.
O(log n)
log(2n) = log(2) + log(n)
N1 ≈ c + N0
O(n)
2n = 2(n)
N1 ≈ 2N0
O(n²)
(2n)^2 = 4n^2 = 4(n^2)
N1 ≈ 4N0
O(2ⁿ)
2^(2n) = 2^(n*2) = (2^n)^2
N1 ≈ (N0)²
Since O(f(n)) ~ k * f(n) (almost by definition), you want to look at what happens when you put 2n in for n. In each case:
N1 ≈ k*log 2n = k*(log 2 + log n) = k*log n + k*log 2 ≈ N0 + c where c = k*log 2
N1 ≈ k*(2n) = 2*k*n ≈ 2N0
N1 ≈ k*(2n)^2 = 4*k*n^2 ≈ 4N0
N1 ≈ k*2^(2n) = k*(2^n)^2 ≈ N0*2^n ≈ N0^2/k
So the last one is not quite right, anyway. Keep in mind that these relationships are only true asymptotically, so the approximations will be more accurate as n gets larger. Also, f(n) = O(g(n)) only means g(n) is an upper bound for f(n) for large enough n. So f(n) = O(g(n)) does not necessarily mean f(n) ~ k*g(n). Ideally, you want that to be true, since your big-O bound will be tight when that is the case.
This question already has answers here:
What is a plain English explanation of "Big O" notation?
(43 answers)
Closed 9 years ago.
The definition in a book said
The "Big-Oh" Notation
Let f(n) and g(n) be functions mapping nonnegative integers to real numbers. We say that f(n) is O(g(n)) if there is a real constant c > 0 and an real constant n0 ≥ 1 such that
f(n) ≤cg(n), for n ≥ n0.
I couldn't able to understand terminologies used in formula and definition can somebody explain in plain English.
Basically, f(n) is O(g(n)) then g(n) is proportional to the worst-case scenario of f(x).
For example, binary search is O(log n) (or O(ln n), which is equivalent). Why?
(Binary search works like this: take the middle element, and compare to the target. If it's the one, you're done. If it's bigger than the target, throw out the second half of the list and repeat on the first half; if it's smaller than the target, throw out the first half and repeat the search on the second half.)
Because you need 1 operation to find something in a list that is 3 elements long; 2 operations when it's 7 elements long; 3 if it is 15 elements long. Thus, when number of elements n is (2^x - 1) for any x, the number of operations is x; turn it around, and you'd say for number of elements n, number of operations is log_2 n. And say that each operation lasts 2 seconds (say you're comparing stuff by hand), and the worst time to search is log_2 n * 2 seconds. log_2 n can be rewritten as ln n / ln 2, so the formula becomes:
worst search time(n) = (ln n / ln 2) * 2 seconds
= (2 seconds / ln 2) * ln n
Now, 2 seconds / ln 2 is a constant; let's call it c. Let's call "search time for n elements" f(n). And let's call ln n as g(n).
We said before, if n = 3, g(3) <= c * ln 3 (because c * ln 3 is worst search time, real search time is always less or equal to that; but we could always find it on our first try). If n = 7, g(7) <= c * ln 7; etc.
The bit about n0 is just a guard that says the complexity we calculate for the small n might be a deviation, an anomaly, an exception from the rule, and if we go with big enough data (i.e. n >= n0), the rule becomes obvious and inviolate. In this case, the rule works pretty much from the start, but some algorithms might have extra costs that throw off the calculation on small numbers.
Translation to "plain English": Imagine that f(n) are g(n) function that take a positive number or zero as input, and give a real number as output (no imaginary numbers).
Big-Oh allows us to compare two functions to see if one is bounded by the other. For example, an exponential function f(n) would not be bounded by a linear function g(n), so f(n) would not be O(g(n)).
We can say that f(n) is O(g(n)) if the following is possible: f(n) ≤ c * g(n) for n ≥ n0. If there is some way to solve the equation by plugging in for c and n0, then f(n) is O(g(n)).
For example (same as above), let f(n) = 2^n, g(n) = n. Is the following solvable: 2^n ≤ c * n for n ≥ n0? The answer is no. No matter what value is plugged into c, the left side will always be bigger than the right side as n approaches infinity. There is no way to make the left side smaller than the right side for all values n ≥ n0.
On the other hand, if f(n) = 2n, g(n) = n, then the condition is 2n ≤ c * n for n ≥ n0. This is solvable: c = 2, n0 = 0.
Let f(n) and g(n) be functions mapping nonnegative integers to real numbers.
Let f(n) and g(n) be functions where the values of n i.e. domain is 0 or positive integers, the values of f(n) and g(n) for those values of n may be real numbers.
We say that f(n) is O(g(n)) if there is a real constant c > 0 and an real constant n0 ≥ 1 such that:
f(n) ≤cg(n), for n ≥ n0.
f(n) = O(g(n)) if there exist positive constants c and n0 such that
0 <= f(n) <= cg(n) for all n >= n0. Actually , it means that f(n) is asymptotically less than or equal to g(n).
For example, consider f(n) = 3 * n^2 + 5. We can show that f(n) is O(n^2) by choosing c = 4 and n0 = 2. This is because for all values of n greater than 2:
3 * n^2 + 5 <= 4 * n^2
f(n) is not O(n), because whatever constant c and value n0 you choose, I can always find a value of n greater than n0 so that 3 * n^2 + 5 is greater than c * n.
I'm reading a textbook right now for my Java III class. We're reading about Big-Oh and I'm a little confused by its formal definition.
Formal Definition: "A function f(n) is of order at most g(n) - that is, f(n) = O(g(n)) - if a positive real number c and positive integer N exist such that f(n) <= c g(n) for all n >= N. That is, c g(n) is an upper bound on f(n) when n is sufficiently large."
Ok, that makes sense. But hold on, keep reading...the book gave me this example:
"In segment 9.14, we said that an
algorithm that uses 5n + 3 operations
is O(n). We now can show that 5n + 3 =
O(n) by using the formal definition of
Big Oh.
When n >= 3, 5n + 3 <= 5n + n = 6n.
Thus, if we let f(n) = 5n + 3, g(n) =
n, c = 6, N = 3, we have shown that
f(n) <= 6 g(n) for n >= 3, or 5n + 3 =
O(n). That is, if an algorithm
requires time directly proportional to
5n + 3, it is O(n)."
Ok, this kind of makes sense to me. They're saying that if n = 3 or greater, 5n + 3 takes less time than if n was less than 3 - thus 5n + n = 6n - right? Makes sense, since if n was 2, 5n + 3 = 13 while 6n = 12 but when n is 3 or greater 5n + 3 will always be less than or equal to 6n.
Here's where I get confused. They give me another example:
Example 2: "Let's show that 4n^2 + 50n
- 10 = O(n^2). It is easy to see that: 4n^2 + 50n - 10 <= 4n^2 + 50n
for any n. Since 50n <= 50n^2 for n
= 50, 4n^2 + 50n - 10 <= 4n^2 + 50n^2 = 54n^2 for n >= 50. Thus, with c = 54 and N = 50, we have shown that 4n^2
+ 50n - 10 = O(n^2)."
This statement doesn't make sense: 50n <= 50n^2 for n >= 50.
Isn't any n going to make the 50n less than 50n^2? Not just greater than or equal to 50? Why did they even mention that 50n <= 50n^2? What does that have to do with the problem?
Also, 4n^2 + 50n - 10 <= 4n^2 + 50n^2 = 54n^2 for n >= 50 is going to be true no matter what n is.
And how in the world does picking numbers show that f(n) = O(g(n))?
Keep in mind that you're looking for "an upper bound on f(n) when n is sufficiently large." Thus, if you can show that f(n) is less than or equal to some cg(n) for values of n greater than N, this means cg(n) is an upper bound for f(n) and f(n)'s complexity is therefore O(g(n)).
The examples given are intended to show that the given function f(n) can never grow beyond c*g(n) for any n > N. By manipulating an initial upper bound so it can be expressed more simply (if 4n^2 + 50n is an upper bound on f(n) then so is 4n^2 + 50n^2, which is equal to 54n^2, which becomes your 54*g(n) where c = 54 and g(n) = n^2), the authors can show that f(n) is bounded by c*g(n), which has complexity O(g(n)) and therefore so does f(n).
The whole thing about picking numbers is just this: To make it easier. Because you're allowed to pick any numbers you like for N and c, the author just picks something, where it's most easy to see. And that's what you can also do (when writing an exam etc).
So while it would often be possible to use a smaller N, the reasoning would become a little bit harder (often requiring some previous knowledge about analysis - we've all learnt years before, that x doesn't grow as fast as x^2... But do you want to write down the analysis proof?)
Keep it simple, is the message :-) It's just a bit strange to get used to this at first.
50n <= 50n^2 for n >= 50
because if n is 50, then 50n is the same as n^2, because 50*50 equals 50^2.
Substituting n^2 for 50n we get
n^2 <= 50n^2 for n >= 50
which is obvious.
Probably the reason that they said 50n<=50n^2 for n>=50 is that if n is less than 1, than n^2 < n. Of course, if n is a positive integer, then yes 50n<=50n^2. In this case, it seems that n is assumed to be a positive integer, although the formal definition they give doesn't state that explicitly.
I can see why saying 50n<=50n^2 for n>=50 may seem a little silly. But it's still true. The book doesn't say 50n<=50n^2 holds ONLY for n>=50; that would be false.
As an analogy, if I say "all of my siblings speak English", that would be true, even though there are a lot of people who speak English who are not my siblings.
Regarding the proof, we might split it into different statements.
(1): 4n^2 + 50n - 10 <= 4n^2 + 50n (for all n)
(2): 4n^2 + 50n <= 4n^2 + 50n^2 (for all n>=50)
(3): 4n^2 + 50n^2 = 54 n^2 (for all n, including all n>=50)
(4): Therefore, 4n^2 + 50n - 10 <= 54n^2 for all n>=50
(5): Therefore, for f(n)=4n^2 + 50n - 10, g(n)=n^2, N=50, and c=54,
the statement f(n) <= c g(n) for all n >= N is true
(6): Therefore, by definition 4n^2 + 50n - 10=O(n^2).
It should be clear that each of these statements is true, either on its own (1,2,3), or as a result of the previous statements.
Formal definition:
f(n) = O(g(n)) means there exist c > 0 and n0 such that for any n >= n0 f(n) <= c*g(n)
f(n) = o(g(n)) means for any c > 0 there exist n0 such that for any n >= n0 f(n) <= c*g(n)
As you can note there are slightly different :)