In a tutorial on the Big-O notation it is said that if T(n) = 4n2-2n+2, then T(n)=O(n2). However, we know that f(x) = O(g(x)) if there exists N and C such that |f(x)| <= C|g(x)| for all x>N.
But the thing is that n2 < 4n2-2n+2 for any n. Shouldn't we say that n2 = O(4n2-2n+2) in this case?
All of the below statements are true:
n2 ∈ O(n2)
n2 ∈ O(4n2-2n+2)
4n2-2n+2 ∈ O(4n2-2n+2)
4n2-2n+2 ∈ O(n2)
However talking about O(4n2-2n+2) does not make much sense as it's the exact same set as O(n2)*, so since the latter is simpler, there's no reason to refer to it as the former.
* For every function f such that ∃N,C: ∀x>N: |f(x)| ≤ C|4n2-2n+2|, it is also true that ∃N,C: ∀x>N: |f(x)| ≤ C|n2| and vice versa.
The thing about big-O notation and questions like this is to consider which term of the equation dominates as n (or x or other suitable variable name) gets really big. That is, which term contributes most to the overall shape of the equation graph. What term should you plot against the equation result to get the closest approximation of a straight line (approximately one-to-one correspondence).
With regards to the rest of your question, it doesn’t say that C>1. I presume C>0. As n grows, 2n + 2 becomes tiny in comparison with the squared term.
With regards as to why it’s relevant to coding: how long will your code take to run? Can you make it run faster? Which equation/code is more efficient? How big do your variables need to be (i.e. int or long choices in C). I presume if there is a "big-o" tag, there’s been at least one question on this before.
Related
I need an explanation of this section of Chapter 3 in Introduction to Algorithms.
As an example, consider any quadratic function f(n) = an² + bn + c,
where a, b, and c are constants and a > 0. Throwing away the
lower-order terms and ignoring the constant yields f(n) = Θ(n²).
Formally, to show the same thing, we take the constants c₁ = a/4, c₂ = 7a/4, and n₀ = 2 · max(|b|/a, √(|c|/a)). You may verify that 0 ≤ c₁n² ≤ an²+bn+c ≤ c₂n² for all n ≥ n₀.
It makes complete sense to me that any quadratic function will be Θ(n²) because you drop the lower order terms and the coefficient of the highest order term. But, in this proof, why are these constant values chosen for c₁, c₂, and n₀? They seem arbitrary but perhaps there is something I'm missing. I generally understand the definition of Θ but not this proof as it relates to quadratic functions.
The parameters seem arbitrary, because they are. The definition only says that such parameters exists and normally there are a lot of parameters that work. Just choose a c and calculate a n for that specific c.
Do you think the following information is true?
If Θ(f(n)) = Θ(g(n)) AND g(n) > 0 everywhere THEN f(n)/g(n) ∈ Θ(1)
We are having bit of argument with our prof
f(n) = Θ(g(n)) means there's c, d, n0 such that cg(n) <= f(n) <= dg(n) for n > n0.
Then, since g(n) > 0, c <= f(n)/g(n) <= d for n > n0.
So f(n)/g(n) = Θ(1).
Dividing functions f(n),g(n) is not the same as dividing their Big-O. For example let:
f(n) = n^3 + n^2 + n
g(n) = n^3
so:
O(f(n)) = n^3
O(g(n)) = n^3
but:
f(n)/g(n) = 1 + 1/n + 1/n^2 != constant !!!
[Edit1]
but as kfx pointed you are comparing with complexity so you want:
O(f(n)/g(n)) = O(1 + 1/n + 1/n^2) = O(1)
So the answer is Yes.
But beware complexity theory is not really my cup of tea and also I do not have any context to the question of yours.
Using definitions for Landau notation https://en.wikipedia.org/wiki/Big_O_notation, it's easy to conclude that this is true, the limit of division must be less than infinity but larger than 0.
It does not have to be exactly 1 but it has to be a finite constant, which is Θ(1).
A counter example would be nice, and should be easy to be given if the statement isn't true. A positive rigorous proof would probably need to go from definition of limes with respect to series, to prove equivalence of formal and limit definitions.
I use this definition and haven't seen it proven wrong. I suppose the disagreement might lie in exact definition of Θ, it is known that people use those colloquially with minor differences, especially Big O. Or maybe some tricky cases. For positively defined functions and series, I don't think it fails.
Basically there are three options for any pair of functions f, g: Either the first grows asymptotically slower and we write f=o(g) (notice I'm using small o), the first grows asymptotically faster: f=ω(g) (again, small omega) or they are asymptotically tightly bound: f=Θ(g).
What f=o(g) means is stricter then big O in that it doesn't allow for f=Θ(g) to be true; f=Θ(g) implies both f=O(g) and f=Ω(g), but o, Θ and ω are exclusive.
To find out whether f=o(g) it's sufficient to evaluate limit for n going to infinity f(n)/g(n) and if it is zero, f=o(g) is true, if it is infinity f=ω(g) is true and if it is any real finite number, f=Θ(g) is your answer. This is not a definition, but merely a way to evaluate a statement. (One assumption I made here was that both f and g are positive.)
Special case is if limit for n goint to infinity f(n)/1 = f(n) is finite number, it means f(n)=Θ(1) (basically we chose constant function for g).
Now we're getting to your problem: Since f=g(Θ)implies f=O(g), we know that there exists c>0 and n0 such that f(n) <= c*g(n)for all n>n0. Thus we know that f(n)/g(n) <= (c*g(n))/g(n) = cfor all n>n0. The same can be done for Ω just with opposite unequality signs. Thus we get that f(n)/g(n)is between c1and c2 from some n0 which are known to be finite numbers because of how Θ is defined. Because we know our new function is somewhere in there we also know that its limit is finite number, thus proving it is indeed constant.
Conclusion, I believe you were right and I would like your professor to offer counterexample to dispruve the statement. If something didn't make sense feel free to ask more in the comments, I'll try to clarify.
I have two algorithms.
The complexity of the first one is somewhere between Ω(n^2*(logn)^2) and O(n^3).
The complexity of the second is ω(n*log(logn)).
I know that O(n^3) tells me that it can't be worse than n^3, but I don't know the difference between Ω and ω. Can someone please explain?
Big-O: The asymptotic worst case performance of an algorithm. The function n happens to be the lowest valued function that will always have a higher value than the actual running of the algorithm. [constant factors are ignored because they are meaningless as n reaches infinity]
Big-Ω: The opposite of Big-O. The asymptotic best case performance of an algorithm. The function n happens to be the highest valued function that will always have a lower value than the actual running of the algorithm. [constant factors are ignored because they are meaningless as n reaches infinity]
Big-Θ: The algorithm is so nicely behaved that some function n can describe both the algorithm's upper and lower bounds within the range defined by some constant value c. An algorithm could then have something like this: BigTheta(n), O(c1n), BigOmega(-c2n) where n == n throughout.
Little-o: Is like Big-O but sloppy. Big-O and the actual algorithm performance will actually become nearly identical as you head out to infinity. little-o is just some function that will always be bigger than the actual performance. Example: o(n^7) is a valid little-o for a function that might actually have linear or O(n) performance.
Little-ω: Is just the opposite. w(1) [constant time] would be a valid little omega for the same above function that might actually exihbit BigOmega(n) performance.
Big omega (Ω) lower bound:
A function f is an element of the set Ω(g) (which is often written as f(n) = Ω(g(n))) if and only if there exists c > 0, and there exists n0 > 0 (probably depending on the c), such that for every n >= n0 the following inequality is true:
f(n) >= c * g(n)
Little omega (ω) lower bound:
A function f is an element of the set ω(g) (which is often written as f(n) = ω(g(n))) if and only for each c > 0 we can find n0 > 0 (depending on the c), such that for every n >= n0 the following inequality is true:
f(n) >= c * g(n)
You can see that it's actually the same inequality in both cases, the difference is only in how we define or choose the constant c. This slight difference means that the ω(...) is conceptually similar to the little o(...). Even more - if f(n) = ω(g(n)), then g(n) = o(f(n)) and vice versa.
Returning to your two algorithms - the algorithm #1 is bounded from both sides, so it looks more promising to me. The algorithm #2 can work longer than c * n * log(log(n)) for any (arbitrarily large) c, so it might eventually loose to the algorithm #1 for some n. Remember, it's only asymptotic analysis - so all depends on actual values of these constants and the problem size which has some practical meaning.
The informal idea of Big-O is described as "it's the highest order of growth of a function" ie f(n) = 3n^2 + 5n + 50 is just O(n^2).
I do understand that Big-O is just a way of saying "guaranteed to not be worse than this period". Formally, it appears the definition is f(n) -> O(g(n)) iff f(n) <= c * g(n) where c is positive
First some mathy stuff.. if f(n) = 5n^2, g(n)=n I should be able to show 5n^2 isn't O(g(n)) by doing
5n^2 <= cn
5n <= c
If the idea is that is that c isn't a constant(I have no idea if that's a requirement), and that is proof f(n) isnt in O(g(n)), what about if g(n) were n^3 (of which it surely should be contained)?
5n^2 <= cn^3
5/n <= c
I have a misunderstanding of how the math works out for all of this I assume, so I ask:
How does all this fancy stuff work
How does it connect to the simple definition given in my data structures class?
Thanks for any help
n is a positive integer, which means that 1<=n and therefore 5/n<=5/1=5, so you can pick c=5.
A more complete definition also allows you to pick n0 and a, both constants, and only prove that f(n)<=a+c*g(n) for all n0<n
c is a constant (i.e. independent of n)
In your first example (it's proof by contradiction):
i.e. assume
5n^2 <= cn
5n <= c
But for any fixed constant c, we can find a value of n that makes it untrue.
For example pick c = 1000000, then a value of n = 200001 would be a contradiction.
In your second example, we know that f(n) is O(n^2), therefore it is also O(n^3) and above. If you are bounded by k(n^2), you are also bounded by j(n^3)
The informal idea of Big-O is described as "it's the highest order of growth of a function" ie f(n) = 3n^2 + 5n + 50 is just O(n^2).
I wouldn't say that this is the idea behind Big-O. Informally Big-O is some rough estimation of what a given function cannot exceed. And it's usage is mostly approximating how something will grow for big numbers.
For example, if we take a 6 digit number, we can definitely say that it's less than million without looking for its digits. There are a lot of cases when this is enough and we don't need to analyse all digits.
For analysing function growth two factors play their role:
we only interested in function behaviour for very large numbers
if f is bigger than g, but we can fix it with multiplying g by some big constant, that's means f's advantage is not because of growth
This leads us to two parts of the definition: (1) some constant and (2) for big enough n
And for polynomials indeed, the higher order component defines grow speed.
My question refers to the big-Oh notation in algorithm analysis. While Big-Oh seems to be a math question, it's much useful in algorithm analysis.
Suppose two functions are defined below:
f(n) = 2( to the power n) when n is even
f(n) = n when n is odd
g(n) = n when n is even
g(n) = 2( to the power n) when n is odd.
For the above two functions which one is big-Oh of other? Or whether any function is not a Big-Oh of another function.
Thanks!
In this case,
f ∉ O(g), and
g ∉ O(f).
This is because no matter what constants N and k you pick,
there exists i ≥ N such that f(i) > k g(i), and
there exists j ≥ N such that g(j) > k f(j).
The Big-Oh relationship is quite specific in that one function is, after a finite n, always larger than the other.
Is this true here? If so, give such a n. If not, you should prove it.
Usually Big-O and Big-Theta notations get confused.
A layman attempt at definition could be that Big-O means that one function is growing as fast or faster than another one, i.e. that given a large enough n, f(n)<=k*g(n) where k is some constant. That means that if f(x) = 2x^3, then it is in O(x^3), O(x^4), O(2^x), O(x!) etc..
Big-Theta means that one function is growing as fast as another one, with neither one being able to "outgrow" the other, or, k1*g(n)<=f(n)<=k2*g(n) for some k1 and k2. In programming terms that means that these two functions have the same level of complexity. If f(x) = 2x^3, then it is in Θ(x^3), as for example, if k1=1, and k2=3, 1*x^3 < 2*x^3 < 3*x^3
In my experience whenever programmers are talking about Big-O, the discussion is actually about Big-Θ, as we are concerned more with the as fast as part more, than in the no faster than part.
That said, if two functions with different Θ's are combined, as in your example, the larger one - (Θ(2^n) - swallows the smaller - Θ(n), so both f and g have the exact same Big-O and Big-Θ complexities. In this case, it's both correct that
f(n) = O(g(n)), also f(n) = Θ(g(n))
g(n) = O(f(n)), also g(n) = Θ(f(n))
so, as they have the same complexity, they are O and Θ bound by each other.