Here's an example from Wikipedia:
Let f(x) = 6x^4 - 2x^3 + 5, then f(x) is a "big O" of x^4.
Then I swap the coefficients between x^3 and x^4. Now that f(x) = -2x^4 + 6x^3 + 5. Will f(x) still be a "big O" of x^4? I assume not, since f(x) will take less time as x becomes larger. My question is:
What is the big-o complexity of the above equation? At the moment, I'm ignoring -2x^4, which makes f(x) is a "big O" of x^3, but I'm not sure about that.
The expression -2x^4 + 6x^3 + 5 is still O(x^4), because the leading constant is not part of the big-O expression. Empirically, it should be clear that as x gets arbitrarily large, the x^3 term will basically become irrelevant. Whether the x^4 terms becomes a very large positive or negative number has no bearing on the running time.
Big-O and related asymptotic classes are usually defined in terms of the absolute value of the function, so that it doesn't matter whether it is increasing or decreasing asymptotically.
So it would still be Theta(x^4), which also implies O(x^4).
In context of time complexity analysis this isn't really relevant though, since you cannot have a program/algorithm taking negative time, so the function you are asking about cannot be a function describing the execution time of a program/algorithm.
Related
I‘m trying to wrap my head around the meaning of the Landau-Notation in the context of analysing an algorithm‘s complexity.
What exactly does the O formally mean in Big-O-Notation?
So the way I understand it is that O(g(x)) gives a set of functions which grow as rapidly or slower as g(x), meaning, for example in the case of O(n^2):
where t(x) could be, for instance, x + 3 or x^2 + 5. Is my understanding correct?
Furthermore, are the following notations correct?
I saw the following written down by a tutor. What does this mean? How can you use less or equal, if the O-Notation returns a set?
Could I also write something like this?
So the way I understand it is that O(g(x)) gives a set of functions which grow as rapidly or slower as g(x).
This explanation of Big-Oh notation is correct.
f(n) = n^2 + 5n - 2, f(n) is an element of O(n^2)
Yes, we can say that. O(n^2) in plain English, represents "set of all functions that grow as rapidly as or slower than n^2". So, f(n) satisfies that requirement.
O(n) is a subset of O(n^2), O(n^2) is a subset of O(2^n)
This notation is correct and it comes from the definition. Any function that is in O(n), is also in O(n^2) since growth rate of it is slower than n^2. 2^n is an exponential time complexity, whereas n^2 is polynomial. You can take limit of n^2 / 2^n as n goes to infinity and prove that O(n^2) is a subset of O(2^n) since 2^n grows bigger.
O(n) <= O(n^2) <= O(2^n)
This notation is tricky. As explained here, we don't have "less than or equal to" for sets. I think tutor meant that time complexity for the functions belonging to the set O(n) is less than (or equal to) the time complexity for the functions belonging to the set O(n^2). Anyways, this notation doesn't really seem familiar, and it's best to avoid such ambiguities in textbooks.
O(g(x)) gives a set of functions which grow as rapidly or slower as g(x)
That's technically right, but a bit imprecise. A better description contains the addenda
O(g(x)) gives the set of functions which are asymptotically bounded above by g(x), up to constant factors.
This may seem like a nitpick, but one inference from the imprecise definition is wrong.
The 'fixed version' of your first equation, if you make the variables match up and have one limit sign, seems to be:
This is incorrect: the ratio only has to be less than or equal to some fixed constant c > 0.
Here is the correct version:
where c is some fixed positive real number, that does not depend on n.
For example, f(x) = 3 (n^2) is in O(n^2): one constant c that works for this f is c = 4. Note that the requirement isn't 'for all c > 0', but rather 'for at least one constant c > 0'
The rest of your remarks are accurate. The <= signs in that expression are an unusual usage, but it's true if <= means set inclusion. I wouldn't worry about that expression's meaning.
There's other, more subtle reasons to talk about 'boundedness' rather than growth rates. For instance, consider the cosine function. |cos(x)| is in O(1), but its derivative fluctuates from negative one to positive one even as x increases to infinity.
If you take 'growth rate' to mean something like the derivative, example like these become tricky to talk about, but saying |cos(x)| is bounded by 2 is clear.
For an even better example, consider the logistic curve. The logistic function is O(1), however, its derivative and growth rate (on positive numbers) is positive. It is strictly increasing/always growing, while 1 has a growth rate of 0. This seems to conflict with the first definition without lots of additional clarifying remarks of what 'grow' means.
An always growing function in O(1) (image from the Wikipedia link):
I'm taking the "Intro To Algorithms" course on Coursera, and I've arrived at the video which deals with Big-Theta, Big-Omega and Big-O notation. The end-of-video quiz presents the following question:
Q: Which of the following functions is O(N^3)?
a) 11N + 15lgN + 100
b) (N^2)/3
c) 25,000*(N^3)
d) All of the above
I answered "c" and was told my answer was incorrect, and that the correct answer is actually "d". The explanation provided by the course wasn't much help:
Recall that big-Oh notation provides only an upper bound on the growth
rate of a function as N gets large. In this course, we primarily use
tilde notation because it more accurately describes the function—it
provides both an upper and lower bound on the function as well as the
coefficient of the leading term.
I was under the impression that one should drop the lesser-order terms (i.e. "15lgN + 100") and focus only on the highest-order terms. Furthermore, I can't see how N^3 could be the upper bound on a quadratic (as opposed to a cubic) function like N^2.
So my question is, why are "a" and "b" classified as O(N^3) in this case?
Do you know, f(n) = O(g(n)) implies f(n) <= constant* g(n), right?
In other words, it means, when you plot the graph of f(n) and g(n) then after some value of, g(n) will always be more than f(n).
Here g(n) is N^3 and remaining comes in f(n). Now, N^3 is always >= options a, b, c. hence answer id D :)
Edit:
Following statements are true,
n=O(n)
n=O(n^2)
n=O(n^3)
But only n = O(n) is tight upper bound and that is what we should use in time complexity derivation of algorithms. If we are using 2nd and 3rd option, then we are misusing the Big-O notation or let's say they are upper bounds but not tightly bounded!
Edit 2: See following image
G(x) is tight upper bound for F(x) and H(x) is upper of F(x) but not tight! Still we would say, F(x)=O(G(x)) & F(x)=O(H(x)). When somebody in exam/interview ask for time complexity they are asking for tight bounds, but not an upper bound. Unfortunately, tight upper bound and upper bound terms are used in exams/interviews interchangeably.
The explanation says it: "Recall that big-Oh notation provides only an upper bound on the growth
rate of a function as N gets large."
In this particular context, the upper bound can be read as "does not grow faster than N³".
It is true that 11N + 15lgN + 100 does not grow faster than N³.
Think of O(N^2) also being O(n^3), O(n^4) and so on. O(N^2) is always bound under O(n^3), therefore O(n^2) is indeed O(n^3).
http://en.wikipedia.org/wiki/Big_O_notation#/media/File:Big-O-notation.png
As many have already quoted a function f(n) which has upper bound say O(n) complexity is also O(n^2) , O(n^3), O(n^4)... etc
Does that make sense or if it gets confused, think in absolute layman terms.
Suppose an process takes max upper bound of 10 secs to execute, come whatever be the input we can conclude this :-
Whatever be the input the execution will complete in less or equal to 10 seconds.
If that is true, even the following is true :-
Whatever be the input the execution will complete in less or equal to 100 seconds.
Whatever be the input the execution will complete in less or equal to 1000 seconds.
and so on.......
And thus you can correlate the answer. Hope that gave you a glimpse.
I know in Big O Notation we only consider the highest order, leading polynomial term because we are basically placing this theoretic worst case bound on compute-time complexity but sometimes I get confused on when we can legally drop/consider terms as constants. For example if our equation ran in
O((n^3)/3) --> we pull out the "1/3" fraction, treat it as a constant, drop it, then say our algo runs in O(n^3) time.
What about in the case of O((n^3)/((log(n))^2))? In this case could we pull out the 1/((log(n)^2)) term, treat it as a constant, drop it, and then ultimately conclude our algorithm is O(n^3). It does not look like we can, but what differentiates this case from the above case? both can be treated as constants because their values are relatively small in comparison to the leading polynomial term in the numerator but in the second case, the denominator term really brings down the worst case bound (convergence) as n values get larger and larger.
At this point, it starts to be a good idea to go back and look at the formal definition for big O notation. Essentially, when we say that f(x) is O(g(x)) we mean that there exists a constant factor a and a starting input n0 such that for all x >= n0 then f(x) <= a*g(x).
For a concrete example, to prove that 3*x^2 + 17 is O(n^2) we can use a = 20 and n0 = 1.
From this definition it becomes easy to see why the constant factors get dropped off - its just a matter of adjusting the a to compensate. As for your 1/log(n) question, if we have f(x) is O(g(x)) and g(x) is O(h(x)) then we also have f(x) is O(h(x)). So yes, 10*n^3/log(n) + x is O(n^3) but that is not a tighter upper bound and it is a weaker statement than saying that 10*n^3/log(n) + x is O(n^3/log(n)). For a tight bounds you would want to use big-Theta notation instead.
Any value which is fixed and does not depend on a variable (like n) can be treated as constant. You can separate the constants, remove the lower order terms and classify the result as the complexity. Also big O notation states if
f(x) <= c*g(x)
Then f(x) ~ O(g(x)). For example-
n^3 * 5 -> here 5 is a constant. Complexity is O(n^3)
4*(n^3)/((log(n))^2 + 7 -> here 7 and 4 are constants. Complexity is O(n^3/(logn)^2)
I have used the Master Theorem to solve recurrence relations. I have gotten it down to Θ(3n2-9n). Does this equal Θ(n2)? I have another recurrence for which the solution is Θ(2n3 - 1002). In BigTheta notation do you always use only the largest term? So my second one would be Θ(n3)? It just seems like 100n2 would be more important in the second case. So will it matter if I discard it?
Any suggestions?
Yes. Your assumptions are correct. The first one is Θ(n2) and the second one is Θ(n3). When you are using Θ notation you only require the largest term.
In case of your second recurrence consider the n = 1000, then n3 = 1000000000. Where as 100n2 is just 100000000. As the value of n increases, n3 becomes more and more predominant than 100n2.
For theoretical purpose you don't need to consider the constant, how ever large it might be. But practical applications might prefer an algorithm with a small constant even if the complexity is high. For example it might be better to use an algorithm having complexity 0.01n 3 over an algorithm having 10000n2 complexity if the value of n is not very large.
if we have function f(n) = 3n^2-9n , lower order terms and costants can be ignored, we consider higher order terms ,because they play major role in growth of function.
By considering only higher order term we can easily find the upper bound, here is the example.
f(n)= 3n^2-9n
For all sufficient large value of n>=1,
3n^2<=3n^2
and -9n <= n^2
thus, f(n)=3n^2 -9n <= 3n^2 + n^2
<= 4n^2
*The upper bound of f(n) is 4n^2 , that means for all sufficient large
value of n>=1, the value of f(n) wouldn't be greater than 4n^2.*
therefore, f(n)= Θ(n^2) where c=4 and n0=1
we can directly find the upper bound by saying to ignore lower order terms and constants in the equation f(n)= 3n^2-9n , result will be the same Θ(n^2)
I know that the big-o complexity of the following equation is O(n^3)
4n^3 + 6n + 6
because the n^3 is the dominant term.
Would the big-o complexity be the same for the same function, but with a negative coefficient?
-4n^3 + 6n + 6
Actually if you have negative terms in a big-O computation, you can ignore them because they make you win time.
In this case, the complexity would be O(n).
Don't know to what kind of algorithm something like that could correspond though, but to answer the general question, you could have something like O(an^2 - bn) which would give a O(n^2)complexity.
Edit:
Found a funny related question, about time travelling in algorithm solving.
Formally we analyse monotonically increasing function.
It's implied by formal definitions of asymptotic complexity.
Let's look at one of definitions, at wikipedia:
One writes
f(x) = O(g(x)) as x -> inf
if and only if there is a positive constant M such that for all
sufficiently large values of x, f(x) is at most M multiplied by g(x)
in absolute value. That is, f(x) = O(g(x)) if and only if there exists
a positive real number M and a real number x0 such that
|f(x)| <= M|g(x)| for all x>x0
As you can see, this definition works on absolute values.
In some other sources (like books about data structures and algorithms) you might find definitions without absolute value, but somewhere assumption that analysed functions are monotonically increasing (warning: sometimes assumptions are hidden in book references or implied by properties of analysed universe).
To sum it up: asymptotic analysis are designed to be used on monotonically increasing functions. Sometimes it's enforced by assumption, sometimes by absolute value in equation.
You may find other agrugments like this another SO answer , but with same conclusion.