I know that in terms of complexity, O(logn) is faster than O(n), which is faster than O(nlogn), which is faster than O(n2).
But what about O(n2) and O(n2log), or O(n2.001) and O(n2log):
T1(n)=n^2 + n^2logn
What is the big Oh and omega of this function? Also, what's little oh?
versus:
T2(n)=n^2.001 + n^2logn
Is there any difference in big Oh now?
I'm having trouble understanding how to compare logn with powers of n. As in, is logn approximately n^0.000000...1 or n^1.000000...1?
O(n^k) is faster than O(n^k') for all k, k' >= 0 and k' > k
O(n^2) would be faster than O(n^2*logn)
Note that you can only ignore constants, nothing involving the input size can be ignored.
Thus, complexity of T(n)=n^2 + n^2logn would be the worse of the two, which is O(n^2logn).
Little-oh
Little oh in loose terms is a guaranteed upper bound. Yes, it is called little, and it is more restrictive.
n^2 = O(n^k) for k >= 2 but n^2 = o(n^k) for k > 2
Practically, it is Big-Oh which takes most of the limelight.
What about T(n)= n^2.001 + n^2logn?
We have n2.001 = n2*n0.001 and n2 * log(n).
To settle the question, we need to figure out what would eventually be bigger, n0.001 or log(n).
It turns out that a function of the form nk with k > 0 will eventually take over log(n) for a sufficiently large n.
Same is the case here, and thus T(n) = O(n2.001).
Practically though, log(n) will be larger than n0.001.
(103300)0.001 < log(103300) (1995.6 < 3300), and the sufficiently large n in this case would be just around 103650, an astronomical number.
Worth mentioning again, 103650. There are 1082 atoms in the universe.
T(n)=n^2 + n^2logn
What is the big Oh and omega of this function? Also, what's little oh?
Quoting a previous answer:
Don't forget big O notation represents a set. O(g(n)) is the set of
of all function f such that f does not grows faster than g,
formally is the same is saying that there exists C and n0 such
that we have |f(n)| <= C|g(n)| for every n >= n0. The expression
f(n) = O(g(n)) is a shorthand for saying that f(n) is in the set
O(g(n))
Also you can think of big O as ≤ and of small o as < (reference). So you care of more of finding relevant big O bound than small o. In your case it's even appropriate to use big theta which is =. Since n^2 log n dominates n^2 it's true that
T1(n)=n^2 + n^2logn = Ө(n^2 logn)
Now the second part. log n grows so slowly that even n^e, e > 0 dominates it. Interestingly, you can even prove that lim n^e/(logn)^k=inf as n goes to infinity. From this you have that n^0.001 dominates log n then
T2(n)=n^2.001 + n^2logn = Ө(n^2.001).
If f(n) = Ө(g(n)) it's also true that f(n) = O(g(n)) so to answer your question:
T1(n)=O(n^2 logn)
T2(n)=O(n^2.001)
Related
I wanted to ask if someone could answer this questions and also check my solutions. I don't really get the Big-O of these functions.
e(n)= n! + 2^n
f(n)= log10(n) * n/2 + n
g(n) = n2 + n * log(n)
h(n) = 2^30n * 4^log2(n)
I thought that:
e(n) n! because n! is exponentially rising.
f(n) n(log)n but I don't really know why
g(n) n^2
h(n) n
I would appreciate any answer. Thanks.
Big O notation allows us to evaluate the growth of a function with respect to some quantity as it tends towards a infinity.
The Big O notation of a function is that of it's fastest growing constituent. Big O notation bounds the growth of a function.
For example if a function is said to be O(n).
There exists some k such that f(n) <= k * n for all n.
We ignore all other constituents of the equation as we are looking at values that tend towards infinity. As n tends towards infinity the other constituents of the equation are drowned out.
We ignore any constants as we are describing the general relationship of the function as it tends towards some value. Constants make things harder to analyze.
e(n) = n! + 2^n. A factorial is the fastest growing constituent in the equation so the Big O notation is O(n!).
f(n) = log10(n) * n/2 + n. The fast growing constituent is log10(n) * n/2. we say that the Big O notation is O(nlogn). It does not matter what base of log we use as we can convert between log bases by using a constant factor.
g(n) = n^2 + n * log(n). The Big O notation is n^2. This is because n^2 grows faster than n * log(n) with respect to n.
h(n) = (2^30) * n * 4 ^ log2(n). The time complexity is O(nlogn). The constants in this equation are 2^30 and 4 so we ignore these values. When doing this you can clearly see that the time complexity is O(nlogn).
Two questions:
First, if f(n) = n(3n + nlog(n)) then why is f(n) Ω(n2)?
Second, why is n2log(n) not O(n2)?
These are both consequences of the fact that log(n) tends to infinity as n tends to infinity.
1) n(3n + nlog(n)) is omega(n^2) because for large n the 3n is negligible and n^2log(n) is bounded below by n^2
2) n^2log(n) is not O(n^2) since, for any constant K > 0, for any n > e^K you have that n^2log(n) > Kn^2, so no K satisfies n^2log(n) < Kn^2 for all but finitely many n.
Ω( ) is for putting the bound under the function. That means that the function will have running time complexity at least more that what is mentioned between the parentheses in Ω( ). Now, for the function f(n) = n(3n + nlog(n)), the dominating function out of the two functions involved (3n2 and n2Log(n)) is n2Log(n).
Therefore, any function smaller than the dominating function can act as the lower bound even if it may not be the tightest possible lower bound. So, f(n) is Ω(n2).
Next, growth rate of n2Log(n) is higher than n2 (as I already mentioned above which one is dominating). Therefore, n2Log(n) is a tighter bound (and therefore a better determinant) about the Big - O of f(n). Hence, f(n) is O(n2Log(n)) instead of O(n2).
So the answer to this question What is the difference between Θ(n) and O(n)?
states that "Basically when we say an algorithm is of O(n), it's also O(n2), O(n1000000), O(2n), ... but a Θ(n) algorithm is not Θ(n2)."
I understand Big O to represent upper bound or worst case with that I don't understand how O(n) is also O(n2) and the other cases worse than O(n).
Perhaps I have some fundamental misunderstandings. Please help me understand this as I have been struggling for a while.
Thanks.
It's helpful to think of what big-Oh means: if a function is O(n), then c*n, where c is some positive number, is the upper-bound. If c*n is an upper-bound, it's clear that for integers, c*n^2 would also be an upper-bound. Also c*n^3, c*n^4, c*n^1000, etc.
The below graph shows the growth of functions, which are upper bounds of the function "to the right" of it; i.e., it grows faster on smaller n.
Suppose the running time of your algorithm is T(n) = 3n + 6 (i.e., an arbitrary polynomial of order 1).
It's true that T(n) = O(n) because 3n + 6 < 4n for all n > 5 (to use the definition of big-oh notation). It's also true that T(n) = O(n^2) because 3n + 6 < n^2 for all n > 5 (to use the defintion again).
It's also true that T(n) = Θ(n) because, in addition to the proof that it was O(n), it is true that 3n + 6 > n for all n > 1. However, you cannot prove that 3n + 6 > c n^2 for any value of c for arbitrarily large n. (Proof sketch: lim (cn^2 - 3n - 6) > 0 as n -> infinity).
I understand Big O to represent upper bound or worst case with that I don't understand how O(n) is also O(n2) and the other cases worse than O(n).
Intuitively, an "upper bound of x" means that something will always be less than or equal to x. If something is less than or equal to x, it is also less than or equal to x^2 and x^1000, for large enough values of x. So x^2 and x^1000 can also be upper bounds.
This is what Big-oh represents: upper bounds.
When we say that f(n) = O(g(n)), we mean only that for all sufficiently large n, there exists a constant c such that f(n) <= cg(n). Note that if f(n) = O(g(n)), we can always choose a function h(n) bigger than g(n) and since g(n) is eventually less than h(n), we have f(n) <= cg(n) <= ch(n), so f(n) = O(h(n)) as well.
Note that the O bound is not tight. The theta bound is the intersection of O(g(n)) and Omega(g(n)), where Omega gives the lower bound (it's like O, the upper bound, but bounds from below instead). If f(n) is bounded below by g(n), and h(n) is bigger than g(n), then if follows that f(n) is not (necessarily) bounded below by h(n).
for(i: 1 to n^2)
x = x + 1;
return x + 1;
N is the number of inputs. N>1 and tends to infinity
I understand that the worst (and the best) case running time is n^2 + 1. Hence, it'll be O(n^2). However, how do I find if it is a tight, big O expression? How do we find a tight big O expression? What is that?
Let your function be f(n)
You already worked out that the best case is n^2 (ommit +1 since constants indepently from the input are ignored).
More formally: Ω(n^2)
--> means you can find a function u(n) and a constant k such that k * u(n) <= f(n) in terms of complexity.
As you said the worst case is n^2 too.
Again formally: O(n^2)
--> means you can find a function v(n) and a constant k such that k * v(n) >= f(n) in terms of complexity.
Since Ω and O are sets of functions Theta is defined as the intersection of Ω and O. "Theta is upper and lower bound."
The intersection is n^2 --> Theta (n^2)
In an descriptive manner:
f(n) grows not much faster (or equal) than u(n).
f(n) grows not much faster (or equal) than v(n).
--> f(n) grows exactly like n^2
Keep in mind that mor often Ω and O are sets of different classes of functions.
E.g. Ω(log n)(searching in a tree) and O(n) (searching in a String) When dealing with such a case you are unable to specify an exact Theta value.
I was reading Intro to Algorithms, by Thomas H. Corman when I encountered this statement (in Asymptotic Notations)
when a>0, any linear function an+b is in O(n^2) which is essentially verified by taking c = a + |b| and no = max(1, -b/a)
I can't understand why O(n^2) and not O(n). When will O(n) upper bound fail.
For example, for 3n+2, according to the book
3n+2 <= (5)n^2 n>=1
but this also holds good
3n+2 <= 5n n>=1
So why is the upper bound in terms of n^2?
Well I found the relevant part of the book. Indeed the excerpt comes from the chapter introducing big-O notation and relatives.
The formal definition of the big-O is that the function in question does not grow asymptotically faster than the comparison function. It does not say anything about whether the function grows asymptotically slower, so:
f(n) = n is in O(n), O(n^2) and also O(e^n) because n does not grow asymptotically faster than any of these. But n is not in O(1).
Any function in O(n) is also in O(n^2) and O(e^n).
If you want to describe the tight asymptotic bound, you would use the big-Θ notation, which is introduced just before the big-O notation in the book. f(n) ∊ Θ(g(n)) means that f(n) does not grow asymptotically faster than g(n) and the other way around. So f(n) ∊ Θ(g(n)) is equivalent to f(n) ∊ O(g(n)) and g(n) ∊ O(f(n)).
So f(n) = n is in Θ(n) but not in Θ(n^2) or Θ(e^n) or Θ(1).
Another example: f(n) = n^2 + 2 is in O(n^3) but not in Θ(n^3), it is in Θ(n^2).
You need to think of O(...) as a set (which is why the set theoretic "element-of"-symbol is used). O(g(n)) is the set of all functions that do not grow asymptotically faster than g(n), while Θ(g(n)) is the set of functions that neither grow asymptotically faster nor slower than g(n). So a logical consequence is that Θ(g(n)) is a subset of O(g(n)).
Often = is used instead of the ∊ symbol, which really is misleading. It is pure notation and does not share any properties with the actual =. For example 1 = O(1) and 2 = O(1), but not 1 = O(1) = 2. It would be better to avoid using = for the big-O notation. Nonetheless you will later see that the = notation is useful, for example if you want to express the complexity of rest terms, for example: f(n) = 2*n^3 + 1/2*n - sqrt(n) + 3 = 2*n^3 + O(n), meaning that asymptotically the function behaves like 2*n^3 and the neglected part does asymptotically not grow faster than n.
All of this is kind of against the typically usage of big-O notation. You often find the time/memory complexity of an algorithm defined by it, when really it should be defined by big-Θ notation. For example if you have an algorithm in O(n^2) and one in O(n), then the first one could actually still be asymptotically faster, because it might also be in Θ(1). The reason for this may sometimes be that a tight Θ-bound does not exist or is not known for given algorithm, so at least the big-O gives you a guarantee that things won't take longer than the given bound. By convention you always try to give the lowest known big-O bound, while this is not formally necessary.
The formal definition (from Wikipedia) of the big O notation says that:
f(x) = O(g(x)) as x → ∞
if and only if there is a positive constant M such that for all
sufficiently large values of x, f(x) is at most M multiplied by g(x)
in absolute value. That is, f(x) = O(g(x)) if and only if there exists
a positive real number M and a real number x0 such that
|f(x)|≤ M|g(x)| for all x > x₀ (mean for x big enough)
In our case, we can easily show that
|an + b| < |an + n| (for n sufficiently big, ie when n > b)
Then |an + b| < (a+1)|n|
Since a+1 is constant (corresponds to M in the formal definition), definitely
an + b = O(n)
Your were right to doubt.