I have a problem that my professor gave us:
Algorithms A and B spend exactly TA(n) = 0.1n^2log10(n) and TB(n) =
2.5n^2 microseconds, respectively, for a problem of size n. Choose the algorithm, which is better in the Big-Oh sense, and find out a problem
size n0 such that for any larger size n > n0 the chosen algorithm
outperforms the other. If your problems are of the size n ≤ 10^9,
which algorithm will you recommend to use?
At first, I thought algorithm A would be better in the Big-Oh sense, but his answer says A is better. My reasoning for A being better is that it grows slower than B. Am I correct or is my professor correct?
Here's his answer:
In the Big-Oh sense, the algorithm B is better. It
outperforms the algorithm A when TB(n) ≤ TA(n), that is, when 2.5n^2 ≤
0.1n^2log10(n). This inequality reduces to log10(n) ≥ 25, or n ≥ n0 = 1025. If n≤ 10^9, the algorithm of choice is A
Is he correct or am I correct?
B is better in big-o terms because it takes time proportional to n squared, but A is proportional to n squared times log n which is bigger.
So for large enough values of n B will be faster.
Related
I was reading about Big-O Notation
So, any algorithm that is O(N) is also an O(N^2).
It seems confusing to me, I know that Big-O gives upper bound only.
But how can an O(N) algorithm also be an O(N^2) algorithm.
Is there any examples where it is the case?
I can't think of any.
Can anyone explain it to me?
"Upper bound" means the algorithm takes no longer than (i.e. <=) that long (as the input size tends to infinity, with relevant constant factors considered).
It does not mean it will ever actually take that long.
Something that's O(n) is also O(n log n), O(n2), O(n3), O(2n) and also anything else that's asymptotically bigger than n.
If you're comfortable with the relevant mathematics, you can also see this from the formal definition.
O notation can be naively read as "less than".
In numbers if I tell you x < 4 well then obviously x<5 and x< 6 and so on.
O(n) means that, if the input size of an algorithm is n (n could be the number of elements, or the size of an element or anything else that mathematically describes the size of the input) then the algorithm runs "about n iterations".
More formally it means that the number of steps x in the algorithm satisfies that:
x < k*n + C where K and C are real positive numbers
In other words, for all possible inputs, if the size of the input is n, then the algorithm executes no more than k*n + C steps.
O(n^2) is similar except the bound is kn^2 + C. Since n is a natural number n^2 >= n so the definition still holds. It is true that, because x < kn + C then x < k*n^2 + C.
So an O(n) algorithm is an O(n^2) algorithm, and an O(N^3 algorithm) and an O(n^n) algorithm and so on.
For something to be O(N), it means that for large N, it is less than the function f(N)=k*N for some fixed k. But it's also less than k*N^2. So O(N) implies O(N^2), or more generally, O(N^m) for all m>1.
*I assumed that N>=1, which is indeed the case for large N.
Big-O notation describes the upper bound, but it is not wrong to say that O(n) is also O(n^2). O(n) alghoritms are subset of O(n^2) alghoritms. It's the same that squares are subsets of all rectangles, but not every rectangle is a square. So technically it is correct to say that O(n) alghoritm is O(n^2) alghoritm even if it is not precise.
Definition of big-O:
Some function f(x) is O(g(x)) iff |f(x)| <= M|g(x)| for all x >= x0.
Clearly if g1(x) <= g2(x) then |f(x)| <= M|g1(x)| <= M|g2(x)|.
For an algorithm with just a single Loop will get a O(n) and algorithm with a nested loop will get a O(n^2).
Now consider the Bubble sort algorithm it uses the nested loop in it,
If we give an already sort set of inputs to a bubble sort algorithm the inner loop will never get executed so for a scenario like this it gets O(n) and for the other cases it gets O(n^2).
Talking about Big O notations, if one algorithm time complexity is O(N) and other's is O(2N), which one is faster?
The definition of big O is:
O(f(n)) = { g | there exist N and c > 0 such that g(n) < c * f(n) for all n > N }
In English, O(f(n)) is the set of all functions that have an eventual growth rate less than or equal to that of f.
So O(n) = O(2n). Neither is "faster" than the other in terms of asymptotic complexity. They represent the same growth rates - namely, the "linear" growth rate.
Proof:
O(n) is a subset of O(2n): Let g be a function in O(n). Then there are N and c > 0 such that g(n) < c * n for all n > N. So g(n) < (c / 2) * 2n for all n > N. Thus g is in O(2n).
O(2n) is a subset of O(n): Let g be a function in O(2n). Then there are N and c > 0 such that g(n) < c * 2n for all n > N. So g(n) < 2c * n for all n > N. Thus g is in O(n).
Typically, when people refer to an asymptotic complexity ("big O"), they refer to the canonical forms. For example:
logarithmic: O(log n)
linear: O(n)
linearithmic: O(n log n)
quadratic: O(n2)
exponential: O(cn) for some fixed c > 1
(Here's a fuller list: Table of common time complexities)
So usually you would write O(n), not O(2n); O(n log n), not O(3 n log n + 15 n + 5 log n).
Timothy Shield's answer is absolutely correct, that O(n) and O(2n) refer to the same set of functions, and so one is not "faster" than the other. It's important to note, though, that faster isn't a great term to apply here.
Wikipedia's article on "Big O notation" uses the term "slower-growing" where you might have used "faster", which is better practice. These algorithms are defined by how they grow as n increases.
One could easily imagine a O(n^2) function that is faster than O(n) in practice, particularly when n is small or if the O(n) function requires a complex transformation. The notation indicates that for twice as much input, one can expect the O(n^2) function to take roughly 4 times as long as it had before, where the O(n) function would take roughly twice as long as it had before.
It depends on the constants hidden by the asymptotic notation. For example, an algorithm that takes 3n + 5 steps is in the class O(n). So is an algorithm that takes 2 + n/1000 steps. But 2n is less than 3n + 5 and more than 2 + n/1000...
It's a bit like asking if 5 is less than some unspecified number between 1 and 10. It depends on the unspecified number. Just knowing that an algorithm runs in O(n) steps is not enough information to decide if an algorithm that takes 2n steps will complete faster or not.
Actually, it's even worse than that: you're asking if some unspecified number between 1 and 10 is larger than some other unspecified number between 1 and 10. The sets you pick from being the same doesn't mean the numbers you happen to pick will be equal! O(n) and O(2n) are sets of algorithms, and because the definition of Big-O cancels out multiplicative factors they are the same set. Individual members of the sets may be faster or slower than other members, but the sets are the same.
Theoretically O(N) and O(2N) are the same.
But practically, O(N) will definitely have a shorter running time, but not significant. When N is large enough, the running time of both will be identical.
O(N) and O(2N) will show significant difference in growth for small numbers of N, But as N value increases O(N) will dominate the growth and coefficient 2 becomes insignificant. So we can say algorithm complexity as O(N).
Example:
Let's take this function
T(n) = 3n^2 + 8n + 2089
For n= 1 or 2, the constant 2089 seems to be the dominant part of function but for larger values of n, we can ignore the constants and 8n and can just concentrate on 3n^2 as it will contribute more to the growth, If the n value still increases the coefficient 3 also seems insignificant and we can say complexity is O(n^2).
For detailed explanation refer here
O(n) is faster however you need to understand that when we talk about Big O, we are measuring the complexity of a function/algorithm, not its speed. And we measure this complexity asymptotically. In lay man terms, when we talk about asymptotic analysis, we take immensely huge values for n. So if you plot the graph for O(n) and O(2n), the values will stay in some particular range from each other for any value of n. They are much closer compared to the other canonical forms like O(nlogn) or O(1), so by convention we approximate the complexity to the canonical form O(n).
Is O(n Log n) in polynomial time? If so, could you explain why?
I am interested in a mathematical proof, but I would be grateful for any strong intuition as well.
Thanks!
Yes, O(nlogn) is polynomial time.
From http://mathworld.wolfram.com/PolynomialTime.html,
An algorithm is said to be solvable in polynomial time if the number
of steps required to complete the algorithm for a given input is
O(n^m) for some nonnegative integer m, where n is the complexity of
the input.
From http://en.wikipedia.org/wiki/Big_O_notation,
f is O(g) iff
I will now prove that n log n is O(n^m) for some m which means that n log n is polynomial time.
Indeed, take m=2. (this means I will prove that n log n is O(n^2))
For the proof, take k=2. (This could be smaller, but it doesn't have to.)
There exists an n_0 such that for all larger n the following holds.
n_0 * f(n) <= g(n) * k
Take n_0 = 1 (this is sufficient)
It is now easy to see that
n log n <= 2n*n
log n <= 2n
n > 0 (assumption)
Click here if you're not sure about this.
This proof could be a lot nicer in latex math mode, but I don't think stackoverflow supports that.
It is, because it is upper-bounded by a polynomial (n).
You could take a look at the graphs and go from there, but I can't formulate a mathematical proof other than that :P
EDIT: From the wikipedia page, "An algorithm is said to be of polynomial time if its running time is upper bounded by a polynomial expression in the size of the input for the algorithm".
It is at least not worse than polynomial time. And still not better: n < n log n < n*n.
Yes. What's the limit of nlogn as n goes to infinity? Intuitively, for large n, n >> logn and you can consider the product dominated by n and so nlogn ~ n, which is clearly polynomial time. A more rigorous proof is by using the the Sandwich theorem which Inspired did:
n^1 < nlogn < n^2.
Hence nlogn is bounded above (and below) by a sequence which is polynomial time.
Given a list of complexities:
How do you order then in their Big O order?
I think the answer is below?
Question now is how does log(n!) become n log(n). Also I don't know if I got the n! and (n-1)! right. Is it possible that c^n can be bigger than n!? When c > n?
In general how do I visualize such Big O problem ... it took me quite long to do this ... compared to coding so far ... Any resources, videos MIT Open Courseware resources, something with explaination
You might want to see how the functions grow. Here's a quick plot from Wolfram Alpha:
link
In general, n^n grows much faster than c^n for any n greater than some n_0 (because n will overtake c at some point, even if c is extremely large). log grows much slower than quadratic or exponential, and slightly faster than linear.
For O(log(n!)) = O(nlogn), I believe there was something called Stirling's Approximation. It boils down to seeing that O(n!) = O(n^n) as n! = n*(n-1)*(n-2)*...*2*1, so n^n = n*n*n*...*n is an upper bound. It can be proven that is it a lower bound as well, but you don't need that.
Since log(n^n) = nlogn by log rules, O(log(n!) = O(log(n^n)) = O(nlogn).
If algorithm A has complexity O(n) and algorithm B has complexity o(n^2), what, if anything, can we say about the relationship between A and B? Note: the complexity of A is expressed using big-Oh, and the complexity of B is expressed using little-Oh.
As wikipedia expressively puts it,
"f(x) is little-o of g(x)" ... means
that g(x) grows much faster than
f(x)
(my ellipsis and emphasis) -- putting it in the somber way that "mathematician wannabes" seem to require, the limit of f(x)/g(x) tends to 0 at the limit when x tends to infinity.
So, there's not all that much we can say: A grows no faster than n, B grows much slower than n squared, it might even be the same algorithm you're talking about;-) -- just as if you were expressing the same constraints verbally ("A is no worse than linear, B is better than quadratic").
Indeed, anything O(n) will be o(n squared), though not vice versa; for example, x to the power of K for some constant K is going to be O(n) for any K <= 1, but it will be o(n squared) for any K < 2. Other functions that are o(n squared) but not O(N) include the commonly-encountered n log n.