Assuming some algorithm has a polynomial time complexity T(n), is it possible for any of the terms to have a negative coefficient? Intuitively, the answer seems like an obvious "No" since there is no part of any algorithm that reduces the existing amount of time taken by previous steps but I want to be certain.
When talking about polynomial complexity, only the coefficient with the highest degree counts.
But I think you can have T(n) = n*n - n = n*(n-1). The n-1 would represent something you don't do on the first or last iteration.
Anyway, the complexity would still be n*n.
It is possible for an algorithm to have a negative coefficient in its time complexity, but overall the algorithm will have some positive time complexity. As an example from Wikipedia, take the function f(x)=6x^4-2x^3+5. They solve for the complexity of O(x^4) as follows:
for some suitable choice of x0 and M and for all x > x0. To prove this, let x0 = 1 and M = 13. Then, for all x > x0:
So,
That is, even if there are negative coefficients in the original equation, there is still some positive overall time complexity based on the term with the highest order of power.
What about for lower bounds? By definition, we can find the lower bound of any function by using the following definition: As n goes to infinity, then for some constant k and some n0 we have that the following holds for all n>n0:
Let's guess that the above function f(x) is also Omega(x^4). This means that:
6x^4 - 2x^3 + 5 >= kx^4
Solving for k:
k <= (6x^4 - 2x^3 + 5)/(x^4)
k <= 6 - 2x^-1 + 5x^-4
The term (2/x) approaches 0, as does (5/x^4) so we can choose k=2 for some large x0=30. To show that this holds, we show that:
6x^4 - 2x^3 + 5 >= 2x^4 where x > 30
4x^4 - 2x^3 + 5 >= 0
Which holds. So f(x) is Omega(x^4), and we can also conclude that we have found a tight bound such that f(x) is Theta(x^4).
Why does this work, even though the coefficient was negative? For both Big O and Big Omega notation, we are looking for a bound such that after some point one function dominates another. That is, as these graphs illustrate:
(source: Alistair.Rendell at cs.anu.edu.au)
-- Big O
(source: Alistair.Rendell at cs.anu.edu.au)
-- Big Omega
Thinking about our original f(x), 6x^4grows faster than 2x^4 (our kg(x) function). After some point, the 6x^4 term will outstrip the growth of 2x^4 in such a way that it is always greater than 2x^4. Graphically, the two functions look like this:
Despite the negative coefficient, clearly kg(x) is a lower bound of f(x).
Now, is this always true for any polynomial function with any negative coefficient--that a function f(x) with any coefficients will be bound by its highest degree polynomials? No. If the term with the highest degree has the negative coefficient, then the bounds aren't quite the same. Take f(x) = -2x^2. We can show that f(x) = O(x^2):
-2x^2 <= cx^2
-2 <= c
Which can be satisfied by any c>0 (as c is by definition a positive constant). However, if we try to do the same for lower bound:
-2x^2 >= cx^2
-2 <= c
Then we can't find the right c because again c must be non-negative.
Related
Given integer variable x, ranging from 0 to n. We have two functions f(x) and g(x) with the following properties:
f(x) is a strictly increasing function; with x1 > x2, we have f(x1) > f(x2)
g(x) is a strictly decreasing function; with x1 > x2, we have g(x1) < g(x2)
f(x) and g(x) are black-box functions, and have constant time complexity O(1)
The problem is to solve an optimization problem and determine optimal x:
minimize f(x) + g(x)
An easy approach is a simple linear scan to test all x from 0 to n with time complexity of O(n). I am curious if there is an approach to solve it with O(log n).
There is no such solution.
Start with f(i) = 2i. And g(i) = 2n - 2i. These meet your requirements, and the minimum is going to be 2n.
Now at one point k replace g(k) with 2n - 2k - 1. This still meets your requirements, the minimum is now going to be 2n-1, and you only can get this knowledge from asking about the kth. No amount of other questions give you any information that is different than the original one. So there is no way around asking n questions to notice a difference between the modified and original functions.
I doubt the problem in such general shape has an answer.
Let f(x)=2x for even x and 2x+1 for odd,
and g(x)=-2x-1.
Then f+g oscillates between 0 and 1 for integer arguments and every odd x is a local minimum.
And, similarly to example by #btilly, a small variation in the g(x) definition may introduce a global minimum anywhere.
I already marked one response as the solution. With certain special cases, you have to brutal force all x to get the optimal value. However, my real intention is see if there is any early stopping criteria when we observe a specific pattern. An example early stopping solution is as follows.
First we solve the boundary conditions at 0 and n, we have f(0), f(n), and g(0), g(n), with any x, we have:
f(0) < f(x) < f(n)
g(0) > g(x) > g(n)
Given two trials x and y, y > x, if we observe:
f(y) + g(y) > f(x) + g(x) // x solution is better
f(y) - f(x) > g(y) - g(n) // no more room to improve after y
Then, there is no need to test solutions after y.
I am reading an example where the following are equivalent to O(N):
O(N + P), where P < N/2
O(N + log N)
Can someone explain in laymen terms how it is that the two examples above are the same thing as O(N)?
We always take the greater one in case of addition.
In both the cases N is bigger than the other part.
In first case P < N/2 < N
In second case log N < N
Hence the complexity is O(N) in both the cases.
Let f and g be two functions defined on some subset of the real numbers. One writes
f(x) = O(g(x)) as x -> infinite
if and only if there is a positive constant M such that for all sufficiently large values of x, the absolute value of f(x) is at most M multiplied by the absolute value of g(x). That is, f(x) = O(g(x)) if and only if there exists a positive real number M and a real number x0 such that
|f(x)| <= M |g(x)| for all x > x0
So in your case 1:
f(N) = N + P <= N + N/2
We could set M = 2 Then:
|f(N)| <= 3/2|N| <= 2|N| (N0 could any number)
So:
N+p = O(N)
In your second case, we could also set M=2 and N0=1 to satify that:
|N + logN| <= 2 |N| for N > 1
Big O notation usually only provides an upper bound on the growth rate of the function, wiki. Meaning for your both cases, as P < N and logN < N. So that O(N + P) = O(2N) = O(N), The same to O(N + log N) = O(2N) = O(N). Hope that can answer your question.
For the sake of understanding you can assume that O(n) represents that the complexity is of the order of n and also that O notation represents the upper bound(or the complexity in worst case). So, when I say that O(n+p) it represents that the order of n+p.
Let's assume that in worst case p = n/2, then what would be order of n+n/2? It would still be O(n), that is, linear because constants do form a part of the Big-O notation.
Similary, for O(n+logn) because logn can never be greater than n. So, overall complexity turns out to be linear.
In short
If N is a function and C is a constant:
O(N+N/2):
If C=2, then for any N>1 :
(C=2)*N > N+N/2,
2*N>3*N/2,
2> 3/2 (true)
O(N+logN):
If C=2, then for any N>2 :
(C=2)*N > N+logN,
2*N > N+logN,
2>(N+logN)/N,
2> 1 + logN/N (limit logN/N is 0),
2>1+0 (true)
Counterexample O(N^2):
No C exists such that C*N > N^2 :
C > N^2/N,
C>N (contradiction).
Boring mathematical part
I think the source of confusion is that equals sign in O(f(x))=O(N) does not mean equality! Usually if x=y then y=x. However consider O(x)=O(x^2) which is true, but reverse is false: O(x^2) != O(x)!
O(f(x)) is an upper bound of how fast a function is growing.
Upper bound is not an exact value.
If g(x)=x is an upper bound for some function f(x), then function 2*g(x) (and in general anything growing faster than g(x)) is also an upper bound for f(x).
The formal definition is: for function f(x) to be bound by some other function g(x) if you chose any constant C then, starting from some x_0 g(x) is always greater than f(x).
f(x)=N+N/2 is the same as 3*N/2=1.5*N. If we take g(x)=N and our constant C=2 then 2*g(x)=2*N is growing faster than 1.5*N:
If C=2 and x_0=1 then for any n>(x_0=1) 2*N > 1.5*N.
same applies to N+log(N):
C*N>N+log(N)
C>(N+logN)/N
C>1+log(N)/N
...take n_0=2
C>1+1/2
C>3/2=1.5
use C=2: 2*N>N+log(N) for any N>(n_0=2),
e.g.
2*3>3+log(3), 6>3+1.58=4.68
...
2*100>100+log(100), 200>100+6.64
...
Now interesting part is: no such constant exist for N & N^2. E.g. N squared grows faster than N:
C*N > N^2
C > N^2/N
C > N
obviously no single constant exists which is greater than a variable. Imagine such a constant exists C=C_0. Then starting from N=C_0+1 function N is greater than constant C, therefore such constant does not exist.
Why is this useful in computer science?
In most cases calculating exact algorithm time or space does not make sense as it would depend on hardware speed, language overhead, algorithm implementation details and many other factors.
Big O notation provides means to estimate which algorithm is better independently from real world complications. It's easy to see that O(N) is better than O(N^2) starting from some n_0 no matter which constants are there in front of two functions.
Another benefit is ability to estimate algorithm complexity by just glancing at program and using Big O properties:
for x in range(N):
sub-calc with O(C)
has complexity of O(N) and
for x in range(N):
sub-calc with O(C_0)
sub-calc with O(C_1)
still has complexity of O(N) because of "multiplication by constant rule".
for x in range(N):
sub-calc with O(N)
has complexity of O(N*N)=O(N^2) by "product rule".
for x in range(N):
sub-calc with O(C_0)
for y in range(N):
sub-calc with O(C_1)
has complexity of O(N+N)=O(2*N)=O(N) by "definition (just take C=2*C_original)".
for x in range(N):
sub-calc with O(C)
for x in range(N):
sub-calc with O(N)
has complexity of O(N^2) because "the fastest growing term determines O(f(x)) if f(x) is a sum of other functions" (see explanation in the mathematical section).
Final words
There is much more to Big-O than I can write here! For example in some real world applications and algorithms beneficial n_0 might be so big that an algorithm with worse complexity works faster on real data.
CPU cache might introduce unexpected hidden factor into otherwise asymptotically good algorithm.
Etc...
It is for a homework assignment and I'm just getting thrown a little bit by the negative sign.
Express the following in terms of big-O notation. Use the tightest bounds possible. For instance, n5 is technically O(n1000), but this is not as tight as O(n5).
n2 ā500nā2
n2 - 500 n - 2
<= n2 - 500 n
<= n2 for all n > 0
which is O(n2)
For Big O notation what you need to remember is that it only matters for some number x0 and all numbers above that. Specifically f(x)= O(g(x)) as x approaches infinity if there is some number M and some real number x0 such that |f(x)| <= M|g(x)| for all x >= x0. (Source for equations, wikipedia).
Basically, we only need to consider large values of x and you can pick an arbitrarily large value. So large in fact that n^2 will overshadow a subtraction by 500n. To be more technical if I pick M to be 2 and x0 to be 100000000000000000. Then the above equation holds. I'm being lazy and picking an x0 that is extremely large but the equation lets me. For an M equal to 2 a much smaller value of x0 would work, but again, it doesn't matter.
Finally, your answer of O(n^2) is correct
Yes O(n^2) is correct. The negative sign should not bother you. Yes, if n = 10, then it'd be a negative number, but what if the n is sufficiently large?
E.g. see these two graphs: link - n^2 for sufficiently large n is always larger than n^2-500n-2.
I am working on an assignment but a friend of mine disagree with the answer to one part.
f(n) = 2n-2n^3
I find the complexity to be f(n) = O(n^3)
Am I wrong?
You aren't wrong, since O(n^3) does not provide a tight bound. However, typically you assume that f is increasing and try to find the smallest function g for which f=O(g) is true. Consider a simple function f=n+3. It's correct to say that f=O(n^3), since n+3 < n^3 for all n > 2 (just to pick an arbitrary constant). However, it's "more" correct to say that f=O(n), since n+3 < 2n for all n > 3, and this gives you a better feel for how f behaves as n increases.
In your case, f is decreasing as n increases, so it is true that f = O(g) for any function that stays positive as n increases. The "smallest" (or rather, slowest growing) such function is a constant function for some positive constant, and we usually write that as 2n - 2n^3 = O(1), since 2n - 2n^3 < 1 for all n>0.
You could even find some function of n that is decreasing as n increases, but decreases more slowly than your f, but such usage is rare. Big-O notation is most commonly used to describe algorithm running times as the input size increases, so n is almost universally assumed to be positive.
In n-element array sorting processing takes;
in X algorithm: 10-8n2 sec,
in Y algoritm 10-6n log2n sec,
in Z algoritm 10-5 sec.
My question is how do i compare them. For example for y works faster according to x, Which should I choose the number of elements ?
When comparing Big-Oh notations, you ignore all constants:
N^2 has a higher growth rate than N*log(N) which still grows more quickly than O(1) [constant].
The power of N determines the growth rate.
Example:
O(n^3 + 2n + 10) > O(200n^2 + 1000n + 5000)
Ignoring the constants (as you should for pure big-Oh comparison) this reduces to:
O(n^3 + n) > O(n^2 + n)
Further reduction ignoring lower order terms yields:
O(n^3) > O(n^2)
because the power of N 3 > 2.
Big-Oh follows a hierarchy that goes something like this:
O(1) < O(log[n]) < O(n) < O(n*log[n]) < O(n^x) < O(x^n) < O(n!)
(Where x is any amount greater than 1, even the tiniest bit.)
You can compare any other expression in terms of n via some rules which I will not post here, but should be looked up in Wikipedia. I list O(n*log[n]) because it is rather common in sorting algorithms; for details regarding logarithms with different bases or different powers, check a reference source (did I mention Wikipedia?)
Give the wiki article a shot: http://en.wikipedia.org/wiki/Big_O_notation
I propose this different solution since there is not an accepted answer yet.
If you want to see at what value of n does one algorithm perform better than another, you should set the algorthim times equal to each other and solve for n.
For Example:
X = Z
10^-8 n^2 = 10^-5
n^2 = 10^3
n = sqrt(10^3)
let c = sqrt(10^3)
So when comparing X and Z, choose X if n is less than c, and Z if n is greater than c. This can be repeating between the other two pairs.