Suppose f(n) is runtime of algorithm.
According to function definition of O(n), if f(n)<=c*g(n) then f(n)=O(g(n)) where n0<=n.
What is the range of values constant c can take?
By definition (e.g. here), any positive number, as long as it's constant.
For instance, n^2 is not in O(n) because there is no positive number c such that n^2 = cn for all n; that equality is trivially solved to c = n, but by definition n is not a constant.
C can be anything (above zero, obviously). Doesn't matter: 0.1 or 1 or 1.000.000. The only thing: it must be constant - i.e. may be defined once and for all. It must not depend from n. Of course, C will affect total algorithm performance - but the purpose of big-O is to estimate performance, not calculate it precisely (well, that goes from definition)
It can be any positive number. If it is 0 you do nothing, if it is negative you break something.
To speak simply c is an constant consisted of two halves:
Algorithm half. For example your algo has to iterate 5 times through entire input collection. So constant will be 5. If your another algorithm iterates 3 times. Then constant will be 3, but both of your algo will have complexity O(n).
Hardware half. It is time needed to compute one operation on your pc. If you start app implementing your algo upon the same collection on Pentium 1 and on modern Xeon it is obvious that Xeon will compute the result much faster.
Any range.
c being a constance means that it does not depend on the size n of the problem. Whenever the value of n changes, c remains the same.
While c is independent of n, one is allowed to exclude some special cases for small n and determine the c only for n ≥ n0.
For instance,
n ≤ n^2 for n ≥ 1
but also
n ≤ 0.01*n^2 for n ≥ 100
Suppose f(n)=n+1 and g(n)=n^2
We try to prove that f(n)=O(g(n))
For n=1,f(n)=2,g(n)=1 f(n)<=c * g(n) if c=2 and now c>n
For n=2,f(n)=3,g(n)=4 f(n)<=c * g(n) if c=1 and now c< n
So comparison between c and n is senseless.
NOTE: c is a constant (its value can never change)
So we can say,f(n)=O(g(n) for n>=1 if c=2
But f(n)=O(g(n) for n>=2 if c=1
Related
we have RNG which generate No. between [0.0,1000.0[
in a while loop we count to find out how long does it take to produce a No. < 1.0.
code:
n =0
while ( RNG1000() >= 1.0){
n =+
}
question:
what is the O(?)
A function f of variable n is said to be big-Oh of a function g of variable n - often denoted f(n) = O(g(n)), or f in O(g) - if there exists an n0 such that for all n greater than n0, f(n) <= g(n).
Suppose our algorithm's runtime were big-Oh of some function g of n. Then there is an n0 such that for n > n0, our algorithm's runtime is less than or equal to g(n). Let us choose some fixed n' > n0 and consider what this means: it means that on input n', our algorithm is guaranteed to terminate within g(n') steps. We must have that g(n') is some number; let's call it t'. So for input n', our loop cannot take more than t' steps. But this is patently false, since - assuming that a whole loop iteration takes 1 step - the probability of iterating for more than t' steps is finite but non-zero (assuming the RNG is a true RNG). This is a contradiction, so our assumption that the runtime is big-Oh of g was wrong - that is, there is no upper bound on the runtime.
It is tempting to say that this is O(1) in terms of n, since the loop condition doesn't depend on n, but that is technically incorrect, given the above analysis. There really is no upper bound here.
"Big O notation is a mathematical notation that describes the limiting behavior of a function when the argument tends towards a particular value or infinity."
The variable n in your case does not constrain the upper limit of a function. In fact there is no function in this case (at least not one that returns repeatable results.) I would suggest that there is no Big O notation to describe this and it is undefined. However, some might argue that the worst case scenario is simply O(∞). What is confusing here is that you are not actually using your variable, n, to constrain your behavior.
I am working on an assignment but a friend of mine disagree with the answer to one part.
f(n) = 2n-2n^3
I find the complexity to be f(n) = O(n^3)
Am I wrong?
You aren't wrong, since O(n^3) does not provide a tight bound. However, typically you assume that f is increasing and try to find the smallest function g for which f=O(g) is true. Consider a simple function f=n+3. It's correct to say that f=O(n^3), since n+3 < n^3 for all n > 2 (just to pick an arbitrary constant). However, it's "more" correct to say that f=O(n), since n+3 < 2n for all n > 3, and this gives you a better feel for how f behaves as n increases.
In your case, f is decreasing as n increases, so it is true that f = O(g) for any function that stays positive as n increases. The "smallest" (or rather, slowest growing) such function is a constant function for some positive constant, and we usually write that as 2n - 2n^3 = O(1), since 2n - 2n^3 < 1 for all n>0.
You could even find some function of n that is decreasing as n increases, but decreases more slowly than your f, but such usage is rare. Big-O notation is most commonly used to describe algorithm running times as the input size increases, so n is almost universally assumed to be positive.
In this example: http://www.wolframalpha.com/input/?i=2%5E%281000000000%2F2%29+%3C+%283%2F2%29%5E1000000000
I noticed that those two equations are pretty similar no matter how high you go in n. Do all algorithms with a constant to the n fall in the same time complexity category? Such as 2^n, 3^n, 4^n, etc.
They are in the same category, This does not mean their complexity is the same. They are exponential running time algorithms. Obviously 2^n < 4^n
We can see 4^n/2^n = 2^2n/2^n = 2^n
This means 4^n algorithm exponential slower(2^n times) than 2^n
The same thing happens with 3^n which is 1.5^n.
But this does not mean 2^n is something far less than 4^n, It is still exponential and will not be feasible when n>50.
Note this is happening due to n is not in the base. If they were in the base like this:
4n^k vs n^k then this 2 algorithms are asymptotically the same(as long as n is relatively small than actually data size). They would be different by linear time, just like O(n) vs c * O(n)
The time complexities O(an) and O(bn) are not the same if 1 < a < b. As a quick proof, we can use the formal definition of big-O notation to show that bn ≠ O(an).
This works by contradiction. Suppose that bn = O(an) and that 1 < a < b. Then there must be some c and n0 such that for any n ≥ n0, we have that bn ≤ c · an. This means that bn / an ≤ c for any n ≥ n0. Since b > a, it should start to become clear that this is impossible - as n grows larger, bn / an = (b / a)n will get larger and larger. In particular, if we pick any n ≥ n0 such that n > logb / a c, then we will have that
(b / a)n > (b / a)log(b/a) c = c
So, if we pick n = max{n0, c + 1}, then it's not true that bn ≤ c · an, contradicting our assumption that bn = O(an).
This means, in particular, that O(2n) ≠ O(1.5n) and that O(3n) ≠ O(2n). This is why when using big-O notation, it's still necessary to specify the base of any exponents that end up getting used.
One more thing to notice - although it looks like 21000000000/2 is approximately 1.41000000000/2, notice that these are totally different numbers. The first is of the form 10108.1ish and the second of the form 10108.2ish. That might not seem like a big difference, but it's absolutely colossal. Take, for example, 10101 and 10102. This first number is 1010, which is 10 billion and takes ten digits to write out. The second is 10100, one googol, which takes 100 digits to write out. There's a huge difference between them - the first of them is close to the world population, while the second is about the total number of atoms in the universe!
Hope this helps!
Suppose I have the following : T(n) = 5n^2 +2n
The asymtotic tight bound of this is theta n^2. I want to understand the reason behind dropping the 5. I understand why we ignore the lower order terms.
Consult the definition of big-O.
Keeping things simple[*], let's define that a function g is O(f) if there exist constants C and M such that for n > M, 0 <= g(n) < Cf(n).
The presence of a positive constant multiplier in f doesn't affect this, just choose C appropriately. Your example T is O(n^2) by choosing a value of C greater than 5, and a value of M big enough that the +2n is irrelevant. For for example, for n > 2 it's a fact that 5n^2 + 2n < 6n^2 (because n^2 > 2n), so with C= 6 and M = 2 we see that T(n) is O(n^2).
So it's true that T(n) is O(n^2), and also true that it's O(5n^2), and O(5n^2 + 2n). The most interesting of those facts is that it's O(n^2), since it's the simplest expression and the other two are logically equivalent. If we want to compare the complexities of different functions, then we want to use simple expressions.
For big-Theta just note that we can play the same trick when f and g are the other way around. The relation "g is Theta(f)" is an equivalence relation, so what are we going to choose as the representative member of the equivalence class of T? The simplest one.
[*] Keeping things less simple, we cope with negative numbers by using a limsup rather than a plain limit. My definition above is actually sufficient but not necessary.
wEverything comes back to the concept of "Order of Magnitute". Given something like
5n^2 +2n
You think that the 5 is significant, however when you break it down and think about the numbers, the constant really doesn't matter (graph it and you will see why). For example .. say n is 50.
5 * 50^2 + 2 * 50 --> 5 * 2500 + 2 * 50 --> 12,600
As you mentioned, the 2*n is insignificant when compared to n^2. This same concept applies when viewing the constant as well... you might think that 2500 vs 125,000 is a big difference; however consider if the algorithm was n^3... you are now looking at 12,600 vs 625,100
So the factor that will have the most significant difference on the cost of an algorithm is just n^2.
The constant is dropped because it does not affect which complexity class the function belongs to.
If you have two functions f(n) = c1 * n^2 and g(n) = c2 * n^3, where c1 and c2 are constants, it doesn't matter how large c1 is and how small c2 is, g(n) will ALWAYS overtake and outgrow f(n) at some value of n.
According to the definition of big O f(n) <= C*g(n)(which means f(n) = O(g(n)), it could be deduced that:
f(n) <= C
f(n) <= 2C
I think there are no big differences between these two. What I could come up with is:
f(n) = 1 - 1 / n
f(n) = 2 - 1 / n
C = 1
But what differs this two complexities,since both are constant complexity?
Could you show some real world code to demonstrate the differences between O(1) and O(2).
There is no difference between O(1) and O(2). Algorithms classifying as O(1) are O(2) and vice versa. In fact, O(c1) is O(c2) for any positive constants c1 and c2.
O(c) where c is a positive constants simply means that the runtime is bounded independent of the input or problem size. From this it is clear (informally) that O(1) and O(2) are equal.
Formally, consider a function f in O(1). Then there is a constant c such that f(n) <= c * 1 for all n. Let d = c / 2. Then f(n) <= c = (c / 2) * 2 = d * 2 which shows that f is O(2). Similarly if g is O(2) there is a constant c such that g(n) <= c * 2 for all n. Let d = 2 * c. Then g(n) <= c * 2 = d = d * 1 which shows that g is O(1). Therefore O(1) = O(2).
O(1) and O(2) are the same, as is any O(constant value).
The point being that neither rely on some function of N.
There is no difference.
In the graph below, the red line represents O(n) and the green curve represents O(n2).
As you can see by the red line, the 2 and the 1 become insignificant as x increases (the green curve grows at a much faster rate). This is what Big-O notation is trying to capture; constants are relatively meaningless.
Maybe they meant that both algorithms execute in constant time regardless of input size (usually denoted as N), but one of them is twice as fast. But it's an abuse of the big-O notation.
There is no difference between O(1) and O(2).
The order-of notation is unique up to a constant. O(f(x)) means that there is some constant k such that the time would be less than kf(x).
If something is O(2), then there is some constant k that the program takes less than 2k. Therefore, there's another constant, k' = 2k that works for O(1).
There is not difference between O(1) and O(2). In fact you would not use this notation. It is more like O(N) or O(n^2), O(log(N)) etc. This is just a indication of the order of magnitude of the algorithm. Put in other words, O(1) would be constant in time. O(N) would be linear in the number of items (N), O(n^2) would be exponential in time, etc.
Big-O notation is generally used for asymptotic analysis of algorithm complexity, ie analysis of how the algorithm performs as n increases toward infinity. The highest order term in n of the function f(n) will be the dominant part of the function in this case.
As such, lower order terms in n are usually dropped from the function when expressed in big-O (for example, f(n)=2n^2+4 will result in O(n^2) asymptotic big-O complexity).
In the case of the highest term being constant and not dependent on n, then all these constants are effectively the same asymptotically speaking and are usually simply reduced to O(1).
Hence, O(2) would be considered equivalent to O(1).
You typically do not write O(2) or O(2n) but O(1) and O(n) instead. The difference is in actual speed, of course - 5s vs 10s for instance. I found your question a bit confusing.