Hey, the title is probably a bit off, so please correct it if you know how to put it better.
As a homework assignment I have been given several assignments along the following:
Let f(n) and g(n) be asymptotically positive functions. Prove or disprove each of the following conjectures.
a. f(n) = O(g(n)) implies g(n) = O(f(n))
Now, my real question is - how would you go about proving this in a formal way? I know that the above would be easy as I could easily provide a counter example to disprove it, but for the sake of the argument let's say that we want to do this without counter examples, as of course this continues on with some other examples where this will not work.
I am a bit stuck, I have the following inequalities written up (with <= being less than or equal to)
f(n) <= c1 * g(n)
g(n) <= c2 * f(n)
But I am uncertain of how I would combine these 2 inequations into a single (in)equation and disprove it. I am most certain that this is something quite easy that I have simply overlooked and that I am being rather stupid at the moment - but any pointers / concrete examples of how to do this would be great, so that I should be able to work the rest of these questions out on my own.
Why do you want to disprove it without using a counterexample? That is the most direct way to disprove a claim.
If you had to prove it instead, of course you would not be able to use a counterexample. In this case, contrapositive can work very well - assume that the claim is false, and then show how that leads to a logical inconsistency.
In this case, you start with f(n) <= c1 * g(n) being true, since this is what is meant by f(n) = O(g(n)). Now you want to assume that g(n) <= c2 * f(n) is true for all f and g (this last part is very important, because you can certainly pick f and g such that it is true), and show why this can't work. My hint for you: pick an f and a g such that it can't work, and show that it can't work by your choice of c1 and c2.
A few hints:
Don't forget that f(n) = O(g(n)) is a set notation and you can convert it to a mathematical form of inequalities.
Simple operations you can do with the O-notation:
f(n) = O(f(n))
c * O(f(n)) = O(f(n)), if c is constant
O(f(n)) + O(f(n)) = O(f(n))
O(O(f(n))) = O(f(n))
O(f(n)) * O(g(n)) = O(f(n)g(n))
O(f(n)g(n)) = f(n) * O(g(n))
(The Art of Computer Programming, vol 1 - The O-Notation)
Related
Could please help me to understand notation's that mention in the picture?, I try to understand "Big O notation" in that under the "Family of Bachmann–Landau notations" Table there is "Formal Definition" column, in that, there are lot's notation with equation, i did't come across these notation before. could any one familiar with this ? https://en.wikipedia.org/wiki/Big_O_notation#Family_of_Bachmann–Landau_notations
The logic behind that definitions are actually quite simple, it basically says that no matter what constants are multiplying the result, from some point where n is big enough, the one of the function will start to being bigger/smaller and it remains that way.
To see real difference, I will explain th small-o (which says that some function has smaller complexity than other), it says that for all k bigger than zero you can find some value of n called n_0 for which all n bigger than n_0 follows this pattern: f(n) <= k*g(n).
So you have two functions and you put there n as a parameter. Then no matter what you put as k, you always find value of n for which f(n) <= k*g(n) and all value that are bigger than the one you have find will also fit into this equation.
Consider for example:
f(n) = n * 100
g(n) = n^2
So if you try to put i.e. n=5 there, it does not say you what has bigger complexity, because 5*100=500 and 5^2=25. If you put number big enough, i.e. n=100, then f(n)=100*100=10000 and g(n)=100^2=100*100=10000. So we get to the same value. If you try to put anything bigger than that, the g(n) will become bigger and bigger.
It also have to follow the equation f(n) <= k*g(n). In example, if I put i.e. k=0.1 then
100*n <= 0.1*n^2 *10
1000n <= n^2 /n
1000 < n
So with that functions, you can see that for k=0.1 you have n_0 = 1000 to fulfill the equations, but it is enough. All n > 1000 will be bigger and the function g(n) will always be bigger, therefore it has higher complexity. (ok, the real proof is not that easy, but you can see the pattern). The point is, no matter what k will be, even if it is equal k=0.000000001, there always be breaking point of n_0 and from that point, all g(n) will be bigger than f(n)
We can also try some negative equations to see whats difference between O(n) and O(n^2).
Lets take:
f(n) = n
g(n) = 10*n
So in standard algebra the g(n) > f(n), right? But in complexity theory we need to know if it grows bigger and if so, if it grows bigger than just multiplying it with constant.
So if we consider that k=0.01, then you can see that no matter how big the n will be, you never find n_0 that fulfills the f(n) <= k*g(n), so the f(n) != o(g(n))
In terms of complexity theory you can take the notations as smaller/bigger, so
f(n) = o(g(n)) -> f(n) < g(n)
f(n) = O(g(n)) -> f(n) <= g(n)
f(n) = Big-Theta(g(n)) -> f(n) === g(n)
//... etc, remember these euqations are not algebraic, just for complexity
For this question, I thought it's true because I thought the question is basically asking f(n) is greater than or equal to g(n) then is 2^(f(n)) greater than or equal to 2^(g(n))
So if we take an instance of f(n) = 2n and g(n) = n, f(n) is > g(n). Then 2^2n is greater than 2^n.
But my friend said that's not correct, can someone give me some insight? I think I might have some misunderstanding of the problem.
You're interested in proving or disproving this claim:
If f(n) = Ω(g(n)), then 2f(n) = Ω(2g(n)).
When you see a statement like this, it's often helpful to clarify what f and g are here. Specifically, the statement above really means the following:
For any functions f and g, if f(n) = Ω(g(n)), then 2f(n) = Ω(2g(n))
So in this sense, if you want to prove that this statement is true, you'd need to approach it by showing that this statement is true for any possible choice of f and g, not just by picking a single f and a single function g and confirming that the relationship holds for those particular functions. In this sense, your friend is correct.
(On the other hand, if you want to disprove this claim, you just need to give examples of functions f and g where f(n) = Ω(g(n)) but 2f(n) ≠ Ω(2g(n)).)
As a hint for this question: asymptotic notations like O, Ω, and Θ all completely ignore constant factors. If f(n) = Ω(g(n)), then you can scale either f or g by any constant factor that you'd like and the relationship will still hold. On the other hand, constant factors in an exponent radically change the properties of that exponent. for example, the function en grows exponentially slower than the function e2n, since e2n = (e2)n, which is an exponential function with a higher base. In other words, you can't scale exponents by a constant factor without completely changing their rates of growth.
Based on this disconnect - that Ω notation can't tell apart functions that differ by a constant factor, but that exponential functions are very sensitive to constant factors - do you think this statement is true or false? Based on the advice above, how would you prove a statement like that?
For this question, I thought it's true because I thought the question is basically asking f(n) is greater than or equal to g(n) then is 2^(f(n)) greater than or equal to 2^(g(n))
Nope. That's not what big-omega notation means at all. f(n) = Ω(g(n)) means that for sufficiently large n, the ratio f(n)/g(n) is bounded below by a positive constant.
To see that f(n) = Ω(g(n)) does not imply 2^f(n) = Ω(2^g(n)), consider f(n) = n - log(n) and g(n) = n. Then 2^f(n) = (2^n)/n and 2^g(n) = 2^n, and 2^f(n) != Ω(2^g(n)).
To answer your question, the statement is true.
We can find constants that work by the definition of big Omega. Specifically, let C_2 = 2^C_1 and let n >= max(n_1, n_2).
Proof
Do you think the following information is true?
If Θ(f(n)) = Θ(g(n)) AND g(n) > 0 everywhere THEN f(n)/g(n) ∈ Θ(1)
We are having bit of argument with our prof
f(n) = Θ(g(n)) means there's c, d, n0 such that cg(n) <= f(n) <= dg(n) for n > n0.
Then, since g(n) > 0, c <= f(n)/g(n) <= d for n > n0.
So f(n)/g(n) = Θ(1).
Dividing functions f(n),g(n) is not the same as dividing their Big-O. For example let:
f(n) = n^3 + n^2 + n
g(n) = n^3
so:
O(f(n)) = n^3
O(g(n)) = n^3
but:
f(n)/g(n) = 1 + 1/n + 1/n^2 != constant !!!
[Edit1]
but as kfx pointed you are comparing with complexity so you want:
O(f(n)/g(n)) = O(1 + 1/n + 1/n^2) = O(1)
So the answer is Yes.
But beware complexity theory is not really my cup of tea and also I do not have any context to the question of yours.
Using definitions for Landau notation https://en.wikipedia.org/wiki/Big_O_notation, it's easy to conclude that this is true, the limit of division must be less than infinity but larger than 0.
It does not have to be exactly 1 but it has to be a finite constant, which is Θ(1).
A counter example would be nice, and should be easy to be given if the statement isn't true. A positive rigorous proof would probably need to go from definition of limes with respect to series, to prove equivalence of formal and limit definitions.
I use this definition and haven't seen it proven wrong. I suppose the disagreement might lie in exact definition of Θ, it is known that people use those colloquially with minor differences, especially Big O. Or maybe some tricky cases. For positively defined functions and series, I don't think it fails.
Basically there are three options for any pair of functions f, g: Either the first grows asymptotically slower and we write f=o(g) (notice I'm using small o), the first grows asymptotically faster: f=ω(g) (again, small omega) or they are asymptotically tightly bound: f=Θ(g).
What f=o(g) means is stricter then big O in that it doesn't allow for f=Θ(g) to be true; f=Θ(g) implies both f=O(g) and f=Ω(g), but o, Θ and ω are exclusive.
To find out whether f=o(g) it's sufficient to evaluate limit for n going to infinity f(n)/g(n) and if it is zero, f=o(g) is true, if it is infinity f=ω(g) is true and if it is any real finite number, f=Θ(g) is your answer. This is not a definition, but merely a way to evaluate a statement. (One assumption I made here was that both f and g are positive.)
Special case is if limit for n goint to infinity f(n)/1 = f(n) is finite number, it means f(n)=Θ(1) (basically we chose constant function for g).
Now we're getting to your problem: Since f=g(Θ)implies f=O(g), we know that there exists c>0 and n0 such that f(n) <= c*g(n)for all n>n0. Thus we know that f(n)/g(n) <= (c*g(n))/g(n) = cfor all n>n0. The same can be done for Ω just with opposite unequality signs. Thus we get that f(n)/g(n)is between c1and c2 from some n0 which are known to be finite numbers because of how Θ is defined. Because we know our new function is somewhere in there we also know that its limit is finite number, thus proving it is indeed constant.
Conclusion, I believe you were right and I would like your professor to offer counterexample to dispruve the statement. If something didn't make sense feel free to ask more in the comments, I'll try to clarify.
Imagine that we have three functions f(n), g(n) and h(n) and that f(n) = O(g(n))
If g(n) = (1/5)h(n) is it possible that f(n) = O(h(n))?
Why is it possible / not possible?
Look at the formal Definition of big-O. You should understand what 1/5 represents. Hint: It is clearly a constant. I will not provide a formal proof as I guess is part of this homework, but you may try to show equality in big-O between these functions.
The informal idea of Big-O is described as "it's the highest order of growth of a function" ie f(n) = 3n^2 + 5n + 50 is just O(n^2).
I do understand that Big-O is just a way of saying "guaranteed to not be worse than this period". Formally, it appears the definition is f(n) -> O(g(n)) iff f(n) <= c * g(n) where c is positive
First some mathy stuff.. if f(n) = 5n^2, g(n)=n I should be able to show 5n^2 isn't O(g(n)) by doing
5n^2 <= cn
5n <= c
If the idea is that is that c isn't a constant(I have no idea if that's a requirement), and that is proof f(n) isnt in O(g(n)), what about if g(n) were n^3 (of which it surely should be contained)?
5n^2 <= cn^3
5/n <= c
I have a misunderstanding of how the math works out for all of this I assume, so I ask:
How does all this fancy stuff work
How does it connect to the simple definition given in my data structures class?
Thanks for any help
n is a positive integer, which means that 1<=n and therefore 5/n<=5/1=5, so you can pick c=5.
A more complete definition also allows you to pick n0 and a, both constants, and only prove that f(n)<=a+c*g(n) for all n0<n
c is a constant (i.e. independent of n)
In your first example (it's proof by contradiction):
i.e. assume
5n^2 <= cn
5n <= c
But for any fixed constant c, we can find a value of n that makes it untrue.
For example pick c = 1000000, then a value of n = 200001 would be a contradiction.
In your second example, we know that f(n) is O(n^2), therefore it is also O(n^3) and above. If you are bounded by k(n^2), you are also bounded by j(n^3)
The informal idea of Big-O is described as "it's the highest order of growth of a function" ie f(n) = 3n^2 + 5n + 50 is just O(n^2).
I wouldn't say that this is the idea behind Big-O. Informally Big-O is some rough estimation of what a given function cannot exceed. And it's usage is mostly approximating how something will grow for big numbers.
For example, if we take a 6 digit number, we can definitely say that it's less than million without looking for its digits. There are a lot of cases when this is enough and we don't need to analyse all digits.
For analysing function growth two factors play their role:
we only interested in function behaviour for very large numbers
if f is bigger than g, but we can fix it with multiplying g by some big constant, that's means f's advantage is not because of growth
This leads us to two parts of the definition: (1) some constant and (2) for big enough n
And for polynomials indeed, the higher order component defines grow speed.