I need help in this question. I really don't understand how to do it.
Show, either mathematically or by an example, that if f(n) is O(g(n)), a*f(n) is O(g(n)), for any constant a > 0.
I'll give you this. It should help you look in the right direction:
definition of O(n):
a function f(n) who satisfies f(n) <= C*n for an arbitrary constant number C and for every n above an arbitrary constant number N will be noted f(n) = O(n).
This is the formal definition for big-o notation, it should be simple to take this and turn it into a solution.
Related
I understand that classifying in Big-O is essentially having an "upper-bound" of sort so I understand it graphically, I don't understand is how to use the Big-O formal definition to solve this types of problems. Any help is appreciated.
Although there are platforms better suited for this question than SO, since this gets purely mathematical, understanding this is fundamental for using big-O also in a Computer Science context, so I like the question. And understanding your particular example will likely shed some light on what big-O is in general, as well as provide some practical intuition. So I will try to explain:
We have a function f(n) = n2. To show that it is not in O(1), we have to show that it grows faster than a function g(n) = c where c is some constant. In other words, that f(n) > g(n) for sufficiently large n.
What does sufficiently large mean? It means that for any constant c, we can find an N so that f(n) > g(n) for all n > N.
This is how we define asymptotic behaviour. Your constant function may be larger than f(n), but as n grows enough, you will eventually reach a point where f(n) remains larger forever. How to prove it?
Whatever constant c you choose - however large - we can construct our N. Let us say N = √c. Since c is a constant, so is √c.
Now, we have f(n) = n2 > c = g(n) whenever n > √c.
Therefore, f(n) is not in O(1). □
The key here is that we found an constant N (that is, which depends on our constant c but not on our variable n) such that this inequivalence holds. It can take some wrapping one's head around it, but doing a few other simple examples can help get the intuition.
I have been through this Big-Oh explanation, understanding the complexity of two loops, difference between big theta and big-oh and also through this question.
I understand that we cannot say that Big-oh is used as the worst case, Omega as Best case and theta as average case. Big-oh has its own best, worst and average cases. But how we find out that any specific algorithm belongs from Big-oh, Big-theta or Big-Omega. or how we can check that if any algorithm belongs from all of these.
A function f(n) is Big-Oh of a function g(n), written f(n) = O(g(n)), if there exist a positive constant c and a natural number n0 such that for n > n0, f(n) <= c * g(n). A function f(n) is Big-Omega of g(n), written f(n) = Omega(g(n)), if and only if g(n) = O(f(n)). A function f(n) is Theta of a function g(n), written f(n) = Theta(g(n)), if and only if f(n) = O(g(n)) and f(n) = Omega(g(n)).
To prove any of the free, you do it by showing some function(s) are Big-Oh of some other functions. To show that one function is Big-Oh of another is a difficult problem in the general case. Any form of mathematical proof may be helpful. Induction proofs in conjunction with intuition for the base cases are not uncommon. Basically, guess at values for c and n0 and see if they work. Other options involve choosing one of the two and working out a reasonable value for another.
Note that a function may not be Big-Theta of any other function, if its tightest bounds from above and below are functions with different asymptotic rates of growth. However, I think it's usually a safe bet that most functions are going to be Big-Oh of something reasonably uncomplicated, and all functions typically looked at from this perspective are at least constant-time in the best case - Omega(1).
I've been watching MIT lectures for the algorithms course and the definition for the Big O notation says
f(n) = O(g(n)) such that for some constants c and n0
0 < f(n) < c.g(n) for all n>n0
Then the instructor proceeded to give an example,
2n2=O(n3)
Now I get that Big O gives the upper bound on the function but I am confused as to what exactly does the function f(n) correspond to here? What is its significance? As per my understanding goes, g(n) is the function representing the algorithm we are trying to analyse, but what is the purpose of f(n) or as in the example 2n2?
Need some clarification on this, I've been stuck here for hours.
In the formal definition of big-O notation, the functions f(n) and g(n) are placeholders for other functions, the same way that, say, in the quadratic formula, the letters a, b, and c are placeholders for the actual coefficients in the quadratic equation.
In your example, the instructor was talking about how 2n2 = O(n3). You have a formal definition that talks about what it means, in general, for f(n) = O(g(n)) to be true. So let's pattern-match that against the math above. It looks like f(n) is the thing on the left and g(n) is the thing on the right, so in this example f(n) = 2n2 and g(n) = n3.
The previous paragraph gives a superficial explanation of what f(n) and g(n) are by just looking at one example, but it's better to talk about what they really mean. Mathematically, f(n) and g(n) really can be any functions you'd like, but typically when you're using big-O notation in the context of the analysis of algorithms, you'll usually let f(n) be the true amount of work done by the algorithm in question (or its runtime, or its space usage, or really just about anything else) and will pick g(n) to be some "nice" function that's easier to reason about. For example, it might be the case that some function you're analyzing has a true runtime, as a function of n, as 16n3 - 2n2 - 9n + 137. That would be your function f(n). Since the whole point behind big-O notation is to be able to (mathematically rigorously and safely) discard constant factors and low-order terms, we'll try to pick a g(n) that grows at the same rate as f(n) but is easier to reason about - say, g(n) = n3. So now we can try to determine whether f(n) = O(g(n)) by seeing whether we can find the constants c and n0 talked about in the formal definition of big-O notation.
So to recap:
f(n) and g(n) in the definition given are just placeholders for other functions.
In practical usage, f(n) will be the true runtime of the algorithm in question, and g(n) will be something a lot simpler that grows at the same rate.
f(n) is the function that gives you the exact values of the thing you are trying to measure (be that time, number of processor instructions, number of iterations steps, amount of memory used, whatever).
g(n) is another function that approximates the growth of f(n).
In the usual case you don't really know f(n) or it's really hard to compute. For example for time it depends on the processor speed, memory access patterns, system load, compiler optimizations and other. g(n) is usually really simple and it's easier to understand if f(n) = O(N) that if you double n you will roughly double the runtime, in the worst case. Since it's an upper bound g(n) doesn't have to be the minimum, but usually people try to avoid inflating it if it's not necessary. In your example O(n^3) is an upper bound for 2n^2, but so is O(n^2) and O(n!).
I have a homework question asks
Given f(n) is O(k(n)) and g(n) is O(k(n)), prove f(n)+g(n) is also O(k(n))
I'm not sure where to start with this, any help to guide me of how to work on this?
Try and work through it logically. f(n) increases at a linear rate. So does g(n). Therefore
O(n) + O(n) = O(2n)
When attempting to find the big O classification of a function, constants don't count.
I'll leave the rest (including the why) as an exercise for you. (Getting the full answer on SO would be cheating!)
Refer to the Rules for Big-Oh Notation.
Sum Rule: If f(n) is O(h(n)) and g(n) is O(p(n)), then f(n)+g(n) is O(h(n)+p(n)).
Using this rule for your case the complexity would be O(2k(n)), which is nothing but O(k(n)).
So, f(n) is O(g(n)) iff f(n) is less than or equal to some positive constant multiple of g(n) for arbitrarily large values of n (so this: f(n) <= cg(n) for n >= n_0). Usually, to prove something is O(g(n)), we provide some c and n_0 to show that it is true.
In your case, I would start by using that definition, so you could say f(n) <= ck(n) and g(n) <= dk(n). I don't want to totally answer the question for you, but you are basically just going to try to show that f(n)+g(n) <= tk(n).
*c, d, and t are all just arbitrary, positive constants. If you need more help, just comment, and I will gladly provide more info.
Question 1: Under what circumstances would O(f(n)) = O(k f(n)) be the most appropriate form of time-complexity analysis?
Question 2: Working from mathematical definition of O notation, how to show that O(f(n)) = O(k f(n)), for positive constant k?
For the first Question I think it is average case and worst case form of time-complexity. Am I right? And what else should I write in that?
For the second Question, I think we need to define the function mathematically. So is the answer something like because the multiplication by a constant just corresponds to a readjustment of value of the arbitrary constant k in definition of O?
My view: For the first one I think it
is average case and worst case form of
time-complexity. am i right? and what
else do i write in that?
No! Big O notation has NOTHING to do with average case or worst case. It is only about the order of growth of a function - particularly, how quickly a function grows relative to another one. A function f can be O(n) in the average case and O(n^2) in the worst case - this just means the function behaves differently depending on its inputs, and so the two cases must be accounted for separately.
Regarding question 2, it is obvious to me from the wording of the question that you need to start with the mathematical definition of Big O. For completeness's sake, it is:
Formal Definition: f(n) = O(g(n))
means there are positive constants c
and k, such that 0 ≤ f(n) ≤ cg(n) for
all n ≥ k. The values of c and k must
be fixed for the function f and must
not depend on n.
(source http://www.itl.nist.gov/div897/sqg/dads/HTML/bigOnotation.html)
So, you need to work from this definition and write a mathematical proof showing that f(n) = O(k(n)). Start by substituting O(g(n)) with O(k*f(n)) in the definition above; the rest should be quite easy.
Question 1 is a little vague, but your answer for question 2 is definitely lacking. The question says "working from the mathematical definition of O notation". This means that your instructor wants you to use the mathematical definition:
f(x) = O(g(x)) if and only if limit [x -> a+] |f(x)/g(x)| < infinity, for some a
And he wants you to plug in g(x) = k f(x) and prove that that inequality holds.
The general argument you posted might get you partial credit, but it is reasoning rather than mathematics, and the question is asking for mathematics.