Let's suppose that a recursive formula is a big-o(n^2), and at the same time a big-omega(n^2). Does this imply that the recursion is a big-Theta(n^2)?
To make the long story short: the answer is Yes, it does. See proof below.
Though everybody have heard about big-o notation lets recall what exactly does these notations mean with a help of Introduction to Algorithms. For a general case it is said Ο(g(n)), Ω(g(n)), Θ(g(n)), but we will consider yours.
Ο(n2)
Ο(n2) notation defines a set of functions for each of which the following statement holds: There exists such positive constants c and n0 that 0 ≤ f(n) ≤ cn2 holds for all n ≥ n0.
So f(n) is just a function from Ο(n2). Examples 13n, -5, 4n2 + 5. All these pertain to Ο(n2).
Ω(n2)
Ω(n2) notation defines a set of functions for each of which the following statement holds: There exists such positive constants c and n0 that 0 ≤ cn2 ≤ f(n) holds for all n ≥ n0.
So f(n) is just a function from Ω(n2). Examples n4 + n - 1, 3n, n2 - 12. All these pertain to Ω(n2).
Θ(n2)
Θ(n2) notation defines a set of functions for each of which the following statement holds: There exists such positive constants c1, c2 and n0 that 0 ≤ c1n2 ≤ f(n) ≤ c2n2 holds for all n ≥ n0.
Again f(n) is just a function from Θ(n2). These are its representatives n2/2 + 3, 5n2.
Proof
I bet saying that a recursive formula is a big-o(n^2), and at the same time a big-omega(n^2) you meant there is a function (lets call it) f(n) pertaining to
Ω(n2) and Ο(n2).
From Ω(n2) we have existence of c1 that c1n2 ≤ f(n) holds. From Ο(n2) we have existence of c2 that f(n) ≤ c2n2 holds. Consequently we have existence of c1 and c2 that c1n2 ≤ f(n) ≤ c2n2, that is exactly what Θ(n2) is about.
Related
I would like to prove this statement:
If f(n) is Theta(h(n)) and g(n) = O((h(n)) then f(n) + g(n) is O(h(n)).
All functions are assumed to be non-negative and monotonically non-decreasing.
My attempted proof explains that if f(n) and g(n) are O(h(n)) then there exists:
positive constants n0, c0 such that f(n) <= c0*h(n) for all n >= n0
positive constants n1, c1 such that f(n) <= c1*h(n) for all n >= n1
Therefore it can be deduced that g(n) + f(n) <= (c1 + c0) * h(n) for all n >= max(n1, n0) meaning that g(n) + f(n) is O(h(n))
Is this correct? Is there a more rigourous proof ... for example, by contradiction?
Your proof is correct and rigorous. It is direct and constructive proof in that you not only show that suitable c and n0 exist, but you say how to calculate them from the assumptions. No other simpler methods of proof come to mind here.
I have been given the problem:
f(n) are asymptotically positive functions. Prove f(n) = Θ(g(n)) iff g(n) = Θ(f(n)).
Everything I have found points to this statement being invalid. For example an answer I've come across states:
f(n) = O(g(n)) implies g(n) = O(f(n))
f(n) = O(g(n)) means g(n) grows faster than f(n). It cannot imply that f(n) grows
faster than g(n). Hence not true.
Another states:
If f(n) = O(g(n)) then O(f(n)). This is false. If f(n) = 1 and g(n) = n
for all natural numbers n, then f(n) <= g(n) for all natural numbers n, so
f(n) = O(g(n)). However, suppose g(n) = O(f(n)). Then there are natural
numbers n0 and a constant c > 0 such that n=g(n) <= cf(n) = c for all n >=
n0 which is impossible.
I understand that there are slight differences between my exact question and the examples I have found, but I've only been able to come up with solutions that do not prove it. I am correct in thinking that it is not able to be proved or am I looking over some detail?
You can start from here:
Formal Definition: f(n) = Θ (g(n)) means there are positive constants c1, c2, and k, such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ k.
Because you have that iff, you need to start from the left side and to prove the right side, and then start from the right side and prove the left side.
Left -> right
We consider that:
f(n) = Θ(g(n))
and we want to prove that
g(n) = Θ(f(n))
So, we have some positive constants c1, c2 and k such that:
0 ≤ c1*g(n) ≤ f(n) ≤ c2*g(n), for all n ≥ k
The first relation between f and g is:
c1*g(n) ≤ f(n) => g(n) ≤ 1/c1*f(n) (1)
The second relation between f and g is:
f(n) ≤ c2*g(n) => 1/c2*f(n) ≤ g(n) (2)
If we combine (1) and (2), we obtain:
1/c2*f(n) ≤ g(n) ≤ 1/c1*f(n)
If you consider c3 = 1/c2 and c4 = 1/c1, they exist and are positive (because the denominators are positive). And this is true for all n ≥ k (where k can be the same).
So, we have some positive constants c3, c4, k such that:
c3*f(n) ≤ g(n) ≤ c4*f(n), for all n ≥ k
which means that g(n) = Θ(f(n)).
Analogous for right -> left.
I'm fairly new to the Big-O stuff and I'm wondering what's the complexity of the algorithm.
I understand that every addition, if statement and variable initialization is O(1).
From my understanding first 'i' loop will run 'n' times and the second 'j' loop will run 'n^2' times. Now, the third 'k' loop is where I'm having issues.
Is it running '(n^3)/2' times since the average value of 'j' will be half of 'n'?
Does it mean the Big-O is O((n^3)/2)?
We can use Sigma notation to calculate the number of iterations of the inner-most basic operation of you algorithm, where we consider the sum = sum + A[k] to be a basic operation.
Now, how do we infer that T(n) is in O(n^3) in the last step, you ask?
Let's loosely define what we mean by Big-O notation:
f(n) = O(g(n)) means c · g(n) is an upper bound on f(n). Thus
there exists some constant c such that f(n) is always ≤ c · g(n),
for sufficiently large n (i.e. , n ≥ n0 for some constant n0).
I.e., we want to find some (non-unique) set of positive constants c and n0 such that the following holds
|f(n)| ≤ c · |g(n)|, for some constant c>0 (+)
for n sufficiently large (say, n>n0)
for some function g(n), which will show that f(n) is in O(g(n)).
Now, in our case, f(n) = T(n) = (n^3 - n^2) / 2, and we have:
f(n) = 0.5·n^3 - 0.5·n^2
{ n > 0 } => f(n) = 0.5·n^3 - 0.5·n^2 ≤ 0.5·n^3 ≤ n^3
=> f(n) ≤ 1·n^3 (++)
Now (++) is exactly (+) with c=1 (and choose n0 as, say, 1, n>n0=1), and hence, we have shown that f(n) = T(n) is in O(n^3).
From the somewhat formal derivation above it's apparent that any constants in function g(n) can just be extracted and included in the constant c in (+), hence you'll never (at least should not) see time complexity described as e.g. O((n^3)/2). When using Big-O notation, we're describing an upper bound on the asymptotic behaviour of the algorithm, hence only the dominant term is of interest (however not how this is scaled with constants).
I know the definitions of both of them, but what is the reason sometimes I see O(1) and other times Θ(1) written in textbooks?
Thanks.
O(1) and Θ(1) aren't necessarily the same if you are talking about functions over real numbers. For example, consider the function f(n) = 1/n. This function is O(1) because for any n ≥ 1, f(n) ≤ 1. However, it is not Θ(1) for the following reason: one definition of f(n) = Θ(g(n)) is that the limit of |f(n) / g(n)| as n goes to infinity is some finite value L satisfying 0 < L. Plugging in f(n) = 1/n and g(n) = 1, we take the limit of |1/n| as n goes to infinity and get that it's 0. Therefore, f(n) ≠ Θ(1).
Hope this helps!
Big-O notation expresses an asymptotic upper bound, whereas Big-Theta notation additionally expresses an asymptotic lower bound. Often, the upper bound is what people are interested in, so they write O(something), even when Theta(something) would also be true. For example, if you wanted to count the number of things that are equal to x in an unsorted list, you might say that it can be done in linear time and is O(n), because what matters to you is that it won't take any longer than that. However, it would also be true that it's Omega(n) and therefore Theta(n), since you have to examine all of the elements in the list - it can't be done in sub-linear time.
UPDATE:
Formally:
f in O(g) iff there exists a c and an n0 such that for all n > n0, f(n) <= c * g(n).
f in Omega(g) iff there exists a c and an n0 such that for all n > n0, f(n) >= c * g(n).
f in Theta(g) iff f in O(g) and f in Omega(g), i.e. iff there exist a c1, a c2 and an n0 such that for all n > n0, c1 * g(n) <= f(n) <= c2 * g(n).
In this Big-O / Computational Complexity problem
given that a and b are positive constants greater than 1 and n is a variable parameter.
I assumed that an+1 = O(an) , abn = O(an) & an+b = O(an)
First I need to know if I am correct in assuming this.
If so, how would I prove that f(n) = O(f(n)).
Recall the definition of big-O:
f(n) ∈ O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.
Let g=f, c=1 and k=0, then you have a trivial demonstration of f(n) ∈ O(f(n))
Similarly, from an+1=a⋅an, let f(n)=an+1, g(n)=an, c=a, k=0, again the proof of O(an+1)=O(an) is trivial. The proof for O(an+b)=O(an) is identical.
O(abn) is not equal to O(an) with a,b>1, see Exponential growth in big-o notation