I don't understand this situation.
f(n) ∈ O(g(n)),
g(n) ∈ Θ(f(n))
for these situations why is are the following the correct answers.
f(n) <= g(n) for all n>1, neither always true or false
g(n) ∈ Ω(f(n)), always true
f(n)<= Θ(g(n)), always true
My logic is since g(n) ∈ Θ(f(n)), g(n) and f(n) would have to have the same highest power (for example: n=n, n^2=n^2). In that case, wouldn't all 3 statements be true?
I don't understand why the first one is neither always true or false and third one is always true.
Big-O, Big-Ω and Big-Θ notation, in mathematics, describes the asymptotic behaviour of a function, as an upper bound, lower bound and a tight bound (upper and lower bound), respectively. On SE, in the context of programming, we usually use these notations to describe the asymptotic behaviour of algorithms, with regard to the size of the problem that the algorithm is intended to solve (typically this size is denoted n).
For reference, see e.g.
https://www.khanacademy.org/computing/computer-science/algorithms/asymptotic-notation/a/asymptotic-notation
For answering your question, we shall treat this subject in the context of the asymptotic (/limiting) behaviour or functions. Lets cover your two properties of and three statements of f(n) and g(n) one by one.
Property i) f(n) ∈ O(g(n))
Given is that
f(n) ∈ O(g(n)).
For some constant k>0, (the dominant term(s) of) g(n) provides an upper bound to the asymptotic behaviour of f(n), i.e.,
f(n) < k*g(n), n sufficiently large. (*)
As a quick example of what I mean by dominant terms: if g(n) is a some polynomial, we have
O(g(n)) = O(a_0 + a_1*n + ... + a_j*n^j) = O(n^j),
i.e., asymptotically dominant term n^j.
Property ii) g(n) ∈ Θ(f(n))
Given is that
g(n) ∈ Θ(f(n)).
For some constants k_1>0 and k_2>0, k_1*f(n) and k_2*f(n) provides lower and upper bounds, respectively, on the asymptotic behaviour of g(n), i.e.,
k_1*f(n) < g(n) < k_2*f(n), n sufficiently large. (**)
Again, as we describe asymptotic behaviour, it's really the dominant terms of g(n) and f(n) that are of interest.
Assume from now on that i) and ii) both holds, for all sufficiently large n
We move only to your three statements.
Statement a) f(n) <= g(n) for all n>1, either always true or false
First of all, given i) and ii), we cannot draw any conclusion of the behaviour of f(n) and g(n) for "n smaller than sufficiently large n", i.e., we cannot say any statements regarding f(n) and g(n) for all n>1. Properties in i) and ii) only describe asymptotic behaviours of f(n) and g(n). If we adjust the statement to
f(n) <= g(n) for all sufficiently large n; either always true or false,
we can analyse it. Assume the following holds true (for n sufficiently large):
f(n) <= g(n). (a1)
For n sufficiently large, we also know that (*) and (**) holds, that is
(*) f(n) < k*g(n), for some constant k>0, (a2)
(**) f(n) < (1/k_1)*g(n), for some constant k_1>0, (a3)
g(n) < k_2*f(n), for some constant k_2>0, (a4)
Since (a1) holds, by assumption, we can consider (a2) and (a3) as redundant by choosing some k=(1/k_1)>1. This leaves us with
f(n) <= g(n) < k_2*f(n), for some constant k_2>0. (a1, a4)
This is simply property ii) above, g(n) ∈ Θ(f(n)), where we've found that constant k_1 = 1 (or, strictly, k_1 very near 1) satisfy the left hand side of (**).
On the other hand, if we assume that f(n) <= g(n) is always false (sufficiently large n), we arrive at the result
g(n) < f(n), (a1.false)
g(n) < k_2*f(n), for some constant k_2>0. (a4)
Which, naturally, holds (k_2=1).
To round off, statement a) is kind of weird.
Statement b) g(n) ∈ Ω(f(n)), always true
Much like in i) (for upper bound), given that
g(n) ∈ Ω(f(n)),
then, for sufficiently large n,
k*f(n) < g(n), for some constant k>0, (b1)
holds. We already know this to be true from ii), as this is given in the left hand side of (**)
(**) k_1*f(n) < g(n), for some constant k_1>0. (b2)
Hence, given ii), it is trivial that g(n) ∈ Ω(f(n)) holds.
Statement c) f(n)<= Θ(g(n)), always true
Recall Big-Θ from ii); f(n) ∈ Θ(g(n)) can be described as
k_1*g(n) < f(n) < k_2*g(n), n sufficiently large, (c1)
for some constants k_1>0, k_2>0.
In this context, f(n) <= Θ(g(n)) does not make much sense, as O(g(n)) either describes a property of f(n) or set of functions f(n) that conforms to the property O(g(n)) (in context of asymptotic behaviour. From Wikipedia article of Big O notation (https://en.wikipedia.org/wiki/Big_O_notation):
"the notation O(g(n)) is also used to denote the set of all functions
f(n) that satisfy the relation f(n)=O(g(n)); f(n) ∈ Θ(g(n))"
Perhaps the leq operator "<=" has some special meaning in the context of Big-...-notation, but it's not something I have ever encountered myself.
Related
When we talk about Θ(g) are we referring to the highest order term of g(n) or are we referring to g(n) exactly as it is?
For example if f(n) = n3. And g(n)=1000n3+n does Θ(g) mean Θ(1000n3+n) or Θ(n3)?
In this scenario can we say that f(n) is Θ(g)?
Θ(g) yields sets of functions that are all of the same complexity class. Θ(1000n3+n) is equal to Θ(n3) because both of these result in the same set.
For simplicity's sake one will usually drop the non-significant terms and multiplicative constants. The lower order additive terms don't change the complexity, nor do any multipliers, so there's no reason to write them out.
Since Θ(g) is a set, you would say that f(n) ∈ Θ(g).
NB: Many CS teachers, textbooks, and other resources muddy the waters by using imprecise notation. Lots of people say that f(n)=n3 is O(n3), rather than f(n)=n3 is in O(n3). They use = when they mean ∈.
theta(g(n)) lies between O(g(n)) and omega(g(n))
if g(n) = 1000n^3 + n
first lets find O(g(n)) upper bound
It could be n^3, n^4, n^5 but we choose the closest one which is O(n^3).
O(n^3) is valid because we can find a constant c such that for some value of n
1000n^3 + n < c.n^3
second lets see omega(g(n)) which is lower bound
omega says f(n) > c.g(n)
we can find a constant c such that
1000.n^3 + n > c.n^3
Now we have upper bound which is O(n^3) and lower bound which is omega(n^3).
therefore we have theta which bounds both upper and lower using same function.
By rule : if f(n) = O(g(n)) and f(n) = omega(g(n)) therefore f(n) = theta(g(n))
1000.n^3 + n = theta(n^3)
I'm fairly new to the Big-O stuff and I'm wondering what's the complexity of the algorithm.
I understand that every addition, if statement and variable initialization is O(1).
From my understanding first 'i' loop will run 'n' times and the second 'j' loop will run 'n^2' times. Now, the third 'k' loop is where I'm having issues.
Is it running '(n^3)/2' times since the average value of 'j' will be half of 'n'?
Does it mean the Big-O is O((n^3)/2)?
We can use Sigma notation to calculate the number of iterations of the inner-most basic operation of you algorithm, where we consider the sum = sum + A[k] to be a basic operation.
Now, how do we infer that T(n) is in O(n^3) in the last step, you ask?
Let's loosely define what we mean by Big-O notation:
f(n) = O(g(n)) means c · g(n) is an upper bound on f(n). Thus
there exists some constant c such that f(n) is always ≤ c · g(n),
for sufficiently large n (i.e. , n ≥ n0 for some constant n0).
I.e., we want to find some (non-unique) set of positive constants c and n0 such that the following holds
|f(n)| ≤ c · |g(n)|, for some constant c>0 (+)
for n sufficiently large (say, n>n0)
for some function g(n), which will show that f(n) is in O(g(n)).
Now, in our case, f(n) = T(n) = (n^3 - n^2) / 2, and we have:
f(n) = 0.5·n^3 - 0.5·n^2
{ n > 0 } => f(n) = 0.5·n^3 - 0.5·n^2 ≤ 0.5·n^3 ≤ n^3
=> f(n) ≤ 1·n^3 (++)
Now (++) is exactly (+) with c=1 (and choose n0 as, say, 1, n>n0=1), and hence, we have shown that f(n) = T(n) is in O(n^3).
From the somewhat formal derivation above it's apparent that any constants in function g(n) can just be extracted and included in the constant c in (+), hence you'll never (at least should not) see time complexity described as e.g. O((n^3)/2). When using Big-O notation, we're describing an upper bound on the asymptotic behaviour of the algorithm, hence only the dominant term is of interest (however not how this is scaled with constants).
I was going through basics of Big-O notation.
f(n) = Ω(g(n)) means c.g(n) is a lower bound on f(n) such that f(n) is always ≥ c.g(n)
f(n) = O(g(n)) means c.g(n) is an upper bound on f(n) such that f(n) is always
≤ c.g(n)
for all n ≥ n0
upper and lower bound is clear in graph above, but why is f(n) and upperbound intersecting? when its clear from above definition? does that have meaning or I am just pointing out unnecessarily?
Source: The Algorith Design Manual by Skiena
Based on the first two definitions, there should not be an intersection because of the word always
f(n) = Ω(g(n)) means c.g(n) is a lower bound on f(n) such that f(n) is
always ≥ c.g(n)
f(n) = O(g(n)) means c.g(n) is an upper bound on f(n) such that f(n)
is always ≤ c.g(n)
These definitions are not exactly correct. Because the idea for big-O notation is to check the number of operations when n is really big. In layman terms it means that you start checking the complexity only after some number which you consider big enough. This is outlined on your picture:
Upper and lower bounds valid for n > n0 ...
and this is why on the picture you have a vertical line n0. So you do not care about anything before this line, because you consider only numbers after n0 big enough.
To make these definitions exactly correct, just add for n > n0 at the end of both of them.
The definition is simply inaccurate. Big-O notation is about asymptotic growth. As such, it's properties are considered for "large enough N", which means it might not hold true for small N's.
In the chart, a "large enough N" is marked as N0, after which the limiting property is maintained.
In addition to what has been already said in other answers, the inequalities in the definition are incorrect as well, they should be reversed:
f(n) = Ω(g(n)) means c.g(n) is a lower bound on f(n) such that f(n) is always ≥ c.g(n)
f(n) = O(g(n)) means c.g(n) is an upper bound on f(n) such that f(n) is always ≤ c.g(n)
If I have already known f(n) is O(g(n)). From the definition of little-oh, how to prove that f(n) is o(n * g(n))?
Given: f(n) is in O(g(n)).
Using the definition of big-O notation, we can write this as:
f(n) is in O(g(n))
=> |f(n)| ≤ k*|g(n)|, for some constant k>0 (+)
for n sufficiently large (say, n>N)
For the definition of big-O used as above, see e.g.
https://www.khanacademy.org/computing/computer-science/algorithms/asymptotic-notation/a/big-o-notation
Prove: Given (+), then f(n) is in o(n*g(n)).
Lets first state what little-o notation means:
Formally, f(n) = o(g(n)) (or f(n) ∈ o(g(n))) as n → ∞ means that
for every positive constant ε there exists a constant N such that
|f(n)| ≤ ε*|g(n)|, for all n > N (++)
From https://en.wikipedia.org/wiki/Big_O_notation#Little-o_notation.
Now, using (+), we can write
|f(n)| ≤ k*|g(n)|, som k>0, n sufficiently large
<=> { n > 0 } <=> n*|f(n)| ≤ k*n*|g(n)|
<=> n*|f(n)| ≤ k*|n*g(n)|
<=> |f(n)| ≤ (k/n)*|n*g(n)| (+++)
Return to the definition of little-o, specifically (++), and let, without loss of generality, k be fixed. Now, every positive constant ε can be described as
ε = k/C, for some constant C>0 (with k fixed, k>0) (*)
Now, assume, without loss of generality, that n is larger than this C, i.e., n>C. Then, (*) and (+++) yields
|f(n)| ≤ (k/n)*|n*g(n)| < (k/C)*|n*g(n)| = ε*|n*g(n)| (**)
^ ^
| |
since `n>C` (*)
Since we're studying asymptotic behaviour, we can choose to to assign a lower bound to n to any value larger than C (in fact, that's in the definition of both big-O and little-o, "n sufficiently large"), and hence---by the definition of little-oh above---, we have:
- As shown above, (+) implies (**)
- By the definition of little-o, (**) shows that f(n) is in o(n*g(n))
- Subsequently, we've shown that, given (+), then: f(n) is in o(n*g(n))
Result: If f(n) is in O(g(n)), then f(n) is in o(n*g(n)), where these two relations refer big-O and litte-O asymptotic bounds, respectively.
Comment: The result is, in fact, quite trivial. The big-O and little-o notation differ only in one of the two constants used in proving the upper bounds, i.e., we can write the definitions of big-O and little-O as:
f(n) is said to be in O(g(n)) if we can find a set of positive constants (k, N), such that f(n) < k*g(n) holds for all n>N.
f(n) is said to be in o(g(n)) if we can find a positive constant N, such that f(n) < ε*g(n) holds for all n>N, and for every positive constant ε.
The latter is obvious a stricter constraint, but if we can make use of one extra power of n in the left-hand-side of f(n) < ε*g(n) (i.e., f(n) < ε*n*g(n)), then even for infinitesimal values of ε, we can always choose the other constant N freely to be sufficiently large for ε*n to provide us any constant k that can be used to show that f(n) is in O(g(n)) (as, recall, n>N).
There is a lot of Explanation about big-0, But i'm really confused on this part.
Acoording to the definition of Big-O in this function
f (n) ≤ c ·g(n), for n ≥ n0
“ f (n) is big-Oh of g(n).”
But A description of a function in terms of big O notation usually only provides an upper bound on the growth rate of the function.
so for e.g here 34 is a upper bound for the set { 5, 10, 34 }
So if in this graph how f(n) is O(g(n)) because if i get the upper bound of g(n) function it's value would be different than what is mentioned here for n>=n0 ..
Beyond n0, f(n) will not grow faster than g(n). f(n)'s rate of growth as a function of n is at most g(n).
g(n)'s rate of growth is said to be an upper-bound of f(n)'s rate of growth of f(n) is Big-O of g(n).
The worst case rate of growth of f(n) will be at most g(n) since f(n) is Big-O of g(n).
This is all about knowing just how big f(n) can grow relative to another known function.
For example, if f(n) = n^2, and g(n) is n^3, then trivially f(n) is Big-O of g(n) since n^2 will never grow faster than n^3.
"c" is used for mathematical proofs - it's just a linear scaling variable. We can't just go around and claim something is Big-O of something else. If we choose n0 and c for a given g(n), and this equation holds
f(n) ≤ c ·g(n), for n ≥ n0
then we can show that truly f(n) is Big-O of g(n).
Example:
f(n) = n^2;
g(n) = n^3;
We can choose n0 = 1, and c = 1 such that
f(n) ≤ 1 ·g(n), for n ≥ 1
which becomes
n^2 ≤ 1 ·n^3, for n ≥ 1
which always holds, thus f(n) is proven to be Big-O of g(n).
Proofs can get more complicated, for instance this, but this is the gist of it.