Just started learning algorithms. So the exercise is to find if statement is always/sometimes true or false. Em, where does my logic fails here?
f(n) != O(g(n)) and g(n) != O(f(n))
O-notation is 0 <= f(n) <= cg(n) where c is some constant. So not equal here means:
f(n) > cg(n) and g(n) > cf(n)
If f(n) = g(n) = 1, and let's say c = 1/2:
1 > (1/2)*1 and 1 > (1/2)*1
So it is true in this case. But the book says it's false in this particular case. What part do I misunderstand?
Big-O is not 0 <= f(n) <= c g(n) for some constant, per se. It's that there exists a number c such that the relation holds for "large enough" values of n. (This is the "asymptotic" that we refer to when we call Big-O an asymptotic notation, the other common ones being Big-Theta and Big-Omega.)
For example, let's say there's an algorithm that operates on some data structure with n elements, and takes 3n^2 + 7n + 18 steps. Call this f(n). We say that the Big-O of this expression is O(n^2) because there exists a constant (in this case anything larger than 3) such that for all "large enough" values of n, f(n) <= c n^2.
Related
Suppose that f(n)=4^n and g(n)=n^n, will it be right to conclude that f(n)=Θ(g(n)).
In my opinion it's a correct claim but I'm not 100% sure.
It is incorrect. f(n) = Theta(g(n)) if and only if both f(n) = O(g(n)) and g(n) = O(f(n)). It is true that f(n) = O(g(n)). We will show that it is not the case that g(n) = O(f(n)).
Assume g(n) = O(f(n)). Then there exists a positive real constant c and a positive natural number n0 such that for all n > n0, g(n) <= c * f(n). For our functions, this implies n^n <= c * 4^n. If we take the nth root of both sides of this inequality we get n <= 4c^(1/n). We are free to assume c >= 1 and n0 >= since if we found a smaller value that worked a larger value would work too. For all c > 1 and n > 1, 4c^(1/n) is strictly less than 4c. But then if we choose n > 4c, the inequality is false. So, there cannot be an n0 such that for all n at least n0 the condition holds. This is a contradiction; our initial assumption is disproven.
I am learning about complexity theory and have a question asking to show truth/falsehood of a number of Big-O statements.
I've done the first few e.g. showing 2^(n+1) is in O(2^n) by finding a constant and N value. But now they are asking more abstract things, for example:
If f(n) is O(g(n)), is log f(n) in O(log g(n))?
Is 2^(f(n)) in O(2^(g(n)))
These both seem like they would be true but I don't know how to express them formally with a constant and a N value. If you can give an example of how I could show these I can go do the rest of the problems.
The comments are both accurate. Here are some notes along the lines you are probably looking for.
Assume f(n) is O(g(n)). Then there exist n0 and c such that f(n) <= cg(n) for n >= n0. Take the logarithm of both sides. log(f(n)) <= log(cg(n)). We can use the laws of logarithms to rewrite this as log(f(n)) <= log(c) + log(g(n)). If g(n) is greater than 1, then log(c) + log(g(n)) <= (1+log(c))*log(g(n)), so we can choose c' = 1 + log(c) and get the desired result. Otherwise, note that for g(n) = 1 we're still good since any choice for c' works.
The second one is not true. Choose f(n) = 2n and g(n) = n. We see f(n) is O(g(n)) by choosing c = 3. However, 2^(2n) = 4^n is not O(2^n). To see that, assume we had n0 and c. Then 4^n <= c*2^n. Dividing by 2^n gives 2^n <= c. But this can't be correct since n can increase indefinitely whereas c is fixed.
Let's say you were given two logarithmic functions like
and you were asked to find if f(n) is O(g(n)) Ω(g(n)) or Θ(g(n)), how would you go about it? I found questions like these easier when you were comparing two exponential equations, because for example with x(n) = n^2 and p(n) = n^2 you could find a c > 0 (ex 3) where x(n) <= cp(n) for all n greater than some n>0 and that would prove that x(n) = O(p(n)). However, comparing two logarithmic functions seems much more difficult for some reason. Any help is appreciated, thanks!
f(n) is O(g(n)) iff there is a constant c and n_0 such that f(n) <= c * g(n) for each n >= n_0.
f(n) is Ω(g(n)) iff there is a constant c and n_0 such that f(n) >= c * g(n) for each n >= n_0.
Now, f(n) is Θ(g(n)) iff f(n) is O(g(n)) and f(n) is Ω(g(n)).
So, in your cases, we have:
f(n) = log (n^2) = 2logn
which means, g(n) is logn and c = 2, which means f(n) <= 2 * logn and f(n) >= 2 * logn, which makes it Ω(logn).
Btw. its also f(n) <= n and f(n) >= 1, so f(n) can be O(n), but we don't use it, since we can find a better O(g(n)). In this case we don't have the same function in both notations, to for those values we don't have Ω. However, we just need one option for g(n) to declare Ω. In cases we can't find it, we say its not Ω. Note the word "we say".
In second case, we care only for "highest growing value", logn part. Now, c = 1, and g = log(n), so in this case, its also Ω(logn).
I was given the function 5n^3+2n+8 to prove for big-O and big-Omega. I finished big-O, but for big-Omega I end up with a single-term function. I canceled out 2n and 8 because they're positive and make my function larger, so I just end up with 5n^3. How do I choose C and n_0? or is it simply trivial in this case?
From Big-Ω (Big-Omega) notation (slightly modified):
If a running time of some function f(n) is Ω(g(n)), then for large
enough n, say n > n_0 > 0, the running time of f(n) is at least
C⋅g(n), for some constant C > 0.
Hence, if f(n) is in Ω(g(n)), then there exists some positive constants C and n_0 such at the following holds
f(n) ≥ C⋅g(n), for all n > n_0 (+)
Now, the choice of C and n_0 is not unique, it suffices that you can show one such set of constants (such that (+) holds) to be able to describe the running time using the Big-Omega notation, as posted above.
Hence, you are indeed almost there
f(n) = 5n^3+2n+8 > 5n^3 holds for all n larger than say, 1
=> f(n) ≥ 5⋅n^3 for all n > n_0 = 1 (++)
Finally, (++) is just (+) for g(n) = n^3 and C=5, and hence, by (+), f(n) is in Ω(n^3).
I'm fairly new to the Big-O stuff and I'm wondering what's the complexity of the algorithm.
I understand that every addition, if statement and variable initialization is O(1).
From my understanding first 'i' loop will run 'n' times and the second 'j' loop will run 'n^2' times. Now, the third 'k' loop is where I'm having issues.
Is it running '(n^3)/2' times since the average value of 'j' will be half of 'n'?
Does it mean the Big-O is O((n^3)/2)?
We can use Sigma notation to calculate the number of iterations of the inner-most basic operation of you algorithm, where we consider the sum = sum + A[k] to be a basic operation.
Now, how do we infer that T(n) is in O(n^3) in the last step, you ask?
Let's loosely define what we mean by Big-O notation:
f(n) = O(g(n)) means c · g(n) is an upper bound on f(n). Thus
there exists some constant c such that f(n) is always ≤ c · g(n),
for sufficiently large n (i.e. , n ≥ n0 for some constant n0).
I.e., we want to find some (non-unique) set of positive constants c and n0 such that the following holds
|f(n)| ≤ c · |g(n)|, for some constant c>0 (+)
for n sufficiently large (say, n>n0)
for some function g(n), which will show that f(n) is in O(g(n)).
Now, in our case, f(n) = T(n) = (n^3 - n^2) / 2, and we have:
f(n) = 0.5·n^3 - 0.5·n^2
{ n > 0 } => f(n) = 0.5·n^3 - 0.5·n^2 ≤ 0.5·n^3 ≤ n^3
=> f(n) ≤ 1·n^3 (++)
Now (++) is exactly (+) with c=1 (and choose n0 as, say, 1, n>n0=1), and hence, we have shown that f(n) = T(n) is in O(n^3).
From the somewhat formal derivation above it's apparent that any constants in function g(n) can just be extracted and included in the constant c in (+), hence you'll never (at least should not) see time complexity described as e.g. O((n^3)/2). When using Big-O notation, we're describing an upper bound on the asymptotic behaviour of the algorithm, hence only the dominant term is of interest (however not how this is scaled with constants).