i am stuck on this one and i don't know how to solve it, no matter what i try i just can't find a way to play with the function so i can represent it in a way that will allow me to find a g(n), so that g(n) is T(n)∈Θ(g(n))
the function i am having trouble with is:
$T(n)=4n^4T(\sqrt n) +(n^6lgn+3lg^7n)(2n^2lgn+lg^3n)$
additionally, if you can - could you please check if i am on the right path with:
$T(n)=T(n-1)+\frac{1}{n}+\frac{1}{n^2}$
to solve it i tried to use: $T(n)-T(n-1)=\frac{1}{n}+\frac{1}{n^2}$ iff $(T(n)-T(n-1))+(T(n-1)-T(n-2))+\ldots+(T(2)-T(1))=\frac{1}{n}+\frac{1}{n-1}+...+\frac{1}{n^2}+\frac{1}{\left(n-1\right)^2}+....$ iff $(T(n)-T(n-1))+(T(n-1)-T(n-2))+\ldots+(T(2)-T(1))=T(n)=T(1)+\sum_{k=2}^n\frac{1}{n}+\sum_{k=2}^n\frac{1}{n^2}$ and then using the harmonic series formula. however i don't know how to continue and finish it and find the asymptotic boundaries to solve it
i hope that on the second i am on the right path. however i don't know how to solve the first one at all. if i've done any mistakes, please show me the right way so i can improve my mistakes.
thank you very much for your help
sorry that for some reason math doesn't show correctly here
Following on from the comments:
Solving (2) first since it is more straightforward.
Your expansion attempt is correct. Writing it slightly differently:
A, harmonic series - asymptotically equal to the natural logarithm:
γ = 0.57721... is the Euler-Mascheroni constant.
B, sum of inverse squares - the infinite sum is the famous Basel problem:
Which is 1.6449.... Therefore since B is monotonically increasing, it will always be O(1).
The total complexity of (2) is simply Θ(log n).
(1) is a little more tedious.
Little-o notation: strictly lower complexity class, i.e.:
Assume a set of N functions {F_i} is ordered in decreasing order of complexity, i.e. F2 = o(F1) etc. Take a linear combination of them:
Thus a sum of different functions is asymptotically equal to the one with the highest growth rate.
To sort the terms in the expansion of the two parentheses, note that
Provable by applying L'Hopital's rule. So the only asymptotically significant term is n^6 log n * 2n^2 log n = 2n^8 log^2 n.
Expand the summation as before, note that i) the factor 4n^4 accumulates, ii) the parameter for the m-th expansion is n^(1/(2^m)) (repeated square-root).
The new term added by the m-th expansion is therefore (will assume you know how to do this since you were able to do the same for (2)):
Rather surprisingly, each added term is precisely equal to the first.
Assume that the stopping condition for recursive expansion is n < 2 (which of course rounds down to T(1)):
Since each added term t_m is always the same, simply multiply by the maximum number of expansions:
Function (1) is
So when we do an iterative solution to find the nth number in a Fibonacci sequence, we run a for loop (n-2) times. This would mean that the time complexity would be O(n). Is this correct or would it actually be pseudo-polynomial depending on the number of bits of the input, much like the Knapsack problem?
Here, I assume Fib(n) is an iterative version of a program that computes Fibonacci numbers. Perhaps something like:
def Fib(n):
a, b = 0, 1
for _ in xrange(n):
a, b = b, a + b
return a
"Fib(n) is pseudo-polynomial" means in this context that computing Fib is bounded by a polynomial of its argument, n, but isn't bounded by a polynomial function of the size of the argument, log(n). That's true in this case.
"Fib(n) is O(n)" is a statement about the running time of Fib with respect to the value of its argument. There's sometimes ambiguity what "n" is, but here there's none -- it's the input to Fib, otherwise "n" would refer to two different things in the original statement. That's true here (although see the technical side-note below).
"Fib is O(n)" is ambiguous. There are people who will tell you that n clearly refers to the argument, and there's others who will tell you that n always refers to the size of the argument. The truth is that it's ambiguous and if it's not clear in context you should say what you mean (or ask what it means if you hear it and are confused). One context where it's not ambiguous is when you're talking about classes of P/NP problems -- there it's assumed that complexities are always relative to the size of the input.
A technical side-note
The iterative version of Fib(n) performs O(n) arithmetic operations, but whether it's O(n) time depends on your computational model, and specifically whether it can perform arbitrary integer arithmetic operations in O(1) time. Personally, I'd be careful and say "Fib(n) performs O(n) arithmetic operations" rather than "Fib(n) is O(n)" -- and if you plot the running time of Fib(n), you'll find it's not linear time in practice, as real bignum implementations are certainly not O(1) for all basic operations.
Yes, it is infact O(n). The time complexity of Knapsack Problem is a really weird one and is an exception.
I came across this passage on page 47 of Introduction to Algorithms by Cormen et al.:
The number of anonymous functions in an expression is understood to be equal to the number of times the asymptotic notation appearrs. For example in the expression:
Σ (i=1 to n) O(i)
there is only a single anonymous function (a function of i). This expression is not the same as O(1) + O(2) + ... + O(n), which doesn't really have a clean interpretation.
What does this mean?
I think they're saying that when they use that notation (sum of a big-O), it means there's a single O(i) function (call it f(i)), and then the expression refers to the sum from 1 to n of that function.
This isn't the same thing as if there were n different functions (call them f_1(i) to f_n(i)), each of which is O(i), and then the expression refers to the sum of f_1(1) + f_2(2) + ... + f_n(n). That latter thing is not what the notation means.
i think you should understand "The number of anonymous functions in an expression is understood to be equal to the number of times the asymptotic notation appears" at first.
it means if
Σ (i=1 to n) O(i)
so anonymous functions is one equals to O(i)
such as Σ (i=1 to n) f1(i)
and if Σ (i=1 to n) O(i)+O(i)
anonymous functions should have two functions equals to O(i)+O(i)
such as Σ (i=1 to n) f1(i)+f2(i)
The following is according to my understanding:
The passage simply states that.
This is what everyone first understands and it is the correct understanding of that cryptic passage, but the problem is WHY?, since we all are used to expand the sigma notations in the usual (and correct) way as in the expression above.
It turns out there is an exception when it comes to asymptotic notations like , etc. Recall that these asymptotic notations are all rough approximations for some anonymous function; due to this fact (the passage intends to convey that), before you expand and compute those sigma notations containing some asymptotic notation, you must firstly
figure out/evaluate/find out exactly that anonymous function and
substitute it for the asymptotic notation in the sigma expression and
only then expand and compute the sigma notation.
Thus:
But now let’s suppose, for the sake of argument, that the expression
is valid. But immediately we will face a problem here: since each of O(1), O(2), O(3),…, O(n) has a different value within its parentheses (1, 2, etc) and each subsequent number within parentheses is bigger than the previous one, it appears that each of these Big-Os represent a different anonymous function and each of these different anonymous functions grow faster than the previous different function.
are there a limited amount of basic O Notations, considering you are meant to 'distil' them down to their most important part?
O(n^2):
O(n):
O(1):
O(log n) logarithmic
O(n!) factorial
O(na) polynomial
Or are you expected to work out variations such as O(n^4) etc... and if so, is that the only exception? the power of X one?
Generally, you distill Big-O notation (and related Bachman-Landau notations like Big-Theta and Big-Omega) down to the operation of the fastest-growing N term. So, you remove/simplify lesser terms (N2 + N == O(N2)) and nonvariable coefficients of the term (O(4N2) == O(N2)), but NOT powers or exponent bases (O(34N) == O(3N)). You also don't strip variable coefficients; NlogN is NlogN, NOT logN or N.
So, you will normally only see numbers in a Big-Oh notation if if the complexity is polynomial (power of N) or exponential (Nth power of a base). The most common Big-Oh notations are much as you show, with the addition of NlogN (VERY common).
However, if you are differentiating between two algorithms of equal general complexity, you MAY add lesser terms and/or coefficients back in to demonstrate the relative difference; an algorithm that performs linearly but has double the instructions of another might be described as O(2N) when comparing it with the other O(N) algorithm. However, taken individually, both algorithms are linear (O(N)).
Some Big-O notations are not algebraic, and may involve multiple variables in therir simplest general-case form. The counting sort, for instance, is complexity O(Max(N,M)), where N is the number of elements in the list, and M is the range of those elements. Often it is possible to reduce this in specific cases by defining M in terms of N and thus reducing to a single variable (if the list in question is of the first N squares, M = N2-1), but in the general case both variables are independent and significant. BucketSort's complexity is officially O(N), but really it's more like O(NlogM) where M is the maximum value of the list of N elements. M is usually considered insignificant, but that depends on the values you normally sort (sorting 5 values each in the billions will require more loops to compare each power of 10 than traversals through the list to put them in the buckets) and on the radix used (RadixSort is a base-2 BucketSort; again, sorting values with a greater log2 value will require more loops than traversals).
The Big-O notation is a way to provide an upper bound on the limiting behaviour of a function. There are no restrictions on its functional form. However, there are certain conventions, as explained by Wikipedia:
In typical usage, the formal definition of O notation is not used directly; rather, the O notation for a function f(x) is derived by the following simplification rules:
If f(x) is a sum of several terms, the one with the largest growth rate is kept, and all others omitted.
If f(x) is a product of several factors, any constants (terms in the product that do not depend on x) are omitted.
There are, of course, some functional forms that show up more frequently that others. Some common classes are listed here.
No, the number of different O-classes is not finite.
As you already mentioned O(n^x) describes a different set for every x. And that is not the only "exception". O(x^n) is also a different set for every x. Likewise O(n^n), O(n^n^n), O(n^n^n^n) etc. are all different sets (and you can of course continue that ad infinitum).
In general, you split the expression into a sum of products, keep the largest term, and divide by constants to simplify it as much as possible.
ex:
n(2n+3log(n)) => 2n^2+3nlog(n) => 2n^2 => n^2
(n+1)(2nlog(n)+n) => 2n^2log(n)+n^2+2nlog(n)+n => 2n^2log(n) => n^2log(n)
So given the following program:
Is the time complexity of this program O(0)? In other words, is 0 O(0)?
I thought answering this in a separate question would shed some light on this question.
EDIT: Lots of good answers here! We all agree that 0 is O(1). The question is, is 0 O(0) as well?
From Wikipedia:
A description of a function in terms of big O notation usually only provides an upper bound on the growth rate of the function.
From this description, since the empty algorithm requires 0 time to execute, it has an upper bound performance of O(0). This means, it's also O(1), which happens to be a larger upper bound.
Edit:
More formally from CLR (1ed, pg 26):
For a given function g(n), we denote O(g(n)) the set of functions
O(g(n)) = { f(n): there exist positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 }
The asymptotic time performance of the empty algorithm, executing in 0 time regardless of the input, is therefore a member of O(0).
Edit 2:
We all agree that 0 is O(1). The question is, is 0 O(0) as well?
Based on the definitions, I say yes.
Furthermore, I think there's a bit more significance to the question than many answers indicate. By itself the empty algorithm is probably meaningless. However, whenever a non-trivial algorithm is specified, the empty algorithm could be thought of as lying between consecutive steps of the algorithm being specified as well as before and after the algorithm steps. It's nice to know that "nothingness" does not impact the algorithm's asymptotic time performance.
Edit 3:
Adam Crume makes the following claim:
For any function f(x), f(x) is in O(f(x)).
Proof: let S be a subset of R and T be a subset of R* (the non-negative real numbers) and let f(x):S ->T and c ≥ 1. Then 0 ≤ f(x) ≤ f(x) which leads to 0 ≤ f(x) ≤ cf(x) for all x∈S. Therefore f(x) ∈ O(f(x)).
Specifically, if f(x) = 0 then f(x) ∈ O(0).
It takes the same amount of time to run regardless of the input, therefore it is O(1) by definition.
Several answers say that the complexity is O(1) because the time is a constant and the time is bounded by the product of some coefficient and 1. Well, it is true that the time is a constant and it is bounded that way, but that doesn't mean that the best answer is O(1).
Consider an algorithm that runs in linear time. It is ordinarily designated as O(n) but let's play devil's advocate. The time is bounded by the product of some coefficient and n^2. If we consider O(n^2) to be a set, the set of all algorithms whose complexity is small enough, then linear algorithms are in that set. But it doesn't mean that the best answer is O(n^2).
The empty algorithm is in O(n^2) and in O(n) and in O(1) and in O(0). I vote for O(0).
I have a very simple argument for the empty algorithm being O(0): For any function f(x), f(x) is in O(f(x)). Simply let f(x)=0, and we have that 0 (the runtime of the empty algorithm) is in O(0).
On a side note, I hate it when people write f(x) = O(g(x)), when it should be f(x) ∈ O(g(x)).
Big O is asymptotic notation. To use big O, you need a function - in other words, the expression must be parametrized by n, even if n is not used. It makes no sense to say that the number 5 is O(n), it's the constant function f(n) = 5 that is O(n).
So, to analyze time complexity in terms of big O you need a function of n. Your algorithm always makes arguably 0 steps, but without a varying parameter talking about asymptotic behaviour makes no sense. Assume that your algorithm is parametrized by n. Only now you may use asymptotic notation. It makes no sense to say that it is O(n2), or even O(1), if you don't specify what is n (or the variable hidden in O(1))!
As soon as you settle on the number of steps, it's a matter of the definition of big O: the function f(n) = 0 is O(0).
Since this is a low-level question it depends on the model of computation.
Under "idealistic" assumptions, it is possible you don't do anything.
But in Python, you cannot say def f(x):, but only def f(x): pass. If you assume that every instruction, even pass (NOP), takes time, then the complexity is f(n) = c for some constant c, and unless c != 0 you can only say that f is O(1), not O(0).
It's worth noting big O by itself does not have anything to do with algorithms. For example, you may say sin x = x + O(x3) when discussing Taylor expansion. Also, O(1) does not mean constant, it means bounded by constant.
All of the answers so far address the question as if there is a right and a wrong answer. But there isn't. The question is a matter of definition. Usually in complexity theory the time cost is an integer --- although that too is just a definition. You're free to say that the empty algorithm that quits immediately takes 0 time steps or 1 time step. It's an abstract question because time complexity is an abstract definition. In the real world, you don't even have time steps, you have continuous physical time; it may be true that one CPU has clock cycles, but a parallel computer could easily have asynchronoous clocks and in any case a clock cycle is extremely small.
That said, I would say that it's more reasonable to say that the halt operation takes 1 time step rather than that it takes 0 time steps. It does seem more realistic. For many situations it's arguably very conservative, because the overhead of initialization is typically far greater than executing one arithmetic or logical operation. Giving the empty algorithm 0 time steps would only be reasonable to model, for example, a function call that is deleted by an optimizing compiler that knows that the function won't do anything.
It should be O(1). The coefficient is always 1.
Consider:
If something grows like 5n, you don't say O(5n), you say O(n) [in other words, O(1n)]
If something grows like 7n^2, you don't say O(7n^2), you say O(n^2) [in other words, O(1n^2)]
Likewise you should say O(1), not O(some other constant)
There is no such thing as O(0). Even an oracle machine or a hypercomputer require the time for one operation, i.e. solve(the_goldbach_conjecture), ergo:
All machines, theoretical or real, finite or infinite produce algorithms with a minimum time complexity of O(1).
But then again, this code right here is O(0):
// Hello world!
:)
I would say it's O(1) by definition, but O(0) if you want to get technical about it: since O(k1g(n)) is equivalent to O(k2g(n)) for any constants k1 and k2, it follows that O(1 * 1) is equivalent to O(0 * 1), and therefore O(0) is equivalent to O(1).
However, the empty algorithm is not like, for example, the identity function, whose definition is something like "return your input". The empty algorithm is more like an empty statement, or whatever happens between two statements. Its definition is "do absolutely nothing with your input", presumably without even the implied overhead of simply having input.
Consequently, the complexity of the empty algorithm is unique in that O(0) has a complexity of zero times whatever function strikes your fancy, or simply zero. It follows that since the whole business is so wacky, and since O(0) doesn't already mean something useful, and since it's slightly ridiculous to even discuss such things, a reasonable special case for O(0) is something like this:
The complexity of the empty algorithm is O(0) in time and space. An algorithm with time complexity O(0) is equivalent to the empty algorithm.
So there you go.
Given the formal definition of Big O:
Let f(x) and g(x) be two functions defined over the set of real numbers. Then, we write:
f(x) = O(g(x)) as x approaches infinity iff there exists a real M and a real x0 so that:
|f(x)| <= M * |g(x)| for every x > x0
As I see it, if we substitute g(x) = 0 (in order to have a program with complexity O(0)), we must have:
|f(x)| <= 0, for every x > x0 (the constraint of existence of a real M and x0 is practically lifted here)
which can only be true when f(x) = 0.
So I would say that not only the empty program is O(0), but it is the only one for which that holds. Intuitively, this should've been true since O(1) encompasses all algorithms that require a constant number of steps regardless of the size of its task, including 0. It's essentially useless to talk about O(0); it's already in O(1). I suspect it's purely out of simplicity of definition that we use O(1), where it could as well be O(c) or something similar.
0 = O(f) for all function f, since 0 <= |f|, so it is also O(0).
Not only is this a perfectly sensible question, but it is important in certain situations involving amortized analysis, especially when "cost" means something other than "time" (for example, "atomic instructions").
Let's say there is a datastructure featuring multiple operation types, for which an amortized analysis is being conducted. It could well happen that one type of operation can always be funded fully using "coins" deposited during previous operations.
There is a simple example of this: the "multipop queue" described in Cormen, Leiserson, Rivest, Stein [CLRS09, 17.2, p. 457], and also on Wikipedia. Each time an item is pushed, a coin is put on the item, for a total amortized cost of 2. When (multi) pops occur, they can be fully paid for by taking one coin from each item popped, so the amortized cost of MULTIPOP(k) is O(0). To wit:
Note that the amortized cost of MULTIPOP is a constant (0)
...
Moreover, we can also charge MULTIPOP operations nothing. To pop the
first plate, we take the dollar of credit off the plate and use it to
pay the actual cost of a POP operation. To pop a second plate, we
again have a dollar of credit on the plate to pay for the POP
operation, and so on. Thus, we have always charged enough up front to
pay for MULTIPOP operations. In other words, since each plate on the
stack has 1 dollar of credit on it, and the stack always has a
nonnegative number of plates, we have ensured that the amount of
credit is always nonnegative.
Thus O(0) is an important "complexity class" for certain amortized operations.
O(1) means the algorithm's time complexity is always constant.
Let's say we have this algorithm (in C):
void doSomething(int[] n)
{
int x = n[0]; // This line is accessing an array position, so it is time consuming.
int y = n[1]; // Same here.
return x + y;
}
I am ignoring the fact that the array could have less than 2 positions, just to keep it simple.
If we count the 2 most expensive lines, we have a total time of 2.
2 = O(1), because:
2 <= c * 1, if c = 2, for every n > 1
If we have this code:
public void doNothing(){}
And we count it as having 0 expansive lines, there is no difference in saying it has O(0) O(1), or O(1000), because for every one of these functions, we can prove the same theorem.
Normally, if the algorithm takes a constant number of steps to complete, we say it has O(1) time complexity.
I guess this is just a convention, because you could use any constant number to represent the function inside the O().
No. It's O(c) by convention whenever you don't have dependence on input size, where c is any positive constant (typically 1 is used - O(1) = O(12.37)).