How the complexity of an algorithm involved with combinatorial operations is classified.
Let's say the input is m, n, and the complexity is determined by C(m,n). (C is the combination function of choosing m from n). The question is how the complexity should be categorized instead of just giving C(m,n).
I mean, to give an idea of the running time of an algorithm, you can say the algorithm is of polynomial, exponential time complexity. But what to do with C(m,n) ?
I know factorials can be approximated by using Stirling's approximation, but the result is still too complex to put it in a complexity class.
If you insist on retaining both m and n, then it's going to be hard to do better than Stirling's approximation. The upper bound for m alone is C(m, m/2), which is asymptotic to 2m/√m and thus exponential.
You can't "put it into a complexity class" because it isn't a single-variable running time.
It's "asymptotic behaviour" is undefined, because which variable should we consider to approach infinite? Even if we said both approach infinite lim {n->inf, m->inf} nCm is undefined because their relative values are undefined. I.e. the behaviour depends not just on n and m being greater than a certain value, but their relative values as well.
The complexity depends on two variables, and nCm is a perfectly valid complexity function.
If you have a reasonable approximation for m relative to n then you can class it more easily. Maybe it's worthwhile working out cases, where m = O(n), m = O(1).
Or where m = [kn] and 0 <= k <= 1 is constant + Stilring's formula, gives you a nice relation in one variable while still being able to consider values of m relative to k.
Related
This is identical to the question found on Check if one integer is an integer power of another, but I am wondering about the complexity of a method that I came up with to solve this problem.
Given an integer n and another integer m, is n = m^p for some integer p. Note that ^ here is exponentiation and not xor.
There is a simple O(log_m n) solution based on dividing n repeatedly by m until it's 1 or until there's a non-zero remainder.
I'm thinking of a method inspired by binary search, and it's not clear to me how complexity should be calculated in this case.
Essentially you start with m, then you go to m^2, then m^4, m^8, m^16, .....
When you find that m^{2^k} > n, you check the range bounded by m^{2^{k-1}} and m^{2^k}. Is this solution O(log_2 (log_m(n)))?
Somewhat related to this, if I do something like
m^2 * m^2
vs.
m * m * m * m
Do these 2 have the same complexity? If they do, then I think the algorithm I came up with is still O(log_m (n))
Not quite. First of all, let's assume that multiplication is O(1), and exponentiation a^b is O(log b) (using exponentiation by squaring).
Now using your method of doubling the exponent p_candidate and then doing a binary search, you can find the real p in log(p) steps (or observe that p does not exist). But each try within the binary search requires you to compute m^p_candidate, which is bounded by m^p, which by assumption is O(log(p)). So the overall time complexity is O(log^2(p)).
But we want to express the time complexity in terms of the inputs n and m. From the relationship n = m^p, we get p = log(n)/log(m), and hence log(p) = log(log(n)/log(m)). Hence the overall time complexity is
O(log^2(log(n)/log(m)))
If you want to get rid of the m, you can provide a looser upper bound by using
O(log^2(log(n)))
which is close to, but not quite O(log(log(n))).
(Note that you can always omit the logarithmic bases in the O-notation since all logarithmic functions differ only by a constant factor.)
Now, the interesting question is: is this algorithm better than one that is O(log(n))? I haven't proved it, but I'm pretty certain it is the case that O(log^2(log(n))) is in O(log(n)) but not vice versa. Anyone cares to prove it?
I am studying the time complexity of an algorithm to find if a string n contains a substring m where m <= n. The result of the analysis is that the time complexity is O(nm). So taking this time complexity as starting point and knowing that m <= n, thus mn <= n^2. Can we say that time complexity in big-O notation is O(n^2).
It is indeed the case that if the runtime of a function is O(mn) and you know for a fact that m ≤ n, then the runtime of the function is O(n2).
However, this may not be the best way of characterizing the runtime. If m is a tunable parameter that can range from, say, 0 to n, then O(mn) could be a "better" way of characterizing the runtime. For example, suppose this is an algorithm that finds the m smallest elements out of an n-element array. If you want to use this algorithm to find the top five elements (m = 5) out of ten million (n = 106), then characterizing the runtime as O(n2) would greatly overestimate the actual amount of work done, while O(mn) would give a better bound on the runtime.
In the case of string searching, the fact that the two strings can have wildly different lengths is a good reason to characterize the runtime as O(mn), as it shows you how the runtime scales with the runtime of both of the input strings, not just one of the strings.
Hope this helps!
Could someone explain the difference between polynomial-time, non-polynomial-time, and exponential-time algorithms?
For example, if an algorithm takes O(n^2) time, then which category is it in?
Below are some common Big-O functions while analyzing algorithms.
O(1) - Constant time
O(log(n)) - Logarithmic time
O(n log(n)) - Linearithmic time
O((log(n))c) - Polylogarithmic time
O(n) - Linear time
O(n2) - Quadratic time
O(nc) - Polynomial time
O(cn) - Exponential time
O(n!) - Factorial time
(n = size of input, c = some constant)
Here is the model graph representing Big-O complexity of some functions
graph credits http://bigocheatsheet.com/
Check this out.
Exponential is worse than polynomial.
O(n^2) falls into the quadratic category, which is a type of polynomial (the special case of the exponent being equal to 2) and better than exponential.
Exponential is much worse than polynomial. Look at how the functions grow
n = 10 | 100 | 1000
n^2 = 100 | 10000 | 1000000
k^n = k^10 | k^100 | k^1000
k^1000 is exceptionally huge unless k is smaller than something like 1.1. Like, something like every particle in the universe would have to do 100 billion billion billion operations per second for trillions of billions of billions of years to get that done.
I didn't calculate it out, but ITS THAT BIG.
O(n^2) is polynomial time. The polynomial is f(n) = n^2. On the other hand, O(2^n) is exponential time, where the exponential function implied is f(n) = 2^n. The difference is whether the function of n places n in the base of an exponentiation, or in the exponent itself.
Any exponential growth function will grow significantly faster (long term) than any polynomial function, so the distinction is relevant to the efficiency of an algorithm, especially for large values of n.
Polynomial time.
A polynomial is a sum of terms that look like Constant * x^k
Exponential means something like Constant * k^x
(in both cases, k is a constant and x is a variable).
The execution time of exponential algorithms grows much faster than that of polynomial ones.
Exponential (You have an exponential function if MINIMAL ONE EXPONENT is dependent on a parameter):
E.g. f(x) = constant ^ x
Polynomial (You have a polynomial function if NO EXPONENT is dependent on some function parameters):
E.g. f(x) = x ^ constant
More precise definition of exponential
The definition of polynomial is pretty much universal and straightforward so I won't discuss it further.
The definition of Big O is also quite universal, you just have to think carefully about the M and the x0 in the Wikipedia definition and work through some examples.
So in this answer I would like to focus on the precise definition of the exponential as it requires a bit more thought/is less well known/is less universal, especially when you start to think about some edge cases. I will then contrast it with polynomials a bit further below
https://cstheory.stackexchange.com/questions/22588/is-it-right-to-call-2-sqrtn-exponential
https://math.stackexchange.com/questions/55468/how-to-prove-that-exponential-grows-faster-than-polynomial
The most common definition of exponential time is:
2^{polymonial(n)}
where polynomial is a polynomial that:
is not constant, e.g. 1, otherwise the time is also constant
the highest order term has a positive coefficient, otherwise it goes to zero at infinity, e.g. 2^{-n^2 + 2n + 1}
so a polynomial such as this would be good:
2^{n^2 + 2n + 1}
Note that the base 2 could be any number > 1 and the definition would still be valid because we can transform the base by multiplying the exponent, e.g.:
8^{polymonial(n)} = (2^3)^{polymonial(n)} = 2^{3 * polymonial(n)}
and 3 * polymonial(n) is also a polynomial.
Also note that constant addition does not matter, e.g. 2^{n + 1} = 2 * 2^{n} and so the + 1 does not matter for big O notation.
Therefore, two possible nice big O equivalent choices for a canonical "smallest exponential" would be for any small positive e either of:
(1 + e)^{n}
2^{en}
for very small e.
The highest order term of the polynomial in the exponent in both cases is n^1, order one, and therefore the smallest possible non-constant polynomial.
Those two choices are equivalent, because as saw earlier, we can transform base changes into an exponent multiplier.
Superpolynomial and sub-exponential
But note that the above definition excludes some still very big things that show up in practice and that we would be tempted to call "exponential", e.g.:
2^{n^{1/2}}. This is a bit like a polynomial, but it is not a polynomial because polynomial powers must be integers, and here we have 1/2
2^{log_2(n)^2}
Those functions are still very large, because they grow faster than any polynomial.
But strictly speaking, they are big O smaller than the exponentials in our strict definition of exponential!
This motivates the following definitions:
superpolynomial: grows faster than any polynomial
subexponential: grows less fast than any exponential, i.e. (1 + e)^{n}
and all the examples given above in this section fall into both of those categories. TODO proof.
Keep in mind that if you put something very small on the exponential, it might go back to polynomial of course, e.g.:
2^{log_2(n)} = n
And that is also true for anything smaller than log_2, e.g.:
2^{log_2(log_2(n))} = log_2(n)
is sub-polynomial.
Important superpolynomial and sub-exponential examples
the general number field sieve the fastest 2020-known algorithm for integer factorization, see also: What is the fastest integer factorization algorithm? That algorithm has complexity of the form:
e^{(k + o(1))(ln(n)^(1/3) * ln(ln(n)))^(2/3)}
where n is the factored number, and the little-o notation o(1) means a term that goes to 0 at infinity.
That complexity even has a named generalization as it presumably occurs in other analyses: L-notation.
Note that the above expression itself is clearly polynomial in n, because it is smaller than e^{ln(n)^(1/3) * ln(n))^(2/3)} = e^{ln(n)} = n.
However, in the context of factorization, what really matters is note n, but rather "the number of digits of n", because cryptography parties can easily generate crypto keys that are twice as large. And the number of digits grows as log_2. So in that complexity, what we really care about is something like:
e^{(k + o(1))(n^(1/3) * ln(n)^(2/3)}
which is of course both superpolynomial and sub-exponential.
The fantastic answer at: What would cause an algorithm to have O(log log n) complexity? gives an intuitive explanation of where the O(log log n) comes from: while log n comes from an algorithm that removes half of the options at each step, and log log n comes from an algorithm that reduces the options to the square root of the total at each step!
https://quantumalgorithmzoo.org/ contains a list of algorithms which might be of interest to quantum computers, and in most cases, the quantum speedup relative to a classical computer is not strictly exponential, but rather superpolynomial. However, as this answer will have hopefully highlighted, this is still extremely significant and revolutionary. Understanding that repository is what originally motivated this answer :-)
It is also worth noting that we currently do not expect quantum computers to solve NP-complete problems, which are also generally expected to require exponential time to solve. But there is no proof otherwise either. See also: https://cs.stackexchange.com/questions/130470/can-quantum-computing-help-solve-np-complete-problems
https://math.stackexchange.com/questions/3975382/what-problems-are-known-to-be-require-superpolynomial-time-or-greater-to-solve asks about any interesting algorithms that have been proven superpolynomial (and presumably with proof of optimality, otherwise the general number sieve would be an obvious choice, but we don't 2020-know if it is optimal or not)
Proof that exponential is always larger than polynomial at infinity
https://math.stackexchange.com/questions/3975382/what-problems-are-known-to-be-require-superpolynomial-time-or-greater-to-solve
Discussions of different possible definitions of sub-exponential
https://cstheory.stackexchange.com/questions/22588/is-it-right-to-call-2-sqrtn-exponential
https://math.stackexchange.com/questions/55468/how-to-prove-that-exponential-grows-faster-than-polynomial
https://en.wikipedia.org/w/index.php?title=Time_complexity&oldid=1026049783#Sub-exponential_time
polynomial time O(n)^k means Number of operations are proportional to power k of the size of input
exponential time O(k)^n means Number of operations are proportional to the exponent of the size of input
Polynomial examples: n^2, n^3, n^100, 5n^7, etc….
Exponential examples: 2^n, 3^n, 100^n, 5^(7n), etc….
o(n sequre) is polynimal time complexity while o(2^n) is exponential time complexity
if p=np when best case , in the worst case p=np not equal becasue when input size n grow so long or input sizer increase so longer its going to worst case and handling so complexity growth rate increase and depend on n size of input when input is small it is polynimal when input size large and large so p=np not equal it means growth rate depend on size of input "N".
optimization, sat, clique, and independ set also met in exponential to polynimal.
Here's the most simplest explaination for newbies:
A Polynomial:
if an expression contains or function is equal to when a constant is the power of a variable e.g.
f(n) = 2 ^ n
while
An Exponential:
if an expression contains or function is qual to when a variable is the power of a constant e.g.
f(n) = n ^ 2
for i = 0 to size(arr)
for o = i + 1 to size(arr)
do stuff here
What's the worst-time complexity of this? It's not N^2, because the second one decreases by one every i loop. It's not N, it should be bigger. N-1 + N-2 + N-3 + ... + N-N+1.
It is N ^ 2, since it's the product of two linear complexities.
(There's a reason asymptotic complexity is called asymptotic and not identical...)
See Wikipedia's explanation on the simplifications made.
Think of it like you are working with a n x n matrix. You are approximately working on half of the elements in the matrix, but O(n^2/2) is the same as O(n^2).
When you want to determine the complexity class of an algorithm, all you need is to find the fastest growing term in the complexity function of the algorithm. For example, if you have complexity function f(n)=n^2-10000*n+400, to find O(f(n)), you just have to find the "strongest" term in the function. Why? Because for n big enough, only that term dictates the behavior of the entire function. Having said that, it is easy to see that both f1(n)=n^2-n-4 and f2(n)=n^2 are in O(n^2). However, they, for the same input size n, don't run for the same amount of time.
In your algorithm, if n=size(arr), the do stuff here code will run f(n)=n+(n-1)+(n-2)+...+2+1 times. It is easy to see that f(n) represents a sum of an arithmetic series, which means f(n)=n*(n+1)/2, i.e. f(n)=0.5*n^2+0.5*n. If we assume that do stuff here is O(1), then your algorithm has O(n^2) complexity.
for i = 0 to size(arr)
I assumed that the loop ends when i becomes greater than size(arr), not equal to. However, if the latter is the case, than f(n)=0.5*n^2-0.5*n, and it is still in O(n^2). Remember that O(1),O(n),0(n^2),... are complexity classes, and that complexity functions of algorithms are functions that describe, for the input size n, how many steps there is in the algorithm.
It's n*(n-1)/2 which is equal to O(n^2).
Could someone explain the difference between polynomial-time, non-polynomial-time, and exponential-time algorithms?
For example, if an algorithm takes O(n^2) time, then which category is it in?
Below are some common Big-O functions while analyzing algorithms.
O(1) - Constant time
O(log(n)) - Logarithmic time
O(n log(n)) - Linearithmic time
O((log(n))c) - Polylogarithmic time
O(n) - Linear time
O(n2) - Quadratic time
O(nc) - Polynomial time
O(cn) - Exponential time
O(n!) - Factorial time
(n = size of input, c = some constant)
Here is the model graph representing Big-O complexity of some functions
graph credits http://bigocheatsheet.com/
Check this out.
Exponential is worse than polynomial.
O(n^2) falls into the quadratic category, which is a type of polynomial (the special case of the exponent being equal to 2) and better than exponential.
Exponential is much worse than polynomial. Look at how the functions grow
n = 10 | 100 | 1000
n^2 = 100 | 10000 | 1000000
k^n = k^10 | k^100 | k^1000
k^1000 is exceptionally huge unless k is smaller than something like 1.1. Like, something like every particle in the universe would have to do 100 billion billion billion operations per second for trillions of billions of billions of years to get that done.
I didn't calculate it out, but ITS THAT BIG.
O(n^2) is polynomial time. The polynomial is f(n) = n^2. On the other hand, O(2^n) is exponential time, where the exponential function implied is f(n) = 2^n. The difference is whether the function of n places n in the base of an exponentiation, or in the exponent itself.
Any exponential growth function will grow significantly faster (long term) than any polynomial function, so the distinction is relevant to the efficiency of an algorithm, especially for large values of n.
Polynomial time.
A polynomial is a sum of terms that look like Constant * x^k
Exponential means something like Constant * k^x
(in both cases, k is a constant and x is a variable).
The execution time of exponential algorithms grows much faster than that of polynomial ones.
Exponential (You have an exponential function if MINIMAL ONE EXPONENT is dependent on a parameter):
E.g. f(x) = constant ^ x
Polynomial (You have a polynomial function if NO EXPONENT is dependent on some function parameters):
E.g. f(x) = x ^ constant
More precise definition of exponential
The definition of polynomial is pretty much universal and straightforward so I won't discuss it further.
The definition of Big O is also quite universal, you just have to think carefully about the M and the x0 in the Wikipedia definition and work through some examples.
So in this answer I would like to focus on the precise definition of the exponential as it requires a bit more thought/is less well known/is less universal, especially when you start to think about some edge cases. I will then contrast it with polynomials a bit further below
https://cstheory.stackexchange.com/questions/22588/is-it-right-to-call-2-sqrtn-exponential
https://math.stackexchange.com/questions/55468/how-to-prove-that-exponential-grows-faster-than-polynomial
The most common definition of exponential time is:
2^{polymonial(n)}
where polynomial is a polynomial that:
is not constant, e.g. 1, otherwise the time is also constant
the highest order term has a positive coefficient, otherwise it goes to zero at infinity, e.g. 2^{-n^2 + 2n + 1}
so a polynomial such as this would be good:
2^{n^2 + 2n + 1}
Note that the base 2 could be any number > 1 and the definition would still be valid because we can transform the base by multiplying the exponent, e.g.:
8^{polymonial(n)} = (2^3)^{polymonial(n)} = 2^{3 * polymonial(n)}
and 3 * polymonial(n) is also a polynomial.
Also note that constant addition does not matter, e.g. 2^{n + 1} = 2 * 2^{n} and so the + 1 does not matter for big O notation.
Therefore, two possible nice big O equivalent choices for a canonical "smallest exponential" would be for any small positive e either of:
(1 + e)^{n}
2^{en}
for very small e.
The highest order term of the polynomial in the exponent in both cases is n^1, order one, and therefore the smallest possible non-constant polynomial.
Those two choices are equivalent, because as saw earlier, we can transform base changes into an exponent multiplier.
Superpolynomial and sub-exponential
But note that the above definition excludes some still very big things that show up in practice and that we would be tempted to call "exponential", e.g.:
2^{n^{1/2}}. This is a bit like a polynomial, but it is not a polynomial because polynomial powers must be integers, and here we have 1/2
2^{log_2(n)^2}
Those functions are still very large, because they grow faster than any polynomial.
But strictly speaking, they are big O smaller than the exponentials in our strict definition of exponential!
This motivates the following definitions:
superpolynomial: grows faster than any polynomial
subexponential: grows less fast than any exponential, i.e. (1 + e)^{n}
and all the examples given above in this section fall into both of those categories. TODO proof.
Keep in mind that if you put something very small on the exponential, it might go back to polynomial of course, e.g.:
2^{log_2(n)} = n
And that is also true for anything smaller than log_2, e.g.:
2^{log_2(log_2(n))} = log_2(n)
is sub-polynomial.
Important superpolynomial and sub-exponential examples
the general number field sieve the fastest 2020-known algorithm for integer factorization, see also: What is the fastest integer factorization algorithm? That algorithm has complexity of the form:
e^{(k + o(1))(ln(n)^(1/3) * ln(ln(n)))^(2/3)}
where n is the factored number, and the little-o notation o(1) means a term that goes to 0 at infinity.
That complexity even has a named generalization as it presumably occurs in other analyses: L-notation.
Note that the above expression itself is clearly polynomial in n, because it is smaller than e^{ln(n)^(1/3) * ln(n))^(2/3)} = e^{ln(n)} = n.
However, in the context of factorization, what really matters is note n, but rather "the number of digits of n", because cryptography parties can easily generate crypto keys that are twice as large. And the number of digits grows as log_2. So in that complexity, what we really care about is something like:
e^{(k + o(1))(n^(1/3) * ln(n)^(2/3)}
which is of course both superpolynomial and sub-exponential.
The fantastic answer at: What would cause an algorithm to have O(log log n) complexity? gives an intuitive explanation of where the O(log log n) comes from: while log n comes from an algorithm that removes half of the options at each step, and log log n comes from an algorithm that reduces the options to the square root of the total at each step!
https://quantumalgorithmzoo.org/ contains a list of algorithms which might be of interest to quantum computers, and in most cases, the quantum speedup relative to a classical computer is not strictly exponential, but rather superpolynomial. However, as this answer will have hopefully highlighted, this is still extremely significant and revolutionary. Understanding that repository is what originally motivated this answer :-)
It is also worth noting that we currently do not expect quantum computers to solve NP-complete problems, which are also generally expected to require exponential time to solve. But there is no proof otherwise either. See also: https://cs.stackexchange.com/questions/130470/can-quantum-computing-help-solve-np-complete-problems
https://math.stackexchange.com/questions/3975382/what-problems-are-known-to-be-require-superpolynomial-time-or-greater-to-solve asks about any interesting algorithms that have been proven superpolynomial (and presumably with proof of optimality, otherwise the general number sieve would be an obvious choice, but we don't 2020-know if it is optimal or not)
Proof that exponential is always larger than polynomial at infinity
https://math.stackexchange.com/questions/3975382/what-problems-are-known-to-be-require-superpolynomial-time-or-greater-to-solve
Discussions of different possible definitions of sub-exponential
https://cstheory.stackexchange.com/questions/22588/is-it-right-to-call-2-sqrtn-exponential
https://math.stackexchange.com/questions/55468/how-to-prove-that-exponential-grows-faster-than-polynomial
https://en.wikipedia.org/w/index.php?title=Time_complexity&oldid=1026049783#Sub-exponential_time
polynomial time O(n)^k means Number of operations are proportional to power k of the size of input
exponential time O(k)^n means Number of operations are proportional to the exponent of the size of input
Polynomial examples: n^2, n^3, n^100, 5n^7, etc….
Exponential examples: 2^n, 3^n, 100^n, 5^(7n), etc….
o(n sequre) is polynimal time complexity while o(2^n) is exponential time complexity
if p=np when best case , in the worst case p=np not equal becasue when input size n grow so long or input sizer increase so longer its going to worst case and handling so complexity growth rate increase and depend on n size of input when input is small it is polynimal when input size large and large so p=np not equal it means growth rate depend on size of input "N".
optimization, sat, clique, and independ set also met in exponential to polynimal.
Here's the most simplest explaination for newbies:
A Polynomial:
if an expression contains or function is equal to when a constant is the power of a variable e.g.
f(n) = 2 ^ n
while
An Exponential:
if an expression contains or function is qual to when a variable is the power of a constant e.g.
f(n) = n ^ 2