I found this question in my Optimization Algorithm course, the full question is this:
If we can prove all Knapsack problems with capacity limited to 100 can be solved in polynomial time, then all Knapsack problems belong to P. Is this sentence true or false? Justify.
With my book and some research I came out with something like this:
First of all KP is an NP-complete problem. With Dynamic programming it can reach a pseudopolynomial time, but it's not enough.
If, absurdly, we can prove that KP with capacity limited to 100 can be solved in polynomial time then we can assume that KP belongs to P.
What do you think about my answer? I think the absurd is not so right in the last sentence.
Proving that all knapsack problems with a limited capacity can be solved in polynomial time does not prove that all knapsack problems are in P. If a problem is in P, that means that it can be solved in polynomial time. This means that it can be solved in O(n^k) where k is some integer. Big O is an upper bound, meaning that, if an algorithm is O(n), as n approaches infinity, the time it takes to do the algorithms will never be longer than n. By proving that all problems with n<100 can be solved in polynomial time, this makes no guarantee for much larger n. Therefore we cannot say that there is an algorithm that runs in O(n^k) and is therefore in P.
Related
The algorithm needs to generate all possible combinations from a given list (empty set excluded).
list => [1, 2, 3]
combinations => [{1}, {2}, {3}, {1,2}, {1,3}, {2,3}, {1,2,3}]
This algorithm would take O(2n) time complexity to generate all combinations. However, I'm not sure if an improved algorithm can bring this time complexity down. If an improved algorithm exists, please do share your knowledge!
In the case that it takes O(2n) which is exponential, I would like some insight regarding which class this algorithm belongs to P, NP, NP-Complete, or NP-Hard. Thanks in advance :)
P, NP, NP-complete, and NP-hard are all classes of decision problems, none of which contain problems that involve non-binary output (such as this enumeration problem).
Often people refer colloquially to problems in FNP as being in NP. This problem is not in FNP either because the length of the output string for the relation must be bounded by some polynomial function of the input length. It might be FNP-hard, but we're getting into the weeds that even a graduate CS education doesn't cover. Worth asking on the CS Stack Exchange if you care enough.
This problem is in none of them except, arguably, NP-hard.
It is not in P because there is no polynomial time algorithm to do it. You cannot generate an exponential number of things in polynomial time.
It is not in NP because there is no polynomial time algorithm to validate the answer. You cannot process an exponential number of things in polynomial time.
It is not in NP-complete because everything in NP-complete must be in NP and it is not.
The argument for it being in NP-hard goes like this. You can say anything that you want about the members of the empty set. Including that they make monkeys fly out of your nose and can solve any problem in NP in polynomial time. So if we could find a polynomial solution, we can solve any NP problem fast, and therefore it meets the definition of NP-hard. But uselessly so - we know that no polynomial solution exists.
I've found here a statement that Algorithm X for sudoku has O(N^3) time complexity where N is a board size.
That's maybe logical, since for sudoku the binary matrix to compute has N^3 rows. But that makes sudoku problem solvable in a polynomial time, and sudoku is known to be NP problem, that means (as I understand it)
not possible to always solve in a polynomial time
possible to verify a solution in a polynomial time
So what is the time complexity of Algorithm X for sudoku,
and is it possible to solve a sudoku in a polynomial time or not ?
Thank you!
Mathematics of Sudoku explains this pretty well:
The general problem of solving Sudoku puzzles on n^2×n^2 grids of n×n
blocks is known to be NP-complete.
The runtime complexity of any algorithm solving Sudoku is thus at least exponential in n. For a normal Sudoku (n = 3) this means O(N^3) is perfectly reasonable.
For a complete analysis of running time see: https://11011110.github.io/blog/2008/01/10/analyzing-algorithm-x.html
There it is stated that
even the stupidest version of Algorithm X, one that avoids any chance to backtrack and always chooses its pivots in such a way as to maximize its running time, takes at most O(3n/3)
which places the algorithm in exponential running time (EXPTIME).
Is an approximation algorithm the same as a Polynomial Time Approximation Algorithm (PTAS)? E.g. It can be shown that A(I) <= 2 * OPT(I) for vertex cover. Does it mean that Vertex Cover has a 2-polynomial time approximation algorithm or a PTAS?
Thanks!
Note: The text in Italics is the edit I made after I posted my question.
No, this isn't necessarily the case. A PTAS is an algorithm where given any ε > 0, you can approximate the answer to a factor of (1 + ε) in polynomial time. In other words, you can get arbitrarily good approximations.
Some problems are known (for example, MAX-3SAT) that have approximation algorithms for specific factors (for example, 5/8), but where it's known that unless P = NP there is a hard limit to how well the problem can be approximated in polynomial time. For example, the PCP theorem says that MAX-3SAT doesn't have a polynomial-time 7/8 approximation unless P = NP. It's therefore possible that MAX-3SAT has a PTAS, but only if P = NP.
Hope this helps!
Vertex cover having 2-approximation algorithm is not same as having a PTAS algorithm. Sometimes, there are problems where much better approximation is possible. These problems then admit PTAS.
Such algorithms take an instance of the problem as input, with another input parameter epsilon>0. And it gives an output whose value is at most (1+epsilon).OPT for minimisation problem; and (1/(1+epsilon)).OPT for maximisation problem.
Run time of a PTAS algorithm is polynomial in n (size of problem instance). Sometimes, the runtime is also polynomial in epsilon, then its called to admit FPTAS(fully PTAS).
Example:
Dynamic programming algorithm for KNAPSACK with integer-profits gives optimal solution.
While, KNAPSACK problem with real-valued profits do not admit polynomial-time algorithm. But it admits a FPTAS, where real-value profits are converted into integer profits; and DP algorithm is used to calculate the solution with "rounded" profits.
Another example, Max Independent Set does not admit a PTAS or FPTAS. Because, in this case, we can set a value for epsilon, which will always give optimal solution for any graph using that PTAS algorithm; which is not possible until P=NP.
Some problems that are NP-hard are also fixed-parameter tractable, or FPT. Wikipedia describes a problem as fixed-parameter tractable if there's an algorithm that solves it in time f(k) · |x|O(1).
What does this mean? Why is this concept useful?
To begin with, under the assumption that P ≠ NP, there are no polynomial-time, exact algorithms for any NP-hard problem. Although we don't know whether P = NP or P ≠ NP, we don't have any polynomial-time algorithms for any NP-hard problems.
The idea behind fixed-parameter tractability is to take an NP-hard problem, which we don't know any polynomial-time algorithms for, and to try to separate out the complexity into two pieces - some piece that depends purely on the size of the input, and some piece that depends on some "parameter" to the problem.
As an example, consider the 0/1 knapsack problem. In this problem, you're given a list of n objects that have associated weights and values, along with some maximum weight W that you're allowed to carry. The question is to determine the maximum amount of value that you can carry. This problem is NP-hard, meaning that there's no polynomial-time algorithm that solves it. A brute-force method will take time around O(2n) by considering all possible subsets of the items, which is extremely slow for large n. However, it is possible to solve this problem in time O(nW), where n is the number of elements and W is the amount of weight you can carry. If you look at the runtime O(nW), you'll notice that it's split into two parts: a component that's linear in the number of elements (the n part) and a component that's linear in the weight (the W part). If W is any fixed constant, then the runtime of this algorithm will be O(n), which is linear-time, even though the problem in general is NP-hard. This means that if we treat W as some tunable "parameter" of the problem, for any fixed value of this parameter, the problem ends up running in polynomial time (which is "tractable," in the complexity theory sense of the word.)
As another example, consider the problem of finding long, simple paths in a graph. This problem is also NP-hard, and the naive algorithm for finding simple paths of length k in a graph takes time O(n! / (n - k)!), which for large k ends up being superexponential. However, using the technique of color-coding, it's possible to solve this problem in time O((2e)kn3 log n), where k is the length of the path to find and n is the number of nodes in the input graph. Notice that this runtime also has two "components:" one component that's a polynomial in the number of nodes in the input graph (the n3 log n part) and one component that's exponential in k (the (2e)k part). This means that for any fixed value of k, there's a polynomial-time algorithm for finding length-k paths in the graph; the runtime will be O(n3 log n).
In both of these cases, we can take a problem for which we have an exponential-time solution (or worse) and find a new solution whose runtime is some polynomial in n times some crazy-looking function of some extra "parameter." In the case of the knapsack problem, that parameter is the maximum amount of weight we can carry; in the case of finding long paths, the parameter is the length of the path to find. Generally speaking, a problem is called fixed-parameter tractable if there is some algorithm for solving the problem defined in terms of two quantities: n, the size of the input, and k, some "parameter," where the runtime is
O(p(n) · f(k))
Where p(n) is some polynomial function and f(k) is an arbitrary function in k. Intuitively, this means that the complexity of the problem scales polynomially with n (meaning that as only the problem size increases, the runtime will scale nicely), but can scale arbitrarily badly with the parameter k. This separates out the "inherent hardness" of the problem such that the "hard part" of the problem is blamed on the parameter k, while the "easy part" of the problem is charged to the size of the input.
Once you have a runtime that looks like O(p(n) · f(k)), we immediately get polynomial-time algorithms for solving the problem for any fixed k. Specifically, if k is fixed, then f(k) is some constant, so O(p(n) · f(k)) is just O(p(n)). This is a polynomial-time algorithm. Therefore, if we "fix" the parameter, we get back some "tractable" algorithm for solving the problem. This is the origin of the term fixed-parameter tractable.
(A note: Wikipedia's definition of fixed-parameter tractability says that the algorithm should have runtime f(k) · |x|O(1). Here, |x| refers to the size of the input, which I've called n here. This means that Wikipedia's definition is the same as saying that the runtime is f(k) · nO(1). As mentioned in this earlier answer, nO(1) means "some polynomial in n," and so this definition ends up being equivalent to the one I've given here).
Fixed-parameter tractability has enormous practical implications for a problem. It's common to encounter problems that are NP-hard. If you find a problem that's fixed-parameter tractable and the parameter is low, it can be significantly more efficient to use the fixed-parameter tractable algorithm than to use the normal brute-force algorithm. The color-coding example above for finding long paths in a graph, for example, has been used to great success in computational biology to find sequencing pathways in yeast cells, and the 0/1 knapsack solution is used frequently because common values of W are low enough for it to be practical.
Hope this helps!
I believe that the explanation of #templatetypedef was already quite comprehensive of the generality of FPT.
I would like to add that in practice, it appears quite often that the class of problem one is trying to solve is FPT, such as above examples.
In the case of problems expressed as set of constraints (e.g. SAT, CSP, ILP, etc.) a very common parameter is treewidth, which basically explicits how much your problem is organized as a tree.
This allows to split ones problem into a tree of subproblems which can then be solved more individually using dynamic programming.
In such case, many problems are linear-time fixed-parameter tractable, that is the complexity grows linearly with the number of components (i.e. the size of the system) by exponentially in the size of its biggest component.
Although the use of explicit techniques is possible to solve sub-problems is possible, in order to scale-up to more reasonnable instances, using symbolic representations is recomended.
Sorry for stupid question. I cannot jog my memory and googling did not help me answer this question.
So basically given a graph G(V,E), I know that O(|V|^2) or O(|E|^2 + |V|^2) is considered to be polynomial complexity, so is O(|E|*|V|) polynomial as well? If not, what kind of complexity is it? I believe it's not pseudo-polynomial either.
Another question is: is O(m*n) considered polynomial as well, given m and n are the sizes of two INDEPENDENT inputs to a problem? I just want to clarify the concept of polynomial time in here and want to know if O(m*n) has a different name for its type of complexity.
it is polynomial O(|V|^3) since the number of edges is bounded O(|V|^2)