When learning for exam in Algorithms and Data Structures i have stumbled upon a question, what does it mean if an algorithm has pseudo polynomial time efficiency( analysis)
Did a lot of searching but turned empty handed
It means that the algorithm is polynomial with respect to the size of the input, but the input actually grows exponentially.
For example take the subset sum problem. We have a set S of n integers and we want to find a subset which sums up to t.
To solve this problem you can just check the sum of each subset, so it is O(P) where P is the number of subsets. However in fact the number of subsets is 2^n so the algorithm has exponential complexity.
I hope this introduction helps to understand the wikipedia's article about it http://en.wikipedia.org/wiki/Pseudo-polynomial_time :)
Related
Once I read in a scientific paper:
The computational complexity of the algorithm is O(N^d), where N
is the number of data, d is the dimension. Hence with fixed
dimension, the algorithm complexity is polynomial.
Now, this made me think, that (if I'm not mistaken), big-O notation is defined in the number of binary inputs. Thus if I fix the dimension of data, it is natural to arrive to polynomial solution. Moreover, if I would also fix N, the number of input, I would arrive to an O(1) solution, see the connected post:
Algorithm complexity with input is fix-sized
My question is, if you think that this is a valid argument for polynomial complexity? Can one really fix one dimension and the input data and claim polynomial complexity?
Yes, that's a reasonable thing to do.
It really depends on the initial problem, but in most cases I would say fixing number of dimensions is reasonable. I would expect the paper to claim something like "polynomial complexity for practical purposes" or something like that or have some arguments presented why limiting d is reasonable.
You can compare with a solution with complexity O(d^N) where fixing the number of dimensions doesn't mean that the solution is polynomial. So the one presented is clearly better when d is small.
As a quick recall from university time.
Big-O notation is just a UPPER bound of how your algorithm perform.
Mathematically, f(x) is O(g(x)) means that there exists a constant k>0 and x0 such that
f(x) <= kg(x) for all x>x0
To answer your question, you cannot fix the N, which is the independent variable.
If you fix N, says <100, we can surely arrive O(1),
because according to the definition. We can set a large K to ensure f(N) <= kG(N) for all x (<100)
This only works for some algorithms. It is not clear to me, what the "dimension" should be in some cases.
E.g. SubSetSum is is NP-complete, therefor there is no algorithm known with polynomial complexity. But the input is just N numbers. You could also see it as N numbers of bit length d. but the algorithm still has a polynomial complexity.
Same holds for Shortest Vector Problem (SVP) for lattices. The input is a N x N Basis (lets say with integer entries) and you look for the shortest non zero vector. This is also a hard problem and no algorithm with polynomial complexity is known yet.
For many problems its not just the size of the input data that makes the problem difficult, but certain properties or parameters of that data. E.g. many graph problems have complexity given in the number of nodes and edges separately.
Sometimes, the difference between this parameters might be dramatic, for example if you have something like O(n^d) the complexity is just polynomial when n grows, but exponential when d grows.
If you now happen to have an application, where you know that the value of a parameter like the dimension is always the same or there is a (small) maximal value, then regarding this parameter as fixed can give you useful inside. So statements like these are very common in scientific papers.
However, you can not just fix any parameter, e.g. your memory is finite, therefore sorting of data is constant time, because the bound on that parameter is so large that viewing it as fixed does not give you any useful insight.
So fixing all parameters is usually not an option because there has to be one aspect in which the size of your data varies. It can be an option if your complexity is very slow growing.
E.g. data structures with O(log n) operations are sometimes considered to have effectively constant complexity if the constant is also quite small. Or data structures as union-find-structures where amortized complexity of the operations is O(α(n)) where α is the inverse of the Ackermann-function, a function growing so slowly that it is impossible to get above 10 or for any size n any hardware imaginable could possible ever handle.
I would like to quote from Wikipedia
In mathematics, the minimum k-cut, is a combinatorial optimization
problem that requires finding a set of edges whose removal would
partition the graph to k connected components.
It is said to be the minimum cut if the set of edges is minimal.
For a k = 2, It would mean Finding the set of edges whose removal would Disconnect the graph into 2 connected components.
However, The same article of Wikipedia says that:
For a fixed k, the problem is polynomial time solvable in O(|V|^(k^2))
My question is Does this mean that minimum 2-cut is a problem that belongs to complexity class P?
The min-cut problem is solvable in polynomial time and thus yes it is true that it belongs to complexity class P. Another article related to this particular problem is the Max-flow min-cut theorem.
First of all, the time complexity an algorithm should be evaluated by expressing the number of steps the algorithm requires to finish as a function of the length of the input (see Time complexity). More or less formally, if you vary the length of the input, how would the number of steps required by the algorithm to finish vary?
Second of all, the time complexity of an algorithm is not exactly the same thing as to what complexity class does the problem the algorithm solves belong to. For one problem there can be multiple algorithms to solve it. The primality test problem (i.e. testing if a number is a prime or not) is in P, but some (most) of the algorithms used in practice are actually not polynomial.
Third of all, in the case of most algorithms you'll find on the Internet evaluating the time complexity is not done by definition (i.e. not as a function of the length of the input, at least not expressed directly as such). Lets take the good old naive primality test algorithm (the one in which you take n as input and you check for division by 2,3...n-1). How many steps does this algo take? One way to put it is O(n) steps. This is correct. So is this algorithm polynomial? Well, it is linear in n, so it is polynomial in n. But, if you take a look at what time complexity means, the algorithm is actually exponential. First, what is the length of the input to your problem? Well, if you provide the input n as an array of bits (the usual in practice) then the length of the input is, roughly said, L = log n. Your algorithm thus takes O(n)=O(2^log n)=O(2^L) steps, so exponential in L. So the naive primality test is in the same time linear in n, but exponential in the length of the input L. Both correct. Btw, the AKS primality test algorithm is polynomial in the size of input (thus, the primality test problem is in P).
Fourth of all, what is P in the first place? Well, it is a class of problems that contains all decision problems that can be solved in polynomial time. What is a decision problem? A problem that can be answered with yes or no. Check these two Wikipedia pages for more details: P (complexity) and decision problems.
Coming back to your question, the answer is no (but pretty close to yes :p). The minimum 2-cut problem is in P if formulated as a decision problem (your formulation requires an answer that is not just a yes-or-no). In the same time the algorithm that solves the problem in O(|V|^4) steps is a polynomial algorithm in the size of the input. Why? Well, the input to the problem is the graph (i.e. vertices, edges and weights), to keep it simple lets assume we use an adjacency/weights matrix (i.e. the length of the input is at least quadratic in |V|). So solving the problem in O(|V|^4) steps means polynomial in the size of the input. The algorithm that accomplishes this is a proof that the minimum 2-cut problem (if formulated as decision problem) is in P.
A class related to P is FP and your problem (as you formulated it) belongs to this class.
Some problems that are NP-hard are also fixed-parameter tractable, or FPT. Wikipedia describes a problem as fixed-parameter tractable if there's an algorithm that solves it in time f(k) · |x|O(1).
What does this mean? Why is this concept useful?
To begin with, under the assumption that P ≠ NP, there are no polynomial-time, exact algorithms for any NP-hard problem. Although we don't know whether P = NP or P ≠ NP, we don't have any polynomial-time algorithms for any NP-hard problems.
The idea behind fixed-parameter tractability is to take an NP-hard problem, which we don't know any polynomial-time algorithms for, and to try to separate out the complexity into two pieces - some piece that depends purely on the size of the input, and some piece that depends on some "parameter" to the problem.
As an example, consider the 0/1 knapsack problem. In this problem, you're given a list of n objects that have associated weights and values, along with some maximum weight W that you're allowed to carry. The question is to determine the maximum amount of value that you can carry. This problem is NP-hard, meaning that there's no polynomial-time algorithm that solves it. A brute-force method will take time around O(2n) by considering all possible subsets of the items, which is extremely slow for large n. However, it is possible to solve this problem in time O(nW), where n is the number of elements and W is the amount of weight you can carry. If you look at the runtime O(nW), you'll notice that it's split into two parts: a component that's linear in the number of elements (the n part) and a component that's linear in the weight (the W part). If W is any fixed constant, then the runtime of this algorithm will be O(n), which is linear-time, even though the problem in general is NP-hard. This means that if we treat W as some tunable "parameter" of the problem, for any fixed value of this parameter, the problem ends up running in polynomial time (which is "tractable," in the complexity theory sense of the word.)
As another example, consider the problem of finding long, simple paths in a graph. This problem is also NP-hard, and the naive algorithm for finding simple paths of length k in a graph takes time O(n! / (n - k)!), which for large k ends up being superexponential. However, using the technique of color-coding, it's possible to solve this problem in time O((2e)kn3 log n), where k is the length of the path to find and n is the number of nodes in the input graph. Notice that this runtime also has two "components:" one component that's a polynomial in the number of nodes in the input graph (the n3 log n part) and one component that's exponential in k (the (2e)k part). This means that for any fixed value of k, there's a polynomial-time algorithm for finding length-k paths in the graph; the runtime will be O(n3 log n).
In both of these cases, we can take a problem for which we have an exponential-time solution (or worse) and find a new solution whose runtime is some polynomial in n times some crazy-looking function of some extra "parameter." In the case of the knapsack problem, that parameter is the maximum amount of weight we can carry; in the case of finding long paths, the parameter is the length of the path to find. Generally speaking, a problem is called fixed-parameter tractable if there is some algorithm for solving the problem defined in terms of two quantities: n, the size of the input, and k, some "parameter," where the runtime is
O(p(n) · f(k))
Where p(n) is some polynomial function and f(k) is an arbitrary function in k. Intuitively, this means that the complexity of the problem scales polynomially with n (meaning that as only the problem size increases, the runtime will scale nicely), but can scale arbitrarily badly with the parameter k. This separates out the "inherent hardness" of the problem such that the "hard part" of the problem is blamed on the parameter k, while the "easy part" of the problem is charged to the size of the input.
Once you have a runtime that looks like O(p(n) · f(k)), we immediately get polynomial-time algorithms for solving the problem for any fixed k. Specifically, if k is fixed, then f(k) is some constant, so O(p(n) · f(k)) is just O(p(n)). This is a polynomial-time algorithm. Therefore, if we "fix" the parameter, we get back some "tractable" algorithm for solving the problem. This is the origin of the term fixed-parameter tractable.
(A note: Wikipedia's definition of fixed-parameter tractability says that the algorithm should have runtime f(k) · |x|O(1). Here, |x| refers to the size of the input, which I've called n here. This means that Wikipedia's definition is the same as saying that the runtime is f(k) · nO(1). As mentioned in this earlier answer, nO(1) means "some polynomial in n," and so this definition ends up being equivalent to the one I've given here).
Fixed-parameter tractability has enormous practical implications for a problem. It's common to encounter problems that are NP-hard. If you find a problem that's fixed-parameter tractable and the parameter is low, it can be significantly more efficient to use the fixed-parameter tractable algorithm than to use the normal brute-force algorithm. The color-coding example above for finding long paths in a graph, for example, has been used to great success in computational biology to find sequencing pathways in yeast cells, and the 0/1 knapsack solution is used frequently because common values of W are low enough for it to be practical.
Hope this helps!
I believe that the explanation of #templatetypedef was already quite comprehensive of the generality of FPT.
I would like to add that in practice, it appears quite often that the class of problem one is trying to solve is FPT, such as above examples.
In the case of problems expressed as set of constraints (e.g. SAT, CSP, ILP, etc.) a very common parameter is treewidth, which basically explicits how much your problem is organized as a tree.
This allows to split ones problem into a tree of subproblems which can then be solved more individually using dynamic programming.
In such case, many problems are linear-time fixed-parameter tractable, that is the complexity grows linearly with the number of components (i.e. the size of the system) by exponentially in the size of its biggest component.
Although the use of explicit techniques is possible to solve sub-problems is possible, in order to scale-up to more reasonnable instances, using symbolic representations is recomended.
I am attempting to prepare a presentation to explain the basics of algorithm analysis to my co-workers - some of them have never had a lecture on the subject before, but everyone has at least a few years programming behind them and good math backgrounds, so I think I can teach this. I can explain the concepts fine, but I need concrete examples of some code structures or patterns that result in factors so I can demonstrate them.
Geometric factors (n, n^2, n^3, etc) are easy, embedded loops using the same sentinel, but I am getting lost on how to describe and show off some of the less common ones.
I would like to incorporate exponential (2^n or c^n), logarithmic (n log(n) or just log(n)) and factoral (n!) factors in the presentation. What are some short, teachable ways to get these in code?
A divide-and-conquer algorithm that does a constant amount of work for each time it divides the problem in half is O(log n). For example a binary search.
A divide-and-conquer algorithm that does a linear amount of work for each time it divides the problem in half is O(n * log n). For example a merge sort.
Exponential and factorial are probably best illustrated by iterating respectively over all subsets of a set, or all permutations of a set.
Exponential: naive Fibonacci implementation.
n log(n) or just log(n): Sorting and Binary seach
Factorial: Naive traveling salesman solutions. Many naive solutions to NP-complete problems.
n! problems are pretty simple. There are many NP-complete n! time problems such as the travelling salesman problem
In doubt pick one of the Sort algorithms - everyone knows what they're supposed to do and therefore they're easy to explain in relation to the complexity stuff: Wikipedia has a quite good overview
I read this article, it suggests (page 1025 last paragraph) that there is a polynomial time algorithm to find the optimum of a k-tsp problem using binary search.
Using binary search would suggest there exists an algorithm for checking if a solution exists with cost<X and this algorithm is used for the binary search.
I 'googled' around for this and the only algorithm i could find was a non deterministic one (which is pretty trivial), but obviously i'm looking for a deterministic one.
I am interested in this for learning purposes,
Any help/links would be appreciated.
EDIT
I am referring to finding the value of the optimal solution and not about finding the solution itself.
Since TSP is a special case of k-TSP where k = number of nodes in the graph. If you had a solution for "what's the cheapest k-TSP route" in polynomial in relation to graph size, then you'd have a polynomial solution to decision problem version of TSP which would imply that P = NP.
So the answer is no. Deterministic polynomial algorithm for both decision problem and optimization version (they're essentially the same) of k-TSP doesn't exist (yet).
The paper you mentioned proposes a polynomial-time approximation algorithm for the directed k-TSP problem.
Approximation algorithms are those which are guaranteed to yield solutions with a limited deviation from the optimal solution value. There are examples of polynomial-time approximation algorithms for NP-Hard problems: the Christofides Algorithm yields, in time O(n³), solutions to the metric TSP problem whose values are at most 3/2 the value of the optimal solution.
David Karger, in a lecture (link)
mentions a randomized algorithm for efficient k-TSP problem which runs in polynomial time in n (but exponential in k). It is based on the idea of color coding: color each of the node with a random color in [1..k], and find a shortest chromatic path (where each color appears exactly once). With a simple dynamic programming algorithm, this approach gives a runtime of O(n^2 2^k) and it succeeds (in finding the path with minimal cost) with probability e^{-k}. By repeating e^k times, one achieves an algorithm that finds the minimum k-TSP with high probability.