UPDATED
After more reading, the solution can be given with the following recurrence relation:
(a) When i = 1 and j = 2, l(i; j) = dist(pi; pj )
(b) When i < j - 1; l(i; j) = l(i; j - 1) + dist(pj-1; pj)
(c) When i = j - 1 and j > 2, min 1<=k<i (l(k; i) + dist(pk; pj ))
This is now starting to make sense, except for part C. How would I go about determining the minimum value k? I suppose it means you can iterate through all possible k values and just store the minimum result of ( l(k,i) + dist(pk,pj)?
Yes, definitely a problem I was studying at school. We are studying bitonic tours for the traveling salesman problem.
Anyway, say I have 5 vertices {0,1,2,3,4}. I know my first step is to sort these in order of increasing x-coordinates. From there, I am a bit confused on how this would be done with dynamic programming.
I am reading that I should scan the list of sorted nodes, and maintain optimal paths for both parts (initial path and the return path). I am confused as to how I will calculate these optimal paths. For instance, how will I know if I should include a given node in the initial path or the return path, since it cannot be in both (except for the endpoints). Thinking back to Fibonacci in dynamic programming, you basically start with your base case and work your way forward. I guess what I am asking is how would I get started with the bitonic traveling salesman problem?
For something like the Fibonacci numbers, a dynamic programming approached is quite clear. However, I don't know if I am just being dense or what but I am quite confused trying to wrap my head around this problem.
Thanks for looking!
NOTE: I am not looking for complete solutions, but at least some good tips to get my started. For example, if this were the Fibonacci problem, one could illustrate how the first few numbers are calculated. Please let me know how I can improve the question as well.
Clarification on your algorithm.
The l(i,j) recursive function should compute the minimum distance of a bitonic tour i -> 1 -> j visiting all nodes that are smaller than i. So, the solution to the initial problem will be l(n,n)!
Important notes:
we can assume that the nodes are ordered by their x coordinate and labeled accordingly (p1.x < p2.x < p3.x ... < pn.x). It they weren't ordered, we could sort them in O(nlogn) time.
l(i,j) = l(j,i). The reason is that in the lhs, we have a i ->...-> 1 -> ... -> j tour which is optimal. However traversing this route backward will give us the same distance, and won't broke bitonic property.
Now the easy cases (note the changes!):
(a) When i = 1 and j = 2, l(i; j) = dist(pi; pj ) = dist(1,2)
Here we have the following tour : 1->1->...->2. Trivially this is equivalent to the length of the path 1->...->2. Since points are ordered by their .x coordinate, there is no point between 1 and 2, so the straight line connecting them will be the optimal one. ( Choosing any number of other points to visit before 2 would result in a longer path! )
(b) When i < j - 1; l(i; j) = l(i; j - 1) + dist(pj-1; pj)
In this case, j-1 must be on the part of the path 1 -> ... -> j, because the part i -> ... -> 1 can not contain nodes with an index bigger than i. Because all nodes in the path 1 -> ... -> j are in increasing order of index, there can be none between j-1 and j. So, this is equivalent to the tour: i -> ... -> 1 -> .... -> j-1 -> j, which is equivalent to l(i,j-1) + dist(pj-1,pj)!
Anf finally the interesting part comes:
(c) When i = j - 1 or i = j, min 1<=k<i (l(k; i) + dist(pk; pj ))
Here we know that we have to get from i to 1, but there is no clue on the backward sweep! The key idea here is that we must think of the node just before j on our backward route. It may be any of the nodes from 1 to j-1! Let us assume that this node is k.
Now we have a tour: i -> ... -> 1 -> .... -> k -> j, right? The cost of this tour is l(i,k) + dist(pk,pj).
Hope you got it.
Implementation.
You will need a 2-dimensional array say BT[1..n][1..n]. Let i be the row index, j be the column index. How should we fill in this table?
In the first row we know BT[1][1] = 0, BT[1][2] = d(1,2), so we have only i,j indexes left that fall into the (b) category.
In the remainin rows, we fill the elements from the diagonal till the end.
Here is a sample C++ code (not tested):
void ComputeBitonicTSPCost( const std::vector< std::vector<int> >& dist, int* opt ) {
int n = dist.size();
std::vector< std::vector< int > > BT;
BT.resize(n);
for ( int i = 0; i < n; ++i )
BT.at(i).resize(n);
BT.at(0).at(0) = 0; // p1 to p1 bitonic distance is 0
BT.at(0).at(1) = dist.at(0).at(1); // p1 to p2 bitonic distance is d(2,1)
// fill the first row
for ( int j = 2; j < n; ++j )
BT.at(0).at(j) = BT.at(0).at(j-1) + dist.at(j-1).at(j);
// fill the remaining rows
int temp, min;
for ( int i = 1; i < n; ++i ) {
for ( int j = i; j < n; ++j ) {
BT.at(i).at(j) = -1;
min = std::numeric_limits<int>::max();
if ( i == j || i == j -1 ) {
for( int k = 0; k < i; ++k ) {
temp = BT.at(k).at(i) + dist.at(k).at(j);
min = ( temp < min ) ? temp : min;
}
BT.at(i).at(j) = min;
} else {
BT.at(i).at(j) = BT.at(i).at(j-1) + dist.at(j-1).at(j);
}
}
}
*opt = BT.at(n-1).at(n-1);
}
Okay, the key notions in a dynamic programming solution are:
you pre-compute smaller problems
you have a rule to let you combine smaller problems to find solutions for bigger problems
you have a known property of the problems that let's you prove the solution is really optimal under some measure of optimality. (In this case, shortest.)
The essential property of a bitonic tour is that a vertical line in the coordinate system crosses a side of the closed polygon at most twice. So, what is a bitonic tour of exactly two points? Clearly, any two points form a (degenerate) bitonic tour. Three points have two bitonic tours ("clockwise" and "counterclockwise").
Now, how can you pre-compute the various smaller bitonic tours and combine them until you have all points included and still have a bitonic tour?
Okay, you're on the righ track with your update. But now, in a dynamic programming solution, what you do with work it bottom-up: pre-compute and memoize (not "memorize") the optimal subproblems.
Related
Context
I am currently working on path reconstruction in some of my graph algorithms. For the single-source-shortest-paths problem I used an array of predecessors to reconstruct the shortest path from one source node to all the other nodes.
Quick example:
[0, 3, 0, 0]
The shortest path from source 0 to target 1 would be [0, 3, 1] because starting from the target node 1 the path can be constructed by going backwards using the 'parent' array. 1 has been reached over 3 and 3 has been reached over 0. 0 is the source. Done.
The next algorithms are all-pairs-shortest-paths algorithms. The easiest example has been the Floyd-Warshall algorithm which results in a matrix containing all 'successor'-nodes. A good example for the reconstruction pseudo code can be found on Wikipedia - Floyd Warshall.
To summarize it: A matrix is used to store each successor from one specific source node. It basically follows the same approach as before just for each node as a source and going forward instead of backwards.
Question - How to create the matrix of successors in case of the pseudo multiply algorithm?
Let's have a look at the algorithm first:
for(int m = 0; m < nodeCount - 1; m++) {
Matrix nextResultMatrix = new Matrix(nodeCount, nodeCount, Integer.MAX_VALUE);
for(int i = 0; i < nodeCount; i++) {
for(int j = 0; j < nodeCount; j++) {
int value = Integer.MAX_VALUE;
for(int k = 0; k < nodeCount; k++) {
value = Math.min(
value,
resultMatrix.at(i, k) + sourceMatrix.at(k, j)
);
}
nextResultMatrix.setAt(i, j, value);
}
}
resultMatrix = nextResultMatrix;
}
In each iteration the matrix for the shortest paths of length m will be calculated. The inner loop is pretty similar to the matrix multiplication itself. In the most inner loop the algorithm checks wether the current path is shorter than the path from source i over k to target j. After the inner k-loop has finished the value inside of the new result matrix is set. Which leads to the problem:
In case of the Floyd-Warshall algorithm it was easier to identify wether the path has been shorter and which node is now the successor. In this case the value that has been calculated in the k-loop will be set anyway. Is it possible to determine the successor here?
Thoughts about a possible solution
The pseudo multiply algorithm provides a matrix for each iteration which represents the shortest paths of length m. Might this help to find a solution without increasing the - already quite bad - time complexity and without having to store each matrix at the same time.
I found an interesting idea in a comment here on stackoverflow which might lead to a solution reference. From what is stated there it seems to be quite heavy lifting for keeping track of the shortest paths. I haven't fully wrapped my head around the idea and how to implement this though.
My solution
So after stepping through the algorithm and making clear what each step exactly means I was finally able to figure out a solution. I will try to explain the changes in the code here but first let me present the solution:
for(int m = 0; m < nodeCount - 1; m++) {
Matrix nextResultMatrix = new Matrix(nodeCount, nodeCount, Integer.MAX_VALUE);
for(int i = 0; i < nodeCount; i++) {
for(int j = 0; j < nodeCount; j++) {
int value = resultMatrix.at(i, j);
int shorterPathFoundOverNode = prevMatrix.at(i, j);
// new shortest path from node i to j is
// the minimum path that can be found from node i over k to j
// k will be the node itself and every other node
for(int k = 0; k < nodeCount; k++) {
if(resultMatrix.at(i, k) != Graph.NO_EDGE && sourceMatrix.at(k, j) != Graph.NO_EDGE) {
if(value > resultMatrix.at(i, k) + sourceMatrix.at(k, j)) {
// update value if path is shorter
value = resultMatrix.at(i, k) + sourceMatrix.at(k, j);
shorterPathFoundOverNode = k;
}
}
}
nextResultMatrix.setAt(i, j, value);
prevMatrix.setAt(i, j, shorterPathFoundOverNode);
}
}
resultMatrix = nextResultMatrix;
}
A very basic but important idea was to replace the initialization value of value inside the j loop from Integer.MAX with the value that has previously been found or at first iteration the value that has been used to initialize the matrix (Integer.MAX). This was also important because the condition would have been true once per iteration which did not cause any problems before but now - since we perform more actions inside of the condition - it matters.
It was necessary to replace the Math.min method with an if condition to have the ability to do more than just setting the value.
To reconstruct the shortest paths the method of keeping track of the previous nodes is used. This is very similar to the single-source-shortest-paths problem as stated in the question. Of course it is required to use a matrix in this case because all nodes will be the source node.
To summarize the idea: Setup an additional matrix that keeps track of the previous node for each target node. When iterating through the k loop save the previous node if a shorter path has been found (Important: Only if it is actually shorter than the previous path).
I have been trying to solve this problem my professor has given me but couldn't make a proper solution. The following is the problem
Problem:
A rectangular circuit board has two parallel sides with width W between them. There are m terminals on the upper side of the board and n terminals (n < m) on the lower side. Let U1 < U[2] < … < U[m] be the distances from the left end of the board to the m terminals on the upper side, respectively. Let L1 < L[2] < … < L[n] be the distances from the left end of the board to the n terminals on the lower side, respectively. Now, we need to select n terminals from the m terminals on the upper side to be connected to the n terminals on the lower side by n straight line segments, respectively, such that the total length of the n line segments is minimized. The following figure illustrates the problem for m = 8 and n = 4.
(a) Prove that, in an optimal solution, any two line segments will not intersect.
(b) Design an O(mn) dynamic programming algorithm to solve this minimization problem. You need to define sub-problems, show the inductive formula, initial conditions, and a pseudocode. You can use d(i, j) to denote the distance between U[i] and L[j], 1 ≤ i ≤ m, 1 ≤ j ≤ n. (The calculation of d(i, j) = ) can be omitted.
My Approach:
For the above problem, my approach was first to make a matrix d(i,j) where i are the terminals on the bottom and j are the terminals on the top. d(i,j) has all the distances from any two circuits.Then iterating through each row I will find the smallest distance and mark the respective terminal. But I am not sure this would work if the top circuits are all to the extreme right of the side. So can anyone provide me with a better approach.
I have written a recursive Dynamic Programming solution that uses memoisation, the complexity is O(mn), here at each recursive level we can either choose to join the current point defined in the U[] array with the point defined in the L[] array, or we can move forward without doing so:
#include<iostream>
#define INF 1e9
using namespace std;
int n, m, d[100][100], dp[100][100];
int solve(int idx1, int idx2){
if(idx1 > m){
if(idx2 < n) return INF;
else return 0;
}
if(idx2 > n) return 0;
if(dp[idx1][idx2] != -1) return dp[idx1][idx2];
int v1, v2;
//include current
v1 = solve(idx1 + 1, idx2 + 1) + d[idx1][idx2];
//do not include current
v2 = solve(idx1 + 1, idx2);
return dp[idx1][idx2] = min(v1, v2);
}
int main(){
//enter the the distances
for(int i = 0;i < 100;i++) for(int j = 0;j < 100;j++) dp[i][j] = -1;
cout << solve(1, 1) << endl;
return 0;
}
For the part (a) of your question, let us assume that 2 line segments do intersect, then we cannot have an optimal solution because if we just swapped the 2 end points of the line segments defined by the L[] array then the distance would reduce, hence giving us a better solution.
I recently encountered this question in an interview. I couldn't really come up with an algorithm for this.
Given an array of unsorted integers, we have to find the minimum cost in which this array can be converted to an Arithmetic Progression where a cost of 1 unit is incurred if any element is changed in the array. Also, the value of the element ranges between (-inf,inf).
I sort of realised that DP can be used here, but I couldn't solve the equation. There were some constraints on the values, but I don't remember them. I am just looking for high level pseudo code.
EDIT
Here's a correct solution, unfortunately, while simple to understand it's not very efficient at O(n^3).
function costAP(arr) {
if(arr.length < 3) { return 0; }
var minCost = arr.length;
for(var i = 0; i < arr.length - 1; i++) {
for(var j = i + 1; j < arr.length; j++) {
var delta = (arr[j] - arr[i]) / (j - i);
var cost = 0;
for(var k = 0; k < arr.length; k++) {
if(k == i) { continue; }
if((arr[k] + delta * (i - k)) != arr[i]) { cost++; }
}
if(cost < minCost) { minCost = cost; }
}
}
return minCost;
}
Find the relative delta between every distinct pair of indices in the array
Use the relative delta to test the cost of transforming the whole array to AP using that delta
Return the minimum cost
Louis Ricci had the right basic idea of looking for the largest existing arithmetic progression, but assumed that it would have to appear in a single run, when in fact the elements of this progression can appear in any subset of the positions, e.g.:
1 42 3 69 5 1111 2222 8
requires just 4 changes:
42 69 1111 2222
1 3 5 8
To calculate this, notice that every AP has a rightmost element. We can suppose each element i of the input vector to be the rightmost AP position in turn, and for each such i consider all positions j to the left of i, determining the step size implied for each (i, j) combination and, when this is integer (indicating a valid AP), add one to the the number of elements that imply this step size and end at position i -- since all such elements belong to the same AP. The overall maximum is then the longest AP:
struct solution {
int len;
int pos;
int step;
};
solution longestArithProg(vector<int> const& v) {
solution best = { -1, 0, 0 };
for (int i = 1; i < v.size(); ++i) {
unordered_map<int, int> bestForStep;
for (int j = 0; j < i; ++j) {
int step = (v[i] - v[j]) / (i - j);
if (step * (i - j) == v[i] - v[j]) {
// This j gives an integer step size: record that j lies on this AP
int len = ++bestForStep[step];
if (len > best.len) {
best.len = len;
best.pos = i;
best.step = step;
}
}
}
}
++best.len; // We never counted the final element in the AP
return best;
}
The above C++ code uses O(n^2) time and O(n) space, since it loops over every pair of positions i and j, performing a single hash read and write for each. To answer the original problem:
int howManyChangesNeeded(vector<int> const& v) {
return v.size() - longestArithProg(v).len;
}
This problem has a simple geometric interpretation, which shows that it can be solved in O(n^2) time and probably can't be solved any faster than that (reduction from 3SUM). Suppose our array is [1, 2, 10, 3, 5]. We can write that array as a sequence of points
(0,1), (1,2), (2,10), (3,3), (4,5)
in which the x-value is the index of the array item and the y-value is the value of the array item. The question now becomes one of finding a line which passes the maximum possible number of points in that set. The cost of converting the array is the number of points not on a line, which is minimized when the number of points on a line is maximized.
A fairly definitive answer to that question is given in this SO posting: What is the most efficient algorithm to find a straight line that goes through most points?
The idea: for each point P in the set from left to right, find the line passing through that point and a maximum number of points to the right of P. (We don't need to look at points to the left of P because they would have been caught in an earlier iteration).
To find the maximum number of P-collinear points to the right of P, for each such point Q calculate the slope of the line segment PQ. Tally up the different slopes in a hash map. The slope which maps to the maximum number of hits is what you're looking for.
Technical issue: you probably don't want to use floating point arithmetic to calculate the slopes. On the other hand, if you use rational numbers, you potentially have to calculate the greatest common divisor in order to compare fractions by comparing numerator and denominator, which multiplies running time by a factor of log n. Instead, you should check equality of rational numbers a/b and c/d by testing whether ad == bc.
The SO posting referenced above gives a reduction from 3SUM, i.e., this problem is 3SUM-hard which shows that if this problem could be solved substantially faster than O(n^2), then 3SUM could also be solved substantially faster than O(n^2). This is where the condition that the integers are in (-inf,inf) comes in. If it is known that the integers are from a bounded set, the reduction from 3SUM is not definitive.
An interesting further question is whether the idea in the Wikipedia for solving 3SUM in O(n + N log N) time when the integers are in the bounded set (-N,N) can be used to solve the minimum cost to convert an array to an AP problem in time faster than O(n^2).
Given the array a = [a_1, a_2, ..., a_n] of unsorted integers, let diffs = [a_2-a_1, a_3-a_2, ..., a_n-a_(n-1)].
Find the maximum occurring value in diffs and adjust any values in a necessary so that all neighboring values differ by this amount.
Interestingly,even I had the same question in my campus recruitment test today.While doing the test itself,I realised that this logic of altering elements based on most frequent differences between 2 subsequent elements in the array fails in some cases.
Eg-4,5,8,9 .According to the logic of a2-a1,a3-a2 as proposed above,answer shud be 1 which is not the case.
As you suggested DP,I feel it can be on the lines of considering 2 values for each element in array-cost when it is modified as well as when it is not modified and return minimum of the 2.Finally terminate when you reach end of the array.
I want to find number of path of length N in a graph where the vertex can be any natural number. However two vertex are connected only if the product of the two vertices is less than some natural number P. If the product of two vertexes are greater than P than those are not connected and can't be reached from one other.
I can obviously run two nested loops (<= P) and create an adjacency matrix, but P can be extremely large and this approach would be extremely slow. Can anyone think of some optimal approach to solve the problem? Can we solve it using Dynamic Programming?
I agree with Ante's recurrence, although I used a slightly simplified version. Note that I'm using the letter P to name the maximum product, as it is used in the original problem statement:
f(1,x) = 1
f(i,x) = sum(f(i-1, y) for y in {1, ..., floor(P/x)})
f(i,x) is the number of sequences of length i that end with x. The answer to the question is then f(n+1, 1).
Of course since P can be up to 10^9 in this task, a straightforward implementation with a DP table is out of the question. However, there are only up to m < 70000 possible different values of floor(P/i). So let's find the maximal segments aj ... bj, where floor(P/aj) = floor(P/bj). We can find those segments in O(number of segments * log P) using binary search.
Imagine the full DP table for f. Since there are only m different values for floor(P/x), every row of f consists of m contiguous ranges that have the same value.
So let's compute the compressed DP table, where we represent the rows as list of (length, value) pairs. We start with f(1) = [(P, 1)] and we can compute f(i+1) from f(i) by processing the segments in increasing order and computing prefix sums of the lengths stored in f(i).
The total runtime of my implementation of this approach is O(m (log P + n)). This is the code I used:
using ll=long long;
const int mod = 1000000007;
void add(int& x, ll y) { x = (x+y)%mod; }
int main() {
int n, P;
cin >> n >> P;
int x = 1;
vector<pair<int,int>> segments;
while(x <= P) {
int y = x+1, hi = P+1;
while(y<hi) {
int mid = (y+hi)/2;
if (P/mid < P/x) hi=mid;
else y=mid+1;
}
segments.push_back(make_pair(P/x, y-x));
x = y;
}
reverse(begin(segments), end(segments));
vector<pair<int,int>> dp;
dp.push_back(make_pair(P,1));
for (int i = 1; i <= n; ++i) {
int j = 0;
int sum_smaller = 0, cnt_smaller = 0;
vector<pair<int,int>> dp2;
for (auto it : segments) {
int value = it.first, cnt = it.second;
while (cnt_smaller + dp[j].first <= value) {
cnt_smaller += dp[j].first;
add(sum_smaller,(ll)dp[j].first*dp[j].second);
j++;
}
int pref_sum = sum_smaller;
if (value > cnt_smaller)
add(pref_sum, (ll)(value - cnt_smaller)*dp[j].second);
dp2.push_back(make_pair(cnt, pref_sum));
}
dp = dp2;
reverse(begin(dp),end(dp));
}
cout << dp[0].second << endl;
}
I needed to do some micro-optimizations with the handling of the arrays to get AC, but those aren't really relevant, so I left them away.
If number of vertices is small than adjacency matrix (A) can help. Since sum of elements in A^N is number of distinct paths, if paths are oriented. If not than number of paths i sum of elements / 2. That is due an element (i,j) represents number of paths from vertex i to vertex j.
In this case, same approach can be done by DP, using reasoning that number of paths of length n from vertex v is sum of numbers of paths of length n-1 of all it's neighbours. Neigbours of vertex i are vertices from 1 to floor(Q/i). With that we can construct function N(vertex, length) which represent number of paths from given vertex with given length:
N(i, 1) = floor(Q/i),
N(i, n) = sum( N(j, n-1) for j in {1, ..., floor(Q/i)}.
Number of all oriented paths of length is sum( N(i,N) ).
Given:
array of integers
value K,M
Question:
Find the maximum sum which we can obtain from all K element subsets of given array such that sum is less than value M?
is there a non dynamic programming solution available to this problem?
or if it is only dp[i][j][k] can only solve this type of problem!
can you please explain the algorithm.
Many people have commented correctly that the answer below from years ago, which uses dynamic programming, incorrectly encodes solutions allowing an element of the array to appear in a "subset" multiple times. Luckily there is still hope for a DP based approach.
Let dp[i][j][k] = true if there exists a size k subset of the first i elements of the input array summing up to j
Our base case is dp[0][0][0] = true
Now, either the size k subset of the first i elements uses a[i + 1], or it does not, giving the recurrence
dp[i + 1][j][k] = dp[i][j - a[i + 1]][k - 1] OR dp[i][j][k]
Put everything together:
given A[1...N]
initialize dp[0...N][0...M][0...K] to false
dp[0][0][0] = true
for i = 0 to N - 1:
for j = 0 to M:
for k = 0 to K:
if dp[i][j][k]:
dp[i + 1][j][k] = true
if j >= A[i] and k >= 1 and dp[i][j - A[i + 1]][k - 1]:
dp[i + 1][j][k] = true
max_sum = 0
for j = 0 to M:
if dp[N][j][K]:
max_sum = j
return max_sum
giving O(NMK) time and space complexity.
Stepping back, we've made one assumption here implicitly which is that A[1...i] are all non-negative. With negative numbers, initializing the second dimension 0...M is not correct. Consider a size K subset made up of a size K - 1 subset with sum exceeding M and one other sufficiently negative element of A[] such that overall sum no longer exceeds M. Similarly, our size K - 1 subset could sum to some extremely negative number and then with a sufficiently positive element of A[] sum to M. In order for our algorithm to still work in both cases we would need to increase the second dimension from M to the difference between the sum of all positive elements in A[] and the sum of all negative elements (the sum of the absolute values of all elements in A[]).
As for whether a non dynamic programming solution exists, certainly there is the naive exponential time brute force solution and variations that optimize the constant factor in the exponent.
Beyond that? Well your problem is closely related to subset sum and the literature for the big name NP complete problems is rather extensive. And as a general principle algorithms can come in all shapes and sizes -- it's not impossible for me to imagine doing say, randomization, approximation, (just choose the error parameter to be sufficiently small!) plain old reductions to other NP complete problems (convert your problem into a giant boolean circuit and run a SAT solver). Yes these are different algorithms. Are they faster than a dynamic programming solution? Some of them, probably. Are they as simple to understand or implement, without say training beyond standard introduction to algorithms material? Probably not.
This is a variant of the Knapsack or subset-problem, where in terms of time (at the cost of exponential growing space requirements as the input size grows), dynamic programming is the most efficient method that CORRECTLY solves this problem. See Is this variant of the subset sum problem easier to solve? for a similar question to yours.
However, since your problem is not exactly the same, I'll provide an explanation anyways. Let dp[i][j] = true, if there is a subset of length i that sums to j and false if there isn't. The idea is that dp[][] will encode the sums of all possible subsets for every possible length. We can then simply find the largest j <= M such that dp[K][j] is true. Our base case dp[0][0] = true because we can always make a subset that sums to 0 by picking one of size 0.
The recurrence is also fairly straightforward. Suppose we've calculated the values of dp[][] using the first n values of the array. To find all possible subsets of the first n+1 values of the array, we can simply take the n+1_th value and add it to all the subsets we've seen before. More concretely, we have the following code:
initialize dp[0..K][0..M] to false
dp[0][0] = true
for i = 0 to N:
for s = 0 to K - 1:
for j = M to 0:
if dp[s][j] && A[i] + j < M:
dp[s + 1][j + A[i]] = true
for j = M to 0:
if dp[K][j]:
print j
break
We're looking for a subset of K elements for which the sum of the elements is a maximum, but less than M.
We can place bounds [X, Y] on the largest element in the subset as follows.
First we sort the (N) integers, values[0] ... values[N-1], with the element values[0] is the smallest.
The lower bound X is the largest integer for which
values[X] + values[X-1] + .... + values[X-(K-1)] < M.
(If X is N-1, then we've found the answer.)
The upper bound Y is the largest integer less than N for which
values[0] + values[1] + ... + values[K-2] + values[Y] < M.
With this observation, we can now bound the second-highest term for each value of the highest term Z, where
X <= Z <= Y.
We can use exactly the same method, since the form of the problem is exactly the same. The reduced problem is finding a subset of K-1 elements, taken from values[0] ... values[Z-1], for which the sum of the elements is a maximum, but less than M - values[Z].
Once we've bound that value in the same way, we can put bounds on the third-largest value for each pair of the two highest values. And so on.
This gives us a tree structure to search, hopefully with much fewer combinations to search than N choose K.
Felix is correct that this is a special case of the knapsack problem. His dynamic programming algorithm takes O(K*M) size and O(K*K*M) amount of time. I believe his use of the variable N really should be K.
There are two books devoted to the knapsack problem. The latest one, by Kellerer, Pferschy and Pisinger [2004, Springer-Verlag, ISBN 3-540-40286-1] gives an improved dynamic programming algorithm on their page 76, Figure 4.2 that takes O(K+M) space and O(KM) time, which is huge reduction compared to the dynamic programming algorithm given by Felix. Note that there is a typo on the book's last line of the algorithm where it should be c-bar := c-bar - w_(r(c-bar)).
My C# implementation is below. I cannot say that I have extensively tested it, and I welcome feedback on this. I used BitArray to implement the concept of the sets given in the algorithm in the book. In my code, c is the capacity (which in the original post was called M), and I used w instead of A as the array that holds the weights.
An example of its use is:
int[] optimal_indexes_for_ssp = new SubsetSumProblem(12, new List<int> { 1, 3, 5, 6 }).SolveSubsetSumProblem();
where the array optimal_indexes_for_ssp contains [0,2,3] corresponding to the elements 1, 5, 6.
using System;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
public class SubsetSumProblem
{
private int[] w;
private int c;
public SubsetSumProblem(int c, IEnumerable<int> w)
{
if (c < 0) throw new ArgumentOutOfRangeException("Capacity for subset sum problem must be at least 0, but input was: " + c.ToString());
int n = w.Count();
this.w = new int[n];
this.c = c;
IEnumerator<int> pwi = w.GetEnumerator();
pwi.MoveNext();
for (int i = 0; i < n; i++, pwi.MoveNext())
this.w[i] = pwi.Current;
}
public int[] SolveSubsetSumProblem()
{
int n = w.Length;
int[] r = new int[c+1];
BitArray R = new BitArray(c+1);
R[0] = true;
BitArray Rp = new BitArray(c+1);
for (int d =0; d<=c ; d++) r[d] = 0;
for (int j = 0; j < n; j++)
{
Rp.SetAll(false);
for (int k = 0; k <= c; k++)
if (R[k] && k + w[j] <= c) Rp[k + w[j]] = true;
for (int k = w[j]; k <= c; k++) // since Rp[k]=false for k<w[j]
if (Rp[k])
{
if (!R[k]) r[k] = j;
R[k] = true;
}
}
int capacity_used= 0;
for(int d=c; d>=0; d--)
if (R[d])
{
capacity_used = d;
break;
}
List<int> result = new List<int>();
while (capacity_used > 0)
{
result.Add(r[capacity_used]);
capacity_used -= w[r[capacity_used]];
} ;
if (capacity_used < 0) throw new Exception("Subset sum program has an internal logic error");
return result.ToArray();
}
}