I am trying to devise a pseudo-code for Knapsack algorithms, where a single item can be selected multiple times. The classical algorithm is
OPT(i, w) = max(OPT(i-1, w) or vi + OPT(i-1, w-wi))
In order to meet the requirements, I am modifying it to
k=1;
max = maximum(OPT(i-1, w))
while(OPT(i-1, w - k*wi) > 0) {
maximum = max(maximum, k*vi + OPT(i-1, w - k*wi))
k++
}
OPT(i, w) = maximum
Does this seem to be an appropriate solution? Or any better solution exists?
Please let me know if any additional information is required.
Rest all remains the same, vi denotes the value of ith element and wi denotes the weight of ith element.
If you want to be able to chose multiple items, all you have to do is to change the recursion when selecting an element:
OPT(i, w) = max(OPT(i-1, w) or vi + OPT(i, w-wi))
^ ^
removed the element not removing the element
The idea is, you allow readding it on next iteration. You add an element as much as you want, and you stop adding it when you "chose" to use the first condition in the stop recursive formula.
The recursion will still not be infinite because you must stop once w<0.
Time complexity does not change - O(nW)
Based on Algorithms DVP, the solution to Knapsack with repetition is like below:
K(0)=0
for w=1 to W:
K(w) = max{K(w - w_i) + v_i, w_i < w}
return K(W)
Here, W is the capacity; w_i is the weight of item i; v_i is the value of item i; K(w) is the max value achievable with knapsack of capacity w.
Your solution seems like 0-1 Knapsack though.
Related
I am trying to write a python algorithm to do the following.
Given a set of positive integers S, find the subset with the smallest sum, greater or equal to k.
For example:
S = [50, 103, 85, 21, 30]
k = 140
subset = [85, 50, 21] (with sum = 146)
The numbers in the initial set are all integers, and k can be arbitrarily large. Usually there will be about 100 numbers in the set.
Of course there's the brute force solution of going through all possible subsets, but that runs in O(2^n) which is unfeasable. I have been told that this problem is NP-Complete, but that there should be a Dynamic Programing approach that allows it to run in pseudo-polynomial time, like the knapsack problem, but so far, attempting to use DP still leads me to solutions that are O(2^n).
Is there such a way to appy DP to this problem? If so, how? I find DP hard to understand so I might have missed something.
Any help is much appreciated.
Well seeing that numbers are not integers but reals, best I can think of is O(2^(n/2) log (2^(n/2)).
It might look worse at first glance but notice that 2^(n/2) == sqrt(2^n)
So to achieve such complexity we will use technique known as meet in the middle:
Split set into 2 parts of sizes n/2 and n-n/2
Use brute force to generate all subsets (including empty one) and store them in arrays, let's call them A and B
Let's sort array B
Now for each element a in A, if B[-1] + a >=k we can use binary search to find smallest element b in B that satisfies a + b >= k
out of all such a + b pairs we found choose the smallest
OP changed question a little now its integers so here goes dynamic solution:
well not much to say, classical knapsack.
for each i in [1,n] we have 2 options for set item i:
1. Include in subset, state changes from (i, w) to (i+1, w + S[i])
2. Skip it, state changes from (i, w) to (i+1, w)
Every time we reach some w that`s >= k, we update answer
Pseudo-code:
visited = Set() //some set/hashtable object to store visited states
S = [...]//set of integers from input
int ats = -1;
void solve(int i, int w) //theres atmost n*k different states so complexity is O(n*k)
{
if(w >= k)
{
if(ats==-1)ats=w;
else ats=min(ats,w);
return;
}
if(i>n)return;
if(visited.count(i,w))return; //we already visited this state, can skip
visited.insert(i,w)=1;
solve(i+1, w + S[i]); //take item
solve(i+1, w); //skip item
}
solve(1,0);
print(ats);
I'm working on a program to solve a variant of the 0/1 Knapsack problem.
The original problem is described here: https://en.wikipedia.org/wiki/Knapsack_problem.
In case the link goes missing in the future, I will give you a summary of the 0/1 Knapsack problem (if you are familiar with it, jump this paragraph):
Let's say we have n items, each with weight wi and value vi. We want to put items in a bag, that supports a maximum weight W, so that the total value inside the bag is the maximum possible without overweighting the bag. Items cannot have multiple instances (i.e., we only have one of each). The objective of the problem is to maximize SUM(vi.xi) so that SUM(wi.xi) <= W and xi = 0, 1 (xi represents the state of an item being or not in the bag).
For my case, there are small differences in both conditions and objective:
The weight of all items is 1, wi = 1, i = 1...n
I always want to put exactly half the items in the bag. So, the maximum weight capacity of the bag is half (rounded up) of the number of items.W = ceil[n/2] or W = floor[(n+1)/2].
Also, the weight inside the bag must be equal to its maximum capacity SUM(wi.xi) = W
Finally, instead of maximizing the value of the items inside the bag, the objective is that the value of the items inside is as close as possible to the value of the items outside. Hence, my objective is to minimize |SUM(vi.-xi) - SUM[vi(1-xi)]|, which simplifies into something like minimize |SUM[vi(2xi - 1)]|.
Now, there is a pseudo-code for the original 0/1 Knapsack problem in the Wikipedia page above (you can find it on the bottom of this text), but I am having trouble adapting it to my scenario. Can someone help? (I am not asking for code, just for an idea, so language is irrelevant)
Thanks!
Wikipedia's pseudo-code for 0/1 Knapsack problem:
Assume w1, w2, ..., wn, W are strictly positive integers. Define
m[i,w] to be the maximum value that can be attained with weight less
than or equal to w using items up to i (first i items).
We can define m[i,w] recursively as follows:
m[0, w]=0
m[i, w] = m[i-1, w] if wi > w (the new item is more than the current weight limit)
m[i, w]= max(m[i-1, w], m[i-1, w-wi] + vi) if wi <= w.
The solution can then be found by calculating m[n,W].
// Input:
// Values (stored in array v)
// Weights (stored in array w)
// Number of distinct items (n)
// Knapsack capacity (W)
for j from 0 to W do:
m[0, j] := 0
for i from 1 to n do:
for j from 0 to W do:
if w[i-1] <= j then:
m[i, j] := max(m[i-1, j], m[i-1, j-w[i-1]] + v[i-1])
else:
m[i, j] := m[i-1, j]
Thanks to #harold, it seems like this problem is not a Knapsack problem, but a Partition problem. Part of the pseudo-code I was seeking is in the corresponding Wikipedia page: https://en.wikipedia.org/wiki/Partition_problem
EDIT: well, actually, Partition problem algorithms tell you whether a Set of items can be partitioned in 2 sets of equal value or not. Suppose it can't, you have approximation algorithms, which say whether you can have the set partiotioned in 2 sets with the difference their values being lower than d.
BUT, they don't tell you the resulting sub-sets, and that's what I was seeking.
I ended up finding a question here asking for that (here: Balanced partition), with a code example which I have tested and works fine.
This is essentially the problem of connecting n destinations with the minimal amount of road possible.
The input is a set of vertices (a,b, ... , n)
The weight of an edge between two vertices is easily calculated (example the cartesian distance between the two vertices)
I would like an algorithm that given a set of vertices in euclidian space, returns a set of edges that would constitute a connected graph and whose total weight of edges is as small as it could be.
In graph language, this is the Minimum Spanning Tree of a Connected Graph.
With brute force I would have:
Define all possible edges between all vertices - say you have n
vertices, then you have n(n-1)/2 edges in the complete graph
A possible edge can be on or off (2 states)
Go through all possible edge on/off
combinations: 2^(n(n-1)/2)!
Ignore all those that would not connect the
graph
From the remaining combinations, find the one whose sum of
edge weights is the smallest of all
I understand this is an NP-Hard problem. However, realistically for my application, I will have a maximum of 11 vertices. I would like to be able to solve this on a typical modern smart phone, or at the very least on a small server size.
As a second variation, I would like to obtain the same goal, with the restriction that each vertex is connected to a maximum of one other vertex. Essentially obtaining a single trace, starting from any point, and finishing at any other point, as long as the graph is connected. There is no need to go back to where you started. In graph language, this is the Open Euclidian Traveling Salesman Problem.
Some pseudocode algorithms would be much helpful.
Ok for the first problem you have to build a Minimum Spanning Tree. There are several algorithms to do so, Prim and Kruskal. But take a look also in the first link to the treatment for complete graphs that it is your case.
For the second problem, it becomes a little more complicated. The problem becomes an Open Traveling Salesman Problem (oTSP). Reading the previous link maybe focused on Euclidean and Asymmetric.
Regards
Maybee you could try a greedy algorithm:
1. Create a list sortedList that stores each pair of nodes i and j and is sorted by the
weight w(i,j).
2. Create a HashSet connectedNodes that is empty at the beginning
3. while (connectedNodes.size() < n)
element := first element of sortedList
if (connectedNodes.isEmpty())
connectedNodes.put(element.nodeI);
connectedNodes.put(element.nodeJ);
delete element from sortedList
else
for(element in sortedList) //start again with the first
if(connectedNodes.get(element.nodeI) || connectedNodes.get(element.nodeJ))
if(!(connectedNodes.get(element.nodeI) && connectedNodes.get(element.nodeJ)))
//so it does not include already both nodes
connectedNodes.put(element.nodeI);
connectedNodes.put(element.nodeJ);
delete element from sortedList
break;
else
continue;
So I explain step 3 a little bit:
You add as long nodes till all nodes are connected to one other. It is sure that the graph is connected, because you just add a node, if he has a connection to an other one already in the connectedNodes list.
So this algorithm is greedy what means, it does not make sure, that the solution is optimal. But it is a quite good approximation, because it always takes the shortest edge (because sortedList is sorted by the weight of the edge).
Yo don't get duplicates in connectedNodes, because it is a HashSet, which also make the runtime faster.
All in all the runtime should be O(n^2) for the sorting at the beginning and below its around O(n^3), because in worst case you run in every step through the whole list that has size of n^2 and you do it n times, because you add one element in each step.
But more likely is, that you find an element much faster than O(n^2), i think in most cases it is O(n).
You can solve the travelsalesman problem and the hamilton path problem with the optimap tsp solver fron gebweb or a linear program solver. But the first question seems to ask for a minimum spanning tree maybe the question tag is wrong?
For the first problem, there is an O(n^2 * 2^n) time algorithm. Basically, you can use dynamic programming to reduce the search space. Let's say the set of all vertices is V, so the state space consists of all subsets of V, and the objective function f(S) is the minimum sum of weights of the edges connecting vertices in S. For each state S, you may enumerate over all edges (u, v) where u is in S and v is in V - S, and update f(S + {v}). After checking all possible states, the optimal answer is then f(V).
Below is the sample code to illustrate the idea, but it is implemented in a backward approach.
const int n = 11;
int weight[n][n];
int f[1 << n];
for (int state = 0; state < (1 << n); ++state)
{
int res = INF;
for (int i = 0; i < n; ++i)
{
if ((state & (1 << i)) == 0) continue;
for (int j = 0; j < n; ++j)
{
if (j == i || (state & (1 << j)) == 0) continue;
if (res > f[state - (1 << j)] + weight[i][j])
{
res = f[state - (1 << j)] + weight[i][j];
}
}
}
f[state] = res;
}
printf("%d\n", f[(1 << n) - 1]);
For the second problem, sorry I don't quite understand it. Maybe you should provide some examples?
The dynamic programming algorithm to optimally fill a knapsack works well in the case of one knapsack. But is there an efficient known algorithm that will optimally fill 2 knapsacks (capacities can be unequal)?
I have tried the following two approaches and neither of them is correct.
First fill the first knapsack using the original DP algorithm to fill one knapsack and then fill the other knapsack.
First fill a knapsack of size W1 + W2 and then split the solution into two solutions (where W1 and W2 are the capacities of the two knapsacks).
Problem statement (see also Knapsack Problem at Wikipedia):
We have to fill the knapsack with a set of items (each item has a weight and a value) so as to maximize the value that we can get from the items while having a total weight less than or equal to the knapsack size.
We cannot use an item multiple times.
We cannot use a part of an item. We cannot take a fraction of an item. (Every item must be either fully included or not).
I will assume each of the n items can only be used once, and you must maximize your profit.
Original knapsack is dp[i] = best profit you can obtain for weight i
for i = 1 to n do
for w = maxW down to a[i].weight do
if dp[w] < dp[w - a[i].weight] + a[i].gain
dp[w] = dp[w - a[i].weight] + a[i].gain
Now, since we have two knapsacks, we can use dp[i, j] = best profit you can obtain for weight i in knapsack 1 and j in knapsack 2
for i = 1 to n do
for w1 = maxW1 down to a[i].weight do
for w2 = maxW2 down to a[i].weight do
dp[w1, w2] = max
{
dp[w1, w2], <- we already have the best choice for this pair
dp[w1 - a[i].weight, w2] + a[i].gain <- put in knapsack 1
dp[w1, w2 - a[i].weight] + a[i].gain <- put in knapsack 2
}
Time complexity is O(n * maxW1 * maxW2), where maxW is the maximum weight the knapsack can carry. Note that this isn't very efficient if the capacities are large.
The original DP assumes you mark in the dp array that values which you can obtain in the knapsack, and updates are done by consequently considering the elements.
In case of 2 knapsacks you can use 2-dimensional dynamic array, so dp[ i ][ j ] = 1 when you can put weight i to first and weight j to second knapsack. Update is similar to original DP case.
The recursive formula is anybody is looking:
Given n items, such that item i has weight wi and value pi. The two knapsacks havk capacities of W1 and W2.
For every 0<=i<=n, 0<=a<=W1, 0<=b<=W2, denote M[i,a,b] the maximal value.
for a<0 or b<0 - M[i,a,b] = −∞
for i=0, or a,b=0 - M[i,a,b] = 0
The formula:
M[i,a,b] = max{M[i-1,a,b], M[i-1,a-wi,b] + pi, M[i-1,a,b-wi] + pi}
Every solution to the problem with i items either has item i in knapsack 1, in knapsack 2, or in none of them.
In wikipedia the algorithm for Knapsack is as follows:
for i from 1 to n do
for j from 0 to W do
if j >= w[i] then
T[i, j] := max(T[i-1, j], T[i-1, j-w[i]] + v[i]) [18]
else
T[i, j] := T[i-1, j]
end if
end for
end for
And it is the same structures on all examples I found online.
What I can not understand is how does this code take into account the fact that perhaps the max value comes from a smaller knapsack? E.g. if the knapsack capacity is 8 then perhaps max value comes from capacity 7 (8 - 1).
I could not find anywhere logic to consider that perhaps the max value comes from a smaller knapsack. Is this wrong idea?
The Dynamic Programming solution of knapsack is basically recursive:
T(i,j) = max{ T(i-1,j) , T(i-1,j-w[i]) + v[i] }
// ^ ^
// ignore the element add the element, your value is increase
// by v[i] and the additional weight you can
// carry is decreased by w[i]
(The else condition is redundant in the recursive form if you set T(i,j) = -infinity for each j < 0).
The idea is exhaustive search, you start from one element and you have two possibilities: add it, or don't.
You check both options, and chose the best of those.
Since it is done recursively - you effectively checking ALL possibilities to assign the elements to the knapsack.
Note that the solution in wikipedia is basically a bottom-up solution for the same recursive formula
As I see, you have misunderstood the concept of knapsack. which I will describe here in details till we reach the code part.
First, there are two versions of the problem:
0-1 knapsack problem: here, the Items are indivisible, you either take an item or not. and can be solved with dynamic programming. //and this one is the one yo are facing problems with
Fractional knapsack problem: don't care about this one now.
For the first problem you can understand it as the following:
Given a knapsack with maximum capacity W, and a set S consisting of n items
Each item i has some weight wi and benefit value bi (all wi and W are integer values).
SO, How to pack the knapsack to achieve maximum total value of packed
items?
and in mathematical mouth:
and to solve this problem using Dynamic Programming We set up a table V[0..k, 0..W] with one row for each available item, and one column for each weight from 0 to W.
We need to carefully identify the sub-problems,
The sub-problem then will be to compute V[k,w], i.e., to find an optimal solution for
Sk= {items labeled 1, 2, .. k} in a knapsack of size w (maximum value achievable given capacity w and items 1,…, k)
So, we found this formula to solve our problem:
This algorithm only finds the max possible value that can be carried in the knapsack
i.e., the value in V[n,W]
To know the items that make this maximum value, this will be another topic.
I really hope that this answer will help you. I have an pp presentation that walks with you to fill the table and to show you the algorithm step by step. But I don't know how can I upload it to stackoverflow. let me know if any help needed.