As far as I know, knapsack problem uses dynamic programming to find the best solution of each item depending on its previous items. This hypothesis assumes that the solution is depending on the order of items. Why the final solution is not depend on the order?
No. The dynamic programming solution for knapsack problem does not depend on the previous items.
When considering whether to put an item into the knapsack or not, we just need to consider the remaining capacity of the knapsack before and after selecting the item. So we loop through all the possible remaining capacity and choose the best one.
dp[i][c] = max(dp[i-1][c+w[i]] + v[i], dp[i-1][c])
where dp[i][c] indicates the maximum possible value after considering the i-th item with the remaining capacity of the knapsack equals to c. w[i] indicates the weight(or volume) of the i-th item, and v[i] indicates the value of the i-th item.
It's not necessary to consider the items in order. Considering the items in order is just for convenience. You can also consider selecting the items in a reversed order, or in a random order.
While standard knapsack problem can be solved by dynamic programming, I am trying to twist the problem a bit to clear my concept, however I found it maybe harder than I thought.
Original knapsack problem is that given a knapsack with size W, and a list of items which weight w[i] and has a value v[i], find the subset of items which can fit in the knapsack with highest total value.
To my understanding, this can be done by O(Wn) with dynamic programming, where n is the number of items.
Now if I try to add m constrains, each of them is a pair of items which can only be picked mutual exclusively (i.e. if there exist a constrain of item A and item B, then I can only take either one of them but not both)
Under such constrains, can this problem still be solved by dynamic programming in O(Wn)?
Assumption: Each element is included in atmost one constraint.
For the usual Knapsack problem, the optimal substructure that the problem exhibits is as follows:
For each item there can be two cases:
1. The item is included in the solution
2. The item not included in the solution.
Hence, the optimal solution for n items is given by max of following two values.
1. Maximum value obtained by n-1 items and W weight.
2. v_n + maximum value obtained by n-1 items and W-w_n weight.
Now if we add the constraint that either of nth or (n-1)th item can exist in the solution, then the optimal solution for n items is given by max of following three values.
1. Maximum value obtained by n-2 items and W weight.
2. v_n + maximum value obtained by n-2 items and W-w_n weight.
3. v_(n-1) + maximum value obtained by n-2 items and W-w_(n-1) weight.
So we treat each pair of elements in the constraint as a single element and execute the dynamic programming algorithm in O(Wn) time.
Consider the problem definition of a knapsack problem. Given a set S of objects - each having a profit and weight associated with it, I have to find a subset T of S, which gives me the maximum profit but has a total weight less than or equal to a constant W. Now consider an extra constraint. In the above problem the profit of one object is independent of another. Suppose I say they're interdependent, say I've a factor 0<= S_ij <=1 for two objects i and j. This factor diminishes the effect of the item with minimum profit. Effectively
profit({i,j})=max(profit(i),profit(j))+S_ij * min(profit(i),profit(j))
This keeps the effective sum between max(profit(i),profit(j)) and profit(i)+profit(j) -> "Atleast as good as the best one but not as good as using both simultaneously". Now I'm tyring to extend it for n>2. Is this a standard problem of some variation of knapsack ? Can I formulate an LP(?) or NLP for this ?
UPDATE:
The set T is a strict subset of S. So you can only use objects in S(use duplicates if it exists in S).
As for the objective function, I'm still not sure about how to go about it. Above I've calculated the score for a 2 object sack considering the interactions between them. Now i want extend it over to more than 2 objects, and I'm not sure how to do it. The letter 'n' is the size of sack. For n=2 I've defined a way of calculating the total profit of the sack but for n>2 I'm not quite clear.
I am familiar with the 0-1 knapsack problem and when you are given a certain number of copies from each item but I can figure out how to solve it when you are given infinite copies of each item using dynamic programming. I am trying to solve it by hand right now so I am not interested in any particular code. For example here is how I solve the 0-1 problem. How do I need to modify this if I am given an infinity amount of copies of each item?
Edit: I am aware there is a second solution to the problem containing items 1,2, and 3 with the same total value.
One possibility would be to provide a suitable number of multiplicities of the items. For item i, there can be at most
m_i := K / w_i
choices of that item, where K denotes the knapsack capacity and w_i denotes the weight of the i-th item. Furthermore, for each weight value which occurs in the instance, there is at most one item type necessary, namely the one with maximum profit with respect to the weight.
Equivalently, one could modify the evaluation of the dynamic program to reflect the different number of items to be taken, instead of just distinguishing between a choice of 0 and 1.
I'm looking in to a kind-of bin-packing problem, but not quite the same.
The problem asks to put n items into minimum number of bins without total weight exceeding capacity of bins. (classical definition)
The difference is:
Each item has a weight and bound, and the capacity of the bin is dynamically determined by the minimum bound of items in that bin.
E.g.,
I have four items A[11,12], B[1,10], C[3,4], D[20,22] ([weight,bound]).
Now, if I put item A into a bin, call it b1, then the capacity of b1 become 12. Now I try to put item B into b1, but failed because the total weight is 11+1 =12, and the capacity of b1 become 10, which is smaller than total weight. So, B is put into bin b2, whose capacity become 10. Now, put item C into b2, because the total weight is 1+3 =4, and the capacity of b2 become 4.
I don't know whether this question has been solved in some areas with some name. Or it is a variant of bin-packing that has been discussed somewhere.
I don't know whether this is the right place to post the question, any helps are appreciated!
Usually with algorithm design for NP-hard problems, it's necessary to reuse techniques rather than whole algorithms. Here, the algorithms for standard bin packing that use branch-and-bound with column generation carry over well.
The idea is that we formulate an enormous set cover instance where the sets are the sets of items that fit into a single bin. Integer programming is a good technique for normal set cover, but there are so many sets that we need to do something else, i.e., column generation. There is a one-to-one correspondence between sets and columns, so we rip out the part of the linear programming solver that uses brute force to find a good column to enter and replace it with a solver for what turns out to be the knapsack analog of this problem.
This modified knapsack problem is, given items with weights, profits, and bounds, find the most profitable set of items whose total weight is less than the minimum bound. The dynamic program for solving knapsack with small integer weights happily transfers over with no loss of efficiency. Just sort the items by descending bounds; then, when forming sets involving the most recent item, the weight limit is just that item's bound.
The following is based on Anony-mouse's answer. I am not an algorithm expert, so consider the following as "just my two cents", for what they are worth.
I think Anony-mouse is correct in starting with the smallest items (by bound). This is because a bin tends to get smaller in capacity the more items you add to it; a bin's maximum capacity is determined with the first item placed in it, it can never get larger after that point.
So instead of starting with a large bin and have its capacity slowly reduced, and having to worry about taking out too-large items that previously fit, let's jut try to keep bins' capacities as constant as possible. If we can keep the bins' capacities stable, we can use "standard" algorithms that know nothing about "bound".
So I'd suggest this:
Group all items by bound.
This will allow you to use a standard bin packing algorithm per group because if all items have the same bound (i.e. bound is constant), it can essentially be disregarded. All that the bound means now is that you know the resulting bins' capacity in advance.
Start with the group with the smallest bound and perform a standard bin packing for its items.
This will result in 1 or more bins that have a capacity equal to the bound of all items in them.
Proceed with the item group having the next-larger bound. See if there are any items that could still be put in an already existing bin (i.e. a bin produced by the previous steps).
Note that bound can again be ignored; since all pre-existing bins already have a smaller capacity than these additional items' bound, the bins' capacity cannot be affected; only weight is relevant, so you can use "standard" algorithms.
I suspect this step is an instance of the (multiple) knapsack problem, so look towards knapsack algorithms to determine how to distribute these items over and into the pre-existing, partially filled bins.
It's possible that the item group from the previous group has only been partially processed, there might be items left. These will go into one or more new bins: Basically, repeat step 3.
Repeat the above steps (from 3 onwards) until no more items are left.
It can still be written as an ILP instance, like so:
Make a binary variable x_{i,j} signifying whether item j goes into bin i, helper variables y_i that signify whether bin i is used, helper variables c_i that determine the capacity of bin i, and there are constants s_j (size of item j) b_j (bound of item j) and M (a large enough constant), now
minimize sum[j] y_j
subject to:
1: for all j:
(sum[i] x_{i,j}) = 1
2: for all i,j:
y_i ≥ x_{i,j}
3: for all i:
(sum[j] s_j * x_{i,j}) ≤ c_i
4: for all i,j:
c_i ≤ b_j + (M - M * x_{i,j})
5: x_{i,j} ϵ {0,1}
6: y_i ϵ {0,1}
The constraints mean
any item is in exactly one bin
if an item is in a bin, then that bin is used
the items in a bin do not exceed the capacity of that bin
the capacity of a bin is no more than the lowest bound of the items that are in it (the thing with the big M prevents items that are not in the bin from changing the capacity, provided you choose M no less than the highest bound)
and 6., variables are binary.
But the integrality gap can be atrocious.
First of all i might be totally wrong and there might exist an algorithm that is even better than mine.
Bin packing is NP-hard and is efficiently solved using classic algorithms like First Fit etc.There are some improvements to this too.Korf's algorithm
I aim to reduce this to normal bin packing by sorting the items by thier bound.The steps are
Sort items by bound :Sorting items by bound will help us in arranging the bins as limiting condition is minimum of bound.
Insert smallest item(by bound) into a bin
Check whether the next item(sorted by bound) can coexist in this bin.If it can then keep the item in the bin too.If not then try putting it in another bin or create another bin for it.
Repeat the procedure till all elements are arranged. The procedure is repeated in ascending order of bounds.
I think this pretty much solves the problem.Please inform me if it doesn't.I am trying to implement the same.And if there are any suggestions or improvements inform me that too. :) Thank you