I need to create algorithm for extended Knapsack problem, where we have special items with normal items and I can pack only several special items.
So, we have N items and we know weight/value/isSpecial of each one. Also we have defined S, which is maximum number of special items.
Any idea?
The problem can be modeled as a non-geometric knapsack problem where each item has two weights (namely the actual weight and a second weight which is 1 if and only if it is special); the first weight capacity is the knapsack capacity and the second weight capacity is the desired maximum number of special items. According to this Wikipedia article, the problem is known to be NP-complete and does not admit an FPTAS unless P=NP, but admits an FPTAS. Apparently the two-dimensional version also admits a dynamic programming algorithm similar to the one-dimensional version. The semantics of the state space would be as follows, where each item has two weights w1[i] and w2[i] and profit p[i] and where C1 and C2 are the respective capacities.
P[i,j,k] := maximum profit attainable for items in {1,...,i} with weight
in the first component at most j and weight in the second
component at most k
for any i in {1,...,N}
j in {1,...,C1}
k in {1,...,C2}
The recurrence relation would be as follows.
P[i,j,k] = max{ P[i-1,j-w1[i],k-w2[k]] + p[i], // item i occurs
P[i-1,j,k] // item i does not occur
}
In an actual implementation, some index checking would be necessary to determine whether the first case can actuallty occur.
As far as I know, knapsack problem uses dynamic programming to find the best solution of each item depending on its previous items. This hypothesis assumes that the solution is depending on the order of items. Why the final solution is not depend on the order?
No. The dynamic programming solution for knapsack problem does not depend on the previous items.
When considering whether to put an item into the knapsack or not, we just need to consider the remaining capacity of the knapsack before and after selecting the item. So we loop through all the possible remaining capacity and choose the best one.
dp[i][c] = max(dp[i-1][c+w[i]] + v[i], dp[i-1][c])
where dp[i][c] indicates the maximum possible value after considering the i-th item with the remaining capacity of the knapsack equals to c. w[i] indicates the weight(or volume) of the i-th item, and v[i] indicates the value of the i-th item.
It's not necessary to consider the items in order. Considering the items in order is just for convenience. You can also consider selecting the items in a reversed order, or in a random order.
While standard knapsack problem can be solved by dynamic programming, I am trying to twist the problem a bit to clear my concept, however I found it maybe harder than I thought.
Original knapsack problem is that given a knapsack with size W, and a list of items which weight w[i] and has a value v[i], find the subset of items which can fit in the knapsack with highest total value.
To my understanding, this can be done by O(Wn) with dynamic programming, where n is the number of items.
Now if I try to add m constrains, each of them is a pair of items which can only be picked mutual exclusively (i.e. if there exist a constrain of item A and item B, then I can only take either one of them but not both)
Under such constrains, can this problem still be solved by dynamic programming in O(Wn)?
Assumption: Each element is included in atmost one constraint.
For the usual Knapsack problem, the optimal substructure that the problem exhibits is as follows:
For each item there can be two cases:
1. The item is included in the solution
2. The item not included in the solution.
Hence, the optimal solution for n items is given by max of following two values.
1. Maximum value obtained by n-1 items and W weight.
2. v_n + maximum value obtained by n-1 items and W-w_n weight.
Now if we add the constraint that either of nth or (n-1)th item can exist in the solution, then the optimal solution for n items is given by max of following three values.
1. Maximum value obtained by n-2 items and W weight.
2. v_n + maximum value obtained by n-2 items and W-w_n weight.
3. v_(n-1) + maximum value obtained by n-2 items and W-w_(n-1) weight.
So we treat each pair of elements in the constraint as a single element and execute the dynamic programming algorithm in O(Wn) time.
Consider the problem definition of a knapsack problem. Given a set S of objects - each having a profit and weight associated with it, I have to find a subset T of S, which gives me the maximum profit but has a total weight less than or equal to a constant W. Now consider an extra constraint. In the above problem the profit of one object is independent of another. Suppose I say they're interdependent, say I've a factor 0<= S_ij <=1 for two objects i and j. This factor diminishes the effect of the item with minimum profit. Effectively
profit({i,j})=max(profit(i),profit(j))+S_ij * min(profit(i),profit(j))
This keeps the effective sum between max(profit(i),profit(j)) and profit(i)+profit(j) -> "Atleast as good as the best one but not as good as using both simultaneously". Now I'm tyring to extend it for n>2. Is this a standard problem of some variation of knapsack ? Can I formulate an LP(?) or NLP for this ?
UPDATE:
The set T is a strict subset of S. So you can only use objects in S(use duplicates if it exists in S).
As for the objective function, I'm still not sure about how to go about it. Above I've calculated the score for a 2 object sack considering the interactions between them. Now i want extend it over to more than 2 objects, and I'm not sure how to do it. The letter 'n' is the size of sack. For n=2 I've defined a way of calculating the total profit of the sack but for n>2 I'm not quite clear.
I'm looking in to a kind-of bin-packing problem, but not quite the same.
The problem asks to put n items into minimum number of bins without total weight exceeding capacity of bins. (classical definition)
The difference is:
Each item has a weight and bound, and the capacity of the bin is dynamically determined by the minimum bound of items in that bin.
E.g.,
I have four items A[11,12], B[1,10], C[3,4], D[20,22] ([weight,bound]).
Now, if I put item A into a bin, call it b1, then the capacity of b1 become 12. Now I try to put item B into b1, but failed because the total weight is 11+1 =12, and the capacity of b1 become 10, which is smaller than total weight. So, B is put into bin b2, whose capacity become 10. Now, put item C into b2, because the total weight is 1+3 =4, and the capacity of b2 become 4.
I don't know whether this question has been solved in some areas with some name. Or it is a variant of bin-packing that has been discussed somewhere.
I don't know whether this is the right place to post the question, any helps are appreciated!
Usually with algorithm design for NP-hard problems, it's necessary to reuse techniques rather than whole algorithms. Here, the algorithms for standard bin packing that use branch-and-bound with column generation carry over well.
The idea is that we formulate an enormous set cover instance where the sets are the sets of items that fit into a single bin. Integer programming is a good technique for normal set cover, but there are so many sets that we need to do something else, i.e., column generation. There is a one-to-one correspondence between sets and columns, so we rip out the part of the linear programming solver that uses brute force to find a good column to enter and replace it with a solver for what turns out to be the knapsack analog of this problem.
This modified knapsack problem is, given items with weights, profits, and bounds, find the most profitable set of items whose total weight is less than the minimum bound. The dynamic program for solving knapsack with small integer weights happily transfers over with no loss of efficiency. Just sort the items by descending bounds; then, when forming sets involving the most recent item, the weight limit is just that item's bound.
The following is based on Anony-mouse's answer. I am not an algorithm expert, so consider the following as "just my two cents", for what they are worth.
I think Anony-mouse is correct in starting with the smallest items (by bound). This is because a bin tends to get smaller in capacity the more items you add to it; a bin's maximum capacity is determined with the first item placed in it, it can never get larger after that point.
So instead of starting with a large bin and have its capacity slowly reduced, and having to worry about taking out too-large items that previously fit, let's jut try to keep bins' capacities as constant as possible. If we can keep the bins' capacities stable, we can use "standard" algorithms that know nothing about "bound".
So I'd suggest this:
Group all items by bound.
This will allow you to use a standard bin packing algorithm per group because if all items have the same bound (i.e. bound is constant), it can essentially be disregarded. All that the bound means now is that you know the resulting bins' capacity in advance.
Start with the group with the smallest bound and perform a standard bin packing for its items.
This will result in 1 or more bins that have a capacity equal to the bound of all items in them.
Proceed with the item group having the next-larger bound. See if there are any items that could still be put in an already existing bin (i.e. a bin produced by the previous steps).
Note that bound can again be ignored; since all pre-existing bins already have a smaller capacity than these additional items' bound, the bins' capacity cannot be affected; only weight is relevant, so you can use "standard" algorithms.
I suspect this step is an instance of the (multiple) knapsack problem, so look towards knapsack algorithms to determine how to distribute these items over and into the pre-existing, partially filled bins.
It's possible that the item group from the previous group has only been partially processed, there might be items left. These will go into one or more new bins: Basically, repeat step 3.
Repeat the above steps (from 3 onwards) until no more items are left.
It can still be written as an ILP instance, like so:
Make a binary variable x_{i,j} signifying whether item j goes into bin i, helper variables y_i that signify whether bin i is used, helper variables c_i that determine the capacity of bin i, and there are constants s_j (size of item j) b_j (bound of item j) and M (a large enough constant), now
minimize sum[j] y_j
subject to:
1: for all j:
(sum[i] x_{i,j}) = 1
2: for all i,j:
y_i ≥ x_{i,j}
3: for all i:
(sum[j] s_j * x_{i,j}) ≤ c_i
4: for all i,j:
c_i ≤ b_j + (M - M * x_{i,j})
5: x_{i,j} ϵ {0,1}
6: y_i ϵ {0,1}
The constraints mean
any item is in exactly one bin
if an item is in a bin, then that bin is used
the items in a bin do not exceed the capacity of that bin
the capacity of a bin is no more than the lowest bound of the items that are in it (the thing with the big M prevents items that are not in the bin from changing the capacity, provided you choose M no less than the highest bound)
and 6., variables are binary.
But the integrality gap can be atrocious.
First of all i might be totally wrong and there might exist an algorithm that is even better than mine.
Bin packing is NP-hard and is efficiently solved using classic algorithms like First Fit etc.There are some improvements to this too.Korf's algorithm
I aim to reduce this to normal bin packing by sorting the items by thier bound.The steps are
Sort items by bound :Sorting items by bound will help us in arranging the bins as limiting condition is minimum of bound.
Insert smallest item(by bound) into a bin
Check whether the next item(sorted by bound) can coexist in this bin.If it can then keep the item in the bin too.If not then try putting it in another bin or create another bin for it.
Repeat the procedure till all elements are arranged. The procedure is repeated in ascending order of bounds.
I think this pretty much solves the problem.Please inform me if it doesn't.I am trying to implement the same.And if there are any suggestions or improvements inform me that too. :) Thank you