Given an array of items, each of which has a value and cost, what's the best algorithm determine the items required to reach a minimum value at the minimum cost? eg:
Item: Value -> Cost
-------------------
A 20 -> 11
B 7 -> 5
C 1 -> 2
MinValue = 30
naive solution: A + B + C + C + C. Value: 30, Cost 22
best option: A + B + B. Value: 34, Cost 21
Note that the overall value:cost ratio at the end is irrelevant (A + A would give you the best value for money, but A + B + B is a cheaper option which hits the minimum value).
This is the knapsack problem. (That is, the decision version of this problem is the same as the decision version of the knapsack problem, although the optimization version of the knapsack problem is usually stated differently.) It is NP-hard (which means no algorithm is known that is polynomial in the "size" -- number of bits -- in the input). But if your numbers are small (the largest "value" in the input, say; the costs don't matter), then there is a simple dynamic programming solution.
Let best[v] be the minimum cost to get a value of (exactly) v. Then you can calculate the values best[] for all v, by (initializing all best[v] to infinity and):
best[0] = 0
best[v] = min_(items i){cost[i] + best[v-value[i]]}
Then look at best[v] for values upto the minimum you want plus the largest value; the smallest of those will give you the cost.
If you want the actual items (and not just the minimum cost), you can either maintain some extra data, or just look through the array of best[]s and infer from it.
This problem is known as integer linear programming. It's NP-hard.
However, for small problems like your example, it's trivial to make a quick few lines of code to simply brute force all the low combinations of purchase choices.
NP-harddoesn't mean impossible or even expensive, it means your problem becomes rapidly slower to solve with larger scale problems. In your case with just three items, you can solve this in mere microseconds.
For the exact question of what's the best algorithm in general.. there are entire textbooks on it. A good start is good old Wikipedia.
Edit This answer is redacted on account of being factually incorrect. Following the advice in this will only cause you harm.
This is not actually the knapsack problem, because it assumes that you cannot pack more items than there is space for in some container. In you case you want to find the cheapest combination that will fill up the space, allowing for the fact that overflow may occur.
My solution, which I don't know is the optimal but it should be pretty close, would be to compute for each item the cost benefit ratio, find the item with the highest cost benefit and fill the structure with this item until there isn't space for one more item. Then I would test to see if there was a combination with any of the other available items that could fill the available slot for less that the cost of one of the cheapest items and then if such a solution exist, use that combination otherwise use one more of the cheapest items.
Amenddum This may actually also be NP-complete, but I am not sure yet. Anyway for all practical purposes this variation should be much faster than the naive solution.
Related
Consider the problem definition of a knapsack problem. Given a set S of objects - each having a profit and weight associated with it, I have to find a subset T of S, which gives me the maximum profit but has a total weight less than or equal to a constant W. Now consider an extra constraint. In the above problem the profit of one object is independent of another. Suppose I say they're interdependent, say I've a factor 0<= S_ij <=1 for two objects i and j. This factor diminishes the effect of the item with minimum profit. Effectively
profit({i,j})=max(profit(i),profit(j))+S_ij * min(profit(i),profit(j))
This keeps the effective sum between max(profit(i),profit(j)) and profit(i)+profit(j) -> "Atleast as good as the best one but not as good as using both simultaneously". Now I'm tyring to extend it for n>2. Is this a standard problem of some variation of knapsack ? Can I formulate an LP(?) or NLP for this ?
UPDATE:
The set T is a strict subset of S. So you can only use objects in S(use duplicates if it exists in S).
As for the objective function, I'm still not sure about how to go about it. Above I've calculated the score for a 2 object sack considering the interactions between them. Now i want extend it over to more than 2 objects, and I'm not sure how to do it. The letter 'n' is the size of sack. For n=2 I've defined a way of calculating the total profit of the sack but for n>2 I'm not quite clear.
I'm looking in to a kind-of bin-packing problem, but not quite the same.
The problem asks to put n items into minimum number of bins without total weight exceeding capacity of bins. (classical definition)
The difference is:
Each item has a weight and bound, and the capacity of the bin is dynamically determined by the minimum bound of items in that bin.
E.g.,
I have four items A[11,12], B[1,10], C[3,4], D[20,22] ([weight,bound]).
Now, if I put item A into a bin, call it b1, then the capacity of b1 become 12. Now I try to put item B into b1, but failed because the total weight is 11+1 =12, and the capacity of b1 become 10, which is smaller than total weight. So, B is put into bin b2, whose capacity become 10. Now, put item C into b2, because the total weight is 1+3 =4, and the capacity of b2 become 4.
I don't know whether this question has been solved in some areas with some name. Or it is a variant of bin-packing that has been discussed somewhere.
I don't know whether this is the right place to post the question, any helps are appreciated!
Usually with algorithm design for NP-hard problems, it's necessary to reuse techniques rather than whole algorithms. Here, the algorithms for standard bin packing that use branch-and-bound with column generation carry over well.
The idea is that we formulate an enormous set cover instance where the sets are the sets of items that fit into a single bin. Integer programming is a good technique for normal set cover, but there are so many sets that we need to do something else, i.e., column generation. There is a one-to-one correspondence between sets and columns, so we rip out the part of the linear programming solver that uses brute force to find a good column to enter and replace it with a solver for what turns out to be the knapsack analog of this problem.
This modified knapsack problem is, given items with weights, profits, and bounds, find the most profitable set of items whose total weight is less than the minimum bound. The dynamic program for solving knapsack with small integer weights happily transfers over with no loss of efficiency. Just sort the items by descending bounds; then, when forming sets involving the most recent item, the weight limit is just that item's bound.
The following is based on Anony-mouse's answer. I am not an algorithm expert, so consider the following as "just my two cents", for what they are worth.
I think Anony-mouse is correct in starting with the smallest items (by bound). This is because a bin tends to get smaller in capacity the more items you add to it; a bin's maximum capacity is determined with the first item placed in it, it can never get larger after that point.
So instead of starting with a large bin and have its capacity slowly reduced, and having to worry about taking out too-large items that previously fit, let's jut try to keep bins' capacities as constant as possible. If we can keep the bins' capacities stable, we can use "standard" algorithms that know nothing about "bound".
So I'd suggest this:
Group all items by bound.
This will allow you to use a standard bin packing algorithm per group because if all items have the same bound (i.e. bound is constant), it can essentially be disregarded. All that the bound means now is that you know the resulting bins' capacity in advance.
Start with the group with the smallest bound and perform a standard bin packing for its items.
This will result in 1 or more bins that have a capacity equal to the bound of all items in them.
Proceed with the item group having the next-larger bound. See if there are any items that could still be put in an already existing bin (i.e. a bin produced by the previous steps).
Note that bound can again be ignored; since all pre-existing bins already have a smaller capacity than these additional items' bound, the bins' capacity cannot be affected; only weight is relevant, so you can use "standard" algorithms.
I suspect this step is an instance of the (multiple) knapsack problem, so look towards knapsack algorithms to determine how to distribute these items over and into the pre-existing, partially filled bins.
It's possible that the item group from the previous group has only been partially processed, there might be items left. These will go into one or more new bins: Basically, repeat step 3.
Repeat the above steps (from 3 onwards) until no more items are left.
It can still be written as an ILP instance, like so:
Make a binary variable x_{i,j} signifying whether item j goes into bin i, helper variables y_i that signify whether bin i is used, helper variables c_i that determine the capacity of bin i, and there are constants s_j (size of item j) b_j (bound of item j) and M (a large enough constant), now
minimize sum[j] y_j
subject to:
1: for all j:
(sum[i] x_{i,j}) = 1
2: for all i,j:
y_i ≥ x_{i,j}
3: for all i:
(sum[j] s_j * x_{i,j}) ≤ c_i
4: for all i,j:
c_i ≤ b_j + (M - M * x_{i,j})
5: x_{i,j} ϵ {0,1}
6: y_i ϵ {0,1}
The constraints mean
any item is in exactly one bin
if an item is in a bin, then that bin is used
the items in a bin do not exceed the capacity of that bin
the capacity of a bin is no more than the lowest bound of the items that are in it (the thing with the big M prevents items that are not in the bin from changing the capacity, provided you choose M no less than the highest bound)
and 6., variables are binary.
But the integrality gap can be atrocious.
First of all i might be totally wrong and there might exist an algorithm that is even better than mine.
Bin packing is NP-hard and is efficiently solved using classic algorithms like First Fit etc.There are some improvements to this too.Korf's algorithm
I aim to reduce this to normal bin packing by sorting the items by thier bound.The steps are
Sort items by bound :Sorting items by bound will help us in arranging the bins as limiting condition is minimum of bound.
Insert smallest item(by bound) into a bin
Check whether the next item(sorted by bound) can coexist in this bin.If it can then keep the item in the bin too.If not then try putting it in another bin or create another bin for it.
Repeat the procedure till all elements are arranged. The procedure is repeated in ascending order of bounds.
I think this pretty much solves the problem.Please inform me if it doesn't.I am trying to implement the same.And if there are any suggestions or improvements inform me that too. :) Thank you
I have a simple algorithmic question. I would be grateful if you could help me.
We have some 2 dimensional points. A positive weight is associated to them (a sample problem is attached). We want to select a subset of them which maximizes the weights and neither of two selected points overlap each other (for example, in the attached file, we cannot select both A and C because they are in the same row, and in the same way we cannot select both A and B, because they are in the same column.) If there is any greedy (or dynamic) approach I can use. I'm aware of non-overlapping interval selection algorithm, but I cannot use it here, because my problem is 2 dimensional.
Any reference or note is appreciated.
Regards
Attachment:
A simple sample of the problem:
A (30$) -------- B (10$)
|
|
|
|
C (8$)
If you are OK with a good solution, and do not demand the best solution - you can use heuristical algorithms to solve this.
Let S be the set of points, and w(s) - the weightening function.
Create a weight function W:2^S->R (from the subsets of S to real numbers):
W(U) = - INFINITY is the solution is not feasible
Sigma(w(u)) for each u in U otherwise
Also create a function next:2^S -> 2^2^S (a function that gets a subset of S, and returns a set of subsets of S)
next(U) = V you can get V from U by adding/removing one element to/from U
Now, given that data - you can invoke any optimization algorithm in the Artificial Intelligence book, such as Genetic Algorithm or Hill Climbing.
For example, Hill Climbing with random restarts, will be something like that:
1. best<- -INFINITY
2. while there is more time
3. choose a random subset s
4. NEXT <- next(s)
5. if max{ W(v) | for each v in NEXT} < W(s): //s is a local maximum
5.1. if W(s) > best: best <- W(s) //if s is better then the previous result - store it.
5.2. go to 2. //restart the hill climbing from a different random point.
6. else:
6.1. s <- max { NEXT }
6.2. goto 4.
7. return best //when out of time, return the best solution found so far.
The above algorithm is anytime - meaning it will produce better results if given more time.
This can be treated as a linear assignment problem, which can be solved using an algorithm like the Hungarian algorithm. The algorithm tries to minimize the sum of costs, so just negate your weights, and use them as the costs. The assignment of rows to columns will give you the subset of points that you need. There are sparse variants for cases where not every (row,column) pair has an associated point, but you can also just use a large positive cost for these.
Well you can think of this as a binary constraint optimization problem, and there are various algorithms. The easiest algorithm for this problem is backtracking and arc propogation. However, it takes exponential time in the worst case. I am not sure if there are any specific algorithms to take advantage of the geometrical nature of the problem.
This can be solved by a pretty straight forward dynamic programming approach with a exponential time complexity
s = {A, B, C ...}
getMaxSum(s) = max( A.value + getMaxSum(compatibleSubSet(s, A)),
B.value + getMaxSum(compatibleSubSet(s, B)),
...)
where compatibleSubSet(s, A) gets the subset of s that does not overlap with A
To optimize it, you can memorize the result for each subset
Some way to do it:
Write a function that generates subsets ordered from the subset off maximum weight to the subset off minimum weight while ignoring the constraints.
Then call this function repeatedly until a subset that honors the constraints pops up.
In order to improve the performance, you can write a not so dumb generator function that for instance honors the not-on-the-same-row constraint but that ignores the not-on-the-same-column one.
I watched Dynamic Programming - Kapsack Problem (YouTube). However, I am solving a slightly different problem where the constraint is the budget, price, in double, not integer. So I am wondering how can I modify that? Double is "continuous" unlike integer where I can have 1,2,3 .... I don't suppose I do 0.0, 0.1, 0.2 ...?
UPDATE 1
I thought of converting double to int by multiply by 100. Money is only 2 decimal places. But that will mean the range of values will be very large?
UPDATE 2
The problem I need to solve is:
Items have a price (double) & satisfaction (integer) value. I have a budget as a constraint and I need to maximize satisfaction value.
In the youtube video, the author created two 2d array like int[numItems][possibleCapacity(weight)]. Here, I can't as budget is a double not integer
If you want to use floating point numbers with arbitrary precision (i.e., don't have a fixed number of decimals), and these are not fractions, dynamic programming won't work.
The basis of dynamic programming is to store previous results of a calculation for specific inputs. Therefore, if you used floating point numbers with arbitrary precision, you would need practically infinite memory for each of the possible floating point numbers and, of course, do infinite calculations, something that is impossible and non-optimal.
However, if these numbers have a fixed precision (as with the money, which only have two decimal numbers), you can convert these into integers by multiplying them (as you've mentioned), and then solve the knapsack problem as usual.
You will have to do what you said in UPDATE 1: express the budget and item prices in cents (assuming we are talking about dollars). Then we're not talking about arbitrary precision or continuous numbers. Every price (and the budget) will be an integer, it's just that that integer will represent cents.
To make things easier let's assume the budget is $10. The problem is that the Knapsack Capacity will have to take all the values in:
[0.00, 0.01, 0.02, 0.03, ..., 9.99, 10.00]
The values are two many. Each line of the SOLUTION MATRIX and the KEEP MATRIX will have 1001 columns so you won't be able to solve the problem by hand (if the budget is millions of dollars even a computer might have a hard time) but that is inherent to the original problem (you can't do anything about it).
Your best bet is to use some existing code about KNAPSACK or maybe write your own (I don't advice that).
If you can't find existing code about KNAPSACK and are familiar with Linux/Mac I suggest you install the GNU Linear Programming Kit (GLPK) and express the problem as an Integer Linear Program or a Binary Linear Program (if you're trying to solve the 0-1 Knapsack). It will solve the problem for you (plus you can use it through C, C++, Python and maybe Java if you need to). For help using GLPK check this awesome article (you'll probably need part 2, where it talks about integer variables). If you need more help with GLPK please leave a comment.
EDIT:
Basically, what I'm trying to say is that your constraint is not continuous, it's discrete (cents), your problem is that the budget might be too many cents so you won't be able to solve it by hand.
Don't get intimidated because your budget might be several dollars -> several hundreds of cents. If your budget is just 18 cents your problem's size will be comparable to the one in the YouTube video. The guy in the video wouldn't be able to solve his problem either (by hand) if his knapsack size was 1800 (or even 180).
This is not an answer to your question, but might as well be what you are looking for:
Linear Programming
I've used Microsoft's Solver Foundation 3 to make a simple code that solves the problem you described. It doesn't use the knapsack algorithm, but a simplex method.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Common;
using Microsoft.SolverFoundation.Services;
namespace LPOptimizer
{
class Item
{
public String ItemName { get; set; }
public double Price { get; set; }
public double Satisfaction { get; set; }
static void Main(string[] args)
{
//Our data, budget and items with respective satisfaction and price values
double budget = 100.00;
List<Item> items = new List<Item>()
{
new Item(){
ItemName="Product_1",
Price=20.1,
Satisfaction=2.01
},
new Item(){
ItemName="Product_2",
Price=1.4,
Satisfaction=0.14
},
new Item(){
ItemName="Product_3",
Price=22.1,
Satisfaction=2.21
}
};
//variables for solving the problem.
SolverContext context = SolverContext.GetContext();
Model model = context.CreateModel();
Term goal = 0;
Term constraint = 0;
foreach (Item i in items)
{
Decision decision = new Decision(Domain.IntegerNonnegative, i.ItemName);
model.AddDecision(decision); //each item is a decision - should the algorithm increase this item or not?
goal += i.Satisfaction * decision; //decision will contain quantity.
constraint += i.Price * decision;
}
constraint = constraint <= budget; //this now says: "item_1_price * item_1_quantity + ... + item_n_price * item_n_quantity <= budget";
model.AddConstraints("Budget", constraint);
model.AddGoals("Satisfaction", GoalKind.Maximize, goal); //goal says: "Maximize: item_1_satisfaction * item_1_quantity + ... + item_n_satisfaction * item_n_quantity"
Solution solution = context.Solve(new SimplexDirective());
Report report = solution.GetReport();
Console.Write("{0}", report);
Console.ReadLine();
}
}
}
This finds the optimum max for the number of items (integers) with prices (doubles), with a budget constraint (double).
From the code, it is obvious that you could have some items quantities in real values (double). This will probably also be faster than a knapsack with a large range (if you decide to use the *100 you mentioned).
You can easily specify additional constraints (such as number of certain items, etc...). The code above is adapted from this MSDN How-to, where it shows how you can easily define constraints.
Edit
It has occurred to me that you may not be using C#, in this case I believe there are a number of libraries for linear programming in many languages, and are all relatively simple to use: You specify constraints and a goal.
Edit2
According to your Update 2, I've updated this code to include satisfaction.
Have your looked at this.
Sorry, I don't have comment privilege.
Edit 1
Are you saying constraint is the budget instead of knapsack weight?
This still remains a knapsack problem.
Or are your saying instead of Item Values as Integers(0-1 knapsack problem) your have fractions. Then Greedy approach should do fine.
Edit 2
If I understand your problem correctly.. It states
We have n kinds of items, 1 through n. Each kind of item i has a value vi and a price pi. We usually assume that all values and pricess are nonnegative. The Budget is B.
The most common formulation of the problem is the 0-1 knapsack problem, which restricts the number xi of copies of each kind of item to zero or one. Mathematically the 0-1-knapsack problem can be formulated as:
n
maximize E(vi.xi)
i=i
n
subject to E(pi.xi) <= B, xi is a subset of {0,1}
i=1
Neo Adonis's answer is spot on here.. Dynamic programming wont work for arbitrary precision in practice.
But if you are willing to limit the precision say to 2 decimal places.. then carry on as explained in video.. your table should look something like this..
+------+--------+--------+--------+--------+--------+--------+
| Vi,Pi| 0.00 | 0.01 | 0.02 | 0.03 | 0.04 ... B |
+------+--------+--------+--------+--------+--------+--------+
|4,0.23| | | | | | |
|2,2.93| | | | | | |
|7,9.11| | | | | | |
| ... | | | | | | |
| Vn,Pn| | | | | | answer |
+------+--------+--------+--------+--------+--------+--------+
you can even convert real numbers to int as you have mentioned.
Yes, the range of values is very large, and you also have to understand knapsack is NP-complete, i.e, there is no efficient algorithm to solve this. only pseudo polynomial solution using DP. see this and this.
A question recently posted to sci.op-research offered me a welcome respite from some tedious work that I’d rather not think about and you’d rather not hear about. We know that the greedy heuristic solves the continuous knapsack problem
maximizec′xs.t.a′x≤bx≤ux∈ℜ+n(1)
to optimality. (The proof, using duality theory, is quite easy.) Suppose that we add what I’ll call a count constraint, yielding
maximizec′xs.t.a′x≤be′x=b˜x≤ux∈ℜ+n(2)
where e=(1,…,1) . Can it be solved by something other than the simplex method, such as a variant of the greedy heuristic?
The answer is yes, although I’m not at all sure that what I came up with is any easier to program or more efficient than the simplex method. Personally, I would link to a library with a linear programming solver and use simplex, but it was amusing to find an alternative even if the alternative may not be an improvement over simplex.
The method I’ll present relies on duality, specifically a well known result that if a feasible solution to a linear program and a feasible solution to its dual satisfy the complementary slackness condition, then both are optimal in their respective problems. I will denote the dual variables for the knapsack and count constraints λ and μ respectively. Note that λ≥0 but μ is unrestricted in sign. Essentially the same method stated below would work with an inequality count constraint (e′x≤b˜ ), and would in fact be slightly easier, since we would know a priori the sign of μ (nonnegative). The poster of the original question specified an equality count constraint, so that’s what I’ll use. There are also dual variables (ρ≥0 ) for the upper bounds. The dual problem is
minimizebλ+b˜μ+u′ρs.t.λa+μe+ρ≥cλ,ρ≥0.(3)
This being a blog post and not a dissertation, I’ll assume that (2) is feasible, that all parameters are strictly positive, and that the optimal solution is unique and not degenerate. Uniqueness and degeneracy will not cause invalidate the algorithm, but they would complicate the presentation. In an optimal basic feasible solution to (2), there will be either one or two basic variables — one if the knapsack constraint is nonbinding, two if it is binding — with every other variable nonbasic at either its lower or upper bound. Suppose that (λ,μ,ρ) is an optimal solution to the dual of (2). The reduced cost of any variable xi is ri=ci−λai−μ . If the knapsack constraint is nonbinding, then λ=0 and the optimal solution is
xi=uiri>0b˜−∑rj>0ujri=00ri<0.(4)
If the knapsack constraint is binding, there will be two items (j , k ) whose variables are basic, with rj=rk=0 . (By assuming away degeneracy, I’ve assumed away the possibility of the slack variable in the knapsack constraint being basic with value 0). Set
xi=uiri>00ri<0(5)
and let b′=b−∑i∉{j,k}aixi and b˜′=b˜−∑i∉{j,k}xi . The two basic variables are given by
xj=b′−akb˜′aj−akxk=b′−ajb˜′ak−aj.(6)
The algorithm will proceed in two stages, first looking for a solution with the knapsack nonbinding (one basic x variable) and then looking for a solution with the knapsack binding (two basic x variables). Note that the first time we find feasible primal and dual solutions obeying complementary slackness, both must be optimal, so we are done. Also note that, given any μ and any λ≥0 , we can complete it to obtain a feasible solution to (3) by setting ρi=ci−λai−μ+ . So we will always be dealing with a feasible dual solution, and the algorithm will construct primal solutions that satisfy complementary slackness. The stopping criterion therefore reduces to the constructed primal solution being feasible.
For the first phase, we sort the variables so that c1≥⋯≥cn . Since λ=0 and there is a single basic variable (xh ), whose reduced cost must be zero, obviously μ=ch . That means the reduced cost ri=ci−λai−μ=ci−ch of xi is nonnegative for ih . If the solution given by (3) is feasible — that is, if ∑ih . Thus we can use a bisection search to complete this phase. If we assume a large value of n , the initial sort can be done in O(nlogn ) time and the remainder of the phase requires O(logn) iterations, each of which uses O(n) time.
Unfortunately, I don’t see a way to apply the bisection search to the second phase, in which we look for solutions where the knapsack constraint is binding and λ>0 . We will again search on the value of μ , but this time monotonically. First apply the greedy heuristic to problem (1), retaining the knapsack constraint but ignoring the count constraint. If the solutions happens by chance to satisfy the count constraint, we are done. In most cases, though, the count constraint will be violated. If the count exceeds b˜ , then we can deduce that the optimal value of μ in (4) is positive; if the count falls short of b˜ , the optimal value of μ is negative. We start the second phase with μ=0 and move in the direction of the optimal value.
Since the greedy heuristic sorts items so that c1/a1≥⋯≥cn/an , and since we are starting with μ=0 , our current sort order has (c1−μ)/a1≥⋯≥(cn−μ)/an . We will preserve that ordering (resorting as needed) as we search for the optimal value of μ . To avoid confusion (I hope), let me assume that the optimal value of μ is positive, so that we will be increasing μ as we go. We are looking for values of (λ,μ) where two of the x variables are basic, which means two have reduced cost 0. Suppose that occurs for xi and xj ; then
ri=0=rj⟹ci−λai−μ=0=cj−λaj−μ(7)⟹ci−μai=λ=cj−μaj.
It is easy to show (left to the reader as an exercise) that if (c1−μ)/a1≥⋯≥(cn−μ)/an for the current value of μ , then the next higher (lower) value of μ which creates a tie in (7) must involve consecutive a consecutive pair of items (j=i+1 ). Moreover, again waving off degeneracy (in this case meaning more than two items with reduced cost 0), if we nudge μ slightly beyond the value at which items i and i+1 have reduced cost 0, the only change to the sort order is that items i and i+1 swap places. No further movement in that direction will cause i and i+1 to tie again, but of course either of them may end up tied with their new neighbor down the road.
The second phase, starting from μ=0 , proceeds as follows. For each pair (i,i+1) compute the value μi of μ at which (ci−μ)/ai=(ci+1−μ)/ai+1 ; replace that value with ∞ if it is less than the current value of μ (indicating the tie occurs in the wrong direction). Update μ to miniμi , compute λ from (7), and compute x from (5) and (6). If x is primal feasible (which reduces to 0≤xi≤ui and 0≤xi+1≤ui+1 ), stop: x is optimal. Otherwise swap i and i+1 in the sort order, set μi=∞ (the reindexed items i and i+1 will not tie again) and recompute μi−1 and μi+1 (no other μj are affected by the swap).
If the first phase did not find an optimum (and if the greedy heuristic at the start of the second phase did not get lucky), the second phase must terminate with an optimum before it runs out of values of μ to check (all μj=∞ ). Degeneracy can be handled either with a little extra effort in coding (for instance, checking multiple combinations of i and j in the second phase when three-way or higher ties occur) or by making small perturbations to break the degeneracy.
The answers are not quite correct.
You can implement a dynamic programm that solves the knapsack problem with integer satisfaction and arbitrary real number prizes like doubles.
First the standard solution of the problem with integer prizes:
Define K[0..M, 0..n] where K[j, i] = optimal value of items in knapsack of size j, using only items 1, ..., i
for j = 0 to M do K[j,0] = 0
for i = 1 to n do
for j = 0 to M do
//Default case: Do not take item i
K[j,1] = K[j, i-1]
if j >= w_i and v_i+K[j-w, i-1] > K[j, i] then
//Take item i
K[j,i] = v_i + K[j-w_i, i-1]
This creates a table where the solution can be found by following the recursion for entry K[M, n].
Now the solution for the problem with real number weight:
Define L[0..S, 0..N] where L[j, i] = minimal weight of items in knapsack of total value >= j, using only items 1, ..., i
and S = total value of all items
for j = 0 to S do L[j, 0] = 0
for i = 0 to n do
for j = 0 to S do
//Default case: Do not take item i
L[j,i] = L[j, i-1]
if j >= v_i and L[j-v_i, i-1] + w_i < L[j, i] then
//Take item i
L[j, i] = L[j -v_i, i-1] + w_i
The solution can now be found similiar to the other version. Instead of using the weight as first dimension we now use the total value of the items that lead to the minimal weight.
The code has more or less the same runtime O(S * N) whereas the other has O(M * N).
The answer to your question depends on several factors:
How large is the value of constraint (if scaled to cants and converted to integers).
How many items are there.
What kind of knapsack problem is to be solved
What is required precision.
If you have very large constraint value (much more than millions) and very many items (much more than thousands)
Then the only option is Greedy approximation algorithm. Sort the items in decreasing order of value per unit of weight and pack them in this order.
If you want to use a simple algorithm and do not require high precision
Then again try to use greedy algorithm. "Satisfaction value" itself may be very rough approximation, so why bother inventing complex solutions when simple approximation may be enough.
If you have very large (or even continuous) constraint value but pretty small number of items (less than thousands)
Then use branch and bound approach. You don't need to implement it from scratch. Try GNU GLPK. Its branch-and-cut solver is not perfect, but may be enough to solve small problems.
If both constraint value and number of items are small
Use any approach (DP, branch and bound, or just brute-force).
If constraint value is pretty small (less than millions) but there are too many (like millions) items
Then DP algorithms are possible.
Simplest case is the unbounded knapsack problem when there is no upper bound on the number of copies of each kind of item. This wikipedia article contains a good description how to simplify the problem: Dominance relations in the UKP and how to solve it: Unbounded knapsack problem.
More difficult is the 0-1 knapsack problem when you can pack each kind of item only zero times or one time. And the bounded knapsack problem, allowing to pack each kind of item up to some integer limit times is even more difficult. Internet offers lots of implementations for these problems, there are several suggestions in the same article. But I don't know which one is good or bad.
I have several numbers. I need to group them in several groups, so that sums of all numbers in one group are between predefined min and max. The point is to left as few numbers ungrouped as possible.
Input:
min, max: range for sum of numbers
N1, N2, N3 ... Ni: numbers to group
Output:
[N1,N3,N5],[Ni,Nj,Nk,Nm...]...: groups where sum of numbers is between min and max
Na,Nb,Nc...: numbers, left ingrouped.
This problem could be viewed as bin packing into bins of size max, with a funny objective: minimize the number of items not packed into bins holding at least min. One idea from the bin-packing literature is that the "small" items (in this case, items that are small relative to max - min) are easy to pack but are accountable for most of the combinatorial explosion of possibilities. Thus some approximation algorithms for bin packing do something clever for big items and then fill in with the small. Another way to reduce the number of possibilities is to round the numbers to belong to a smaller set. It's somewhat obvious how to do that for bin packing (round up), but it's not clear what to do for this problem.
Okay, I'll give an example of how these ideas could be instantiated. Suppose that max = 1 and min = 1/2. Let's try to find a solution that's competitive with the optimum for when max = 2 and min = 1/2. (That may sound terrible, but this sort of approximation guarantee where OPT is held to higher standards is sometimes used in the literature.)
First round every item's size up to a power of 2. Very large items, of size 4 or greater, can't be packed. Large items, of size 2 or 1 or 1/2, are given their own bins. Small items, of size 1/4 or less, are dealt with as follows. Whenever two items of size 1/4 or less have the same size, combine them into one super-item. Pack all of the new items of size 1/2 into their own bins. The remainder has total size less than 1/2. If there is space in another bin, put them there. Otherwise, give them their own bin.
The quality of the resulting solution for max = 2 is at least as good as the quality of OPT for max = 1. Take the optimal solution for max = 1 and round the item sizes. The set of bad bins remains the same, because no item is smaller, and each bin stores less than 2 because each item is less than twice as large as it used to be. Now it suffices to show that the packing algorithm I gave for powers of 2 is optimal. I'll leave that as an exercise.
I don't expect this instantly to generalize into a full algorithm. I have to get back to work, but the approach I would take would be to force OPT to deal with max = 1 while ALG gets to use max = 1 + epsilon, substitute powers of (1 + epsilon) for powers of two in the rounding step, and then figure out how to pack the small items, probably using a dynamic program since greed likely won't work.
If you're not worried about efficiency, simply generate each possible grouping and choose the one that is correct and optimal in the sense you describe. Clearly, this works for any finite list of numbers (and is, by definition, optimal).
If efficiency is desired, the problem seems to become somewhat more difficult. :D I'll keep thinking.
EDIT: Come to think of it, this problem seems at least as hard as "subset sum" and, as such, I don't think there is a solution significantly better than the one I give (i.e., no known polynomial-time algorithm can solve it, if it is NP-Hard.