I looked at many resources and also this question, but am still confused why we need Dynamic Programming to solve 0/1 knapsack?
The question is: I have N items, each item with value Vi, and each item has weight Wi. We have a bag of total weight W. How to select items to get best total of values over limitation of weight.
I am confused with the dynamic programming approach: why not just calculate the fraction of (value / weight) for each item and select the item with best fraction which has less weight than remaining weight in bag?
For your fraction-based approach you can easily find a counterexample.
Consider
W=[3, 3, 5]
V=[4, 4, 7]
Wmax=6
Your approach gives optimal value Vopt=7 (we're taking the last item since 7/5 > 4/3), but taking the first two items gives us Vopt=8.
As other answers pointed out, there are edge cases with your approach.
To explain the recursive solution a bit better, and perhaps to understand it better I suggest you approach it with this reasoning:
For each "subsack"
If we have no fitting element there is no best element
If we only have one fitting element, the best choice is that element
If we have more than one fitting element, we take each element and calculate the best fit for its "subsack". The best choice is the highest valued element/subsack combination.
This algorithm works because it spans all the possible combinations of fitting elements and finds the one with the highest value.
A direct solution, instead, is not possible as the problem is NP-hard.
Just look at this counterexample:
Weight 7, W/V pairs (3/10),(4/12),(5/21)
Greedy algorithm fails when there is unit ratio case. for example consider the following example:
n= 1 2, P= 4 18, W= 2 18, P/W= 2 1
Knapsack capacity=18
According to greedy algorithm it will consider the first item since it's P/W ratio is greater and hence the total profit will be 4 (since it cannot insert the second item after first as the capacity reduces to 16 after inserting the first item).
But the actual answer is 18.
Hence there are multiple corner cases where greedy fails to give optimal solution, that's why we use Dynamic programming in 0/1 knapsack problem.
Related
I have to reorder a sequence of elements based on the similarity between each other (expressed by a coefficient) so that each element is the most similar possible to each of its neighbors. I have to find an algorithm rather than a code.
Example with 10 elements and similarity coefficients calculated for each pair of the elements below :
The excel file can be find here : https://1drv.ms/x/s!AtmZN4-kjgrPms99fqgaDwAS_F4uYw
What I have tried :
Find a pair with the highest coefficient. In the example : 0.98 for T3 (left-end) and T5 (right-end)
Find maximum coefficient between the left-end and the remaining elements
Find maximum coefficient between the right-end and the remaining elements
Take the maximum between 2. and 3.
If maximum is 2. add on the left the element corresponding to the maximum coefficient for the left-end. Else, add on the right the element corresponding to the maximum coefficient for the right-end
Repeat points 2 - 6 until no elements left.
Here is the result :
The result isn't bad. One of the disadvantages I see is that 0.99>0.98 is considered in the same way as 0.99>0.01.
The second option I thought about was maximizing the sum of coefficients between all neighbors, but don't really know where to start from. Especially if there are significantly more than 10 elements. More, it could result in a more "flat" order where while having better similarities overall some extremely similar elements could be placed far from each other.
Being really new to this kind of problems I am pretty sure this should be a rather standard issue with existing solutions. Could you please point to those?
Thank you!
After researching I have found that my problem can be seen as the "Travelling Salesman Problem" (TSP). More here : https://en.wikipedia.org/wiki/Travelling_salesman_problem
To apply it you can see "elements" in my example as "cities" in TSP and (1-Similarity coefficient) as "distances".
Question: I have a sack which can carry some weight, and number of items with weight and i want to put as much weight as possible in the sack to carry, after some thought I have got into a conclusion, I take the highest weight every time and put into the sack, intuitivaly that it will work if the weights that are given are incremented atleast by multiplication of 2. For e.g. 2 4 8 16 32 64..
Can anyone help me prove if I am right or wrong about that? I have also an intuition about that, would love to hear urs.
Note: thought about saying that the sum of the previous numbers wont be bigger of the current nunber.
Yes, described greedy algorithm will work for powers of two.
Note that partial sum of geometric sequence 1,2,4,8,16..2^(k-1) is 2^k-1, that is why you always should choose the largest possible item - it is always bigger than any sum of smaller items.
In mathematical sense set of 2's powers forms matroid
But it would fail in general case (example - 3,3,4 and sum 6). You can learn for dynamic programming to solve this problem with integer weights. It is similar to knapsack problem with unit item costs.
Consider the problem definition of a knapsack problem. Given a set S of objects - each having a profit and weight associated with it, I have to find a subset T of S, which gives me the maximum profit but has a total weight less than or equal to a constant W. Now consider an extra constraint. In the above problem the profit of one object is independent of another. Suppose I say they're interdependent, say I've a factor 0<= S_ij <=1 for two objects i and j. This factor diminishes the effect of the item with minimum profit. Effectively
profit({i,j})=max(profit(i),profit(j))+S_ij * min(profit(i),profit(j))
This keeps the effective sum between max(profit(i),profit(j)) and profit(i)+profit(j) -> "Atleast as good as the best one but not as good as using both simultaneously". Now I'm tyring to extend it for n>2. Is this a standard problem of some variation of knapsack ? Can I formulate an LP(?) or NLP for this ?
UPDATE:
The set T is a strict subset of S. So you can only use objects in S(use duplicates if it exists in S).
As for the objective function, I'm still not sure about how to go about it. Above I've calculated the score for a 2 object sack considering the interactions between them. Now i want extend it over to more than 2 objects, and I'm not sure how to do it. The letter 'n' is the size of sack. For n=2 I've defined a way of calculating the total profit of the sack but for n>2 I'm not quite clear.
I am new to algorithms and am currently studying using you-tube video tutorials/lectures and a book, I firstly watch the video and then read the book and finally try a question from the book to make sure I have learned the topic correctly. I am currently up to greedy algorithms and it is very confusing.
Inside the book there are various problems but I am having trouble understanding and answering a particular one.
Firstly it gives the problem which is (I've just copied the text).
there is a set of n objects of sizes {x1; x2;..... xn} and a bin with
capacity B. All these are positive integers. Try to find a subset of these objects
so that their total size is smaller than or equal to B, but as close to B as possible.
All objects are 1-dimensional. For example, if the objects have sizes 4, 7, 10, 12, 15, and
B = 20, then we should choose 4 and 15 with total size 19 (or equivalently, 7 and 12).
For each of the following greedy algorithms, show that they are not optimal by creating
a counter-example. try to make your examples as bad as you can, where "badness"
is measured by the ratio between the optimal and greedy solutions. Thus if the best
solution has value 10 and the greedy solution has value 5, then the ratio is 2.
how do I do this for the following?
1) Always choose the object with the largest size so that the total size of this and all
other objects already chosen does not exceed B. Repeat this for the remaining objects.
Assume the following instance of the problem:
You have a box of size 2n, one element of size n+1 and the rest are of size n.
It is easy to see that the optimal is 2 elements of size n, while the greedy will get you one element of size n+1.
Since it is true for each n, it actually gives you a desired ratio of at least using this greedy approach 2.
This sounds similar to the 0-1 Knapsack problem where each item has a different size but the same value, which means any one item doesn't have any preference to being placed into the bin other than its size. In your code, you need to examine each item and calculate the maximum total size that results whether or not putting it into the bin without exceeding the capacity of the bin.
Given an array of items, each of which has a value and cost, what's the best algorithm determine the items required to reach a minimum value at the minimum cost? eg:
Item: Value -> Cost
-------------------
A 20 -> 11
B 7 -> 5
C 1 -> 2
MinValue = 30
naive solution: A + B + C + C + C. Value: 30, Cost 22
best option: A + B + B. Value: 34, Cost 21
Note that the overall value:cost ratio at the end is irrelevant (A + A would give you the best value for money, but A + B + B is a cheaper option which hits the minimum value).
This is the knapsack problem. (That is, the decision version of this problem is the same as the decision version of the knapsack problem, although the optimization version of the knapsack problem is usually stated differently.) It is NP-hard (which means no algorithm is known that is polynomial in the "size" -- number of bits -- in the input). But if your numbers are small (the largest "value" in the input, say; the costs don't matter), then there is a simple dynamic programming solution.
Let best[v] be the minimum cost to get a value of (exactly) v. Then you can calculate the values best[] for all v, by (initializing all best[v] to infinity and):
best[0] = 0
best[v] = min_(items i){cost[i] + best[v-value[i]]}
Then look at best[v] for values upto the minimum you want plus the largest value; the smallest of those will give you the cost.
If you want the actual items (and not just the minimum cost), you can either maintain some extra data, or just look through the array of best[]s and infer from it.
This problem is known as integer linear programming. It's NP-hard.
However, for small problems like your example, it's trivial to make a quick few lines of code to simply brute force all the low combinations of purchase choices.
NP-harddoesn't mean impossible or even expensive, it means your problem becomes rapidly slower to solve with larger scale problems. In your case with just three items, you can solve this in mere microseconds.
For the exact question of what's the best algorithm in general.. there are entire textbooks on it. A good start is good old Wikipedia.
Edit This answer is redacted on account of being factually incorrect. Following the advice in this will only cause you harm.
This is not actually the knapsack problem, because it assumes that you cannot pack more items than there is space for in some container. In you case you want to find the cheapest combination that will fill up the space, allowing for the fact that overflow may occur.
My solution, which I don't know is the optimal but it should be pretty close, would be to compute for each item the cost benefit ratio, find the item with the highest cost benefit and fill the structure with this item until there isn't space for one more item. Then I would test to see if there was a combination with any of the other available items that could fill the available slot for less that the cost of one of the cheapest items and then if such a solution exist, use that combination otherwise use one more of the cheapest items.
Amenddum This may actually also be NP-complete, but I am not sure yet. Anyway for all practical purposes this variation should be much faster than the naive solution.