I am trying to solve a variant of the multidimensional multiple knapsack problem which tries to optimize the values in each knapsack so that a percentage of each of them can be “taken” and added to create a “final knapsack” with the ideal values. See this question below.
https://cs.stackexchange.com/questions/14163/linear-programming-algorithm-to-check-if-ratios-can-be-combined-with-n-bottles
The problem I linked to says "given n bottles find a solution where you can take a ratio of every bottle and add them to equal the predetermined values (A, B, C)." The problem I have is "given a set of values organize them in the n bottles in such a way that there is a solution where you can take a ratio of every bottle and add them to equal the predetermined values (A, B, C)."
Essentially I would like to create an algorithm that organizes incoming values knapsacks (or bottles as they call it in the question above) so that they could be combined in a way that could guarantee the desired result.
The main difference between the multiple multidimensional knapsack problem and what I am trying to do is what I am trying to maximize. Instead of trying to maximize the total value of all the knapsacks, I want to first multiply each knapsack by a variable Lambda[i] and add them together so that they equal the item F which is a vector with constants (A, B, C, D). I want the knapsacks optimized in such a way that the combination gives back the most amount of item F.
Here are the variables
I: number of bottles
J: set of items
v[j] =[a(j), b(j), c(j), d(j)]: value of item j
p[i] = [a(i), b(i), c(i), d(i)]: value of bottle after items added to it
n[i] = weight of each bottle after items are added to it
m[i]: capacity of bottle i
Lambda[i]: lambda variable to multiply each bottle by
F =[A, B, C, D]: Optimal bottle value
N: Weight of final blend
M: Weight capacity of final bottle
The objective function I am trying to maximize N = Σ n(i) * Lambda(i)
some constraints I have are
n[i] <= m[i]
Σ a(i) * Lambda[i] = A
Σ b(i) * Lambda[i] = B
Σ C(i) * Lambda[i] = C
Σ D(i) * Lambda[i] = D
0 <= Lambda[i] <= 1
I have tried implementing this solution in Gurobi and OR-Tools but the problem I'm having is that the weight of each bin is only found after optimizing so there is no way for me to maximize the function I need.
Ultimately I would like to solve an Online version of this problem where the algorithm wouldn't be able to reject any item coming in but I figured starting with the offline version with a dataset would be easier.
Does this mean this algorithm is not a linear programming problem or I am just missing a step? If it isn't a linear program is there any other method like machine learning that could help me solve this?
Any help would be greatly appreciated.
It looks to me to be probably a linear programming problem, but you are having a problem with the objective being not-linear? There are ways to reformulate some of these not-linear terms to make them solveable: See for example Erwin Kalvelagen's excellent mathematical programming examples such as http://yetanothermathprogrammingconsultant.blogspot.com/2017/05/linearizing-average.html which is probably not quite the same as you need but may give you enough ideas to help.
Related
I've been trying to wrap my head around the fact that a lot of DP questions that involve bottom up tabulation via a 2D Matrix can be simplified into a 1D Array to save on space since you only rely on the previous two rows but I don't really understand the why/how/intuition behind this.
Just wondering if anyone could offer the most dumb downed version of why this works...
Generally, we use can apply DP when the optimal solution to a given problem can be determined from the optimal solutions of its subproblems. When coming up with a solution to some algorithm, it usually helps to come up with a recursive one first. From there, if we observe that recursive subproblems are re-calculated multiple times, we can just memoize the intermediate results for fast reference to them later.
In some special cases, we don't actually need to remember the solution to all subproblems at once; we just need to know a certain subset at a time.
The space optimization described above seems to best answer to the question you're asking - how does one condense the total set of solutions as a 2D matrix into a single 1D array? Well, at a given time, we don't actually store all solutions (the 2D matrix) in any single point in time; we just store what is needed to calculate the next round of intermediate/final outputs in the algorithm.
Perhaps walking through an example application may help reinforce this description.
A nice example is the generalized stock trading problem. Basically, we have an input array prices of a stock on a given day and would like to calculate the maximum profit that can be earned if k buy-sell transactions are made, where one stock may be bought and held at any given time.
The trickiest part in my opinion is figuring out how to move from the one transaction case to the two transaction case. I'll assume we're proficient enough in dynamic programming to move immediately to the k transaction case. Notice that a nice formulation of the problem in terms of subproblems is the following:
prices = input array of prices, length is n
Define dp[k][i] = maximum profit earned by day i, after having made k transactions
dp[k][i] = max(dp[k][i - 1], (prices[i] + effectivePrice)
effectivePrice = max(dp[k - 1][i] - prices[i], effectivePrice) (compute on the fly for each i)
Now in this particular case our "naive" dp solution has a 2D matrix with k rows and n columns. The space reduction here is that, in order to calculate the result for k transactions, we only need knowledge of the case for k - 1 transactions. Therefore, it is certainly possible to solve the problem using two 1D arrays of size n.
Let oldDp = solution for the k - 1 case
Let newDp = solution for the k case (computed on the fly)
for each transaction:
for each day i:
newDp[i] = max(newDp[i - 1], (prices[i] + effectivePrice)
effectivePrice = max(oldDp[i] - prices[i], effectivePrice)
// Set up for next iteration
oldDp = newDp
newDp = blank array of size n
As we can see, we managed to save a lot of space - we went from having to use a 2D matrix with k rows and n columns to two 1D arrays of size n. An even better optimization is to just use a single 1D array; this is possible since the only indices that we examine in oldDp is the current one i when calculating effectivePrice. Because we only need to temporarily remember the old result for day i, we can just make use of a temporary variable. Thus, the optimized pseudocode (for our "naive" approach) appears as below:
Let dp = maximum profit, so that dp[i] = maximum profit after k transactions (built iteratively) on day i.
for each transaction:
for each day i:
// At this point, dp[i] is equivalent to dp[k - 1][i], yet
// for all j < i, dp[j] is equivalent to dp[k][j]!
temp = dp[i]
dp[i] = max(dp[i - 1], prices[i] + effectivePrice)
effectivePrice = max(temp - prices[i], effectivePrice)
And so, using the "naive" idea of determining the optimal solution after k transactions from k - 1 transactions, we optimize space by going from a 2D matrix of size kn to a 1D array of size n.
I'm working on a problem that requires an array (dA[j], j=-N..N) to be calculated from the values of another array (A[i], i=-N..N) based on a conservation of momentum rule (x+y=z+j). This means that for a given index j for all the valid combinations of (x,y,z) I calculate A[x]A[y]A[z]. dA[j] is equal to the sum of these values.
I'm currently precomputing the valid indices for each dA[j] by looping x=-N...+N,y=-N...+N and calculating z=x+y-j and storing the indices if abs(z) <= N.
Is there a more efficient method of computing this?
The reason I ask is that in future I'd like to also be able to efficiently find for each dA[j] all the terms that have a specific A[i]. Essentially to be able to compute the Jacobian of dA[j] with respect to dA[i].
Update
For the sake of completeness I figured out a way of doing this without any if statements: if you parametrize the equation x+y=z+j given that j is a constant you get the equation for a plane. The constraint that x,y,z need to be integers between -N..N create boundaries on this plane. The points that define this boundary are functions of N and j. So all you have to do is loop over your parametrized variables (s,t) within these boundaries and you'll generate all the valid points by using the vectors defined by the plane (s*u + t*v + j*[0,0,1]).
For example, if you choose u=[1,0,-1] and v=[0,1,1] all the valid solutions for every value of j are bounded by a 6 sided polygon with points (-N,-N),(-N,-j),(j,N),(N,N),(N,-j), and (j,-N).
So for each j, you go through all (2N)^2 combinations to find the correct x's and y's such that x+y= z+j; the running time of your application (per j) is O(N^2). I don't think your current idea is bad (and after playing with some pseudocode for this, I couldn't improve it significantly). I would like to note that once you've picked a j and a z, there is at most 2N choices for x's and y's. So overall, the best algorithm would still complete in O(N^2).
But consider the following improvement by a factor of 2 (for the overall program, not per j): if z+j= x+y, then (-z)+(-j)= (-x)+(-y) also.
So I've come across an interesting problem I'd like to solve. It came across when I was trying to solve a game with nondeterminstic transitions. If you've ever heard of this problem or know if it has a name/papers written about it let me know! Here it is.
Given n boxes and m elements where n1 has i1 elements, n2 has i2 elements, etc (i.e i1 + i2 + ... + in = m). Each element has a weight w and value v. Find a selection of exactly one element from each n boxes (solution size = n) such that the value is maximized and the weight <= k (some input parameter).
The first thing I noticed is there are i1*i2...*in solutions. This is less than m choose n, which is less than 2^m, so does this mean the problem is in P (sorry my math is a little fuzzy)? Does anyone have any idea of an algorithm that does not involve iterating over every solution? Approximations are fine!
Edit: Okay so this problem is actually identical to the knapsack problem, so it's NP-hard. Let the boxes have two elements each, one of zero size and zero value, and one of nonzero size and nonzero value. This is identical to knapsack. Can anyone think of a clever pseudopolynomial time algorithm/conversion to knapsack?
This looks close enough to http://en.wikipedia.org/wiki/Knapsack_problem#0.2F1_Knapsack_Problem that almost the same definition of m[i, w] as given there will work - let m[i, w] be the maximum value that can be obtained with weight <= w using items up to i. The only difference is that at each stage instead of considering either taking an item or not, consider which of the possible items at each stage you should take.
I have a simple algorithmic question. I would be grateful if you could help me.
We have some 2 dimensional points. A positive weight is associated to them (a sample problem is attached). We want to select a subset of them which maximizes the weights and neither of two selected points overlap each other (for example, in the attached file, we cannot select both A and C because they are in the same row, and in the same way we cannot select both A and B, because they are in the same column.) If there is any greedy (or dynamic) approach I can use. I'm aware of non-overlapping interval selection algorithm, but I cannot use it here, because my problem is 2 dimensional.
Any reference or note is appreciated.
Regards
Attachment:
A simple sample of the problem:
A (30$) -------- B (10$)
|
|
|
|
C (8$)
If you are OK with a good solution, and do not demand the best solution - you can use heuristical algorithms to solve this.
Let S be the set of points, and w(s) - the weightening function.
Create a weight function W:2^S->R (from the subsets of S to real numbers):
W(U) = - INFINITY is the solution is not feasible
Sigma(w(u)) for each u in U otherwise
Also create a function next:2^S -> 2^2^S (a function that gets a subset of S, and returns a set of subsets of S)
next(U) = V you can get V from U by adding/removing one element to/from U
Now, given that data - you can invoke any optimization algorithm in the Artificial Intelligence book, such as Genetic Algorithm or Hill Climbing.
For example, Hill Climbing with random restarts, will be something like that:
1. best<- -INFINITY
2. while there is more time
3. choose a random subset s
4. NEXT <- next(s)
5. if max{ W(v) | for each v in NEXT} < W(s): //s is a local maximum
5.1. if W(s) > best: best <- W(s) //if s is better then the previous result - store it.
5.2. go to 2. //restart the hill climbing from a different random point.
6. else:
6.1. s <- max { NEXT }
6.2. goto 4.
7. return best //when out of time, return the best solution found so far.
The above algorithm is anytime - meaning it will produce better results if given more time.
This can be treated as a linear assignment problem, which can be solved using an algorithm like the Hungarian algorithm. The algorithm tries to minimize the sum of costs, so just negate your weights, and use them as the costs. The assignment of rows to columns will give you the subset of points that you need. There are sparse variants for cases where not every (row,column) pair has an associated point, but you can also just use a large positive cost for these.
Well you can think of this as a binary constraint optimization problem, and there are various algorithms. The easiest algorithm for this problem is backtracking and arc propogation. However, it takes exponential time in the worst case. I am not sure if there are any specific algorithms to take advantage of the geometrical nature of the problem.
This can be solved by a pretty straight forward dynamic programming approach with a exponential time complexity
s = {A, B, C ...}
getMaxSum(s) = max( A.value + getMaxSum(compatibleSubSet(s, A)),
B.value + getMaxSum(compatibleSubSet(s, B)),
...)
where compatibleSubSet(s, A) gets the subset of s that does not overlap with A
To optimize it, you can memorize the result for each subset
Some way to do it:
Write a function that generates subsets ordered from the subset off maximum weight to the subset off minimum weight while ignoring the constraints.
Then call this function repeatedly until a subset that honors the constraints pops up.
In order to improve the performance, you can write a not so dumb generator function that for instance honors the not-on-the-same-row constraint but that ignores the not-on-the-same-column one.
I have a set of n real numbers. I also have a set of functions,
f_1, f_2, ..., f_m.
Each of these functions takes a list of numbers as its argument. I also have a set of m ranges,
[l_1, u_1], [l_2, u_2], ..., [l_m, u_m].
I want to repeatedly choose a subset {r_1, r_2, ..., r_k} of k elements such that
l_i <= f_i({r_1, r_2, ..., r_k}) <= u_i for 1 <= i <= m.
Note that the functions are smooth. Changing one element in {r_1, r_2, ..., r_k} will not change f_i({r_1, r_2, ..., r_k}) by much. average and variance are two f_i that are commonly used.
These are the m constraints that I need to satisfy.
Moreover I want to do this so that the set of subsets I choose is uniformly distributed over the set of all subsets of size k that satisfy these m constraints. Not only that, but I want to do this in an efficient manner. How quickly it runs will depend on the density of solutions within the space of all possible solutions (if this is 0.0, then the algorithm can run forever). (Assume that f_i (for any i) can be computed in a constant amount of time.)
Note that n is large enough that I cannot brute-force the problem. That is, I cannot just iterate through all k-element subsets and find which ones satisfy the m constraints.
Is there a way to do this?
What sorts of techniques are commonly used for a CSP like this? Can someone point me in the direction of good books or articles that talk about problems like this (not just CSPs in general, but CSPs involving continuous, as opposed to discrete values)?
Assuming you're looking to write your own application and use existing libraries to do this, there are choices in many languages, like Python-constraint, or Cream or Choco for Java, or CSP for C++. The way you've described the problem it sound like you're looking for a general purpose CSP solver. Are there any properties of your functions that may help reduce the complexity, such as being monotonic?
Given the problem as you've described it, you can pick from each range r_i uniformly and throw away any m-dimensional point that fails to meet the criterion. It will be uniformly distributed because the original is uniformly distributed and the set of subsets is a binary mask over the original.
Without knowing more about the shape of f, you can't make any guarantees about whether time is polynomial or not (or even have any idea of how to hit a spot that meets the constraint). After all, if f_1 = (x^2 + y^2 - 1) and f_2 = (1 - x^2 - y^2) and the constraints are f_1 < 0 and f_2 < 0, you can't satisfy this at all (and without access to the analytic form of the functions, you could never know for sure).
Given the information in your message, I'm not sure it can be done at all...
Consider:
numbers = {1....100}
m = 1 (keep it simple)
F1 = Average
L1 = 10
U1 = 50
Now, how many subset of {1...100} can you come up with that produces an average between 10 & 50?
This looks like a very hard problem. For the simplest case with linear functions you could take a look at linear programming.