Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I wish to use integer programming to enumerate pareto optimal solutions.
I would like to implement an algorithm that uses gurobi or a similar single-objective integer programming solver to do this, but I don't know any such algorithms. Could someone please propose an algorithm for enumerating the efficient frontier?
In this answer, I'll address how to enumerate all pareto efficient solutions of 2-objective pure integer optimization problems of the form
min_x {g'x, h'x}
s.t. Ax <= b
x integer
Algorithm
We start the algorithm by optimizing for one of the objectives (we'll use g here). Since this is a standard single-objective integer optimization problem, it can be easily solved with gurobi or any other LP solver:
min_x g'x
s.t. Ax <= b
x integer
We initialize a set P, which will eventually contain all the pareto efficient solutions, to P = {x*}, where x* is the optimal solution of this model. To get the next point on the efficient frontier, which has the second-smallest g'x value and an improved h'x value, we can add the following constraints to our model:
Remove x* from the feasible set of solutions (details on these x != x* constraints are provided later in the answer).
Add a constraint that h'x <= h'x*
The new optimization model that we need to solve is:
min_x g'x
s.t. Ax <= b
x != x* for all x* in P
h'x <= h'x* for all x* in P
x integer
Again, this is a single-objective integer optimization model that can be solved with gurobi or another solver (once you follow the details below on how to model x != x* constraints). As you repeatedly solve this model, adding the optimal solutions to P, solutions will get progressively larger (worse) g'x values and progressively smaller (better) h'x values. Eventually, the model will become infeasible, which means no more points on the pareto frontier exist, and the algorithm terminates.
At this point, there may be some pairs of solutions x, y in P for which g'x = g'y and h'x > h'y, in which case x is dominated by y and can be removed. After filtering in this way, the set P represents the full pareto efficient frontier.
x != x* Constraints
All that remains is to model constraints of the form x != x*, where x and x* are n-dimensional vectors of integer variables. Even in one dimension this is a non-convex constraint (see here for details), so we need to add auxiliary variables to help us model the constraint.
Denote the n variables stored in the optimization model (collectively denoted x) as x_1, x_2, ..., x_n, and similarly denote the variable values in x* as x*_1, x*_2, ..., x*_n. We can add new binary variables y_1, y_2, ..., y_n to the model, where y_i is set to 1 when x_i > x*_i. Because x_i and x*_i are integer valued, this is the same as saying x_i >= x*_i + 1, and this can be implemented with the following constraints (M is a large constant):
x_i - x*_i >= 1 - M(1-y_i) for all i = 1, ..., n
Similarly, we can add new binary variables z_1, z_2, ..., z_n to the model, where z_i is set to 1 when x_i < x*_i. Because x_i and x*_i are integer valued, this is the same as saying x_i <= x*_i - 1, and this can be implemented with the following big-M constraints:
x_i - x*_i <= -1 + M(1-z_i) for all i = 1, ..., n
If at least one of the y or z variables is set, then we know that our x != x* constraint is satisfied. Therefore, we can replace x != x* with:
y_1 + y_2 + ... + y_n + z_1 + z_2 + ... + z_n >= 1
In short, each x != x* constraint can be handled by adding 2n binary variables and 2n+1 constraints to the model, where n is the number of variables in the original model.
PolySCIP is an academic open-source solver for multi-objective mixed integer linear programs. In case you do not want to implement your own solver or want to compare yours with another one.
Related
I am reading about linear programming using simplex method in book Introduction to Algorithm Design and analysis.
I am having difficulty in understanding text. Here is text snippet.
The principal advantage of the standard form lies in the simple
mechanism it provides for identifying extreme points of the feasible
region. For the general case of a problem with m equations in n
unknowns (n ≥ m), n − m variables need to be set to zero to get a
system of m equations in m unknowns. If the system obtained has a
unique solution—as any nondegenerate system of linear equations with
the number of equations equal to the number of unknowns does—we have a
basic solution; its coordinates set to zero before solving the system
are called nonbasic, and its coordinates obtained by solving the
system are called basic.
My questions on above text are
What is nondegenerate system of linear equations?
What does author mean by unique solution here in above context?
As author says, "For the general case of a problem with m equations in n unknowns (n ≥ m), (n − m) variables need to be set to zero to get a system of m equations in m unknowns." So when you set (n-m) variables as zero, you will have a system of m equations with m variables.
Let me explain with an example and basic math. Suppose that m = 2, and the system of equations you have is :
x + y = 2 & x - y = 0
So this system of equations has "unique solution", where (x = 1 & y = 1)
Now suppose that m = 3, and the system of equations you have is :
x + y + z = 3, x + y - z = 1, & x + y = 2
In this case, any point with z = 1 and satisfying equation x + y = 2 will satisfy all the equations. For example, (x = 1, y = 1, & z = 1), (x = 2, y = 0, & z = 1), and (x = 4, y = - 2, & z = 1). So here you have more than one solution to this system of equations, so you will say that this system of equations has "multiple solutions".
Now let us talk about degeneracy. Assuming that you get unique solution to the system of equations, If you solve system of m equations with m variables, you will get numerical value for all m variables. If any of these m variables have their numerical value equal to zero, you will say that solution is degenerate. By non-degenerate, author means that all of the variables have non-zero value in solution. Degeneracy tends to increase the number of simplex iterations before reaching the optimal solution.
Now let us talk a little about simplex method. Suppose you have set (n-m) out of n variables as zero (as author says), and you get an unique non-degenerate solution. You say this solution as basic solution. But their is catch here, you don't know which of (n-m) variables out of n variables have to be set as zero to find optimal solution. In theory, you will have n_combinatorial_(n-m) basic solutions to evaluate. This is where simplex method helps. It helps you in identifying optimal set of variables to be set as zero in order to find optimal solution. Once you have optimally identified variables to be set as zero, and your reduced system of equations have unique solution, then solution to this system of equations is the optimal solution.
I hope it helps :-)
This is part of a bigger question. Its actually a mathematical problem. So it would be really great if someone can direct me to any algorithm to obtain the solution of this problem or a pseudo code will be of help.
The question. Given an equation check if it has an integral solution.
For example:
(26a+5)/32=b
Here a is an integer. Is there an algorithm to predict or find if b can be an integer. I need a general solution not specific to this question. The equation can vary. Thanks
Your problem is an example of a linear Diophantine equation. About that, Wikipedia says:
This Diophantine equation [i.e., a x + b y = c] has a solution (where x and y are integers) if and only if c is a multiple of the greatest common divisor of a and b. Moreover, if (x, y) is a solution, then the other solutions have the form (x + k v, y - k u), where k is an arbitrary integer, and u and v are the quotients of a and b (respectively) by the greatest common divisor of a and b.
In this case, (26 a + 5)/32 = b is equivalent to 26 a - 32 b = -5. The gcd of the coefficients of the unknowns is gcd(26, -32) = 2. Since -5 is not a multiple of 2, there is no solution.
A general Diophantine equation is a polynomial in the unknowns, and can only be solved (if at all) by more complex methods. A web search might turn up specialized software for that problem.
Linear Diophantine equations take the form ax + by = c. If c is the greatest common divisor of a and b this means a=z'c and b=z''c then this is Bézout's identity of the form
with a=z' and b=z'' and the equation has an infinite number of solutions. So instead of trial searching method you can check if c is the greatest common divisor (GCD) of a and b
If indeed a and b are multiples of c then x and y can be computed using extended Euclidean algorithm which finds integers x and y (one of which is typically negative) that satisfy Bézout's identity
(as a side note: this holds also for any other Euclidean domain, i.e. polynomial ring & every Euclidean domain is unique factorization domain). You can use Iterative Method to find these solutions:
Integral solution to equation `a + bx = c + dy`
Currently I have this linear programming model:
Max X
such that:
Max_a(Min_b(F(a,b,X))) <= some constant
*Max_a meaning to maximize the following equation by just changing a, and the same applies to Min_b
Now, the problem becomes how to linearize the constraint part. Most of the current Minmax linearization papers talks about Minmax as an objective. But how to linearize it if it was an constraint??
Thanks
Preliminary remark: the problem you describe is not a "linear programming model", and there is no way to transform it into a linear model directly (which doesn't mean it can't be solved).
First, note that the Max in the constraint is not necessary, i.e. your problem can be reformulated as:
Max X
subject to: Min_b F(a, b, X) <= K forall a
Now, since you are speaking of 'linear model', I assume that at least F is linear, i.e.:
F(a, b, X) = Fa.a + Fb.b + FX.X
And the constraint can obviously be written:
Fa.a + Min_b Fb.b + FX.X <= k forall a
The interesting point is that the minimum on b does not depend on the value of a and X. Hence, it can be solved beforehand: first find u = Min_b Fb.b, and then solve
Max X
subject to Fa.a + FX.X <= k - u forall a
This assume, of course, that the domain of a and b are independant (of the form AxB): if there are other constraints coupling a and b, it is a different problem (in that case please write the complete problem in the question).
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
I read the method to calculate the square root of any number and the algorithm is as follows:
double findSquareRoot(int n) {
double x = n;
double y = 1;
double e = 0.00001;
while(x-y >= e) {
x = (x+y)/2;
y = n/x;
}
return x;
}
My question regarding this method are
How it calculates the square root? I didn't understand the mathematics behind this. How x=(x+y)/2 and y=n/x converges to square root of n. Explain this mathematics.
What is the complexity of this algorithm?
It is easy to see if you do some runs and print the successive values of x and y. For example for 100:
50.5 1.9801980198019802
26.24009900990099 3.8109612300726345
15.025530119986813 6.655339226067038
10.840434673026925 9.224722348894286
10.032578510960604 9.96752728032478
10.000052895642693 9.999947104637101
10.000000000139897 9.999999999860103
See, the trick is that if x is not the square root of n, then it is above or below the real root, and n/x is always on the other side. So if you calculate the midpoint of x and n/x it will be somewhat nearer to the real root.
And about the complexity, it is actually unbounded, because the real root will never reached. That's why you have the e parameter.
This is a typical application of Newton's method for calculating the square root of n. You're calculating the limit of the sequence:
x_0 = n
x_{i+1} = (x_i + n / x_i) / 2
Your variable x is the current term x_i and your variable y is n / x_i.
To understand why you have to calculate this limit, you need to think of the function:
f(x) = x^2 - n
You want to find the root of this function. Its derivative is
f'(x) = 2 * x
and Newton's method gives you the formula:
x_{i+1} = x_i - f(x_i) / f'(x_1) = ... = (x_i + n / x_i) / 2
For completeness, I'm copying here the rationale from #rodrigo's answer, combined with my comment to it. This is helpful if you want to forget about Newton's method and try to understand this algorithm alone.
The trick is that if x is not the square root of n, then it is
an approximation which lies either above or below the real root, and y = n/x is always on the
other side. So if you calculate the midpoint of (x+y)/2, it will be
nearer to the real root than the worst of these two approximations
(x or y). When x and y are close enough, you're done.
This will also help you find the complexity of the algorithm. Say that d is the distance of the worst of the two approximations to the real root r. Then the distance between the midpoint (x+y)/2 and r is at most d/2 (it will help you if you draw a line to visualize this). This means that, with each iteration, the distance is halved. Therefore, the worst-case complexity is logarithmic w.r.t. to the distance of the initial approximation and the precision that is sought. For the given program, it is
log(|n-sqrt(n)|/epsilon)
I think all information can be found in wikipedia.
The basic idea is that if x is an overestimate to the square root of a non-negative real number S then S/x, will be an underestimate and so the average of these two numbers may reasonably be expected to provide a better approximation.
With each iteration this algorithm doubles correct digits in answer, so complexity is linear to desired accuracy's logarithm.
Why does it work? As stated here, if you will do infinite iterations you'll get some value, let's name it L. L has to satisfy equasion L = (L + N/L)/2 (as in algorithm), so L = sqrt(N). If you're worried about convergence, you may calculate squared relative errors for each iteration (Ek is error, Ak is computed value):
Ek = (Ak/sqrt(N) - 1)²
if:
Ak = (Ak-1 + N/Ak-1)/2 and Ak = sqrt(N)(sqrt(Ek) + 1)
you may derive recurrence relation for Ek:
Ek = Ek-1²/[4(sqrt(Ek-1) + 1)²]
and limit of it is 0, so limit of A1,A2... sequence is sqrt(N).
The mathematical explanation is that, over a small range, the arithmetic mean is a reasonable approximation to the geometric mean, which is used to calculate the square root. As the iterations get closer to the true square root, the difference between the arithmetic mean and the geometric mean vanishes, and the approximation gets very close. Here is my favorite version of Heron's algorithm, which first normalizes the input n over the range 1 ≤ n < 4, then unrolls the loop for a fixed number of iterations that is guaranteed to converge.
def root(n):
if n < 1: return root(n*4) / 2
if 4 <= n: return root(n/4) * 2
x = (n+1) / 2
x = (x + n/x) / 2
x = (x + n/x) / 2
x = (x + n/x) / 2
x = (x + n/x) / 2
x = (x + n/x) / 2
return x
I discuss several programs to calculate the square root at my blog.
Imagine that I'm a bakery trying to maximize the number of pies I can produce with my limited quantities of ingredients.
Each of the following pie recipes A, B, C, and D produce exactly 1 pie:
A = i + j + k
B = t + z
C = 2z
D = 2j + 2k
*The recipes always have linear form, like above.
I have the following ingredients:
4 of i
5 of z
4 of j
2 of k
1 of t
I want an algorithm to maximize my pie production given my limited amount of ingredients.
The optimal solution of these example inputs would yield me the following quantities of pies:
2 x A
1 x B
2 x C
0 x D
= a total of 5 pies
I can solve this easily enough by taking the maximal producer of all combinations, but the number
of combos becomes prohibitive as the quantities of ingredients increases. I feel like there must
be generalizations of this type of optimization problem, I just don't know where to start.
While I can only bake whole pies, I would be still be interested in seeing a method which may produce non integer results.
You can define the linear programming problem. I'll show the usage on the example, but it can of course be generalized to any data.
Denote your pies as your variables (x1 = A, x2 = B, ...) and the LP problem will be as follows:
maximize x1 + x2 + x3 + x4
s.t. x1 <= 4 (needed i's)
x1 + 2x4 <= 4 (needed j's)
x1 + 2x4 <= 2 (needed k's)
x2 <= 1 (needed t's)
x2 + 2x3 <= 5 (needed z's)
and x1,x2,x3,x4 >= 0
The fractional solution to this problem is solveable polynomially, but the integer linear programming is NP-Complete.
The problem is indeed NP-Complete, because given an integer linear programming problem, you can reduce the problem to "maximize the number of pies" using the same approach, where each constraint is an ingredient in the pie and the variables are the number of pies.
For the integers problem - there are a lot of approximation techniques in the literature for the problem if you can do with "close up to a certain bound", (for example local ratio technique or primal-dual are often used) or if you need an exact solution - exponential solution is probably your best shot. (Unless of course, P=NP)
Since all your functions are linear, it sounds like you're looking for either linear programming (if continuous values are acceptable) or integer programming (if you require your variables to be integers).
Linear programming is a standard technique, and is efficiently solvable. A traditional algorithm for doing this is the simplex method.
Integer programming is intractable in general, because adding integral constraints allows it to describe intractable combinatorial problems. There seems to be a large number of approximation techniques (for example, you might try just using regular linear programming to see what that gets you), but of course they depend on the specific nature of your problem.