Unexpected results from sum using gekko variable - gekko

I am optimizing a simple problem where I am summing intermediate variables for a constraint where the sum needs to be lower than a certain budget.
When I print the sum, either using sum or np.sum, I get the following results:(((((((((((((((((((((((((((((i429+i430)+i431)+i432)+i433)+i434)+i435)+i436)+i437)+i438)+i439)+i440)+i441)+i442)+i443)+i444)+i445)+i446)+i447)+i448)+i449)+i450)+i451)+i452)+i453)+i454)+i455)+i456)+i457)+i458)
Here is the command to create the variables and the sum.
x = m.Array(m.Var, (len(bounds)),integer=True)
sums = [m.Intermediate(objective_inverse2(x,y)) for x,y in zip(x,reg_feats)]
My understanding of the intermediate variable is a variable which is dynamically calculated based on the value of x, which are decision variables.
Here is the summing function for the max budget constraint.
m.Equation(np.sum(sums) < max_budget)
Solving the problem returns an error saying there are no feasible solution, even through trivial solutions exist. Furthermore, removing this constraint returns a solution which naturally does not violate the max budget constraint.
What am I misunderstanding about the intermediate variable and how to sum them.

It is difficult to diagnose the problem without a complete, minimal problem. Here is an attempt to recreate the problem:
from gekko import GEKKO
import numpy as np
m = GEKKO()
nb = 5
x = m.Array(m.Var,nb,value=1,lb=0,ub=1,integer=True)
y = m.Array(m.Var,nb,lb=0)
i = [] # intermediate list
for xi,yi in zip(x,y):
i.append(m.Intermediate(xi*yi))
m.Maximize(m.sum(i))
m.Equation(m.sum(i)<=100)
m.options.SOLVER = 1
m.solve()
print(x)
print(y)
Instead of creating a list of Intermediates, the summation can also happen with the result of the list comprehension. This way, only one Intermediate value is created.
from gekko import GEKKO
import numpy as np
m = GEKKO()
nb = 5
x = m.Array(m.Var,nb,value=1,lb=0,ub=1,integer=True)
y = m.Array(m.Var,nb,lb=0)
sums = m.Intermediate(m.sum([xi*yi for xi,yi in zip(x,y)]))
m.Maximize(sums)
m.Equation(sums<=100)
m.options.SOLVER = 1
m.solve()
print(sums.value)
print(x)
print(y)
In both cases, the optimal solution is:
---------------------------------------------------
Solver : APOPT (v1.0)
Solution time : 1.560000001336448E-002 sec
Objective : -100.000000000000
Successful solution
---------------------------------------------------
[100.0]
[[1.0] [1.0] [1.0] [1.0] [1.0]]
[[20.0] [20.0] [20.0] [20.0] [20.0]]
Try using the Gekko m.sum() function to improve solution efficiency, especially for large problems.

Related

Curve fitting with nonlinear constraints with scipy.optimize.minimize

Let's say that we want to fit a nonlinear model to our data (that include measurement errors) with certain constraints on the fit parameters among themselves. Without these constraints, scipy.optimize.curve_fit does the job as it supports by default the optimization of the
chisq = sum((r / sigma) ** 2)
where r is the difference between the true and predicted value. Long story short, the user defines a function and pass it to the solver which minimizes the chisq loss. The situation gets a bit more complex when there are other restrictions imposed on the function parameters, e.g. for a linear model y=ax+b, we can demand a fixed nonlinear condition such as a^2+b=1. A function scipy.optimization.minimize is just what the doctor ordered, provided that we pass the chisq as argument. Below is the code for using minimize found here scipy.optimize with non linear constraints. I'm struggling very hard in building the chisq properly to make the code work. The most general question would how does one defines in the most pythonic way a custom loss that would be minimized?
Help please.
from math import cos, atan
import numpy as np
from scipy.optimize import minimize
def f(x):
return 0.1 * x[0] * x[1]
def ineq_constraint(x):
return x[0]**2 + x[1]**2 - (5. + 2.2 * cos(10 * atan(x[0] / x[1])))**2
con = {'type': 'ineq', 'fun': ineq_constraint}
x0 = [1, 1]
res = minimize(f, x0, method='SLSQP', constraints=con)
Since we minimize the chisq function that is found by optimizing our targeted function, I guess the chisq will be defined in a very elegant way in a form of a wrapper since for arguments it takes the targeted function, a column of true values and a column of uncertanties e.g.
chisq(target_function, (y, y_err)).

GEKKO: Array size as a model variable

I'm quite new to Gekko. Is it possible to vary the size of a model array as part of an optimization? I am running a simple problem where various numbers of torsional springs engage at different angles, and I would like to allow the model to change the number of engagement angles. Each spring has several component variables, which I am also attempting to define as arrays of variables. However, the size definition of the array theta_engage, below, has not accepted int(n_engage.value). I get the following error:
TypeError: int() argument must be a string, a bytes-like object or a number, not 'GK_Value'
Relevant code:
n_engage = m.Var(2, lb=1, ub=10, integer=True)
theta_engage = m.Array(m.Var, (int(n_engage.value)))
theta_engage[0].value = 0.0
theta_engage[0].lower = 0.0
theta_engage[0].upper = 85.0
theta_engage[1].value = 15.0
theta_engage[1].lower = 0.0
theta_engage[1].upper = 85.0
If I try to define the size of theta_engage only by n_engage.value, I get this error:
TypeError: expected sequence object with len >= 0 or a single integer
I suppose I could define the array at the maximum size I am willing to accept and allow the number of springs to have a lower bound of 0, but I would have to enforce a minimum number of total springs somehow in the constraints. If Gekko is capable of varying the size of the arrays this way it seems to me the more elegant solution.
Any help is much appreciated.
The problem structure can't be changed iteration-to-iteration. However, it is easy to define a binary variable b that either activates or deactivates those parts of the model that should be included or excluded.
from gekko import GEKKO
import numpy as np
m = GEKKO()
# number of springs
n = 10
# number of engaged springs (1-10)
nb = m.Var(2, lb=1, ub=n, integer=True)
# engaged springs (binary, 0-1)
b = m.Array(m.Var,n,lb=0,ub=1,integer=True)
# angle of engaged springs
θ = m.Array(m.Param,n,lb=0,ub=85)
# initialize values
t0 = [0,15,20,25,30,15,30,25,10,50]
for i,ti in enumerate(t0):
θ[i].value = ti
# contributing spring forces
F = [m.Intermediate(b[i]*m.cos((np.pi/180.0)*θ[i])) \
for i in range(10)]
# force constraint
m.Equation(m.sum(F)>=3)
# engaged springs
m.Equation(nb==m.sum(b))
# minimize engaged springs
m.Minimize(nb)
# optimize with APOPT solver
m.options.SOLVER=1
m.solve()
# print solution
print(b)
This gives a solution in 0.079 sec that springs 1, 3, 9, and 10 should be engaged. It selects the minimum number of springs (4) to achieve the required force that is equivalent to 3 springs at 0 angle.
Successful solution
---------------------------------------------------
Solver : APOPT (v1.0)
Solution time : 7.959999999729916E-002 sec
Objective : 4.00000000000000
Successful solution
---------------------------------------------------
[[1.0] [0.0] [1.0] [0.0] [0.0] [0.0] [0.0] [0.0] [1.0] [1.0]]

Computing a single element of the adjugate or inverse of a symbolic binary matrix

I'm trying to get a single element of an adjugate A_adj of a matrix A, both of which need to be symbolic expressions, where the symbols x_i are binary and the matrix A is symmetric and sparse. Python's sympy works great for small problems:
from sympy import zeros, symbols
size = 4
A = zeros(size,size)
x_i = [x for x in symbols(f'x0:{size}')]
for i in range(size-1):
A[i,i] += 0.5*x_i[i]
A[i+1,i+1] += 0.5*x_i[i]
A[i,i+1] = A[i+1,i] = -0.3*(i+1)*x_i[i]
A_adj_0 = A[1:,1:].det()
A_adj_0
This calculates the first element A_adj_0 of the cofactor matrix (which is the corresponding minor) and correctly gives me 0.125x_0x_1x_2 - 0.28x_2x_2^2 - 0.055x_1^2x_2 - 0.28x_1x_2^2, which is the expression I need, but there are two issues:
This is completely unfeasible for larger matrices (I need this for sizes of ~100).
The x_i are binary variables (i.e. either 0 or 1) and there seems to be no way for sympy to simplify expressions of binary variables, i.e. simplifying polynomials x_i^n = x_i.
The first issue can be partly addressed by instead solving a linear equation system Ay = b, where b is set to the first basis vector [1, 0, 0, 0], such that y is the first column of the inverse of A. The first entry of y is the first element of the inverse of A:
b = zeros(size,1)
b[0] = 1
y = A.LUsolve(b)
s = {x_i[i]: 1 for i in range(size)}
print(y[0].subs(s) * A.subs(s).det())
print(A_adj_0.subs(s))
The problem here is that the expression for the first element of y is extremely complicated, even after using simplify() and so on. It would be a very simple expression with simplification of binary expressions as mentioned in point 2 above. It's a faster method, but still unfeasible for larger matrices.
This boils down to my actual question:
Is there an efficient way to compute a single element of the adjugate of a sparse and symmetric symbolic matrix, where the symbols are binary values?
I'm open to using other software as well.
Addendum 1:
It seems simplifying binary expressions in sympy is possible with a simple custom substitution which I wasn't aware of:
A_subs = A_adj_0
for i in range(size):
A_subs = A_subs.subs(x_i[i]*x_i[i], x_i[i])
A_subs
You should make sure to use Rational rather than floats in sympy so S(1)/2 or Rational(1, 2) rather than 0.5.
There is a new (undocumented and for the moment internal) implementation of matrices in sympy called DomainMatrix. It is likely to be a lot faster for a problem like this and always produces polynomial results in a fully expanded form. I expect that it will be much faster for this kind of problem but it still seems to be fairly slow for this because is is not sparse internally (yet - that will probably change in the next release) and it does not take advantage of the simplification from the symbols being binary-valued. It can be made to work over GF(2) but not with symbols that are assumed to be in GF(2) which is something different.
In case it is helpful though this is how you would use it in sympy 1.7.1:
from sympy import zeros, symbols, Rational
from sympy.polys.domainmatrix import DomainMatrix
size = 10
A = zeros(size,size)
x_i = [x for x in symbols(f'x0:{size}')]
for i in range(size-1):
A[i,i] += Rational(1, 2)*x_i[i]
A[i+1,i+1] += Rational(1, 2)*x_i[i]
A[i,i+1] = A[i+1,i] = -Rational(3, 10)*(i+1)*x_i[i]
# Convert to DomainMatrix:
dM = DomainMatrix.from_list_sympy(size-1, size-1, A[1:, 1:].tolist())
# Compute determinant and convert back to normal sympy expression:
# Could also use dM.det().as_expr() although it might be slower
A_adj_0 = dM.charpoly()[-1].as_expr()
# Reduce powers:
A_adj_0 = A_adj_0.replace(lambda e: e.is_Pow, lambda e: e.args[0])
print(A_adj_0)

Python Gekko - How to use built-in maximum function with sequential solvers?

When solving models sequentially in Python GEKKO (i.e. with IMODE >= 4) fails when using the max2 and max3 functions that come with GEKKO.
This is for use cases, where np.maximum or the standard max function treat a GEKKO parameter like an array, which is not always the intended usage or can create errors when comparing against integers for example.
minimal code example:
from gekko import GEKKO
import numpy as np
m = GEKKO()
m.time = np.arange(0,20)
y = m.Var(value=5)
forcing = m.Param(value=np.arange(-5,15))
m.Equation(y.dt()== m.max2(forcing,0) * y)
m.options.IMODE=4
m.solve(disp=False)
returns:
Exception: #error: Degrees of Freedom
* Error: DOF must be zero for this mode
STOPPING...
I know from looking at the code that both max2 and max3 use inequality expressions in the equations, which understandably introduces the degrees of freedoms, so was this functionality never intended? Could there be some workaround to fix this?
Any help would be much appreciated!
Note:
I hope this is not a duplicate of How to define maximum of Intermediate and another value in Python Gekko, when using sequential solver?, but instead asking a more concise & different question, about essentially the same issue.
You can get a successful solution by switching to IMODE=6. IMODE=4 (simultaneous simulation) or IMODE=7 sequential simulation requires zero degrees of freedom. Both m.max2() and m.max3() require degrees of freedom and an optimizer to solve.
from gekko import GEKKO
import numpy as np
m = GEKKO(remote=False)
m.time = np.arange(0,20)
y = m.Var(value=5)
forcing = m.Param(value=np.arange(-5,15))
m.Equation(y.dt()== -m.max2(forcing,0) * y)
m.options.IMODE=6
m.solve(disp=True)
The equation y.dt()== -m.max2(forcing,0) * y exponentially increases beyond machine precision so I switched the equation to something that can solve.

sympy regression example, solve after partial derivative

I try to solve some matrix calculus problem with sympy and get stuck at the solver after differentiation with respect to a vector.
As a short example let's take ordinary least squares regression.
E.g. the sum of the squared differences between target y and prediction y_hat. Where the prediction y_hat = X.T * w is linear combination and thus a matrix vector multiplication.
We therefore want to minimize the LMS Error with respect to the weight vector w.
By hand we can derive that from:
Err(w) = norm(y - X.T * w)^2
follows after differentiation, setting to zero and solving for w
w_opt = (X*X.T)^-1 * X * y
How can we derive w_opt using sympy?
My rather naïve approach was:
from sympy import *
# setup matrix and vectors
X = MatrixSymbol('X',3,5)
y = MatrixSymbol('y',5,1)
w = MatrixSymbol('w',3,1)
# define error function
E = (y - X.T*w).T * (y - X.T*w)
# derivate
Edw = [E.diff(wi) for wi in w]
# solve for w
solve(Edw,w)
At solve(Edw,w) however i get the attribute error: 'Mul' object has no attribute 'shape'
I also tried to set E.as_explicit() before differentiating. This however resultet in the attribute error: 'str' object has no attribute 'is_Piecewise'
I know by calculating by hand, that after derivation the result should be -2*X*y + 2*X*X.T*w. The derivation in Edw is listed but not performed. How can I verify this step in between? My first guess was the .doit() method, which unfortunately is not defined in that case.

Resources