When solving models sequentially in Python GEKKO (i.e. with IMODE >= 4) fails when using the max2 and max3 functions that come with GEKKO.
This is for use cases, where np.maximum or the standard max function treat a GEKKO parameter like an array, which is not always the intended usage or can create errors when comparing against integers for example.
minimal code example:
from gekko import GEKKO
import numpy as np
m = GEKKO()
m.time = np.arange(0,20)
y = m.Var(value=5)
forcing = m.Param(value=np.arange(-5,15))
m.Equation(y.dt()== m.max2(forcing,0) * y)
m.options.IMODE=4
m.solve(disp=False)
returns:
Exception: #error: Degrees of Freedom
* Error: DOF must be zero for this mode
STOPPING...
I know from looking at the code that both max2 and max3 use inequality expressions in the equations, which understandably introduces the degrees of freedoms, so was this functionality never intended? Could there be some workaround to fix this?
Any help would be much appreciated!
Note:
I hope this is not a duplicate of How to define maximum of Intermediate and another value in Python Gekko, when using sequential solver?, but instead asking a more concise & different question, about essentially the same issue.
You can get a successful solution by switching to IMODE=6. IMODE=4 (simultaneous simulation) or IMODE=7 sequential simulation requires zero degrees of freedom. Both m.max2() and m.max3() require degrees of freedom and an optimizer to solve.
from gekko import GEKKO
import numpy as np
m = GEKKO(remote=False)
m.time = np.arange(0,20)
y = m.Var(value=5)
forcing = m.Param(value=np.arange(-5,15))
m.Equation(y.dt()== -m.max2(forcing,0) * y)
m.options.IMODE=6
m.solve(disp=True)
The equation y.dt()== -m.max2(forcing,0) * y exponentially increases beyond machine precision so I switched the equation to something that can solve.
Related
Let's say that we want to fit a nonlinear model to our data (that include measurement errors) with certain constraints on the fit parameters among themselves. Without these constraints, scipy.optimize.curve_fit does the job as it supports by default the optimization of the
chisq = sum((r / sigma) ** 2)
where r is the difference between the true and predicted value. Long story short, the user defines a function and pass it to the solver which minimizes the chisq loss. The situation gets a bit more complex when there are other restrictions imposed on the function parameters, e.g. for a linear model y=ax+b, we can demand a fixed nonlinear condition such as a^2+b=1. A function scipy.optimization.minimize is just what the doctor ordered, provided that we pass the chisq as argument. Below is the code for using minimize found here scipy.optimize with non linear constraints. I'm struggling very hard in building the chisq properly to make the code work. The most general question would how does one defines in the most pythonic way a custom loss that would be minimized?
Help please.
from math import cos, atan
import numpy as np
from scipy.optimize import minimize
def f(x):
return 0.1 * x[0] * x[1]
def ineq_constraint(x):
return x[0]**2 + x[1]**2 - (5. + 2.2 * cos(10 * atan(x[0] / x[1])))**2
con = {'type': 'ineq', 'fun': ineq_constraint}
x0 = [1, 1]
res = minimize(f, x0, method='SLSQP', constraints=con)
Since we minimize the chisq function that is found by optimizing our targeted function, I guess the chisq will be defined in a very elegant way in a form of a wrapper since for arguments it takes the targeted function, a column of true values and a column of uncertanties e.g.
chisq(target_function, (y, y_err)).
I am optimizing a simple problem where I am summing intermediate variables for a constraint where the sum needs to be lower than a certain budget.
When I print the sum, either using sum or np.sum, I get the following results:(((((((((((((((((((((((((((((i429+i430)+i431)+i432)+i433)+i434)+i435)+i436)+i437)+i438)+i439)+i440)+i441)+i442)+i443)+i444)+i445)+i446)+i447)+i448)+i449)+i450)+i451)+i452)+i453)+i454)+i455)+i456)+i457)+i458)
Here is the command to create the variables and the sum.
x = m.Array(m.Var, (len(bounds)),integer=True)
sums = [m.Intermediate(objective_inverse2(x,y)) for x,y in zip(x,reg_feats)]
My understanding of the intermediate variable is a variable which is dynamically calculated based on the value of x, which are decision variables.
Here is the summing function for the max budget constraint.
m.Equation(np.sum(sums) < max_budget)
Solving the problem returns an error saying there are no feasible solution, even through trivial solutions exist. Furthermore, removing this constraint returns a solution which naturally does not violate the max budget constraint.
What am I misunderstanding about the intermediate variable and how to sum them.
It is difficult to diagnose the problem without a complete, minimal problem. Here is an attempt to recreate the problem:
from gekko import GEKKO
import numpy as np
m = GEKKO()
nb = 5
x = m.Array(m.Var,nb,value=1,lb=0,ub=1,integer=True)
y = m.Array(m.Var,nb,lb=0)
i = [] # intermediate list
for xi,yi in zip(x,y):
i.append(m.Intermediate(xi*yi))
m.Maximize(m.sum(i))
m.Equation(m.sum(i)<=100)
m.options.SOLVER = 1
m.solve()
print(x)
print(y)
Instead of creating a list of Intermediates, the summation can also happen with the result of the list comprehension. This way, only one Intermediate value is created.
from gekko import GEKKO
import numpy as np
m = GEKKO()
nb = 5
x = m.Array(m.Var,nb,value=1,lb=0,ub=1,integer=True)
y = m.Array(m.Var,nb,lb=0)
sums = m.Intermediate(m.sum([xi*yi for xi,yi in zip(x,y)]))
m.Maximize(sums)
m.Equation(sums<=100)
m.options.SOLVER = 1
m.solve()
print(sums.value)
print(x)
print(y)
In both cases, the optimal solution is:
---------------------------------------------------
Solver : APOPT (v1.0)
Solution time : 1.560000001336448E-002 sec
Objective : -100.000000000000
Successful solution
---------------------------------------------------
[100.0]
[[1.0] [1.0] [1.0] [1.0] [1.0]]
[[20.0] [20.0] [20.0] [20.0] [20.0]]
Try using the Gekko m.sum() function to improve solution efficiency, especially for large problems.
I try to solve some matrix calculus problem with sympy and get stuck at the solver after differentiation with respect to a vector.
As a short example let's take ordinary least squares regression.
E.g. the sum of the squared differences between target y and prediction y_hat. Where the prediction y_hat = X.T * w is linear combination and thus a matrix vector multiplication.
We therefore want to minimize the LMS Error with respect to the weight vector w.
By hand we can derive that from:
Err(w) = norm(y - X.T * w)^2
follows after differentiation, setting to zero and solving for w
w_opt = (X*X.T)^-1 * X * y
How can we derive w_opt using sympy?
My rather naïve approach was:
from sympy import *
# setup matrix and vectors
X = MatrixSymbol('X',3,5)
y = MatrixSymbol('y',5,1)
w = MatrixSymbol('w',3,1)
# define error function
E = (y - X.T*w).T * (y - X.T*w)
# derivate
Edw = [E.diff(wi) for wi in w]
# solve for w
solve(Edw,w)
At solve(Edw,w) however i get the attribute error: 'Mul' object has no attribute 'shape'
I also tried to set E.as_explicit() before differentiating. This however resultet in the attribute error: 'str' object has no attribute 'is_Piecewise'
I know by calculating by hand, that after derivation the result should be -2*X*y + 2*X*X.T*w. The derivation in Edw is listed but not performed. How can I verify this step in between? My first guess was the .doit() method, which unfortunately is not defined in that case.
How can I calculate the conditional expectation of a random variable in sympy? I read this and tried:
from sympy.stats import *
v = Uniform("v",0,1)
E(v)
this returns correctly 1/2, but then:
E(v, v>1/2)
returns NaN. I also tried:
E(v, where(v > 1/2))
it returned 1/2, which is incorrect (it should be 3/4).
What am I doing wrong?
This issue (which I see you already reported) is specific to uniformly distributed random variables. (There's also an older issue involving Uniform.) For other distributions, what you did works correctly:
>>> from sympy.stats import *
>>> x = Exponential("x", 1)
>>> E(x, x < 2)
-3/(-1 + exp(2)) + exp(2)/(-1 + exp(2))
As for the uniform type, a workaround for now is to remember that conditioning a uniformly distributed random variable to some interval creates another uniformly distributed random variable.
So the value of E(v, v > 1/2) can be found by computing
E(Uniform("x", 1/2, 1))
which returns 0.75.
Caution: if working interactively, one may want to eventually import from core SymPy, in addition to its stats module. Since E stands for Euler's number 2.718... in SymPy, one may end up unable to compute expectations with
TypeError: 'Exp1' object is not callable
So one either has to be more specific about what to import, or use namespace for one or both modules. My preferred solution is
from sympy import *
import sympy.stats as st
So that st.E is expectation while E is 2.718...
I edit my code including all the parameters and variables involved:
(D is a numpy matrix imported from Python)
import pyomo
from pyomo.environ import *
from array import *
import numpy as np
import scipy as sp
from diff_matrix import D ##N=10????
print(D)
m =ConcreteModel()
...
m.n = Param(initialize = 10, within = Integers)
m.Ns = Set(initialize = range(0,value(m.n)))
m.x1 = Var(m.N, domain = Reals)
m.D = Param(m.N, m.N, initialize=D)
m.f_x1 = Var(m.N)
def f_x1_definition(model,i):
return m.f_x1[i] == sum(m.x1[j]*m.D[i,j] for j in range(value(m.n)))
m.f_x1_const = Constraint(m.Ns, rule = f_x1_definition)
But I get the next error:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Any help?
The simplest thing is to just use the Python sum() function instead of the Pyomo summation() function:
def f_x1_definition(model,i):
return model.f_x1[i] == sum(model.x1[j]*model.D[i,j] for j in range(value(model.n)))
Also, note that I reversed the order of the Pyomo Var (m.x1) and the matrix (m.D). Based on your other questions (Importing a matrix from Python to Pyomo), I am assuming that the matrix is a NumPy matrix. When multiplying a NumPy value and a Pyomo component (Var or Param), always put the Pyomo object first. This is due to a conflict between the NumPy operator overloading and the Pyomo operator overloading in current versions of Pyomo (up through at least 5.1).
EDIT 1: Note on reversing the order of operands: in your original question, it was not clear that m.D was being defined as a Pyomo Param. There is no concern with the order of Pyomo objects in expressions. The operator overloading problem mentioned above is only when multiplying NumPy objects with Pyomo components. Further, at this time (up through Pyomo 5.1), Pyomo does not support matrix algebra - that is, operations like matrix-matrix or matrix-vector products. Since every expression is a scalar expression, the ordering of the terms in a commutative operation (+, *) does not change the meaning of the expression.
EDIT 2: Your error has nothing to do with the sum/summation you originally posted. The problem is with how you are initializing your Param. At this time (up through Pyomo 5.1), you cannot directly initialize a Param from a numpy.ndarray. You need to first convert the NumPy object into a Python dictionary with something like:
m.D = Param(m.N, m.N, initialize=dict(((i,j),D[i,j]) for i in m.N for j in m.N))