I want to transform LINGO code to Python GEKKO code. Here is Lingo code, lingo results and gekko codes. I cant write second and third constraints. It returns indexing error but, I dont understand why? Can someone help? (It's a graph coloring problem)
from gekko import GEKKO
import numpy as np
m = GEKKO(remote=False)
# x = m.Array(m.Var,(7,5),lb=0,ub=1,integer=True)
x = m.Array(m.Var,(6,6),lb=0,ub=1,integer=True)
y= np.array([1, 2, 3, 4, 5, 6])
country=6
arcs=np.array([[1,3],
[5,4],
[3,6],
[2,4],
[2,5],
[2,6],
[4,5],
[4,6]])
for i in range(6):
m.Minimize(y)
for i in range(6):
# for j in range(2):
# m.Equation(m.sum(x[i,j])==1)
m.Equation(m.sum(x[i,:])==1)
for k in range (6):
for i in range(8):
m.Equation(x[arcs[i,1],k]+x[arcs[i,2],k]<=1)
# m.Equation(x[arcs[i,1],k]+x[arcs[i,2],k])<=1)
# m.Equation(m.sum(x[arcs[i,1],k],x[arcs[i,2],k]))<=1)
Revised version is=
from gekko import GEKKO
import numpy as np
m = GEKKO(remote=False)
x = m.Array(m.Var,(6,6),lb=0,ub=1,integer=True)
y = m.Array(m.Var,6,lb=0,ub=1,integer=True)
y= np.array([1, 2, 3, 4, 5, 6])
country=6
arcs=np.array([[1,3],
[1,4],
[3,4],
[3,4],
[4,5],
[2,6],
[4,5],
[4,6]])
for i in range(6):
m.Minimize(y[i])
for i in range(6):
m.Equation(m.sum(x[i,:])==1)
for k in range (6):
for i in range(8):
m.Equation(x[arcs[i,0]-1,k-1]+x[arcs[i,1]-1,k-1]<=1)
for i in range(6):
m.Equation(m.sum(x[i,:]<=y[i])
m.options.solver = 1
m.solve()
print('Objective Function: ' + str(m.options.objfcnval))
print(x)
print(y)
now it gives invalid syntax error for m.solve and m options?
A couple things that you need to consider for Python:
Lists and arrays are zero-index so you need to shift them by -1 relative to the LINDO / LINGO language.
The objective function y is a list of constants. Gekko generates warnings that there are no variables in that expression.
Here is a corrected version of your Python script that you probably need to supplement with a correct objective statement and any additional equations that are needed.
from gekko import GEKKO
import numpy as np
m = GEKKO(remote=False)
x = m.Array(m.Var,(6,6),lb=0,ub=1,integer=True)
y= np.array([1, 2, 3, 4, 5, 6])
country=6
arcs=np.array([[1,3],
[1,4],
[3,4],
[2,4],
[2,5],
[2,6],
[4,5],
[4,6]])
for i in range(6):
m.Minimize(y[i])
for i in range(6):
m.Equation(m.sum(x[i,:])==1)
for k in range (6):
for i in range(8):
m.Equation(x[arcs[i,0]-1,k]\
+x[arcs[i,1]-1,k]<=1)
m.solve()
Response to Edit
The revised version is missing a closing parenthesis on the m.sum(). Here is a corrected version.
from gekko import GEKKO
import numpy as np
m = GEKKO(remote=False)
x = m.Array(m.Var,(6,6),lb=0,ub=1,integer=True)
y = m.Array(m.Var,6,lb=0,ub=1,integer=True)
y= np.array([1, 2, 3, 4, 5, 6])
country=6
arcs=np.array([[1,3],[1,4],[3,4],[3,4],[4,5],[2,6],[4,5],[4,6]])
for i in range(6):
m.Minimize(y[i])
for i in range(6):
m.Equation(m.sum(x[i,:])==1)
for k in range (6):
for i in range(8):
m.Equation(x[arcs[i,0]-1,k-1]+x[arcs[i,1]-1,k-1]<=1)
for i in range(6):
m.Equation(m.sum(x[i,:])<=y[i])
m.options.solver = 1
m.solve()
print('Objective Function: ' + str(m.options.objfcnval))
print(x)
print(y)
You can find additional tips on troubleshooting gekko applications with tutorial 18.
Related
I have 2 tensors. The first tensor is 1D (e.g. a tensor of 3 values). The second tensor is 2D, with the first dim as the IDs to first tensor in a one-many relationship (e.g. a tensor with a shape of 6, 2)
# e.g. simple example of dot product
import torch
a = torch.tensor([2, 4, 3])
b = torch.tensor([[0, 2], [0, 3], [0, 1], [1, 4], [2, 3], [2, 1]]) # 1st column is the index to tensor a, 2nd column is the value
output = [(2*2)+(2*3)+(2*1),(4*4),(3*3)+(3*1)]
output = [12, 16, 12]
Current what I have is to find the size of each id in b (e.g. [3,1,2]) then using torch.split to group them into a list of tensors and running a for loop through the groups. It is fine for a small tensor, but when the size of the tensors are in millions, with tens of thousands of arbitrary-sized groups, it became very slow.
Any better solutions?
You can use numpy.bincount or torch.bincount to sum the elements of b by key:
import numpy as np
a = np.array([2,4,3])
b = np.array([[0,2], [0,3], [0,1], [1,4], [2,3], [2,1]])
print( np.bincount(b[:,0], b[:,1]) )
# [6. 4. 4.]
print( a * np.bincount(b[:,0], b[:,1]) )
# [12. 16. 12.]
import torch
a = torch.tensor([2,4,3])
b = torch.tensor([[0,2], [0,3], [0,1], [1,4], [2,3], [2,1]])
torch.bincount(b[:,0], b[:,1])
# tensor([6., 4., 4.], dtype=torch.float64)
a * torch.bincount(b[:,0], b[:,1])
# tensor([12., 16., 12.], dtype=torch.float64)
References:
numpy.bincount official documentation;
torch.bincount official documentation;
How can I reduce a numpy array based on a key rather than an axis?
Another alternative in pytorch if gradient is needed.
import torch
a = torch.tensor([2,4,3])
b = torch.tensor([[0,2], [0,3], [0,1], [1,4], [2,3], [2,1]])
output = torch.zeros(a.shape[0], dtype=torch.long).index_add_(0, b[:, 0], b[:, 1]) * a
alternatively, torch.tensor.scatter_add also works.
I am trying to extract the lower triangular part of a SymPy matrix. Since I could not find a tril method in SymPy, I defined:
def tril (M):
m = M.copy()
for row_index in range (m.rows):
for col_index in range (row_index + 1, m.cols):
m[row_index, col_index] = 0
return (m)
It seems to work:
Is there a more elegant way to extract the lower triangular part of a SymPy matrix?
Is .copy() the recommended way to ensure the integrity of the original matrix?
In SymPy, M.lower_triangular(k) will give the lower triangular elements below the kth diagonal. The default is k=0.
In [99]: M
Out[99]:
⎡a b c⎤
⎢ ⎥
⎢d e f⎥
⎢ ⎥
⎣g h i⎦
The other answer suggest using the np.tril function:
In [100]: np.tril(M)
Out[100]:
array([[a, 0, 0],
[d, e, 0],
[g, h, i]], dtype=object)
That converts M into a numpy array - object dtype because of the symbols. And the result is also a numpy array.
Your function returns a sympy.Matrix.
In [101]: def tril (M):
...: m = M.copy()
...: for row_index in range (m.rows):
...: for col_index in range (row_index + 1, m.cols):
...: m[row_index, col_index] = 0
...: return (m)
...:
In [102]: tril(M)
Out[102]:
⎡a 0 0⎤
⎢ ⎥
⎢d e 0⎥
⎢ ⎥
⎣g h i⎦
As a general rule mixing sympy and numpy leads to confusion, if not errors. numpy is best for numeric work. It can handle non-numeric objects like symbols, but the math is hit-or-miss.
The np.tri... functions are built on the np.tri function:
In [114]: np.tri(3).astype(int)
Out[114]:
array([[1, 0, 0],
[1, 1, 0],
[1, 1, 1]])
We can make a symbolic Matrix from this:
In [115]: m1 = Matrix(np.tri(3).astype(int))
In [116]: m1
Out[116]:
⎡1 0 0⎤
⎢ ⎥
⎢1 1 0⎥
⎢ ⎥
⎣1 1 1⎦
and do element-wise multiplication:
In [117]: M.multiply_elementwise(m1)
Out[117]:
⎡a 0 0⎤
⎢ ⎥
⎢d e 0⎥
⎢ ⎥
⎣g h i⎦
np.tri works by comparing a column array with a row:
In [123]: np.arange(3)[:,None]>=np.arange(3)
Out[123]:
array([[ True, False, False],
[ True, True, False],
[ True, True, True]])
In [124]: _.astype(int)
Out[124]:
array([[1, 0, 0],
[1, 1, 0],
[1, 1, 1]])
Another answer suggests lower_triangular. It's interesting to look at its code:
def entry(i, j):
return self[i, j] if i + k >= j else self.zero
return self._new(self.rows, self.cols, entry)
It applies an i>=j test to each element. _new must be iterating on the rows and columns.
You can simply use numpy function:
import numpy as np
np.tril(M)
*of course, as noted below, you should convert back to sympy.Matrix(np.tril(M)). But it depends on what you're going to do next.
I want to have multiple types of seaborn plots using the same y axis but with different x coordinates (see image below).
I've tried doing this multiple different ways with specifying the X-axis coordinates differently but can't seem to get it to work.
Here is an example of almost working code
x=[1,2,3,3,3,4,4,5,5,6] # first violin
y=[4,4,5,5,5,5,6] # second violin
z=[5,5,6] # swarmplot over second violin
for data,label in [(x,'x'),(y,'y'),(z,'z')]:
for i in data:
c2v['value'].append(i)
c2v['key'].append(label)
data=pd.DataFrame(c2v)
data.head()
print(data.loc[data.key=='z'])
fig,ax=plt.subplots(1,figsize=(5,5),dpi=200)
ax = sns.violinplot(data=data.loc[data.key.isin(['x','y'])], x='key', y='value',palette=['honeydew','lightgreen'])
sns.swarmplot(x=['swarmplot']*len(data), y=data['value'], order=ax.get_xticklabels() + ['swarmplot'], ax=ax) #.loc[data.key=='z',:]
ax.set_xlabel('')
It produces the following image:
However, it is plotting all values associated with x/y/z instead of just z. When I slice the dataframe to only 'z' in the swarmplot as below, I get an error:
sns.swarmplot(x=['swarmplot']*len(data), y=data.loc[data.key=='z',:]['value'], order=ax.get_xticklabels() + ['swarmplot'], ax=ax)
KeyError: 'swarmplot'
Any suggestions?
To draw a second plot onto the same x-axis, you can use order= giving a list of existing tick labels, appending the new labels.
Here is an example:
import seaborn as sns
tips = sns.load_dataset('tips')
ax = sns.swarmplot(data=tips, x='day', y='total_bill')
sns.violinplot(x=['violin']*len(tips), y=tips['total_bill'], order=ax.get_xticklabels() + ['violin'], ax=ax)
ax.set_xlabel('')
The problem with the code in the new question, is that the x= and y= of the swarmplot need the same number of elements. It also seems the swarmplot resets the y limits, so I added some code to readjust those:
from matplotlib import pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
x = [1, 2, 3, 3, 3, 4, 4, 5, 5, 6] # first violin
y = [4, 4, 5, 5, 5, 5, 6] # second violin
z = [5, 5, 6] # swarmplot over second violin
data = pd.DataFrame({'value': np.concatenate([x, y, z]),
'key': ['x'] * len(x) + ['y'] * len(y) + ['z'] * len(z)})
fig, ax = plt.subplots(1, figsize=(5, 5))
ax = sns.violinplot(data=data.loc[data.key.isin(['x', 'y'])], x='key', y='value', palette=['honeydew', 'lightgreen'])
ymin1, ymax1 = ax.get_ylim()
swarm_data = data.loc[data.key == 'z', :]['value']
sns.swarmplot(x=['swarmplot'] * len(swarm_data), y=swarm_data, order=ax.get_xticklabels() + ['swarmplot'], ax=ax)
ymin2, ymax2 = ax.get_ylim()
ax.set_ylim(min(ymin1, ymin2), max(ymax1, ymax2))
ax.set_xlabel('')
ax.set_xticks(np.arange(3))
ax.set_xticklabels(['x', 'y', 'swarmplot'])
plt.show()
You can simplify things by directly using the data without creating a dataframe:
x = [1, 2, 3, 3, 3, 4, 4, 5, 5, 6] # first violin
y = [4, 4, 5, 5, 5, 5, 6] # second violin
z = [5, 5, 6] # swarmplot over second violin
fig, ax = plt.subplots(1, figsize=(5, 5))
ax = sns.violinplot(x=['x']*len(x) + ['y']*len(y), y=x + y, palette=['honeydew', 'lightgreen'])
ymin1, ymax1 = ax.get_ylim()
sns.swarmplot(x=['swarmplot'] * len(z), y=z, order=ax.get_xticklabels() + ['swarmplot'], ax=ax)
ymin2, ymax2 = ax.get_ylim()
ax.set_ylim(min(ymin1, ymin2), max(ymax1, ymax2))
ax.set_xticks(np.arange(3))
ax.set_xticklabels(['x', 'y', 'swarmplot'])
plt.show()
I tried to train a LightGBM binary classifier using the Python API the relation -
if feature > 5, then 1 else 0
import pandas as pd
import numpy as np
import lightgbm as lgb
x_train = pd.DataFrame([4, 7, 2, 6, 3, 1, 9])
y_train = pd.DataFrame([0, 1, 0, 1, 0, 0, 1])
x_test = pd.DataFrame([8, 2])
y_test = pd.DataFrame([1, 0])
lgb_train = lgb.Dataset(x_train, y_train)
lgb_eval = lgb.Dataset(x_test, y_test, reference=lgb_train)
params = { 'objective': 'binary', 'metric': {'binary_logloss', 'auc'}}
gbm = lgb.train(params, lgb_train, valid_sets=lgb_eval)
y_pred = gbm.predict(x_test, num_iteration=gbm.best_iteration)
y_pred
array([0.42857143, 0.42857143])
np.where((y_pred > 0.5), 1, 0)
array([0, 0])
Clearly it failed to predict the first test 8. Can anyone see what went wrong?
LightGBM's parameter defaults are set with the expectation of moderate-sized training data, and might not work well on extremely small datasets like the one in this question.
There are two in particular that are impacting your result:
min_data_in_leaf: minimum number of samples that must fall into a leaf node
min_sum_hessian_in_leaf: basically, the minimum contribution to the loss function for one leaf node
Setting these to the lowest possible values can force LightGBM to overfit to such a small dataset.
import pandas as pd
import numpy as np
import lightgbm as lgb
x_train = pd.DataFrame([4, 7, 2, 6, 3, 1, 9])
y_train = pd.DataFrame([0, 1, 0, 1, 0, 0, 1])
x_test = pd.DataFrame([8, 2])
y_test = pd.DataFrame([1, 0])
lgb_train = lgb.Dataset(x_train, y_train)
lgb_eval = lgb.Dataset(x_test, y_test, reference=lgb_train)
params = {
'objective': 'binary',
'metric': {'binary_logloss', 'auc'},
'min_data_in_leaf': 1,
'min_sum_hessian_in_leaf': 0
}
gbm = lgb.train(params, lgb_train, valid_sets=lgb_eval)
y_pred = gbm.predict(x_test, num_iteration=gbm.best_iteration)
y_pred
# array([6.66660313e-01, 1.89048958e-05])
np.where((y_pred > 0.5), 1, 0)
# array([1, 0])
For details on all the parameters and their defaults, see https://lightgbm.readthedocs.io/en/latest/Parameters.html.
I have the following code
a_alpha_beta = zeros((2, len( atime ) ))
for i in range( len( atime ) ):
alpha_beta = transf.clarke(signal[0][i], signal[1][i], signal[2][i])
a_alpha_beta[0][i] = alpha_beta[0][0]
a_alpha_beta[1][i] = alpha_beta[1][0]
How can I optimize the code above, for example how can I copy alpha_beta to a_alpha_beta?
I don't exactly know what the function transf.clarke does, but copy operations that you desire can be done as follows:
import numpy as np
# generate a test input
x = np.arange(0, 10).reshape((2, 5))
print x
# simply copy 2d array
y = x.copy()
print y
# some new data (i.e., alpha_beta in the original code)
z = np.array([[10, 11, 12],
[13, 14, 15]])
print z
# replace the iteration by numpy slicing to obtain the same result
x[0, :] = z[0, 0]
x[1, :] = z[1, 0]
print x