Using GEKKO for Moving Horizon Estimation online 3 - gekko
This is the following question after Using GEKKO for Moving Horizon Estimation online 2.
My code:
from gekko import GEKKO
import numpy as np
import matplotlib.pyplot as plt
import matplotlib; matplotlib.use('TkAgg')
class Observer():
def __init__(self, window_size, r_init, alpha_init):
self.m = GEKKO(remote=False)
self.dt = 0.05
self.m.time = [i*self.dt for i in range(window_size)]
#Parameters
self.m.u = self.m.MV()
#Variables
self.m.r = self.m.CV(lb=0.01) # value=r_init) #ub=20 can be over 20
self.m.alpha = self.m.CV() # value=alpha_init) #ub lb for angle?
#Equations
self.m.Equation(self.m.r.dt() == -self.m.cos(self.m.alpha))
self.m.Equation(self.m.alpha.dt() == self.m.sin(self.m.alpha)/self.m.r - self.m.u) # differential equation
#Options
# self.m.options.MV_STEP_HOR = 2
self.m.options.IMODE = 5 # dynamic estimation
self.m.options.EV_TYPE = 1 #Default 1: absolute error form 2: squared error form
self.m.options.DIAGLEVEL = 0 #diagnostic level
self.m.options.NODES = 3 #nodes # collocation nodes default:2
self.m.options.SOLVER = 2 #solver_num
# STATUS = 0, optimizer doesn't adjust value
# STATUS = 1, optimizer can adjust
self.m.u.STATUS = 0
self.m.r.STATUS = 1
self.m.alpha.STATUS = 1
# FSTATUS = 0, no measurement
# FSTATUS = 1, measurement used to update model
self.m.u.FSTATUS = 1 #default
self.m.r.FSTATUS = 1
self.m.alpha.FSTATUS = 1
self.m.r.MEAS_GAP = 2
self.m.alpha.MEAS_GAP = 1
self.m.r.TR_INIT = 1
self.m.alpha.TR_INIT = 1
self.count = 0
def MHE(self, observed_state, u_data):
self.count =+ 1
self.m.u.MEAS = u_data
self.m.r.MEAS = observed_state[0]
self.m.alpha.MEAS = observed_state[1]
self.m.solve(disp=True)
return self.m.r.MODEL, self.m.alpha.MODEL
if __name__=="__main__":
FILE_PATH00 = '/home/shane16/Project/model_guard/uav_paper/adversarial/SA_PPO/src/DATA/4end_estimation_results_r.csv'
FILE_PATH01 = '/home/shane16/Project/model_guard/uav_paper/adversarial/SA_PPO/src/DATA/4end_estimation_results_alpha.csv'
FILE_PATH02 = '/home/shane16/Project/model_guard/uav_paper/adversarial/SA_PPO/src/DATA/4end_action_buffer_eps0.0_sig0.0.csv'
cycles = 250
x = np.arange(cycles) # 1...300
matrix00 = np.loadtxt(FILE_PATH00, delimiter=',')
matrix01 = np.loadtxt(FILE_PATH01, delimiter=',')
matrix02 = np.loadtxt(FILE_PATH02, delimiter=',')
vanilla_action_sigma_0 = matrix02
vanilla_estimation_matrix_r = np.zeros(cycles)
vanilla_estimation_matrix_alpha = np.zeros(cycles)
# sigma = 0.0
# vanilla model true/observed states
r_vanilla_sigma_0_true = matrix00[0, 3:] # from step 1
r_vanilla_sigma_0_observed = matrix00[1, 3:] # from step1
alpha_vanilla_sigma_0_true = matrix01[0, 3:]
alpha_vanilla_sigma_0_observed = matrix01[1, 3:]
# initialize estimator
sigma = 0.0 #1.0
solver_num = 2
nodes = 5
# for window_size in [5, 10, 20, 30, 40, 50]:
window_size = 8
observer = Observer(window_size, r_vanilla_sigma_0_observed[0], alpha_vanilla_sigma_0_observed[0])
for i in range(cycles):
if i % 100 == 0:
print('cylcle: {}'.format(i))
vanilla_observed_states = np.hstack((r_vanilla_sigma_0_observed[i], alpha_vanilla_sigma_0_observed[i])) # from current observed state
r_hat, alpha_hat = observer.MHE(vanilla_observed_states, vanilla_action_sigma_0[i]) # and current action -> estimate current state
vanilla_estimation_matrix_r[i] = r_hat
vanilla_estimation_matrix_alpha[i] = alpha_hat
#plot vanilla
plt.figure()
plt.subplot(3,1,1)
plt.title('Vanilla model_sig{}'.format(sigma))
plt.plot(x, vanilla_action_sigma_0[:cycles],'b:',label='action (w)')
plt.legend()
plt.subplot(3,1,2)
plt.ylabel('r')
plt.plot(x, r_vanilla_sigma_0_true[:cycles], 'k-', label='true_r')
plt.plot(x, r_vanilla_sigma_0_observed[:cycles], 'gx', label='observed_r')
plt.plot(x, vanilla_estimation_matrix_r, 'r--', label='time window: 10')
# plt.legend()
plt.subplot(3,1,3)
plt.xlabel('time steps')
plt.ylabel('alpha')
plt.plot(x, alpha_vanilla_sigma_0_true[:cycles], 'k-', label='true_alpha')
plt.plot(x, alpha_vanilla_sigma_0_observed[:cycles], 'gx', label='observed_alpha')
plt.plot(x, vanilla_estimation_matrix_alpha, 'r--', label='time window: {}'.format(window_size))
plt.legend()
plt.savefig('plot/revision/4estimated_STATES_vanilla_sig{}_window{}_cycles{}_solver{}_nodes{}.png'.format(sigma, window_size,cycles, solver_num, nodes))
plt.show()
I have studied and did set the problem as the example https://apmonitor.com/do/index.php/Main/EstimatorTypes.
But when I set window_size in my code over 10, the solution is not found:
(result of terminal when options.SOLVER = 0)
apm 165.132.48.158_gk_model0 <br><pre>
---------------------------------------------------------------- APMonitor, Version 1.0.1 APMonitor Optimization Suite
----------------------------------------------------------------
--------- APM Model Size ------------ Each time step contains Objects : 0 Constants : 0 Variables : 3 Intermediates: 0 Connections : 0 Equations : 2 Residuals : 2 Warning: CV( 1 ) on at cycle 1 with no MVs on Warning: CV( 2 ) on at cycle 1 with no MVs on Number of state variables: 261 Number of total equations: - 261 Number of slack variables: - 0 --------------------------------------- Degrees of freedom : 0 ---------------------------------------------- Dynamic Estimation with APOPT Solver
---------------------------------------------- Iter Objective Convergence
0 5.30547E+02 1.04898E+01
1 3.70823E+02 3.25000E-01
2 3.70724E+02 7.49963E-02
3 3.70689E+02 3.74995E-01
4 3.70716E+02 1.00000E-01
5 3.70717E+02 2.00000E-01
6 2.33359E+03 1.73260E-01
7 2.56047E+04 1.00000E-01
8 2.56261E+04 1.18140E-01
9 1.52748E+21 1.39682E+00 Iter Objective Convergence 10 1.08739E+30 1.39682E+00 11 1.06063E+30 1.39682E+00 12
1.03450E+30 1.39682E+00 13 2.40101E+32 1.39682E+00 14 2.34191E+32 1.39682E+00 15 2.28418E+32 1.39682E+00 16 1.68770E+32 1.39682E+00 17 1.64616E+32 1.39682E+00 18 1.60560E+32 1.39682E+00 19 1.72636E+32 1.39682E+00 Iter Objective Convergence 20 1.68387E+32 1.39682E+00 21
1.64239E+32 1.39682E+00 22 1.72644E+32 1.39682E+00 23 1.68395E+32 1.39682E+00 24 1.64243E+32 1.39682E+00 25 1.72645E+32 1.39682E+00 26 1.68395E+32 1.39682E+00 27 1.64243E+32 1.39682E+00 28 1.72645E+32 1.39682E+00 29 1.68395E+32 1.39682E+00 Iter Objective Convergence 30 1.64243E+32 1.39682E+00 31 1.72645E+32 1.39682E+00 32 1.68395E+32 1.39682E+00 33 1.64244E+32 1.39682E+00 34 1.72645E+32 1.39682E+00 35 1.68395E+32 1.39682E+00 36 1.64244E+32 1.39682E+00 37 1.72645E+32 1.39682E+00 38 1.68395E+32 1.39682E+00 39 1.64244E+32 1.39682E+00 Iter Objective Convergence 40 1.72645E+32 1.39682E+00 41
1.68395E+32 1.39682E+00 42 1.64244E+32 1.39682E+00 43 1.72645E+32 1.39682E+00 44 1.68395E+32 1.39682E+00 45 1.64245E+32 1.39682E+00 46 1.72645E+32 1.39682E+00 47 1.68395E+32 1.39682E+00 48 1.64243E+32 1.39682E+00 49 1.72645E+32 1.39682E+00 Iter Objective Convergence 50 1.68395E+32 1.39682E+00 51 1.64243E+32 1.39682E+00 52 1.72645E+32 1.39682E+00 53 1.68395E+32 1.39682E+00 54 1.64244E+32 1.39682E+00 55 1.72645E+32 1.39682E+00 56 1.68395E+32 1.39682E+00 57 1.64244E+32 1.39682E+00 58 1.72645E+32 1.39682E+00 59 1.68395E+32 1.39682E+00 Iter Objective Convergence 60 1.64244E+32 1.39682E+00 61
1.72645E+32 1.39682E+00 62 1.68395E+32 1.39682E+00 63 1.64243E+32 1.39682E+00 64 1.72645E+32 1.39682E+00 65 1.68395E+32 1.39682E+00 66 1.64243E+32 1.39682E+00 67 1.72645E+32 1.39682E+00 68 1.68395E+32 1.39682E+00 69 1.64244E+32 1.39682E+00 Iter Objective Convergence 70 1.72645E+32 1.39682E+00 71 1.68395E+32 1.39682E+00
... 81
1.64244E+32 1.39682E+00 82 1.72645E+32 1.39682E+00 83 1.68395E+32 1.39682E+00 84 1.64243E+32 1.39682E+00 85 1.72645E+32 1.39682E+00 86 1.68395E+32 1.39682E+00 87 1.64243E+32 1.39682E+00 88 1.72645E+32 1.39682E+00 89 1.68395E+32 1.39682E+00 Iter Objective Convergence 90 1.64243E+32 1.39682E+00 91 1.72645E+32 1.39682E+00 92 1.68395E+32 1.39682E+00 93 1.64243E+32 1.39682E+00 94 1.72645E+32 1.39682E+00 95 1.68395E+32 1.39682E+00 96 1.64243E+32 1.39682E+00 97 1.72645E+32 1.39682E+00 98 1.68395E+32 1.39682E+00 99 1.64243E+32 1.39682E+00 Iter Objective Convergence 100 1.72645E+32 1.39682E+00 101
1.68395E+32 1.39682E+00 102 1.64244E+32 1.39682E+00 103 1.72645E+32 1.39682E+00 104 1.68395E+32 1.39682E+00 105 1.64244E+32 1.39682E+00 106 1.72645E+32 1.39682E+00 107 1.68395E+32 1.39682E+00 108 1.64244E+32 1.39682E+00 109 1.72645E+32 1.39682E+00 Iter Objective Convergence 110 1.68395E+32 1.39682E+00 111 1.64243E+32 1.39682E+00 112 1.72645E+32 1.39682E+00 113 1.68395E+32 1.39682E+00 114 1.64243E+32 1.39682E+00 115 1.72645E+32 1.39682E+00 116 1.68395E+32 1.39682E+00 117 1.64244E+32 1.39682E+00 118 1.72645E+32 1.39682E+00 119 1.68395E+32 1.39682E+00 Iter Objective Convergence 120 1.64244E+32 1.39682E+00 121
1.72645E+32 1.39682E+00 122 1.68395E+32 1.39682E+00 123 1.64243E+32 1.39682E+00 124 1.72645E+32 1.39682E+00 125 1.68395E+32 1.39682E+00 126 1.64244E+32 1.39682E+00 127 1.72645E+32 1.39682E+00 128 1.68395E+32 1.39682E+00 129 1.64243E+32 1.39682E+00 Iter Objective Convergence 130 1.72645E+32 1.39682E+00 131 1.68395E+32 1.39682E+00 132 1.64244E+32 1.39682E+00 133 1.72645E+32 1.39682E+00 134 1.68395E+32 1.39682E+00 135 1.64243E+32 1.39682E+00 136 1.72645E+32 1.39682E+00 137 1.68395E+32 1.39682E+00 138 1.64243E+32 1.39682E+00 139 1.72645E+32 1.39682E+00 Iter Objective Convergence 140 1.68395E+32 1.39682E+00 141
1.64243E+32 1.39682E+00 142 1.72645E+32 1.39682E+00 143 1.68395E+32 1.39682E+00 144 1.64243E+32 1.39682E+00 145 1.72645E+32 1.39682E+00 146 1.68395E+32 1.39682E+00 147 1.64244E+32 1.39682E+00 148 1.72645E+32 1.39682E+00 149 1.68395E+32 1.39682E+00 Iter Objective Convergence 150 1.64243E+32 1.39682E+00 151 1.72645E+32 1.39682E+00 152 1.68395E+32 1.39682E+00 153 1.64244E+32 1.39682E+00 154 1.72645E+32 1.39682E+00 155 1.68395E+32 1.39682E+00 156 1.64244E+32 1.39682E+00 157 1.72645E+32 1.39682E+00 158 1.68395E+32 1.39682E+00 159 1.64243E+32 1.39682E+00 Iter Objective Convergence 160 1.72645E+32 1.39682E+00 161
1.68395E+32 1.39682E+00 162 1.64243E+32 1.39682E+00 163 1.72645E+32 1.39682E+00 164 1.68395E+32 1.39682E+00 165 1.64244E+32 1.39682E+00 166 1.72645E+32 1.39682E+00 167 1.68395E+32 1.39682E+00 168 1.64244E+32 1.39682E+00 169 1.72645E+32 1.39682E+00 Iter Objective Convergence 170 1.68395E+32 1.39682E+00 171 1.64243E+32 1.39682E+00 172 1.72645E+32 1.39682E+00 173 1.68395E+32 1.39682E+00 174 1.64244E+32 1.39682E+00 175 1.72645E+32 1.39682E+00 176 1.68395E+32 1.39682E+00 177 1.64243E+32 1.39682E+00 178 1.72645E+32 1.39682E+00 179 1.68395E+32 1.39682E+00 Iter Objective Convergence 180 1.64244E+32 1.39682E+00 181
1.72645E+32 1.39682E+00 182 1.68395E+32 1.39682E+00 183 1.64243E+32 1.39682E+00 184 1.72645E+32 1.39682E+00 185 1.68395E+32 1.39682E+00 186 1.64243E+32 1.39682E+00 187 1.72645E+32 1.39682E+00 188 1.68395E+32 1.39682E+00 189 1.64243E+32 1.39682E+00 Iter Objective Convergence 190 1.72645E+32 1.39682E+00 191 1.68395E+32 1.39682E+00 192 1.64243E+32 1.39682E+00 193 1.72645E+32 1.39682E+00 194 1.68395E+32 1.39682E+00 195 1.64244E+32 1.39682E+00 196 1.72645E+32 1.39682E+00 197 1.68395E+32 1.39682E+00 198 1.64243E+32 1.39682E+00 199 1.72645E+32 1.39682E+00 Iter Objective Convergence 200 1.68395E+32 1.39682E+00 201
1.64244E+32 1.39682E+00 202 1.72645E+32 1.39682E+00 203 1.68395E+32 1.39682E+00 204 1.64244E+32 1.39682E+00 205 1.72645E+32 1.39682E+00 206 1.68395E+32 1.39682E+00 207 1.64243E+32 1.39682E+00 208 1.72645E+32 1.39682E+00 209 1.68395E+32 1.39682E+00 Iter Objective Convergence 210 1.64243E+32 1.39682E+00 211 1.72645E+32 1.39682E+00 212 1.68395E+32 1.39682E+00 213 1.64244E+32 1.39682E+00 214 1.72645E+32 1.39682E+00 215 1.68395E+32 1.39682E+00 216 1.64244E+32 1.39682E+00 217 1.72645E+32 1.39682E+00 218 1.68395E+32 1.39682E+00 219 1.64243E+32 1.39682E+00 Iter Objective Convergence 220 1.72645E+32 1.39682E+00 221
1.68395E+32 1.39682E+00 222 1.64244E+32 1.39682E+00 223 1.72645E+32 1.39682E+00 224 1.68395E+32 1.39682E+00 225 1.64243E+32 1.39682E+00 226 1.72645E+32 1.39682E+00 227 1.68395E+32 1.39682E+00 228 1.64244E+32 1.39682E+00 229 1.72645E+32 1.39682E+00 Iter Objective Convergence 230 1.68395E+32 1.39682E+00 231 1.64243E+32 1.39682E+00 232 1.72645E+32 1.39682E+00 233 1.68395E+32 1.39682E+00 234 1.64243E+32 1.39682E+00 235 1.72645E+32 1.39682E+00 236 1.68395E+32 1.39682E+00 237 1.64243E+32 1.39682E+00 238 1.72645E+32 1.39682E+00 239 1.68395E+32 1.39682E+00 Iter Objective Convergence 240 1.64243E+32 1.39682E+00 241
1.72645E+32 1.39682E+00 242 1.68395E+32 1.39682E+00 243 1.64244E+32 1.39682E+00 244 1.72645E+32 1.39682E+00 245 1.68395E+32 1.39682E+00 246 1.64243E+32 1.39682E+00 247 1.72645E+32 1.39682E+00 248 1.68395E+32 1.39682E+00 249 1.64244E+32 1.39682E+00 Iter Objective Convergence 250 1.72645E+32 1.39682E+00 Maximum iterations --------------------------------------------------- Solver : APOPT (v1.0) Solution time : 5.02409999999873 sec Objective : 217.772371342953 Unsuccessful with error code 0
--------------------------------------------------- ---------------------------------------------- Dynamic Estimation with BPOPT Solver ----------------------------------------------
----------------------------------------------------- BPOPT Solver v1.0.6
----------------------------------------------------- Adjusting variables to interior region of bounds Iter Objective Convergence
0 -1.85748E+02 3.38018E+01
1 -1.05777E+02 2.71532E+01
2 1.26905E+02 2.76867E+02
3 4.04640E+03 7.58592E+02
4 1.22410E+08 1.22407E+04
5 1.88476E+03 1.95147E+04
-1.77050E+00 4.25888E+02
9.23784E+02 9.96666E+03
1.38670E+03 1.47370E+04
1.61835E+03 1.71234E+04
1.74542E+03 1.83168E+04
6 1.81543E+03 1.89153E+04
2.56695E+02 1.43715E+03
1.01483E+03 1.00785E+04
1.40039E+03 1.44630E+04
1.59401E+03 1.66629E+04
1.69819E+03 1.77867E+04
7 1.75705E+03 1.83500E+04
8 6.72094E+08 6.72094E+04
5.65658E+02 6.07154E+03
3.69827E+03 3.62246E+04
5.28094E+03 5.14597E+04
6.09958E+03 5.93489E+04
6.50835E+03 6.32875E+04
9 6.71243E+03 6.52535E+04 10 3.42908E+02 3.74813E+03 11 8.25957E+03 3.79110E+03 12 4.75089E+07 4.75089E+03 13 1.75564E+04 1.44171E+04
7.14522E+07 7.14522E+03
9.29139E+07 9.29140E+03
1.14506E+08 1.14506E+04
1.26204E+08 1.26204E+04
1.32081E+08 1.32081E+04 14 1.37395E+08 1.37395E+04 Iter Objective Convergence 15 9.73361E+07 9.73356E+03 16
5.10815E+08 5.10812E+04
3.34519E+03 1.54614E+04
5.68490E+03 3.27431E+04
6.90872E+03 4.19233E+04
7.53574E+03 4.66639E+04
7.82826E+03 4.88168E+04 17 7.97497E+03 4.98979E+04 18 1.02392E+09 1.02392E+05 19 7.77468E+09 7.77468E+05 20 9.02530E+09 1.30137E+06 21 8.22402E+12 8.22402E+08
8.27849E+07 7.85323E+08
6.67868E+07 6.46502E+08
6.68851E+07 6.58066E+08
7.40486E+07 7.34991E+08
7.80379E+07 7.77529E+08 22 8.00766E+07 7.99239E+08 23 8.85462E+12 8.85424E+08
3.10775E+08 2.89271E+08
3.88079E+08 5.70603E+08
4.28405E+08 7.28006E+08
4.48567E+08 8.06703E+08
4.58649E+08 8.46059E+08 24 4.63690E+08 8.65737E+08 25 1.86029E+07 1.48320E+08
1.17713E+10 1.17712E+06
7.44218E+11 7.44216E+07
1.11371E+12 1.11370E+08
1.29845E+12 1.29844E+08
1.39082E+12 1.39082E+08 26 1.43700E+12 1.43700E+08
9.97937E+09 9.97933E+05
7.20196E+11 7.20194E+07
1.07860E+12 1.07860E+08
1.25780E+12 1.25780E+08
1.34740E+12 1.34740E+08 27 1.39221E+12 1.39220E+08
4.87457E+09 4.87453E+05
6.97347E+11 6.97346E+07
1.04477E+12 1.04477E+08
1.21849E+12 1.21848E+08
1.30535E+12 1.30534E+08 28 1.34878E+12 1.34877E+08
7.02507E+04 6.33467E+05
8.46417E+06 6.74433E+07
1.26923E+07 1.01160E+08
1.48064E+07 1.18019E+08
1.58634E+07 1.26448E+08 29 1.63920E+07 1.30662E+08 Iter Objective Convergence 30 6.88703E+05 1.60107E+06
1.15163E+08 8.31826E+05
1.68185E+08 1.21658E+06
1.94691E+08 1.40891E+06
2.07914E+08 1.50487E+06
2.14555E+08 1.55306E+06 31 2.17836E+08 1.57686E+06 *** WARNING MESSAGE FROM SUBROUTINE MA27BD *** INFO(1) = 3
MATRIX IS SINGULAR. RANK= 683 Problem with linear solver, INFO: 3 --------------------------------------------------- Solver : BPOPT (v1.0) Solution time : 4.249999999956344E-002 sec Objective : 528860.935305211 Unsuccessful with error code 0 ---------------------------------------------------
********************************************** Dynamic Estimation with Interior Point Solver
**********************************************
Info: Exact Hessian
****************************************************************************** This program contains Ipopt, a library for large-scale nonlinear optimization. Ipopt is released as open source code under the Eclipse Public License (EPL).
For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************
This is Ipopt version 3.12.10, running with linear solver ma57.
Number of nonzeros in equality constraint Jacobian...: 257 Number of nonzeros in inequality constraint Jacobian.: 324 Number of nonzeros in Lagrangian Hessian.............: 54
Total number of variables............................: 261
variables with only lower bounds: 180
variables with lower and upper bounds: 0
variables with only upper bounds: 0 Total number of equality constraints.................: 99 Total number of inequality constraints...............: 162
inequality constraints with only lower bounds: 162 inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 0 8.8000164e-01 1.05e+01 9.00e+00 0.0 0.00e+00 -
0.00e+00 0.00e+00 0 1 1.0609900e+00 1.05e+01 9.02e+00 -6.7 3.88e+01 - 1.84e-03 1.78e-03h 1 2 9.3143493e+00 1.02e+01 2.21e+01 0.4 3.87e+01 - 1.79e-02 1.98e-02f 1 3 9.3142593e+00 1.02e+01 1.15e+04 -0.3 3.79e+01 - 2.86e-01 2.02e-04h 1 4r 9.3142593e+00 1.02e+01 1.00e+03 1.0 0.00e+00 - 0.00e+00 2.53e-07R 4 5r 5.3654124e+01 8.37e+00 1.16e+03 1.6 2.57e+02 - 2.05e-03 7.22e-03f 1 6 5.3675353e+01 8.36e+00 2.99e+02 -6.2 4.01e+01 - 2.12e-01 7.00e-04h 1 7r 5.3675353e+01 8.36e+00 9.99e+02 0.9 0.00e+00 - 0.00e+00 4.38e-07R 5 8r 1.6618000e+02 2.86e+00 9.90e+02 1.0 6.80e+02 - 2.20e-02 8.16e-03f 1 9 1.6613771e+02 2.86e+00 7.01e+02 -6.2 3.78e+01 - 4.05e-01 5.72e-04h 1 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 10r 1.6613771e+02
2.86e+00 9.99e+02 0.5 0.00e+00 - 0.00e+00 3.58e-07R 5 11r 2.2518286e+02 1.89e+00 9.97e+02 1.8 1.58e+03 - 5.31e-03 1.80e-03f 1 12 2.2512388e+02 1.89e+00 1.12e+03 -6.1 3.77e+01 - 4.42e-01 3.90e-04f 1 13r 2.2512388e+02 1.89e+00 1.00e+03 0.3 0.00e+00 - 0.00e+00 4.87e-07R 4 14r 2.2721384e+02 1.87e+00 9.91e+02 -5.9 6.07e+01 - 3.73e-03 7.55e-03f 1 15r 2.2764670e+02 6.34e-01 9.89e+02 1.5 2.10e+04 - 3.96e-03 1.18e-03f 1 16 2.2763736e+02 6.34e-01 5.40e+03 -6.1 1.48e+02 - 4.36e-01 7.99e-05h 1 17r 2.2763736e+02 6.34e-01 1.00e+03 -0.0 0.00e+00 - 0.00e+00 4.00e-07R 2 18r 2.3088007e+02 5.58e-01 9.95e+02 1.9 9.63e+02 - 7.10e-03 2.87e-03f 1 19 2.3089464e+02 5.58e-01 5.63e+04 0.1 1.19e+02 - 4.79e-01 4.76e-04h 1 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 20r 2.3089464e+02 5.58e-01 1.00e+03
-0.3 0.00e+00 - 0.00e+00 2.98e-07R 5 21r 2.3319530e+02 5.24e-01 1.31e+03 1.7 1.33e+03 - 1.50e-02 3.54e-03f 1 22r 2.5899287e+02 3.78e-01 1.77e+03 1.9 1.04e+02 - 5.59e-02 4.32e-02f 1 23 2.5842489e+02 3.78e-01 6.85e+02 0.2 6.86e+01 - 8.90e-01 1.48e-03f 3 24 2.5843511e+02 3.82e-01 3.36e+04 0.2 6.84e+01 - 2.20e-01 4.46e-03f 1 25 2.5843515e+02 3.82e-01 1.36e+08 0.2 6.48e+01 - 1.80e-01 4.47e-05h 1 26r 2.5843515e+02 3.82e-01 1.00e+03 0.2 0.00e+00 - 0.00e+00 2.24e-07R 2 27r 2.6284498e+02 2.08e-01 1.34e+03 1.9 1.55e+02 - 7.63e-02 5.34e-03f 1 28 2.6244784e+02 2.08e-01 4.34e+03 0.2 4.59e+01 - 9.14e-01 9.61e-04h 1 29r 2.6244784e+02 2.08e-01 1.00e+03 0.2 0.00e+00 - 0.00e+00 3.01e-07R 6 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 30r
2.7090035e+02 2.28e-01 9.50e+02 1.6 7.37e+01 - 2.96e-01 1.63e-02f 1 31r 2.7157286e+02 2.27e-01 9.22e+02 0.4 2.45e+00 - 4.94e-01 3.05e-02f 1 32r 2.7252200e+02 2.29e-01 5.15e+02 -1.1 1.18e+00 - 8.32e-01 4.55e-01f 1 33r 2.8772420e+02 2.25e-01 9.40e+01 -0.7 2.22e+00 - 9.83e-01 9.15e-01f 1 34r 2.8769913e+02 1.95e+00 1.26e+02 -6.6 1.40e+03 - 1.66e-04 1.08e-03f 1 35r 2.9223102e+02 1.54e+00 1.29e+03 -1.2 1.05e+00 0.0 3.03e-01 8.71e-01f 1 36r 2.9225465e+02 1.07e+00 9.01e+02 -0.9 5.35e-01 2.2 4.81e-01 6.15e-01f 1 37r 2.9231013e+02 5.11e-01 1.06e+03 -0.4 2.19e-01 2.7 1.36e-01 1.00e+00f 1 38r 2.9231181e+02 4.23e-01 9.15e+02 -0.6 1.86e-01 3.1 9.06e-02 1.77e-01f 1 39r 2.9231198e+02 4.04e-01 1.12e+03 -0.6 2.11e-01 3.5 2.88e-01 4.46e-02f 1 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 40r 2.9231360e+02 3.01e-01
9.53e+02 -0.6 2.60e-01 3.0 6.51e-02 1.35e-01f 1 41r 2.9231693e+02 2.72e-01 1.66e+03 -0.6 2.67e-01 2.6 1.00e+00 9.33e-02f 1 42r 2.9232719e+02 1.98e-01 2.41e+02 -0.6 1.29e-01 3.0 2.20e-01 7.65e-01h 1 43r 2.9243143e+02 2.09e-01 5.65e+02 0.3 2.98e-01 2.5 1.00e+00 2.52e-01f 1 44r 2.9246350e+02 2.10e-01 8.61e+02 -0.4 2.92e-01 2.0 9.92e-01 2.01e-01f 1 45r 2.9245480e+02 2.09e-01 1.93e+02 -1.4 1.20e-01 2.5 1.00e+00 7.37e-01f 1 46r 2.9259452e+02 2.03e-01 1.27e+03 0.2 1.80e+00 2.0 7.20e-01 1.20e-01f 1 47r 2.9277900e+02 2.04e-01 7.72e+02 -0.3 1.25e+00 1.5 3.93e-01 2.34e-01f 1 48r 2.9280994e+02 2.04e-01 5.33e+02 -0.3 1.78e+00 1.0 3.76e-01 1.16e-02h 1 49r 2.9962583e+02 2.28e-01 3.45e+02 -0.3 7.89e-01 0.5 7.13e-01 1.00e+00f 1 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 50r 3.2677031e+02 4.59e-01 7.44e+01
-0.3 6.06e-01 - 1.00e+00 7.82e-01f 1 51r 4.2678794e+02 4.44e-01 1.12e+01 -0.3 4.39e+00 - 1.00e+00 1.00e+00f 1 52r 3.4495773e+02 1.99e-01 1.01e+01 -1.0 2.60e+00 - 8.72e-01 1.00e+00f 1 53r 3.4520580e+02 2.23e-01 3.63e+02 -1.0 4.22e-01 0.1 1.00e+00 3.00e-01f 2 54r 3.2340242e+02 2.25e-01 2.00e+00 -1.1 1.11e+00 - 1.00e+00 1.00e+00f 1 55r 2.9817629e+02 2.31e-01 6.43e+01 -2.2 7.79e-01 - 9.99e-01 7.57e-01f 1 56r 2.8673872e+02 2.25e-01 4.58e-01 -2.7 2.41e-01 - 1.00e+00 1.00e+00f 1 57r 2.7428826e+02 2.23e-01 8.35e-02 -3.9 3.26e-01 - 1.00e+00 1.00e+00f 1 58r 2.7154701e+02 2.21e-01 1.36e+00 -4.8 2.86e+00 - 9.99e-01 6.82e-01f 1 59r 2.7155377e+02 2.21e-01 5.33e-03 -4.2 2.28e-02 -0.4 1.00e+00 1.00e+00h 1 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 60r
2.7041602e+02 2.21e-01 8.88e+00 -4.9 4.62e+00 - 9.97e-01 1.64e-01h 1 61r 2.7039401e+02 2.21e-01 1.33e-02 -6.0 7.91e-02 -0.9 1.00e+00 9.97e-01h 1 62r 2.7037446e+02 2.21e-01 1.52e-02 -7.6 3.26e-01 -1.4 1.00e+00 1.00e+00f 1 63r 2.7036542e+02 2.21e-01 1.24e-01 -8.6 1.25e+00 -1.8 1.00e+00 1.00e+00f 1 64r 2.7035672e+02 2.21e-01 3.74e+00 -9.0 7.13e+00 -2.3 1.00e+00 1.00e+00f 1 65r 2.7035462e+02 2.21e-01 7.83e-01 -8.7 3.27e+00 -1.9 1.00e+00 1.00e+00h 1 66r 2.7034622e+02 5.19e-01 3.07e+01 -8.5 2.10e+01 -2.4 1.00e+00 1.00e+00f 1 67r 2.7034197e+02 2.66e-01 7.12e+00 -8.8 1.02e+01 -1.9 1.00e+00 1.00e+00f 1 68r 2.7033259e+02 3.97e+01 2.03e+02 -8.9 9.24e+01 -2.4 1.00e+00 1.00e+00f 1 69r 2.6360194e+02 1.39e+01 5.98e+02 -7.3 4.14e+01 - 1.00e+00 1.00e+00h 1 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 70r 2.6287479e+02 1.17e+01
4.25e+02 -7.1 1.33e+01 - 1.00e+00 1.00e+00h 1 71r 2.6399459e+02 4.20e+00 3.71e+02 -6.2 1.24e+01 - 1.00e+00 6.63e-01h 1 72r 2.6277485e+02 2.15e-01 1.34e+00 -6.7 5.00e+00 - 1.00e+00 1.00e+00h 1 73r 2.6249804e+02 2.14e-01 8.71e-02 -8.0 2.35e-01 - 1.00e+00 1.00e+00h 1 74r 2.6246138e+02 2.14e-01 1.06e-01 -9.0 1.43e-01 - 1.00e+00 1.00e+00h 1 75r 2.6248456e+02 2.15e-01 3.45e-03 -9.0 2.17e-02 - 1.00e+00 1.00e+00h 1 76r 2.6248458e+02 2.15e-01 3.74e-03 -9.0 2.15e-03 - 1.00e+00 1.00e+00h 1 77r 2.6248458e+02 2.15e-01 1.26e-04 -9.0 4.26e-04 - 1.00e+00 1.00e+00h 1
Number of Iterations....: 77
(scaled) (unscaled) Objective...............: 2.6248457642008947e+02
2.6248457642008947e+02 Dual infeasibility......: 1.9999999998163066e+01 1.9999999998163066e+01 Constraint violation....: 2.1450238157252022e-01 2.1450238157252022e-01 Complementarity.........: 1.0000004294468675e-09
1.0000004294468675e-09 Overall NLP error.......: 1.9999999998163066e+01 1.9999999998163066e+01
Number of objective function evaluations = 118 Number of objective gradient evaluations = 29 Number of equality constraint evaluations = 118 Number of inequality constraint evaluations = 118 Number of equality constraint Jacobian evaluations = 87 Number of inequality constraint Jacobian evaluations = 87 Number of Lagrangian Hessian evaluations
= 78 Total CPU secs in IPOPT (w/o function evaluations) = 0.131 Total CPU secs in NLP function evaluations = 0.053
EXIT: Converged to a point of local infeasibility. Problem may be infeasible. An error occured. The error code is 2
and if I set window_size = 5, the estimation result does not follow true states:
What am I doing wrong?
How should I fix this?
Sorry if I'm asking something that is easily known through studying example, but I still don't get the solution.
Thank you for your kind instruction.
Sincerely, Shane K.
The objective function value from the solver iterations gives a clue that one of the variables is getting very large. It could be related to the divide-by-zero issue that is in the response to Using GEKKO for Moving Horizon Estimation online 2.
One of the equations may be the source of the problem:
self.m.alpha.dt() == self.m.sin(self.m.alpha)/self.m.r - self.m.u
The initial value of r is zero (default) because no initial value is provided when it is declared as self.m.r = self.m.CV(lb=0). A comment suggests that it was formerly initialized with value r_init. The zero value creates a divide-by-zero for that equation. Try rearranging the equation into an equivalent form that avoids the potential for divide-by-zero either with the initial guess or when the solver is iterating.
self.m.r*self.m.alpha.dt() == self.m.sin(self.m.alpha) - self.m.r*self.m.u
The initialization strategies article also gives suggestions on methods to obtain a successful solution:
Safdarnejad, S.M., Hedengren, J.D., Lewis, N.R., Haseltine, E., Initialization Strategies for Optimization of Dynamic Systems, Computers and Chemical Engineering, 2015, Vol. 78, pp. 39-50, DOI: 10.1016/j.compchemeng.2015.04.016.
Related
Create a file according to sort contend
I have a list of more than 100000 records. per example the values from 21 to 84 are continuous, then it will be 21-84 but if it is not continuous as the case 84 87, then it need to be 84,87 separated by , at beginning of each line will be the value 11111. The values from the list will be in the column range of 21 to 80 with, at last. The length of each row need to be maximum 80. here is the input file. 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 87 85 86 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 108 111 109 112 110 113 115 114 117 116 118 124 125 120 122 123 126 132 127 133 128 130 131 135 136 137 138 139 140 141 142 143 144 145 146 148 147 149 150 151 152 153 154 155 156 158 157 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 184 183 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 214 here in the output file desired. 111111 21-84,87,85-86,88-106,108,111,109,112,110,113,115,114,117, 111111 116,118,124-125,120,122-123,126,132,127,133,128,130-131, 111111 135-146,148,147,149-156,158,157,159-182,184,183,185-212,214, thanks in advance
Presented without explanation: check the man pages for the commands used and come back with questions:
awk '
function printrange() { print start (start == last ? "" : "-" last) }
NR == 1 {start=last=$1; next}
$1 == last+1 {last=$1; next}
{printrange(); start=last=$1}
END {printrange()}
' file | paste -sd" " | fold -sw 60 | tr ' ' ',' | sed 's/^/111111 /'
111111 21-84,87,85-86,88-106,108,111,109,112,110,113,115,114,117,
111111 116,118,124-125,120,122-123,126,132,127,133,128,130-131,
111111 135-146,148,147,149-156,158,157,159-182,184,183,185-212,214
Keep lines based on ratio between lines
I have a sort -g k9 command on a file that gives me this in the bash standard output: 55.19 645 156 15 9 520 58 702 0.0 661 55.50 636 159 16 9 520 58 693 0.0 654 55.19 645 156 15 9 520 58 702 0.0 658 56.52 644 147 16 9 520 59 701 0.0 669 55.97 645 151 15 9 520 65 709 0.0 672 55.97 645 151 15 9 520 65 709 4e-124 674 28.32 671 301 32 1 507 48 702 3e-49 183 28.32 671 301 32 1 507 47 701 3e-49 183 31.40 516 247 24 86 507 196 698 1e-46 176 31.41 519 243 25 86 507 196 698 5e-46 175 27.72 588 290 26 19 481 98 675 2e-39 154 30.56 337 170 17 101 413 302 598 5e-20 96.3 30.56 337 170 17 101 413 302 598 8e-20 95.5 I would like to cut my data based on the 9th column. The idea would be to compare the value of the 9th column on line i, divide it by the value of the 9th column on line i+1, and if the ratio is 0 OR 0/0 OR > 1e-50, line i and i+1 are kept. As soon as one of these conditions is not filled, stop reading. The desired output would be: 55.19 645 156 15 9 520 58 702 0.0 661 55.50 636 159 16 9 520 58 693 0.0 654 55.19 645 156 15 9 520 58 702 0.0 658 56.52 644 147 16 9 520 59 701 0.0 669 55.97 645 151 15 9 520 65 709 0.0 672 55.97 645 151 15 9 520 65 709 4e-124 674 I can obtain this output with head -n 6 but this is obviously not based on the condition on values in the 9th column. Please note that the values are in 'scientific' format. I know how to do this in Python (write the standard output to a file, calculate ratios, etc.) but for commodity reasons I'd prefer a shell-based solution (with awk or sort for instance) although I don't know if that's possible. Thanks for your help!
Just exit the script when the condition is not accomplished; otherwise, print the previous line and store the 9th field to compare on the next loop:
$ awk '($9 && prev/$9>1e-50) {exit} {print stored; prev=$9; stored=$0}' file
55.19 645 156 15 9 520 58 702 0.0 661
55.50 636 159 16 9 520 58 693 0.0 654
55.19 645 156 15 9 520 58 702 0.0 658
56.52 644 147 16 9 520 59 701 0.0 669
55.97 645 151 15 9 520 65 709 0.0 672
55.97 645 151 15 9 520 65 709 4e-124 674
awk find the closest match of a list in a matrix
I look for common elements in two files or which row of a matrix has the most elements from a given row. what I understood until now is how to compare fields. I receive the lines which hold the same value in the same fieldnumber.
But how can I open the search to the other field numbers?
awk 'NR==FNR{a[$1];next}$1 in a{print $1" "FNR}' file1 file2
104 3
Expected output:
104 3 111 4 117 2 134 2 148 - 156 4 166 4 176 3 186 - 198 1 221 6 236 -
best match row 4 with 3 elements common.
file 1
104 111 117 134 148 156 166 176 186 198 221 236
file 2
102 108 116 124 132 141 151 162 173 185 198 211
103 109 117 125 134 143 153 163 175 187 200 213
104 110 118 126 135 144 154 165 176 188 201 215
105 111 119 127 136 145 156 166 178 190 203 217
106 112 120 128 137 147 157 168 179 192 205 219
107 113 121 130 139 148 158 169 181 193 207 221
108 114 122 131 140 150 160 171 183 195 208 200
This solution assumes 1) that file1 contains unique values as shown in the provided example and 2) there is only one top ranked line in file2.
awk -v string=$(cat file1 | tr " " ",") \
'{split(string,array,","); cnt=0;
for(i in array) {for(j=1;j<=NF;j++) if(array[i]==$j) cnt++};
if(cnt>cntmax) {cntmax=cnt; NRmax=NR}} END{print NRmax}' file2
4
Parsing the wbxml get from Exchange2013 server
Below is a full section of bytes stream I got from Exchange2013 server after sending activesync command 'itemoperations' from iPhone. 31 139 8 0 0 0 0 0 4 0 237 189 7 96 28 73 150 37 38 47 109 202 123 127 74 245 74 215 224 116 161 8 128 96 19 36 216 144 64 16 236 193 136 205 230 146 236 29 105 71 35 41 171 42 129 202 101 86 101 93 102 22 64 204 237 157 188 247 222 123 239 189 247 222 123 239 189 247 186 59 157 78 39 247 223 255 63 92 102 100 1 108 246 206 74 218 201 158 33 128 170 200 31 63 126 124 31 63 34 126 237 95 243 167 127 141 95 227 183 58 253 226 215 222 253 53 126 205 23 207 248 199 175 241 155 255 196 175 125 255 119 191 151 61 220 167 127 240 247 111 245 123 253 26 191 249 119 127 237 54 127 215 222 93 149 89 177 196 71 207 127 237 215 95 61 124 253 229 79 31 191 251 226 233 233 15 126 234 167 143 175 190 248 233 183 87 95 60 61 198 255 119 190 124 243 19 59 47 62 255 226 7 191 207 155 159 250 233 23 139 223 231 222 139 167 211 221 23 63 248 226 250 167 126 250 247 217 161 191 247 126 234 233 239 115 255 197 226 171 189 159 250 233 239 44 94 252 244 23 244 115 122 253 226 167 207 246 190 120 67 127 63 125 187 67 237 174 95 44 78 247 94 188 249 125 126 240 226 7 63 177 75 144 232 253 179 123 47 126 154 127 18 220 47 222 125 177 248 234 7 218 159 254 255 212 251 221 252 127 26 126 255 6 248 30 95 253 212 27 243 25 193 249 193 219 31 124 241 211 223 41 191 248 193 233 15 8 151 125 134 251 70 254 254 242 233 23 215 95 60 125 50 255 125 246 126 159 61 250 219 194 122 241 148 218 253 128 218 253 244 23 192 237 7 47 8 159 23 63 125 122 77 227 218 37 24 10 231 39 8 206 23 87 244 61 189 251 157 183 47 126 250 43 140 135 218 31 83 59 124 126 76 99 195 223 103 123 47 208 15 141 235 139 31 224 221 51 130 71 125 124 206 116 217 121 241 131 175 238 125 241 131 87 229 139 207 127 31 162 171 224 255 133 197 95 255 14 199 255 238 133 163 129 155 31 249 158 254 254 170 67 163 83 31 30 254 255 217 175 241 107 226 249 127 0 225 220 243 27 28 2 0 0 I don't know why it is not normal and expect it sholud start like this: 3 1 106 0 0 20 69 77 3 49 0 1 78 70 77 3 49 0 1 0 17 81 3 82 103 65 65 65 65 65 117 50 72 120 85 112 65 .......... I think the response must have been encrypted, but what's the encryption algorithm? Will be very appreciated for any ideas.
I have got the answer, the stream 31 139 8 0 0 0 0 0 4 0 237 189 7 96 28 73 150 37 ...... has been compressed by Gzip
R compare all list elements for duplicates
I am looking at all possible paths through a graph. I have written a DFS algorithm that finds all these paths. I want to make sure that my algorithm works correctly and that no two paths are identical. My algorithm returns a list that looks as follows: .... [[2770]] [1] 1 2 3 52 53 54 55 56 57 58 59 60 12 11 10 9 8 78 79 80 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 [38] 130 131 132 133 134 137 138 139 140 141 142 143 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 [[2771]] [1] 1 2 3 52 53 54 55 56 57 58 59 60 12 11 10 9 8 78 79 80 113 114 115 143 144 145 146 147 148 149 150 151 152 153 154 155 156 [38] 157 158 159 160 161 162 163 164 165 166 [[2772]] [1] 1 2 3 52 53 54 55 56 57 58 59 60 12 11 10 9 8 78 79 80 113 114 115 143 150 151 152 153 154 155 156 157 158 159 160 161 162 [38] 163 164 165 166 As you can see, the list is 2772 elements long. This means there are 2,772 paths through this graph. How can I easily compare all the list elements to make sure there are no duplicates. Just to be clear, the same set of numbers but in a different ordering represents a different path and is not a duplicate! Thank you for your help!
maybe something like test<-list(1:2,3:4,5:7,1:10,3:4,4:3) dups<-duplicated(test) idups<-seq_along(test)[dups]