It seems like with one image - I can create 36 pictures, but for some reason, I cannot create 36 different images and display them on Canvas. Only one random image is shown in position 30, for the reason I do not quite get :)
There will be an image added. It seems like generating a random image in the for loop does not work. I have tried to move it around - does not help.
Here is what I get
from tkinter import *
import math
import random
time_to_remember = 60
suit = ["clubs","diamonds","spades","hearts"]
names_cards = ["6","7","8","9","10","jack","ace","king"]
def countdown(count):
count_min = math.floor(count / 60)
count_sec = math.floor(count % 60)
if count_sec < 10:
count_sec = f"0{math.floor(count % 60)}"
timer_text.config( text=f"{count_min}:{count_sec}")
if count < 10:
timer_text.config( fg ="red")
if count > 0:
global timer
timer = window.after(1000, countdown, count - 1)
print(count)
window = Tk()
window.minsize(1000, 800)
canvas = Canvas(height=1000, width = 1000)
canvas.grid(row = 1, rowspan=6, column=0,columnspan=10 )
b_img = PhotoImage(file= "/Users/oleksandrzozulia/PycharmProjects/memory_project/Images/Screenshot 2022-08-27 at 11.48.49.png",
height=130,
width=80)
y_cor = 20
x_cor = 90
leng = 10
count = 0
ii = []
for i in range(0,4):
if count == 3:
leng = 6
for i in range(0,leng):
i = canvas.create_image(x_cor,y_cor, image=b_img, anchor="ne")
x_cor += 100
count +=1
x_cor = 90
y_cor += 150
#Display Random cards==================================================================
y_n = 20
x_n = 90
leng_n = 10
count_n = 0
for i in range(0,3):
if count_n == 3:
leng_n = 6
for i in range(0,leng_n):
img_n = PhotoImage(
file = f"Images/PNG-cards-1.3/{random.choice(names_cards)}_of_{random.choice(suit)}.png",
height=130,
width = 80)
i = canvas.create_image(x_n,y_n, image=img_n, anchor="ne")
x_n += 100
count +=1
x_n = 90
y_n += 150
everyone and Professor John
We are using gekko to do MPC on tclab simulation model. We try to emulate the situation that on site the actuator deviates from MV calculated by gekko because of the problems of actuator.
If the deviation is in the fixed pattern, for example a quite big constant deviation happens for a long time and may come back then work well for a long time. We can deal with it by extra logic to detect deviation and add the deviation value to the mv calculated by gekko.
one day, I noticed that there could be meas for MV when fstatus = 1. So I gave it a try. I hope gekko could deal with the deviation by itself. for example, if mv from gekko is 10 and the measurement is 5 and the pattern continues, gekko may spit out a higher MV value than 10, for example 15 and measurement is 10.
In the simulation, when I set MV's fstatus=1, the MV's curve becomes to :
q1a is the q1 with manual deviation. In the above pic, q1a == q1. It looks like gekko takes one more step thinking about the MV's effect.
In the below pic, there are two times range, one with "q1a == q1+20" and the other with "q1a == q1 -20". q1a's value is fed to tclab and mv(q1)'s meas.
I do not understand why the q1 calculated by gekko is going up or going down when meas deviates despite the t1 is going far away from sp.
Edit: Example Code
See the screen shot below from "normal" HMI. The sluggish MV disappeared, so it maybe caused by bug in my code. But the up-going or down-going could still be seen.
See my code below:
from random import random
from random import randrange
import tclab
from tclab import labtime
from tclab import TCLabModel
import numpy as np
import time
import matplotlib.pyplot as plt
from gekko import GEKKO
import json
from tclab import TCLabModel
make_mp4 = True
if make_mp4:
import imageio # required to make animation
import os
try:
os.mkdir('./figures')
except:
pass
class tclab_heaterpipe():
def __init__(self,d1,d2,model):
if(d1 >= 1 and d2 >=1):
self.delay_q1_step = int(d1)
self.delay_q2_step = int(d2)
self.q1_buffer = [0] * self.delay_q1_step
self.q2_buffer = [0] * self.delay_q2_step
self.m = model
else:
self.delay_q1_step =0
self.delay_q2_step =0
return
def Q1_delay(self,q1):
if(self.delay_q1_step == 0):
self.m.Q1(q1)
self.q1_buffer.insert(0,q1)
self.m.Q1(self.q1_buffer.pop())
def Q2_delay(self,q2):
if(self.delay_q2_step == 0):
self.m.Q1(q2)
self.q2_buffer.insert(0,q2)
self.m.Q2(self.q2_buffer.pop())
# Connect to Arduino
connected = False
theta1 = 1
theta2 = 1
T = tclab.setup(connected)
a = T()
tclab_delay = tclab_heaterpipe(theta1,theta2,a)
# Turn LED on
print('LED On')
a.LED(100)
# Simulate a time delay
# Run time in minutes
run_time = 80.0
# Number of cycles
loops = int(60.0*run_time)
# Temperature (K)
t1sp = 45.0
t2sp = 35.0
#########################################################
# Initialize Model
#########################################################
# use remote=True for MacOS
m = GEKKO(name='tclab-mpc',remote=False)
m.time = np.linspace(0,400,41)
step = 10
T1 = np.ones(int(loops/step)+1) * a.T1 # temperature (degC)
T2 = np.ones(int(loops/step)+1) * a.T2 # temperature (degC)
Tsp1 = np.ones(int(loops/step)+1) * t1sp # set point (degC)
Tsp2 = np.ones(int(loops/step)+1) * t2sp # set point (degC)
# heater values
Q1s = np.ones(int(loops/step)+1) * 0.0
Q2s = np.ones(int(loops/step)+1) * 0.0
# Parameters
Q1_ss = m.Param(value=0)
TC1_ss = m.Param(value=a.T1)
Q2_ss = m.Param(value=0)
TC2_ss = m.Param(value=a.T2)
Kp1 = m.Param(value= 0.7)
tau1 = m.Param(value=160.0)
Kp2 = m.Param(value=0.05)
tau2 = m.Param(value=160.0)
Kp3= m.Param(value=0.05)
tau3 = m.Param(value=160.0)
Kp4 = m.Param(value=0.4)
tau4 = m.Param(value=200.0)
sp1 = m.Param(value=a.T1)
sp2 = m.Param(value=a.T2)
# Manipulated variable
Q1 = m.MV(value=0, name='q1')
Q1.STATUS = 1 # use to control temperature
Q1.FSTATUS = 1 # no feedback measurement
Q1.LOWER = 0.0
Q1.UPPER = 100.0
Q1.DMAX = 10.0
Q1.DCOST = 5.0
Q2 = m.MV(value=0, name='q2')
Q2.STATUS = 1 # use to control temperature
Q2.FSTATUS = 1 # no feedback measurement
Q2.LOWER = 0.0
Q2.UPPER = 100.0
Q2.DMAX = 10.0
Q2.DCOST = 5.0
# Controlled variable
TC1 = m.CV(value=a.T1, name='tc1')
TC1.STATUS = 1 # minimize error with setpoint range
TC1.FSTATUS = 1 # receive measurement
TC1.TR_INIT = 2 # reference trajectory
# TC1.COST = 0.1
TC1.WSPHI = 20
TC1.WSPLO = 20
TC1.TAU = 50 # time constant for response
#TC1.TR_OPEN = 3
TC2 = m.CV(value=a.T2, name='tc2')
TC2.STATUS = 1 # minimize error with setpoint range
TC2.FSTATUS = 1 # receive measurement
TC2.TR_INIT = 2 # reference trajectory
# TC2.COST = 0.1
TC2.WSPHI = 20
TC2.WSPLO = 20
TC2.TAU = 30 # time constant for response
#kTC2.TR_OPEN = 3
# 添加延时
Q1d=m.Var()
m.delay(Q1, Q1d, theta1)
Q2d=m.Var()
m.delay(Q2, Q2d, theta2)
# Equation
#m.Equation(tau1 * TC1.dt() + (TC1 - TC1_ss) == Kp1 * (Q1d - Q1_ss))
# m.Equation(tau2 * TC2.dt() + (TC2 - TC2_ss) == Kp2 * (Q1d - Q1_ss))
# m.Equation(tau3 * TC1.dt() + (TC1 - TC1_ss) == Kp3 * (Q2d - Q2_ss))
# m.Equation(tau2 * TC2.dt() + (TC2 - TC2_ss) == Kp4 * (Q2d - Q2_ss))
m.Equation(0.5 * (tau1 * TC1.dt() + (TC1 - TC1_ss) + tau3 * TC1.dt() + (TC1 - TC1_ss)) == Kp1 * (Q1d - Q1_ss) + Kp3 * (Q2d -Q2_ss))
m.Equation(0.5 * (tau2 * TC2.dt() + (TC2 - TC2_ss) + tau4 * TC2.dt() + (TC2 - TC2_ss)) == Kp4 * (Q2d - Q2_ss) + Kp2 * (Q1d - Q1_ss))
# Steady-state initializations
m.options.IMODE = 1
m.options.SOLVER = 1 # 1=APOPT, 3=IPOPT
m.solve()
sp1.VALUE = 45
sp2.VALUE = 35
# Global Options
m.options.IMODE = 6 # MPC
m.options.CV_TYPE = 3 # Objective type
m.options.NODES = 2 # Collocation nodes
m.options.MAX_TIME = 10
m.options.SOLVER = 1 # 1=APOPT, 3=IPOPT
#m.options.CV_WGT_START = 2*theta
#m.options.CV_WGT_SLOPE = theta
# m.options.MV_STEP_HOR = 5
##################################################################
# Create plot
plt.figure()
plt.ion()
plt.show()
# Main Loop
a.Q1(0)
a.Q2(0)
Q2s[0:] = 0
start_time = time.time()
tm = np.linspace(1,loops,int(loops/step)+1)
j=0
try:
time_start = time.time()
labtime_start = labtime.time()
if(not connected):
labtime.set_rate(10)
for i in tclab.clock(loops,adaptive=False):
i = int(i)
if(i == 0):
continue
print("-----------------------")
t_real = time.time() - time_start
t_lab = labtime.time() - labtime_start
print("real time = {0:4.1f} lab time = {1:4.1f} m.time = {1:4.1f}".format(t_real, t_lab,m.time))
#print("real time = {0:4.1f} m.time = {1:4.1f}".format(t_real, m.time))
if(i%step != 0):
continue
j = i/step
j = int(j)
print(j)
T1[j:] = a.T1
T2[j:] = a.T2
tm[j] = i
###############################
### MPC CONTROLLER ###
###############################
TC1.MEAS = T1[j]
TC2.MEAS = T2[j]
print("T1 meas:{0:4.1f} ".format(a.T1))
print("T2 meas:{0:4.1f} ".format(a.T2))
# input setpoint with deadband +/- DT
DT =0.5
TC1.SPHI = Tsp1[j] +DT
TC1.SPLO = Tsp1[j] -DT
TC2.SPHI = Tsp2[j] +DT
TC2.SPLO = Tsp2[j] -DT
try:
# stop model time to solve MPC in cast the solver takes too much time
if(not connected):
labtime.stop()
m.solve(disp=False)
#start model time
if(not connected):
labtime.start()
except Exception as e:
if(not connected):
if(not labtime.running):
labtime.start()
print("sovle's exception:")
print(e)
if(j != 0):
Q1s[j] = Q1s[j-1]
Q2s[j] = Q2s[j-1]
continue
# test for successful solution
if (m.options.APPSTATUS==1):
# retrieve the first Q value
Q1s[j:] = np.ones(len(Q1s)-j) * Q1.NEWVAL
Q2s[j:] = np.ones(len(Q2s)-j) * Q2.NEWVAL
#a.Q1(Q1.NEWVAL)
#a.Q2(Q2.NEWVAL)
print("Q1 applied with delay: {0:4.1f} ".format(Q1.NEWVAL))
print("Q2 applied with delay: {0:4.1f} ".format(Q2.NEWVAL))
with open(m.path+'//results.json') as f:
results = json.load(f)
else:
# not successful, set heater to zero
print("APPSTATUS is not 1,set Q to 0")
#Q1s[j] = 0
#Q2s[j] = 0
if i> 300 and i < 600:
Q1s[j] = Q1s[j] - 20
Q2s[j] = Q2s[j] - 20
if i>= 600:
Q1s[j] = Q1s[j] + 20
Q2s[j] = Q2s[j] + 20
Q1.meas= Q1s[j]
Q2.meas= Q2s[j]
tclab_delay.Q1_delay(Q1s[j])
tclab_delay.Q2_delay(Q2s[j])
print("calc:"+str(Q1s[j]))
print("calc:"+str(Q2s[j]))
#apply disturbance on 50s, 200s,
#if(i == 600):
# Q2s[j] = 100
#if(i == 1400):
# Q2s[j] = 0
#Q2s[j] = 20 - randrange(20)
#Q2s[j:] = np.ones(len(Q2s)-j) * Q2s[j]
#restore Q2 to 0
#if(i == 300):
#Q2s[j:] = 0
#a.Q2(Q2s[j])
#tclab_delay.Q2_delay(Q2s[j])
#take Q2 to FV
#Q2.MEAS = Q2s[j]
if(not connected):
labtime.stop()
# Plot
try:
plt.clf()
ax=plt.subplot(2,1,1)
ax.grid()
plt.plot(tm[0:j],T1[0:j],'ro',markersize=3,label=r'$T_1$')
plt.plot(tm[0:j],Tsp1[0:j],'r-',markersize=3,label=r'$T_1 Setpoint$')
plt.plot(tm[0:j],T2[0:j],'bo',markersize=3,label=r'$T_2$')
plt.plot(tm[0:j],Tsp2[0:j],'b-',markersize=3,label=r'$T_2 Setpoint$')
plt.plot(tm[j]+m.time,results['tc1.bcv'],'r-.',markersize=1,\
label=r'$T_1$ predicted',linewidth=1)
plt.plot(tm[j]+m.time,results['tc2.bcv'],'b-.',markersize=1,\
label=r'$T_2$ predicted',linewidth=1)
plt.plot(tm[j]+m.time,results['tc1.tr_hi'],'k--',\
label=r'$T_1$ trajectory')
plt.plot(tm[j]+m.time,results['tc1.tr_lo'],'k--')
plt.plot(tm[j]+m.time,results['tc2.tr_hi'],'k--',\
label=r'$T_2$ trajectory')
plt.plot(tm[j]+m.time,results['tc2.tr_lo'],'k--')
plt.ylabel('Temperature (degC)')
plt.legend(loc='best')
ax=plt.subplot(2,1,2)
ax.grid()
plt.plot(tm[0:j],Q1s[0:j],'r-',linewidth=3,label=r'$Q_1$')
plt.plot(tm[0:j],Q2s[0:j],'b-',linewidth=3,label=r'$Q_2$')
plt.plot(tm[j]+m.time,Q1.value,'r-.',\
label=r'$Q_1$ plan',linewidth=1)
plt.plot(tm[j]+m.time,Q2.value,'b-.',\
label=r'$Q_2$ plan',linewidth=1)
#plt.plot(tm[0:i],Q2s[0:i],'b:',LineWidth=3,label=r'$Q_2$')
plt.ylabel('Heaters')
plt.xlabel('Time (sec)')
plt.legend(loc='best')
plt.draw()
plt.pause(0.05)
if make_mp4:
filename='./figures/plot_'+str(j+10000)+'.png'
plt.savefig(filename)
except Exception as e:
print(e)
pass
if(not connected):
labtime.start()
# Turn off heaters
a.Q1(0)
a.Q2(0)
print('Shutting down')
input("Press Enter to continue...")
a.close()
# Allow user to end loop with Ctrl-C
except KeyboardInterrupt:
# Disconnect from Arduino
a.Q1(0)
a.Q2(0)
print('Shutting down')
a.close()
if make_mp4:
images = []
iset = 0
for i in range(1,int(loops/step)+1):
filename='./figures/plot_'+str(i+10000)+'.png'
if os.path.exists(filename):
images.append(imageio.imread(filename))
if ((i+1)%350)==0:
imageio.mimsave('results_'+str(iset)+'.mp4', images)
iset += 1
images = []
if images!=[]:
imageio.mimsave('results_'+str(iset)+'.mp4', images)
# Make sure serial connection still closes when there's an error
except:
# Disconnect from Arduino
a.Q1(0)
a.Q2(0)
print('Error: Shutting down')
a.close()
raise
Regards
Tibalt
Is the FSTATUS also ON for the CVs such as t1.FSTATUS=1? If you update the measurement such as:
t1.MEAS = lab.T1
t2.MEAS = lab.T2
then this updates the BIAS for t1 and t2 (BIAS documentation). This should take care of any process / model mismatch that you are introducing by arbitrarily increasing or decreasing the heater by 20%. If t1.FSTATUS is OFF (0) then it is not able to compensate for the mismatch.
Another thing to try is to adjust the reference trajectory. The controller can appear sluggish if TAU is too high. Here is an example application with MPC and a linear model.
One additional way to compensate for the mismatch is to use Moving Horizon Estimation as shown here.
It looks like you have created a nice interface!
Response to Edit
Thanks for adding the code. The problem is that Q1.DMAX=10 and Q2.DMAX=10. When the Q1 and Q2 values are shifted up by 20 each cycle, the most that the controller can shift down is 20-10=10 so the controller appears that it is ramping in the wrong direction. Changing to DMAX=100 fixes the problem. There is still offset from the setpoint because the recommended Q1 and Q2 are shifted each cycle. The true recommended values are never implemented. Another thing to try is to impose an offset on the measured values such as TC1.MEAS = T1[j] + 20. The model bias will remove the offset in this case.
from random import random
from random import randrange
import tclab
from tclab import labtime
from tclab import TCLabModel
import numpy as np
import time
import matplotlib.pyplot as plt
from gekko import GEKKO
import json
from tclab import TCLabModel
make_gif = True
make_mp4 = True
if make_gif or make_mp4:
# pip install imageio-ffmpeg with imageio to make MP4
import imageio # required to make animation
import os
try:
os.mkdir('./figures')
except:
pass
class tclab_heaterpipe():
def __init__(self,d1,d2,model):
if(d1 >= 1 and d2 >=1):
self.delay_q1_step = int(d1)
self.delay_q2_step = int(d2)
self.q1_buffer = [0] * self.delay_q1_step
self.q2_buffer = [0] * self.delay_q2_step
self.m = model
else:
self.delay_q1_step =0
self.delay_q2_step =0
return
def Q1_delay(self,q1):
if(self.delay_q1_step == 0):
self.m.Q1(q1)
self.q1_buffer.insert(0,q1)
self.m.Q1(self.q1_buffer.pop())
def Q2_delay(self,q2):
if(self.delay_q2_step == 0):
self.m.Q1(q2)
self.q2_buffer.insert(0,q2)
self.m.Q2(self.q2_buffer.pop())
# Connect to Arduino
connected = False # switch to connected=True with physical hardware
theta1 = 1
theta2 = 1
T = tclab.setup(connected)
a = T()
tclab_delay = tclab_heaterpipe(theta1,theta2,a)
# Turn LED on
print('LED On')
a.LED(100)
# Simulate a time delay
# Run time in minutes
run_time = 20.0
# Number of cycles
loops = int(60.0*run_time)
# Temperature (K)
t1sp = 45.0
t2sp = 35.0
#########################################################
# Initialize Model
#########################################################
# use remote=True for MacOS
m = GEKKO(name='tclab-mpc',remote=False)
m.time = np.linspace(0,400,41)
step = 10
T1 = np.ones(int(loops/step)+1) * a.T1 # temperature (degC)
T2 = np.ones(int(loops/step)+1) * a.T2 # temperature (degC)
Tsp1 = np.ones(int(loops/step)+1) * t1sp # set point (degC)
Tsp2 = np.ones(int(loops/step)+1) * t2sp # set point (degC)
# heater values
Q1s = np.ones(int(loops/step)+1) * 0.0
Q2s = np.ones(int(loops/step)+1) * 0.0
# Parameters
Q1_ss = m.Param(value=0)
TC1_ss = m.Param(value=a.T1)
Q2_ss = m.Param(value=0)
TC2_ss = m.Param(value=a.T2)
Kp1 = m.Param(value= 0.7)
tau1 = m.Param(value=160.0)
Kp2 = m.Param(value=0.05)
tau2 = m.Param(value=160.0)
Kp3= m.Param(value=0.05)
tau3 = m.Param(value=160.0)
Kp4 = m.Param(value=0.4)
tau4 = m.Param(value=200.0)
sp1 = m.Param(value=a.T1)
sp2 = m.Param(value=a.T2)
# Manipulated variable
Q1 = m.MV(value=0, name='q1')
Q1.STATUS = 1 # use to control temperature
Q1.FSTATUS = 1 # no feedback measurement
Q1.LOWER = 0.0
Q1.UPPER = 100.0
Q1.DMAX = 100.0
Q1.DCOST = 1e-3
Q2 = m.MV(value=0, name='q2')
Q2.STATUS = 1 # use to control temperature
Q2.FSTATUS = 1 # no feedback measurement
Q2.LOWER = 0.0
Q2.UPPER = 100.0
Q2.DMAX = 100.0
Q2.DCOST = 1e-3
# Controlled variable
TC1 = m.CV(value=a.T1, name='tc1')
TC1.STATUS = 1 # minimize error with setpoint range
TC1.FSTATUS = 1 # receive measurement
TC1.TR_INIT = 2 # reference trajectory
# TC1.COST = 0.1
TC1.WSPHI = 20
TC1.WSPLO = 20
TC1.TAU = 50 # time constant for response
#TC1.TR_OPEN = 3
TC2 = m.CV(value=a.T2, name='tc2')
TC2.STATUS = 1 # minimize error with setpoint range
TC2.FSTATUS = 1 # receive measurement
TC2.TR_INIT = 2 # reference trajectory
# TC2.COST = 0.1
TC2.WSPHI = 20
TC2.WSPLO = 20
TC2.TAU = 30 # time constant for response
#kTC2.TR_OPEN = 3
# 添加延时
Q1d=m.Var()
m.delay(Q1, Q1d, theta1)
Q2d=m.Var()
m.delay(Q2, Q2d, theta2)
# Equation
#m.Equation(tau1 * TC1.dt() + (TC1 - TC1_ss) == Kp1 * (Q1d - Q1_ss))
# m.Equation(tau2 * TC2.dt() + (TC2 - TC2_ss) == Kp2 * (Q1d - Q1_ss))
# m.Equation(tau3 * TC1.dt() + (TC1 - TC1_ss) == Kp3 * (Q2d - Q2_ss))
# m.Equation(tau2 * TC2.dt() + (TC2 - TC2_ss) == Kp4 * (Q2d - Q2_ss))
m.Equation(0.5 * (tau1 * TC1.dt() + (TC1 - TC1_ss) + tau3 * TC1.dt() + (TC1 - TC1_ss)) == Kp1 * (Q1d - Q1_ss) + Kp3 * (Q2d -Q2_ss))
m.Equation(0.5 * (tau2 * TC2.dt() + (TC2 - TC2_ss) + tau4 * TC2.dt() + (TC2 - TC2_ss)) == Kp4 * (Q2d - Q2_ss) + Kp2 * (Q1d - Q1_ss))
# Steady-state initializations
m.options.IMODE = 1
m.options.SOLVER = 1 # 1=APOPT, 3=IPOPT
m.solve()
sp1.VALUE = 45
sp2.VALUE = 35
# Global Options
m.options.IMODE = 6 # MPC
m.options.CV_TYPE = 3 # Objective type
m.options.NODES = 2 # Collocation nodes
m.options.MAX_TIME = 10
m.options.SOLVER = 1 # 1=APOPT, 3=IPOPT
#m.options.CV_WGT_START = 2*theta
#m.options.CV_WGT_SLOPE = theta
# m.options.MV_STEP_HOR = 5
##################################################################
# Create plot
plt.figure(figsize=(12,8))
plt.ion()
plt.show()
# Main Loop
a.Q1(0)
a.Q2(0)
Q2s[0:] = 0
start_time = time.time()
tm = np.linspace(1,loops,int(loops/step)+1)
j=0
try:
time_start = time.time()
labtime_start = labtime.time()
if(not connected):
labtime.set_rate(10)
for i in tclab.clock(loops,adaptive=False):
i = int(i)
if(i == 0):
continue
print("-----------------------")
t_real = time.time() - time_start
t_lab = labtime.time() - labtime_start
print("real time = {0:4.1f} lab time = {1:4.1f} m.time = {1:4.1f}".format(t_real, t_lab,m.time))
#print("real time = {0:4.1f} m.time = {1:4.1f}".format(t_real, m.time))
if(i%step != 0):
continue
j = i/step
j = int(j)
print(j)
T1[j:] = a.T1
T2[j:] = a.T2
tm[j] = i
###############################
### MPC CONTROLLER ###
###############################
TC1.MEAS = T1[j]
TC2.MEAS = T2[j]
print("T1 meas:{0:4.1f} ".format(a.T1))
print("T2 meas:{0:4.1f} ".format(a.T2))
# input setpoint with deadband +/- DT
DT =0.5
TC1.SPHI = Tsp1[j] +DT
TC1.SPLO = Tsp1[j] -DT
TC2.SPHI = Tsp2[j] +DT
TC2.SPLO = Tsp2[j] -DT
try:
# stop model time to solve MPC in cast the solver takes too much time
if(not connected):
labtime.stop()
m.solve(disp=False)
#start model time
if(not connected):
labtime.start()
except Exception as e:
if(not connected):
if(not labtime.running):
labtime.start()
print("sovle's exception:")
print(e)
if(j != 0):
Q1s[j] = Q1s[j-1]
Q2s[j] = Q2s[j-1]
continue
# test for successful solution
if (m.options.APPSTATUS==1):
# retrieve the first Q value
Q1s[j:] = np.ones(len(Q1s)-j) * Q1.NEWVAL
Q2s[j:] = np.ones(len(Q2s)-j) * Q2.NEWVAL
#a.Q1(Q1.NEWVAL)
#a.Q2(Q2.NEWVAL)
print("Q1 applied with delay: {0:4.1f} ".format(Q1.NEWVAL))
print("Q2 applied with delay: {0:4.1f} ".format(Q2.NEWVAL))
with open(m.path+'//results.json') as f:
results = json.load(f)
else:
# not successful, set heater to zero
print("APPSTATUS is not 1,set Q to 0")
#Q1s[j] = 0
#Q2s[j] = 0
if i> 300 and i < 600:
Q1s[j] = max(0,Q1s[j] - 20)
Q2s[j] = max(0,Q2s[j] - 20)
if i>= 600:
Q1s[j] = min(100,Q1s[j] + 20)
Q2s[j] = min(100,Q2s[j] + 20)
Q1.meas= Q1s[j]
Q2.meas= Q2s[j]
tclab_delay.Q1_delay(Q1s[j])
tclab_delay.Q2_delay(Q2s[j])
print("calc:"+str(Q1s[j]))
print("calc:"+str(Q2s[j]))
if(not connected):
labtime.stop()
# Plot
try:
plt.clf()
ax=plt.subplot(2,1,1)
ax.grid()
plt.plot(tm[0:j],T1[0:j],'ro',markersize=3,label=r'$T_1$')
plt.plot(tm[0:j],Tsp1[0:j],'r-',markersize=3,label=r'$T_1 Setpoint$')
plt.plot(tm[0:j],T2[0:j],'bo',markersize=3,label=r'$T_2$')
plt.plot(tm[0:j],Tsp2[0:j],'b-',markersize=3,label=r'$T_2 Setpoint$')
plt.plot(tm[j]+m.time,results['tc1.bcv'],'r-.',markersize=1,\
label=r'$T_1$ predicted',linewidth=1)
plt.plot(tm[j]+m.time,results['tc2.bcv'],'b-.',markersize=1,\
label=r'$T_2$ predicted',linewidth=1)
plt.plot(tm[j]+m.time,results['tc1.tr_hi'],'k--',\
label=r'$T_1$ trajectory')
plt.plot(tm[j]+m.time,results['tc1.tr_lo'],'k--')
plt.plot(tm[j]+m.time,results['tc2.tr_hi'],'k--',\
label=r'$T_2$ trajectory')
plt.plot(tm[j]+m.time,results['tc2.tr_lo'],'k--')
plt.ylabel('Temperature (degC)')
plt.legend(loc=1)
ax=plt.subplot(2,1,2)
ax.grid()
plt.plot(tm[0:j],Q1s[0:j],'r-',linewidth=3,label=r'$Q_1$')
plt.plot(tm[0:j],Q2s[0:j],'b-',linewidth=3,label=r'$Q_2$')
plt.plot(tm[j]+m.time,Q1.value,'r-.',\
label=r'$Q_1$ plan',linewidth=1)
plt.plot(tm[j]+m.time,Q2.value,'b-.',\
label=r'$Q_2$ plan',linewidth=1)
#plt.plot(tm[0:i],Q2s[0:i],'b:',LineWidth=3,label=r'$Q_2$')
plt.ylabel('Heaters')
plt.xlabel('Time (sec)')
plt.legend(loc=1)
plt.draw()
plt.pause(0.05)
if make_mp4:
filename='./figures/plot_'+str(j+10000)+'.png'
plt.savefig(filename)
except Exception as e:
print(e)
pass
if(not connected):
labtime.start()
# Turn off heaters
a.Q1(0)
a.Q2(0)
print('Shutting down')
input("Press Enter to continue...")
a.close()
# make gif
if make_gif:
images = []
iset = 0
for i in range(1,int(loops/step)+1):
filename='./figures/plot_'+str(i+10000)+'.png'
if os.path.exists(filename):
images.append(imageio.imread(filename))
if ((i+1)%350)==0:
imageio.mimsave('results_'+str(iset)+'.gif', images)
iset += 1
images = []
if images!=[]:
imageio.mimsave('results_'+str(iset)+'.gif', images)
if make_mp4:
images = []
iset = 0
for i in range(1,int(loops/step)+1):
filename='./figures/plot_'+str(i+10000)+'.png'
if os.path.exists(filename):
images.append(imageio.imread(filename))
if ((i+1)%350)==0:
imageio.mimsave('results_'+str(iset)+'.gif', images)
iset += 1
images = []
if images!=[]:
imageio.mimsave('results_'+str(iset)+'.gif', images)
# Allow user to end loop with Ctrl-C
except KeyboardInterrupt:
# Disconnect from Arduino
a.Q1(0)
a.Q2(0)
print('Shutting down')
a.close()
if make_gif:
images = []
iset = 0
for i in range(1,int(loops/step)+1):
filename='./figures/plot_'+str(i+10000)+'.png'
if os.path.exists(filename):
images.append(imageio.imread(filename))
if ((i+1)%350)==0:
imageio.mimsave('results_'+str(iset)+'.gif', images)
iset += 1
images = []
if images!=[]:
imageio.mimsave('results_'+str(iset)+'.gif', images)
if make_mp4:
images = []
iset = 0
for i in range(1,int(loops/step)+1):
filename='./figures/plot_'+str(i+10000)+'.png'
if os.path.exists(filename):
images.append(imageio.imread(filename))
if ((i+1)%350)==0:
imageio.mimsave('results_'+str(iset)+'.mp4', images)
iset += 1
images = []
if images!=[]:
imageio.mimsave('results_'+str(iset)+'.mp4', images)
# Make sure serial connection still closes when there's an error
except:
# Disconnect from Arduino
a.Q1(0)
a.Q2(0)
print('Error: Shutting down')
a.close()
raise
I am trying to create a binary CNN classifier for a dataset (class 0 = 77 images, class 1 = 41 images), which I want to do 5-Fold cross validation. In each fold, using the validation sets to save best model, and sharing same model, Hyperparameters, and training strategy. And here is my results.
fold - test sets accuracy
fold0 - 0.68
fold1 - 0.71
fold2 - 0.91
fold3 - 0.96
fold4 - 0.64
My question is:
Fine tuning by changing the Hyperparameters. It was found that fold2 and fold3 performed better each time, but fold0 and fold4 performed poorly. What is willing to cause it and what should I do.
The possible problem is that each initialization is random.
Thank you all for your answers.
import os
import torch
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from torch.utils.data.sampler import WeightedRandomSampler
import monai
from monai.data import NiftiDataset
from monai.transforms import Compose, AddChannel, ScaleIntensity, RandFlip, RandRotate, ToTensor
from monai.data import CSVSaver
from data_process import read_csv, get_sample_weights
def train(train_file, val_file, stage='exp0'):
'''
:param train_file:
:param val_file:
:param stage:
:return:
'''
os.environ['CUDA_VISIBLE_DEVICES'] = '0,1'
img_src_path = '../samples/T1c_images/' #
img_list_train, label_list_train = read_csv(train_file)
img_list_val, label_list_val = read_csv(val_file)
img_train = [os.path.join(img_src_path, i) for i in img_list_train]
labels_train = [int(i) for i in label_list_train]
img_val = [os.path.join(img_src_path, i) for i in img_list_val]
labels_val = [int(i) for i in label_list_val]
print('val images: ', len(img_val))
# Define transforms
# train_transforms = Compose([ScaleIntensity(), AddChannel(), Resize((182, 218, 182)), RandRotate90(), ToTensor()])
# val_transforms = Compose([ScaleIntensity(), AddChannel(), Resize((182, 218, 182)), ToTensor()])
train_transforms = Compose([ScaleIntensity(), RandRotate(range_x=45, range_y=45, range_z=45, prob=0.5),
RandFlip(prob=0.5, spatial_axis=1),
AddChannel(), ToTensor()]) # if x=y=z RandRotate90()
val_transforms = Compose([ScaleIntensity(), AddChannel(), ToTensor()])
train_ds = NiftiDataset(image_files=img_train, labels=labels_train, transform=train_transforms, image_only=False)
train_loader = DataLoader(train_ds, batch_size=4, shuffle=True, num_workers=2,
pin_memory=torch.cuda.is_available())
# create a validation data_process loader
val_ds = NiftiDataset(image_files=img_val, labels=labels_val, transform=val_transforms, image_only=False)
val_loader = DataLoader(val_ds, batch_size=4, num_workers=2, pin_memory=torch.cuda.is_available())
# Create DenseNet121, CrossEntropyLoss and Adam optimizer
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = monai.networks.nets.densenet.densenet121(spatial_dims=3, in_channels=1, out_channels=2).to(device)
model = torch.nn.DataParallel(model)
loss_function = torch.nn.CrossEntropyLoss(weight=torch.Tensor([1, 1.2])).cuda()
optimizer = torch.optim.Adam(model.parameters(), 1e-5)
# start a typical PyTorch training
epochs = 50
val_interval = 1
best_metric = -1
best_metric_epoch = -1
writer = SummaryWriter()
for epoch in range(epochs):
print("-" * 10)
print(f"epoch {epoch + 1}/{epochs}")
model.train()
epoch_loss = 0
step = 0
t_metric_count = 0
t_num_correct = 0
for batch_data in train_loader:
step += 1
# ptrint images name
# print('image name', batch_data[2]['filename_or_obj'])
inputs = batch_data[0].to(device)
# print(inputs.shape)
labels = batch_data[1].to(device)
optimizer.zero_grad()
outputs = model(inputs)
loss = loss_function(outputs, labels)
loss.backward()
optimizer.step()
epoch_loss += loss.item()
epoch_len = len(train_ds) // train_loader.batch_size
# train acc
t_value = torch.eq(outputs.argmax(dim=1), labels)
t_metric_count += len(t_value) #
t_num_correct += t_value.sum().item() #
# print(f"{step}/{epoch_len}, train_loss: {loss.item():.4f}")
epoch_loss /= step
t_metric = t_num_correct / t_metric_count
writer.add_scalar("train_loss", epoch_loss, epoch + 1)
writer.add_scalar("train_acc", t_metric, epoch + 1)
print(f"epoch {epoch + 1} average loss: {epoch_loss:.4f}")
if (epoch + 1) % val_interval == 0:
model.eval()
with torch.no_grad():
num_correct = 0.0
metric_count = 0
for val_data in val_loader:
val_images, val_labels = val_data[0].to(device), val_data[1].to(device)
val_outputs = model(val_images)
value = torch.eq(val_outputs.argmax(dim=1), val_labels)
metric_count += len(value) #
num_correct += value.sum().item() #
metric = num_correct / metric_count
if metric > best_metric:
best_metric = metric
best_metric_epoch = epoch + 1
save_path = 'checkpoint_07201/' + stage + '_' + str(epoch + 1) + "_best_metric_model.pth"
torch.save(model.state_dict(), save_path)
print("saved new best metric model")
print(
"current epoch: {} current accuracy: {:.4f} best val accuracy: {:.4f} at epoch {}".format(
epoch + 1, metric, best_metric, best_metric_epoch
))
print('current train accuracy: {:.4f}, num_correct: {}, num_count:{}'.
format(t_metric, t_num_correct, t_metric_count ))
writer.add_scalar("val_accuracy", metric, epoch + 1)
print(f"train completed, best_metric: {best_metric:.4f} at epoch: {best_metric_epoch}")
writer.close()
if __name__ == "__main__":
# 5 folder
for i in range(5):
folder = 'exp'+str(i)
train_path = './data/'+ folder +'/train.csv'
val_path = './data/'+ folder + '/val.csv'
train(train_path, val_path, stage=folder)
I am using healpy.sphtfunc.smoothing for apodization of my binary mask and I am having problem that, if I have temperature cut of 100K and I made a binary mask corresponding to cut, then after apodization of mask using above routine when I apply it on my map I get 120K or number more than 100K. So I am confuse that does one do apodization on Binary mask or (map*Binary_mask)
def getMapValue(map, ra, dec, theta):
nSide = hp.pixelfunc.npix2nside(map.size)
# Extract the region around the source
vec = hp.pixelfunc.ang2vec(np.pi / 2 - np.deg2rad(dec), np.deg2rad(ra))
vec = np.array(vec)
innerPixels = hp.query_disc(nSide, vec, radius=np.radians(theta))
return innerPixels
def masking_map(map1, nside, npix, limit):
mask = np.ones(hp.nside2npix(nside), dtype=np.float64)
index = (map1 > limit)
mask[index] = 0.0
for ipix in xrange(0, npix):
theta1, phi = hp.pixelfunc.pix2ang(nside, ipix)
if 70. < np.degrees(theta1) < 110.:
mask[ipix] = 0.0
inner_pix = getMapValue(map1,329.6, 17.5, 54.0)
outer_pix = getMapValue(map1,329.6, 17.5, 62.0)
index = np.setdiff1d(outer_pix, inner_pix)
index1 = []
for ipix1 in index:
theta, phi = hp.pixelfunc.pix2ang(nside, ipix1)
if np.degrees(theta) < 90.0:
if 0.0 < np.degrees(phi)< 60.0:
index1.append(ipix1)
if 320.0 < np.degrees(phi)< 360.0:
index1.append(ipix1)
index1=np.asarray(index1)
mask[index1]=0.0
return mask
def apodiz(mask, theta):
apodiz_mask = hp.sphtfunc.smoothing(mask, fwhm=np.radians(theta),
iter=3, use_weights=True,
verbose=False)
index = (apodiz_mask < 0.0)
apodiz_mask[index] = 0.000
return apodiz_mask
def main():
filename = 'haslam408_dsds_Remazeilles2014.fits'
NSIDE = 512
input_map = loadMap(fname)
NPIX = hp.pixelfunc.nside2npix(NSIDE)
LIMIT = 50 # for 50K cut
theta_ap = 5.0 # FWHM for Apodization in degrees
Binary_mask = masking_map(input_map, NSIDE, NPIX, LIMIT)
imp_map = apodiz(Binary_mask, theta_ap)
masked_map = input_map*imp_map
hp.mollview(apoMask, xsize=2000, coord=['G'], unit=r'$T_{B}(K)$', nest=False, title='%s' % key[count])
hp.mollview(imp_map, xsize=2000, coord=['G'], unit=r'$T_{B}(K)$', nest=False, title='%s' % key[count])
hp.mollview(masked_map, xsize=2000, coord=['G'], unit=r'$T_{B}(K)$', nest=False, title='408 MHz,%s' % key[count])
hp.mollview(input_map*Binary_mask, xsize=2000, coord=['G'], unit=r'$T_{B}(K)$', nest=False, title='408 MHz,%s' % key[count])
count+=1
if __name__ == "__main__":
This is resulting map without apodization of binar mask
This is resulting map after 5 degree apodized of Binary mask
I am running the loop in this method for around 1 million times but it is taking a lot of time maybe due O(n^2) , so is there any way to improve these two modules :-
def genIndexList(length,ID):
indexInfoList = []
id = list(str(ID))
for i in range(length):
i3 = (str(decimalToBase3(i)))
while len(i3) != 12:
i3 = '0' + i3
p = (int(str(ID)[0]) + int(i3[0]) + int(i3[2]) + int(i3[4]) + int(i3[6]) + int(i3[8]) + int(i3[10]))%3
indexInfoList.append(str(ID)+i3+str(p))
return indexInfoList
and here is the method for to convert number to base3 :-
def decimalToBase3(num):
i = 0
if num != 0 and num != 1 and num != 2:
number = ""
while num != 0 :
remainder = num % 3
num = num / 3
number = str(remainder) + number
return int(number)
else:
return num
I am using python to make a software and these 2 functions are a part of it.Please suggest why these 2 methods are so slow and how to improve efficiency of these methods.
The first function can be reduced to:
def genIndexList(length, ID):
indexInfoList = []
id0 = str(ID)[0]
for i in xrange(length):
i3 = format(decimalToBase3(i), '012d')
p = sum(map(int, id0 + i3[::2])) % 3
indexInfoList.append('{}{}{}'.format(ID, i3, p))
return indexInfoList
You may want to make it a generator instead:
def genIndexList(length, ID):
id0 = str(ID)[0]
for i in xrange(length):
i3 = format(decimalToBase3(i), '012d')
p = sum(map(int, id0 + i3[::2])) % 3
yield '{}{}{}'.format(ID, i3, p)
The second function could be:
def decimalToBase3(num):
if 0 <= num < 3: return num
result = ""
while num:
num, digit = divmod(num, 3)
result = str(digit) + result
return int(result)
Next step; you are just generating a sequence of base-3 digits. Just generate these directly:
from itertools import product, imap
def base3sequence(l=12, digits='012'):
return imap(''.join, product(digits, repeat=l))
This produces base3 values, 0-padded to 12 digits:
>>> gen = base3sequence()
>>> for i in range(10):
... print next(gen)
...
000000000000
000000000001
000000000002
000000000010
000000000011
000000000012
000000000020
000000000021
000000000022
000000000100
and genIndexList() becomes:
from itertools import islice
def genIndexList(length, ID):
id0 = str(ID)[0]
for i3 in islice(base3sequence(), length):
p = sum(map(int, id0 + i3[::2])) % 3
yield '{}{}{}'.format(ID, i3, p)