I have the following model in pymc2:
import pymc
from scipy.stats import gamma
alpha = pymc.Uniform('alpha', 0.01, 2.0)
scale = pymc.Uniform('scale', 1.0, 4.0)
#pymc.deterministic(plot=False)
def beta(scale=scale):
return 1.0 / scale
#pymc.potential
def p_factor(alpha=alpha, scale=scale, lmin=lmin, n=len(sample)):
dist = gamma(alpha, loc=0., scale=scale)
fp = 1.0 - dist.cdf(lmin)
return -(n+1)*np.log(fp)
obs = pymc.Gamma("obs", alpha=alpha, beta=beta, value=sample, observed=True)
The physical background of this model is the luminosity function of galaxies (LF), i.e., the probability of a galaxy having luminosity L. For some types of galaxies, the LF is just a gamma function. The potential accounts for data truncation, as galaxy surveys usually miss a substantial fraction of the targets, particularly those of low luminosity. In this model I miss everything below lmin
Details of this method can be found in this paper by Kelly et al.
This model works: I run MAP and MCMC on the model and I can recover the parameters alpha and scale from my simulated data sample, with increased uncertainty as lmin grows.
Now I would like to insert gaussian measurement errors. For simplicity all the data has the same precision. I'm not modifying the potential to include the errors also.
alpha = pymc.Uniform('alpha', 0.01, 2.0)
scale = pymc.Uniform('scale',1.0, 4.0)
sig = 0.1
tau = math.pow(sig, -2.0)
#pymc.deterministic(plot=False)
def beta(scale=scale):
return 1.0 / scale
#pymc.potential
def p_factor(alpha=alpha, scale=scale, lmin=lmin, n=len(sample)):
dist = gamma(alpha, loc=0., scale=scale)
fp = 1.0 - dist.cdf(lmin)
return -(n+1) * np.log(fp)
dist = pymc.Gamma("dist", alpha=alpha, beta=beta)
obs = pymc.Normal("obs", mu=dist, tau=tau, value=sample, observed=True)
But surely I'm doing something wrong here because this model does not work.
When I run pymc.MAPon this model I recover the initial values of alpha and scale
vals = {'alpha': alpha, 'scale': scale, 'beta': beta,
'p_factor': p_factor, 'obs': obs, 'dist': dist}
M2 = pymc.MAP(vals)
M2.fit()
print M2.alpha.value, M2.scale.value
>>> (array(0.010000000006018368), array(1.000000000833973))
When I run pymc.MCMC, alpha and beta are no traced at all.
M = pymc.MCMC(vals)
M.sample(10000, burn=5000)
...
M.stats()['alpha']
>>> {'95% HPD interval': array([ 0.01000001, 0.01000502]),
'mc error': 2.1442678276712383e-07,
'mean': 0.010001588137798096,
'n': 5000,
'quantiles': {2.5: 0.0100000088679046,
25: 0.010000382359859467,
50: 0.010001100377476166,
75: 0.010001668672799679,
97.5: 0.0100050194240779},
'standard deviation': 2.189828287191421e-06}
again initial values. In fact if I change alpha to start in, say, 0.02, the recovered values of alpha is 0.02.
This is a notebook with the working model plus simulated data.
This is a notebook with the error model plus simulated data.
Any guidance on making this work would be really appreciated.
It seems that is enough to change
dist = pymc.Gamma("dist", alpha=alpha, beta=beta)
by
dist = pymc.Gamma("dist", alpha=alpha, beta=beta, value=sample)
The sampled data is a reasonable initial value for dist. Anyway, I do no get the logic, as other initial values (such as an array of zeros) bring back the problem of not sampling alpha and beta again.
Related
Attached is a simple python Kalman filter example of a free-fall object (g=-9.8m/s^2)
Alas, I have a problem. The state vector x contains both the position and the velocity but the z vector (measurement) contains only the position.
If I set a wrong initial position value, the algorithm coverages to the true value even with noisy measurements (see picture below)
However, if I sent the wrong initial velocity value, the algorithm does not converge even though the motion model is defined correctly.
Attached is the python code:
kalman.py
In your code I see two problems.
You set the Q-Matrix to zero. It means you trust too much in your model and give the filter no chance to improve the estimation through the measurement. Your filter becomes to stiff. You can think of it like a low pass filter with a very big time constant.
In my code I set the Q-Matrix to
Q = np.array([[1,0],[0,0.1]])
The second issue is your measurement noise. You simulate the noisy measurements with R=100 but communicate to the filter R=4. The filter trusts the measurement more than it should be. This issue is not really relevant to your question but still it should be corrected.
Now even if I set the initial velocity to 20, the position estimation works fine.
Here is the estimation for R = 4:
And for R = 100:
UPDATE
The velocity estimation works wrong, because you have some mistakes in your matrix operations. Please note, the matrix multiplication goes through np.dot(), not through *.
Here is a correct result for v0 = 20:
Many thanks, Anton.
Attached below is the corrected code for your convenience:
Roi
import numpy as np
import matplotlib.pyplot as plt
%matplotlib notebook
from numpy.linalg import inv
N = 1000 # number of time steps
dt = 0.01 # Sampling time (s)
t = dt*np.arange(N)
F = np.array([[1, dt],[ 0, 1]])# system matrix - state
B = np.array([[-1/2*dt**2],[ -dt]])# system matrix - input
H = np.array([[1, 0]])#; % observation matrix
Q = np.array([[1,0],[0,1]])
u = 9.80665# % input = acceleration due to gravity (m/s^2)
I = np.array([[1,0],[0,1]]) #identity matrix
# Define the initial position and velocity
y0 = 100; # m
v0 = 0; # m/s
G2 = np.array([-1/2*dt**2, -dt])# system matrix - input
# Initialize the state vector (true state)
xt = np.zeros((2, N)) # True state vector
xt[:,0] = [y0,v0]
for k in range(1,N):
xt[:,k] = np.dot(F,xt[:,k-1]) +G2*u
#Generate the noisy measurement from the true state
R = 4 # % m^2/s^2
v = np.sqrt(R)*np.random.randn(N) #% measurement noise
z = np.dot(H,xt) + v; #% noisy measurement
R2=4
#% Initialize the covariance matrix
P = np.array([[10, 0], [0, 0.1]])# Covariance for initial state error
#% Loop through and perform the Kalman filter equations recursively
x_list =[]
x_kalman= np.array([[117],[290]])
x_list.append(x_kalman)
print(-B*u)
for k in range(1,N):
x_kalman=np.dot(F,x_kalman) +B*u
P = np.dot(np.dot(F,P),F.T) +Q
S=(np.dot(np.dot(H,P),H.T) + R2)
S2 = inv(S)
K = np.dot(P,H.T)*S2
x_kalman = x_kalman +K*((z[:,k]- np.dot(H,x_kalman)))
P = np.dot((I - K*H),P)
x_list.append(x_kalman)
x_array = np.array(x_list)
print(x_array.shape)
plt.figure()
plt.plot(t,z[0,:], label="measurment", color='LIME', linewidth=1)
plt.plot(t,x_array[:,0,:],label="kalman",linewidth=5)
plt.plot(t,xt[0,:],linestyle='--', label = "Truth",linewidth=6)
plt.legend(fontsize=30)
plt.grid(True)
plt.xlabel("t[s]")
plt.title("Position Estimation", fontsize=20)
plt.ylabel("$X_t$ = h[m]")
plt.gca().set( ylim=(0, 110))
plt.gca().set(xlim=(0,6))
plt.figure()
#plt.plot(t,z, label="measurment", color='LIME')
plt.plot(t,x_array[:,1,:],label="kalman",linewidth=4)
plt.plot(t,xt[1,:],linestyle='--', label = "Truth",linewidth=2)
plt.legend()
plt.grid(True)
plt.xlabel("t[s]")
plt.title("Velocity Estimation")
plt.ylabel("$X_t$ = h[m]")
I'm new to python, I try to give some adjustment to the data, but when I get the graph, only the original data appears and with the message "Optimal parameters not found: Number of calls to function has reached maxfev = 1000." Could you help me find my mistake?
%matplotlib inline
import matplotlib.pylab as m
from scipy.optimize import curve_fit
import numpy as num
import scipy.optimize as optimize
xData=num.array([0,0,100,200,250,300,400], dtype="float")
yData=num.array([0,0,0,0,75,100,100], dtype="float")
m.plot(xData, yData, 'ro', label='Datos originales')
def fun(x, a, b):
return a + b * num.log(x)
popt,pcov=optimize.curve_fit(fun, xData, yData,p0=[1,1], maxfev=1000)
print=popt
x=num.linspace(1,400,7)
m.plot(x,fun(x, *popt), label='Función ajustada')
m.xlabel('concentración')
m.ylabel('% mortalidad')
m.legend()
m.grid()
The model in your code is "a + b * num.log(x)". Because your data contains an x value of 0.0, the evaluation of log(0.0) gives errors and will not allow the fitting software to function. Sometimes these x values of 0.0 can be replaced with very small numbers, as log(small number) will not fail - but in this case the equation and data do not appear to match and so using that technique alone would not be sufficient here.
My thought is that a different equation would be a better model for this data. I performed an equation search using your data, and found that several different sigmoidal type equations gave suspiciously good fits to this data set - which is not surprising because of the small number of data points.
The sigmoidal equations I tried were all extremely sensitive to the initial parameter estimates. Here is a graphical Python fitter using scipy's Differential Evolution genetic algorithm module to determine the initial parameter estimates for curve_fit's non-linear solver. That scipy module uses the Latin Hypercube algorithm to ensure a thorough search of parameter space, requiring bounds within which to search. Here those bounds are taken from the data maximum and minimun values.
I personally would not use this fit precisely because the small number of data points is giving such suspiciously good fits, and strongly recommend taking additional data points if at all possible. I could however not find any equations with less than three parameters that would fit the data.
import numpy, scipy, matplotlib
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
from scipy.optimize import differential_evolution
import warnings
xData=numpy.array([0,0,100,200,250,300,400], dtype="float")
yData=numpy.array([0,0,0,0,75,100,100], dtype="float")
def func(x, a, b, c): # Sigmoid B equation from zunzun.com
return a / (1.0 + numpy.exp(-1.0 * (x - b) / c))
# function for genetic algorithm to minimize (sum of squared error)
def sumOfSquaredError(parameterTuple):
warnings.filterwarnings("ignore") # do not print warnings by genetic algorithm
val = func(xData, *parameterTuple)
return numpy.sum((yData - val) ** 2.0)
def generate_Initial_Parameters():
# min and max used for bounds
maxX = max(xData)
minX = min(xData)
parameterBounds = []
parameterBounds.append([minX, maxX]) # search bounds for a
parameterBounds.append([minX, maxX]) # search bounds for b
parameterBounds.append([0.0, 2.0]) # search bounds for c
# "seed" the numpy random number generator for repeatable results
result = differential_evolution(sumOfSquaredError, parameterBounds, seed=3)
return result.x
# by default, differential_evolution completes by calling curve_fit() using parameter bounds
geneticParameters = generate_Initial_Parameters()
# now call curve_fit without passing bounds from the genetic algorithm,
# just in case the best fit parameters are aoutside those bounds
fittedParameters, pcov = curve_fit(func, xData, yData, geneticParameters)
print('Fitted parameters:', fittedParameters)
print()
modelPredictions = func(xData, *fittedParameters)
absError = modelPredictions - yData
SE = numpy.square(absError) # squared errors
MSE = numpy.mean(SE) # mean squared errors
RMSE = numpy.sqrt(MSE) # Root Mean Squared Error, RMSE
Rsquared = 1.0 - (numpy.var(absError) / numpy.var(yData))
print()
print('RMSE:', RMSE)
print('R-squared:', Rsquared)
print()
##########################################################
# graphics output section
def ModelAndScatterPlot(graphWidth, graphHeight):
f = plt.figure(figsize=(graphWidth/100.0, graphHeight/100.0), dpi=100)
axes = f.add_subplot(111)
# first the raw data as a scatter plot
axes.plot(xData, yData, 'D')
# create data for the fitted equation plot
xModel = numpy.linspace(min(xData), max(xData), 100)
yModel = func(xModel, *fittedParameters)
# now the model as a line plot
axes.plot(xModel, yModel)
axes.set_xlabel('X Data') # X axis data label
axes.set_ylabel('Y Data') # Y axis data label
plt.show()
plt.close('all') # clean up after using pyplot
graphWidth = 800
graphHeight = 600
ModelAndScatterPlot(graphWidth, graphHeight)
I need help with curve fitting a given set of points. The points form a parabola and I ought to find the peak point of the result. Issue is when I do a curve fit, it sometimes doesn't touch the max y-coordinate even if the actual point is given in the input array.
Following is the code snippet. Here 1.88 is the actual peak y-coordinate (13.05,1.88). But the graph generated by the code does not touch the point due to curve fitting. So is there a way to fit the curve making sure that it touches the max point given in the input array?
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit, minimize_scalar
fig = plt.gcf()
#fig.set_size_inches(18.5, 10.5)
x = [4.59,9.02,13.05,18.47,20.3]
y = [1.7,1.84,1.88,1.7,1.64]
def f(x, p1, p2, p3):
return p3*(p1/((x-p2)**2 + (p1/2)**2))
plt.plot(x,y,"ro")
popt, pcov = curve_fit(f, x, y)
# find the peak
fm = lambda x: -f(x, *popt)
r = minimize_scalar(fm, bounds=(1, 5))
print( "maximum:", r["x"], f(r["x"], *popt) ) #maximum: 2.99846874275 18.3928199902
plt.text(1,1.9,'maximum '+str(round(r["x"],2))+'( #'+str(round(f(r["x"], *popt),2)) + ' )')
x_curve = np.linspace(min(x), max(x), 50)
plt.plot(x_curve, f(x_curve, *popt))
plt.plot(r['x'], f(r['x'], *popt), 'ko')
plt.show()
Here is a graphical code example using your equation with weighted fitting, where I have made the max point larger to more easily see the effect of the weighting. In non-weighted curve fitting, all weights are implicitly 1.0 as all data points have equal weight. Scipy's curve_fit routine uses weights in the form of uncertainties, so that giving a point a very small uncertainty (which I have done) is like giving the point a very large weight. This technique can be used to make a fit pass arbitrarily close to any single data point by any software that can perform weghted fitting.
import numpy, scipy, matplotlib
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
x = [4.59,9.02,13.05,18.47,20.3]
y = [1.7,1.84,2.0,1.7,1.64]
# note the single very small uncertainty - try making this value 1.0
uncertainties = numpy.array([1.0, 1.0, 1.0E-6, 1.0, 1.0])
# rename data to use previous example
xData = numpy.array(x)
yData = numpy.array(y)
def func(x, p1, p2, p3):
return p3*(p1/((x-p2)**2 + (p1/2)**2))
# these are the same as the scipy defaults
initialParameters = numpy.array([1.0, 1.0, 1.0])
# curve fit the test data, first without uncertainties to
# get us closer to initial starting parameters
ssqParameters, pcov = curve_fit(func, xData, yData, p0 = initialParameters)
# now that we have better starting parameters, use uncertainties
fittedParameters, pcov = curve_fit(func, xData, yData, p0 = ssqParameters, sigma=uncertainties, absolute_sigma=True)
modelPredictions = func(xData, *fittedParameters)
absError = modelPredictions - yData
SE = numpy.square(absError) # squared errors
MSE = numpy.mean(SE) # mean squared errors
RMSE = numpy.sqrt(MSE) # Root Mean Squared Error, RMSE
Rsquared = 1.0 - (numpy.var(absError) / numpy.var(yData))
print('Parameters:', fittedParameters)
print('RMSE:', RMSE)
print('R-squared:', Rsquared)
print()
##########################################################
# graphics output section
def ModelAndScatterPlot(graphWidth, graphHeight):
f = plt.figure(figsize=(graphWidth/100.0, graphHeight/100.0), dpi=100)
axes = f.add_subplot(111)
# first the raw data as a scatter plot
axes.plot(xData, yData, 'D')
# create data for the fitted equation plot
xModel = numpy.linspace(min(xData), max(xData))
yModel = func(xModel, *fittedParameters)
# now the model as a line plot
axes.plot(xModel, yModel)
axes.set_xlabel('X Data') # X axis data label
axes.set_ylabel('Y Data') # Y axis data label
plt.show()
plt.close('all') # clean up after using pyplot
graphWidth = 800
graphHeight = 600
ModelAndScatterPlot(graphWidth, graphHeight)
I'm trying to implement a perceptual-based image searching engine, that will allow users to find pictures, containing objects of relatively same or close colours to the user-specified template(object from the sample image).
The goal for now is not to match a precise object, but rather to find any significant areas that are close in color to the template. I am stuck with indexing my dataset.
I have tried some clustering algorithms, such as k-means from sklearn.cluster (as I've read from this article), to select centroids from the sample image as my features, that are eventually in CIELab color space to acquire more perceptual uniformity. But it doesn't seem to work well, as cluster centres are generated randomly and thus I've got poor metrics results even on an object and image, from which that same object was extracted!
As far as I'm concerned, a common algorithm in simple image searching programs is using distance between histograms, which is not acceptable as I try to sustain perceptually-valid colour difference, and by that I mean that I can only manage two separate colours (and maybe some additional values) to calculate metrics in CIELab colour space. I am using CMCl:c metric of my own implementation, and it produced good results so far.
Maybe someone can help me and recommend an algorithm more suitable for my purpose.
Some code that I've done so far:
import cv2 as cv
import numpy as np
from sklearn.cluster import KMeans, MiniBatchKMeans
from imageproc.color_metrics import *
def feature_extraction(image, features_length=6):
width, height, dimensions = tuple(image.shape)
image = cv.cvtColor(image, cv.COLOR_BGR2LAB)
image = cv.medianBlur(image, 7)
image = np.reshape(image, (width * height, dimensions))
clustering_handler = MiniBatchKMeans(n_init=40, tol=0.0, n_clusters=features_length, compute_labels=False,
max_no_improvement=10, max_iter=200, reassignment_ratio=0.01)
clustering_handler.fit(image)
features = np.array(clustering_handler.cluster_centers_, dtype=np.float64)
features[:, :1] /= 255.0
features[:, :1] *= 100.0
features[:, 1:2] -= 128.0
features[:, 2:3] -= 128.0
return features
if __name__ == '__main__':
first_image_name = object_image_name
second_image_name = image_name
sample_features = list()
reference_features = list()
for name, features in zip([first_image_name, second_image_name], [sample_features, reference_features]):
image = cv.imread(name)
features.extend(feature_extraction(image, 6))
distance_matrix = np.ndarray((6, 6))
distance_mappings = {}
for n, i in enumerate(sample_features):
for k, j in enumerate(reference_features):
distance_matrix[n][k] = calculate_cmc_distance(i, j)
distance_mappings.update({distance_matrix[n][k]: (i, j)})
minimal_distances = []
for i in distance_matrix:
minimal_distances.append(min(i))
minimal_distances = sorted(minimal_distances)
print(minimal_distances)
for ii in minimal_distances:
i, j = distance_mappings[ii]
color_plate1 = np.zeros((300, 300, 3), np.float32)
color_plate2 = np.zeros((300, 300, 3), np.float32)
color1 = cv.cvtColor(np.float32([[i]]), cv.COLOR_LAB2BGR)[0][0]
color2 = cv.cvtColor(np.float32([[j]]), cv.COLOR_LAB2BGR)[0][0]
color_plate1[:] = color1
color_plate2[:] = color2
cv.imshow("s", np.hstack((color_plate1, color_plate2)))
cv.waitKey()
print(sum(minimal_distances))
The usual approach would be to cluster only once, with a representative sample from all images.
This is a preprocessing step, to generate your "dictionary".
Then for feature extraction, you would map points to the fixed cluster centers, that are now shared across all images. This is a simple nearest-neighbor mapping, no clustering.
Let's have a corrupted Image C, Bias profile, B and True Image A. So if we can define a model,
C = A * B;
We can get the original Image back as,
A = C / B;
in the log domain,
log A = log C - log B.
Now let's say, I have true image, A and I am introducing the bias B and I am getting the corrupted image C. Now I can correct this biased Image, C using the polynomial regression. I will fit the surface once I convert the corrupted image C in the log domain, and I can subtract the bias profile from it as shown above. After the subtraction, I don't need to apply exp(log C - log B) as obvious. Onlu normalization is needed to get [0 255] range.
Algorithm:
Original Image without any bias field is introduced with the polynomial profile, which results in an image having non-uniform illumination.
Biased image is converted in log domain and surface is approximated using polynomial fit
approximated surface is subtracted from the Biased image which results in original image back with no bias fields.(Ideally).
measure RMSE between approximated surface and introduced polynomial field in step 1. Measure RMSE between the Biased Image and the image we get back at the end after subtraction.
Code:
clear;clc;close all;
%read the image, on which profile is to be generated
I = ones(300);
I = padarray(I,[20,20],'both','symmetric'); % padding
%%
%creating a bias profile using polynomial modeling
[x,y] = meshgrid(1:size(I,1),1:size(I,2));
profile = -2.5.*x.^3 - 2.5.* y.^3 + 0.25 .*(x.* y.^2) - 0.3*( x.^2 .* y ) - 0.5.* x .* y - x + y - 2.5*( x.^2) - y.^2 + 0.5 .* x .*y + 1;
% come to scale [0 1]
profile = profile - min(profile(:));
profile = profile / max(profile(:));
figure,imshow(profile,[]); %introduced bias profile
%% corrupt the image
biasedImage = (I .* profile);
figure,imshow(biasedImage,[]); %biased Image
cImage = log(biasedImage+1);% conversion to log domain/ +1 is needed to avoid infinite values in case of 0 intensty values in corrupted image.
%% forming the input for prediction of surface
colorChannel = cImage;
rows = size(colorChannel, 1);
columns = size(colorChannel, 2);
[X,Y] = meshgrid(1:columns, 1:rows);
z = colorChannel;
x1d = reshape(X, numel(X), 1);
y1d = reshape(Y, numel(Y), 1);
z1d = double(reshape(z, numel(z), 1)); %dependent variables
x = [x1d y1d]; % two regressors
%% surface fitting
m = 3; %define the order of polynomial
p = polyfitn(x,z1d,m); %prediction step
zg = polyvaln(p, x);
modeledColorChannel = reshape(zg, [rows columns]); % predicted surface
%modeledColorChannel = exp(modeledColorChannel)-1; if we use this step then the step below will be division instead of subtraction
%f = biasedImage./ modeledColorChannel; Same as the step below but as we are using exponential, it will be devision.
%% correction
f = cImage- modeledColorChannel; %getting the original image back.
%grayImage = f(21:end-20,21:end-20);
%modeledColorChannel = modeledColorChannel(21:end-20,21:end-20); %to remove the padding
figure,imshow(f,[]);
figure,imshow(modeledColorChannel,[]);
%% measure the RMSE for image
y = (I - f);
RMSE = sqrt(mean(y(:).^2));
disp(RMSE);
% RMSE for profile
z = (modeledColorChannel - profile);
RMSE = sqrt(mean(z(:).^2));
disp(RMSE);
Results:
In case of: f = cImage- modeledColorChannel
1.0000
0.2127
Corrected Image: enter image description here
In case of division: f = cImage ./ modeledColorChannel (although it is not correct as per theory.)
0.0190
0.2127
Corrected Image:enter image description here
Now, the question is: I am getting lower RMSE value at the end if I do division in the log domain instead of subtraction as I am doing here(See %% correction section). How does it possible to have higher RMSE for subtraction where it is theoriticaly correct? As per my understanding if I keep all of my calculation in log domain image division will become image subtraction.It is obvious if you run the code and see the image f at the end of the correction for division and subtraction in log domain.
Note: In both the cases, RMSE between the introduced and perceived profile is same as I am doing my estimation in log domain in both the cases.Either image division or in image subtraction.
See this for polyfitn tool box.
www.mathworks.com/matlabcentral/fileexchange/34765-polyfitn
Let me add to the answer to my question as I found my mistake, in case anybody faces same issue in future.
Mistake 1: After the subtraction, I don't need to apply exp(log C - log B) as obvious. Only normalisation is needed to get [0 255] range.
My intuition was, I don't need to apply exp() to get the original values back. But in fact I have to apply exp(). Log-log on LHS and RHS never cancels each other.
if log(A) = log(B), to get the values of A back I need A = exp(log(B)).
Mistake 2: In log domain, I subtract two images so I do not have to face infinity problem in the log domain, we usually face in case of division.
So simply while converting image in log domain, I could just do,
cImage = log(biasedImage);
instead of,
cImage = log(biasedImage+1);
Here, adding +1 is creating the unwanted blur in the images because in estimation while predicting surface it will push the surface to high values in the dark areas.