Related
I am going to implement a salient object detection method based on a simple linear feedback control system (LFCS). The control system model is represented as in the following equation:
I've come up with the following program codes but the result would not be what should be. Specifically, the output should be something like the following image:
But the code produces this output:
The codes are as follows.
%Calculation of euclidian distance between adjacent superpixels stores in variable of Euc
A = imread('aa.jpg');
[rows, columns, cnumberOfColorChannels] = size(A);
[L,N] = superpixels(A,400);
%% Determination of adjacent superpixels
glcms = graycomatrix(L,'NumLevels',N,'GrayLimits',[1,N],'Offset',[0,1;1,0]); %Create gray-level co-occurrence matrix from image
glcms = sum(glcms,3); % add together the two matrices
glcms = glcms + glcms.'; % add upper and lower triangles together, make it symmetric
glcms(1:N+1:end) = 0; % set the diagonal to zero, we don't want to see "1 is neighbor of 1"
idx = label2idx(L); % Convert label matrix to cell array of linear indices
numRows = size(A,1);
numCols = size(A,2);
%%Mean color in Lab color space for each channel
data = zeros(N,3);
for labelVal = 1:N
redIdx = idx{labelVal};
greenIdx = idx{labelVal}+numRows*numCols;
blueIdx = idx{labelVal}+2*numRows*numCols;
data(labelVal,1) = mean(A(redIdx));
data(labelVal,2) = mean(A(greenIdx));
data(labelVal,3) = mean(A(blueIdx));
end
Euc=zeros(N);
%%Calculation of euclidian distance between adjacent superpixels stores in Euc
for a=1:N
for b=1:N
if glcms(a,b)~=0
Euc(a,b)=sqrt(((data(a,1)-data(b,1))^2)+((data(a,2)-data(b,2))^2)+((data(a,3)-data(b,3))^2));
end
end
end
%%Creation of Connectivity matrix "W" between adjacent superpixels
W=zeros(N);
W_num=zeros(N);
W_den=zeros(N);
OMG1=0.1;
for c=1:N
for d=1:N
if(Euc(c,d)~=0)
W_num(c,d)=exp(-OMG1*(Euc(c,d)));
W_den(c,c)=W_num(c,d)+W_den(c,c); %
end
end
end
%Connectivity matrix W between adjacent superpixels
for e=1:N
for f=1:N
if(Euc(e,f)~=0)
W(e,f)=(W_num(e,f))/(W_den(e,e));
end
end
end
%%calculation of geodesic distance between nonadjacent superpixels stores in variable "s_star_temp"
s_star_temp=zeros(N); %temporary variable for geodesic distance measurement
W_sparse=zeros(N);
W_sparse=sparse(W);
for g=1:N
for h=1:N
if W(g,h)==0 & g~=h;
s_star_temp(g,h)=graphshortestpath(W_sparse,g,h,'directed',false);
end
end
end
%%Calculation of connectivity matrix for nonadjacent superpixels stores in "S_star" variable"
S_star=zeros(N);
OMG2=8;
for i=1:N
for j=1:N
if s_star_temp(i,j)~=0
S_star(i,j)=exp(-OMG2*s_star_temp(i,j));
end
end
end
%%Calculation of connectivity matrix "S" for measuring connectivity between all superpixels
S=zeros(N);
S=S_star+W;
%% Defining non-isolation level for connectivity matrix "W"
g_star=zeros(N);
for k=1:N
g_star(k,k)=max(W(k,:));
end
%%Limiting the range of g_star and calculation of isolation cue matrix "G"
alpha1=0.15;
alpha2=0.85;
G=zeros(N);
for l=1:N
G(l,l)=alpha1*(g_star(l,l)- min(g_star(:)))/(max(g_star(:))- min(g_star(:)))+(alpha2 - alpha1);
end
%%Determining the supperpixels that surrounding the image boundary
lr = L([1,end],:);
tb = L(:,[1,end]);
labels = unique([lr(:);tb(:)]);
%% Calculation of background likelihood for each superpixels stores in"BgLike"
sum_temp=0;
temp=zeros(1,N);
BgLike=zeros(N,1);
BgLike_num=zeros(N);
BgLike_den=zeros(N);
for m=1:N
for n=1:N
if ismember(n,labels)==1
BgLike_num(m,m)=S(m,n)+ BgLike_num(m,m);
end
end
end
for o=1:N
for p=1:N
for q=1:N
if W(p,q)~=0
temp(q)=S(o,p)-S(o,q);
end
end
sum_temp=max(temp)+sum_temp;
temp=0;
end
BgLike_den(o,o)=sum_temp;
sum_temp=0;
end
for r=1:N
BgLike(r,1)= BgLike_num(r,r)/BgLike_den(r,r);
end
%%%%Calculation of Foreground likelihood for each superpixels stores in "FgLike"
FgLike=zeros(N,1);
for s=1:N
for t=1:N
FgLike(s,1)=(exp(-BgLike(t,1))) * Euc(s,t)+ FgLike(s,1);
end
end
The above codes are prerequisite for the following sections (in fact, they produce necessary data and matrices for the next section. The aforementioned codes provided to make the whole process reproducible).
Specifically, I think that this section did not give the desired results. I'm afraid I did not properly simulate the parallelism using for loops. Moreover, the terminating conditions (employed with for and if statements to simulate do-while loop) are never satisfied and the loops continue until the last iteration (instead terminating when a specified condition occurs). A major concern here is that if the terminating conditions are properly implemented.
The pseudo algorithm for the following code is as the image below:
%%parallel operations for background and foreground implemented here
T0 = 0 ;
Tf = 20 ;
Ts = 0.1 ;
Ti = T0:Ts:Tf ;
Nt=numel(Ti);
Y_Bg=zeros(N,Nt);
Y_Fg=zeros(N,Nt);
P_Back_Bg=zeros(N,N);
P_Back_Fg=zeros(N,N);
u_Bg=zeros(N,Nt);
u_Fg=zeros(N,Nt);
u_Bg_Star=zeros(N,Nt);
u_Fg_Star=zeros(N,Nt);
u_Bg_Normalized=zeros(N,Nt);
u_Fg_Normalized=zeros(N,Nt);
tau=0.1;
sigma_Bg=zeros(Nt,N);
Temp_Bg=0;
Temp_Fg=0;
C_Bg=zeros(Nt,N);
C_Fg=zeros(Nt,N);
%%System Initialization
for u=1:N
u_Bg(u,1)=(BgLike(u,1)- min(BgLike(:)))/(max(BgLike(:))- min(BgLike(:)));
u_Fg(u,1)=(FgLike(u,1)- min(FgLike(:)))/(max(FgLike(:))- min(FgLike(:)));
end
%% P_state and P_input
P_state=G*W;
P_input=eye(N)-G;
% State Initialization
X_Bg=zeros(N,Nt);
X_Fg=zeros(N,Nt);
for v=1:20 % v starts from 1 because we have no matrices with 0th column number
%The first column of X_Bg and X_Fg is 0 for system initialization
X_Bg(:,v+1)=P_state*X_Bg(:,v) + P_input*u_Bg(:,v);
X_Fg(:,v+1)=P_state*X_Fg(:,v) + P_input*u_Fg(:,v);
v=v+1;
if v==2
C_Bg(1,:)=1;
C_Fg(1,:)=1;
else
for w=1:N
for x=1:N
Temp_Fg=S(w,x)*X_Fg(x,v-1)+Temp_Fg;
Temp_Bg=S(w,x)*X_Bg(x,v-1)+Temp_Bg;
end
C_Fg(v-1,w)=inv(X_Fg(w,v-1)+((Temp_Bg)/(Temp_Fg)*(1-X_Fg(w,v-1))));
C_Bg(v-1,w)=inv(X_Bg(w,v-1)+((Temp_Fg)/(Temp_Bg))*(1-X_Bg(w,v-1)));
Temp_Bg=0;
Temp_Fg=0;
end
end
P_Bg=diag(C_Bg(v-1,:));
P_Fg=diag(C_Fg(v-1,:));
Y_Bg(:,v)= P_Bg*X_Bg(:,v);
Y_Fg(:,v)= P_Fg*X_Fg(:,v);
for y=1:N
Temp_sig_Bg=0;
Temp_sig_Fg=0;
for z=1:N
Temp_sig_Bg = Temp_sig_Bg +S(y,z)*abs(Y_Bg(y,v)- Y_Bg(z,v));
Temp_sig_Fg = Temp_sig_Fg +S(y,z)*abs(Y_Fg(y,v)- Y_Fg(z,v));
end
if Y_Bg(y,v)>= Y_Bg(y,v-1)
sign_Bg=1;
else
sign_Bg=-1;
end
if Y_Fg(y,v)>= Y_Fg(y,v-1)
sign_Fg=1;
else
sign_Fg=-1;
end
sigma_Bg(v-1,y)=sign_Bg*Temp_sig_Bg;
sigma_Fg(v-1,y)=sign_Fg*Temp_sig_Fg;
end
%Calculation of P_Back for background and foreground
P_Back_Bg=tau*diag(sigma_Bg(v-1,:));
P_Back_Fg=tau*diag(sigma_Fg(v-1,:));
u_Bg_Star(:,v)=u_Bg(:,v-1)+P_Back_Bg*Y_Bg(:,v);
u_Fg_Star(:,v)=u_Fg(:,v-1)+P_Back_Fg*Y_Fg(:,v);
for aa=1:N %Normalization of u_Bg and u_Fg
u_Bg(aa,v)=(u_Bg_Star(aa,v)- min(u_Bg_Star(:,v)))/(max(u_Bg_Star(:,v))-min(u_Bg_Star(:,v)));
u_Fg(aa,v)=(u_Fg_Star(aa,v)- min(u_Fg_Star(:,v)))/(max(u_Fg_Star(:,v))-min(u_Fg_Star(:,v)));
end
if (max(abs(Y_Fg(:,v)-Y_Fg(:,v-1)))<=0.0118) &&(max(abs(Y_Bg(:,v)-Y_Bg(:,v-1)))<=0.0118) %% epsilon= 0.0118
break;
end
end
Finally, the saliency map will be generated by using the following codes.
K=4;
T=0.4;
phi_1=(2-(1-T)^(K-1))/((1-T)^(K-2));
phi_2=(1-T)^(K-1);
phi_3=1-phi_1;
for bb=1:N
Y_Output_Preliminary(bb,1)=Y_Fg(bb,v)/((Y_Fg(bb,v)+Y_Bg(bb,v)));
end
for hh=1:N
Y_Output(hh,1)=(phi_1*(T^K))/(phi_2*(1-Y_Output_Preliminary(hh,1))^K+(T^K))+phi_3;
end
V_rs=zeros(N);
V_Final=zeros(rows,columns);
for cc=1:rows
for dd=1:columns
V_rs(cc,dd)=Y_Output(L(cc,dd),1);
end
end
maxDist = 10; % Maximum chessboard distance from image
wSF=zeros(rows,columns);
wSB=zeros(rows,columns);
% Get the range of x and y indices who's chessboard distance from pixel (0,0) are less than 'maxDist'
xRange = (-(maxDist-1)):(maxDist-1);
yRange = (-(maxDist-1)):(maxDist-1);
% Create a mesgrid to get the pairs of (x,y) of the pixels
[pointsX, pointsY] = meshgrid(xRange, yRange);
pointsX = pointsX(:);
pointsY = pointsY(:);
% Remove pixel (0,0)
pixIndToRemove = (pointsX == 0 & pointsY == 0);
pointsX(pixIndToRemove) = [];
pointsY(pixIndToRemove) = [];
for ee=1:rows
for ff=1:columns
% Get a shifted copy of 'pointsX' and 'pointsY' that is centered
% around (x, y)
pointsX1 = pointsX + ee;
pointsY1 = pointsY + ff;
% Remove the the pixels that are out of the image bounds
inBounds =...
pointsX1 >= 1 & pointsX1 <= rows &...
pointsY1 >= 1 & pointsY1 <= columns;
pointsX1 = pointsX1(inBounds);
pointsY1 = pointsY1(inBounds);
% Do stuff with 'pointsX1' and 'pointsY1'
wSF_temp=0;
wSB_temp=0;
for gg=1:size(pointsX1)
Temp=exp(-OMG1*(sqrt(double(A(pointsX1(gg),pointsY1(gg),1))-double(A(ee,ff,1)))^2+(double(A(pointsX1(gg),pointsY1(gg),2))-double(A(ee,ff,2)))^2 + (double(A(pointsX1(gg),pointsY1(gg),3))-double(A(ee,ff,3)))^2));
wSF_temp=wSF_temp+(Temp*V_rs(pointsX1(gg),pointsY1(gg)));
wSB_temp=wSB_temp+(Temp*(1-V_rs(pointsX1(gg),pointsY1(gg))));
end
wSF(ee,ff)= wSF_temp;
wSB(ee,ff)= wSB_temp;
V_Final(ee,ff)=V_rs(ee,ff)/(V_rs(ee,ff)+(wSB(ee,ff)/wSF(ee,ff))*(1-V_rs(ee,ff)));
end
end
imshow(V_Final,[]); %%Saliency map of the image
Part of your terminating criterion is this:
max(abs(Y_a(:,t)-Y_a(:,t-1)))<=eps
Say Y_a tends to 2. You are really close... In fact, the closest you can get without subsequent values being identical is Y_a(t)-Y_a(t-1) == 4.4409e-16. If the two values were any closer, their difference would be 0, because this is the precision with which floating-point values can be represented. So you have reached this fantastic level of closeness to your goal. Subsequent iterations are changing the target value by the smallest possible amount, 4.4409e-16. But your test is returning false! Why? Because eps == 2.2204e-16!
eps is short-hand for eps(1), the difference between 1 and the next representable larger value. Because how floating-point values are represented, this difference is half the difference between 2 and the next representable larger value (which is given by eps(2).
However, if Y_a tends to 1e-16, subsequent iterations could double or halve the value of Y_a and you'd still meet the stopping criterion!
Thus, what you need is to come up with a reasonable stopping criterion that is a fraction of the target value, something like this:
max(abs(Y_a(:,t)-Y_a(:,t-1))) <= 1e6 * eps(max(abs(Y_a(:,t))))
Unsolicited advice
You should really look into vectorized operations in MATLAB. For example,
for y=1:N
Temp_sig_a=0;
for z=1:N
Temp_sig_a = Temp_sig_a + abs(Y_a(y,t)- Y_a(z,t));
end
sigma_a(t-1,y)= Temp_sig_a;
end
can be written as
for y=1:N
sigma_a(t-1,y) = sum(abs(Y_a(y,t) - Y_a(:,t)));
end
which in turn can be written as
sigma_a(t-1,:) = sum(abs(Y_a(:,t).' - Y_a(:,t)));
Avoiding loops is not only usually more efficient, but it also leads to shorter code that is easier to read.
Also, this:
P_FB_a = diag(sigma_a(t-1,:));
u_a(:,t) = u_a(:,t-1) + P_FB_a * Y_a(:,t);
is the same as
u_a(:,t) = u_a(:,t-1) + sigma_a(t-1,:).' .* Y_a(:,t);
but of course creating a diagonal matrix and doing a matrix multiplication with so many zeros is much more expensive than directly computing an element-wise multiplication.
Hi I'm trying to implement Gradient Descent algorithm for a function:
My starting point for the algorithm is w = (u,v) = (2,2). The learning rate is eta = 0.01 and bound = 10^-14. Here is my MATLAB code:
function [resultTable, boundIter] = gradientDescent(w, iters, bound, eta)
% FUNCTION [resultTable, boundIter] = gradientDescent(w, its, bound, eta)
%
% DESCRIPTION:
% - This function will do gradient descent error minimization for the
% function E(u,v) = (u*exp(v) - 2*v*exp(-u))^2.
%
% INPUTS:
% 'w' a 1-by-2 vector indicating initial weights w = [u,v]
% 'its' a positive integer indicating the number of gradient descent
% iterations
% 'bound' a real number indicating an error lower bound
% 'eta' a positive real number indicating the learning rate of GD algorithm
%
% OUTPUTS:
% 'resultTable' a iters+1-by-6 table indicating the error, partial
% derivatives and weights for each GD iteration
% 'boundIter' a positive integer specifying the GD iteration when the error
% function got below the given error bound 'bound'
%
% The error function
E = #(u,v) (u*exp(v) - 2*v*exp(-u))^2;
% Partial derivative of E with respect to u
pEpu = #(u,v) 2*(u*exp(v) - 2*v*exp(-u))*(exp(v) + 2*v*exp(-u));
% Partial derivative of E with respect to v
pEpv = #(u,v) 2*(u*exp(v) - 2*v*exp(-u))*(u*exp(v) - 2*exp(-u));
% Initialize boundIter
boundIter = 0;
% Create a table for holding the results
resultTable = zeros(iters+1, 6);
% Iteration number
resultTable(1, 1) = 0;
% Error at iteration i
resultTable(1, 2) = E(w(1), w(2));
% The value of pEpu at initial w = (u,v)
resultTable(1, 3) = pEpu(w(1), w(2));
% The value of pEpv at initial w = (u,v)
resultTable(1, 4) = pEpv(w(1), w(2));
% Initial u
resultTable(1, 5) = w(1);
% Initial v
resultTable(1, 6) = w(2);
% Loop all the iterations
for i = 2:iters+1
% Save the iteration number
resultTable(i, 1) = i-1;
% Update the weights
temp1 = w(1) - eta*(pEpu(w(1), w(2)));
temp2 = w(2) - eta*(pEpv(w(1), w(2)));
w(1) = temp1;
w(2) = temp2;
% Evaluate the error function at new weights
resultTable(i, 2) = E(w(1), w(2));
% Evaluate pEpu at the new point
resultTable(i, 3) = pEpu(w(1), w(2));
% Evaluate pEpv at the new point
resultTable(i, 4) = pEpv(w(1), w(2));
% Save the new weights
resultTable(i, 5) = w(1);
resultTable(i, 6) = w(2);
% If the error function is below a specified bound save this iteration
% index
if E(w(1), w(2)) < bound
boundIter = i-1;
end
end
This is an exercise in my machine learning course, but for some reason my results are all wrong. There must be something wrong in the code. I have tried debugging and debugging it and haven't found anything wrong...can someone identify what is my problem here?...In other words can you check that the code is valid gradient descent algorithm for the given function?
Please let me know if my question is too unclear or if you need more info :)
Thank you for your effort and help! =)
Here is my results for five iterations and what other people got:
PARAMETERS: w = [2,2], eta = 0.01, bound = 10^-14, iters = 5
As discussed below the question: I would say the others are wrong... your minimization leads to smaller values of E(u,v), check:
E(1.4,1.6) = 37.8 >> 3.6 = E(0.63, -1.67)
Not a complete answer but lets go for it:
I added a plotting part in your code, so you can see whats going on.
u1=resultTable(:,5);
v1=resultTable(:,6);
E1=E(u1,v1);
E1(E1<bound)=NaN;
[x,y]=meshgrid(-1:0.1:5,-5:0.1:2);Z=E(x,y);
surf(x,y,Z)
hold on
plot3(u1,v1,E1,'r')
plot3(u1,v1,E1,'r*')
The result shows that your algorithm is doing the right thing for that function. So, as other said, or all the others are wrong, or you are not using the right equation from the beggining.
(I apologize for not just commenting, but I'm new to SO and cannot comment.)
It appears that your algorithm is doing the right thing. What you want to be sure is that at each step the energy is shrinking (which it is). There are several reasons why your data points may not agree with the others in the class: they could be wrong (you or others in the class), they perhaps started at a different point, they perhaps used a different step size (what you are calling eta I believe).
Ideally, you don't want to hard-code the number of iterations. You want to continue until you reach a local minimum (which hopefully is the global minimum). To check this, you want both partial derivatives to be zero (or very close). In addition, to make sure you're at a local min (not a local max, or saddle point) you should check the sign of E_uu*E_vv - E_uv^2 and the sign of E_uu look at: http://en.wikipedia.org/wiki/Second_partial_derivative_test for details (the second derivative test, at the top). If you find yourself at a local max or saddle point, your gradient will tell you not to move (since the partial derivatives are 0). Since you know this isn't optimal, you have to just perturb your solution (sometimes called simulated annealing).
Hope this helps.
I built a binary chessboard (tab):
Now I have to put in several pieces (pawns), just like this:
P_1 and P_2 have the same dimension that each block of the chessboard (301x301 px)
like it follows:
P_1 :
P_2:
In another exercise I had to invert the images of row 2, doing this:
P1_neg=double(P_1)*-1+255;
P2_neg=double(P_2)*-1+255;
I obtain this images:
P1_neg:
P2_neg
In the second row I have to paste two kind of pieces: a black pawn with black background (P_1), and a black pawn with white background (P_2).
Each block of the chessboard has 301x301 pixels, and each piece has the same measurements. I have to use only conditions and loops, but I don't know how paste the pieces there.
So, I know for the second row: when the row (i) it even and the column(j) is odd, I have to paste P_1; and when the row is even and the column is also even I have to paste P_2.
My code, is the following one, I hope you can help me :)
for i=1:8 % from 1 to 8 because it have 8 blocks
for j=1:8
if mod(i,2)==0 && mod (j,2)~=0
%paste P_1 (I don't know how do this)
elseif mod(i,2)==0 && mod(j,2)==0
%paste P_2
end
end
end
figure,imshow() % I don't know if I show the chessboard or I have to save the data in another matrix.
I couldn't upload the chessboard with the original resolution, but here it's the code I use
tab=zeros(2408);
for i=0:7
for j=0:7
if mod(i,2)~=0 && mod(j,2)~=0
for avanza_fila=(i*301)+1:(i+1)*301
for avanza_columna= (j*301)+1:(j+1)*301
tab(avanza_fila,avanza_columna)=255;
end
end
elseif mod(i,2)==0 && mod(j,2)==0
for avanza_fila=(i*301)+1:(i+1)*301
for avanza_columna= (j*301)+1:(j+1)*301
tab(avanza_fila,avanza_columna)=255;
end
end
end
end
end
figure, imshow(tab)
Here's my solution to your problem. I would personally avoid for loops, but you said your assignment needs it. I've created a high resolution version of your board that you can get here:
This board is 2408 x 2408, as each chess piece is 301 x 301, on an 8 x 8 grid. You almost have the logic correct. You just need to figure out how exactly to place the tiles on the board. I wrote two for loops where the first for loop uses rows 2 and 7, and the second for loop goes through columns 1 to 8. For either the second or seventh row, we check to see whether the column is even or odd and place the correct chess piece in its corresponding block. I also converted the board and the images to black and white to get rid of any quantization artifacts, as I saw that in the images you have posted. Also, I have renamed your images to how they're named in your post, so it'll be up to you to change the relevant names according to your code that you have written. I have also made a copy of the board in the code so that we don't want to modify the original board called boardCopy. This copy will contain the placed chess pieces in the board. Without further ado, here's the code:
board = imread('boardHigh.png'); %// High-res board
P_1 = im2bw(imread('P_1.png')); %// Load in the pawn pieces
P_2 = im2bw(imread('P_2.png'));
P1_neg = im2bw(imread('P1_neg.png'));
P2_neg = im2bw(imread('P2_neg.png'));
boardCopy = board; %// Make a copy of the board
for row = [2 7]
for col = 1 : 8
if row == 2
if mod(col,2) ~= 0
boardCopy((row-1)*301 + 1 : row*301, (col-1)*301 + 1 : col*301) = P_2;
else
boardCopy((row-1)*301 + 1 : row*301, (col-1)*301 + 1 : col*301) = P_1;
end
else
if mod(col,2) ~= 0
boardCopy((row-1)*301 + 1 : row*301, (col-1)*301 + 1 : col*301) = P1_neg;
else
boardCopy((row-1)*301 + 1 : row*301, (col-1)*301 + 1 : col*301) = P2_neg;
end
end
end
end
figure;
imshow(boardCopy);
This is the board I get with the chess pieces:
Hope this helps and good luck!
I have a 3 X 1000 (and later 3 X 10 000) matrix cord given, which contains the three dimensional coordinates for my pixels.
My intention is to calculate the distance between all the pixels, and I do it with a for loop (see below), but I will have to calculate this for huge matrices soon, and am wondering if I could vectorize the code for making it faster...?
dist = zeros(size(cord,2),size(cord,2));
for i = 1:size(cord,2)
for j = 1:size(cord,2)
dist(i,j) = norm(cord(:,i)-cord(:,j));
dist(j,i) = dist(i,j);
end
end
pdist does exactly that. squareform is needed to get the result in the form of a square, symmetric matrix:
dist = squareform(pdist(cord.'));
Approach 1 (Vectorized apprach with bsxfun ) -
squeeze(sqrt(sum(bsxfun(#minus,cord,permute(cord,[1 3 2])).^2)))
Not sure if this will be faster though.
Approach 2 -
Inspired by this very smart approach and all credits to the poster. The code posted here is just slightly customized for your case and hopefully slightly better in terms of runtime. Here it is -
A = cord'; %//'
numA = size(cord,2);
helpA = ones(numA,9);
helpB = ones(numA,9);
for idx = 1:3
sqA_idx = A(:,idx).^2;
helpA(:,3*idx-1:3*idx) = [-2*A(:,idx), sqA_idx ];
helpB(:,3*idx-2:3*idx-1) = [sqA_idx , A(:,idx)];
end
dist1 = sqrt(helpA * helpB'); %// desired output
From your code, you have recognized that the dist matrix is symmetric
dist(i,j) = norm(cord(:,i)-cord(:,j));
dist(j,i) = dist(i,j);
You could change the inner loop to account for this and reduce by roughly one half the number of calculations needed
for j = i:size(cord,2)
Further, we can avoid the dist(j,i) = dist(i,j); at each iteration and just do that at the end by extracting the upper triangle part of dist and adding its transpose to the dist matrix to account for the symmetry
dist = zeros(size(cord,2),size(cord,2));
for i = 1:size(cord,2)
for j = i:size(cord,2)
dist(i,j) = norm(cord(:,i)-cord(:,j));
end
end
dist = dist + triu(dist)';
The above addition is fine since the main diagonal is all zeros.
It still performs poorly though and so we should take advantage of vectorization. We can do that as follows against the inner loop
dist = zeros(size(cord,2),size(cord,2));
for i = 1:size(cord,2)
dist(i,i+1:end) = sum((repmat(cord(:,i),1,size(cord,2)-i)-cord(:,i+1:end)).^2);
end
dist = dist + triu(dist)';
dist = sqrt(dist);
For every element in cord we need to calculate its distance with all other elements that follow it. We reproduce the element with repmat so that we can subtract it from every element that follows without the need for the loop. The differences are squared and summed and assigned to the dist matrix. We take care of the symmetry and then take the square root of the matrix to complete the norm operation.
With tic and toc, the original distance calculation with a random cord (cord = rand(3,num);) took ~93 seconds. This version took ~2.8.
I would like to know why the sampler is incredibly slow when sampling step by step.
For example, if I run:
mcmc = MCMC(model)
mcmc.sample(1000)
the sampling is fast. However, if I run:
mcmc = MCMC(model)
for i in arange(1000):
mcmc.sample(1)
the sampling is slower (and the more it samples, the slower it is).
If you are wondering why I am asking this.. well, I need a step by step sampling because I want to perform some operations on the values of the variables after each step of the sampler.
Is there a way to speed it up?
Thank you in advance!
------------------ EDIT -------------------------------------------------------------
Here I present the specific problem in more details:
I have two models in competition and they are part of a bigger model that has a categorical variable functioning as a 'switch' between the two.
In this toy example, I have the observed vector 'Y', that could be explained by a Poisson or a Geometric distribution. The Categorical variable 'switch_model' selects the Geometric model when = 0 and the Poisson model when =1.
After each sample, if switch_model selects the Geometric model, I want the variables of the Poisson model NOT to be updated, because they are not influencing the likelihood and therefore they are just drifting away. The opposite is true if the switch_model selects the Poisson model.
Basically what I do at each step is to 'change' the value of the non-selected model by bringing it manually one step back.
I hope that my explanation and the commented code will be clear enough. Let me know if you need further details.
import numpy as np
import pymc as pm
import pandas as pd
import matplotlib.pyplot as plt
# OBSERVED VALUES
Y = np.array([0, 1, 2, 3, 8])
# PRIOR ON THE MODELS
pi = (0.5, 0.5)
switch_model = pm.Categorical("switch_model", p = pi)
# switch_model = 0 for Geometric, switch_model = 1 for Poisson
p = pm.Uniform('p', lower = 0, upper = 1) # Prior of the parameter of the geometric distribution
mu = pm.Uniform('mu', lower = 0, upper = 10) # Prior of the parameter of the Poisson distribution
# LIKELIHOOD
#pm.observed
def Ylike(value = Y, mu = mu, p = p, M = switch_model):
if M == 0:
out = pm.geometric_like(value+1, p)
elif M == 1:
out = pm.poisson_like(value, mu)
return out
model = pm.Model([Ylike, p, mu, switch_model])
mcmc = pm.MCMC(model)
n_samples = 5000
traces = {}
for var in mcmc.stochastics:
traces[str(var)] = np.zeros(n_samples)
bar = pm.progressbar.progress_bar(n_samples)
bar.update(0)
mcmc.sample(1, progress_bar=False)
for var in mcmc.stochastics:
traces[str(var)][0] = mcmc.trace(var)[-1]
for i in np.arange(1,n_samples):
mcmc.sample(1, progress_bar=False)
bar.update(i)
for var in mcmc.stochastics:
traces[str(var)][i] = mcmc.trace(var)[-1]
if mcmc.trace('switch_model')[-1] == 0: # Gemetric wins
traces['mu'][i] = traces['mu'][i-1] # One step back for the sampler of the Poisson parameter
mu.value = traces['mu'][i-1]
elif mcmc.trace('switch_model')[-1] == 1: # Poisson wins
traces['p'][i] = traces['p'][i-1] # One step back for the sampler of the Geometric parameter
p.value = traces['p'][i-1]
print '\n\n'
traces=pd.DataFrame(traces)
traces['mu'][traces['switch_model'] == 0] = np.nan
traces['p'][traces['switch_model'] == 1] = np.nan
print traces.describe()
traces.plot()
plt.show()
The reason this is so slow is that Python's for loops are pretty slow, especially when they are compared to FORTRAN loops (Which is what PyMC is written in basically.) If you could show more detailed code, it might be easier to see what you are trying to do and to provide faster alternative algorithms.
Actually I found a 'crazy' solution, and I have the suspect to know why it works. I would still like to get an expert opinion on my trick.
Basically if I modify the for loop in the following way, adding a 'reset of the mcmc' every 1000 loops, the sampling fires up again:
for i in np.arange(1,n_samples):
mcmc.sample(1, progress_bar=False)
bar.update(i)
for var in mcmc.stochastics:
traces[str(var)][i] = mcmc.trace(var)[-1]
if mcmc.trace('switch_model')[-1] == 0: # Gemetric wins
traces['mu'][i] = traces['mu'][i-1] # One step back for the sampler of the Poisson parameter
mu.value = traces['mu'][i-1]
elif mcmc.trace('switch_model')[-1] == 1: # Poisson wins
traces['p'][i] = traces['p'][i-1] # One step back for the sampler of the Geometric parameter
p.value = traces['p'][i-1]
if i%1000 == 0:
mcmc = pm.MCMC(model)
In practice this trick erases the traces and the database of the sampler every 1000 steps. It looks like the sampler does not like having a long database, although I do not really understand why. (of course 1000 steps is arbitrary, too short it adds too much overhead, too long it will cause the traces and database to be too long).
I find this hack a bit crazy and definitely not elegant.. does any of the experts or developers have a comment on it? Thank you!