when working with images, usually they include 3 layers, (RGB). In order to do some computation, I need to convert each layer of the image into a vector.
I1 = ones(70,50,3); % the first image
I2 = 0.4 * ones(70,50,3); % the second image
for dd = 1:3
ILayer1 = I1(:,:,dd);
ILayerLinear1 = ILayer1(:);
ILayer2 = I2(:,:,dd);
ILayerLinear2 = ILayer2(:);
comp = ILayerLinear1 * ILayerLinear1.';
end
Here I have replaced the main computation part with a very simple computation, but that is not the point.
Is there a better way to not repeat the matrix-to-vector conversion, or do it more efficiently? Because it may happen multiple times through the code.
Update:
I can also define a function as follows to pass an Image and retrieve a vector, but it still is not improving the code.
function V = I2V(I)
[r,c,d] = size(I);
V = zeros(d,r*c);
for dd = 1:d
layer = I(:,:,dd);
V(dd,:) = layer(:);
end
end
I'm not sure about the outer product but, here's everything else.
I1 = reshape(1:70*50*3, 70,50,3);
I2 = 0.4*reshape(1:70*50*3, 70,50,3);
i1 = reshape(I1, [], 3);
i2 = reshape(I2, [], 3);
Related
I got an assignment in a video processing course - to implement the Lucas-Kanade algorithm. Since we have to do it in the pyramidal model, I first build a pyramid for each of the 2 input images, and then for each level I perform a number of LK iterations. in each step (iteration), the following code runs (note: the images are zero-padded so I can handle the image edges easily):
function [du,dv]= LucasKanadeStep(I1,I2,WindowSize)
It = I2-I1;
[Ix, Iy] = imgradientxy(I2);
Ixx = imfilter(Ix.*Ix, ones(5));
Iyy = imfilter(Iy.*Iy, ones(5));
Ixy = imfilter(Ix.*Iy, ones(5));
Ixt = imfilter(Ix.*It, ones(5));
Iyt = imfilter(Iy.*It, ones(5));
half_win = floor(WindowSize/2);
du = zeros(size(It));
dv = zeros(size(It));
A = zeros(2);
b = zeros(2,1);
%iterate only on the relevant parts of the images
for i = 1+half_win : size(It,1)-half_win
for j = 1+half_win : size(It,2)-half_win
A(1,1) = Ixx(i,j);
A(2,2) = Iyy(i,j);
A(1,2) = Ixy(i,j);
A(2,1) = Ixy(i,j);
b(1,1) = -Ixt(i,j);
b(2,1) = -Iyt(i,j);
U = pinv(A)*b;
du(i,j) = U(1);
dv(i,j) = U(2);
end
end
end
mathematically what I'm doing is calculating for every pixel (i,j) the following optical flow:
as you can see, in the code I am calculating this for each pixel, which takes quite a long time (the whole processing for 2 images - including building 3 levels pyramids and 3 LK steps like the one above on each level - takes about 25 seconds (!) on a remote connection to my university servers).
My question: Is there a way to calculate this single LK step without the nested for loops? it must be more efficient because the next step of the assignment is to stabilize a short video using this algorithm.. thanks.
I ran your code on my system and did profiling. Here is what I got.
As you can see inverting the matrix(pinv) is taking most of the time. You can try and vectorise your code I guess, but I am not sure how to do it. But I do know a trick to improve the compute time. You have to exploit the minimum variance of the matrix A. That is, compute the inverse only if the minimum variance of A is greater than some threshold. This will improve the speed as you won't be inverting the matrix for all the pixel.
You do this by modifying your code to the one shown below.
function [du,dv]= LucasKanadeStep(I1,I2,WindowSize)
It = double(I2-I1);
[Ix, Iy] = imgradientxy(I2);
Ixx = imfilter(Ix.*Ix, ones(5));
Iyy = imfilter(Iy.*Iy, ones(5));
Ixy = imfilter(Ix.*Iy, ones(5));
Ixt = imfilter(Ix.*It, ones(5));
Iyt = imfilter(Iy.*It, ones(5));
half_win = floor(WindowSize/2);
du = zeros(size(It));
dv = zeros(size(It));
A = zeros(2);
B = zeros(2,1);
%iterate only on the relevant parts of the images
for i = 1+half_win : size(It,1)-half_win
for j = 1+half_win : size(It,2)-half_win
A(1,1) = Ixx(i,j);
A(2,2) = Iyy(i,j);
A(1,2) = Ixy(i,j);
A(2,1) = Ixy(i,j);
B(1,1) = -Ixt(i,j);
B(2,1) = -Iyt(i,j);
% +++++++++++++++++++++++++++++++++++++++++++++++++++
% Code I added , threshold better be outside the loop.
lambda = eig(A);
threshold = 0.2
if (min(lambda)> threshold)
U = A\B;
du(i,j) = U(1);
dv(i,j) = U(2);
end
% end of addendum
% +++++++++++++++++++++++++++++++++++++++++++++++++++
% U = pinv(A)*B;
% du(i,j) = U(1);
% dv(i,j) = U(2);
end
end
end
I have set the threshold to 0.2. You can experiment with it. By using eigen value trick I was able to get the compute time from 37 seconds to 10 seconds(shown below). Using eigen, pinv hardly takes up the time like before.
Hope this helped. Good luck :)
Eventually I was able to find a much more efficient solution to this problem.
It is based on the formula shown in the question. The last 3 lines are what makes the difference - we get a loop-free code that works way faster. There were negligible differences from the looped version (~10^-18 or less in terms of absolute difference between the result matrices, ignoring the padding zone).
Here is the code:
function [du,dv]= LucasKanadeStep(I1,I2,WindowSize)
half_win = floor(WindowSize/2);
% pad frames with mirror reflections of itself
I1 = padarray(I1, [half_win half_win], 'symmetric');
I2 = padarray(I2, [half_win half_win], 'symmetric');
% create derivatives (time and space)
It = I2-I1;
[Ix, Iy] = imgradientxy(I2, 'prewitt');
% calculate dP = (du, dv) according to the formula
Ixx = imfilter(Ix.*Ix, ones(WindowSize));
Iyy = imfilter(Iy.*Iy, ones(WindowSize));
Ixy = imfilter(Ix.*Iy, ones(WindowSize));
Ixt = imfilter(Ix.*It, ones(WindowSize));
Iyt = imfilter(Iy.*It, ones(WindowSize));
% calculate the whole du,dv matrices AT ONCE!
invdet = (Ixx.*Iyy - Ixy.*Ixy).^-1;
du = invdet.*(-Iyy.*Ixt + Ixy.*Iyt);
dv = invdet.*(Ixy.*Ixt - Ixx.*Iyt);
end
I have a cell array of size m x 1 and each cell is again s x t cell array (size varies). I would like to concatenate vertically. The code is as follows:
function(cell_out) = vert_cat(cell_in)
[row,col] = cellfun(#size,cell_in,'Uni',0);
fcn_vert = #(x)([x,repmat({''},size(x,1),max(cell2mat(col))-size(x,2))]);
cell_out = cellfun(fcn_vert,cell_in,'Uni',0); % Taking up lot of time
cell_out = vertcat(cell_out{:});
end
Step 3 takes a lot of time. Is it the right way to do or is there any another faster way to achieve this?
cellfun has been found to be slower than loops (kind of old, but agrees with what I have seen).
In addition, repmat has also been a performance hit in the past (though that may be different now).
Try this two-loop code that aims to accomplish your task:
function cellOut = vert_cat(c)
nElem = length(c);
colPad = zeros(nElem,1);
nRow = zeros(nElem,1);
for k = 1:nElem
[nRow(k),colPad(k)] = size(c{k});
end
colMax = max(colPad);
colPad = colMax - colPad;
cellOut = cell(sum(nRow),colMax);
bottom = cumsum(nRow) - nRow + 1;
top = bottom + nRow - 1;
for k = 1:nElem
cellOut(bottom(k):top(k),:) = [c{k},cell(nRow(k),colPad(k))];
end
end
My test for this code was
A = rand(20,20);
A = mat2cell(A,ones(20,1),ones(20,1));
C = arrayfun(#(c) A(1:c,1:c),randi([1,15],1,5),'UniformOutput',false);
ccat = vert_cat(c);
I used this pice of code to generate data:
%generating some dummy data
m=1000;
s=100;
t=100;
cell_in=cell(m,1);
for idx=1:m
cell_in{idx}=cell(randi(s),randi(t));
end
Applying some minor modifications, I was able to speed up the code by a factor of 5
%Minor modifications of the original code
%use arrays instead of cells for row and col
[row,col] = cellfun(#size,cell_in);
%claculate max(col) once
tcol=max(col);
%use cell instead of repmat to generate an empty cell
fcn_vert = #(x)([x,cell(size(x,1),tcol-size(x,2))]);
cell_out = cellfun(fcn_vert,cell_in,'Uni',0); % Taking up lot of time
cell_out = vertcat(cell_out{:});
Using simply a for loop is even faster, because the data is only moved once
%new approac. Basic idea: move every data only once
[row,col] = cellfun(#size,cell_in);
trow=sum(row);
tcol=max(col);
r=1;
cell_out2 = cell(trow,tcol);
for idx=1:numel(cell_in)
cell_out2(r:r+row(idx)-1,1:col(idx))=cell_in{idx};
r=r+row(idx);
end
Currently i am recognzing a face, means i have to find a face which we have to test is in training database or not..! So, i have to decide yes or no..
Yes means find image, and no means print message that NO IMAGE IN DATABASE. I have a program, Currently this program is finding a correct image correctly, but even when there is no image, even it shows other image which not matches.. Actually it should print NO IMAGE IN DATABASE.
So, How to do..?
Here is a Test and training images data on this link.
http://www.fileconvoy.com/dfl.php?id=g6e59fe8105a6e6389994740914b7b2fc99eb3e445
My Program is in terms of different four .m files, and it is here,we have to run only first code.. and remaining 3 are functions, it is also given here..**
clear all
clc
close all
TrainDatabasePath = uigetdir('D:\Program Files\MATLAB\R2006a\work', 'Select training database path' );
TestDatabasePath = uigetdir('D:\Program Files\MATLAB\R2006a\work', 'Select test database path');
prompt = {'Enter test image name (a number between 1 to 10):'};
dlg_title = 'Input of PCA-Based Face Recognition System';
num_lines= 1;
def = {'1'};
TestImage = inputdlg(prompt,dlg_title,num_lines,def);
TestImage = strcat(TestDatabasePath,'\',char(TestImage),'.jpg');
im = imread(TestImage);
T = CreateDatabase(TrainDatabasePath);
[m, A, Eigenfaces] = EigenfaceCore(T);
OutputName = Recognition(TestImage, m, A, Eigenfaces);
SelectedImage = strcat(TrainDatabasePath,'\',OutputName);
SelectedImage = imread(SelectedImage);
imshow(im)
title('Test Image');
figure,imshow(SelectedImage);
title('Equivalent Image');
str = strcat('Matched image is : ',OutputName);
disp(str)
function T = CreateDatabase(TrainDatabasePath)
TrainFiles = dir(TrainDatabasePath);
Train_Number = 0;
for i = 1:size(TrainFiles,1)
if
not(strcmp(TrainFiles(i).name,'.')|strcmp(TrainFiles(i).name,'..')|strcmp(TrainFiles(i).name,'Thu mbs.db'))
Train_Number = Train_Number + 1; % Number of all images in the training database
end
end
T = [];
for i = 1 : Train_Number
str = int2str(i);
str = strcat('\',str,'.jpg');
str = strcat(TrainDatabasePath,str);
img = imread(str);
img = rgb2gray(img);
[irow icol] = size(img);
temp = reshape(img',irow*icol,1); % Reshaping 2D images into 1D image vectors
T = [T temp]; % 'T' grows after each turn
end
function [m, A, Eigenfaces] = EigenfaceCore(T)
m = mean(T,2); % Computing the average face image m = (1/P)*sum(Tj's) (j = 1 : P)
Train_Number = size(T,2);
A = [];
for i = 1 : Train_Number
temp = double(T(:,i)) - m;
Ai = Ti - m
A = [A temp]; % Merging all centered images
end
L = A'*A; % L is the surrogate of covariance matrix C=A*A'.
[V D] = eig(L); % Diagonal elements of D are the eigenvalues for both L=A'*A and C=A*A'.
L_eig_vec = [];
for i = 1 : size(V,2)
if( D(i,i)>1 )
L_eig_vec = [L_eig_vec V(:,i)];
end
end
Eigenfaces = A * L_eig_vec; % A: centered image vectors
function OutputName = Recognition(TestImage, m, A, Eigenfaces)
ProjectedImages = [];
Train_Number = size(Eigenfaces,2);
for i = 1 : Train_Number
temp = Eigenfaces'*A(:,i); % Projection of centered images into facespace
ProjectedImages = [ProjectedImages temp];
end
InputImage = imread(TestImage);
temp = InputImage(:,:,1);
[irow icol] = size(temp);
InImage = reshape(temp',irow*icol,1);
Difference = double(InImage)-m; % Centered test image
ProjectedTestImage = Eigenfaces'*Difference; % Test image feature vector
Euc_dist = [];
for i = 1 : Train_Number
q = ProjectedImages(:,i);
temp = ( norm( ProjectedTestImage - q ) )^2;
Euc_dist = [Euc_dist temp];
end
[Euc_dist_min , Recognized_index] = min(Euc_dist);
OutputName = strcat(int2str(Recognized_index),'.jpg');
So, how to generate error massege when no image matches..?
At the moment, your application appears to find the most similar image (you appear to be using Euclidean distance as you measure of similarity), and return it. There doesn't seem to be any concept of whether the image "matches" or not.
Define a threshold on similarity, and then determine whether your most similar image meets that threshold. If it does, return it, otherwise display an error message.
:) I'm trying to code a Least Squares algorithm and I've come up with this:
function [y] = ex1_Least_Squares(xValues,yValues,x) % a + b*x + c*x^2 = y
points = size(xValues,1);
A = ones(points,3);
b = zeros(points,1);
for i=1:points
A(i,1) = 1;
A(i,2) = xValues(i);
A(i,3) = xValues(i)^2;
b(i) = yValues(i);
end
constants = (A'*A)\(A'*b);
y = constants(1) + constants(2)*x + constants(3)*x^2;
When I use this matlab script for linear functions, it works fine I think. However, when I'm passing 12 points of the sin(x) function I get really bad results.
These are the points I pass to the function:
xValues = [ -180; -144; -108; -72; -36; 0; 36; 72; 108; 144; 160; 180];
yValues = [sind(-180); sind(-144); sind(-108); sind(-72); sind(-36); sind(0); sind(36); sind(72); sind(108); sind(144); sind(160); sind(180) ];
And the result is sin(165°) = 0.559935259380508, when it should be sin(165°) = 0.258819
There is no reason why fitting a parabola to a full period of a sinusoid should give good results. These two curves are unrelated.
MATLAB already contains a least square polynomial fitting function, polyfit and a complementary function, polyval. Although you are probably supposed to write your own, trying out something like the following will be educational:
xValues = [ -180; -144; -108; -72; -36; 0; 36; 72; 108; 144; 160; 180];
% you may want to experiment with different ranges of xValues
yValues = sind(xValues);
% try this with different values of n, say 2, 3, and 4
p = polyfit(xValues,yValues,n);
x = -180:36:180;
y = polyval(p,x);
plot(xValues,yValues);
hold on
plot(x,y,'r');
Also, more generically, you should avoid using loops as you have in your code. This should be equivalent:
points = size(xValues,1);
A = ones(points,3);
A(:,2) = xValues;
A(:,3) = xValues.^2; % .^ and ^ are different
The part of the loop involving b is equivalent to doing b = yValues; either name the incoming variable b or just use the variable yValues, there's no need to make a copy of it.
I have tried to make a Gaussian filter in Matlab without using imfilter() and fspecial().
I have tried this but result is not like the one I have with imfilter and fspecial.
Here is my codes.
function Gaussian_filtered = Gauss(image_x, sigma)
% for single axis
% http://en.wikipedia.org/wiki/Gaussian_filter
Gaussian_filtered = exp(-image_x^2/(2*sigma^2)) / (sigma*sqrt(2*pi));
end
for 2D Gaussian,
function h = Gaussian2D(hsize, sigma)
n1 = hsize;
n2 = hsize;
for i = 1 : n2
for j = 1 : n1
% size is 10;
% -5<center<5 area is covered.
c = [j-(n1+1)/2 i-(n2+1)/2]';
% A product of both axes is 2D Gaussian filtering
h(i,j) = Gauss(c(1), sigma)*Gauss(c(2), sigma);
end
end
end
and the final one is
function Filtered = GaussianFilter(ImageData, hsize, sigma)
%Get the result of Gaussian
filter_ = Gaussian2D(hsize, sigma);
%check image
[r, c] = size(ImageData);
Filtered = zeros(r, c);
for i=1:r
for j=1:c
for k=1:hsize
for m=1:hsize
Filtered = Filtered + ImageData(i,j).*filter_(k,m);
end
end
end
end
end
But the processed image is almost same as the input image. I wonder the last function GaussianFiltered() is problematic...
Thanks.
here's an alternative:
Create the 2D-Gaussian:
function f=gaussian2d(N,sigma)
% N is grid size, sigma speaks for itself
[x y]=meshgrid(round(-N/2):round(N/2), round(-N/2):round(N/2));
f=exp(-x.^2/(2*sigma^2)-y.^2/(2*sigma^2));
f=f./sum(f(:));
Filtered image, given your image is called Im:
filtered_signal=conv2(Im,gaussian2d(N,sig),'same');
Here's some plots:
imagesc(gaussian2d(7,2.5))
Im=rand(100);subplot(1,2,1);imagesc(Im)
subplot(1,2,2);imagesc(conv2(Im,gaussian2d(7,2.5),'same'));
This example code is slow because of the for-loops. In matlab you can better use conv2, as suggested by user:bla, or just use filter2.
I = imread('peppers.png'); %load example data
I = I(:,:,1);
N=5; %must be odd
sigma=1;
figure(1);imagesc(I);colormap gray
x=1:N;
X=exp(-(x-((N+1)/2)).^2/(2*sigma^2));
h=X'*X;
h=h./sum(h(:));
%I=filter2(h,I); %this is faster
[is,js]=size(I);
Ib = NaN(is+N-1,js+N-1); %add borders
b=(N-1)/2 +1;
Ib(b:b+is-1,b:b+js-1)=I;
I=zeros(size(I));
for i = 1:is
for j = 1:js
I(i,j)=sum(sum(Ib(i:i+N-1,j:j+N-1).*h,'omitnan'));
end
end
figure(2);imagesc(I);colormap gray