Construct by hand a Perceptron which correctly classifies the following data - algorithm

I am beginner in Machine learning. I am trying to solve perceptron problem by hand for following data
Training example [x1 x2] [0 1] [2 0] [1 0] Class [1 -1]
using a learning rate of 1.0 and initial weight values of w0 = -1.5 w1 = 0 w2 = 2
I have solved this for epochs but it is not converging. I am unable to figure out where I am doing wrong Please see attached picture I solved by hand
I tried to solve this way

Related

Maximizing the efficiency of a simple algorithm in MATLAB

So here is what I'm trying to do in MATLAB:
I have an array of n, 2D images. I need to go through pixel by pixel, and find which picture has the brightest pixel at each point, then store the index of that image in another array at that point.
As in, if I have three pictures (n=1,2,3) and picture 2 has the brightest pixel at [1,1], then the value of max_pixels[1,1] would be 2, the index of the picture with that brightest pixel.
I know how to do this with for loops,
%not my actual code:
max_pixels = zeroes(x_max, y_max)
for i:x_max
for j:y_max
[~ , max_pixels(i, j)] = max(pic_arr(i, j))
end
end
But my question is, can it be done faster with some of the special functionality in MATLAB? I have heard that MATLAB isn't too friendly when it comes to nested loops, and the functionality of : should be used wherever possible. Is there any way to get this more efficient?
-PK
You can use max(...) with a dimension specified to get the maximum along the 3rd dimension.
[max_picture, indexOfMax] = max(pic_arr,[],3)
You can get the matrix of maximum values in this way, using memory instead of high performance of processor:
a = [1 2 3];
b = [3 4 2];
c = [0 4 1];
[max_matrix, index_max] = arrayfun(#(x,y,z) max([x y z]), a,b,c);
a,b,c can be matrices also.
It returns the matrix with max values and the matrix of indexes (in which matrix is found each max value).

Calculating translation value and rotation angle of a rotated 2D image

I have two images which one of them is the Original image and the second one is Transformed image.
I have to find out how many degrees Transformed image was rotated using 3x3 transformation matrix. Plus, I need to find how far translated from origin.
Both images are grayscaled and held in matrix variables. Their sizes are same [350 500].
I have found a few lecture notes like this.
Lecture notes say that I should use the following matrix formula for rotation:
For translation matrix the formula is given:
Everything is good. But there are two problems:
I could not imagine how to implement the formulas using MATLAB.
The formulas are shaped to find x',y' values but I already have got x,x',y,y' values. I need to find rotation angle (theta) and tx and ty.
I want to know the equivailence of x, x', y, y' in the the matrix.
I have got the following code:
rotationMatrix = [ cos(theta) sin(theta) 0 ; ...
-sin(theta) cos(theta) 0 ; ...
0 0 1];
translationMatrix = [ 1 0 tx; ...
0 1 ty; ...
0 0 1];
But as you can see, tx, ty, theta variables are not defined before used. How can I calculate theta, tx and ty?
PS: It is forbidden to use Image Processing Toolbox functions.
This is essentially a homography recovery problem. What you are doing is given co-ordinates in one image and the corresponding co-ordinates in the other image, you are trying to recover the combined translation and rotation matrix that was used to warp the points from the one image to the other.
You can essentially combine the rotation and translation into a single matrix by multiplying the two matrices together. Multiplying is simply compositing the two operations together. You would this get:
H = [cos(theta) -sin(theta) tx]
[sin(theta) cos(theta) ty]
[ 0 0 1]
The idea behind this is to find the parameters by minimizing the error through least squares between each pair of points.
Basically, what you want to find is the following relationship:
xi_after = H*xi_before
H is the combined rotation and translation matrix required to map the co-ordinates from the one image to the other. H is also a 3 x 3 matrix, and knowing that the lower right entry (row 3, column 3) is 1, it makes things easier. Also, assuming that your points are in the augmented co-ordinate system, we essentially want to find this relationship for each pair of co-ordinates from the first image (x_i, y_i) to the other (x_i', y_i'):
[p_i*x_i'] [h11 h12 h13] [x_i]
[p_i*y_i'] = [h21 h22 h23] * [y_i]
[ p_i ] [h31 h32 1 ] [ 1 ]
The scale of p_i is to account for homography scaling and vanishing points. Let's perform a matrix-vector multiplication of this equation. We can ignore the 3rd element as it isn't useful to us (for now):
p_i*x_i' = h11*x_i + h12*y_i + h13
p_i*y_i' = h21*x_i + h22*y_i + h23
Now let's take a look at the 3rd element. We know that p_i = h31*x_i + h32*y_i + 1. As such, substituting p_i into each of the equations, and rearranging to solve for x_i' and y_i', we thus get:
x_i' = h11*x_i + h12*y_i + h13 - h31*x_i*x_i' - h32*y_i*x_i'
y_i' = h21*x_i + h22*y_i + h23 - h31*x_i*y_i' - h32*y_i*y_i'
What you have here now are two equations for each unique pair of points. What we can do now is build an over-determined system of equations. Take each pair and build two equations out of them. You will then put it into matrix form, i.e.:
Ah = b
A would be a matrix of coefficients that were built from each set of equations using the co-ordinates from the first image, b would be each pair of points for the second image and h would be the parameters you are solving for. Ultimately, you are finally solving this linear system of equations reformulated in matrix form:
You would solve for the vector h which can be performed through least squares. In MATLAB, you can do this via:
h = A \ b;
A sidenote for you: If the movement between images is truly just a rotation and translation, then h31 and h32 will both be zero after we solve for the parameters. However, I always like to be thorough and so I will solve for h31 and h32 anyway.
NB: This method will only work if you have at least 4 unique pairs of points. Because there are 8 parameters to solve for, and there are 2 equations per point, A must have at least a rank of 8 in order for the system to be consistent (if you want to throw in some linear algebra terminology in the loop). You will not be able to solve this problem if you have less than 4 points.
If you want some MATLAB code, let's assume that your points are stored in sourcePoints and targetPoints. sourcePoints are from the first image and targetPoints are for the second image. Obviously, there should be the same number of points between both images. It is assumed that both sourcePoints and targetPoints are stored as M x 2 matrices. The first columns contain your x co-ordinates while the second columns contain your y co-ordinates.
numPoints = size(sourcePoints, 1);
%// Cast data to double to be sure
sourcePoints = double(sourcePoints);
targetPoints = double(targetPoints);
%//Extract relevant data
xSource = sourcePoints(:,1);
ySource = sourcePoints(:,2);
xTarget = targetPoints(:,1);
yTarget = targetPoints(:,2);
%//Create helper vectors
vec0 = zeros(numPoints, 1);
vec1 = ones(numPoints, 1);
xSourcexTarget = -xSource.*xTarget;
ySourcexTarget = -ySource.*xTarget;
xSourceyTarget = -xSource.*yTarget;
ySourceyTarget = -ySource.*yTarget;
%//Build matrix
A = [xSource ySource vec1 vec0 vec0 vec0 xSourcexTarget ySourcexTarget; ...
vec0 vec0 vec0 xSource ySource vec1 xSourceyTarget ySourceyTarget];
%//Build RHS vector
b = [xTarget; yTarget];
%//Solve homography by least squares
h = A \ b;
%// Reshape to a 3 x 3 matrix (optional)
%// Must transpose as reshape is performed
%// in column major format
h(9) = 1; %// Add in that h33 is 1 before we reshape
hmatrix = reshape(h, 3, 3)';
Once you are finished, you have a combined rotation and translation matrix. If you want the x and y translations, simply pick off column 3, rows 1 and 2 in hmatrix. However, we can also work with the vector of h itself, and so h13 would be element 3, and h23 would be element number 6. If you want the angle of rotation, simply take the appropriate inverse trigonometric function to rows 1, 2 and columns 1, 2. For the h vector, this would be elements 1, 2, 4 and 5. There will be a bit of inconsistency depending on which elements you choose as this was solved by least squares. One way to get a good overall angle would perhaps be to find the angles of all 4 elements then do some sort of average. Either way, this is a good starting point.
References
I learned about homography a while ago through Leow Wee Kheng's Computer Vision course. What I have told you is based on his slides: http://www.comp.nus.edu.sg/~cs4243/lecture/camera.pdf. Take a look at slides 30-32 if you want to know where I pulled this material from. However, the MATLAB code I wrote myself :)

How to draw/plot with multiple matrices

Hi I am trying to draw an image.
I have three matrices:
Matrix A:
X coordinates
Matrix B:
Y coordinates
Matrix C:
Image gray scale
For example:
A = [1, 1; B = [1, 2; C = [1, 2;
2, 2] 1, 2] 3, 4]
I will plot a point with value of C(1) at X(1), Y(1).
Value 1 is drawn at (1,1)
Value 2 is drawn at (1,2)
Value 3 is drawn at (2,1)
Value 4 is drawn at (2,2)
Is there a function that I can use to plot this, or do I have to implement this? Any suggestion how to implement this would be appreciated it. Thank you.
Is it a full image? And A, B, and C are 1D, right? If so you could make a 2D array with the values of Matrix C at the corresponding indices, convert it to an image and display the images.
img = zeros(max(max(B)),max(max(A))); %initialize the new matrix
for i = 1:numel(C) %for each element in C
img(B(i),A(i)) = C(i); %fill the matrix one element at a time
end
img = mat2gray(img); %optional. More information in edit
imshow(img); %display the image
This assumes that the minimum index value is 1. If it is 0 instead, you'll have to add 1 to all of the indices.
My matlab is a little rusty but that should work.
edit: Is there any reason why they are two dimensional arrays to start? Regardless, I've updated my answer to work in either case.
edit2: mat2gray will scale your values between 0 and 1. If your values are already grayscale this is unnecessary. If your values range another scale but do not necessarily contain the min and max values, you can specify the min and max. For example if your range is 0 to 255, use mat2gray(img,[0,255]);

Error in MATLAB genetic algorithm (ga) from global optimization toolbox with seeding initial population

I am running into a puzzling error when when using the ga function in MATLAB's global optimization toolbox, the error message is pasted below to see if anyone can decipher what this means.
I am trying to seed the ga with an initial population, a matrix of pop-by-nvar. However, this produces the error below. I can only get the ga to run if I pass in an initial population vector of (pop-1)-by-nvar. This is a little inconvenient since I want to specify the entire population. I hope someone who is familiar with the global optimization toolbox and the ga function can shed some light here!
??? Index exceeds matrix dimensions.
Error in ==> gacreationlinearfeasible>lhsLambda at 213
[lambda(i,:),f,e] = fmincon(fun,lambda(i,:),[],[],Aeq,beq,lb,ub,[],opts);
Error in ==> gacreationlinearfeasible>feasibleLHS at 180
initialPopulation(boundary_points+1:interior_points+boundary_points,:) = ...
Error in ==> gacreationlinearfeasible at 42
feasiblePop = feasibleLHS(individualsToCreate,GenomeLength,options);
Error in ==> makeState at 30
state.Population = feval(options.CreationFcn,GenomeLength,FitnessFcn,options,options.CreationFcnArgs{:});
Error in ==> galincon at 18
state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);
Error in ==> ga at 282
[x,fval,exitFlag,output,population,scores] = galincon(FitnessFcn,nvars, ...
Here is an example of using GA with specifying an initial population:
%# find minima
initPop = bsxfun(#plus, [2 3], randn(20,2)*2); %# 20-by-2 matrix
opts = gaoptimset('InitialPopulation',initPop);
[x, fx, flag, out, pop] = ga(#rastriginsfcn, 2, [],[], [],[], [],[], [], opts);
%# plot solution
figure('Renderer','opengl')
ezsurfc(#(x,y) rastriginsfcn([x,y])), colormap hot
line(x(1), x(2), fx, 'Marker','.', 'MarkerSize',50, 'Color','b')
view(3)
Here is the solution I get after 72 generations:
Note that the Rastrigin function has the global minimum at (0,0)
I had the same problem when fully defining the initial population (population size is equal to the number of rows of the initial population matrix).
It was caused by the violation of the linear constraints by one of the individuals of the initial population.
So check if the manually assigned individuals do not violate the linear constraints.

Marking a specific point on a graph in MATLAB

I guess this is a very basic question.
I have a graph which I created in MATLAB. This is a graph of Power (y-axis) versus Frequency (x-axis).
The range of my x-axis is from 0 to 1000. Now here is my problem. I want to draw a line from specific points on the x-axis to the graph. For example, for points 40, 400, 950.
By using set(gca, 'XTick', [40 400 950]); I am able to mark these particular points. But I want to make it more visible by drawing straight vertical lines from these points.
Any help will be greatly appreciated. Thank you.
Use plot with endpoints with the same x value and different y values. (and don't forget to use myaa to beautify the output).
x = 0:0.1:2*pi;
y = sin(x);
plot(x,y);
hold on;
plot([0.6 0.6], [-1 1], 'Color', [0.7 0.7 0.7], 'LineWidth', 2);
plot([3.6 3.6], [-1 1], 'Color', [0.7 0.7 0.7], 'LineWidth', 2);
If you do this often I would recommend you a great submission from the FileExchange:
hline and vline
Just do:
vline([40 400 950])
Read the function documentation, if you want the line to have different properties than default.
I typically do this using something like this (is powers is a row vector).
powers = randn(1,1000)+40;
plot([1;1]*[40 400 950], [[0 0 0]; [powers([40 400 950])]],'k-')

Resources