I need to know how to align an image in Matlab for further work.
for example I have the next license plate image and I want to recognize all
the digits.
my program works for straight images so, I need to align the image and then
preform the optical recognition system.
The method should be as much as universal that fits for all kinds of plates and in all kinds of angles.
EDIT: I tried to do this with Hough Transform but I didn't Succeed. anybody can help me do to this?
any help will be greatly appreciated.
The solution was first hinted at by #AruniRC in the comments, then implemented by #belisarius in Mathematica. The following is my interpretation in MATLAB.
The idea is basically the same: detect edges using Canny method, find prominent lines using Hough Transform, compute line angles, finally perform a Shearing Transform to align the image.
%# read and crop image
I = imread('http://i.stack.imgur.com/CJHaA.png');
I = I(:,1:end-3,:); %# remove small white band on the side
%# egde detection
BW = edge(rgb2gray(I), 'canny');
%# hough transform
[H T R] = hough(BW);
P = houghpeaks(H, 4, 'threshold',ceil(0.75*max(H(:))));
lines = houghlines(BW, T, R, P);
%# shearing transforma
slopes = vertcat(lines.point2) - vertcat(lines.point1);
slopes = slopes(:,2) ./ slopes(:,1);
TFORM = maketform('affine', [1 -slopes(1) 0 ; 0 1 0 ; 0 0 1]);
II = imtransform(I, TFORM);
Now lets see the results
%# show edges
figure, imshow(BW)
%# show accumlation matrix and peaks
figure, imshow(imadjust(mat2gray(H)), [], 'XData',T, 'YData',R, 'InitialMagnification','fit')
xlabel('\theta (degrees)'), ylabel('\rho'), colormap(hot), colorbar
hold on, plot(T(P(:,2)), R(P(:,1)), 'gs', 'LineWidth',2), hold off
axis on, axis normal
%# show image with lines overlayed, and the aligned/rotated image
figure
subplot(121), imshow(I), hold on
for k = 1:length(lines)
xy = [lines(k).point1; lines(k).point2];
plot(xy(:,1), xy(:,2), 'g.-', 'LineWidth',2);
end, hold off
subplot(122), imshow(II)
In Mathematica, using Edge Detection and Hough Transform:
If you are using some kind of machine learning toolbox for text recognition, try to learn from ALL plates - not only aligned ones. Recognition results should be equally well if you transform the plate or dont, since by transforming, no new informations according to the true number will enhance the image.
If all the images have a dark background like that one, you could binarize the image, fit lines to the top or bottom of the bright area and calculate an affine projection matrix from the line gradient.
Related
I have an image rotated with imrotate as follow:
Im_requete=imread('lena.jpg');
Im_requete_G=rgb2gray(Im_requete);
Im_requete_G_scale_rot = imresize(imrotate(Im_requete_G,-20), 1.2);
I'm trying to get the coordinates (x, y) of the four corners of the rotated image as illustrated in the image below (red circle represents the desired corner):
This is my code:
stat = regionprops(Im_requete_G_scale_rot,'Extrema'); %extrema detection of the image.
point = stat.Extrema;
hold on
figure,imshow(Im_requete_G_scale_rot)
hold on
for i = 2:2:length(point)
x = point(i,1);
y = point(i,2);
plot(x,y,'o');
text(x,y,num2str(i),'color','r')
end
But the resulting coordinates are somewhere along the edges and not where I wanted them to be, as illustrated in the second image:
Can someone please tell me what's wrong with this code?
I don't have a good explanation for this, but I suppose regionprops gets confused by the grayscale tones in the image. If we turn the rotated Lena into a logical array, your algorithm works properly:
Im_requete_G_scale_rot = logical(imresize(imrotate(Im_requete_G,-20), 1.2)); % 3rd line
I have a hard problem that need your help. I have a binary image that maintains some unwanted region (small white dot) and hole regions (in figure 1).My idea is that the first I will remove unwanted region by calculating area these region and then filter with small area value.At the second step, I fill in hole region to make clear image.What do you think best method to fill in hole region. Do you have any idea to resolve it. Could you help me implement it by matlab. Thank you so much. This is my reference code for remove unwanted region. But it need threshold term. You can download image test at here
function exImage=rmUnwantedRegion(Img,threshold)
lb = bwlabel(Img);
st = regionprops(lb, 'Area', 'PixelIdxList' );
toRemove = [st.Area] <threshold; % fix your threshold here
exImage = Img;
exImage( vertcat(st(toRemove).PixelIdxList ) ) = 0; % remove
end
Here is an example implementation based on my comment:
subplot(1,3,1), imshow(input);
title('Original Image');
Calculating the opening of the image:
openInput=bwareaopen(input, 20);
subplot(1,3,2), imshow(bwareaopen(input, 20));
title('Opened Image');
And the subsequent closing:
ClosedInput = imclose(openInput,ones(10));
subplot(1,3,3), imshow(ClosedInput);
title('Closed Image');
Result:
Assuming white pixel is 1
Black is 0
Step 1:
Use convultion matrix (http://en.wikipedia.org/wiki/Kernel_%28image_processing%29)
with blur filter
Step 2:
Treshold each pixel with some static value (for example 0.5)
if pixel is >0.5 pixel = 1
else pixel = 0
This looks like a job for binary dilation and erosion. Generally an erosion is done first to remove unwanted noise and then dilation is performed with the same structuring element to fill in the gaps left by the erosion. Matlab uses strel to create structuring elements for morphological operations. You can also read about morphological operators here
Example:
SE=strel('square',5);
im_eroded=imerode(im,SE);
im_dilated=imdilate(im_eroded,SE);
You need to do an erosion (Wikipedia or Matlab) followed by a dilation (Wikipedia or Matlab). This is done using the imerode and the imdilate functions in Matlab.
Doing this require to specify the size of the element eroding and dilating using the strel function with a shape ('square', 'disk', 'octagon', etc.) and a size.
SE=strel('disk',5);
im_eroded=imerode(im,SE);
im_dilated=imdilate(im_eroded,SE);
What is the preferred way of converting from axis coordinates (e.g. those taken in by plot or those output in point1 and point2 of houghlines) to pixel coordinates in an image?
I see the function axes2pix in the Mathworks documentation, but it is unclear how it works. Specifically, what is the third argument? The examples just pass in 30, but it is unclear where this value comes from. The explanations depend on a knowledge of several other functions, which I don't know.
The related question: Axis coordinates to pixel coordinates? suggests using poly2mask, which would work for a polygon, but how do I do the same thing for a single point, or a list of points?
That question also links to Scripts to Convert Image to and from Graph Coordinates, but that code threw an exception:
Error using /
Matrix dimensions must agree.
Consider the following code. It shows how to convert from axes coordinates to image pixel coordinates.
This is especially useful if you plot the image using custom XData/YData locations other than the default 1:width and 1:height. I am shifting by 100 and 200 pixels in the x/y directions in the example below.
function imageExample()
%# RGB image
img = imread('peppers.png');
sz = size(img);
%# show image
hFig = figure();
hAx = axes();
image([1 sz(2)]+100, [1 sz(1)]+200, img) %# shifted XData/YData
%# hook-up mouse button-down event
set(hFig, 'WindowButtonDownFcn',#mouseDown)
function mouseDown(o,e)
%# get current point
p = get(hAx,'CurrentPoint');
p = p(1,1:2);
%# convert axes coordinates to image pixel coordinates
%# I am also rounding to integers
x = round( axes2pix(sz(2), [1 sz(2)], p(1)) );
y = round( axes2pix(sz(1), [1 sz(1)], p(2)) );
%# show (x,y) pixel in title
title( sprintf('image pixel = (%d,%d)',x,y) )
end
end
(note how the axis limits do not start at (1,1), thus the need for axes2pix)
There may be a built-in way that I haven't heard of, but this shouldn't be hard to do from scratch...
set(axes_handle,'units','pixels');
pos = get(axes_handle,'position');
xlim = get(axes_handle,'xlim');
ylim = get(axes_handle,'ylim');
Using these values, you can convert from axes coordinates to pixels easily.
x_in_pixels = pos(1) + pos(3) * (x_in_axes-xlim(1))/(xlim(2)-xlim(1));
%# etc...
The above uses pos(1) as the x-offset of the axes within the figure. If you don't care about this, don't use it. Likewise, if you want it in screen coordinates, add the x-offset of the position obtained by get(figure_handle,'position')
I've been using a function file [ret]=drawellipse(x,y,a,b,angle,steps,color,img). Calling the function through a script file to draw random ellipses in image. But once i set the random center point(x,y), and random a, b, there is high possibility that the ellipses intersection would occur. How can i prevent the intersection? (I'm supposed to draw the ellipses that are all separate from each other)
Well, over here i have a function file which is to check whether the ellipses got overlap or not,overlap = overlap_ellipses(x0,y0,a0,b0,angle0,x1,y1,a1,b1,angle1). If the two ellipses are overlap, then the 'overlap=1', otherwise 'overlap=0'.
Based on all these, i tested in the command window:
x=rand(4,1)*400; % x and y are the random coodinates for the center of ellipses
y=rand(4,1)*400;
a=[50 69 30 60]; % major axis for a and b, i intend to use random also in the future
b=[20 40 10 40]; % minor axis
angle=[30 90 45 0]; % angle of ellipse
steps=10000;
color=[255 0 0]; % inputs for another function file to draw the ellipse
img=zeros(500,500,3);
The following i want to dispaly the ellipses if overlap==0, and 'if overlap==1', decrease the a and b, till there is no intersection. Lastly, to imshow the img.
for i=1:length(x)
img=drawellipse(x(i),y(i),a(i),b(i),angle(i),steps,color,img);
end
For me now, i have difficulty in coding the middle part. How can i use the if statement to get the value of overlap and how to make the index corresponding to the ellipse i need to draw.
i tested a bit like
for k=1:(length(x)-1)
overlap = overlap_ellipses(x(1),y(1),a(1),b(1),angle(1),x(1+k),y(1+k),a(1+k),b(1+k),angle(1+k))
end
it returns
overlap=0
overlap=0
overlap=1
it is not [0 0 1]. I can't figure it out, thus stuck in the process.
The final image shoule look like the picture in this voronoi diagram of ellipses.
(There is no intersection between any two ellipses)
Assuming you are drawing the ellipses into a raster graphics image, you could calculate the pixels you would have to draw for an ellipse, check whether these pixels in the image are still of the background color, and draw the ellipse only if the answer is yes, otherwise reject it (because something else, i.e. another ellipse, is in the way) and try other x,y,a and b.
Alternatively, you could split your image into rectangles (not neccessarily of equal size) and place one ellipse in each of those, picking x,y,a,b such that no ellipse exceeds its rectangle - then the ellipses cannot overlap either, but it depends on how much "randomness" your ellipse placing should have whether this suffices.
The mathematically rigorous way would be to store x,y,a,b of each drawn ellipse and for each new ellipse, do pairwise checks with each of those whether they have common points by solving a system of two quadratic equations. However, this might be a bit complicated, especially once the angle is not 0.
Edit in response to the added code: Instead of fixing all x's and y's before the loop, you can determine them inside the loop. Since you know how many ellipses you want, but not how many you have to sample, you need a while loop. The test loop you give may come in handy, but you need to compare all previous ellipses to the one created in the loop iteration, not the first one.
i=1;
while (i<=4) %# or length(a), or, more elegantly, some pre-defined max
x(i) = rand*400; y(i) = rand*400; %# or take x and y as givren and decrease a and b
%# now, check overlap for given center
overlap = false;
for k=1:(i-1)
overlap = overlap || overlap_ellipses(x(i),y(i),a(i),b(i),angle(i),x(k),y(k),a(k),b(k),angle(k))
end
if (~overlap)
img = drawellipse(x(i),y(i),a(i),b(i),angle(i),steps,color,img);
i = i+1; %# determine next ellipse
end %# else x(i) and y(i) will be overwritten in next while loop iteration
end
Of course, if a and b are fixed, it may happen that no ellipse fits the image dimensions if the already present ones are unfortunately placed, resulting in an infinite loop.
Regarding your plan of leaving the center fixed and decreasing the ellipse's size until it fits: where does your overlap_ellipses method come from? Maybe itcan be adapted to return a factor by which one ellipse needs to be shrinked to fit next to the other (and 1 if it fits already)?
The solution proposed by #arne.b (the first one) is a good way to rasterize non-overlapping ellipses.
Let me illustrate that idea with an example. I will be extending my previous answer:
%# color image
I = imread('pears.png');
sz = size(I);
%# parameters of ellipses
num = 7;
h = zeros(1,num);
clr = lines(num); %# color of each ellipse
x = rand(num,1) .* sz(2); %# center x-coords
y = rand(num,1) .* sz(1); %# center y-coords
a = rand(num,1) .* 200; %# major axis length
b = rand(num,1) .* 200; %# minor axis length
angle = rand(num,1) .* 360; %# angle of rotation
%# label image, used to hold rasterized ellipses
BW = zeros(sz(1),sz(2));
%# randomly place ellipses one-at-a-time, skip if overlaps previous ones
figure, imshow(I)
axis on, hold on
for i=1:num
%# ellipse we would like to draw directly on image matrix
[ex,ey] = calculateEllipse(x(i),y(i), a(i),b(i), angle(i), 100);
%# lets plot the ellipse (overlayed)
h(i) = plot(ex,ey, 'LineWidth',2, 'Color',clr(i,:));
%# create mask for image pixels inside the ellipse polygon
mask = poly2mask(ex,ey,sz(1),sz(2));
%# get the perimter of this mask
mask = bwperim(mask,8);
%# skip if there is an existing overlapping ellipse
if any( BW(mask)~=0 ), continue, end
%# use the mask to place the ellipse in the label image
BW(mask) = i;
end
hold off
legend(h, cellstr(num2str((1:num)','Line%d')), 'Location','BestOutside') %'
%# set pixels corresponding to ellipses using specified colors
clr = im2uint8(clr);
II = I;
for i=1:num
BW_ind = bsxfun(#plus, find(BW==i), prod(sz(1:2)).*(0:2));
II(BW_ind) = repmat(clr(i,:), [size(BW_ind,1) 1]);
end
figure, imshow(II, 'InitialMagnification',100, 'Border','tight')
Note how the overlap test is performed in the order the ellipses are added, thus after Line1 (blue) and Line2 (green) are drawn, Line3 (red) will be skipped because it overlaps one of the previous ones, and so on for the rest...
One option is to keep track of all the ellipses already drawn, and to make sure the next set of [x,y,a,b] does not produce a new ellipse which intersects with the existing ones. You can either invoke random numbers until you come up with a set that fulfills the condition, or once you have a set which violates the condition, decrease the values of a and/or b until no intersection occurs.
So I have an image like this:
I want to get something like this (I haven't drawn all lines I want but I hope you can get my idea):
I want to use SURF ( (Speeded Up Robust Features) is a robust image descriptor, first presented by Herbert Bay et al. in 2006 ) or something that is based on sums of 2D Haar wavelet responses and makes an efficient use of integral images for finding all straight lines on image. I want to get relative to picture pixel coords start and end points of lines.
So on this picture to find all lines between tiles and those 2 black lines on top.
Is there any such Code Example (with lines search capability) to start from?
I love C and C++ but any other readable code will probably work for me=)
The following is a complete example of applying Hough Transform to detect lines. I am using MATLAB for the job..
The trick is to divide the image into regions and process each differently; this is because you have different "textures" in your scene (tiles on the upper region of the wall are quite different from the darker ones on the bottom, and processing the image all at once wont be optimal).
As a working example, consider this one:
%# load image, blur it, then find edges
I0 = rgb2gray( imread('http://www.de-viz.ru/catalog/new2/Holm/hvannaya.jpg') );
I = imcrop(I0, [577 156 220 292]); %# select a region of interest
I = imfilter(I, fspecial('gaussian', [7 7], 1), 'symmetric');
BW = edge(I, 'canny');
%# Hough Transform and show accumulated matrix
[H T R] = hough(BW, 'RhoResolution',2, 'Theta',-90:0.5:89.5);
imshow(imadjust(mat2gray(H)), [], 'XData',T, 'YData',R, ...
'InitialMagnification','fit')
xlabel('\theta (degrees)'), ylabel('\rho')
axis on, axis normal, colormap(hot), colorbar, hold on
%# detect peaks
P = houghpeaks(H, 20, 'threshold',ceil(0.5*max(H(:))));
plot(T(P(:,2)), R(P(:,1)), 'gs', 'LineWidth',2);
%# detect lines and overlay on top of image
lines = houghlines(BW, T, R, P, 'FillGap',50, 'MinLength',5);
figure, imshow(I), hold on
for k = 1:length(lines)
xy = [lines(k).point1; lines(k).point2];
plot(xy(:,1), xy(:,2), 'g.-', 'LineWidth',2);
end
hold off
You could try the same procedure for other regions while tuning the parameters to get good results..
Have you tried a simpler approach such as the Hough transform for finding lines? A function to perform this and example are included in OpenCV called cvHoughLines2.
Two-dimensional wavelet transforms are implemented in R using the package waveslim. Specifically, the function dwt2D() uses a C "backend" for speed. You can then apply thresholding to find the lines.