I want to align image X against image Y. Each image contains circles and Hough transform is used to detect them.
Lets assume Hough transform detects different number of circles in image X and image Y.
I want to find transformation of image X (rotation, translation) such that most circles in image X match some circle in image Y. Can you tell me how would you proceed?
An approach:
Get some properties from the circles of images X and Y, such as radius or others (color?).
Find putative matches between the circles of X with those of Y. It is to say, two circles with a similar radius are a putative match. You may obtain several matches, correct and incorrect ones.
Use a RANSAC-like algorithm to find the transformation:
Select two random pairs of matching circles and compute the translation and rotation that these define.
Check how many putative matches are ok if you assume the previous transformation (translation + rotation) was ok.
Repeat.
Keep the transformation that maximizes the number of matching circles.
Related
I am using matlab's built in function called Procrustes to see the rotation translation and scale between two images. But, I am just using coordinates of the brightest points in the image and rotating these coordinates about the center of the image. Procrustes compares two matrices and gives you the rotation, translation, and scale. However, procrustes only works correctly if the matrices are in the same order for comparison.
I am given an image and a separate comparison coordinate matrix. The end goal is to find how much the image has been rotated, translated, and scaled compared to the coordinate matrix. I can just use Procrustes for this, but I need to correctly order the coordinates found from the image to match the order in the comparison coordinate matrix. My thought was to compare the distance between every possible combination of points in the coordinate matrix and compare it to the coordinates that I find in the picture. I just do not know how to write this code due to the fact if there is n coordinates, there will be n! possible combinations.
Just searching for the shortest distance is not so hard.
A = rand(1E4,2);
B = rand(1E4,2);
tic
idx = nan(1,1E4);
for ct = 1:size(A,1)
d = sum((A(ct,:)-B).^2,2);
idx(ct) = find(d==min(d));
end
toc
plot(A(1:10,1),A(1:10,2),'.r',B(idx(1:10),1),B(idx(1:10),2),'.b')
takes half a second on my PC.
The problems can start when two points in set A are matched to the same location in set B.
length(unique(idx))==length(idx)
This can be solved in several ways. The best (imho) is to determine a probability that point B matches with point A based on the distance (usually something that decreases exponentially), and solve for the most probable situation.
A simpler method (but more error prone) is to remove the matched point from set B.
I have 4 Point values: TopLeft, TopRight, BottomLeft, BottomRight. These define a 4 sided shape (like a distorted rectangle) on my monitor. These are the point a Tobii gaze device thinks I am looking at when in fact I am looking at the four corners of my monitor.
This picture shows a bitmap on the left representing my monitor, and the points the Tobii device tells me I am looking at when I am in fact looking at the corners of the screen. (It's a representation, not real).
I want to use those four calibration points to take a screen X,Y position that is from an inaccurate gaze position and correct it so that it is positioned as per the image on the right.
Edit: New solution for the edited question is at the end.
This problem is call bilinear interpolation.
Once you grasp the idea, it will be very easy and you would remember it for the rest of your life.
It would be quite long to post all detail here, but I will try.
First, I will name the point on the left to be (x,y) and the right to be (X,Y).
Let (x1,y1), (x1,y2), (x2,y1), (x2,y2) be the corner points on the left rectangle.
Secondly, let's split the problem into 2 bilinear interpolation problems:
want to find X
want to find Y
Let's find them one by one (X or Y).
Define : Qxx are the value of X or Y of the four corner in the right rectangle.
Suppose that we want to find the value of the unknown function f at
the point (x, y). It is assumed that we know the value of f at the
four points Q11 = (x1, y1), Q12 = (x1, y2), Q21 = (x2, y1), and Q22 =
(x2, y2).
The f(x,y) of your problem is X or Y in your question.
Then you interpolate f(x,y1) and f(x,y2) to be f(x,y) in the same way.
Finally, you will got X or Y=f(x,y)
Reference : All pictures/formulas/text here are copied from the wiki link (some with modification).
Edit: After the question has been edited, it become very different.
The new one is opposite, and it is called "inverse bilinear interpolation" which is far harder.
For more information, please read http://www.iquilezles.org/www/articles/ibilinear/ibilinear.htm
You can define a unique Linear Transform using 6 equations. The 3 points which have to align provide those 6 equations, as each pair of matching points provides two equations in x and y.
If you want to pursue this, I can provide the matrix equation which defines the Linear Transform based on how it maps three points. You invert this matrix and it will provide the linear transform.
But having done that, the transform is completely specified. You have no control over where the corner points of the original quadrilateral will go. In general, you can't even define a linear transform to map one quadrilateral onto another; this gives 8 equations (2 for each corner) with only 6 unknowns. Its over-specified. In fact a Linear Transform must always map a rectangle to a parallelogram, so in general you can't define a Linear Transform which maps one quadrilateral to another.
So if it can't be a Linear Transform, can it be a non-Linear Transform? Well, yes, but non-Linear Transforms don't necessarily map straight lines to straight lines, so the mapped edges of the quadrilateral won't be straight. Or any other lines. And you still have 14 equations (2 for each point and corner) for which you have to invent some non-Linear transform with 14 unknowns.
So the problem as stated cannot be solved with a Linear Transform; its over specified. Using a non-Linear transform will require you to devise a non-Linear transform which has 14 free variables (vs the 6 in a Linear Transform), this will map the 7 points correctly but straight lines will no longer be straight. Adding this requirement in adds an infinite number of constraints (one for every point in the line) and you won't even be able to use continuous functions.
There may be some solution to what you are doing in terms of what you are really trying to do (ie the underlying application need), but as a mathematical problem it is unsolvable.
Let me know if you want the matrix equation to produce a Linear Transform based on how it transforms 3 points.
I have many images like the following (only white and black):
My final problem is to find well matching ellipses. Unfortunately the real used images are not always that nice like this. They could be deformed a bit, which makes ellipse matching probably harder.
My idea is to find "break points". I markes them in the following picture:
Maybe these points could help to make a matching for the ellipses. The end result should be something like this:
Has someone an idea what algorithm may be used to find these break points? Or even better to make good ellipse matching?
Thank you very much
Sample the circumference points
Just scan your image and select All Black pixels with any White neighbor. You can do this by recoloring the remaining black pixels to any unused color (Blue).
After whole image is done you can recolor the inside back from unused color (Blue) to white.
form a list of ordered circumference points per cluster/ellipse
Just scan your image and find first black pixel. Then use A* to order the circumference points and store the path in some array or list pnt[] and handle it as circular array.
Find the "break points"
They can be detect by peak in the angle between neighbors of found points. something like
float a0=atan2(pnt[i].y-pnt[i-1].y,pnt[i].x-pnt[i-1].x);
float a1=atan2(pnt[i+1].y-pnt[i].y,pnt[i+1].x-pnt[i].x);
float da=fabs(a0-a1); if (da>M_PI) da=2.0*M_PI-da;
if (da>treshold) pnt[i] is break point;
or use the fact that on break point the slope angle delta change sign:
float a1=atan2(pnt[i-1].y-pnt[i-2].y,pnt[i-1].x-pnt[i-2].x);
float a1=atan2(pnt[i ].y-pnt[i-1].y,pnt[i ].x-pnt[i-1].x);
float a2=atan2(pnt[i+1].y-pnt[i ].y,pnt[i+1].x-pnt[i ].x);
float da0=a1-a0; if (da0>M_PI) da0=2.0*M_PI-da0; if (da0<-M_PI) da0=2.0*M_PI+da0;
float da1=a2-a1; if (da1>M_PI) da1=2.0*M_PI-da1; if (da1<-M_PI) da1=2.0*M_PI+da1;
if (da0*da1<0.0) pnt[i] is break point;
fit ellipses
so if no break points found you can fit the entire pnt[] as single ellipse. For example Find bounding box. It's center is center of ellipse and its size gives you semi-axises.
If break points found then first find the bounding box of whole pnt[] to obtain limits for semi-axises and center position area search. Then divide the pnt[] to parts between break points. Handle each part as separate part of ellipse and fit.
After all the pnt[] parts are fitted check if some ellipses are not the same for example if they are overlapped by another ellipse the they would be divided... So merge the identical ones (or average to enhance precision). Then recolor all pnt[i] points to white, clear the pnt[] list and loop #2 until no more black pixel is found.
how to fit ellipse from selection of points?
algebraically
use ellipse equation with "evenly" dispersed known points to form system of equations to compute ellipse parameters (x0,y0,rx,ry,angle).
geometrically
for example if you detect slope 0,90,180 or 270 degrees then you are at semi-axis intersection with circumference. So if you got two such points (one for each semi-axis) that is all you need for fitting (if it is axis-aligned ellipse).
for non-axis-aligned ellipses you need to have big enough portion of the circumference available. You can exploit the fact that center of bounding box is also the center of ellipse. So if you got the whole ellipse you know also the center. The semi-axises intersections with circumference can be detected with biggest and smallest tangent change. If you got center and two points its all you need. In case you got only partial center (only x, or y coordinate) you can combine with more axis points (find 3 or 4)... or approximate the missing info.
Also the half H,V lines axis is intersecting ellipse center so it can be used to detect it if not whole ellipse in the pnt[] list.
approximation search
You can loop through "all" possible combination of ellipse parameters within limits found in #4 and select the one that is closest to your points. That would be insanely slow of coarse so use binary search like approach something like mine approx class. Also see
Curve fitting with y points on repeated x positions (Galaxy Spiral arms)
on how it is used for similar fit to yours.
hybrid
You can combine geometrical and approximation approach. First compute what you can by geometrical approach. And then compute the rest with approximation search. you can also increase precision of the found values.
In rare case when two ellipses are merged without break point the fitted ellipse will not match your points. So if such case detected you have to subdivide the used points into groups until their fits matches ...
This is what I have in mind with this:
You probably need something like this:
https://en.wikipedia.org/wiki/Circle_Hough_Transform
Your edge points are simply black pixels with at least one white 4-neighbor.
Unfortunately, though, you say that your ellipses may be “tilted”. Generic ellipses are described by quadratic equations like
x² + Ay² + Bxy + Cx + Dy + E = 0
with B² < 4A (⇒ A > 0). This means that, compared to the circle problem, you don't have 3 dimensions but 5. This causes the Hough transform to be considerably harder. Luckily, your example suggests that you don't need a high resolution.
See also: algorithm for detecting a circle in an image
EDIT
The above idea for an algorithm was too optimistic, at least if applied in a straightforward way. The good news is that it seems that two smart guys (Yonghong Xie and Qiang Ji) have already done the homework for us:
https://www.ecse.rpi.edu/~cvrl/Publication/pdf/Xie2002.pdf
I'm not sure I would create my own algorithm. Why not leverage the work other teams have done to figure out all that curve fitting of bitmaps?
INKSCAPE (App Link)
Inkscape is an open source tool which specializes in vector graphics editing with some ability to work with raster (bitmap) parts too.
Here is a link to a starting point for Inkscape's API:
http://wiki.inkscape.org/wiki/index.php/Script_extensions
It looks like you can script within Inkscape, or access Inkscape via external scripts.
You also may be able to do something with zero scripting, from the inkscape command line interface:
http://wiki.inkscape.org/wiki/index.php/Frequently_asked_questions#Can_Inkscape_be_used_from_the_command_line.3F
COREL DRAW (App Link)
Corel Draw is recognized as the premier industry solution for vector graphics, and has some great tools for converting rasterized images into vector images.
Here's a link to their API:
https://community.coreldraw.com/sdk/api
Here's a link to Corel Draw batch image processing (non-script solution):
http://howto.corel.com/en/c/Automating_tasks_and_batch-processing_images_in_Corel_PHOTO-PAINT
I am writing a function to draw an approximate circle on a square array (in Matlab, but the problem is mainly algorithmic).
The goal is to produce a mask for integrating light that falls on a portion of a CCD sensor from a diffraction-limited point source (whose diameter corresponds to a few pixels on the CCD array). In summary, the CCD sensor sees a pattern with revolution-symmetry, that has of course no obligation to be centered on one particular pixel of the CCD (see example image below).
Here is the algorithm that I currently use to produce my discretized circular mask, and which works partially (Matlab/Octave code):
xt = linspace(-xmax, xmax, npixels_cam); % in physical coordinates (meters)
[X Y] = meshgrid(xt-center(1), xt-center(2)); % shifted coordinate matrices
[Theta R] = cart2pol(X,Y);
R = R'; % cart2pol uses a different convention for lines/columns
mask = (R<=radius);
As you can see, my algorithm selects (sets to 1) all the pixels whose physical distance (in meters) is smaller or equal to a radius, which doesn't need to be an integer.
I feel like my algorithm may not be the best solution to this problem. In particular, I would like it to include the pixel in which the center is present, even when the radius is very small.
Any ideas ?
(See http://i.stack.imgur.com/3mZ5X.png for an example image of a diffraction-limited spot on a CCD camera).
if you like to select pixels if and only if they contain any part of the circle C:
in each pixel place a small circle A with the radius = halv size of the pixel, and another one around it with R=sqrt(2)*half size of the circle (a circumscribed circle)
To test if two circles touch each other you just calculate the center to center distances and subtract the sum of the two radii.
If the test circle C is within A then you select the pixel. If it's within B but not C you need to test all four pixel sides for overlap like this Circle line-segment collision detection algorithm?
A brute force approximate method is to make a much finer grid within each pixel and test each center point in that grid.
This is a well-studied problem. Several levels of optimization are possible:
You can brute-force check if every pixel is inside the circle. (r^2 >= (x-x0)^2 + (y-y0)^2)
You can brute-force check if every pixel in a square bounding the circle is inside the circle. (r^2 >= (x-x0)^2 + (y-y0)^2 where |x-x0| < r and |y-y0| < r)
You can go line-by-line (where |y-y0| < r) and calculate the starting x ending x and fill all the lines in between. (Although square roots aren't cheap.)
There's an infinite possibility of more sophisticated algorithms. Here's a common one: http://en.wikipedia.org/wiki/Midpoint_circle_algorithm (filling in the circle is left as an exercise)
It really depends on how sophisticated you want to be based on how imperative good performance is.
As a followup to my previous question about determining camera parameters I have formulated a new problem.
I have two pictures of the same rectangle:
The first is an image without any transformations and shows the rectangle as it is.
The second image shows the rectangle after some 3d transformation (XYZ-rotation, scaling, XY-translation) is applied. This has caused the rectangle to look a trapezoid.
I hope the following picture describes my problem:
alt text http://wilco.menge.nl/application.data/cms/upload/transformation%20matrix.png
How do determine what transformations (more specifically: what transformation matrix) have caused this tranformation?
I know the pixel locations of the corners in both images, hence i also know the distances between the corners.
I'm confused. Is this a 2d or a 3d problem?
The way I understand it, you have a flat rectangle embedded in 3d space, and you're looking at two 2d "pictures" of it - one of the original version and one based on the transformed version. Is this correct?
If this is correct, then there is not enough information to solve the problem. For example, suppose the two pictures look exactly the same. This could be because the translation is the identity, or it could be because the translation moves the rectangle twice as far away from the camera and doubles its size (thus making it look exactly the same).
This is a math problem, not programming ..
you need to define a set of equations (your transformation matrix, my guess is 3 equations) and then solve it for the 4 transformations of the corner-points.
I've only ever described this using German words ... so the above will sound strange ..
Based on the information you have, this is not that easy. I will give you some ideas to play with, however. If you had the 3D coordinates of the corners, you'd have an easier time. Here's the basic idea.
Move a corner to the origin. Thereafter, rotations will take place about the origin.
Determine vectors of the axes. Do this by subtracting the adjacent corners from the origin point. These will be a local x and y axis for your world.
Determine angles using the vectors. You can use the dot and cross products to determine the angle between the local x axis and the global x axis (1, 0, 0).
Rotate by the angle in step 3. This will give you a new x axis which should match the global x axis and a new local y axis. You can then determine another rotation about the x axis which will bring the y axis into alignment with the global y axis.
Without the z coordinates, you can see that this will be difficult, but this is the general process. I hope this helps.
The solution will not be unique, as Alex319 points out.
If the second image is really a trapezoid as you say, then this won't be too hard. It is a trapezoid (not a parallelogram) because of perspective, so it must be an isosceles trapezoid.
Draw the two diagonals. They intersect at the center of the rectangle, so that takes care of the translation.
Rotate the trapezoid until its parallel sides are parallel to two sides of the original rectangle. (Which two? It doesn't matter.)
Draw a third parallel through the center. Scale this to the sides of the rectangle you chose.
Now for the rotation out of the plane. Measure the distance from the center to one of the parallel sides and use the law of sines.
If it's not a trapezoid, just a quadralateral, then it'll be harder, you'll have to use the angles between the diagonals to find the axis of rotation.