Is it possible to get a rectangle distortion from few fixed points?
This example will explain better what I mean:
Suppose I've got this image with a rectangle and two points, the two points are recognized in the other image where the image is distorted
How can I reproduce the distortion knowing the position of the two(or maybe three) previous points??
My purpose is to get the distorted rectangle border. It's not an easy image as the one in the example so I can't just filter colors, I need to find a way to get the distorted image border.
I believe what you're looking for can be described as an affine transform. If you want general transform of a planar surface, you may want perspective transform instead.
You can find the OpenCV implementation here. The relevant functions are cv::getAffineTransform which requires 3 pairs of points or cv::getPerspectiveTransform which requires 4 pairs of points.
Note: if you're using an automatic feature detector/matcher, it would be best to use far more point pairs than the minimum and use a robust outlier rejection algorithm like RANSAC.
shift and rotation need - 2 points
Affine tranform need - 3 points
Perspective tranform need - 4 points
Related
I have many images like the following (only white and black):
My final problem is to find well matching ellipses. Unfortunately the real used images are not always that nice like this. They could be deformed a bit, which makes ellipse matching probably harder.
My idea is to find "break points". I markes them in the following picture:
Maybe these points could help to make a matching for the ellipses. The end result should be something like this:
Has someone an idea what algorithm may be used to find these break points? Or even better to make good ellipse matching?
Thank you very much
Sample the circumference points
Just scan your image and select All Black pixels with any White neighbor. You can do this by recoloring the remaining black pixels to any unused color (Blue).
After whole image is done you can recolor the inside back from unused color (Blue) to white.
form a list of ordered circumference points per cluster/ellipse
Just scan your image and find first black pixel. Then use A* to order the circumference points and store the path in some array or list pnt[] and handle it as circular array.
Find the "break points"
They can be detect by peak in the angle between neighbors of found points. something like
float a0=atan2(pnt[i].y-pnt[i-1].y,pnt[i].x-pnt[i-1].x);
float a1=atan2(pnt[i+1].y-pnt[i].y,pnt[i+1].x-pnt[i].x);
float da=fabs(a0-a1); if (da>M_PI) da=2.0*M_PI-da;
if (da>treshold) pnt[i] is break point;
or use the fact that on break point the slope angle delta change sign:
float a1=atan2(pnt[i-1].y-pnt[i-2].y,pnt[i-1].x-pnt[i-2].x);
float a1=atan2(pnt[i ].y-pnt[i-1].y,pnt[i ].x-pnt[i-1].x);
float a2=atan2(pnt[i+1].y-pnt[i ].y,pnt[i+1].x-pnt[i ].x);
float da0=a1-a0; if (da0>M_PI) da0=2.0*M_PI-da0; if (da0<-M_PI) da0=2.0*M_PI+da0;
float da1=a2-a1; if (da1>M_PI) da1=2.0*M_PI-da1; if (da1<-M_PI) da1=2.0*M_PI+da1;
if (da0*da1<0.0) pnt[i] is break point;
fit ellipses
so if no break points found you can fit the entire pnt[] as single ellipse. For example Find bounding box. It's center is center of ellipse and its size gives you semi-axises.
If break points found then first find the bounding box of whole pnt[] to obtain limits for semi-axises and center position area search. Then divide the pnt[] to parts between break points. Handle each part as separate part of ellipse and fit.
After all the pnt[] parts are fitted check if some ellipses are not the same for example if they are overlapped by another ellipse the they would be divided... So merge the identical ones (or average to enhance precision). Then recolor all pnt[i] points to white, clear the pnt[] list and loop #2 until no more black pixel is found.
how to fit ellipse from selection of points?
algebraically
use ellipse equation with "evenly" dispersed known points to form system of equations to compute ellipse parameters (x0,y0,rx,ry,angle).
geometrically
for example if you detect slope 0,90,180 or 270 degrees then you are at semi-axis intersection with circumference. So if you got two such points (one for each semi-axis) that is all you need for fitting (if it is axis-aligned ellipse).
for non-axis-aligned ellipses you need to have big enough portion of the circumference available. You can exploit the fact that center of bounding box is also the center of ellipse. So if you got the whole ellipse you know also the center. The semi-axises intersections with circumference can be detected with biggest and smallest tangent change. If you got center and two points its all you need. In case you got only partial center (only x, or y coordinate) you can combine with more axis points (find 3 or 4)... or approximate the missing info.
Also the half H,V lines axis is intersecting ellipse center so it can be used to detect it if not whole ellipse in the pnt[] list.
approximation search
You can loop through "all" possible combination of ellipse parameters within limits found in #4 and select the one that is closest to your points. That would be insanely slow of coarse so use binary search like approach something like mine approx class. Also see
Curve fitting with y points on repeated x positions (Galaxy Spiral arms)
on how it is used for similar fit to yours.
hybrid
You can combine geometrical and approximation approach. First compute what you can by geometrical approach. And then compute the rest with approximation search. you can also increase precision of the found values.
In rare case when two ellipses are merged without break point the fitted ellipse will not match your points. So if such case detected you have to subdivide the used points into groups until their fits matches ...
This is what I have in mind with this:
You probably need something like this:
https://en.wikipedia.org/wiki/Circle_Hough_Transform
Your edge points are simply black pixels with at least one white 4-neighbor.
Unfortunately, though, you say that your ellipses may be “tilted”. Generic ellipses are described by quadratic equations like
x² + Ay² + Bxy + Cx + Dy + E = 0
with B² < 4A (⇒ A > 0). This means that, compared to the circle problem, you don't have 3 dimensions but 5. This causes the Hough transform to be considerably harder. Luckily, your example suggests that you don't need a high resolution.
See also: algorithm for detecting a circle in an image
EDIT
The above idea for an algorithm was too optimistic, at least if applied in a straightforward way. The good news is that it seems that two smart guys (Yonghong Xie and Qiang Ji) have already done the homework for us:
https://www.ecse.rpi.edu/~cvrl/Publication/pdf/Xie2002.pdf
I'm not sure I would create my own algorithm. Why not leverage the work other teams have done to figure out all that curve fitting of bitmaps?
INKSCAPE (App Link)
Inkscape is an open source tool which specializes in vector graphics editing with some ability to work with raster (bitmap) parts too.
Here is a link to a starting point for Inkscape's API:
http://wiki.inkscape.org/wiki/index.php/Script_extensions
It looks like you can script within Inkscape, or access Inkscape via external scripts.
You also may be able to do something with zero scripting, from the inkscape command line interface:
http://wiki.inkscape.org/wiki/index.php/Frequently_asked_questions#Can_Inkscape_be_used_from_the_command_line.3F
COREL DRAW (App Link)
Corel Draw is recognized as the premier industry solution for vector graphics, and has some great tools for converting rasterized images into vector images.
Here's a link to their API:
https://community.coreldraw.com/sdk/api
Here's a link to Corel Draw batch image processing (non-script solution):
http://howto.corel.com/en/c/Automating_tasks_and_batch-processing_images_in_Corel_PHOTO-PAINT
With a camera inside a cylinder I capture a image. I want to transform that image into a plane 2d. The image inside the cylinder have a lot of dots which forms a grid.
What I tried to do was estimating the transformation. With blob analysis I can detect the center of each point and obtain the coordinates in pixels. I save this in matrix called ImCilynder. After that i create a matrix with coordinates of that points in the plane with the name Im2d.
I calculate the transformation (H) solving the equation:
Imcilynder * H= Im2d;
H= matrix [9x1]
H=pinv(Imcilynder) * Im2d
But, when i'm doing the test with the same points, the result is completely random, so i'm doing something wrong.
Is there a better way to solve this? Can you help me?
Explaining better,
I'm trying to find the transformation which transforms the image above to this image:
So, to clarify, I want the projection of the points which i see in the first image to a plane. Basically i want o unwrap the cylinder.
After the calculation of the transformation matrix. I'm expecting to multiply the first image with the transformation matrix and obtain the points in the plane. Or to multiply the coordinates of the center of the black dots and obtain the coordinates of that dots in the plane. Is this possibly?
Thank you very much,
Afonso
Well, what do yo wish to have in a plane? the circles forming a grid? Because if this is the case you need to remove the radial distortion, these kind of models are represented by some parameters, are non-linear by the way. May be if you can find a very good algorithm, you are going to obtain something like this:
If this is not your idea, you need to apply an elastic transformation and this kind of transformation needs to use a kind of grid that is the model of the transformation and you need to propose your model of grid. If you want to do this automatically you need to resort to elastic registration algorithms and you can use a model like this one:
Any ways, this is not a trivial task, there are a lot of research about complex transformations of course if you want to automatically obtain the transformation. Otherwise you can use photoshop ;).
On a discrete grid-based plane (think: pixels of an image), I have a closed contour that can be expressed either by:
a set of 2D points (x1,y1);(x2,y2);(x3,y3);...
or a 4-connected Freeman code, with a starting point: (x1,y1) + 00001112...
I know how to switch from one to the other of these representations. This will be the input data.
I want to get the set of grid coordinates that are bounded by the contour.
Consider this example, where the red coordinates are the contour, and the gray one the starting point:
If the gray coordinate is, say, at (0,0), then I want a vector holding:
(1,1),(2,1),(3,1),(3,2)
Order is not important, and the output vector can also hold the contour itself.
Language of choice is C++, but I'm open to any existing code, algorithm, library, pointer, whatever...
I though that maybe CGAL would have something like this, but I am unfamiliar with it and couldn't find my way through the manual, so I'm not even sure.
I also looked toward Opencv but I think it does not provide this algorithm (but I can be wrong?).
I was thinking about finding the bounding rectangle, then checking each of the points in the rectangle to see if they are inside/outside, but this seems suboptimal. Any idea ?
One way to solve this is drawContours, and you have contours points with you.
Create blank Mat and draw contour with thickness = 1(boundary).
Create another blank Mat and draw contour with thickness = CV_FILLED(whole area including boundary).
Now bitwise_and between above two(you got filled area excluding boundary).
Finally check for non-zero pixel.
Suppose I have an image of a scene as depicted above. A sort of a pole with a blob on it next to possibly similar objects with no blobs.
How can I find the blob marked by the red circle (a binary image indicating which pixels belong to the blob).
Note that the pole together with the blob may be rotated arbitrarily and also size may vary.
Can you try to do it in below 4 steps?
Circle detection like: writing robust (color and size invariant) circle detection with opencv (based on Hough transform or other features)
Line detection, like: Finding location of rectangles in an image with OpenCV
Identify rectangle position by combining neighboring lines (For each line segment you have the start and end point position, you also know the direction of each line segment. So that you can figure out if two connecting line segments (whose endpoints are close) are orthogonal. Your goal is to find 3 such segments for each rectangle.)
Check the relative position of each circle and rectangle to see if any pair can form the knob shape.
One approach could be using Viola-Jones object detection framework.
Though the framework is mostly used for face detection - it is actually designed for generic objects you feed to the algorithm.
The algorithm basic idea is to feed samples of "good object" (what you are looking for) and "bad objects" to a machine learning algorithm - which generates patterns from the images as its features.
During Classification - using a sliding window the algorithm will search for a "match" to the object (the classifier returned a positive answer).
The algorithm uses supervised learning and thus requires a labeled set of examples (both positive and negative ones)
I'm sure there is some boundary-map algorithm in image processing to do this.
Otherwise, here is a quick fix: pick a pixel at the center of the
"undiscovered zone", which initially is the whole image.
trace the horizantal and vertical lines at 4 directions each ending at the
borders of the zone and find the value changes from 0 to 1 or the vice verse.
Trace each such value switch and complete the boundary of each figure (Step-A).
Do the same for the zones
that still are undiscovered: start at some center
point and skim thru the lines connecting the center to the image border or to a
pixel at the boundary of a known zone.
In Step-A, you can also check to see whether the boundary you traced is
a line or a curve. Whenever it is a curve, you need only two points on it--
points at some distance from one another for the accuracy of the calculation.
The lines perpendicular each to these two points of tangency
intersect at the center of the circle red in your figure.
You can segment the image. Then use only the pixels in the segments to contribute to a Hough-transform to find the circles.
Then you will only have segments with circle in them. You can use a modified hough transform to find rectangles. The 'best' rectangle and square combination will then be your match. This is very computationally intentsive.
Another approach, if you already have these binary pictures, is to transform to a (for example 256 bin) sample by taking the distance to the centroid compared to the distance travelled along the edge. If you start at the point furthest away from the centroid you have a fairly rotational robust featurevector.
I am interested in using shapes like these:
Usually a tangram is made of 7 shapes(5 triangles, 1 square and 1 parallelogram).
What I want to do is fill a shape only with tangram shapes, so at this point,
the size and repetition of shapes shouldn't matter.
Here's something I manually tried:
I am a bit lost on how to approach this.
Assuming I have a path (an ordered list/array of points of the outline),
I imagine I should try to do some sort of triangulation.
Is there such a thing as Deulanay triangulation with triangles constrained to 45 degrees
right angled triangles ?
A more 'brute' approach would be to add a bunch of triangles(45 degrees) and use SAT
for collision detection to 'fix' overlaps, and hopefully gaps will be avoided.
Since the square and parallelogram can be made of triangles(45 degrees) too, I imagine there
would be a nice clean geometric solution, right ?
How do I pack triangles(45 degrees) inside an arbitrary shape ?
Any ideas are welcome.
A few random thoughts (maybe they help you find a better solution) if you're using only the original sizes of the shapes:
as you point out, all shapes in the tangram can be made composed of e.g. the yellow or pink triangle (d-g-c), so try also thinking of a bottom-up approach such as first trying to place as many yellow triangles into your shape and then combine them into larger shapes if possible. In the worst case, you'll end up with a set of these smallest triangles.
any kind triangulation of non-polygons (such as the half-moon in your example) probably does not work very well...
It looks like you require that the shapes can only have a few discrete orientations. To find the best fit of these triangles into the given shape, I'd propose the following approximate solution: draw a grid of triangles (i.e. a square grid with diagonal lines) across the shape and take those triangles which are fully contained. This most likely will not give you the optimal coverage but then you could repeatedly shift the grid by a tenth of the grid size in horizontal and vertical direction and see whether you'll find something which covers a larger fraction of the original shape (or you could go in steps of 1/2 then 1/4 etc. of the original grid size in the spirit of a binary search).
If you allow any arbitrary scaling of the shapes you could approximate any (reasonably smooth ?) shape to arbitrary precision by adding smaller and smaller shapes. E.g. if you have a raster image, you can e.g. choose the size of the yellow triangle such that two of them make a pixel on the image and then you can represent any such raster image.