I am learning camera matrix stuff. I already known that I can get the homography of the camera (3*3 matrix) by using four points in a plane in object space. I want to know if we can get the homagraphy with four points not in a plane? If yes, how can I get the matrix? What formulas should I look at?
I also confused homography with another concept: I only need to know three points if I want to convert from points from one coordinate to another coordinate system. So why we need four points in computing homography?
Homography maps points
1. On plane to points at another plane
2. Projections of points in 3D (no obligatory lying on the same plane) during a pure camera rotation or zoom.
The latter can be easily verified if you look at the rays that connect points while sensor plane rotates: green are two sensor positions and black is a 3d object
Since Homography is between projections and not between objects in 3D you don’t care what these projections represent. But this can be confusing, I agree. For example you can point your camera at 3D scene (that is not flat!), then rotate your camera and the two resulting pictures of the scene will be related by homography. This is, by the way, a foundation for image panoramas.
Three point correspondences you mentioned may be reladte to a transformation called Affine (happens during large zooms when a perspective effects disappears) or to the finding a rigid rotation and translation in 3D space. Both require 3 point correspondences but the former needs only 2D points while the latter needs 3D points. The latter case has 6DOF ( 3 for rotation and 3 for translation) while each correspondence provides 2DOF, hence 6/2=3 correspondences. Homography has 8 DOF so there should be 8/2=4 correspondences;
Below is a little diagram that explains the difference between affine and homographs transformation when the original square tilts forward. In affine case the perspective effect is negligible that is far side has the same length as a near one. In the case of Homography the far side is shorter.
If you only have 4 points - and they're not on the same plane - then computing a homography will not work.
If you have a loads of points, and 4 of them do lie on a plane but some don't, there are filters you can use to try to remove the ones not lying on a plane. The filters implemented by OpenCV are called RANSAC and LMeDs.
Also as Hammer says in a comment under your question - The 4th point is there to figure out perspective.
Homography is a 3X3 matrix, which consists of 8 independent unknowns which means it requires 4 equations to solve these unknowns. So, in order to calculate homography we need at least 4 points.
In homography we assume that Z=0 in world scene, so the image projected is assumed as 2D. In a very famous journal named ORB-SLAM, the author formulated a scene-selective approach depending on motion parallax in scene.
Homography is the relation between two planes and the degree of freedom in case of homography transform is 7; hence you need minimum 4 corresponding points.
4 points will give you 4 pair of (x,y) hence you can calculate 7 variables. Homography is homogines transfrom hence the (3,3) value in homography matrix is always 1.
So your first question that can you calculate homography with 3 points in the plane and 4th not on the plane : it's not possible. You need projection of that point on the plane and then you can calculate the homography.
Your 2nd question about how to calculate homography matrix, you can see implemetation of findHomography() in opencv.
Related
Apologies in advance for my feeble maths.
I'm trying to be able to find the corners of a plane in space based on the equation of that plane. Here's what I know. I know three points on the plane and I know where they fall in the 2d coordinate space of the plane (x,y) and where they are in 3d space. I know the width and height of the plane and I can now calculate the equation of the plane. The plane sits on the inside of a large sphere that surrounds the origin so, in theory, it should more or less face where the camera is (though in my diagram it doesn't face the origin as it's just for illustrative purposes)
But it's not clear to me how I can use that to figure out another point. One thought I had was to find the transform that moves the plane parallel to the xy axis and rotate it round one of the points (so it stays in the same place), find the position of the new point, and then rotate it by the inverse of that transform. But it's not clear to me how I would find that transform matrix or how to use it. Could I do this using the normal and vector maths? I understand what normals are, but I'm fuzzy about how to use them.
I currently have two images of a plane in real life from straight above. One to use as a reference image, and another when the plane has undergone a rotation fixed at the centre of the plane thus changing its orientation. The camera stays at a constant position.
I was wondering if I found the homography matrix of this rotation in opencv and then decomposed the homography matrix in order to find the rotation matrix whether this would yield accurate results and I would be able to find the three angles needed to describe the planes rotation in euclidean coordinates to a reasonable degree of accuracy.
Thanks
I have a 2D shape (a circle) that I want to extrude along a 3D curve to create a 3D tube mesh.
Currently the way I generate cross-sections along the curve (which form the basis of the resulting mesh) is to take every control point along the curve, create a 3D transform matrix for it, then multiply the 2D points of my circle by those curve-point matrices to determine their location in 3D space along the curve.
To create the matrix (from 3 vectors), I use the tangent on the curve as the up vector, world-up ([0,1,0]) as the forward vector, and the cross product of the up/forward vectors as the right vector. All three vectors are also orthogonalized during the process to create the final matrix.
The problem comes when my curve tangent is identical to the world-up axis. Ie, my tangent vector is [0,1,0] and the world-up is [0,1,0]....since the cross product of two parallel vectors is not explicit....the resulting extruded mesh has artifacts along those areas of the curve (pinching, twisting, etc).
I thought a potential solution would be to use the dot product of the curve tangent and the world-up as an interpolation value to shift my forward vector from world-up to world-right...in other words, as a curve tangent approaches [0,1,0], my forward vector approaches [1,0,0]...but that results in unwanted twisting along the final mesh as well.
How can I extrude my shape along a curve in a consistent manner that has no flipping/artifacts/twisting? I know it's possible since various off-the-shelf 3D applications can do it...I'm just not sure how.
One way I would approach this is to consider my tangent vector to the 3D curve as actually being a normal vector of the plane I am interested into.
Let's say, the tangent vector is
All you need now is two other vectors that are othoghonal to it, so let's.
Let's construct v like so:
(rotating the coordinates). Because v is the result of the cross product of u and something else, you know that v is orthogonal to u.
(This method will not work if u have equal x,y,z coordinates, in that case, construct the other vector by adding random numbers to at least two variables, rince&repeat).
Then you can simply construct w like before:
normalize and go.
So we have such situation:
In this illustration, the first quadrilateral is shown on the Image Plane and the second quadrilateral is shown on the World Plane. [1]
In my particular case the Image Plane has 3 quadrilaterals - projections of real world squares, which, as we know, have same size, lying on the same plane, with same rotation relative to the plane they are lying on, and are not situated on same line on plane.
I wonder if we can get rotation angles of Image Plane to World Plane knowing stuff described?
In my case as input I have such data structures: original image (RGB pixels), objects (squares) with angles points in pixels (x,y) on Image Plane.
Take a look at Sections 2 and 3 of Algorithms for plane-based pose estimation.
The methods described there assume that you know the (x,y) coordinates of the features in question - in this case the red squares.
The problem you are describing is generally known as pose estimation - determining the 3D orientation and position of an object relative to a camera from a 2D view. For you, the object is a plane. Googling 'pose estimation plane' should give you more sources.
As a followup to my previous question about determining camera parameters I have formulated a new problem.
I have two pictures of the same rectangle:
The first is an image without any transformations and shows the rectangle as it is.
The second image shows the rectangle after some 3d transformation (XYZ-rotation, scaling, XY-translation) is applied. This has caused the rectangle to look a trapezoid.
I hope the following picture describes my problem:
alt text http://wilco.menge.nl/application.data/cms/upload/transformation%20matrix.png
How do determine what transformations (more specifically: what transformation matrix) have caused this tranformation?
I know the pixel locations of the corners in both images, hence i also know the distances between the corners.
I'm confused. Is this a 2d or a 3d problem?
The way I understand it, you have a flat rectangle embedded in 3d space, and you're looking at two 2d "pictures" of it - one of the original version and one based on the transformed version. Is this correct?
If this is correct, then there is not enough information to solve the problem. For example, suppose the two pictures look exactly the same. This could be because the translation is the identity, or it could be because the translation moves the rectangle twice as far away from the camera and doubles its size (thus making it look exactly the same).
This is a math problem, not programming ..
you need to define a set of equations (your transformation matrix, my guess is 3 equations) and then solve it for the 4 transformations of the corner-points.
I've only ever described this using German words ... so the above will sound strange ..
Based on the information you have, this is not that easy. I will give you some ideas to play with, however. If you had the 3D coordinates of the corners, you'd have an easier time. Here's the basic idea.
Move a corner to the origin. Thereafter, rotations will take place about the origin.
Determine vectors of the axes. Do this by subtracting the adjacent corners from the origin point. These will be a local x and y axis for your world.
Determine angles using the vectors. You can use the dot and cross products to determine the angle between the local x axis and the global x axis (1, 0, 0).
Rotate by the angle in step 3. This will give you a new x axis which should match the global x axis and a new local y axis. You can then determine another rotation about the x axis which will bring the y axis into alignment with the global y axis.
Without the z coordinates, you can see that this will be difficult, but this is the general process. I hope this helps.
The solution will not be unique, as Alex319 points out.
If the second image is really a trapezoid as you say, then this won't be too hard. It is a trapezoid (not a parallelogram) because of perspective, so it must be an isosceles trapezoid.
Draw the two diagonals. They intersect at the center of the rectangle, so that takes care of the translation.
Rotate the trapezoid until its parallel sides are parallel to two sides of the original rectangle. (Which two? It doesn't matter.)
Draw a third parallel through the center. Scale this to the sides of the rectangle you chose.
Now for the rotation out of the plane. Measure the distance from the center to one of the parallel sides and use the law of sines.
If it's not a trapezoid, just a quadralateral, then it'll be harder, you'll have to use the angles between the diagonals to find the axis of rotation.