I have two calibrated cameras C1 and C2 and assuming that their optic axes are perfectly parallel and the baseline is known, can someone please explain how I can calculate the x,y and z of a point from the pixel coordinates from C1 and C2 for the same point?
Related
A similar question was asked before, unfortunately I cannot comment Samgaks answer so I open up a new post with this one. Here is the link to the old question:
How to calculate ray in real-world coordinate system from image using projection matrix?
My goal is to map from image coordinates to world coordinates. In fact I am trying to do this with the Camera Intrinsics Parameters of the HoloLens Camera.
Of course this mapping will only give me a ray connecting the Camera Optical Centre and all points, which can lie on that ray. For the mapping from image coordinates to world coordinates we can use the inverse camera matrix which is:
K^-1 = [1/fx 0 -cx/fx; 0 1/fy -cy/fy; 0 0 1]
Pcam = K^-1 * Ppix;
Pcam_x = P_pix_x/fx - cx/fx;
Pcam_y = P_pix_y/fy - cy/fy;
Pcam_z = 1
Orientation of Camera Coordinate System and Image Plane
In this specific case the image plane is probably at Z = -1 (However, I am a bit uncertain about this). The Section Pixel to Application-specified Coordinate System on page HoloLens CameraProjectionTransform describes how to go form pixel coordinates to world coordinates. To what I understand two signs in the K^-1 are flipped s.t. we calculate the coordinates as follows:
Pcam_x = (Ppix_x/fx) - (cx*(-1)/fx) = P_pix_x/fx + cx/fx;
Pcam_y = (Ppix_y/fy) - (cy*(-1)/fy) = P_pix_y/fy + cy/fy;
Pcam_z = -1
Pcam = (Pcam_x, Pcam_y, -1)
CameraOpticalCentre = (0,0,0)
Ray = Pcam - CameraOpticalCentre
I do not understand how to create the Camera Intrinsics for the case of the image plane being at a negative Z-coordinate. And I would like to have a mathematical explanation or intuitive understanding of why we have the sign flip (P_pix_x/fx + cx/fx instead of P_pix_x/fx - cx/fx).
Edit: I read in another post that the thirst column of the camera matrix has to be negated for the case that the camera is facing down the negative z-direction. This would explain the sign flip. However, why do we need to change the sign of the third column. I would like to have a intuitive understanding of this.
Here the link to the post Negation of third column
Thanks a lot in advance,
Lisa
why do we need to change the sign of the third column
To understand why we need to negate the third column of K (i.e. negate the principal points of the intrinsic matrix) let's first understand how to get the pixel coordinates of a 3D point already in the camera coordinates frame. After that, it is easier to understand why -z requires negating things.
let's imagine a Camera c, and one point B in the space (w.r.t. the camera coordinate frame), let's put the camera sensor (i.e. image) at E' as in the image below. Therefore f (in red) will be the focal length and ? (in blue) will be the x coordinate in pixels of B (from the center of the image). To simplify things let's place B at the corner of the field of view (i.e. in the corner of the image)
We need to calculate the coordinates of B projected into the sensor d (which is the same as the 2d image). Because the triangles AEB and AE'B' are similar triangles then ?/f = X/Z therefore ? = X*f/Z. X*f is the first operation of the K matrix is. We can multiply K*B (with B as a column vector) to check.
This will give us coordinates in pixels w.r.t. the center of the image. Let's imagine the image is size 480x480. Therefore B' will look like this in the image below. Keep in mind that in image coordinates, the y-axis increases going down and the x-axis increases going right.
In images, the pixel at coordinates 0,0 is in the top left corner, therefore we need to add half of the width of the image to the point we have. then px = X*f/Z + cx. Where cx is the principal point in the x-axis, usually W/2. px = X*f/Z + cx is exactly as doing K * B / Z. So X*f/Z was -240, if we add cx (W/2 = 480/2 = 240) and therefore X*f/Z + cx = 0, same with the Y. The final pixel coordinates in the image are 0,0 (i.e. top left corner)
Now in the case where we use z as negative, when we divide X and Y by Z, because Z is negative, it will change the sign of X and Y, therefore it will be projected to B'' at the opposite quadrant as in the image below.
Now the second image will instead be:
Because of this, instead of adding the principal point, we need to subtract it. That is the same as negating the last column of K.
So we have 240 - 240 = 0 (where the second 240 is the principal point in x, cx) and the same for Y. The pixel coordinates are 0,0 as in the example when z was positive. If we do not negate the last column we will end up with 480,480 instead of 0,0.
Hope this helped a little bit
I have two cameras I have calibrated the cameras considering there position at the same point. But actually the positions of the cameras is slightly different than considered during calibration. This caused a parallax error. Now when I capture a point with these two cameras I get a misalignment in the images due to parallax Now I want to calculate this misalignment in pixels.
I tried to calculate the misalignment in m
Z(measured) = Z(calib) + (Du /tan a1 + tan a2)
Z(measured) is actual distance from cam to object in m
Z(calib) is distance from camera to calibration marker point.
Du is distance between the projected point of the object captured by two cameras on image plane in meters
tan a1 = (distance between camera position during calibration and actual camera 1 position/ distance between camera position during calibration and position of calibration marker point)
tan a2 = (distance between camera position during calibration and actual camera 2 position/ distance between camera position during calibration and position of calibration marker point)
How can I now convert this value of Du in meters to pixels
If you know what the ground sample distance of your image you can use that to determine how much distance a pixel represents and use that number to convert meters to pixels.
Ground sample distance is calculated as:
GSD = D/F* PS
GSD = Ground sample distance
D = Distance to object (from camera)
F = Focal Length
PS = Pixel size (calculated using Photo dimension/Camera Sensor Dimension.
PS should be almost if not exactly the same when comparing Width and Height result.
Having GSD you can then work backwards to determine number of pixels based on distance in meters (note this means you will want units to all be in meters).
I have an array of diamonds as shown in the image and I know the position of every diamond and the distance from the origin of the diamond to any vertex (They are all the same distance from the center). I am also given a point. Given that information what is the most efficient method to find which diamond the point is in.
I know that I can just check the distance of the point from the position of every diamond but that seems way too cpu intensive as I have to do this multiple times.
Also, this shouldn't matter, but I am using C# and Unity 3D to do this.
If your diamonds form a regular pattern as in your picture, then just perform coordinate transformation to rotate the whole thing 45 degrees CW or CCW with (0, 0) as the origin. After that the problem becomes trivial: locating a point in a regular orthogonal grid.
Diamonds border line have equations
x + y = a0 + u * Size
y - x = b0 + v * Size
where a0, b0 are coordinates of the some vertex of base diamond (that has cell coordinates 0, 0), u and v are cell coordinates, Size is edge length. So to find what diamond point (px, py) belongs to, you can calculate
u = Floor((px + py - a0) / Size))
v = Floor((py - px - b0) / Size))
I would like to check the intersection of two squares which are not axis-aligned. I know how to do it for axis-aligned squares. Can I extend the same idea?
Basically I want to distribute squares by gaussian distribution on the x-y plane in +ve quadrant say, but two squares should not be intersecting so I have to shift the original center of the square. Suggestions are welcome.
Separating axis theorem (link2) is suitable for effective checking of convex polygon intersection. For squares prerequisite (normals and so on) calculation becomes especially simple.
After almost 4-5 hours of thinking, I got one click. Just sharing if anyone needs this. PseudoCode
Input: Square1 and Square2
Bool(Sqaure1,square2)
for vertex v0 to v3 of sqaure 1
angle = 0;
for vertex P0 to P3 of square 2 (In anticlockwise of clockwise way)
angle = angle + angle(Pj,vi,P((j+1)%4));
if(abs(angle-360) == 0)
return true;
return false;
IDEA: point of one square inside or on the line of another square will have angle sum of 360 with all points if two squares intersects.So you can call function twice by swapping arguments and if anyone returns true then answer is yes.(Call twice or check if angle sum is 0 or 360 exactly)
If I am right, you can proceed as follows:
rotate both Sa and Sb so that Sa becomes axis-aligned;
check overlap of Sa' with the bounding box of Sb';
rotate both Sa and Sb so that Sb becomes axis-aligned;
check overlap of the bounding box of Sa" with Sb".
If there is a configuration with no overlap, then the squares are disjoint.
In analytical terms, the no-overlap conditions will have a form like
C1 + C2 (|cos(t1+t2)| + |sin(t1+t2)|) < 2D Min(|cos(t1)|, (|sin(t1)|)
where C1, C2 are the sides, D the distance between centers and t1, t2 are the angles between sides and the center line, and similar by exchange of 1/2.
I'm using Electro in Lua for some 3D simulations, and I'm running in to something of a mathematical/algorithmic/physics snag.
I'm trying to figure out how I would find the "spin" of a sphere of a sphere that is spinning on some axis. By "spin" I mean a vector along the axis that the sphere is spinning on with a magnitude relative to the speed at which it is spinning. The reason I need this information is to be able to slow down the spin of the sphere by applying reverse torque to the sphere until it stops spinning.
The only information I have access to is the X, Y, and Z unit vectors relative to the sphere. That is, each frame, I can call three different functions, each of which returns a unit vector pointing in the direction of the sphere model's local X, Y and Z axes, respectively. I can keep track of how each of these change by essentially keeping the "previous" value of each vector and comparing it to the "new" value each frame. The question, then, is how would I use this information to determine the sphere's spin? I'm stumped.
Any help would be great. Thanks!
My first answer was wrong. This is my edited answer.
Your unit vectors X,Y,Z can be put together to form a 3x3 matrix:
A = [[x1 y1 z1],
[x2 y2 z2],
[x3 y3 z3]]
Since X,Y,Z change with time, A also changes with time.
A is a rotation matrix!
After all, if you let i=(1,0,0) be the unit vector along the x-axis, then
A i = X so A rotates i into X. Similarly, it rotates the y-axis into Y and the
z-axis into Z.
A is called the direction cosine matrix (DCM).
So using the DCM to Euler axis formula
Compute
theta = arccos((A_11 + A_22 + A_33 - 1)/2)
theta is the Euler angle of rotation.
The magnitude of the angular velocity, |w|, equals
w = d(theta)/dt ~= (theta(t+dt)-theta(t)) / dt
The axis of rotation is given by e = (e1,e2,e3) where
e1 = (A_32 - A_23)/(2 sin(theta))
e2 = (A_13 - A_31)/(2 sin(theta))
e3 = (A_21 - A_12)/(2 sin(theta))
I applaud ~unutbu's, answer, but I think there's a simpler approach that will suffice for this problem.
Take the X unit vector at three successive frames, and compare them to get two deltas:
deltaX1 = X2 - X1
deltaX2 = X3 - X2
(These are vector equations. X1 is a vector, the X vector at time 1, not a number.)
Now take the cross-product of the deltas and you'll get a vector in the direction of the rotation vector.
Now for the magnitude. The angle between the two deltas is the angle swept out in one time interval, so use the dot product:
dx1 = deltaX1/|deltaX1|
dx2 = deltax2/|deltaX2|
costheta = dx1.dx2
theta = acos(costheta)
w = theta/dt
For the sake of precision you should choose the unit vector (X, Y or Z) that changes the most.