Despite searching and trying everything for days now, I have a (simple?) problem I can't solve...
Say I have a car going up and down hills. How would I build a 4x4 rotation matrix based on the surface normal of the landscape where the car is? Y is up and I'm not using OpenGL. I just want to apply a rotation matrix to my car's transformation matrix so it looks proper but I can't seem to be able to make such rotation matrix from a surface normal. Any help is greatly appreciated, thank you.
I've tried the following pseudo code but I'm getting weird results.
UpVector = (0,1,0) Y is up
Z = The Surface Normal
X = Cross(Z, UpVector)
Y = Cross(X, Z)
Matrix4x4 =
X.x, X.y, X.z, 0,
Y.x, Y.y, Y.z, 0,
Z.x, Z.y, Z.z, 0,
0, 0, 0, 1
Related
I have a stationary camera, which is 640mm above the ground and is tilted slightly forward (about the x-axis?) roughly 30 degrees, so it's looking slightly down towards a flat ground plane.
My goal is to determine the distance from the camera to any small objects that it detects on the ground, for example, if my camera detects an object which is located at pixel [314, 203], I would like to know where on the ground that object would be in world coordinates [x, y, z] with y=0, and the distance to that object.
I've drawn a diagram to better visualize the problem:
camera plane diagram
I have my rotation matrix and translation vector and also my intrinsic matrix, but I'm not sure 1) if the rotation matrix and translation vector are correct given the information/diagram above, and 2) how to proceed with figuring out a mathematical formula for finding distance and real world location. Here is what I have so far:
Rotation matrix R (generated here https://www.andre-gaschler.com/rotationconverter/) from the orientation [-30, 0, 0] (degrees)
R =
[ 1.0000000, 0.0000000, 0.0000000;
0.0000000, 0.8660254, -0.5000000;
0.0000000, 0.5000000, 0.8660254 ]
Camera is 640mm above ground plane
t =
[0, 640, 0]
Intrinsic matrix from calibration information given by camera
fx=349.595, fy=349.505, cx=328.875, cy=178.204
K =
[ 349.595, 0.0000, 328.875;
0.0000, 349.505, 178.204;
0.0000, 0.0000, 1.000 ]
I also have these distortion parameters, I'm unsure what to do with them or if they are relevant to K
k1=-0.170901, k2=0.0255027, k3=-9.30328e-11, p1=0.000117187, p2=6.42836e-05
I get about this far and then I'm lost, any help would be much appreciated.
Also sorry if this is a lot of information or if it's confusing in any way, I'm very much a beginner when it comes to projection matrices
UPDATE:
After some more research and testing on my own I found a formula that seems to give me a somewhat decent approximation. Given pixel [x, y], I find (I think) the direction vector from the camera origin to the pixel coordinate with:
dir_x = (x - cx) / fx
dir_y = (y - cy) / fy
dir_z = 1
which I then multiply by rotation matrix R, which gives me the real-world vector. I then divide my camera height (640mm) by the y-value of that vector, which gives me (I think) the distance to the specified pixel in the real-world. After some testing and measuring by hand, this seems to be an adequate method for finding the distance, but I'm not sure if I'm missing steps for accuracy or if I'm actually doing this completely wrong.
Again, any insight is greatly appreciated.
A similar question was asked before, unfortunately I cannot comment Samgaks answer so I open up a new post with this one. Here is the link to the old question:
How to calculate ray in real-world coordinate system from image using projection matrix?
My goal is to map from image coordinates to world coordinates. In fact I am trying to do this with the Camera Intrinsics Parameters of the HoloLens Camera.
Of course this mapping will only give me a ray connecting the Camera Optical Centre and all points, which can lie on that ray. For the mapping from image coordinates to world coordinates we can use the inverse camera matrix which is:
K^-1 = [1/fx 0 -cx/fx; 0 1/fy -cy/fy; 0 0 1]
Pcam = K^-1 * Ppix;
Pcam_x = P_pix_x/fx - cx/fx;
Pcam_y = P_pix_y/fy - cy/fy;
Pcam_z = 1
Orientation of Camera Coordinate System and Image Plane
In this specific case the image plane is probably at Z = -1 (However, I am a bit uncertain about this). The Section Pixel to Application-specified Coordinate System on page HoloLens CameraProjectionTransform describes how to go form pixel coordinates to world coordinates. To what I understand two signs in the K^-1 are flipped s.t. we calculate the coordinates as follows:
Pcam_x = (Ppix_x/fx) - (cx*(-1)/fx) = P_pix_x/fx + cx/fx;
Pcam_y = (Ppix_y/fy) - (cy*(-1)/fy) = P_pix_y/fy + cy/fy;
Pcam_z = -1
Pcam = (Pcam_x, Pcam_y, -1)
CameraOpticalCentre = (0,0,0)
Ray = Pcam - CameraOpticalCentre
I do not understand how to create the Camera Intrinsics for the case of the image plane being at a negative Z-coordinate. And I would like to have a mathematical explanation or intuitive understanding of why we have the sign flip (P_pix_x/fx + cx/fx instead of P_pix_x/fx - cx/fx).
Edit: I read in another post that the thirst column of the camera matrix has to be negated for the case that the camera is facing down the negative z-direction. This would explain the sign flip. However, why do we need to change the sign of the third column. I would like to have a intuitive understanding of this.
Here the link to the post Negation of third column
Thanks a lot in advance,
Lisa
why do we need to change the sign of the third column
To understand why we need to negate the third column of K (i.e. negate the principal points of the intrinsic matrix) let's first understand how to get the pixel coordinates of a 3D point already in the camera coordinates frame. After that, it is easier to understand why -z requires negating things.
let's imagine a Camera c, and one point B in the space (w.r.t. the camera coordinate frame), let's put the camera sensor (i.e. image) at E' as in the image below. Therefore f (in red) will be the focal length and ? (in blue) will be the x coordinate in pixels of B (from the center of the image). To simplify things let's place B at the corner of the field of view (i.e. in the corner of the image)
We need to calculate the coordinates of B projected into the sensor d (which is the same as the 2d image). Because the triangles AEB and AE'B' are similar triangles then ?/f = X/Z therefore ? = X*f/Z. X*f is the first operation of the K matrix is. We can multiply K*B (with B as a column vector) to check.
This will give us coordinates in pixels w.r.t. the center of the image. Let's imagine the image is size 480x480. Therefore B' will look like this in the image below. Keep in mind that in image coordinates, the y-axis increases going down and the x-axis increases going right.
In images, the pixel at coordinates 0,0 is in the top left corner, therefore we need to add half of the width of the image to the point we have. then px = X*f/Z + cx. Where cx is the principal point in the x-axis, usually W/2. px = X*f/Z + cx is exactly as doing K * B / Z. So X*f/Z was -240, if we add cx (W/2 = 480/2 = 240) and therefore X*f/Z + cx = 0, same with the Y. The final pixel coordinates in the image are 0,0 (i.e. top left corner)
Now in the case where we use z as negative, when we divide X and Y by Z, because Z is negative, it will change the sign of X and Y, therefore it will be projected to B'' at the opposite quadrant as in the image below.
Now the second image will instead be:
Because of this, instead of adding the principal point, we need to subtract it. That is the same as negating the last column of K.
So we have 240 - 240 = 0 (where the second 240 is the principal point in x, cx) and the same for Y. The pixel coordinates are 0,0 as in the example when z was positive. If we do not negate the last column we will end up with 480,480 instead of 0,0.
Hope this helped a little bit
I have initiated a PIXI js canvas:
g_App = new PIXI.Application(800, 600, { backgroundColor: 0x1099bb });
Set up a container:
container = new PIXI.Container();
g_App.stage.addChild(container);
Put a background texture (2000x2000) into the container:
var texture = PIXI.Texture.fromImage('picBottom.png');
var back = new PIXI.Sprite(texture);
container.addChild(back);
Set the global:
var g_Container = container;
I do various pivot points and rotations on container and canvas stage element:
// Set the focus point of the container
g_App.stage.x = Math.floor(400);
g_App.stage.y = Math.floor(500); // Note this one is not central
g_Container.pivot.set(1000, 1000);
g_Container.rotation = 1.5; // radians
Now I need to be able to convert a canvas pixel to the pixel on the background texture.
g_Container has an element transform which in turn has several elements localTransform, pivot, position, scale ands skew. Similarly g_App.stage has the same transform element.
In Maths this is simple, you just have vector point and do matix operations on them. Then to go back the other way you just find inverses of those matrices and multiply backwards.
So what do I do here in pixi.js?
How do I convert a pixel on the canvas and see what pixel it is on the background container?
Note: The following is written using the USA convention of using matrices. They have row vectors on the left and multiply them by the matrix on the right. (Us pesky Brits in the UK do the opposite. We have column vectors on the right and multiply it by the matrix on the left. This means UK and USA matrices to do the same job will look slightly different.)
Now I have confused you all, on with the answer.
g_Container.transform.localTransform - this matrix takes the world coords to the scaled/transposed/rotated COORDS
g_App.stage.transform.localTransform - this matrix takes the rotated world coords and outputs screen (or more accurately) html canvas coords
So for example the Container matrix is:
MatContainer = [g_Container.transform.localTransform.a, g_Container.transform.localTransform.b, 0]
[g_Container.transform.localTransform.c, g_Container.transform.localTransform.d, 0]
[g_Container.transform.localTransform.tx, g_Container.transform.localTransform.ty, 1]
and the rotated container matrix to screen is:
MatToScreen = [g_App.stage.transform.localTransform.a, g_App.stage.transform.localTransform.b, 0]
[g_App.stage.transform.localTransform.c, g_App.stage.transform.localTransform.d, 0]
[g_App.stage.transform.localTransform.tx, g_App.stage.transform.localTransform.ty, 1]
So to get from World Coordinates to Screen Coordinates (noting our vector will be a row on the left, so the first operation matrix that acts first on the World coordinates must also be on the left), we would need to multiply the vector by:
MatAll = MatContainer * MatToScreen
So if you have a world coordinate vector vectWorld = [worldX, worldY, 1.0] (I'll explain the 1.0 at the end), then to get to the screen coords you would do the following:
vectScreen = vectWorld * MatAll
So to get screen coords and to get to world coords we first need to calculate the inverse matrix of MatAll, call it invMatAll. (There are loads of places that tell you how to do this, so I will not do it here.)
So if we have screen (canvas) coordinates screenX and screenY, we need to create a vector vectScreen = [screenX, screenY, 1.0] (again I will explain the 1.0 later), then to get to world coordinates worldX and worldY we do:
vectWorld = vectScreen * invMatAll
And that is it.
So what about the 1.0?
In a 2D system you can do rotations, scaling with 2x2 matrices. Unfortunately you cannot do a 2D translations with a 2x2 matrix. Consequently you need 3x3 matrices to fully describe all 2D scaling, rotations and translations. This means you need to make your vector 3D as well, and you need to put a 1.0 in the third position in order to do the translations properly. This 1.0 will also be 1.0 after any matrix operation as well.
Note: If we were working in a 3D system we would need 4x4 matrices and put a dummy 1.0 in our 4D vectors for exactly the same reasons.
I'm trying to achieve same rotation as in scene editor but with code so object always rotate around selected axis, however if I look at angles(x,y,z) in editor they changes quite randomly
![Local node axis][1]
I've tried to use quaternions, but can't get it working
PS. my bad was using rotation property instead of orientation both SCNVector4, have read doc properly)
Seems you was really close, you had to swap parameters in GLKQuaternionMultiply call. I used solution in https://stackoverflow.com/a/39813058/4124265 to achieve rotation only by Z axis:
let orientation = modelNode.orientation
var glQuaternion = GLKQuaternionMake(orientation.x, orientation.y, orientation.z, orientation.w)
// Rotate around Z axis
let multiplier = GLKQuaternionMakeWithAngleAndAxis(0.5, 0, 0, 1)
glQuaternion = GLKQuaternionMultiply(glQuaternion, multiplier)
modelNode.orientation = SCNQuaternion(x: glQuaternion.x, y: glQuaternion.y, z: glQuaternion.z, w: glQuaternion.w)
To rotate arround Y:
// Rotate around Y axis
let multiplier = GLKQuaternionMakeWithAngleAndAxis(0.5, 0, 1, 0)
glQuaternion = GLKQuaternionMultiply(glQuaternion, multiplier)
By setting the rotation property, the absolute rotation of the object is changed, rather than the relative rotation.
Here's some pseudo code
Compute the quaternion represents the relative rotation. I would use the GLKQuaternionMakeWithAngleAndAxis function to do so.
Apply the rotation to the orientation property:
let initial_object_orientation = rotateNode.orientation;
new_orientation = GLKQuaternionMultiply(rotation_quaternion, initial_object_orientation)
Assign the new orientation
rotatNode.orientation = new_orientation
Hope it helps.
I'm attempting to use Java3D to rotate a vector. My goal is create a transform that will make the vector parallel with the y-axis. To do this, I calculated the angle between the original vector and an identical vector except that it has a z value of 0 (original x, original y, 0 for z-value). I then did the same thing for the y-axis (original x, 0 for y-value, original z). I then used each angle to create two Transform3D objects, multiply them together and apply to the vector. My code is as follows:
Transform3D yRotation = new Transform3D();
Transform3D zRotation = new Transform3D();
//create new normal vector
Vector3f normPoint = new Vector3f (normal.getX(), normal.getY(), normal.getZ());
//****Z rotation methods*****
Vector3f newNormPointZ = new Vector3f(normal.getX(), normal.getY(),0.0F);
float zAngle = normPoint.angle(newNormPointZ);
zRotation.rotZ(zAngle);
//****Y rotation methods*****
Vector3f newNormPointY = new Vector3f(normal.getX(),0.0F, normal.getZ());
float yAngle = normPoint.angle(newNormPointY);
yRotation.rotY(yAngle);
//combine the two rotations
yRotation.mul(zRotation);
System.out.println("before trans normal = " +normPoint.x + ", "+normPoint.y+", "+normPoint.z);
//PRINT STATEMENT RETURNS: before trans normal = 0.069842085, 0.99316376, 0.09353002
//perform transform
yRotation.transform(normPoint);
System.out.println("normal trans = " +normPoint.x + ", "+normPoint.y+", "+normPoint.z);
//PRINT STATEMENT RETURNS: normal trans = 0.09016449, 0.99534255, 0.03411238
I was hoping the transform would produce x and z values of or very close to 0. While the logic makes sense to me, I'm obviously missing something..
If your goal is to rotate a vector parallel to the y axis, why can't you just manually set it using the magnitude of the vector and setting your vector to <0, MAGNITUDE, 0>?
Also, you should know that rotating a vector to be directly pointing +Y or -Y can cause some rotation implementations to break, since they operate according to the "world up" vector, or, <0,1,0>. You can solve this by building your own rotation system and using the "world out" vector <0,0,1> when rotating directly up.
If you have some other purpose for this, fastgraph helped me with building rotation matrices.
It's best to understand the math of what's going on so that you know what to do in the future.