I'm creating a 3D object in Three.js with 6 faces. Each face has a mesh which uses a THREE.PlaneGeometry(width and height both are 256). On the mesh I'm using a JPEG picture which is 256 by 256 for the texture. I'm trying to find a way to find the world coordinate of a pixel coordinate(for example 200,250 is the pixel coordinate) on the Object3D's PlaneGeometry corresponding to where that picture was used as texture.
Object hierarchy:-
Object3D-->face(object3d) (total 6 faces)-->Each face has a mesh(planegeometry) and uses a jpeg file as texture.
Picture1 pixel coordinate-->Used to create texture for Plane1-->World Coordinate corresponding to that pixel coordinate.
Can someone please help me.
Additional information:-
Thanks for the answer. I'm trying to compare 2 results.
Method 1:- One yaw/pitch is obtained by clicking on a specific point in the 3d object(e.g, center of a particular car headlight which is the front face) using a mouse and getting the point of intersection with the front face using raycasting.
Method 2:-The other yaw/pitch is obtained by taking the pixel coordinate of the same point(center of a particular car headlight) and calculating the world space coordinate for that pixel point. Pls note that pixel coordinate is taken from the JPEG file that was used as texture to create the PlaneGeometry for the mesh(which is a child of the front face).
Do you think the above comparison approach is supposed to produce the same results, assuming all other parameters are identical between the 2 approaches?
Well assuming your planes are PlaneGeometry(1,1) then the local coordinate X/Y/ZZ for a given pixel is pixelX / 256, pixelY / 256 and the Z is 0.5
so something like:
var localPoint = new THREE.Vector3(px/256,py/256,0.5)
var worldPoint = thePlaneObject.localToWorld(localPoint)
Related
I am trying to solve a question related to transformation of coordinates in 3-D space but not sure how to approach it.
Lets a vertex point named P is drawn at the origin with a 4x4 transformation matrix. It's then views through a camera that's positioned with a model view matrix and then through a simple projective transform matrix.
How do I calculate the new screen coordinates of P' (x,y,z)?
Before explain of pipeline, you need to know is how pipeline do process to draw on screen.
Everything between process is just matrix multiplication with vector
Model - World - Camera - Projection(or Nomalized Coordinate) - Screen
First step, we call it 'Model Space' because of (0,0,0) is based in model.
And we need to move model space to world space because of we are gonna place model to world so
we need to do transform will be (translate, rotation, scale)TRS * Model(Vector4) because definition of world transform will be different
After do it, model place in world.
Thrid, need to render on camrea space because what we see is through the camera. in world, camera also has position, viewport size and
rotation.. It needs to project from the camera. see
General Formula for Perspective Projection Matrix
After this job done, you will get nomalized coordinate which is Techinically 0-1 coordinates.
Finaly, Screen space. suppose that we are gonna make vido game for mobile. mobile has a lot of screen resolution. so how to get it done?
Simple, scale and translate to get result in screen space coordinate. Because of origin and screen size is different.
So what you are tring to do is 'step 4'.
If you want to get screen position P1 from world, formula will be "Screen Matrix * projection matrix * Camera matrix * P1"
If you want to get position from camera space it would be "Screen Matrix * projection matrix * P1".
There are useful links to understand matrix and calculation so see above links.
https://answers.unity.com/questions/1359718/what-do-the-values-in-the-matrix4x4-for-cameraproj.html
https://www.google.com/search?q=unity+camera+to+screen+matrix&newwindow=1&rlz=1C5CHFA_enKR857KR857&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjk5qfQ18DlAhUXfnAKHabECRUQ_AUIEigB&biw=1905&bih=744#imgrc=Y8AkoYg3wS4PeM:
I am working on fusing Lidar and Camera images in order to perform a classification object algorithm using CNN.
I want to use the KITTI Dataset which provide synchronized lidar and rgb image data. Lidar are 3D-scanners, so the output is a 3D Point Cloud.
I want to use depth information from point cloud as a channel for the CNN. But I have never work with point cloud so I am asking for some help. Is projecting the point cloud into the camera image plane (using projection matrix provide by Kitti) will give me the depth map that I want? Is Python libray pcl useful or I should move to c++ libraries?
If you have any suggestions, thanks you in advance
I'm not sure what projection matrix provide by Kitti includes, so the answer is it depends. If this projection matrix only contains a transformation matrix, you cannot generate depth map from it. The 2D image has distortion that comes from the 2D camera and the point cloud usually doesn't have distortion, so you cannot "precisely" map point cloud to rgb image without intrinsic and extrinsic parameters.
PCL is not required to do this.
Depth map essentially is mapping depth value to rgb image. You can treat each point in point cloud(each laser of lider) as a pixel of the rgb image. Therefore, I think all you need to do is finding which point in point cloud corresponding to the first pixel(top left corner) of the rgb image. Then read the depth value from point cloud based on rgb image resolution.
You have nothing to do with camera. This is all about point cloud data. Lets say you have 10 million of points and each point has x,y,z in meters. If the data is not in meters first convert it. Then you need the position of the lidar. When you subtract position of car from all the points one by one, you will take the position of lidar to the (0,0,0) point, then you can just print the point on a white image. The rest is simple math, there may be many ways to do it. First that comes to my mind: think rgb as binary numbers. Lets say 1cm is scaled to change in 1 blue, 256cm change equals to change in 1 green and 256x256 which is 65536 cm change equals change in 1 red. We know that cam is (0,0,0) if rgb of the point is 1,0,0 then that means 256x256x1+0x256+0x1=65536 cm away from the camera. This could be done in C++. Also you can use interpolation and closest point algorithms to fill blanks if there are
Say we have a web cam image of a road. We have picked 4 connected 2d lines of a rectangle, and we know there are 90deg angles between them. We have set their real life dimentions. How one can get 3d camera position relative to that rectangle from such data?
Having 4 pairs of corresponding coordinates - real word ones and coordinates at camera matrix (in photo image coordinate system), one can calcalate matrix of perspective transformation, for example, with OpenCV function getPerspectiveTransform.
Then apply decomposeProjectionMatrix
If you are using another means/libraries for image acquisition/treatment, they might contain something similar
I'm trying to get top view image from the image captured by a camera in perspective. Already got the intrinsic and extrinsic parameters of the camera to the plane , the ground. The camera is positioned on a robot pointed to the ground at a certain height and tilt. For this camera position got:
Intrinsic matrix (3X3):
KK= [624.2745,0,327.0749;0,622.0777,232.3448;0,0,1]
The translation vector and rotation matrix:
T = [-323.708466;-66.960728;1336.693284]
R =[0.0029,1.0000,-0.0034;0.3850,-0.0042,-0.9229;-0.9229,0.0013,-0.3850]
How can I get a Homography matrix to get top view image of the ground (chess set in the ground), with the information I have?
I'm using matlab. I've done the code to apply the matrix H to the captured image, but I still haven´t i way the get this H matrix properly.
Thanks in advance.
I have 6 textures I would like to load on 6 different faces of a cube. I'm trying to make a new texture by using GLGE.TextureCube();. And then I load all six images to the faces the supposedly should be on the cube like so
mapTex = new GLGE.TextureCube();
mapTex.setSrcNegX("models/map/negx.jpg"); // they are all 1024x1024
mapTex.setSrcNegY("models/map/negy.jpg");
mapTex.setSrcNegZ("models/map/negz.jpg");
mapTex.setSrcPosX("models/map/posx.jpg");
mapTex.setSrcPosY("models/map/posy.jpg");
mapTex.setSrcPosZ("models/map/posz.jpg");
And then I add the texture to the Wavefront object. However, it seems only one of the 6 texture images is getting mapped and its mapped incorrectly.
My guess is that when it creates the new texture map out of the other 6, it tiles them beside each other so the new texture map's co-ordinates no longer correspond to that my obj file.
How can I properly combine 6 textures to one map to be used with GLGE? Or is there a way to manually load a texture on a face of a Mesh?
Cube maps are somewhat special, as the usual UV (ST) texture coordinates don't work for them. A cube map, the name suggests it, consists of 6 quadratic textures, arranged as the faces of a cube. The texture coordinates are not absolute positions on the cube's faces, but directions from the center of the cube away, and the position where a ray from the center in the given direction hits the cube, is the position of the texture on that particular face.
If you apply texture coordinates with the third coordinate being zero, like those in Wavefront, you will address only a slice of the cube's face, namely the part that intersects with the XY plane. If you want to see a working cubemap in action, use the object's smooth normals as texture coordinates.
You'll need to use a different texture coordinate, eg:
materialLayer.setMapinput(GLGE.MAP_OBJ)
depending on what you want try GLGE.MAP_OBJ,GLGE.MAP_NORM or GLGE.MAP_ENV