I'm using Tango motion tracking and it is very easy to get the pose of the device relative to the TANGO_START_OF_SERVICE. For the translation that works fine for me, but I'd like my orientation to be aligned with gravity, so that the yaw and roll angles are aligned with gravity rather than with the arbitrary position at which the Tango service started. I'm fine with an arbitrary azimuth angle.
I can do this by using the accelerometer data to get the absolute orientation at one point in time and then use that going forward, but is there an easier way?
I think the Z axis of TANGO_COORDINATE_FRAME_CAMERA_DEPTH frame is always aligned with gravity.
Related
I am trying to create a 3D Visualization of an RC airplane in Threebox. The RC plane sends live telemetry, including:
GPS Coordinates
Gyro sensor data, showing the pitch, roll and heading of the plane.
I have now loaded a Model of an airplane in Threebox, no problems with that.
My problem comes down to the rotation of the plane. I want the plane object to represent the current orientation of the RC plane. Since I have the live telemetry from the flight controller, this should be possible.
In the Documentation, I have found this method, which seemed like exactly what i needed:
plane.setRotation({x: roll, y: pitch, z: yaw/heading})
And it basically works. I can rotate the Plane around its axes. But things get messed up when I combine the rotations.
For example: When I just update the roll axis, the Object behaves just like I want it to. However, when i change the heading of the plane by 90 degrees, the roll axis suddenly becomes the pitch axis. It seems to me, that the axes of the Plane object don't rotate with the plane itself.
I've prepared a recreation of the issue on jsfiddle. You can change the heading of the plane using the slider in the bottom right.
I've been stuck on this for days, would be super happy for any help!
There are lots of issues with your jsfiddle that prevent it from running. To isolate an issue and make it easier to test you should eliminate as many variables as possible - you're using two third-party libraries that will play a big hand in how transformations behave, particularly threebox.
I would recommend sticking with three.js's built in transformation tools unless you specifically need some lat/lng transformations, or other transformations to move between a local cartesian space and a global coordinate system. In this case, a very basic plane.setRotationFromEuler(new THREE.Euler(yaw, pitch, roll)) should do the trick. Be aware of how much order in euler rotations can affect the outcome, and that three.js uses radians for all its rotations, not degrees.
for a personal project, I've created a simple 3D engine in python using as little libraries as possible. I did what I wanted - I am able to render simple polygons, and have a movable camera. However, there is a problem:
I implemented a simple flat shader, but in order for it to work, I need to know the camera location (the camera is my light source). However, the problem is that I have no way of knowing the camera's location in the world space. At any point, I am able to display my view matrix, but I am unsure about how to extract the camera's location from it, especially after I rotate the camera. Here is a screenshot of my engine with the view matrix. The camera has not been rotated yet and it is very simple to extract its location (0, 1, 4).
However, upon moving the camera to a point between the X and Z axes and pointing it upwards (and staying at the same height), the view matrix changes to this:
It is obvious now that the last column cannot be taken directly to determine the camera location (it should be something like (4,1,4) on the last picture).
I have tried a lot of math, but I can't figure out the way to determine the camera x,y,z location from the view matrix. I will appreciate any and all help in solving this, as it seems to be a simple problem, yet whose solution eludes me. Thank you.
EDIT:
I was advised to transform a vertex (0,0,0,1) by my view matrix. This, however, does not work. See the example (the vertex obviously is not located at the printed coordinates):
Just take the transform of the vector (0,0,0,1) with the modelview matrix: Which is simply the rightmost column of the modelview matrix.
EDIT: #ampersander: I wonder why you're trying to work with the camera location in the first place, if you assume the source of illumination to be located at the camera's position. In that case, just be aware, that in OpenGL there is no such thing as a camera, and in fact, what the "view" transform does, is move everything in the world around so that where you assume your camera to be ends up at the coordinate origin (0,0,0).
Or in other words: After the modelview transform, the transformed vertex position is in fact the vector from the camera to the vertex, in view space. Which means that for your assumed illumination calculation the direction toward the light source, is the negative vertex position. Take that, normalize it to unit length and stick it into the illumination term.
I'm working on my raytracer and it seems I can't manage to handle the case where the direction vector of my camera is parallel to the vector (0,1,0).
I think it is linked to my way to compute the vector up and right for camera but I can't manage to find a work around.
Here is how I do it:
cam_up = vector_cross(cam_dir, {0, 1, 0});
camp_right = vector_cross(cam_right, cam_dir);
Can somebody enlighten me?
You have the correct formula for calculation of an orthogonal axis from a single cameraOut vector. However, as has been stated this formula will not account for the camera roll, which could be any direction in the plane perpendicular to the camera direction. This will be apparent when moving a camera across the pole (y-axis) as there will be undesireable behavior (yes it will be correctly aimed, but no doubt the roll won't be desired).
For more information, look into gimbal lock.
The roll itself is not really incorrect, however in reality for this camera transition to be smooth and appear correct (rather than suddenly flip or spin as it's direction becomes 0,1,0), you need to correct any roll incurred. This is a rotation about the cameraOut axis and ideally should be relative to the previous cameraAlong. This means in order to maintain the correct roll (or perceived correct roll) you need to consider the camera POSE (position and orientation) from the previous frame and ensure the roll is mitigated. Of course, if the camera doesn't move (i.e. your rendering a frame with a static camera position) you do not have a previous camera state so the position cannot be calculated and instead must be explicitly defined as part of the scene definition.
Personally I store an entire orthogonal axis for a camera so the orientation and roll is always clearly defined. This is only for completeness, to be honest you don't need to store the entire axis, 2 vectors cameraOut and cameraAlong (the third one being cameraUp) are enough. cameraAlong is dependant on the handed-ness of your coordinate system (e.g. for initial camera position say position (0,0,0) in left hand coordinate system, the cameraAlong direction will be in the right direction in relation to the viewer, for right hand system the cameraAlong would be the other way around. The cameraUp and cameraOut would are the same in both coordinate systems).
Hope this helps.
P.S This isn't ray tracing specific and the same principles apply for OpenGL/DirectX or any 3D representation.
my use case is only concerned with locationing, in fact only 2-d locationing. so a lot of the cool capabilities in tango are probably not useful to me. so I'm trying to see if i could implement the location algorithm myself.
from teardown reports it seems the 9dof sensors are pretty commodity hardware. the basic integration-based location algorithm (even with magnetic field calibration) has been mature knowledge. what algorithm does tango use?
from the description it seems that tango tries to aid in navigation by using the images it sees as a reference, sort of like the "terrain-following" mode in cruise missiles, is this right? this would be too ccomplex for me to implemente
You may easily get 2D position using the TangoPoseData with the correct coordinate system:
Project Tango uses a right-handed, local-level frame for the START_OF_SERVICE and AREA_DESCRIPTION coordinate frames. This convention sets the Z-axis aligned with gravity, with Z+ pointed upwards, and the X-Y plane is perpendicular to gravity and locally level with the ground plane. This local-level convention is based on the local east-north-up (ENU) earth-based coordinate system. Instead of true north, Project Tango uses the direction the back of the device is pointed when the service started as the Y axis, and the X axis is pointed to the right. The START_OF_SERVICE and AREA_DESCRIPTION base coordinate frames of the API will use this local-level frame convention.
Said more simply, use the pose data y/x coordinates for your space as you would latitude/longitude for the earth.
Heading data is also derived from the TangoPoseData and can be converted from quaternion to euler angles. Euler angles may be easier for you to use in your 2D location app.
Tango uses 3D to increase the confidence of its position within the space...even if you don't need 3D. I would let Tango do the hard stuff and extract the 2D position so you can focus on your app.
Tango uses the camera images to detect any change in position. And uses the IMU for device rotation and acceleration. Try blocking the camera and using the Motion Tracking app, it will fail.
A quick introduction:
We're developing a positioning system that works the following way. Our camera is situated on a robot and is pointed upwards (looking at the ceiling). On the ceiling we have something like landmarks, thanks to whom we can compute the position of the robot. It looks like this:
Our problem:
The camera is tilted a bit (0-4 degrees I think), because the surface of the robot is not perfectly even. That means, when the robot turns around but stays at the same coordinates, the camera looks at a different position on the ceiling and therefore our positioning program yields a different position of the robot, even though it only turned around and wasn't moved a bit.
Our current (hardcoded) solution:
We've taken some test photos from the camera, turning it around the lens axis. From the pictures we've deduced that it's tilted ca. 4 degrees in the "up direction" of the picture. Using some simple geometrical transformations we've managed to reduce the tilt effect and find the real camera position. On the following pictures the grey dot marks the center of the picture, the black dot is the real place on the ceiling under which the camera is situated. The black dot was transformed from the grey dot (its position was computed correcting the grey dot position). As you can easily notice, the grey dots form a circle on the ceiling and the black dot is the center of this circle.
The problem with our solution:
Our approach is completely unportable. If we moved the camera to a new robot, the angle and direction of tilt would have to be completely recalibrated. Therefore we wanted to leave the calibration phase to the user, that would demand takings some pictures, assessing the tilt parameters by him and then setting them in the program. My question to you is: can you think of any better (more automatic) solution to computing the tilt parameters or correcting the tilt on the pictures?
Nice work. To have an automatic calibration is a nice challenge.
An idea would be to use the parallel lines from the roof tiles:
If the camera is perfectly level, then all lines will be parallel in the picture too.
If the camera is tilted, then all lines will be secant (they intersect in the vanishing point).
Now, this is probably very hard to implement. With the camera you're using, distortion needs to be corrected first so that lines are indeed straight.
Your practical approach is probably simpler and more robust. As you describe it, it seems it can be automated to become user friendly. Make the robot turn on itself and identify pragmatically which point remains at the same place in the picture.