three.js / web-vr-boilerplate / polyfill - HMD to controlled object:Axis re-mapping does not re-map also rotation order/rules - three.js

I'm using your webvr-boilerplate and trying to map it to a human face mesh.
The way I do is is:
1) attach the camera to an eye bone
main js script:
//add camera to eye
mesh.skeleton.bones[ 22 ].add(camera);
//resets camera rotation
camera.rotation.set(0,0,0);
//looks at mesh up direction to face front
camera.lookAt( mesh.up );
//moves camera to middle of eyes
camera.position.set(10,10,0);
2) change the webvr-manager.js to update the neck bone ( passed as argument on initialization ) position and rotation and in index.php I swap the axis to match the HMD ones with the ones of the bone:
webvr-manager.js:
if ( state.orientation !== null ) {
object.quaternion.copy( state.orientation );
}
if ( state.position !== null ) {
object.position.copy( state.position ).multiplyScalar( scope.scale );
}
main js script:
/* INSIDE UPDATE CYCLE */
// mesh.rotation.y+=0.1;
controls.update();
//resets bone position to default
mesh.skeleton.bones[ neckVRControlBone ].position.set(neckInitPosition.x,neckInitPosition.y,neckInitPosition.z) ;
//ROTATION SWAP
mesh.skeleton.bones[ neckVRControlBone ].rotation.x = pivot.rotation.y;
mesh.skeleton.bones[ neckVRControlBone ].rotation.y = - pivot.rotation.z;
mesh.skeleton.bones[ neckVRControlBone ].rotation.z = - tempRotation;
UPDATE 28/10/2015:
to simplify and after some extra debug realised is not a clamp problem..
The restated problem is:
To map the VR controls to an object that has a different axis configuration of the HMD/Cardboard and keep the correct rotation rules.
Example of object axis:
* x - up
* y - depth
* z - side
Swapping the rotations by just
object .rotation.x = object .rotation.z results that, after updating the controls, rotating to the side makes an undesired rotation after 45º.
The rotation rules for each axis are different :
x rotates until PI and after that inverts signal and keeps changing in the same direction it was;
y rotates until PI/2 and after inverts the direction (when increasing, starts decreasing)
z is equal to x.
Changed webvr-polyfill.js and got it fixed for keyboard/mouse with this:
MouseKeyboardPositionSensorVRDevice.prototype.getState = function() {
// this.euler.set(this.phi, this.theta, 0, 'YXZ');
this.euler.set( this.theta , 0, - this.phi, 'YXZ');
But no way similar line to other controllers (HMD, cardboard, etc.).
Maybe it would be nice the rotation order and mapping could be available to the user.
Thanks
Example - try an set swappedAxis = true in the js console and rotate the neck.

The main problem you are running into is gimbal lock because you are using Euler rotations. Use Quaternions to avoid this problem.
Additionally, the axes on your mesh appear to be flipped, so you have to account for that.
Instead of setting components of the rotation, just set the quaternion:
mesh.skeleton.bones[neckVRControlBone].quaternion.set(
pivot.quaternion.y,
-pivot.quaternion.z,
-pivot.quaternion.x,
pivot.quaternion.w
);

Related

threejs rotate the object gradually to where camera is looking using orbit control

I'm planning to use Orbit Control to do a simple 3rd person camera view,
But I cant seem to figure out how to do it.
when I rotate the camera around an object, and press say “W” key to move forward, I want the object “look” to gradually rotate and move to the new direction the camera is facing.
How can I do that?
It's possible to do exactly that by gradually rotating the object to the camera direction.
Made a codepen here which uses a generic replacement to orbit controls for simplicity:
https://codepen.io/cdeep/pen/QWMWyYW
// Get the X-Z plane in which camera is looking to move the player
camera.getWorldDirection(tempCameraVector);
const cameraDirection = tempCameraVector.setY(0).normalize();
// Get the X-Z plane in which player is looking to compare with camera
model.getWorldDirection(tempModelVector);
const playerDirection = tempModelVector.setY(0).normalize();
// Get the angle to x-axis. z component is used to compare if the angle is clockwise or anticlockwise since angleTo returns a positive value
const cameraAngle = cameraDirection.angleTo(xAxis) * (cameraDirection.z > 0 ? 1 : -1);
const playerAngle = playerDirection.angleTo(xAxis) * (playerDirection.z > 0 ? 1 : -1);
// Get the angle to rotate the player to face the camera. Clockwise positive
const angleToRotate = playerAngle - cameraAngle;
// Get the shortest angle from clockwise angle to ensure the player always rotates the shortest angle
let sanitisedAngle = angleToRotate;
if(angleToRotate > Math.PI) {
sanitisedAngle = angleToRotate - 2 * Math.PI
}
if(angleToRotate < -Math.PI) {
sanitisedAngle = angleToRotate + 2 * Math.PI
}
// Rotate the model by a tiny value towards the camera direction
model.rotateY(
Math.max(-0.05, Math.min(sanitisedAngle, 0.05))
);

How to apply two-finger touch anywhere in the image, such that the point stays at the same location

I have a 2D texture image that is zoomed in/out via 2-finger touch and pinch.
Currently the image is not panned, i.e. the center of the image is always in the middle.
I want the center point between the twoFinger touch to stay between the 2 fingers.
If I pinch exactly in the center of the image, then the center image point will stay between the fingers - good!
But if I pinch near the corner of the image the point will move away, relative to the 2 fingers, because of the zoom.
So I need to apply some pan in addition to the zoom, to make the point appear in the same place.
I basically need to transftorm the camera position such that, for every zoom, the same world coordinate is projected to the same screen coord.
Figures 1-3 illustrate the problem.
Figure1 is the original image.
Currently when I zoom in the camera stays in the same position, so the object between the 2 fingers (the cat's eye on the right) is drifted from being between the 2 fingers, as the image zooms (Figure 2).
I want to pan the camera such that the object between the 2 fingers stays between the 2 fingers even after the zooming the image (Figure 3).
I used the code below, but the object still drifts as the image zooms in/out.
How should I calculate the amount of shift that needs to be applied to the camera?
Thanks
Code to calculate the amount of shift that needs to be applied to the camera
handleTwoFingerTouchMove( p3_inScreenCoord) {
// normalize the screen coord to be in the range of [-1, 1]
// (See method1 in https://stackoverflow.com/questions/13542175/three-js-ray-intersect-fails-by-adding-div/)
let point2dNormalizedX = ( ( p3_inScreenCoord.x - windowOffset.left ) / windowWidth) * 2 - 1;
let point2dNormalizedY = -( ( p3_inScreenCoord.y - windowOffset.top ) / windowHeight) * 2 + 1;
// calc p3 before zoom (in world coords)
let p3_beforeZoom = new THREE_Vector3( point2dNormalizedX, point2dNormalizedY, -1 ).unproject( this.camera );
// Apply zoom
this.dollyInOut( this.getZoomScale(), true );
// calc p3 after zoom (in world coords)
let p3_afterZoom = new THREE_Vector3( point2dNormalizedX, point2dNormalizedY, -1 ).unproject( this.camera );
// calc the required shift in camera position
let deltaX = p3_afterZoom.x - p3_beforeZoom.x;
let deltaZ = p3_afterZoom.z - p3_beforeZoom.z;
// shift in camera position
this.pan( deltaX, deltaZ );
};
I was able to solve my problem. Here is my solution in the hope that it helps others.
When first applying a 2-finger touch (i.e. on touchstart event), the code computes:
the world-coordinate of the object pointed at (e.g. the cat's eye on the right), when starting two-finger touch
centerPoint3d_inWorldCoord0 (Vector3)
the screen-coordinate anchor for zooming via two-finger touch
centerPoint2dBetweenTwoFingerTouch_inScreenCoordNormalized (Vector2)
While zooming in/out via two-finger pinch in/out (on touchmove event)
in the event listener function, immediately after the applying the zoom,
I call the following code:
// Calculate centerPoint3d_inWorldCoord2, which is the new world-coordinate, for
// centerPoint2dBetweenTwoFingerTouch_inScreenCoordNormalized given the new zoom setting.
let centerPoint3d_inWorldCoord2 = new THREE_Vector3( centerPoint2dBetweenTwoFingerTouch_inScreenCoordNormalized.x,
centerPoint2dBetweenTwoFingerTouch_inScreenCoordNormalized.y,
-1 ).unproject( camera );
// compute the shift in world-coordinate between the new vs the original world-coordinate
let delta_inWorldCoords = new THREE_Vector2(centerPoint3d_inWorldCoord2.x - centerPoint3d_inWorldCoord0.x,
centerPoint3d_inWorldCoord2.z - centerPoint3d_inWorldCoord0.z);
// pan the camera, to compensate the shift
pan_usingWorldCoords( delta_inWorldCoords );
The function pan_usingWorldCoords shifts the camera in the x axis (panLeft), and then in the y axis (panUp)
pan_usingWorldCoords( delta_inWorldCoord ) {
panLeft( delta_inWorldCoord.x );
panUp( delta_inWorldCoord.y );
};
The functions panLeft, panUp are similar to the functions that are used in three.js-r114/examples/jsm/controls/OrbitControls.js
Initially the object pointed at was drifting from being between the 2 fingers, as the image zoomed in/out.
I added this.camera.updateProjectionMatrix() at the end of each function.
This updates the projection matrix at the end of panLeft, before using it again in panUp.
With the code above and after updating the projection matrix at the end of panLeft, panUp, the object pointed at (e.g. the eye on the right), when starting two-finger touch, is kept between the 2 fingers, while zooming via two-finger pinch.

In A-Frame/THREE.js, is there a method like the Camera.ScreenToWorldPoint() from Unity?

I know a method from Unity whichs is very useful to convert a screen position to a world position : https://docs.unity3d.com/ScriptReference/Camera.ScreenToWorldPoint.html
I've been looking for something similar in A-Frame/THREE.js, but I didn't find anything.
Is there an easy way to convert a screen position to a world position in a plane which is positioned a given distance from the camera ?
This is typically done using Raycaster. An equivalent function using three.js would be written like this:
function screenToWorldPoint(screenSpaceCoord, target = new THREE.Vector3()) {
// convert the screen-space coordinates to normalized device coordinates
// (x and y ranging from -1 to 1):
const ndc = new THREE.Vector2()
ndc.x = 2 * screenSpaceCoord.x / screenWidth - 1;
ndc.y = 2 * screenSpaceCoord.y / screenHeight - 1;
// `Raycaster` can be used to convert this into a ray:
const raycaster = new THREE.Raycaster();
raycaster.setFromCamera(ndc, camera);
// finally, apply the distance:
return raycaster.ray.at(screenSpaceCoord.z, target);
}
Note that coordinates in browsers are usually measured from the top/left corner with y pointing downwards. In that case, the NDC calculation should be:
ndc.y = 1 - 2 * screenSpaceCoord.y / screenHeight;
Another note: instead of using a set distance in screenSpaceCoord.z you could also let three.js compute an intersection with any Object in your scene. For that you can use raycaster.intersectObject() and get a precise depth for the point of intersection with that object. See the documentation and various examples linked here: https://threejs.org/docs/#api/core/Raycaster

How to setup a camera that follows a circle path?

I'm trying to create a camera that follows an object that rotates on a orbit around a sphere. But everytime the camera reaches the polar coordinates of the orbit, the direction changes. I just set the position of the camera according to the object that is has to follow and calling lookAt afterwards:
function render() {
rotation += 0.002;
// set the marker position
pt = path.getPoint( t );
// set the marker position
marker.position.set( pt.x, pt.y, pt.z );
marker.lookAt( new THREE.Vector3(0,0,0) );
// rotate the mesh that illustrates the orbit
mesh.rotation.y = rotation
// set the camera position
var cameraPt = cameraPath.getPoint( t );
camera.position.set( cameraPt.x, cameraPt.y, cameraPt.z );
camera.lookAt( marker.position );
t = (t >= 1) ? 0 : t += 0.002;
renderer.render( scene, camera );
}
Here's a complete fiddle: http://jsfiddle.net/krw8nwLn/69/
I've created another fiddle with a second cube which represents the desired camera behaviour: http://jsfiddle.net/krw8nwLn/70/
What happens is that the camera's lookAt function will always try to align the camera with the horizontal plane (so that the "up" direction is always (0, 1, 0). And when you reach the top and bottom of the ellipse path, the camera will instantaneously rotate 180° so that up is still up. You can also see this in your "desired behaviour" example, as the camera cube rotates so that the colors on the other side are shown.
A solution is to not use lookAt for this case, because it does not support cameras doing flips like this. Instead set the camera's rotation vector directly. (Which requires some math, but you look like a math guy.)

Rotate camera X on local axis using Three.js

I'm new to Three.js and fairly new to 3d space engines and what I'm trying to achieve is a 360 equirectangular image viewer.
What my script does so far is to create a camera at 0,0,0 and a sphere mesh at the same location with normals inverted and an emission map of my 360 image.
Representation of scene using Blender's viewport.
The user should be enabled to rotate the camera using mouse drag or keyboard arrows, so using mouse listeners I created the drag feature which calculates the amount of rotation in the camera's Y axis (blue) and X axis (red) at each render frame. I also created min and max rotation limit on X (so the user couldn't spin backward), as follows:
var render = function () {
requestAnimationFrame( render );
if((camera.rotation.x < Math.PI/6 && speedX >= 0) || (camera.rotation.x > -Math.PI/6 && speedX <= 0))
camera.rotation.x += speedX * (Math.PI/180);
camera.rotation.y += speedY * (Math.PI/180);
renderer.render(scene, camera);
};
Where speedX and speedY represent the amount of rotation in each axis.
So far so good, but since those rotation coordinates are relative to the world and not the camera itself the X rotation makes the camera go wild, since after a couple of rotated degrees in the Y axis, the camera's X axis is no longer the same as the world's X axis.
My question, finally, is: how do I rotate the camera on it's own X axis at each frame?
If you want a camera's rotation to have meaning in terms of yaw (heading), pitch, and roll, you need set:
camera.rotation.order = 'YXZ'; // default is 'XYZ'
For more information, see this stackoverflow answer.
three.js r.82

Resources