threejs - raycasting in AR with controller after repositioning - three.js

I'm rather new to threejs, so what I'm doing might not be the most efficient way.
I have an object in AR on a mobile device and I want to know if I intersect with it when touching on the screen.
I use the following code to generate the raycast, and it works initally.
const tempMatrix = new THREE.Matrix4();
tempMatrix.identity().extractRotation(this.controller.matrixWorld);
this.raycaster.ray.origin.setFromMatrixPosition(this.controller.matrixWorld);
this.raycaster.ray.direction.set(0, 0, -1).applyMatrix4(tempMatrix);
However, I have the ability to reposition the object (i.e. reset the position so the object is in front, relative to the current camera direction and position) by moving and rotating the whole scene.
After the repositioning, the raycasting is completely offset and is not casting rays anywhere near where I touch the screen.
Repositioning is done like this (while it works, if there's a better way, let me know!) :
public handleReposition(): void {
const xRotation = Math.abs(this.camera.rotation.x) > Math.PI / 2 ? -Math.PI : 0;
const yRotation = this.camera.rotation.y;
this.scene.rotation.set(xRotation, yRotation, xRotation);
this.scene.position.set(this.camera.position.x, this.camera.position.y, this.camera.position.z);
}
How can I achieve to raycast to the correct new location?
Thanks!

Assuming this.scene is actually the main threejs Scene, it's usually a bad idea to change its rotation or position, since it will affect everything inside the scene, including the controller. I'd suggest moving your object instead, or add your object(s) to a Group and move that.

Related

Make children of imported Three.js group follow pointer when their canvas gets hovered

In Three.js, I have a group of meshes that is loaded from outside with help of FBX loader. The group has six meshes inside. My task is to make this meshes follow pointer when they get hovered. More precisely, I'd like to have a sort of magnetic effect (just like navbar items in this pen, but with meshes in Three.js).
I think, firstly, I have to detect, where currently pointer is, i.e. get position of cursor in world coordinates system, and then translate meshes towards it. But when I try to get the position of cursor, it seems to be wrong.
Having said that, I have two questions:
How to get proper cursor's position relative to the world coordinates?
How to change position of each of the group's meshes so that they get translated against the cursor?
Here is what have I done so far:
Hi everyone.
In Three.js, I have a group of meshes that is loaded from outside with help of FBX loader. The group has six meshes inside. My task is to make this meshes follow pointer when their canvas get hovered. More precisely, I'd like to have a sort of magnetic effect (just like navbar items in this pen, but with meshes of Three.js).
I think, firstly, I have to detect, where currently pointer on canvas is, i.e. get position of cursor in world coordinates system, and then translate meshes towards it. But when I try to get the position of cursor, it seems to be wrong.
Having said that, I have two questions:
How to get proper cursor's position relative to the world coordinates?
How to change position of each of the group's meshes so that they get translated against the cursor?
Here is what have I done so far. Function that translates meshes isn't written yet. Mousemove callback returns pretty big digits, though:
// Load object and play a third-party animation
loader.load("Object_001.fbx", (object) => {
mixer = new THREE.AnimationMixer(object);
const action = mixer.clipAction(object.animations[0]);
action.play();
object.traverse((child) => {
if (child.isMesh) {
child.material.map = texture;
child.material.needsUpdate = true;
}
});
scene.add(object);
});
// log coordinates of the pointer
const mouse = new THREE.Vector3();
const position = new THREE.Vector3();
function onMouseMove(event) {
mouse.set(
(event.clientX / window.innerWidth) * 2 - 1,
-(event.clientY / window.innerHeight) * 2 + 1,
0.5
);
mouse.unproject(camera);
mouse.sub(camera.position).normalize();
const distance = -camera.position.z / mouse.z;
position.copy(camera.position).add(mouse.multiplyScalar(distance));
console.log(position);
}
wrapperElement.addEventListener("mousemove", onMouseMove);
Thanks in advance.
Made a codepen here:
https://codepen.io/cdeep/pen/YzxPPZQ
The cursor only exists in the canvas dom element which is a rendering of the camera view frustum.
The easiest way to make the object follow a mouse is to get the point of intersection of the mouse with another object in the 3d scene and set the object position to the intersection point. The above codepen showcases that.
raycaster.setFromCamera( mouse, camera );
const intersects = raycaster.intersectObjects([ground]);
if(intersects.length) {
const { point } = intersects[0];
cube.position.copy(point.setY(0.5));
}
You could also position it at a fixed distance from the mouse but it looks odd in my opinion:
const distance = 10;
raycaster.setFromCamera( mouse, camera );
const { origin, direction } = raycaster.ray;
cube.position.copy(origin.clone().add(direction.multiplyScalar(distance)));
Documentation for raycaster:
https://threejs.org/docs/index.html?q=ray#api/en/core/Raycaster
Raycasting is used for mouse picking (working out what objects in the
3d space the mouse is over) amongst other things.

Raycasting in threeJs, when target is over object

I'm fairly new to three.js and trying to get a better understanding of ray casting. I have used it so far in a game to show when an object collides with another on the page which works perfectly. In the game I'm building this is to take health from the hero as it crashes into walls.
I am now trying to implement a target which when hovered over some objects it will auto shoot. However the target only registers a target hit once the object passes through the target mesh rather than when its ray is cast through it.
to further detail this, the ray is cast from the camera through the target, if the target is on a mesh (stored as an object array) then I want it to trigger a function.
within my update function I have this:
var ray = new THREE.Raycaster();
var crossHairClone = crossHair.position.clone();
var coards = {};
coards.x = crossHairClone.x
coards.y = crossHairClone.y
ray.setFromCamera(coards, camera);
var collisionResults = ray.intersectObjects( collidableMeshList );
if ( collisionResults.length > 0 ) {
console.log('Target Hit!', collisionResults)
}
The console log is only triggered when the collidableMeshList actually touches the target mesh rather than when it is aiming at it.
How do I extend the ray to pass through the target (which I thought it was already doing) so that if anything hits the ray then it triggers my console log.
Edit
I've added a URL to the game in progress. There are other wider issues with the game, my current focus is just the target ray casting.
Game Link
Many thanks to everyone who helped me along the way on this one but I finally solved it, only not through the means that I had expected to. Upon reading into the setFromCamera() function it looks like it is mainly used around mouse coards which in my instance was not what I wanted. I wanted a target on the page and the ray to pass through this. For some reason my Ray was shooting vertically up from the centre of the whole scene.
In the end I tried a different approach and set the Rays position and direction directly rather rely on setting it from the cameras position.
var vector = new THREE.Vector3(crossHair.position.x, crossHair.position.y, crossHair.position.z);
var targetRay = new THREE.Raycaster(camera.position, vector.sub(camera.position).normalize());
var enemyHit = targetRay.intersectObject( enemy );
if ( enemyHit.length > 0 ) {
console.log('Target Hit!', targetRay)
}
I'll leave this up for a while before accepting this answer in case anyone has a better approach than this or has any corrections over what I have said in regards to setFromCamera().

Change facing direction of CSS3DObject

I have a 3D scene with a bunch of CSS object that I want to rotate so that they are all pointing towards a point in the space.
My CSS objects are simple rectangles that are a lot wider than they are high:
var element = document.createElement('div');
element.innerHTML = "test";
element.style.width = "75px";
element.style.height = "10px";
var object = new THREE.CSS3DObject(element);
object.position.x = x;
object.position.y = y;
object.position.z = z;
Per default, the created objects are defined as if they are "facing" the z-axis. This means that if I use the lookAt() function, the objects will rotate so that the "test" text face the point.
My problem is that I would rather rotate so that the "right edge" of the div is pointing towards the desired point. I've tried fiddling with the up-vector, but I feel like that wont work because I still want the up-vector to point up. I also tried rotating the object Math.PI/2 along the y axis first, but lookAt() seems to ignore any prior set rotation.
It seems like I need to redefine the objects local z-vector instead, so that it runs along with the global x-vector. That way the objects "looking at"-direction would be to the right in the scene, and then lookAt() would orient it properly.
Sorry for probably mangling terminology, newbie 3D programmer here.
Object.lookAt( point ) will orient the object so that the object's internal positive z-axis points in the direction of the desired point.
If you want the object's internal positive x-axis to point in the direction of the desired point, you can use this pattern:
object.lookAt( point );
object.rotateY( - Math.PI / 2 );
three.js r.84

threejs raycaster cannot intersect in stereoscopic mode

I am trying to make use of Raycaster in a ThreeJS scene to create a sort of VR interaction.
Everything works fine in normal mode, but not when I enable stereo effect.
I am using the following snippet of code.
// "camera" is a ThreeJS camera, "objectContainer" contains objects (Object3D) that I want to interact with
var raycaster = new THREE.Raycaster(),
origin = new THREE.Vector2();
origin.x = 0; origin.y = 0;
raycaster.setFromCamera(origin, camera);
var intersects = raycaster.intersectObjects(objectContainer.children, true);
if (intersects.length > 0 && intersects[0].object.visible === true) {
// trigger some function myFunc()
}
So basically when I try the above snippet of code in normal mode, myFunc gets triggered whenever I am looking at any of the concerned 3d objects.
However as soon as I switch to stereo mode, it stops working; i.e., myFunc never gets triggered.
I tried updating the value of origin.x to -0.5. I did that because in VR mode, the screen gets split into two halves. However that didn't work either.
What should I do to make the raycaster intersect the 3D objects in VR mode (when stereo effect is turned on)?
Could you please provide a jsfiddle with the code?
Basically, if you are using stereo in your app, it means you are using 2 cameras, therefore you need to check your intersects on both cameras views, this could become an expensive process.
var cameras =
{ 'camera1': new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 1, 10000),
'camera2': new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 1, 10000)
};
for (var cam in cameras) {
raycaster.setFromCamera(origin, cameras[cam]);
//continue your logic
}
You could use a vector object that simulates the camera intersection to avoid checking twice, but this depends on what you are trying to achieve.
I encountered a similar problem, I eventually found the reason. Actually in StereoEffect THREE.js displays the meshes on the two eyes, but in reality is actually adds only one mesh to the scene, exactly in the middle of the line left-eye-mesh <-> right-eye-mesh, hidden to the viewer.
So when you use the raycaster, you need to use it on the real mesh on the middle, not the illusion displayed on each eye !
I detailled here how to do it
Three.js StereoEffect displays meshes across 2 eyes
Hopes it solves your problem !
You can use my StereoEffect.js file in your project for resolving problem. See example of using. See my Raycaster stereo pull request also.

Three.js - move custom geometry to origin

I have some custom geometries obtained from a STEP file conversion and I use the mouse to rotate them. They rotate around the origin of the scene, but since they are far from it, they seem rotating on a virtual sphere. How can I move them to the origin so that they don't seem "floating" around (I mean that I'd like to reduce to zero the radius of the virtual sphere). This is the example I'd like to move. I've tried setting their position to (0, 0, 0) doing:
object.position.x = 0;
object.position.y = 0;
object.position.z = 0;
but it didin't work.
The typical solution to this problem is to translate the geometry right after it is created. You do that by applying a translation matrix to the geometry like so:
geometry.applyMatrix( new THREE.Matrix4().makeTranslation( distX, distY, distZ ) );
EDIT: You can simply do this, instead:
geometry.translate( distX, distY, distZ ); // three.js r.72
The function geometry.computeBoundingBox() may be of help to you in determining an amount to translate.
However, I see in your case, you have multiple geometries, so it it a bit more complicated, but doable. You will need to translate each geometry by the same amount.
EDIT
Tip: Instead of adding each object to the scene, create a parent object, add it to the scene, and then add the objects to the parent.
var parent;
parent = new THREE.Object3D();
scene.add( parent );
parent.add( object1 );
parent.add( object2 );
// and so on...
Then in your render function, just rotate the parent, not the individual objects.

Resources