Three.js raycaster intersection with sprites is completely off to the left - three.js

I have sprites with text on screen, placed in spherical pattern and I want to allow user to click on individual words and highlight them.
Now the problem is that when I do raycasting and do raycaster.intersectObjects() it returns sprites that are somewhere completely different where the click happened (usually it highlights words that are to the left of the words clicked). For debugging purposes I actually drew the rays from the raycaster object, and they are pretty much going trough the words I clicked.
In this picture I clicked the words "emma", "universe" and "inspector legrasse", but the words that got highlighted are in the red, I also rotated the camera so we can see the lines.
here is the relevant code:
Sprite creation:
var canvas = document.createElement('canvas');
var context = canvas.getContext('2d');
canvas.height = 256;
canvas.width = 1024;
...
context.fillStyle = "rgba(0, 0, 0, 1.0)";
context.fillText(message, borderThickness, fontsize + borderThickness);
var texture = new THREE.Texture(canvas);
texture.needsUpdate = true;
var spriteMaterial = new THREE.SpriteMaterial({map: texture});
var sprite = new THREE.Sprite(spriteMaterial);
sprite.scale.set(400, 100, 1.0);
Individual sprites are then added to "sprites" array, and then added to the scene.
Click detection:
function clickOnSprite(event) {
mouse.x = ( event.clientX / renderer.domElement.clientWidth ) * 2 - 1;
mouse.y = -( event.clientY / renderer.domElement.clientHeight ) * 2 + 1;
raycaster.setFromCamera(mouse, camera);
drawRaycastLine(raycaster);
var intersects = raycaster.intersectObjects(sprites);
if (intersects.length > 0) {
intersects[0].object.material.color.set(0xff0000);
console.log(intersects[0]);
}
}
I am using perspective camera with orbit controls.

I came to the conclusion, that this cannot be currently done with sprites (at least not currently).
As suggested in the comments, I ended up using plane geometries instead of sprites.

Related

Painting the faces of a cube when user clicks on that face

I am trying to do what the title says.
When a user clicks on a face of the cube, that face will change colour.
This is my code snippet:
// create a cube
var cubeGeometry = new THREE.BoxGeometry(20, 20, 20);
var cubeMaterial = new THREE.MeshLambertMaterial({color: 0xffff00 }); //0xF7F7F7 = gray
cube = new THREE.Mesh(cubeGeometry, cubeMaterial);
cube.userData.originalColor = 0xffff00;
function onDocumentMouseClick(event) //if we detect a click event
{
// the following line would stop any other event handler from firing
// (such as the mouse's TrackballControls)
event.preventDefault();
// update the mouse variable
mouse.x = (event.clientX / window.innerWidth) * 2 - 1;
mouse.y = -(event.clientY / window.innerHeight) * 2 + 1;
var vector = new THREE.Vector3( ( event.clientX / window.innerWidth ) * 2 - 1, - ( event.clientY / window.innerHeight ) * 2 + 1, 0.5 );
vector.unproject( camera );
raycaster.set( camera.position, vector.sub( camera.position ).normalize() );
var intersects = raycaster.intersectObject( cube );
if ( intersects.length > 0 )
{
var index = Math.floor( intersects[0].faceIndex / 2 );
switch (index)
{
case 0:
case 1:
case 2:
case 3:
case 4:
case 5:
}
}
}
This code is incomplete.
My problems are these:
I don't know if this strategy is correct and works
I don't know what code to use inside the different cases, in order to paint that side of the cube.
First, consider breaking the cube's faces into draw groups. This answer delves into how to do that.
The idea is that all of the cube's faces will have the same color, but not the same material. This will allow you to change the material's colors individually, and as such you will individually change the color of the cube faces.
When you get a response back from the Raycaster, look for the faceIndex property. This will tell you the index of the face that was intersected. Find which draw group that index belongs to, and you can then reference that group's material index, and thus change the color of the material.

Three js RayCast between two object in scene

I know how raycast object in scene when click on mouse, but now i need to know if two object in scene can raycast each other.
This is, i load a 3D Object in scene for example Two Rooms in OBJ object, then i add three mesh box in some points, for example two point on first room and one point on second room.
Then two points on first room can raycast each other(have direct vision), but two point for first room can't raycast point on second room.(they don't have vision through room wall).
I attached code used for load scene and points, any sugestion hwo to do?
//LOAD MAIN 3D OBJECT
var objLoader = new THREE.OBJLoader();
objLoader.setMaterials(materials);
objLoader.setPath('./asset/3d/');
objLoader.load("model01.obj", function(object){
var mesh = object.children[0];
mesh.castShadow = true;
mesh.receiveShadow = true;
mesh.rotation.x = Math.PI / 2;
var box = new THREE.Box3().setFromObject( object )
var ox = -(box.max.x + box.min.x) / 2;
var oy = -(box.max.y + box.min.y) / 2;
var oz = -(box.max.z + box.min.z) / 2;
mesh.position.set(ox, oy, oz);
_scene.add(mesh);
render();
setTimeout(render, 1000);
}
//LOAD count_points inside scene
for(var i=0;i<cta_points;i++){
var c_r = 2;
var c_geometry = new THREE.BoxBufferGeometry( c_r, c_r, c_r );
var c_material = new THREE.MeshLambertMaterial( { color: new THREE.Color("rgb("+(40 + 30)+", 0, 0)"),opacity: 0.0,
transparent: true} );
var c_mesh = new THREE.Mesh( c_geometry, c_material );
var position = get_positions(i);
c_mesh.position.copy(position);
c_mesh.name="BOX";
scene.add( c_mesh );
}
Possibly take a look at:
How to detect collision in three.js?
Usually, to solve this problem, you would make a collision mask with a collision group.
The collision group is added per object, and is represented by a "bit" in a bitmask,
The wall could be in a separate collision group, like 4 (binary 100)
and the objects could be in another group, say 2 (binary 10)
Then you just need to check collisions of the object against the mask.
(check if the collision group matches against a bitmask (the masks above could be 10, 100,) to check for collisions).
So that way, you can call THREE.Raycaster().intersectObjects(args), where the arguments are the ones that pass the bitmask test ( mask == object.collision_group ).
That way, you won't need to include the wall for collision detection testing, since it is using a separate bitmask.

what's wrong with it - or how to find correct THREE.PerspectiveCamera settings

I have a simple THREE.Scene where the main content is a THREE.Line mesh that visualizes the keyframe based path that the camera will follow for some scripted animation. There is then one THREE.SphereGeometry based mesh that is always repositioned to the current camera location.
The currently WRONG result looks like this (the fractal background is rendered independently but using the same keyframe input - and ultimately the idea is that the "camera path" visualization ends up in the same scale/projection as the respective fractal background...):
The base is an array of keyframes, each of which represents the modelViewMatrix for a specific camera position/orientation and is directly used to drive the vertexshader for the background, e.g.:
varying vec3 eye, dir;
void main() {
gl_Position = vec4(position, 1.0);
eye = vec3(modelViewMatrix[3]);
dir = vec3(modelViewMatrix * vec4(position.x , position.y , 1, 0));
}
(it is my understanding that "eye" is basically the camera position while "dir" reflects the orientation of the camera and by the way it is used during the ray marching implicitly leads to a perspective projection)
The respective mesh objects are created like this:
visualizeCameraPath: function(scene) {
// debug: visualize the camera path
var n= this.getNumberOfKeyFrames();
var material = new THREE.LineBasicMaterial({
color: 0xffffff
});
var geometry = new THREE.Geometry();
for (var i= 0; i<n; i++) {
var m= this.getKeyFrameMatrix(true, i);
var pos= new THREE.Vector3();
var q= new THREE.Quaternion();
var scale= new THREE.Vector3();
m.decompose(pos,q,scale);
geometry.vertices.push( new THREE.Vector3( pos.x, pos.y, pos.z ));
}
this.camPath = new THREE.Line( geometry, material );
this.camPath.frustumCulled = false; // Avoid getting clipped - does not seem to help one little bit
scene.add( this.camPath );
var radius= 0.04;
var g = new THREE.SphereGeometry(radius, 10, 10, 0, Math.PI * 2, 0, Math.PI * 2);
this.marker = new THREE.Mesh(g, new THREE.MeshNormalMaterial());
scene.add(this.marker);
}
in order to play the animation I update the camera and the marker position like this (I guess it is already wrong how I use the input matrix "m" directly on the "shadowCamera" - eventhough I think that it contains the correct position):
syncShadowCamera(m) {
var pos= new THREE.Vector3();
var q= new THREE.Quaternion();
var scale= new THREE.Vector3();
m.decompose(pos,q,scale);
this.applyMatrix(m, this.shadowCamera); // also sets camera position to "pos"
// highlight current camera-position on the camera-path-line
if (this.marker != null) this.marker.position.set(pos.x, pos.y, pos.z);
},
applyMatrix: function(m, targetObj3d) {
var pos= new THREE.Vector3();
var q= new THREE.Quaternion();
var scale= new THREE.Vector3();
m.decompose(pos,q,scale);
targetObj3d.position.set(pos.x, pos.y, pos.z);
targetObj3d.quaternion.set(q.x, q.y, q.z, q.w);
targetObj3d.scale= scale;
targetObj3d.updateMatrix(); // this.matrix.compose( this.position, this.quaternion, this.scale );
targetObj3d.updateMatrixWorld(true);
},
I've tried multiple things with regard to the camera and the screenshot reflects the output with disabled "this.projectionMatrix" (see below code).
createShadowCamera: function() {
var speed = 0.00039507;
var z_near = Math.abs(speed);
var z_far = speed * 65535.0;
var fH = Math.tan( this.FOV_Y * Math.PI / 360.0 ) * z_near;
var fW = Math.tan( this.FOV_X * Math.PI / 360.0 ) * z_near;
// orig opengl used: glFrustum(-fW, fW, -fH, fH, z_near, z_far);
var camera= new THREE.PerspectiveCamera();
camera.updateProjectionMatrix = function() {
// this.projectionMatrix.makePerspective( -fW, fW, fH, -fH, z_near, z_far );
this.projectionMatrix= new THREE.Matrix4(); // hack: fallback to no projection
};
camera.updateProjectionMatrix();
return camera;
},
My initial attempt had been to use the same kind of settings that the opengl shader for the fractal background had been using (see glFrustum above). Unfortunately it seems that I have yet managed to correctly map the input "modelViewMatrix" (and the projection implicitly performed by the raymarching in the shader) to equivalent THREE.PerspectiveCamera settings (orientation/projectionMatrix).
Is there any matrix calculation expert here, that knows how to obtain the correct transformations?
Finally I have found one hack that works.
Actually the problem was made up of two parts:
1) Row- vs column-major order of modelViewMatrix: The order expected by the vertex shader is the oposite of what the remaining THREE.js expects..
2) Object3D-hierarchy: i.e. Scene, Mesh, Geometry, Line vertices + Camera: where to put the modelViewMatrix data so that it creates the desired result (i.e. the same result that the old bloody opengl application produced): I am not happy with the hack that I found here - but so far it is the only one that seems to work:
I DO NOT touch the Camera.. it stays at 0/0/0
I directly move all the vertices of my "line"-Geometry relative to the real camera position (see "position" from the modelViewMatrix)
I then disable "matrixAutoUpdate" on the Mesh that contains my "line" Geometry and copy the modelViewMatrix (in which I first zeroed out the "position") into the "matrix" field.
BINGO.. then it works. (All of my attempts to achieve the same result by rotating/displacing the Camera or by displacing/rotating any of the Object3Ds involved have failed miserably..)
EDIT: I found a better way than updating the vertices and at least keeping all the manipulations on the Mesh level (I am still moving the world around - like the old OpenGL app would have done..). To get the right sequence of translation/rotation one can also use ("m" is still the original OpenGL modelViewMatrix - with 0/0/0 position info):
var t= new THREE.Matrix4().makeTranslation(-cameraPos.x, -cameraPos.y, -cameraPos.z);
t.premultiply ( m );
obj3d.matrixAutoUpdate=false;
obj3d.matrix.copy(t);
If somebody knows a better way that also works (one where the Camera is updated without having to directly manipulate object matrices) I'd certainly be interested to hear it.

Three.js StereoEffect displays meshes across 2 eyes

I have a THREE.js scene using StereoEffect renderer. However, when I add new meshes to the scene, they are displayed across the two eyes instead of being duplicated for each eye. I believe THREE.js is supposed to do it automatically and I don't have to duplicate them myself ? (I tried duplicating them but it is a lot of annoying calculation and I could not manage it)
My meshes are actually transparent planes, and I add a DOM element on the top of them to have a flat display.
Illustrating Example
OK I finally found it ! I tried putting back on a texture (not invisible) on my meshes and I discovered the problem.
When we use StereoEffect and we see that our mesh is duplicated on both views, it is actually an illusion : THREE.JS puts an image there, but the actual object is invisible, put exactly at the middle of the two images !
See image here : explanation
If you use raycaster for instance, it will tell you there's not an instersection where you see the mesh, but at the center of the line from left image to right image ! Same for mesh.position.
So what I did was keep an invisible texture, and create two div tags that I placed symmetrically around the mesh position :
var middleX = window.offsetWidth/2;
//left div
if(this.element.id.indexOf("-2") == -1){
var posL = coords2d.x - middleX/2;
this.element.style.left = posL + 'px';
//Hide if gets on the right part of the screen
if(posL > middleX) this.element.style.display = 'none';
}
//right div
else{
var posR = coords2d.x + middleX/2;
this.element.style.left = posR + 'px';
//Hide if gets on the left part of the screen
if(posR < middleX) this.element.style.display = 'none';
}
That gives the illusion that my mesh is there, put it is just empty divs.
Then, to check if someone clicks on my mesh, I do the opposite : I go back to the real position of the mesh before sending it to raycaster !
function hasClicked(e) {
e.preventDefault();
var clientX,clientY;
//GET REAL POSITION OF OBJECT
var middleX = window.offsetWidth/2;
//Right screen
if(e.clientX>middleX){
clientX = e.clientX - middleX/2;
}
//Left screen
else {
clientX = e.clientX + middleX/2;
}
clientY = e.clientY; //Keep same Y coordinate
//TRANSFORM THESE COORDS ON SCREEN INTO THREE.JS COORDS
var mouse = new THREE.Vector2();
mouse.x = (clientX / window.innerWidth) * 2 - 1;
mouse.y = -(clientY / window.innerHeight) * 2 + 1;
//USE RAYCASTER
raycaster.setFromCamera(mouse, camera);
var intersects = raycaster.intersectObjects(arrowManager.storage);
if (intersects.length > 0) {
//It works !!!
}
}
And it works perfectly fine !! Hope this can help someone else, I was getting really desperate here ;)

Mouse / Canvas X, Y to Three.js World X, Y, Z

I've searched around for an example that matches my use case but cannot find one. I'm trying to convert screen mouse co-ordinates into 3D world co-ordinates taking into account the camera.
Solutions I've found all do ray intersection to achieve object picking.
What I am trying to do is position the center of a Three.js object at the co-ordinates that the mouse is currently "over".
My camera is at x:0, y:0, z:500 (although it will move during the simulation) and all my objects are at z = 0 with varying x and y values so I need to know the world X, Y based on assuming a z = 0 for the object that will follow the mouse position.
This question looks like a similar issue but doesn't have a solution: Getting coordinates of the mouse in relation to 3D space in THREE.js
Given the mouse position on screen with a range of "top-left = 0, 0 | bottom-right = window.innerWidth, window.innerHeight", can anyone provide a solution to move a Three.js object to the mouse co-ordinates along z = 0?
You do not need to have any objects in your scene to do this.
You already know the camera position.
Using vector.unproject( camera ) you can get a ray pointing in the direction you want.
You just need to extend that ray, from the camera position, until the z-coordinate of the tip of the ray is zero.
You can do that like so:
var vec = new THREE.Vector3(); // create once and reuse
var pos = new THREE.Vector3(); // create once and reuse
vec.set(
( event.clientX / window.innerWidth ) * 2 - 1,
- ( event.clientY / window.innerHeight ) * 2 + 1,
0.5 );
vec.unproject( camera );
vec.sub( camera.position ).normalize();
var distance = - camera.position.z / vec.z;
pos.copy( camera.position ).add( vec.multiplyScalar( distance ) );
The variable pos is the position of the point in 3D space, "under the mouse", and in the plane z=0.
EDIT: If you need the point "under the mouse" and in the plane z = targetZ, replace the distance computation with:
var distance = ( targetZ - camera.position.z ) / vec.z;
three.js r.98
This worked for me when using an orthographic camera
let vector = new THREE.Vector3();
vector.set(
(event.clientX / window.innerWidth) * 2 - 1,
- (event.clientY / window.innerHeight) * 2 + 1,
0
);
vector.unproject(camera);
WebGL three.js r.89
In r.58 this code works for me:
var planeZ = new THREE.Plane(new THREE.Vector3(0, 0, 1), 0);
var mv = new THREE.Vector3(
(event.clientX / window.innerWidth) * 2 - 1,
-(event.clientY / window.innerHeight) * 2 + 1,
0.5 );
var raycaster = projector.pickingRay(mv, camera);
var pos = raycaster.ray.intersectPlane(planeZ);
console.log("x: " + pos.x + ", y: " + pos.y);
Below is an ES6 class I wrote based on WestLangley's reply, which works perfectly for me in THREE.js r77.
Note that it assumes your render viewport takes up your entire browser viewport.
class CProjectMousePosToXYPlaneHelper
{
constructor()
{
this.m_vPos = new THREE.Vector3();
this.m_vDir = new THREE.Vector3();
}
Compute( nMouseX, nMouseY, Camera, vOutPos )
{
let vPos = this.m_vPos;
let vDir = this.m_vDir;
vPos.set(
-1.0 + 2.0 * nMouseX / window.innerWidth,
-1.0 + 2.0 * nMouseY / window.innerHeight,
0.5
).unproject( Camera );
// Calculate a unit vector from the camera to the projected position
vDir.copy( vPos ).sub( Camera.position ).normalize();
// Project onto z=0
let flDistance = -Camera.position.z / vDir.z;
vOutPos.copy( Camera.position ).add( vDir.multiplyScalar( flDistance ) );
}
}
You can use the class like this:
// Instantiate the helper and output pos once.
let Helper = new CProjectMousePosToXYPlaneHelper();
let vProjectedMousePos = new THREE.Vector3();
...
// In your event handler/tick function, do the projection.
Helper.Compute( e.clientX, e.clientY, Camera, vProjectedMousePos );
vProjectedMousePos now contains the projected mouse position on the z=0 plane.
to get the mouse coordinates of a 3d object use projectVector:
var width = 640, height = 480;
var widthHalf = width / 2, heightHalf = height / 2;
var projector = new THREE.Projector();
var vector = projector.projectVector( object.matrixWorld.getPosition().clone(), camera );
vector.x = ( vector.x * widthHalf ) + widthHalf;
vector.y = - ( vector.y * heightHalf ) + heightHalf;
to get the three.js 3D coordinates that relate to specific mouse coordinates, use the opposite, unprojectVector:
var elem = renderer.domElement,
boundingRect = elem.getBoundingClientRect(),
x = (event.clientX - boundingRect.left) * (elem.width / boundingRect.width),
y = (event.clientY - boundingRect.top) * (elem.height / boundingRect.height);
var vector = new THREE.Vector3(
( x / WIDTH ) * 2 - 1,
- ( y / HEIGHT ) * 2 + 1,
0.5
);
projector.unprojectVector( vector, camera );
var ray = new THREE.Ray( camera.position, vector.subSelf( camera.position ).normalize() );
var intersects = ray.intersectObjects( scene.children );
There is a great example here. However, to use project vector, there must be an object where the user clicked. intersects will be an array of all objects at the location of the mouse, regardless of their depth.
I had a canvas that was smaller than my full window, and needed to determine the world coordinates of a click:
// get the position of a canvas event in world coords
function getWorldCoords(e) {
// get x,y coords into canvas where click occurred
var rect = canvas.getBoundingClientRect(),
x = e.clientX - rect.left,
y = e.clientY - rect.top;
// convert x,y to clip space; coords from top left, clockwise:
// (-1,1), (1,1), (-1,-1), (1, -1)
var mouse = new THREE.Vector3();
mouse.x = ( (x / canvas.clientWidth ) * 2) - 1;
mouse.y = (-(y / canvas.clientHeight) * 2) + 1;
mouse.z = 0.5; // set to z position of mesh objects
// reverse projection from 3D to screen
mouse.unproject(camera);
// convert from point to a direction
mouse.sub(camera.position).normalize();
// scale the projected ray
var distance = -camera.position.z / mouse.z,
scaled = mouse.multiplyScalar(distance),
coords = camera.position.clone().add(scaled);
return coords;
}
var canvas = renderer.domElement;
canvas.addEventListener('click', getWorldCoords);
Here's an example. Click the same region of the donut before and after sliding and you'll find the coords remain constant (check the browser console):
// three.js boilerplate
var container = document.querySelector('body'),
w = container.clientWidth,
h = container.clientHeight,
scene = new THREE.Scene(),
camera = new THREE.PerspectiveCamera(75, w/h, 0.001, 100),
controls = new THREE.MapControls(camera, container),
renderConfig = {antialias: true, alpha: true},
renderer = new THREE.WebGLRenderer(renderConfig);
controls.panSpeed = 0.4;
camera.position.set(0, 0, -10);
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(w, h);
container.appendChild(renderer.domElement);
window.addEventListener('resize', function() {
w = container.clientWidth;
h = container.clientHeight;
camera.aspect = w/h;
camera.updateProjectionMatrix();
renderer.setSize(w, h);
})
function render() {
requestAnimationFrame(render);
renderer.render(scene, camera);
controls.update();
}
// draw some geometries
var geometry = new THREE.TorusGeometry( 10, 3, 16, 100, );
var material = new THREE.MeshNormalMaterial( { color: 0xffff00, } );
var torus = new THREE.Mesh( geometry, material, );
scene.add( torus );
// convert click coords to world space
// get the position of a canvas event in world coords
function getWorldCoords(e) {
// get x,y coords into canvas where click occurred
var rect = canvas.getBoundingClientRect(),
x = e.clientX - rect.left,
y = e.clientY - rect.top;
// convert x,y to clip space; coords from top left, clockwise:
// (-1,1), (1,1), (-1,-1), (1, -1)
var mouse = new THREE.Vector3();
mouse.x = ( (x / canvas.clientWidth ) * 2) - 1;
mouse.y = (-(y / canvas.clientHeight) * 2) + 1;
mouse.z = 0.0; // set to z position of mesh objects
// reverse projection from 3D to screen
mouse.unproject(camera);
// convert from point to a direction
mouse.sub(camera.position).normalize();
// scale the projected ray
var distance = -camera.position.z / mouse.z,
scaled = mouse.multiplyScalar(distance),
coords = camera.position.clone().add(scaled);
console.log(mouse, coords.x, coords.y, coords.z);
}
var canvas = renderer.domElement;
canvas.addEventListener('click', getWorldCoords);
render();
html,
body {
width: 100%;
height: 100%;
background: #000;
}
body {
margin: 0;
overflow: hidden;
}
canvas {
width: 100%;
height: 100%;
}
<script src='https://cdnjs.cloudflare.com/ajax/libs/three.js/97/three.min.js'></script>
<script src=' https://threejs.org/examples/js/controls/MapControls.js'></script>
ThreeJS is slowly mowing away from Projector.(Un)ProjectVector and the solution with projector.pickingRay() doesn't work anymore, just finished updating my own code.. so the most recent working version should be as follow:
var rayVector = new THREE.Vector3(0, 0, 0.5);
var camera = new THREE.PerspectiveCamera(fov,this.offsetWidth/this.offsetHeight,0.1,farFrustum);
var raycaster = new THREE.Raycaster();
var scene = new THREE.Scene();
//...
function intersectObjects(x, y, planeOnly) {
rayVector.set(((x/this.offsetWidth)*2-1), (1-(y/this.offsetHeight)*2), 1).unproject(camera);
raycaster.set(camera.position, rayVector.sub(camera.position ).normalize());
var intersects = raycaster.intersectObjects(scene.children);
return intersects;
}
For those using #react-three/fiber (aka r3f and react-three-fiber), I found this discussion and it's associated code samples by Matt Rossman helpful. In particular, many examples using the methods above are for simple orthographic views, not for when OrbitControls are in play.
Discussion: https://github.com/pmndrs/react-three-fiber/discussions/857
Simple example using Matt's technique: https://codesandbox.io/s/r3f-mouse-to-world-elh73?file=/src/index.js
More generalizable example: https://codesandbox.io/s/react-three-draggable-cxu37?file=/src/App.js
Here is my take at creating an es6 class out of it. Working with Three.js r83. The method of using rayCaster comes from mrdoob here: Three.js Projector and Ray objects
export default class RaycasterHelper
{
constructor (camera, scene) {
this.camera = camera
this.scene = scene
this.rayCaster = new THREE.Raycaster()
this.tapPos3D = new THREE.Vector3()
this.getIntersectsFromTap = this.getIntersectsFromTap.bind(this)
}
// objects arg below needs to be an array of Three objects in the scene
getIntersectsFromTap (tapX, tapY, objects) {
this.tapPos3D.set((tapX / window.innerWidth) * 2 - 1, -(tapY /
window.innerHeight) * 2 + 1, 0.5) // z = 0.5 important!
this.tapPos3D.unproject(this.camera)
this.rayCaster.set(this.camera.position,
this.tapPos3D.sub(this.camera.position).normalize())
return this.rayCaster.intersectObjects(objects, false)
}
}
You would use it like this if you wanted to check against all your objects in the scene for hits. I made the recursive flag false above because for my uses I did not need it to be.
var helper = new RaycasterHelper(camera, scene)
var intersects = helper.getIntersectsFromTap(tapX, tapY,
this.scene.children)
...
Although the provided answers can be useful in some scenarios, I hardly can imagine those scenarios (maybe games or animations) because they are not precise at all (guessing around target's NDC z?). You can't use those methods to unproject screen coordinates to the world ones if you know target z-plane. But for the most scenarios, you should know this plane.
For example, if you draw sphere by center (known point in model space) and radius - you need to get radius as delta of unprojected mouse coordinates - but you can't! With all due respect #WestLangley's method with targetZ doesn't work, it gives incorrect results (I can provide jsfiddle if needed). Another example - you need to set orbit controls target by mouse double click, but without "real" raycasting with scene objects (when you have nothing to pick).
The solution for me is to just create the virtual plane in target point along z-axis and use raycasting with this plane afterward. Target point can be current orbit controls target or vertex of object you need to draw step by step in existing model space etc. This works perfectly and it is simple (example in typescript):
screenToWorld(v2D: THREE.Vector2, camera: THREE.PerspectiveCamera = null, target: THREE.Vector3 = null): THREE.Vector3 {
const self = this;
const vNdc = self.toNdc(v2D);
return self.ndcToWorld(vNdc, camera, target);
}
//get normalized device cartesian coordinates (NDC) with center (0, 0) and ranging from (-1, -1) to (1, 1)
toNdc(v: THREE.Vector2): THREE.Vector2 {
const self = this;
const canvasEl = self.renderers.WebGL.domElement;
const bounds = canvasEl.getBoundingClientRect();
let x = v.x - bounds.left;
let y = v.y - bounds.top;
x = (x / bounds.width) * 2 - 1;
y = - (y / bounds.height) * 2 + 1;
return new THREE.Vector2(x, y);
}
ndcToWorld(vNdc: THREE.Vector2, camera: THREE.PerspectiveCamera = null, target: THREE.Vector3 = null): THREE.Vector3 {
const self = this;
if (!camera) {
camera = self.camera;
}
if (!target) {
target = self.getTarget();
}
const position = camera.position.clone();
const origin = self.scene.position.clone();
const v3D = target.clone();
self.raycaster.setFromCamera(vNdc, camera);
const normal = new THREE.Vector3(0, 0, 1);
const distance = normal.dot(origin.sub(v3D));
const plane = new THREE.Plane(normal, distance);
self.raycaster.ray.intersectPlane(plane, v3D);
return v3D;
}

Resources