So I'm trying to render the depth map of a scene using three.js
An example can be found here: http://threejs.org/docs/#Reference/Materials/MeshDepthMaterial
The depth map looks good when the scene is setup carefully, for example:
camera.position.z = 30;
camera.near = 0.1;
camera.far = 50;
object.position.z = 0;
However, if the camera is setup in some other ways, for example:
camera.position.z = 600;
camera.near = 550;
camera.far = 650;
object.position.z = 0;
In the second example, the depth resolution is all taken up by the depth range from -45 to -50 in the world coordinate system. This means for -50 < objects.position.z < -45, you can still see grayscale values in the depth map. Anything that has depth ranging from -45 to 50 will appear completely white in the depth map...
Ideally, I would like to see depth values uniformly distributed in the -50 to 50 range.
Is there a way to get the depth map I want?
Is there a way to increase the precision of depth map?
Eventually, I would like to save the Depth Map of the three.js scene as a .png image on to local disk for some analysis. Methods that don't use three.js are also welcome. Thanks.
Together with MeshDepthMaterial and the logarithmicDepthBuffer flag, I can get pretty nice looking depth map with the weird camera setup.
var renderer = new THREE.WebGLRenderer({logarithmicDepthBuffer: true });
Related
I'd like to enable a user to rotate a texture on a rectangle while keeping the aspect ratio of the texture image intact. I'm doing the rotation of a 1:1 aspect ratio image on a surface that is rectangular (say width: 2 and length: 1)
Steps to reproduce:
In the below texture rotation example
https://threejs.org/examples/?q=rotation#webgl_materials_texture_rotation
If we change one of the faces of the geometry like below:
https://github.com/mrdoob/three.js/blob/master/examples/webgl_materials_texture_rotation.html#L57
var geometry = new THREE.BoxBufferGeometry( 20, 10, 10 );
Then you can see that as you play around with the rotation control, the image aspect ratio is distorted. (form a square to a weird shape)
At 0 degree:
At some angle between 0 and 90:
I understand that by changing the repeatX and repeatY factor I can control this. It's also easy to see what the values would be at 0 degree, 90 degree rotations.
But I'm struggling to come up with the formula for repeatX and repeatY that works for any texture rotation given length and width of the rectangular face.
Unfortunately when stretching geometry like that, you'll get a distortion in 3D space, not UV space. In this example, one UV.x unit occupies twice as much 3D space as one UV.y unit:
This is giving you those horizontally-skewed diamonds when in between rotations:
Sadly, there's no way to solve this with texture matrix transforms. The horizontal stretching will be applied after the texture transform, in 3D space, so texture.repeat won't help you avoid this. The only way to solve this is by modifying the UVs so the UV.x units take up as much 3D space as UV.y units:
With complex models, you'd do this kind of "equalizing" in a 3D editor, but since the geometry is simple enough, we can do it via code. See the example below. I'm using a width/height ratio variable to use in my UV.y remapping, that way the UV transformations will match up, regardless of how much wider it is.
//////// Boilerplate Three setup
const renderer = new THREE.WebGLRenderer({canvas: document.querySelector("canvas")});
const camera = new THREE.PerspectiveCamera(50, 1, 1, 100);
camera.position.z = 3;
const scene = new THREE.Scene();
/////////////////// CREATE GEOM & MATERIAL
const width = 2;
const height = 1;
const ratio= width / height; // <- magic number that will help with UV remapping
const geometry = new THREE.BoxBufferGeometry(width, height, width);
let uvY;
const uvArray = geometry.getAttribute("uv").array;
// Re-map UVs to avoid distortion
for (let i2 = 0; i2 < uvArray.length; i2 += 2){
uvY = uvArray[i2 + 1]; // Extract Y value,
uvY -= 0.5; // center around 0
uvY /= ratio; // divide by w/h ratio
uvY += 0.5; // remove center around 0
uvArray[i2 + 1] = uvY;
}
geometry.getAttribute("uv").needsUpdate = true;
const uvMap = new THREE.TextureLoader().load("https://raw.githubusercontent.com/mrdoob/three.js/dev/examples/textures/uv_grid_opengl.jpg");
// Now we can apply texture transformations as expected
uvMap.center.set(0.5, 0.5);
uvMap.repeat.set(0.25, 0.5);
uvMap.anisotropy = 16;
const material = new THREE.MeshBasicMaterial({map: uvMap});
const mesh = new THREE.Mesh(geometry, material);
scene.add(mesh);
window.addEventListener("mousemove", onMouseMove);
window.addEventListener("resize", resize);
// Add rotation on mousemove
function onMouseMove(ev) {
uvMap.rotation = (ev.clientX / window.innerWidth) * Math.PI * 2;
}
function resize() {
const width = window.innerWidth;
const height = window.innerHeight;
renderer.setSize(width, height);
camera.aspect = width / height;
camera.updateProjectionMatrix();
}
function animate(time) {
mesh.rotation.y = Math.cos(time/ 3000) * 2;
renderer.render(scene, camera);
requestAnimationFrame(animate);
}
resize();
requestAnimationFrame(animate);
body { margin: 0; }
canvas { width: 100vw; height: 100vh; display: block; }
<script src="https://threejs.org/build/three.js"></script>
<canvas></canvas>
First of all, I agree with the solution #Marquizzo provided to your problem. And setting UV explicitly to the geometry should be the easiest way to solve your problem.
But #Marquizzo did not answer why changing the matrix of the texture (set repeatX and repeatY) does not work.
We all know the 2D rotation matrix R
cos -sin
sin cos
UVs are calculated in the shader with a transform matrix T, which is the texture matrix from your question.
T * UV = new UV
To simplify the question, we only consider rotation. And assume we have another additional matrix X for calculating the new UV. Then we have
X * R * UV = new UV
The question now is whether we can find a solution ofX, so that with any rotation, new UV of any points in your question can be calculated correctly. If there is a solution of X, then we can simply use
var X = new Matrix3();
//X.set(x,y,z,...)
texture.matrix.premultiply(X);
Otherwise, we can't find the approach you expected.
Let's create several equations to figure out X.
In the pic below, ABCD is one face of your geometry, and the transparent green is the texture. The UV of point A is (0,1), point B is (0,0), and (1,0), (1,1) for C and D respectively.
The first equation comes from the consideration, without any rotation, the original UV should never be changed (UV for A is always (0,1)). So we should have
X * I * (0, 1) = (0, 1) // I is the identity matrix
From here we can see X should also be an identity matrix.
Then let's see whether the identity matrix X can satisfy the second equation. What's the second equation? Simplify again, let B be the rotation centre(origin) and rotate the texture 90 degrees(counterclockwise). We use -90 to calculate UV though we rotate 90 degrees.
The new UV for point A after rotating the texture 90 degrees should be the current value of E. The value of E is (a/b, 0). Then we have
From this equation we can see X should not be an identity matrix, which means, WE ARE NOT ABLE TO FIND A SOLUTION OF X TO SOLVE YOUR PROBLEM WITH
X * R * UV = new UV
Certainly, you can change the shader of calculating new UVs, but it's even harder than the way #Marquizzo provided.
I know a method from Unity whichs is very useful to convert a screen position to a world position : https://docs.unity3d.com/ScriptReference/Camera.ScreenToWorldPoint.html
I've been looking for something similar in A-Frame/THREE.js, but I didn't find anything.
Is there an easy way to convert a screen position to a world position in a plane which is positioned a given distance from the camera ?
This is typically done using Raycaster. An equivalent function using three.js would be written like this:
function screenToWorldPoint(screenSpaceCoord, target = new THREE.Vector3()) {
// convert the screen-space coordinates to normalized device coordinates
// (x and y ranging from -1 to 1):
const ndc = new THREE.Vector2()
ndc.x = 2 * screenSpaceCoord.x / screenWidth - 1;
ndc.y = 2 * screenSpaceCoord.y / screenHeight - 1;
// `Raycaster` can be used to convert this into a ray:
const raycaster = new THREE.Raycaster();
raycaster.setFromCamera(ndc, camera);
// finally, apply the distance:
return raycaster.ray.at(screenSpaceCoord.z, target);
}
Note that coordinates in browsers are usually measured from the top/left corner with y pointing downwards. In that case, the NDC calculation should be:
ndc.y = 1 - 2 * screenSpaceCoord.y / screenHeight;
Another note: instead of using a set distance in screenSpaceCoord.z you could also let three.js compute an intersection with any Object in your scene. For that you can use raycaster.intersectObject() and get a precise depth for the point of intersection with that object. See the documentation and various examples linked here: https://threejs.org/docs/#api/core/Raycaster
I've run into an issue after switching to a logarithmic depth buffer in Three.js. Everything runs nicely except for nearby culling of the ground as described in the following photos:
As you can see, the camera is elevated above the ground significantly. The character box that is shown is about 2 units above the ground, and my camera is set up as such:
var WIDTH = window.innerWidth
, HEIGHT = window.innerHeight;
var VIEW_ANGLE = 70
, ASPECT = WIDTH / HEIGHT
, NEAR = 1e-6
, FAR = 9000;
var aspect = WIDTH / HEIGHT;
var camera = new THREE.PerspectiveCamera(VIEW_ANGLE, ASPECT, NEAR, FAR);
camera.rotation.order = 'YXZ';
So my NEAR parameter is nowhere near 2, the distance between the camera and the ground. You can see in the second image that I even move up the camera with my PointerLockControls and still run into the issue.
Can anyone diagnose my issue?
I also tested my issue by seeing if this bug occurred with a static camera as well. It does.
Additionally, this problem only happens with the logarithmic depth buffer, as it doesn't happen with the default depth buffer.
I have my camera as a child to a controls object, which is defined as follows:
controls = new THREE.PointerLockControls(camera);
controls.getObject().position.set(strtx, 50, strtz);
scene.add(controls.getObject());
camera.position.z += 2;
camera.position.y += .1;
Here's the relevant code for PointerLockControls:
var pitchObject, yawObject;
var v = new THREE.Vector3(0, 0, -1);
THREE.PointerLockControls = function(camera){
var scope = this;
camera.rotation.set(0, 0, 0);
pitchObject = new THREE.Object3D();
pitchObject.rotation.x -= 0.3;
pitchObject.add(camera);
yawObject = new THREE.Object3D();
yawObject.position.y = 10;
yawObject.add(pitchObject);
var PI_2 = Math.PI / 2;
var onMouseMove = function(event){
if (scope.enabled === false) return;
var movementX = event.movementX || event.mozMovementX || event.webkitMovementX || 0;
var movementY = event.movementY || event.mozMovementY || event.webkitMovementY || 0;
yawObject.rotation.y -= movementX * 0.002;
pitchObject.rotation.x -= movementY * 0.002;
pitchObject.rotation.x = Math.max( - PI_2, Math.min( PI_2, pitchObject.rotation.x ) );
};
this.dispose = function() {
document.removeEventListener( 'mousemove', onMouseMove, false );
};
document.addEventListener( 'mousemove', onMouseMove, false );
this.enabled = false;
this.getObject = function () {
return yawObject;
};
this.getDirection = function() {
// assumes the camera itself is not rotated
var rotation = new THREE.Euler(0, 0, 0, "YXZ");
var direction = new THREE.Vector3(0, 0, -1);
return function() {
rotation.set(pitchObject.rotation.x, yawObject.rotation.y, 0);
v.copy(direction).applyEuler(rotation);
return v;
};
}();
};
You'll also notice that it's only the ground that is being culled, not other objects
Edit:
I've whipped up an isolated environment that shows the larger issue. In the first image, I have a flat PlaneBufferGeometry that has 400 segments for both width and height, defined by var g = new THREE.PlaneBufferGeometry(380, 380, 400, 400);. Even getting very close to the surface, no clipping is present:
However, if I provide only 1 segment, var g = new THREE.PlaneBufferGeometry(380, 380, 1, 1);, the clipping is present
I'm not sure if this intended in Three.js/WebGL, but it seems that I'll need to do something to work around it.
I don't think this is a bug, I think this is a feature of how the depthbuffer in the different settings works. Look at this example. On the right, the depthbuffer can't make up its mind between the letters in "microscopic" and the sphere. This is because it has lower precision at very small scales and starts doing rounding that oscilates between one object and another, and favoring draw order over z-depth.
It's always a tradeoff. If you want to forgo this issue, you can try raising the scale of your scene overall, so that the 'near' of the camera will never be so close to something that it can round it off - so just work in a number range that won't be rounded in the exponential model of the logarithmic z-buffer.
Also another question - how is the blue defined, because maybe what you're seeing is not clipping from being too close, but confusion between whether blue or the ground is closer. If it's just a blue box encompassing everything, you could try making it bigger and more distant from the ground.
EDIT:
Okay, this looks like it should work. so I would start looking for edge cases. What can you do to change the scene so that it does work? What can you do to make other things start breaking?
try moving the landscape far down/ far up (does the issue persist when looking up instead of down at it, does it persist even when it's unquestionably far away?)
try rotating the landscape
try changing the camera FOV
try changing the camera far plane
try changing the camera near plane from 1e-x notation to .000001, .0001,.01,.1, etc. see what effect it has.
console.log the camera object in your render function, and make sure that the fov, near, far etc, is as you set on setup and that it's not being overwritten and reset to default. check what it prints out in chrome's developer tools, you can browse the whole object, check position, parent name, all that stuff.
basically i don't see a blatant mistake, so I would guess it's something hard to spot, or it's working exactly as it should. Figure out what you can do to improve the effect/ make it worse, and that will clarify a direction to go.
A good rule of thumb for debugging is to try and just take things to an extreme, without trying to fix it, or keep the code true to its purpose, and just see in what way it breaks further/changes. report back when you find something.
I have seams between horizontal faces of the cube when use texture atlas in three.js.
This is demo: http://jsfiddle.net/rnix/gtxcj3qh/7/ or http://jsfiddle.net/gtxcj3qh/8/ (from comments)
Screenshot of the problem:
Here I use repeat and offset:
var materials = [];
var t = [];
var imgData = document.getElementById("texture_atlas").src;
for ( var i = 0; i < 6; i ++ ) {
t[i] = THREE.ImageUtils.loadTexture( imgData ); //2048x256
t[i].repeat.x = 1 / 8;
t[i].offset.x = i / 8;
//t[i].magFilter = THREE.NearestFilter;
t[i].minFilter = THREE.NearestFilter;
t[i].generateMipmaps = false;
materials.push( new THREE.MeshBasicMaterial( { map: t[i], overdraw: 0.5 } ) );
}
var skyBox = new THREE.Mesh( new THREE.CubeGeometry( 1024, 1024, 1024), new THREE.MeshFaceMaterial(materials) );
skyBox.applyMatrix( new THREE.Matrix4().makeScale( 1, 1, -1 ) );
scene.add( skyBox );
The atlas has size 2048x256 (power of two). I also tried manual UV-mapping instead of repeat, but the result is the same. I use 8 tiles instead of 6 because I have thought precision of division 1/6 causes the problem, but not.
Pixels on this line are from next tile in atlas. I tried completly white atlas and there was not any artefacts. This explains why there are not seams on vertical borders of Z-faces. I have played with filters, wrapT, wrapS and mipmaps but it does not help. Increasing resolution does not help. There is 8192x1024 atlas http://s.getid.org/jsfiddle/atlas.png I tried another atlas, the result is the same.
I know that I can split atlas into separate files and it works perfectly but it is not convenient.
Whats wrong?
I think the issue is the filtering problem with texture sheets. On image borders in a texture sheet, the gpu may pick the texel from either the correct image or the neighbor image due to limited precision. Because the colors are usually very different, this results in the visible seams. In regular textures, this is solved with CLAMP_TO_EDGE.
If you must use texture alias, then you need to fake CLAMP_TO_EDGE behavior by padding the image borders. See this answer https://gamedev.stackexchange.com/questions/61796/sprite-sheet-textures-picking-up-edges-of-adjacent-texture. It should look something like this: (exaggerated borders for clarity)
Otherwise, the simpler solution is to use a different texture for each face. Webgl supports the cube texture and that is usually used the majority of the time to implement skyboxes.
Hack the uv, replace all value 1.0 with 0.999, replace all value 0 with 0.001 will fakely resolve part of this problem.
I am trying to use the Three.js library to display a large number of colored points on the screen (about half a million to million for example). I am trying to use the Canvas renderer rather than the WebGL renderer if possible (The web pages would also be displayed in the Google Earth Client bubbles, which seems to work with Canvas renderer but not the WebGL renderer.)
While I have the problem solved for a small number of points (tens of thousands) by modifying the code from here, I am having trouble scaling it beyond that.
But in the the following code using WebGL and the Particle System I can render half a million random points, but without colors.
...
var particles = new THREE.Geometry();
var pMaterial = new THREE.ParticleBasicMaterial({
color: 0xFFFFFF,
size: 1,
sizeAttenuation : false
});
// now create the individual particles
for (var p = 0; p < particleCount; p++) {
// create a particle with randon position values,
// -250 -> 250
var pX = Math.random() * POSITION_RANGE - (POSITION_RANGE / 2),
pY = Math.random() * POSITION_RANGE - (POSITION_RANGE / 2),
pZ = Math.random() * POSITION_RANGE - (POSITION_RANGE / 2),
particle = new THREE.Vertex(
new THREE.Vector3(pX, pY, pZ)
);
// add it to the geometry
particles.vertices.push(particle);
}
var particleSystem = new THREE.ParticleSystem(
particles, pMaterial);
scene.add(particleSystem);
...
Is the reason for the better performance of the above code due to the Particle System? From what I have read in the documentation it seems the Particle System can only be used by the WebGL renderer.
So my question(s) are
a) Can I render such large number of particles using the Canvas renderer or is it always going to be slower than the WebGL/ParticleSystem version? If so, how do I go about doing that? What objects and or tricks do I use to improve performance?
b) Is there a compromise I can reach if I give up some features? In other words, can I still use the Canvas renderer for the large dataset if I give up the need to color the individual points?
c) If I have to give up the Canvas and use the WebGL version, is it possible to change the colors of the individual points? It seems the color is set by the material passed to the ParticleSystem and that sets the color for all the points.
EDIT: ParticleSystem and PointCloud has been renamed to Points. In addition, ParticleBasicMaterial and PointCloudMaterial has been renamed to PointsMaterial.
This answer only applies to versions of three.js prior to r.125.
To have a different color for each particle, you need to have a color array as a property of the geometry, and then set vertexColors to THREE.VertexColors in the material, like so:
// vertex colors
var colors = [];
for( var i = 0; i < geometry.vertices.length; i++ ) {
// random color
colors[i] = new THREE.Color();
colors[i].setHSL( Math.random(), 1.0, 0.5 );
}
geometry.colors = colors;
// material
material = new THREE.PointsMaterial( {
size: 10,
transparent: true,
opacity: 0.7,
vertexColors: THREE.VertexColors
} );
// point cloud
pointCloud = new THREE.Points( geometry, material );
Your other questions are a little too general for me to answer, and besides, it depends on exactly what you are trying to do and what your requirements are. Yes, you can expect Canvas to be slower.
EDIT: Updated for three.js r.124