Removing moire patterns produced by GLSL shaders - three.js

I have setup this minimal test case, which you can easily see the moire patterns produced by undersampling the oscilating red colour using a custom fragment shader (jsfiddle).
What is the general technique for removing such patterns using GLSL? I assume it involves the derivatives extension, but I've never quite understood how to implement it. I basically have to do anti-aliasing, I think?
var canvas = document.getElementById('canvas');
var scene = new THREE.Scene();
var renderer = new THREE.WebGLRenderer({canvas: canvas, antialias: true});
var camera = new THREE.PerspectiveCamera(75, canvas.clientWidth / canvas.clientWidth, 1, 1000);
var geometry = new THREE.SphereGeometry(50, 50, 50);
var material = new THREE.ShaderMaterial({
vertexShader: document.getElementById('vertex-shader').textContent,
fragmentShader: document.getElementById('fragment-shader').textContent
});
var sphere = new THREE.Mesh(geometry, material);
scene.add(sphere);
camera.position.z = 100;
var period = 30;
var clock = new THREE.Clock();
render();
function render() {
requestAnimationFrame(render);
if (canvas.width !== canvas.clientWidth || canvas.height !== canvas.clientHeight) {
renderer.setSize(canvas.clientWidth, canvas.clientHeight, false);
camera.aspect = canvas.clientWidth / canvas.clientHeight;
camera.updateProjectionMatrix();
}
sphere.rotation.y -= clock.getDelta() * 2 * Math.PI / period;
renderer.render(scene, camera);
}
html, body, #canvas {
margin: 0;
padding: 0;
width: 100%;
height: 100%;
overflow: hidden;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r73/three.min.js"></script>
<canvas id="canvas"></canvas>
<script id="vertex-shader" type="x-shader/x-vertex">
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
</script>
<script id="fragment-shader" type="x-shader/x-fragment">
#define M_TAU 6.2831853071795864769252867665590
varying vec2 vUv;
void main() {
float w = sin(500.0 * M_TAU * vUv.x) / 2.0 + 0.5;
vec3 color = vec3(w, 0.0, 0.0);
gl_FragColor = vec4(color, 1.0);
}
</script>
Update: I've tried to implement super-sampling, not sure if I have implemented it correctly but it doesn't seem to help too much.

Unfortunately, the moire pattern here is a result of the high-contrast lines approaching the Nyquist Frequency. In other words, there's no good way to have a 1- or 2-pixel-wide high-contrast line smoothly shift to the next pixel over, without either introducing such artifacts, or blurring the lines to be indistinguishable.
You mentioned the derivatives extension, and indeed that extension can be used to figure out how quickly your UVs are changing in screen space, and thus, figure out how much blurring is needed to sort of sweep this problem under the rug. In the modified version of your own example below, I attempt to use fwidth to turn the sphere red where the noise gets bad. Try playing with some of the floats that are defined to constants here, see what you can find.
var canvas = document.getElementById('canvas');
var scene = new THREE.Scene();
var renderer = new THREE.WebGLRenderer({canvas: canvas, antialias: true});
var camera = new THREE.PerspectiveCamera(75, canvas.clientWidth / canvas.clientWidth, 1, 1000);
var geometry = new THREE.SphereGeometry(50, 50, 50);
var material = new THREE.ShaderMaterial({
vertexShader: document.getElementById('vertex-shader').textContent,
fragmentShader: document.getElementById('fragment-shader').textContent
});
var sphere = new THREE.Mesh(geometry, material);
scene.add(sphere);
camera.position.z = 100;
var period = 30;
var clock = new THREE.Clock();
render();
function render() {
requestAnimationFrame(render);
if (canvas.width !== canvas.clientWidth || canvas.height !== canvas.clientHeight) {
renderer.setSize(canvas.clientWidth, canvas.clientHeight, false);
camera.aspect = canvas.clientWidth / canvas.clientHeight;
camera.updateProjectionMatrix();
}
sphere.rotation.y -= clock.getDelta() * 2 * Math.PI / period;
renderer.render(scene, camera);
}
html, body, #canvas {
margin: 0;
padding: 0;
width: 100%;
height: 100%;
overflow: hidden;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r73/three.min.js"></script>
<canvas id="canvas"></canvas>
<script id="vertex-shader" type="x-shader/x-vertex">
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
</script>
<script id="fragment-shader" type="x-shader/x-fragment">
#extension GL_OES_standard_derivatives : enable
#define M_TAU 6.2831853071795864769252867665590
varying vec2 vUv;
void main() {
float linecount = 200.0;
float thickness = 0.0;
float blendregion = 2.8;
// Loosely based on https://github.com/AnalyticalGraphicsInc/cesium/blob/1.16/Source/Shaders/Materials/GridMaterial.glsl#L17-L34
float scaledWidth = fract(linecount * vUv.s);
scaledWidth = abs(scaledWidth - floor(scaledWidth + 0.5));
vec2 dF = fwidth(vUv) * linecount;
float value = 1.0 - smoothstep(dF.s * thickness, dF.s * (thickness + blendregion), scaledWidth);
gl_FragColor = vec4(value, 0.0, 0.0, 1.0);
}
</script>

Related

GLSL Three.js vertex and fragment shader is not working [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed last year.
Improve this question
I am new to Three.js and GLSL and I am trying to load my vertex and fragment shader in HTML format. However, it was not working, and I am facing the error
Uncaught TypeError: Cannot read properties of null (reading 'textContent')
Is there a reason why this happen and how can I solve this?
I tried to do my vertex and fragment shader inline with the HTML document.
File index.html
<!DOCTYPE html>
<html lang="en">
<head>
<title>OBJ loader</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0">
<style>
body {
font-family: Monospace;
background-color: #FFF;
color: #FFF;
margin: 0px;
overflow: hidden;
}
#info {
color: #FFF;
position: absolute;
top: 10px;
width: 100%;
text-align: center;
z-index: 100;
display: block;
}
#info a, .button {
color: #F00;
font-weight: bold;
text-decoration:
underline;
cursor: pointer
}
</style>
</head>
<body>
<script src="http://threejs.org/build/three.min.js"></script>
<script src="http://threejs.org/examples/js/loaders/OBJLoader.js"></script>
<script src="http://threejs.org/examples/js/libs/stats.min.js"></script>
<script id="vertex_shader" type="x-shader/x-vertex">
layout (location = 0) in vec2 pos;
void main() {
gl_Position = vec4(pos, 0.0f, 1);
}
</script>
<script id="fragment_shader" type="x-shader/x-fragment">
uniform vec2 uResolution;
uniform float uTime;
out vec4 outColor;
precision highp float;
uniform sampler2D tex;
void main()
{
vec2 uv = gl_FragCoord.xy/uResolution;
float time = uTime * 0.4;
// Apply pixelate effect
//vec2 uv_pixel = uv;
vec2 uv_pixel = floor(uv * (uResolution/4)) / (uResolution/4);
vec4 col1 = vec4(0.510, 0.776, 0.486, 1.0);
vec4 col2 = vec4(0.200, 0.604, 0.318, 1.0);
vec4 col3 = vec4(0.145, 0.490, 0.278, 1.0);
vec4 col4 = vec4(0.059, 0.255, 0.251, 1.0);
// Displacement on top of y
vec3 displace = texture(tex, vec2(uv_pixel.x, (uv_pixel.y + time) * 0.05)).xyz;
displace *= 0.5;
displace.x -= 1.0;
displace.y -= 1.0;
displace.y *= 0.5;
// Color
vec2 uv_tmp = uv_pixel;
uv_tmp.y *= 0.2;
uv_tmp.y += time;
vec4 color = texture(tex, uv_tmp + displace.xy);
// Match to colors
vec4 noise = floor(color * 10.0) / 5.0;
vec4 dark = mix(col1, col2, uv.y);
vec4 bright = mix(col3, col4, uv.y);
color = mix(dark, bright, noise);
// Add gradients (top dark and transparent, bottom bright)
float inv_uv = 1.0 - uv_pixel.y;
color.xyz -= 0.45 * pow(uv_pixel.y, 8.0);
color.a -= 0.2 * pow(uv_pixel.y, 8.0);
color += pow(inv_uv, 8.0);
// Make the waterfall transparent
color.a -= 0.2;
outColor = vec4(color);
}
</script>
<script>
var clock = new THREE.Clock();
var delta = clock.getDelta(); // Seconds.
var rotateAngle = Math.PI / 2 * delta; // pi/2 radians (90 degrees) per second
var container, stats;
var camera, scene, renderer, texture;
var mouseX = 0, mouseY = 0;
var windowHalfX = window.innerWidth / 2;
var windowHalfY = window.innerHeight / 2;
const customMaterial = new THREE.ShaderMaterial({
uniforms: {
time: { value: 1.0 },
map: { value: texture },
resolution: { value: new THREE.Vector2() }
},
vertexShader: document.getElementById('vertexshader').textContent,
fragmentShader: document.getElementById('fragmentshader').textContent,
});
init();
animate();
//var texture = new THREE.Texture();
new THREE.OBJLoader().load('https://s3-us-west-2.amazonaws.com/s.cdpn.io/39255/ladybug.gltf', function (object) {
object.traverse(function (child) {
if (child instanceof THREE.Mesh) {
child.material = customMaterial;
}
});
scene.add(object);
});
function init() {
container = document.createElement('div');
document.body.appendChild(container);
camera = new THREE.PerspectiveCamera(15, window.innerWidth / window.innerHeight, 1, 2000);
camera.position.z = 100;
// Scene
scene = new THREE.Scene();
var ambient = new THREE.AmbientLight(0x111130);
scene.add(ambient);
var directionalLight = new THREE.DirectionalLight(0xFFEEFF);
directionalLight.position.set(1, 1, 0.5);
scene.add(directionalLight);
renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
container.appendChild(renderer.domElement);
document.addEventListener('mousemove', onDocumentMouseMove, false);
window.addEventListener('resize', onWindowResize, false);
}
function onWindowResize() {
windowHalfX = window.innerWidth / 2;
windowHalfY = window.innerHeight / 2;
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(window.innerWidth, window.innerHeight);
}
function onDocumentMouseMove(event) {
mouseX = (event.clientX - windowHalfX) / 2;
mouseY = (event.clientY - windowHalfY) / 2;
}
function animate() {
requestAnimationFrame(animate);
render();
}
function render() {
camera.position.x += (mouseX - camera.position.x) * .05;
camera.position.y += (- mouseY - camera.position.y) * .05;
camera.lookAt(scene.position);
renderer.render(scene, camera);
}
</script>
</body>
</html>
There are typos in your code. The IDs of your shader script tags are vertex_shader and fragment_shader but you are using vertexshader and fragmentshader in your JavaScript code. You are missing the underscores.

Not getting expect result from THREE.js/GLSL code

Apologies for the vague question but I wasn't sure how to phrase this. I'm trying to write some THREE.js/GLSL code that produces a circular gradient (for some SDF stuff). With the code below I would expect to see the gradient on the plane, but the plane remains white and nothing else renders on it. Can anyone tell me where I'm going wrong?
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>My first three.js app</title>
<style>
body { margin: 0; }
canvas { width: 100%; height: 100% }
</style>
</head>
<body>
<script type="x-shader/x-vertex" id="sdfVS">
varying vec2 vUv; // pass the uv coordinates of each pixel to the frag shader
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
</script>
<script type="x-shader/x-fragment" id="sdfFS">
precision mediump float;
uniform vec2 u_resolution;
varying vec2 vUv;
float circle(vec2 pos, float radius)
{
return distance(pos, vec2(radius));
}
void main()
{
vec2 pos = (gl_FragCoord.xy / vUv) * 2.0 - 1.0;
float circle1 = circle(pos, 0.5);
vec3 color = vec3(circle1);
gl_FragColor = vec4(color, 1.0);
}
</script>
<script src="../../js/three.min.js"></script>
<script>
var scene, camera, renderer, aspect, geometry, material, plane;
var container;
var frustumSize = 2;
init();
animate();
function init() {
container = document.createElement( 'div' );
document.body.appendChild( container );
scene = new THREE.Scene();
scene.background = new THREE.Color(0x0000ff);
aspect = window.innerWidth / window.innerHeight;
camera = new THREE.OrthographicCamera( 0.5 * frustumSize * aspect / - 2, 0.5 * frustumSize * aspect / 2, frustumSize / 2, frustumSize / - 2, 0.1, 1 );
cameraOrthoHelper = new THREE.CameraHelper( camera );
scene.add( cameraOrthoHelper );
var width = 1;
var height = 1;
geometry = new THREE.PlaneGeometry(width, height);
material = new THREE.ShaderMaterial( {
vertexShader: document.getElementById('sdfVS').textContent,
fragmentShader: document.getElementById('sdfFS').textContent,
side: THREE.DoubleSide
} );
plane = new THREE.Mesh( geometry, material );
plane.rotation.x = 0;
plane.rotation.y = THREE.Math.degToRad( -90 );
plane.rotation.z = 0;
scene.add( plane )
renderer = new THREE.WebGLRenderer();
renderer.setPixelRatio( window.devicePixelRatio );
renderer.setSize( window.innerWidth, window.innerHeight );
container.appendChild( renderer.domElement );
window.addEventListener( 'resize', onWindowResize, false );
}
function onWindowResize() {
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize( window.innerWidth, window.innerHeight );
}
function animate() {
requestAnimationFrame( animate );
render();
}
function render() {
camera.position.x = -1;
camera.position.y = 0;
camera.position.z = 0;
camera.lookAt( scene.position );
camera.updateMatrixWorld();
renderer.render( scene, camera );
}
</script>
</body>
</html>
Can't get this line:
vec2 pos = (gl_FragCoord.xy / vUv) * 2.0 - 1.0;
Seems like you want to calculate uv coordinates, but if so, then you already have uvs (passed in vUv).
You can do the thing this way, as an option:
var scene, camera, renderer, aspect, geometry, material, plane;
var container;
var clock = new THREE.Clock();
var frustumSize = 2;
init();
animate();
function init() {
container = document.createElement('div');
document.body.appendChild(container);
scene = new THREE.Scene();
scene.background = new THREE.Color(0x0000ff);
aspect = window.innerWidth / window.innerHeight;
camera = new THREE.OrthographicCamera(0.5 * frustumSize * aspect / -2, 0.5 * frustumSize * aspect / 2, frustumSize / 2, frustumSize / -2, 0.1, 1);
cameraOrthoHelper = new THREE.CameraHelper(camera);
scene.add(cameraOrthoHelper);
var width = 1;
var height = 1;
geometry = new THREE.PlaneGeometry(width, height);
material = new THREE.ShaderMaterial({
uniforms: {time: {value: 0}},
vertexShader: document.getElementById('sdfVS').textContent,
fragmentShader: document.getElementById('sdfFS').textContent,
side: THREE.DoubleSide
});
plane = new THREE.Mesh(geometry, material);
plane.rotation.x = 0;
plane.rotation.y = THREE.Math.degToRad(-90);
plane.rotation.z = 0;
scene.add(plane)
renderer = new THREE.WebGLRenderer();
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
container.appendChild(renderer.domElement);
window.addEventListener('resize', onWindowResize, false);
}
function onWindowResize() {
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(window.innerWidth, window.innerHeight);
}
function animate() {
requestAnimationFrame(animate);
render();
}
function render() {
camera.position.x = -1;
camera.position.y = 0;
camera.position.z = 0;
camera.lookAt(scene.position);
camera.updateMatrixWorld();
var t = clock.getElapsedTime();
material.uniforms.time.value = t;
renderer.render(scene, camera);
}
body {
overflow: hidden;
margin: 0;
}
<script src="https://threejs.org/build/three.min.js"></script>
<script type="x-shader/x-vertex" id="sdfVS">
varying vec2 vUv; // pass the uv coordinates of each pixel to the frag shader
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
</script>
<script type="x-shader/x-fragment" id="sdfFS">
precision mediump float;
uniform float time;
uniform vec2 u_resolution;
varying vec2 vUv;
float circle(vec2 uv, vec2 pos, float radius) {
return smoothstep(radius, 0., length(uv - pos));
}
void main()
{
vec2 uv = vUv * 2. - 1.;
vec2 pos = vec2(cos(time), sin(time));
float circle1 = circle(uv, pos, 1.0);
vec3 color = vec3(circle1);
gl_FragColor = vec4(color, 1.0);
}
</script>

GLSL/THREE.js Math to turn a shape into a cube issue

I am making code to turn a sphere into a cube in a vertex shader, but it seems to turn into this weird shape, my logic was this:
The commented out code was the iterative version.
vec3 p = position;
if(true)
{
if(p.y<s&&p.y>-s){
p.x = -(p.x-s);//p.x-=(p.x-s)*t*0.1;
}
if(p.x<s&&p.x>-s){
p.y = -(p.y-s);//p.y-=(p.y-s)*t*0.1;
}
}
gl_Position = projectionMatrix * modelViewMatrix * vec4( p, 1.0 );
But then that turns this:
Into this:
Any help appreciated.
Use THREE.BoxBufferGeometry() as a base, then add another buffer attribute with coodinates for a sphere formation, then interpolate (mix) those coordinates of the box and the sphere in the shader:
var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera(60, window.innerWidth / window.innerHeight, 1, 100);
camera.position.set(1, 3, 5);
var renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
var controls = new THREE.OrbitControls(camera, renderer.domElement);
scene.add(new THREE.GridHelper(10, 10));
var side = 2;
var rad = Math.sqrt(3) * 0.5 * side; // radius of the sphere is a half of cube's diagonal
var geom = new THREE.BoxBufferGeometry(side, side, side, 10, 10, 10);
var pos = geom.attributes.position;
var spherePos = []; // array of coordinates for the sphere formation
var vec3 = new THREE.Vector3(); // vector for re-use
for (let i = 0; i < pos.count; i++) {
vec3.fromBufferAttribute(pos, i).setLength(rad); // create coordinate for the sphere formation
spherePos.push(vec3.x, vec3.y, vec3.z);
}
geom.addAttribute("spherePos", new THREE.BufferAttribute(new Float32Array(spherePos), 3));
var mat = new THREE.ShaderMaterial({
uniforms: {
mixShapes: {
value: 0
}
},
vertexShader: `
uniform float mixShapes;
attribute vec3 spherePos;
void main() {
vec3 pos = mix(position, spherePos, mixShapes); // interpolation between shapes
gl_Position = projectionMatrix * modelViewMatrix * vec4(pos, 1.0);
}
`,
fragmentShader: `
void main() {
gl_FragColor = vec4(1.0, 0.0, 1.0,1.0);
}
`,
wireframe: true
});
var shape = new THREE.Mesh(geom, mat);
scene.add(shape);
var gui = new dat.GUI();
gui.add(mat.uniforms.mixShapes, "value", 0.0, 1.0).name("mixShapes");
renderer.setAnimationLoop(() => {
renderer.render(scene, camera)
});
body {
overflow: hidden;
margin: 0;
}
<script src="https://threejs.org/build/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/dat-gui/0.7.6/dat.gui.min.js"></script>
If you want to get a cube from a sphere, you can clamp vertices to min and max vectors of a bounding box you need (but accuracy of this approach depends on the amount of vertices of the sphere):
var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera(60, window.innerWidth / window.innerHeight, 1, 100);
camera.position.set(1, 3, 5);
var renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
var controls = new THREE.OrbitControls(camera, renderer.domElement);
scene.add(new THREE.GridHelper(10, 10));
var side = 2;
var rad = Math.sqrt(3) * 0.5 * side; // radius of the sphere is a half of cube's diagonal
var geom = new THREE.SphereBufferGeometry(rad, 36, 36);
var mat = new THREE.ShaderMaterial({
uniforms: {
mixShapes: {
value: 0
}
},
vertexShader: `
uniform float mixShapes;
attribute vec3 spherePos;
void main() {
vec3 pos = clamp(position, vec3(${-side * 0.5}), vec3(${side * 0.5})); // clamp to min and max vectors
gl_Position = projectionMatrix * modelViewMatrix * vec4(pos, 1.0);
}
`,
fragmentShader: `
void main() {
gl_FragColor = vec4(1.0, 0.0, 1.0,1.0);
}
`,
wireframe: true
});
var shape = new THREE.Mesh(geom, mat);
scene.add(shape);
renderer.setAnimationLoop(() => {
renderer.render(scene, camera)
});
body {
overflow: hidden;
margin: 0;
}
<script src="https://threejs.org/build/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>

ThreeJS - Material which shows surface structure in a scene without light

In my scene I render complex objects, which have a complex surface structure. Furthermore I am not using light in my scene and I am trying to avoid it.
For now I am using the MeshNormalMaterial which shows perfectly the surface structures of my objects.
object with MeshNormalMaterial:
But I want to render certain objects with a unique color (e.g. from dark red to light red based on the surface structure/ similar to the MeshNormalMaterial).
I tried the MeshDepthMaterial for one object, but it rendered the whole object in almost color (no/ sparse color gradation) and not as expected like in this example. Independent of the camera position.
Same object from above with: MeshDepthMaterial
I am using a THREE.PerspectiveCamera with THREE.OrbitControls. Camera properties:
//camera attributes
public fieldOfView: number = 60;
public nearClippingPane: number = 0.1;
public farClippingPane: number = 50000;
Does the MeshNormalMaterial require light or why is this the case? Can I somehow amplify the depth effect of MeshNormalMaterial?
Is ist possible to restrict the RGB Colors of MeshNormalMaterial or do I have to use another Material for my purpose?
I just slightly modified the code of the fragment shader from this SO answer, so all credits to Rabbid76:
var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera(60, window.innerWidth / window.innerHeight, 0.01, 1000);
camera.position.set(0, 0, 10);
var renderer = new THREE.WebGLRenderer({
antialias: true
});
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
var controls = new THREE.OrbitControls(camera, renderer.domElement);
var colors = {
color1: "#ff0000",
color2: "#ffaaaa"
}
var geometry = new THREE.TorusKnotBufferGeometry(2, 0.5, 100, 16);
var material = new THREE.ShaderMaterial({
uniforms: {
color1: {
value: new THREE.Color(colors.color1)
},
color2: {
value: new THREE.Color(colors.color2)
}
},
vertexShader: vertShader,
fragmentShader: fragShader
});
var mesh = new THREE.Mesh(geometry, material);
scene.add(mesh);
var gui = new dat.GUI();
gui.addColor(colors, "color1").onChange(function(value) {
material.uniforms.color1.value.set(value);
});
gui.addColor(colors, "color2").onChange(function(value) {
material.uniforms.color2.value.set(value);
});
render();
function render() {
requestAnimationFrame(render);
renderer.render(scene, camera);
}
body {
overflow: hidden;
margin: 0;
}
<script src="https://threejs.org/build/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>
<script src="https://threejs.org/examples/js/libs/dat.gui.min.js"></script>
<script>
var vertShader = `
varying vec3 vNormal;
void main(void)
{
vNormal = normalMatrix * normalize(normal);
gl_Position = projectionMatrix * modelViewMatrix * vec4(position,1.0);
}
`;
var fragShader = `
uniform vec3 color1;
uniform vec3 color2;
varying vec3 vNormal;
void main(void)
{
vec3 view_nv = normalize(vNormal);
vec3 nv_color = view_nv * 0.5 + 0.5;
vec3 c = mix(color1, color2, nv_color.r);
gl_FragColor = vec4(c, 1.0);
}
`;
</script>

gpu picking - invisible pixels around sprites

I'm rendering a picking scene that contains sprites. As my cursor gets close to the sprite, it registers as a color and gets "picked". This invisible border gets larger closer you zoom into the sprites.
Open up your console to see the IDs printed in real time. Move your cursor closer and further away to large and small sprites. You'll see that sprites get selected on an invisible border. This behavior does not happen with regular geometry, just with sprites.
It's weird because I'm rendering out what renderer.readRenderTargetPixels actually sees.
How can I get rid of the invisible borders for more accurate picking?
var renderer, scene, camera, controls;
var particles, uniforms;
var PARTICLE_SIZE = 50;
var raycaster, intersects;
var mouse, INTERSECTED;
var pickingTexture;
var numOfVertices;
init();
animate();
function init() {
container = document.getElementById('container');
scene = new THREE.Scene();
camera = new THREE.PerspectiveCamera(45, window.innerWidth / window.innerHeight, 1, 10000);
camera.position.z = 150;
//
var geometry1 = new THREE.BoxGeometry(200, 200, 200, 4, 4, 4);
var vertices = geometry1.vertices;
numOfVertices = vertices.length;
var positions = new Float32Array(vertices.length * 3);
var colors = new Float32Array(vertices.length * 3);
var sizes = new Float32Array(vertices.length);
var vertex;
var color = new THREE.Color();
for (var i = 0, l = vertices.length; i < l; i++) {
vertex = vertices[i];
vertex.toArray(positions, i * 3);
color.setHex(i + 1);
color.toArray(colors, i * 3);
sizes[i] = PARTICLE_SIZE * 0.5;
}
var geometry = new THREE.BufferGeometry();
geometry.addAttribute('position', new THREE.BufferAttribute(positions, 3));
geometry.addAttribute('customColor', new THREE.BufferAttribute(colors, 3));
geometry.addAttribute('size', new THREE.BufferAttribute(sizes, 1));
//
var material = new THREE.ShaderMaterial({
uniforms: {
// texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/circle.png")}
texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/disc.png")}
},
vertexShader: document.getElementById('vertexshader').textContent,
fragmentShader: document.getElementById('fragmentshader').textContent,
depthTest: false,
transparent: false
// alphaTest: 0.9
});
//
particles = new THREE.Points(geometry, material);
scene.add(particles);
//
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true
});
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.setClearColor(0xffffff);
container.appendChild(renderer.domElement);
//
raycaster = new THREE.Raycaster();
mouse = new THREE.Vector2();
//
//
window.addEventListener('resize', onWindowResize, false);
document.addEventListener('mousemove', onDocumentMouseMove, false);
// defaults are on the right (except minFilter)
var options = {
format: THREE.RGBAFormat, // THREE.RGBAFormat
type: THREE.UnsignedByteType, // THREE.UnsignedByteType
anisotropy: 1, // 1
magFilter: THREE.LinearFilter, // THREE.LinearFilter
minFilter: THREE.LinearFilter, // THREE.LinearFilter
depthBuffer: true, // true
stencilBuffer: true // true
};
pickingTexture = new THREE.WebGLRenderTarget(window.innerWidth, window.innerHeight, options);
pickingTexture.texture.generateMipmaps = false;
controls = new THREE.OrbitControls(camera, container);
controls.damping = 0.2;
controls.enableDamping = false;
}
function onDocumentMouseMove(e) {
// event.preventDefault();
mouse.x = e.clientX;
mouse.y = e.clientY;
}
function onWindowResize() {
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(window.innerWidth, window.innerHeight);
}
function animate() {
requestAnimationFrame(animate);
controls.update();
render();
}
function render() {
pick();
renderer.render(scene, camera);
}
function pick() {
renderer.render(scene, camera, pickingTexture);
//create buffer for reading single pixel
var pixelBuffer = new Uint8Array(4);
//read the pixel under the mouse from the texture
renderer.readRenderTargetPixels(pickingTexture, mouse.x, pickingTexture.height - mouse.y, 1, 1, pixelBuffer);
//interpret the pixel as an ID
var id = ( pixelBuffer[0] << 16 ) | ( pixelBuffer[1] << 8 ) | ( pixelBuffer[2] );
if (id <= numOfVertices) console.log(id);
}
body {
color: #ffffff;
background-color: #000000;
margin: 0px;
overflow: hidden;
}
<script src="http://threejs.org/build/three.min.js"></script>
<script src="http://threejs.org/examples/js/controls/OrbitControls.js"></script>
<script type="x-shader/x-fragment" id="fragmentshader">
uniform sampler2D texture;
varying vec3 vColor;
void main() {
// solid squares of color
gl_FragColor = vec4( vColor, 1.0 );
}
</script>
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) );
gl_Position = projectionMatrix * mvPosition;
}
</script>
<div id="container"></div>
The problem is you're on a device that has a devicePixelRatio != 1.0 and three.js lying about the size.
Because you called renderer.setPixelRatio now magic happens behind the scenes. Your canvas is not the size you requested it's some other size based on some formula hidden in the three.js code.
So, what happens. Your canvas is one size but your render target is a different size. Your shader uses gl_PointSize to draw its points. That size is in device pixels. Because your render target is a different size the size of the points are different in your render target than they are on screen.
Remove the call to render.setPixelRatio and it will start working.
IMO the correct way to fix this is to use devicePixelRatio yourself because that way everything that is happening is 100% visible to you. No magic happening behind the scenes.
So,
Get rid of the container and use a canvas directly
<canvas id="c"></canvas>
set the canvas to use 100vw for width, 100vh for height and made the body margin: 0;
canvas { width: 100vw; height: 100vh; display: block; }
body { margin: 0; }
This will make your canvas stretch automatically to fill the window.
Use the size the browser stretched the canvas to choose the size its drawingBuffer should be and multiply by devicePixelRatio. That assumes you actually want to support device pixel ratio. No need to do this twice so following D.R.Y. so just do it in onWindowResize.
canvas = document.getElementById("c");
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true,
canvas: canvas,
});
pickingTexture = new THREE.WebGLRenderTarget(1, 1, options);
onWindowResize();
...
function onWindowResize() {
var width = canvas.clientWidth * window.devicePixelRatio;
var height = canvas.clientHeight * window.devicePixelRatio;
camera.aspect = width / height;
camera.updateProjectionMatrix();
renderer.setSize(width, height, false); // YOU MUST PASS FALSE HERE otherwise three.js will muck with the CSS
pickingTexture.setSize(width, height);
}
Convert the mouse coordinates into device coordinates
renderer.readRenderTargetPixels(
pickingTexture,
mouse.x * window.devicePixelRatio,
pickingTexture.height - mouse.y * window.devicePixelRatio,
1, 1, pixelBuffer);
Here's that solution
var renderer, scene, camera, controls;
var particles, uniforms;
var PARTICLE_SIZE = 50;
var raycaster, intersects;
var mouse, INTERSECTED;
var pickingTexture;
var numOfVertices;
var info = document.querySelector('#info');
init();
animate();
function init() {
canvas = document.getElementById('c');
scene = new THREE.Scene();
camera = new THREE.PerspectiveCamera(45, 1, 1, 10000);
camera.position.z = 150;
//
var geometry1 = new THREE.BoxGeometry(200, 200, 200, 4, 4, 4);
var vertices = geometry1.vertices;
numOfVertices = vertices.length;
var positions = new Float32Array(vertices.length * 3);
var colors = new Float32Array(vertices.length * 3);
var sizes = new Float32Array(vertices.length);
var vertex;
var color = new THREE.Color();
for (var i = 0, l = vertices.length; i < l; i++) {
vertex = vertices[i];
vertex.toArray(positions, i * 3);
color.setHex(i + 1);
color.toArray(colors, i * 3);
sizes[i] = PARTICLE_SIZE * 0.5;
}
var geometry = new THREE.BufferGeometry();
geometry.setAttribute('position', new THREE.BufferAttribute(positions, 3));
geometry.setAttribute('customColor', new THREE.BufferAttribute(colors, 3));
geometry.setAttribute('size', new THREE.BufferAttribute(sizes, 1));
//
var loader = new THREE.TextureLoader();
var material = new THREE.ShaderMaterial({
uniforms: {
// texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/circle.png")}
texture: {value: loader.load("https://i.imgur.com/iXT97XR.png")}
},
vertexShader: document.getElementById('vertexshader').textContent,
fragmentShader: document.getElementById('fragmentshader').textContent,
depthTest: false,
transparent: false
// alphaTest: 0.9
});
//
particles = new THREE.Points(geometry, material);
scene.add(particles);
//
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true,
canvas: canvas,
});
renderer.setClearColor(0xffffff);
//
raycaster = new THREE.Raycaster();
mouse = new THREE.Vector2();
//
//
window.addEventListener('resize', onWindowResize, false);
document.addEventListener('mousemove', onDocumentMouseMove, false);
// defaults are on the right (except minFilter)
var options = {
format: THREE.RGBAFormat, // THREE.RGBAFormat
type: THREE.UnsignedByteType, // THREE.UnsignedByteType
anisotropy: 1, // 1
magFilter: THREE.LinearFilter, // THREE.LinearFilter
minFilter: THREE.LinearFilter, // THREE.LinearFilter
depthBuffer: true, // true
stencilBuffer: true // true
};
pickingTexture = new THREE.WebGLRenderTarget(1, 1, options);
pickingTexture.texture.generateMipmaps = false;
controls = new THREE.OrbitControls(camera, canvas);
controls.damping = 0.2;
controls.enableDamping = false;
onWindowResize();
}
function onDocumentMouseMove(e) {
// event.preventDefault();
mouse.x = e.clientX;
mouse.y = e.clientY;
}
function onWindowResize() {
var width = canvas.clientWidth * window.devicePixelRatio;
var height = canvas.clientHeight * window.devicePixelRatio;
camera.aspect = width / height;
camera.updateProjectionMatrix();
renderer.setSize(width, height, false); // YOU MUST PASS FALSE HERE!
pickingTexture.setSize(width, height);
}
function animate() {
requestAnimationFrame(animate);
controls.update();
render();
}
function render() {
pick();
renderer.render(scene, camera);
}
function pick() {
renderer.setRenderTarget(pickingTexture);
renderer.setClearColor(0);
renderer.render(scene, camera);
renderer.setClearColor(0xFFFFFF);
renderer.setRenderTarget(null)
//create buffer for reading single pixel
var pixelBuffer = new Uint8Array(4);
//read the pixel under the mouse from the texture
renderer.readRenderTargetPixels(pickingTexture, mouse.x * window.devicePixelRatio, pickingTexture.height - mouse.y * window.devicePixelRatio, 1, 1, pixelBuffer);
//interpret the pixel as an ID
var id = ( pixelBuffer[0] << 16 ) | ( pixelBuffer[1] << 8 ) | ( pixelBuffer[2] );
//if (id > 0) console.log(id);
info.textContent = id;
}
body {
color: #ffffff;
background-color: #000000;
margin: 0;
}
canvas { width: 100vw; height: 100vh; display: block; }
#info { position: absolute; left: 0; top: 0; color: red; background: black; padding: 0.5em; font-family: monospace; }
<script src="https://threejs.org/build/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>
<script type="x-shader/x-fragment" id="fragmentshader">
uniform sampler2D texture;
varying vec3 vColor;
void main() {
// solid squares of color
gl_FragColor = vec4( vColor, 1.0 );
}
</script>
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) );
gl_Position = projectionMatrix * mvPosition;
}
</script>
<canvas id="c"></canvas>
<div id="info"></div>
Note a few other things.
I'd guess you really want to clear the picking texture to zero instead of white. That way 0 = nothing there, anything else = something there.
renderer.setClearColor(0);
renderer.render(scene, camera, pickingTexture);
renderer.setClearColor(0xFFFFFF);
No idea what the id <= numOfVertices means
So given that it's clearing to zero now the code is just
if (id) console.log(id);
I don't set the renderer size, the pickingTexture size nor the camera aspect at init time.
Why repeat myself. onWindowResize already sets it
You need to resize the pickingTexture render target when the canvas is resizes so it matches in size.
I removed most references to window.innerWidth and window.innerHeight
I would have removed all of them but I didn't want to change even more code for this example. Using window.innerWidth ties the code to the window. If you ever want to use the code in something that's not the fullsize of the window, for example lets say you make an editor. You'll have to change the code.
It's not any harder to write the code in a way that works in more situations so why make more work for yourself later.
Other solutions I didn't chose
You could call render.setPixelRatio and then set the pickingTexture render target's size with window.devicePixelRatio
I didn't pick this solution because you have to guess what three.js is doing behind the scenes. Your guess might be correct today but wrong tomorrow. It seems better if you tell three.js make something width by height it should just make it width by height and not make it something else. Similarly you'd have to guess when three.js is going to apply pixelRatio and when it's not. As you noticed above it doesn't apply it to the size of the render target and it can't because it doesn't know what your purpose is. Are you making a render target for picking? For a fullscreen effect? For capture? for a non-fullscreen effect? Since it can't know it can't apply the pixelRatio for you. This happens all over the three.js code. Some places it applies pixelRatio, other places it doesn't. You're left guessing. If you never set pixelRatio that problem disappears.
You could pass in devicePixelRatio into your shader
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
uniform float devicePixelRatio; // added
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) ) * devicePixelRatio;
gl_Position = projectionMatrix * mvPosition;
}
</script>
and of course you'd need to set devicePixelRatio in your uniforms.
I might pick this solution. The minor problem is if the pickingTexture is not the same resolution as the canvas's backbuffer you can get off by 1 errors. In this case if the canvas was 2x the pickingTexture then 3 of every 4 pixels in the canvas don't exist in the pickingTexture. Depending on your application that might be ok though. You can't pick 1/2 pixels, at least not with the mouse.
Another other reason I would probably not pick this solution is it just leaves the issue to pop up other places. lineWidth is one, gl_FragCoord is another. So are the viewport and scissor settings. It seems better to make the render target size match that canvas so that everything is the same rather than make more and more workarounds and have to remember where to use one size vs another. Tomorrow I start using the PointsMaterial. It also has issues with devicePixelRatio. By not calling renderer.setPixelRatio those problems go away.

Resources