Sphere Impostor on ThreeJS - three.js

I am developing a sphere impostor shader on GLSL with ThreeJS. My algorithm is based on the publication from Sigg et al. named "GPU-Based Ray-Casting of Quadratic Surfaces".
When using a classic geometry approach, you need dozens or even hundreds triangles to represent each sphere. It may cause memory overload if you need to show thousands of spheres. The sphere impostor allows you to store only positions and radius on the geometry to show a sphere, giving much more performance than the previous technique.
For now, I succeeded to develop the shader, even by using ThreeJS shader chunks to ensure a full ThreeJS compatibility. You can find a demo page here. However, there is a last thing not working on this implementation.
When moving the objects on the scene, it seems that the object using the sphere impostor is delayed compared to a normal mesh. You can also notice that some times, the spheres are "cut" like on this picture.
This second bug makes me think that the sprite is nicely placed into the scene by the vertex shader but the fragment shader is computing wrong coordinates. I suspect two pieces of code where the problem could be :
Two varyings provided by the vertex shader to the fragment shader that should give the same value for each pixel of a sprite. I don't know how to verify this.
varying float projMatrix11;
varying float projMatrix22;
I don't know if I'm doing well to update my shader uniforms
group.traverse(function(o) {
if (!o.material) { return; }
var u = o.material.uniforms;
if (!u) { return; }
modelViewMatrixInverse.getInverse(
o.modelViewMatrix
);
if (u.projectionMatrixInverse) {
u.projectionMatrixInverse.value = projectionMatrixInverse;
}
if (u.projectionMatrixTranspose) {
u.projectionMatrixTranspose.value = projectionMatrixTranspose;
}
if (u.modelViewMatrixInverse) {
u.modelViewMatrixInverse.value = modelViewMatrixInverse;
}
if (u.viewport) {
u.viewport.value = viewport;
}
});
I wasn't able to debug the problem and hope someone knowing better ThreeJS than I can give me some clues about it.
I really hope we can solve this problem, so we may be able to propose this feature to the whole community of ThreeJS ;)
Note : I delayed the calls of requestAnimationFrame for you to facilitate debugging
EDIT : After digging more, the problem may come from how I'm updating custom uniforms. One of it uses the modelViewMatrix to get it's inverse. But the modelViewMatrix is updated only during the render call of the WebGLRenderer, so the frame delay may come from there. How can I update a uniform which is depending to other uniforms and keep them synchronized on ThreeJS ?

Answer found alone, I will explain it here if someone encounters the same trouble.
The problem is that I was updating modelViewMatrixInverse uniform by using the modelViewMatrix provided by ThreeJS. This uniform is only updated during the call of render() method of the WebGLRenderer and my modelViewMatrixInverse was one frame late at each render call. That's why my custom shader was everytime one frame late than ThreeJS native shaders.

Related

ThreeJS Points (Point Cloud) with Lighting using custom Shader Material

Coded using:
Using ThreeJS v0.130.1
Framework: Angular 12, but that's not relevant to the issue.
Testing on Chrome browser.
I am building an application that gets more than 100K points. I use these points to render a THREE.Points object on the screen.
I found that default THREE.PointsMaterial does not support lighting (the points are visible with or without adding lights to the scene).
So I tried to implement a custom ShaderMaterial. But I could not find a way to add lighting to the rendered object.
Here is a sample of what my code is doing:
Sample App on StackBlitz showing my current attempt
In this code, I am using sample values for point cloud data, normals and color but everything else is similar to my actual application. I can see the 3D object, but need more proper lighting using normals.
I need help or guidance to implement the following:
Add lighting to custom shader material. I have Googled and tried many things, no success so far.
Using normals, show the effects of lighting (In this sample code, the normals are fixed to Y-axis direction, but I am calculating them based on some vector logic in actual application). So calculating normals is already done, but I want to use them to show light shine/shading effect in the custom shader material.
And in this sample, color attribute is set to fixed red color, but in actual application I am able to apply colors using UV range from a texture to color attribute.
Please advise how/if I can get lighting based on normals for Point Cloud. Thanks.
Note: I looked at this Stackoveflow question but it only deals with changing the alpha/transparency of points and not lighting.
Adding lighting to a custom material is a very complex process. Especially since you could use Phong, Lambert, or Physical lighting methods, and there's a lot of calculations that need to pass from the vertex to the fragment shader. For instance, this segment of shader code is just a small part of what you'd need.
Instead of trying to re-create lighting from scratch, I recommend you create a PlaneGeometry with the material you'd like (Phong, Lambert, Physical, etc...) and use an InstancedMesh to create thousands of instances, just like in this example.
Based on that example, the pseudo-code of how you could achieve a similar effect is something like this:
const count = 100000;
const geometry = new PlaneGeometry();
const material = new THREE.MeshPhongMaterial();
mesh = new THREE.InstancedMesh( geometry, material, count );
mesh.instanceMatrix.setUsage( THREE.DynamicDrawUsage ); // will be updated every frame
scene.add( mesh );
const dummy = new THREE.Object3D();
update() {
// Sets the rotation so it's always perpendicular to camera
dummy.lookAt(camera);
// Updates positions of each plane
for (let i = 0; i < count; i++){
dummy.position.set( x, y, z );
dummy.updateMatrix();
mesh.setMatrixAt( i ++, dummy.matrix );
}
}
The for() loop would be the most expensive part of each frame, so if you need to update it on each frame, you might want to calculate this in the vertex shader, but that's another question altogether.

How can I light emission per vertex and per vertex lighting in ThreeJS?

I want to see a chart with color specified per vertex and to get little bit of shading too.
But if I use MeshBasicMaterial I only get VertexColor with no dynamic shading.
On the other hand, if I use MeshPhongMaterial I just get shading but without emissiveness from my vertex colors.
As the THREE.JS PhongMaterial supports vertexColors, giving you a nice combination of dynamic lighting and vertex colors, I'm not quite sure I understand your question. Perhaps that is something you should investigate more?
However, as an alternative to writing a custom shader you could try rendering your model in multiple passes.
This will not give you as much control over the way the vertex colors and phong lighting are combined as a shader would, but often a simple add/multiply blend can give pretty decent results.
Algorithm:
- create two meshes for the BufferGeometry, one with the BasicMaterial and one with the PhongMaterial
- for the PhongMaterial, set
depthFunc = THREE.EqualDepth
transparent = true;
blending = THREE.AdditiveBlending(or MultiplyBlending)
- render the first mesh
- render the second mesh at the exact same spot

Fixed texture size in Three.js

I am building quite a complex 3D environment in Three.js (FPS-a-like). For this purpose I wanted to structure the loading of textures and materials in an object oriƫnted way. For example; materials.wood.brownplank is a reusable material with a certain texture and other properties. Below is a simplified visualisation of the process where models uses materials and materials uses textures.
loadTextures();
loadMaterials();
loadModels();
//start doing stuff in the scene
I want to use that material on differently sized objects. However, in Three.js you can't (AFAIK) set a certain texture scale. You will have to set the repeat to scale it appropiate to your object. But I don't want to do that for every plane of every object I use.
Here is how it looks now
As you can see, the textures are not uniform in size.
Is there an easy way achieve this? So cloning the texture and/or material every time and setting the repeat according to the geometry won't do :)
I hope someone can help me.
Conclusion:
There is no real easy way to do this. I ended up changing my loading methods, where things like materials.wood.brownplank are now for example getMaterial('wood', 'brownplank') In the function new objects are instantiated
You should be able to do this by modifying your geometry UV coordinates according to the "real" dimensions of each face.
In Three.js, UV coordinates are relative to the face and texture (as in, 0.0 = one edge, 1.0 = other edge), no matter what the actual size of texture or face is. But by modifying the UVs in geometry (multiply them by some factor based on face physical size), you can use the same material and texture in different sizes (and orientations) per face.
You just need to figure out the mapping between UVs, geometry scale and your desired working units (eg. mm or m). Sorry I don't have, or know a ready algorithm to do it, but that's the approach you probably need to take. Should be quite doable with a bit of experimentation and google-fu.

three.js - Adjusting opacity of individual particles

I am trying to vary the opacity of particles as a function of their distance from a plane.
This issue describes my problem, and the answer a year ago was essentially "you can't". Opacity is apparently a parameter of a material, not an element, and hence individual particle opacity is not possible.
Has anything changed, is there any way I could achieve this? If individual particle colouring is possible, I imagine this isn't out of reach.
Cheers
EDIT - This answer shows how to set per-point opacity using a custom ShaderMaterial. See https://stackoverflow.com/a/67892506/1461008 for an approach using PointsMaterial.
ParticleSystem has been renamed to PointCloud and then to Points.
Yes, you can create a Point Cloud and vary the alpha value of each particle's color dynamically.
In three.js, you can do this by setting the Point Cloud's material to be a ShaderMaterial having an attribute equal to the desired alpha value for each particle.
If ShaderMaterials, vertex shaders and fragment shaders are new to you, here is a really simple Fiddle that implements a Point Cloud with dynamic alphas: https://jsfiddle.net/9Lvrnpwc/.
EDIT: Updated fiddle
three.js r.148
Not sure why, but proposed solution didn't work for me. I used somewhat tricky shading to make points round and blurry at edges. So the corners of points were supposed to be transparent, but they appeared black: http://jsfiddle.net/5kz64ero/1/
Relevant part of my fragment shader:
// Distance from 0.0 to 0.5 from the center of the point
float d = distance(gl_PointCoord, vec2(0.5, 0.5));
// Applying sigmoid to smoothen the edge
float opacity = 1.0 / (1.0 + exp(16.0 * (d - 0.25)));
gl_FragColor = vec4(opacity * vColor, opacity);
I figured that traditionally this is solved by depth-sorting (with farthest points coming first), and I found some evidence that some older implementations of ParticleSystem in Three contained sortParticles attribute. But it's not there anymore. And in my case sorting would really involve redoing that every time camera position changes. Instead I set depthWrite: false and it seems to solve the issue.
The result: http://jsfiddle.net/5kz64ero/6/

Efficiently sending values to GLSL shader

I am trying to write my particle system for OpenGL ES 2.0. Each particle is made up of 4 vertexes, forming the little square where a transparent texture is drawn.
The problem is: each particle has its own properties (color, position, size), that are constant across the 4 vertexes of that particle. The only variation for each vertex is what corner of the square it is.
If I am to send the properties of the particle via uniform variables, I must do:
for(each particle) { // do maaaany times
glUniform*(...);
glDrawArray(...); // only draw 4 vertexes
};
this is clearly inefficient, since I will only draw 4 vertexes per glDrawArray call.
If I send this properties via attribute variables, I must fill the same information 4 times for each fragment in the attribute buffer:
struct particle buf[n];
for(each particle) {
struct particle p;
p = ...; // Update particle
buf[i+0] = buf[i+1] = buf[i+2] = buf[i+3] = p;
};
glBufferData(..., buf, ...);
// then draw everithing once afterwards...
what is memory inefficient and seems very ugly to me. So what is the solution to this problem? What is the right way to pass parameters that change for each few vertexes to the shader?
Use point sprites. The introduction is very explicit about how to solve your problem.
You can also combine the use of point sprites with another extension, point_size_array.
...
As Christian Rau has commented, the point_size_array is no more usefull using programmable pipeline: set the maximum point size as usual, then discard fragments basing on their distance from the point center, derived from texture coordinates generated by OpenGL. The particle size shall be sent via additional attribute.
GL ES doesn't really have a good solution to this. Desktop OpenGL allows for instancing and various other tricks, but ES just doesn't have those.
You can use a Uniform Buffer Object. Note that this feature is only available on D3D10+ hardware.
Send the information via a texture. I'm not sure that texture sampling is supported in opengl-es 2.0 vertex shaders, but if it is, then that would be optimal.

Resources