Basic, I can't get raycasting to work with them. My guess is my matrix coordinate calculation method is wrong. Don't know how to do it right.
I set vertex position and offset in vertexShader, and in InstancedMesh, I set the same offset, expecting the the raycast can get the an instanceID, but nothing intersects. You can find my entire code here.
I tried to adapt an official raycasting example here, but can't figure out where I did wrong. My hodgepodge uses: InstancedMesh, InstancedBufferGeometry, custom shader together. My objective is to learn how it works.
My question is where I did wrong?
My vertex shader:
precision highp float;
uniform mat4 modelViewMatrix;
uniform mat4 projectionMatrix;
attribute vec3 position;
attribute vec4 color;
attribute vec3 offset;
varying vec3 vPosition;
varying vec4 vColor;
void main() {
vColor = vec4(color);
vPosition = offset*1.0 + position;
gl_Position = projectionMatrix * modelViewMatrix * vec4( vPosition, 1.0 );
// if gl_Position not set, nothing is shown
}
My InstancedMesh matrix setting:
for(let i = 0; i < SQUARE_COUNT; i++) {
transform.position.set(offsets[i], offsets[i+1], offsets[i+2] )
transform.updateMatrix()
mesh.setMatrixAt(i, transform.matrix)
}
The offsets is set before as following:
for(let i = 0; i < SQUARE_COUNT; i++ ) {
offsets.push( 0 + i*0.05, 0 + i*0.05, 0 + i*0.05); // same is set in InstancedMesh
colors.push( Math.random(), Math.random(), Math.random(), Math.random() );
}
The raycaster has no awareness of any nonstandard transformation that you do in your vertex shader. It's just the way it works. It has no way of knowing that you are doing:
vPosition = offset*1.0 + position;
in your shader.
It works by assuming that you are running the bog standard vertex shader with no additional transforms. It assumes that every object you are casting against has a well defined/computed bounding box as well.
If you are going to use raycasting, you may have to make a non-rendered scene that represents your objects in their final rendered positions, and cast against that.
Related
I have lots of models that ain't unwrapped (they don't have UV coordinates). They are quite complex to unwrap them. Thus, I decided to texture them using a seamless cubemap:
[VERT]
attribute vec4 a_position;
varying vec3 texCoord;
uniform mat4 u_worldTrans;
uniform mat4 u_projTrans;
...
void main()
{
gl_Position = u_projTrans * u_worldTrans * a_position;
texCoord = vec3(a_position);
}
[FRAG]
varying vec3 texCoord;
uniform samplerCube u_cubemapTex;
void main()
{
gl_FragColor = textureCube(u_cubemapTex, texCoord);
}
It works, but the result is quite weird due to texturing depends on the vertices position. If my model is more complex than a cube or sphere, I see visible seams and low resolution of the texture on some parts of the object.
Reflection is mapped good on the model, but it has a mirror effect.
Reflection:
[VERT]
attribute vec3 a_normal;
varying vec3 v_reflection;
uniform mat4 u_matViewInverseTranspose;
uniform vec3 u_cameraPos;
...
void main()
{
mat3 normalMatrix = mat3(u_matViewInverseTranspose);
vec3 n = normalize(normalMatrix * a_normal);
//calculate reflection
vec3 vView = a_position.xyz - u_cameraPos.xyz;
v_reflection = reflect(vView, n);
...
}
How to implement something like a reflection, but with “sticky” effect, which means that it’s as if the texture is attached to a certain vertex (not moving). Each side of the model must display its own side of the cubemap, and as a result it should look like a common 2D texturing. Any advice will be appreciated.
UPDATE 1
I summed up all comments and decided to calculate cubemap UV. Since I use LibGDX, some names may differ from OpenGL ones.
Shader class:
public class CubemapUVShader implements com.badlogic.gdx.graphics.g3d.Shader {
ShaderProgram program;
Camera camera;
RenderContext context;
Matrix4 viewInvTraMatrix, viewInv;
Texture texture;
Cubemap cubemapTex;
...
#Override
public void begin(Camera camera, RenderContext context) {
this.camera = camera;
this.context = context;
program.begin();
program.setUniformMatrix("u_matProj", camera.projection);
program.setUniformMatrix("u_matView", camera.view);
cubemapTex.bind(1);
program.setUniformi("u_textureCubemap", 1);
texture.bind(0);
program.setUniformi("u_texture", 0);
context.setDepthTest(GL20.GL_LEQUAL);
context.setCullFace(GL20.GL_BACK);
}
#Override
public void render(Renderable renderable) {
program.setUniformMatrix("u_matModel", renderable.worldTransform);
viewInvTraMatrix.set(camera.view);
viewInvTraMatrix.mul(renderable.worldTransform);
program.setUniformMatrix("u_matModelView", viewInvTraMatrix);
viewInvTraMatrix.inv();
viewInvTraMatrix.tra();
program.setUniformMatrix("u_matViewInverseTranspose", viewInvTraMatrix);
renderable.meshPart.render(program);
}
...
}
Vertex:
attribute vec4 a_position;
attribute vec2 a_texCoord0;
attribute vec3 a_normal;
attribute vec3 a_tangent;
attribute vec3 a_binormal;
varying vec2 v_texCoord;
varying vec3 v_cubeMapUV;
uniform mat4 u_matProj;
uniform mat4 u_matView;
uniform mat4 u_matModel;
uniform mat4 u_matViewInverseTranspose;
uniform mat4 u_matModelView;
void main()
{
gl_Position = u_matProj * u_matView * u_matModel * a_position;
v_texCoord = a_texCoord0;
//CALCULATE CUBEMAP UV (WRONG!)
//I decided that tm_l2g mentioned in comments is u_matView * u_matModel
v_cubeMapUV = vec3(u_matView * u_matModel * vec4(a_normal, 0.0));
/*
mat3 normalMatrix = mat3(u_matViewInverseTranspose);
vec3 t = normalize(normalMatrix * a_tangent);
vec3 b = normalize(normalMatrix * a_binormal);
vec3 n = normalize(normalMatrix * a_normal);
*/
}
Fragment:
varying vec2 v_texCoord;
varying vec3 v_cubeMapUV;
uniform sampler2D u_texture;
uniform samplerCube u_textureCubemap;
void main()
{
vec3 cubeMapUV = normalize(v_cubeMapUV);
vec4 diffuse = textureCube(u_textureCubemap, cubeMapUV);
gl_FragColor.rgb = diffuse;
}
The result is completely wrong:
I expect something like that:
UPDATE 2
The texture looks stretched on the sides and distorted in some places if I use vertices position as a cubemap coordinates in the vertex shader:
v_cubeMapUV = a_position.xyz;
I uploaded euro.blend, euro.obj and cubemap files to review.
that code works only for meshes that are centered around (0,0,0) if that is not the case or even if (0,0,0) is not inside the mesh then artifacts occur...
I would start with computing BBOX BBOXmin(x0,y0,z0),BBOXmax(x1,y1,z1) of your mesh and translate the position used for texture coordinate so its centered around it:
center = 0.5*(BBOXmin+BBOXmax);
texCoord = vec3(a_position-center);
However non uniform vertex density would still lead to texture scaling artifacts especially if BBOX sides sizes differs too much. Rescaling it to cube would help:
vec3 center = 0.5*(BBOXmin+BBOXmax); // center of BBOX
vec3 size = BBOXmax-BBOXmin; // size of BBOX
vec3 r = a_position-center; // position centered around center of BBOX
r.x/=size.x; // rescale it to cube BBOX
r.y/=size.y;
r.z/=size.z;
texCoord = r;
Again if the center of BBOX is not inside mesh then this would not work ...
The reflection part is not clear to me do you got some images/screenshots ?
[Edit1] simple example
I see it like this (without the center offsetting and aspect ratio corrections mentioned above):
[Vertex]
//------------------------------------------------------------------
#version 420 core
//------------------------------------------------------------------
uniform mat4x4 tm_l2g;
uniform mat4x4 tm_g2s;
layout(location=0) in vec3 pos;
layout(location=1) in vec4 col;
out smooth vec4 pixel_col;
out smooth vec3 pixel_txr;
//------------------------------------------------------------------
void main(void)
{
pixel_col=col;
pixel_txr=(tm_l2g*vec4(pos,0.0)).xyz;
gl_Position=tm_g2s*tm_l2g*vec4(pos,1.0);
}
//------------------------------------------------------------------
[Fragment]
//------------------------------------------------------------------
#version 420 core
//------------------------------------------------------------------
in smooth vec4 pixel_col;
in smooth vec3 pixel_txr;
uniform samplerCube txr_skybox;
out layout(location=0) vec4 frag_col;
//------------------------------------------------------------------
void main(void)
{
frag_col=texture(txr_skybox,pixel_txr);
}
//------------------------------------------------------------------
And here preview:
The white torus in first few frames are using fixed function and the rest is using shaders. As you can see the only input I use is the vertex position,color and transform matrices tm_l2g which converts from mesh coordinates to global world and tm_g2s which holds the perspective projection...
As you can see I render BBOX with the same CUBE MAP texture as I use for rendering the model so it looks like cool reflection/transparency effect :) (which was not intentional).
Anyway When I change the line
pixel_txr=(tm_l2g*vec4(pos,0.0)).xyz;
into:
pixel_txr=pos;
In my vertex shader the object will be solid again:
You can combine both by passing two texture coordinate vectors and fetching two texels in fragment adding them with some ratio together. Of coarse you would need to pass 2 Cube map textures one for object and one for skybox ...
The red warnings are from my CPU side code reminding me that I am trying to set uniforms that are not present in the shaders (as I did this from the bump mapping example without changing CPU side code...)
[Edit1] here preview of your mesh with offset
The Vertex changes a bit (just added the offsetting described in the answer):
//------------------------------------------------------------------
#version 420 core
//------------------------------------------------------------------
uniform mat4x4 tm_l2g;
uniform mat4x4 tm_g2s;
uniform vec3 center=vec3(0.0,0.0,2.0);
layout(location=0) in vec3 pos;
layout(location=1) in vec4 col;
out smooth vec4 pixel_col;
out smooth vec3 pixel_txr;
//------------------------------------------------------------------
void main(void)
{
pixel_col=col;
pixel_txr=pos-center;
gl_Position=tm_g2s*tm_l2g*vec4(pos,1.0);
}
//------------------------------------------------------------------
So by offsetting the center point you can get rid of the singular point distortion however as I mentioned in comments for arbitrary meshes there will be always some distortions with cheap texturing tricks instead of proper texture coordinates.
Beware my mesh was resized/normalized (sadly I do not remeber if its <-1,+1> range or different ona and too lazy to dig in my source code of the GLSL engine I tested this in) so the offset might have different magnitude in your environment to achieve the same result.
I have been working on implementing SSAO into the engine I am writing, and a major problem has arrived. Everything was going quite well until I realized that my SSAO was not working correctly. There are two things that I can find that are wrong with my SSAO and I am unable to figure out how to remedy them.
My shader code is at the end of this post, before that I will be describing the problems with images.
Firstly, as seen in the below screenshot, there are some wierd artifacts showing up based on the angle of viewing. So far I am assuming the way I am implementing the View matrix is wrong. I have done a lot of research about how this all should work and I understand it in theory. However, in practice things are not changing as I would expect.
Secondly, whenever I get close to the blocks, I get very odd triangle shadows that appear around the edges of the screen, as shown in the next screenshot.
[![Odd triangle shadows around screen][2]][2]
These two images show the main issues I am having. I am using a deferred type Renderer to render the geometry to a few textures (Position, normals, color) the importing these textures and using them to manipulate the final output. The first two codeblocks are the vertex and fragment shaders respectively for translating the geometry to textures.
Vertex Shader
#version 430 core
layout(location=0) in mat4 modelMatrix;
layout(location=4) in vec4 VertexPosition;
layout(location=5) in vec4 VertexNormal;
layout(location=6) in vec3 VertexColor;
layout(location=7) in vec2 TextureCoords;
out vec4 vNormal;
out vec3 vColor;
out vec4 shaderCoord;
out vec2 texCoords;
layout(location=8) uniform mat4 V;
layout(location=12) uniform mat4 P;
void main()
{
shaderCoord = (V*modelMatrix * VertexPosition);
mat4 normalMatrix = transpose(inverse(V*modelMatrix));
vNormal = (normalMatrix*VertexNormal);
texCoords = TextureCoords;
vColor = VertexColor;
gl_Position = P*shaderCoord;
}
Fragment Shader
#version 430 core
in vec4 vNormal;
in vec3 vColor;
in vec4 shaderCoord;
in vec2 texCoords;
layout (location=0) out vec4 NormalBuffer;
layout (location=1) out vec4 ColorBuffer;
layout (location=2) out vec4 PositionBuffer;
layout (location=3) out vec4 TextureCoordBuffer;
out float fragDepth;
//Start of the main function.
void main()
{
NormalBuffer = vec4(normalize(vNormal).xyz, 1.0);
ColorBuffer = vec4(vColor, 1.0);
PositionBuffer = vec4(shaderCoord.xyz, 1.0);
TextureCoordBuffer = vec4(texCoords, 0.0, 1.0);
fragDepth = gl_FragCoord.z;
}
As you can see, I am translating everything from world space to view space before I write them to the textures. I would much prefer to keep them in world space but when I do, the entire screen looks white with occasional hints of shadows, but the background swaps between white and black depending on camera angle.
Next are my SSAO shaders, In order to implement these I followed a few tutorials, so they probably look familiar. If the tutorial was correct, the next two shaders should work correctly but they are not.
Vetex shader that just creates a quad, and applies the final texture to it.
#version 430 core
layout (location=0) in vec3 VertexPosition;
layout (location=1) in vec2 TextureCoords;
out vec2 texCoords;
void main (){
texCoords = TextureCoords;
gl_Position = vec4(VertexPosition, 1.0);
}
Fragment shader for SSAO
#version 430 core
in vec2 texCoords;
layout (location=0) out vec4 fColor;
uniform sampler2D NormalBuffer;
uniform sampler2D positionBuffer;
uniform sampler2DArrayShadow shadowMap;
uniform sampler1D SSAOKernelMap;
uniform sampler2D SSAONoiseMap;
layout(location=12) uniform mat4 P;
layout(location=8) uniform mat4 V;
uniform uint kernelSize;
uniform vec2 windowSize;
//Define Variables for SSAO Processing.
float radius = 0.5;
float SSAOBias = 0.025;
float power = 1.5;
//mat4 biasMatrix = mat4(0.5,0.0,0.0,0.0,0.0,0.5,0.0,0.0,0.0,0.0,0.5,0.0,0.5,0.5,0.5,1.0);
void main()
{
//Retrieve from textures
vec3 shaderCoord = (texture(positionBuffer, texCoords)).xyz;
vec3 vNormal = normalize((texture(NormalBuffer, texCoords)).rgb);
//process SSAO
vec2 NoiseScale = vec2(windowSize.x/4.0, windowSize.y/4.0);
vec3 randVec = normalize(texture(SSAONoiseMap, texCoords*NoiseScale).xyz);
vec3 tangent = normalize(randVec - vNormal * dot(randVec, vNormal));
vec3 bitTangent = cross(vNormal, tangent);
mat3 TBN = mat3(tangent, bitTangent, vNormal);
//Begin Processing of SSAO with inputed Kernel Samples
float Occlusion = 0.0;
for(int i=0; i<kernelSize; i++){
vec4 kernelSample = texture(SSAOKernelMap, i);
vec3 TSample = TBN*kernelSample.rgb;
TSample = shaderCoord + TSample * radius;
vec4 newCoord = vec4(TSample, 1.0);
newCoord = P*newCoord;
newCoord.xyz /= newCoord.w;
newCoord.xyz = newCoord.xyz * 0.5 + 0.5;
float sampleDepth = texture(positionBuffer,newCoord.xy).z;
//float rangeCheck = smoothstep(0.0,1.0, radius / abs(shaderCoord.z-sampleDepth));
Occlusion += (sampleDepth >= TSample.z+SSAOBias?1.0:0.0);
}
Occlusion = 1.0 - (Occlusion/kernelSize);
fColor = vec4(vec3(Occlusion),1.0f);
}
That is all the information I can think to provide initially. Any help you guys can provide would be immensely helpful! If any other information would help, please let me know and I will be happy to provide.
EDIT:
I figured out that one of my issues was the way that I was accessing the 1D texture above. This made all the kernel samples very strange. I fixed that and now I am getting something like the image below, where half the screen is darker and half the screen is lighter on one side and darker on the other. The contrast line moves with the camera.
Any help with this issue would be immensely appreciated!
I have found two things that were wrong that mostly resolved the issue that this current post is about.
Firstly, the format which I was passing in the kernelMap was off and so all the values were quite skewed.
Secondly, I was unable to figure out why but when I passed the position and normal values to the Lightingfragment shader in world space and then applied the view and projection matrices to them, they would turn out very strangely. However if I applied the view and projection matrices to the position and normal values in the BaseGeometry shader, then reverted that application in the Lighting shader everything works perfectly.
If i find out any more information I will happily post here to update any future searchers.
I'm implementing the xToon shader(pdf) in glsl to use as a shader with Three.js.
I'm getting some rendering artifacts, and I think the problem is due to webgl strangeness that I am not knowledgable about, perhaps relating to a Nan or Inf or something... I'm pulling my hair out.
I'll include the complete fragment and vertex shaders below, but I think this is the offending code located in the fragment shader:
....
vec3 n = normalize(vNormal);
vec3 l = normalize(lightDir);
float d = dot(n, l) * 0.5 + 0.5;
//vec2 texLookUp = vec2(d, loa);
vec2 texLookUp = vec2(d, 0.055);
vec4 dColor = texture2D(texture, texLookUp);
gl_FragColor = dColor;
....
To the best of my debugging efforts there seems to be some problem with using the value d as a component of the texture look up vector. This code produces these strange artifacts:
There shouldn't be those yellow "lines" on those contours...
As you may have noted, I'm not actually using the "loa" value in this code. For a while I thought that this problem was in the way I was calculating loa, but it seems that this bug is independent of loa.
Any help would be much appreciated!
The fragment shader:
uniform vec3 lightDir;
uniform sampler2D texture;
varying vec3 vNormal;
varying vec3 vPosition;
varying vec2 vUv;
// loa calculation for texture lookup
varying highp float loa;
void main() {
vec3 n = normalize(vNormal);
vec3 l = normalize(lightDir);
float d = dot(n, l) * 0.5 + 0.5;
//vec2 texLookUp = vec2(d, loa);
vec2 texLookUp = vec2(d, 0.055);
vec4 dColor = texture2D(texture, texLookUp);
gl_FragColor = dColor;
}
And the vertex shader:
uniform vec3 cameraPos;
uniform vec3 lightDir;
uniform vec3 focalPos;
uniform float inflate;
uniform float zmin;
uniform float r;
varying vec3 vNormal;
varying vec2 vUv;
varying float loa;
void main() {
vec3 n = normalize(normal);
// euclidiean distance to point from camera pos
float depth = length(cameraPos - position);
// 1. detail mapping correcting for perspective projection
float z = depth / zmin;
loa = 1.0 - (log2(z)/log2(r));
loa = clamp(loa, 0.055, 0.9);
vNormal = n;
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(normal * inflate + position, 1.0 );
}
I solved the problem by setting the texture to ClampToEdgeWrapping instead of RepeatWrapping. I was led to this answer by this stack overflow question:
Using floor() function in GLSL when sampling a texture leaves glitch
The solution is explained very well in this blog post:
http://webglfundamentals.org/webgl/lessons/webgl-3d-textures.html
And the functions to deal with this in THREEjs are members of the Texture and are explained in the THREEjs docs here.
Also I needed to set the min filter to Nearest to fully get rid of the artifacts.
I'm creating a glow effect for car stop lights and found a shader that makes it possible to always face the camera:
uniform vec3 viewVector;
uniform float c;
uniform float p;
varying float intensity;
void main() {
vec3 vNormal = normalize( normalMatrix * normal );
vec3 vNormel = normalize( normalMatrix * -viewVector );
intensity = pow( c - dot(vNormal, vNormel), p );
gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
}
This solution is quite simple and almost works. It reacts to camera movement and it would be great. BUT this element is a child of a car. The car itself is moving around and when it rotates the material stops pointing directly at the camera.
I don't want to use SpritePlugin or LensFlarePlugin because they slow down my game by 20fps so I'll stick to this lightweight solution.
I found a solution for Direct 3d that you have to remove rotation data from tranformation matrix, but I don't know how to do this in THREE.js
I guess that instead of adding calculations with car transformation there must be a way to simplify this shader instead.
How to simplify this shader so the material always faces the camera?
From the link below: "To do spherical billboarding, just remove all rotations by setting the identity matrix". How to do it ShaderMaterial in THREE.js?
http://www.geeks3d.com/20140807/billboarding-vertex-shader-glsl/
The problem here I think is intercepting transformation matrix from ShaderMaterial before it's passed to the shader, but I'm not sure.
Probably irrelevant but here's also fragment shader:
uniform vec3 glowColor;
varying float intensity;
void main() {
vec3 glow = glowColor * intensity;
gl_FragColor = vec4( glow, 1.0 );
}
edit: for now I found a workaround which is eliminating parent's rotation influence by setting opposite quaternion. Not perfect and it's happening in CPU not GPU
this.quaternion._x = -this.parent.quaternion._x;
this.quaternion._y = -this.parent.quaternion._y;
this.quaternion._z = -this.parent.quaternion._z;
this.quaternion._w = -this.parent.quaternion._w;
Are you looking for an implementation of billboarding? (make a 2D sprite always face camera) If so, all you need to do is this:
"vec3 billboard(vec2 v, mat4 view){",
" vec3 up = vec3(view[0][1], view[1][1], view[2][1]);",
" vec3 right = vec3(view[0][0], view[1][0], view[2][0]);",
" vec3 p = right * v.x + up * v.y;",
" return p;",
"}"
v is the offset from the center, basically the 4 vertices in a plane that faces the z-axis. Eg. (1.0, 1.0), (1.0, -1.0), (-1.0, 1.0), and (-1.0, -1.0).
Use it like so:
"vec3 worldPos = billboard(a_offset, u_view);"
// then do whatever else.
I'm trying to get this tutorial to work but I ran into two issues, one of which can be found here. The other one is the following.
For convenience this is the code that is supposed to work and here's a jsfiddle.
Vertex-shader:
uniform mat4 projectionMatrix;
uniform mat4 modelViewMatrix;
attribute vec3 position;
uniform vec3 normal;
varying vec3 vNormal;
void main() {
test = 0.5;
vNormal = normal;
gl_Position = projectionMatrix *
modelViewMatrix *
vec4(position,1.0);
}
Fragment-shader:
varying mediump vec3 vNormal;
void main() {
mediump vec3 light = vec3(0.5, 0.2, 1.0);
// ensure it's normalized
light = normalize(light);
// calculate the dot product of
// the light to the vertex normal
mediump float dProd = max(0.0, dot(vNormal, light));
// feed into our frag colour
gl_FragColor = vec4(dProd, // R
dProd, // G
dProd, // B
1.0); // A
}
The values for normal in the vertex shader or at least the values for vNormal in the fragment shader seem to be 0. The sphere that is supposed to show up stays black. As soon as I change the values for gl_FragColor manually the sphere changes colors. Can anybody tell me why this is not working?
In your vertex shader the vec3 normal should be an attribute (since each vertex has a normal) not a uniform:
attribute vec3 normal;
Here is the working version of your code.