Im trying to reduce the number of post process textures I have to draw in my scene. The end goal is to support an SSAO shader. The shader requires depth, postion and normal data. Currently I am storing the depth and normals in 1 float texture and the position in another.
I've been doing some reading, and it seems possible that you can get the position by simply using the depth stored in the normal texture. You have to unproject the x and y and multiply it by the depth value. I can't seem to get this right however and its probably due to my lack of understanding...
So currently my positions are drawn to a position texture. This is what it looks like (this is currently working correctly)
So is my new method. I pass the normal texture that stores the normal x,y and z in the RGB channels and the depth in the w. In the SSAO shader I need to get the position and so this is how im doing it:
//viewport is a vec2 of the viewport width and height
//invProj is a mat4 using camera.projectionMatrixInverse (camera.projectionMatrixInverse.getInverse( camera.projectionMatrix );)
vec3 get_eye_normal()
{
vec2 frag_coord = gl_FragCoord.xy/viewport;
frag_coord = (frag_coord-0.5)*2.0;
vec4 device_normal = vec4(frag_coord, 0.0, 1.0);
return normalize((invProj * device_normal).xyz);
}
...
float srcDepth = texture2D(tNormalsTex, vUv).w;
vec3 eye_ray = get_eye_normal();
vec3 srcPosition = vec3( eye_ray.x * srcDepth , eye_ray.y * srcDepth , eye_ray.z * srcDepth );
//Previously was doing this:
//vec3 srcPosition = texture2D(tPositionTex, vUv).xyz;
However when I render out the positions it looks like this:
The SSAO looks very messed up using the new method. Any help would be greatly appreciated.
I was able to find a solution to this. You need to multiply the ray normal by the camera far - near (I was using the normalized depth value - but you need the world depth value.)
I created a function to extract the position from the normal/depth texture like so:
First in the depth capture pass (fragment shader)
float ld = length(vPosition) / linearDepth; //linearDepth is cam.far - cam.near
gl_FragColor = vec4( normalize( vNormal ).xyz, ld );
And now in the shader trying to extract the position...
/// <summary>
/// This function will get the 3d world position from the Normal texture containing depth in its w component
/// <summary>
vec3 get_world_pos( vec2 uv )
{
vec2 frag_coord = uv;
float depth = texture2D(tNormals, frag_coord).w;
float unprojDepth = depth * linearDepth - 1.0;
frag_coord = (frag_coord-0.5)*2.0;
vec4 device_normal = vec4(frag_coord, 0.0, 1.0);
vec3 eye_ray = normalize((invProj * device_normal).xyz);
vec3 pos = vec3( eye_ray.x * unprojDepth, eye_ray.y * unprojDepth, eye_ray.z * unprojDepth );
return pos;
}
Related
I'm creating a glow effect for car stop lights and found a shader that makes it possible to always face the camera:
uniform vec3 viewVector;
uniform float c;
uniform float p;
varying float intensity;
void main() {
vec3 vNormal = normalize( normalMatrix * normal );
vec3 vNormel = normalize( normalMatrix * -viewVector );
intensity = pow( c - dot(vNormal, vNormel), p );
gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
}
This solution is quite simple and almost works. It reacts to camera movement and it would be great. BUT this element is a child of a car. The car itself is moving around and when it rotates the material stops pointing directly at the camera.
I don't want to use SpritePlugin or LensFlarePlugin because they slow down my game by 20fps so I'll stick to this lightweight solution.
I found a solution for Direct 3d that you have to remove rotation data from tranformation matrix, but I don't know how to do this in THREE.js
I guess that instead of adding calculations with car transformation there must be a way to simplify this shader instead.
How to simplify this shader so the material always faces the camera?
From the link below: "To do spherical billboarding, just remove all rotations by setting the identity matrix". How to do it ShaderMaterial in THREE.js?
http://www.geeks3d.com/20140807/billboarding-vertex-shader-glsl/
The problem here I think is intercepting transformation matrix from ShaderMaterial before it's passed to the shader, but I'm not sure.
Probably irrelevant but here's also fragment shader:
uniform vec3 glowColor;
varying float intensity;
void main() {
vec3 glow = glowColor * intensity;
gl_FragColor = vec4( glow, 1.0 );
}
edit: for now I found a workaround which is eliminating parent's rotation influence by setting opposite quaternion. Not perfect and it's happening in CPU not GPU
this.quaternion._x = -this.parent.quaternion._x;
this.quaternion._y = -this.parent.quaternion._y;
this.quaternion._z = -this.parent.quaternion._z;
this.quaternion._w = -this.parent.quaternion._w;
Are you looking for an implementation of billboarding? (make a 2D sprite always face camera) If so, all you need to do is this:
"vec3 billboard(vec2 v, mat4 view){",
" vec3 up = vec3(view[0][1], view[1][1], view[2][1]);",
" vec3 right = vec3(view[0][0], view[1][0], view[2][0]);",
" vec3 p = right * v.x + up * v.y;",
" return p;",
"}"
v is the offset from the center, basically the 4 vertices in a plane that faces the z-axis. Eg. (1.0, 1.0), (1.0, -1.0), (-1.0, 1.0), and (-1.0, -1.0).
Use it like so:
"vec3 worldPos = billboard(a_offset, u_view);"
// then do whatever else.
I am trying to visualize 2d matrices using Three js. These matrices are the states of the neurons in a neural network. The matrices are not huge (64 x 32) The values in these matrices will change and I want those new values to be displayed in the visualization.
For the 2d matrix I want a plane of neurons.
I have tried creating a particle system using a plane geometry with as many vertices as neurons in the data matrix.
var width = 32;
var height = 64;
var planeGeometry = new THREE.PlaneGeometry( width, height, width - 1 , height - 1 );
var particlePlane = new THREE.ParticleSystem( planeGeometry, shaderMaterial );
In the fragment shader each particle is given a base texture (a white circle)
gl_FragColor = texture2D(baseTexture, gl_PointCoord);
And then I use a second texture containing the data matrix values (greyscale pixel values) to modify each base texture.
// Sets particle texture to desired color
// vertexPosition is a vec2 in coordinates local to the plane
gl_FragColor = gl_FragColor * texture2D( dataTexture, vertexPosition );
To calculate vertexPosition in the vertex share I do the following (irrelevant lines ommitted):
uniform float width;
uniform float height;
varying vec2 vertexPosition;
void main()
{
vertexPosition = vec2( position.x / width, position.y / height );
}
This is where I'm getting caught up. The vertexPosition does not seem to be mapping properly to the dataTexture pixels. I want a one to one correspondence between particles and pixels.
How do I properly map from the location of particles/vertexes on a plane to equivalent pixel locations in a texture?
I am new to three js, so please feel free to tell me my approach is totally off.
To get texture coordinates, there are ready to use projection matrix in glsl, here is what I would use as a vertex shader
varying vec2 vertexPosition;
void main() {
vertexPosition = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
Then you have the xy position to use in the fragment in the varying vertexPosition.
If you pass a varying view-space position from the vertex shader to a fragment shader then the fragment shader can know the fragment's position relative to the camera (0,0,0 in view-space). This is useful for per-pixel lighting etc. E.g.:
precision mediump float;
attribute vec3 vertex;
uniform mat4 pMatrix, mvMatrix;
varying vec4 pos;
void main() {
pos = (mvMatrix * vec4(vertex,1.0));
gl_Position = pMatrix * pos;
}
However, if you are rendering gl_POINTS and setting the gl_PointSize in the vertex shader, how can the fragment shader determine each fragment's position (as the pos passed in the example above will be for the sprite's centre-point)?
Simple answer: stop using point sprites. Really, they're terrible.
Less simple answer: stop passing the view-space position to the fragment shader entirely. Instead, use gl_FragCoord to compute the view-space position, based on viewport data and so forth. There's even sample GLSL code for it:
vec4 ndcPos;
ndcPos.xy = ((2.0 * gl_FragCoord.xy) - (2.0 * viewport.xy)) / (viewport.zw) - 1;
ndcPos.z = (2.0 * gl_FragCoord.z - gl_DepthRange.near - gl_DepthRange.far) /
(gl_DepthRange.far - gl_DepthRange.near);
ndcPos.w = 1.0;
vec4 clipPos = ndcPos / gl_FragCoord.w;
vec4 eyePos = invPersMatrix * clipPos;
You'll need to give your fragment shader the viewport and invPersMatrix values. gl_DepthRange is built-in. eyePos is what you're looking for.
There's probably a faster way to do it that takes advantage of the fact that you're drawing a screen-aligned quad. It would involve the point size and using gl_PointCoord.
I'm working with a GPU based particle system.
There are 1 million particles computed by passing in the x,y,z positions as rgb values on a 1024*1024 texture. The same is being done for their velocities.
I'm trying to make them move from an arbitrary point to a point on sphere.
My current shader, which I'm using for the computation, is moving from one point to another directly.
I'm not using the mass or velocity texture at the moment
// float mass = texture2D( posArray, texCoord.st).a;
vec3 p = texture2D( posArray, texCoord.st).rgb;
// vec3 v = texture2D( velArray, texCoord.st).rgb;
// map into 'cinder space'
p = (p * - 1.0) + 0.5;
// vec3 acc = -0.0002*p; // Centripetal force
// vec3 ayAcc = 0.00001*normalize(cross(vec3(0, 1 ,0),p)); // Angular force
// vec3 new_v = v + mass*(acc+ayAcc);
vec3 new_p = p + ((moveToPos - p) / duration);
// map out of 'cinder space'
new_p = (new_p - 0.5) * -1.0;
gl_FragData[0] = vec4(new_p.x, new_p.y, new_p.z, mass);
//gl_FragData[1] = vec4(new_v.x, new_v.y, new_v.z, 1.0);
moveToPos is the mouse pointer as a float (0.0f > 1.0f)
the coordinate system is being translated from (0.5,0.5 > -0.5,-0.5) to (0.0,0.0 > 1.0,1.0)
I'm completely new to vector maths, and the calculations that are confusing me. I know I need to use the formula:
x=Rsinϕcosθ
y=Rsinϕsinθ
z=Rcosϕ
but calculating the angles from moveToPos(xyz) > p(xyz) is remaining a problem
I wrote the original version of this GPU-particles shader a few years back (now #: https://github.com/num3ric/Cinder-Particles). Here is one possible approach to your problem.
I would start with a fragment shader applying a spring force to the particles so that they more or less are constrained to the surface of a sphere. Something like this:
uniform sampler2D posArray;
uniform sampler2D velArray;
varying vec4 texCoord;
void main(void)
{
float mass = texture2D( posArray, texCoord.st).a;
vec3 p = texture2D( posArray, texCoord.st).rgb;
vec3 v = texture2D( velArray, texCoord.st).rgb;
float x0 = 0.5; //distance from center of sphere to be maintaned
float x = distance(p, vec3(0,0,0)); // current distance
vec3 acc = -0.0002*(x - x0)*p; //apply spring force (hooke's law)
vec3 new_v = v + mass*(acc);
new_v = 0.999*new_v; // friction to slow down velocities over time
vec3 new_p = p + new_v;
//Render to positions texture
gl_FragData[0] = vec4(new_p.x, new_p.y, new_p.z, mass);
//Render to velocities texture
gl_FragData[1] = vec4(new_v.x, new_v.y, new_v.z, 1.0);
}
Then, I would pass a new vec3 uniform for the mouse position intersecting a sphere of the same radius (done outside the shader in Cinder).
Now, combining this with the previous soft spring constraint. You could add a tangential force towards this attraction point. Start with a simple (mousePos - p) acceleration, and then figure out a way to make this force exclusively tangential using cross-products.
I'm not sure how the spherical coordinates approach would work here.
x=Rsinϕcosθ
y=Rsinϕsinθ
z=Rcosϕ
Where do you get ϕ and θ? The textures stores the positions and velocities in cartesian coordinates. Plus, converting back and forth is not really an option.
My explanation could be too advanced if you are not comfortable with vectors. Unfortunately, shaders and particle animation are very mathematical by nature.
Here is a solution that I've worked out - it works, however if I move the center point of the spheres outside their own bounds, I lose particles.
#define NPEOPLE 5
uniform sampler2D posArray;
uniform sampler2D velArray;
uniform vec3 centerPoint[NPEOPLE];
uniform float radius[NPEOPLE];
uniform float duration;
varying vec4 texCoord;
void main(void) {
float personToGet = texture2D( posArray, texCoord.st).a;
vec3 p = texture2D( posArray, texCoord.st).rgb;
float mass = texture2D( velArray, texCoord.st).a;
vec3 v = texture2D( velArray, texCoord.st).rgb;
// map into 'cinder space'
p = (p * - 1.0) + 0.5;
vec3 vec_p = p - centerPoint[int(personToGet)];
float len_vec_p = sqrt( ( vec_p.x * vec_p.x ) + (vec_p.y * vec_p.y) + (vec_p.z * vec_p.z) );
vec_p = ( ( radius[int(personToGet)] /* mass */ ) / len_vec_p ) * vec_p;
vec3 new_p = ( vec_p + centerPoint[int(personToGet)] );
new_p = p + ( (new_p - p) / (duration) );
// map out of 'cinder space'
new_p = (new_p - 0.5) * -1.0;
vec3 new_v = v;
gl_FragData[0] = vec4(new_p.x, new_p.y, new_p.z, personToGet);
gl_FragData[1] = vec4(new_v.x, new_v.y, new_v.z, mass);
}
I'm passing in arrays of 5 vec3f's and a float mapped as 5 center points and radii.
The particles are setup with a random position at the beginning and move towards the number in the array mapped to the alpha value of the position array.
My aim is to pass in blob data from openCV and map the spheres to people on a camera feed.
It's really uninteresting visually at the moment, so will need to use the velocity texture to add to the behaviour of the particles.
So I look onto OpenGL ES shader specs but do not see such...
For example - I created simple "pinch to zoon" and "rotate to turn around" and "move to move center" HYDRA pixel bender filter. it can be executed in flash. It is based on default pixel bender twirl example and this:
<languageVersion: 1.0;>
kernel zoomandrotate
< namespace : "Pixel Bender Samples";
vendor : "Kabumbus";
version : 3;
description : "rotate and zoom an image around"; >
{
// define PI for the degrees to radians calculation
const float PI = 3.14159265;
// An input parameter to specify the center of the twirl effect.
// As above, we're using metadata to indicate the minimum,
// maximum, and default values, so that the tools can set the values
// in the correctly in the UI for the filter.
parameter float2 center
<
minValue:float2(0.0, 0.0);
maxValue:float2(2048.0, 2048.0);
defaultValue:float2(256.0, 256.0);
>;
// An input parameter to specify the angle that we would like to twirl.
// For this parameter, we're using metadata to indicate the minimum,
// maximum, and default values, so that the tools can set the values
// in the correctly in the UI for the filter.
parameter float twirlAngle
<
minValue:float(0.0);
maxValue:float(360.0);
defaultValue:float(90.0);
>;
parameter float zoomAmount
<
minValue:float(0.01);
maxValue:float(10.0);
defaultValue:float(1);
>;
// An input parameter that indicates how we want to vary the twirling
// within the radius. We've added support to modulate by one of two
// functions, a gaussian or a sinc function. Since Flash does not support
// bool parameters, we instead are using this as an int with two possible
// values. Setting this parameter to be 1 will
// cause the gaussian function to be used, unchecking it will cause
// the sinc function to be used.
parameter int gaussOrSinc
<
minValue:int(0);
maxValue:int(1);
defaultValue:int(0);
>;
input image4 oImage;
output float4 outputColor;
// evaluatePixel(): The function of the filter that actually does the
// processing of the image. This function is called once
// for each pixel of the output image.
void
evaluatePixel()
{
// convert the angle to radians
float twirlAngleRadians = radians(twirlAngle);
// calculate where we are relative to the center of the twirl
float2 relativePos = outCoord() - center;
// calculate the absolute distance from the center normalized
// by the twirl radius.
float distFromCenter = length( relativePos );
distFromCenter = 1.0;
// modulate the angle based on either a gaussian or a sync.
float adjustedRadians;
// precalculate either the gaussian or the sinc weight
float sincWeight = sin( distFromCenter ) * twirlAngleRadians / ( distFromCenter );
float gaussWeight = exp( -1.0 * distFromCenter * distFromCenter ) * twirlAngleRadians;
// protect the algorithm from a 1 / 0 error
adjustedRadians = (distFromCenter == 0.0) ? twirlAngleRadians : sincWeight;
// switch between a gaussian falloff or a sinc fallof
adjustedRadians = (gaussOrSinc == 1) ? adjustedRadians : gaussWeight;
// rotate the pixel sample location.
float cosAngle = cos( adjustedRadians );
float sinAngle = sin( adjustedRadians );
float2x2 rotationMat = float2x2(
cosAngle, sinAngle,
-sinAngle, cosAngle
);
relativePos = rotationMat * relativePos;
float scale = zoomAmount;
// sample and set as the output color. since relativePos
// is related to the center location, we need to add it back in.
// We use linear sampling to smooth out some of the pixelation.
outputColor = sampleLinear( oImage, relativePos/scale + center );
}
}
So now I want to port it into OpenGL ES shader. math and parameters are convertable into OpenGL ES shader language, but what to do with sampleLinear? what is analog for it in openGL ES shader languge?
update:
So I had created something similar to my HYDRA filter... compatable with webGL and OpenGL ES shaders...
#ifdef GL_ES
precision highp float;
#endif
uniform vec2 resolution;
uniform float time;
uniform sampler2D tex0;
void main(void)
{
vec2 p = -1.0 + 2.0 * gl_FragCoord.xy / resolution.xy;
// a rotozoom
vec2 cst = vec2( cos(.5*time), sin(.5*time) );
mat2 rot = 0.5*cst.x*mat2(cst.x,-cst.y,cst.y,cst.x);
vec3 col = texture2D(tex0,0.5*rot*p+sin(0.1*time)).xyz;
gl_FragColor = vec4(col,1.0);
}
To see how it works get modern browser, navigate to shadertoy provide it with one texture ( http://www.iquilezles.org/apps/shadertoy/presets/tex4.jpg for example), paste my code into editable text aeria and hit ... Have fun. So.. now I have another problem... I want to have one image and black around it not copies of that same image... Any one knows how to do that?
Per Adobe's Pixel Blender Reference, sampleLinear "Handles coordinates not at pixel centers by performing bilinear interpolation on the adjacent pixel values."
The correct way to achieve that in OpenGL is to use texture2D, as you already are, but to set the texture environment for linear filtering via glTexParameter.
You can use the step function and multiply by its result to get black for out-of-bounds pixels, or give your texture a single pixel black border and switch to clamping rather than repeat, also via glTexParameter.
If you want to do it in code, try:
#ifdef GL_ES
precision highp float;
#endif
uniform vec2 resolution;
uniform float time;
uniform sampler2D tex0;
void main(void)
{
vec2 p = -1.0 + 2.0 * gl_FragCoord.xy / resolution.xy;
// a rotozoom
vec2 cst = vec2( cos(.5*time), sin(.5*time) );
mat2 rot = 0.5*cst.x*mat2(cst.x,-cst.y,cst.y,cst.x);
vec2 samplePos = 0.5*rot*p+sin(0.1*time);
float mask = step(samplePos.x, 0.0) * step(samplePos.y, 0.0) * (1.0 - step(samplePos.x, 1.0)) * (1.0 - step(samplePos.y, 1.0));
vec3 col = texture2D(tex0,samplePos).xyz;
gl_FragColor = vec4(col*mask,1.0);
}
That'd restrict colours to coming from the box from (0, 0) to (1, 1), but it looks like the shader heads off to some significantly askew places, so I'm not sure exactly what you want.