I am attempting to repurpose the heightmap shader example found here into one that will work with 32-bits of precision instead of 8. The work-in-progress code is on github: https://github.com/bgourlie/three_heightmap
The height map is being generated in .NET. The heights are within 0f...200f and converted into a 32-bit color value (Unity's Color struct) using the following method:
private static Color DepthToColor(float height)
{
var depthBytes = BitConverter.GetBytes(height);
int enc = BitConverter.ToInt32(depthBytes, 0);
return new Color((enc >> 24 & 255)/255f, (enc >> 16 & 255)/255f, (enc >> 8 & 255)/255f,
(enc & 255)/255f);
}
The color data is encoded as a png, with the result looking like this:
The vertex shader is taking this image data and coverting the RBGA values back to the original height value (using the technique answered in my question here):
uniform sampler2D bumpTexture; // defined in heightmap.js
varying float vAmount;
varying vec2 vUV;
void main()
{
vUV = uv;
vec4 bumpData = texture2D( bumpTexture, uv );
vAmount = dot(bumpData, vec4(1.0, 255.0, 65025.0, 16581375.0));
// Uncomment to see a "flatter" version
//vAmount = dot(bumpData, vec4(1.0, 1.0/255.0, 1.0/65025.0, 1.0/160581375.0));
// move the position along the normal
vec3 newPosition = position + normal * vAmount;
gl_Position = projectionMatrix * modelViewMatrix * vec4( newPosition, 1.0 );
}
The result is definitely messed up:
I can make it flatter by changing this line:
vAmount = dot(bumpData, vec4(1.0, 1.0/255.0, 1.0/65025.0, 1.0/16581375.0));
This will give me a much flatter image, which at least shows a nice outline of the generated terrain, but with an almost entirely flat plane (there is slight, albeit unnoticeable variation):
I assume I'm doing a few things wrong, I just don't know what. I'm not sure if I'm encoding the original float correctly. I'm not sure if I'm decoding it correctly in the vertex shader (the value I'm getting is certainly outside the range of 0...200). I'm also not very experienced in 3d graphics in general. So any pointers as to what I'm doing wrong, or generally how to achieve this would be greatly appreciated.
Again, the self contained work-in-progress code can be found here: https://github.com/bgourlie/three_heightmap
Your colour:
return new Color((enc >> 24 & 255)/255f, (enc >> 16 & 255)/255f, (enc >> 8 & 255)/255f,
(enc & 255)/255f);
... contains the most significant byte of enc in r, the second most significant in g, etc.
This:
vAmount = dot(bumpData, vec4(1.0, 255.0, 65025.0, 160581375.0));
builds vAmount with r in the least significant byte, g in the next-least significant, etc (though the multiplicands should be 256, 65536, etc*). So the bytes are in the incorrect order. The flatter version:
vAmount = dot(bumpData, vec4(1.0, 1.0/255.0, 1.0/65025.0, 1.0/160581375.0));
gets the bytes in the correct order but scales the output values into the range [0.0, 1.0], which is probably why it looks essentially flat.
So switch the order of encoding or of decoding the bytes and pick an appropriate scale.
(*) think about it this way: the smallest number that can go on any channel is 1.0 / 255.0. The least significant channel will be in the range [0, 1.0] — from 0 / 255.0 to 255.0 / 255.0. You want to scale the next channel so that its smallest value is the next thing on that scale. So its smallest value should be 256 / 255.0. So you need to turn 1.0 / 255.0 into 256.0 / 255.0. You achieve that by multiplying by 256, not by 255.
If you encode a wide integer into the components of a RGBA vector it's essential that you turn off filtering so that no interpolation happens between the values. Also OpenGL may internally convert to a different format, but that should only reduce your sample depth.
Related
I am working on a WebGL scene with ~100 different 2048 x 2048 px textures. I'm rendering points primitives, and each point has a texture index and texture uv offsets that indicate the region of the given texture that should be used on the point.
Initially, I attempted to pass each point's texture index as a varying value, then I attempted to pull the given texture from a sampler2D array using that index position. However, this yielded an error that one can only fetch sampler2D array values with a "constant integer expression", so now I'm using a gnarly if conditional to assign each point's texture index:
/**
* The fragment shader's main() function must define `gl_FragColor`,
* which describes the pixel color of each pixel on the screen.
*
* To do so, we can use uniforms passed into the shader and varyings
* passed from the vertex shader.
*
* Attempting to read a varying not generated by the vertex shader will
* throw a warning but won't prevent shader compiling.
**/
// set float precision
precision highp float;
// repeat identifies the size of each image in an atlas
uniform vec2 repeat;
// textures contains an array of textures with length n textures
uniform sampler2D textures[42];
// identify the uv values as a varying attribute
varying vec2 vUv; // blueprint uv coords
varying vec2 vTexOffset; // instance uv offsets
varying float vTexture; // set index of each object's vertex
void main() {
int textureIndex = int(floor(vTexture));
vec2 uv = vec2( gl_PointCoord.x, 1.0 - gl_PointCoord.y );
// The block below is automatically generated
if (textureIndex == 0) {vec4 color = texture2D(textures[0], uv * repeat + vTexOffset ); }
else if (textureIndex == 1) { vec4 color = texture2D(textures[1], uv * repeat + vTexOffset ); }
else if (textureIndex == 2) { vec4 color = texture2D(textures[2], uv * repeat + vTexOffset ); }
else if (textureIndex == 3) { vec4 color = texture2D(textures[3], uv * repeat + vTexOffset ); }
[ more lines of the same ... ]
gl_FragColor = color;
}
If the number of textures is small, this works fine. But if the number of textures is large (e.g. 40) this approach throws:
ERROR: 0:58: '[' : memory exhausted
I've tried reading around on this error but still am not sure what it means. Have I surpassed the max RAM in the GPU? If anyone knows what this error means, and/or what I can do to resolve the problem, I'd be grateful for any tips you can provide.
More details:
Total size of all textures to be loaded: 58MB
Browser: recent Chrome
Graphics card: AMD Radeon R9 M370X 2048 MB graphics (stock 2015 OSX card)
There is a limit on how many samplers a fragment shader can access. It can be obtained via gl.getParameter(gl.MAX_TEXTURE_IMAGE_UNITS). It is guaranteed to be at least 8, and is typically 16 or 32.
To circumvent the limit, texture arrays are available in WebGL2, which also allow indexing layers with any variable. In WebGL1 your only option are atlases, but since your textures are already 2048 by 2048, you can't make ghem any bigger.
If you don't want to limit yourself to WebGL2, you would have to split your rendering into multiple draw calls with diffferent textures set.
Also consider that having 100 8-bit RGBA 2048x2048 textures uses up 1.6 gigabytes of VRAM. Texture compression via WEBGL_compressed_texture_s3tc can reduce that by 8x or 4x, depending on how much alpha precision you need.
I wrote some WebGL code that is based on floating point textures. But while testing it on a few more devices I found that support for the OES_texture_float extension isn't as widespread as I had thought. So I'm looking for a fallback.
I have currently a luminance floating point texture with values between -1.0 and 1.0. I'd like to encode this data in a texture format that is available in WebGL without any extensions, so probably a simple RGBA unsigned byte texture.
I'm a bit worried about the potential performance overhead because the cases where this fallback is needed are older smartphones or tablets which already have much weaker GPUs than a modern desktop computer.
How can I emulate floating point textures on a device that doesn't support them in WebGL?
If you know your range is -1 to +1 the simplest way is to just to convert that to some integer range and then convert back. Using the code from this answer which packs a value that goes from 0 to 1 into a 32bit color
const vec4 bitSh = vec4(256. * 256. * 256., 256. * 256., 256., 1.);
const vec4 bitMsk = vec4(0.,vec3(1./256.0));
const vec4 bitShifts = vec4(1.) / bitSh;
vec4 pack (float value) {
vec4 comp = fract(value * bitSh);
comp -= comp.xxyz * bitMsk;
return comp;
}
float unpack (vec4 color) {
return dot(color , bitShifts);
}
Then
const float rangeMin = -1.;
const float rangeMax = -1.;
vec4 convertFromRangeToColor(float value) {
float zeroToOne = (value - rangeMin) / (rangeMax - rangeMin);
return pack(value);
}
float convertFromColorToRange(vec4 color) {
float zeroToOne = unpack(color);
return rangeMin + zeroToOne * (rangeMax - rangeMin);
}
This should be a good starting point: http://aras-p.info/blog/2009/07/30/encoding-floats-to-rgba-the-final/
It's intended for encoding to 0.0 to 1.0, but should be straightforward to remap to your required range.
I am now using opengl-es and I use the gl shading language. I hope to render to texture but I found a loss of precision. For example, when I write a float value of 0.5 to the texture, I found the actual value stored in the texture was approximately 0.498. What should I do to achieve higher precision?
You probably should consider storing your values (if just one value per pixel/texel) via packing-unpacking your values:
vec4 packFloat(const float value) {
const vec4 bitSh = vec4(256.0 * 256.0 * 256.0, 256.0 * 256.0, 256.0, 1.0);
const vec4 bitMsk = vec4(0.0, 1.0 / 256.0, 1.0 / 256.0, 1.0 / 256.0);
vec4 res = fract(value * bitSh);
res -= res.xxyz * bitMsk;
return res;
}
float unpackFloat(const vec4 value) {
const vec4 bitSh = vec4(1.0 / (256.0 * 256.0 * 256.0), 1.0 / (256.0 * 256.0), 1.0 / 256.0, 1.0);
return (dot(value, bitSh));
}
This might be okay for storing values for something like depth-maps
And this would be kind of a 32 bit range for each pixel/texel
Try adding the highp precision qualifier in front of your variables.
Render to a texture that uses more than 8 bits per component. If you don't have the appropriate OpenGL ES extensions for that, then generally there's not much you can do.
Even the next higher precision might not be enough, because the final stage of the rendering pipeline scales the pixel values to a range of 0..1, the end points inclusive. Thus 1 will be represented as 255, which suggest a factor of 1/255 instead of 1/256.
The same applies to all precisions: 0.5 can't be represented exactly.
I am learning to use shaders in OpenGL ES.
As an example: Here's my playground fragment shader which takes the current video frame and makes it grayscale:
varying highp vec2 textureCoordinate;
uniform sampler2D videoFrame;
void main() {
highp vec4 theColor = texture2D(videoFrame, textureCoordinate);
highp float avrg = (theColor[0] + theColor[1] + theColor[2]) / 3.0;
theColor[0] = avrg; // r
theColor[1] = avrg; // g
theColor[2] = avrg; // b
gl_FragColor = theColor;
}
theColor represents the current pixel. It would be cool to also get access to the previous pixel at this same coordinate.
For sake of curiousity, I would like to add or multiply the color of the current pixel to the color of the pixel in the previous render frame.
How could I keep the previous pixels around and pass them in to my fragment shader in order to do something with them?
Note: It's OpenGL ES 2.0 on the iPhone.
You need to render the previous frame to a texture, using a Framebuffer Object (FBO), then you can read this texture in your fragment shader.
The dot intrinsic function that Damon refers to is a code implementation of the mathematical dot product. I'm not supremely familiar with OpenGL so I'm not sure what the exact function call is, but mathematically a dot product goes like this :
Given a vector a and a vector b, the 'dot' product a 'dot' b produces a scalar result c:
c = a.x * b.x + a.y * b.y + a.z * b.z
Most modern graphics hardware (and CPUs, for that matter) are capable of performing this kind of operation in one pass. In your particular case, you could compute your average easily with a dot product like so:
highp vec4 = (1/3, 1/3, 1/3, 0) //or zero
I always get the 4th component in homogeneous vectors and matrices mixed up for some reason.
highp float avg = theColor DOT vec4
This will multiple each component of theColor by 1/3 (and the 4th component by 0), and then add them together.
Im having issue with loss of precision in my SAVSM setup.
when you see the light moving around the effect is very striking; there is a lot of noise with fragments going black and white all the time. This can be somewhat lessened by using the minvariance (thus ignoring anything below a certain threshold) but then we get even worse effects with incorrect falloff (see my other post).
Im using GLSL 1.2 because I'm on a mac so I dont have access to the modf function in order to split the precision across two channels as described in GPU Gems 3 Chapter 8.
Im using GL_RGBA32F_ARB textures with a Framebuffer object and ping ponging two textures to generate a summed area table which i use with the VSM algorithm.
Moments / Depth Shader to create the basis for the tables
varying vec4 v_position;
varying float tDepth;
float g_DistributeFactor = 1024.0;
void main()
{
// Is this linear depth? I would say yes but one can't be utterly sure.
// Could try a divide by the far plane?
float depth = v_position.z / v_position.w ;
depth = depth * 0.5 + 0.5; //Don't forget to move away from unit cube ([-1,1]) to [0,1] coordinate system
vec2 moments = vec2(depth, depth * depth);
// Adjusting moments (this is sort of bias per pixel) using derivative
float dx = dFdx(depth);
float dy = dFdy(depth);
moments.y += 0.25 * (dx*dx+dy*dy);
// Subtract 0.5 off now so we can get this into our summed area table calc
//moments -= 0.5;
// Split the moments into rg and ba for EVEN MORE PRECISION
// float FactorInv = 1.0 / g_DistributeFactor;
// gl_FragColor = vec4(floor(moments.x) * FactorInv, fract(moments.x ) * g_DistributeFactor,
// floor(moments.y) * FactorInv, fract(moments.y) * g_DistributeFactor);
gl_FragColor = vec4(moments,0.0,0.0);
}
The shadowmap shader
varying vec4 v_position;
varying float tDepth;
float g_DistributeFactor = 1024.0;
void main()
{
// Is this linear depth? I would say yes but one can't be utterly sure.
// Could try a divide by the far plane?
float depth = v_position.z / v_position.w ;
depth = depth * 0.5 + 0.5; //Don't forget to move away from unit cube ([-1,1]) to [0,1] coordinate system
vec2 moments = vec2(depth, depth * depth);
// Adjusting moments (this is sort of bias per pixel) using derivative
float dx = dFdx(depth);
float dy = dFdy(depth);
moments.y += 0.25 * (dx*dx+dy*dy);
// Subtract 0.5 off now so we can get this into our summed area table calc
//moments -= 0.5;
// Split the moments into rg and ba for EVEN MORE PRECISION
// float FactorInv = 1.0 / g_DistributeFactor;
// gl_FragColor = vec4(floor(moments.x) * FactorInv, fract(moments.x ) * g_DistributeFactor,
// floor(moments.y) * FactorInv, fract(moments.y) * g_DistributeFactor);
gl_FragColor = vec4(moments,0.0,0.0);
}
The Summed tables do seem to be working. I know this because I have a function that converts back from the summed table to the original depth map and the two images do look pretty much the same. Im also using the -0.5 + 0.5 trick in order to get some more precision but it doesnt seem to be helping
My question is this, given that im on a mac which has GLSL 1.2 only, how can I split the precision over two channels? If I could use these extra channels for space in the summed table then maybe that would work? Ive seen some stuff that uses modf but that isnt available to me.
Also, people have suggested 32 bit integer buffers but I dont think I have support for these on my macbook pro.