Quite new at shaders, so please bear with me if I am doing something silly here. :)
I am trying to render the depth buffer of a scene to a texture using opengl ES 2.0 on iOS, but I do not seem to get entirely accurate results unless the models have a relatively high density of polygons showing on the display.
So, for example if I render a large plane consisting of only four vertices, I get very inaccurate results, but if I subdivide this plane the results get more accurate for each subdivision, and ultimately I get a correctly rendered depth buffer.
This reminds me a lot about affine versus perspective projected texture mapping issues, and I guess I need to play around with the ".w" component somehow to fix this. But I thought the "varying" variables should take this into account already, so I am a bit at loss here.
This is my vertex and fragment shader:
[vert]
uniform mat4 uMVPMatrix;
attribute vec4 aPosition;
varying float objectDepth;
void main()
{
gl_Position=uMVPMatrix * aPosition;
objectDepth=gl_Position.z;
}
[frag]
precision mediump float;
varying float objectDepth;
void main()
{
//Divide by scene clip range, set to a constant 200 here
float grayscale=objectDepth/200.0;
gl_FragColor = vec4(grayscale,grayscale,grayscale,1.0);
}
Please note that this shader is simplified a lot just to highlight the method I am using. Although for the naked eye it seems to work well in most cases, I am in fact rendering to 32 bit textures (by packing a float into ARGB), and I need very high accuracy for later processing or I get noticeable artifacts.
I can achieve pretty high precision by cranking up the polygon count, but that drives my framerate down a lot, so, is there a better way?
You need to divide z by the w component.
This is very simple, the depth is not linear so you can not use linear interpolation for z ... you will solve it very easily if you interpolate 1/z instead of z. You can also perform some w math, exactly as suggested by rasmus.
You can read more about coordinate interpolation at http://www.luki.webzdarma.cz/eng_05_en.htm (page about implementing a simple software renderer)
Related
I am implementing a feature extraction algorithm with OpenGL ES 3.0 (given an input texture with some 1's and mostly 0's, produce an output texture that has feature regions labeled). The problem I face in my fragment shader is how to perform a “lookup” on an intermediate vec or float rather than a sampler.
Conceptually every vec or float is just a texel-valued function, so there ought to be a way to get its value given texel coordinates, something like textureLikeLookup(texel_valued, v_color) - but I haven’t found anything that parallels the texture* functions.
The options I see are:
Render my vector to a framebuffer and pass that as a texture into another shader/rendering pass - undesirable because I have many such passes in a loop, and I want to avoid interleaving CPU calls;
Switch to ES 3.1 and take advantage of imageStore (https://www.khronos.org/registry/OpenGL-Refpages/es3.1/html/imageStore.xhtml) - it seems clear that if I can update an intermediate image within my shader then I can achieve this within the fragment shader (cf. https://www.reddit.com/r/opengl/comments/279fc7/glsl_frag_update_values_to_a_texturemap_within/), but I would rather stick to 3.0 if possible.
Is there a better/natural way to deal with this problem? In other words, do something like this
// works, because tex_sampler is a sampler2D
vec4 texel_valued = texture(tex_sampler, v_color);
when the data is not a sampler2D but a vec:
// doesn't work, because texel_valued is not a sampler but a vec4
vec4 oops = texture(texel_valued, v_color);
I am trying to alter the texture coordinates in the fragment shader.
Precision is set to medium.
It works on some devices but causes severe pixelation on some (cheaper) devices.
I assume this is a precision problem but its odd because I have set default to medium which should be available on all devices. Any idea? Thanks for your time.
+"mediump vec2 coords=v_TexCoordinate; \n"
+"coords+=0.1; \n"
mediump is only defined in the spec in terms of a minimum allowable precision. There are devices out there which can implement mediump as a highp (i.e. as more or less a single precision 32 bit float).
There may be other devices which genuinely have a mediump (half precision float alike) and separate highp in hardware, but use some heuristic to notice that it's a texture coordinate and decide that you're better off with the higher precision.
Other devices still, may move your trivial tex coordinate modification to the vertex shader, and will use the texture coordinates output by the vertex shader to prefetch the texels before the fragment shader even executes (and thus avoid the fragment shader's shonky precision altogether).
So just because you ask for mediump everywhere, it's not unusual that you can only repro precision issues on a subset of your devices.
In terms of fixing it - from what you've included in your question, you could just do the tex coordinate modification on the vertex shader.
I'm working on emulating in glsl an effect I've seen used pretty widely (example in image). I am new to glsl, but have a decent amount of experience in max msp and jitter, so that is where I am trying to implement it (syntax should be similar enough though, but it's glsl version 2.1).
Currently, I am taking in one texture, then feeding the output back in as the second texture. Using a luma threshold, I am selecting pixels to be sorted, but from here, I can't figure out how to continue. I assume that I need to:
Take the color value from the initial, luma selected pixel.
Apply that color to subsequent pixel along a certain axis.
And now I can't figure out how to continue. I'll include the code that I do have, which obviously just has nothing after the luma threshold stuff. Any advice or resources would be greatly appreciated.
uniform sampler2DRect tex0;
uniform sampler2DRect tex1;
varying vec2 texcoord0;
varying vec2 texcoord1;
uniform float thresh;
const vec4 lumcoeff = vec4(0.299,0.587,0.114,0.0);
void main()
{
vec4 input0 = texture2DRect(tex0, texcoord0);
vec4 input1 = texture2DRect(tex1, texcoord1);
float luma = dot(input0,lumcoeff);
if(luma > thresh){
gl_FragColor
}else{
gl_FragColor = input0;
}
}
I actually made the original image you posted up above. Sorting on the GPU can be a complicated task because we can't easily make comparisons the same was that a CPU can.
I'd take a look at this nvidia paper on a bitonic merge sort. I implemented it in glsl here if you'd like to take a look, though I think chrome auto play killed it so you may need to try in FF or Safari.
There's also a few examples on shadertoy if you search around. I think this one might be pretty close to what you're hoping for
Good luck!
I am in the process of converting my webgl deferred renderer to one that uses high dynamic range. I've read a lot about the subject from various sources online and I have a few questions that I hope could be clarified. Most of the reading I have done covers HDR image rendering, but my questions pertain to how a renderer might have to change to support HDR.
As I understand it, HDR is essentially trying to capture higher light ranges so that we can see detail in both extremely lit or dark scenes. Typically in games we use an intensity of 1 to represent white light and 0 black. But in HDR / the real world, the ranges are far more varied. I.e. a sun in the engine might be 10000 times brighter than a lightbulb of 10.
To cope with these larger ranges you have to convert your renderer to use floating point render targets (or ideally half floats as they use less memory) for its light passes.
My first question is on the lighting. Besides the floating point render targets, does this simply mean that if previously I had a light representing the sun, which was of intensity 1, it could/should now be represented as 10000? I.e.
float spec = calcSpec();
vec4 diff = texture2D( sampler, uv );
vec4 color = diff * max(0.0, dot( N, L )) * lightIntensity + spec; //Where lightIntensity is now 10000?
return color;
Are there any other fundamental changes to the lighting system (other than float textures and higher ranges)?
Following on from this, we now have a float render target that has additively accumulated all the light values (in the higher ranges as described). At this point I might do some post processing on the render target with things like bloom. Once complete it now needs to be tone-mapped before it can be sent to the screen. This is because the light ranges must be converted back to the range of our monitors.
So for the tone-mapping phase, I would presumably use a post process and then using a tone-mapping formula convert the HDR lighting to a low dynamic range. The technique I chose was John Hables from Uncharted 2:
const float A = 0.15;
const float B = 0.50;
const float C = 0.10;
const float D = 0.20;
const float E = 0.02;
const float F = 0.30;
const float W = 11.2;
vec3 Uncharted2Tonemap(vec3 x)
{
return ((x*(A*x+C*B)+D*E)/(x*(A*x+B)+D*F))-E/F;
}
... // in main pixel shader
vec4 texColor = texture2D(lightSample, texCoord );
texColor *= 16; // Hardcoded Exposure Adjustment
float ExposureBias = 2.0;
vec3 curr = Uncharted2Tonemap( ExposureBias * texColor.xyz );
vec3 whiteScale = 1.0 / Uncharted2Tonemap(W);
vec3 color = curr * whiteScale;
// Gama correction
color.x = pow( color.x, 1.0 /2.2 );
color.y = pow( color.y, 1.0 /2.2 );
color.z = pow( color.z, 1.0 /2.2 );
return vec4( color, 1.0 );
Tone mapping article
My second question is related to this tone mapping phase. Is there much more to it than simply this technique? Is simply using higher light intensities and tweaking the exposure all thats required to be considered HDR - or is there more to it? I understand that some games have auto exposure functionality to figure out the average luminescence, but at the most basic level is this needed? Presumably you can just use manually tweak the exposure?
Something else thats discussed in a lot of the documents is that of gama correction. The gama correction seems to be done in two areas. First when textures are read and then once again when they are sent to the screen. When textures are read they must simply be changed to something like this:
vec4 diff = pow( texture2D( sampler, uv), 2.2 );
Then in the above tone mapping technique the output correction is done by:
pow(color,1/2.2);
From John Hables presentation he says that not all textures must be corrected like this. Diffuse textures must be, but things like normal maps don't necessarily have to.
My third question is on this gama correction. Is this necessary in order for it to work? Does it mean I have to change my engine in all places where diffuse maps are read?
That is my current understanding of whats involved for this conversion. Is it correct and is there anything I have misunderstood or got wrong?
Light Calculation / Accumulation
Yes, you are generally able to keep your lightning calculation the same and increasing say the intensity of directional lights over 1.0 is certainly fine. Another way the value can exceed one is simply by adding the contributions of several lights together.
Tone Mapping
You certainly understood the concept. There are quite a few different ways to do the actual mapping, from the more simple / naive one color = clamp(hdrColor * exposure) to the more sophisticated (and better) one you posted.
Adaptive tone mapping can quickly become more complicated. Again the naive way is to simply normalize colors by diving with the brightest pixel, which will certainly make it hard/impossible to perceive details in the darker parts of the image. You can also average the brightness and clamp. Or you can save whole histograms of the several last frames and use those in your mapping.
Another method is to normalize each pixel only with the values of the neighbouring pixels, i.e. "local tone mapping". This one is not usually done in real-time rendering.
While it may sound complicated the formula you posted will generate very good results, so it is fine to go with it. Once you have a working implementation feel free to experiment here. There are also great papers available :)
Gamma
Now gamma-correction is important, even if you do not use hdr rendering. But never worry, it is not hard.
The most important thing is to be always aware in what color space you are working. Just like a number without unit, a color without color space just makes seldom sense. Now we like to work in linear (rgb) color space in our shaders, meaning a color with twice the rgb-values should be twice as bright. However this is not how monitors work.
Cameras and photo-editing software often simply hide all this from us and simply save pictures in the format the monitor likes (called sRGB).
There is an additional advantage in sRGB and that is compression. We usually save image with 8/16/32 bit per pixel per channel. If you save pictures in linear space and you have small but very bright spots in the image your 8/16/32 bit may not be precise enough to save brightness differences in the darker parts of the image and if you are displaying them again (of course gamma correct) details may be lost in the dark.
You are able to change the color space your images are saved in many cameras and programs, even if it is sometimes a bit hidden. So if you tell your artists to save all images in linear (rgb) color space you do not need to gamma-correct images at all. Since most programs like sRGB and sRGB offers better compression it is generally a good idea to save images that describe color in sRGB, those therefore need to be gamma corrected. Images that describe values/data like normal maps or bump maps are usually saved in linear color space (if your normal [1.0, 0.5, 0.0] just does not have a 45 degree angle everybody will be confused; the compression advantage is also naught with non-colors).
If you want to use a sRGB Texture just tell OpenGL so and it will convert it to a linear color space for you, without performance hit.
void glTexImage2D( GLenum target,
GLint level,
GLint internalFormat, // Use **GL_SRGB** here
GLsizei width,
GLsizei height,
GLint border,
GLenum format,
GLenum type,
const GLvoid * data);
Oh and of course you have to gamma-correct everything you send to your display (so change from linear to sRGB or gamma 2.2). You can do this in your tone mapping or another post-process step. Or let OpenGL do it for you; see glEnable(GL_FRAMEBUFFER_SRGB)
I'm working on a WebGL batch renderer (question still valid in OpenGL land). Aka all graphics in a scene drawn in the fewest possible drawArrays/drawElements calls (ideally 1). Part of this involves allowing textures to be determined based on attributes.
Therefore in my fragment shader I'm contemplating two scenarios:
1. Draw texture 0 to the screen and use attributes to determine the "frame" in which the texture lies on a sprite sheet that's in memory. The fragment shader would look something like:
precision mediump float;
uniform sampler2D u_spriteSheet;
// Represents position that's framed.
varying vec4 v_texturePosition;
void main() {
gl_FragColor = texture2D(u_spriteSheet, v_texturePosition);
}
2. Do "if" statements in the shader to determine which uniform sampler2d to utilize. The fragment shader would look something like:
precision mediump float;
uniform sampler2D u_image1;
uniform sampler2D u_image2;
uniform sampler2D u_image3;
uniform sampler2D u_image4;
....
uniform sampler2D u_image32;
varying uint v_imageId;
// Represents texture position that's framed
varying vec4 v_texturePosition;
void main() {
if(v_imageId == 1) {
gl_FragColor = texture2D(u_image1, v_texturePosition);
}
else if (v_imageId == 2) {
gl_FragColor = texture2D(u_image2, v_texturePosition);
}
...
else if (v_imageId == 32) {
gl_FragColor = texture2D(u_image32, v_texturePosition);
}
}
I understand that with option 1 i'm limited by the max texture size and by approach 2 i'm limited by the number of texture registers available. For the sake of discussion lets assume that these limits will never be passed.
I'm trying to determine the more performant approach before investing a significant amount of time into either one... Sooo any thoughts?
If statements in shaders are generally slow, because on normal GPU hardware
shaders are executed as SIMD, i.e. many fragments are processed in parallel,
instruction per instruction. Simplified speaking, in case of an if all threads
process the then part whereby only threads with a positive if-condition
really execute and store the result while the other threads are waiting
(or even do the calculation but not store the result). Afterwards all
threads do the else part and all threads with positive condition are waiting.
So in your solution #2 the fragment shader on many cards would execute all
32 parts and not just one as in solution #1 (On some cards it is said that
they stop executing an if branch if there is no thread following that part any
more, so it may be less than 32).
So I would expect that solution #1 is faster w.r.t the fragment shader. However your solution might have other bottlenecks, so that the performance of the fragment shader
might become irrelevant for the overall performance.
Additional thoughts are that many GPUs allow less than 32 textures, so you
probably cannot use 32 if you want to stay compatible with many devices.
According to webglstats.com 99% have 16 textures units and since most
scenes have more than 16 textures there is probably no way around
implementing something like your solution #1.
On the other hand when you hit the maximal texture size you might need
something like #2 as well.