How to convert to a HDR renderer? - opengl-es

Question

I am in the process of converting my webgl deferred renderer to one that uses high dynamic range. I've read a lot about the subject from various sources online and I have a few questions that I hope could be clarified. Most of the reading I have done covers HDR image rendering, but my questions pertain to how a renderer might have to change to support HDR.
As I understand it, HDR is essentially trying to capture higher light ranges so that we can see detail in both extremely lit or dark scenes. Typically in games we use an intensity of 1 to represent white light and 0 black. But in HDR / the real world, the ranges are far more varied. I.e. a sun in the engine might be 10000 times brighter than a lightbulb of 10.
To cope with these larger ranges you have to convert your renderer to use floating point render targets (or ideally half floats as they use less memory) for its light passes.
My first question is on the lighting. Besides the floating point render targets, does this simply mean that if previously I had a light representing the sun, which was of intensity 1, it could/should now be represented as 10000? I.e.
float spec = calcSpec();
vec4 diff = texture2D( sampler, uv );
vec4 color = diff * max(0.0, dot( N, L )) * lightIntensity + spec; //Where lightIntensity is now 10000?
return color;
Are there any other fundamental changes to the lighting system (other than float textures and higher ranges)?
Following on from this, we now have a float render target that has additively accumulated all the light values (in the higher ranges as described). At this point I might do some post processing on the render target with things like bloom. Once complete it now needs to be tone-mapped before it can be sent to the screen. This is because the light ranges must be converted back to the range of our monitors.
So for the tone-mapping phase, I would presumably use a post process and then using a tone-mapping formula convert the HDR lighting to a low dynamic range. The technique I chose was John Hables from Uncharted 2:
const float A = 0.15;
const float B = 0.50;
const float C = 0.10;
const float D = 0.20;
const float E = 0.02;
const float F = 0.30;
const float W = 11.2;
vec3 Uncharted2Tonemap(vec3 x)
{
return ((x*(A*x+C*B)+D*E)/(x*(A*x+B)+D*F))-E/F;
}
... // in main pixel shader
vec4 texColor = texture2D(lightSample, texCoord );
texColor *= 16; // Hardcoded Exposure Adjustment
float ExposureBias = 2.0;
vec3 curr = Uncharted2Tonemap( ExposureBias * texColor.xyz );
vec3 whiteScale = 1.0 / Uncharted2Tonemap(W);
vec3 color = curr * whiteScale;
// Gama correction
color.x = pow( color.x, 1.0 /2.2 );
color.y = pow( color.y, 1.0 /2.2 );
color.z = pow( color.z, 1.0 /2.2 );
return vec4( color, 1.0 );
Tone mapping article
My second question is related to this tone mapping phase. Is there much more to it than simply this technique? Is simply using higher light intensities and tweaking the exposure all thats required to be considered HDR - or is there more to it? I understand that some games have auto exposure functionality to figure out the average luminescence, but at the most basic level is this needed? Presumably you can just use manually tweak the exposure?
Something else thats discussed in a lot of the documents is that of gama correction. The gama correction seems to be done in two areas. First when textures are read and then once again when they are sent to the screen. When textures are read they must simply be changed to something like this:
vec4 diff = pow( texture2D( sampler, uv), 2.2 );
Then in the above tone mapping technique the output correction is done by:
pow(color,1/2.2);
From John Hables presentation he says that not all textures must be corrected like this. Diffuse textures must be, but things like normal maps don't necessarily have to.
My third question is on this gama correction. Is this necessary in order for it to work? Does it mean I have to change my engine in all places where diffuse maps are read?
That is my current understanding of whats involved for this conversion. Is it correct and is there anything I have misunderstood or got wrong?

Answer 1

Light Calculation / Accumulation
Yes, you are generally able to keep your lightning calculation the same and increasing say the intensity of directional lights over 1.0 is certainly fine. Another way the value can exceed one is simply by adding the contributions of several lights together.
Tone Mapping
You certainly understood the concept. There are quite a few different ways to do the actual mapping, from the more simple / naive one color = clamp(hdrColor * exposure) to the more sophisticated (and better) one you posted.
Adaptive tone mapping can quickly become more complicated. Again the naive way is to simply normalize colors by diving with the brightest pixel, which will certainly make it hard/impossible to perceive details in the darker parts of the image. You can also average the brightness and clamp. Or you can save whole histograms of the several last frames and use those in your mapping.
Another method is to normalize each pixel only with the values of the neighbouring pixels, i.e. "local tone mapping". This one is not usually done in real-time rendering.
While it may sound complicated the formula you posted will generate very good results, so it is fine to go with it. Once you have a working implementation feel free to experiment here. There are also great papers available :)
Gamma
Now gamma-correction is important, even if you do not use hdr rendering. But never worry, it is not hard.
The most important thing is to be always aware in what color space you are working. Just like a number without unit, a color without color space just makes seldom sense. Now we like to work in linear (rgb) color space in our shaders, meaning a color with twice the rgb-values should be twice as bright. However this is not how monitors work.
Cameras and photo-editing software often simply hide all this from us and simply save pictures in the format the monitor likes (called sRGB).
There is an additional advantage in sRGB and that is compression. We usually save image with 8/16/32 bit per pixel per channel. If you save pictures in linear space and you have small but very bright spots in the image your 8/16/32 bit may not be precise enough to save brightness differences in the darker parts of the image and if you are displaying them again (of course gamma correct) details may be lost in the dark.
You are able to change the color space your images are saved in many cameras and programs, even if it is sometimes a bit hidden. So if you tell your artists to save all images in linear (rgb) color space you do not need to gamma-correct images at all. Since most programs like sRGB and sRGB offers better compression it is generally a good idea to save images that describe color in sRGB, those therefore need to be gamma corrected. Images that describe values/data like normal maps or bump maps are usually saved in linear color space (if your normal [1.0, 0.5, 0.0] just does not have a 45 degree angle everybody will be confused; the compression advantage is also naught with non-colors).
If you want to use a sRGB Texture just tell OpenGL so and it will convert it to a linear color space for you, without performance hit.
void glTexImage2D( GLenum target,
GLint level,
GLint internalFormat, // Use **GL_SRGB** here
GLsizei width,
GLsizei height,
GLint border,
GLenum format,
GLenum type,
const GLvoid * data);
Oh and of course you have to gamma-correct everything you send to your display (so change from linear to sRGB or gamma 2.2). You can do this in your tone mapping or another post-process step. Or let OpenGL do it for you; see glEnable(GL_FRAMEBUFFER_SRGB)