Blurring (two-pass shader) an object with a transparent background? - opengl-es

Question

I have an object I'm trying to blur.
Render it to a transparent (glClear with 1, 1, 1, 0) FBO.
Render it to a second transparent FBO with a vertical blur shader.
Render it to the screen with a horizontal blur shader.
Here is what an example looks like not blurred, and then blurred with this technique:
Obviously the issue is that white glow around the blurred object.
I think I grasp the basic concept of why this is happening. While the pixels around the object in the FBO are transparent, they still hold the color (1,1,1) and as a result, that color is mixed into the blur.
I just don't know what I would do to remedy this?
Here is my horizontal blur shader, vertical is much of the same:
hBlur.vert
uniform mat4 u_projTrans;
uniform float u_blurPixels;
uniform float u_texelWidth;
attribute vec4 a_position;
attribute vec2 a_texCoord0;
attribute vec4 a_color;
varying vec2 v_texCoord;
varying vec2 v_blurTexCoords[14];
void main()
{
v_texCoord = a_texCoord0;
gl_Position = u_projTrans * a_position;
float blurDistance6 = u_blurPixels * u_texelWidth;
float blurDistance5 = blurDistance6 * 0.84;
float blurDistance4 = blurDistance6 * 0.70;
float blurDistance3 = blurDistance6 * 0.56;
float blurDistance2 = blurDistance6 * 0.42;
float blurDistance1 = blurDistance6 * 0.28;
float blurDistance0 = blurDistance6 * 0.14;
v_blurTexCoords[ 0] = v_texCoord + vec2(-blurDistance6, 0.0);
v_blurTexCoords[ 1] = v_texCoord + vec2(-blurDistance5, 0.0);
v_blurTexCoords[ 2] = v_texCoord + vec2(-blurDistance4, 0.0);
v_blurTexCoords[ 3] = v_texCoord + vec2(-blurDistance3, 0.0);
v_blurTexCoords[ 4] = v_texCoord + vec2(-blurDistance2, 0.0);
v_blurTexCoords[ 5] = v_texCoord + vec2(-blurDistance1, 0.0);
v_blurTexCoords[ 6] = v_texCoord + vec2(-blurDistance0, 0.0);
v_blurTexCoords[ 7] = v_texCoord + vec2( blurDistance0, 0.0);
v_blurTexCoords[ 8] = v_texCoord + vec2( blurDistance1, 0.0);
v_blurTexCoords[ 9] = v_texCoord + vec2( blurDistance2, 0.0);
v_blurTexCoords[10] = v_texCoord + vec2( blurDistance3, 0.0);
v_blurTexCoords[11] = v_texCoord + vec2( blurDistance4, 0.0);
v_blurTexCoords[12] = v_texCoord + vec2( blurDistance5, 0.0);
v_blurTexCoords[13] = v_texCoord + vec2( blurDistance6, 0.0);
}
blur.frag
uniform sampler2D u_texture;
varying vec2 v_texCoord;
varying vec2 v_blurTexCoords[14];
void main()
{
gl_FragColor = vec4(0.0);
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 0]) * 0.0044299121055113265;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 1]) * 0.00895781211794;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 2]) * 0.0215963866053;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 3]) * 0.0443683338718;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 4]) * 0.0776744219933;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 5]) * 0.115876621105;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 6]) * 0.147308056121;
gl_FragColor += texture2D(u_texture, v_texCoord ) * 0.159576912161;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 7]) * 0.147308056121;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 8]) * 0.115876621105;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[ 9]) * 0.0776744219933;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[10]) * 0.0443683338718;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[11]) * 0.0215963866053;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[12]) * 0.00895781211794;
gl_FragColor += texture2D(u_texture, v_blurTexCoords[13]) * 0.0044299121055113265;
}
I'd lie if I said I was completely certain what this code is doing. But in summary it's just sampling pixels from within a radius of u_blurPixels and summing up the resulting color for gl_FragColor with pre-determined gaussian weights.
How would I modify this to prevent the white glow due to a transparent background?

Answer 1

This blur procedure is really not meant for transparent images so some adjustment is needed.
To see what you are doing in your code is you are applying surrounding pixel taking effect depending on their distance when you then normalize them, take their average, whatever you want to call it... What I am talking about is that your factors 0.0044299121055113265, 0.00895781211794 are normalized so that the sum of them is always 1. More naturally these values might be for instance (using only 3 pixels) scales = [1, 5, 1] where then the result is (pix[0]*scales[0] + pix[1]*scales[1] + pix[2]*scales[2])/(scales[0]+scales[1]+scales[2]).
So if we take a step back your code can be transformed into:
int offset = 7; // Maximum range taking effect
float offsetScales[offset+1] = { // +1 is for the zero offset
0.159576912161,
0.147308056121,
...
};
float sumOfScales = offsetScales[0];
for(int i=0; i<offset; i++) sumOfScales += offsetScales[i+1]*2.0;
gl_FragColor = texture2D(u_texture, v_texCoord)*offsetScales[0];
for(int i=0; i<offset; i++) {
gl_FragColor += texture2D(u_texture, v_blurTexCoords[6-i]) * offsetScales[i+1];
gl_FragColor += texture2D(u_texture, v_blurTexCoords[7+i]) * offsetScales[i+1];
}
gl_FragColor /= sumOfScales; // sumOfScales in current case is always 1.0
Now unless I made some mistakes this code should do exactly the same as your code. It is a bit more agile though because if you added another pixel in range (offset = 8) you could simply add it and set its value to 0.0022 and your color will never overflow. Where in your case you need to adjust all of the scales so their sum equals to 1.0. But never mind that, your way is closer to optimal so keep using it, I am explaining this to take a step back and find the solution to your problem...
Ok so now that the code is a bit more maintainable let's see what happens when an alpha channel needs to take effect; When a pixel is transparent or semitransparent it should take less or no effect in computing the overall color. It should take effect on the end alpha though. That means that next to those scales we need to also take the alpha scale. But doing so we need to adjust the sum of applied colors:
int offset = 7; // Maximum range taking effect
float offsetScales[offset+1] = { // +1 is for the zero offset
0.159576912161,
0.147308056121,
...
};
highp vec4 summedColor = vec4(0.0); // We best take a large value now
highp float overallAlpha = 0.0; // The actual end alpha (previously sumOfScales)
highp float overallScale = 0.0; // We need this one so alpha doesn't overflow. If the sum of original scales is 1.0 then this factor is 1.0 and is not needed at all.
vec4 fetchedColor = texture2D(u_texture, v_texCoord);
float scaleWithAlpha = fetchedColor.a * offsetScales[0];
overallScale += offsetScales[0];
summedColor += fetchedColor*scaleWithAlpha;
overallAlpha += scaleWithAlpha;
for(int i=0; i<offset; i++) {
vec4 fetchedColor = texture2D(u_texture, v_blurTexCoords[6-i]);
float scaleWithAlpha = fetchedColor.a * offsetScales[i+1];
overallScale += offsetScales[i+1];
summedColor += fetchedColor*scaleWithAlpha;
overallAlpha += scaleWithAlpha;
fetchedColor = texture2D(u_texture, v_blurTexCoords[7+i]);
scaleWithAlpha = fetchedColor.a * offsetScales[i+1];
overallScale += offsetScales[7+i];
summedColor += fetchedColor*scaleWithAlpha;
overallAlpha += scaleWithAlpha;
}
overallAlpha /= overallScale;
summedColor /= overallAlpha; // TODO: if overallAlpha is 0.0 then discard or use clear color
gl_FragColor = vec4(summedColor.xyz, overallAlpha);
Some adjustment may still be needed to this code but I hope this gets you on the right tracks. After you make it work I suggest you again lose all the loops and do it like you originally did (with the new logic). It would also be nice if you posted the code you ended up with.
Feel free to ask any questions...

Answer 2

The problem is that OpenGL ES use post-multipled alpha (it's cheaper in hardware), whereas doing it "properly" needs premultiplied alpha.
You can do the pre-multiplication maths in the shader for each sample you blur:
premult.rgb = source.rgb * source.a;
... but then you incur run-time cost for every texture sample you are blending. It's generally better to premultiply your input art assets offline during texture creation/compression.
If you need post-multiplied data for lighting computation, etc, you can make the error less visible by extruding the object color into the neighboring "transparent" pixels. E.g.:
Note if you shaders are emitting premultipled alpha fragmetn colors you'll need to fix your OpenGL blend equation (use GL_ONE for srcAlpha, not the alpha value).