Three.js fragment shader with ambient light intensity - three.js

Question

I have a Three.js shader material that chromakeys a video texture so that the green becomes transparent. That part works fine.
Now i am trying to modify it so that it is affected by the intensity of the ambient light, basically what i want is that when the ambient light's intensity is lower, the video playing becomes darker.
On images I can do that fine by simply adding a Standard Material so i've tried adding two separate materials to the video (the chromakey shader material and a standard one) but that didn't help.
So I started doing some research and digging into the code of the chromakey shader (which was not written by me) and i made the following changes:
I've merged the original uniforms with the ones from THREE.UniformsLib["lights"]
I've enabled the lights in the shader material's parameters
Now the question is, how do I access the ambient light's intensity value (which is constantly updating by the way) inside the fragment shader, and how do I make the pixels darker depending on the intensity value (which is between 0 and 1).
Shader Material
var uniforms = THREE.UniformsUtils.clone(THREE.UniformsLib["lights"]);
uniforms['color'] = { type: 'c', value: data.color };
uniforms['texture'] = { type: 't', value: videoTexture };
this.material = new THREE.ShaderMaterial({
uniforms: uniforms,
vertexShader: this.vertexShader,
fragmentShader: this.fragmentShader,
lights: true
});
Vertex Shader
varying vec2 vUv;
void main(void)
{
vUv = uv;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_Position = projectionMatrix * mvPosition;
}
Fragment Shader
uniform sampler2D texture;
uniform vec3 color;
varying vec2 vUv;
void main(void)
{
vec3 tColor = texture2D( texture, vUv ).rgb;
float a = (length(tColor - color) - 0.5) * 7.0;
gl_FragColor = vec4(tColor, a);
}
I basically need to modify tColor according to the light's intensity but like I said, I have no idea how to access that value and how to darken/brighten the color according to it.

Answer 1

Adding brightness control to you fragment shader
If its just a simple brightness control you can multiply the frag color by a scalar.
Example frag shader
uniform sampler2D texture;
uniform vec3 color;
uniform float brightness; // added uniform to control brightness
varying vec2 vUv;
void main(void) {
vec3 tColor = texture2D( texture, vUv ).rgb;
float a = (length(tColor - color) - 0.5) * 7.0;
gl_FragColor = vec4(tColor * brightness, a); // scale brightness of rgb channels
}
Then in Javascript code to support the new uniform
const BRIGHTNESS_MAX = 1; // default value. MAX brightness DO not change this
// DO NOT CHANGE this value to set upper limit
// Use the const LUX_MAX (see below) to set upper limit
const BRIGHTNESS_MIN = 0.7; // The darkest you want A value of 0 is black
// add brightness The same uniforms object as you
// got with uniforms = THREE.UniformsUtils.clone(THREE.UniformsLib["lights"]);
uniforms.brightness = {value: BRIGHTNESS_MAX}; // default to max
To change the value of the uniform
uniforms.brightness.value = brightness; // set the new value
From Three Docs
"All uniforms values can be changed freely (e.g. colors, textures, opacity, etc), values are sent to the shader every frame."
So that is all that is needed to add the brightness control.
Using Ambient sensor
I will assume you have access to the sensor value. The sensor holds the light level as an absolute value LUX 10 is dark ~707 is normal and 10000 plus is bright.
You will have to calibrate the sensor reading by changing what LUX corresponds to BRIGHTNESS_MAX and setting the BRIGHTNESS_MIN to the darkest you want the image to become.
As the the scaling and dynamic range of the light sensor and display device are very different the following function makes the assumption the MAX_LUX and white on the rendered image are the same brightness
The following function will convert from a LUX value to a brightness value
const MAX_LUX = 5000; // This LUX value and above will set brightness to max
function LUX2Brightness(lux) {
if (lux >= MAX_LUX) { return BRIGHTNESS_MAX }
const MIN = (BRIGHTNESS_MIN ** 2.2) * MAX_LUX; // do not manually set this value
// Set BRIGHTNESS_MIN to control
// low light level
if (lux <= MIN) { return BRIGHTNESS_MIN }
return (lux ** (1 / 2.2)) / (MAX_LUX ** (1 / 2.2));
}
To use the above function with the shader
// luminosity is value from ambient light sensor event
uniforms.brightness.value = LUX2Brightness(luminosity);
The assumption is that you set MAX_LUX to the actual LUX output of the all white rendered image (best of luck with that).
IMPORTANT!!
The is no absolute solution to levels.
Human vision is adaptive. How you calibrate the min and max will change depending on how your eyes have adapted to the current light levels, the current brightness, color, (and more) setting on the device displaying the rendered content, the current setting of the camera, exposure, white balance (and so on), your personal artist preferences.
All of these things are usually set automatically so any setting that looks good now may not be what is desired in the morning, or when you come back from a coffee break.
All the code
Fragment shader
uniform sampler2D texture;
uniform vec3 color;
uniform float brightness;
varying vec2 vUv;
void main(void) {
vec3 tColor = texture2D( texture, vUv ).rgb;
float a = (length(tColor - color) - 0.5) * 7.0;
gl_FragColor = vec4(tColor * brightness, a);
}
JavaScript settup code
const BRIGHTNESS_MAX = 1; // Don't change this value
const BRIGHTNESS_MIN = 0.7;
const MAX_LUX = 2000;
uniforms.brightness = {value: BRIGHTNESS_MAX};
function LUX2Brightness(lux) {
if (lux >= MAX_LUX) { return BRIGHTNESS_MAX }
const MIN = (BRIGHTNESS_MIN ** 2.2) * MAX_LUX;
if (lux <= MIN) { return BRIGHTNESS_MIN }
return (lux ** (1 / 2.2)) / (MAX_LUX ** (1 / 2.2));
}
Sensor reading
Put following line in sensor event. Eg the "devicelight" event listener.
uniforms.brightness.value = LUX2Brightness(event.value);