How to achieve brightness + contrast in WebGL? - opengl-es

I am trying to apply window level similar to this example ,drag your mouse over the image to see the brightness + contrast effect.
But I want to achieve the same using WebGL ,as GPU will process them more faster than CPU.
Here's what I have done:
Vertex Shader:
attribute vec3 attrVertexPos;
attribute vec2 attrTextureCoord;
varying highp vec2 vTextureCoord;
void main(void) {
gl_Position = vec4(attrVertexPos, 0.81);
vTextureCoord = attrTextureCoord;
}
Fragment Shader:
#ifdef GL_ES
precision highp float;
#endif
varying highp vec2 vTextureCoord;
uniform sampler2D uImage;
uniform float brightnessFactor;
uniform float contrastFactor;
void main(void) {
gl_FragColor = texture2D(uImage, vTextureCoord) *(brightnessFactor/contrastFactor);
}
But it doesn't work as expected according the link given above.
Here's the Javascript code:
var isMouseDown = false;
document.getElementById('canvas').onmousedown = function() { isMouseDown = true };
document.getElementById('canvas').onmouseup = function() { isMouseDown = false };
document.getElementById('canvas').onmousemove = function(e) {
if (isMouseDown) {
var intWidth = $('canvas').innerWidth();
var intHeight = $('canvas').innerHeight();
var x = (e.clientX/intWidth)*100;
var y = (e.clientY/intHeight)*100;
console.log(x/10 + ' :: ' + y/10);
brightnessVal = x/10;
gl.uniform1f(gl.getUniformLocation(program, "contrastFactor"), brightnessVal);
contrastVal = y/10;
gl.uniform1f(gl.getUniformLocation(program, "brightnessFactor"), contrastVal);
}
};

The difference is in pixel manipulation. You're just multiplying by a coefficient (also, the names of the coefficient you use are incorrect), where as in the linked exampled something a bit more complicated is happening. Some colour range (described by its center and width) from the source image gets expanded to full [0,1] range:
newColor = (oldColor - rangeCenter) / rangeWidth + 0.5
Why is it doing this is beyond my knowledge (the page is an example for medical imaging library and I don't know anything about it). Nevertheless, I've managed to port the formula to your code. First, fragment shader changes:
#ifdef GL_FRAGMENT_PRECISION_HIGH
precision highp float;
#else
precision mediump float;
#endif
varying highp vec2 vTextureCoord;
uniform sampler2D uImage;
// New uniforms
uniform float rangeCenter;
uniform float rangeWidth;
void main(void) {
vec3 c = texture2D(uImage, vTextureCoord).rgb;
gl_FragColor = vec4(
// The above formula, clamped to [0, 1]
clamp((c - windowCenter) / windowWidth + 0.5, 0.0, 1.0),
// Also, let's not screw alpha
1
);
}
As for JavaScript, I've taken the liberty to make it a bit closer to linked example:
var isMouseDown = false,
// Initially we set "equality" colour mapping
rangeCenter = 0.5,
ragneWidth = 1,
lastX, lastY;
document.getElementById('canvas').onmousedown = function(e) {
lastX = e.clientX;
lastY = e.clientY;
isMouseDown = true
};
document.getElementById('canvas').onmouseup = function() {
isMouseDown = false
};
document.getElementById('canvas').onmousemove = function(e) {
if (isMouseDown) {
var intWidth = $('canvas').innerWidth();
var intHeight = $('canvas').innerHeight();
// Change params according to cursor coords delta
rangeWidth += (e.clientX - lastX) / intWidth;
rangeCenter += (e.clientY - lastY) / intHeight;
gl.uniform1f(
gl.getUniformLocation(program, "rangeWidth"),
rangeWidth
);
gl.uniform1f(
gl.getUniformLocation(program, "rangeCenter"),
rangeCenter
);
lastX = e.clientX;
lastY = e.clientY;
}
};
Old answer:
The problem is that, according to your code, you don't redraw the
picture after mousemove event. Just insert a draw call (or several
draw calls) with appropriate parameters after setting uniforms. For
example, it may look like this:
document.getElementById('canvas').onmousemove = function(e) {
if (isMouseDown) {
// your code
gl.drawArrays(/* params, e.g gl.TRIANGLES, 0 and vertex count */);
}
};
A better way would be using requestAnimationFrame callback to
redraw. To do that, define a draw function, which will make the draw
call for you and use it a requestAnimationFrame callback:
function draw () {
/* your draw calls here */
}
document.getElementById('canvas').onmousemove = function(e) {
if (isMouseDown) {
// your code
requestAnimationFrame(draw);
}
};
P.S. Also, I'm intersted, where does this #ifdef GL_ES thing come from? It's incorrect to set highp precision based on GL_ES macro. Hardware doesn't have to support it according to the standard. The right way would be:
#ifdef GL_FRAGMENT_PRECISION_HIGH
precision highp float;
#else
precision mediump float;
#endif

Related

Decompose a GLSL mat4 to original RTS values within vertex shader to calculate a View UV Offset

I need to get the rotation differences between the model and the camera, convert the values to radians/degrees, and pass it to the fragment shader.
For that I need to decompose and the Model rotation matrix and maybe the camera view matrix as well. I cannot seem to find a way to decompose mechanism suitable within a shader.
The rotation details goes into fragment shader to calculate uv offset.
original_rotation + viewing_angles to calculate a final sprite-like offset of the following texture and shown as billboards.
Ultimately UV should offset downwards (ex:H3 to A3) looking from down, upwards looking from up (ex:A3 to H3), left to right looking and viceversa looking from sides (ex: D1 to D8 and viceversa).
const vertex_shader = `
precision highp float;
uniform mat4 modelViewMatrix;
uniform mat4 projectionMatrix;
attribute vec3 position;
attribute vec2 uv;
attribute mat4 instanceMatrix;
attribute float index;
attribute float texture_index;
uniform vec2 rows_cols;
uniform vec3 camera_location;
varying float vTexIndex;
varying vec2 vUv;
varying vec4 transformed_normal;
float normal_to_orbit(vec3 rotation_vector, vec3 view_vector){
rotation_vector = normalize(rotation_vector);
view_vector = normalize(view_vector);
vec3 x_direction = vec3(1.0,0,0);
vec3 y_direction = vec3(0,1.0,0);
vec3 z_direction = vec3(0,0,1.0);
float rotation_x_length = dot(rotation_vector, x_direction);
float rotation_y_length = dot(rotation_vector, y_direction);
float rotation_z_length = dot(rotation_vector, z_direction);
float view_x_length = dot(view_vector, x_direction);
float view_y_length = dot(view_vector, y_direction);
float view_z_length = dot(view_vector, z_direction);
//TOP
float top_rotation = degrees(atan(rotation_x_length, rotation_z_length));
float top_view = degrees(atan(view_x_length, view_z_length));
float top_final = top_view-top_rotation;
float top_idx = floor(top_final/(360.0/rows_cols.x));
//FRONT
float front_rotation = degrees(atan(rotation_x_length, rotation_z_length));
float front_view = degrees(atan(view_x_length, view_z_length));
float front_final = front_view-front_rotation;
float front_idx = floor(front_final/(360.0/rows_cols.y));
return abs((front_idx*rows_cols.x)+top_idx);
}
vec3 extractEulerAngleXYZ(mat4 mat) {
vec3 rotangles = vec3(0,0,0);
rotangles.x = atan(mat[2].z, -mat[1].z);
float cosYangle = sqrt(pow(mat[0].x, 2.0) + pow(mat[0].y, 2.0));
rotangles.y = atan(cosYangle, mat[0].z);
float sinXangle = sin(rotangles.x);
float cosXangle = cos(rotangles.x);
rotangles.z = atan(cosXangle * mat[1].y + sinXangle * mat[2].y, cosXangle * mat[1].x + sinXangle * mat[2].x);
return rotangles;
}
float view_index(vec3 position, mat4 mv_matrix, mat4 rot_matrix){
vec4 posInView = mv_matrix * vec4(0.0, 0.0, 0.0, 1.0);
// posInView /= posInView[3];
vec3 VinView = normalize(-posInView.xyz); // (0, 0, 0) - posInView
// vec4 NinView = normalize(rot_matrix * vec4(0.0, 0.0, 1.0, 1.0));
// float NdotV = dot(NinView, VinView);
vec4 view_normal = rot_matrix * vec4(VinView.xyz, 1.0);
float view_x_length = dot(view_normal.xyz, vec3(1.0,0,0));
float view_y_length = dot(view_normal.xyz, vec3(0,1.0,0));
float view_z_length = dot(view_normal.xyz, vec3(0,0,1.0));
// float radians = atan(-view_x_length, -view_z_length);
float radians = atan(view_x_length, view_z_length);
// float angle = radians/PI*180.0 + 180.0;
float angle = degrees(radians);
if (radians < 0.0) { angle += 360.0; }
if (0.0<=angle && angle<=360.0){
return floor(angle/(360.0/rows_cols.x));
}
return 0.0;
}
void main(){
vec4 original_normal = vec4(0.0, 0.0, 1.0, 1.0);
// transformed_normal = modelViewMatrix * instanceMatrix * original_normal;
vec3 rotangles = extractEulerAngleXYZ(modelViewMatrix * instanceMatrix);
// transformed_normal = vec4(rotangles.xyz, 1.0);
transformed_normal = vec4(camera_location.xyz, 1.0);
vec4 v = (modelViewMatrix* instanceMatrix* vec4(0.0, 0.0, 0.0, 1.0)) + vec4(position.x, position.y, 0.0, 0.0) * vec4(1.0, 1.0, 1.0, 1.0);
vec4 model_center = (modelViewMatrix* instanceMatrix* vec4(0.0, 0.0, 0.0, 1.0));
vec4 model_normal = (modelViewMatrix* instanceMatrix* vec4(0.0, 0.0, 1.0, 1.0));
vec4 cam_loc = vec4(camera_location.xyz, 1.0);
vec4 view_vector = normalize((cam_loc-model_center));
//float findex = normal_to_orbit(model_normal.xyz, view_vector.xyz);
float findex = view_index(position, base_matrix, combined_rot);
vTexIndex = texture_index;
vUv = vec2(mod(findex,rows_cols.x)/rows_cols.x, floor(findex/rows_cols.x)/rows_cols.y) + (uv / rows_cols);
//vUv = vec2(mod(index,rows_cols.x)/rows_cols.x, floor(index/rows_cols.x)/rows_cols.y) + (uv / rows_cols);
gl_Position = projectionMatrix * v;
// gl_Position = projectionMatrix * modelViewMatrix * instanceMatrix * vec4(position, 1.0);
}
`
const fragment_shader = (texture_count) => {
var fragShader = `
precision highp float;
uniform sampler2D textures[${texture_count}];
varying float vTexIndex;
varying vec2 vUv;
varying vec4 transformed_normal;
void main() {
vec4 finalColor;
`;
for (var i = 0; i < texture_count; i++) {
if (i == 0) {
fragShader += `if (vTexIndex < ${i}.5) {
finalColor = texture2D(textures[${i}], vUv);
}
`
} else {
fragShader += `else if (vTexIndex < ${i}.5) {
finalColor = texture2D(textures[${i}], vUv);
}
`
}
}
//fragShader += `gl_FragColor = finalColor * transformed_normal; }`;
fragShader += `gl_FragColor = finalColor; }`;
// fragShader += `gl_FragColor = startColor * finalColor; }`;
// int index = int(v_TexIndex+0.5); //https://stackoverflow.com/questions/60896915/texture-slot-not-getting-picked-properly-in-shader-issue
//console.log('frag shader: ', fragShader)
return fragShader;
}
function reset_instance_positions() {
const dummy = new THREE.Object3D();
const offset = 500*4
for (var i = 0; i < max_instances; i++) {
dummy.position.set(offset-(Math.floor(i % 8)*500), offset-(Math.floor(i / 8)*500), 0);
dummy.updateMatrix();
mesh.setMatrixAt(i, dummy.matrix);
}
mesh.instanceMatrix.needsUpdate = true;
}
function setup_geometry() {
const geometry = new THREE.InstancedBufferGeometry().copy(new THREE.PlaneBufferGeometry(400, 400));
const index = new Float32Array(max_instances * 1); // index
for (let i = 0; i < max_instances; i++) {
index[i] = (i % max_instances) * 1.0 /* index[i] = 0.0 */
}
geometry.setAttribute("index", new THREE.InstancedBufferAttribute(index, 1));
const texture_index = new Float32Array(max_instances * 1); // texture_index
const max_maps = 1
for (let i = 0; i < max_instances; i++) {
texture_index[i] = (Math.floor(i / max_instances) % max_maps) * 1.0 /* index[i] = 0.0 */
}
geometry.setAttribute("texture_index", new THREE.InstancedBufferAttribute(texture_index, 1));
const textures = [texture]
const grid_xy = new THREE.Vector2(8, 8)
mesh = new THREE.InstancedMesh(geometry,
new THREE.RawShaderMaterial({
uniforms: {
textures: {
type: 'tv',
value: textures
},
rows_cols: {
value: new THREE.Vector2(grid_xy.x * 1.0, grid_xy.y * 1.0)
},
camera_location: {
value: camera.position
}
},
vertexShader: vertex_shader,
fragmentShader: fragment_shader(textures.length),
side: THREE.DoubleSide,
// transparent: true,
}), max_instances);
scene.add(mesh);
reset_instance_positions()
}
var camera, scene, mesh, renderer;
const max_instances = 64
function init() {
camera = new THREE.PerspectiveCamera(60, window.innerWidth / window.innerHeight,1, 10000 );
camera.position.z = 1024;
scene = new THREE.Scene();
scene.background = new THREE.Color(0xffffff);
setup_geometry()
var canvas = document.createElement('canvas');
var context = canvas.getContext('webgl2');
renderer = new THREE.WebGLRenderer({
canvas: canvas,
context: context
});
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
window.addEventListener('resize', onWindowResize, false);
var controls = new THREE.OrbitControls(camera, renderer.domElement);
}
function onWindowResize() {
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(window.innerWidth, window.innerHeight);
}
function animate() {
requestAnimationFrame(animate);
renderer.render(scene, camera);
}
var dataurl = "https://i.stack.imgur.com/accaU.png"
var texture;
var imageElement = document.createElement('img');
imageElement.onload = function(e) {
texture = new THREE.Texture(this);
texture.needsUpdate = true;
init();
animate();
};
imageElement.src = dataurl;
JSFiddle of work so far
So You got 4x4 transform matrix M used on xy plane QUAD and want to map its 4 corners (p0,p1,p2,p3) to your texture with "repaeat" like manner (crossing border from left/right/up/down will return right/left/down/up) based on direction of Z axis of the matrix.
You face 2 problems...
M rotation is 3 DOF and you want just 2 DOF (yaw,pitch) so if roll present the result might be questionable
if texture crosses borders you need to handle this in GLSL to avoid seems
so either do this in geometry shader and divide the quad to more if needed or use enlarged texture where you have the needed overlaps ...
Now if I did not miss something the conversion is like this:
const float pi=3.1415926535897932384626433832795;
vec3 d = normalize(z axis from M);
vec2 dd = normalize(d.xy);
u = atan2(dd.y,dd.x);
v = acos(d.z);
u = (u+pi)/(2.0*pi);
v = v/pi
The z axis extraction is just simple copy of 3th column/row (depends on your notation) from your matrix 'M' or transforming (1,0,0,0) by it. For more info see:
Understanding 4x4 homogenous transform matrices
In case of overlapped texture you need to add also this:
const float ov = 1.0/8.0; // overlap size
u = ov + (u/(ov+ov+1.0));
v = ov + (v/(ov+ov+1.0));
And the texture would look like:
In case your quads cover more than 1/8 of your original texture you need to enlarge the overlap ...
Now to handle the corners of QUAD instead of just axis you could translate the quad by distance l in Z+ direction in mesh local coordinates, apply the M on them and use those 4 points as directions to compute u,v in vertex shader. The l will affect how much of the texture area is used for quad ... This approach might even handle roll but did not test any of this yet...
After implementing it my fears was well grounded as any 2 euler angles affect each other so the result is OK on most of the directions but in edge cases the stuff get mirrored and or jumped in one or both axises probably due to area coverage difference between 3 DOF and 2 DOF (unless I made a bug in my code or the math was not computed correctly in vertex which happened to me before due to bug in drivers)
If you going for azimut/elevation that should be fine as its 2 DOF too the equation above shoul dwork for them too +/- some range conversion if needed.

Black screen when trying to use multiple textures in Expo

I'm trying to use multiple textures in expo, but for some reason whenever I try to bind more than one texture, even if the texture is unused in the shader, nothing renders to the screen.
Code that binds textures:
gl.useProgram(program);
gl.bindVertexArray(vao);
// Set texture sampler uniform
const TEXTURE_UNIT = 0;
const TEXTURE_UNIT2 = 1;
gl.uniform1i(uniformLocations.get('cameraTexture'), TEXTURE_UNIT);
gl.uniform1i(uniformLocations.get('segmentationTexture'), TEXTURE_UNIT2);
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, cameraTexture);
gl.activeTexture(gl.TEXTURE1);
gl.bindTexture(gl.TEXTURE_2D, segmentationTexture);
gl.bindFramebuffer(gl.FRAMEBUFFER, null);
gl.viewport(0, 0, dims.width, dims.height);
gl.drawArrays(gl.TRIANGLES, 0, vertices.length / 2);
//console.log("draws")
gl.bindVertexArray(null);
gl.bindTexture(gl.TEXTURE_2D, null);
gl.useProgram(null);
Fragment shader:
#version 300 es
precision highp float;
uniform sampler2D cameraTexture;
uniform sampler2D segmentationTexture;
in vec2 uv;
out vec4 fragColor;
void main() {
fragColor = texture(cameraTexture, uv); //segmentationTexture.r *
}
Vertex Shader:
const horizontalScale = flipHorizontal ? -1 : 1;
return `#version 300 es
precision highp float;
in vec2 position;
in vec2 texCoords;
out vec2 uv;
void main() {
// Invert geometry to match the image orientation from the camera.
gl_Position = vec4(position * vec2(${horizontalScale}., -1.), 0, 1);
uv = texCoords;
}`
I have checked both textures by commenting out the code the binds the other texture, e.g.
const TEXTURE_UNIT = 0;
//const TEXTURE_UNIT2 = 1;
gl.uniform1i(uniformLocations.get('cameraTexture'), TEXTURE_UNIT);
//gl.uniform1i(uniformLocations.get('segmentationTexture'), TEXTURE_UNIT2);
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, cameraTexture);
//gl.activeTexture(gl.TEXTURE1);
//gl.bindTexture(gl.TEXTURE_2D, segmentationTexture);
or
//const TEXTURE_UNIT = 0;
const TEXTURE_UNIT2 = 1;
//gl.uniform1i(uniformLocations.get('cameraTexture'), TEXTURE_UNIT);
gl.uniform1i(uniformLocations.get('segmentationTexture'), TEXTURE_UNIT2);
//gl.activeTexture(gl.TEXTURE0);
//gl.bindTexture(gl.TEXTURE_2D, cameraTexture);
gl.activeTexture(gl.TEXTURE1);
gl.bindTexture(gl.TEXTURE_2D, segmentationTexture);
and both render fine individually.
However, the act of trying to bind multiple textures, even though only one is used in frag shader, leaves me with only a black screen.

apply LUT to an image GLSL

I am very new to CG and am trying to implement a fragment shader that applies a png LUT to a picture, but I don't get the expected result, right now my code makes the picture very blue-ish.
Here is an example LUT :
[![enter image description here][1]][1]
When I apply the LUT using the following code to some image the whole picture just turns very blue-ish.
Code :
precision mediump float;
uniform sampler2D u_image;
uniform sampler2D u_lut;
// LUT resolution for one component (4, 8, 16, ...)
uniform float u_resolution;
layout(location = 0) out vec4 fragColor;
in vec2 v_uv;
void main(void)
{
vec2 tiles = vec2(u_resolution, u_resolution);
vec2 tilesSize = vec2(u_resolution * u_resolution);
vec3 imageColor = texture(u_image, v_uv).rgb;
// min and max are used to interpolate between 2 tiles in the LUT
float index = imageColor.b * (tiles.x * tiles.y - 1.0);
float index_min = min(u_resolution - 2.0, floor(index));
float index_max = index_min + 1.0;
vec2 tileIndex_min;
tileIndex_min.y = floor(index_min / tiles.x);
tileIndex_min.x = floor(index_min - tileIndex_min.y * tiles.x);
vec2 tileIndex_max;
tileIndex_max.y = floor(index_max / tiles.x);
tileIndex_max.x = floor(index_max - tileIndex_max.y * tiles.x);
vec2 tileUV = mix(0.5/tilesSize, (tilesSize - 0.5)/tilesSize, imageColor.rg);
vec2 tableUV_1 = tileIndex_min / tiles + tileUV / tiles;
vec2 tableUV_2 = tileIndex_max / tiles + tileUV / tiles;
vec3 lookUpColor_1 = texture(u_lut, tableUV_1).rgb;
vec3 lookUpColor_2 = texture(u_lut, tableUV_2).rgb;
vec3 lookUpColor = mix(lookUpColor_1, lookUpColor_2, index - index_min);
fragColor = vec4(lookUpColor, 1.0);
}
Since you're using WebGL2 you can just use a 3D texture
#version 300 es
precision highp float;
in vec2 vUV;
uniform sampler2D uImage;
uniform mediump sampler3D uLUT;
out vec4 outColor;
void main() {
vec4 color = texture(uImage, vUV);
vec3 lutSize = vec3(textureSize(uLUT, 0));
vec3 uvw = (color.rgb * float(lutSize - 1.0) + 0.5) / lutSize;
outColor = texture(uLUT, uvw);
}
And you can use UNPACK_ROW_LENGTH and UNPACK_SKIP_PIXELS to load slice of a PNG into a 3D texture
function createLUTTexture(gl, img, filter, size = 8) {
const tex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_3D, tex);
gl.texStorage3D(gl.TEXTURE_3D, 1, gl.RGBA8, size, size, size);
// grab slices
for (let z = 0; z < size; ++z) {
gl.pixelStorei(gl.UNPACK_SKIP_PIXELS, z * size);
gl.pixelStorei(gl.UNPACK_ROW_LENGTH, img.width);
gl.texSubImage3D(
gl.TEXTURE_3D,
0, // mip level
0, // x
0, // y
z, // z
size, // width,
size, // height,
1, // depth
gl.RGBA,
gl.UNSIGNED_BYTE,
img,
);
}
gl.texParameteri(gl.TEXTURE_3D, gl.TEXTURE_MIN_FILTER, filter);
gl.texParameteri(gl.TEXTURE_3D, gl.TEXTURE_MAG_FILTER, filter);
return tex;
}
Example:
const fs = `#version 300 es
precision highp float;
in vec2 vUV;
uniform sampler2D uImage;
uniform mediump sampler3D uLUT;
out vec4 outColor;
void main() {
vec4 color = texture(uImage, vUV);
vec3 lutSize = vec3(textureSize(uLUT, 0));
vec3 uvw = (color.rgb * float(lutSize - 1.0) + 0.5) / lutSize;
outColor = texture(uLUT, uvw);
}
`;
const vs = `#version 300 es
in vec4 position;
in vec2 texcoord;
out vec2 vUV;
void main() {
gl_Position = position;
vUV = texcoord;
}
`;
const lutURLs = [
'default.png',
'bgy.png',
'-black-white.png',
'blues.png',
'color-negative.png',
'funky-contrast.png',
'googley.png',
'high-contrast-bw.png',
'hue-minus-60.png',
'hue-plus-60.png',
'hue-plus-180.png',
'infrared.png',
'inverse.png',
'monochrome.png',
'nightvision.png',
'-posterize-3-lab.png',
'-posterize-3-rgb.png',
'-posterize-4-lab.png',
'-posterize-more.png',
'-posterize.png',
'radioactive.png',
'red-to-cyan.png',
'saturated.png',
'sepia.png',
'thermal.png',
];
let luts = {};
const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
async function main() {
const gl = document.querySelector('canvas').getContext('webgl2');
if (!gl) {
alert('need WebGL2');
return;
}
const img = await loadImage('https://i.imgur.com/CwQSMv9.jpg');
document.querySelector('#img').append(img);
const imgTexture = twgl.createTexture(gl, {src: img, yFlip: true});
// compile shaders, link program, lookup locatios
const programInfo = twgl.createProgramInfo(gl, [vs, fs]);
// calls gl.createBuffer, gl.bindBuffer, gl.bufferData for
// a plane with positions, and texcoords
const bufferInfo = twgl.primitives.createXYQuadBufferInfo(gl, 2);
gl.useProgram(programInfo.program);
// calls gl.bindBuffer, gl.enableVertexAttribArray, gl.vertexAttribPointer
twgl.setBuffersAndAttributes(gl, programInfo, bufferInfo);
gl.activeTexture(gl.TEXTURE0 + 1);
for (;;) {
for (let name of lutURLs) {
let lut = luts[name];
if (!lut) {
let url = name;
let filter = gl.LINEAR;
if (url.startsWith('-')) {
filter = gl.NEAREST;
url = url.substr(1);
}
const lutImg = await loadImage(`https://webglsamples.org/color-adjust/adjustments/${url}`);
lut = {
name: url,
texture: createLUTTexture(gl, lutImg, filter),
};
luts[name] = lut;
}
document.querySelector('#info').textContent = lut.name;
// calls gl.uniformXXX, gl.activeTexture, gl.bindTexture
twgl.setUniformsAndBindTextures(programInfo, {
uImg: imgTexture,
uLUT: lut.texture,
});
// calls gl.drawArrays or gl.drawElements
twgl.drawBufferInfo(gl, bufferInfo);
await wait(1000);
}
}
}
main();
function createLUTTexture(gl, img, filter, size = 8) {
const tex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_3D, tex);
gl.texStorage3D(gl.TEXTURE_3D, 1, gl.RGBA8, size, size, size);
// grab slices
for (let z = 0; z < size; ++z) {
gl.pixelStorei(gl.UNPACK_SKIP_PIXELS, z * size);
gl.pixelStorei(gl.UNPACK_ROW_LENGTH, img.width);
gl.pixelStorei(gl.UNPACK_SKIP_PIXELS, z * size);
gl.pixelStorei(gl.UNPACK_ROW_LENGTH, img.width);
gl.texSubImage3D(
gl.TEXTURE_3D,
0, // mip level
0, // x
0, // y
z, // z
size, // width,
size, // height,
1, // depth
gl.RGBA,
gl.UNSIGNED_BYTE,
img,
);
}
gl.texParameteri(gl.TEXTURE_3D, gl.TEXTURE_MIN_FILTER, filter);
gl.texParameteri(gl.TEXTURE_3D, gl.TEXTURE_MAG_FILTER, filter);
return tex;
}
function loadImage(url) {
return new Promise((resolve, reject) => {
const img = new Image();
img.onerror = reject;
img.onload = () => resolve(img);
img.crossOrigin = "anonymous";
img.src = url;
});
}
.split { display: flex; }
.split>div { padding: 5px; }
img { width: 150px; }
<div class="split">
<div>
<div id="img"></div>
<div>original</div>
</div>
<div>
<canvas width="150" height="198"></canvas>
<div>LUT Applied: <span id="info"></span></div>
</div>
</div>
<script src="https://twgljs.org/dist/4.x/twgl-full.min.js"></script>
As for doing it in 2D there's this which as a video explaining it linked at the top. There's also this if you want to look at a shader that works.

Storing data as a texture for use in Vertex Shader for Instanced Geometry (THREE JS / GLSL)

I'm using a THREE.InstancedBufferGeometry, and I wish to access data in the Vertex Shader, encoded into a Texture.
What I want to do, is create a Data Texture with one pixel per instance, which will store position data for each instance (then at a later stage, I can update the texture using a simulation with a flow field to animate the positions).
I'm struggling to access the data from the texture in the Vertex Shader.
const INSTANCES_COUNT = 5000;
// FOR EVERY INSTANCE, GIVE IT A RANDOM X, Y, Z OFFSET, AND SAVE IT IN DATA TEXTURE
const data = new Uint8Array(4 * INSTANCES_COUNT);
for (let i = 0; i < INSTANCES_COUNT; i++) {
const stride = i * 4;
data[stride] = (Math.random() - 0.5);
data[stride + 1] = (Math.random() - 0.5);
data[stride + 2] = (Math.random() - 0.5);
data[stride + 3] = 0.0;
}
const offsetTexture = new THREE.DataTexture( data, INSTANCES, 1, THREE.RGBAFormat, THREE.FloatType );
offsetTexture.minFilter = THREE.NearestFilter;
offsetTexture.magFilter = THREE.NearestFilter;
offsetTexture.generateMipmaps = false;
offsetTexture.needsUpdate = true;
// CREATE MY INSTANCED GEOMETRY
const geometry = new THREE.InstancedBufferGeometry();
geometry.maxInstancedCount = INSTANCES_COUNT;
geometry.addAttribute( 'position', new THREE.Float32BufferAttribute([5, -5, 0, -5, 5, 0, 0, 0, 5], 3 )); // SIMPLE TRIANGLE
const vertexShader = `
precision highp float;
uniform vec3 color;
uniform sampler2D offsetTexture;
uniform mat4 modelViewMatrix;
uniform mat4 projectionMatrix;
attribute vec3 position;
varying vec3 vPosition;
varying vec3 vColor;
void main(){
vPosition = position;
vec4 orientation = vec4(.0, .0, .0, .0);
vec3 vcV = cross( orientation.xyz, vPosition );
vPosition = vcV * ( 2.0 * orientation.w ) + ( cross( orientation.xyz, vcV ) * 2.0 + vPosition );
vec2 uv = position.xy;
vec4 data = texture2D( offsetTexture, uv );
vec3 particlePosition = data.xyz * 1000.0;
gl_Position = projectionMatrix * modelViewMatrix * vec4( vPosition + particlePosition, 1.0 );
}
`;
const fragmentShader = `
precision highp float;
varying vec3 vColor;
void main() {
gl_FragColor = vec4(vColor, 1.0);
}
`;
const uniforms = {
size: { value: 1.0 },
color: {
type: 'c',
value: new THREE.Color(0x3db230),
},
offsetTexture: {
type: 't',
value: offsetTexture,
},
};
// CREATE MY MATERIAL
const material = new THREE.RawShaderMaterial({
uniforms,
vertexShader,
fragmentShader,
side: THREE.DoubleSide,
transparent: false,
});
scene.add(new THREE.Mesh(geometry, material));
At the moment it seems that the data from the image isn't accessible in the vertex shader (if I just set the vUv to vec2(1.0, 0.0), for example, and change the offset positions, nothing changes), and also I'm not sure how to go about making sure that the instance can reference the correct texel in the texture.
So, my two issues are:
1) How to correctly set the Data Image Texture, and access that data in the Vertex Shader
2) How to correctly reference the texel storing the data for each particular instance (e.g, instance 1000 should use vec2(1000,1), etc
Also, do I have to normalize the data (0.0-1.0, or 0–255, or -1 – +1)
Thanks
You need to compute some kind of an index into the texture per instance.
Meaning, you need an attribute that is going to be shared by each instance.
if your triangle is
[a,b,c]
your index should be
[0,0,0]
Lets say you have 1024 instances and a 1024x1 px texture.
attribute float aIndex;
vec2 myIndex = ((aIndex + 0.5)/1024.,1.);
vec4 myRes = texture2D( mySampler, myIndex);

Shader wireframe of an object

I want to see a wireframe of an object without the diagonals like
Currently, I add lines according to the vertices, the problem is after I have several of those I experience a major performance degradation.
The examples here are either too new for my version of Three or don't work (I commented there about it).
So I want to try to implement a shader instead.
I tried to use this shader: https://stackoverflow.com/a/31610464/4279201 but it breaks the shape to parts and I'm getting WebGL errors.
That's how I use it:
const vertexShader = `
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position,1.0);
}
`
const fragmentShader = `
#version 150 compatibility
flat in float diffuse;
flat in float specular;
flat in vec3 edge_mask;
in vec2 bary;
uniform float mesh_width = 1.0;
uniform vec3 mesh_color = vec3(0.0, 0.0, 0.0);
uniform bool lighting = true;
out vec4 frag_color ;
float edge_factor(){
vec3 bary3 = vec3(bary.x, bary.y, 1.0 - bary.x - bary.y);
vec3 d = fwidth(bary3);
vec3 a3 = smoothstep(vec3(0.0, 0.0, 0.0), d * mesh_width, bary3);
a3 = vec3(1.0, 1.0, 1.0) - edge_mask + edge_mask * a3;
return min(min(a3.x, a3.y), a3.z);
}
void main() {
float s = (lighting && gl_FrontFacing) ? 1.0 : -1.0;
vec4 Kdiff = gl_FrontFacing ?
gl_FrontMaterial.diffuse : gl_BackMaterial.diffuse;
float sdiffuse = s * diffuse;
vec4 result = vec4(0.1, 0.1, 0.1, 1.0);
if (sdiffuse > 0.0) {
result += sdiffuse * Kdiff +
specular * gl_FrontMaterial.specular;
}
frag_color = (mesh_width != 0.0) ?
mix(vec4(mesh_color, 1.0), result, edge_factor()) :
result;
}`
...
const uniforms = {
color: {
value: new THREE.Vector4(0, 0, 1, 1),
type: 'v4'
}
}
const material = new THREE.ShaderMaterial({
fragmentShader: data.fragmentShader,
vertexShader: data.vertexShader,
uniforms
})
this._viewer.impl.matman().addMaterial(
data.name, material, true)
const fragList = this._viewer.model.getFragmentList()
this.toArray(fragIds).forEach((fragId) => {
fragList.setMaterial(fragId, material)
})
So to implement this shader, is the right approach would be to basically check the angle between every two vertices, and draw a line if the degree is 90?
How can I have access to all the vertices of the shape from the vertex shader?
And how can I tell the fragment shader to draw a line between two vertices that match the above condition? (also to leave the default shading for everything else as is)
I'm using Autodesk viewer that uses Three.js rev 71.
// -- Vertex Shader --
precision mediump float;
// Input from buffers
attribute vec3 aPosition;
attribute vec2 aBaryCoord;
// Value interpolated accross pixels and passed to the fragment shader
varying vec2 vBaryCoord;
// Uniforms
uniform mat4 uModelMatrix;
uniform mat4 uViewMatrix;
uniform mat4 uProjMatrix;
void main() {
vBaryCoord = aBaryCoord;
gl_Position = uProjMatrix * uViewMatrix * uModelMatrix * vec4(aPosition,1.0);
}
// ---------------------
// -- Fragment Shader --
// This shader doesn't perform any lighting
precision mediump float;
varying vec2 vBaryCoord;
uniform vec3 uMeshColour;
float edgeFactor() {
vec3 d = fwidth(vBaryCoord);
vec3 a3 = smoothstep(vec3(0.0,0.0,0.0),d * 1.5,vBaryCoord);
return min(min(a3.x,a3.y),a3.z);
}
void main() {
gl_FragColor = vec4(uMeshColour,(1.0 - edgeFactor()) * 0.95);
}
// ---------------------
/*
This code isn't tested so take it with a grain of salt
Idea taken from
http://codeflow.org/entries/2012/aug/02/easy-wireframe-display-with-barycentric-coordinates/
*/

Resources