What command in webgl cause texture memory to be processed - performance

Question

When setting up a single texture WebGL program what command actually causes "work" to be done?
It seems to me that the textImage2D command upload html image data to the GPU:
gl.texImage2D(target, level, internalformat, format, type, HTMLImageElement);
However once that data is uploaded and bound to a texture, that texture still needs to be "bound" to a sampler.
setActiveTexture(gl, 0, this['textureRef0']);
var samplerRef = gl.getUniformLocation(program, 'sampler0');
gl.uniform1i(samplerRef, 0);
Does any memory need to be allocated to bind the texture to the sampler? Or is it just a pointer that changes which points the sampler to the texture data?
Also what about binding textures to frame buffers?
gl.bindFramebuffer(gl.FRAMEBUFFER, this.globalFB);
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, textureRef0, 0);
Does that act alone cause any significant performance issues? Or is "real" work only done when the program is called and data is rendered into that texture?

Answer 1

texImage2D allocates memory because the driver needs to make a copy of the data you pass it as the moment texImage2D returns you can change your memory.
framebuffers don't allocate much memory, the memory is the attachments. But, framebuffers need to be validated so it's better to make multiple framebuffers with every combination of attachments you need rather than change the attachments on a single framebuffer
In other words, for example if you're doing ping ponging of textures for post processing for example
// init time
const fb = gl.createFramebuffer();
// render time
for each pass
gl.framebufferTexture2D(...., dstTex); // bad!
...
gl.drawXXX
const t = srcTex; srcTex = dstTex; dstTex = t; // swap textures
}
vs
// init time
let srcFB = gl.createFramebuffer();
gl.framebufferTexture(...., srcTex);
let dstFB = gl.createFramebuffer();
gl.framebufferTexture(...., dstTex);
// render time
for each pass
gl.bindFramebuffer(dstFB); // good
...
gl.drawXXX
const t = srcFB; srcFB = dstFB; dstFB = t; // swap framebuffers
}
textures also have the issue that because of the API design GL has a bunch of work to do the first time you draw with a texture (and any time you change it's contents).
Consider this is a normal sequence in WebGL to supply mips
texImage2D level 0, 16x16
texImage2D level 1, 8x8
texImage2D level 2, 4x4
texImage2D level 3, 2x2
texImage2D level 4, 1x1
But this is also completely valid API calls
texImage2D level 0, 16x16
texImage2D level 1, 8x8
texImage2D level 2, 137x324 // nothing in the spec prevents this. It's fully valid
texImage2D level 3, 2x2
texImage2D level 4, 1x1
texImage2D level 2, 4x4 // fix level 2 before drawing
That call to level 2 with some strange size is valid. It's not allowed to give an error. Of course if you don't replace level 2 before drawing it will fail to draw but uploading the data is not wrong according to the API. That means it isn't until the texture is actually used that the driver can look at the data, formats, and sizes for each mip, check if they are all correct, and then finally arrange the data on the GPU.
texStorage was added to fix that issue (available in WebGL2/OpenGL ES 3.0)
Calling activeTexture, binding textures with bindTexture, and setting uniforms take no memory and have no significant performance issues.