I'm implementing camera filter application using GLES like GPUImage on mobile devices.
In order to create final filtered image, need to render sub-filters on the previously filtered image. My approach is to create multiple FBOs for each filter step.
For example, If i use three filters to get final result then create 3 FBOs. Bind FBO1 and draw, next bind FBO2 using FBO1 texture draw and so on.
However, since I don't need FBO1's content when I draw on to FBO3 using FBO2 texture, if I re-use FBO1 instead FBO3 then I don't need to create many FBOs.
I'm not sure which one would better or any performance benefit, just create multiple FBOs for each filter step? or switching between first and second FBOs. Thanks in advance.
I show some code example..
glBindFramebuffer(fbo1);
glClear();
glBindTexture(camera-texture);
draw();
glBindFramebuffer(fbo2);
glClear();
glBindTexture(fbo1-texture);
draw();
glBindFramebuffer(fbo3);
glClear();
glBindTexture(fbo2-texture);
draw();
...
or
If i swtich FBOs..
glBindFramebuffer(fbo1);
glBindTexture(camera-texture);
glClear();
draw();
glBindFramebuffer(fbo2);
glClear();
glBindTexture(fbo1-texture);
draw();
glBindFramebuffer(fbo1);
glClear();
glBindTexture(fbo2-texture);
draw();
...
For performance, I doubt it really makes any difference - you're doing the same number of API calls.
The big difference is likely to be memory footprint - cycling between a small number of FBOs and so reusing their attachments multiple times within a single frame, will reduce the total number of surfaces you need allocated compared to a scheme where you have unique attachments per render pass.
Related
PIXI.js has Container#cacheAsBitmap which causes the container to "render" itself to an image, save that, render the image instead of its children and when a child is added or removed or updated, the cache is updated.
What's the alternative for Three.js (but instead of an image it would be a mesh)?
I may not be understanding your question properly, but your reply to Sabee's answer was helpful. It sounds like you're looking to either merge multiple geometries into a single mesh or implement a form of model instancing, with the goal of reducing draw calls.
There is more than one way to accomplish this, depending on your requirements. You can merge multiple geometries into a single geometry object, and provide either one material or an array of materials (where each index corresponds to one of the merged geometries). You can also use GPU-accelerated instancing to achieve a similar effect with only a single copy of the geometry.
I'll refer to Dusan Bosnjak's excellent Medium series on instancing, which starts here: https://medium.com/#pailhead011/instancing-with-three-js-36b4b62bc127
As well, here are the three.js examples regarding instancing: https://threejs.org/examples/?q=instanc#webgl_buffergeometry_instancing_dynamic
Pixi.js is a 2D javascript library, using WebGL to render the images(frames) into html5 canvas. Three.js allows the creation of Graphical Processing Unit (GPU)-accelerated 3D animations using WebGL.
The browser cannot store rendered 3D frames, this work allows the GPU Accelerated Render Cache, which depends on what hardware's they run. Helpful post understanding what's going on behind the scenes.
But you can cahce your assets in browser like images, json objects of 3D models and etc.
In Three.js Cache class is a global object, used by assets loaders (TextureLoader, ImageLoader, AudioLoader ...), by default is disabled (false). To enable it you can set THREE.Cache.enabled = true ;
I think by default the browser should cache the textures for performance reasons, but if you want to be sure simply enable the cache by force code it in Three.js. Also the creator of Three.js answered this question.
Echarts custom series are really flexibel and convenient.
But unfortunately the performance is really slow (compared to the "native" series).
e.g. here is a jsfiddle example (using echarts 4.8.0) that draws 5 custom series (with only 500 points each).
To test with more data, you can simply adjust the variables at the top of the jsfiddle code:
/**
* the number of data-samples
*/
var dataCount = 2000;
/**
* we create one custom series for each item in the csColors array
*/
var csColors = [
'#FF9800', '#9C27B0', '#512DA8', '#4CAF50', '#448AFF'
// , '#d32f2f', '#F1C40F', '#8bc6ff', '#00bc91', '#992f1c'
];
Even with this little data, zoomig (using the mouse wheel) or using the brush is already really slow.
For our application we would need up to 10 charts with ~5 series and 1k data samples each. And with that many samples the custom series is just not usable, because rendering takes way too long.
Any ideas how we could improve the performance?
e.g. when we use the brush it seems that almost every mouse-move redraws the whole series: I guess it may be related to the emphasis settings: is there a way to deactivate this?
Or is there maybe another way we could get fast custom series (i.e. directly draw on the canvas, ..)?
In Windows World, a dedicated render thread would loop something similar to this:
void RenderThread()
{
while (!quit)
{
UpdateStates();
RenderToDirect3D();
// Can either present with no synchronisation,
// or synchronise after 1-4 vertical blanks.
// See docs for IDXGISwapChain::Present
PresentToSwapChain();
}
}
What is the equivalent in Cocoa with CAMetalLayer? All the examples deal with updates being done in the main thread, either using MTKView (with it's internal timer) or using CADisplayLink in the iOS examples.
I want to be in control of the whole render loop, rather than just receiving a callback at some non-specified interval (and ideally blocking for V-Sync if it's enabled).
At some level, you're going to be throttled by the availability of drawables. A CAMetalLayer has a fixed pool of drawables available, and calling nextDrawable will block the current thread until a drawable becomes available. This doesn't imply you have to call nextDrawable at the top of your render loop, though.
If you want to draw on your own schedule without getting blocked waiting on a drawable, render to an off-screen renderbuffer (i.e., a MTLTexture with dimensions matching your drawable size), and then blit from the most-recently-drawn texture to a drawable's texture and present on whatever cadence you prefer. This can be useful for getting frame timings, but every frame you draw and then don't display is wasted work. It also increases the risk of judder.
Your options are limited when it comes to getting callbacks that match the v-sync cadence. Your best is almost certainly a CVDisplayLink scheduled in the default and tracking run loop modes, though this has caveats.
You could use something like a counting semaphore in concert with a display link if you want to free-run without getting too far ahead.
If your application is able to maintain a real-time framerate, you'll normally be rendering a frame or two ahead of what's going on the glass, so you don't want to literally block on v-sync; you just want to inform the window server that you'd like presentation to match v-sync. On macOS, you do this by setting the layer's displaySyncEnabled to true (the default). Turning this off may cause tearing on certain displays.
At the point where you want to render to screen, you obtain the drawable from the layer by calling nextDrawable. You obtain the drawable's texture from its texture property. You use that texture to set up the render target (color attachment) of a MTLRenderPassDescriptor. For example:
id<CAMetalDrawable> drawable = layer.nextDrawable;
id<MTLTexture> texture = drawable.texture;
MTLRenderPassDescriptor *desc = [MTLRenderPassDescriptor renderPassDescriptor];
desc.colorAttachments[0].texture = texture;
From here, it's pretty similar to what you do in an MTKView's drawRect: method. You create a command buffer (if you don't already have one), create a render command encoder using the descriptor, encode drawing commands, end encoding, tell the command buffer to present the drawable (using a -presentDrawable:... method), and commit the command buffer. Whatever was drawn to the drawable's texture is what will end up on-screen when it's presented.
I agree with Warren that you probably don't really want to sync your loop with the display refresh. You want parallelism. You want the CPU to be working on the next frame while the GPU is rendering the most current frame (and the display is showing the last frame).
The fact that there's a limit on how many drawables may be in flight at once and that nextDrawable will block waiting for one will prevent your render loop from getting too far ahead. (You'll probably use some other synchronization before that, like for managing a small pool of buffers.) If you want only double-buffering and not triple-buffering, you can set the layer's maximumDrawableCount to 2 instead of its default value of 3.
I'm implementing in my program the gaussian blur effect. To do the job I need to render the first blur information (the one on Y axis) in a specific texture (let's call it tex_1) and use this same information contained in tex_1 as input information for a second render pass (for the X axis) to fill an other texture (let's call it tex_2) containing the final gaussian blur result.
A good practice should be to create 2 frame buffers (FBOs) with a texture attached for each of them and linked both to GL_COLOR_ATTACHMENT0 (for example). But I just wonder one thing:
Is it possible to fill these 2 textures using the same FBO ?
So I will have to enable GL_COLOR_ATTACHMENT0 and GL_COLOR_ATTACHMENT1 and bind the desired texture to the correct render pass as follow :
Pseudo code:
FrameBuffer->Bind()
{
FrameBuffer->GetTexture(GL_COLOR_ATTACHMENT0)->Bind(); //tex_1
{
//BIND external texture to blur
//DRAW code (Y axis blur pass) here...
//-> Write the result in texture COLOR_ATTACHEMENT0 (tex_1)
}
FrameBuffer->GetTexture(GL_COLOR_ATTACHMENT1)->Bind(); //tex_2
{
//BIND here first texture (tex_1) filled above in the first render pass
//Draw code (X axis blur pass) here...
//-> Use this texture in FS to compute the final result
//within COLOR_ATTACHEMENT1 (tex_2) -> The final result
}
}
FrameBuffer->Unbind()
But in my mind there is a problem because I need for each render pass to bind an external texture as an input in my fragment shader. Consequently, the first binding of the texture (the color_attachment) is lost!
So does it exist a way to solve my problem using one FBO or do I need to use 2 separate FBOs ?
I can think of at least 3 distinct options to do this. Where the 3rd one will actually not work in OpenGL ES, but I'll explain it anyway because you might be tempted to try it otherwise, and it is supported in desktop OpenGL.
I'm going to use pseudo-code as well to cut down on typing and improve readability.
2 FBOs, 1 attachment each
This is the most straightforward approach. You use a separate FBO for each texture. During setup, you would have:
attach(fbo1, ATTACHMENT0, tex1)
attach(fbo2, ATTACHMENT0, tex2)
Then for rendering:
bindFbo(fbo1)
render pass 1
bindFbo(fbo2)
bindTexture(tex1)
render pass 2
1 FBO, 1 attachment
In this approach, you use one FBO, and attach the texture you want to render to each time. During setup, you only create the FBO, without attaching anything yet.
Then for rendering:
bindFbo(fbo1)
attach(fbo1, ATTACHMENT0, tex1)
render pass 1
attach(fbo1, ATTACHMENT0, tex2)
bindTexture(tex1)
render pass 2
1 FBO, 2 attachments
This seems to be what you had in mind. You have one FBO, and attach both textures to different attachment points of this FBO. During setup:
attach(fbo1, ATTACHMENT0, tex1)
attach(fbo1, ATTACHMENT1, tex2)
Then for rendering:
bindFbo(fbo1)
drawBuffer(ATTACHMENT0)
render pass 1
drawBuffer(ATTACHMENT1)
bindTexture(tex1)
render pass 2
This renders to tex2 in pass 2 because it is attached to ATTACHMENT1, and we set the draw buffer to ATTACHMENT1.
The major caveat is that this does not work with OpenGL ES. In ES 2.0 (without using extensions) it's a non-starter because it only supports a single color buffer.
In ES 3.0/3.1, there is a more subtle restriction: They do not have the glDrawBuffer() call from full OpenGL, only glDrawBuffers(). The call you would try is:
GLenum bufs[1] = {GL_COLOR_ATTACHMENT1};
glDrawBuffers(bufs, 1);
This is totally valid in full OpenGL, but will produce an error in ES 3.0/3.1 because it violates the following constraint from the spec:
If the GL is bound to a draw framebuffer object, the ith buffer listed in bufs must be COLOR_ATTACHMENTi or NONE.
In other words, the only way to render to GL_COLOR_ATTACHMENT1 is to have at least two draw buffers. The following call is valid:
GLenum bufs[2] = {GL_NONE, GL_COLOR_ATTACHMENT1};
glDrawBuffers(bufs, 2);
But to make this actually work, you'll need a fragment shader that produces two outputs, where the first one will not be used. By now, you hopefully agree that this approach is not appealing for OpenGL ES.
Conclusion
For OpenGL ES, the first two approaches above will work, and are both absolutely fine to use. I don't think there's a very strong reason to choose one over the other. I would recommend the first approach, though.
You might think that using only one FBO would save resources. But keep in mind that FBOs are objects that contain only state, so they use very little memory. Creating an additional FBO is insignificant.
Most people would probably prefer the first approach. The thinking is that you can configure both FBOs during setup, and then only need glBindFramebuffer() calls to switch between them. Binding a different object is generally considered cheaper than modifying an existing object, which you need for the second approach.
Consequently, the first binding of the texture (the color_attachment)
is lost!
No, it isn't. Maybe your framebuffer class works that way, but then, it would be a very bad abstraction. The GL won't detach a texture from an FBO just because you bind this texture to some texture unit. You might get some undefined results if you create a feedback loop (rendering to a texture you are reading from).
EDIT
However, as #Reto Koradi pointed out in his excellent answer, (and his comment to this one), you can't simply render to a single color attachment in unextended GLES1/2, and need some tricks in GLES3. As a result, The fact I'm pointing out here is still true, but not really helpful for the ultimate goal you are trying to achieve.
As per my need I want to redraw only some part of the scene for each frame instead
of redrawing the entire scene only if some portion of it is updated.
Is there a way to do that in OpenGL ES 2.0?
Please any input on this will be really helpful
OpenGL does not really support incremental rendering. You need to draw the entire frame every time you are asked to redraw.
The closest I can think of is that you render your static data to an offscreen framebuffer, using a FBO (Frame Buffer Object). You should be able to find plenty of examples online and in books if you look for keywords like "OpenGL FBO". You will be using calls like glGenFramebuffers(), glBindFramebuffer(), glFramebufferTexture2D(), etc.
Once you rendered the static content into an FBO, you can copy it to the default framebuffer at the start of each redraw, and then render the dynamic content on top of it. This can be a worthwhile method if rendering the static content is very expensive. Otherwise, doing the copy from FBO to default framebuffer can be more expensive than simply re-rendering the static content each time.
The above is pretty easy if the static content is in the background, and the dynamic content is completely in front of it. If static and dynamic content overlap, it gets trickier. You will then have to restore the depth buffer resulting from rendering the static content each time before starting to render the dynamic content. I can't think of a good way to do that in ES 2.0. The features to do this relatively smoothly (depth textures, glBlitFramebuffer) are only in ES 3.0 and later.
There is one other option that I don't think is very appealing, but I wanted to mention it for completeness sake: EGL defines a EGL_SWAP_BEHAVIOR attribute that can be set to EGL_BUFFER_PRESERVED. One big caveat is that it's optional, and not supported on all devices. It also only preserves the color buffer, and not auxiliary buffers, like the depth buffer. If you want to read up on it anyway, see eglSwapBuffers and eglSurfaceAttrib.