I'm trying to implement a cross-platform UI library that takes as little system resource as possible. I'm considering to either use my own software renderer or opengl.
For stationary controls everything's fine, I can repaint only when it's needed. However, when it comes to implementing animations, especially animated blinking carets like the 'phase' caret in sublime text, I don't see a easy way to balance resource usage and performance.
For a blinking caret, it's required that the caret be redrawn very frequently(15-20 times per sec at least, I guess). On one hand, the software renderer supports partial redraw but is far too slow to be practical(3-4 fps for large redraw regions, say, 1000x800, which makes it impossible to implement animations). On the other hand, opengl doesn't support partial redraw very well as far as I know, which means the whole screen needs to be rendered at 15-20 fps constantly.
So my question is:
How are carets usually implemented in various UI systems?
Is there any way to have opengl to render to only a proportion of the screen?
I know that glViewport enables rendering to part of the screen, but due to double buffering or other stuff the rest of the screen is not kept as it was. In this way I still need to render the whole screen again.
First you need to ask yourself.
Do I really need to partially redraw the screen?
OpenGL or better said the GPU can draw thousands of triangles at ease. So before you start fiddling with partial redrawing of the screen, then you should instead benchmark and see whether it's worth looking into at all.
This doesn't however imply that you have to redraw the screen endlessly. You can still just redraw it when changes happen.
Thus if you have a cursor blinking every 500 ms, then you redraw once every 500 ms. If you have an animation running, then you continuously redraw while that animation is playing (or every time the animation does a change that requires redrawing).
This is what Chrome, Firefox, etc does. You can see this if you open the Developer Tools (F12) and go to the Timeline tab.
Take a look at the following screenshot. The first row of the timeline shows how often Chrome redraws the windows.
The first section shows a lot continuously redrawing. Which was because I was scrolling around on the page.
The last section shows a single redraw every few 500 ms. Which was the cursor blinking in a textbox.
Open the image in a new tab, to see better what's going on.
Note that it doesn't tell whether Chrome is fully redrawing the window or only that parts of it. It is just showing the frequency of the redrawing. (If you want to see the redrawn regions, then both Firefox and Chrome has "Show Paint Rectangles".)
To circumvent the problem with double buffering and partially redrawing. Then you could instead draw to a framebuffer object. Now you can utilize glScissor() as much as you want. If you have various things that are static and only a few dynamic things. Then you could have multiple framebuffer objects and only draw the static contents once and continuously update the framebuffer containing the dynamic content.
However (and I can't emphasize this enough) benchmark and check if this is even needed. Having two framebuffer objects could be more expensive than just always redrawing everything. The same goes for say having a buffer for each rectangle, in contrast to packing all rectangles in a single buffer.
Lastly to give an example let's take NanoGUI (a minimalistic GUI library for OpenGL). NanoGUI continuously redraws the screen.
The problem with not just continuously redrawing the screen is that now you need a system for issuing a redraw. Now calling setText() on a label needs to callback and tell the window to redraw. Now what if the parent panel the label is added to isn't visible? Then setText() just issued a redundant redrawing of the screen.
The point I'm trying to make is that if you have a system for issuing redrawing of the screen. Then that might be more prone to errors. Thus unless continuously redrawing is an issue, then that is definitely a more optimal starting point.
Related
I've made a game on pixi.js where it is very simple and should run smoothly but doesn't. Since the game is big and has lots of elements I don't know where the bottleneck is. I believe some part/s of the is buggy resulting in lots of resource usage.
I want to learn the method on how to identify the functions I wrote that use up a lot of resources.
When I use default chrome tools they never show the part of the code that I wrote but always the library ticker: https://prnt.sc/qiplql.
This doesn't help me, I want to know which functions I wrote that is being called many times and uses up a lot of resources.
What is a way I can do this?
Here is our debugging server where the code isn't minimized.
Zoom in
To get the most out of the performance monitor you must zoom into the timeline and investigate small sections of code.
The image below shows the section of the timeline, and the Call tree's representation of that small section of time
Find a single frame
However this still is too much information. Zoom right in to find a single frame that is running long (over 16ms)
Use the graphical call tree (open fold main)
The next image shows a long frame (25ms) Opening the fold main (on left) will show a graphical representation of the call tree.
The frame is started with a long task triggered by a mouse up event. At the bottom of that stack the reason the event takes so long is the DOM recalculating the style.
The recalculation of the style meant that the next requestAnimationFrame callback had some extra baggage before (reflow) and after the script (DOM rendering and compositing) meaning the frame was late to present.
Zooming in on the requestAnimationFrame there are no problems. Just as an example I zoomed so you can see how the graphical representation relates to the Call tree. I have drawn some lines connecting graphical function calls with the same call in the call tree.
Many reasons
To find your bottle necks you will have to zoom right in on the slow frames. There may be many different reasons that frames are slow.
If the content of the last frame isn't changed on receiving WM_PAINT, is it possible to simply direct the operating system to redraw the window using the old back buffer instead of redrawing the whole scene again to the new back buffer and swapping it?
No. There is no such "backbuffer". And when drawing occurs you don't know what areas may be covered by other windows. The clipping area isn't a real good indicator.
The only thing you know is that such areas need to be redrawn. Each window cares about its own client area. If you want to buffer something, you have to do it on your own.
The reason is simple: Imagine you have hundreds of windows. To hold a buffer for each window is inefficient, when just a view on the top are visible. So the Windows makers decide not to store any windows content and just notify windows on the top to redraw themselves.
OK. Since we have a DWM (Dynamic Window Manager) things changed a lot. But the principle is still: You are responsible to draw. If you want to buffer something, you have to do it on your own.
I am drawing an animation using double-buffered GDI on a window, on a system where DWM composition is enabled, and seeing clearly visible tearing onscreen. Is there a way to prevent this?
Details
The animation takes the same image, and moves it right to left over the screen; the number of pixels across is determined by the difference between the current time and the time the animation started and the time to end, to get a fraction complete which is applied to the whole window width, using timeGetTime with a 1ms resolution. The animation draws in a loop without processing application messages; it calls the (VCL library) method Repaint which internally invalidates and then calls UpdateWindow for the window in question, directly calling into the message procedure with WM_PAINT. The VCL implementation of the paint handler uses BeginBufferedPaint. Painting is itself double-buffered.
The aim of this is to have as high a frame-rate as possible to get a smooth animation across the screen. (The drawing uses double-buffering to remove flickering and to ensure a whole image or frame is onscreen at any one time. It invalidates and updates directly by calling into the message procedure, without doing other message processing. Painting is implemented using modern techniques (eg BeginBufferedPaint) for Aero composition.) Within this, painting is done in a couple of BitBlt calls (one for the left side of the animation, ie what's moving offscreen, and one for the right side of the animation, ie what's moving onscreen.)
When watching the animation, there is clearly visible tearing. This occurs on Windows Vista, 7 and 8.1 on multiple systems with different graphics cards.
My approach to handle this has been to reduce the rate at which it is drawing, or to try to wait for VSync before painting again. This might be the wrong approach, so the answer to this question might be "Do something else completely: X". If so, great :)
(What I'd really like is a way to ask the DWM to compose / use only fully-painted frames for this specific window.)
I've tried the following approaches, none of which remove all visible tearing. Therefore the question is, Is it possible to avoid tearing when using DWM composition, and if so how?
Approaches tried:
Getting the monitor refresh rate via GetDeviceCaps(Application.MainForm.Handle, VREFRESH); sleeping for 1 / refresh rate milliseconds. Slightly improved over painting as fast as possible, but may be wishful thinking. Perceptually slightly less smooth animation rate. (Tweaks: normal Sleep and a high-resolution spin-wait using timeGetTime.)
Using DwmSetPresentParameters to try to limit updating to the same rate at which the code draws. (Variations: lots of buffers (cBuffer = 8) (no visible effect); specifying a source rate of monitor refresh rate / 1 and sleeping using the above code (the same as just trying the sleeping approach); specifying a refresh per frame of 1, 10, etc (no visible effect); changing the source frame coverage (no visible effect.)
Using DwmGetCompositionTimingInfo in a variety of ways:
While cFramesPending > 0, spin;
Get cFrame (frame composed) and spin while this number doesn't change;
Get cFrameDisplayed and spin while this doesn't change;
Calculating a time to sleep to by adding qpcVBlank + qpcRefreshPeriod, and then while QueryPerformanceCounter returns a time less than this, spin
All these approaches have also been varied by painting, then spinning/sleeping before painting again; or the reverse: sleeping and then painting.
Few seem to have any visible effect and what effect there is is hard to qualify and may just be a result of a lower frame rate. None prevent tearing, ie none make the DWM compose the window with a "whole" copy of the contents of the window's DC.
Advice appreciated :)
Since you're using BitBlt, make sure your DIBs are 4-bytes / pixel. With 3 bytes / pixel, GDI is horribly slow while DWM is running, that could be the source of your tearing. Another BitBlt issue I've run into, if your DIB is somewhat larger, than the BitBlt call make take an unexpectedly long time. If you split up one call into smaller calls than only draw a portion of the data, it might help. Both of these items helped me for my case, only because BitBlt itself was running too slow, thus leading to video artifacts.
How does Flash deal with elements that are off-stage?
Obviously Flash doesn't actually render them (because they don't appear anywhere on-screen), but is the process of rendering them still existent, slowing down my game as much as it would if the elements were on-screen?
Or does Flash intelligently ignore elements who don't fall into a renderable area?
Should I manually manage removing objects off the DisplayList and adding them back on as the exit and enter the stage, or is this going to be irrelevant?
Yes, they are slowing down your game.
In one of my early experiments I've developed a sidescroller game with many NPCs scattered around the map, not all visible in the same screen. I still had to calculate stuff but they weren't on the screen. The performance was significantly better when I handled their removal off the display list when irrelevant (by simply checking their X in relation to the 'camera'). Again, I'm not talking about additional code and events that may be attached to them, just plain graphical children of a movieclip.
The best practice though, in my experience, is drawing the objects in bitmaps. Of course if you're too deep into your game already this may be irrelevant, but if you have the time to invest, this is one of the best ways to get the most out of AS3 regarding 2D games. I found some of the greatest tutorials regarding bitmaps and AS3 in 8bitrocket
http://www.8bitrocket.com/books/the-essential-guide-to-flash-games/ I can elaborate on the subject if you want, but I think I'm going off topic here.
Even if some display objects are out of the stage area, they are still executed. If they have any animation playing in them, that might slow down the performance.
The question arises, why do we need to keep unused items outside the stage area? if you need to 'cache' the movieClips for faster loading , then load them in a keyframe where the control will never go. for eg. load the display objects which you want to show in frame 1, then put a stop() in the actions panel of the frame, make it a key frame, and in frame 2 load the unused animations. since there is a stop() in frame 1, the control never goes in frame 2, but the display objects are cached.
Or, if you have codes in the unsused displayobjects, and thus need to load them along with the main game components, then, try putting stop() in the frames of the unused display objects so that they don't animate.
Or to ask it another way, how OnEraseBkgnd() works?
I'm building a custom control and I hit on this problem. Childs are rectangles, as usual. I had to disable OnEraseBkgnd() and I use only the OnPaint(). What I need is to efficiently clear the area behind the childs and without flickering. Techniques like using back buffers are not an option.
Edit: I am very interested in the algorithm that's under the hood of OnEraseBkgnd(). But any helpful answer will also be accepted.
Usually in Windows, the easiest (but not most effective) means of reducing flicker, is to turn off the WM_ERASEBKGND notification handling. This is because if you erase the background in the notification handler, then paint the window in the WM_PAINT handler, there is a short delay between the two - this delay is seen as a flicker.
Instead, if you do all of the erasing and drawing in the WM_PAINT handler, you will tend to see a lot less flicker. This is because the delay between the two is reduced. You will still see some flicker, especially when resizing because there is still a small delay between the two actions, and you cannot always get in all of the drawing before the next occurance of the vertical blanking interrupt for the monitor. If you cannot use double buffering, then this is probably the most effective method you will be able to use.
You can get better drawing performance by following most of the usual recommendations around client area invalidation - do not invalidate the whole window unless you really need to. Try to invalidate only the areas that have changed. Also, you should use the BeginDeferWindowPos functions if you are updating the positions of a collection of child windows at the same time.