I develop audio plugins, which are run inside their hosts and work realtime. Each plugin has its own window with controls, which often contains some kind of analysis pane, a pretty big rectangle that gets repeatedly painted (e.g. 20-50x per second). This is all working well.
The trouble comes when the user adjusts a parameter - the plugin uses WM_MOUSEMOVE to track mouse movements and on each change calls ::InvalidateRect to make the relevant portion of the window be redrawn. If you move quickly enough, the window really gets quickly repainted, however there seems no time for the host and other windows to be redrawn and these usually perform some kind of analysis feedback too, so it is really not ideal.
No my questions:
1) Assuming the host and other window are using ::InvalidateRect too, why mine is prioritized?
2) How to make ::InvalidateRect not prioritized, meaning the window needs to be invalidated, but it may be later, the rest of the system must get time for their redrawing too.
Thanks in advance!
Related
Some years ago (2008) I wrote an online Child Support Calculator using PHP/Javascript/Ajax/css Advanced Child Support Calculator - May help to explain what I am replicating.
Some time after (2009 I think) I started writing an equivalent for Windows and am now revisiting this. The fundamentals of this version are working.
However, I have an issue with the window noticeably flickering/changing when controls are dynamically added and the window is redrawn/rebuilt.
Note! This calculator is specific to Australia.
In short I'm looking for a way, if possible, to only refresh/re-display/redraw the window after all components have been added.
Basically, the windows controls need to be dynamically added/removed depending upon the scenario (number of adults and children involved).
Adding or removing a child or adult or performing the calculation, results in a complete rebuild of the window. That is, all existing controls are destroyed and then all valid controls are added (this could perhaps be minimised via complex logic).
The issue is that controls are removed but briefly reappear (in an ordered fashion) causing the display to flicker (for want of a better description).
The following screen shots demonstrate the complexity factor (such as a child has a drop down for each defined adult), but not the flicckering(sic).
Here's a screen shot of the Initial display (OK pretty ugly at present) :-
And then if an Adult is added (note that the child now has an extra dropdown for Adult 3, the new adult):-
And now, an Adult (as above) and a Child:-
Coding wise, there is a RebuildAll function. This has two main stages. (1) The removal of the controls. (2) The rebuild (recreation) of the appropriate of the controls (Create Window, SendMessage's and then ShowWindow)
At a minimum there are 61 controls. The number of controls is
23 + ((#children * 12) - 2) + (#children * (#Adults * 8 )) + ((#adults * 10) -4). It's likely suffice to say that the number of controls increases rapidly/exponentially.
I suspect that it might be possible to postpone the ShowWindow untill after all of the builds have been done. Is the solution, as simple (theoretically) as this, assuming that this is feasible or is there another way that would circumvent the need to change all the code to remove the 'ShowWindow' and add a ShowWindow at the end of the builds?
Note replacing the individual ShowWindow's with one significantly reduced the "flicker" but didn't eliminate it. (as per Update below).
As an endnote I'm pretty sure that something should be feasible as the
windows program I have to date is a very poor reflection of the speed
of the browser/javascript version, which basically does the same thing
on windows (albeit 64bit).
Update
I went through and commented out the ShowWindows in all the AddItem???? functions and added a ShowWindows to the function that calls all the functions. This has improved matters. However, the numerous DestroyWindow calls still causes flickering when removing all of the controls. So now I guess that I'm looking for something that can disable them apparently doing an equivalent of ShowWindow.
Update 2
I have found SendMessage(hwnd, WM_SETREDRAW, FALSE, 0); (TRUE to turn drawing back on). However, this appears to supress the DestroyWindow in that the display aspect of the control remains (the controls themselves don't appear to respond though).
By a process of elimination I have cured the flicker with a combination of ShowWindow's and WM_SETREDRAW's at pertinent points.
To briefly re-describe the issue. When adding a child or adult, or performing a calculation. The entire display is rebuilt. The rebuild basically consists of 2 phases. First, destroying the existing control windows and then adding the new/replacement set of window's controls.
The resolution was to :-
Issue a ShowWindow(hwnd,SW_SHOW); (for the main/containing
window) after all the pre-existing windows control's had been
destroyed.
Issue a WM_SETREDRAW FALSE message as per SendMessage(hwnd,
WM_SETREDRAW, FALSE, 0); before rebuilding the windows controls.
Without any ShowWindow's being used.
Build the windows controls via the various functions.
Finally, when all the windows controls have been built, issue a
ShowWindow(hwnd,SW_SHOW); (for the main/containing window).
Rather than seeing the controls appear and then disappear, "the flickering". The window is blank for a while and then the controls are displayed all at once.
I'm still confused about what is actually going on though, due to:-
After some thought I decided to use a MessageBox after all the destroys. When the message was displayed the window was blank. However, when the button was clicked and processing resumed then, in situations when the number of controls was relatively high, controls would appear and then disappear in an orderly fashion, albeit it momentarily.
My guess, is that creating a control in an area where an existing
control had been, was causing that previous control to momentarily
display.
I have a cocoa application window (NSWindow) which position on the screen should be updated frequently (depending on some calculation). As noticed in the documentation, UI changes should be made on the main thread:
void calculationThread()
{
while(true)
{
calculatePosition();
if(positionChanged)
{
dispatch_async(dispatch_get_main_queue(), ^{ setWindowPos(); });
}
}
}
void setWindowPos()
{
[window setFrame:_newFrame display:YES];
}
Now the problem I have is that the window movement is very slow and delayed. After making some profiling I see that the calculation process takes about 40mSec, meaning that I'm queueing up a backlog of UI updates 25 times a second.
I've read here that this might be faster than they can be processed and timer should be used to fire the changes every tenth of a second or so. But, wouldn't it be too slow for the human eye (I mean, in that case the movement wouldn't be delayed but would be lagged causing pretty much the same affect).
I will appreciate some knowledge sharing on this. Actually my main 2 questions are:
Are 25-30 UI updates per second really to much?
If yes, what is the recommended UI changes frequency?
The frequency at which a window can be moved around onscreen without problems will of course depend upon the speed of the user's machine, the video card they have, the size of the window, and probably a bunch of other factors. There is no single good answer to this. However, if you just drag a window around on your screen, you will notice that it can probably be moved very smoothly (unless your machine is very busy or very low on memory or something); I would not expect 25 times per second to produce a problem on a modern Mac. Not even close, in fact.
#RobNapier's points about Core Animation etc. are fine, but overstated I think; there is nothing inherently wrong with changing your UI using a timer or other periodic update if that is what you actually want to do. CoreAnimation is a toolkit for making some types of animation easier; using it is not required, and it is not suited to every problem. Similarly, if you want to make changes that are actually synched to screen refresh then CVDisplayLink is useful, but it doesn't really sound like that's what you want to do.
For your purposes, your basic approach seems fine, although I would suggest adding an NSDate check in order to skip updates if the previous update was less than, say, 1/60th of a second previous. After all, the calculation appears to take 40mSec on your machine, but it might be much faster on some other machine; you want to throttle your drawing to a reasonable rate just to be a good citizen.
So what is the problem, then? I suspect the issue might actually be your call [window setFrame:_newFrame display:YES]. If you look at Apple's docs for that method, they state "When YES the window sends a displayIfNeeded message down its view hierarchy, thus redrawing all views." Each time you call that method, then, you are not only moving your window (which I gather is your intention); you are redrawing all of the contents of the window, too, and that is slow. If you don't need to do that, then that is the overhead you need to eliminate. Call setFrameOrigin: or setFrameTopLeftPoint: instead (which make the semantics clear, that you are moving the window without resizing it or redrawing it), or perhaps just setFrame:display: passing NO instead of YES, and I'm guessing your performance problem will vanish.
If you do in fact need to redraw the window contents every time, then please edit the problem description to reflect that. In that case, the solution will have to involve profiling why your window drawing is slow, and figuring out ways to optimize that, which is an entirely different problem.
As you've discovered, you should never try to drive the UI from a tight loop. You should let the UI drive you. There are three primary tools for that.
For simple problems, AppKit is capable of moving windows around the screen. Just call [NSWindow setFrame:display:animate:]. You can override animationResizeTime: to modify the timing.
In many cases AppKit doesn't give enough control. In those case, the best tool is almost always Core Animation. You should tell the system using Core Animation how you where you want UI elements to wind up, and over what period and path, and let it do the work of getting them there. See the Core Animation Programming Guide for extensive documentation on how to use that. It focuses on animating CALayer, but the techniques are similar for NSWindow. You'll use [NSWindow setAnimations:] to add your animation. Look at the NSAnimatablePropertyContainer protocol (which NSWindow conforms to) for more information. For a simple sample project of animating NSWindow, see Just Say No from CIMGF.
In a few cases, you really do need to update the screen manually at the screen update frequency. I must stress how rare this situation is. In almost all cases, Core Animation is the correct tool. But in those rare case (some kinds of video for instance), you can use a CVDisplayLink to handle this. That will call you each time the screen would like to refresh, giving you an opportunity to update your content to match.
It is possible to use the SetWindowPos API on Windows to keep a windows always on top of other windows, and there are many questions on StackOverflow dealing with this.
It is possible to keep only part of a Window always visible? I.e. specify a clipping region inside an existing window, and keep only that part visible?
A use case would be the following (on Windows):
User clicks on icon to run app.
User highlights a portion of the screen to focus on (similar to the Snipping Tool on Windows 7)
The highlighted part of the screen remains always visible, even when other windows/programs are moved over the selected region.
I know the issues that would spring up with having other applications that are also set to being topmost. Just curious if this is even possible?
Even if you change part of your window to be transparent to what's below (with a clipping region) it's still going to take all the mouse clicks, etc. that occur over the transparent part.
Your best bet is to create a new smaller window and make it top-most while hiding the main one.
I have been all over the web looking for an answer to this, and my question is this:
How does a GUI framework work? for instance how does Qt work, is there any books or wibsites on the topic of writing a GUI framework from scratch? and also does the framework have to call methods from the operating systems GUI framework?
-- Thank you to any one who takes the time to try to answer this question, and forgive me if i misspelled anything.
In the old days we did a lot of GUI programming from scratch. It is not as hard as it seems, but it requires a few weeks to come with results.
First you need a good drawing library. Minimal functionality for this library is drawing clipped rectangles (using patterns), lines, bitmaps, and fonts. You can cheat by creating fonts as bitmaps, and clipped rectangle is just a bunch of horizontal lines.
Now you need at least drivers for mouse, keyboard, and timer (if not already provided by the operating system). In general, you will need to detect keys, symbol keys (such as shift, etc.), mouse moves and mouse clicks. Basic timer functions will allow you to detect double clicks.
Then you need to create a window data structure. This data structure needs to have coordinates i.e. a rectangle, link to parent window (if not top window), and window function i.e. the function that will be called when this window should handle some events.
Once you can draw on screen you need some rectangle algebra functions. You need at least good function to calculate intersection of rectangles, and a quick resolution of relative to absolute coordinates. For example - if your child window has parent then its' x and y should recursively be added to parent x and y until you reach top window.
At this point you have your:
- primitive graphical functions,
- window structure,
- mouse driver, keyboard driver, and timer,
- rectangle arithmetic.
Now you can write your main event harvesting function. This function will run all the time. It's purpose will be to detect events and send messages to correct windows. What is an event? Well, when you start your program, store mouse x and y coordinates. Then in a loop check if they have changed. If they have changed, find the window at that position ... and send WM_MOUSEMOVE event to it. Your harvesting function should handle:
- mouse moves
- mouse clicks
- mouse double clicks (remember last click and position, measure time and decide if it is a double click or not)
- timer events
- keyboard buffer changes
...
Now you should be able to send events to windows. But you really need a mechanism for it. It is a combination of message queue, and window procedure. It usually works like this: each window has a window procedure which commonly accepts four arguments: message id (i.e. is it mouse move, is it paint message), window handle, parameter 1 and parameter 2. You can call this window procedure directly using something like a send_message functions. Or you can send this window a message via post_message function. This will put message to the queue and window will process messages one by one, eventually receiving this one. So why should you call one messages directly and put others to the queue? Because of priority. You see, a keyboard click can wait some time before being processed. But a window redraw must complete immediately to prevent flicker and wrong data on screen.
So your harvest_events function sends messages to windows using post_message, and send_message. And your window message pump gets them using typical message pump like this:
while (pmsg = get_message() != NULL) send_message(pmsg->id, pmsg->hwnd, pmsg->p1, pmsg->p2);
get_message simply obtains message from the queue, and calls send message. Simple, huh? Well, not quite so. This way you would only receive driver messages to windows, but you also need some functions to redraw windows, move them, etc.. When you create move_window function, resize_window, show_window, and hide_window function, your window coordinates will change. Parts of other windows will be uncovered (if top window is moved or closed).You need to calculate which windows are affected by coordinate changes and send paint message to those windows (to repaint only the parts that were uncovered - remember, you have clipping drawing functions so this will work).
These functions introduces messages msg_paint, msg_move, msg_resize, msg_hide...
Last, you need to maintain hierarchy of windows. Your top window should be the desktop. It should have child windows (application top windows). These windows may have further child windows (buttons, edit boxes, etc.) The obvious structure for holding these is the window tree. When you detect mouse click you have to traverse window tree and do it in a smart way (finding out who has focus, who is modal, etc.) to send message to the right window. And when you draw you also must traverse all children to see who is uncovered and who is not. Last but not least, you need to handle mouse rectangle as top window to prevent flickering the mouse as windows are re-drawn or (using timers and msg_paint events) animated.
That's roughly it.
A GUI framework like Qt generally works by taking the existing OS's primitive objects (windows, fonts, bitmaps, etc), wrapping them in more platform-neutral and less clunky classes/structures/handles, and giving you the functionality you'll need to manipulate them. Yes, that almost always involves using the OS's own functions, but it doesn't HAVE to -- if you're designing an API to draw an OpenGL UI, for example, most of the underlying OS's GUI stuff won't even work, and you'll be doing just about everything on your own.
Either way, it's not for the faint of heart. If you have to ask how a GUI framework works, you're not even close to ready to design one. You're better off sticking with an existing framework and extending it to do the spiffy stuff it doesn't do already.
We have a fairly complex GUI, so when certain windows are resized their Redraw() is set to false till the operation is completed. The problem with this is that if the OS "Show window content while dragging" setting is checked, when decreasing the window's size the windows behind it are not repainted. This means I have to force the repaint myself so the remains of the resized window are deleted. I have no problem getting the dimensions of the region that was uncovered. What I'm looking for is best way to cause all windows within that region to repaint their part.
Not being much of a GUI programmer, I can traverse the uncovered region and list the windows in it. Then, I can ask each one of them to repaint its part. But I'm quite certain there has to be a better way to do this...
It is worth mentioning the app is written in PowerBuilder. This means I can call whatever Win32 function I'd like, but have limited control over the GUI behavior and the message handling. If there's a better way to prevent the window's content resize from being visible, or there's a way to make a non-redrawn window clean after itself, I'd love to hear it (just have the limitations above in mind).
I'm curious what version of PowerBuilder you are working in? I do resizing all the time and never run into issues like you are describing.
Maybe you can lay out some more detail on why you need to set your redraws to false within the PowerBuilder environment.
Hope I can help.