What is the recommended frequency for UI changes? - macos

I have a cocoa application window (NSWindow) which position on the screen should be updated frequently (depending on some calculation). As noticed in the documentation, UI changes should be made on the main thread:
void calculationThread()
{
while(true)
{
calculatePosition();
if(positionChanged)
{
dispatch_async(dispatch_get_main_queue(), ^{ setWindowPos(); });
}
}
}
void setWindowPos()
{
[window setFrame:_newFrame display:YES];
}
Now the problem I have is that the window movement is very slow and delayed. After making some profiling I see that the calculation process takes about 40mSec, meaning that I'm queueing up a backlog of UI updates 25 times a second.
I've read here that this might be faster than they can be processed and timer should be used to fire the changes every tenth of a second or so. But, wouldn't it be too slow for the human eye (I mean, in that case the movement wouldn't be delayed but would be lagged causing pretty much the same affect).
I will appreciate some knowledge sharing on this. Actually my main 2 questions are:
Are 25-30 UI updates per second really to much?
If yes, what is the recommended UI changes frequency?

The frequency at which a window can be moved around onscreen without problems will of course depend upon the speed of the user's machine, the video card they have, the size of the window, and probably a bunch of other factors. There is no single good answer to this. However, if you just drag a window around on your screen, you will notice that it can probably be moved very smoothly (unless your machine is very busy or very low on memory or something); I would not expect 25 times per second to produce a problem on a modern Mac. Not even close, in fact.
#RobNapier's points about Core Animation etc. are fine, but overstated I think; there is nothing inherently wrong with changing your UI using a timer or other periodic update if that is what you actually want to do. CoreAnimation is a toolkit for making some types of animation easier; using it is not required, and it is not suited to every problem. Similarly, if you want to make changes that are actually synched to screen refresh then CVDisplayLink is useful, but it doesn't really sound like that's what you want to do.
For your purposes, your basic approach seems fine, although I would suggest adding an NSDate check in order to skip updates if the previous update was less than, say, 1/60th of a second previous. After all, the calculation appears to take 40mSec on your machine, but it might be much faster on some other machine; you want to throttle your drawing to a reasonable rate just to be a good citizen.
So what is the problem, then? I suspect the issue might actually be your call [window setFrame:_newFrame display:YES]. If you look at Apple's docs for that method, they state "When YES the window sends a displayIfNeeded message down its view hierarchy, thus redrawing all views." Each time you call that method, then, you are not only moving your window (which I gather is your intention); you are redrawing all of the contents of the window, too, and that is slow. If you don't need to do that, then that is the overhead you need to eliminate. Call setFrameOrigin: or setFrameTopLeftPoint: instead (which make the semantics clear, that you are moving the window without resizing it or redrawing it), or perhaps just setFrame:display: passing NO instead of YES, and I'm guessing your performance problem will vanish.
If you do in fact need to redraw the window contents every time, then please edit the problem description to reflect that. In that case, the solution will have to involve profiling why your window drawing is slow, and figuring out ways to optimize that, which is an entirely different problem.

As you've discovered, you should never try to drive the UI from a tight loop. You should let the UI drive you. There are three primary tools for that.
For simple problems, AppKit is capable of moving windows around the screen. Just call [NSWindow setFrame:display:animate:]. You can override animationResizeTime: to modify the timing.
In many cases AppKit doesn't give enough control. In those case, the best tool is almost always Core Animation. You should tell the system using Core Animation how you where you want UI elements to wind up, and over what period and path, and let it do the work of getting them there. See the Core Animation Programming Guide for extensive documentation on how to use that. It focuses on animating CALayer, but the techniques are similar for NSWindow. You'll use [NSWindow setAnimations:] to add your animation. Look at the NSAnimatablePropertyContainer protocol (which NSWindow conforms to) for more information. For a simple sample project of animating NSWindow, see Just Say No from CIMGF.
In a few cases, you really do need to update the screen manually at the screen update frequency. I must stress how rare this situation is. In almost all cases, Core Animation is the correct tool. But in those rare case (some kinds of video for instance), you can use a CVDisplayLink to handle this. That will call you each time the screen would like to refresh, giving you an opportunity to update your content to match.

Related

what is the best approach to scheduling repaint for winapi window?

I already tried multiple ways of rendering simple animations on winapi windows using wingdi, and realized that its fairly slow even for primitive animations, and inconsistent if i do it by spamming SendMessageW when there are no events for this window. So I thought that instead of rendering when I can i should schedule it with a fixed interval hence improving framerate and quality of the animation. I now wondering if this can be done with registering some callback with windows or there is no such functionality and i do have to spin another thread to function as a timer? Is it generally a better idea? What is the commonly accepted way to do things like this, when event triggered redraws just wont cut it?
Concerning the scheduled animations, there is SetTimer() and KillTimer(), if I remember their names correctly. In any case, "timer" is what you're looking for.
Concerning the painting in the timer callbacks, don't. Instead, adjust animation parameters (positions, colours etc) and trigger a redraw using InvalidateRect(). This will in turn invoke the regular drawing event handler. The difference is that this will not waste any CPU if your window is hidden or minimized. Also, in any case, when the window is un-hidden, the drawing event handler has to be able to draw the right window content anyway.
There is a derived version of this where you update an image in a memory DC and only blit to the window in the drawing event handler. It's unclear whether that is necessary in your case though. It's only when drawing takes an extended amount of time and you want to keep your UI responsive.
Well as my luck usually goes i found this article on MSDN, and it has a very nice example. Putting it here if someone will have same question

Updating contents of multiple NSTextView objects in a single operation

The database application I am working on can have a window with multiple NSTextView elements for displaying and editing data. When the current spot in the database is repositioned, all of the NSTextView objects in the window need to be updated with new contents. This is done with a loop that scans each object and checks to see if it needs to be updated. If it does, the new value is calculated, then updated by using the [NSTextView setString:] method. Here is a simplified version of the code involved.
for formObject in formObjectsInWindow {
NSTextView * objectTextView = [formObject textView];
NSString * updatedValue = [formObject calculateValue];
[objectTextView setString: updatedValue];
}
This works, but if there are a lot of objects, it is somewhat slow. Probably related, the display does not update all at once, you can actually see a "ripple" as the objects are updated, as illustrated in this movie (this movie has been slowed down to 1/4 speed to make the ripple effect more pronounced, but it is definitely visible at full speed).
If you've gotten this far, you might suspect that the calculateValue method is slow, but that isn't the problem. In other places the same code is used and runs at tens of thousands of operations per second. Also, this delay only occurs during update operations, it doesn't occur when the window is first opened, even though the same calculations are required at that time. Here is an example. Notice that when I switch back to the detail view all the NSTextView objects update instantaneously, even though the record changed and all of the values are different.
My suspicion is that the [NSTextView setString:] method is updating the off-screen buffer, then immediately copying that to the on-screen buffer, so that this double buffering is happening over and over again for each item, causing the delay and ripple. If so, I'm sure there must be some way to prevent this so that the screen is only updated at the end after all of the values have been updated. It's probably something simple that I am missing, but I'm afraid I am stumped as to how this is supposed to be done.
By the way, this application does not use layer-backed views, and is not linked against the QuartzCore framework.
I brought up this question with Apple engineers at the WWDC 2018 labs. It turns out the problem is that the setString: method does not mark the NSTextView object as needing display. The system does eventually notice that the text has changed and updates the display, but this happens in an asynchronous process, hence the "ripple" effect. So the workaround is simply to add a call to setNeedsDisplay after calling setString.
[objectTextView setString: updatedValue]
[objectTextView setNeedsDisplay:YES];
Adding this one line of code fixed the problem, no more ripple effect.
I'm told that this is actually a bug, so hopefully this extra line won't be needed in future versions of macOS. As requested, a radar has been filed (41273611 if any Apple engineers are reading this).

InvalidateRect called too frequently blocks other windows from redrawing

I develop audio plugins, which are run inside their hosts and work realtime. Each plugin has its own window with controls, which often contains some kind of analysis pane, a pretty big rectangle that gets repeatedly painted (e.g. 20-50x per second). This is all working well.
The trouble comes when the user adjusts a parameter - the plugin uses WM_MOUSEMOVE to track mouse movements and on each change calls ::InvalidateRect to make the relevant portion of the window be redrawn. If you move quickly enough, the window really gets quickly repainted, however there seems no time for the host and other windows to be redrawn and these usually perform some kind of analysis feedback too, so it is really not ideal.
No my questions:
1) Assuming the host and other window are using ::InvalidateRect too, why mine is prioritized?
2) How to make ::InvalidateRect not prioritized, meaning the window needs to be invalidated, but it may be later, the rest of the system must get time for their redrawing too.
Thanks in advance!

What could possibily slow down my flex mobile application?

I am working with flex for the last two years on some desktop apps. Until now I never had any performance related issues but today as we completed a mobile application for the iPad, I'm facing a challenge, the application is incredibly slow on the iPad.
http://i.stack.imgur.com/qkbWn.png
Slow, means that when I press a button in the menu to change the splitview I must wait something like 5s. Then scrolling is really slow two, with less than one fps and my TextInput starts to bug (the text is not in his box anymore).
I started to read a lot of blog post and presentation about optimisation for the mobile platform and then I rewrite some of the components I use. I removed the SkinnableContainer for instance and replaced it by a VGroup including some actionScript based drawing.
Now what you see is a VGroup (the dark grey one) containing some others VGroup (the group with title here) and then each widget is an HGroup with a label and a Widget. I only use Label and TextInput for the text.
Creation time is slow even (several seconds to create the view) for another page where there is only 4 text widget on it, or another one with only a list with a custom item renderer where each row is a set of 4 labels.
The whole things is cabled with RobotLegs, with nothing fancy, one models is injected in the view and at the beginning I set a member variable on the view with this object to bind my variables.
Frankly my thinking right now is : it smells fishy because if I've done everything right it is impossible to have such low performance and thinks that flex is competitive on the mobile platform. So right now I'm trying to disable the application piece by piece to try to locate what could slow it like that. I've got a couple suspects to check, for instance I've got some binding warning to check, and then see if robotlegs has got its share of the problem.
So my main question here is what do you think, and could you have some ideas about "is there a problem" and "how do we solve it".
Thanks
Run profiler for startup and separatelly for each operation that takes longed that it needs. Then prioritize the problems and try to solve them with basic optimization techniques.
Some problems you will not be able to solve fast - e.g. time for creating big components. The only option there is to rewrite those components with AS3 without MXML, styles and anything. I'm sure that flash.text.TextField is created many times faster than mx.controls.Label. The same for other components.
When component is created, it can be reused at a very low price. In your app there must be a lot of places where you recreate while you can reuse old components. It will save you memory and time.
Layouts tend to redraw even when it's not needed. If you have a lot of nested layouts, find the most critical places and replace a series of layouts with one custom layout or even component.
This all is very developer time consuming. At the end you will not get a smooth app anyway, but I believe that it can become usable.

Updating the region behind a resized window

We have a fairly complex GUI, so when certain windows are resized their Redraw() is set to false till the operation is completed. The problem with this is that if the OS "Show window content while dragging" setting is checked, when decreasing the window's size the windows behind it are not repainted. This means I have to force the repaint myself so the remains of the resized window are deleted. I have no problem getting the dimensions of the region that was uncovered. What I'm looking for is best way to cause all windows within that region to repaint their part.
Not being much of a GUI programmer, I can traverse the uncovered region and list the windows in it. Then, I can ask each one of them to repaint its part. But I'm quite certain there has to be a better way to do this...
It is worth mentioning the app is written in PowerBuilder. This means I can call whatever Win32 function I'd like, but have limited control over the GUI behavior and the message handling. If there's a better way to prevent the window's content resize from being visible, or there's a way to make a non-redrawn window clean after itself, I'd love to hear it (just have the limitations above in mind).
I'm curious what version of PowerBuilder you are working in? I do resizing all the time and never run into issues like you are describing.
Maybe you can lay out some more detail on why you need to set your redraws to false within the PowerBuilder environment.
Hope I can help.

Resources