I'm trying to get an animation to run smoothly, at the refresh rate of the display without screen tearing. The animation is rendered using Metal. As far as I understand Apple tells you to use CVDisplayLink based timers for this, and that is what I've done.
Everything works fine on desktop computers and on laptops when they are connected to a power adaptor. However, when the laptops run on battery, especially when they battery isn't full, I can see very noticeable stuttering in the animation. There is no tearing, though. It seems like the timer is not fired on every screen refresh.
I'm pretty sure this is not because the CPU is throttled down. The CPU utilisation is below 10% and the animation takes less than 2ms to calculate and render; and at 60Hz it would have 16ms to do so.
For what it's worth, this is how I set up the timer:
private func makeDisplayLink(window: NSWindow) -> CVDisplayLink
{
func displayLinkOutputCallback(_ displayLink: CVDisplayLink, _ inNow: UnsafePointer<CVTimeStamp>, _ inOutputTime: UnsafePointer<CVTimeStamp>, _ flagsIn: CVOptionFlags, _ flagsOut: UnsafeMutablePointer<CVOptionFlags>, _ displayLinkContext: UnsafeMutableRawPointer?) -> CVReturn {
unsafeBitCast(displayLinkContext, to: MetalScreenSaverView.self).animateOneFrame()
return kCVReturnSuccess
}
var link: CVDisplayLink?
let screensID = UInt32(window.screen!.deviceDescription["NSScreenNumber"] as! Int)
CVDisplayLinkCreateWithCGDisplay(screensID, &link)
CVDisplayLinkSetOutputCallback(link!, displayLinkOutputCallback, UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque()))
return link!
}
And later I'm starting the link by calling CVDisplayLinkStart with the link as the argument. The full code, if you are interested can be found at: https://github.com/thoughtworks/dancing-glyphs/blob/master/Library/MetalScreenSaverView.swift
Any ideas? Can I tell OS X somehow to ensure that the timer is fired on every screen refresh? Is this an issue with Metal? I've seen games and screen savers that run fine on battery, but I assume the use OpenGL.
If I'm reading your code correctly, you're assuming that CVDisplayLink is called at some unchangeable interval. That's not promised at all. The system is absolutely free to modify the refresh interval or drop frames. All real-time systems must include the ability to drop frames. That's the heart of what it means to be "real-time."
You're passed the "currently displayed" time and the "target output" time. You're supposed to use those to compute the correct frame for the target output. I don't see you making use of the output time in your code.
Related
In older versions of MacOSX, one would use
UpdateSystemActivity(UsrActivity);
In order to reset the screensaver timer.
In modern versions of MacOSX, the most commonly recommended solution is:
static IOPMAssertionID activity_assertion_id = kIOPMNullAssertionID;
IOReturn r = IOPMAssertionDeclareUserActivity(CFSTR("FractalUserActivity"),
kIOPMUserActiveLocal, &activity_assertion_id);
or
IOReturn result = IOPMAssertionCreateWithName(
kIOPMAssertionTypeNoDisplaySleep,
kIOPMAssertionLevelOn,
CFSTR("FractalNewFrameActivity"),
&power_assertion_id);
IOPMAssertionRelease(power_assertion_id);
However, neither of these in my testing appear to actually reset the MacOSX Screensaver timer. When I set my screensaver to 1minute, but run the above code in a loop at 60FPS, the screensaver still turns on eventually.
Often it's said that IOPMAssertionRelease should only be called after the screen no longer wants to be held awake, but I don't think that's the functionality I need. I need to simply reset the 1min screensaver timer. Because, the timer should be reset every single time a framebuffer gets rendered [due to the application changing what's being displayed]. But, if no frame gets rendered in a 1minute interval, the screensaver should be displayed.
Is there no way to do this in modern versions of MacOSX? Chromium currently exhibits the exact feature that I desire. When watching a YouTube Video, 1minute after the video ends, is when the screensaver will turn on, regardless of how long the youtube video is, even if it's e.g. a 10min Youtube Video.
~ I don't want to hardcode 1minute, since obviously it should regardless of what your screensaver time setting is.
I have a cocoa application window (NSWindow) which position on the screen should be updated frequently (depending on some calculation). As noticed in the documentation, UI changes should be made on the main thread:
void calculationThread()
{
while(true)
{
calculatePosition();
if(positionChanged)
{
dispatch_async(dispatch_get_main_queue(), ^{ setWindowPos(); });
}
}
}
void setWindowPos()
{
[window setFrame:_newFrame display:YES];
}
Now the problem I have is that the window movement is very slow and delayed. After making some profiling I see that the calculation process takes about 40mSec, meaning that I'm queueing up a backlog of UI updates 25 times a second.
I've read here that this might be faster than they can be processed and timer should be used to fire the changes every tenth of a second or so. But, wouldn't it be too slow for the human eye (I mean, in that case the movement wouldn't be delayed but would be lagged causing pretty much the same affect).
I will appreciate some knowledge sharing on this. Actually my main 2 questions are:
Are 25-30 UI updates per second really to much?
If yes, what is the recommended UI changes frequency?
The frequency at which a window can be moved around onscreen without problems will of course depend upon the speed of the user's machine, the video card they have, the size of the window, and probably a bunch of other factors. There is no single good answer to this. However, if you just drag a window around on your screen, you will notice that it can probably be moved very smoothly (unless your machine is very busy or very low on memory or something); I would not expect 25 times per second to produce a problem on a modern Mac. Not even close, in fact.
#RobNapier's points about Core Animation etc. are fine, but overstated I think; there is nothing inherently wrong with changing your UI using a timer or other periodic update if that is what you actually want to do. CoreAnimation is a toolkit for making some types of animation easier; using it is not required, and it is not suited to every problem. Similarly, if you want to make changes that are actually synched to screen refresh then CVDisplayLink is useful, but it doesn't really sound like that's what you want to do.
For your purposes, your basic approach seems fine, although I would suggest adding an NSDate check in order to skip updates if the previous update was less than, say, 1/60th of a second previous. After all, the calculation appears to take 40mSec on your machine, but it might be much faster on some other machine; you want to throttle your drawing to a reasonable rate just to be a good citizen.
So what is the problem, then? I suspect the issue might actually be your call [window setFrame:_newFrame display:YES]. If you look at Apple's docs for that method, they state "When YES the window sends a displayIfNeeded message down its view hierarchy, thus redrawing all views." Each time you call that method, then, you are not only moving your window (which I gather is your intention); you are redrawing all of the contents of the window, too, and that is slow. If you don't need to do that, then that is the overhead you need to eliminate. Call setFrameOrigin: or setFrameTopLeftPoint: instead (which make the semantics clear, that you are moving the window without resizing it or redrawing it), or perhaps just setFrame:display: passing NO instead of YES, and I'm guessing your performance problem will vanish.
If you do in fact need to redraw the window contents every time, then please edit the problem description to reflect that. In that case, the solution will have to involve profiling why your window drawing is slow, and figuring out ways to optimize that, which is an entirely different problem.
As you've discovered, you should never try to drive the UI from a tight loop. You should let the UI drive you. There are three primary tools for that.
For simple problems, AppKit is capable of moving windows around the screen. Just call [NSWindow setFrame:display:animate:]. You can override animationResizeTime: to modify the timing.
In many cases AppKit doesn't give enough control. In those case, the best tool is almost always Core Animation. You should tell the system using Core Animation how you where you want UI elements to wind up, and over what period and path, and let it do the work of getting them there. See the Core Animation Programming Guide for extensive documentation on how to use that. It focuses on animating CALayer, but the techniques are similar for NSWindow. You'll use [NSWindow setAnimations:] to add your animation. Look at the NSAnimatablePropertyContainer protocol (which NSWindow conforms to) for more information. For a simple sample project of animating NSWindow, see Just Say No from CIMGF.
In a few cases, you really do need to update the screen manually at the screen update frequency. I must stress how rare this situation is. In almost all cases, Core Animation is the correct tool. But in those rare case (some kinds of video for instance), you can use a CVDisplayLink to handle this. That will call you each time the screen would like to refresh, giving you an opportunity to update your content to match.
I'm making a fairly complex sprite kit game. I recently added support for OS X. I get 60 fps always, regardless of how my game is scaled when the window is resized (even when resized to max screen space). However, the moment I make my App enter "Full Screen," the fps drops to 30-40 fps and stays that way? But if I take my mouse cursor and reveal the menu bar while full screen is enabled, the fps goes back up to 60 fps!
You can even test this bug by making a sprite kit game for mac in Xcode using the default template. Here are the screen shots I took of the default game template for mac.
I suggest trying it out for yourself, you don't even have to write any code if you use Apple's default sprite kit template for OS X.
Max Window (No FPS Problems: 59-60 FPS)
Full Screen Mode (FPS Drops to 30-40 FPS)
Full Screen Mode With Mouse At Top Revealing Menu Bar (Surprisingly, NO FPS Issues: 59-60 FPS)
Anyone have any idea what might be causing this problem. I don't want to release my App with full screen mode if it means users will lose performance. You would think full screen mode could better optimize drawing but apparently it's quite the opposite. I'm running this on Yosemite.
Ok, after weeks looking into this issue I have found some workarounds to this issue. Before I begin, let me start by explaining my setup. I'm using an NSViewController in a storyboard which holds an SKView. I've tested the workaround on MacBook Pro (Retina, 15-inch, Early 2013), I have no idea if the workarounds I present below will work on other Macs. I believe it should, when I get the chance I will test and see if the workarounds below work.
So before I begin, lets recap what the issue is. The issue is that making your App enter fullscreen by clicking the fullscreen button causes a massive drop in FPS. Below is how you enable the fullscreen button:
self.view.window!.collectionBehavior = .FullScreenPrimary
So then I searched around and found a different way of entering fullscreen using this code:
self.view.enterFullScreenMode(NSScreen.mainScreen()!, withOptions: nil)
But I still had a massive drop in FPS. Keep in mind, I had no fps issues when in maximized window mode or even full screen with the menu bar visible! (see pictures in question).
So then I tried a less high-level approach to going full screen. I found a guide by Apple here
Using some of the code from the guide, I managed to enter fullscreen by setting the window size to the size of the display, and positioning the window above all OS X UI. The code for this is as follows:
self.view.window!.styleMask = NSBorderlessWindowMask
self.view.window!.level = Int(CGWindowLevelForKey(Int32(kCGMainMenuWindowLevelKey))) + 1
self.view.window!.opaque = true
self.view.window!.hidesOnDeactivate = true
let size = NSScreen.mainScreen()!.frame.size
self.view.window!.setFrame(CGRect(x: 0, y: 0, width: size.width, height: size.height), display:true)
But, sadly, same problem... The FPS just dropped just like before.
So then I thought what if I mess with the size/position of the window. So I tried moving the window down so that just the menu bar was visible as shown below. AND THIS WORKED. I no longer had a drop in fps. But obviously it's not truly fullscreen because the menu bar is visible
self.view.window!.setFrame(CGRect(x: 0, y: 0, width: size.width, height: size.height-NSApplication.sharedApplication().mainMenu!.menuBarHeight), display:true)
In fact, as it turns out, just be adjusting the window size by 1 point fixes the drop in fps. Thus the bug must be related to an optimization (how ironic) apple does when your window size matches the screen size.
Don't believe me? Here is a quote from the link.
OS X v10.6 and later automatically optimize the performance of
screen-sized windows
So to fix the issue all we need to do is make our window size height 1 point larger which will prevent OS X from trying to optimize our window. This will cause your App to get slightly cut off on the top but 1 pixel shouldn't be noticeable at all. And in the worst case you could adjust your nodes position by 1 point to account for this.
For your convenience, listed below are the 2 workarounds. Both of these workarounds do not cause any drop in FPS. Your App should function just like it did in maximized window mode. The first workaround puts your App in fullscreen and displays the menu bar at the top. The second workaround puts your App in complete full screen with no menu bar.
Workaround 1: Fullscreen with Menu Bar
self.view.window!.styleMask = NSBorderlessWindowMask
self.view.window!.level = Int(CGWindowLevelForKey(Int32(kCGMainMenuWindowLevelKey))) + 1
self.view.window!.opaque = true
self.view.window!.hidesOnDeactivate = true
let size = NSScreen.mainScreen()!.frame.size
self.view.window!.setFrame(CGRect(x: 0, y: 0, width: size.width, height: size.height-NSApplication.sharedApplication().mainMenu!.menuBarHeight), display:true)
Workaround 2: Fullscreen with no Menu Bar
self.view.window!.styleMask = NSBorderlessWindowMask
self.view.window!.level = Int(CGWindowLevelForKey(Int32(kCGMainMenuWindowLevelKey))) + 1
self.view.window!.opaque = true
self.view.window!.hidesOnDeactivate = true
let size = NSScreen.mainScreen()!.frame.size
NSMenu.setMenuBarVisible(false)
self.view.window!.setFrame(CGRect(x: 0, y: 0, width: size.width, height: size.height+1), display:true)
If for some reason these workarounds don't work, try messing some more with the size/position of the window. Also you may need to change the window level depending on if you have other views such as dialogues that your App should not overlap. Also please remember to file bug reports with Apple.
Additional Info About NSBorderlessWindowMask
These workarounds use an NSBorderlessWindowMask. These type of windows do not accept keyboard input when the key window changes. So if your game uses keyboard input, you should override the following. See here
class CustomWindow: NSWindow {
override var canBecomeKeyWindow: Bool {
get {
return true
}
}
override var canBecomeMainWindow: Bool {
get {
return true
}
}
}
Update: Some bad news
Tested this workaround on Mac Book Air, and it did not work unless about 100 points were subtracted (which obviously is extremely noticeable). I have no idea why. Same goes for andyvn22's solution. I also have noticed that very rarely, perhaps once every 60 launches the workarounds provided simply don't work on the Mac Book Air at all. And the only way to fix is to relaunch the App. Maybe the Max Book Air is a special case. Maybe lack of a graphics card has to do with the issue. Hopefully Apple gets the issue sorted out. I'm now torn between supporting fullscreen and not supporting fullscreen. I really want users to be able to enter fullscreen mode, but at the same time I don't want to risk users loosing half their FPS.
Based on Epic Byte's very helpful work, I found an even easier way to disable Apple's full screen "optimization". You can still use OS X's built in full screen capability; all you have to do is implement the following method in your window's delegate:
func window(window: NSWindow, willUseFullScreenContentSize proposedSize: NSSize) -> NSSize {
return NSSize(width: proposedSize.width, height: proposedSize.height - 1)
}
Unfortunately adding one pixel doesn't seem to work this way, only subtracting one, so you lose a row of screen space. Totally worth it to me, though, to continue using the built-in full screen function, especially while just waiting for Apple to fix their optimization bug.
I believe this problem occurs on all apps using OpenGL to render. MPV (video player) with the following video config has the same issues: vo=opengl hwdec=no
Cpu usage - windowed: average 42%
Cpu usage - fullscreen (native): 62%
Cpu usage - fullscreen (non-native/in app): 60%
Cpu usage - fullscreen (native with menu bar): 45%
Cpu usage - offscreen (using native full screen): 95%
This also occurs on PPSSPP with OpenGL backend except with increased GPU instead of cpu usage:
Gpu usage - windowed: average 20%
Gpu usage - fullscreen (with menu bar): 20%
Gpu usage - fullscreen (native): 35%
Gpu usage - offscreen (using native full screen): 90%
This problem however does not seem to occur when developers implement their own "Special" fullscreen. In the case of Enter the Gungeon, where cpu usage and gpu usage shows no difference between windowed and FS. Although I haven't had time to check how they've implemented fullscreen yet.
Tested on MBP Late 2015 13' on OSX 10.11.6
The slightly increased usage during fullscreen is a bit annoying as you've said and can cause framedrops, but what's worrying me the most is the near 100% usage of both CPU and GPU in openGL applications when in background. (Note: it's 90% on ppsspp no matter what it's doing, even when paused).
According to this article by Microsoft the screen refresh rate set by the user can be (and is mostly) a fractional number. The user sets 59Hz but the screen runs according to the on screen display at 60Hz, but in reality it's 59.94Hz. What I need for a extremely smooth animation is the 59.94Hz.
Using IDirect3DDevice9::GetDisplayMode I only get an int value which cannot by definition represent the real timing (same goes for EnumDisplaySettings). I encounter a visible stutter about every second because it reports the rounded/truncated 59. If I manually correct the reported timing in my application to 59.94 it runs smooth.
Anybody knows how I can retrieve the real screen refresh rate?
My current workaround is mapping 60Hz and 59Hz both to constant 59.94Hz but that's not satisfying.
If you are targeting Windows Vista or later, the answer depends on the mode in which your app is running.
If it is a windowed app (or windowed full-screen), refresh rate is controlled via the Desktop Window Manager (DWM) according to user settings and other factors. Use DwmGetCompositionTimingInfo and look at DWM_TIMING_INFO::rateRefresh to get the monitor refresh rate.
If the app is true full-screen, then the full-screen swap chain you create overrides the system default. However, your selected refresh rate (DXGI_SWAP_CHAIN_FULLSCREEN_DESC::RefreshRate) should match one of the monitor-supported refresh rates. You can get the list of supported refresh rates using IDXGIOutput::GetDisplayModeList. Here's an example of how to do so:
UINT numModes = 0;
dxgiOutput->GetDisplayModeList(DXGI_FORMAT_B8G8R8A8_UNORM, 0, &numModes, NULL);
DXGI_MODE_DESC* modes = new DXGI_MODE_DESC[numModes];
dxgiOutput->GetDisplayModeList(DXGI_FORMAT_B8G8R8A8_UNORM, 0, &numModes, modes);
// see modes[i].RefreshRate
In any case, you shouldn't see glitching if you're triple-buffered. You should just present as fast as you can and the OS will present on time. If you combine triple-buffering with custom managed frame timing, you're guaranteed to not actually get triple-buffering, and you'll get glitches any time there's drift in the vblank phase (which happens gradually even if you have a perfect value for refresh rate). If you want to stick with triple-buffering, just present as fast as you can and let the OS take care of presentation timing. If you're using your own timing to drive Present()s (for example, to get low-latency response), you should throw in a call to IDXGIOutput::WaitForVBlank on another thread to help synchronize frame timings. If you end up doing that, you should also use IDXGISwapChain::GetFrameStatistics to make sure you recover from any spurious glitches, otherwise you'll end up a frame behind.
Good luck!
I am trying to implement frame independent movement in SFML, but am having trouble getting it to work. When I had this problem earlier with SDL, I found somewhere that Windows pauses the main thread of an application whenever it is moved (by dragging the title-bar). The problem comes because when the window is being dragged, the clock updates but the movement is not drawn until I let go of the window. When I move the window, the window is no longer being drawn to, but the time is still increasing. Thus, when I let go of the window, the units immediately jump to where they would be if the window had not been dragged.
I tried thinking of a solution, and since Windows only pauses the main thread, I considered just running the entire game in a separate thread, and launching it in main() but that does not appear to work as the same result occurs. I also thought about the extremely low FPS's I get as a result, but I would have no way of being able to differentiate between someone dragging the window and if their game is just naturally running slowly... There has to be a way to either prevent windows from pausing the main thread, or doing something that prevents this issue, but I haven't found any sort of solution on the internet...
Here is a link to a zip file which demonstrates the problem. Both Demo0 and Demo1 are the same, except Demo1 uses a second thread to run the program, yet the same effect occurs. Just run both and watch as the delta value is output to command line. Then drag the window and move it to some other part of the screen. When you let go, you should see a very large delta value and the circle should jump ahead depending on how long you had the window suspended. The source code is all there (in the "src" folder), so I hope people can understand the exact problem: http://www.sendspace.com/file/4er8f4
I see two solutions to the problem:
You can try to see if the sf::Events LostFocus or Resized picks up a window drag, and if they do, simply pause the clock on the game. More information can be found here: http://www.sfml-dev.org/tutorials/2.0/window-events.php
However, if that doesn't work, I would simply add an upper-cap on your delta. Meaning, if the game goes above a certain threshold of delta (1/60 or 1/30), you set delta to a lower value. In your situation though, this cap could probably be really big, like 1/15.
if(delta > 1/15.0f)
delta = 1/15.0f;
Chances are you don't expect your game to be playable at 15fps anyways, and if the user drags the window while moving, the worst you'll have to deal with is a 15fps delta on resume.