NSView performance of wantsLayer - macos

If I create a blank Mac XCode project and layout 500 simple NSView objects side by side in the main window it loads pretty damn fast. If I set wantsLayer=YES on each subview, performance dramatically drops, by several seconds. Why is this the case conceptually? It seems that layers would be faster not slower than regular old NSViews.

You're giving the system more work to do by layer-backing so many views. Layer-backing allows graphic acceleration (for drawing) but it adds a bit of overhead to things like layout, not to mention just creating them and putting them on screen. If used properly, it's not really much of a problem.
Typically, if you had so many "things" to manage on screen at once, you'd have one layer-backed hosting view that manages its own tree of sublayers. "But what about view-based table views?" you ask. Trickery, trickery, I say! Table views don't actually keep all the cell views they manage around; they efficiently reuse them, keeping around only enough to represent what's on screen and/or animating around.
So I'd say this isn't really a problem since it's not a particularly good approach to throw 500+ layer-backed views up for layout and drawing to begin with. :-)

as of 2021, Joshua Nozzi's answer is just the half of the truth.
When you want to utilise that much layers, you should make use of the power of CALayers with Sublayers instead of using NSViews over and over again, each with its own NSCell and possible CALayer backed as you forced it to with -wantsLayer:.
There is nothing wrong with 500 sublayers. Sublayers allow you to be fast when properly structured.
If you want to speed up even more, turn off autoanimation by force on each Layer that does not need it. Yes a lot examples out there show that you can turn off Autoanimation each time you call some property to be changed, slowing down your drawing even more, because it involves even more object messaging.
Keep in mind the following example turns off Auto-animated Keys by their Key name and kicks out the whole action and does not apply onto all Sublayers on purpose. If your structure of Sublayers of Sublayers need it you could call this on according ParentLayer to wipe out the CPU eating Autoanimation. Its much better to let the autoanimation do its job on the layers you actually want it to. Which turns around Apples default paradigm to animate everything. If most action keys are turned off be aware that your layers do show up kind of like a state machine. In the example here #"frame" animations are not wiped out..
void turnOffAutoAnimationsForLayerAndSublayers(CALayer* layer) {
NSNull *no = [NSNull null]; //(id)kCFNull
NSDictionary *dict = [NSDictionary dictionaryWithObjects:#[no,no,no,no,no,no,no,no]
forKeys:#[#"bounds",#"position",#"contents",#"hidden",#"backgroundColor",#"foregroundColor",#"borderWidth",#"borderColor"]];
for (CALayer *layertochange in layer.sublayers ) {
if (layertochange.sublayers.count) {
for (CALayer *sublayertochange in layertochange.sublayers) {
sublayertochange.actions = dict;
}
}
layertochange.actions = dict;
}
}
and use it in your CALayer backed NSView -init like..
CALayer *layer = [CALayer layer];
layer.frame = self.frame;
//...backgroundColor, contents, etc etc..
// here you could even do a for loop adding as much sublayers you want..
for (int y=0; y<100; y++) {
CALayer *sub = [CALayer layer];
//...frame, backgroundColor, contents, of sublayers etc etc..
layer.frame = CGRectMake(0, 0, 10, y*10);
[layer addSublayer:sub];
}
// structure is ready, add it to the base layer
self.layer = layer;
// now tell NSView you really want to make use this layer structure
self.wantsLayer = YES;
// turn off AutoAnimations
turnOffAutoAnimationsForLayerAndSublayers(self.layer);
For layers that are basically all the same there is even a specialised Subclass called CAReplicatorLayer. Allowing you to add even more with less internal draw calls.
You should also know that layers that are hidden are not calculated at all (meaning drawing here). You can still change properties even on hidden Layers. Unhide them, when needed, not earlier. So in example a custom CATextLayer that will change its string property to lets say #"", aka nothing, will still be drawn. But if you Subclass CATextLayer and change its string implementation to lets say..
#interface YourTextAutoHideLayer : CATextLayer
-(void)setString:(id _Nullable )string;
#end
#implementation YourTextAutoHideLayer
-(void)setString:(id _Nullable)string {
self.hidden = [string isEqual:#""];
super.string = string;
}
#end
you speed up drawing even more. A) because less object messaging to hide the layer that has no text anyway and B) once hidden it is not part of the internal draw calls effective speeding up your main CALayers drawing.
On NSTableViews that are NSCell based you usually never use CALayers. There is no need to, NSTableViews manage the visible Cells on purpose to keep in example scrolling smoothly. And still they (NSTableViewCells) have a reuse-mechanism to speed things up. You will very likely not re-invent a tableview with CALayers anyway. But in case you do reinvent, there is a CAScrollLayer class too.
And if that is not enough speed it is worth thinking about a MetalView utilising the power of GPUs.
Edit: code your CALayers not in -drawRect:. -drawRect: is called anytime something changes, in example frame/position on screen or bounds etc.. so try to avoid coding CALayers in there. You can set [self setNeedsLayout:YES]; & [self setNeedsDisplay:YES]; on purpose for a good reason, the reason is you want to avoid too much drawing for basically no change at all. -drawRect: in example was/is such method that is supposed to handle a backing with its own context to draw in. As CALayers have their very own mechanism you can let the drawRect block blank, doing nothing in there, maybe even erasing the method at all if you dont need it.
CALayers are not part of the auto layout system unless you code in -layout.. so again, keep CALayer drawing outside such and you are good. and pssst a lot CALayers properties can be changed even outside the main thread, some definitely not like layer.string.

Related

NSImageView always displays on top of any other kind of view

I'm working on a database application that has a graphical editor that is very similar to Interface Builder. Unlike IB, however, this editor can be flipped from editor mode to live mode, where the user interface is fully operational (buttons can be clicked, text edited, etc.) To do this, the graphical editor uses standard Appkit interface classes -- NSButton, NSTextView, etc. The editor itself is implemented with a custom subclass of NSView. All of the user interface elements are subviews of this custom NSView, new elements are added using the addSubview: method, making the new element the topmost visible element (note -- the views are not layer-backed, just regular views). The user can also use Bring-to-Front and Send-to-Back commands to change the ordering of the subviews. This movie shows two overlapping NSButton elements (for illustration purposes, of course normally you would never overlap them), and how the program can re-arrange the subview to change the Z-order of the user interface elements.
The problem is, this works with every kind of interface element except NSImageView. In the movie before there are two elements, an NSButton and an NSImageView. The NSButton is actually "on top" the whole time, the NSImageView element should appear behind the button, but no matter the order of the subviews, the NSImageView always appears on top.
If there are two overlapping NSImageView objects, the visible stacking order between them is unpredictable, but they will always appear above all other objects, no matter what the order of the subviews is.
A possibly useful clue is that if I implement my own custom view that draws an image directly in it's drawRect: method, that works fine. So that's one possible solution, but I am reluctant because that means re-implementing a large swath of useful features that NSImageView normally takes care of, some of which are quite complex like supporting animated GIF display. Other than this layering/z-order issue, everything else about NSImageView works fine.
Perhaps NSImageView is using layer-backing without my asking for it, so it isn't mixing in properly with my other objects? I can't find any documentation that indicates this. I am not linking against the QuartzCore framework.
Here is the code that adds the NSImageView element as a subview to the graphic editor view.
- (void)objectDidAppearBelow:(NSView *)nextView
{
FormView * formView = [FormWindowController currentFormView]; // get view element will be placed into
NSScrollView * imageContainer = [[NSScrollView alloc] initWithFrame:insideBorderRect];
ImageView * ixView = [[ImageView alloc] initWithFrame:[self insideFormObjectBorder:objectRectangle]];
[ixView setOwnerObject:self];
[imageContainer setDocumentView:ixView];
[imageContainer setAutoresizesSubviews:YES];
[shapeView addSubview:imageContainer Below:nextView];
imageDocumentView = ixView; // save weak reference to image view so it can be manipulated
}
In other places there is nearly identical code for NSButton (in several variations for push buttons, radio buttons, etc.), NSTextView, NSTableView (for lists and matrixes), NSSlider, NSScroller, NSSegmentedControl, even WebView. All the others work correctly with overlapping objects, including WebView, only NSImageView doesn't work as expected.
For my reference this is #429 in the Panorama X issue tracker.
I discussed this problem with Apple engineers at WWDC 2018. It turns out that as I suspected, in some cases Appkit will use layer backing for NSImageView even if you did not ask for it! So the best solution is to switch all views to layer backing (which will happen automatically with Mojave).
In this particular case the NSImageView was inside a NSScrollView, which I didn't mention because I didn't think it was important (my bad). It turns out, this is the case where Appkit thinks using layer backing would be a good idea (to optimize scrolling). So another way to fix this is to subclass NSImageView (which I had already done for other reasons) and add this method (written on the spot by an Apple engineer who prefers to remain uncredited).
+ (BOOL)isCompatibleWithResponsiveScrolling {
if (NSAppKitVersionNumber <= 1561. /* NSAppKitVersionNumber10_13 */) {
return NO;
} else {
return YES;
}
}
I was assured that this is all part of the public, documented API, though the documentation is minimal (surprise surprise). There is some discussion of responsive scrolling in the What's new in OS X 10.9 release notes. The check with NSAppKitVersionNumber is to make sure to turn off this patch when running on Mojave, since everything is layer backed.

Are layer-hosting NSViews allowed to have subviews?

Layer-hosting NSViews (so NSViews that you supply a CALayer instance for and set it with setLayer:) can obviously contain subviews. Why obviously? Because in Apple's own Cocoa Slides sample code project, you can check a checkbox that switches the AssetCollectionView from being layer-backed to being layer-hosting:
- (void)setUsesQuartzCompositionBackground:(BOOL)flag {
if (usesQuartzCompositionBackground != flag) {
usesQuartzCompositionBackground = flag;
/* We can display a Quartz Composition in a layer-backed view tree by
substituting our own QCCompositionLayer in place of the default automanaged
layer that AppKit would otherwise create for the view. Eventually, hosting of
QCViews in a layer-backed view subtree may be made more automatic, rendering
this unnecessary. To minimize visual glitches during the transition,
temporarily suspend window updates during the switch, and toggle layer-backed
view rendering temporarily off and back on again while we prepare and set the
layer.
*/
[[self window] disableScreenUpdatesUntilFlush];
[self setWantsLayer:NO];
if (usesQuartzCompositionBackground) {
QCCompositionLayer *qcLayer = [QCCompositionLayer compositionLayerWithFile:[[NSBundle mainBundle] pathForResource:#"Cells" ofType:#"qtz"]];
[self setLayer:qcLayer];
} else {
[self setLayer:nil]; // Discard the QCCompositionLayer we were using, and let AppKit automatically create self's backing layer instead.
}
[self setWantsLayer:YES];
}
}
In the same AssetCollectionView class, subviews are added for each image that should be displayed:
- (AssetCollectionViewNode *)insertNodeForAssetAtIndex:(NSUInteger)index {
Asset *asset = [[[self assetCollection] assets] objectAtIndex:index];
AssetCollectionViewNode *node = [[AssetCollectionViewNode alloc] init];
[node setAsset:asset];
[[self animator] addSubview:[node rootView]];
[nodes addObject:node];
return [node autorelease];
}
When I build and run the app and play around with it, everything seems to be fine.
However, in Apple's NSView Class Reference for the setWantsLayer: method it reads:
When using a layer-hosting view you should not rely on the view for
drawing, nor should you add subviews to the layer-hosting view.
What is true? Is the sample code incorrect and it's just a coincidence that it works? Or is the documentation false (which I doubt)? Or is it OK because the subviews are added through the animator proxy?
When AppKit is "layer hosting" we assume you may (or may not) have a whole subtree of layers that AppKit doesn't know about.
If you add a subview to the layer hosted view, then it might not come out in the right sibling order that you want. Plus, we sometimes add and remove them, so it might change depending on when you call setLayer:, setWantsLayer: or when the view is added or removed from the superview. On Lion (and before) we remove the layers that we "own" (ie: layer backed) when the view is removed from the window (or superview).
It is okay to add subviews...their children-sibling-order in the sublayers array just might not be deterministic if you have sibling-layers that aren't NSViews.
I don't know what's the "right" answer to this. But I do think that the CocoaSlides example works within the boundaries of what the docs say you "shouldn't" do. In the example, look at where the insertNodeForAssetAtIndex: method is called, and you'll see that it only happens when the view is being populated, before it ever is assigned a layer or has setWantsLayer: called on it.
The docs don't say that a layer-hosted view can't contain any subviews, they just say that you can't add and subviews to one. At the point in time when those subviews are added, the main view hasn't yet become a layer-hosting view. After it has been turned into a layer-hosting view by having a manually created layer assigned to it, no more subviews are added.
So there's really no contradiction between the docs and this particular example. That being said, it could be interesting to explore this further, maybe by switching on the QC background layer right from the start, e.g. by sticking a [self setUsesQuartzCompositionBackground:YES]; right inside initWithFrame:.
SPOLIER ALERT:
It seems to work just fine. The creation of the display is a bit slower (not surprising with all that QC animation going on), but apart from that it's smooth sailing.
One comment about this code from Apple: it's busted.
When you first start the app up, note the nice gradient background. Turn QC on, then off.
Poof, no more gradient background.

CALayer Live Resize Poor Performance

I have a UI where the content of an NSCollectionViewItem's View is drawn programmatically through CALayers. I am using a CAConstraintLayoutManager to keep the layot of the sublayers consistent when resizing, but I am getting very poor performance when doing so. It seems that resizing the window, which causes the resize of two CATextLayers so that they fit the root layer's width, and the repositioning of one CATextLayer so that it stays right-aligned, is causing the application to spend most of its time executing the CGSScanConvolveAndIntegrateRGB function (I have used the Time Profiler instrument).
The most "expensive" layer (the one that causes the most stuttering even if it's the only one displayed) is a wrapped multiline CATextLayer. I have absolutely no idea how to get better performance (I have tried not using a CAConstraintLayoutManager and going with layer alignments but I'm getting the same thing). Has anyone had this problem? Is there a way around it?
PS: I have subclassed the layout manager and disabled all the animations during the execution of - (void)layoutSublayersOfLayer:(CALayer *)layer by setting YES to kCATransactionDisableActions in the CATransaction but it doesn't seems to help.
Edit: I have disabled Font Smoothing for the Text Layers and performance has increased a little bit (very little), but it spends an awful amount of time in _ZL9view_drawP7_CAViewdPK11CVTimeStampb (which is something that gets called by a thread of the ATI Radeon driver, I suppose).
I solved it. Kind of. It still seems like a dirty hack to me, but I couldn't find out how to make setNeedsDisplayInRect work so I ended up doing it like this:
In the NSWindow delegate:
-(void)windowWillStartLiveResize:(NSNotification *)notification
{
[[NSNotificationCenter defaultCenter] postNotificationName:#"beginResize" object:nil];
}
-(void)windowDidEndLiveResize:(NSNotification *)notification
{
[[NSNotificationCenter defaultCenter] postNotificationName:#"endResize" object:nil];
}
In my Custom View those two notifications call, respectively, the -(void)beginResize and -(void)endResize selectors. The first one sets a BOOL inLiveResize variable to YES, while the second one sets it to NO and calls setFrameSize again with the new frame size.
I overrode (overridden? Not native english speaker, sorry) the -(void)setFrameSize:(NSSize)newSize method like this:
-(void)setFrameSize:(NSSize)newSize
{
if (inLiveResize) {
NSRect scrollFrame = [[[self superview] enclosingScrollView] documentVisibleRect];
BOOL condition1 = (self.frame.origin.y > (scrollFrame.origin.y - self.frame.size.height));
BOOL condition2 = (self.frame.origin.y < (scrollFrame.origin.y + scrollFrame.size.height + self.frame.size.height));
if (condition1 && condition2)
[super setFrameSize:newSize];
}
else {
[super setFrameSize:newSize]; }}
That's it. This way, only the visible views resize live with the window, while the others get redrawn at the end of the operation. It works, but I don't like how 'dirty' it is, I'm sure there is a more elegant, built-in(ish) way to do this by using the setNeedsDisplayInRect method. I will research more.

Core animation code structure/conventions

In learning Core Animation, I learned very quickly that if you don't do it right, you get really weird undefined behavior. To that end, I have a few questions that will help me conceptually understand it better.
My NSView subclass declares the following in it's init. This view is a subview of normal layer backed view.
[self setLayer:[CALayer layer]];
[self setWantsLayer:NO];
After this, when and in what situations should I refer to self as opposed to [self layer]? I have been ONLY manipulating the layer with explicit and implicit animations, staying away from [self setFrame:] etc. and using [[self layer] setPosition] etc.
The problem with this approach is that the actual frame of the view stays in one spot throughout any and all animations applied. What if my view is supposed to recieve mouse events? For example, I have a view that uses core animation and it is dragged around by the mouse. Is there a way I can somehow keep the view frame synced with the current state of the presentation layer so I can receive proper mouse events?
About the presentation layer, apparently it's only available when an actual animation is in progress. Is there any sort of property of the layer that can tell me where it's ACTUALLY visually at even when an animation's not in progress?
I think you need to re-phrase your question a little. It seems there is some underlying misunderstanding, but you're not really expressing it very clearly. You're question title suggests you're looking to understand something more theoretical, but your actual question suggests you're looking for something more concrete. Let me see if I can clarify a few things.
The presentationLayer provides information about the layer's current state while "in-flight".
When there is no animation occurring, the presentationLayer and the layer information will be identical. Query the layer's bounds, frame, or position to find out where it is currently in its parents coordinate space.
NSViews must have layer backing enabled to be able to perform animations.
Make sure you're not just animating with an explicit animation and not actually setting the layer value that you're animating. Animations don't automatically change the properties of the layers they're animating. You have to change the property to the ending value yourself or it will just "snap back" to the starting value.
If you want to animate the view, as opposed to a layer, you can use the animator proxy, like [[view animator] setFrame:newFrame];
Wrap calls to the animator in a CATrasaction to alter things like animation duration.
Let me know if you need some clarification by updating your question. Providing some pertinent code would really help identify the problems you're having trouble solving.
Firstly, you want to use [self setWantsLayer: YES]. Also, it's only important to call -setLayer: before -setWantsLayer: if you want to provide a specific CALayer subclass (such as a CAScrollLayer); if you just want a regular CALayer you just call -setWantsLayer: and it'll be created for you. Even better, just check the 'wants layer' option in Interface Builder.
Secondly, the entire point of using a layer-backed view is that you can continue to use the regular NSView methods and get the free CoreAnimation 'tweening' effects. If you want to use CoreAnimation as your only means of moving items around, then the correct way to do so is to create a layer backed view which contains your pure-CALayer presentation hierarchy.
I've not looked at any freely-available CoreAnimation tutorials, but I can definitely recommend the Pragmatic Programmers' book on the subject. They also have a screencast available by the book's author.

Is there a proper way to handle overlapping NSView siblings?

I'm working on a Cocoa application, and I've run into a situation where I would like to have two NSView objects overlap. I have a parent NSView which contains two subviews (NSView A and NSView B), each of which can have several subviews of their own.
Is there a proper way to handle this kind of overlap? NSView B would always be "above" NSView A, so I want the overlapped portions of NSView A to be masked out.
Chris, the only solution is to use CALayers. That is definitely the one and only solution.
NSViews are plain broken in OSX (Sept 2010): siblings don't work properly. One or the other will randomly appear on top.
Just to repeat, the problem is with siblings.
To test this: using NSViews and/or nsimageviews. Make an app with a view that is one large image (1000x1000 say). In the view, put three or four small images/NSViews here and there. Now put another large 1000x1000 image on top. Build and launch the app repeatedly - you'll see it is plain broken. Often the underneath (small) layers will appear on top of the large covering layer. if you turn on layer-backing on the NSViews, it does not help, no matter what combo you try. So that's the definitive test.
You have to abandon NSViews and use CALayers and that's that.
The only annoyance with CALayers is that you can't use the IB to set up your stuff. You have to set all the layer positions in code,
yy = [CALayer layer];
yy.frame = CGRectMake(300,300, 300,300);
Make one NSView only, who's only purpose is to hold your first CALayer (perhaps called 'rear'), and then just put all your CALayers inside rear.
rear = [CALayer layer];
rear.backgroundColor = CGColorCreateGenericRGB( 0.75, 0.75, 0.75, 1.0 );
[yourOnlyNsView setLayer:rear]; // these two lines must be in this order
[yourOnlyNsView setWantsLayer:YES]; // these two lines must be in this order
[rear addSublayer:rr];
[rear addSublayer:yy];
[yy addSublayer:s1];
[yy addSublayer:s2];
[yy addSublayer:s3];
[yy addSublayer:s4];
[rear addSublayer:tt];
[rear addSublayer:ff];
everything then works utterly perfectly, you can nest and group anything you want and it all works flawlessly with everything properly appearing above/below everything it should appear above/below, no matter how complex your structure. Later you can do anything to the layers, or shuffle things around in the typcial manner,
-(void) shuff
{
[CATransaction begin];
[CATransaction setValue:[NSNumber numberWithFloat:0.0f]
forKey:kCATransactionAnimationDuration];
if ..
[rear insertSublayer:ff below:yy];
else
[rear insertSublayer:ff above:yy];
[CATransaction commit];
}
(The only reason for the annoying 'in zero seconds' wrapper for everything you do, is to prevent the animation which is given to you for free - unless you want the animation!)
By the way in this quote from Apple,
For performance reasons, Cocoa does
not enforce clipping among sibling
views or guarantee correct
invalidation and drawing behavior when
sibling views overlap.
Their following sentence ...
If you want a view to be drawn in
front of another view, you should make
the front view a subview (or
descendant) of the rear view.
Is largely nonsensical (you can't necessarily replace siblings with subs; and the obvious bug described in the test above still exists).
So it's CALayers! Enjoy!
If your application is 10.5-only, turn on layers for the views and it should just work.
If you're meaning to support 10.4 and below, you'll need to find a way not to have the views overlap, because overlapping sibling views is undefined behavior. As the View Programming Guide says:
For performance reasons, Cocoa does not enforce clipping among sibling views or guarantee correct invalidation and drawing behavior when sibling views overlap. If you want a view to be drawn in front of another view, you should make the front view a subview (or descendant) of the rear view.
I've seen some hacks that can make it kinda work sometimes, but it's not anything you can rely on. You'll need to either make View A a subview of View B or make one giant view that handles both of their duties.
There is a way to do it without using CALayers and an App I have been working on can prove it. Create two windows and use this:
[mainWindow addChildWindow:otherWindow ordered:NSWindowAbove];
To remove "otherWindow" use:
[mainWindow removeChildWindow:otherWindow];
[otherWindow orderOut:nil];
And you will probably want to take the window's title bar with:
[otherWindow setStyleMask:NSBorderlessWindowMask];
As Nate wrote, one can use:
self.addSubview(btn2, positioned: NSWindowOrderingMode.Above, relativeTo: btn1)
However, the ordering of views is not respected as soon as you ask either of the views to redraw it self through the "needDisplay = true" call
Siblings wont get the drawRect call, only the views direct hierarchy will.
Update 1
To solve this problem I had to dig deep, very deep. Probably a week of research and ive spread my findings over a few articles. The final break-through is in this article: http://eon.codes/blog/2015/12/24/The-odd-case-of-luck/
Update 2
Be warned though the concept is hard to understand, but it works, and it works great. Here is the end result and code to support it, links to the github repo etc: http://eon.codes/blog/2015/12/30/Graphic-framework-for-OSX/
In order to ensure that NSView B always overlaps NSView A, make sure that you use the correct NSWindowOrderingMode when you are adding the subview:
[parentView addSubview:B positioned:NSWindowAbove relativeTo:A];
You should also keep in mind that the hidden portions of A will not be asked to redraw if view B is 100% opaque.
If you are moving the subviews, you also need to make sure that you call
-setNeedsDisplayInRect: for for the areas of the view you are uncovering.

Resources