In Windows World, a dedicated render thread would loop something similar to this:
void RenderThread()
{
while (!quit)
{
UpdateStates();
RenderToDirect3D();
// Can either present with no synchronisation,
// or synchronise after 1-4 vertical blanks.
// See docs for IDXGISwapChain::Present
PresentToSwapChain();
}
}
What is the equivalent in Cocoa with CAMetalLayer? All the examples deal with updates being done in the main thread, either using MTKView (with it's internal timer) or using CADisplayLink in the iOS examples.
I want to be in control of the whole render loop, rather than just receiving a callback at some non-specified interval (and ideally blocking for V-Sync if it's enabled).
At some level, you're going to be throttled by the availability of drawables. A CAMetalLayer has a fixed pool of drawables available, and calling nextDrawable will block the current thread until a drawable becomes available. This doesn't imply you have to call nextDrawable at the top of your render loop, though.
If you want to draw on your own schedule without getting blocked waiting on a drawable, render to an off-screen renderbuffer (i.e., a MTLTexture with dimensions matching your drawable size), and then blit from the most-recently-drawn texture to a drawable's texture and present on whatever cadence you prefer. This can be useful for getting frame timings, but every frame you draw and then don't display is wasted work. It also increases the risk of judder.
Your options are limited when it comes to getting callbacks that match the v-sync cadence. Your best is almost certainly a CVDisplayLink scheduled in the default and tracking run loop modes, though this has caveats.
You could use something like a counting semaphore in concert with a display link if you want to free-run without getting too far ahead.
If your application is able to maintain a real-time framerate, you'll normally be rendering a frame or two ahead of what's going on the glass, so you don't want to literally block on v-sync; you just want to inform the window server that you'd like presentation to match v-sync. On macOS, you do this by setting the layer's displaySyncEnabled to true (the default). Turning this off may cause tearing on certain displays.
At the point where you want to render to screen, you obtain the drawable from the layer by calling nextDrawable. You obtain the drawable's texture from its texture property. You use that texture to set up the render target (color attachment) of a MTLRenderPassDescriptor. For example:
id<CAMetalDrawable> drawable = layer.nextDrawable;
id<MTLTexture> texture = drawable.texture;
MTLRenderPassDescriptor *desc = [MTLRenderPassDescriptor renderPassDescriptor];
desc.colorAttachments[0].texture = texture;
From here, it's pretty similar to what you do in an MTKView's drawRect: method. You create a command buffer (if you don't already have one), create a render command encoder using the descriptor, encode drawing commands, end encoding, tell the command buffer to present the drawable (using a -presentDrawable:... method), and commit the command buffer. Whatever was drawn to the drawable's texture is what will end up on-screen when it's presented.
I agree with Warren that you probably don't really want to sync your loop with the display refresh. You want parallelism. You want the CPU to be working on the next frame while the GPU is rendering the most current frame (and the display is showing the last frame).
The fact that there's a limit on how many drawables may be in flight at once and that nextDrawable will block waiting for one will prevent your render loop from getting too far ahead. (You'll probably use some other synchronization before that, like for managing a small pool of buffers.) If you want only double-buffering and not triple-buffering, you can set the layer's maximumDrawableCount to 2 instead of its default value of 3.
I am trying to scale images/text etc using MM_ANISTROPIC and what I've done is the following (by the way if the syntax is a little weird, it's originally from delphi so treat the following as pseudocode)
I would expect the following code to produce a rectangle that is 70% of the width of the PaintBox and 30% of the height, yet it doesn't, instead it it noticeably too small.
SetMapMode(hdc,MM_ANISOTROPIC);
SetWindowExtEx(hdc,100,100,0);
SetViewportExtEx(hdc,70,30,0);
Rectangle(hdc, 0,0,PaintBox.width-1,PaintBox.Height-1);
if, on the other hand I change the code so that the SetWindowExtEx has 91 instead of 100 as its parameters (as shown below) then it works, which makes no sense to me at all...
SetMapMode(hdc,MM_ANISOTROPIC);
SetWindowExtEx(hdc,91,91,0);
SetViewportExtEx(hdc,70,30,0);
Rectangle(hdc, 0,0,PaintBox.width-1,PaintBox.Height-1);
My sanity test case was to add the following pseudocode
SetMapMode(hdc,MM_TEXT);
DrawLine(hdc,Round(PaintBox.width*0.7),0,Round(PaintBox.width*0.7),PaintBox.Height-1);
DrawLine(hdc,0,Round(PaintBox.height*0.3),PaintBox.width-1,Round(PaintBox.height*0.3));
I would have expected this to overwrite the lower / bottom edges of my original Rectangle but it does not unless I uses that 91,91 SetWindowExtEx.
Can anyone duplicate this?
FURTHER EDIT: Here is my exact original code I had given pseudo code before to make the question more accessible to non-delphi users but one of my commenters wanted full code to see if my contention that it was a delphi quirk was true or not.
The entire project consisted of a VCL form with a rectangular paintbox dropped on it centered so there was space all around it, and its onPaint event was set to the code below resulting in this image::
procedure TForm11.PaintBox2Paint(Sender: TObject);
var
hdc:THandle;
res:TPoint;
procedure SetupMapMode;
begin
SetMapMode(hdc,MM_ANISOTROPIC);
SetWindowExtEx(hdc,100,100,0);
SetViewportExtEx(hdc,70,30,0);
// These lines are required when we're painting to a TPaintBox but can't be used
// if we're paiting to a TPanel and they were NOT in my original question but only
// got added as part of the answer
// SetViewportOrgEx(hdc,PaintBox2.Left,PaintBox2.Top,#ZeroPoint);
// SetWindowOrgEx(hdc,0,0,#ZeroPoint);
end;
begin
//draw a rectangle to frame the Paintbox Surface
PaintBox2.Canvas.Pen.Style:=psSolid;
PaintBox2.Canvas.Pen.width:=2;
PaintBox2.Canvas.Pen.Color:=clGreen;
PaintBox2.Canvas.Brush.Style:=bsClear;
PaintBox2.Canvas.Rectangle(0,0,PaintBox2.Width-1,PaintBox2.Height-1);
PaintBox2.Canvas.Brush.Style:=bsSolid;
//initialize convenience variable
hdc:=PaintBox2.Canvas.Handle;
SetTextAlign(hdc,TA_LEFT);
//as doing things to the PaintBox2.Canvas via Delphi's interface tends to reset
//everything, I'm ensuring the map mode gets set **immediately** before
//each drawing call
SetupMapMode;
/// Draw Text at the bottom of the PaintBox2.Canvas (though as it's mapped it
/// SHOULD be 1/3 of the way down and much smaller instead)
TextOut(hdc,200,PaintBox2.Height-PaintBox2.Canvas.TextHeight('Ap'),'Hello, World!',13);
PaintBox2.Canvas.Pen.Color:=clblue;
PaintBox2.Canvas.Brush.Style:=bsClear;
//ensure it's set before doing the rectangle
SetupMapMode;
// Redraw the same rectangle as before but in the mapped mode
Rectangle(hdc, 0,0,PaintBox2.Width-1,PaintBox2.Height-1);
PaintBox2.Canvas.Brush.Style:=bsSolid;
//reset the map mode to normal
SetMapMode(hdc,MM_Text);
//draw text at the "same" position as before but unmapped...
TextOut(hdc,200,PaintBox2.Height-PaintBox2.Canvas.TextHeight('Ap'),'Goodbye, World!',15);
//Draw lines exactly at 70% of the way across and 30% of the way down
//if this works as expected they should overwrite the right and bottom
//borders of the rectangle drawn in the mapped mode
PaintBox2.Canvas.Pen.Color:=RGB(0,255,255);
PaintBox2.Canvas.MoveTo(Round(PaintBox2.Width*0.7),0);
PaintBox2.Canvas.LineTo(Round(PaintBox2.Width*0.7),PaintBox2.Height);
PaintBox2.Canvas.MoveTo(0,Round(PaintBox2.Height*0.3));
PaintBox2.Canvas.LineTo(PaintBox2.Width,Round(PaintBox2.Height*0.3));
end;
Okay, I don't know WHY the following is necessary -- it may be a Delphi quirk, the fact that I'm using a TPaintBox with is a TGraphicControl rather than a Component, or if I'm missing out on some fundamental concept on how this whole mapping mode works, BUT if I add the following code:
ZeroPoint:=TPoint.Zero;
SetViewportOrgEx(hdc,PaintBox1.Left,PaintBox1.Top,#ZeroPoint);
SetWindowOrgEx(hdc,0,0,#ZeroPoint);
Then it all displays as expected. Anyone have any explanations as to why this is necessary?
EDIT: Okay, I've got a PARTIAL explanation. It has to do with the control I was painting on being a TPaintBox, which is a TGraphic control rather than a TWinControl. To wit:
TGraphicControl is the base class for all lightweight controls.
TGraphicControl supports simple lightweight controls that do not need the ability to accept keyboard input or contain other controls. Since lightweight controls do not wrap Windows screen objects, they are faster and user fewer resources than controls based on TWinControl.
As such, although they APPEAR to have a separate canvas, I have this sneaking feeling that they are really sharing the form's canvas which is why, when I switched to a TWinControl descendant, which DOES own its own Windows DC, then the display worked as expected without setting the ViewpointOrg.
So it was a Delphi quirk after all...!
I am making a drawing on NSView using a timer that is set to update every .02 seconds. On update some physical simulation makes a step, and then Canvas!.needsDisplay = true. It works when app is in foreground (usually), but when some lags happen, simulation progresses forward despite the fact that view hasn't reflected it yet. How do I pause the timer during these times to make simulation happen only when NSView can show it? I do not want to call step_over from inside drawRect, cause it seems like a bad idea, because then it would be harder to stop the simulation.
Generally this kind of update should be done the other way around, by letting the display ask you for frames as it can display them. This is done with a CADisplayLink CVDisplayLink (this is Mac; CADisplayLink is iOS). Configure it with a method you want to be called when a frame can be drawn.
Generally you do want your simulation to keep moving forward, even if it means dropping frames occasionally. For that, you check the timestamp and use that to work out what time to use for your new frame. But if you only want to move forward when the display can show it, then just update once per call.
Note that generating at 50fps is often going to mismatch the system that's trying to draw at 60fps, so you're going to wind up missing frames occasionally. That's one of several reasons not to try to push drawing with a timer.
See also Alternative of CADisplayLink for Mac OS X. Note that trying to draw at 50fps with Core Graphics usually isn't going to give good results in any case. The right tool here in OS X is Core Animation (or SpriteKit for games on 10.10, or OpenGL for more advanced high-speed rendering). You can do very basic animations with an NSTimer (and we did for years before Core Animation came along), but it's not really a tool for complex drawing.
I would like to write a program that can mirror a portion of the main display into a new window. Ideally this new window could then be displayed on an external monitor. I have seen this uiltity for a flightsim that does this on a pc (a multifunction display extractor).
CLick here for a screenshot of the program (MFD Extractor)
This would be a live window ie. constantaly updated video display not just a static graphic.
I have looked at screen magnifiers or vnc clients for ideas but I think I need to write something from scratch. I have tried to do some reading on osx programing but where do I start in terms of gaining access to the display? I somehow need to extract the graphics from a particular program. Is it best to go near the final output stage (the individual pixels sent to the display) or somewhere nearer the window management stage.
Any ideas or pointers would be much appreciated. I just need somewhere to start from.
Regards,
There are a few ways to do this:
Quartz Display Services will let you get access to the video memory for a screen.
Quartz Window Services (a.k.a. CGWindow) will let you create an image of everything that lies below a window. If you create a borderless, transparent, empty, high-level window whose frame occupies an entire screen, everything below it will be everything on that screen. (Of course, you could create a smaller window in order to copy a section of the screen.)
There's also a way to do it using OpenGL that I never fully understood. That technique is demonstrated by a couple of code samples, OpenGLScreenSnapshot and OpenGLCaptureToMovie. It's more or less obsoleted by CGWindow, though.
Each of those will get you an image that you can then show or write to a file or something.
To show an image, use NSImageView or IKImageView. If you want to magnify it, IKImageView has a zoomFactor property, but if you want nearest-neighbor scaling (like Pixie, DigitalColor Meter, or xScope), I think you'll need to write a custom view for that (but even that isn't all that hard).
I'm writing a video annotation application with Qt4 in which users need to be able to seek to various points in a video, putting markers on various objects and then setting keypoints for those markers so that they stay on the objects in the video as they move around. QGraphicsItemAnimation seems like a great place to start for these markers, however they need to be able to appear and disappear at specific times, which I can't figure out how to do with the QGraphicsItemAnimation. I could set the scale at 0 to make the objects disappear, but that seems like a pretty hacky solution, and I'm guessing that the paint engine would still waste cpu cycles trying to draw those invisible objects. Does anyone have a better solution than this? I'm using Qt 4.5.3 right now, but I'm willing to upgrade to 4.6 if it makes things easier. Thanks!!
It seems like the functionality you want of showing/hiding QGraphicsItem objects is beyond the scope of the simple "tweening" that the animation class performs. It is only for one object at a time, and any appearance or disappearance you have to write yourself.
You still might get some mileage out of QGraphicsItemAnimation (although the fact that it uses its own timer instead of being locked to the frame clock of your video is a little dodgy).
Neglecting "seeking" for a moment, there is a QTimeLine::finished() signal. If you let the end of an annotation's active animation timeline represent the point where you want it to disappear, you can trigger QGraphicsItem::hide() at that point. When it comes time to turn it back on, you would construct a new QGraphicsItemAnimation (based on the next run of keyframe data for that object) and call QGraphicsItem::show().
Note that one of the headlining features of Qt 4.6 is the QtAnimation framework, which is more sophisticated but also rather complex. I've not used it yet, but looking over the examples it seems like you might be able to "animate" a visibility or opacity property.