Incorrect drawing with overlapping siblings and WM_EX_TRANSPARENT - winapi

I have an issue where overlapping child windows are not being painted correctly when one window is updated below another window that has the WS_EX_TRANSPARENT style.
The parent window has two children, W1 and W2, that overlap with W1 being above W2 in the z-order. W1 has the extended style WS_EX_TRANSPARENT. The issue is that when W2 paints in its client area (I'm testing this by invalidating W2), the W2 entirely paints over the overlapping portion of W1. If the parent is redrawn then both overlapping children are drawn in the correct order.
The child windows are drawn correctly when WS_CLIPSIBLINGS is set on W2 and WS_EX_TRANSPARENT is removed from W1. If W2's client area is invalidated in this situation, it paints but does not draw over the portion obscured by W1.
WS_EX_TRANSPARENT is being used because the first child window is being used as an overlay for the second child window, and I would like to avoid having mouse clicks blocked by the overlay.
Is there a way to have overlapping windows draw correctly with the WS_EX_TRANSPARENT style or can I get the same behavior using some sort of workaround?

WS_EX_TRANSPARENT is flaky. The KB article has a cop-out, "we don't actually support transparent windows". But does nothing to explain the operation, nor does the SDK, nor does Raymond Chen. I've been bitten once too many so just don't use it anymore.
But I think the rule is that it will take care of getting the overlayed window to paint first when the upper window needs painting to ensure that the background pixels are set. So invalidating W1 will invalidate W2 as well and then the normal Z-order painting takes care of things. Ignoring the inevitable flicker problem. But not the other way around, invalidating the overlayed window will not force a repaint for the upper one. Bit of an hang-up in your case.
Regardless of my guessing, a fix in your case that ought to work is to invalidate W1 as well when you invalidate W2. Or just invalidate W1 when it completely overlaps W2. Hope that works, good luck with it.

Related

DirectX Resize shows win32 background at Edges

When resizing down in DirectX I noticed a flicker along the bottom/right edges.
If you fix the size of the swapchain (in red), and set the win32 background to green you will get the following on a resize down:
It appears that the window size lags behind the mouse position ("Drag Rectangle") but the area DirectX fills (in red and black) matches the Drag Rectangle, with the remainder to be painted with the window background color (green).
The DirectX Utility Toolkit (DXUT) didn't have this problem. Experimenting with their settings I found that setting DXGI_SWAP_CHAIN_DESC::SwapEffect = DXGI_SWAP_EFFECT_DISCARD would just paint the screen green (the window background color) during a sizemove. I assume that DXUT has their own special handling for when win32 enters a modal sizemove loop in order to display DirectX content during a sizemove.
DXGI_SWAP_EFFECT_DISCARD is part of the old blit mode presentation model. Notably it cant have DXGI_SWAP_CHAIN_DESC1::Scaling = DXGI_SCALING_NONE the scaling must be stretched to the window size. So my best guess is this behavior is due to the implementation of the flip presentation model (it flips a rectangle too small (black) and then covers by painting the rest green).
Does anyone know how to stop the win32 background from showing?
Edit:
Thanks to IInspectable! I can confirm the WS_EX_NOREDIRECTIONBITMAP extended style works: it stops the artifact from the win32 background showing
I reproduced the exact same behavior when using CreateSwapChainForComposition() with DirectComposition as I got using CreateSwapChainForHwnd().
This means that there are two path for pixels to get to the screen. The green pixels are going through the Redirection Surface. Hence, explicitly requesting WS_EX_NOREDIRECTIONBITMAP prevents any drawing through a Redirection Surface stopping the win32 background from showing.
The other path is the flip presentation behavior which is how the red and black pixels are get displayed. Hence, not using flip presentation when resizing also stops the win32 background from being shown.
There must be a bug with DWM: when there is a Redirection Surface, the clipping to prevent the swapchain's contents extending past the window is smaller than the redirection surface allowing it to be seen along the bottom/right edges.
There are two observations of interest:
when using WS_EX_LAYOUTRTL or manually positioning a IDCompositionVisual to the right edge of the screen with GetClientRect(), the swapchains contents are correctly position, but still clipped.
when using WS_EX_NOREDIRECTIONBITMAP the window`s nonclient area lines up with the contents of the swapchain rather than the swapchains contents being clipped
These observations would seam to imply that the cause of the issue is that DWM is using it's prefered size for the window size sometimes and the size of the redirection surface othertimes.

Windows GDI - FillRect vs MoveToEx & LineTo?

When reading source code which draws lines in Windows using GDI it is relatively common to see FillRect() being used despite the only purpose being to draw a line. But the end product of drawing a line with a width value and a filled rectangle are quite similar aren't they?
FillRect() is 1 function call, Using MoveToEx & LineTo requires 2
Which is more efficient when needing to draw a line, using FillRect() or MoveToEx() and LineTo()?
In the most common cases, FillRect will do the same thing as MoveToEx and LineTo for perfectly horizontal and vertical rectangles. Nowadays, there are so many layers of indirection between GDI and the screen that the performance difference is almost certainly not relevant.
Drawing operations in GDI typically depend on the current "state" of the device context (DC). Lines are drawn with whichever pen is currently selected into the DC. The pen determines the color, style (solid, dashed, etc.), thickness, end caps, etc.
FillRect, however, doesn't depend on much of the DC state. All drawing primitives depend on the mapping mode and clipping region, but, unlike lines, FillRect doesn't even depend on the selected brush, since you get to provide one right in the call.
Changing state (which objects are selected into the DC) can be a lot of work. If you know you want a horizontal line, 2 pixels thick, in blue, it's a tad easier to use FillRect than to first create a pen, select it into the DC, draw your line, select the pen back out, and then decide how to manage the lifetime of that pen (when do you delete it?). If the rest of the drawing is a bunch of dashed yellow lines with round endcaps, not having to keep switching state can make the code simpler.

Changing size of non-client area in NCCALCSIZE when window is maximized causes flickering when maximizing with Aero Snap only

I'm trying to make a borderless window where the maximized window properly fits the work area of the screen when maximized. This has turned out to be more difficult than I could've ever imagined, and every solution I can find has some sort of downside or glitch that prevents it from being perfect. Some of the glitches are very minor and probably wouldn't be a big deal to most people, but it bothers me.
I know it's possible to do this somehow because I've used a few programs that clearly draw their own non-client area but have proper maximize behavior and appear to be without glitches. Steam and Twitch clients are two examples. However, I have no idea how much code or what sort of tricks they used to get their windows to have that behavior.
Background Info:
When maximizing a window, the window manager positions the window in such a way that the non-client area falls just outside of the visible work area so that the non-client area is not visible when the window is maximized. I remove the non-client area by returning 0 from WM_NCCALCSIZE because I want to be able to draw my own non-client area. The result is that the window manager is trying to account for a non-client area that doesn't exist and my maximized borderless window extends off the screen approximately 7 or 8 pixels in all directions.
What I've tried to fix this:
I've tried handling WM_MINMAXINFO, which doesn't work because the window manager ignores any dimensions passed unless they are smaller than the work area or monitor area. For example, if I subtract 1 from the ptMaxSize.x coordinate, the window fits almost perfectly except there's a 1 pixel transparent space where you can click whatever is behind the window. See this post for more info.
What almost works:
I found this post which contains a solution that almost works perfectly, except that when maximizing the window with Aero Snap, there's a very brief flicker where you can see everything behind the window. This doesn't occur when maximizing the window by double clicking the title bar or using the maximize buttons, even when adjusting the size of the non-client area on maximize. Here's a video that demonstrates the issue.
Video explanation:
First, I maximize the window several times with Aero Snap without the adjustments being made to the non-client area to demonstrate that the maximize transition is flicker-free. Then I show that the window flickers after having adjusted the size of the non-client area on maximizing.
My Question:
Is there a way to prevent the flicker from happening when using this method to fit the maximized borderless window to the work area? Or is there a better way to go about making the maximized window fit the screen? Handling WM_MINMAXINFO seemed like the best hack-free approach, but because of the quirky way windows responds to dimensions that are greater than or equal to the work area, it doesn't seem to work.

Custom shaped NSButton

I am trying to create eight custom buttons (NSButton) in Xcode 4.6.3. Those are the segments of a circle. I used a standard rectangular button for each of them, adding a custom image for each segment. However, when I put the pieces together in one circle, there is no way to click some of these buttons, as the rectangular areas around each of them overlap, and prevent from reaching the other half of the buttons.
I was wondering if there is any way to make the button shape at least triangular, such that I can click on all of these buttons?
From the documentation "View Programming Guide":
Note: For performance reasons, Cocoa does not enforce clipping among sibling views or guarantee correct invalidation and drawing behavior when sibling views overlap. If you want a view to be drawn in front of another view, you should make the front view a subview (or descendant) of the rear view.
In other words, you can't expect overlapping views to process mouse events properly. There's no way of getting around the fact that views occupy rectangular frames. You have to make a single view which performs the work of all of your circle segments (including drawing and event handling, and optionally mouse moved events). YOu will have to use trigonometry to calculate which segment a mouse click occurs in, and respond appropriately as though a button were pressed, by re-drawing the segment and invoking the desired action.

How to get the size in pixels of the Resize Corners of a Window

Is there a way (API) of getting the size (vertical and horizontal) in pixels of the resize corners?
I am referring to the area at each of the corners of a window where you can resize the window in both directions (Left-to-Right and Top-to-Bottom) at the same time using your mouse. You will know you are there with your mouse cursor when you hover over the corners of the window and the mouse cursor is a Diagonal Resizing cursor.
Thank you
Edit:
An example: Hover your mouse over the right edge of a sizable window. Start in the middle (vertically) of the window and move the mouse up along the edge until the horizontal sizing cursor changes to a diagonal sizing cursor. How do I determine by asking the OS how far that position when the cursor changes, is from the top of the window.
I would suggest to use the size of the scrollbars. Call GetSystemMetrics with SM_CYHSCROLL and SM_CXVSCROLL. May be also SM_CYSIZEFRAME and SM_CXSIZEFRAME sizes can be combined.
But I think a better value is to use the height of the status bar. However even Microsoft Windows seems to use some fixed value as can seen on the screenshot.
Comparing the results of GetClientRect and GetWindowRect will tell you how wide the non-client (border) area is along each edge of the window.
If you're concerned that it might not all be active for sizing (true especially along the top), or you want to distinguish the diagonal sizing areas from edge sizing areas, you can take the coordinates discovered in step 1 and pass them to SendMessage(WM_NCHITTEST) See its documentation for the various return codes. There's no problem sending this message repeatedly -- it's designed to be called for each mouse move event and therefore is very fast.

Resources