Dealing with cursor masks - winapi

Question

I'm trying to get the data (byte[]) of the current mouse cursor, but one type of cursor (known as MaskedColor) needs special treatment.
http://elektronotdienst-nuernberg.de/bugs/cursor.html
It's the grab and grabbing cursors from this page.
I wonder if there's any way to combine somehow the mask with the color data.
The color data comes with alpha = 0, so there's no transparency (which is available in the mask).
The mask looks like this (32x32, 128 bytes):
And the color info is this (32x32, 4096 bytes, but with alpha=0):
Getting the cursor references:
var cursorInfo = new CursorInfo(false); //Create struct.
if (!User32.GetCursorInfo(out cursorInfo))
return;
if (cursorInfo.Flags != ScreenToGif.Native.Constants.CursorShowing)
{
Gdi32.DeleteObject(cursorInfo.CursorHandle);
return;
}
var iconHandle = User32.CopyIcon(cursorInfo.CursorHandle);
if (iconHandle == IntPtr.Zero)
{
Gdi32.DeleteObject(cursorInfo.CursorHandle);
return;
}
if (!User32.GetIconInfo(iconHandle, out var iconInfo))
{
User32.DestroyIcon(iconHandle);
Gdi32.DeleteObject(cursorInfo.CursorHandle);
return;
}
Getting the cursor color and mask buffers:
try
{
//Color.
var colorHeader = new BitmapInfoHeader(false);
Gdi32.GetDIBits(_windowDeviceContext, iconInfo.Color, 0, 0, null, ref colorHeader, DibColorModes.RgbColors);
//Mask.
var maskHeader = new BitmapInfoHeader(false);
Gdi32.GetDIBits(_windowDeviceContext, iconInfo.Mask, 0, 0, null, ref maskHeader, DibColorModes.RgbColors);
Whenever there's a color buffer, draw to a bitmap to get the cursor animation steps.
if (colorHeader.Height != 0)
{
//Create bitmap.
var compatibleBitmap = Gdi32.CreateCompatibleBitmap(_windowDeviceContext, colorHeader.Width, colorHeader.Height);
var oldBitmap = Gdi32.SelectObject(_compatibleDeviceContext, compatibleBitmap);
//Draw image.
var ok = User32.DrawIconEx(_compatibleDeviceContext, 0, 0, cursorInfo.CursorHandle, 0, 0, _cursorStep, IntPtr.Zero, DrawIconFlags.Image);
if (!ok)
{
_cursorStep = 0;
User32.DrawIconEx(_compatibleDeviceContext, 0, 0, cursorInfo.CursorHandle, 0, 0, _cursorStep, IntPtr.Zero, DrawIconFlags.Image);
}
else
_cursorStep++;
Here comes the tricky part, if there's a mask, I need to apply to that cursor color buffer to know which pixels are transparent or not.
I tried to call DrawIconEx() passing Normal which Mask + Color in the last parameter, but without any difference.
if (maskHeader.SizeImage > 0)
{
//User32.DrawIconEx(_compatibleDeviceContext, 0, 0, cursorInfo.CursorHandle, 0, 0, _cursorStep, IntPtr.Zero, DrawIconFlags.Mask);
var ok2 = Gdi32.MaskBlt(_compatibleDeviceContext, 0, 0, colorHeader.Width, colorHeader.Height, compatibleBitmap, 0, 0, iconInfo.Mask, 0, 0, (int)CopyPixelOperations.SourceErase);
}
The last part is to actually get the final cursor as a byte[].
colorHeader.Height *= -1;
var colorBuffer = new byte[colorHeader.SizeImage];
Gdi32.GetDIBits(_windowDeviceContext, compatibleBitmap, 0, (uint)(colorHeader.Height * -1), colorBuffer, ref colorHeader, DibColorModes.RgbColors);
//Send colorBuffer
//Erase bitmaps.
Gdi32.SelectObject(_compatibleDeviceContext, oldBitmap);
Gdi32.DeleteObject(compatibleBitmap);
return;
}
For the mask monochrome cursor, just get the mask itself as byte[].
var maskBuffer = new byte[maskHeader.SizeImage];
maskHeader.Height *= -1;
Gdi32.GetDIBits(_windowDeviceContext, iconInfo.Mask, 0, (uint)(maskHeader.Height * -1), maskBuffer, ref maskHeader, DibColorModes.RgbColors);
//Send maskBuffer
}
finally
{
Gdi32.DeleteObject(iconInfo.Color);
Gdi32.DeleteObject(iconInfo.Mask);
User32.DestroyIcon(iconHandle);
Gdi32.DeleteObject(cursorInfo.CursorHandle);
}
Edit
Well, I can merge the mask buffer with the color buffer manually, but the issue is that I don't have a straightforward way to detect if the cursor needs the mask or not.
It works for the masked-color cursors:
But it butchers normal cursors (semi-transparent pixels turn opaque):
Both have a mask and color data, but the only difference is that the masked cursor will have its color buffer with all alpha = 0.
var colorHeight = colorHeader.Height * -1;
var colorWidth = colorHeader.Width; //Bug: For some reason, after calling GetDIBits() for the mask, the width of the color struct shifts.
var maskBuffer2 = new byte[maskHeader.SizeImage];
maskHeader.Height *= -1;
Gdi32.GetDIBits(_windowDeviceContext, iconInfo.Mask, 0, (uint)(maskHeader.Height * -1), maskBuffer2, ref maskHeader, DibColorModes.RgbColors);
var targetPitch = colorBuffer.Length / colorHeight;
var cursorPitch = maskBuffer2.Length / maskHeader.Height * -1;
//Merge mask with color.
for (var row = 0; row < colorWidth; row++)
{
//128 in binary.
byte mask = 0x80;
for (var col = 0; col < colorHeight; col++)
{
//Reads current pixel and merge with mask.
//Each mask byte holds information for 8 pixels.
var targetIndex = row * targetPitch + col * 4;
var xor = (maskBuffer2[row * cursorPitch + col / 8] & mask) == mask;
colorBuffer[targetIndex + 3] = (byte) (xor ? 255 : 0);
//Shifts the mask around until it reaches 1, then resets it back to 128.
if (mask == 0x01)
mask = 0x80;
else
mask = (byte)(mask >> 1);
}
}
The non-elegant way to know for sure that I need the mask is to check all alphas in the cursor buffer, if all zero, then I need the mask.
var needsMask = true;
for (var index = 0; index < colorBuffer.Length; index += 4)
{
if (colorBuffer[index] == 0) continue;
//If there's any non-zero alpha value, it means that the mask is not necessary.
needsMask = false;
break;
}
if (!needsMask)
{
//Send colorBuffer.
return;
}
//Send colorBuffer + maskBuffer.
At least it gives me the correct result: