Related
I'm making progress developing a '3d desktop' directx app that needs to display the current contents of a desktop window (e.g. "Calculator") as a 2D texture on a rectangular surface in directx (11). I'm sooo close but really struggling with the screenshot BMP -> Texture2D step. I do have screenshot->HBITMAP and DDSFile->rendered texture successfully working but can't complete the screenshot->rendered texture.
So far I have working the 'capture the window as a screenshot' bit:
RECT user_window_rectangle;
HWND user_window = FindWindow(NULL, TEXT("Calculator"));
GetClientRect(user_window, &user_window_rectangle);
HDC hdcScreen = GetDC(NULL);
HDC hdc = CreateCompatibleDC(hdcScreen);
UINT screenshot_width = user_window_rectangle.right - user_window_rectangle.left;
UINT screenshot_height = user_window_rectangle.bottom - user_window_rectangle.top;
hbmp = CreateCompatibleBitmap(hdcScreen, screenshot_width, screenshot_height);
SelectObject(hdc, hbmp);
PrintWindow(user_window, hdc, PW_CLIENTONLY);
At this point I have the window bitmap referenced by HBITMAP hbmp.
Also working is my code to render a DDS file as a texture on a directx/3d rectangle:
ID3D11Device *dev;
ID3D11DeviceContext *dev_context;
...
dev_context->PSSetShaderResources(0, 1, &shader_resource_view);
dev_context->PSSetSamplers(0, 1, &tex_sampler_state);
...
DirectX::TexMetadata tex_metadata;
DirectX::ScratchImage image;
hr = LoadFromDDSFile(L"Earth.dds", DirectX::DDS_FLAGS_NONE, &tex_metadata, image);
hr = CreateShaderResourceView(dev, image.GetImages(), image.GetImageCount(), tex_metadata, &shader_resource_view);
Pixel shader is:
Texture2D ObjTexture
SamplerState ObjSamplerState
float4 PShader(float4 pos : SV_POSITION, float4 color : COLOR, float2 tex : TEXCOORD) : SV_TARGET\
{
return ObjTexture.Sample( ObjSamplerState, tex );
}
The samplerstate (defaulting to linear) is:
D3D11_SAMPLER_DESC sampler_desc;
ZeroMemory(&sampler_desc, sizeof(sampler_desc));
sampler_desc.AddressU = D3D11_TEXTURE_ADDRESS_WRAP;
sampler_desc.AddressV = D3D11_TEXTURE_ADDRESS_WRAP;
sampler_desc.AddressW = D3D11_TEXTURE_ADDRESS_WRAP;
sampler_desc.MinLOD = 0;
sampler_desc.MaxLOD = D3D11_FLOAT32_MAX;
hr = dev->CreateSamplerState(&sampler_desc, &tex_sampler_state);
Question: how do I replace the LoadFromDDSFile bit with some equivalent that takes the HBITMAP from the windows screencapture and ends up with it on the graphics card as ObjTexture ?
Below is my best shot of bridging from the screenshot HBITMAP hbmp to the shader resource screenshot_texture, but it gives a memory access violation from the graphics driver (I think due to my "data.pSysmem = &bmp.bmBits", but no idea really):
GetObject(hbmp, sizeof(BITMAP), (LPSTR)&bmp)
D3D11_TEXTURE2D_DESC screenshot_desc = CD3D11_TEXTURE2D_DESC(DXGI_FORMAT_R8G8B8A8_UNORM, bmp.bmWidth, bmp.bmHeight, 1,
1,
D3D11_BIND_SHADER_RESOURCE
);
int bytes_per_pixel = 4;
D3D11_SUBRESOURCE_DATA data;
ZeroMemory(&data, sizeof(D3D11_SUBRESOURCE_DATA));
data.pSysMem = &bmp.bmBits; //pixel buffer
data.SysMemPitch = bytes_per_pixel * bmp.bmWidth;// line size in byte
data.SysMemSlicePitch = bytes_per_pixel * bmp.bmWidth * bmp.bmHeight;// total buffer size in byte
hr = dev->CreateTexture2D(
&screenshot_desc, //texture format
&data, // pixel buffer use to fill the texture
&screenshot_texture // created texture
);
:::::::::::::::::::::::::SOLUTION::::::::::::::::::::::::::::::::::::::::::
The main issue was trying to use &bmp.bmBits directly as a pixel buffer caused memory conflicts within the graphics driver - this was resolved by using 'malloc' to allocate an appropriately sized block of memory to store the pixel data. Thanks to Chuck Walbourn for helping with my poking around in the dark to work out how the pixel data is actually stored (it was actually 32 bits/pixel by default). It's still possible/likely some of code is relying on luck to read the pixel data correctly, but it's been improved with Chuck's input.
My basic technique was;
FindWindow to get the client window on the desktop
CreateCompatibleBitmap and SelectObject and PrintWindow to get a HBITMAP to the snapshot
malloc to allocate the correct amount of space for a (byte*)pixel buffer
GetDIBits to populate the (byte*)pixel buffer from the HBITMAP
CreateTexture2D to build the texture buffer
CreateShaderResourceView to map the texture to the graphics pixel shader
So working code to screenshot a windows desktop window and pass that as a texture to a direct3d app is:
RECT user_window_rectangle;
HWND user_window = FindWindow(NULL, TEXT("Calculator")); //the window can't be min
if (user_window == NULL)
{
MessageBoxA(NULL, "Can't find Calculator", "Camvas", MB_OK);
return;
}
GetClientRect(user_window, &user_window_rectangle);
//create
HDC hdcScreen = GetDC(NULL);
HDC hdc = CreateCompatibleDC(hdcScreen);
UINT screenshot_width = user_window_rectangle.right - user_window_rectangle.left;
UINT screenshot_height = user_window_rectangle.bottom - user_window_rectangle.top;
hbmp = CreateCompatibleBitmap(hdcScreen, screenshot_width, screenshot_height);
SelectObject(hdc, hbmp);
//Print to memory hdc
PrintWindow(user_window, hdc, PW_CLIENTONLY);
BITMAPINFOHEADER bmih;
ZeroMemory(&bmih, sizeof(BITMAPINFOHEADER));
bmih.biSize = sizeof(BITMAPINFOHEADER);
bmih.biPlanes = 1;
bmih.biBitCount = 32;
bmih.biWidth = screenshot_width;
bmih.biHeight = 0-screenshot_height;
bmih.biCompression = BI_RGB;
bmih.biSizeImage = 0;
int bytes_per_pixel = bmih.biBitCount / 8;
BYTE *pixels = (BYTE*)malloc(bytes_per_pixel * screenshot_width * screenshot_height);
BITMAPINFO bmi = { 0 };
bmi.bmiHeader = bmih;
int row_count = GetDIBits(hdc, hbmp, 0, screenshot_height, pixels, &bmi, DIB_RGB_COLORS);
D3D11_TEXTURE2D_DESC screenshot_desc = CD3D11_TEXTURE2D_DESC(
DXGI_FORMAT_B8G8R8A8_UNORM, // format
screenshot_width, // width
screenshot_height, // height
1, // arraySize
1, // mipLevels
D3D11_BIND_SHADER_RESOURCE, // bindFlags
D3D11_USAGE_DYNAMIC, // usage
D3D11_CPU_ACCESS_WRITE, // cpuaccessFlags
1, // sampleCount
0, // sampleQuality
0 // miscFlags
);
D3D11_SUBRESOURCE_DATA data;
ZeroMemory(&data, sizeof(D3D11_SUBRESOURCE_DATA));
data.pSysMem = pixels; // texArray; // &bmp.bmBits; //pixel buffer
data.SysMemPitch = bytes_per_pixel * screenshot_width;// line size in byte
data.SysMemSlicePitch = bytes_per_pixel * screenshot_width * screenshot_height;
hr = dev->CreateTexture2D(
&screenshot_desc, //texture format
&data, // pixel buffer use to fill the texture
&screenshot_texture // created texture
);
D3D11_SHADER_RESOURCE_VIEW_DESC srvDesc;
srvDesc.Format = screenshot_desc.Format;
srvDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
srvDesc.Texture2D.MostDetailedMip = 0;
srvDesc.Texture2D.MostDetailedMip = screenshot_desc.MipLevels;
dev->CreateShaderResourceView(screenshot_texture, NULL, &shader_resource_view);
You are making a lot of assumptions here that the BITMAP returned is actually in 32-bit RGBA form. It is likely not at all in that format, and in any case you need to validate the contents of bmPlanes to be 1 and bmBitsPixel to be 32 if you are assuming it is 4-bytes per pixel. You should read more about the BMP format.
BMPs uses BGRA order, so you can use DXGI_FORMAT_B8G8R8A8_UNORM for the case of bmBitsPixel being 32.
Secondly, you need to derive pitch from bmWidthBytes and not bmWidth.
data.pSysMem = &bmp.bmBits; //pixel buffer
data.SysMemPitch = bmp.bmWidthBytes;// line size in byte
data.SysMemSlicePitch = bmp.bmWidthBytes * bmp.bmHeight;// total buffer size in byte
If bmBitsPixel is 24, there is no DXGI format equivalent to that. You have to copy the data to a 32-bit format such as DXGI_FORMAT_B8G8R8X8_UNORM.
If bmBitsPixel is 15 or 16, you can use DXGI_FORMAT_B5G5R5A1_UNORM on a system with Direct3D 11.1, but remember that 16-bit DXGI formats are not always supported depending on the driver. Otherwise you'll have to convert this data to something else.
For bmBitsPixel values of 1, 2, 4, or 8 you have to convert them as there are no DXGI texture formats that are equivalent.
The main issue was trying to use &bmp.bmBits directly as a pixel buffer caused memory conflicts within the graphics driver - this was resolved by using 'malloc' to allocate an appropriately sized block of memory to store the pixel data. Thanks to Chuck Walbourn for helping with my poking around in the dark to work out how the pixel data is actually stored (it was actually 32 bits/pixel by default). It's still possible/likely some of code is relying on luck to read the pixel data correctly, but it's been improved with Chuck's input.
My basic technique was;
FindWindow to get the client window on the desktop
CreateCompatibleBitmap and SelectObject and PrintWindow to get a HBITMAP to the snapshot
malloc to allocate the correct amount of space for a (byte*)pixel buffer
GetDIBits to populate the (byte*)pixel buffer from the HBITMAP
CreateTexture2D to build the texture buffer
CreateShaderResourceView to map the texture to the graphics pixel shader
So working code to screenshot a windows desktop window and pass that as a texture to a direct3d app is:
RECT user_window_rectangle;
HWND user_window = FindWindow(NULL, TEXT("Calculator")); //the window can't be min
if (user_window == NULL)
{
MessageBoxA(NULL, "Can't find Calculator", "Camvas", MB_OK);
return;
}
GetClientRect(user_window, &user_window_rectangle);
//create
HDC hdcScreen = GetDC(NULL);
HDC hdc = CreateCompatibleDC(hdcScreen);
UINT screenshot_width = user_window_rectangle.right - user_window_rectangle.left;
UINT screenshot_height = user_window_rectangle.bottom - user_window_rectangle.top;
hbmp = CreateCompatibleBitmap(hdcScreen, screenshot_width, screenshot_height);
SelectObject(hdc, hbmp);
//Print to memory hdc
PrintWindow(user_window, hdc, PW_CLIENTONLY);
BITMAPINFOHEADER bmih;
ZeroMemory(&bmih, sizeof(BITMAPINFOHEADER));
bmih.biSize = sizeof(BITMAPINFOHEADER);
bmih.biPlanes = 1;
bmih.biBitCount = 32;
bmih.biWidth = screenshot_width;
bmih.biHeight = 0-screenshot_height;
bmih.biCompression = BI_RGB;
bmih.biSizeImage = 0;
int bytes_per_pixel = bmih.biBitCount / 8;
BYTE *pixels = (BYTE*)malloc(bytes_per_pixel * screenshot_width * screenshot_height);
BITMAPINFO bmi = { 0 };
bmi.bmiHeader = bmih;
int row_count = GetDIBits(hdc, hbmp, 0, screenshot_height, pixels, &bmi, DIB_RGB_COLORS);
D3D11_TEXTURE2D_DESC screenshot_desc = CD3D11_TEXTURE2D_DESC(
DXGI_FORMAT_B8G8R8A8_UNORM, // format
screenshot_width, // width
screenshot_height, // height
1, // arraySize
1, // mipLevels
D3D11_BIND_SHADER_RESOURCE, // bindFlags
D3D11_USAGE_DYNAMIC, // usage
D3D11_CPU_ACCESS_WRITE, // cpuaccessFlags
1, // sampleCount
0, // sampleQuality
0 // miscFlags
);
D3D11_SUBRESOURCE_DATA data;
ZeroMemory(&data, sizeof(D3D11_SUBRESOURCE_DATA));
data.pSysMem = pixels; // texArray; // &bmp.bmBits; //pixel buffer
data.SysMemPitch = bytes_per_pixel * screenshot_width;// line size in byte
data.SysMemSlicePitch = bytes_per_pixel * screenshot_width * screenshot_height;
hr = dev->CreateTexture2D(
&screenshot_desc, //texture format
&data, // pixel buffer use to fill the texture
&screenshot_texture // created texture
);
D3D11_SHADER_RESOURCE_VIEW_DESC srvDesc;
srvDesc.Format = screenshot_desc.Format;
srvDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
srvDesc.Texture2D.MostDetailedMip = 0;
srvDesc.Texture2D.MostDetailedMip = screenshot_desc.MipLevels;
dev->CreateShaderResourceView(screenshot_texture, NULL, &shader_resource_view);
I want to resize a screen captured using the Desktop Duplication API in SharpDX. I am using the Screen Capture sample code from the SharpDX Samples repository, relevant portion follows:.
SharpDX.DXGI.Resource screenResource;
OutputDuplicateFrameInformation duplicateFrameInformation;
// Try to get duplicated frame within given time
duplicatedOutput.AcquireNextFrame(10000, out duplicateFrameInformation, out screenResource);
if (i > 0)
{
// copy resource into memory that can be accessed by the CPU
using (var screenTexture2D = screenResource.QueryInterface<Texture2D>())
device.ImmediateContext.CopyResource(screenTexture2D, screenTexture);
// Get the desktop capture texture
var mapSource = device.ImmediateContext.MapSubresource(screenTexture, 0, MapMode.Read, MapFlags.None);
System.Diagnostics.Debug.WriteLine(watch.Elapsed);
// Create Drawing.Bitmap
var bitmap = new System.Drawing.Bitmap(width, height, PixelFormat.Format32bppArgb);
var boundsRect = new System.Drawing.Rectangle(0, 0, width, height);
// Copy pixels from screen capture Texture to GDI bitmap
var mapDest = bitmap.LockBits(boundsRect, ImageLockMode.WriteOnly, bitmap.PixelFormat);
var sourcePtr = mapSource.DataPointer;
var destPtr = mapDest.Scan0;
for (int y = 0; y < height; y++)
{
// Iterate and write to bitmap...
I would like to resize the image much smaller than the actual screen size before processing it as a byte array. I do not need to save the image, just get at the bytes. I would like to do this relatively quickly and efficiently (e.g. leveraging GPU if possible).
I'm not able to scale during CopyResource, as the output dimensions are required to be the same as the input dimensions. Can I perform another copy from my screenTexture2D to scale? How exactly do I scale the resource - do I use a Swap Chain, Matrix transform, or something else?
If you are fine resizing to a power of two from the screen, you can do it by:
Create a smaller texture with RenderTarget/ShaderResource usage, and options GenerateMipMaps, same size of screen, mipcount > 1 (2 for having size /2, 3 for having /4...etc.).
Copy the first mipmap of the screen texture to the smaller texture
DeviceContext.GenerateMipMaps on the smaller texture
Copy the selected mimap of the smaller texture (1: /2, 2: /4...etc.) to the staging texture (that should also be declared smaller, i.e. same size as the mipmap that is going to be used)
A quick hack on the original code to generate a /2 texture would be like this:
[STAThread]
private static void Main()
{
// # of graphics card adapter
const int numAdapter = 0;
// # of output device (i.e. monitor)
const int numOutput = 0;
const string outputFileName = "ScreenCapture.bmp";
// Create DXGI Factory1
var factory = new Factory1();
var adapter = factory.GetAdapter1(numAdapter);
// Create device from Adapter
var device = new Device(adapter);
// Get DXGI.Output
var output = adapter.GetOutput(numOutput);
var output1 = output.QueryInterface<Output1>();
// Width/Height of desktop to capture
int width = output.Description.DesktopBounds.Width;
int height = output.Description.DesktopBounds.Height;
// Create Staging texture CPU-accessible
var textureDesc = new Texture2DDescription
{
CpuAccessFlags = CpuAccessFlags.Read,
BindFlags = BindFlags.None,
Format = Format.B8G8R8A8_UNorm,
Width = width/2,
Height = height/2,
OptionFlags = ResourceOptionFlags.None,
MipLevels = 1,
ArraySize = 1,
SampleDescription = { Count = 1, Quality = 0 },
Usage = ResourceUsage.Staging
};
var stagingTexture = new Texture2D(device, textureDesc);
// Create Staging texture CPU-accessible
var smallerTextureDesc = new Texture2DDescription
{
CpuAccessFlags = CpuAccessFlags.None,
BindFlags = BindFlags.RenderTarget | BindFlags.ShaderResource,
Format = Format.B8G8R8A8_UNorm,
Width = width,
Height = height,
OptionFlags = ResourceOptionFlags.GenerateMipMaps,
MipLevels = 4,
ArraySize = 1,
SampleDescription = { Count = 1, Quality = 0 },
Usage = ResourceUsage.Default
};
var smallerTexture = new Texture2D(device, smallerTextureDesc);
var smallerTextureView = new ShaderResourceView(device, smallerTexture);
// Duplicate the output
var duplicatedOutput = output1.DuplicateOutput(device);
bool captureDone = false;
for (int i = 0; !captureDone; i++)
{
try
{
SharpDX.DXGI.Resource screenResource;
OutputDuplicateFrameInformation duplicateFrameInformation;
// Try to get duplicated frame within given time
duplicatedOutput.AcquireNextFrame(10000, out duplicateFrameInformation, out screenResource);
if (i > 0)
{
// copy resource into memory that can be accessed by the CPU
using (var screenTexture2D = screenResource.QueryInterface<Texture2D>())
device.ImmediateContext.CopySubresourceRegion(screenTexture2D, 0, null, smallerTexture, 0);
// Generates the mipmap of the screen
device.ImmediateContext.GenerateMips(smallerTextureView);
// Copy the mipmap 1 of smallerTexture (size/2) to the staging texture
device.ImmediateContext.CopySubresourceRegion(smallerTexture, 1, null, stagingTexture, 0);
// Get the desktop capture texture
var mapSource = device.ImmediateContext.MapSubresource(stagingTexture, 0, MapMode.Read, MapFlags.None);
// Create Drawing.Bitmap
var bitmap = new System.Drawing.Bitmap(width/2, height/2, PixelFormat.Format32bppArgb);
var boundsRect = new System.Drawing.Rectangle(0, 0, width/2, height/2);
// Copy pixels from screen capture Texture to GDI bitmap
var mapDest = bitmap.LockBits(boundsRect, ImageLockMode.WriteOnly, bitmap.PixelFormat);
var sourcePtr = mapSource.DataPointer;
var destPtr = mapDest.Scan0;
for (int y = 0; y < height/2; y++)
{
// Copy a single line
Utilities.CopyMemory(destPtr, sourcePtr, width/2 * 4);
// Advance pointers
sourcePtr = IntPtr.Add(sourcePtr, mapSource.RowPitch);
destPtr = IntPtr.Add(destPtr, mapDest.Stride);
}
// Release source and dest locks
bitmap.UnlockBits(mapDest);
device.ImmediateContext.UnmapSubresource(stagingTexture, 0);
// Save the output
bitmap.Save(outputFileName);
// Capture done
captureDone = true;
}
screenResource.Dispose();
duplicatedOutput.ReleaseFrame();
}
catch (SharpDXException e)
{
if (e.ResultCode.Code != SharpDX.DXGI.ResultCode.WaitTimeout.Result.Code)
{
throw e;
}
}
}
// Display the texture using system associated viewer
System.Diagnostics.Process.Start(Path.GetFullPath(Path.Combine(Environment.CurrentDirectory, outputFileName)));
// TODO: We should cleanp up all allocated COM objects here
}
You need to take your original source surface in GPU memory and Draw() it on to a smaller surface. This involves simple vector/pixel shaders, which some folks with simple needs would rather bypass.
I would look to see if someone made a sprite lib for sharpdx. It should be a common "thing"...or using Direct2D (which is much more fun). Since D2D is just a user-mode library over D3D, it interops with D3D very easily.
I've never used SharpDx, but fFrom memory you would do something like this:
1.) Create an ID2D1Device, wrapping your existing DXGI Device (make sure your dxgi device creation flag has D3D11_CREATE_DEVICE_BGRA_SUPPORT)
2.) Get the ID2D1DeviceContext from your ID2D1Device
3.) Wrap your source and destination DXGI surfaces into D2D bitmaps with ID2D1DeviceContext::CreateBitmapFromDxgiSurface
4.) ID2D1DeviceContext::SetTarget of your destination surface
5.) BeginDraw, ID2D1DeviceContext::DrawBitmap, passing your source D2D bitmap. EndDraw
6.) Save your destination
Here is a pixelate example...
d2d_device_context_h()->BeginDraw();
d2d_device_context_h()->SetTarget(mp_ppBitmap1.Get());
D2D1_SIZE_F rtSize = mp_ppBitmap1->GetSize();
rtSize.height *= (1.0f / cbpx.iPixelsize.y);
rtSize.width *= (1.0f / cbpx.iPixelsize.x);
D2D1_RECT_F rtRect = { 0.0f, 0.0f, rtSize.width, rtSize.height };
D2D1_SIZE_F rsSize = mp_ppBitmap0->GetSize();
D2D1_RECT_F rsRect = { 0.0f, 0.0f, rsSize.width, rsSize.height };
d2d_device_context_h()->DrawBitmap(mp_ppBitmap0.Get(), &rtRect, 1.0f,
D2D1_BITMAP_INTERPOLATION_MODE_LINEAR, &rsRect);
d2d_device_context_h()->SetTarget(mp_ppBitmap0.Get());
d2d_device_context_h()->DrawBitmap(mp_ppBitmap1.Get(), &rsRect, 1.0f,
D2D1_BITMAP_INTERPOLATION_MODE_NEAREST_NEIGHBOR, &rtRect);
d2d_device_context_h()->EndDraw();
Where iPixelsize.xy is the size of the "pixelated pixel", note that i just use linear interpolation when shrinking the bmp and NOT when i reenlarge. This will generate a pixelation effect.
I am capturing particular portion of desktop window in a bitmap and trying to print
BGR pixel color value. This captured portion of desktop window is completely filled with 16, 0, 16 color.
When I capture and print the data when my window color depth is 32 everything is right,
but if my window is in 24/16 bit color mode then I am getting different pixel values instead of 16, 0, 16.
I am capturing screen left = 150, top = 150, right = 200, bottom = 200.
*********CAPTURING AN IMAGE FROM DESKTOP*********
iLeft = 150;
iTop = 150;
iRight = iLeft + 50;
iBottom = iTop + 50;
/*
HDC hdcScreen;
HDC hdcWindow;
HDC hdcMemDC = NULL;
HBITMAP hbmScreen = NULL;
BITMAP bmpScreen;
// Retrieve the handle to a display device context for the client
// area of the window.
hdcScreen = GetDC(NULL);
hdcWindow = GetDC(hWnd);
// Create a compatible DC which is used in a BitBlt from the window DC
hdcMemDC = CreateCompatibleDC(hdcWindow);
if(!hdcMemDC)
{
MessageBox(hWnd, L"CreateCompatibleDC has failed",L"Failed", MB_OK);
goto done;
}
// Get the client area for size calculation
RECT rcClient;
GetClientRect(hWnd, &rcClient);
//This is the best stretch mode
SetStretchBltMode(hdcWindow,HALFTONE);
//The source DC is the entire screen and the destination DC is the current window (HWND)
if(!StretchBlt(hdcWindow,
0,0,
rcClient.right, rcClient.bottom,
hdcScreen,
0,0,
GetSystemMetrics (SM_CXSCREEN),
GetSystemMetrics (SM_CYSCREEN),
SRCCOPY))
{
MessageBox(hWnd, L"StretchBlt has failed",L"Failed", MB_OK);
goto done;
}
// Create a compatible bitmap from the Window DC
hbmScreen = CreateCompatibleBitmap(hdcWindow, rcClient.right-rcClient.left, rcClient.bottom-rcClient.top);
if(!hbmScreen)
{
MessageBox(hWnd, L"CreateCompatibleBitmap Failed",L"Failed", MB_OK);
goto done;
}
// Select the compatible bitmap into the compatible memory DC.
SelectObject(hdcMemDC,hbmScreen);
// Bit block transfer into our compatible memory DC.
if(!BitBlt(hdcMemDC,
0,0,
rcClient.right-rcClient.left, rcClient.bottom-rcClient.top,
hdcWindow,
0,0,
SRCCOPY))
{
MessageBox(hWnd, L"BitBlt has failed", L"Failed", MB_OK);
goto done;
}
// Get the BITMAP from the HBITMAP
GetObject(hbmScreen,sizeof(BITMAP),&bmpScreen);
BITMAPFILEHEADER bmfHeader;
BITMAPINFOHEADER bi;
bi.biSize = sizeof(BITMAPINFOHEADER);
bi.biWidth = bmpScreen.bmWidth;
bi.biHeight = bmpScreen.bmHeight;
bi.biPlanes = 1;
bi.biBitCount = 32;
bi.biCompression = BI_RGB;
bi.biSizeImage = 0;
bi.biXPelsPerMeter = 0;
bi.biYPelsPerMeter = 0;
bi.biClrUsed = 0;
bi.biClrImportant = 0;
DWORD dwBmpSize = ((bmpScreen.bmWidth * bi.biBitCount + 31) / 32) * 4 * bmpScreen.bmHeight;
// Starting with 32-bit Windows, GlobalAlloc and LocalAlloc are implemented as wrapper functions that
// call HeapAlloc using a handle to the process's default heap. Therefore, GlobalAlloc and LocalAlloc
// have greater overhead than HeapAlloc.
HANDLE hDIB = GlobalAlloc(GHND,dwBmpSize);
char *lpbitmap = (char *)GlobalLock(hDIB);
// Gets the "bits" from the bitmap and copies them into a buffer
// which is pointed to by lpbitmap.
GetDIBits(hdcWindow, hbmScreen, 0,
(UINT)bmpScreen.bmHeight,
lpbitmap,
(BITMAPINFO *)&bi, DIB_RGB_COLORS);
// A file is created, this is where we will save the screen capture.
HANDLE hFile = CreateFile(L"captureqwsx.bmp",
GENERIC_WRITE,
0,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL, NULL);
// Add the size of the headers to the size of the bitmap to get the total file size
DWORD dwSizeofDIB = dwBmpSize + sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);
//Offset to where the actual bitmap bits start.
bmfHeader.bfOffBits = (DWORD)sizeof(BITMAPFILEHEADER) + (DWORD)sizeof(BITMAPINFOHEADER);
//Size of the file
bmfHeader.bfSize = dwSizeofDIB;
//bfType must always be BM for Bitmaps
bmfHeader.bfType = 0x4D42; //BM
DWORD dwBytesWritten = 0;
WriteFile(hFile, (LPSTR)&bmfHeader, sizeof(BITMAPFILEHEADER), &dwBytesWritten, NULL);
WriteFile(hFile, (LPSTR)&bi, sizeof(BITMAPINFOHEADER), &dwBytesWritten, NULL);
WriteFile(hFile, (LPSTR)lpbitmap, dwBmpSize, &dwBytesWritten, NULL);
//Unlock and Free the DIB from the heap
GlobalUnlock(hDIB);
GlobalFree(hDIB);
//Close the handle for the file that was created
CloseHandle(hFile);
//Clean up
done:
DeleteObject(hbmScreen);
DeleteObject(hdcMemDC);
ReleaseDC(NULL,hdcScreen);
ReleaseDC(hWnd,hdcWindow);
return 0;
*/
//#if 0
HDC hdcScreen;
// HDC hdcWindow;
HDC hdcMemDC = NULL;
HBITMAP hbmScreen = NULL;
HRGN rgn = NULL;
BITMAP bmpScreen;
// Retrieve the handle to a display device context for the client
// area of the window.
hdcScreen = GetDC(NULL);
//hdcWindow = GetDC(hWnd);
//if(g_hdcMemDC == NULL)
{
// Create a compatible DC which is used in a BitBlt from the window DC
hdcMemDC = CreateCompatibleDC(hdcScreen);
if(!hdcMemDC)
{
//MessageBox(hWnd, L"CreateCompatibleDC has failed",L"Failed", MB_OK);
goto done;
}
// Get the client area for size calculation
//RECT rcClient;
//GetClientRect(hWnd, &rcClient);
//This is the best stretch mode
SetStretchBltMode(hdcMemDC,HALFTONE);
// Create a compatible bitmap from the Window DC
hbmScreen = CreateCompatibleBitmap(hdcScreen, iRight - iLeft, iBottom - iTop);
if(!hbmScreen)
{
//MessageBox(hWnd, L"CreateCompatibleBitmap Failed",L"Failed", MB_OK);
goto done;
}
// Select the compatible bitmap into the compatible memory DC.
SelectObject(hdcMemDC,hbmScreen);
}
//The source DC is the entire screen and the destination DC is the current window (HWND)
/* if(!StretchBlt(hdcWindow,
0,0,
rcClient.right, rcClient.bottom,
hdcScreen,
0,0,
GetSystemMetrics (SM_CXSCREEN),
GetSystemMetrics (SM_CYSCREEN),
SRCCOPY))
{
MessageBox(hWnd, L"StretchBlt has failed",L"Failed", MB_OK);
goto done;
}
*/
// Bit block transfer into our compatible memory DC.
if(!StretchBlt(hdcMemDC,
0,0,
iRight - iLeft,
iBottom - iTop,
hdcScreen,
iLeft,iTop,
iRight - iLeft,
iBottom - iTop,
SRCCOPY))
{
// MessageBox(hWnd, L"StretchBlt has failed",L"Failed", MB_OK);
goto done;
}
}
*******************FUNCTION FOR GETTING THE BITMAP RAW DATA POINTER***********
BYTE* Get24BitPixels(HDC dcDesktop, HBITMAP pBitmap, WORD *pwWidth, WORD *pwHeight, WORD * pReminderWidth)
{
// a bitmap object just to get bitmap width and height
BITMAP bmpBmp;
// pointer to original bitmap info
LPBITMAPINFO pbmiInfo;
// bitmap info will hold the new 24bit bitmap info
BITMAPINFO bmiInfo;
// width and height of the bitmap
WORD wBmpWidth ; WORD wBmpHeight;
// ---------------------------------------------------------
// get some info from the bitmap
// ---------------------------------------------------------
GetObject(pBitmap, sizeof(bmpBmp),&bmpBmp);
pbmiInfo = (LPBITMAPINFO)&bmpBmp;
//get width and height
wBmpWidth = (WORD)pbmiInfo->bmiHeader.biWidth;
int iReminderWidth = (wBmpWidth%4);
//wBmpWidth -= (wBmpWidth%4); // width is 4 byte boundary aligned.
wBmpHeight = (WORD)pbmiInfo->bmiHeader.biHeight;
// copy to caller width and height parms
*pwWidth = wBmpWidth;
*pwHeight = wBmpHeight;
wBmpWidth += (4 - iReminderWidth); // width is 4 byte boundary aligned, thereby increasing the width
//so that it will be fully divible by four , it will cause some extra bytes to be filled in with garbage value
//beyond the actual width of the bitmap, we will be discrading this extra padding pixels data while processign each pixel.
*pReminderWidth = 4 - iReminderWidth;
// ---------------------------------------------------------
// allocate width * height * 24bits pixels
BYTE * pPixels = new BYTE[wBmpWidth*wBmpHeight*3];
if (!pPixels) return NULL;
// get user desktop device context to get pixels from
//HDC hDC = GetWindowDC(NULL);
// fill desired structure
bmiInfo.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
bmiInfo.bmiHeader.biWidth = wBmpWidth;
bmiInfo.bmiHeader.biHeight = -wBmpHeight;
bmiInfo.bmiHeader.biPlanes = 1;
bmiInfo.bmiHeader.biBitCount = 24;
bmiInfo.bmiHeader.biCompression = BI_RGB;
bmiInfo.bmiHeader.biSizeImage = wBmpWidth*wBmpHeight*3;
bmiInfo.bmiHeader.biXPelsPerMeter = 0;
bmiInfo.bmiHeader.biYPelsPerMeter = 0;
bmiInfo.bmiHeader.biClrUsed = 0;
bmiInfo.bmiHeader.biClrImportant = 0;
// get pixels from the original bitmap converted to 24bits
int iRes = GetDIBits(dcDesktop,pBitmap,0,wBmpHeight,(LPVOID)pPixels,&bmiInfo,DIB_RGB_COLORS);
// release the device context
//ReleaseDC(NULL,hDC);
// if failed, cancel the operation.
if (!iRes)
{
delete [] pPixels;
pPixels = NULL;
return NULL;
};
// return the pixel array
return pPixels;
}
Fortunately I got the below mention post on StackOverflow.
Many thanks to Vodemki who posted this answer.
Get Pixel color fastest way?
How would you draw text on a IMFMediaBuffer object, and write it out to another IMFMediaBuffer object?
The context is that I'm building an MFT, and initially I tried using Direct2D and Direct3D11 to achieve this, but to no avail.
I was able to accomplish this with Windows GDI calls. The MFT I created has RGB32 input/output types, which allows me to copy to/from a Bitmap object. I copy the part of the frame I want to overlay with text into a Windows bitmap I create, draw the text, then copy back to the IMFMediaBuffer. Here's my code:
#define BREAK_ON_FAIL(val) { if ( FAILED(hr = (val)) ) { break; } }
HRESULT AddOverlay(IMFSample* pSample)
{
HRESULT hr = S_OK;
IMFMediaBuffer * pBuffer;
HDC hDC, hMemDC;
HBITMAP hBitmap, hOldBitmap;
do
{
BYTE * pBufferData;
UINT32 nWidth, nHeight;
BITMAPV5HEADER bi;
BYTE * pBitmapData;
UINT32 nXOffset, nYOffset;
RECT OverlayRect;
OverlayRect.left = 0;
OverlayRect.top = 0;
OverlayRect.right = 400;
OverlayRect.bottom = 32;
DWORD nOverlayWidth = OverlayRect.right - OverlayRect.left;
DWORD nOverlayHeight = OverlayRect.bottom - OverlayRect.top;
LONG lFrameStride;
// Get the frame dimensions and stride
MFGetAttributeSize(m_pInputType, MF_MT_FRAME_SIZE, &nWidth, &nHeight);
m_pInputType->GetUINT32(MF_MT_DEFAULT_STRIDE, (UINT32*)&lFrameStride))
// Setup offset for the overlay area into the video frame
nXOffset = (nWidth - nOverlayWidth) / 2;
nYOffset = nOverlayHeight-1;
// Set up the bitmap header
ZeroMemory(&bi, sizeof(BITMAPV5HEADER));
bi.bV5Size = sizeof(BITMAPV5HEADER);
bi.bV5Width = nOverlayWidth;
// If the stride is negative, the bitmap is bottom-up, which is designated by a negative height
bi.bV5Height = (lFrameStride > 0) ? nOverlayHeight : -(LONG)nOverlayHeight;
bi.bV5Planes = 1;
bi.bV5BitCount = 32;
bi.bV5Compression = BI_RGB;
// The following mask specification specifies a supported 32 BPP
// alpha format for Windows XP.
bi.bV5RedMask = 0x00FF0000;
bi.bV5GreenMask = 0x0000FF00;
bi.bV5BlueMask = 0x000000FF;
bi.bV5AlphaMask = 0xFF000000;
// Create a DIB section with an alpha channel, along with
// a memory device context
hDC = GetDC(NULL);
hBitmap = CreateDIBSection(hDC, (BITMAPINFO*)&bi, DIB_RGB_COLORS, (void**)&pBitmapData, NULL, 0);
hMemDC = CreateCompatibleDC(hDC);
ReleaseDC(NULL, hDC);
// Lock the media buffer for our use
BREAK_ON_FAIL( pSample->GetBufferByIndex(0, &pBuffer) );
BREAK_ON_FAIL( pBuffer->Lock(&pBufferData, NULL, NULL) );
// Copy the video frame to the bitmap (to support transparency)
MFCopyImage(pBitmapData, nOverlayWidth*sizeof(RGBQUAD),
pBufferData + nYOffset*abs(lFrameStride) + nXOffset*sizeof(RGBQUAD), lFrameStride,
nOverlayWidth*sizeof(RGBQUAD), nOverlayHeight);
// Draw on the bitmap
hOldBitmap = (HBITMAP)SelectObject(hMemDC, hBitmap);
//FillRect(hMemDC, &OverlayRect, WHITE_BRUSH);
SetTextColor(hMemDC, RGB(255,0,0));
SetBkMode(hMemDC, TRANSPARENT);
DrawText(hMemDC, _T("Hello World!"), 12, &OverlayRect, DT_CENTER);
SelectObject(hMemDC, hOldBitmap);
// Copy the bitmap to the buffer
MFCopyImage(pBufferData + nYOffset*abs(lFrameStride) + nXOffset*sizeof(RGBQUAD), lFrameStride,
pBitmapData, nOverlayWidth*sizeof(RGBQUAD),
nOverlayWidth*sizeof(RGBQUAD), nOverlayHeight);
BREAK_ON_FAIL( pBuffer->Unlock() );
} while(false);
DeleteDC(hMemDC);
DeleteObject(hBitmap);
SafeRelease(&pBuffer);
return hr;
}
References:
http://support.microsoft.com/kb/318876
I have a raw image buffer in the RGB format. I need to draw it to CGContext so that I get a new buffer of the format ARGB. I accomplish this in the following way:
Create a data provider out of raw buffer using CGDataProviderCreateWithData and then create image out of the data provider with the api: CGImageCreate.
Now if I write this image back to the CGBitmapContext using CGContextImageDraw.
Instead of creating an intermediate image, is there any way of writing the buffer directly to CGContext so that I can avoid the image creation phase?
Thanks
If all you want is to take RGB data with no alpha component and turn it into ARGB data with full opacity (alpha = 1.0 at all points), why not just copy the data yourself into a new buffer?
// assuming 24-bit RGB (1 byte per color component)
unsigned char *rgb = /* ... */;
size_t rgb_bytes = /* ... */;
const size_t bpp_rgb = 3; // bytes per pixel - rgb
const size_t bpp_argb = 4; // bytes per pixel - argb
const size_t npixels = rgb_bytes / bpp_rgb;
unsigned char *argb = malloc(npixels * bpp_argb);
for (size_t i = 0; i < npixels; ++i) {
const size_t argbi = bpp_argb * i;
const size_t rgbi = bpp_rgb * i;
argb[argbi] = 0xFF; // alpha - full opacity
argb[argbi + 1] = rgb[rgbi]; // r
argb[argbi + 2] = rgb[rgbi + 1]; // g
argb[argbi + 3] = rgb[rgbi + 2]; // b
}
If you are using a CGBitmapContext then you can get a pointer to the bitmap buffer using the CGBitmapContextGetData() function. You can then write your data directly to the buffer.