I'm not sure I understand how to free a bitmap context.
I'm doing the following:
CGContextRef context = CGBitmapContextCreate(nil, size.width, size.height, 8, 0, CGColorSpaceCreateDeviceRGB(), kCGBitmapAlphaInfoMask);
.
. // (All straightforward code)
.
CGContextRelease(context);
Xcode Analyze still gives me a "potential memory leak" on the CGBitmapContextCreate line.
What am I doing wrong?
As you don't assign the result of CGColorSpaceCreateDeviceRGB() to a variable, you loose reference to the object created by that method.
You need that reference later to release the colorspace object. Core Graphics follows the Core Foundation memory management rules.
You can find more about that here.
Fixed version of your code:
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(nil, size.width, size.height, 8, 0, colorSpace, kCGBitmapAlphaInfoMask);
.
. // (All straightforward code)
.
CGContextRelease(context);
CGColorSpaceRelease(colorSpace);
If you click the blue icon that the code analyzer places in your source, you get an arrow graph that shows you the origin of the leak. (I assume it would point to the line where you create the color space)
You are leaking the color space object from CGColorSpaceCreateDeviceRGB() call. You need to release the color space too.
Related
I'm having trouble getting a rendered video's colors to match the source content's colors. I'm rendering images into a CGContext, converting the backing data into a CVPixelBuffer and appending that as a frame to an AVAssetWriterInputPixelBufferAdaptor. This causes slight color differences between the images that I'm drawing into the CGContext and the resulting video file.
It seems like there are 3 things that need to be addressed:
tell AVFoundation what colorspace the video is in.
make the AVAssetWriterInputPixelBufferAdaptor and the CVPixelBuffers I append to it match that color space.
use the same colorspace for the CGContext.
The documentation is terrible, so I'd appreciate any guidance on how to do these things or if there is something else I need to do to make the colors be preserved throughout this entire process.
Full code:
AVAssetWriter *_assetWriter;
AVAssetWriterInput *_assetInput;
AVAssetWriterInputPixelBufferAdaptor *_assetInputAdaptor;
NSDictionary *outputSettings = #{ AVVideoCodecKey :AVVideoCodecH264,
AVVideoWidthKey :#(outputWidth),
AVVideoHeightKey:#(outputHeight)};
_assetInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo
outputSettings:outputSettings];
NSDictionary *bufferAttributes = #{å(NSString*)kCVPixelBufferPixelFormatTypeKey:#(kCVPixelFormatType_32ARGB)};
_assetInputAdaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:_assetInput
sourcePixelBufferAttributes:bufferAttributes];
_assetWriter = [AVAssetWriter assetWriterWithURL:aURL fileType:AVFileTypeMPEG4 error:nil];
[_assetWriter addInput:_assetInput];
[_assetWriter startWriting];
[_assetWriter startSessionAtSourceTime:kCMTimeZero];
NSInteger bytesPerRow = outputWidth * 4;
long size = bytesPerRow * outputHeight;
CGColorSpaceRef srgbSpace = CGColorSpaceCreateWithName(kCGColorSpaceSRGB);
UInt8 *data = (UInt8 *)calloc(size, 1);
CGContextRef ctx = CGBitmapContextCreateWithData(data, outputWidth, outputHeight, 8, bytesPerRow, srgbSpace, kCGImageAlphaPremultipliedFirst, NULL, NULL);
// draw everything into ctx
CVPixelBufferRef pixelBuffer;
CVPixelBufferCreateWithBytes(kCFAllocatorSystemDefault,
outputWidth, outputHeight,
k32ARGBPixelFormat,
data,
bytesPerRow,
ReleaseCVPixelBufferForCVPixelBufferCreateWithBytes,
NULL,
NULL,
&pixelBuffer);
NSDictionary *pbAttachements = #{(id)kCVImageBufferCGColorSpaceKey : (__bridge id)srgbSpace};
CVBufferSetAttachments(pixelBuffer, (__bridge CFDictionaryRef)pbAttachements, kCVAttachmentMode_ShouldPropagate);
[_assetInputAdaptor appendPixelBuffer:pixelBuffer withPresentationTime:CMTimeMake(0, 60)];
CGColorSpaceRelease(srgbSpace);
[_assetInput markAsFinished];
[_assetWriter finishWritingWithCompletionHandler:^{}];
This is quite a confusing subject and the Apple docs really do not help all that much. I am going to describe the solution I have settled on based on using the BT.709 colorspace, I am sure someone will have an objection based on Colorimetric correctness and the weirdness of various video standards, but this is complex topic. First off, don't use kCVPixelFormatType_32ARGB as the pixel type. Always pass kCVPixelFormatType_32BGRA instead, since BGRA is the native pixel layout on both MacOSX and iPhone hardware and it BGRA is just faster. Next, when you create a CGBitmapContext to render into use the BT.709 colorspace (kCGColorSpaceITUR_709). Also, don't render into a malloc() buffer, render directly into the CoreVideo pixel buffer by creating a bitmap context over the same memory, CoreGraphics will handle the colorspace and gamma conversion from whatever your input image is to BT.709 and its associated gamma. Then you need to tell AVFoundation the colorspace of the pixel buffer, do that by making an ICC profile copy and setting the kCVImageBufferICCProfileKey on the CoreVideo pixel buffer. That takes care of your issues 1 and 2, you do not need to have input images in this same colorspace with this approach. Now, this is of course complex and actual working source code (yes actually working) is hard to come by. Here is a github link to a small project that does these exact steps, the code is BSD licensed, so feel free to use it. Note specifically the H264Encoder class which wraps all this horror up into a reusable module. You can find calling code in encode_h264.m, it is a little MacOSX command line util to encode PNG to M4V. Also attached 3 keys Apple docs related to this subject 1, 2, 3.
MetalBT709Decoder
I created an app in which I want to display text on top of google maps. I chose to use custom markers, but they can only be images, so I decided to create an image from my text utilizing SkiaSharp.
private static ImageSource CreateImageSource(string text)
{
int numberSize = 20;
int margin = 5;
SKBitmap bitmap = new SKBitmap(30, numberSize + margin * 2, SKImageInfo.PlatformColorType, SKAlphaType.Premul);
SKCanvas canvas = new SKCanvas(bitmap);
SKPaint paint = new SKPaint
{
Style = SKPaintStyle.StrokeAndFill,
TextSize = numberSize,
Color = SKColors.Red,
StrokeWidth = 1,
};
canvas.DrawText(text.ToString(), 0, numberSize, paint);
SKImage skImage = SKImage.FromBitmap(bitmap);
SKData data = skImage.Encode(SKEncodedImageFormat.Png, 100);
return ImageSource.FromStream(data.AsStream);
}
The images I create however have ugly artifacts on the top of the resulting image and my feeling is that they get worse if I create multiple images.
I built an example app, that shows the artifacts and the code I used to draw the text. It can be found here:
https://github.com/hot33331/SkiaSharpExample
How can I get rid of those artifacts. Am I using skia wrong?
I got the following answer from Matthew Leibowitz on the SkiaSharp GitHub:
The chances are you are not clearing the canvas/bitmap first.
You can either do bitmap.Erase(SKColors.Transparent) or canvas.Clear(SKColors.Transparent) (you can use any color).
The reason for this is performance. When creating a new bitmap, the computer has no way of knowing what background color you want. So, if it was to go transparent and you wanted white, then there would be two draw operations to clear the pixels (and this may be very expensive for large images).
During the allocation of the bitmap, the memory is provided, but the actual data is untouched. If there was anything there previously (which there will be), this data appears as colored pixels.
When I've seen that before, it's been because the memory passed to SkiaSharp was not zeroed. As an optimization, though, Skia assumes that the memory block passed to it is pre zeroed. Resultingly, if your first operation is a clear, it will ignore that operation, because it thinks that the state is already clean. To resolve this issue, you can manually zero the memory passed to SkiaSharp.
public static SKSurface CreateSurface(int width, int height)
{
// create a block of unmanaged native memory for use as the Skia bitmap buffer.
// unfortunately, this may not be zeroed in some circumstances.
IntPtr buff = System.Runtime.InteropServices.Marshal.AllocCoTaskMem(width * height * 4);
byte[] empty = new byte[width * height * 4];
// copy in zeroed memory.
// maybe there's a more sanctioned way to do this.
System.Runtime.InteropServices.Marshal.Copy(empty, 0, buff, width * height * 4);
// create the actual SkiaSharp surface.
var colorSpace = CGColorSpace.CreateDeviceRGB();
var bContext = new CGBitmapContext(buff, width, height, 8, width * 4, colorSpace, (CGImageAlphaInfo)bitmapInfo);
var surface = SKSurface.Create(width, height, SKColorType.Rgba8888, SKAlphaType.Premul, bitmap.Data, width * 4);
return surface;
}
Edit: btw, I assume this is a bug in SkiaSharp. The samples/apis that create the buffer for you should probably be zeroing it out. Depending on the platform it can be hard to repro as the memory alloc behaves differently. More or less likely to provide you untouched memory.
i'm handing off to this method and using the resultant CGImageRef adding it to a CGImageDestinationRef that is setup for finalizing as a kUTTypeJPEG:
- (CGImageRef)scaleImage:(CGImageRef)fullImageRef originalSize:(CGSize)originalSize scale:(CGFloat)scale;
{
CGSize imageSize = originalSize;
CGRect scaledRect = CGRectMake(0.0, 0.0, imageSize.width * scale, imageSize.height * scale);
CGColorSpaceRef colorSpace = CGImageGetColorSpace(fullImageRef);
CGContextRef context = CGBitmapContextCreate(NULL, scaledRect.size.width, scaledRect.size.height, 8, 0, colorSpace, CGImageGetAlphaInfo(fullImageRef));
CGContextSetInterpolationQuality(context, kCGInterpolationHigh);
CGImageRef subImage = CGImageCreateWithImageInRect(fullImageRef, scaledRect);
CGContextDrawImage(context, scaledRect, subImage); // offscreen
CGImageRef scaledImage = CGBitmapContextCreateImage(context);
CGImageRelease(subImage);
subImage = NULL;
subImage = scaledImage;
CGImageRelease(scaledImage);
CGContextFlush(context);
CGContextRelease(context);
return subImage;
}
i need to be sure that the resulting JPEG is the best possible color match for the original.
what i am unclear about is the actual role of CGImageAlphaInfo constant for my purposes.
i've read Quartz docs, and also the excellent Programming With Quartz book (by Gelphman and Laden), but i'm still not sure of my methods.
i've set the property kCGImageDestinationLossyCompressionQuality to 1.0 in the jpeg's property dictionary, along with capturing other properties.
should i be doing something more to maintain the color integrity of the original?
this is macos, and using gc.
It doesn't much matter, because JPEG doesn't support alpha (at least, not without external masking or an extension), so you can expect that CGImageDestination will throw away the alpha channel at the export stage.
I would try the original image's alpha info first, and if I can't create the bitmap context that way, I would use either kCGImageAlphaNoneSkipFirst or kCGImageAlphaNoneSkipLast. For other destination file formats, of course, I'd use one of the alpha-with-premultiplied-colors pixel formats.
This page has the full list of combinations supported by CGBitmapContext.
I am creating a brush using CreatePatternBrush with a bitmap created with CreateBitmap.
The bitmap is 1 pixel wide and 24 pixels tall, I have the RGB value for each pixel, so I create an array of rgbquads and pass that to CreateBitmap.
This works fine when the screen color depth is 32bpp, since the bitmap I create is also 32bpp.
When the screen color depth is not 32bpp, this fails, and I understand why it does, since I should be creating a compatible bitmap instead.
It seems I should use CreateCompatibleBitmap instead, but how do I put the pixel data I have into that bitmap?
I have also read about CreateDIBPatternBrushPt, CreateDIBitmap, CreateDIBSection, etc.
I don´t understand what is a DIBSection, and find the subject generally confusing.
I do understand that I need a bitmap with the same color depth as the screen, but how do I create it having only the 32bpp pixel data?
You could create a DIB because you can use a Device Independent Bitmap independently of the screen color depth. See CreateDIBSection().
How can you create it having only the 32bpp pixel data? A DIB can be created with 32bpp data. As you can read in the documentation:
The CreateDIBSection function creates
a DIB that applications can write to
directly. The function gives you a
pointer to the location of the bitmap
bit values.
If hSection is NULL, the system
allocates memory for the DIB. If the
function succeeds, the return value is
a handle to the newly created DIB, and
*ppvBits points to the bitmap bit values.
Try something like this:
VOID *ppvBits = NULL;
BITMAPINFO BitmapInfo;
memset(&BitmapInfo, 0, sizeof(BITMAPINFOHEADER));
BitmapInfo.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
BitmapInfo.bmiHeader.biWidth = 1;
BitmapInfo.bmiHeader.biHeight = 24;
BitmapInfo.bmiHeader.biPlanes = 1;
BitmapInfo.bmiHeader.biBitCount = 32;
BitmapInfo.bmiHeader.biCompression = BI_RGB;
HBITMAP hBitmap = CreateDIBSection(hDC, &BitmapInfo, DIB_RGB_COLORS, &ppvBits, NULL, 0);
In our case *ppvBits points to 1 * 24 * (32 / 8) allocated bytes.
It is important to know that if biHeight is positive, the bitmap is a bottom-up DIB and its origin is the lower-left corner. See BITMAPINFOHEADER Structure for more info.
I solved it by using CreateCompatibleBitmap and SetPixel. Not the best option I guess, but it works.
EDIT: I've offered a bounty, since I doubt I'll be getting any answers otherwise.
Lately I've been working with listviews and I've decided to add an icon for each item indicating whether it's input or output. The icons add fine, but they're not transparent:
As can be seen, the icons are clearly not transparent. I'm currently doing something like this load the icons:
hImageList = ImageList_Create(16, 16, ILC_MASK | ILC_COLOR32, 2, 2);
if (hImageList != NULL)
{
iIN = ImageList_AddIcon(hImageList, LoadIcon(hInstance, MAKEINTRESOURCE(101)));
iOUT = ImageList_AddIcon(hImageList, LoadIcon(hInstance, MAKEINTRESOURCE(102)));
}
I've tried messing with the flags for ImageList_Create & LoadIcon/LoadImage but have had no luck and to be honest I've run out of ideas.
Any help would be very appreciated.
First up, ImageList_ReplaceIcon copies the icon data when adding it to an image list. So the HICON needs to be released afterwards.
Next, imagelists are natively bitmaps, not icons. And the way you are creating your imagelist makes the conversion of icon to bitmap very ambiguous. ILC_COLOR32 implies the imagelist should be created as a 32bit dib section, which typically contain transparency information via an embedded alpha channel. ILC_MASK instead implies that the internal bitmaps are DDB bitmaps, with the transparency information stored as a 1bpp mask bitmap.
The quickest solution to your problem - take your two icons:
Merge them into a single bitmap resource thats 32 pels wide by 16 high. Fill the background with a mask color :- purple or something.
Create the bitmap using ILC_COLOR|ILC_MASK
Load the bitmap being sure NOT to use LR_TRANSPARENT.
Add the bitmap using ImageList_AddMasked passing in a COLORREF that represents the mask color.
OR, for a better visual effect...
export your PNG data as a 32x16 32bpp bitmap file containing pre-multiplied alpha channel data.
Create the imagelist using the ILC_COLOR32 value.
LoadImage() with LR_CREATEDIBSECTION to load the bitmap as a 32bpp dib section.
Add the image using ImageList_Add()
(the last option is kind of tricky as the number of tools that support writing out 32bit bmp files with properly pre multiplied alpha channels is rather low).
Edited to add the following code sample. Using a 4bpp bitmap created in the dev environment this works just great :-
HWND hwndCtl = CreateWindowEx(0,WC_LISTVIEW,TEXT("ListView1"),WS_CHILD|WS_VISIBLE|WS_HSCROLL|WS_VSCROLL,0,0,cx,cy,hWnd,(HMENU)101,hModule,NULL);
HBITMAP hbm = (HBITMAP)LoadImage(hModule,MAKEINTRESOURCE(IDB_BITMAP1),IMAGE_BITMAP,0,0,0);
COLORREF crMask=RGB(255,0,255);
HIMAGELIST himl = ImageList_Create(16,16,ILC_COLOR|ILC_MASK,2,0);
ImageList_AddMasked(himl,hbm,crMask);
ListView_SetImageList(hwndCtl,himl,LVSIL_NORMAL);
You want to make your icons have a background color that isn't used anywhere else in the icon, like a really ugly purple, and then use LoadImage(..., LR_LOADTRANSPARENT); The flag says look at the first pixel at 0,0 and make everything that color transparent.
Your code looks fine to me, I always use LoadImage instead of LoadIcon but I suspect that doesn't matter. Have you checked that the icons do indeed have transparent areas and don't themselves have a solid background?
My LoadImage calls look like:
HICON hIcon = (HICON)LoadImage(hinstResources,MAKEINTRESOURCE(IDI_ICON),IMAGE_ICON,16,16,LR_DEFAULTCOLOR);
Here... Create an ImageList, as suggested, make your icons into a Bitmap, 16 pixels high, by 16*n long, where n= the number of icons...
Set the background color to 255, 0, 255, like you have done.
Then, load it, and add it to the image list as I did here:
m_ImageList.Create(16, 16, ILC_COLOR16 | ILC_MASK, 7, 1);
CBitmap bm;
bm.LoadBitmap(IDB_SUPERTREEICONS);
m_ImageList.Add(&bm, RGB(255, 0, 255));
GetTreeCtrl().SetImageList(&m_ImageList, TVSIL_NORMAL);
Of course, this was written in MFC, but as you know, it's just a wrapper to Win32...
Outside of this, you are going to have to go to a custom draw control, in which you draw the icon over whatever background the icon happens to be sitting on. There isn't really any magic "transparent" color, that I know of, in any of these controls.
In the case of a custom draw, you need to use code like the following:
#define TRANSPARENT_COLOR (255,0,255)
UINT iBitmap = IDB_ICON_UP
CDC *dc = GetDC();
int x = 0, y = 0;
CDC *pDisplayMemDC = new CDC;
CDC *pMaskDC = new CDC;
CDC *pMemDC = new CDC;
CBitmap *pBitmap = new CBitmap;
CBitmap *pMaskBitmap = new CBitmap;
CBitmap *pMemBitmap = new CBitmap;
int cxLogo, cyLogo;
BITMAP bm;
pBitmap->LoadBitmap(iBitmap);
pDisplayMemDC->CreateCompatibleDC(dc);
CBitmap *pOldBitmap = (CBitmap *)pDisplayMemDC->SelectObject(pBitmap);
pBitmap->GetObject(sizeof(bm), &bm);
cxLogo = bm.bmWidth;
cyLogo = bm.bmHeight;
pMaskBitmap->CreateBitmap(cxLogo, cyLogo, 1, 1, NULL);
pMaskDC->CreateCompatibleDC(dc);
CBitmap *pOldMask = (CBitmap *)pMaskDC->SelectObject(pMaskBitmap);
COLORREF oldBkColor = pDisplayMemDC->SetBkColor(TRANSPARENT_COLOR);
pMaskDC->BitBlt(0, 0, cxLogo, cyLogo, pDisplayMemDC, 0, 0, SRCCOPY);
pMemBitmap->CreateCompatibleBitmap(dc, cxLogo, cyLogo);
pMemDC->CreateCompatibleDC(dc);
CBitmap *pOldMem = (CBitmap *)pMemDC->SelectObject(pMemBitmap);
pMemDC->BitBlt(0, 0, cxLogo, cyLogo, dc, x, y, SRCCOPY);
pMemDC->BitBlt(0, 0, cxLogo, cyLogo, pDisplayMemDC, 0, 0, SRCINVERT);
pMemDC->BitBlt(0, 0, cxLogo, cyLogo, pMaskDC, 0, 0, SRCAND);
pMemDC->BitBlt(0, 0, cxLogo, cyLogo, pDisplayMemDC, 0, 0, SRCINVERT);
dc->BitBlt(x, y, cxLogo, cyLogo, pMemDC, 0, 0, SRCCOPY);
delete pMemDC->SelectObject(pOldMem);
delete pMemDC;
delete pMaskDC->SelectObject(pOldMask);
delete pMaskDC;
delete pDisplayMemDC->SelectObject(pOldBitmap);
delete pDisplayMemDC;
This code decides where to draw the icon, and takes a snapshot of the background, creates a mask for the icon, and then draws it over the background, giving it a fully transparent background...
Hope that helps somewhat. If not, please explain in more detail what you are trying to make happen, and what you are seeing, or what you are NOT seeing...
I struggled with the same issue using an ImageList in a Tree View. I eventually got Chris Becke's second solution to work, creating an ImageList using the ILC_COLOR32 flag and using LoadImage() with the LR_CREATEDIBSECTION flag. This solution, and probably also the first solution, requires what is described below.
Transparency (and themes) are only supported with comctl32.dll version 6+, to use the correct version, the pre-processor directive on this page worked for me:
https://learn.microsoft.com/en-us/windows/win32/controls/cookbook-overview