I'd LOVE to know what I'm doing wrong here. I'm a bit of a newbie with CGImageRefs so any advice would help.
I'm trying to create a bitmap image that has as it's pixel values a weighted sum of the pixels from another bitmap, and both bitmaps are 16bits per channel. For some reason I had no trouble getting this to work with 8bit images but it fails miserably with 16bit. My guess is that I'm just not setting things up correctly. I've tried using CGFloats, floats and UInt16s as the data types but nothing has worked. The input image has no alpha channel. The output image I get looks liked colored snow.
relevant stuff from the header:
UInt16 *inBaseAddress;
UInt16 *outBaseAddress;
CGFloat inAlpha[5];
CGFloat inRed[5];
CGFloat inGreen[5];
CGFloat inBlue[5];
CGFloat alphaSum, redSum, greenSum, blueSum;
int shifts[5];
CGFloat weight[5];
CGFloat weightSum;
I create the context for the input bitmap (a CGImageRef created with CGImageSourceCreateImageAtIndex(source, 0, NULL)) using:
size_t width = CGImageGetWidth(inBitmap);
size_t height = CGImageGetHeight(inBitmap);
size_t bitmapBitsPerComponent = CGImageGetBitsPerComponent(inBitmap);
size_t bitmapBytesPerRow = (pixelsWide * 4 * bitmapBitsPerComponent / 8);
CGColorSpaceRef colorSpace = CGImageGetColorSpace(inImage);
CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault | kCGImageAlphaNoneSkipLast;
CGContextRef inContext = CGBitmapContextCreate (NULL,width,height,bitmapBitsPerComponent,bitmapBytesPerRow,colorSpace,bitmapInfo);
The context for the output bitmap is created in the same way. I draw the inBitmap into the inContext using:
CGRect rect = {{0,0},{width,height}};
CGContextDrawImage(inContext, rect, inBitmap);
Then I initialize the inBaseAddress and outBaseAddress like so:
inBaseAddress = CGBitmapContextGetData(inContext);
outBaseAddress = CGBitmapContextGetData(outContext);
Then I fill the outBaseAddress with values from the inBaseAddress:
for (n = 0; n < 5; n++)
{
inRed[n] = inBaseAddress[inSpot + 0 + shifts[n]];
inGreen[n] = inBaseAddress[inSpot + 1 + shifts[n]];
inBlue[n] = inBaseAddress[inSpot + 2 + shifts[n]];
inAlpha[n] = inBaseAddress[inSpot + 3 + shifts[n]];
}
alphaSum = 0.0;
redSum = 0.0;
greenSum = 0.0;
blueSum = 0.0;
for (n = 0; n < 5; n++)
{
redSum += inRed[n] * weight[n];
greenSum += inGreen[n] * weight[n];
blueSum += inBlue[n] * weight[n];
alphaSum += inAlpha[n] * weight[n];
}
outBaseAddress[outSpot + 0] = (UInt16)roundf(redSum);
outBaseAddress[outSpot + 1] = (UInt16)roundf(greenSum);
outBaseAddress[outSpot + 2] = (UInt16)roundf(blueSum);
outBaseAddress[outSpot + 3] = (UInt16)roundf(alphaSum);
As a simple check I've tried:
outBaseAddress[outSpot + 0] = inBaseAddress[inSpot + 0];
outBaseAddress[outSpot + 1] = inBaseAddress[inSpot + 1];
outBaseAddress[outSpot + 2] = inBaseAddress[inSpot + 2];
outBaseAddress[outSpot + 3] = inBaseAddress[inSpot + 3];
which works and at least means that the contexts and pointers to the bitmap data are working.
Thanks for any input. This has been pretty frustrating since it worked just fine with 8bit images.
OK, I've got it figured out. I needed to set the bitmapInfo to kCGBitmapByteOrder16Little for the 16bit images and to kCGBitmapByteOrder32Little for the 8bit images. I'm a bit surprised by this actually as would have expected it to be the other way around (32Little for 16 bit and 16Little for 8bit).
I also needed to type def the pointers to the bitmaps as UInt8* and UInt16*. It also appears that I have to include an alpha channel in the bitmapContext. I'm not sure why but the context returned was always nil without it.
It sounds like a byte ordering problem
Have you checked that CGImageGetBitsPerComponent is returning 16? As a matter of style, if you're assuming you're creating a bitmap context with 16 bits per pixel (since you treat the data as UInt16*), you should set explicitly set size_t bitmapBitsPerComponent = 16.
What is your shifts array for? It seems like the most likely place for error, since it's affecting the address you're reading from, but you don't explain it at all. Are the values in shifts multiples of 16?
Related
I am trying to manipulate pixel data for a school assignment. In one of the sections, I am trying to obtain the pixel data of an image and copy it into a buffer. However, the buffer looks like this beside the original image, where the buffer is flattened significantly and contains 2 of the same image. Why is this happening?
Original image on left, buffer on the right (grey rect is just the background):
The buffer is 512x512, exactly the same as the original image, when I checked it in console, yet it looks like that.
Code I used for the copying:
imgs[0].loadPixels();
avgImg.loadPixels();
for(var y = 0; y < imgs[0].height; ++y) {
for(var x = 0; x < imgs[0].width; ++x) {
var index = (y * imgs[0].width + x) * 4;
avgImg.pixels[index] = imgs[0].pixels[index];
avgImg.pixels[index + 1] = imgs[0].pixels[index + 1]
avgImg.pixels[index + 2] = imgs[0].pixels[index + 2]
avgImg.pixels[index + 3] = imgs[0].pixels[index + 3]
}
}
avgImg.updatePixels();
image(avgImg, avgImg.width, 0);
What am I doing wrong?
I found the source of the problem, although I do not really understand why.
This was the setup() originally:
function setup() {
createCanvas(imgs[0].width * 2, imgs[0].height);
avgImg = createGraphics(imgs[0].width, imgs[0].height);
pixelDensity(1);
}
After swapping the positions of pixelDensity() and the declaration for avgImg, it worked.
function setup() {
createCanvas(imgs[0].width * 2, imgs[0].height);
pixelDensity(1);
avgImg = createGraphics(imgs[0].width, imgs[0].height);
}
EDIT: I worked around this issue but to rephrase my question - is it possible to create the CVPixelBuffer to match the CGImage? For example I would like to have 16 bits per component instead of 8.
As the title says. For example, when I NSLog my buffer it has 4544 bytes per row. But when I NSLog the actual CGImage it has 9000. This is the logs:
buffer: <CVPixelBuffer 0x282b8b020 width=1125 height=2436 bytesPerRow=4544 pixelFormat=BGRA iosurface=0x0 attributes={ PixelFormatDescription = {
BitsPerBlock = 32;
BitsPerComponent = 8;
BlackBlock = {length = 4, bytes = 0x000000ff};
CGBitmapContextCompatibility = 1;
CGBitmapInfo = 8196;
CGImageCompatibility = 1;
ComponentRange = FullRange;
ContainsAlpha = 1;
ContainsGrayscale = 0;
ContainsRGB = 1;
ContainsYCbCr = 0;
FillExtendedPixelsCallback = {length = 24, bytes = 0x00000000000000008c3c1da7010000000000000000000000};
IOSurfaceCoreAnimationCompatibility = 1;
IOSurfaceOpenGLESFBOCompatibility = 1;
IOSurfaceOpenGLESTextureCompatibility = 1;
OpenGLESCompatibility = 1;
PixelFormat = 1111970369;
};
} propagatedAttachments={
} nonPropagatedAttachments={
}>
And the image
image: <CGImage 0x1044625d0> (IP) <<CGColorSpace 0x283e9ea60> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; Display P3)>
width = 1125, height = 2436, bpc = 16, bpp = 64, row bytes = 9000
kCGImageAlphaNoneSkipLast | kCGImageByteOrder16Little | kCGImagePixelFormatPacked
is mask? No, has masking color? No, has soft mask? No, has matte? No, should interpolate? Yes
This is the code I use
CGImageRef cgImageRef = [image CGImage];
CGImageRetain(cgImageRef);
CVPixelBufferRef pixelBuffer = NULL;
size_t w = CGImageGetWidth(cgImageRef);
size_t h = CGImageGetHeight(cgImageRef);
CGDataProviderRef x = CGImageGetDataProvider(cgImageRef);
CGDataProviderRetain(x);
CFDataRef da = CGDataProviderCopyData(x);
CFIndex len = CFDataGetLength(da);
const uint8_t* src = CFDataGetBytePtr(da);
CVPixelBufferCreate(kCFAllocatorDefault,
w,
h,
kCVPixelFormatType_32BGRA,
nil,
&pixelBuffer);
CVPixelBufferLockBaseAddress( pixelBuffer, 0 );
void* base = CVPixelBufferGetBaseAddress(pixelBuffer);
SNLog(#"buffer: %#", pixelBuffer);
SNLog(#"image: %#", cgImageRef);
I am later trying to go through the CGImage address and manually copy the bytes into the destination pixel buffer. But it crashes and I assume it is because I am at some point trying to read from "after" the actual image.
I guess my initial question was not exactly clear, or perhaps I should have rephrased it slightly. The problem is more like - I would like to create my buffer to match the image - with the same amount of bytes/bits per component.
I worked around this by looping through the buffer and picking individually bytes I want from the image. Basically ignoring some bits of each pixel.
In the end the image looks fine still, I suspect the increased amount of bits is used for better quality on more capable screens. Although I am not sure.
[The final fix, which works unconditionally: use SetDIBitsToDevice, not BitBlt, to copy out the post-text-draw image data. With this change, all occurrences of the problem are gone.]
I fixed the problem I'm having, but for the life of me I can't figure out why it occurred.
Create a bitmap with CreateDIBitmap. Get a pointer to the bitmap bits.
Select the bitmap into a memory DC.
Background fill the bitmap by directly writing the bitmap memory.
TextOut.
No text displays.
What fixed the problem: change item 3. from direct fill to a call to FillRect. All is well, it works perfectly.
This is under Windows 10 but from what little I could find on the web, it spans all versions of Windows. NO operations work on the bitmap - even calling FillRect - after the manual write. No savvy, Kimosabe. Elsewhere in the app, I even build gradient fills by directly writing to that bitmap memory and there is no problem. But once TextOut is called after the manual fill, the bitmap is locked (effectively) and no further functions work on it - nor do any return an error.
I'm using a font with a 90 degree escapement. Have not tried it with a "normal" font, 0 degree escapement. DrawTextEx with DT_CALCRECT specifically states it only works on 0 degree escapement fonts so I had to use TextOut for this reason.
Very bizarre.
No, there were no stupid mistakes like using the same text color as the background color. I've spent too long on this for that. One option people have available is that the endless energy that would normally be spent destroying the question and/or the person who asked it could instead be used to write a few lines of code and try it for yourself.
Here's a function to make a bitmap. Don't pass a plain colour, pass a gradient fill, say going from white to pinkish.
Does it display correctly? If so, does the TextOut call on top of that work?
static HBITMAP MakeBitmap(unsigned char *rgba, int width, int height, VOID **buff)
{
VOID *pvBits; // pointer to DIB section
HBITMAP answer;
BITMAPINFO bmi;
HDC hdc;
int x, y;
int red, green, blue, alpha;
// setup bitmap info
bmi.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
bmi.bmiHeader.biWidth = width;
bmi.bmiHeader.biHeight = height;
bmi.bmiHeader.biPlanes = 1;
bmi.bmiHeader.biBitCount = 32; // four 8-bit components
bmi.bmiHeader.biCompression = BI_RGB;
bmi.bmiHeader.biSizeImage = width * height * 4;
hdc = CreateCompatibleDC(GetDC(0));
answer = CreateDIBSection(hdc, &bmi, DIB_RGB_COLORS, &pvBits, NULL, 0x0);
for (y = 0; y < height; y++)
{
for (x = 0; x < width; x++)
{
red = rgba[(y*width + x) * 4];
green = rgba[(y*width + x) * 4 + 1];
blue = rgba[(y*width + x) * 4 + 2];
alpha = rgba[(y*width + x) * 4 + 3];
red = (red * alpha) >> 8;
green = (green * alpha) >> 8;
blue = (blue * alpha) >> 8;
((UINT32 *)pvBits)[(height - y - 1) * width + x] = (alpha << 24) | (red << 16) | (green << 8) | blue;
}
}
DeleteDC(hdc);
*buff = pvBits;
return answer;
}
I'm trying to pull the YUV pixel data from an AVFrame, modify the pixels, and put it back into FFmpeg.
I'm currently using this to retrieve the YUV buffer
const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(base->format);
int baseSize = av_image_get_buffer_size(base->format, base->width, base->height, 32);
uint8_t *baseBuffer = (uint8_t*)malloc(baseSize);
av_image_copy_to_buffer(baseBuffer, baseSize, base->data, base->linesize, base->format, base->width, base->height, 32);
But I can't seem to correctly target pixels in that buffer. From the source code they seem to be stacking the planes on top of each other, leading me to attempt this
int width = base->width;
int height = base->height;
int chroma2h = desc->log2_chroma_h;
int linesizeY = base->linesize[0];
int linesizeU = base->linesize[1];
int linesizeV = base->linesize[2];
int chromaHeight = (height + (1 << chroma2h) -1) >> chroma2h;
int x = 100;
int y = 100;
uint8_t *vY = base;
uint8_t *vU = base +(linesizeY*height);
uint8_t *vV = base +((linesizeY*height) + (linesizeU*chromaHeight));
vY+= x + (y * linesizeY);
vU+= x + (y * linesizeU);
vV+= x + (y * linesizeV);
Using that, if I try to modify pixels from a range of 300,300-400,400 I get a small box darker than the rest of the video, along with horizontal stripes of darkness along the video. The original color is still there, so I think I'm still touching the Y plane on all 3 pointers.
How can I actually hit the pixels I want to hit?
Hey Guys i am currently trying to iterate through all pixels of an UIImage but the way i implemented it it takes sooo much time. So i thought it is the wrong way i implemented it.
This is my method how i get the RGBA Values of an Pixel :
+(NSArray*)getRGBAsFromImage:(UIImage*)image atX:(int)xx andY:(int)yy count:(int)count
{
// Initializing the result array
NSMutableArray *result = [NSMutableArray arrayWithCapacity:count];
// First get the image into your data buffer
CGImageRef imageRef = [image CGImage]; // creating an Instance of
NSUInteger width = CGImageGetWidth(imageRef); // Get width of our Image
NSUInteger height = CGImageGetHeight(imageRef); // Get height of our Image
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); // creating our colour Space
// Getting that raw Data out of an image
unsigned char *rawData = (unsigned char*) calloc(height * width * 4, sizeof(unsigned char));
NSUInteger bytesPerPixel = 4; // Bytes per pixel defined
NSUInteger bytesPerRow = bytesPerPixel * width; // Bytes per row
NSUInteger bitsPerComponent = 8; // Bytes per component
CGContextRef context = CGBitmapContextCreate(rawData, width, height,
bitsPerComponent, bytesPerRow, colorSpace,
kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace); // releasing the color space
CGContextDrawImage(context, CGRectMake(0, 0, width, height), imageRef);
CGContextRelease(context);
// Now your rawData contains the image data in the RGBA8888 pixel format.
int byteIndex = (bytesPerRow * yy) + xx * bytesPerPixel;
for (int ii = 0 ; ii < count ; ++ii)
{
CGFloat red = (rawData[byteIndex] * 1.0) / 255.0;
CGFloat green = (rawData[byteIndex + 1] * 1.0) / 255.0;
CGFloat blue = (rawData[byteIndex + 2] * 1.0) / 255.0;
CGFloat alpha = (rawData[byteIndex + 3] * 1.0) / 255.0;
byteIndex += 4;
UIColor *acolor = [UIColor colorWithRed:red green:green blue:blue alpha:alpha];
[result addObject:acolor];
}
free(rawData);
return result;
}
And this is the code how i parse through all the pixels :
for (NSUInteger y = 0 ; y < self.originalPictureWidth; y++) {
for (NSUInteger x = 0 ; x < self.originalPictureHeight; x++) {
NSArray * originalRGBA = [ComputerVisionHelperClass getRGBAsFromImage:self.originalPicture atX:(int)x andY:(int)y count:1];
NSArray * referenceRGBA = [ComputerVisionHelperClass getRGBAsFromImage:self.referencePicture atX:(int)referenceIndexX andY:(int)referenceIndexY count:1];
// Do something else ....
}
}
Is there a faster way of getting all RGBA values of an uiimage instance ?
For every pixel, you're generating a new copy of the image and then throwing it away. Yes, it would be much faster by just getting the data once and then processing on that byte array.
But it heavily depends on what is in "Do something else." There are many CoreImage and vImage functions that can do image processing very quickly, but you may need to approach the problem differently. It depends on what you're doing.