How to DEBUG OpenGL a gray/black texture box? - debugging

I'm altering someone else's code. They used PNG's which are loaded via BufferedImage. I need to load a TGA instead, which is just simply a 18 byte header and BGR codes. I have the textures loaded and running, but I get a gray box instead of the texture. I don't even know how to DEBUG this.
Textures are loaded in a ByteBuffer:
final static int datasize = (WIDTH*HEIGHT*3) *2; // Double buffer size for OpenGL // not +18 no header
static ByteBuffer buffer = ByteBuffer.allocateDirect(datasize);
FileInputStream fin = new FileInputStream("/Volumes/RAMDisk/shot00021.tga");
FileChannel inc = fin.getChannel();
inc.position(18); // skip header
buffer.clear(); // prepare for read
int ret = inc.read(buffer);
fin.close();
I've followed this: [how-to-manage-memory-with-texture-in-opengl][1] ... because I am updating the texture once per frame, like video.
Called once:
GL11.glBindTexture(GL11.GL_TEXTURE_2D, textureID);
GL11.glTexParameteri(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_WRAP_S, GL11.GL_CLAMP);
GL11.glTexParameteri(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_WRAP_T, GL11.GL_CLAMP);
GL11.glTexParameteri(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_MAG_FILTER, GL11.GL_NEAREST);
GL11.glTexParameteri(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_MIN_FILTER, GL11.GL_NEAREST);
GL11.glTexImage2D(GL11.GL_TEXTURE_2D, 0, GL11.GL_RGB, width, height, 0, GL11.GL_RGB, GL11.GL_UNSIGNED_BYTE, (ByteBuffer) null);
assert(GL11.GL_NO_ERROR == GL11.glGetError());
Called repeatedly:
GL11.glBindTexture(GL11.GL_TEXTURE_2D, textureID);
GL11.glTexSubImage2D(GL11.GL_TEXTURE_2D, 0, 0, 0, width, height, GL11.GL_RGB, GL11.GL_UNSIGNED_BYTE, byteBuffer);
assert(GL11.GL_NO_ERROR == GL11.glGetError());
return textureID;
The render code hasn't changed and is based on:
GL11.glDrawArrays(GL11.GL_TRIANGLES, 0, this.vertexCount);

Make sure you set the texture sampling mode. Especially min filter: glTexParameteri ( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR). The default setting is mip mapped (GL_NEAREST_MIPMAP_LINEAR) so unless you upload mip maps you will get a white read result.
So either set the texture to no mip or generate them. One way to do that is to call glGenerateMipmap after the tex img call.
(see https://www.khronos.org/opengles/sdk/docs/man/xhtml/glTexParameter.xml).
It's a very common gl pitfall and something people just tend to know after getting bitten by it a few times.
There is no easy way to debug stuff like this. There are good gl debugging tools in for example xcode but they will not tell you about this case.

Debugging GPU code is always a hassle. I would bet my money on a big industry progress in this area as more companies discover the power of GPU. Until then; I'll share my two best GPU debugging friends:
1) Define a function to print OGL errors:
int printOglError(const char *file, int line)
{
/* Returns 1 if an OpenGL error occurred, 0 otherwise. */
GLenum glErr;
int retCode = 0;
glErr = glGetError();
while (glErr != GL_NO_ERROR) {
printf("glError in file %s # line %d: %s\n", file, line, gluErrorString(glErr));
retCode = 1;
glErr = glGetError();
}
return retCode;
}
#define printOpenGLError() printOglError(__FILE__, __LINE__)
And call it after your render draw calls (possible earlier errors will also show up):
GL11.glDrawArrays(GL11.GL_TRIANGLES, 0, this.vertexCount);
printOpenGLError();
This alerts if you make some invalid operations (which might just be your case) but you usually have to find where the error occurs by trial and error.
2) Check out gDEBugger, free software with tons of GPU memory information.
[Edit]:
I would also recommend using the opensource lib DevIL - its quite competent in loading various image formats.

Thanks to Felix, by not calling glTexSubImage2D (leaving the memory valid, but uninitialized) I noticed a remnant pattern left by the default memory. This indicated that the texture is being displayed, but the load is most likely the problem.
**UPDATE:
The, problem with the code above is essentially the buffer. The buffer is 1024*1024, but it is only partially filled in by the read, leaving the limit marker of the ByteBuffer at 2359296(1024*768*3) instead of 3145728(1024*1024*3). This gives the error:
Number of remaining buffer elements is must be ... at least ...
I thought that OpenGL needed space to return data, so I doubled the size of the buffer.
The buffer size is doubled to compensate for the error.
final static int datasize = (WIDTH*HEIGHT*3) *2; // Double buffer size for OpenGL // not +18 no header
This is wrong, what is needed is the flip() function (Big THANKS to Reto Koradi for the small hint to the buffer rewind) to put the ByteBuffer in read mode. Since the buffer is only semi-full, the OpenGL buffer check gives an error. The correct thing to do is not double the buffer size; use buffer.position(buffer.capacity()) to fill the buffer before doing a flip().
final static int datasize = (WIDTH*HEIGHT*3); // not +18 no header
buffer.clear(); // prepare for read
int ret = inc.read(buffer);
fin.close();
buffer.position(buffer.capacity()); // make sure buffer is completely FILLED!
buffer.flip(); // flip buffer to read mode
To figure this out, it is helpful to hardcode the memory of the buffer to make sure the OpenGL calls are working, isolating the load problem. Then when the OpenGL calls are correct, concentrate on the loading of the buffer. As suggested by Felix K, it is good to make sure one texture has been drawn correctly before calling glTexSubImage2D repeatedly.

Some ideas which might cause the issue:
Your texture is disposed somewhere. I don't know the whole code but I guess somewhere there is a glDeleteTextures and this could cause some issues if called at the wrong time.
Are the texture width and height powers of two? If not this might be an issue depending on your hardware. Old hardware sometimes won't support non-power of two images.
The texture parameters changed between the draw calls at some other point ( Make a debug check of the parameters with glGetTexParameter ).
There could be a loading issue when loading the next image ( edit: or even the first image ). Check if the first image is displayed without loading the next images. If so it must be one of the cases above.

Related

GDI: Original DC bitmap changes, and can't restore

I'm seeing an issue where I could not re-select the original bitmap on a DC, causing a memory leak. The pointer to the original bitmap stayed the same throughout the program, but the data (from CBitmap::GetBitmap) changes from monochrome to something else. I don't know when the bitmap actually changes, but something in the system is causing it.
CBitmap* cMyClass::mpOldBitmap;
CDC cMyClass::mCanvasDc;
CBitmap cMyClass::mCanvasBmp;
void cMyClass::Init()
{
// One-time initialization
CDC* pDc = GetDC();
mCanvasDc.CreateCompatibleDC(pDc);
mCanvasBmp.CreateCompatibleBitmap(pDc, 10, 10);
mpOldBitmap = mCanvasDc.SelectObject(&mCanvasBmp);
ReleaseDC(pDc);
BITMAP bitmap;
mpOldBitmap->GetBitmap(&bitmap); // A monochrome bitmap, as expected.
}
void cMyClass::Recreate(int newW, int newH)
{
// 1. Delete existing bitmap:
if (mpOldBitmap)
{
BITMAP bitmap;
mpOldBitmap->GetBitmap(&bitmap); // This is no longer the monochrome bitmap. It is 8bpp, with random size.
CBitmap* pCurrBmp = mCanvasDc.SelectObject(mpOldBitmap); // This fails (NULL). I can't de-select my bitmap.
mCanvasBmp.DeleteObject(); // This fails too, causing memory leak. Actually, it fails in CE6, but not in Win32. Regardless, both platforms will have a memory leak.
}
// 2. Recreate the bitmap with new size:
{
CDC* pDc = GetDC();
mCanvasBmp.CreateCompatibleBitmap(pDc, newW, newH);
ReleaseDC(pDc);
}
// 3. Finalize
mpOldBitmap = mCanvasDc.SelectObject(&mCanvasBmp);
}
Any known scenarios where this can happen?
Any debugging tips to break when the bitmap data changes?
Note: In code, I mentioned "this fails". I remove the assert on the returned values to make the code readable.
Edit: The solution that I am using to fix it is to use CDC:SaveDC and CDC::RestoreDC instead of stashing the pointer. The memory leak went away, and every GDI call passed. But I am still curious why the original code leaked. The pointer to the default bitmap, as far as I know, should be a default monochrome bitmap that is probably global in GDI world
Let's see the code from OP.
mpOldBitmap = mCanvasDc.SelectObject(mCanvasBmp);
Because mCanvasBmp is a CBitmap object (not a pointer to CBitmap), first is called the HGDIOBJ operator, then CDC::SelectObject(HGDIOBJ) which returns HGDIOBJ and not CBitmap*. This should give a conversion compiler error. If cast the returned value to CBitmap* is also wrong.
The correct way to get rid of problem is to pass a pointer.
mpOldBitmap = mCanvasDc.SelectObject(& mCanvasBmp);
This case will be called CDC::SelectObject(CBitmap* pBitmap) which returns a CBitmap*.
// I hope it's pretty clear. :)

Direct2D fails when drawing a single-channel bitmap

I'm an experienced programmer specialized in Computer Graphics, mainly using Direct3D 9.0c, OpenGL and general algorithms. Currently, I am evaluating Direct2D as rendering technology for a professional application dealing with medical image data. As for rendering, it is a x64 desktop application in windowed mode (not fullscreen).
Already with my very initial steps I struggle with a task I thought would be a no-brainer: Rendering a single-channel bitmap on screen.
Running on a Windows 8.1 machine, I create an ID2D1DeviceContext with a Direct3D swap chain buffer surface as render target. The swap chain is created from a HWND and buffer format DXGI_FORMAT_B8G8R8A8_UNORM. Note: See also the code snippets at the end.
Afterwards, I create a bitmap with pixel format DXGI_FORMAT_R8_UNORM and alpha mode D2d1_ALPHA_MODE_IGNORE. When calling DrawBitmap(...) on the device context, a debug break point is triggered with the debug message "D2d DEBUG ERROR - This operation is not compatible with the pixel format of the bitmap".
I know that this output is quite clear. Also, when changing the pixel format to DXGI_FORMAT_R8G8B8A8_UNORM with DXGI_ALPHA_MODE_IGNORE everything works well and I see the bitmap rendered. However, I simply cannot believe that! Graphics cards support single-channel textures ever since - every 3D graphics application can use them without thinking twice. This goes without speaking.
I tried to find anything here and at Google, without success. The only hint I could find was the MSDN Direct2D page with the (supported pixel formats). The documentation suggests - by not mentioning it - that DXGI_FORMAT_R8_UNORM is indeed not supported as bitmap format. I also find posts talking about alpha masks (using DXGI_FORMAT_A8_UNORM), but that's not what I'm after.
What am I missing that I can't convince Direct2D to create and draw a grayscale bitmap? Or is it really true that Direct2D doesn't support drawing of R8 or R16 bitmaps??
Any help is really appreciated as I don't know how to solve this. If I can't get this trivial basics to work, I think I'd have to stop digging deeper into Direct2D :-(.
And here is the code snippets of relevance. Please note that they might not compile since I ported this on the fly from my C++/CLI code to plain C++. Also, I threw away all error checking and other noise:
Device, Device Context and Swap Chain Creation (D3D and Direct2D):
// Direct2D factory creation
D2D1_FACTORY_OPTIONS options = {};
options.debugLevel = D2D1_DEBUG_LEVEL_INFORMATION;
ID2D1Factory1* d2dFactory;
D2D1CreateFactory(D2D1_FACTORY_TYPE_MULTI_THREADED, options, &d2dFactory);
// Direct3D device creation
const auto type = D3D_DRIVER_TYPE_HARDWARE;
const auto flags = D3D11_CREATE_DEVICE_BGRA_SUPPORT;
ID3D11Device* d3dDevice;
D3D11CreateDevice(nullptr, type, nullptr, flags, nullptr, 0, D3D11_SDK_VERSION, &d3dDevice, nullptr, nullptr);
// Direct2D device creation
IDXGIDevice* dxgiDevice;
d3dDevice->QueryInterface(__uuidof(IDXGIDevice), reinterpret_cast<void**>(&dxgiDevice));
ID2D1Device* d2dDevice;
d2dFactory->CreateDevice(dxgiDevice, &d2dDevice);
// Swap chain creation
DXGI_SWAP_CHAIN_DESC1 desc = {};
desc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
desc.BufferCount = 2;
IDXGIAdapter* dxgiAdapter;
dxgiDevice->GetAdapter(&dxgiAdapter);
IDXGIFactory2* dxgiFactory;
dxgiAdapter->GetParent(__uuidof(IDXGIFactory), reinterpret_cast<void **>(&dxgiFactory));
IDXGISwapChain1* swapChain;
dxgiFactory->CreateSwapChainForHwnd(d3dDevice, hwnd, &swapChainDesc, nullptr, nullptr, &swapChain);
// Direct2D device context creation
const auto options = D2D1_DEVICE_CONTEXT_OPTIONS_NONE;
ID2D1DeviceContext* deviceContext;
d2dDevice->CreateDeviceContext(options, &deviceContext);
// create render target bitmap from swap chain
IDXGISurface* swapChainSurface;
swapChain->GetBuffer(0, __uuidof(swapChainSurface), reinterpret_cast<void **>(&swapChainSurface));
D2D1_BITMAP_PROPERTIES1 bitmapProperties;
bitmapProperties.dpiX = 0.0f;
bitmapProperties.dpiY = 0.0f;
bitmapProperties.bitmapOptions = D2D1_BITMAP_OPTIONS_TARGET | D2D1_BITMAP_OPTIONS_CANNOT_DRAW;
bitmapProperties.pixelFormat.format = DXGI_FORMAT_B8G8R8A8_UNORM;
bitmapProperties.pixelFormat.alphaMode = D2D1_ALPHA_MODE_IGNORE;
bitmapProperties.colorContext = nullptr;
ID2D1Bitmap1* swapChainBitmap = nullptr;
deviceContext->CreateBitmapFromDxgiSurface(swapChainSurface, &bitmapProperties, &swapChainBitmap);
// set swap chain bitmap as render target of D2D device context
deviceContext->SetTarget(swapChainBitmap);
D2D single-channel Bitmap Creation:
const D2D1_SIZE_U size = { 512, 512 };
const UINT32 pitch = 512;
D2D1_BITMAP_PROPERTIES1 d2dProperties;
ZeroMemory(&d2dProperties, sizeof(D2D1_BITMAP_PROPERTIES1));
d2dProperties.pixelFormat.alphaMode = D2D1_ALPHA_MODE_IGNORE;
d2dProperties.pixelFormat.format = DXGI_FORMAT_R8_UNORM;
char* sourceData = new char[512*512];
ID2D1Bitmap1* d2dBitmap;
deviceContext->DeviceContextPointer->CreateBitmap(size, sourceData, pitch, d2dProperties, &d2dBitmap);
Bitmap drawing (FAILING):
deviceContext->BeginDraw();
D2D1_COLOR_F d2dColor = {};
deviceContext->Clear(d2dColor);
// THIS LINE FAILS WITH THE DEBUG BREAKPOINT IF SINGLE CHANNELED
deviceContext->DrawBitmap(bitmap, nullptr, 1.0f, D2D1_INTERPOLATION_MODE_LINEAR, nullptr);
swapChain->Present(1, 0);
deviceContext->EndDraw();
From my little experience, Direct2D seems very limited, indeed.
Have you tried Direct2D effects (ID2D1Effect)? You can write your own [it seems comparatively complicated], or use one of the built-in effects [which is rather simple].
There is one called Color matrix effect (CLSID_D2D1ColorMatrix). It might work to have your DXGI_FORMAT_R8_UNORM (or DXGI_FORMAT_A8_UNORM, any single-channel would do) as input (inputs to effects are ID2D1Image, and ID2D1Bitmap inherits from ID2D1Image). Then set the D2D1_COLORMATRIX_PROP_COLOR_MATRIX for copying the input channel to all output channels. Have not tried it, though.

Direct2D API calls stall at specific intervals

I am working on migrating the drawing code of an application from GDI/GDI+ to Direct2D. So far things have been going well - however, while testing the new code, I have noticed some bizarre performance. The flow of execution I have been investigating is as follows (I have done my best to remove irrelevant code):
Create D2D Factory (on creation of app)
HRESULT hr = S_OK;
hr = D2D1CreateFactory(D2D1_FACTORY_TYPE_MULTI_THREADED, &m_pD2DFactory);
if (hr == S_FALSE) {
ASSERT(FALSE);
throw Exception(CExtString(_T("Failed to create Direct2D factory")));
}
OnDraw Callback
HWND hwnd = GetSafeHwnd();
RECT rc;
GetClientRect(&rc);
D2D1_SIZE_U size = D2D1::SizeU(rc.right - rc.left, rc.bottom - rc.top);
// Create a render target if it has been destroyed
if (!m_pRT) {
D2D1_RENDER_TARGET_PROPERTIES props = D2D1::RenderTargetProperties(
D2D1_RENDER_TARGET_TYPE_DEFAULT,
D2D1::PixelFormat(
DXGI_FORMAT_B8G8R8A8_UNORM,
D2D1_ALPHA_MODE_IGNORE),
0,
0,
D2D1_RENDER_TARGET_USAGE_NONE,
D2D1_FEATURE_LEVEL_DEFAULT);
GetD2DFactory()->CreateHwndRenderTarget(props,
D2D1::HwndRenderTargetProperties(hwnd, size),
&m_pRT);
}
m_pRT->Resize(size);
m_pRT->BeginDraw();
// Begin drawing the layers, given the
// transformation matrix and some geometric information
Draw(m_pRT, matrixD2D, rectClipWorld, rectClipDP);
HRESULT hr = m_pRT->EndDraw();
if (hr == D2DERR_RECREATE_TARGET) {
SafeRelease(m_pRT);
}
The contents of the Draw method
The draw method does a lot of fluff that is largely irrelevant to this test (as I have turned all extraneous layers off), but it eventually draws a layer that executes this method several thousand times:
void DrawStringWithEffects(ID2D1RenderTarget* m_pRT, const CString& text, const D2D1_POINT_2F& point, const COLORREF rgbFore, const COLORREF rgbBack, IDWriteTextFormat* pfont) {
// The text will be vertically centered around point.y, with point.x on the left hand side
// Create a TextLayout for the string
IDWriteTextLayout* textLayout = NULL;
GetDWriteFactory()->CreateTextLayout(text,
text.GetLength(),
pfont,
std::numeric_limits<float>::infinity(),
std::numeric_limits<float>::infinity(),
&textLayout);
DWRITE_TEXT_METRICS metrics = {0};
textLayout->GetMetrics(&metrics);
D2D1_RECT_F rect = D2D1::RectF(point.x, point.y - metrics.height/2, point.x + metrics.width, point.y + metrics.height/2);
D2D1_POINT_2F pointDraw = point;
pointDraw.y -= metrics.height/2;
ID2D1SolidColorBrush* brush = NULL;
m_pRT->CreateSolidColorBrush(ColorD2DFromCOLORREF(rgbBack), &brush);
m_pRT->FillRectangle(rect, brush);
// ^^ this is sometimes very slow!
brush->SetColor(ColorD2DFromCOLORREF(rgbFore));
m_pRT->DrawTextLayout(pointDraw, textLayout, brush, D2D1_DRAW_TEXT_OPTIONS_NONE);
// ^^ this is also sometimes very slow!
SafeRelease(&brush);
SafeRelease(&textLayout);
The vast majority of the time, the Direct2D calls are executing ~3-4 times faster than the GDI+ equivalents, which is great (generally 0.1ms compared to ~0.35ms). For some reason, though, the function calls will occasionally stall for a long period of time - upwards of 200ms combined. The offending calls are straight from the Direct2D API - FillRectangle and DrawTextLayout. Strangely, these stalls appear in the same location every time I run the application - the 73rd occurrence of the loop, then the 218th, then the 290th and so on (there is somewhat of a pattern in the differences, alternating between every ~73rd and every ~145th cycle). This is independent of the data that it draws (when I told it to skip drawing the 73rd cycle, the next cycle simply becomes the 73rd and thus stalls).
I thought this may be a GPU/CPU communication issue, so I set the render target (I am using an HWnd target) to software mode (D2D1_RENDER_TARGET_TYPE_SOFTWARE), and the results were even more strange. The stall times dropped from ~200ms to ~20ms (still not great, but hey), but there were two instances that stalled for over 2500ms! (These two, like the rest of the stalls, are completely reproducible in terms of being the n'th API call).
This is rather frustrating, as 99% of the loop is several times faster than the old implementation, but the (less than) 1% remaining hang for an abnormally long time.
To any Direct2D experts out there - what type of problem might this stalling be a symptom of? What, in general, could be causing this disconnect between my code and what D2D is doing in the background?
Direct2D buffers drawing commands (presumably to optimize them). You can't look at the performance of an individual drawing command, you must look at the total time between BeginDraw() and EndDraw(). If you want to force each drawing command to execute immediately, you must follow each one with a call to Flush(). That's probably a bad idea for performance though.
https://msdn.microsoft.com/en-us/library/windows/desktop/dd371768(v=vs.85).aspx
After BeginDraw is called, a render target will normally build up a
batch of rendering commands, but defer processing of these commands
until either an internal buffer is full, the Flush method is called,
or until EndDraw is called.

glTexSubImage2D with GL_PIXEL_UNPACK_BUFFER gives GL_INVALID_OPERATION

Currently I am attempting to use PBOs to get video data to textures. I'm not sure if what I'm trying to do is even possible, or a good way to do it if it IS possible... I have 3 textures with the GL_RED format (one for each channel, not using Alpha currently). All three of these will be filled out in a single call to an external library.
Here's binding the buffer, etc:
void LockTexture(const TextureID& id, void ** ppbData)
{
Texture& tex = textures.getArray()[id];
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, tex.glBufID);
glBufferData(GL_PIXEL_UNPACK_BUFFER, tex.width * tex.height, NULL, GL_STREAM_DRAW);
*ppbData = glMapBuffer(GL_PIXEL_UNPACK_BUFFER, GL_WRITE_ONLY);
}
This is done for the 3 textures, the buffers are then filled by the external library. Then I attempt to push them to the texture, like so:
void UnlockTexture(const TextureID& id)
{
Texture& tex = textures.getArray()[id];
glUnmapBuffer(GL_PIXEL_UNPACK_BUFFER);
glBindTexture(tex.glTarget, tex.glTexID);
glCheckForErrors(); // <--- NO ERROR
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, tex.width, tex.height, GL_RED, GL_UNSIGNED_BYTE, 0);
glCheckForErrors(); // <--- ERROR
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, 0);
glBindTexture(tex.glTarget, 0);
}
Going through the list of reasons the error could be generated, this is what I know:
texture array has been defined
type is correct
data param (offset) is good at 0
not executed between glBegin/glEnd
This one I'm not sure about:
error is generated if a non-zero buffer object name is bound to the GL_PIXEL_UNPACK_BUFFER target and the data would be unpacked from the buffer object such that the memory reads required would exceed the data store size.
This one seems like it could be an issue, but I'd have no idea how else to handle this:
error is generated if a non-zero buffer object name is bound to the GL_PIXEL_UNPACK_BUFFER target and the buffer object's data store is currently mapped.
Am I correct in saying that this glUnmapBuffer is unmapping the last-mapped buffer, so the correct buffer is still mapped?
GL version is 3.2
I would greatly appreciate any help on this one, thanks!
glUnmapBuffer(target) will unmap the buffer which is currently bound to target. From the code you posted, it is unclear if there will still be the same binding as at the time you did the map call. Your wordings suggests that you do the mapping for all 3 right after each other, and when you try to unmap it, you only unmap the last one mapped because you forget to rebind the other ones, which would lead to this error for the first two of your textures.

Getting PIX_FMT_YUYV422 out of libswscale

I'm trying to learn to use the different ffmpeg libs with Cocoa, and I'm trying to get frames to display with help of Core Video. It seems I have gotten the CV callbacks to work, and it gets frames which I try to put in a CVImageBufferRef that I later draw with Core Image.
The problem is I'm trying to get PIX_FMT_YUYV422 to work with libswscale, but as soon as I change the pixel format to anything other than PIX_FMT_YUV420P it crashes with EXC_BAD_ACCESS.
As long as I use YUV420P the program runs, allthough it doesn't display properly. I suspected that the pixel format isn't supported, so I wanted to try PIX_FMT_YUYV422.
I have had it running before and successfully wrote PPM files with PIX_FMT_RGB24. For some reason it just crashes on me now, and I don't see what might be wrong.
I'm a bit in over my head here, but that is how I prefer to learn. :)
Here's how I allocate the AVFrames:
inFrame = avcodec_alloc_frame();
outFrame = avcodec_alloc_frame();
int frameBytes = avpicture_get_size(PIX_FMT_YUYV422, cdcCtx->width, cdcCtx->height);
uint8_t *frameBuffer = malloc(frameBytes);
avpicture_fill((AVPicture *)outFrame, frameBuffer, PIX_FMT_YUYV422, cdcCtx->width, cdcCtx->height);
Then I try to run it through swscale like so:
static struct SwsContext *convertContext;
if (convertContext == NULL) {
int w = cdcCtx->width;
int h = cdcCtx->height;
convertContext = sws_getContext(w, h, cdcCtx->pix_fmt, outWidth, outHeight, PIX_FMT_YUYV422, SWS_BICUBIC, NULL, NULL, NULL);
if (convertContext == NULL) {
NSLog(#"Cannot initialize the conversion context!");
return NO;
}
}
sws_scale(convertContext, inFrame->data, inFrame->linesize, 0, outHeight, outFrame->data, outFrame->linesize);
And finally I try to write it to a pixel buffer for use with Core Image:
int ret = CVPixelBufferCreateWithBytes(0, outWidth, outHeight, kYUVSPixelFormat, outFrame->data[0], outFrame->linesize[0], 0, 0, 0, &currentFrame);
With 420P it runs, but it doesnt match up with the kYUVSPixelformat for the pixel buffer, and as I understand it doesnt accept YUV420.
I would really appreciate any help, no matter how small, as it might help me struggle on. :)
This certainly isn't a complete code sample, since you never decode anything into the input frame. If you were to do that, it looks correct.
You also don't need to fill the output picture, or even allocate an AVFrame for it, really.
YUV420P is a planar format. Therefore, AVFrame.data[0] is not the whole story. I see a mistake in
int ret = CVPixelBufferCreateWithBytes(0, outWidth, outHeight, kYUVSPixelFormat, outFrame->data[0], outFrame->linesize[0], 0, 0, 0, &currentFrame);
For planar formats, you will have to read data blocks from AVFrame.data[0] up to AVFrame.data[3]

Resources