Previously, the swapchain setting was RGB32 format, which worked fine.
I want to receive yv12 data and render it.
So I changed swapChain format to DXGI_FORMAT_420_OPAQUE.
DXGI_SWAP_CHAIN_DESC1 swapChainDesc = { 0 };
swapChainDesc.Width = 0;
swapChainDesc.Height = 0;
swapChainDesc.Format = DXGI_FORMAT_420_OPAQUE;// DXGI_FORMAT_B8G8R8A8_UNORM
swapChainDesc.Stereo = false;
swapChainDesc.SampleDesc.Count = 1;
swapChainDesc.SampleDesc.Quality = 0;
swapChainDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
swapChainDesc.BufferCount = 2;
swapChainDesc.Scaling = DXGI_SCALING_STRETCH;
swapChainDesc.SwapEffect = DXGI_SWAP_EFFECT_DISCARD;
swapChainDesc.Flags = 0;
dxgiFactory->CreateSwapChainForHwnd(_direct3DDevice, _hwnd, &swapChainDesc, nullptr, nullptr, &_DXGISwapChain);
IDXGISurface *dxgiBackBuffer;
_DXGISwapChain->GetBuffer(0, IID_PPV_ARGS(&dxgiBackBuffer));
Then in dxgiBackBuffer I get a NullReferenceException.
I think that you can not create back buffer according to yv12 format.
Do you know if I'm wrong?
Supported swapchain formats are those listed as 'display out' in the DXGI hardware feature levels format tables. No Direct3D hardware feature level supports a YUV format for display out. You have to perform a YUV to RGB colorspace conversion in a shader somewhere.
Related
I am working on Visual Studio 12 (Windows 10 OS) and trying to display an image buffer using StretchDIBits(), but it's failing with return 0.
I don't have any header file for that buffer. I don't know how to draw the client area using only buffer data. If anyone knows, please share your thoughts.
I have added sample snippet below:
void DShow::set framebuffer(HDC hDC){
BITMAPINFO m_bi;
DWORD result = 0;
m_bi.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
m_bi.bmiHeader.biWidth = m_curResolutionWidth;
m_bi.bmiHeader.biHeight = m_curResolutionHeight;
m_bi.bmiHeader.biPlanes = (WORD)1;
m_bi.bmiHeader.biBitCount = (WORD)24;
m_bi.bmiHeader.biCompression = 0;
m_bi.bmiHeader.biSizeImage = (m_curResolutionWidth*m_curResolutionHeight)*(24/8);
m_bi.bmiHeader.biXPelsPerMeter = 0;
m_bi.bmiHeader.biYPelsPerMeter = 0;
m_bi.bmiHeader.biClrUsed = 0;
m_bi.bmiHeader.biClrImportant = 0;
result = StretchDIBits(hDC,
m_TaniaDestRect.left, m_TaniaDestRect.top,
m_TaniaDestRect.right - m_TaniaDestRect.left, m_TaniaDestRect.bottom - m_TaniaDestRect.top,
0, 0,
m_curResolutionWidth, abs(m_curResolutionHeight),
g_pBuffer, &m_bi,
DIB_RGB_COLORS, SRCCOPY);
}
I'm currently doing some tests to see if my app runs correctly on Retina Macs. I have installed Quartz Debug for this purpose and I'm currently running a scaled mode. My screen mode is now 960x540 but of course the physical size of the monitor is still Full HD, i.e. 1920x1080 pixels.
When querying the monitor database using CGGetActiveDisplayList() and then using CGDisplayBounds() on the individual monitors in the list, the returned monitor size is 960x540. This is what I expected because CGDisplayBounds() is said to use the global display coordinate space, not pixels.
To my surprise, however, CGDisplayPixelsWide() and CGDisplayPixelsHigh() also return 960x540, although they're explicitly said to return pixels so I'd expect them to return 1920x1080 instead. But they don't.
This leaves me wondering how can I retrieve the real physical resolution of the monitor instead of the scaled mode using the CGDisplay APIs, i.e. 1920x1080 instead of 960x540 in my case? Is there any way to get the scaling coefficient for a CGDisplay so that I can compute the real physical resolution on my own?
I know I can get this scaling coefficient using the backingScaleFactor method but this is only possible for NSScreen, how can I get the scaling coefficient for a CGDisplay?
You need to examine the mode of the display, not just the display itself. Use CGDisplayCopyDisplayMode() and then CGDisplayModeGetPixelWidth() and CGDisplayModeGetPixelHeight(). These last two are relatively newer functions and the documentation primarily exists in the headers.
And, of course, don't forget to CGDisplayModeRelease() the mode object.
From Ken's answer it is not obvious how you find the native mode(s). To do this, call CGDisplayModeGetIOFlags and choose from the modes that have the kDisplayModeNativeFlag set (see IOKit/IOGraphicsTypes.h, the value is 0x02000000).
const int kFlagNativeMode = 0x2000000; // see IOGraphicsTypes.h
const CGFloat kNoSize = 100000.0;
NSScreen *screen = NSScreen.mainScreen;
NSDictionary *desc = screen.deviceDescription;
unsigned int displayID = [[desc objectForKey:#"NSScreenNumber"] unsignedIntValue];
CGSize displaySizeMM = CGDisplayScreenSize(displayID);
CGSize nativeSize = CGSizeMake(kNoSize, kNoSize);
CFStringRef keys[1] = { kCGDisplayShowDuplicateLowResolutionModes };
CFBooleanRef values[1] = { kCFBooleanTrue };
CFDictionaryRef options = CFDictionaryCreate(kCFAllocatorDefault, (const void**)keys, (const void**)values, 1, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks );
CFArrayRef modes = CGDisplayCopyAllDisplayModes(displayID, options);
int n = CFArrayGetCount(modes);
for (int i = 0; i < n; i++) {
CGDisplayModeRef mode = (CGDisplayModeRef) CFArrayGetValueAtIndex(modes, i);
if (CGDisplayModeGetIOFlags(mode) & kFlagNativeMode) {
int w = CGDisplayModeGetWidth(mode);
// We get both high resolution (screen.backingScaleFactor > 1)
// and the "low" resolution, in CGFloat units. Since screen.frame
// is CGFloat units, we want the lowest native resolution.
if (w < nativeSize.width) {
nativeSize.width = w;
nativeSize.height = CGDisplayModeGetHeight(mode);
}
}
// printf("mode: %dx%d %f dpi 0x%x\n", (int)CGDisplayModeGetWidth(mode), (int)CGDisplayModeGetHeight(mode), CGDisplayModeGetWidth(mode) / displaySizeMM.width * 25.4, CGDisplayModeGetIOFlags(mode));
}
if (nativeSize.width == kNoSize) {
nativeSize = screen.frame.size;
}
CFRelease(modes);
CFRelease(options);
float scaleFactor = screen.frame.size.width / nativeSize.width;
I have an audio related app that has multichannel mixer to play m4a files at a time.
I'm using the AudioToolBox framework to stream audio, but on iOS9 the framework throws me exception in mixer rendering callback where i am streaming the audio files.
Interestingly apps compiled with the iOS9 SDK continue to stream the same file perfectly on iOS7/8 devices, but not iOS9.
Now i can't figure out if Apple broke something in iOS9, or we have the files encoded wrong on our end, but they play just fine on both iOS 7/8 but not 9.
Exception:
malloc: *** error for object 0x7fac74056e08: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
It works for all other formats does not give any exception or any kind of memory errors but does not work for m4a format which is very surprising.
Here is a code to load files which works for wav,aif etc formats but not for m4a:
- (void)loadFiles{
AVAudioFormat *clientFormat = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatFloat32
sampleRate:kGraphSampleRate
channels:1
interleaved:NO];
for (int i = 0; i < numFiles && i < maxBufs; i++) {
ExtAudioFileRef xafref = 0;
// open one of the two source files
OSStatus result = ExtAudioFileOpenURL(sourceURL[i], &xafref);
if (result || !xafref) {break; }
// get the file data format, this represents the file's actual data format
AudioStreamBasicDescription fileFormat;
UInt32 propSize = sizeof(fileFormat);
result = ExtAudioFileGetProperty(xafref, kExtAudioFileProperty_FileDataFormat, &propSize, &fileFormat);
if (result) { break; }
// set the client format - this is the format we want back from ExtAudioFile and corresponds to the format
// we will be providing to the input callback of the mixer, therefore the data type must be the same
double rateRatio = kGraphSampleRate / fileFormat.mSampleRate;
propSize = sizeof(AudioStreamBasicDescription);
result = ExtAudioFileSetProperty(xafref, kExtAudioFileProperty_ClientDataFormat, propSize, clientFormat.streamDescription);
if (result) { break; }
// get the file's length in sample frames
UInt64 numFrames = 0;
propSize = sizeof(numFrames);
result = ExtAudioFileGetProperty(xafref, kExtAudioFileProperty_FileLengthFrames, &propSize, &numFrames);
if (result) { break; }
if(i==metronomeBusIndex)
numFrames = (numFrames+6484)*4;
//numFrames = (numFrames * rateRatio); // account for any sample rate conversion
numFrames *= rateRatio;
// set up our buffer
mSoundBuffer[i].numFrames = (UInt32)numFrames;
mSoundBuffer[i].asbd = *(clientFormat.streamDescription);
UInt32 samples = (UInt32)numFrames * mSoundBuffer[i].asbd.mChannelsPerFrame;
mSoundBuffer[i].data = (Float32 *)calloc(samples, sizeof(Float32));
mSoundBuffer[i].sampleNum = 0;
// set up a AudioBufferList to read data into
AudioBufferList bufList;
bufList.mNumberBuffers = 1;
bufList.mBuffers[0].mNumberChannels = 1;
bufList.mBuffers[0].mData = mSoundBuffer[i].data;
bufList.mBuffers[0].mDataByteSize = samples * sizeof(Float32);
// perform a synchronous sequential read of the audio data out of the file into our allocated data buffer
UInt32 numPackets = (UInt32)numFrames;
result = ExtAudioFileRead(xafref, &numPackets, &bufList);
if (result) {
free(mSoundBuffer[i].data);
mSoundBuffer[i].data = 0;
}
// close the file and dispose the ExtAudioFileRef
ExtAudioFileDispose(xafref);
}
// [clientFormat release];
}
If anyone could point me in the right direction, how do i go about debugging the issue?
Do we need to re-encode our files in some specific way?
I tried it on iOS 9.1.beta3 yesterday and things seem to be back to normal.
Try it out. Let us know if it works out for you too.
I am using a recent windows (Jan 2011) ffmpeg build and trying to record video in H264. It is recording fine in MPEG4 using the following settings:
c->codec_id = CODEC_ID_MPEG4;
c->codec_type = AVMEDIA_TYPE_VIDEO;
c->width = VIDEO_WIDTH;
c->height = VIDEO_HEIGHT;
c->bit_rate = c->width * c->height * 4;
c->time_base.den = FRAME_RATE;
c->time_base.num = 1;
c->gop_size = 12;
c->pix_fmt = PIX_FMT_YUV420P;
Simply changing CODEC Id to H264 causes avcodec_open() to fail (-1). I found a list of possible settings How to encode h.264 with libavcodec/x264?. I have tried these, without setting pix_fmt, avcodec_open() still fails but if I additionally set c->pix_fmt = PIX_FMT_YUV420P; then I get a divide by zero exception.
I then came across a few posts on here that say I should set nothing (with exception of code_id, codec_type, width, height and perhaps bit_rate and pix_fmt) as the library now chooses the best settings itself. I have tried various combinations, still avcode_open() fails.
Does anyone have some advice on what to do or some settings that are current?
Thanks.
Here are one set of H264 settings which give the issue I describe:
static AVStream* AddVideoStream(AVFormatContext *pOutputFmtCtx,
int frameWidth, int frameHeight, int fps)
{
AVCodecContext* ctx;
AVStream* stream;
stream = av_new_stream(pOutputFmtCtx, 0);
if (!stream)
{
return NULL;
}
ctx = stream->codec;
ctx->codec_id = pOutputFmtCtx->oformat->video_codec; //CODEC_ID_H264
ctx->codec_type = AVMEDIA_TYPE_VIDEO;
ctx->width = frameWidth; //704
ctx->height = frameHeight; //576
ctx->bit_rate = frameWidth * frameHeight * 4;
ctx->coder_type = 1; // coder = 1
ctx->flags|=CODEC_FLAG_LOOP_FILTER; // flags=+loop
ctx->me_cmp|= 1; // cmp=+chroma, where CHROMA = 1
ctx->partitions|=X264_PART_I8X8+X264_PART_I4X4+X264_PART_P8X8+X264_PART_B8X8; // partitions=+parti8x8+parti4x4+partp8x8+partb8x8
ctx->me_method=ME_HEX; // me_method=hex
ctx->me_subpel_quality = 7; // subq=7
ctx->me_range = 16; // me_range=16
ctx->gop_size = 250; // g=250
ctx->keyint_min = 25; // keyint_min=25
ctx->scenechange_threshold = 40; // sc_threshold=40
ctx->i_quant_factor = 0.71; // i_qfactor=0.71
ctx->b_frame_strategy = 1; // b_strategy=1
ctx->qcompress = 0.6; // qcomp=0.6
ctx->qmin = 10; // qmin=10
ctx->qmax = 51; // qmax=51
ctx->max_qdiff = 4; // qdiff=4
ctx->max_b_frames = 3; // bf=3
ctx->refs = 3; // refs=3
ctx->directpred = 1; // directpred=1
ctx->trellis = 1; // trellis=1
ctx->flags2|=CODEC_FLAG2_BPYRAMID+CODEC_FLAG2_MIXED_REFS+CODEC_FLAG2_WPRED+CODEC_FLAG2_8X8DCT+CODEC_FLAG2_FASTPSKIP; // flags2=+bpyramid+mixed_refs+wpred+dct8x8+fastpskip
ctx->weighted_p_pred = 2; // wpredp=2
// libx264-main.ffpreset preset
ctx->flags2|=CODEC_FLAG2_8X8DCT;
ctx->flags2^=CODEC_FLAG2_8X8DCT; // flags2=-dct8x8
// if set this get divide by 0 error on avcodec_open()
// if don't set it get -1 error on avcodec_open()
//ctx->pix_fmt = PIX_FMT_YUV420P;
return stream;
}
In my experience you should give FFMPEG the least amount of information when initialising your codec as possible. This may seem counter intuitive but it means that FFMPEG will use it's default settings that are more likely to work than your own guesses. See what I would include below:
AVStream *stream;
m_video_codec = avcodec_find_encoder(AV_CODEC_ID_H264);
stream = avformat_new_stream(_outputCodec, m_video_codec);
ctx = stream->codec;
ctx->codec_id = m_fmt->video_codec;
ctx->bit_rate = m_AVIMOV_BPS; //Bits Per Second
ctx->width = m_AVIMOV_WIDTH; //Note Resolution must be a multiple of 2!!
ctx->height = m_AVIMOV_HEIGHT; //Note Resolution must be a multiple of 2!!
ctx->time_base.den = m_AVIMOV_FPS; //Frames per second
ctx->time_base.num = 1;
ctx->gop_size = m_AVIMOV_GOB; // Intra frames per x P frames
ctx->pix_fmt = AV_PIX_FMT_YUV420P;//Do not change this, H264 needs YUV format not RGB
As in previous answers, here is a working example of the FFMPEG library encoding RGB frames to a H264 video:
http://www.imc-store.com.au/Articles.asp?ID=276
An extra thought on your code though:
Have you called register all like below?
avcodec_register_all();
av_register_all();
If you don't call these two functions near the start of your code your subsequent calls to FFMPEG will fail and you'll most likely seg-fault.
Have a look at the linked example, I tested it on VC++2010 and it works perfectly.
I'm using D3DXSaveSurfaceToFile to save windowed Direct3D 9 surfaces to PNG, BMP and JPG files. There are no errors returned from the D3DXSaveSurfaceToFile call and all files open fine in Windows Photo Viewer and Paint. But they will not open in a higher end image editing program such as Paint Shop Pro or Photoshop. The error messages from these programs basically say that the file is corrupted. If I open the files in Paint and then save them in the same file format with a different file name, then they'll open fine in the other programs.
This leads me to believe that D3DXSaveSurfaceToFile is writing out non-standard versions of these file formats. Is there some way I can get this function to write out files that can be opened in programs like Photoshop without the intermediate step of resaving the files in Paint? Or is there another function I should be using that does a better job of saving a Direct3D surfaces to an image?
Take a look at the file in a image meta viewer. What does it tell you?
Unfortunately D3DXSaveSurfaceToFile() isn't the most stable (it's also exceptionally slow). Personally I do something like the below code. It works even on Anti-aliased displays by doing an offscreen render to take the screenshot then getting it into a buffer. It also supports only the most common of the pixel formats. Sorry for any errors in it, pulled it out of an app I used to work on.
You can then, in your code and probably in another thread, then convert said 'bitmap' to anything you like using a variety of different code.
void HandleScreenshot(IDirect3DDevice9* device)
{
DWORD tcHandleScreenshot = GetTickCount();
LPDIRECT3DSURFACE9 pd3dsBack = NULL;
LPDIRECT3DSURFACE9 pd3dsTemp = NULL;
// Grab the back buffer into a surface
if ( SUCCEEDED ( device->GetBackBuffer(0, 0, D3DBACKBUFFER_TYPE_MONO, &pd3dsBack) ))
{
D3DSURFACE_DESC desc;
pd3dsBack->GetDesc(&desc);
LPDIRECT3DSURFACE9 pd3dsCopy = NULL;
if (desc.MultiSampleType != D3DMULTISAMPLE_NONE)
{
if (SUCCEEDED(device->CreateRenderTarget(desc.Width, desc.Height, desc.Format, D3DMULTISAMPLE_NONE, 0, FALSE, &pd3dsCopy, NULL)))
{
if (SUCCEEDED(device->StretchRect(pd3dsBack, NULL, pd3dsCopy, NULL, D3DTEXF_NONE)))
{
pd3dsBack->Release();
pd3dsBack = pd3dsCopy;
}
else
{
pd3dsCopy->Release();
}
}
}
if (SUCCEEDED(device->CreateOffscreenPlainSurface(desc.Width, desc.Height, desc.Format, D3DPOOL_SYSTEMMEM, &pd3dsTemp, NULL)))
{
DWORD tmpTimeGRTD = GetTickCount();
if (SUCCEEDED(device->GetRenderTargetData(pd3dsBack, pd3dsTemp)))
{
D3DLOCKED_RECT lockedSrcRect;
if (SUCCEEDED(pd3dsTemp->LockRect(&lockedSrcRect, NULL, D3DLOCK_READONLY | D3DLOCK_NOSYSLOCK | D3DLOCK_NO_DIRTY_UPDATE)))
{
int nSize = desc.Width * desc.Height * 3;
BYTE* pixels = new BYTE[nSize +1];
int iSrcPitch = lockedSrcRect.Pitch;
BYTE* pSrcRow = (BYTE*)lockedSrcRect.pBits;
LPBYTE lpDest = pixels;
LPDWORD lpSrc;
switch (desc.Format)
{
case D3DFMT_A8R8G8B8:
case D3DFMT_X8R8G8B8:
for (int y = desc.Height - 1; y >= 0; y--)
{
lpSrc = reinterpret_cast<LPDWORD>(lockedSrcRect.pBits) + y * desc.Width;
for (unsigned int x = 0; x < desc.Width; x++)
{
*reinterpret_cast<LPDWORD>(lpDest) = *lpSrc;
lpSrc++; // increment source pointer by 1 DWORD
lpDest += 3; // increment destination pointer by 3 bytes
}
}
break;
default:
ZeroMemory(pixels, nSize);
}
pd3dsTemp->UnlockRect();
BITMAPINFOHEADER header;
header.biWidth = desc.Width;
header.biHeight = desc.Height;
header.biSizeImage = nSize;
header.biSize = sizeof(BITMAPINFOHEADER);
header.biPlanes = 1;
header.biBitCount = 3 * 8; // RGB
header.biCompression = 0;
header.biXPelsPerMeter = 0;
header.biYPelsPerMeter = 0;
header.biClrUsed = 0;
header.biClrImportant = 0;
BITMAPFILEHEADER bfh = {0};
bfh.bfType = 0x4d42;
bfh.bfOffBits = sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);
bfh.bfSize = bfh.bfOffBits + nSize;
unsigned int rough_size = sizeof(BITMAPINFOHEADER) + sizeof(BITMAPFILEHEADER) + nSize;
unsigned char* p = new unsigned char[rough_size]
memcpy(p, &bfh, sizeof(BITMAPFILEHEADER));
p += sizeof(BITMAPFILEHEADER);
memcpy(p, &header, sizeof(BITMAPINFOHEADER));
p += sizeof(BITMAPINFOHEADER);
memcpy(p, pixels, nSize);
delete [] pixels;
/**********************************************/
// p now has a full BMP file, write it out here
}
}
pd3dsTemp->Release();
}
pd3dsBack->Release();
}
}
Turns out that it was a combination of a bug in my code and Paint being more forgiving than Photoshop when it comes to reading files. The bug in my code caused the files to be saved with the wrong extension (i.e. Image.bmp was actually saved using D3DXIFF_JPG). When opening a file that contained a JPG image, but had a BMP extension, Photoshop just failed the file. I guess Paint worked since it ignored the file extension and just decoded the file contents.
Looking at a file in an image meta viewer helped me to see the problem.