Extending Media Foundation Encoder to support 10-bit video encoding - winapi

Question

The HEVC Media Foundation Encoder in Windows will only encode 8-bit video. My GFX NVidia card also supports 10-bit HDR and alpha-mode video encoding, so I decided to create my own IMFTransform to use the NVidia SDK.
I 've registered my DLL using MFTRegister:
MFT_REGISTER_TYPE_INFO aMediaTypesIn[] =
{
{ MFMediaType_Video, MFVideoFormat_ARGB32 },
{ MFMediaType_Video, MFVideoFormat_RGB32 },
{ MFMediaType_Video, MFVideoFormat_RGB32 },
{ MFMediaType_Video, MFVideoFormat_RGB10 },
{ MFMediaType_Video, MyFakeFmt },
};
MFT_REGISTER_TYPE_INFO aMediaTypesOut[] =
{
{ MFMediaType_Video, MFVideoFormat_H264 },
{ MFMediaType_Video, MFVideoFormat_H265 },
{ MFMediaType_Video, MFVideoFormat_HEVC },
};
// Size of the array.
const DWORD cNumMediaTypesI = ARRAY_SIZE(aMediaTypesIn);
const DWORD cNumMediaTypesO = ARRAY_SIZE(aMediaTypesOut);
hr = MFTRegister(
GUID_NVidiaEncoder, // CLSID.
MFT_CATEGORY_VIDEO_ENCODER, // Category.
The MyFakeFmt is a non existing input type to fool the Sink Writer to pick my encoder when calling SetInputMediaType instead of the predefined Microsoft's transform. This works OK.
int wi = 1920;
int he = 1080;
int fps = 30;
int br = 4000;
auto fmt = MFVideoFormat_H264;
bool Our = 1;
const wchar_t* fil = L"r:\\1.mp4";
std::vector<DWORD> frame;
frame.resize(wi * he);
// Test
CComPtr<IMFSinkWriter> wr;
DeleteFile(fil);
CComPtr<IMFAttributes> attrs;
MFCreateAttributes(&attrs, 0);
auto hr = MFCreateSinkWriterFromURL(fil, 0, attrs, &wr);
DWORD str = (DWORD)-1;
CComPtr<IMFMediaType> mt2;
MFCreateMediaType(&mt2);
mt2->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);
mt2->SetGUID(MF_MT_SUBTYPE, fmt);
MFSetAttributeRatio(mt2, MF_MT_FRAME_RATE, fps, 1);
hr = MFSetAttributeSize(mt2, MF_MT_FRAME_SIZE,wi, he);
MFSetAttributeRatio(mt2, MF_MT_PIXEL_ASPECT_RATIO, 1, 1);
mt2->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive);
mt2->SetUINT32(MF_MT_VIDEO_NOMINAL_RANGE, MFNominalRange_Normal);
mt2->SetUINT32(MF_MT_AVG_BITRATE, br*1000);
hr = wr->AddStream(mt2, &str);
CComPtr<IMFMediaType> mt1;
MFCreateMediaType(&mt1);
mt1->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);
mt1->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_ARGB32);
hr = MFSetAttributeSize(mt1, MF_MT_FRAME_SIZE, wi, he);
// Force our selection
if (Our)
{
mt1->SetGUID(MF_MT_SUBTYPE, MyFakeFmt);
hr = wr->SetInputMediaType(str, mt1, 0);
}
mt1->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_ARGB32);
hr = wr->SetInputMediaType(str, mt1, 0);
hr = wr->BeginWriting();
for(int i = 0 ; i < 15 ; i++)
{
auto i2 = i % 5;
if (i2 == 0) Frm(frame, wi, he, 0xFFFFFFFF);
if (i2 == 1 || i2 == 4) Frm(frame, wi, he, 0xFF0000FF); // some colors
if (i2 == 2) Frm(frame, wi, he, 0xFFFF00FF); //
if (i2 == 3) Frm(frame, wi, he, 0xFF00FFFF); //
CComPtr<IMFSample> s;
MFCreateSample(&s);
int secs = 1;
hr = s->SetSampleDuration(10 * 1000 * 1000 * secs);
hr = s->SetSampleTime(10 * 1000 * 1000 * i);
CComPtr<IMFMediaBuffer> b;
MFCreateMemoryBuffer((DWORD)(frame.size() * 4), &b);
b->SetCurrentLength((DWORD)(frame.size() * 4));
BYTE* by = 0;
DWORD ml = 0, cl = 0;
b->Lock(&by, &ml, &cl);
memcpy(by, frame.data(), frame.size() * 4);
b->Unlock();
hr = s->AddBuffer(b);
b = 0;
hr = wr->WriteSample(str, s);
}
hr = wr->Finalize();
wr = 0;
The problems start with the call to Finalize to end the writing. At that point, everything seems to work normally. Note that I have tested the NVidia IMFTransform I 've created with input frames and it encodes and outputs them correctly as raw data.
When I call Finalize and the type is MFVideoFormat_H264 , the call succeeds. However the generated mp4 plays weirdly:
For some reason also, MediaInfo shows 1 FPS. Why?
When the output is MFVideoFormat_HEVC, then Finalize fails with `0xc00d4a45 : Sink could not create valid output file because required headers were not provided to the sink.'.
I 've also tried to convert the raw .h264 file I 'm saving with ffmpeg to mp4, and this works. The mp4 generated plays correctly.
Adding a MF_MT_MPEG_SEQUENCE_HEADER didn't help (besides, I think this is only needed for H.264)
const char* bl4 = "\x00\x00\x00\x01\x67\x42\xC0\x28\x95\xB0\x1E\x00\x89\xF9\x70\x16\xC8\x00\x00\x03\x00\x08\x00\x00\x03\x01\xE0\x6D\x04\x42\x37\x00\x00\x00\x01\x68\xCA\x8F\x20";
mt2->SetBlob(MF_MT_MPEG_SEQUENCE_HEADER, (UINT8*)bl4, 39);
What do you make of all that?
Thanks