Currently using Direct3D11CaptureFramePool setup with a non default dx11 device to capture the window of another process and trying to encode them to a file using MediaTranscoder. However I cant seem to find an API to set what ID3D11Device* the MediaTranscoder should use (like you can configure with Direct3D11CaptureFramePool).
Currently it blows up when passing the MediaTranscoder the first frame in DX11 trying to open the shared texture on the wrong device.
Am I missing something or is this even possible?
FramePoolSetup:
winrt::com_ptr<ID3D11Device> d3dDevice = CreateDeviceOnNonDefaultAdapter();
auto dxgiDevice = d3dDevice.as<IDXGIDevice1>();
winrt::IDirect3DDevice direct3Ddevice = CreateDirect3DDevice(dxgiDevice.get());
m_framePool = winrt::Direct3D11CaptureFramePool::Create(direct3Ddevice, winrt::DirectXPixelFormat::B8G8R8A8UIntNormalized, 1, size);
EncoderSetup:
auto transcoder = winrt::MediaTranscoder();
transcoder.HardwareAccelerationEnabled(true);
auto streamSource = winrt::MediaStreamSource(videoDescriptor);
// setup stream source ...
auto encodingProfile = winrt::MediaEncodingProfile();
// setup encoding profile ...
auto outStream = co_await OpenFile("C:\out.mp4");
auto res = co_await transcoder.PrepareMediaStreamSourceTranscodeAsync(streamSource, outStream, encodingProfile);
co_await res.TranscodeAsync();
Related
I'm an experienced programmer specialized in Computer Graphics, mainly using Direct3D 9.0c, OpenGL and general algorithms. Currently, I am evaluating Direct2D as rendering technology for a professional application dealing with medical image data. As for rendering, it is a x64 desktop application in windowed mode (not fullscreen).
Already with my very initial steps I struggle with a task I thought would be a no-brainer: Rendering a single-channel bitmap on screen.
Running on a Windows 8.1 machine, I create an ID2D1DeviceContext with a Direct3D swap chain buffer surface as render target. The swap chain is created from a HWND and buffer format DXGI_FORMAT_B8G8R8A8_UNORM. Note: See also the code snippets at the end.
Afterwards, I create a bitmap with pixel format DXGI_FORMAT_R8_UNORM and alpha mode D2d1_ALPHA_MODE_IGNORE. When calling DrawBitmap(...) on the device context, a debug break point is triggered with the debug message "D2d DEBUG ERROR - This operation is not compatible with the pixel format of the bitmap".
I know that this output is quite clear. Also, when changing the pixel format to DXGI_FORMAT_R8G8B8A8_UNORM with DXGI_ALPHA_MODE_IGNORE everything works well and I see the bitmap rendered. However, I simply cannot believe that! Graphics cards support single-channel textures ever since - every 3D graphics application can use them without thinking twice. This goes without speaking.
I tried to find anything here and at Google, without success. The only hint I could find was the MSDN Direct2D page with the (supported pixel formats). The documentation suggests - by not mentioning it - that DXGI_FORMAT_R8_UNORM is indeed not supported as bitmap format. I also find posts talking about alpha masks (using DXGI_FORMAT_A8_UNORM), but that's not what I'm after.
What am I missing that I can't convince Direct2D to create and draw a grayscale bitmap? Or is it really true that Direct2D doesn't support drawing of R8 or R16 bitmaps??
Any help is really appreciated as I don't know how to solve this. If I can't get this trivial basics to work, I think I'd have to stop digging deeper into Direct2D :-(.
And here is the code snippets of relevance. Please note that they might not compile since I ported this on the fly from my C++/CLI code to plain C++. Also, I threw away all error checking and other noise:
Device, Device Context and Swap Chain Creation (D3D and Direct2D):
// Direct2D factory creation
D2D1_FACTORY_OPTIONS options = {};
options.debugLevel = D2D1_DEBUG_LEVEL_INFORMATION;
ID2D1Factory1* d2dFactory;
D2D1CreateFactory(D2D1_FACTORY_TYPE_MULTI_THREADED, options, &d2dFactory);
// Direct3D device creation
const auto type = D3D_DRIVER_TYPE_HARDWARE;
const auto flags = D3D11_CREATE_DEVICE_BGRA_SUPPORT;
ID3D11Device* d3dDevice;
D3D11CreateDevice(nullptr, type, nullptr, flags, nullptr, 0, D3D11_SDK_VERSION, &d3dDevice, nullptr, nullptr);
// Direct2D device creation
IDXGIDevice* dxgiDevice;
d3dDevice->QueryInterface(__uuidof(IDXGIDevice), reinterpret_cast<void**>(&dxgiDevice));
ID2D1Device* d2dDevice;
d2dFactory->CreateDevice(dxgiDevice, &d2dDevice);
// Swap chain creation
DXGI_SWAP_CHAIN_DESC1 desc = {};
desc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
desc.BufferCount = 2;
IDXGIAdapter* dxgiAdapter;
dxgiDevice->GetAdapter(&dxgiAdapter);
IDXGIFactory2* dxgiFactory;
dxgiAdapter->GetParent(__uuidof(IDXGIFactory), reinterpret_cast<void **>(&dxgiFactory));
IDXGISwapChain1* swapChain;
dxgiFactory->CreateSwapChainForHwnd(d3dDevice, hwnd, &swapChainDesc, nullptr, nullptr, &swapChain);
// Direct2D device context creation
const auto options = D2D1_DEVICE_CONTEXT_OPTIONS_NONE;
ID2D1DeviceContext* deviceContext;
d2dDevice->CreateDeviceContext(options, &deviceContext);
// create render target bitmap from swap chain
IDXGISurface* swapChainSurface;
swapChain->GetBuffer(0, __uuidof(swapChainSurface), reinterpret_cast<void **>(&swapChainSurface));
D2D1_BITMAP_PROPERTIES1 bitmapProperties;
bitmapProperties.dpiX = 0.0f;
bitmapProperties.dpiY = 0.0f;
bitmapProperties.bitmapOptions = D2D1_BITMAP_OPTIONS_TARGET | D2D1_BITMAP_OPTIONS_CANNOT_DRAW;
bitmapProperties.pixelFormat.format = DXGI_FORMAT_B8G8R8A8_UNORM;
bitmapProperties.pixelFormat.alphaMode = D2D1_ALPHA_MODE_IGNORE;
bitmapProperties.colorContext = nullptr;
ID2D1Bitmap1* swapChainBitmap = nullptr;
deviceContext->CreateBitmapFromDxgiSurface(swapChainSurface, &bitmapProperties, &swapChainBitmap);
// set swap chain bitmap as render target of D2D device context
deviceContext->SetTarget(swapChainBitmap);
D2D single-channel Bitmap Creation:
const D2D1_SIZE_U size = { 512, 512 };
const UINT32 pitch = 512;
D2D1_BITMAP_PROPERTIES1 d2dProperties;
ZeroMemory(&d2dProperties, sizeof(D2D1_BITMAP_PROPERTIES1));
d2dProperties.pixelFormat.alphaMode = D2D1_ALPHA_MODE_IGNORE;
d2dProperties.pixelFormat.format = DXGI_FORMAT_R8_UNORM;
char* sourceData = new char[512*512];
ID2D1Bitmap1* d2dBitmap;
deviceContext->DeviceContextPointer->CreateBitmap(size, sourceData, pitch, d2dProperties, &d2dBitmap);
Bitmap drawing (FAILING):
deviceContext->BeginDraw();
D2D1_COLOR_F d2dColor = {};
deviceContext->Clear(d2dColor);
// THIS LINE FAILS WITH THE DEBUG BREAKPOINT IF SINGLE CHANNELED
deviceContext->DrawBitmap(bitmap, nullptr, 1.0f, D2D1_INTERPOLATION_MODE_LINEAR, nullptr);
swapChain->Present(1, 0);
deviceContext->EndDraw();
From my little experience, Direct2D seems very limited, indeed.
Have you tried Direct2D effects (ID2D1Effect)? You can write your own [it seems comparatively complicated], or use one of the built-in effects [which is rather simple].
There is one called Color matrix effect (CLSID_D2D1ColorMatrix). It might work to have your DXGI_FORMAT_R8_UNORM (or DXGI_FORMAT_A8_UNORM, any single-channel would do) as input (inputs to effects are ID2D1Image, and ID2D1Bitmap inherits from ID2D1Image). Then set the D2D1_COLORMATRIX_PROP_COLOR_MATRIX for copying the input channel to all output channels. Have not tried it, though.
Context: I have a file called libffmpeg.so, that I took from the APK of an Android application that is using FFMPEG to encode and decode files between several Codecs. Thus, I take for grant that this is compiled with encoding options enable and that this .so file is containing all the codecs somewhere. This file is compiled for ARM (what we call ARMEABI profile on Android).
I also have a very complete class with interops to call API from ffmpeg. Whatever is the origin of this static library, all call responses are good and most endpoints exist. If not I add them or fix deprecated one.
When I want to create an ffmpeg Encoder, the returned encoder is correct.
var thisIsSuccessful = avcodec_find_encoder(myAVCodec.id);
Now, I have a problem with Codecs. The problem is that - let's say that out of curiosity - I iterate through the list of all the codecs to see which one I'm able to open with the avcodec_open call ...
AVCodec codec;
var res = FFmpeg.av_codec_next(&codec);
while((res = FFmpeg.av_codec_next(res)) != null)
{
var name = res->longname;
AVCodec* encoder = FFmpeg.avcodec_find_encoder(res->id);
if (encoder != null) {
AVCodecContext c = new AVCodecContext ();
/* put sample parameters */
c.bit_rate = 64000;
c.sample_rate = 22050;
c.channels = 1;
if (FFmpeg.avcodec_open (ref c, encoder) >= 0) {
System.Diagnostics.Debug.WriteLine ("[YES] - " + name);
}
} else {
System.Diagnostics.Debug.WriteLine ("[NO ] - " + name);
}
}
... then only uncompressed codecs are working. (YUV, FFmpeg Video 1, etc)
My hypothesis are these one:
An option that was missing at the time of compiling to the .so file
The av_open_codec calls is acting depending on the properties of the AVCodecContext I've referenced in the call.
I'm really curious about why only a minimum set of uncompressed codecs are returned?
[EDIT]
#ronald-s-bultje answer led me to read AVCodecContext API description, and there are a lot of mendatory fileds with "MUST be set by user" when used on an encoder. Setting a value for these parameters on AVCodecContext made most of the nice codecs available:
c.time_base = new AVRational (); // Output framerate. Here, 30fps
c.time_base.num = 1;
c.time_base.den = 30;
c.me_method = 1; // Motion-estimation mode on compression -> 1 is none
c.width = 640; // Source width
c.height = 480; // Source height
c.gop_size = 30; // Used by h264. Just here for test purposes.
c.bit_rate = c.width * c.height * 4; // Randomly set to that...
c.pix_fmt = FFmpegSharp.Interop.Util.PixelFormat.PIX_FMT_YUV420P; // Source pixel format
The av_open_codec calls is acting depending on the properties of the
AVCodecContext I've referenced in the call.
It's basically that. I mean, for the video encoders, you didn't even set width/height, so most encoders really can't be expected to do anything useful like this, and are right to error right out.
You can set default parameters using e.g. avcodec_get_context_defaults3(), which should help you a long way to getting some useful settings in the AVCodecContext. After that, set typical ones like width/height/pix_fmt to the ones describing your input format (if you want to do audio encoding - which is actually surprisingly unclear from your question, you'll need to set some different ones like sample_fmt/sample_rate/channels, but same idea). And then you should be relatively good to go.
I am trying to encode a particular string into a QR Code. I have been used Zxing lib for this.I am not able to generate the qr code using this piece of code. I have tried using Bitmap but windows 8 doesn't allow System.drawing assembly...
public ZXing.Rendering.PixelData get()
{
string url="abcdefghijkl";
BarcodeWriter _writer = new BarcodeWriter();
_writer.Format = BarcodeFormat.QR_CODE;
_writer.Options.Height = 400;
_writer.Options.Width = 400;
_writer.Options.Margin = 1;
var barcodeImage = _writer.Write("tel:" + url); //tel: prefix for phone numbers
return barcodeImage;
}
You have to use the correct pre-built assembly for your target platform.
There is a binary for WinRT available which doesn't use the Bitmap class.
https://zxingnet.codeplex.com/releases
If you use the version for Windows Metadata (WinMD) you have to convert the
raw bytes of the PixelData to an image. It depends on the application you write
and the things you want to do.
There are sample client projects for nearly every platform available
which show you how to use the library.
Basically, I'm trying to copy the front or back buffer to a texture, grab the 1x1 mipmap level of said texture, and then spew the color of the resulting back to the Arduino to control my room's lighting. Everything else is up and running, and I've already gotten it to work via GetDC(NULL) and StretchBlt. But this was about 15FPS, and the windows GUI ran choppy.
The downsampling is just DEMANDING to be used on a GPU.
In D3D9, it seemed like there was simply GetBackBuffer() or something like that, but I see nothing similar in D3D10. And I'm not even sure it would grab anything from the windows GUI.
Questions:
-What function(s) would I use?:
-Do I need to explicitly create a swapchain beforehand?
-Would this only capture data from other D3D programs?
Okay, here's where I'm at as far as creating the texture goes But I'm not seeing anything that gets me back to textures:
//Create Texture
D3D10_TEXTURE2D_DESC tBufferDesc;
ID3D10Texture2D *tBuffer = NULL;
DXGI_SAMPLE_DESC iBufferSamples = {1,0};
tBufferDesc.Width = iScreenSizeX;
tBufferDesc.Height = iScreenSizeY;
tBufferDesc.MipLevels = 0;
tBufferDesc.ArraySize = 1;
tBufferDesc.Format = DXGI_FORMAT_B8G8R8A8_TYPELESS;
tBufferDesc.SampleDesc = iBufferSamples;
tBufferDesc.Usage = D3D10_USAGE_DEFAULT;
tBufferDesc.BindFlags = D3D10_BIND_SHADER_RESOURCE | D3D10_BIND_RENDER_TARGET;
tBufferDesc.CPUAccessFlags = 0;
tBufferDesc.MiscFlags = D3D10_RESOURCE_MISC_GENERATE_MIPS;
HRESULT tBufferResult = pDevice->CreateTexture2D(&tBufferDesc, NULL , &tBuffer);
hrResult(tBufferResult);
ID3D10RenderTargetView *rtBuffer;
ID3D10Resource *rsBuffer;
pDevice->OMGetRenderTargets(1, &rtBuffer, NULL);
rtBuffer->GetResource(&rsBuffer);
rsBuffer->
I have a 7.1 channel audio output device and a custom kext to drive that. My custom application needs to send 7.1 rear channel audio data to the device but the device receives only 2 channel audio data. I checked "Configure Speaker" option in "Audio MIDI setup" application and it's set to stereo. When I set it to "7.1 Rear Surround" everything works fine. In my final product, I don't want the user to have to do all of this manually. So, the question is - Is there any Core Audio API or any other means of doing this programatically?
Ok, after playing around with some Core Audio APIs, finally I could get this done.
Get the AudioDeviceID:
AudioDeviceID audioDevice = getMyAwesomeDeviceID();
Create an AudioObjectPropertyAddress:
AudioObjectPropertyAddress propertyAddress;
propertyAddress.mSelector = kAudioDevicePropertyPreferredChannelLayout;
propertyAddress.mScope = kAudioDevicePropertyScopeOutput;
propertyAddress.mElement = kAudioObjectPropertyElementMaster;
Query if the audio object has this property:
AudioObjectHasProperty(audioDevice, &propertyAddress)
Get the data size of this property and create AudioChannelLayout:
UInt32 propSize(0);
AudioObjectGetPropertyDataSize(audioDevice, &propertyAddress, 0, NULL, &propSize);
AudioChannelLayout* layout = (AudioChannelLayout*)malloc(propSize);
Configure your AudioChannelLayout structure (eg: stereo layout):
AudioChannelLabel labels[2] = {kAudioChannelLabel_Right, kAudioChannelLabel_Left};
layout->mNumberChannelDescriptions = 2;
for (UInt32 i = 2; i < layout->mNumberChannelDescriptions; i++) {
layout->mChannelDescriptions[i].mChannelLabel = labels[i];
layout->mChannelDescriptions[i].mChannelFlags = kAudioChannelFlags_AllOff;
}
Set the AudioObject property data:
AudioObjectSetPropertyData(audioDevice, &propertyAddress, 0, NULL, propSize, layout);