Is it possible to copy pixel data directly from a windows device context into an openGL rendering context (an openGL texture, to be specific)? I know that I can copy the windows device context pixel data into system memory (take it out of the graphics card), and then later upload it back into my openGL framebuffer, but what I'm looking for is a direct transfer where the data doesn't have to leave the GPU.
I am trying to write an application that is essentially a virtual magnifier. It consists of one window, which displays the contents of any other windows that are open underneath it. The target for my application is machines with windows 8 and higher, and I am using the basic win32 API. The reason why I want to use openGL to display the contents of my window is because I wish to perform various spatial transformations (distortions) with the GPU, not just magnification. Using openGL, I believe I can perform these transformations very fast.
Previously, I thought that all I had to do was to "move" my openGL rendering context onto each "third party" window that I wanted to steal pixel data from, and use glReadPixels() to copy this data into a PBO. I could then switch my rendering context back to my magnifier window, and proceed with rendering. However, I understand that this isn't possible, because openGL doesn't have access to any pixel data that wasn't rendered using openGL itself.
Any help would be greatly appreciated.
Related
I need to capture a snapshots/screenshots of the specific window (HWND) that is using HW acceleration and record them to a video stream.
While using BitBlt or PrintWindow I'm able to capture image data only if this window is not HW accelerated, else I'm getting a black texture.
Tried using User32.dll's undocumented DwmGetDxSharedSurface to get the DirectX surface handle. But it fails with an error:
ERROR_GRAPHICS_PRESENT_REDIRECTION_DISABLED - desktop windowing
management subsystem is off
(Edit: Fails for certain applications, i.e. "calculator.exe")
Tried using Dwmapi.dll's undocumented functions DwmpDxUpdateWindowSharedSurface and DwmpDxGetWindowSharedSurface. I've managed to retrieve what looks like a valid DirectX surface handle. (it's d3dFormat, width and height information was valid) Dx's OpenSharedResource was not complaining and managed to create a valid ID3D11Texture2D. Problem is.. all bytes are zeros (getting a black texture). I might be doing something wrong here or.. undocumented DWM functionas does not work anymore on Windows 10...
Edit: I'm able to get image data for some applications like Windows
explorer, Paint, etc, but for some like i.e. Slack i get all
zeros/black image.
Edit: When capturing i.e. VLC, I get this:
Question:
Is there any other way to capture image data of the HW accelerated window?
Note: I don't want to capture the entire desktop.
You can use PrintWindow with nFlags=2
Or use Magnification API (exclude windows)
Or try to hack dwm.exe.
I don't have any Deep-Color capable hardware attached to my computer, so I haven't been able to experiment myself, and searching online for "win32 deep-color" or "win32 10-bit color" (or 30-bit, 48-bit or 64-bit) yields nothing relevant or recent. The top result is still this NVIDIA PDF from 2009: https://www.nvidia.com/docs/IO/40049/TB-04701-001_v02_new.pdf - it describes using OpenGL and an NVIDIA API for displaying images with more than 8-bits per channel.
I understand how using OpenGL allows 30-bit color images to be displayed: it effectively bypasses the operating system and the OpenGL surface is rendered in deep-color on the GPU and sent directly to the monitor in an appropriate format over DisplayPort or HDMI.
But what options are there outside of OpenGL?
In Win32, after you create a Window with CreateWindow, you render it by handling the WM_PAINT message, and then calling BeginPaint, which gives you a handle to a GDI device-context, which cannot be more than 32bpp (8-bits per channel).
While GDI ostensibly abstracts away implementation details of the rendering device, including color depth, it is impossible to specify a 10bpp RGB value, for example (the COLORREF struct is hardcoded to use 32-bit (8bpp) DWORD values), a leaky-abstraction.
Does this mean it is impossible to display 30bpp / Deep-color content in the Windows Desktop using a program handling WM_PAINT and that OpenGL is the only way?
And what would happen if you attempted to blit from an in-memory OpenGL rendering buffer back to the window surface? (i.e. what happens if you press PrintScreen while displaying a Deep-Color BluRay disc in a BD player, or displaying 30-bit content in Photoshop?)
I'm trying to do pretty much what TangoARScreen does but with multithreaded rendering on in Unity. I did some experiments and I'm stuck.
I tried several things, such as letting Tango render into the OES texture that would be then blitted into a regular Texture2D in Unity, but OpenGL keeps complaining about invalid display when I try to use it. Probably not even OnTangoCameraTextureAvailable is called in the correct GL context? Hard to say when you have no idea how Tango Core works internally.
Is registering a YUV texture via TangoService_Experimental_connectTextureIdUnity the way to go? I'd have to deal with YUV2RGB conversion I assume. Or should I use OnTangoImageMultithreadedAvailable and deal with the buffer? Render it with a custom share for instance? The documentation is pretty blank in these areas and every experiment means several wasted days at least. Did anyone get this working? Could you point me in the right direction? All I need is live camera image rendered into Unity's camera background.
Frome the April 2017: Gankino release notes: "The C API now supports getting the latest camera image's timestamp outside a GL thread .... Unity multithreaded rendering support will get added in a future release.". So I guess we need to wait a little bit.
Multithreaded rendering still can be used in applications without camera feed (with motion tracking only), choosing "YUV Texture and Raw Bytes" as overlay method in Tango Application Script.
Is there any way in Mac OS X to share an OpenGL framebuffer between processes? That is, I want to render to an off-screen target in one process and display it in another.
You can do this with DirectX (actually DXGI) in Windows by creating a surface (the DXGI equivalent of an OpenGL framebuffer) in shared mode, getting an opaque handle for that surface, passing that to another process via whatever means you like, then creating a surface in that other process, but passing in the existing handle. You use the surface as a render target in one process then and use it as a texture in the other to consume as you wish. And in fact the whole compositing Window system works like this from Vista onwards.
If this isn't possible I can of course get the contents of the framebuffer into system memory and use cross-process shared memory to get it to the target process, then upload it again from there, but that would be unnecessarily slow.
Depending on what you're really trying to do this sample code project may be what you want:
MultiGPUIOSurface sample code
It really depends upon the context of how you're using it.
Objects that may be shared between contexts include buffer objects,
program and shader objects, renderbuffer objects, sampler objects,
sync objects, and texture objects (except for the texture objects
named zero).
Some of these objects may contain views (alternate interpretations) of
part or all of the data store of another object. Examples are texture
buffer objects, which contain a view of a buffer object’s data store,
and texture views, which contain a view of another texture object’s
data store. Views act as references on the object whose data store is
viewed.
Objects which contain references to other objects include framebuffer,
program pipeline, query, transform feedback, and vertex array objects.
Such objects are called container objects and are not shared.
Chapter 5 / OpenGL-4.4 core specification
The reason you can do those things on Windows and not OS X is that graphics obviously utilizes an API that allows DirectX contexts to be shared between those processes. If OS X doesn't have the capability within the OpenGL API then you're going to have to come up with your own solution. Take a look at OpenGL Programming Guide for Mac, there's a small section that describes using multiple OpenGL contexts.
I have to get (shared memory or GPU memory) the directX render from a windows application called "Myapp" and apply this render (view) to four directX simple applications (only exactly the same view as the first windows application "Myapp")
Someone tells about backbuffer and anothers tells about FrontBufferData
1) How can I get easily the directX render of a directXWindows application in C++ ?
2) How can i Share easily and quickly this render to 4 another DirectX applications in C++ ?
Thanks in advance
You can never get the rendering data from backbuffer for an 3rd application, the only Interface Microsoft provide is GetFrontBufferData(), this function is the only way to take an antialiased screen shot, and it's very slow.
front buffer contains the data currently displayed on your screen.
back buffer contains the data being draw, but not present yet.
when you call Present, DirecX will swap the two buffers by simply change the buffer pointers, thus the front buffer became back buffer now, and the back buffer became front buffer now. this is called surface flipping.
There are many ways to share memory between processes.
Can I ask a question, what do you want to do with the rendering data?
thanks for your answer.
I just want to post / show the render / view of the application "Myapp" in 4 others directX views without changes (in C++)