I want my program to be able to launch any Windows game, and while the user is playing it, intermittently display some text or pictures in some part of the game window. The game may be in windowed or full-screen mode. From what I have been able to figure out from online resources, this could be done using a graphics library that supports overlays and using Windows Hooks to keep track of the target application's window. In this context I have some questions.
Will the overlays affect the game's performance?
How will hooking the application affect performance?
Is there any other way one could achieve this? For example, how do you think PIX, the DirectX debugging and analysis tool, work?
Fraps is the archetypal example of doing this sort of thing to a fullscreen DirectX application from a third-party app. It works by hooking some system calls and inserting itself into the call-chain between an app and DirectX. There is some performance hit, but in general its minimal.
This page seems to have some details and sample code on how to hook the app in this way.
If I recall correctly, from other forum discussions (can't find the link at the moment. search for things like "how does fraps work", it's a popular question), Fraps hooks a few things to force the app to load its DLL, then hooks Present() calls and executes a device->Clear() call before calling the real Present(), with a list of small rectangles to set to a different color, which can spell out the FPS number that it displays. This has a minimal performance impact and is widely compatible with whatever rendering the app is doing. Overlaying a bitmap would be more complicated since it wouldn't be as easy to do at Present-time. Perhaps if you could hook EndScene, then you could do more, but you would have to be careful to not change the device state.
PIX has privileged access to the DirectX driver, so I wouldn't expect to be able to use that as a model to emulate.
If an the target app is running in windowed mode, hooking DirectX still work, but you could also just use GDI instead.
Edit: I think this is the link I was originally thinking of.
Related
What little information I managed to dig out on developing in-game overlays (similar to what Steam does) mentions having to intercept calls graphics API's frame swapping function, and hook my own drawing routine in it.
This appears to be what Mumble (a gaming VoIP) is doing. Since I've never done anything that involves hooking, and since I don't really have much experience with DirectX, I'm wondering if there is some sort of SDK, or even just a more readable example than Mumble that also implements input, that demonstrates how to implement an interactive in-game overlay. Mumble is great, but I don't seem to be able to wrap my head around it, especially around the more interesting things it does in order to hook its stuff properly.
Also, if you have more detailed info on how to do this on Mac and Linux... :-)
Maybe GLIntercept could give you some inspiration.
It provides an openGL.dll file that you put in your app's folder. Windows loads this dll instead of system32's one because of priority rules. GLIntercept forwards all calls to the system32's dll, but logs them all meanwhile.
So, you could implement your own glSwapBuffers() which renders some more things, and then forwards the call.
Source code is available as well.
I upvote your question, and I'm interested in you future discoveries... feel free to repost when you have more info :)
Start by designing your overlay without intercepting graphics API. Keep in mind that the key input must use global hooks.
Integrate it to the application using a Direct3D interceptor dll. Google it to retrieve a base code.
Edit:
DirectX: Intercept Calls to DirectX with a Proxy DLL will give you downloadable source code.
OpenGL: Chromium might be a good start.
we are developing a skinned application, and under vista/windows 7, on some machines, skinned applications sometimes loses their skin. here's an example for the problem, and here's how the application looks when it's good.
this happens to us whether we develop with native Win32 API or in QT. It happens spontanously, with no event that might explain it. btw, we see it happens sometimes to some other applications, too
we solve it by repainting everything every 2-3 seconds. but this is an ugly hack...
any ideas why this could happen?
thanks _very_much_ for any lead -
Lior
Shot in the dark, but it sounds like a graphics driver problem. I'd check whether the problematic machines all have the same graphics card or the same version of the graphics driver, and how the driver collection on those machines compares with the OK ones.
Shot in the dark #2: You're running out of GDI resources because your app (or another app running on the same machine) is leaking GDI handles.
It's been while since I've had to use any tools for detecting "GDI handle leaks" (Google or Bing on it).
Here's some links to go read up on:
http://msdn.microsoft.com/en-us/magazine/cc301756.aspx
http://www.nirsoft.net/utils/gdi_handles.html
http://msdn.microsoft.com/en-us/magazine/cc188782.aspx
Is it possible to develop an 3D application in .Net (XNA or WPF) that would take advantage of Windows 7's multi touch support.
Is it possible to do so ?
Where is the best place to start ?
First: The nature of what you display is independent of whether your application supports multitouch or not.
Second: I don't believe there's a great value in having 3D interfaces whether using multitouch or not: the thing you touch is probably plain (a monitor or something the like). Better go with 2.5D (use shadows to give the whole thing a pseudo 3D look, stay away from 3D cubes).
Third: Have a well concept of user experience. What will be touchable, how will the application react (animations, etc.). Have a great look (graphics) designed and be prepared to have them changed while implementing a multitouch application. Technically, make yourself familiar with gestures, APIs and whatever it takes (the hardware is a factor as well),
The second point can (and probably should) be seen as a personal taste...
Start with gestures
Then dive into the API
I'm ready to outsource the screen capture functionality of our application, because it's not our core business and I've spent too much time trying to get our code to do all the things I need. Time for a specialist, I think. My requirements, in decreasing order of importance, are below. Does anyone have experience with any commercial or free libraries that meet most or all of these requirements?
Has to work in Windows XP and higher. (But not Win95/98/Me.)
Visual C++ 2005 compatible, where screenshot can be triggered from my code. Preferably a static library, but a DLL or COM object is OK. I'd prefer not to shell out to a standalone EXE because there are some users that will try to tamper with our application and I think having an obvious separate screenshot EXE makes that too easy.
Must be able to take full screenshots of multiple monitor systems. (Preferably the way the "Print Screen" key does in Windows, by making a bitmap of the displays stitched together, but if I have to take the pictures separately and combine them myself that's acceptable.)
Must be able to capture screen correctly when Remote Desktop Client (or any RDP client or VM) is in full screen mode. (Of course, it should also work when RDP is in windowed mode, but that shouldn't be a problem as long as it doesn't operate by simulating a key press that might get transmitted to the remote OS instead of being handled locally.)
Must be able to capture screen correctly when Windows Media Player (or any other media player) is in full screen mode.
Must be able to capture screen correctly when game (e.g. World of Warcraft) is in full screen mode.
Would be nice to be able to capture a few seconds of user activity as a video.
I don't know of a library that would do what you want.
If I had to code your requirements, I would probably use the source code of the TightVNC server as my starting point. I think it has the technology to do everything on your list EXCEPT....
I'm not sure that technically there's ANYTHING that can do a screen capture of somebody's Remote Desktop session. Think about it: There can be multiple remote desktop sessions (the csrss.exe process) occuring using the same physical remote desktop server. If you were sitting in front of the machine looking at the video monitor, you wouldn't see anything happening at all. So what woould you expect to capture. VNC is only going to capture what's happening with the "real" video (the non-remote csrss.exe).
I dont know if this really helps, but the best imaging libraries available are available from
www.accusoft.com and
www.leadtools.com
Both support creatting screenshots, though i don't know, if hey will properly capture movie player output that displays through graphics acceleration or the recording of movies from screen.
If I were you I would investigate WindowsClippings, a pretty mature and extensible application written in C++ that does precisely what you need. It has an API you can extend to your requirements and is pretty cheap at 18$
Some people in our office are using Magick++ (an ImageMagick library) to make screenshots. I don't know all the specs, but you could take a look at it and see if it matches your requirements.
When writing DirectX applications, obviously it's desirable to support the user suspending the application via Alt-Tab in a way that's fast and error-free. What is the best set of practices for ensuring this? Things that need to be addressed include:
The best methods of detecting when your application has been alt-tabbed out of and when it has been returned to.
What DirectX resources are lost when the user alt-tabs, and the best ways to cope with this.
Major things to do and things to avoid in application architecture for purposes of alt-tab support.
Any significant differences between major DirectX versions as they apply to the above.
Interesting tricks and gotchas are also good to hear about.
I will assume you are using C++ for the purposes of my answers, but if you can afford to use C#, XNA (http://creators.xna.com/) is an excellent game platform that handles all of these issues for you.
1]
This article is helpful for windows events in the window procedure to detect when a window loses or gains focus, you could handle this on your main window: http://www.functionx.com/win32/Lesson05.htm. Also, check out the WM_ACTIVATEAPP message here: http://msdn.microsoft.com/en-us/library/ms632614(VS.85).aspx
2]
The graphics device is lost when the application loses focus from full screen mode. Microsoft offers an article on how to handle this: http://msdn.microsoft.com/en-us/library/bb174717(VS.85).aspx This article also has a lost device tutorial: http://www.codesampler.com/dx9src/dx9src_6.htm
DirectInput can also have a device lost error state, here is a link about that: http://www.toymaker.info/Games/html/directinput.html
DirectSound can also have a device lost error state, this article has code that handles that: http://www.eastcoastgames.com/directx/chapter2.html
3]
I would make sure to never disable Alt-Tab. You probably want minimal CPU load while the application is not active because the user probably Alt-Tabbed because they want to do something else, so you could completely pause the application, or reduce the frames rendered per second. If the application is minimzed, you of course don't need to render anything either. After thinking about a network game, my best solution is that you should still reduce the frames rendered per second as well as the amount of network packets handled, possibly even throwing away many of the packets that come in until the game is re-activated.
4]
Honestly I would just stick to DirectX 9.0c (or DirectX 10 if you want to limit your target operating system to Vista and newer) if at all possible :)
Finally, the DirectX sdk has numerous tutorials and samples: http://www.microsoft.com/downloads/details.aspx?FamilyID=24a541d6-0486-4453-8641-1eee9e21b282&displaylang=en
We solved it by not using a fullscreen DirectX device at all - instead we used a full-screen window with the top-most flag to make it hide the task bar. If you Alt-Tab out of that, you can remove the flag and minimize the window. The texture resources are kept alive by the window.
However, this approach doesn't handle the device lost event happening due to 'lock screen', Ctrl+Alt+Delete, remote desktop connections, user switching or similar. But those don't need to be handled extremely fast or efficiently (at least that was the case in our application)
All serious D3D apps should be able to handle lost devices as this is something that can happen for a variety of reasons.
In DX10 under Vista there is a new "Timeout Detection and Recovery" feature that makes it common in my experience for graphics devices to be reset which would cause a lost device for your app. This seems to be improving as drivers mature but you need to handle it anyway.
In DX8 and 9 (and 10?) if you create your resources (vertex and index buffers and textures mainly) using D3DPOOL_MANAGED they will persist across lost devices and will not need reloading. This is because they are stored in system memory and the DX runtime copies to video memory automatically. However there is a performance cost due to the copying and this is not recommended for rapidly changing vertex data. Of course you would profile first to determine if there is a speed issue :-)