What little information I managed to dig out on developing in-game overlays (similar to what Steam does) mentions having to intercept calls graphics API's frame swapping function, and hook my own drawing routine in it.
This appears to be what Mumble (a gaming VoIP) is doing. Since I've never done anything that involves hooking, and since I don't really have much experience with DirectX, I'm wondering if there is some sort of SDK, or even just a more readable example than Mumble that also implements input, that demonstrates how to implement an interactive in-game overlay. Mumble is great, but I don't seem to be able to wrap my head around it, especially around the more interesting things it does in order to hook its stuff properly.
Also, if you have more detailed info on how to do this on Mac and Linux... :-)
Maybe GLIntercept could give you some inspiration.
It provides an openGL.dll file that you put in your app's folder. Windows loads this dll instead of system32's one because of priority rules. GLIntercept forwards all calls to the system32's dll, but logs them all meanwhile.
So, you could implement your own glSwapBuffers() which renders some more things, and then forwards the call.
Source code is available as well.
I upvote your question, and I'm interested in you future discoveries... feel free to repost when you have more info :)
Start by designing your overlay without intercepting graphics API. Keep in mind that the key input must use global hooks.
Integrate it to the application using a Direct3D interceptor dll. Google it to retrieve a base code.
Edit:
DirectX: Intercept Calls to DirectX with a Proxy DLL will give you downloadable source code.
OpenGL: Chromium might be a good start.
Related
On Windows, all disk I/O ultimately happens via Win32 API calls like CreateFile, SetFilePointer, etc.
Now, is it possible to intercept these disk I/O Win32 calls and hook in your own code, at run time, for all dynamically-linked Windows applications? That is, applications that get their CreateFile functionality via a Windows DLL instead of a static, C library.
Some constraints that I have are:
No source code: I won't have the source code for the processes I'd like to intercept.
Thread safety: My hook code may dynamically allocate its own memory. Further, because this memory is going to be shared with multiple intercepted processes (and their threads), I'd like to be able to serialize access to it.
Conditional delegation and overriding : In my hook code, I would like to be able to decide whether to delegate to the original Win32 API functionality, or to use my own functionality, or both. (Much like the optional invocation of the super class method in the overriding method of the subclass in C++ or Java.)
Regular user-space code: I want to be able to accomplish the above without having to write any device-driver, mainly due to the complexity involved in writing one.
If this is possible, I'd appreciate some pointers. Source code is not necessary, but is always welcome!
You may want to look into mhook if Detours isn't what you want.
Here are a couple of problems you may run into while working with hooks:
ASLR can prevent injected code from intercepting the intended calls.
If your hooks are global (using AppInit_DLLs for example), only Kernel32.dll and User32.dll are available when your DLL is loaded. If you want to target functions outside of those modules, you'll need to manually make sure they're available.
I suggest you start with Microsoft Detours. It's free edition also exists and its rather powerful stable as well. For injections you will have to find which injection method will work for your applications in target. Not sure whether you need to code those on your own or not, but a simple tool like "Extreme Injector" would serve you well for testing your approaches. And you definitely do not need any kernel-land drivers to be developed for such a simple task, in my opinion at least. In order to get the full help of me and others, I'd like to see your approach first or list more constraints to the problem at hand or where have you started so far, but had problems. This narrows down a lot chit-chats and can save your time as well.
Now, if you are not familiar with Detours from Microsoft (MSFT) please go ahead and download it from the following link: http://research.microsoft.com/en-us/projects/detours/ once you download it. You are required to compile it yourself. It's very straightforward and it comes with a compiled HTML help file and samples. So far your profiles falls under IAT (Import Address Table) and EAT (Export Address Table).
I hope this non-snippet answer helps you a little bit in your approach to the solution, and if you get stuck come back again and ask. Best of luck!
I want my program to be able to launch any Windows game, and while the user is playing it, intermittently display some text or pictures in some part of the game window. The game may be in windowed or full-screen mode. From what I have been able to figure out from online resources, this could be done using a graphics library that supports overlays and using Windows Hooks to keep track of the target application's window. In this context I have some questions.
Will the overlays affect the game's performance?
How will hooking the application affect performance?
Is there any other way one could achieve this? For example, how do you think PIX, the DirectX debugging and analysis tool, work?
Fraps is the archetypal example of doing this sort of thing to a fullscreen DirectX application from a third-party app. It works by hooking some system calls and inserting itself into the call-chain between an app and DirectX. There is some performance hit, but in general its minimal.
This page seems to have some details and sample code on how to hook the app in this way.
If I recall correctly, from other forum discussions (can't find the link at the moment. search for things like "how does fraps work", it's a popular question), Fraps hooks a few things to force the app to load its DLL, then hooks Present() calls and executes a device->Clear() call before calling the real Present(), with a list of small rectangles to set to a different color, which can spell out the FPS number that it displays. This has a minimal performance impact and is widely compatible with whatever rendering the app is doing. Overlaying a bitmap would be more complicated since it wouldn't be as easy to do at Present-time. Perhaps if you could hook EndScene, then you could do more, but you would have to be careful to not change the device state.
PIX has privileged access to the DirectX driver, so I wouldn't expect to be able to use that as a model to emulate.
If an the target app is running in windowed mode, hooking DirectX still work, but you could also just use GDI instead.
Edit: I think this is the link I was originally thinking of.
I want to write a app which change the microphone input voice and make it like robot or some funny man's voice.It must support send changed voice to all application like IM Software or Game Client. Which technology should I pick up? Windows WaveForm Api? DirectX?
audio driver?
Thank you very much!
There's an MSDN Coding4Fun article that explains how to create a voice changer that operates over Skype, in C# (.NET). The full source code is also hosted as a project on CodePlex. In addition, it should be fairly easy do something else with the audio (as opposed to streaming it via Skype), since the project is based around the NAudio framework, which contains a good level of abstraction. Anyway, it is a reasonably complete (and stable) example - definitely worth checking out in my opinion.
If you want/need to use C++ or some other language for development, then this project should at least give you some ideas about how to go about it. Still, if you can use .NET, then you're in luck I think.
Robot voice is often done with a ring modulator effect, mixing the voice with a sine wave - this is easier. Or use a vocoder effect, modulating the voice onto some other waveform, like rectangle - might be a bit more tricky. Go read up how the effects work, get a program with which you can check out how they sound (Audacity works for the ring modulator, finding and using a vocoder may be a bit harder). Then read how it's done or get a library which will do the processing for you.
You are looking to support VSTi or DXi plugins.
There are tons that also act as vocoders, even for free.
You just need to write the host application.
Take a look here :)
Now that's a neat idea, especially for a mobile app.
I'd probably start off-line by using a .wav file as input to get the effects working the way I wanted. You can use any high level language for this, but you probably want something that will map reasonably well into C/C++.
In terms of a production version, I'd go native and do this in C or C++. You want something fast for real time audio processing & I like to avoid dependencies on things like .net for distribution. (Not that I have anything against .net, it's great for servers and distribution within a company but I'm not so keen on having it as a dependency for shrink wrap software.)
Windows DirectShow would be a tempting option - you could do some interesting effects with multi-media as well if you had the voice morpher implemented as a direct show filter.
What you're looking for is a vocoder. I don't know if any of the technologies listed above has a vocoder effect, but the best chance would be with DirectX.
Try this sample app .I think its useful to you.Link
So I have this program that I really like, and it doesn't support Applescript. I'd like to automate it a little bit. Now, I know that I could use applescript to tell the program to tell the menu to tell the submenu to tell the menuitem to activate or whatever, but frankly I don't like applescript very much anyway.
When I open the NIB file in IB, I can see the messages that are being sent to FirstResponder; for example, the Copy menu item sends "copy:". Is there any way for me to invoke this directly from another program?
No. It's called protected memory for a reason, you know. The other program is completely insulated from your application. There are ways to put code into other apps, but (a) it's very inadvisable (b) requires root privileges, which means the rest of your app needs to be ROCK SOLID AND IMPREGNABLE, and (c) writing such code is a black art requiring knowledge of the operating system kernel interfaces, virtual memory management, the ABI, the internals of the linker/loader, assembler programming, and the operational parameters and other specifics of the particular processor upon which your app happens to be running.
Really, AppleEvents and other such IPC mechanisms are there for a reason.
Your other alternatives (all of which are a bit hacky, to be honest, and give you the fairly significant burden of ensuring the target app is in the state you want/expect) the access the data you're looking for are:
The Accessibility APIs from the ApplicationServices framework, through which you can traverse the UI tree to grab the text from wherever you need it directly, or can activate the menu item. Access for your app has to be explicitly granted by the user, however (although this is much the same as the requirement for UI scripting).
You can use the CoreGraphics APIs (within the ApplicationServices framework again) to send keyboard events to the target application (or just to the system) directly. This would mean sending four events: Command-down, C-down, C-up, Command-up.
None of these are ideal. To be honest, your best approach would be to look at your requirements and figure out how you can best engineer around the problem by changing those requirements in some way, i.e. instead of grabbing something directly, ask the user to provide some input, etc.
You might be interested in SIMBL or in mach_inject. SIMBL is a daemon (in my fork based on mach_inject, in the original version based on injection via some ScriptingAdditions hack) which does the injection for you, so you just need to put a bundle with your code into the SIMBL directory and SIMBL will inject it for you into the target application. Or you can do so yourself via mach_inject. Or probably more convenient, mach_inject_framework which injects and runs code which just loads some framework.
I think Jim may overstate the point a bit; he's not wrong, but it seems misleading. There are lots of ways to cause a Cocoa program to execute its own code under you control (Carbon is harder). The Accessibility API is very commonly used this way (so commonly that I expect it to be repurposed eventually). Fscript can give you all kinds of access to the innards of another Cocoa program. While Input Managers may well exit the scene at some point, SIMBL is still out there today to do this kind of stuff.
Whether you like Applescript or not, Apple Events are the primary way Apple provides for inter-program control. Have you double-checked Script Editor's Open Library function to find out if the program really does have any Applescript support? You can code Apple Events entirely in Objective-C these days using Leopard's Scripting Bridge. I wrote up a tutorial if you like (it's still under-documented by Apple).
Cocoa is a reverse-engineer's dream. The same guys who host SIMBL have a nice intro to the subject. "Wolf" also writes a lot of useful information on this.
Jim's right. Many of these approaches can completely destabilize the system if done incorrectly (sometimes even if done correctly). I don't do much of this stuff on my production systems; I need them to work. But there are a lot of things you can make a Mac app do, and it's a good part of a Mac developer's training to understand how all the pieces really work.
We provide a web application with a frontend completely developed in Adobe Flash. When we chose Flash 6 years ago, we did so for its large number of features for user interaction, like dragging stuff, opening and closing menus, tree navigation elements, popup dialogs etc.
Today it's obvious that AJAX/JS offers roughly the same possibilities and because of the number of frameworks that are readily available, it's very feasible to implement them.
Is there a technical reason one should choose either technology over the other? By "technical", I mean performance, security, portability/compatibility and the like. I don't mean aspects such as the very non-programmer way development is done in Flash or whether it makes sense to switch an app from one to the other.
As I just explained in another question, it seems to me that JS is way ahead in terms of market share and I'm wondering whether we are missing some important point if we stick to Flash.
In addition to what others have said, Flash is constrained in the "rectangle" and cannot be added to a normal html page in an un-obtrusive manner.
#Gulzar I think when more browsers will support the video tag like mozilla 3.1 does we'll see even more adoption of ajax/js over flash.
Adobe Actionscript is a statically typed language, Javascript is dynamically typed. Depending on your point of view, this may be a good thing or a bad thing.
With Javascript/HTML/CSS you're going to be heading into cross-browser compatibility hell, especially if you want to support older browsers. This can be mitigated by the libraries that are available, but it's still a big headache. With Flash, you write the code once and it just works in all browsers.
Even with the libraries available, Flash user controls are simply more advanced than anything you can find in the world of Javascript/HTML. In Javascript, you are not going to find anything that comes close to the simplicity and power of a databound user control that Flash provides.
I don't see how Javascript has more of a "market share" than Flash. Pretty much anyone with a web browser has a Flash plugin installed. I'd be curious to know how many people disable Javascript but have a Flash plugin.
Also keep in mind that you're going to be in for a huge learning curve and lots of development time if you decide to switch your technology base so you'd really better have a good business reason to do it.
This decision also has a lot to do with what your application does and who your install base is.
Edit: I see people have mentioned that the iPhone doesn't have Flash support. I would expect this to change with the install base of the iPhone - Adobe would be crazy not to support it.
Correctly designed AJAX apps are more googleable than Flash
Correctly designed AJAX apps are more easily deep linkable than Flash
AJAX doesn't require a plugin (Flash is pretty ubiquitous, so it's not really a big deal)*
AJAX isn't controlled by a single company the way Flash is
Edited to add:
* Except for the iPhone, as Abdu points out.
JS and Flash both have great presence on the web with overlapping capabilities. One area JS is still lacking is in rendering video.
Flash, used well, allows easy localization and internationalization.
Furthermore, it is much easier to use Flash in an accessible manner; you can feed screen readers the right text, instead of having them iterate over all of the possible form elements.
I think Flash should be limited to online games, videos and animation. Otherwise use html and Ajax. It's a web standard and supported by almost all devices.
AFAIK, the iPhone doesn't support Flash. That's a fast growing segment you're blocking out already. Keep it simple and efficient.
Although flash is pretty ubiquitous on desktop browsers, mobile support is very limited (flash lite? yeah, right). I get really frustrated looking up a restaurant on my phone only to find the entire site is flash based and I can't even get a phone number or address!
One benefit of Flash is that it has a few facilities to help do cross domain type operations safely, which can be helpful. Flash also has (limited) support for some hardware, which is not possible with Javascript.
Personally, I'd try to use as much Ajax as possible before turning to something like Flash. From the UI perspective, it is better in that the controls and basic authoring is a little more developed. The Sound Manager project is a good example of effectively using a small amount of Flash while keeping the remainder in Javascript.
I suspect one of the reasons javascript is becoming more popular is that it's more easy to retrofit into an existing application.
As I can't accept two answers, I'm going to merge Christ Upchurch's and 17 of 26's answers in my own post. I think, these two together pretty much sum up what I wanted to know. Thanks guys!
If you're dealing a lot with polygons, then Flash is still easier to program and debug. With AJAX there are a lot of libraries to handle polygons, but the more libraries your app uses, the slower it gets.