I'm working on a program that needs to record and play back keystrokes. The main process runs as a service, so it needs a configuration program to record the keystrokes.
The problem comes when the system default keyboard layout is (say) English, and the user's keyboard layout of the moment is (say) German. The user enters a 'ü' character into the configuration interface, which is duly recorded and stored.
Now the user runs Notepad and tries to play the keystroke back, which is done by a child of the service. When transforming input to pass to SendInput, we call VkKeyScan, which then tells us there's no way to reproduce that character on this keyboard (remember the default keyboard layout is English).
What I'd really like to do is sync the keyboard layouts of the service and the current foreground application, so the service can reproduce any character the user can with their keyboard. GetKeyboardLayout only works in the current process. This has to run on both 32- and 64-bit, so a message hook is... not preferred :).
To retrieve the keyboard layout for the foreground window:
GetKeyboardLayout(GetWindowThreadProcessId(GetForegroundWindow(), NULL))
It sounds like you're recording characters instead of keystrokes. Instead of sending keystrokes when replaying, can you use SendMessage to send WM_CHAR messages with the recorded characters instead of generating keystrokes?
Related
I haven't find anything relevant in Google or any Microsoft site about it so I decided to ask a question here.
Everybody knows that in Win-based OS there is a virtual keyboard. I also know that *nix based OS, have it too. So, the question is about:
HOW DOES IT WORK INSIDE?
I mean, let's have an example that I opened on screen keyboard in Windows 10. What's the actual difference between:
input via hardware keyboard: when I'm using it, like I press X button
..and using a virtual keyboard, when I press the same button
Imagine, I have an admin access to terminal/computer, is there any option to track/distinguish that in the second time I pressed button not on hardware keyboard, but on-screen (by mouse clicking) version of it?
And there are also many different software, like AutoIt (yes, it's a language, but it's relevant to this example) that emulating pressing the X button. How does they work in Win-based OS? Do they "in-common" with default on-screen keyboard and using the same driver/WinAPI or there is a difference between them?
And the second case, between:
default on-screen keyboard
compilated AutoIt script
..any other software that emulating press X button
I guess the only way to find out "how exactly button was pressed" is to check current processes list via taskmgr and find out have anything been launched or not. Or I'm totally wrong here, and missing something?
THE SCOPE
I have written a node.js script which emulates button pressing behaviour in windows app.
TL:DR business logic short => open notepad.exe and type `Hello world`
And could someone give me any advice/recommend any powershell/bat script (or any other solution) with demonstration of GetAsyncKeyState check behavior? With which I could easily check my own node.js script (not by functional of it, but by triggering press the X button event)
I found an answer for node.js case here: Detecting Key Presses Across Applications in Powershell
SendInput is the preferred method to generate user input in software. The Windows on-screen keyboard probably uses it for everything except Ctrl+Alt+Delete which I believe has some kind of special handling. The on-screen keyboard is only able to generate Ctrl+Alt+Delete in certain configurations.
Software-generated input is merged with normal hardware input in the RIT (Raw Input Thread) in the kernel.
A low-level keyboard hook can detect software-generated input.
I have a third party application (I'll call it GreedyApp for brevity), which holds the mouse and keyboard input hostage when its window gets focus i.e. it hides the standard mouse cursor and replaces it with it's own cursor, and confines the cursor to its window. The only way to get input to other windows is to ALT+TAB away from GreedyApp.
I need to allow the user free use of all of the components of the system (the delivered system will be purely touch-screen), so at the minute the rest of the system becomes unusable once GreedyApp gets focus.
So far, I've hijacked user32.dll for GreedyApp, hooked SetCursor, ShowCursor and ClipCursor, and disabled them. The result is that GreedyApp no longer hides the cursor, and the cursor is free to roam wherever the user moves it, but...
The problem I'm left with, is that no matter where on the screen the cursor is pressed, or what keys on the keyboard are pressed (except ALT+TAB), the input is still directed into GreedyApp, and other windows don't receive any input.
I'm not sure how GreedyApp is achieving this, and therefore I don't yet know which API calls to hook to stop it. I though it might have been using hooks itself, but I've hooked and disabled SetWindowsHookEx, but the problem persists.
So my question is this:
Either:
A) Is there a (relatively straight-forward) way to find out what API calls an application is making at runtime?
or
B) What method is GreedyApp likely to be using to stop other windows from receiving input?
The application was using RegisterRawInputDevices to get raw mouse and keyboard input, and using the flag RIDEV_CAPTUREMOUSE to stop other applications getting focus.
I've hooked the API call and remove the flag before passing the parameters to the Windows API to process. The user now has control over the system :)
Under Win2K(or later), by sending a Virtual-Key VK_LAUNCH_MEDIA_SELECT, can start a player.
If more than one player software installed, how to determine which one it will start?
A sample VBS code:
Wscript.CreateObject("Wscript.Shell").Sendkeys chr(&h88b5)
http://msdn.microsoft.com/en-us/library/windows/desktop/dd375731(v=vs.85).aspx
http://msdn.microsoft.com/zh-cn/library/dd375731
VK_LAUNCH_MEDIA_SELECT
0xB5
Select Media key
VK_LAUNCH_MEDIA_SELECT is actually received by the system and translated into a WM_APPCOMMAND with lParam as APPCOMMAND_LAUNCH_MEDIA_SELECT. So theoretically, any program implementing a handler for this could be launched. This page (albeit old) lists default applications which listen for WM_APPCOMMAND:
Internet Explorer
Windows Help
DVD Player
CD Player
Media Player
Volume Control system tray applet
Obviously, only a few of those are actually applicable for APPCOMMAND_LAUNCH_MEDIA_SELECT.
However, I don't know how the translation mechanism actually works. It appears not every application will receive the translated WM_APPCOMMAND message. On my keyboard, I tried pressing the button which sends VK_VOLUME_UP. The volume goes up as expected because it is handled by the volume control system tray applet. However, I opened an instance of notepad.exe and monitored its messages with Spy++. It did not receive any message even when it was in the foreground. Curiously, VK_MEDIA_PLAY_PAUSE is received through WM_APPCOMMAND if the play/pause button is pressed as long as notepad.exe had the focus. I would test with VK_LAUNCH_MEDIA_SELECT, but I'm not actually sure what button that corresponds to (or whether my keyboard has it).
Thanks, for Mike Kwan's reply.
Through testing a lot of settings a program associated with a variety of audio files.
Finalized, this will depend on the CDA (CD Audio) files associated with which program.
If you set the default association for the CDA (CD Audio) files to Notepad or Paint or any other program, it can still send the same virtual key to open the corresponding program.
The foreground window can do whatever it wants in response to the WM_APPCOMMAND message, if it does not handle the message then shell hooks (HSHELL_APPCOMMAND) gets to handle it, if no hooks handle the message then Windows checks the AppKey key in the registry. (You can use Process Monitor to find the number for a specific key-press)
I'm developing an app and I'd like to trigger functions in my app via keystroke combos when it's not in focus.
Because I'm developing my app in AIR, I do not have access to listen to global Keystrokes. However, I can receive STDOUT from an application. So, I'm looking for a utility that can give me this ability. I'm looking for both Windows and OSX (cross-platform baby!)
For Windows, you could write a simple application that installs a keyboard hook and prints information about the key event to stdout. See SetWindowsHookEx.
For Windows:
I don't know of any app off the top of my head, but here are some ideas that might work within 100 lines of code...
I would avoid SetWindowsHook, as that would inject your code into all apps. (Because I've spent good time debugging crash dumps and bugs as a result of poorly written hooks...)
You could write a console app with DirectInput (old gaming keyboard API). I believe you just pass DISCL_BACKGROUND and DISCL_NONEXCLUSIVE into IDirectInputDevice8::SetCooperativeLevel call. Use IDirectInputDevice8::SetEventNotification to set the event handle so you don't have get into a busy wait loop polling for input. And that should do it. I did this once for my app a long time ago on Windows 98 and it worked really well. But DirectInput is very close to being deprecated technology so YMMV.
Another simple hacked up way to do what you are doing is to have your app create a hidden window, call call RegisterHotkey for all the keyboard, and pump window messages. Your wndproc will get a WM_HOTKEY window message that you can use that to generate a message to stdout.
The simplest way, but will be slightly error prone and cpu-expensive is to have your console app get into a loop and call GetKeyboardState. This will return the entire state of the keyboard of all keys that are up and down. You'll have to figure out how translate each poll into a logical keystroke. I'd recommend sleeping a few milliseconds between polls so you don't kill system-wide performance.
Can't help you on OSX.
For windows, here is a utility that will listen to the keyboard.
http://www.dynamicnetservices.com/~will/academic/textinput/keycapture/
I have made an application already that sends commands to an activated window. I want to be able to use the computer while my process is running because as soon as I switch focus to another window the key strokes being sent via send keys will go to the window I just switched to.
Currently I use FindWindow, IsIconic, and ShowWindow from the Windows API. I have to check to see if the window is there with FindWindow and set my object to the specific window that is returned with that call, I then check if it's minimized with IsIconic and call ShowWindow if it is, and then finally I have to call Interaction.AppActivate to set focus to that window. All of this is done before I even send key strokes. Seems like there should be a way to just send key strokes without having to show the window and activate it. The big thing is while my application is running the key strokes I can't do anything on my computer.
Alright, this is kind of disappointing I'm sure, but you fundamentally cannot do this with 100% reliability.
Windows assumes that the active window is the one getting keyboard input. The proper way to fake keyboard input is with SendInput, and you'll notice that it sends messages to the active window only.
That being said, you can SendMessage WM_KEYUP, WM_CHAR, and WM_KEYDOWN messages and (depending on the WndProc receiving them) maybe get away with it. But remember, its going to break under some circumstances, period.
Sounds like you are using keybd_event() or SendInput(), which both send keystrokes to the currently active window. To direct keystrokes to a specific window, regardless of whether that widnow is focused or not, you need to find its HWND handle first, and then post appropriately-formatted WM_KEYUP/DOWN and WM_CHAR messages directly to it.
once you have the windows HWND, you can directly SendMessage() the WM_KEYDOWN and WM_KEYUP messages to its message queue. The window does not have to be active.
However, understand that this depends on how the target application processes keyboard input. There are several different ways to handle it.
WM_KEYUP/WM_KEYDOWN is most common and some applications only process one or the other (usually WM_KEYDOWN).
WM_CHAR is also fairly common
Some programs use GetAsyncKeyState, GetKeyState, or GetKeyboardState. This is extremely unusual, but effectively prevents keypress injection with SendMessage(). If this is the case fall back to keybd_event() which is directly handled by the keyboard driver. Of course the window will have to be active