I want to create a virtual keyboard in osx. Is it possible? I mean can I make a program that gives same signals than real keyboard. Example of this kind of keyboard would be onscreen keyboard, or keybord viewer (does it have necessary interface btw).
How low should I start? Should I make a device driver e.g. virtual (wireless) keyboard? Or does cocoa etc. have the necessary stuff?
The requirements I have are:
- a list of tuples (time, key_down/key_up, key_code) corresponds to person typing
- virtual keyboard should work side by side with the real one (like touchpad and bluetooh mouse)
- this should work with every program. Hardest examples I can find are: Terminal+vim, remote desktop, games like starcraft
Sample code and links are more than welcome.
edit: The main point is to have programmatic access to keystrokes. There are similar programs but they are with closed source (e.g. http://www.assistiveware.com/keystrokes.php). I want to know what is the best way to make this kind of program.
edit 2: Now I got this party started. Below is a copy-edit-paste-try-something-else code that basically includes all necessary parts to make a virtual keyboard. In this case, every time I press 'a' the virtual keyboard presses 'z'. There is a bug, that there is multiple 'z's added...
#import <ApplicationServices/ApplicationServices.h>
CGEventRef myCGEventCallback(CGEventTapProxy proxy, CGEventType type, CGEventRef event, void *refcon) {
UniChar unicodeString[101];
UniCharCount unicharCount;
char chars[2];
int i,j,charsLen;
CGEventRef zDown;
CGEventRef zUp;
zDown = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)6, true);
zUp = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)6, false);
//printf("%u %u\n", (uint32_t)type, (uint32_t) event);
CGEventKeyboardGetUnicodeString(event, 100, &unicharCount, unicodeString);
for (i=0; i < unicharCount; i++)
{
if (unicodeString[i] > 127) {
chars[0] = (unicodeString[i] >> 8) & (1 << 8) - 1;
chars[1] = unicodeString[i] & (1 << 8) - 1;
charsLen = 2;
} else {
charsLen = 1;
chars[0] = unicodeString[i];
}
//for (j = 0; j < charsLen; j++) printf("%c", chars[j]);
}
if (chars[0] == 'a')
{
CGEventPost(kCGHIDEventTap, zDown);
CGEventPost(kCGHIDEventTap, zUp);
}
return event;
}
int main (int argc, const char * argv[]) {
CFMachPortRef eventTap;
CFRunLoopSourceRef runLoopSource;
eventTap = CGEventTapCreate(kCGSessionEventTap, kCGHeadInsertEventTap, 0, kCGEventMaskForAllEvents, myCGEventCallback, NULL);
runLoopSource = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, eventTap, 0);
CFRunLoopAddSource(CFRunLoopGetCurrent(), runLoopSource, kCFRunLoopCommonModes);
CGEventTapEnable(eventTap, true);
CFRunLoopRun();
return 0;
}
br,
Juha
You could do this using Quartz Event Taps, which provides a:
...C API for event taps, which are
filters used to observe and alter the
stream of low-level user input events
in Mac OS X
On the Mac, there is the keyboard view. Why can't you use that?
Related
I have been using keybd_event for keyboard input by command line. However, want something that can send hardware level input, just like real keyboard.
Is there anything that suits my purpose?
I am using Windows as operating system.
I once used SendInput to control a game character. Game (icy tower?) was using directx input system (I think?) and somehow it was ignoring keybd_event calls but this method worked. I do not know how close to hardware you need to be but did this the trick for me. I used virtual key codes but turned them into scancodes for this answer.
UINT PressKeyScan(WORD scanCode)
{
INPUT input[1] = {0};
input[0].type = INPUT_KEYBOARD;
input[0].ki.wVk = NULL;
input[0].ki.wScan = scanCode;
input[0].ki.dwFlags = KEYEVENTF_SCANCODE;
UINT ret = SendInput(1, input, sizeof(INPUT));
return ret;
}
UINT ReleaseKeyScan(WORD scanCode)
{
INPUT input[1] = {0};
input[0].type = INPUT_KEYBOARD;
input[0].ki.wVk = NULL;
input[0].ki.wScan = scanCode;
input[0].ki.dwFlags = KEYEVENTF_SCANCODE | KEYEVENTF_KEYUP;
UINT ret = SendInput(1, input, sizeof(INPUT));
return ret;
}
To simulate press and release you use them sequentially (or you may create a separate function for press and release that use same INPUT structure).
WORD scanCodeSpace = 0x39;
PressKeyScan(scanCodeSpace);
ReleaseKeyScan(scanCodeSpace)
You can use MapVirtualKeyA to get scan code from virtual key code.
keybd_event() is deprecated, use SendInput() instead.
SendInput() posts its simulated events to the same queue that the hardware driver posts its events to, as shown in the below diagram from Raymond Chen's blog:
When something gets added to a queue, it takes time for it to come out the front of the queue
I have an application that is used as a control system for a presentation, under Linux and using X11. I have a USB presentation remote that acts as a very miniature keyboard (four buttons: Page Up, Page Down, and two others) which can be used to advance and go back in the presentation. I would like to have my presentation application to receive all of the events from this remote regardless of where the mouse focus is. But I would also like to be able to receive the normal mouse and keyboard events if the current window focus is on the presentation application. Using XIGrabDevice() I was able to receive all events from the remote in the presentation application regardless of the current focus but I was not able to receive any events from the mouse or keyboard while the grab was active.
I ended up setting up a separate program to capture the remote's keys, then I relay those keys to my main program. I did it this way because the original program was using the older XInput extension, and I needed to use the newer XInput2 extension, and they do not exist well together. Here's some C++ code (it doesn't do any error checking, but this should be done in a real program):
// Open connection to X Server
Display *dpy = XOpenDisplay(NULL);
// Get opcode for XInput Extension; we'll need it for processing events
int xi_opcode = -1, event, error;
XQueryExtension(dpy, "XInputExtension", &xi_opcode, &event, &error);
// Allow user to select a device
int num_devices;
XIDeviceInfo *info = XIQueryDevice(dpy, XIAllDevices, &num_devices);
for (int i = 0; i < num_devices; ++i)
{
XIDeviceInfo *dev = &info[i];
std::cout << dev->deviceid << " " << dev->name << "\n";
}
XIFreeDeviceInfo(info);
std::cout << "Enter the device number: ";
std::string input;
std::cin >> input;
int deviceid = -1;
std::istringstream istr(input);
istr >> deviceid;
// Create an InputOnly window that is just used to grab events from this device
XSetWindowAttributes attrs;
long attrmask = 0;
memset(&attrs, 0, sizeof(attrs));
attrs.override_redirect = True; // Required to grab device
attrmask |= CWOverrideRedirect;
Window win = XCreateWindow(dpy, DefaultRootWindow(dpy), 0, 0, 1, 1, 0, 0, InputOnly, CopyFromParent, attrmask, &attrs);
// Make window without decorations
PropMotifWmHints hints;
hints.flags = 2;
hints.decorations = 0;
Atom property = XInternAtom(dpy, "_MOTIF_WM_HINTS", True);
XChangeProperty(dpy, win, property, property, 32, PropModeReplace, (unsigned char *)&hints, PROP_MOTIF_WM_HINTS_ELEMENTS);
// We are interested in key presses and hierarchy changes. We also need to get key releases or else we get an infinite stream of key presses.
XIEventMask evmasks[1];
unsigned char mask0[XIMaskLen(XI_LASTEVENT)];
memset(mask0, 0, sizeof(mask0));
XISetMask(mask0, XI_KeyPress);
XISetMask(mask0, XI_KeyRelease);
XISetMask(mask0, XI_HierarchyChanged);
evmasks[0].deviceid = XIAllDevices;
evmasks[0].mask_len = sizeof(mask0);
evmasks[0].mask = mask0;
XISelectEvents(dpy, win, evmasks, 1);
XMapWindow(dpy, win);
XFlush(dpy);
XEvent ev;
bool grab_success = false, grab_changed;
while (1)
{
grab_changed = false;
if (!grab_success)
{
XIEventMask masks[1];
unsigned char mask0[XIMaskLen(XI_LASTEVENT)];
memset(mask0, 0, sizeof(mask0));
XISetMask(mask0, XI_KeyPress);
masks[0].deviceid = deviceid;
masks[0].mask_len = sizeof(mask0);
masks[0].mask = mask0;
XIGrabDevice(dpy, deviceid, win, CurrentTime, None, XIGrabModeAsync, XIGrabModeAsync, XIOwnerEvents, masks);
}
XNextEvent(dpy, &ev);
XGenericEventCookie *cookie = &ev.xcookie;
if (cookie->type == GenericEvent && cookie->extension == xi_opcode && XGetEventData(dpy, cookie))
{
if (cookie->evtype == XI_KeyPress)
{
XIDeviceEvent *de = (XIDeviceEvent*)cookie->data;
std::cout << "found XI_KeyPress event: keycode " << de->detail << "\n";
}
else if (cookie->evtype == XI_HierarchyChanged)
{
// Perhaps a device was unplugged. The client is expected to re-read the list of devices to find out what changed.
std::cout << "found XI_HierarchyChanged event.\n";
grab_changed = true;
}
XFreeEventData(dpy, cookie);
}
if (grab_changed)
{
XIUngrabDevice(dpy, deviceid, CurrentTime);
grab_success = false;
break;
}
}
I found the following links helpful:
Peter Hutterer's 6-part blog on XInput2: 1 2 3 4 5 6
This blog entry was useful to determine which class to cast the cookie->data pointer to, depending on the cookie->evtype: 7
Is there a way to programmatically launch the "Force Quit Applications" app that can be launched from the 'Apple menu' or by pressing Command-Option-Esc.
I haven't been able to find out whether it's a separate app or perhaps something that can be invoked by a command line parameter to Activity Monitor.
I've tried the simulating the keystrokes but it doesn't seem to work:
CGEventFlags flags = kCGEventFlagMaskAlternate | kCGEventFlagMaskCommand;
CGKeyCode virtualKey = kVK_Escape;
CGEventSourceRef source = CGEventSourceCreate(kCGEventSourceStateHIDSystemState);
CGEventRef KbdEvent = CGEventCreateKeyboardEvent(source, virtualKey, YES);
CGEventSetFlags(KbdEvent, (CGEventFlags)flags);
CGEventTapLocation location = kCGHIDEventTap;
CGEventPost(location, KbdEvent);
CFRelease(KbdEvent);
CFRelease(source);
I also noticed that CGEvents cannot trigger the "Force Quit Applications" window, perhaps it works only on the lower IOHID level (like the Xcode Simulator).
The only way to invoke the window I have found is to use System Events:
system("osascript -l JavaScript -e \"Application('System Events').processes['Finder'].menuBars[0].menus['Apple'].menuItems['Force Quit…'].click()\"");
You can check to see if the window is open with the following:
#include <Carbon/Carbon.h>
int IsForceQuitOpen() {
int found = 0;
CFArrayRef windowList = CGWindowListCopyWindowInfo(kCGWindowListOptionOnScreenOnly, kCGNullWindowID);
CFIndex numWindows = CFArrayGetCount(windowList);
for (int i = 0; i < (int)numWindows; i++) {
CFDictionaryRef info = (CFDictionaryRef)CFArrayGetValueAtIndex(windowList, i);
CFStringRef appName = (CFStringRef)CFDictionaryGetValue(info, kCGWindowOwnerName);
if (CFEqual(appName, CFSTR("loginwindow"))) {
found = 1;
}
}
return found;
}
How to find which com-port is occupied by serial mouse
Here is how I detect mouse in C# (adapted code from this answer)
var info = IntPtr.Zero;
try
{
var guid = new Guid("{4d36e96f-e325-11ce-bfc1-08002be10318}"); // mouses
info = SetupDiGetClassDevsW(ref guid, null, IntPtr.Zero, 0);
if ((int)info == -1) // INVALID_HANDLE_VALUE
throw new Exception(string.Format("Error({0}) SetupDiGetClassDevsW", Marshal.GetLastWin32Error()));
// enumerate mouses
var device = new SP_DEVINFO_DATA();
device.cbSize = (UInt32)Marshal.SizeOf(device);
for (uint i = 0; ; i++)
{
// get device info
if (!SetupDiEnumDeviceInfo(info, i, out device))
{
var error = Marshal.GetLastWin32Error();
if (error == 259) // ERROR_NO_MORE_ITEMS
break;
else
throw new Exception(string.Format("Error({0}) SetupDiEnumDeviceInfo", error));
}
string id = GetStringPropertyForDevice(info, device, 1); // SPDRP_HARDWAREID
if (id != null && id.Contains("*PNP0F09")) // Microsoft BallPoint Serial Mouse
{
// ...
// here I want to check com-port, how?
// ...
}
}
}
finally
{
if (info != IntPtr.Zero)
SetupDiDestroyDeviceInfoList(info);
}
Edit
Removing C# tag. Looking for general info (any language).
You can use Process Monitor from SysInternalSuite and open device manager then find out from where does the device manager getting its values
I tried it on USB Mouse and was able to get (on USB Input Device) as shown below
1. Open Mouse Properties (From Control Panel)
2. Open ProcMon
3. Click on the target icon and choose the mouse properties window
4. From the Mouse Properties window open the Hardware tab
5. In ProcMon Click on File-> Captuer Events
6. In ProcMon Edit->Find and look for "com" without quotation mark
7. Double click the found row (If you where able to find it)
Another solution would be to get device information using device manager command line utility devcon and parse the information from the output stream
More information on devcon:
* http://support.microsoft.com/kb/311272
* https://superuser.com/questions/414280/how-do-i-view-a-list-of-devices-from-the-command-line-in-windows
Hope this help
The subroutine that generates the "Location" string in Device Manager is devmgr.dll!GetLocationInformation.
The path in it that interests you - generating the value that is appended in brackets - can be represented with the following code (based on Hex-Rays' decompilation):
int __stdcall GetLocationInformation(DEVINST dnDevInst, wchar_t *lpsResult,
int cchBufferMax, HMACHINE hMachine)
{
int dwUiNumber;
HKEY hKey;
DWORD pulLength;
wchar_t sRawLocationInfo[260];
sRawLocationInfo[0] = 0;
DWORD Type = REG_SZ;
pulLength = 520;
if ( !CM_Open_DevNode_Key_Ex(dnDevInst, KEY_READ, 0, 1u, &hKey, 1u, hMachine) )
{
RegQueryValueExW(hKey, L"LocationInformationOverride", 0, &Type,
sRawLocationInfo, &pulLength);
RegCloseKey(hKey);
}
if ( !sRawLocationInfo[0] )
{
pulLength = 520;
CM_Get_DevNode_Registry_Property_ExW(
dnDevInst,
CM_DRP_LOCATION_INFORMATION,
0,
sRawLocationInfo,
&pulLength,
0,
hMachine);
}
pulLength = 4;
if ( CM_Get_DevNode_Registry_Property_ExW(
dnDevInst,
CM_DRP_UI_NUMBER,
0,
&dwUiNumber,
&pulLength,
0,
hMachine)
|| pulLength <= 0 )
{
<...> //this block always returns
}
else
{
<...>
if ( sRawLocationInfo[0] )
{
lstrcatW(lpsResult, L" (");
lstrcatW(lpsResult, sRawLocationInfo);
lstrcatW(lpsResult, L")");
}
return 0;
}
}
In a nutshell, the bracketed value is the device node's LocationInformationOverride or LocationInformation property and is only produced if the UiNumber property is absent (or bogus).
The CM_Open_DevNode_Key_Ex and CM_Get_DevNode_Registry_Property_ExW functions are marked "reserved" in the docs. You can
find their signatures e.g. in CM_Open_DevNode_Key_Ex - FileLog and Enumerate Properties of an Installed Device - The Code Project, respectively, or
use the publicly-documented CM_Open_DevNode_Key and CM_Get_DevNode_Registry_Property instead. They are exactly the same as the former ones save for missing the hMachine argument (they essentially directly call them substituting NULL for it).
There are corresponding SetupDi equivalents, SetupDiOpenDevRegKey and SetupDiGetDeviceRegistryProperty, as well but note that this API is new in Vista and uses different data types.
Seeing that you use it already though means that this is probably the way to go for you unless you need XP support.
If my guess is right, the "USB Serial Port (COM6)" you see in Device Manager is actually the name of the parent device (=the device this one is connected to as seen in Device Manager in "view devices by connection" mode). If this is correct, the "COM6" is but a part of the name rather than some independent property.
I am developing a multi-platform game that runs on iOS as well as desktops (Windows, Mac, Linux). I want the game to be able to resize certain UI elements depending on the resolution of the screen in inches. The idea is that if a button should be, say, around 1/2 inch across in any interface, it will be scaled automatically that size.
Now for iOS devices this problem is reasonably well solvable using brute force techniques. You can look up the type of the device and use a hard-coded table to determine the screen size in inches for each device. Not the most elegant solution, but sufficient.
Desktops are the tricky ones. What I wish and hope exists is a mechanism by which (some?) monitors report to operating systems their actual screen size in inches. If that mechanism exists and I can access it somehow, I can get good numbers at least for some monitors. But I've never come across any such concept in any of the major OS APIs.
Is there a way to ask for the screen size in inches in Win32? If so, are there monitors that actually provide this information?
(And if the answer is no: Gosh, doesn't this seem awfully useful?)
For Windows, first see SetProcessDPIAware() for a discussion on turning off automatic scaling, and then call GetDeviceCaps( LOGPIXELSX ) and GetDeviceCaps( LOGPIXELSY ) on your HDC to determine the monitor's DPI. Divide the screen resolution on your active monitor by those settings and you've got the size.
Also see this article for a similar discussion on DPI aware apps.
Here is a method I found at the web address "https://ofekshilon.com/2011/11/13/reading-monitor-physical-dimensions-or-getting-the-edid-the-right-way/".
Authour says that measurement is in millimeters.
Does not give you the precise width and height but better approximation than HORSIZE and VERTSIZE. In which I tried on two different monitors and got a max difference of 38 cm (measured screen size - calculated screen size).
#include <SetupApi.h>
#pragma comment(lib, "setupapi.lib")
#define NAME_SIZE 128
const GUID GUID_CLASS_MONITOR = {0x4d36e96e, 0xe325, 0x11ce, 0xbf, 0xc1, 0x08, 0x00, 0x2b, 0xe1, 0x03, 0x18};
// Assumes hDevRegKey is valid
bool GetMonitorSizeFromEDID(const HKEY hDevRegKey, short& WidthMm, short& HeightMm)
{
DWORD dwType, AcutalValueNameLength = NAME_SIZE;
TCHAR valueName[NAME_SIZE];
BYTE EDIDdata[1024];
DWORD edidsize=sizeof(EDIDdata);
for (LONG i = 0, retValue = ERROR_SUCCESS; retValue != ERROR_NO_MORE_ITEMS; ++i)
{
retValue = RegEnumValueA ( hDevRegKey, i, &valueName[0],
&AcutalValueNameLength, NULL, &dwType,
EDIDdata, // buffer
&edidsize); // buffer size
if (retValue != ERROR_SUCCESS || 0 != strcmp(valueName,"EDID"))
continue;
WidthMm = ((EDIDdata[68] & 0xF0) << 4) + EDIDdata[66];
HeightMm = ((EDIDdata[68] & 0x0F) << 8) + EDIDdata[67];
return true; // valid EDID found
}
return false; // EDID not found
}
// strange! Authour requires TargetDevID argument but does not use it
bool GetSizeForDevID(const char *TargetDevID, short& WidthMm, short& HeightMm)
{
HDEVINFO devInfo = SetupDiGetClassDevsExA(
&GUID_CLASS_MONITOR, //class GUID
NULL, //enumerator
NULL, //HWND
DIGCF_PRESENT, // Flags //DIGCF_ALLCLASSES|
NULL, // device info, create a new one.
NULL, // machine name, local machine
NULL);// reserved
if (NULL == devInfo) return false;
bool bRes = false;
for (ULONG i=0; ERROR_NO_MORE_ITEMS != GetLastError(); ++i)
{
SP_DEVINFO_DATA devInfoData;
memset(&devInfoData,0,sizeof(devInfoData));
devInfoData.cbSize = sizeof(devInfoData);
if (SetupDiEnumDeviceInfo(devInfo,i,&devInfoData))
{
HKEY hDevRegKey = SetupDiOpenDevRegKey(devInfo,&devInfoData,DICS_FLAG_GLOBAL, 0, DIREG_DEV, KEY_READ);
if(!hDevRegKey || (hDevRegKey == INVALID_HANDLE_VALUE)) continue;
bRes = GetMonitorSizeFromEDID(hDevRegKey, WidthMm, HeightMm);
RegCloseKey(hDevRegKey);
}
}
SetupDiDestroyDeviceInfoList(devInfo);
return bRes;
}
int main(int argc, CHAR* argv[])
{
short WidthMm, HeightMm;
DISPLAY_DEVICE dd;
dd.cb = sizeof(dd);
DWORD dev = 0; // device index
int id = 1; // monitor number, as used by Display Properties > Settings
char DeviceID[1024];
bool bFoundDevice = false;
while (EnumDisplayDevices(0, dev, &dd, 0) && !bFoundDevice)
{
DISPLAY_DEVICE ddMon = {sizeof(ddMon)};
DWORD devMon = 0;
while (EnumDisplayDevices(dd.DeviceName, devMon, &ddMon, 0) && !bFoundDevice)
{
if (ddMon.StateFlags & DISPLAY_DEVICE_ATTACHED_TO_DESKTOP &&
!(ddMon.StateFlags & DISPLAY_DEVICE_MIRRORING_DRIVER))
{
sprintf(DeviceID,"%s", ddMon.DeviceID+8);
for(auto it=DeviceID; *it; ++it)
if(*it == '\\') { *it = 0; break; }
bFoundDevice = GetSizeForDevID(DeviceID, WidthMm, HeightMm);
}
devMon++;
ZeroMemory(&ddMon, sizeof(ddMon));
ddMon.cb = sizeof(ddMon);
}
ZeroMemory(&dd, sizeof(dd));
dd.cb = sizeof(dd);
dev++;
}
return 0;
}