Ola Folks,
Once again I want to drink from the pool of knowledge shared by people using SO.
I have written a small app for OSX that sends key events to an application. I am targeting OSX 10.5.x and newer. However, the problem exists when I build for 10.6.x as well. Everything works fine except when I send only the modifier keys; Alt, Command, Control and Shift.
The problem is that on the two MacBooks, the events for the modifier keys appear to be cleared as soon as the testers move the cursor using the mouse or touch the touchpad.
On a desktop with XCode installed, everything works fine. Just like it should. On two different MacBooks, the problem occurs. The desktop has a standard 101 key keyboard and a multi-button mouse attached.
When a mouse is connected to the MacBooks, a two button mouse with scrollwheel is used. However, the problem exists when no peripherals are attached and the touchpad is used.
What I expect to happen is that the Modifier Key event is sent to the target application, the user moves the cursor using the mouse / touchpad, press buttons on the mouse / touchpad and/or press keys on the keyboard with the modifier key down event 'active'. Then, when they finish, the key up event for the modifier key is sent.
Here is how I am sending key down events, the Shift key for this example:
case ModKeyShiftDown:
xEventSource = CGEventSourceCreate(kCGEventSourceStatePrivate);
xTheCommand = CGEventCreateKeyboardEvent(xEventSource, kVK_Shift, true);
CGEventSetFlags(xTheCommand, kCGEventFlagMaskAlternate);
CGEventPost(kCGHIDEventTap, xTheCommand);
//CGEventPost(kCGSessionEventTap, xTheCommand);
//CGEventPost(kCGAnnotatedSessionEventTap, xTheCommand);
CFRelease(xTheCommand);
CFRelease(xEventSource);
break;
I have used all three flags for creating the event source (kCGEventSourceStatePrivate, kCGEventSourceStateCombinedSessionState and kCGEventSourceStateHIDSystemState).
I have tried Creating the Keyboard Event with the Event Source as well as null as the first parameter.
I have tried with and without the appropriate flags on the event.
I have tried various combinations of posting the event; kCGHIDEventTap, kCGSessionEventTap and kCGAnnotatedSessionEventTap.
For completeness, here is how I send an up event for the Shift key:
case ModKeyShiftUp:
xEventSource = CGEventSourceCreate(kCGEventSourceStatePrivate);
xTheCommand = CGEventCreateKeyboardEvent(xEventSource, kVK_Shift, false);
CGEventSetFlags(xTheCommand, 0);
CGEventPost(kCGHIDEventTap, xTheCommand);
//CGEventPost(kCGSessionEventTap, xTheCommand);
//CGEventPost(kCGAnnotatedSessionEventTap, xTheCommand);
CFRelease(xTheCommand);
CFRelease(xEventSource);
break;
When the testers trigger a modifier key down event, they can see the cursor change as expected. This lets me know the event is being processed by the target application. However, as soon as they touch the mouse or touchpad, the cursor changes back to a standard cursor and the mouse events are processed as if no modifier key events are active.
I would like to know if there is a problem with the way I am sending the events. I would also like to know if there is an alternate way to send Modifier Key events that is Going To Work.
Sorry if I overtalked this. My excuse is that I only slept a couple of hours. :P
Thanx
-isdi-
Ola,
Interestingly enough, using AXUIElementRef and AXUIElementPostKeyboardEvent seem to work.
For whatever reason, sending key events using the Accessibility object and method above solves the problem for the testers.
Related
First of all hi guys!
I was trying to write a mouse controller app for mac os x which is reading inputs from keyboard and moves the mouse accordingly. By garbage input i will describe the input was intented for a mouse event but it creates text on screen.
Before anyone points to the fact that there is a built in one, It was laggy even in shortest lag setting and cannot registers more than two buttons at the same time (you have to press diagonals to go to the diagonal.) If you accidentally press another button when release of the accident button your motion stops. My first and last reaction was "rubbish!". Adding customization and extra features is my goal.
I want to create a key combination that will block the garbage input to be passed to other programs while it was held. But global monitoring and seems like it always passes the event. And unfortunately I see qqqqqqqwwwwwww like text in unwanted places.
I want to see that when i press q w and up, it will make the mouse go up. But i create qqqqqqqwwwwww mess on the way. My first idea was creating a view on popover and handle events there, but whenever I want to use my mouse from keyboard seeing a popover is anoying and I couldn't find a way to show the popover without leaving any garbage keyboard input.
What should i do in this situation?
You will want to use Quartz Event Taps. Note that for an application to tap keyboard events, it has to be trusted for accessibility (as in System Preferences > Security & Privacy > Privacy > Accessibility). Your app can ask to be made trusted using AXIsProcessTrustedWithOptions().
I have a system with no keyboard. I can connect a keyboard, but ultimately the inputs come from a custom keypad which is not a HID device, it sends serial data which I can interpret and decode to determine if the user pressed Up, Down, Left, Right, or Enter.
Right now all I have is a Fl_Window, with two Fl_Button widgets. Focus is set for one of the buttons and callbacks are defined for these buttons. I know that if I attach a real keyboard, and use the arrow keys I can change focus from button to button. I do have to hit SPACE to activate a button.
My problem is determining how to cause these key presses using code when I decode the outcome form the embedded key pad. Because in deployment, there will be no actual keyboard.
What I've tried is to invoke int Fl_Window::handle(int) and not really had success. I've also tried to invoke int Fl::handle(int, Fl_Window *) and not had success.
Here are code examples:
if((ret = Fl::handle(FL_Left, window)) == 0)
That compiles, but I find that I get zero back, implying that it did not process the event.
if((ret = Fl_Window::handle(FL_Right)) == 0)
That does not compile, informing me that it "cannot call member function virtual int Fl_Window::handle(int) without object"
I'm thinking that the "int event" actually ought to be FL_KEYDOWN.
That logic leaves me to wonder though how I "set" event_key(). For instance, there are API functions to get that when one has a handler, but I do not wish to get that; I wish to cause that event to occur.
Is my only option here to figure out how to emulate a HID or make some type of virtual HID where I then cause the keyboard events to occur?
I do not feel I require a handler function in my application, I'm fine with the default behaviors which occur and cause my callback functions to be invoked. My problem is that I can't "cause" these events to occur.
You need to assign the desired key to e_keysym, then dispatch a FL_KEYDOWN event using Fl::handle_(). (Fl::handle() will not generate the followup FL_SHORTCUT event.)
Fl::e_keysym = FL_Left;
Fl::handle_(FL_KEYDOWN, window);
// sleep() and/or Fl::wait() as appropriate
Fl::handle_(FL_KEYUP, window);
I have a basic keystroke converter app in development. The conversion works with the following:
CFRunLoopSourceRef runLoopSource = NULL;
CFMachPortRef eventTap = CGEventTapCreate(kCGHIDEventTap, kCGHeadInsertEventTap, kCGEventTapOptionDefault, kCGEventMaskForAllEvents, myCGEventCallback, NULL);
runLoopSource = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, eventTap, 0);
CFRunLoopAddSource(CFRunLoopGetCurrent(), runLoopSource, kCFRunLoopCommonModes);
CGEventTapEnable(eventTap, true);
As you might expect, kCGEventMaskForAllEvents is constantly firing for any mouse movement or click in addition to the keyboard, and I suspect tying up system resources. I tried substituting CGEventMaskBit(kCGEventKeyDown), which best I can tell from Quartz Event doc on Event Types is what I want, and would weed out mouse movements and clicks. Unfortunately, using this seems to just eat the keystrokes, rather than convert them.
What am I doing wrong?
The following works, but I still don't understand why CGEventMaskBit(kCGEventKeyUp) by itself isn't the correct implementation.
CGEventMaskBit(kCGEventKeyUp) | CGEventMaskBit(kCGEventKeyDown) | CGEventMaskBit(NX_SYSDEFINED)
because a keystroke key press consists of a keydown and a key up
The discussion section of the CGEventTapCreate doc page says:
Event taps receive key up and key down events if one of the following conditions is true:
The current process is running as the root user.
Access for assistive devices is enabled. In OS X v10.4, you can enable this feature using System Preferences, Universal Access panel, Keyboard view.
Running as the root user definitely worked for me (MacOS Sierra.) I didn't try the assistive devices approach.
To run as root inside XCode (I have 8.3.3 at this time), choose Product/Scheme/Edit Scheme.../Run/Info/Debug Process As: root
In the CGEventTapCreate call, replace the kCGEventMaskForAllEvents argument with CGEventMaskBit(kCGEventKeyDown) | CGEventMaskBit(kCGEventKeyUp). Your callback will now get invoked for most key presses, except for modifier keys: shift, ctrl, cmd, some of teh function keys.
To get the callback invoked for the modifier keys, add CGEventMaskBit(kCGEventKeyDown) | CGEventMaskBit(kCGEventKeyUp) | CGEventMaskBit(NX_SYSDEFINED). For some reason, with this change I also get the callback invoked for mouse button presses. This might be a side effect of how the Logitech mouse driver works -- I didn't investigate. But the volume of calls is much lower than before and doesn't include mouse moves.
Dave Keck's response to this CocoaBuilder thread gets credit for figuring this out.
How can I fire an automatic key press or mouse click event when a color appears on the screen
on other application or browser?
It depends a lot on what you want. Do you want to send the keys to
your Application
another fixed Application
Simulate a global keypress
Simulating keys globally
All of these will cause problems targeting a specific application and the active window changes.
SendKeys Sends Messages to the active app. It's a high level function taking a string which encodes a sequence of keys.
keybd_event is very low level and injects a global keypress. In most cases SendKeys is easier to use.
mouse_event simulates mouse input.
SendInput supersedes these functions. It's more flexible but a bit harder to use.
Sending to a specific window
When working with a fixed target window, sending it messages can work depending on how the window works. But since this doesn't update all states it might not always work. But you don't have a race condition with changing window focus, which is worth a lot.
WM_CHAR sends a character in the basic multilingual plane (16 bit)
WM_UNICHAR sends a character supporting the whole unicode range
WM_KEYDOWN and WM_KEYUP Sends keys which will be translated to characters by the keyboard layout.
My recommendation is when targeting a specific window/application try using messages first, and only if that fails try one of the lower level solutions.
when a color appears on the screen on other application or browser
I made one program using OpenCV and C++ for operating mouse with finger gesture. I used 3 color strips for 3 mouse function.
Yellow color for Left click
Blue color for Right click
Pink color for controlling cursor position
Whenever camera detect these colors, associated function takes place, I have used mouse_event for performing mouse function.
For more information you may read my code, blog, video.
I'm not 100% sure what you want, but if all you are after is running the method linked the the button.Clicked event, then you can manually run the method just like any other method.
You can use the .NET SendKeys class to send keystrokes.
Emulating mouse clicks requires P/Invoke.
I don't know how to detect colors on the screen.
Is it possible to set the virtual key state / mouse button state for all programs on a computer without triggering the associated events at the same time (like setting the left mouse button to be currently pressed without sending a mouse down event). Could this be achieved by setting up a fake keyboard or mouse driver for a fake keyboard or mouse that when queried as to the current state of a key would reply giving an answer of your choice or by any other means?
Edit:
This has to affect programs that I do not have the code for running in other threads ect...
Well, I don't have a complete answer for you but...
The Win32 function SetKeyboard State() changes the state of the keyboard for the thread that called it.
I know this does not affect the entire system, but if you only need the keyboard state changed for applications you are developing, then this could be called by each program, reading in the array passed to it from a temporary file on the harddrive.
Otherwise... don't know of anything else offhand...