Soon I will have to work with OS X and tools like hammerspoon are missing some important capabilities for me. I need to be able to intercept keyboard and mouse events completely from the focused application. Say I ctrl+alt+apple+left_click on an application, I don't want the application to know about that left click. So far the only thing I came up with was to build a transparent fullscreen application, though I'm not sure how feasible that is yet.
Any better idea or hints how to go about this in a language of your choice?
Thanks!
You will need to create an event tap. However, the application will have to run as the root user, or the user will have to authorize that the application has been granted rights to accessibility features.
Apple's documentation can be found here.
Interestingly enough, I am in the process of writing a blog post about how to use event taps (including an ObjectiveC API that I wrote for my own use), but the post won't be made available for another week or so.
Related
I'm looking for a Mac OS X Accessibility API to get the coordinates of the location of the current keyboard (not mouse) focus. According to page 2 of the document I found at http://www.apple.com/accessibility/pdf/Mac_OS_X_Tiger_vpat.pdf, it's doable:
Supported:
Mac OS X exposes the location of the current keyboard and
mouse focus to assistive technologies via the Accessibility API and
also provides a visual indication of the focus on-screen.
Despite the statement above, I can't seem to find the API itself. I'm a seasoned dev (coding since 1982), but have never developed on Mac OS X; please be gentle.
OSX appears to have an asymmetric accessibility API; you can use the NSAccessibilityProtocol to make your own app accessible, but to access accessibility of another app, you have to use a separate set of interfaces/objects, AXUIElement and friends.
I found an article on Retreiving the window that has focus that may be of use here: seems the key steps are:
Use AXUIElementCreateSystemWide to create a 'system wide' accessibility object
Ask that object for the currently focused application by calling AXUIElementCopyAttributeValue asking for kAXFocusedApplicationAttribute
Ask the returned object for the focused window again using AXUIElementCopyAttributeValue, but this time for NSAccessibilityFocusedWindowAttribute - actually looks like you can skip this step below, and go straight from focused application to focused UI Element...
Ask the returned object for the currently focused element using the same API again, but this time with NSAccessibilityFocusedUIElementAttribute
Ask that element for its kAXSizeAttribute / kAXPositionAttribute
You might also want to check out the source code for UIElementInspector which displays information about the element under the mouse pointer (though it doesn't appear to do anything with focus).
Also looks like you'll need to enable the accessibility API either via GUI (see article above) or via terminal for any of the above to work - presumably this is to give users a defense against rogue apps taking control of their desktop.
I haven't used any of these personally (yet); but I'm familiar enough with accessibility APIs to know where to look - hope this helps.
I have a cross platform Qt application that's running into some trouble in OSX. There's a feature that OSX has that I didn't even know existed - the 'Help' key. My MBP doesn't have one, and neither does my Apple wired keyboard purchased a year ago. It seems that this is mostly something that older Macs have. Apparently it generates the same scan code as the Insert key on PC keyboards.
Anyways, when the Help key is pressed, the cursor over our application (or any application that receives the Help key event) turns into a little question mark. This seems to be part of what's called 'context-sensitive help mode', as documented in the NSHelpManager's setContextHelpModeActive: method and in the NSApplication's activateContextHelpMode: method docs. From the docs:
In this mode, the cursor becomes a question mark, and help appears for any user interface item the user clicks.
Most applications don’t use this method. Instead, applications enter
context-sensitive mode when the user presses the Help key.
Applications exit context-sensitive help mode upon the first event
after a help window is displayed.
How many Cocoa developers actually know about this? I'm assuming that clicking on something in the application with this question mark cursor should do something like bring up a help message, but I haven't found a single Cocoa application where it actually does anything at all - not even Apple's apps do anything. In fact, it even seems to put a lot of applications into a strange mode where the cursor text selection is enabled.
The problem is that when we change the application cursor programmatically in Qt when we're in this help-question-cursor-mode, bad things happen. Specifically, our application actually crashes. The crash happens deep inside Cocoa in the NSApplication's NSHelpManager. I'd like to find out why we're seeing this crash, but I'm actually more interested in how we can suppress this 'help' mode. There's nothing in Qt or Cocoa that I can see that would stop it, other than perhaps intercepting and squashing an event, which I haven't tried yet.
Does anyone know any more about this?
I've been searching forever for a solution to this, so I thought I'd seek out the brainpower of greater minds than mine. I'm developing a Cocoa app that uses the Accessibility API to manipulate another program (it's a hotkey app). The app I'm controlling typically has multiple windows open, with some hidden behind others. What I would like to do, if it's possible, is to send mouse events to windows using the Accessibility API in a way that presses a button in the window without bringing it to the foreground (interact with the window but don't activate it). The reason I'm trying to do this is that sending the mouse event to this other window will force it to the foreground and disrupt the user's interaction with the foremost window.
This is possible on Windows - apparently, because apps similar to mine do it there - but I'm getting the feeling that this isn't possible with Cocoa, given the way the window manager works. Am I mistaken?
Accessibility is higher-level than that. You send, for example, AXPress actions to AXButton objects, but “press” is not necessarily a click—pressing the space bar while a view is focused, for example, is also a “press”. AXPress is a high-level action that means “do your thing”, which obviously has meaning for some views (such as buttons) and not others (such as fields).
Accessibility activating the application does make sense when you look at it from its intended purpose: Assistive devices for disabled users. If the user “presses” something by whatever means, they probably intend to activate the application and work in it.
Quartz Event Services will get you almost there: You can create an event tap for the process you want to control, and you can forge events and send them to a tap. The catch is that you can only send events to a tap when the tap fires—i.e., when the application already has an event to deal with. When it doesn't, you're stuck.
I'm facing a problem for an application I'm writing (http://code.google.com/p/blazingstars/issues/detail?id=25), where my program is a menulet (menu bar) application that uses the Accessibility API to interact with and control another program. I do the usual things like registering for the API notifications and getting the window list through API calls, etc., but I realized a while ago that if my program is started in a second Space (virtual desktop) after the program I'm interacting with is started in the first, my program will crash and burn because it can't access any information about its target. (Is there a way around that problem I'm missing?)
A simple solution would be to popup a dialog asking the user to restart the program in the correct Space, but for the life of me I can't figure out how to tell which Space my target is in, either through NSWorkspace or the Accessibility API, so that I can compare it to the Space that I'm in. Any ideas?
Note that setting the collection behaviour to NSWindowCollectionBehaviorCanJoinAllSpaces isn't going to do me any good because I have to do a bunch of work upon launch, so I have to be in the same space as my target right from the start.
I think you can do this with the APIs in CGWindow.h..
Specifically see CGWindowListCopyWindowInfo() and kCGWindowWorkspace.
I've used these APIs to do all types of things like getting window contents, window frames, etc...
If that doesn't work then you might want to try this private API:
extern CGSError CGSGetWindowWorkspace(const CGSConnectionID cid,
CGSWindowID wid,
CGSWorkspaceID *workspace);
The trick would be getting the connection ID of the target process.
You should probably redesign your app so that it delays its initialization until the app you want to control is in the current space.
There is no easy way to do this under Leopard because there are no official "space change" notifications, but the blog post and comments on this page may help.
Using the Apple OS X Cocoa framework, how can I post a sheet (slide-down modal dialog) on the window of another process?
Edit: Clarified a bit:
My application is a Finder extension to do Subversion version control (http://scplugin.tigris.org/). Part of my application is a plug-in (a Contextual Menu Item for Finder); the bulk of my application, however, is in a separate daemon proces. For several reasons, we've chosen to put virtually all the code into the daemon; the plug-in only defines the menu itself, and Apple-Events over to the Daemon.
Sometimes, the daemon needs to prompt the user for further information. It can toss a window on-screen for this, but that's disruptive (randomly positioned), and it seems to me the work flow here is legitimately modal, for example "select a file, pick 'commit' from the menu, provide commit comments, do the operation."
Interprocess cooperation (such as passing a reference of some kind) is acceptable: both processes are mine, but I want to avoid binding the sheet's code into the primary process.
Really, it sounds like you're trying to have your inter-process communication happen at the view level, which isn't really how Cocoa generally works. Things will be much easier if you separate your layers a bit more than that.
Why don't you want to put the sheet code into the other process? It's view code, and view code is inherently process-specific. The right thing to do here is probably to add somewhat generic modal-sheet support to your plugin code, and an IPC call that your daemon can make to summon that code. Trying to ship view objects over to the remote process is going to be nightmarish if you can make it work at all.
You're fighting the frameworks with this approach.
You can't add a sheet to a window in another process, because you have at most only the most restricted access to the windows in the other process.
Please don't do this. Make the interaction nonmodal if at all possible. Especially in something like a commit, it's much nicer to be able to browse around your files while you're writing commit comments.
OS X does have window groups, but I don't think they can (easily) span applications.
Another thing to consider is that in OS X it's possible to have many Finder windows open on the same folder (unlike in OS 9). Even if you did have sufficient privileges/APIs to add a sheet to a Finder window, it's not like the modality of that window would prevent the user from being able to continue working with the files.
(My personal opinion as a long-time Mac user is that this kind of interaction would drive me right up the wall.)