Getting mouse coordinates on Mojave - macos

I have a really basic little command line app that grabs the mouse coordinates the next time the mouse is clicked.
#import <Foundation/Foundation.h>
#import <AppKit/AppKit.h>
CGEventRef myCGEventCallback(CGEventTapProxy proxy, CGEventType type, CGEventRef event, void *refcon) {
CGFloat displayScale = 1.0f;
if ([[NSScreen mainScreen] respondsToSelector:#selector(backingScaleFactor)])
{
displayScale = [NSScreen mainScreen].backingScaleFactor;
}
CGPoint loc = CGEventGetLocation(event);
CFRelease(event);
printf("%dx%d\n", (int)roundf(loc.x * displayScale), (int)roundf(loc.y * displayScale) );
exit(0);
return event;
}
int main(int argc, const char * argv[]) {
#autoreleasepool {
CFMachPortRef eventTap;
CGEventMask eventMask;
CFRunLoopSourceRef runLoopSource;
eventMask = 1 << kCGEventLeftMouseDown;
eventTap = CGEventTapCreate(kCGSessionEventTap, kCGHeadInsertEventTap,
1, eventMask, myCGEventCallback, #"mydata");
runLoopSource = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, eventTap, 0);
CFRunLoopAddSource(CFRunLoopGetCurrent(), runLoopSource,
kCFRunLoopCommonModes);
CGEventTapEnable(eventTap, true);
CFRunLoopRun();
}
return 0;
}
I'm building it with cmake with the following file:
cmake_minimum_required(VERSION 3.0.0)
project (location)
set(CMAKE_C_FLAGS "-arch x86_64 -mmacosx-version-min=10.12 -std=gnu11 -fobjc-arc -fmodules")
This all worked fine until the upgrade to Mojave.
A bit of poking around shows this is down to the latest set of security updates and some hints (except CGEventTapCreate() is not returning null) about settings some values in Info.plist to allow the app to use the accessibility API. But I'm struggling to work out where to put it as I just have a single .m file with the code.
Edit
This needs to run as a none root user (company policy)
if the only way to get it to ask for permission then it can be extended to be a "GUI" app with a minimal UI
This app is just to grab the upper left hand corner of a region of the screen to feed to a second app that streams that area of screen to a second device. The code for the streamer is common across Win/Linux/MacOS so trying to keep the screen coordinate collection totally separate

As you surmise, event taps won't work on Mojave without having accessibility access. From the documentation:
Event taps receive key up and key down events if one of the following
conditions is true: The current process is running as the root user.
Access for assistive devices is enabled. In OS X v10.4, you can enable
this feature using System Preferences, Universal Access panel,
Keyboard view.
A GUI app will prompt the user to enable accessibility the first time it's needed, but it looks like a CLI app doesn't do that (which makes sense).
There is no way to enable this programatically or through a script; the user must do it themselves.
Running your tool as root should work - can you enforce that?
Otherwise, you can direct the user to the correct place in System Preferences:
tell application "System Preferences"
reveal anchor "Privacy_Accessibility" of pane id "com.apple.preference.security"
activate
end tell
It may be possible using Carbon, if your app isn't sandboxed.
Finally, a quick test shows this is at least possible using IOHID. I shameless borrowed the KeyboardWatcher class from this answer. Then, modified the device type:
[self watchDevicesOfType:kHIDUsage_GD_Keyboard];
into:
[self watchDevicesOfType:kHIDUsage_GD_Mouse];
Finally, my callback looks like this:
static void Handle_DeviceEventCallback (void *inContext, IOReturn inResult, void *inSender, IOHIDValueRef value)
{
IOHIDElementRef element = IOHIDValueGetElement(value);
IOHIDElementType elemType = IOHIDElementGetType(element);
if (elemType == kIOHIDElementTypeInput_Button)
{
int elementValue = (int) IOHIDValueGetIntegerValue(value);
// 1 == down 0 == up
if (elementValue == 1)
{
CGEventRef ourEvent = CGEventCreate(NULL);
CGPoint point = CGEventGetLocation(ourEvent);
printf("Mouse Position: %.2f, y = %.2f \n", (float) point.x, (float) point.y);
}
}
}
That is really a quick hack job, but it demonstrates this is possible and hopefully you can refine it to your needs.

I've found the CGEventTap documentation is out of date beginning with Mojave. Running as root used to act as a bypass for certain entitlements, but in Mojave this was tightened down. One bizarre side effect, as you noticed, is that root can still acquire the mach port for the tap; its just that no events can be read from it. If you try your application without running as root you should get the expected popup asking for permission.
If you do not get the popup, or need to run as root for other purposes, you can manually add your application to the trusted TCC database via SystemPreferences -> Security & Privacy -> Privacy -> Accessibility
settings some values in Info.plist to allow the app to use the accessibility API
I believe you mean adding entitlements (which are also a plist). The entitlement that allows an application to use the Accessibility API is the com.apple.private.tcc.allow entitlement (with a value of kTCCServiceAccessibility). As you can probably guess from the name it is only allowed on Apple signed binaries.
You can add these entitlements to your own app if you disable System Integrity Protection (SIP) and boot the kernel with the option amfi_get_out_of_my_way=1, but I wouldn't recommend it (and certainly any customers of yours wouldn't want to). With just SIP disabled you could manually add an entry to the TCC database to grant privileges, but still wouldn't recommend it.
Possible Alternative
You can use an event monitor:
NSEventMask mask = (NSLeftMouseDownMask | NSRightMouseDownMask | NSOtherMouseDownMask);
mouseEventMonitor = [NSEvent addGlobalMonitorForEventsMatchingMask: mask
handler:^(NSEvent *event){
// get the current coordinates with this
NSPoint coords = [NSEvent mouseLocation];
// event cooordinates would be event.absoluteX and event.absoluteY
... do stuff
}];
The documentation does mention:
Key-related events may only be monitored if accessibility is enabled or if your application is trusted for accessibility access (see AXIsProcessTrusted).
But I don't think that applies to mouse events.

Related

Mac Catalyst - control window resize

I have an app for ipad/iphone, now adding also mac support by Mac Catalyst. On Mac, I want to control window resizing, in order to allow only some sizes and aspects. It goes beyond simple minimal height and weight, or aspect. I want to allow user to resize window freely, but when app gets too high and narrow, I want to also seemlessly increase width, to keep some minimal aspect.
I believe that in AppKit it can be done through NSWindowDelegate.windowWillResize() (get user defined size, count required size and return it). However I am getting error "NSWindowDelegate is unavailable in Mac Catalyst" . Is it possible to achieve the result I want by Catalyst means?
Answering my own question. It is NOT possible to create own NSWindowDelegate with windowWillResize() implemented in Catalyst. However, it IS possible to create a new target only for mac, and use it as a plugin from catalyst target.
First I load mac-only plugin (using Bundle.load() ), and instantiate its principalClass. Then I get NSWindow from UIWindow, which is easy through Dynamic library. Then I pass NSWindow to method of a plugin, which then can set own NSWindowDelegate, because it does not run in catalyst.
Sample code:
guard let bundle = Bundle(url: bundleURL) else { return }
let succ = bundle.load()
if (succ) {
let macUtilsClass = bundle.principalClass! as! MacUtilsProtocol.Type
self.macUtils = macUtilsClass.init()
var dnsw: NSObject? = nil
if (ProcessInfo.processInfo.isOperatingSystemAtLeast(
OperatingSystemVersion(majorVersion: 11, minorVersion: 0, patchVersion: 0))) {
dnsw = Dynamic.NSApplication.sharedApplication.delegate.hostWindowForUIWindow(AppDelegate.ref!.window).attachedWindow
}
else {
dnsw = Dynamic.NSApplication.sharedApplication.delegate.hostWindowForUIWindow(AppDelegate.ref!.window)
}
self.macUtils.SetupMainWindow(win: dnsw!)
}

What is the right API to use to remap keys in OSX?

I'm looking at Karabiner, the OSX keyboard remapper for Parallels development. According to this issue this is best fixed by filing a bug with Apple.
What is the alternative (if any) supported API to create such a functionality?
As noted by pkamb on the github issue that OP linked to (added since he posted), Apple technote TN2450 documents the proper API:
The IOKit HID APIs can be used for key remapping. The user will
provide a dictionary of key remapping that the HID event system will
apply to the keyboard. Listing 3 : Key Remapping using IOKit HID
APIs. // compiled with Xcode 8.2.1
#import <Foundation/Foundation.h>
#import <IOKit/hidsystem/IOHIDEventSystemClient.h>
#import <IOKit/hidsystem/IOHIDServiceClient.h>
#import <IOKit/hid/IOHIDUsageTables.h>
int main(int argc, char *argv[])
{
IOHIDEventSystemClientRef system;
CFArrayRef services;
uint64_t aKey = 0x700000004;
uint64_t bKey = 0x700000005;
NSArray *map = #[
#{#kIOHIDKeyboardModifierMappingSrcKey:#(aKey),
#kIOHIDKeyboardModifierMappingDstKey:#(bKey)},
#{#kIOHIDKeyboardModifierMappingSrcKey:#(bKey),
#kIOHIDKeyboardModifierMappingDstKey:#(aKey)},
];
system = IOHIDEventSystemClientCreateSimpleClient(kCFAllocatorDefault);
services = IOHIDEventSystemClientCopyServices(system);
for(CFIndex i = 0; i < CFArrayGetCount(services); i++) {
IOHIDServiceClientRef service = (IOHIDServiceClientRef)CFArrayGetValueAtIndex(services, i);
if(IOHIDServiceClientConformsTo(service, kHIDPage_GenericDesktop, kHIDUsage_GD_Keyboard)) {
IOHIDServiceClientSetProperty(service, CFSTR(kIOHIDUserKeyUsageMapKey), (CFArrayRef)map);
}
}
CFRelease(services);
CFRelease(system);
return 0;
}
Example:
If you are trying to re-map the command / option / control keys on an external USB keyboard, take a look at the enum's defined in IOHIDUsageTables.h. For example, a Z-88 keyboard can be remapped to match the key layout on a MacBookPro using this mapping:
NSArray *remap = #[
#{#kIOHIDKeyboardModifierMappingSrcKey:#(0x7000000E3),
#kIOHIDKeyboardModifierMappingDstKey:#(0x7000000E2)},
#{#kIOHIDKeyboardModifierMappingSrcKey:#(0x7000000E2),
#kIOHIDKeyboardModifierMappingDstKey:#(0x7000000E3)},
#{#kIOHIDKeyboardModifierMappingSrcKey:#(0x7000000E6),
#kIOHIDKeyboardModifierMappingDstKey:#(0x7000000E7)},
#{#kIOHIDKeyboardModifierMappingSrcKey:#(0x700000065),
#kIOHIDKeyboardModifierMappingDstKey:#(0x7000000E6)}
];
LeftGUI (E3) → LeftAlt (E2)
LeftAlt (E2) → LeftGUI (E3)
RightAlt (E6) → RightGUI (E7)
Application (65) → RightAlt (E6)
To undo a mapping, map the same keys back to their original values. Using the above example:
NSArray *unmap = #[
#{#kIOHIDKeyboardModifierMappingSrcKey:#(0x7000000E3),
#kIOHIDKeyboardModifierMappingDstKey:#(0x7000000E3)},
#{#kIOHIDKeyboardModifierMappingSrcKey:#(0x7000000E2),
#kIOHIDKeyboardModifierMappingDstKey:#(0x7000000E2)},
#{#kIOHIDKeyboardModifierMappingSrcKey:#(0x7000000E6),
#kIOHIDKeyboardModifierMappingDstKey:#(0x7000000E6)},
#{#kIOHIDKeyboardModifierMappingSrcKey:#(0x700000065),
#kIOHIDKeyboardModifierMappingDstKey:#(0x700000065)}
];
Karabiner-Elements is solid and versatile despite the sketchy documentation and user interface, excusable for a work in progress. There are many useful examples, e.g., this one which required dedicated tools and the Karabiner error log to generate the tricky JSON rules. Any key can be both a modifier and regular key, run a program, etc., all at once. Public domain on GitHub.

OSX 10.10 file change event from dispatch queue

I try to monitor file changes on OSX 10.10, starting with a fresh Cocoa application in Xcode, just adding the following code.
If I uncomment the last line in the snippet then I receive the file change events perfectly fine. But I can not make this last call because it should be a Cocoa GUI application.
I digged through a lot of documentation and can't find my error. Do I have to initialize or start this whole dispatch subsystem somehow?
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
int fd = open("<FILENAME>", O_EVTONLY);
if (fd == -1) return;
dispatch_queue_t qu = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
if (!qu) {
printf("can not get queue");
return;
}
unsigned long mask =
DISPATCH_VNODE_DELETE |
DISPATCH_VNODE_WRITE |
DISPATCH_VNODE_EXTEND |
DISPATCH_VNODE_ATTRIB |
DISPATCH_VNODE_LINK |
DISPATCH_VNODE_RENAME |
DISPATCH_VNODE_REVOKE;
dispatch_source_t source = dispatch_source_create(DISPATCH_SOURCE_TYPE_VNODE, fd, mask, qu);
printf("source created\n");
if (!source) {
close(fd);
return;
}
printf("source valid\n");
dispatch_source_set_event_handler(source, ^{
printf("FILE CHANGED\n");
});
dispatch_resume(source);
printf("source resumed\n");
// If I call dispatch_main() I will receive the file system events as expected.
// But as a Cocoa application, I must not call this.
// Instead, I was under the impression that NSApplicationMain handles this.
//dispatch_main();
}
Grand Central Dispatch objects, such as dispatch sources, are automatically retained and released by ARC in recent versions of the compiler and frameworks.
At the end of your method, the last strong reference to source is lost and ARC is issuing an automatic dispatch_release(source). (It would also release the queue, but the source has another strong reference to that. So, if the source survived, so would the queue.)
You need to keep a strong reference to the source in an instance variable.

How do you implement global keyboard hooks in Mac OS X?

I know this can be done for Windows and that XGrabKey can be used for X11, but what about Mac OS X? I want create a class that allows setting shortcut keys that can be invoked even when the application windows are inactive.
This is not (yet?) supported in Cocoa. You can still use the old Carbon library for this (which is 64 bit compatible), but unfortunately Apple decided to remove all documentation on the subject.
There's a nice blog article here: http://dbachrach.com/blog/2005/11/program-global-hotkeys-in-cocoa-easily/
The article is a bit lengthy for my taste, so here is the short version:
- (id)init {
self = [super init];
if (self) {
EventHotKeyRef hotKeyRef;
EventHotKeyID hotKeyId;
EventTypeSpec eventType;
eventType.eventClass = kEventClassKeyboard;
eventType.eventKind = kEventHotKeyPressed;
InstallApplicationEventHandler(&mbHotKeyHandler, 1, &eventType, NULL, NULL);
hotKeyId.signature = 'hotk';
hotKeyId.id = 1337;
RegisterEventHotKey(kVK_ANSI_C, cmdKey + shiftKey, hotKeyCopyId, GetApplicationEventTarget(), 0, &hotKeyRef);
}
}
OSStatus mbHotKeyHandler(EventHandlerCallRef nextHandler, EventRef event, void *userData) {
// Your hotkey was pressed!
return noErr;
}
The hotkey is registered with the RegisterEventHotKey(…) call. In this case it registers CMD + Shift + C.
The ANSI keys are defined in HIToolbox/Events.h, so you can look around there for other keys (just press CMD + Shift + O in XCode, and type Events.h to find it).
You have to do a bit more work if you want multiple hotkeys or if you want to call methods from your handler, but that's all in the link near the top of this answer.
I've been looking for a simple answer to this question, so I hope this helps someone else...
Take a look at addGlobalMonitorForEventsMatchingMask:handler: class methods of NSEvent. Also you may find Shortcut Recorder handy.

Window move and resize APIs in OS X

I'm trying to find documented (or, undocumented, if that's my only option) APIs on OS X to query a list of windows from the window server and then cause the windows to move and resize. Can anyone point me in the right direction? I guess I'd be starting with something like FindWindowEx and MoveWindow under Win32.
Note that I want to do this from an external process - I'm not asking how to control just my own app's window size and position.
Use the Accessibility API. Using this API you can connect to a process, obtain a list of windows (actually an array), get the positions and sizes of each window and also change window properties if you like.
However, an application can only be using this API if the user has enabled access for assistive devices in his preferences (System Prefs -> Universal Access), in which case all applications may use this API, or if your application is a trusted assitive application (when it is trusted, it may use the API, even if this option is not checked). The Accessibility API itself offers the necessary functions to make your application trusted - basically you must become root (using security services to request root permissions of the user) and then mark your process as trusted. Once your application has been marked trusted, it must be restarted as the trusted state is only checked on start-up and can't change while the app is running. The trust state is permanent, unless the user moves the application somewhere else or the hash of the application binary changes (e.g. after an update). If the user has assistive devices enabled in his prefs, all applications are treated as if they were trusted. Usually your app would check if this option is enabled, if it is, go on and do your stuff. If not, it would check if it is already trusted, if it is, again just do your stuff. If not try to make itself trusted and then restart the application unless the user declined root authorization. The API offers all necessary functions to check all this.
There exist private functions to do the same using the Mac OS window manager, but the only advantage that would buy you is that you don't need to be a trusted Accessibility application (which is a one time operation on first launch in most cases). The disadvantages are that this API may change any time (it has already changed in the past), it's all undocumented and functions are only known through reverse engineering. The Accessibility however is public, it is documented and it hasn't change much since the first OS X version that introduced it (some new functions were added in 10.4 and again in 10.5, but not much else has changed).
Here's a code example. It will wait 5 seconds, so you can switch to a different window before it does anything else (otherwise it will always work with the terminal window, rather boring for testing). Then it will get the front most process, the front most window of this process, print it's position and size and finally move it by 25 pixels to the right. You compile it on command line like that (assuming it is named test.c)
gcc -framework Carbon -o test test.c
Please note that I do not perform any error checking in the code for simplicity (there are various places that could cause the program to crash if something goes wrong and certain things may/can go wrong). Here's the code:
/* Carbon includes everything necessary for Accessibilty API */
#include <Carbon/Carbon.h>
static bool amIAuthorized ()
{
if (AXAPIEnabled() != 0) {
/* Yehaa, all apps are authorized */
return true;
}
/* Bummer, it's not activated, maybe we are trusted */
if (AXIsProcessTrusted() != 0) {
/* Good news, we are already trusted */
return true;
}
/* Crap, we are not trusted...
* correct behavior would now be to become a root process using
* authorization services and then call AXMakeProcessTrusted() to make
* ourselves trusted, then restart... I'll skip this here for
* simplicity.
*/
return false;
}
static AXUIElementRef getFrontMostApp ()
{
pid_t pid;
ProcessSerialNumber psn;
GetFrontProcess(&psn);
GetProcessPID(&psn, &pid);
return AXUIElementCreateApplication(pid);
}
int main (
int argc,
char ** argv
) {
int i;
AXValueRef temp;
CGSize windowSize;
CGPoint windowPosition;
CFStringRef windowTitle;
AXUIElementRef frontMostApp;
AXUIElementRef frontMostWindow;
if (!amIAuthorized()) {
printf("Can't use accessibility API!\n");
return 1;
}
/* Give the user 5 seconds to switch to another window, otherwise
* only the terminal window will be used
*/
for (i = 0; i < 5; i++) {
sleep(1);
printf("%d", i + 1);
if (i < 4) {
printf("...");
fflush(stdout);
} else {
printf("\n");
}
}
/* Here we go. Find out which process is front-most */
frontMostApp = getFrontMostApp();
/* Get the front most window. We could also get an array of all windows
* of this process and ask each window if it is front most, but that is
* quite inefficient if we only need the front most window.
*/
AXUIElementCopyAttributeValue(
frontMostApp, kAXFocusedWindowAttribute, (CFTypeRef *)&frontMostWindow
);
/* Get the title of the window */
AXUIElementCopyAttributeValue(
frontMostWindow, kAXTitleAttribute, (CFTypeRef *)&windowTitle
);
/* Get the window size and position */
AXUIElementCopyAttributeValue(
frontMostWindow, kAXSizeAttribute, (CFTypeRef *)&temp
);
AXValueGetValue(temp, kAXValueCGSizeType, &windowSize);
CFRelease(temp);
AXUIElementCopyAttributeValue(
frontMostWindow, kAXPositionAttribute, (CFTypeRef *)&temp
);
AXValueGetValue(temp, kAXValueCGPointType, &windowPosition);
CFRelease(temp);
/* Print everything */
printf("\n");
CFShow(windowTitle);
printf(
"Window is at (%f, %f) and has dimension of (%f, %f)\n",
windowPosition.x,
windowPosition.y,
windowSize.width,
windowSize.height
);
/* Move the window to the right by 25 pixels */
windowPosition.x += 25;
temp = AXValueCreate(kAXValueCGPointType, &windowPosition);
AXUIElementSetAttributeValue(frontMostWindow, kAXPositionAttribute, temp);
CFRelease(temp);
/* Clean up */
CFRelease(frontMostWindow);
CFRelease(frontMostApp);
return 0;
}
Sine Ben asked how you get a list of all windows in the comments, here's how:
Instead of kAXFocusedWindowAttribute you use kAXWindowsAttribute for the AXUIElementCopyAttributeValue function. The result is then no AXUIElementRef, but a CFArray of AXUIElementRef elements, one for each window of this application.
I agree that Accessibility is the best way forward. But if you want quick-and-dirty, AppleScript will work as well.

Resources