How to get an array of AXMenuItems from AXMenu? - macos

Question

For my code I am attempting to get an array of AXMenuItems from an AXMenu (AXUIElementRef). The menu logs successfully, and here is my code:
NSArray *anArray = [NSRunningApplication runningApplicationsWithBundleIdentifier:#"com.apple.dock"];
AXUIElementRef anAXDockApp = AXUIElementCreateApplication([[anArray objectAtIndex:0] processIdentifier]);
CFTypeRef aChildren;
AXUIElementCopyAttributeValue(anAXDockApp, kAXChildrenAttribute, &aChildren);
SafeCFRelease(anAXDockApp);
CFTypeRef aMenu = CFArrayGetValueAtIndex(aChildren, 0);
NSLog(#"aMenu: %#", aMenu);
// Get menu items
CFTypeRef aMenuChildren;
AXUIElementCopyAttributeValue(aMenu, kAXVisibleChildrenAttribute, &aMenuChildren);
for (NSInteger i = 0; i < CFArrayGetCount(aMenuChildren); i++) {
AXUIElementRef aMenuItem = [self copyAXUIElementFrom:aMenu role:kAXMenuItemRole atIndex:i];
NSLog(#"aMenuItem: %#", aMenuItem); // logs (null)
CFTypeRef aTitle;
AXUIElementCopyAttributeValue(aMenuItem, kAXTitleAttribute, &aTitle);
if ([(__bridge NSString *)aTitle isEqualToString:#"New Window"] || [(__bridge NSString *)aTitle isEqualToString:#"New Finder Window"]) /* Crashes here (i can see why)*/{
AXUIElementPerformAction(aMenuItem, kAXPressAction);
[NSThread sleepForTimeInterval:1];
break;
}
}
What is the correct way to get the list of AXMenuItems?
Screenshot of the Accessibility Inspector:

Answer 1

I have figured out an answer, using #Willeke answer of using AXUIElementCopyElementAtPosition() to get the menu. Since there were multiple dock orientations and hiding, I had to create enums in the .h file as it would be easier to read than 0, 1, or 2.
// .h
typedef enum {
kDockPositionBottom,
kDockPositionLeft,
kDockPositionRight,
kDockPositionUnknown
} DockPosition;
typedef enum {
kDockAutohideOn,
kDockAutohideOff
} DockAutoHideState;
Then, I added the methods to get these states in the .m
// .m
- (DockPosition)dockPosition
{
NSRect screenRect = [[NSScreen mainScreen] frame];
NSRect visibleRect = [[NSScreen mainScreen] visibleFrame];
// Dont need to remove menubar height
visibleRect.origin.y = 0;
if (visibleRect.origin.x > screenRect.origin.x) {
return kDockPositionLeft;
} else if (visibleRect.size.width < screenRect.size.width) {
return kDockPositionRight;
} else if (visibleRect.size.height < screenRect.size.height) {
return kDockPositionBottom;
}
return kDockPositionUnknown;
}
- (DockAutoHideState)dockHidden
{
NSString *plistPath = [NSHomeDirectory() stringByAppendingPathComponent:#"Library/Preferences/com.apple.dock.plist"];
NSDictionary *dockDict = [NSDictionary dictionaryWithContentsOfFile:plistPath];
CFBooleanRef autohide = CFDictionaryGetValue((__bridge CFDictionaryRef)dockDict, #"autohide");
if (CFBooleanGetValue(autohide) == true) {
return kDockAutohideOn;
}
return kDockAutohideOff;
}
For the dock position unknown, I added in case it was not able to calculate it from the screen positions.
Then, I used a method to get the dock item from the menubar:
- (AXUIElementRef)getDockItemWithName:(NSString *)name
{
NSArray *anArray = [NSRunningApplication runningApplicationsWithBundleIdentifier:#"com.apple.dock"];
AXUIElementRef anAXDockApp = AXUIElementCreateApplication([[anArray objectAtIndex:0] processIdentifier]);
AXUIElementRef aList = [self copyAXUIElementFrom:anAXDockApp role:kAXListRole atIndex:0];
CFTypeRef aChildren;
AXUIElementCopyAttributeValue(aList, kAXChildrenAttribute, &aChildren);
NSInteger itemIndex = -1;
for (NSInteger i = 0; i < CFArrayGetCount(aChildren); i++) {
AXUIElementRef anElement = CFArrayGetValueAtIndex(aChildren, i);
CFTypeRef aResult;
AXUIElementCopyAttributeValue(anElement, kAXTitleAttribute, &aResult);
if ([(__bridge NSString *)aResult isEqualToString:name]) {
itemIndex = i;
}
}
SafeCFRelease(aChildren);
if (itemIndex == -1) return nil;
// We have index now do something with it
AXUIElementRef aReturnItem = [self copyAXUIElementFrom:aList role:kAXDockItemRole atIndex:itemIndex];
SafeCFRelease(aList);
return aReturnItem;
}
This SafeCFRelease() method is a very simple method that checks if the passed value is not nil, then releases (had some issues earlier).
void SafeCFRelease( CFTypeRef cf )
{
if (cf) CFRelease(cf);
}
And this method [copyAXUIElementFrom: role: atIndex:] is a method from #Willeke answer from another one of my questions:
- (AXUIElementRef)copyAXUIElementFrom:(AXUIElementRef)theContainer role:(CFStringRef)theRole atIndex:(NSInteger)theIndex {
AXUIElementRef aResultElement = NULL;
CFTypeRef aChildren;
AXError anAXError = AXUIElementCopyAttributeValue(theContainer, kAXChildrenAttribute, &aChildren);
if (anAXError == kAXErrorSuccess) {
NSUInteger anIndex = -1;
for (id anElement in (__bridge NSArray *)aChildren) {
if (theRole) {
CFTypeRef aRole;
anAXError = AXUIElementCopyAttributeValue((__bridge AXUIElementRef)anElement, kAXRoleAttribute, &aRole);
if (anAXError == kAXErrorSuccess) {
if (CFStringCompare(aRole, theRole, 0) == kCFCompareEqualTo)
anIndex++;
SafeCFRelease(aRole);
}
}
else
anIndex++;
if (anIndex == theIndex) {
aResultElement = (AXUIElementRef)CFRetain((__bridge CFTypeRef)(anElement));
break;
}
}
SafeCFRelease(aChildren);
}
return aResultElement;
}
Taking all this code, I put it into one of my methods:
// Check if in dock (otherwise cant do it)
if ([self isAppOfNameInDock:[appDict objectForKey:#"AppName"]]) {
// Get dock item
AXUIElementRef aDockItem = [self getDockItemWithName:[appDict objectForKey:#"AppName"]];
AXUIElementPerformAction(aDockItem, kAXShowMenuAction);
[NSThread sleepForTimeInterval:0.5];
CGRect aRect;
CFTypeRef aPosition;
AXUIElementCopyAttributeValue(aDockItem, kAXPositionAttribute, &aPosition);
AXValueGetValue(aPosition, kAXValueCGPointType, &aRect.origin);
SafeCFRelease(aPosition);
CFTypeRef aSize;
AXUIElementCopyAttributeValue(aDockItem, kAXSizeAttribute, &aSize);
AXValueGetValue(aSize, kAXValueCGSizeType, &aRect.size);
SafeCFRelease(aSize);
SafeCFRelease(aDockItem);
CGPoint aMenuPoint;
if ([self dockHidden] == kDockAutohideOff) {
switch ([self dockPosition]) {
case kDockPositionRight:
aMenuPoint = CGPointMake(aRect.origin.x - 18, aRect.origin.y + (aRect.size.height / 2));
break;
case kDockPositionLeft:
aMenuPoint = CGPointMake(aRect.origin.x + aRect.size.width + 18, aRect.origin.y + (aRect.size.height / 2));
break;
case kDockPositionBottom:
aMenuPoint = CGPointMake(aRect.origin.x + (aRect.size.width / 2), aRect.origin.y - 18);
break;
case kDockPositionUnknown:
aMenuPoint = CGPointMake(0, 0);
break;
}
} else {
NSRect screenFrame = [[NSScreen mainScreen] frame];
switch ([self dockPosition]) {
case kDockPositionRight:
aMenuPoint = CGPointMake(screenFrame.size.width - 18, aRect.origin.y + (aRect.size.height / 2));
break;
case kDockPositionLeft:
aMenuPoint = CGPointMake(screenFrame.origin.x + 18, aRect.origin.y + (aRect.size.height / 2));
break;
case kDockPositionBottom:
aMenuPoint = CGPointMake(aRect.origin.x + (aRect.size.width / 2), screenFrame.size.height - 18);
break;
case kDockPositionUnknown:
aMenuPoint = CGPointMake(0, 0);
break;
}
}
if ((aMenuPoint.x != 0) && (aMenuPoint.y != 0)) {
AXUIElementRef _systemWideElement = AXUIElementCreateSystemWide();
AXUIElementRef aMenu;
AXUIElementCopyElementAtPosition(_systemWideElement, aMenuPoint.x, aMenuPoint.y, &aMenu);
SafeCFRelease(_systemWideElement);
// Get menu items
CFTypeRef aMenuChildren;
AXUIElementCopyAttributeValue(aMenu, kAXVisibleChildrenAttribute, &aMenuChildren);
NSRunningApplication *app = [[NSRunningApplication runningApplicationsWithBundleIdentifier:[appDict objectForKey:#"BundleID"]] objectAtIndex:0];
for (NSInteger i = 0; i < CFArrayGetCount(aMenuChildren); i++) {
AXUIElementRef aMenuItem = [self copyAXUIElementFrom:aMenu role:kAXMenuItemRole atIndex:i];
CFTypeRef aTitle;
AXUIElementCopyAttributeValue(aMenuItem, kAXTitleAttribute, &aTitle);
// Supports chrome, safari, and finder
if ([(__bridge NSString *)aTitle isEqualToString:#"New Window"] || [(__bridge NSString *)aTitle isEqualToString:#"New Finder Window"]) {
AXUIElementPerformAction(aMenuItem, kAXPressAction);
NSInteger numberOfWindows = [self numberOfWindowsOpenFromApplicationWithPID:[app processIdentifier]];
// Wait until open
while ([self numberOfWindowsOpenFromApplicationWithPID:[app processIdentifier]] <= numberOfWindows) {
}
break;
}
}
SafeCFRelease(aMenu);
SafeCFRelease(aMenuChildren);
}
}
This is pretty complicated, but it works. I probably can't explain it, but I have stress tested this code and it works quite well.

Answer 2

Answer to the question "How to get an array of AXMenuItems from AXMenu?": aMenuChildren is the list of menu items. To be sure you could filter by role kAXMenuItemRole.
Answer to the question "Why does it not work?": The menu isn't a menu. When you inspect the menu in Axccessibility Inspector, it displays a warning "Parent does not report element as one of its children". The Dock app has one child, a list of dock items.