How to get Information of a Selected UI Element? - user-interface

Question

How can I get identify the UI element in AppleScript?
In AppleScript it is possible to make a click on to an UI element like this:
click item 1 of text field 1 of sheet 1 of window 1 of application process "Umsatz" of application "System Events"
But is it also possible to read the active element?
Screenshot shows an active UI element

Answer 1

Here's a handler that will return a record of all the properties, attributes, and actions belonging to a specific UI element:
on info about UIElement
set my text item delimiters to linefeed & linefeed
tell application id "com.apple.SystemEvents" to script Object
property parent : UIElement
property AXAttributes : a reference to my (the ¬
attributes whose name ≠ "AXURL")
property AXValues : value of AXAttributes
property AXRecord : a reference to the ¬
contents of {«class usrf»:my AXList}
property AXList : name of AXAttributes
end script
tell (a reference to the Object's AXList) to set the ¬
contents to paragraphs of (it as text) & ""
tell the Object to repeat with i from 1 to length of its AXValues
set item (i * 2) of its AXList to ¬
item i of its AXValues
end repeat
tell application id "com.apple.SystemEvents" to return ¬
{UI element:the Object's contents} & the properties ¬
of the Object & (the Object's AXRecord as any) & ¬
{_AXActions:every action's name of the Object}
end info
To use it, you pass it the reference of a particular UI element, such as that which is returned by the click or click at commands. In your case, this could be:
tell application id "com.apple.systemevents" to tell process "Umsatz"
return info about window 1's sheet 1's text field 1's ui element 1
It will only work on single elements, so passing it a collection of elements will throw an error. Upon success, you'll get a result like this one returned detailing information about my window in Script Editor when running the following command:
tell application id "com.apple.systemevents" to tell process ¬
"Script Editor" to return my info about window 1
Result:
{UI element:window "Untitled 161.scpt" of application
process "Script Editor" of application "System Events",
minimum value:missing value, orientation:missing value,
position:{1000, 25}, class:window, accessibility
description:missing value, role description:"standard
window", focused:false, title:"Untitled 161.scpt",
size:{623, 1095}, help:missing value, entire contents:{},
enabled:missing value, maximum value:missing value,
role:"AXWindow", value:missing value,
subrole:"AXStandardWindow", selected:missing value,
name:"Untitled 161.scpt", description:"standard window",
AXFocused:false, AXFullScreen:false, AXTitle:"Untitled
161.scpt", AXChildrenInNavigationOrder:{application "System
Events", application "System Events", application "System
Events", application "System Events", application "System
Events", application "System Events", application "System
Events", application "System Events", application "System
Events", application "System Events", application "System
Events"}, AXFrame:{1000, 25, 1623, 1120}, AXPosition:{1000,
25}, AXGrowArea:missing value, AXMinimizeButton:button 3 of
window "Untitled 161.scpt" of application process "Script
Editor" of application "System Events",
AXDocument:"file:///Users/CK/Library/Mobile%20Documents/com~
apple~ScriptEditor2/Documents/Untitled%20161.scpt",
AXSections:{{SectionUniqueID:"AXToolbar",
SectionDescription:"Toolbar"},
{SectionUniqueID:"AXContent",
SectionDescription:"Content"},
{SectionUniqueID:"AXTopLevelNavigator",
SectionDescription:"Top Level Navigator"}},
AXCloseButton:button 1 of window "Untitled 161.scpt" of
application process "Script Editor" of application "System
Events", AXMain:true, AXActivationPoint:{1010, 39},
AXFullScreenButton:button 2 of window "Untitled 161.scpt"
of application process "Script Editor" of application
"System Events", AXProxy:missing value,
AXDefaultButton:missing value, AXMinimized:false,
AXChildren:{group 1 of window "Untitled 161.scpt" of
application process "Script Editor" of application "System
Events", busy indicator 1 of window "Untitled 161.scpt" of
application process "Script Editor" of application "System
Events", static text "Running…" of window "Untitled
161.scpt" of application process "Script Editor" of
application "System Events", radio group 1 of window
"Untitled 161.scpt" of application process "Script Editor"
of application "System Events", splitter group 1 of window
"Untitled 161.scpt" of application process "Script Editor"
of application "System Events", toolbar 1 of window
"Untitled 161.scpt" of application process "Script Editor"
of application "System Events", tab group "tab bar" of
window "Untitled 161.scpt" of application process "Script
Editor" of application "System Events", button 1 of window
"Untitled 161.scpt" of application process "Script Editor"
of application "System Events", button 2 of window
"Untitled 161.scpt" of application process "Script Editor"
of application "System Events", button 3 of window
"Untitled 161.scpt" of application process "Script Editor"
of application "System Events", static text "Untitled
161.scpt" of window "Untitled 161.scpt" of application
process "Script Editor" of application "System Events"},
AXRole:"AXWindow", AXParent:application process "Script
Editor" of application "System Events",
AXTitleUIElement:static text "Untitled 161.scpt" of window
"Untitled 161.scpt" of application process "Script Editor"
of application "System Events", AXCancelButton:missing
value, AXModal:false, AXSubrole:"AXStandardWindow",
AXZoomButton:button 2 of window "Untitled 161.scpt" of
application process "Script Editor" of application "System
Events", AXRoleDescription:"standard window", AXSize:{623,
1095}, AXToolbarButton:missing value,
AXIdentifier:"_NS:794", _AXActions:{"AXRaise"}}

Answer 2

Focused UI Elements of front window may be more than 1. That is, it may be list of focused UI Elements. The element you need will in most cases be the element deepest in the structure.
I tested script in the Safari with Login window of our Stack Overflow site. To test like me, open Login form, tape in the some text field of this form (to make it focused), run script.
on getDeeppestFocusedUIelement(processName)
tell application "System Events" to tell process processName
set frontmost to true
repeat until window 1 exists
delay 0.1
end repeat
repeat with UIElement in reverse of (get entire contents of window 1)
try
if focused of UIElement then return (contents of UIElement)
end try
end repeat
end tell
return {}
end getDeeppestFocusedUIelement
set focusedUIElement to getDeeppestFocusedUIelement("Safari")