AppleScript Not Recognizing Application Index Addresses - applescript

I am creating an AppleScript to control some basic standard actions in SAP and everything as written works fine from the main menu page of available transactions. The current script fills out the Command Text Field and opens a new existing transaction code. However, once SAP has gone to the new transaction code page, when I run the AppleScript a second time it no longer recognizes the index of the Command Text Field. Instead it pastes the transaction code into whatever text field happens to be the current focus. When I run the script, it should always go to the same text field because the overall index address hasn't changed, but it's not.
I've attempted to open the Script Dictionary for SAP, but it appears that my company's flavor of SAP has had the scripting dictionary disabled, so I am forced to use indexing addresses to fill text fields and click buttons
on chooseSAPTransaction()
global sapTransaction, transactionCode
set transactionCode to "" -- intialize the variable
set transactionList to choose from list {"Find Document", "Change Document", "Look up Product", "Correct Product"} with prompt "SAP Action:" default items {""}
if transactionList is false then
error number -128
else
set choosenTransaction to item 1 of transactionList
end if
if choosenTransaction is "Find Document" then
set transactionCode to "code1"
chooseCode1()
end if
if choosenTransaction is "Change Document" then
set transactionCode to "code2"
chooseCode2()
end if
if choosenTransaction is "Look up Product" then
set transactionCode to "code3"
chooseCode3()
end if
if choosenTransaction is "Correct Product" then
set transactionCode to "code4"
chooseCode4()
end if
set sapTransaction to "/n" & transactionCode
activate application "SAP"
delay 0.1
tell application "System Events"
tell process "SAP"
tell front window to tell toolbar 1 to tell text field 1
keystroke sapTransaction
end tell
tell front window to tell toolbar 1 to tell button 1 to perform action "AXPress"
end tell
end tell
end chooseSAPTransaction
It's skipping tell front window to tell toolbar 1 to tell text field 1 and going right to keystroke sapTransaction on subsequent script executions
("/n" & transactionCode creates the string that tells SAP the new transaction to go to. When typing it manually, it always takes you to the new transaction, regardless of how many layers deep you might be in the current transaction)
I expect the indexing tell to go to the same element on every new instance of running the script, but once it's run once, AppleScript isn't finding the element address anymore.
edit:
I just tried moving the pasting action into its own unique handler so that it can be called before attempting to go to the next step, but that didn't work either.
--snip--
if choosenTransaction is "Find Document" then
set transactionCode to "code1"
set sapTransaction to "/n" & transactionCode
pasteTransaction()
chooseCode1()
end if
--snip--
on pasteTransaction()
global serialNumber, sapTransaction, transactionCode
activate application "SAPGUI"
delay 0.1
tell application "System Events"
tell process "SAPGUI"
tell front window to tell toolbar 1 to tell text field 1
keystroke sapTransaction
end tell
tell front window to tell toolbar 1 to tell button 1 to perform action "AXPress"
end tell
end tell
end pasteTransaction
edit 2:
I got it working by forcing SAP to go through the menu. Woo!
on pasteTransaction()
global serialNumber, sapTransaction, transactionCode
activate application "SAPGUI"
delay 0.1
tell application "System Events"
click menu item ¬
"Target Command Field" of menu 1 of menu bar item ¬
"Edit" of menu bar 1 of application process "SAPGUI"
keystroke sapTransaction
tell process "SAPGUI"
tell front window to tell toolbar 1 to tell button 1 to perform action "AXPress"
end tell
end tell
end pasteTransaction
unfortunately, that doesn't fix the bigger issue of AppleScript not finding the correct index address for text fields, because getting this part working is only step one. Now I have to get text strings into various text fields within each transaction.
So if anyone knows how to address that original Index Address issue, that would be great.
I wish I knew how to actually write AppleScript instead of cobbling code together one action at a time from various sources. It's not elegant, and functionality is hit-or-miss.

GUI scripting is always a hack, and without seeing the application in action it's hard to diagnose problems. Obviously something is changing in the interface that is throwing off the view hierarchy you're trying to leverage, but... I will point out that menus and toolbars tend to be dynamic, which makes things dicier. But here's a couple of diagnostics and alternate approaches you can try.
First, let me rewrite the GUI code like this, which I find clearer:
tell application "System Events"
tell process "SAP"
tell front window
tell toolbar 1
tell text field 1
keystroke sapTransaction
end tell
tell button 1
perform action "AXPress"
end tell
end tell
end tell
end tell
end tell
So first — assuming you are targeting the correct textfield — you might try this instead:
tell toolbar 1
tell text field 1
set value to sapTransaction
end tell
tell button 1
perform action "AXPress"
end tell
end tell
keystroke can be intercepted by whatever UI element has the focus, but setting the text field's value ought to go directly into the correct element. If that doesn't work, you can try this diagnostic in Script Editor:
tell toolbar 1
properties of every text field
end tell
This should give you a list in the Script Editor's log of all of the text fields in toolbar 1. With that, you can see if the text field has a name property (which would make addressing easier: i.e. tell text field "someName"), or if some other text field has been added to the toolbar higher in the hierarchy, changing the index of the text field you're after. If worse comes to worst, you can try this:
tell toolbar 1
entire contents
end tell
Which will list out all the UI elements of the toolbar. You can do a before/after comparison to see what (if anything) changed.
Hope that helps.

Related

what means "tab group 1" in AppleScript, why not "tab group 2"?

I'm trying to understand code belong, and can't find anything explain what means "tab group 1"..
And I don't know how to debug to find the value with "tab group 1"
Btw, I test "tab group 0",its ok, but "tab group 2" its error..
set devices to {}
tell application "System Preferences"
reveal pane "声音"
end tell
tell application "System Events"
tell application process "System Preferences"
repeat until exists window "声音"
end repeat
tell tab group 1 of window "声音"
get properties
click radio button "输出"
tell table 1 of scroll area 1
set selected_row to (first UI element whose selected is true)
set currentOutput to value of text field 1 of selected_row as text
repeat with r in rows
try
set deviceName to value of text field 1 of r as text
set end of devices to deviceName
end try
end repeat
end tell
end tell
end tell
end tell
if application "System Preferences" is running then
tell application "System Preferences" to quit
end if
set text item delimiters to "‡"
set devicesStr to devices as text
set comm to "bash ./main.sh" & " \"" & devicesStr & "\"" & " \"" & currentOutput & "\"" & " output"
log comm
#do shell script comm
AppleScript GUI scripting involves working its way down through the view hierarchy of the application's windows. In this case, 'Tab Group [X]' means that there are at least [X] tab groups within the container at that level of the hierarchy, and you need to determine which one contains the lower-level element you're trying to access. Unfortunately, the elements of the view hierarchy aren't always immediately visible (there may be 'hidden' containers and such), and the hierarchy may change significantly from one app update to the next. That can be headache inducing.
You can debug this manually (with a little patience) by working your way down the hierarchy yourself until you find the elements you need, using a series of every UI element of.. commands. I.e., begin with:
tell application "System Events"
tell application process "System Preferences"
tell window "声音"
get every UI element
end tell
end tell
end tell
Then choose a likely UI element from the list it produces and add a new tell block. However, it's easier to use the Accessibility Inspector app, which gives you a look into the details of any applications view hierarchy. Accessibility Inspector is included with Xcode downloads (which is free, and worth having around); I don't know if there's a place to download it separately.
See Apple's Guide of Testing Accessibility.

Click on a specific place on a window, click on a specific button and control if the window changes

Actually I have 3 questions about the same problem: controlling a window with applescript.
What should I do if I would press on button "Close Window" of application "Google Chrome"?
Is it possible to check if the window changes? For example, to see if appear a pop-up or something like that...
What about clicking on a specific place into a window? I mean, I know I can use
tell application "System Events"
click at {x,y}
end tell
but this command use the entire screen as reference system, and I want it works only on a specific window. For example, if at "{x,y}" i put "{1,1}", applescript will click on the first item on the menu bar. Is there a way I can say to "System Events" to click at "{1,1}", but on the window "Google Chrome"?
Here are three examples of how to close the front window of Google Chrome using AppleScript:
Note: The following assumes Google Chrome is running with at least one window open when you test each example AppleScript code in Script Editor.
Example one is the most straight forward way:
tell application "Google Chrome" to close front window
Example two directly clicks the close button:
tell application "System Events" to tell ¬
application process "Google Chrome" to ¬
click button 1 of front window
Example three calculates the center of the close button and clicks there:
activate application "Google Chrome"
delay 0.5
tell application "System Events" to tell ¬
application process "Google Chrome" to tell ¬
front window
set posB1 to (position of button 1)
set szB1 to (size of button 1)
set x to (item 1 of posB1) + (item 1 of szB1) / 2 as integer
set y to (item 2 of posB1) + (item 2 of szB1) / 2 as integer
end tell
tell application "System Events" to click at {x, y}
Note that in the first two examples, the front window of Google Chrome doesn't even need to be the frontmost window on the Desktop; however, with the third example it does, otherwise the click at {x, y} will not go to the intended target.
That said, example three really shouldn't be used when there it a straight forward way, as in example one, to get the job done. Example three was just a proof of concept to get the coordinates to click at. This method may be useful in some fringe cases, especially in an app that doesn't directly support AppleScript.
Note: The example AppleScript code is just that and does not contain any error handling as may be appropriate. The onus is upon the user to add any error handling as may be appropriate, needed or wanted. Have a look at the try statement and error statement in the AppleScript Language Guide. See also, Working with Errors.
In applescript GUI scripting you can simply refer to an element by name or index and tell it to click or to perform an action. For instance to click the close button on the first open window in Chrome you could use:
tell application "System Events"
tell process "Google Chrome"
tell window 1
tell button 1
click
end tell
end tell
end tell
end tell
You don't actually need to know its physical position to click one it; you just need to know that the first button in the window is the close button.
System Events always returns the position of any element in screen pixels, so if you want the position of an element in terms of its window, get the position of the element, get the position of the window, and do some addition or subtraction (e.g., if you want to click at {5,5} in a window whose position is {100, 125}, click at {105, 130})
AppleScript isn't really designed to monitor GUI changes, though if you want to be tricky and you know what change you're looking for you can do something like this:
tell application "System Events"
tell process "..."
tell window 1's pop up button 3
repeat until (exists menu 1)
delay 0.2
end repeat
-- menu 1 now exists, so the pop up button is open
end tell
end tell
end tell
...but note that this will hang the script until the menu is opened. A more elegant way to handle that is to write a script application with an idle handler, like so:
on run
-- whatever initialization is needed
end run
on idle
tell application "System Events"
try
tell process "..."
tell window 1's pop up button 3
if exists menu 1 then
-- menu 1 now exists
-- the pop up button is open
-- do what must be done
end if
end tell
end tell
on error errstr
display alert "Something went wrong" message "The script sent this error: " & errstr
end try
end tell
return 0.2
end idle
You can leave that running in the background watching for specific changes in the GUI (the 'try' statement is in case the app you're watching quits, the window closes, or something unexpected happens to the GUI).
If you haven't already, open the System Events scripting definition in Script Editor and look at the Processes Suite. That will show you all the things you can do with GUI scripting.

Automator script to automatic toggle menu setting in Simulator.app

I need to automate the process of enrolment of Face ID and Touch ID for my UITests. For this purpose, I'm working on an automator script.
My current automator script, which at the moment can automatic click "Enrolled" in the menu:
on run {input, parameters}
if application "Simulator" is running then
tell application "System Events"
set theName to name of the first process whose frontmost is true
end tell
tell application "Simulator" to activate
tell application "System Events"
tell process "Simulator"
tell menu bar 1
tell menu bar item "Hardware"
tell menu "Hardware"
tell menu item "Face ID"
tell menu "Face ID"
click (menu item "Face ID" where its name starts with "Matching")
end tell
end tell
end tell
end tell
end tell
end tell
end tell
tell application theName to activate
end if
return input
end run
The problem is as following. I need to determine, if the device already is enrolled. There is a checkmark, which shows the current state. I have tried to check if the checkmarks is there or not. But I have not been able to make it work yet.
So my question. How can I do, so the script only will press the 'Enrolled' menu item, if the checkmark isn't there?
There's an attribute of menu items called AXMenuItemMarkChar, which is either set to the character representing a check mark next to the menu item (if checked), or missing value.
use application "System Events"
tell application "Simulator" to activate
tell process "Simulator" to tell menu bar 1 to ¬
tell menu bar item "Hardware" to tell menu "Hardware" to ¬
tell menu item "Face ID" to tell menu "Face ID" to ¬
tell menu item "Enrolled"
if the value of the attribute "AXMenuItemMarkChar" is not "✓" then ¬
click it
delay 1 -- !important
return the value of the attribute "AXMenuItemMarkChar"
end tell
In my testing, the attribute returns the correct value if the Simulator app is in focus at the time. The return value of this script should always be "✓", because if the menu item isn't checked, then the script proceeds to click it and put a check mark next to it; and if the menu item is checked, then there's nothing to do but confirm that it is by getting the attribute's value.
One of the issues with your script is that you had the wrong reference to a non-existent menu item (menu item "Face ID" of menu "Face ID" of menu item "Face ID"), and then proceeded to filter it with a where clause against a name that had a different value; hence it would return no value, and you'd have no menu item to click.
Hopefully my script will work for you. Let me know if you have any problems.
The AppleScript contained in this answer was tested on MacOS 10.13. It is an example script to illustrate how to meet your objective. The delay command may need to be adjusted according to your system, and the script may benefit from some error-handling to make it robust.

Selecting Pop Up Menu Buttons in AppleScript

I want to automate clicking a specific pop down menu's item.
For Example, I want to change the Value of "Message receive Sound" to something else. How can I do this with AppleScript? And how can I do this with other pop down menus in AppleScript?
(To open the iMessage Settings menu, shown in the image, type CMD COMMA, once you open iMessage)
Note: I have successfully done this Automator, I just want to do it in applescript.
It's called GUI scripting. You have to identify the reference to the UI element(s).
GUI scripting strongly depends on the system version. If an update changes the UI structure the script will brake.
This selects the sound "Popcorn" in the sound popup menu. It's for El Capitan. In systems < 10.11 the UI elements may be different and the process name might be "iChat"
tell application "System Events"
tell process "Messages"
set frontmost to true
if not (exists (1st window whose value of attribute "AXIdentifier" is "MessagesPreferencesWindow")) then
keystroke "," using command down
repeat until exists (1st window whose value of attribute "AXIdentifier" is "MessagesPreferencesWindow")
delay 0.1
end repeat
end if
tell (1st window whose value of attribute "AXIdentifier" is "MessagesPreferencesWindow")
tell pop up button 4 of group 1
click
delay 0.2
click menu item "Popcorn" of menu 1
end tell
end tell
end tell
end tell

Using AXIdentifier in UI Scripting for Applescript

10.7.4 OSX Lion
Applescript
I am working with an application (built in house and has no Applescript dictionary) that has a static text element I want to copy to the clipboard and send to another app but I'm having a hard time getting it to work.
The script I was using for targeting the element looked like this:
Tell application "System Events" to set frontmost of process "*application*" to true
Tell application "System Events"
Tell process "*application*"
Tell static text 1 of tab view 1 scroll area 1 of splitter group 1 of splitter group 1 of splitter group 1 of window 1
keystroke "a" using command down
delay 0.1
keystroke "c" using command down
delay 0.1
end tell
end tell
end tell
end tell
What would happen was that the wrong text from the wrong element was copied to the clipboard every time I clicked in a different spot on the application (there are numerous text fields).
I noticed in UI Accessor/Accessibility Accessor that each UI element in the application has a unique AXIdentifier value when you mouse over them.
Is there anyway to accomplishing what I am trying to do, using AXIdentifier values to target that element and copy the text from it?
Thanks for all the help this is my first post and I hope it was worthy! ~TheLarkInn
You can do this by using AppleScript's filtering. For example, to get the From: pop-up menu in a message composition window in Apple Mail, there is no accessibility description you can match on, however there is a unique AXIdentifier which you can match as follows:
tell application "System Events"
tell application process "Mail"
tell window 1
get first pop up button whose value of attribute "AXIdentifier" is "popup_from"
end tell
end tell
end tell
This is more efficient than looping in AppleScript as it only involves sending one Apple Event to System Events.
I don't think there is a way to directly do what you're trying to do. It seems like you can only access attributes once you have a handle on an element via selector. Here is a very ugly solution that does what you're asking by iterating over all UI elements, but it is really slow with bigger UIs and probably not ideal for any production level code.
tell application "System Events"
tell process "Some Process"
set tElements to entire contents of window "Some Window"
repeat with tElement in tElements
if (exists attribute "AXIdentifier" of tElement) then
if value of attribute "AXIdentifier" of tElement = "Some AXIdentifier" then set tText to value of tElement
end if
end repeat
end tell
end tell
tText
I think using UIElementInspector or Accessibility Inspector from Xcode to build a selector string is the way to go!
Tell application "*application*" to activate
Tell application "System Events"
Tell application process "*application*"
set textStaticTextValue to value of static text 1 of tab view 1 scroll area 1 of splitter group 1 of splitter group 1 of splitter group 1 of window 1
end tell
end tell

Resources