Its my understanding that UIAutomator is unable to automate any elements where NAF = true in uiautomatorviewer. Ive searched high and low but i cant for the life of me find what NAF stands for. Does anyone know?
"Not Accessibility Friendly"
These are UI elements which are apparently interactive, but which don't have accessibility affordances like content descriptions.
From the source for AccessibilityNodeInfoDumper.java (part of uiautomator):
/**
* We're looking for UI controls that are enabled, clickable but have no
* text nor content-description. Such controls configuration indicate an
* interactive control is present in the UI and is most likely not
* accessibility friendly. We refer to such controls here as NAF controls
* (Not Accessibility Friendly)
*
* #param node
* #return false if a node fails the check, true if all is OK
*/
private static boolean nafCheck(AccessibilityNodeInfo node) {
boolean isNaf = node.isClickable() && node.isEnabled()
&& safeCharSeqToString(node.getContentDescription()).isEmpty()
&& safeCharSeqToString(node.getText()).isEmpty();
if (!isNaf)
return true;
// check children since sometimes the containing element is clickable
// and NAF but a child's text or description is available. Will assume
// such layout as fine.
return childNafCheck(node);
}
NAF stands for " Not Accessibility Friendly" .
Related
I am currently designing an extension to make the notifications in the notification section of the calendar expendable. The goal is to make the noficiation expand like the initial notification on the desktop does. I have changed the type of notification added to the noficiation tray to class NotificationBanner from class NotificationMessage. I am currently using a work-around to make this work, this is what my expand function looks like:
expand(animate) {
this.expanded = true;
this._actionBin.visible = this._actionBin.get_n_children() > 0;
if (this._bodyStack.get_n_children() < 2) {
this._expandedLabel = new MessageList.URLHighlighter(this._bodyText,
true, this._useBodyMarkup);
this.setExpandedBody(this._expandedLabel);
}
if (animate) {
if (!this.clickedByButton && !this.forceExpansion) {
// This is the usual way notifications are expanded, using the layout manager
this._bodyStack.ease_property('#layout.expansion', 1, {
progress_mode: Clutter.AnimationMode.EASE_OUT_QUAD,
duration: MessageTray.ANIMATION_TIME,
});
}
else if (this.forceExpansion || this.clickedByButton) {
// When auto expanding or clicked by button, change height of body
oldHeight = this.bodyLabel.get_height();
const lines = Math.ceil(this._bodyText.length / 54);
this.bodyLabel.set_height(lines * this.bodyLabel.get_height());
}
this._actionBin.scale_y = 0;
this._actionBin.ease({
scale_y: 1,
duration: MessageTray.ANIMATION_TIME,
mode: Clutter.AnimationMode.EASE_OUT_QUAD,
});
} else {
this._bodyStack.layout_manager.expansion = 1;
this._actionBin.scale_y = 1;
}
this.emit('expanded');
}
As you can see, I have 2 options for this extension: Force expand all notifications or make the user use a button to expand. The current solution is not elegant, it simply changes the height of the notification label which manages the body. Furhermore, the notification body still shows the three dots, implying that the body is still not expanded. I believe this to be an issue with the layout manager, since the proper way to expand is to set message._bodyStack.layout_manager.expansion to 1. That does not work in the case of expanding a message in the notification tray. Is anyone familiar with the layout manager or can help me find a different solution? Here is an image of what my current solution looks like:
Image of an automatically expanded notification in the notification tray due to the extension (note the three dots at the end of the first line being still there)
Okay I have found a solution, it is not related to the layout manager. The value of the message message.bodyLabel.clutter_text.ellipsize is set to 3, which is the main cause of the dots appearing on the notification. Setting this value to 0 solves this problem. I would have still loved to find a more elegant approach to displaying the body, but this will do.
I am facing an issue where I have a A.jsff(taskflow A) and a region inside A.jsff called aB.jsff (taskflow aB).
Inside ab.jsff I have the panelTabbed control with two tabs (showDetailItem) tab1 and tab2.
I also have a save button in the A.jsff which commits the changes on the page.
I want the tab1 to be open whenever this taskflow is run.
The problem is, no matter which tab I select and click save(taskflow A), when I come back to this page, the tab which was previously expanded is always disclosed. Even after setting disclosed attribute and persist/dontPersist attribute, I wasn't able to achieve my requirement.
I also found few related discussions on the web which didn't help at all.
Can anyone please help or provide workaround to achieve this.
This is a tech stack error. And I have been provided with a fix which can help others who can face same issue.
Create a binding for the panelTabbed element and add this to its setter function -
public void setPanelTabBinding(RichPanelTabbed panelTabBinding) {
/*
* Tech stack bug fix for default tab selection.
*/
this.panelTabBinding = panelTabBinding;
ComponentChangeFilter[] compChgFilters = null;
if (panelTabBinding != null && compChgFilters.length == 0) {
compChgFilters = panelTabBinding.getComponentChangeFilters();
if (compChgFilters != null)
panelTabBinding.addComponentChangeFilter(new TreeRestrictingChangeFilter(panelTabBinding));
}
}
Situation
Let's say you have a Scene which contains a single complex control that itself contains a TextArea. The complex control is composed in a way, that when it gains the focus the TextArea gains the focus.
The ComplexControl has the ability to make the TextArea non-editable. Then a key input can has another semantic than operating a text. The new semantic can be defined by the ComplexControl or any node soemwhat higher in the scene graph.
The Scene has a global shortcut CTRL+N for opening a new Tab/View. In context of the ComplexControl CRTL+N semantic is creating a new Text document.
------------------------
| Scene |
| ------------------ |
| | ComplexControl | |
| | ------------ | |
| | | TextArea | | |
| | ------------ | |
| ------------------ |
------------------------
Objective
React globally or by a node somewhat higher in the scene graph on KeyEvents / KeyCombinations. Any control somewhat lower can take over the events, so that in a more special context an occurred event has more relevance as globally defined.
Suggestion
Setting a KeyHandler onto the Scene or a higher Node. Any EventHandler closer to the Source/Target of the EventDispatchChain" can consume the event. This way the KeyHandler can prevent the Scene or any higher Node from reacting on the key input, which is what the user intends in the special context of a control. Therefore an EventFilter at the higher place is not suitable.
Trouble
The TextArea consumes always any key event, even if it's not the intention of the design. No EventHandler higher in the EventDispatchChain gets informed.
Problem
How can it be forced to give the event back from the TextArea without consuming it and then let it bubble up the EventDispatchChain?
How can delivering the Event down to the TextArea be prevented without consuming it, so that it not even knows anything about the event.
Thanx to sillyfly, who showed me the way. As an answer to question 2, here's the resulting source. I decided to place the functionality at a central point for reuse and give the opportunity to make the suppression condition based. Please don't blame me for the class name, that's not the point ;-)
public class EventUtil {
/**
* Prevents Events of the given <b>eventTypes</b> to be passed down during the
* event capturing phase, if the <b>condition</b> is met.
*
* #param node
* The Node, whose descendants should not be informed about the Event
* #param eventTypes
* The types of Events for which the prevention should be valid
* #param condition
* The condition that must be met to prevent passing the Event down
*/
public static void preventPassDown(Node node, Supplier<Boolean> condition, EventType<?>... eventTypes) {
for (EventType<?> eventType : eventTypes) {
if (eventTypes == null) {
return;
}
node.addEventFilter(eventType, event -> {
if (condition.get()) {
event.consume();
Parent parent = node.getParent();
if (parent != null) {
Event.fireEvent(parent, event);
}
}
});
}
}
/**
* Prevents Events of the given <b>eventTypes</b> to be passed down during the
* event capturing phase.
*
* #param node
* The Node, whose descendants should not be informed about the Event
* #param eventTypes
* The types of Events for which the prevention should be valid
*/
public static void preventPassDown(Node node, EventType<?>... eventTypes) {
preventPassDown(node, () -> true, eventTypes);
}
If someone finds an answer to question 1, please feel free to jump in.
I need to do something which i expected to be was simple - create a tab control which has 2 tabs, implying 2 modes of operation for my app. When user clicks on Tab1, he'll be presented with some buttons and textboxes, and when he clicks Tab2, some other input method. I noticed that there was a CTabCtrl class thats used in MFC to add tabs.
However, once I added the tab ctrl using the UI designer, I couldn't specify how many tabs there'll be using property window. Searching on the net, I found some examples but all of them required you to derive from CtabCtrl , create 2 or more child dialogs etc and to write your own custom class. My question is, since I want to do something so basic, why couldn't I do it using the familiar Add Event handler/Add member variable wizard and then handle everything else inside my app's class ? Surely, the default CTabCtrl class can do something useful without needing to derive from it ?
Forget about CTabCtrl and use CMFCTabCtrl which is much easier to work with (this is assuming you are working on VS2008 SP1).
Failing that, you seem to misunderstand how the tab control works. It only provides the 'tab strip' at the top and sends messages when the user clicks on another one. It doesn't provide you with 'tab canvases' on which you can put controls. Showing and hiding the controls on the tab is something that the programmer needs to take care of. The resource editor provides little support there. Like Stewart says, the most common way of working is to have child dialogs in your tab and hide all of them except the one of the current tab.
You don't need to derive from CTabCtrl, you can also implement the switching behavior in the window that is the parent of the CTabCtrl.
The MFC tab control is a pretty thin wrapper over the win32 tab control, which works in pretty much the way you describe. It is a window, which provides switching between child windows using tabs. As it happens, in straight win32 this is the most useful way for it to work. If you want to do something more sophisticated than switching between individual windows, you do this by using child dialogs. MFC doesn't do a great deal to help you, but deriving from CTabCtrl and using child dialogs is really not very difficult to do, although if you're used to the way WinForms does tab controls it does seem unnecessary.
If you want the tab control at the root of the dialog, with no other controls along side it, you might want to look at CPropertySheet (http://msdn.microsoft.com/en-us/library/d3fkt014(VS.80).aspx) which is probably simpler to use. Unless you want to use any of the wizard functionality you don't even need to derive from it - you just create a couple of child dialog classes, then in the place where you want to create the property sheet, make an object, add the pages to it and invoke it.
The approach I took with an MFC dialog that contained a CTabCtrl was to derive a small class to manage the tab control and used dialog templates to create the actual tab window contents.
This is still being worked on so the source code is not very clean however here are some pieces. For instance CTabCtrlDialog needs constructor and destructor in order to release object which may have been created.
In the resource file I have a dialog template with a tab control followed by three dialog templates for each of the different tab content windows inserted into the tab control. While the dialog displaying the tab control has the WS_POPUP style, the dialog templates for the tab windows that are inserted into the tab control have a WS_CHILD style instead. This change makes the tab windows child windows so that when the dialog is moved, everything stays lined up properly with no further effort on my part.
In my case the tab windows which are inserted into the tab control display a set of check boxes to indicate various operational parameters. Using a dialog template approach makes it very easy to create the necessary tab window content.
I derive a class from CTabCtrl that extends the standard MFC class with an additional method for inserting into the tab control a tab window based on a specified dialog template id. Since this is just a single dialog, I just put this class into the same files, .h and .cpp, as the dialog components themselves.
class CTabCtrlDialog : public CTabCtrl
{
public:
void InsertItemDialogTemplate (UINT nIDTemplate, int nItem, TCITEM* pTabCtrlItem);
public:
struct {
UINT nIDTemplate;
CDialog *pDialog;
} m_pDialogData[10];
};
The method InsertItemDialogTemplate() looks like:
/*
* InsertItemDialogTemplate ()
*
* Insert into a tab control a tab pane based on the specified dialog template. The
* dialog template describes what the tab pane looks like so far as controls, etc.
*
* NOTE: The STYLE description must be WS_CHILD and not WS_POPUP. Also the dialog
* needs to have as its top coordinate some distance in pixels so that the
* various tab descriptions are visible. For instance an example dialog
* template in the resource file may look like:
* IDD_CASHIER_TAB_ONE DIALOGEX 0, 10, 178, 113
* STYLE DS_SETFONT | DS_FIXEDSYS | WS_CHILD
* FONT 8, "MS Shell Dlg", 400, 0, 0x1
* BEGIN
* LTEXT "Dialog Tab one",IDC_STATIC,6,44,90,17
* END
*
**/
void CTabCtrlDialog::InsertItemDialogTemplate (UINT nIDTemplate, int nItem, TCITEM* pTabCtrlItem)
{
InsertItem (nItem, pTabCtrlItem);
m_pDialogData[nItem].nIDTemplate = nIDTemplate;
m_pDialogData[nItem].pDialog = new CDialog ();
m_pDialogData[nItem].pDialog->Create (nIDTemplate, this);
m_pDialogData[nItem].pDialog->ShowWindow (FALSE);
}
For handling tab selection which displays the various tabs I have the following message map and then the two event handlers in the dialog.
BEGIN_MESSAGE_MAP(CDiaCashierEdit, CDialog)
ON_NOTIFY(TCN_SELCHANGE, IDC_TAB_CASHIER_EDIT_STATUS, &CDiaCashierEdit::OnTcnSelchangeTabCashierEditStatus)
ON_NOTIFY(TCN_SELCHANGING, IDC_TAB_CASHIER_EDIT_STATUS, &CDiaCashierEdit::OnTcnSelchangingTabCashierEditStatus)
END_MESSAGE_MAP()
void CDiaCashierEdit::OnTcnSelchangeTabCashierEditStatus(NMHDR *pNMHDR, LRESULT *pResult)
{
// TODO: Add your control notification handler code here
*pResult = 0;
int i = TabCtrl_GetCurSel(pNMHDR->hwndFrom);
m_TabCtrl.m_pDialogData[i + 1].pDialog->ShowWindow (TRUE);
}
void CDiaCashierEdit::OnTcnSelchangingTabCashierEditStatus(NMHDR *pNMHDR, LRESULT *pResult)
{
// TODO: Add your control notification handler code here
*pResult = 0;
int i = TabCtrl_GetCurSel(pNMHDR->hwndFrom);
m_TabCtrl.m_pDialogData[i + 1].pDialog->ShowWindow (FALSE);
}
In the DoDataExchange() method of the dialog I have the following which creates first the tab control and then creates each of the tab windows and inserts them into the tab control.
void CDiaCashierEdit::DoDataExchange(CDataExchange* pDX)
{
CDialog::DoDataExchange(pDX);
DDX_Control(pDX, IDC_EDIT_CASHIER_NAME, m_CashierName);
DDX_Control(pDX, IDC_EDIT_CASHIER_SUPNO, m_SupervisorId);
DDX_Control(pDX, IDC_EDIT_CASHIER_TEAMNO, m_TeamNumber);
DDX_Control(pDX, IDC_EDIT_CASHIER_GCSTART, m_GuestCheckStart);
DDX_Control(pDX, IDC_EDIT_CASHIER_GCEND, m_GuestCheckEnd);
DDX_Control(pDX, IDC_TAB_CASHIER_EDIT_STATUS, m_TabCtrl);
if (pDX->m_bSaveAndValidate) {
m_CashierName.GetWindowText (m_paraCashier.auchCashierName, 20);
m_paraCashier.usSupervisorID = m_SupervisorId.GetWindowTextAsInt();
m_paraCashier.uchTeamNo = m_TeamNumber.GetWindowTextAsInt();
m_paraCashier.usGstCheckStartNo = m_GuestCheckStart.GetWindowTextAsInt();
m_paraCashier.usGstCheckEndNo = m_GuestCheckEnd.GetWindowTextAsInt();
for (int i = 0; i < sizeof(m_TabItemOneStatus)/sizeof(m_TabItemOneStatus[0]); i++) {
int iTab = m_TabItemOneStatus[i].sTabItem;
int iDlg = m_TabItemOneStatus[i].iDlgItem;
int iOffset = m_TabItemOneStatus[i].sOffset;
CButton *p = (CButton *) m_TabCtrl.m_pDialogData[iTab].pDialog->GetDlgItem(iDlg);
if (p->GetCheck()) {
m_paraCashier.fbCashierStatus[iOffset] |= m_TabItemOneStatus[i].uchBit;
} else {
m_paraCashier.fbCashierStatus[iOffset] &= ~(m_TabItemOneStatus[i].uchBit);
}
}
} else {
m_CashierName.SetWindowText(m_paraCashier.auchCashierName);
m_SupervisorId.SetWindowTextAsInt (m_paraCashier.usSupervisorID);
m_TeamNumber.SetWindowTextAsInt (m_paraCashier.uchTeamNo);
m_GuestCheckStart.SetWindowTextAsInt (m_paraCashier.usGstCheckStartNo);
m_GuestCheckEnd.SetWindowTextAsInt (m_paraCashier.usGstCheckEndNo);
m_TabCtrl.InsertItemDialogTemplate (IDD_CASHIER_TAB_ONE, 1, &m_TabItemOne);
m_TabCtrl.InsertItemDialogTemplate (IDD_CASHIER_TAB_TWO, 2, &m_TabItemTwo);
m_TabCtrl.InsertItemDialogTemplate (IDD_CASHIER_TAB_THREE, 3, &m_TabItemThree);
for (int i = 0; i < sizeof(m_TabItemOneStatus)/sizeof(m_TabItemOneStatus[0]); i++) {
int iTab = m_TabItemOneStatus[i].sTabItem;
int iDlg = m_TabItemOneStatus[i].iDlgItem;
int iOffset = m_TabItemOneStatus[i].sOffset;
CButton *p = (CButton *) m_TabCtrl.m_pDialogData[iTab].pDialog->GetDlgItem(iDlg);
if (m_paraCashier.fbCashierStatus[iOffset] & m_TabItemOneStatus[i].uchBit) {
p->SetCheck (1);
} else {
p->SetCheck (0);
}
}
m_TabCtrl.m_pDialogData[1].pDialog->ShowWindow (TRUE);
}
}
I'm trying to find documented (or, undocumented, if that's my only option) APIs on OS X to query a list of windows from the window server and then cause the windows to move and resize. Can anyone point me in the right direction? I guess I'd be starting with something like FindWindowEx and MoveWindow under Win32.
Note that I want to do this from an external process - I'm not asking how to control just my own app's window size and position.
Use the Accessibility API. Using this API you can connect to a process, obtain a list of windows (actually an array), get the positions and sizes of each window and also change window properties if you like.
However, an application can only be using this API if the user has enabled access for assistive devices in his preferences (System Prefs -> Universal Access), in which case all applications may use this API, or if your application is a trusted assitive application (when it is trusted, it may use the API, even if this option is not checked). The Accessibility API itself offers the necessary functions to make your application trusted - basically you must become root (using security services to request root permissions of the user) and then mark your process as trusted. Once your application has been marked trusted, it must be restarted as the trusted state is only checked on start-up and can't change while the app is running. The trust state is permanent, unless the user moves the application somewhere else or the hash of the application binary changes (e.g. after an update). If the user has assistive devices enabled in his prefs, all applications are treated as if they were trusted. Usually your app would check if this option is enabled, if it is, go on and do your stuff. If not, it would check if it is already trusted, if it is, again just do your stuff. If not try to make itself trusted and then restart the application unless the user declined root authorization. The API offers all necessary functions to check all this.
There exist private functions to do the same using the Mac OS window manager, but the only advantage that would buy you is that you don't need to be a trusted Accessibility application (which is a one time operation on first launch in most cases). The disadvantages are that this API may change any time (it has already changed in the past), it's all undocumented and functions are only known through reverse engineering. The Accessibility however is public, it is documented and it hasn't change much since the first OS X version that introduced it (some new functions were added in 10.4 and again in 10.5, but not much else has changed).
Here's a code example. It will wait 5 seconds, so you can switch to a different window before it does anything else (otherwise it will always work with the terminal window, rather boring for testing). Then it will get the front most process, the front most window of this process, print it's position and size and finally move it by 25 pixels to the right. You compile it on command line like that (assuming it is named test.c)
gcc -framework Carbon -o test test.c
Please note that I do not perform any error checking in the code for simplicity (there are various places that could cause the program to crash if something goes wrong and certain things may/can go wrong). Here's the code:
/* Carbon includes everything necessary for Accessibilty API */
#include <Carbon/Carbon.h>
static bool amIAuthorized ()
{
if (AXAPIEnabled() != 0) {
/* Yehaa, all apps are authorized */
return true;
}
/* Bummer, it's not activated, maybe we are trusted */
if (AXIsProcessTrusted() != 0) {
/* Good news, we are already trusted */
return true;
}
/* Crap, we are not trusted...
* correct behavior would now be to become a root process using
* authorization services and then call AXMakeProcessTrusted() to make
* ourselves trusted, then restart... I'll skip this here for
* simplicity.
*/
return false;
}
static AXUIElementRef getFrontMostApp ()
{
pid_t pid;
ProcessSerialNumber psn;
GetFrontProcess(&psn);
GetProcessPID(&psn, &pid);
return AXUIElementCreateApplication(pid);
}
int main (
int argc,
char ** argv
) {
int i;
AXValueRef temp;
CGSize windowSize;
CGPoint windowPosition;
CFStringRef windowTitle;
AXUIElementRef frontMostApp;
AXUIElementRef frontMostWindow;
if (!amIAuthorized()) {
printf("Can't use accessibility API!\n");
return 1;
}
/* Give the user 5 seconds to switch to another window, otherwise
* only the terminal window will be used
*/
for (i = 0; i < 5; i++) {
sleep(1);
printf("%d", i + 1);
if (i < 4) {
printf("...");
fflush(stdout);
} else {
printf("\n");
}
}
/* Here we go. Find out which process is front-most */
frontMostApp = getFrontMostApp();
/* Get the front most window. We could also get an array of all windows
* of this process and ask each window if it is front most, but that is
* quite inefficient if we only need the front most window.
*/
AXUIElementCopyAttributeValue(
frontMostApp, kAXFocusedWindowAttribute, (CFTypeRef *)&frontMostWindow
);
/* Get the title of the window */
AXUIElementCopyAttributeValue(
frontMostWindow, kAXTitleAttribute, (CFTypeRef *)&windowTitle
);
/* Get the window size and position */
AXUIElementCopyAttributeValue(
frontMostWindow, kAXSizeAttribute, (CFTypeRef *)&temp
);
AXValueGetValue(temp, kAXValueCGSizeType, &windowSize);
CFRelease(temp);
AXUIElementCopyAttributeValue(
frontMostWindow, kAXPositionAttribute, (CFTypeRef *)&temp
);
AXValueGetValue(temp, kAXValueCGPointType, &windowPosition);
CFRelease(temp);
/* Print everything */
printf("\n");
CFShow(windowTitle);
printf(
"Window is at (%f, %f) and has dimension of (%f, %f)\n",
windowPosition.x,
windowPosition.y,
windowSize.width,
windowSize.height
);
/* Move the window to the right by 25 pixels */
windowPosition.x += 25;
temp = AXValueCreate(kAXValueCGPointType, &windowPosition);
AXUIElementSetAttributeValue(frontMostWindow, kAXPositionAttribute, temp);
CFRelease(temp);
/* Clean up */
CFRelease(frontMostWindow);
CFRelease(frontMostApp);
return 0;
}
Sine Ben asked how you get a list of all windows in the comments, here's how:
Instead of kAXFocusedWindowAttribute you use kAXWindowsAttribute for the AXUIElementCopyAttributeValue function. The result is then no AXUIElementRef, but a CFArray of AXUIElementRef elements, one for each window of this application.
I agree that Accessibility is the best way forward. But if you want quick-and-dirty, AppleScript will work as well.