UIPath Robot won't type into a text box, clear selector. Clicking separately and sending hotkeys also does not work - uipath

I've tried nearly anything, The "Type Into" activity won't print plain text into the text box let alone a held variable. The textbox element in question is the update work items comment box in the acme-test website from the Level 3 RPA developer course. I am able to type into the box manually and the robot is able to find it (the cursor moves to the centre of the text box and the program continues). I've tried quite a lot, including using a click activity and then sending the string as hotkeys.

Most probably the issue is related to your selectors. Since you are on level 3 RPA developer course I assume you are using Reframework for the task and I believe because of comprehensive error handling capabilities of this template your application just continuous with the next item instead of crashing when it can't find the element.
To solve the selector issues I usually do the following:
Use partial selectors instead of full selectors
Use wildcards for dynamic parts of your selectors (* for replacing any number of characters, ? for replacing exactly 1 character)
You can also store the page you are working on in a Uipath.Core.Browser type variable to eliminate the need of reselecting browser.
Also keep in mind that if you have used basic recorder functionality of UI path it generates full selectors.

Related

Do I need to use aria-label on elements that have visible labels which are read by screen reader?

My company is trying to become WCAG compliant. One of the things we need to do is add aria-label to interactive elements. My question is do I need to add an aria-label to an element which already has a label which is read by a screen reader.
So basically i definitely need a "close" aria-label here:
<button>x</button>
But in some cases I have a placeholder for example, which is read by the screen reader. Do I need to add an aria-label anyway just in case? The aria-label would be the same as the placeholder.
#mikkel has good advice regarding the first rule of ARIA use.
Regarding your specific question on the placeholder attribute, relying on the placeholder as your label would fail several WCAG checkpoints.
Placeholder text is typically light gray and when displayed on a white background often fails 1.4.3 Contrast Minimum
The placeholder text disappears when you start typing so now the "label" has disappeared and would fail 3.3.2 Labels or Instructions
The placeholder attribute is not specifically referenced in the accessible name calculation (https://www.w3.org/TR/accname-1.1/#step2) although it might be considered in step 2D as "an attribute that defines a text alternative". It's not clear if placeholder is a "text alternative". Personally, I don't think it is. If not, then you would fail 4.1.2 Name, Role, Value
If you look at the spec for placeholder, it has a big disclaimer about not using it as a label.
Warning: Use of the placeholder attribute as a replacement for a label can reduce the accessibility and usability of the control for a range of users including older users and users with cognitive, mobility, fine motor skill or vision impairments. While the hint given by the control’s label is shown at all times, the short hint given in the placeholder attribute is only shown before the user enters a value. Furthermore, placeholder text may be mistaken for a pre-filled value, and as commonly implemented the default color of the placeholder text provides insufficient contrast and the lack of a separate visible label reduces the size of the hit region available for setting focus on the control.
So, all that being said, yes, you should have some kind of visible label associated with your input field even if you are using the placeholder attribute.
Short answer, no.
Long answer, it depends on the situation.
But important – first rule of aria:
If you can use a native HTML element or attribute with the semantics and behavior you require already built in, instead of re-purposing an element and adding an ARIA role, state or property to make it accessible, then do so.
When in doubt, choose native HTML.
https://www.w3.org/TR/using-aria/#rule1
If you have a visible label, then you should connect the label text and the input field via the label-element.
If the input field does not have a visible label, the aria-label is one technique that can add the label semantically.
In your example of the button, you should use an aria-label, as the "X" doesn’t give the user any information. Of course, most users can guess what an "x" dos, but the user should not have to guess.
And here am talking of users who use assistive technology.
I would recement that you get a professional to test your site, for legal reasons but mostly for your users.
I'm a web developer, but have used the last 10 years on web accessibility. So far, it’s the hardest field of web development I hav had to learn and I'm still learning. 😊

Appium: How to differentiate between two different iOS screens?

I am developing a testing algorithm for our iOS apps using Appium. To fully implement this algorithm i need to identify wither i have moved onto different screen or am still on the same screen after performing some action. I need to know, what makes every screen unique/different from other in terms of Appium?
Going through the pageSource of every screen, i found that most screens have xpath attribute in window element. Can i use value of xpath of window element to mark the screen as unique from others, or do i need to do a trivial string comparison between screen's pageSources to mark them different? Or is there any other better solution?
Not sure if xpath would be the best solution for this. Normally the UIAWindow would remain the same, and developers might use different containers within this UIAWindow to render different screens.
So to verify different screens, you might need to figure out what this container is and see if the container's properties change when you move to a new screen (ie a new container)
If you app user a different header for every new screen, then you can use this header to see if the screen is changed. Example: in WhatsApp, you would see a different persons at the top. So in this case, the person's name can be assumed as the header.
If this doesn't work then you can verify some of the other controls, or say list of all the UIAStaticText on the screen. During screen change the entire list of UIAStaticText might change. So this can indicate a screen change.
For our automation suite at work I've implemented a series of screen check steps. Every time we switch screens I do a find_element command for an element on that screen that is unique to that screen. That way if a button or option takes me to a new screen that is incorrect my test will fail as expected. If it does find the element we're expecting it adds minimal time to the test suite.
Anish Pillai made a good suggestion of using the header text if there is any. Otherwise a particular tab, menu text, resource_id, or whatever is unique about the page would suffice. All you would need to do is a find_element call and a failure message if it fails.

Browser shortcut keys for placing focus on arbitrary input controls

When surfing the web, I often find myself having to reach for the mouse to place focus on a particular input control. For example, I might need to click in a text box to enter my log-in information. For reasons of efficiency I would prefer to minimise such mouse usage.
I am aware that one can usually use the 'Tab' key to cycle between such controls, but usually there's a whole bunch of header links that one has to navigate before reaching the control of interest. I'm also aware that some applications (e.g. Gmail) have built-in shortcut keys, however I'm looking for something that will work on generic HTML pages.
Usually, once the first control in a form has focus, it's easy to 'Tab' between the remaining controls in that form before submitting it.
My question: Assuming I'm using Firefox (24.0), is there an existing add-on which will speed up (using keyboard shortcuts) the task of setting focus to an arbitrary input control on the page (e.g. the first text box in a form)?
I imagine that such an add-on would listen for a certain key combination e.g. 'Ctrl-Shift-F', upon which it would display tooltip-like information on each visible input control which specifies the subsequent key (e.g. 'a', 'b', 'c' etc) that will cause the corresponding control/link to be focused.
Does anybody know of such an add-on, or am I perhaps looking at this problem the wrong way?
I found two extensions:
Fox Input
Cycle Input Focus
I've only tried Fox Input, but it's working for me so far. It defaults to Alt-I to move forward through text fields and Alt-J to move backwards, but you can change the hotkeys in the prefs.
I also use VimFX, which can do a tooltip-style selection of links and text fields, but its other features might be overkill for your problem.

Multiple synchronized CKEditor instances on the same page?

I'm working on a document-editing application using CKEditor, where the user can open multiple documents side-by-side in a pair of editor instances.
Most of the time, the user will be editing two different documents, but it's also possible that the two editor instances might contain different views of the same document. That makes things tricky, since I'd like to changes in one editor instance to be immediately reflected in the other instance.
Without hacking the CKEditor core, is something like that possible?
If not, would it be possible to write a plugin that would provide that kind of functionality?
What about if I was willing to get into the core code and hack around a bit? How difficult would it be?
This is a very similar case to a collaborative editing like Google Docs allows. The only differences are that you won't need to synchronize this via network and that it's very unlikely (if not impossible) that the same documents will be modified by two users at the same time. This makes things simpler... a little bit.
A year ago me and my colleague (we are both CKEditor core devs) took part in Node.JS Knockout and our plan was to create a collaborative editor based on CKEditor. It was only 48h, so the result is not impressive, but it worked. The source code is here.
The main problem you'll have is applying changes from editor A to editor B without breaking a caret position in editor B. Basically, you cannot just take data from editor A and set them in editor B, because everything in editor B will be reset including scroll position and caret. Unless this is not a problem, but I assume it is.
So you would need to:
either find a nice algorithm for extracting changes (like diff, but working on a DOM tree, not an HTML string) made in editor A and an algorithm to apply them to editor B (and this is what we implemented on Node.JS Knockout),
or find a way to guess caret position after resetting data in editor B; for example you can remember the caret context in editor B before setting data and try to find it in data that will be loaded.
Both ways are doable, but the first way will give better results if you'll implement it well. However, if your you don't know enough about DOM and contenteditable, then this task may overwhelm you. In this case I would advise to block possibility of opening one document twice.

How Imitate a [Ctrl+Left mouse click] on the center of the form or open another program and type in a word?

Babylon dictionary and a couple of other dictionaries allow to click on any word in any windows program
and automatically recognize the word under the cursor, and at once open the dictionary window while searching for that word in installed dictionaries.
You can on the other hand open your dictionary, type in your word and press Enter, the result will be the same.
There's a Delphi form, containing a text label, for example with the word "Automaton".
My question is:
How to send a word from my Delphi application right into the dictionary window, as if you typed it manually and pressed Enter?
The best solution is to send some message through the Windows mechanism, but if it is too complicated, there's another solution, and so the second answer: as I described, we need to model a [Ctrl+left mouse] click on a form where this word is displayed on a form [ a visual label on the screen of my Delphi application], to be exact, on some central pixel of this label.
Could you kindly give an advice how to do one thing or another in Delphi ?
** edit:
The problem with AppActivate is this: Babylon dict has a daemon part that seats in the tray.
In the task manager a real window where the text should be input also is named 'Babylon'.
So AppActivate('Babylon') tries to bring to front the non-visual part of the application.
Do you have any suggestion how to determine the windows handle or something of a real visual part of the application? In the task manager, I repeat both visual and non-visual parts are named 'Babylon'.
I cannot offer an answer so much as some insight and advice...
There are certain applications which "intercept" keyboard and mouse instructions, and essentially "nullify" them if they are being immitated by software. Generally-speaking, you'd only see this in proper AntiVirus software such as Kaspersky by design... however:
The way some (not many, but some) programs hook keyboard and mouse inputs, as a side-effect, behave the same way. If you have attempted all of the advice given as comments above, and cannot get Babylon to trigger an action as a result, it is likely Babylon behaves as I have described.
If what I suspect is true, then the method you are attempting is simply not possible (at least, not using any simple Pascal code on its own... ASM might be able to do it but that's beyond my knowledge).
A better solution may be to do a little research to see if any of the following options are available to you:
1) Does Babylon have a Pipeline or API you can use to interface your application(s) with it?
2) Is the particular functionality you require of Babylon accessible through one (or more) DLL files distributed as part of Babylon?
3) Is there an alternative to using Babylon for your needs?
I know it's not an answer as such (certainly not one you'd want to hear), but it may point you in a better direction.

Resources