In general how does one test a various parts of a GUI? What are good practices? (Yes I am being overly general here).
Let take for Notepad's Find dialog box:
Notepad's Find dialog box http://img697.imageshack.us/img697/5483/imgp.png
What are some things that can be tested? How does one know its working correctly? What are edge cases to look out for? Stress tests?
Here.
I doubt any good generalization can be made about this - it always depends on the situation.
When someone asks for tests for GUI I always assume that that mean 'this part of application that is accessible via this GUI'. Otherwise it would mean testing the only the GUI without any logic hooked. Dunno why no one never actually asked for testing if the events are fired when button is pressed or is displayed window acquiring focus.
Anyway back to the question. First of all find out about equivalence classes, boundary conditions other testing techniques. Than try to apply it for given problem. Than try to be creative.
All those should be applied when creating following tests:
1) happy path tests - application acts right when given input is good
2) negative tests - application acts right when given input is bad
3) psychotic user behavior (I saw someone use this term, and I find it to be great) - that one user that has nothing better to do than break your application or is to stupid to actually know how bad and horrible things he is doing with your app.
After all this if all tests are passing and you can't figure out other, than you don't know is it working properly, but you can say that it passed all tests and it seems to be working correctly.
As for given GUI example.
1)
Is the application finding string that is in opened file?
Is the application finding character that is in opened file?
How is it reacting to reaching end of file during search?
Is it finding other appearances of given string/character or just one, when there are many of those appearances ?
Is it handling special search characters like * or ? correctly?
Is it searching in desired direction?
Is it 'Mach case ' option working properly?
When opening find setting some criteria, canceling search and launching it again - are search criteria back to default values? Or are they set as you left them when clicking Cancel?
2)
Is it informing user that no mach was found when trying to search for data that is not in opened file?
Is it reacting properly when trying to search down form end of file?
Is it reacting properly when trying to search up form beginning of file?
How search feature is reacting when no file is loaded? (in MS notepad it can be done, but in other editors you can launch editor without opening a file hence this test)
Can I mark both Up and Downs search direction?
3)
Is it working properly on 4GB file?
Can I load 4 GB string in 'Find What:' field and search for it?
Can I provide as input special characters by providing ASCII codes? (it was done like pressing Alt and number of character... or something like that)
Can I search for empty character (there was something like that in character table).
Can I search for characters like end of line or CarretReturn?
Will it search for characters form different languages? (Chinese, or other non-english alphabet characters)
Can I inject something like ') DROP ALL TABLES; (if that would be web based search).
Will I be able to launch proper event twice by really fast double click on search button? (easier on web apps)
With reasonable test suite you know it seems to work correctly.
I think it is better to separate out functional aspects and the usability aspects for the GUI testing.
Let us say in the above example take the use case of user entering some text and hitting the Find button. From the functional aspect I would say your tests should check whether this user action (event) calls the appropriate event handler methods. These can be automated if your code has good separation between the GUI display code and the
functional part.
Testing of usability aspect would involve checking things like whether the display occurs correctly in multiple platforms. I think this needs to be verified manually. But I think there are some tools that automate this kind of GUI testing as well but I've no experience with them.
It's difficult and error-prone to test finished UIs.
But if you are more interested form the programmer's perspective, please have a read of the paper The Humble Dialog. It presents an architecture for creating UIs whose functionality can be tested in code using standard testing frameworks.
Related
I am trying to create a new GUI element within DrRacket's text window, like picts or syntax objects. As far as I can tell, the most standard way of doing this is with a snip%.1
Unfortunately, the documentation for creating new snips, while comprehensive, is a bit impenetrable and leaves some questions to be answered.
For starters, what is the difference between a snip% and a snip-class%? Why do these need to be separated out into two classes, rather than simply being combined into one class? Is it because multiple snips will use one snip class?
Second off, what is snip-reader<%>? Not only why does it need to be a separate class, but why is the module providing it supposed to be installed?2 If it does need to be a new class, why can't it just be referred to directly. Why go through this whole process of constructing and then parsing a string of the form: "(lib ...)\n(lib ...)"?
I mean, there might now be any reason for this design, and it might just be a remnant of an old API. If so, has anyone thought of making a new more consistent API? Or if there is a reason for this design, can you please tell me what it is, as the docs don't seem to make that clear.
I mean, as of right now, I can copy/paste the sample given in the docs on creating a new snip. But I'm having a hard time understanding the design going on here, so I can use them properly.
1I know there are other ways to do it, but I also want to have interactive buttons and whatnot.
2I know it doesn't need to be installed as a library per se, but the documentation seems to strongly push you in that direction.
Okay, I think I finally found the answer. Broadly speaking:
The snip% class includes the methods for drawing the snip, telling the editor how much space to reserve for the picture, and handling events such as mouse clicks.
Next, the snip-class% class is used for encoding and decoding snips. This must be a separate class because when saved to a file, the editor needs to encode what type of snip it is, and for obvious reasons it can't just put the literal snip% class in there. The value it stores in the file is the snip-class%'s 'class name'. This can be anything, and as long as the editor has the classname associated to a snip-class%, it can be loaded. Additionally, if it is of the form "(lib ...)" or "(lib ...) (lib ...)" Racket will just automatically load it into the list for you.
Nothing 'needs' to be installed per se, its just the easiest way to go about it. Otherwise you manually need to tell the editor how to handle the snip before actually loading the file.
I have an activity with spinners (3), an EditText (1), buttons (2). I would like to test whether the order of user interactions with these elements does not matter for the outcome (=display of some string in a TextView). For example it should not matter whether I first type something in the EditText and then select a spinner or whether I first do the spinner selection and then type something in the EditText, etc.
Due to the number of elements, there are many variations of how user interactions could happen. Is there a way to either:
generate automatically all possible combinations of user interactions/testing code
or
Test a single, but random user interaction
?
I am aware of The Monkey, but that seems to be for stress testing only!?
According to this:
generate automatically all possible combinations of user interactions/testing code
Sorry, but I don't see any rational way to test it in the other way than in monkey.
Please read more about this tool here: http://developer.android.com/tools/help/monkey.html
and Trust me, there's no reason to be afraid of this great tool.
Finally, monkey also gives you well-detiled logs, as you can see on this video: https://www.youtube.com/watch?v=wdDVHXo4obY
I have very recently answered to the question about Creating TreeView with nodes and checkboxes.
While I was pondering how to properly process the case when treeview's node is checked when user pressing spacebar I ran into TVN_KEYDOWN notification.
My solution was tested in a dialog box and in a window procedure and both seem to work flawlessly.
Still, I have a dilemma of what should be my returned result. Here is the relevant excerpt from the documentation for TVN_KEYDOWN:
Return value
If the wVKey member of lParam is a character key code, the character
will be used as part of an incremental search. Return nonzero to
exclude the character from the incremental search, or zero to include
the character in the search. For all other keys, the return value is
ignored.
I have tried returning both results when testing if spacebar is pressed and haven't noticed any difference.
So I ask you the following questions:
Can someone explain me what is the incremental search ?
What is the difference when I include or exclude the tested character (spacebar) from incremental search ?
EDIT:
It seems that I have found an answer to first question. I have found an article on Wikipedia that explains what incremental search is.
It leaves only the second question to be answered.
END OF EDIT
Thank you.
Best regards.
It is most visible in a giant TreeView. The best example of one is the left panel in Regedit.exe. Expand HKCR and start typing to see the effect.
The implementation has changed across Windows versions, it used to be a lot less usable in XP. It is a UI blooper, there isn't any good way for the user to see that he mistyped a letter, to correct a typing mistake or to see that a search starts from scratch. Current versions of Windows use a timeout, automatically resetting the partially typed search phrase when you don't hit a key for a couple of seconds. Which is about as practical as it gets. It is certainly useful, just not very usable.
The only sane thing to do with TVN_KEYDOWN is nothing. Never add more ways to make it less predictable than it already is. Intentionally swallowing a keystroke of course makes it a lot less usable if it is a key the user really wanted to use. You certainly don't want to swallow a space, that's of course a valid character in tree node text. If the tree happens to not have any nodes with text that contains a space then you still don't want to swallow it, the control itself already does.
The notification would have been a lot more useful if it also passed the incremental search string that was collected or give a way to reset it. It just doesn't so that's water under the bridge. Consider handling it if you've created some kind of usability trap, very hard to come up with a practical example of one. You know it when you see it.
The only real use is to completely replace the search function. You'd then make your own rules and select a node yourself. And of course always return a non-zero value.
You want only answer to the second question, so this is:
If you exclude spacebar from incremental search, you dont find trewview item with spacebar.
Note: You must disable TVS_CHECKBOXES style, because treeview wndproc handle spacebar different with this style set.
I'm somewhat new to automation, and am learning everything auto-didactically, so forgive me if my terminology is a bit off. I've searched hi and low for an answer to this question, and I can't seem to find anything. I presume it's my small vocabulary when it comes to this stuff... anyway...
I'm attempting to write a test that performs all the actions necessary to complete a tutorial by using the recorder. However, for one particular step, the element ID changes. For example, the ID I'm trying to click is this:
//li[#id='message_661119']/div[2]/div[2]/a/img
However, for each new user that is performing the tutorial "quest", the number of the id changes.
Is there anyway to get Selenium to recognize, or use, wildcards? Example:
//li[#id='message_******']/div[2]/div[2]/a/img
Of course, the example above does not work.
Any advice would be immensely helpful. Thank you!!
You can use starts-with() for this:
//li[starts-with(#id, 'message_')]/div[2]/div[2]/a/img
It's one of the examples mentioned in Locating Techniques in Selenium's docs for starts-with().
In Target field of the command in Selenium IDE where you can see message_123123 click on a dropdownlist and choose an option which is related to xpath:idRelative or if this one doesn't work then try another options which do not include that annoying message_123123 so this way you'll identify webpage element by it's location but not id. I solved my issue this way
Background
Lately I've become a fanatic that everything I type while working on a computer should be compatible with "DRY". If there's anything I have to type more than once in any context, I want some kind of user-aware auto-complete option to do some of the work for me -- always -- no exceptions.
Having to work under Windows, I've looked at GUI solutions to make this insane goal a reality.
The (almost) optimal solution
If you have a moment, open up Firefox 3.0 and type a few keystrokes into the address bar. You will notice that it performs a kind of Incremental Autocomplete based on space-separated sub-strings of whatever you type. Another place in Firefox that does something similar is the about:config URL.
This is sub-optimal, because I don't want this in Firefox only. I want to use this everywhere.
The Question
Does anyone out there know of a widget or app that does nothing but insanely good incremental auto-complete that can be used as a general purpose "run everywhere" tool? Something that allows the user to: 1) maintain one or more "completion candidate files"; 2) pick one of those files as the source for Firefox 3.0 style completion; 3) return the result (or blank if the user canceled), and do those three things only?
Details
Here's how it should work:
STEP1: user saves or more csv file(s) (or other easy-edit format) somewhere in his hard-drive
STEP2: user creates a Windows Script Host script or a batch file (or whatever) instantiates the FilterAsYouType GUI
STEP3: user runs the script file, and the script file instantiates the GUI, telling it which CSV file to use as the source of all potential completions
STEP4: the user either chooses one of the completions, supplies his own text that is not in the list, or cancels out without supplying anything
STEP5: when the user is done the script saves the result to a variable and does something with it
Here is some pseudo-code for the script:
include "GenericTypeaheadWidget";
var gengui = new GenericTypaheadWidget('c:\docs\favorite_foods.csv');
var fave_food = gengui.get_user_input();
if(fave_food != ''){
alert('you chose '+fave_food+'!');
}
The rationale
The goal is to just have a way to always be able to do auto-completions from a list of arbitrary items, even if the list is a couple thousand items, and not have to rely on it being built into some IDE or standalone application that only accepts certain kinds of input or has an overly-complicated API relative to the simplicity of this task.
CSV (or text or sqlite database) would provide a way for me to self-generate "candidate lists" or "history logs" and then just use those logs as the source of the possible completions.
The disclaimer
I've tried several GUI "launcher" programs, command-line engines like power-shell and scripting shells, the regular plain old command-line history with varying degrees of satisfaction. The problem with these is they all do extra superfluous stuff like searching directories or built-in commands. I just want nothing but whatever is in the CSV file I happen to be pointing at.
I'm wondering if there is any simple tool that does nothing but what I'm describing above.
UPDATE: It looks like this question is very closely related to Graphical Command Shell, which captures the essential idea presented here.
You should really try Launchy - it's exactly what you're looking for, a "run anything" with intelligent autocompletion. It completely changes the way you interact with a Windows PC.
And it has open source-code, so you can borrow its autocompletion code if you want to roll your own interface.