I would like to ask if we can also achieved a Page/Window Object Model in AutoIT? Majority of my project assignment was on Web Automation and I'm using Selenium Webdriver with Framework uses Page Object Model. Currently, I'm assigned to a project for GUI automation. I like to implement this kind of approach also in AutoIT if feasible so that I can reuse the objects to other classes. We are planning to use AutoIT standalone. I noticed that most of the example available in the internet was the object created on each class/script.
Your insights are highly appreciated.
Thanks!
General:
That common approach of using the Page Object Model (POM) Design Pattern isn't quit good feasible with AutoIt. Of course you can create a object structure with AutoIt too, but it was not intended for the language. Anyway, some of the goals of POM can be achieved with the following example suggestion of a test structure.
Please notice:
Since you don't provide enough information about your application under test (AUT), I explain a basic structure. The implementation depends on your application (SWING/RCP, WinForm etc.). It's also important which tool support do you need for your page object recognition. Besides WinForm that could be controled by ControlCommand functions in AutoIt, it's a proper way to use UIASpy or au3_uiautomation as helper tools.
UIASpy - UI Automation Spy Tool
au3_uiautomation
It's an advantage to know the POM structure in context with Selenium. I usually include a test case description with behavior driven development BDD (Gherkin syntax with Cucumber or SpecFlow), but this will not be a part of that example here.
Example structure:
The structure consists of two applications under test Calc and VlcPlayer. Both follow the common structure PageObjects and Tests. You should try to devide your page objects (files) in many subfolders to keep an overview. This substructure should be similar for the Tests folder/subfolders.
In the Tests area you could include several test stages or test categories depending on your test goals (Acceptance/UI tests, just functional smoke tests and so on). It's also a good idea to control the execution order by an separat wrapper file, TestCaseExecutionOrder.au3. This should exist for all test categories to avoid a mixing of them.
This wrapper au3 file contains the function calls, it's the processing start/control.
Approach description:
TestCaseExecutionOrder.au3
Calls the functions which are the test cases in the subfolders (Menu, PlaylistContentArea, SideNavigation).
Test case NiceName consists of some test steps.
These test steps have to be included into that script/file by:
#include-once ; this line is optional
#include "Menu\OpenFolder.au3"
Test step OpenFolder.au3 (which is a part of a test case) contains the function(s) to do the folder loading and there content.
In that functions the PageObject MenuItemMedia.au3 will be loaded/included into the script/file by:
#include-once ; this line is optional
#include "..\..\..\PageObjects\Menu\MenuItemMedia.au3"
File MenuItemMedia.au3 should only contain the recognition mechanism for that area and actions.
This could be find menu item Media (as a function).
or find open folder menu item (as a function) and so on.
Func _findMenuItemMedia()
; do the recognition action
; ...
Return $oMenuItem
EndFunc
In the test step OpenFolder.au3 which calls _findMenuItemMedia() like:
Global $oMedia = _findMenuItemMedia()
can a .click executed or something like .getText etc.
The test cases should only #include the files which are necessary (test steps). The test steps should also only #include the necessary files (page objects) and so on. So it's possible to adjust the recognition functions once and it can be used in the corresponding test steps.
Conclusion:
Of course it's hard to explain it in this way, but with this approach you can do a similar way like in Selenium for web testing. Please notice that you properbly have to use Global variables often. You have to be ensure the correct includings and don't lose the overview of your test, which is in OOP test based approaches much easier.
I recommend the usage of VS Code, because you can jump from file to file at the #include statements. That's pretty handy.
I hope this will help you.
Related
I've been using Ruby Selenium-Webdriver for one of the automation scripts I'm developing and I'm being asked to use Page Objects, we use page objects a lot however for this application I am using CSV file instead, I have defined all the xpaths that I'm using in my application in a CSV file and I'm parsing that CSV file in my script to refer to those objects, I would like to know is there much of a difference in using a class for defining Page Objects or using a CSV file instead apart from performance concern? I believe using a CSV file will be an addon for us from configuration standpoint and will make it much easier to maintain, any suggestions on this?
Edit - In our use case, we're actually automating applications built on a cloud based tool, so basically all the applications share same design structure from HTML standpoint so we define xpath patterns in CSV and then we pass certain parameters to some custom methods that we've developed to generate xpath's automatically using the CSV instead of finding those manually as its overhead for us because we already know that all the applications will share similar xpath pattern for all elements.
Thanks
I think, POM is better than CSV approach. In POM, you put elements for a page in a separate class file. So, if any change is to make then it's easier to find where to change/maintain. Moreover, it won't get too messy as CSV file and you don't need to use extra utility function to parse those.
There is also a pageobjects gem that provides a set of libraries over and above webdriver/watir, simplifying the code.
Plus, why xpaths? Its one of the last recommended ways to identify an element.
As for the frameork aspect, csv should be more of a maintenance problem than PageObjects. Its the basic difference between text and code. You enforce Object oriented approach on your elements in PageObjects but that is not possible with csv.
In the best case scenario, you have created a column/separate sheets defining which page that element xpath belongs to. That sounds like an overhead. As your application / suite grows there can be thousands of elements. Imagine parsing/ manually updating a csv with that kind of data.
Instead in PageObjects, your elements will be restricted to the Page. Any changes to the app will also specify which elements may get impacted. Now, when define your element as an object in PageObject, rather than css, you also dont need to explicitly create your elements by reading the csv.
It completely depends on the application and the type of test you might perform.
Since it is an automated test script, you do not have to really worry about the performance of the script (it might take few more milli seconds to parse, which should be OK).
Maintaining all the elements identification properties & corresponding actions in a CSV file will make the maintenance easier and make the framework application independent which are nice. But maintaining your framework is bit difficult to make it more robust. Both approaches have its own pros and cons.
Refer to below posts [examples are in java - but you will get the idea]:
Keyword driven framework
Advanced Page Objects
Update:
If you like both, you can comeup with your implementation to easily integrate these too.
#ObjectRepository(src="/login.csv")
public class LoginPage{
private Map<String, WebElement> elements;
public void login(){
elements.get("username").sendKeys('');
elements.get("password").sendKeys('');
elements.get("signin").click();
}
}
Ie, define all the elements in a config file like csv/json etc. Let the page object refer to the class for the page elements. All the methods will be part of the page class.
My Goal is to create a Page Object Model Automation framework using Cucumber with capybara/selenium for mu client project. Currently my knowledge related to Page Object Model gem is very limited and at the same time i want to show some delivery by creating automation scripts using cucumber and capybara. So i am in the process of creating some feature files and subsequent step definitions.
So my question is, If i move to simple automation to POM framework, how much rework i have to do in terms of scripts and also if this approach is correct. Please guide me as i am pretty new to ruby automation.
Well this can be a very opinionated question, but I'll try to keep it simple:
If you already have well structured and modularised code (cucumber features and step defs), it won't take significant changes to move to POM. Its essentially a different structuring technique to keep your code organised and keep your changes minimal/confined.
Gems like siteprism can help you with creating well structured page objects, if you are open to learn an additional DSL (for siteprism) on top of cucumber / capybara. Or you can roll out your own without any additional libraries, though it would demand a thorough understanding of POM architecture and best practices.
Hope that helps.
I believe you will have a little work to migrate and meet increasingly the tool, the Framework POM only shows speed gain when you have some pages that already are mapped, it might take some time to be shown in reports, but I believe it has much to gain, organization, clarity and speed are some of the advantages of migration.
In my work I help other QAs learning and use POM in one day, a lot of guys use xpath to find objects, I just show how they can reuse that xpaths in a page object.
The second step I show to use method, not only elements, then they can reuse a lot of code and stop repeat steps, find where they break and use a simple method in siteprism page, example:
element :field_login , :xpath, "login"
element :field_password, :xpath, "password"
element :login_button, :xpath, "login_button"
def authenticate (username, password)
field_login.set username
field.password.set password
LoginPage.new if login_button.click
end
My next step is show to leave to use xpath and use css elements.
And after some sprints, they understand the beneficts of the framework, and it is default for all automation, include the DEVs can understand and help in tests now.
Warning this answer is somewhat controversial.
I'd say don't bother using the Page Object Model, its fundamentally flawed. The web is about resources and representations (REST) not about pages. BDD is about behaviour, not pages.
The danger with POM (particularly when you are not experienced) is that you write lots of scenarios to test what is on each page. When really what you should be doing is writing scenarios that confirm that some behaviour exists.
This question could apply to all languages and frameworks but I'm looking for something a bit more 'cocoa specific'. I come from a Java background and I've noticed that learning objective-c is a lot more than just syntax, it's almost a completely different way of thinking.
What I've been having the most trouble with, must be the way one has to organise your classes. Sure all basic OOP(Object Oriented Programming) rules apply, and using MVC patterns where you can is recommended. But with me being used to Java I just need to set a few things straight and make sure I've got the right idea:
So for the sake of simplicity let just focus on one part of an app - Logging a user in. You'd have your .xib file for the UI (called Login.xib), you'd have your class that handles your data (connecting to a web service, called LoginModel.m) and you'd have your controller that acts as the middle man between your front-end and data (Called LoginController.m).
Is this is a pretty good example of applying MVC to a Cocoa app? And if it is, does that mean that you'd have 6 files created for this (since you have header files and implementation files). 6 files just to handle something simple as logging a user in. You can imagine how many you'd end up with for an entire app, even the most simplest of ones...
So my question is - Am I doing something wrong? Do I have the wrong idea? Or is the idea of too many files and too long method names just something I need to get used to since my brain is still working in 'Java Mode'?
Your ideas on how to handle that outlined above are fully correct. There is nothing bad about having a lot of files in the project. It does help a lot when you want to reuse code or if for example login details change and you don't want to edit multiple code locations.
Nevertheless, you may combine in such simple cases model class and controller class, especially if your model data can for example be stored in an NSDictionary and such. Only if you have complex model objects, that will run a lot of their own code, it will be better to separate them out.
Variable and method names can't be too long ;) always use a good name that especially describes the functionality or task. You usually don't have to type them often, but Xcode autocomplete will deal with that easily.
First, Please bear with me with all my questions. I have never used TDD before but more and more I come to realize that I should. I have read a lot of posts and how to guides on TDD but some things are still not clear. Most example used for demonstration are either math calculation or some other simple operations. I also started reading Roy Osherove's book about TDD. Here are some questions I have:
If you have an object in your solution, for instance an Account class, what is the benefit of testing setting a property on it, for example an account name, then you Assert that whatever you set is right. Would this ever fail?
Another example, an account balance, you create an object with balance 300 then you assert that the balance is actually 300. How would that ever fail? What would I be testing here? I can see testing a subtraction operation with different input parameters would be more of a good test.
What should I actually test my objects for? methods or properties? sometime you also have objects as service in an infrastructure layer. In the case of methods, if you have a three tier app and the business layer is calling the data layer for some data. What gets tested in that case? the parameters? the data object not being null? what about in the case of services?
Then on to my question regarding real life project, if you have a green project and you want to start it with TDD. What do you start with first? do you divide your project into features then tdd each one or do you actually pick arbitrarily and you go from there.
For example, I have a new project and it requires a login capability. Do I start with creating User tests or Account tests or Login tests. Which one I start with first? What do I test in that class first?
Let's say I decide to create a User class that has a username and password and some other properties. I'm supposed to create the test first, fix all build error, run the test for it to fail then fix again to get a green light then refactor. So what are the first tests I should create on that class? For example, is it:
Username_Length_Greater_Than_6
Username_Length_Less_Than_12
Password_Complexity
If you assert that length is greater than 6, how is that testing the code? do we test that we throw an error if it's less than 6?
I am sorry if I was repetitive with my questions. I'm just trying to get started with TDD and I have not been able to have a mindset change. Thank you and hopefully someone can help me determine what am I missing here. By the way, does anyone know of any discussion groups or chats regarding TDD that I can join?
Have a look at low-level BDD. This post by Dan North introduces it quite well.
Rather than testing properties, think about the behavior you're looking for. For instance:
Account Behavior:
should allow a user to choose the account name
should allow funds to be added to the account
User Registration Behavior:
should ensure that all usernames are between 6 and 12 characters
should ask the password checker if the password is complex enough <-- you'd use a mock here
These would then become tests for each class, with the "should" becoming the test name. Each test is an example of how the class can be used valuably. Instead of testing methods and properties, you're showing someone else (or your future self) why the class is valuable and how to change it safely.
We also do something in BDD called "outside-in". So start with the GUI (or normally the controller / presenter, since we don't often unit-test the GUI).
You already know how the GUI will use the controller. Now write an example of that. You'll probably have more than one aspect of behavior, so write more examples until the controller works. The controller will have a number of collaborating classes that you haven't written yet, so mock those out - just dependency inject them via an interface. You can write them later.
When you've finished with the controller, replace the next thing you've mocked out in the real system by real code, and test-drive that. Oh, and don't bother mocking out domain objects (like Account) - it'll be a pain in the neck - but do inject any complex behavior into them and mock that out instead.
This way, you're always writing the interface that you wish you had - something that's easy to use - for every class. You're describing the behavior of that class and providing some examples of how to use it. You're making it safe and easy to change, and the appropriate design will emerge (feel free to be guided by patterns, thoughtful common sense and experience).
BTW, with Login, I tend to work out what the user wants to log in for, then code that first. Add Login later - it's usually not very risky and doesn't change much once it's written, so you may not even need to unit-test it. Up to you.
Test until fear is replaced by boredom. Property accessors and constructors are high cost to benefit to write tests against. I usually test them indirectly as part of some other (higher) test.
For a new project, I'd recommend looking at ATDD. Find a user-story that you want to pick first (based on user value). Write an acceptance test that should pass when the user story is done. Now drill down into the types that you'd need to get the AT to pass -- using TDD. The acceptance test will tell you which objects and what behaviors are required. You then implement them one at a time using TDD. When all your tests (incl your acc. test) pass - you pick up the next user story and repeat.
Let's say you pick 'Create user' as your first story. Then you write examples of how that should work. Turn them into automated acceptance tests.
create valid user -> account should be created
create invalid user ( diff combinations that show what is invalid ) -> account shouldn't be created, helpful error shown to the user
AccountsVM.CreateUser(username, password)
AccountsVM.HasUser(username)
AccountsVM.ErrorMessage
The test would show that you need the above. You then go test-drive them them out.
Don't test what is too simple to break.
getters and setters are too simple to be broken, so said, the code is so simple that an error can not happen.
you test the public methods and assert the response is as expected. If the method return void you have to test "collateral consequences" (sometimes is not easy, eg to test a email was sent). When this happens you can use mocks to test not the response but how the method executes (you ask the mockk if the Class Under Test called him the desired way)
I start doing Katas to learn the basics: JUnit and TestNG; then Harmcrest; then read EasyMock or Mockito documentation.
Look for katas at github, or here
http://codekata.pragprog.com
http://codingdojo.org/
The first test should be the easiest one! Maybe one that just force you to create the CUT (class under test)
But again, try katas!
http://codingdojo.org/cgi-bin/wiki.pl?KataFizzBuzz
I consider myself still pretty new to the TDD scene. But find that no matter which method I use (mock framework or stubbing my own objects) I find that I have to write a lot of code to create mock data. I like the idea of loading up objects to create an in-memory database. But what I don't like is cluttering up my tests with a ton of code for the sole purpose of creating mock data. This is especially the case when the data needs to account for all the different cases.
I'd love some suggestions for a better way of doing this.
It would seem to me that I should be able to load the data once into a known state from some data store and then I could use a snapshot of that state which is loaded in the test setup/initialize before each test method is executed. This would satisfy proper testing practices while providing convenience and let me focus on writing tests instead of writing code to create test data "by hand".
May be you could try the NBuilder library. It provides a very fluent interface and is easy to use. You can use it for generating single instances of a class with defualt values or generate lists with default or overriden values. You can have a look at this one.
If your are using .Net Try NDBUnit
You populate your store and then it reverts your DB to a known state at test time, for each test. The Autumn of Agile screen cast series shows this in pretty good detail.
Or you can do this manually...build a stored procedure or whatever to truncate your tables and copy in the data in your teardown method.
You can have Builder class(es) that helps you building the instances you need / in this case ones you would use related to the repository.
Have the Builder use appropiate defaults, and on your tests you can overwride what you need. This helps you avoid needing to put have every single case of "data" mixed up for all the different tests (which introduces problems, because usually there are cases that aren't compatible for different tests).
**Update 1:**Take a look at www.markhneedham.com/blog/2009/01/21/c-builder-pattern-still-useful-for-test-data
I know exactly what you mean. I think a good approach to solving this problem is to actually have a separate MockFramework project that houses all your mock data, outside the test project. This way you can generate mock data separately, store it in memory if you want to, or not, and then reference the mock framework from the test project. If you use a third party framework to do this, all the better, but you can still wrap that third party framework in your own mock framework so you can get all that "glue" that creates the mock data the way you need it out of your tests so the tests can really be only what they need to be.
Thanks for all the suggestions, I think the solution requires a little bit of everything. I don't want these tests to end up being regression tests, but w/o some kind of existing data store everything still boils down to creating the data by "manually" building the objects.
What would really be nice would be a framework that allowed me to use my existing DAL to either script the data to code for me or get the data in memory and access it like an in memory database.
Untils.org covers this way better than I ever could.
Their whole guide is actually very good.
But basically, if your units require "a lot of data" they may not be unit tests anymore. I'd recommend attempting testing the smaller pieces individually.