Is there a way to integrate Visual Studio 2010 Coded UI Test with Specflow?
I am not sure if these 2 can hook somehow.
I have this idea to Create a Data-Driven Coded UI Tests by using some Coded UI API (I don't know yet what) and hook it up into Specflow.
I have to revisit the video Specflow guys had last year to see how all can fit together
and I would appreciate some input, ideas.
Does this has to do with creating unit test providers for Specflow like the one below (since Coded UI work in VS 2010):
MsTest.2010 MsTest version for .Net 4.0. Supporting test-categories.
Will unitTestProvider (generatorProvider and generatorProvider) be that connection?
Or simply these 2 won't mix and are competing or different ways to do integration or acceptance testing.
Is Coded UI Tests actually close to running Specflow via selenium for UI automation test
with additional benefit that the code is generated for you.
I just want o gain the benefit of Coded UI Tests (maximize its feature both at design and run time) and Specflow (again design and run time)
(More about this on Microsoft site)
Create a Data-Driven Coded UI Test
http://msdn.microsoft.com/en-us/library/ee624082(VS.100).aspx
Using Visual Studio Premium or Visual Studio Ultimate, you can create a coded UI test that tests whether the user interface (UI) for an application functions correctly. The coded UI test performs actions on the user interface controls for an application and verifies that specified controls display the correct values. For more information about how to create a coded UI test, see How to: Create a Coded UI Test.
After you create your coded UI test with specified data, you might want to run your coded UI test multiple times with different sets of data to test different conditions. To do this you can add parameter values from a data source to create a data-driven coded UI test. Each row in the data source is an iteration of the coded UI test. The overall result of the test will be based on the outcome for all the iterations. For example, if one test iteration fails, the overall test result is failure.
For more general information about data-driven tests, see How to: Create a Data-Driven Unit Test. (http://msdn.microsoft.com/en-us/library/ms182527.aspx)
Thanks,
Rad
Basically what you need to do is:
Create a custom generator to decorate your test classes properly
Ensure the Coded UI API is initalized during any SpecFlow hooks you call
I have a solution on how to do both of these on the SpecFlow wiki: https://github.com/techtalk/SpecFlow/wiki/Using-SpecFlow-with-CodedUI-API
Related
I am currently working on Microsoft Test Manager. We are not using MTM for any automation, what we do is just pass or fail the test cases manually. Please suggest me if there any possible way for start automating my test cases. If there is any mechanism for record and playing the test cases, that's what i want.
I am in search for getting a guideline to start the automation with the help of MTM. Can anyone help me to start this thing?
This is two (or three) step process
Creating Coded UI test as mentioned above. (Andrew)
a. You have to create Coded UI project using VS ultimate version (I hope that you have it already)
b. Create your test (Eg EnvironmentCheck)
Using automated code using the recorder (Generate code for Coded UI Test) or
Using own code by have [TestMethod] attribute added on top of each test method as follows
[TestMethod]
public void EnvironmentCheck()
{
// your code here
}
More detail can be found on stack overflow question Hand Coding Coded UI Tests c. Verify you can test your method with Visual studio ( Right click inside the method and Click 'Run Tests' or 'Debug Tests')
d. Then you submit your code to TFS Build server and build it successfully
Create Test case at MTM (Microsoft Test Manager)
a. Create Test suite and new Test case at the mtm (as you did on manual test case)
b. New test case name can be 'EnvironmentCheck' and note down the ID (work ID)
c. Associate your work ID with test method using visual studio
More info msdn ==> This will make your test case to automated state at the mtm.
(Create Lab environment or VMs - This is another process. More on msdn)
d. Create a test settings or lab setup
e. Execute the test by clicking Run or Run with options
This will be overall steps, but there will be more steps based on your what you want to automate test. You can refer more on book mentioned by Elena.
What you're looking for are called coded UI tests. You start by creating an action recording of your manual test and then creating a coded UI test from that action recording. Do a google search for "coded UI" and you'll find lots of information.
There is a lot of MSMD articles describing this topic but if you want a guide here is a e-book to download for free that could help you to start:
Testing for Continuous Delivery with Visual Studio 2012
PS: If you are using Visual Studio 2010 this book will be helpful, too, since the approach is similar.
I have an ordered test that contains 4 web tests. The problem is that Visual Studio seems to have a problem loading the test results for an individual test.
I included a screenshot to supplement for the 1000 words. :)
As you may see the 3rd test, called intuitively Webservice03, failed. The screenshot above was taken after I double-clicked on the corresponding line to open the detailed information about that test run. The panels with requests and other useful information are empty.
The issue is quite annoying, because the test is part of a bigger test suite that runs periodically, so when it fails, I have to manually disable the setup and cleanup scripts from the test configuration, run them manually, then run each of the tests manually, instead of just comfortably opening the test results.
Did anyone experienced this problem?
Thanks.
Version details:
Microsoft Visual Studio 2010 Ultimate
Version 10.0.40219.1 SP1Rel
Microsoft .NET Framework
Version 4.0.30319 SP1Rel
I had the same problem and tryed to clarify it via Microsoft forum:
Web test details are not displayed when ran from an ordered test
According to the answer you can not have detailed test result for a web test run as a part of an ordered test.
Here is an alternative to Ordered Tests:
Add the web tests to a load test
In the test mix choose 'Test mix based on sequential test order'. This will achieve the same thing as an Ordered Test.
In the load test run settings, make sure 'Save Log on Test Failure' is set to True:
Set a Constant Load Pattern of 1 user
At the end of the "load" test you will be able to see the number of failures and on the Tables view you can click on the error count which takes you to a dialog that lets you browse the individual web test failures.
Is there an alternate way to test VS 2010 extension projects? Right now I have to spin a new instance of VS 2010 and wait. This takes couple of minutes and would love to know if there's another way of doing it quickly.
Depending on the project type you used to create your package (empty VSIX container or one of the extension projects from the SDK like Editor Margin, Text Adornment, ViewPort Adornment, etc) It will ask you if you want to create unit/integration tests.
The integration tests provide all the methods simulate the shell and allow you to test without actually spinning up the UI. Of course this depends if you need to do any visual verification, but if you're testing package load, service calls, pretty much anything else it's great.
Let me know how it goes.
I've been trying to create and execute webtests for one of my websites using VSTS 2008 SP1. The problem is that during playback VSTS does not play back the javascript. How can i solve this and where can I find comprehensive information on configuring VSTS to do web tests exactly as i want.
Unfortunately webtests work on a request response basis and only validate based on the contents of those. VSTS is not going to be able to run the javascript in the context of the returned html response.
It is theoretically possible to instanciate a browser object from your coded webtest and feed it the returned html+javascript and check modifications made to the html after that, but I don't see that would really end happily.
The "correctness" of your Javascript is also highly browser dependant, so you really need to test this on each of the browsers you require. Like Css and look and feel, this is still going to require manual testing.
However, Visual Studio 2010 now has a coded UI test that will allow you to test your pages in the way you want.
Visual Studio 2008 has a test type called a Load Test. It looks like I can just drop a collection of other tests into it, give it a test profile (say, 50 users constantly hitting the program), a running time, and let it run.
What isn't clear to me, is how to actually setup the tests it consumes. I need to launch a instance of the program to be tested (technically, a HTTP server; but not a web project or anything - just a console app) with some command line parameters, and have it remain up while the actual tests are running.
Basically, how would I setup a Load Test (or any test) in Visual Studio 2008 that would allow me to startup a project, then run a number of tests against it - in a configuration of my choosing as in Load Test.
The best way of doing it is to create a LoadTestPlugin to do what you want at various points in the load test.
The events exposed are:
LoadTestStarting
LoadTestFinished
LoadTestWarmupComplete
TestStarting
TestFinished
ThresholdExceeded
HeartBeat
LoadTestAborted