Is there an elegant way to setup StartUp and Cleanup methods to run for a Load Test in Visual Studio ?
I have a Controller - Agents system for load testing and I need to find a way to run a method before the Test runs and after it finishes but only in the controller.
Plugins can be written for load tests. Basically the plugin provides code to be run during the initialisation of the test run. That code can attach event handlers for the following and hence the test can be manipulated in many ways. (List of events copied from here.)
LoadTestStarting, LoadTestFinished, LoadTestWarmupComplete, TestStarting, TestFinishedTestSelected, ThresholdExceeded, HeartBeat, and LoadTestAborted.
See this Microsoft page (and the pages it links to) for more details on writing load test plugins.
Visual Studio tests also use a ".testsettings" file to specify several aspects of the test. This file allows set-up and clean-up scripts to be specified. These can be seen in the image below "Configure the test controller and roles for remote data collection and execution" and in the "Test Settings: Setup and Cleanup Scripts" parts of this Microsoft page.
Not part of the question, but added for completeness. Plugins can also be written for the web tests. Two forms are provided, web test request plugins and http://msdn.microsoft.com/en-us/library/ms243191(v=vs.110).aspx plugins.
See also this Stackoverflow question VS2010 Load Testing: How can I perform custom action that is run once prior to each load test
Related
I have created Web performance testing using visual studio 2017, Most of the pages are data driven by Login, Change of lists etc is there
I have added extraction rule, and when i do load test of the same WPT it gives me errors 403 and fail's the test
My question here is how should I make it work
Thanks in advance
A 403 error usually means that your VS script is failing at login. It is most likely happens due to one or several dynamic values that VS plays back as recorded. To fix it you need to find and manually correlate them by creating extraction rules and parameters somewhere at the beginning of your script.
It looks like you already created some extractors, but if your script is still failing, chances are that you did not create all of them. Try the manual correlation technique described in my article.
Or check our Web Test Builder for Visual Studio referenced here to automatically create missing extractors and parameter.
We have an established CI process which links 118 Ranorex test cases to MTM test cases and runs them on a dedicated automation VM, which is triggered by a timed build in Visual Studio.
The CI process is still working correctly, but I have just tried to add another test and found that due to changes in the layout of the test case work item page in VSO/MTM, the 'Associated Automation' function is no longer shown.
So you used to click on the ellipses and the list of test cases to be linked would appear: VS CodedUI MTM_VSO work item linking
As you can see in this image, the 'Associated Automation' tab has moved and the linking mechanism is no longer present: Missing MTM_VSO test case mechanism
I think it is tied into this recent Microsoft VSO update hxxps://www.visualstudio.com/en-us/news/2015-dec-10-vso (sorry for posting the link this way but I can't have more than 2 links) but I wondered if anyone else had encountered this issue and how to resolve it.
Sorry for the long post, but I wanted to make everything clear.
There is already a feedback submitted for this on Visual Studio page, refer to this link for details: https://connect.microsoft.com/VisualStudio/Feedback/Details/2137069
Currently, you can create the test case from your test assemblies via following steps first and then update the test cases.
Run "Developer Command Prompt" and navigate to the debug folder via "cd" command. For example: cd C:\xxx\xxx\solution\testproject\bin\debug
Create test case for the test assemblies via "tcm testcase" command: tcm testcase /collection:https:xxxxx/DefaultCollection /teamproject:yourproject /import /storage:xxx.dll
I am currently working on Microsoft Test Manager. We are not using MTM for any automation, what we do is just pass or fail the test cases manually. Please suggest me if there any possible way for start automating my test cases. If there is any mechanism for record and playing the test cases, that's what i want.
I am in search for getting a guideline to start the automation with the help of MTM. Can anyone help me to start this thing?
This is two (or three) step process
Creating Coded UI test as mentioned above. (Andrew)
a. You have to create Coded UI project using VS ultimate version (I hope that you have it already)
b. Create your test (Eg EnvironmentCheck)
Using automated code using the recorder (Generate code for Coded UI Test) or
Using own code by have [TestMethod] attribute added on top of each test method as follows
[TestMethod]
public void EnvironmentCheck()
{
// your code here
}
More detail can be found on stack overflow question Hand Coding Coded UI Tests c. Verify you can test your method with Visual studio ( Right click inside the method and Click 'Run Tests' or 'Debug Tests')
d. Then you submit your code to TFS Build server and build it successfully
Create Test case at MTM (Microsoft Test Manager)
a. Create Test suite and new Test case at the mtm (as you did on manual test case)
b. New test case name can be 'EnvironmentCheck' and note down the ID (work ID)
c. Associate your work ID with test method using visual studio
More info msdn ==> This will make your test case to automated state at the mtm.
(Create Lab environment or VMs - This is another process. More on msdn)
d. Create a test settings or lab setup
e. Execute the test by clicking Run or Run with options
This will be overall steps, but there will be more steps based on your what you want to automate test. You can refer more on book mentioned by Elena.
What you're looking for are called coded UI tests. You start by creating an action recording of your manual test and then creating a coded UI test from that action recording. Do a google search for "coded UI" and you'll find lots of information.
There is a lot of MSMD articles describing this topic but if you want a guide here is a e-book to download for free that could help you to start:
Testing for Continuous Delivery with Visual Studio 2012
PS: If you are using Visual Studio 2010 this book will be helpful, too, since the approach is similar.
I have an ordered test that contains 4 web tests. The problem is that Visual Studio seems to have a problem loading the test results for an individual test.
I included a screenshot to supplement for the 1000 words. :)
As you may see the 3rd test, called intuitively Webservice03, failed. The screenshot above was taken after I double-clicked on the corresponding line to open the detailed information about that test run. The panels with requests and other useful information are empty.
The issue is quite annoying, because the test is part of a bigger test suite that runs periodically, so when it fails, I have to manually disable the setup and cleanup scripts from the test configuration, run them manually, then run each of the tests manually, instead of just comfortably opening the test results.
Did anyone experienced this problem?
Thanks.
Version details:
Microsoft Visual Studio 2010 Ultimate
Version 10.0.40219.1 SP1Rel
Microsoft .NET Framework
Version 4.0.30319 SP1Rel
I had the same problem and tryed to clarify it via Microsoft forum:
Web test details are not displayed when ran from an ordered test
According to the answer you can not have detailed test result for a web test run as a part of an ordered test.
Here is an alternative to Ordered Tests:
Add the web tests to a load test
In the test mix choose 'Test mix based on sequential test order'. This will achieve the same thing as an Ordered Test.
In the load test run settings, make sure 'Save Log on Test Failure' is set to True:
Set a Constant Load Pattern of 1 user
At the end of the "load" test you will be able to see the number of failures and on the Tables view you can click on the error count which takes you to a dialog that lets you browse the individual web test failures.
Visual Studio 2008 has a test type called a Load Test. It looks like I can just drop a collection of other tests into it, give it a test profile (say, 50 users constantly hitting the program), a running time, and let it run.
What isn't clear to me, is how to actually setup the tests it consumes. I need to launch a instance of the program to be tested (technically, a HTTP server; but not a web project or anything - just a console app) with some command line parameters, and have it remain up while the actual tests are running.
Basically, how would I setup a Load Test (or any test) in Visual Studio 2008 that would allow me to startup a project, then run a number of tests against it - in a configuration of my choosing as in Load Test.
The best way of doing it is to create a LoadTestPlugin to do what you want at various points in the load test.
The events exposed are:
LoadTestStarting
LoadTestFinished
LoadTestWarmupComplete
TestStarting
TestFinished
ThresholdExceeded
HeartBeat
LoadTestAborted