I migrated my web performance tests project's build to TFS 2013 build process template for auto nuget restore by following this document.
My solution has .webtest files which are the containers for web performance tests. But VsTest.console.exe doesn't recognize the .webtest tests. I looked at the discoverers and found that none of the ones installed support the .webtest extension.
I tried to find test adapters for running .webtests using vstest.console.exe but found nothing.
This document says the following for vstest.console.exe:
You can run automated unit and coded UI tests from the command line.
And this for mstest.exe:
You can run automated Web performance and load tests from the command line either locally or by using a test controller or test agents.
Why doesn't vstest.console.exe support .webtests ? Wasn't it designed as a replacement to mstest.exe for running any kind of tests using adapters ?
Also, if it means that I need to write a custom adapter that can read and run tests with .webtest extension, what does it take for me to do that ?
vstest.console can run webtests using the following:
"c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" SmokeTest.orderedtest /Settings:..\..\..\local.testsettings"
You need to create an orderedtest and put your webtest in it.
vstest.console can run .webtest Webtests. Enter the path to the exe in quotation marks and then the path to the webtest:
"c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" WebTest1.webtest
Looks like vstest.console.exe cannot run load tests and web tests. My only option is to use MsTest.exe through TFS 2013 by customizing the build process template to suit my needs.
Edit: I'm wrong. You can run web tests using vstest.console.exe.
Related
I've got a Visual Studio Solution File conatining multiple projects. About half of them are Unit Test Projects.
Visual Studio Test Explorer reliably picks up all Test Projects and allows me to run all of them at once.
We also got a CI that should run these Tests. Is there a way to use the Test Explorer functionality from the command line? Currently I have to pass a list of dll's and executables to the vstest.console.exe Tool.
I searched google for solutions but only found solutions that pass dll files to vstest.console.exe. I would expect a possibility to run the tests from a solution also from the command line similar to how dotnet test allows me to do this.
Unfortunately I cannot use dotnet test to reach my goal, as dotnet does not yet support native projects (which are also conatined in my solution).
In a project using official Microsoft unit test framework I can put test in categories however I can't run these categories from the test explorer in Visual Studio.
I specifically want to set one category to Run after build (not on a build server via CI but locally) because I might have specific test I do not want to run that often.
Is there any way to do this or is the testing aspect from within Visual Studio so limited?
When running tests from the test explorer window, you can filter by "Trait".
Entering Trait:"Important" into the search bar will select only tests marked with [TestCategory("Important")].
To exclude certain tests prefix with '-' e.g. -Trait:"Slow"
You can achieve this using vstest (part of Visual Studio), either via CI (Jenkins, etc.), a Windows Scheduled Task, launched batch file, etc.
Simply call something like:
Vstest.console.exe myTestProject.dll /TestCaseFilter:TestCategory=Nightly
Please reference:
https://msdn.microsoft.com/en-us/library/dd286683.aspx
I created a Load Test that I would like to run every night so that I can compare the results over time.
I'm trying to trigger the load test to run from a TFS build. Tests are run by Visual Studio Online agents.
Here is the MSTest command that is being executed, however I'm getting an error and the load tests never run.
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe" /nologo /usestderr /testSettings:"F:\Builds\8269\Tests\LoadTests\src\Tests\xxxxx\Remote.testsettings" /searchpathroot:"F:\Builds\8269\Tests\LoadTests\bin" /resultsfileroot:"F:\Builds\8269\Tests\LoadTests\tst" /testcontainer:"F:\Builds\8269\Tests\LoadTests\bin\LoadTest2.loadtest" /publish:"https://xxxxx.visualstudio.com/defaultcollection" /noprompt /publishbuild:"vstfs:///Build/Build/13578" /teamproject:"Tests" /platform:"Any CPU" /flavor:"Release" /runtitle:"MySweetLoadTests"
Error...
The current SynchronizationContext may not be used as a TaskScheduler.
From here (The current SynchronizationContext may not be used as a TaskScheduler)
It says I need to setup a SynchronizationContext, however I'm not sure how to do that within a load test.
Thanks!
This now supported. Please refer to the blog.
http://blogs.msdn.com/b/visualstudioalm/archive/2015/08/24/cloud-load-test-support-in-mstest-exe-command-line-and-xaml-builds.aspx
I have created some unit tests in Visual Studio 2012 that I need to test out some API functionality. The API is used to create plugins that run within another application.
Does anyone know how I can run Unit Tests outside of the Visual Studio Environment i.e. not using the Test Explorer. The Unit Tests need to be called by a Test Plugin that is run by the main application and in the same process space.
Hope somebody can help.
I assume you running MSTest.
You can run tests outside VS using MSTest.exe.
Call this exe from your plugin with appropriate command line options.
I have a test project that I want to automate on a test server. For now, I have installed the Visual Studio agents (Test Controller, Test Agent and Test Lab) on my local machine, as I hope to try it out here before touching anything on the test server.
The problem I'm having is that I can't find adequate documentation on automating tests using the Visual Studio agent tools. I have successfully set up and configured the test agent and test controller on my local machine, but I'm not sure where I go from here. The test controller is connected to Team Foundation Server. I have a test project contained within TFS - I want to run the tests within it.
I tried launching the VS test manager and attempted to find a way to run tests from there, but I was unsuccessful.
How do I go about doing this? (in simple steps)
My previous experience is with NUnit, and launching automated tests was as simple is launching the command line NUnit program, providing it with the path of the test project and a few other parameters.
Once the Visual Studio agents are set up and configured correctly, it's simply a matter of running tests against a .testsettings file which is connected to the test controller.
When the tests are run (either via the Visual Studio GUI or the mstest.exe command-line program), the active testsettings file will run the tests on any test agent which is connected to the test controller.
So in terms of automating tests, it's just a matter of specifying the correct testsettings file as one of the command-line switches to the MSTest.exe.
A testsetting file can either be created within the Visual Studio solution or alternatively they can be created within Microsoft Test Manager (automated tests can also be scheduled using the MS Test Manager command-line program, tcm.exe).
To select an active testsetting file to use when running tests from Visual Studio 2010, select the 'Test' menu on the toolbar, then 'Select Active Test Settings'.