I have created some unit tests in Visual Studio 2012 that I need to test out some API functionality. The API is used to create plugins that run within another application.
Does anyone know how I can run Unit Tests outside of the Visual Studio Environment i.e. not using the Test Explorer. The Unit Tests need to be called by a Test Plugin that is run by the main application and in the same process space.
Hope somebody can help.
I assume you running MSTest.
You can run tests outside VS using MSTest.exe.
Call this exe from your plugin with appropriate command line options.
Related
In a project using official Microsoft unit test framework I can put test in categories however I can't run these categories from the test explorer in Visual Studio.
I specifically want to set one category to Run after build (not on a build server via CI but locally) because I might have specific test I do not want to run that often.
Is there any way to do this or is the testing aspect from within Visual Studio so limited?
When running tests from the test explorer window, you can filter by "Trait".
Entering Trait:"Important" into the search bar will select only tests marked with [TestCategory("Important")].
To exclude certain tests prefix with '-' e.g. -Trait:"Slow"
You can achieve this using vstest (part of Visual Studio), either via CI (Jenkins, etc.), a Windows Scheduled Task, launched batch file, etc.
Simply call something like:
Vstest.console.exe myTestProject.dll /TestCaseFilter:TestCategory=Nightly
Please reference:
https://msdn.microsoft.com/en-us/library/dd286683.aspx
I migrated my web performance tests project's build to TFS 2013 build process template for auto nuget restore by following this document.
My solution has .webtest files which are the containers for web performance tests. But VsTest.console.exe doesn't recognize the .webtest tests. I looked at the discoverers and found that none of the ones installed support the .webtest extension.
I tried to find test adapters for running .webtests using vstest.console.exe but found nothing.
This document says the following for vstest.console.exe:
You can run automated unit and coded UI tests from the command line.
And this for mstest.exe:
You can run automated Web performance and load tests from the command line either locally or by using a test controller or test agents.
Why doesn't vstest.console.exe support .webtests ? Wasn't it designed as a replacement to mstest.exe for running any kind of tests using adapters ?
Also, if it means that I need to write a custom adapter that can read and run tests with .webtest extension, what does it take for me to do that ?
vstest.console can run webtests using the following:
"c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" SmokeTest.orderedtest /Settings:..\..\..\local.testsettings"
You need to create an orderedtest and put your webtest in it.
vstest.console can run .webtest Webtests. Enter the path to the exe in quotation marks and then the path to the webtest:
"c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" WebTest1.webtest
Looks like vstest.console.exe cannot run load tests and web tests. My only option is to use MsTest.exe through TFS 2013 by customizing the build process template to suit my needs.
Edit: I'm wrong. You can run web tests using vstest.console.exe.
The NUnit Test Adapter for VS 2012 allows you to run NUnit tests from within the Visual Studio 2012 IDE via the Test Explorer. However, the reason I have NUnit tests in the first place is because I have tests written against SharePoint 2010 (I know, these are Integration Tests, not Unit Tests), which, due to the requirement that the SharePoint API be accessed through a 64-bit client means that MSTest/VS can't run them.
I was hoping that with the NUnit Test Adapter, however, I would be able to run the SharePoint tests from within Test Explorer in Visual Studio but I keep getting the error message:
SetUp failed for test fixture xyz
SetUp : System.IO.FileNotFoundException : The Web application at http://sp could not be found. Verify that you have typed the URL correctly. If the URL should be serving existing content, the system administrator may need to add a new request URL mapping to the intended application.
This indicates to me that the NUnit Test Adapter is running as a 32-bit process (I suspect the process is vstest.executionengine.x86.clr20.exe).
Is there any way to run NUnit tests that target SharePoint 2010 using the NUnit Test Adapter and Test Explorer in Visual Studio 2012?
To run tests in 64-bit process:
In Visual Studio 2012, select TEST->Test Settings->Default Processor Architecture->x64 menu item
With vstest.console.exe, specify /Platform:x64 command line option.
You may also be interested in SharePoint Emulators, which allow you run SharePoint integration tests in isolation.
I have a test project that I want to automate on a test server. For now, I have installed the Visual Studio agents (Test Controller, Test Agent and Test Lab) on my local machine, as I hope to try it out here before touching anything on the test server.
The problem I'm having is that I can't find adequate documentation on automating tests using the Visual Studio agent tools. I have successfully set up and configured the test agent and test controller on my local machine, but I'm not sure where I go from here. The test controller is connected to Team Foundation Server. I have a test project contained within TFS - I want to run the tests within it.
I tried launching the VS test manager and attempted to find a way to run tests from there, but I was unsuccessful.
How do I go about doing this? (in simple steps)
My previous experience is with NUnit, and launching automated tests was as simple is launching the command line NUnit program, providing it with the path of the test project and a few other parameters.
Once the Visual Studio agents are set up and configured correctly, it's simply a matter of running tests against a .testsettings file which is connected to the test controller.
When the tests are run (either via the Visual Studio GUI or the mstest.exe command-line program), the active testsettings file will run the tests on any test agent which is connected to the test controller.
So in terms of automating tests, it's just a matter of specifying the correct testsettings file as one of the command-line switches to the MSTest.exe.
A testsetting file can either be created within the Visual Studio solution or alternatively they can be created within Microsoft Test Manager (automated tests can also be scheduled using the MS Test Manager command-line program, tcm.exe).
To select an active testsetting file to use when running tests from Visual Studio 2010, select the 'Test' menu on the toolbar, then 'Select Active Test Settings'.
I have Nunit unit test which i need to run as the part of my MS build.. I know that running all the test will slow up the build So, i need to run only the impacted test is there any way to find that out..
AFAIK running only impacted tests is not possible yet for NUnit tests. But this is possible for MSTests in Visual Studio 2010 Ultimate/Test Professional:
Recommending Tests to Run That are Affected by Code Changes
You can use Visual Studio Ultimate or Visual Studio Test Professional
2010 to help you determine which tests might have to be run, based on
coding changes that were made to the application you are testing. To
be able to use this functionality, you have to use Team Foundation
Build to build your application and use Microsoft Visual Studio 2010
for version control for your source code
Anyway you can use MSBuild Community NUnit Task to run tests from a set of the assemblies. You can do this as dependency target of standard AfterBuild target by specifying DependsOnTargets attribute.
<NUnit Assemblies="..."
IncludeCategory="..."
ExcludeCategory="..."
ToolPath="$(NUnitDllsPath)"
ProjectConfiguration="$(Configuration)"
OutputXmlFile="$(NUnitOutputPath)\UnitTests.xml"
ContinueOnError="true">
To know the impacted tests you need to track the test case code coverage. Only this way you can examine what test is impacted by the changes you are checking in. I don't know of any tool that does what you want besides Microsofts Team Foundation Server.
Running your tests as a part of the build can be done trough the Build Events properties of your project. You can execute the command line tool for NUnit.
But as PVitt already pointed out, I don't know if NUnit can work with Test Impact analysis.