Currently I have a setup of build controllers. I want to run automation test scripts referring to some dll. I tried giving the set-up in Process tab of build definition under automated tests. But nothing happens. Please guide what things are necessary for running automated tests.
This is a good guide to setting up automated unit test in TFS Builds: http://geekswithblogs.net/jakob/archive/2009/06/03/tfs-team-build-2010-running-unit-tests.aspx
Related
I have a generic Selenium Test which needs to run over all projects how would i do that with Visual Studio online? I can get it to work on one project at a test but not sure how to make it so this test can run on all projects?
Generally you need to write the specific Selenium test for the specific project. However if all the projects you mentioned have the same structure/components/features, then you can reuse the Selenium test project for them.
I don't think it's this is applicable if you mean create tests together which want to apply for all the different projects.
You can reference below articles about the Selenium Test in TFS:
Get started with Selenium testing in a CD pipeline
Parallel Testing in a Selenium Grid with VSTS
I have the following project in my Visual Studio 2015 solution. I want to execute jasmine tests as part of our TFS CI build. I can run my tests successfully when I open the SpecRunner.html file although I'm failing to get them to run as part of the TFS build. I've included a screen shot of my TFS build step definition. The build is successful but the tests are not executed.
Please advise, any help greatly appreciated.
Many thanks,
Project and file locations
TFS build step
First make sure your environment on the build sever is set correct. You could manually run the test on the build server.
To use a NuGet package for the Chutzpah test runner , then you can avoid having to manually unpack the VSIX and get it into source control to deploy it to the build host.
About the detail step and build definition you could take a look at this tutorial: nUnit and Jasmine.JS unit tests in TFS/VSO vNext build
I have a C# project in Xamarin for which I've written a few NUnit unit tests. To alert me to regressions as soon as possible I'd like to have Xamarin run my unit tests after every build. I'm not using CI or a build server, everything is local to my dev machine. Is there a way to modify the build task to run my tests after each compilation?
You can write a simple build script in which you invoke nunit-console.exe after your msbuild of your respective Xamarin projects.
Otherwise, I would personally recommend using Cake Build as it's fairly simple to get up and running and provides NUnit runners:
https://github.com/cake-build/cake (http://cakebuild.net/)
https://github.com/Redth/Cake.Xamarin (Xamarin Addin)
Both of these solutions should work fine locally.
Taking elements from #JonDouglas' suggestion I ended up finding the Custom Commands section and making an After Build with nunit-console ${TargetFile}. It's in the Options dialog here:
We use TFS 2012
Through a build definition we execute automated tests from MTM test plans.
How can we ensure TFS uses vstest.console.exe to run each test?
I've read online from other posts that TFS 2012 let's you choose either MSTest or VSTest as the test runner.
I've also read that if you use MTM you effectively are locked into using a test settings file(.testsettings) which means you are locked into using MSTest. This seems to go against TFS 2012 giving you a choice.
Test Manager currently is only supporting MsTest. This is a limitation in Test Manager. It's the same issue that prevents you from associating a xUnit or NUnit test automation on a test case.
So while TFS 2012 gives you the choice, each choice comes with its own set of limitations.
To be clear:
New test runner: Work in Continuous Integration, you specify which tests to run by selecting the new agile test runner (in TFS 2013 this is a fixed choice in the default templates, template customization is required to run mstest). It can run tests during build and you can directly invoke it from the commandline if you want to run tests after deployment. You can use a .runsettings file to specify certain options it should pick up. This option is required to execute 3rd party test frameworks like NUnit, XUnit.NET, Chutzpah etc.
The MTM test runner: Execute automated tests that are associated to a Test Case work item or execute tests using the Test Agent to run tests from a remote system using the Team Test infrastructure. You can use a .testsettings file to specify certain options it should pick up. These tests will be executed using MsTest and cannot contain 3rd party test frameworks.
In your Continuous Integration build you can configure multiple Test Runs, each can be configured to a specific test framework. So you can have both options in your build.
More on the differences and how you can use the different settings files.
Over time all test options will be moved to the new test runner. In TFS 2013 the option to configure a MsTest based test run in your Build Definition has been removed by default. You will need to customize the build process to select a different test runner.
I have Nunit unit test which i need to run as the part of my MS build.. I know that running all the test will slow up the build So, i need to run only the impacted test is there any way to find that out..
AFAIK running only impacted tests is not possible yet for NUnit tests. But this is possible for MSTests in Visual Studio 2010 Ultimate/Test Professional:
Recommending Tests to Run That are Affected by Code Changes
You can use Visual Studio Ultimate or Visual Studio Test Professional
2010 to help you determine which tests might have to be run, based on
coding changes that were made to the application you are testing. To
be able to use this functionality, you have to use Team Foundation
Build to build your application and use Microsoft Visual Studio 2010
for version control for your source code
Anyway you can use MSBuild Community NUnit Task to run tests from a set of the assemblies. You can do this as dependency target of standard AfterBuild target by specifying DependsOnTargets attribute.
<NUnit Assemblies="..."
IncludeCategory="..."
ExcludeCategory="..."
ToolPath="$(NUnitDllsPath)"
ProjectConfiguration="$(Configuration)"
OutputXmlFile="$(NUnitOutputPath)\UnitTests.xml"
ContinueOnError="true">
To know the impacted tests you need to track the test case code coverage. Only this way you can examine what test is impacted by the changes you are checking in. I don't know of any tool that does what you want besides Microsofts Team Foundation Server.
Running your tests as a part of the build can be done trough the Build Events properties of your project. You can execute the command line tool for NUnit.
But as PVitt already pointed out, I don't know if NUnit can work with Test Impact analysis.