I've setup my build as below using the build definition. I'm using XUnit and locally my tests are discovered and run. I've tested the glob **\*spec*.dll and it finds all my test dlls and the build log shows that those dlls are in fact built.
However in the build log I get
Run VS Test Runner
No test found. Make sure that installed test
discoverers & executors, platform & framework
version settings are appropriate and try again.
Which seems to suggest it is trying to use the MSTest test runner instead of the XUnit test runner. How do I tell the build for visual studio online to use the XUnit test runner and discoverer?
This might be out of date now, but this is how I have it setup and working - downvote and let me know if it's wrong and I'll delete this. I got it from a blog post/MSDN page, but I can't locate it any more.
First you need to create a TFVC Team Project (doesn't matter if you don't use it again).
Into $/MyTFVC/BuildProcessTemplate/CustomActivities/
Checking the following files from xunit.net:
Now in VS, click the BUILD, Manage Build Controllers... option. Select the "Hosted Build Controller (Hosted)" and click "Properties...".
Enter the path where you checked in the DLL's into the "Version control path to custom assemblies" field:
You should be good to go.
Related
Seems like a pretty obvious question but I haven't been able to find this anywhere online - but what exactly counts as building something manually? As in if I do Ctrl+Shift+B on Visual Studio is that manually building? Then how could I go from that to automated build (running it from command line?). All I know is that I am supposed to use MSBuild to do automated builds on a project that is currently built 'manually'.
What is Manual Build?
Whether you are using Visual Studio or MSBuild command to build is considered to be manual build. That because you need to build your project manually every time no matter you are using Visual Studio or MSBuild command. And the hot key Ctrl+Shift+B is a quick start mode to build project in Visual Studio.
If you want to automated build, you should consider 'continuous integration' For example, TFS(Team Foundation Server), Teamcity, etc. You can easily search those continuous integration info on the internet.
The biggest difference between manual build and automated build is that you should manually build your project every time when source code changes, but automated build will execute the build automatically by continuous integration tool when source code changes, no need to build your project manually.
Hi I'm trying to run a build with code coverage, I am using Visual Stuido 2017 enterprise I have tried to enable the code coverage in VS but if i click on Edit build definition in team explorer it opens the build definition in TFS.
Apologies for the delay in posting this as an answer.
As a prerequisite to using Code Coverage, the first thing to do is to install Visual Studio Enterprise on the build agent (Which you have already done). Following this, you must then update your builds to specify that Code Coverage is/should be enabled.
To enable code coverage, ensure you have included a Visual Studio Test task (or any equivalent which supports Coverage) in your build definition.
Once you have added the Test task, you'll want to configure this task and set your test assemblies, for this you can use an absolute path to point to a DLL file, or use wildcards like I do here
$(build.sourcesDirectory)\Source\Tests**\Release*Test*.dll
This says that in my sources directory I have a folder called Source, and inside that is a Tests folder which contains all of my unit tests. Then in each Unit Tests folder I use a wildcard to say "any folder at all that contains a Release folder", look inside and take the DLL which has 'Test' in its name.
So now that the testing assemblies are gathered, you can tick the checkbox that says "Code Coverage Enabled", and you're good to go.
I will edit this post later today to include screenshots of my own personal VSTS builds.
I recently installed XUnit and the XUnit VS runner (i.e. xunit.runner.visualstudio) on Visual Studio following the documentation. The installation process uses NuGet and on completion I can successfully see all tests in the MS Test Explorer.
However, if I start a new solution and reference all the required XUnit dll's, i.e., I want to use XUnit without using NuGet to install it, the tests never show up in MS Test Explorer (no problems seeing them in the ReSharper test explorer).
I think this has something to do with the XUnit test adapter that Visual Studio uses to find XUnit tests. It appears that NuGet somehow instructs Visual Studio to create the VisualStudioTestExplorerExtensions folder on each build putting the necessary XUnit test adapter dll's (e.g.xunit.runner.visualstudio.testadapter.dll) in that folder. The MS Test Explorer then uses this test adapter to find XUnit test (see SO Question).
When I attempt to use XUnit without NuGet, the VisualStudioTestExplorerExtensions folder never gets created on build. Adding the folder manually with all XUnit dll's in it also doesn't work. It's almost as if i need to instruct Visual Studio to look in that folder, but I can't see any obvious way of doing that.
How do I use/install XUnit without using NuGet? Or, how do I get VS to look in the `VisualStudioTestExplorerExtensions' when looking for tests?
If you want to install a test runner manually, copy all the necessary dlls to:
C:\Program Files (x86)\Microsoft Visual Studio {version}\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions
Replacing {version} with the appropriate number.
The drawback here is that this is machine wide (maybe desirable in your case), and that you can't quite as easily replicate this setup on a build server. You also will probably have to restart VS whenever you make changes to these files.
I had tried all the way on the web.
But it still not listing NUnit in Tools -> Options -> SpecFlow test runner.
I install "NUnit Test Adapte" already.
I tried to attach NUnit but I can't find "NUnit" in available process.
Is I having wrong way to install NUnit? I downloaded the NUnit.zip and decompressed it in Program files.Did it need more steps to install it?And I also install the NUnit package in VS2013.
What's wrong did I done?Or what isn't done yet?
Please help me.
As the comment said, there are too many things that could be wrong. Here are a few suggestions you could try.
Verify feature files are being converted into tests.
How to Verify: In VS solution explorer, expand the feature file and you should see a feature.cs file.
Solution to fix: install the SpecFlow solution
Verify the test is a nunit test and not mstest
How to Verify: Open the feature.cs file and verify you see NUnit somewhere on that page and your scenario (no spaces) should have [TestAttribute] above it.
Solution to fix: Add nunit reference to the project & in the project's app.config Add NUnit as the provider
Verify NUnit Test Adapter is installed
How to Verify: Write a simple nunit test & see if it shows up in the test explorer window.
Solution to fix: Install it or you can always install SpecRun as an alternative. VS Test explorer should show your tests and they can be run from there.
Verify all your tests are mapped
How to Verify: Your tests should be black except where you have variables. Hitting F12 should go to definition.
Solution to fix: There can be a caching issue but for the most part, verify the regex is correct and make sure the step definition is visible to the feature file. That means it's either in the same project or is included in the app.config as a known step assembly.
I have Nunit unit test which i need to run as the part of my MS build.. I know that running all the test will slow up the build So, i need to run only the impacted test is there any way to find that out..
AFAIK running only impacted tests is not possible yet for NUnit tests. But this is possible for MSTests in Visual Studio 2010 Ultimate/Test Professional:
Recommending Tests to Run That are Affected by Code Changes
You can use Visual Studio Ultimate or Visual Studio Test Professional
2010 to help you determine which tests might have to be run, based on
coding changes that were made to the application you are testing. To
be able to use this functionality, you have to use Team Foundation
Build to build your application and use Microsoft Visual Studio 2010
for version control for your source code
Anyway you can use MSBuild Community NUnit Task to run tests from a set of the assemblies. You can do this as dependency target of standard AfterBuild target by specifying DependsOnTargets attribute.
<NUnit Assemblies="..."
IncludeCategory="..."
ExcludeCategory="..."
ToolPath="$(NUnitDllsPath)"
ProjectConfiguration="$(Configuration)"
OutputXmlFile="$(NUnitOutputPath)\UnitTests.xml"
ContinueOnError="true">
To know the impacted tests you need to track the test case code coverage. Only this way you can examine what test is impacted by the changes you are checking in. I don't know of any tool that does what you want besides Microsofts Team Foundation Server.
Running your tests as a part of the build can be done trough the Build Events properties of your project. You can execute the command line tool for NUnit.
But as PVitt already pointed out, I don't know if NUnit can work with Test Impact analysis.