Hi I'm trying to run a build with code coverage, I am using Visual Stuido 2017 enterprise I have tried to enable the code coverage in VS but if i click on Edit build definition in team explorer it opens the build definition in TFS.
Apologies for the delay in posting this as an answer.
As a prerequisite to using Code Coverage, the first thing to do is to install Visual Studio Enterprise on the build agent (Which you have already done). Following this, you must then update your builds to specify that Code Coverage is/should be enabled.
To enable code coverage, ensure you have included a Visual Studio Test task (or any equivalent which supports Coverage) in your build definition.
Once you have added the Test task, you'll want to configure this task and set your test assemblies, for this you can use an absolute path to point to a DLL file, or use wildcards like I do here
$(build.sourcesDirectory)\Source\Tests**\Release*Test*.dll
This says that in my sources directory I have a folder called Source, and inside that is a Tests folder which contains all of my unit tests. Then in each Unit Tests folder I use a wildcard to say "any folder at all that contains a Release folder", look inside and take the DLL which has 'Test' in its name.
So now that the testing assemblies are gathered, you can tick the checkbox that says "Code Coverage Enabled", and you're good to go.
I will edit this post later today to include screenshots of my own personal VSTS builds.
Related
In VSTS (hosted TFS) I have a build definition which uses MSBuild. What I would really like is to have pull requests annotated with any rule violation detections. However I can't get the analysers to work at all. What I've done so far;
Installed Microsoft.CodeAnalysis.FxCopAnalyzers in one of the projects that gets build
Verified that running "Analyze Code" in Visual Studio does output rule violations
All the *.ruleset files are not available on the self-hosted build agent, as it only has MSBuild installed (no full-fledged Visual Studio). For the same project as in point 1, I've also used a custom ruleset.
I have also build the project on a hosted build agent (which has VS2017 installed), but also to no avail.
With all of the above, the build log / build overview in VSTS doesn't list any rule violations. It's my understanding that the code analysis should be run automatically and the output should appear on the build overview page (i.e. https://MYSITE.visualstudio.com/MYPROJECT/_build/index?buildId=XXX&_a=summary&tab=details).
The Code Analysis setting is based on the configuration and platform, so you need to make sure you build the project with the same configuration and platform that enabled code analysis on build.
You also could specify it in MSBuild Arguments of MSBuild task:
/p:RunCodeAnalysis=true;CodeAnalysisRuleSet=..\RuleSets\MyMixedRecommendedRules.ruleset
I've setup my build as below using the build definition. I'm using XUnit and locally my tests are discovered and run. I've tested the glob **\*spec*.dll and it finds all my test dlls and the build log shows that those dlls are in fact built.
However in the build log I get
Run VS Test Runner
No test found. Make sure that installed test
discoverers & executors, platform & framework
version settings are appropriate and try again.
Which seems to suggest it is trying to use the MSTest test runner instead of the XUnit test runner. How do I tell the build for visual studio online to use the XUnit test runner and discoverer?
This might be out of date now, but this is how I have it setup and working - downvote and let me know if it's wrong and I'll delete this. I got it from a blog post/MSDN page, but I can't locate it any more.
First you need to create a TFVC Team Project (doesn't matter if you don't use it again).
Into $/MyTFVC/BuildProcessTemplate/CustomActivities/
Checking the following files from xunit.net:
Now in VS, click the BUILD, Manage Build Controllers... option. Select the "Hosted Build Controller (Hosted)" and click "Properties...".
Enter the path where you checked in the DLL's into the "Version control path to custom assemblies" field:
You should be good to go.
I have Nunit unit test which i need to run as the part of my MS build.. I know that running all the test will slow up the build So, i need to run only the impacted test is there any way to find that out..
AFAIK running only impacted tests is not possible yet for NUnit tests. But this is possible for MSTests in Visual Studio 2010 Ultimate/Test Professional:
Recommending Tests to Run That are Affected by Code Changes
You can use Visual Studio Ultimate or Visual Studio Test Professional
2010 to help you determine which tests might have to be run, based on
coding changes that were made to the application you are testing. To
be able to use this functionality, you have to use Team Foundation
Build to build your application and use Microsoft Visual Studio 2010
for version control for your source code
Anyway you can use MSBuild Community NUnit Task to run tests from a set of the assemblies. You can do this as dependency target of standard AfterBuild target by specifying DependsOnTargets attribute.
<NUnit Assemblies="..."
IncludeCategory="..."
ExcludeCategory="..."
ToolPath="$(NUnitDllsPath)"
ProjectConfiguration="$(Configuration)"
OutputXmlFile="$(NUnitOutputPath)\UnitTests.xml"
ContinueOnError="true">
To know the impacted tests you need to track the test case code coverage. Only this way you can examine what test is impacted by the changes you are checking in. I don't know of any tool that does what you want besides Microsofts Team Foundation Server.
Running your tests as a part of the build can be done trough the Build Events properties of your project. You can execute the command line tool for NUnit.
But as PVitt already pointed out, I don't know if NUnit can work with Test Impact analysis.
I have set up a TFS 2010 Team Build Configuration to run continuous builds.
I currently have 2 Test Projects that run inside Visual Studio 2010 IDE with no problems.
When I queue the solution (with the test projects) to build, the build log reports:
"No Test Results".
My Build Process parameters Automated Test Rule matches my test assembly names:
Run tests in assemblies matching **\*_Test.dll.
After inspecting the Build folder C:/Builds/2/[ProjectName]/[BuildName]/Binaries, I noticed that there were no .dll/.pdb files for my test projects built; even though all other project required dlls are in here.
The Test Project folders do exist in C:/Builds/2/[ProjectName/[BuildName]/Sources.
My Build Process parameters referenced under "Items to Build" => "Configurations to Build" :
Any CPU|Relase
Under Configuration Manager for my soltuion, for Any CPU and Release, I Do have my Test Projects checked off under the Build column.
From all of the documentation I have read, my tests should be running, but from the above inspection it appears that they are not even building.
Any insight or ideas into getting these unit test projects to compile and run on my TFS 2010 Build Server would be greatly appreciated.
Thanks!
When setting up the build, you can point to the vsmdi file instead of putting in the wildcard. Does that yield the same result?
I have a set of Google Test-based unit tests for a native C++ DLL I'm developing. The DLL is in its own project, and the test project is dependent upon it. The test project has a Post-Build Event script that runs the tests.
My problem: Whenever the test project gets rebuilt, it runs the tests as expected. However, making a change to the dependent DLL does not always cause the test project to be rebuilt. Specifically, it seems that if none of the DLL's header files change, Visual Studio decides that the test project doesn't need to be rebuilt.
Is there any way to force Visual Studio to rebuild the test project, or to run the post-build event, whenever the DLL gets rebuilt?
I can force the tests to be re-run by right-clicking one of the files in the test project and choosing the Compile menu item, then doing a build. I'd prefer to eliminate that manual-and-often-forgotten step.
I suspect I could get the result I want if I were to include the DLL project's source files in my test project, but I'd really like to run the tests against the actual DLL.
Related but unhelpful question: How to setup Google C++ Testing Framework (gtest) on Visual Studio 2005