I have a simple class library that gets tested with MSTest using VS 2010.
I installed the trial version of dotCover 1.2. with no ReSharper installed.
VS only has the menu items "Cover Startup Project" and "Cover Application". They open boxes where I can open all kinds of applications but no class library.
Is it not possible to calculate code coverage for a class library without another application that uses it?
In my case I have a solution containing a class library, and another solution containing the unit tests.
I wanted to see how much of the class library my unit tests covered.
After running Cover Unit Tests from the Solution context menu Dotcover by default shows me the coverage of the methods in the test classes themselves. To see the coverage of the class library you need to unselect Match coverage results with current project structure:
Note that this will only cover class library projects referenced in the tests. If you have a project that doesn't have any tests at all then that project won't show as a symbol in the Coverage Tree.
Thanks to this answer for pointing me in the right direction.
They have a command line tool that can be used without ReSharper: dotCover.exe.
Documentation is not as well as you might expect from jetbrains but it works.
http://blogs.jetbrains.com/dotnet/2010/07/running-code-coverage-from-the-console-with-dotcover/
Related
I have a suite of XUnit Tests in a .NET solution that is built in a TeamCity CI environment. The tests in question generate html templates. In our development environment, they write the results to the Visual Studio console using ITestOutputHelper. In Team City, the output is making the build artifacts way too big, and I want to find a way of suppressing the output as it is not needed in the build.
I imagine this will be achieved using an XUnit console runner command line option (but open to suggestions). I have found this list of command line options fsharp/FAKE#759, but it doesn't include one to suppress output. Is there any other (configuration-based) way of achieving what I want? (There are a lot of test classes and I don't want to have to refactor them all).
How do you replace the 'old' test lists (as in a VS2008 vsmdi file) on a VS2013/TFS2013 build agent? We have VS2013 Professional and Team Foundation Server, and it seems that test lists are no longer supported, and the 'new' test playlists are not supported either for the TFS build agent. MSBUILD no longer seems to support the concept of lists.
Can someone clarify how you should execute selected lists of tests using MSBUILD on a TFS2013 build agent?
Instead of using MSBuild command, you need to choose to use MSTest command. However, by default the TfvcTemplate.12.xaml build template does not support the MSTest.exe option when you select the Test Runner in the build definition. Good news is there is a simple way for you to make it work:
Instead of using TFS2013 build process template, you can choose to use TFS2012 build process template which enables MSTest. You can get it here. Then select to use the TFS2012 build template, click to Automated Tests part, select MSTest Test Metadata File as test runner.
An alternative is to use Test Filters. Tag your unit tests in source control using attributes:
[Category("ListX")]
[TestMethod]
public void Test(){ /* ... */ }
Then you can add a Test Filter to your build definition (sorry for the typo in the screenshot, should read TestCategory=Critical, or ListX in the example above:
See also:
https://jessehouwing.net/xaml-build-staged-execution-of-unit-tests/
I have configured a Nunit test runner build step which successfully runs my testsuite, pointing at a test sub-project of my .Net solution eg. Solution/Solution.Test/bin/debug/Solution.Test.dll.
My solution structure is as follows:
Solution
Solution.Lib
Solution.Model
Solution.Test
Lib and Model dlls are referenced in the test project.
I then turned on dotCover without any assembly filters and it performed code coverage analysis on the above test dll correctly.
I then added a filter, -:Solution.Test, and added +:Solution.Lib, and +:Solution.Model to the assembly filters and ran the build configuration, and the build.log reported the following:
Generate dotCover HTML report
[17:15:41][Generate dotCover HTML report] No source files were found under the build checkout directory W:\TeamCity\Install\buildAgent\work\7136872008cbf3bf. No source files will be included in dotCover report as source code of classes.
No executable code was detected.
The issue could be caused by one of the following:
- Include / exclude patterns are incorrect
- Assemblies are compiled without debugging information
- .pdb files are not available
- Visual Studio code coverage is enabled for MSTest
- .testrunconfig is used for MSTest and Visual Studio code coverage is not disabled (CodeCoverage section with enable="true" is present)
So I'm wondering with the above configuration what am I doing wrong, if the paths are correct?
So, I just experienced this myself. While this may not be your issue, I had a bug in my setup that took me way too long to figure out.
Since my application's namespace (and project file name) is Organization.Product, I used that in my test filter. However, it is the name of the assembly that you're filtering on.
And my assembly's output name is Product name with spaces. Yikes.
Anyway, I was able to get it to work with a simpler filter:
-:*Test*
Your filter could simply be:
-:*.Test
I would also say that you don't need to do both inclusions and exclusions. If you do an exclusion, everything else will be included. If you do an inclusion, everything else will be excluded.
I have been tasked with introducing automated unit test coverage reporting to our CI build environment (we use Hudson and MSBuild). I have used dotCover to produce the coverage data across several projects within a build job, merge that data, produce a single HTML report and integrated that with the Hudson dashboard for that job using the HTML Publisher Hudson add-in.
So far so good. But, what we ultimately want is a graphical display of unit test coverage metrics across the entire system code base (half a dozen build jobs, circa 50 projects) up on one of our team displays; a highly visible 'at a glance' test coverage status. I can build this myself with an app that will merge the dotCover coverage reports for all builds, report to xml and build my own UI around that, but is there a product out there that does this already? We are not married to dotCover, that can change, and it's doesn't have to be free or open source either.
We are using NCover to get the test coverage and a bunch of other code metrics(symbol coverage, branch coverage and cyclomatic complexity).
It is pretty good for highlighting the coverage, you can browse through the source code as part of the reports and see which lines are covered and which are not.
You can also exclude test categories or specific namespaces from your reports.
As far as I know it integrates with Hudson.
OpenCover and PartCover are both open source code coverage tools that can be integrated into build automation systems.
With both you can use ReportGenerator to display results; though PartCover does come with it's own viewer I prefer to use the ReportGenerator one myself as the HTML can be integrated quite nicely into a build status report.
You may also build your own reports using XSLT or such like as both tools have an XML output.
Our C# Test Coverage Tool tool can combine test coverage vectors for separate C# artifacts into a single overall view, out of the box.
In fact, our family of test coverage tools will can combine results from multiple different languages (e.g., C#, VB.net, C++, Java, PHP, COBOL, ...) in the same way.
The tool(s) provide both visual view of the covered code, and reports of the coverage data including in an XML form.
I like the treemap functionality from SonarQube a lot, but this only works for one 'project' in SonarQube, nevertheless, a great tool to look into.
Google for SonarQube and treemap for examples.
Is it possible to enable Resharper to automatically run all unit tests within a solution automatically when performing a local build using Visual Studio? I believe NUnit has an option "re-run last tests run" but I don't see such an option through Resharper or VS.
Thanks in advance
JP
[Edit] This may or may not matter, but I should point out that I am a one-man-team writing a medium sized application.
Posting this one for anyone that comes across it in the future:
If you're running Visual Studio 2012, someone wrote a plugin that will do exactly what you are looking for:
http://visualstudiogallery.msdn.microsoft.com/5dca9c5c-29cf-4fd7-b3ff-573e5776f0bd
I don't think ReSharper has that option, but you could always turn to something like NCrunch or Continuous Testing one of the other continuous testing frameworks for .NET. Just be aware that they are still fairly new and a little rough around the edges.
I suggest creating a build script (MSBuild, nAnt, rake, whatever) - use this script to build your solution then run tests on it.
Separating your build process from your IDE will allow you to add more things to the build (other types of tests, static checks, style checks etc).
See this blog entry on the subject.
You need dotCover, which is technically part of ReSharper Ultimate. dotCover is JetBrains' testing coverage extension. Once you have this extension there is a "Continuous Testing" feature that you can turn on and configure it to run tests on build, save, etc in addition to code coverage.
I think you're on the right track though, test early and as often as possible while you are writing the code.