I have written few Unit test using MStest in visual studio and trying to generated the code coverage for the instrumented assembly which i have marked in the test settings. Although i am getting the code coverage for all of the namespaces in the assembly but one. Is there a setting i need to make to make this namespace appear in the codecoverage window.
Related
I am making changes to an existing code base and ran Visual Studio Code Coverage tool to obtain code coverage against all tests in the test project. I see that areas that have code coverage are highlighted. However, is there an easy way to find which test provided coverage for a portion of code?
Visual Studio CodeLens provides a feature called Associated Unit tests which shows all unit tests associated with the method. This was what I was looking for.
I'm trying to run MSpec tests in Visual Studio's test system using Gallio, but it appears to be ignoring the MSpec plugin from the IDE. What I did:
Installed Gallio
Added the MSpec NuGet package to my test project
Added the Gallio NuGet package to my test project
Copied the three files required for the MSpec Gallio integration to an "MSpec" folder in Gallio's installation directory
Wrote an MSpec test
Ran "All Tests in Solution" in the IDE
This yielded the following error
"No tests were run because no tests are loaded or the selected tests are disabled"
To check Gallio itself, I added a reference to MbUnit and created a simple MbUnit test. I also ran Gallio's other tools to check if the plugin was being picked up at all. I even restarted VS for good measure.
Results
Running the tests using Gallio.Echo or Gallio.Icarus successfully executes both MbUnit and MSpec tests, so the plugin is being picked up. Running the tests from Visual Studio runs the MbUnit test but not the MSpec test, so Gallio integration appears to be working (just not the MSpec plugin). Any thoughts?
Before anyone mentions it: yes, I'd rather be using ReSharper but licenses are not an option for my situation at the moment.
Update: The is now a project to integrate MSpec into VS2012's test system. It's available on the Visual Studio Gallery and the source is on GitHub.
After starting to debug the problem with both project's sources, it appears there are two underlying issues causing the incompatibility between MSpec and Gallio's VS runner. However, both issues stem from the fact that the VS runner uses Cecil to load type information from assemblies (presumably to avoid loading the test assembly into the main VS AppDomain).
I'll update this answer as I uncover more (hopefully resulting in one or more patch to fix the issues).
Issue 1: Nested types are ignored
I had been using nested types for my tests (SubjectSpec+when_something), which are ignored by the Cecil-based reflection used by Gallio's VS runner to avoid loading the test assembly into the app domain.
This ended up being a relatively simple fix, which I've submitted as a patch to the Gallio developers. I'll update this answer if/when I get confirmation of which release it will be part of.
Issue 2: MSpec causes "An exception occurred while invoking a test driver"
If the container type is removed to avoid issue 1, MSpec throws this error. The reason for this is that MSpec is trying to create an instance of the test object, but Gallio is only supplying it a wrapper Type object that can't actually be created.
I'm trying to find a way to migrate from mstest to xunit and still benefit from the IDE integration which made me choose mstest in the first place. Gallio seems to accomplish this noble goal, plus it's free (don't want Reshaper or TestDriven.net). But I can't get code coverage to work.
My solution contains of two projects: project SUT (the assembly I need to test) and project SUT.Tests which is a VS test project (this allows Gallio to display xunit tests in VS's Test View). I have code coverage enabled in Local.testSettings for SUT.dll and instrumentation is in place. After the test run completes there is no code coverage. Code Coverage Results window reports: Empty results generated: none of the instrumented binary was used. Look at test run details for any instrumentation problems. Unfortunately the test run details do not contain any "instrumentation problems". I tried unchecking the instrument assemblies in place checkbox and re-running the unit test; same result.
Any idea what's wrong?
My setup:
- Windows 7 x64
- VS 2010 Premium (SP1)
- xUnit 1.8
- Gallio 3.3.1 x64 (installer, not zip)
Apologies for answering my own question:
Turns out Gallio loads SUT.dll from SUT.Tests\Bin\Debug rather than SUT\Bin\Debug. I added this path to code coverage details, selected it instead of the default SUT.dll path and now it works! This is better described at here, towards the end of the post.
I have configured Specflow to target the MsTest framework (instead of NUnit) by specifying it like this in the app.config of my 'specs' class library:
<configSections>
<section name="specFlow"
type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
</configSections>
<specFlow>
<unitTestProvider name="MsTest.2010" />
</specFlow>
So once it's in place, I can see that my test fixtures are produced correctly by the Specflow custom tool, with correct TestClassAttribute() and methods, etc:
[System.CodeDom.Compiler.GeneratedCodeAttribute("TechTalk.SpecFlow", "1.3.3.0")]
[System.Runtime.CompilerServices.CompilerGeneratedAttribute()]
[Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute()]
...
The specs class builds, but now I cannot run the tests using the Test --> Run --> All Tests in Solution inside Visual Studio 2010 on my vista 64 box. Why doesn't VS recognize these as valid tests to run?
As per Dror Helper and Alex Duggleby you'll want to add the following line to your .csproj file:
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
Add it after the FileAlignment element, and reload the project. It should now be an MS Test project and you get the MS Test functionality in the context of this project. The Guids mean:
{3AC096D0-A1C2-E12C-1390-A8335801FDAB} - Test Project
{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} - C# Class Library
I had to recreate the project as a Test Project and not merely a Class Library -- because I had started development with NUnit and SpecFlow, I had created a vanilla class library to hold my specs that had the NUnit attributes decorated. I thought I could simply change the app.config of this existing project to point at the mstest framework and stop using NUnit, but VS2010 never recognized the tests, despite the correct creation of the stubs by specflow's custom tool.
So...I added a new Test Project to my solution, moved all of my spec code to that new project, then recompiled, and viola, VS2010 recognizes the tests. I'm sure there is a GUID it is looking for in the XML of the .csproj file or something that tells it to wire up the testing framework, but I didn't dig that far.
Hope this helps someone.
To change your class library project template into a test project, modify the .csproj and add the following line:
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
to the first property group element:
<PropertyGroup>
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
I have a Visual Studio 2010 .NET 4 solution with C# projects and a C++/CLI project.
I use Visual Studio's built in unit tests and code coverage.
Other than the fact that Visual Studio 2010 coverage tool for C++/CLI projects seems to be much weaker than Visual Studio 2008 coverage tool, I get weird results.
For example, I get uncovered code in this file:
c:\program files (x86)\microsoft
visual studio 10.0\vc\include\xstring
And some other files in that directory.
I want to exclude this code from coverage results.
Is there a way to put some exclude attributes on that code?
If not, is there a different automatic way to exclude that code from coverage? If not, is there a way to use EXCLUDE option to exclude it? Can it be done automatically within Visual Studio without running the coverage tool from command prompt?
Any other solutions?
Well, the MSDN documentation you linked to says:
EXCLUDE option is supported with code coverage.
To exclude all functions in a namespace use the wildcard character:
MyNamespace::*
Have you tried /EXCLUDE:std::* ? It sounds as if that would be the way to go.