OpenCover showing 0 lines covered with mstest - mstest

'here is output...'
Loading C:\TEMP\BankDemo_mstest\Test_BankDemo\bin\Debug\Test_BankDemo.dll...
Starting execution...
Results Top Level Tests
------- ---------------
Error Test.BankDemo.AccountTest.CreditTest
Error Test.BankDemo.AccountTest.DebitTest
Error Test.BankDemo.AccountTest.FreezeTest
0/3 test(s) Passed, 3 Error
Summary
-------
Test Run Error.
Error 3
--------
Total 3
This is the command I used
OpenCover\OpenCover.Console.exe -register:user
-output:"Codecoverage.xml"
-mergebyhash
-target:"C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe"
targetargs:"/testcontainer:
"C:\TEMP\BankDemo_mstest\Test_BankDemo\bin\Debug\Test_BankDemo.dll"
/noisolation"
-filter:"-[Bank.*]* +[Bank*]* +[Bank.Accounts*]* -[Test.BankDemo*]*"
ReportGenerator\bin\ReportGenerator.exe Codecoverage.xml Coverage HTML
(I have even tried regsvr32 to register the profile and I am using XP)
actually I am beginner to Nunit,mstest and opencoverage and I found sample Unit test case at http://www.nunit.org/index.php?p=quickStart&r=2.4 so
** Nunit test class is as below**
private TestContext testContextInstance;
public TestContext TestContext
{
get { return testContextInstance; }
set { testContextInstance = value; }
}
private int store;
[TestInitialize()]
public void TestFixtureSetUp()
{
store = 1;
}
the above class works fine with Nunit and Opencoverage also showing accurate data but same class after replacing mstest specific attributes didn't worked so after posting this questin I figured it that this method has to be static and added TestContext argument. so I made code changes(in bold) as below and above command worked fine.
MSTest class
private TestContext testContextInstance;
public TestContext TestContext
{
get { return testContextInstance; }
set { testContextInstance = value; }
}
[ClassInitialize()]
public **static** void ClassInit(**TestContext context**)
{
}

Your tests aren't failing -- they're erroring, meaning there appears to be a problem compiling the test project. It stands to reason that you'll get no coverage if the tests cannot be built and executed.

2 reasons could be for this however I suspect your filters are wrong, as described in the usage the filters are
(+/-)[assembly/module filter]namespace.typefilter
and exclusion filters take precedence over inclusion filters
So your -[Bank.*]* is excluding types before the +[Bank.Accounts*]* (and probably +[Bank*]*) can take effect. As the default filter +[*]* is only added if you have no other extra filters, other than the default ones, then you should only need to add filters for the modules you wish to profile i.e. +[Bank.*]*
If you open up the XML output then if a class is filtered out then a reason is supplied via the skippedDueTo attribute.
The other reason could be due to missing PDB files not in the folder of the assembly (some test harnesses copy assemblies to other folders - but I see you are using the /noisolation switch - so this shouldn't be it)
Please feel free to discuss or if you think there is a big raise the issue on the OpenCover GitHub site

Related

Test missing when using AutoFixture with NSubstitute auto data attribute

A test class with the following test is discovered as expected:
[Theory]
[AutoData]
public void MyDiscoveredTest() { }
However, the following test is missing:
[Theory]
[AutoNSubstituteData]
public void MyMissingTest() { }
Interestingly, if I put MyDiscoveredTest after MyMissingTest, then MyDiscoveredTest is also now missing. I have tried both the xUnit visual studio runner and xUnit console runner with the same results.
My AutoNSubstituteData attribute is defined here:
internal class AutoNSubstituteDataAttribute : AutoDataAttribute
{
internal AutoNSubstituteDataAttribute()
: base(new Fixture().Customize(new AutoNSubstituteCustomization()))
{
}
}
A related question: since the AutoNSubstituteDataAttribute above seems like a fairly common attribute, I'm wondering why it's not included with AutoFixture.AutoNSubstitute. Similarly useful would be an InlineAutoNSubstituteDataAttribute. Should I submit a pull request for these?
Nuget package versions used:
AutoFixture 3.30.8
AutoFixture.Xunit2 3.30.8
AutoFixture.AutoNSubstitute 3.30.8
xunit 2.0.0
xunit.runner.visualstudio 2.0.0
xunit.runner.console 2.0.0
NSubstitute 1.8.2.0
I am using Visual Studio 2013 Update 4 and targeting the .NET 4.5.1 Framework
Update: As recommended I tried TestDriven.NET-3.9.2897 Beta 2. The missing test now runs, however it still seems there is some bug. New example:
[Theory]
[AutoData]
public void MyWorkingTest(string s)
{
Assert.NotNull(s); // Pass
}
[Theory]
[AutoNSubstituteData]
public void MyBrokenTest(string s)
{
Assert.NotNull(s); // Fail
}
[Theory]
[AutoData]
public void MyWorkingTestThatIsNowBroken(string s)
{
Assert.NotNull(s); // Fail even though identical to MyWorkingTest above!
}
Both MyBrokenTest and MyWorkingTestThatIsNowBroken fail at Assert.NotNull, while MyWorkingTest passes even though it is identical to MyWorkingTestThatIsNowBroken. So not only does the AutoNSubstituteData attribute not work properly, but it is causing the downstream test to misbehave!
Update2: Changing the definition of AutoNSubstituteDataAttribute to public instead of internal fixes everything. xunit runner now discovers and passes all the tests as does TestDriven.Net. Any idea about this behavior? Is it expected?
Both xUnit visual studio runner and TestDriven.Net runner are causing these weird issues because the AutoNSubstituteDataAttribute class and constructor are internal. Changing these to public resolves all the issues. If the attribute is being ignored I would expect an error like this: System.InvalidOperationException : No data found for ...
This doesn't explain why the downstream tests are affected by the offending AutoNSubstituteData attribute from a totally different test. It seems like the unit test runners should be more robust in this case.
For completeness here is the working implementation of AutoNSubstituteDataAttribute:
public class AutoNSubstituteDataAttribute : AutoDataAttribute
{
public AutoNSubstituteDataAttribute()
: base(new Fixture().Customize(new AutoNSubstituteCustomization()))
{
}
}

Cannot suppress certain warning globally

I have VS2013 with Code Analysis turned on. I want to suppress one type of message globally, so I have following code in GlobalSuppressions.cs
[assembly: System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1707:IdentifiersShouldNotContainUnderscores", Scope = "namespace", Target = "MyProject.MyNamespace")]
But my function name contains underscore still got warned/errored by VS.
[TestMethod()]
public void Do_Something_Successfully_Test()
{
}
Any idea why?

NUnit v3 alpha: trying to get parallel tests to work

I'm trying to get parallel tests to work in NUnit v3, however, the tests don't seem to.
Considering the following test class:
namespace NUnitAlpha3Experimental
{
[TestFixture]
[Parallelizable(ParallelScope.Children)]
class DummyTests
{
[Test]
public void MustSuccess()
{
Assert.IsTrue(true);
FileIO.appendToFile("output.txt", Reflexion.GetCurrentMethodName());
}
[Test]
public void MustFail()
{
Thread.Sleep(500);
FileIO.appendToFile("output.txt", Reflexion.GetCurrentMethodName());
Assert.IsFalse(true);
}
}
}
Whenever I run my tests, "MustFail" is always outputted before "MustSuccess". "MustSuccess" should be outputted first if the tests were ran in parallel. Maybe there's something wrong with my attributes. I don't know.
Please help. Thank you.
edit: I added the /workers=8 to my command line:
[...] \NUnit3\nunit-console NUnitAlpha3Experimental.exe /framework:net-4.5 -workers=8
but still, my tests dont seem to run in parallel.
More info here: https://groups.google.com/forum/#!topic/nunit-discuss/_Zcd3EjiJGo
From the author of NUnit, parallel test cases are not yet implemented. https://groups.google.com/forum/#!topic/nunit-discuss/_Zcd3EjiJGo
Parallel testing of fixtures is implemented thought.

Class files are missing in code coverage result

I have a number of classes in my project. When I check Code Coverage result after running the unit test cases, it does not show all the classes. I am not sure what are the criteria on which the code coverage process the class files.
I read somewhere that if you have not created the test cases for a class file in your Test project it will not be covered in the code coverage. But for me it does not seem true as I can see the the class files even though those are not in the test project.
there are couple of ways to exclude files from code coverage, most popular being an attribute:
[ExcludeFromCodeCoverage]
A bit late in the day, but if you have a class that consists entirely of auto-properties, then this class won't be included in the code coverage stats.
Included
private int _seq;
public int InvoiceSequenceNumber
{
get
{
return _seq;
}
set
{
_seq = value;
}
}
Not included
public int InvoiceSequenceNumber { get; set; }

Duplicate the behaviour of a data driven test

Right now, if you have a test that looks like this:
[TestMethod]
[DeploymentItem("DataSource.csv")]
[DataSource(
Microsoft.VisualStudio.TestTools.DataSource.CSV,
"DataSource.csv",
"DataSource#csv",
DataAccessMethod.Sequential)]
public void TestSomething()
{
string data = TestContext.DataRow["ColumnHeader"].ToString();
/*
do something with the data
*/
}
You'll get as many tests runs as you have data values when you execute this test.
What I'd like to do is duplicate this kind of behaviour in code while still having a datasource. For instance: let's say that I want to run this test against multiple deployed versions of a web service (this is a functional test, so nothing is being mocked - ie. it could very well be a codedui test against a web site deployed to multiple hosts).
[TestMethod]
[DeploymentItem("DataSource.csv")]
[DataSource(
Microsoft.VisualStudio.TestTools.DataSource.CSV,
"DataSource.csv",
"DataSource#csv",
DataAccessMethod.Sequential)]
public void TestSomething()
{
var svc = helper.GetService(/* external file - NOT a datasource */);
string data = TestContext.DataRow["ColumnHeader"].ToString();
/*
do something with the data
*/
}
Now, if I have 2 deployment locations listed in the external file, and 2 values in the datasource for the testmethod, I should get 4 tests.
You might be asking why I don't just add the values to the datasource. The data in the external file will be pulled in via the deployment items in the .testsettings for the test run, because they can and will be defined differently for each person running the tests and I don't want to force a rebuild of the test code in order to run the tests, or explode the number of data files for tests. Each test might/should be able to specify which locations it would like to test against (the types are known at compile-time, not the physical locations).
Likewise, creating a test for each deployment location isn't possible because the deployment locations can and will be dynamic in location, and in quantity.
Can anyone point me to some info that might help me solve this problem of mine?
UPDATE! This works for Visual Studio 2010 but does not seem to work on 2012 and 2013.
I had a similar problem where I had a bunch of files I wanted to use as test data in a data driven test. I solved it by generating a CSV file before executing the data driven test. The generation occurs in a static method decorated with the ClassInitialize attribute.
I guess you could basically do something similar and merge your current data source with your "external file" and output a new CSV data source that your data driven test use.
public TestContext TestContext { get; set; }
const string NameColumn = "NAME";
const string BaseResourceName = "MyAssembly.UnitTests.Regression.Source";
[ClassInitialize()]
public static void Initialize(TestContext context)
{
var path = Path.Combine(context.TestDeploymentDir, "TestCases.csv");
using (var writer = new StreamWriter(path, false))
{
// Write column headers
writer.WriteLine(NameColumn);
string[] resourceNames = typeof(RegressionTests).Assembly.GetManifestResourceNames();
foreach (string resourceName in resourceNames)
{
if (resourceName.StartsWith(BaseResourceName))
{
writer.WriteLine(resourceName);
}
}
}
}
[TestMethod]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "|DataDirectory|\\TestCases.csv", "TestCases#csv", DataAccessMethod.Random)]
public void RegressionTest()
{
var resourceName = TestContext.DataRow[NameColumn].ToString();
// Get testdata from resource and perform test.
}

Resources