I'm trying to get parallel tests to work in NUnit v3, however, the tests don't seem to.
Considering the following test class:
namespace NUnitAlpha3Experimental
{
[TestFixture]
[Parallelizable(ParallelScope.Children)]
class DummyTests
{
[Test]
public void MustSuccess()
{
Assert.IsTrue(true);
FileIO.appendToFile("output.txt", Reflexion.GetCurrentMethodName());
}
[Test]
public void MustFail()
{
Thread.Sleep(500);
FileIO.appendToFile("output.txt", Reflexion.GetCurrentMethodName());
Assert.IsFalse(true);
}
}
}
Whenever I run my tests, "MustFail" is always outputted before "MustSuccess". "MustSuccess" should be outputted first if the tests were ran in parallel. Maybe there's something wrong with my attributes. I don't know.
Please help. Thank you.
edit: I added the /workers=8 to my command line:
[...] \NUnit3\nunit-console NUnitAlpha3Experimental.exe /framework:net-4.5 -workers=8
but still, my tests dont seem to run in parallel.
More info here: https://groups.google.com/forum/#!topic/nunit-discuss/_Zcd3EjiJGo
From the author of NUnit, parallel test cases are not yet implemented. https://groups.google.com/forum/#!topic/nunit-discuss/_Zcd3EjiJGo
Parallel testing of fixtures is implemented thought.
Related
I need to run my xUnit test cases through command line. I have some test cases as below:
[Theory]
[InlineData(2)]
[InlineData(4)]
[InlineData(6)]
public void GivenNumberMustBeAnEvenNumber(int val)
{
Assert.Equal(val%2, 0);
}
How can I run above test case with only 4 as the inline data? I passed -method "MyClass.GivenNumberMustBeAnEvenNumber(4)" to xunit.console but it didn't run. I even tried with -method "MyClass.GivenNumberMustBeAnEvenNumber(val: 4)" but no success.
I also have some test cases that take strings in their parameters as below:
[Theory]
[InlineData("abc")]
[InlineData("xyz")]
public void GivenStringLengthIsAlwaysThree(string val)
{
Assert.Equal(val.Length, 3);
}
How can I run above test case with "xyz" as inline data only.
Please help.
A test class with the following test is discovered as expected:
[Theory]
[AutoData]
public void MyDiscoveredTest() { }
However, the following test is missing:
[Theory]
[AutoNSubstituteData]
public void MyMissingTest() { }
Interestingly, if I put MyDiscoveredTest after MyMissingTest, then MyDiscoveredTest is also now missing. I have tried both the xUnit visual studio runner and xUnit console runner with the same results.
My AutoNSubstituteData attribute is defined here:
internal class AutoNSubstituteDataAttribute : AutoDataAttribute
{
internal AutoNSubstituteDataAttribute()
: base(new Fixture().Customize(new AutoNSubstituteCustomization()))
{
}
}
A related question: since the AutoNSubstituteDataAttribute above seems like a fairly common attribute, I'm wondering why it's not included with AutoFixture.AutoNSubstitute. Similarly useful would be an InlineAutoNSubstituteDataAttribute. Should I submit a pull request for these?
Nuget package versions used:
AutoFixture 3.30.8
AutoFixture.Xunit2 3.30.8
AutoFixture.AutoNSubstitute 3.30.8
xunit 2.0.0
xunit.runner.visualstudio 2.0.0
xunit.runner.console 2.0.0
NSubstitute 1.8.2.0
I am using Visual Studio 2013 Update 4 and targeting the .NET 4.5.1 Framework
Update: As recommended I tried TestDriven.NET-3.9.2897 Beta 2. The missing test now runs, however it still seems there is some bug. New example:
[Theory]
[AutoData]
public void MyWorkingTest(string s)
{
Assert.NotNull(s); // Pass
}
[Theory]
[AutoNSubstituteData]
public void MyBrokenTest(string s)
{
Assert.NotNull(s); // Fail
}
[Theory]
[AutoData]
public void MyWorkingTestThatIsNowBroken(string s)
{
Assert.NotNull(s); // Fail even though identical to MyWorkingTest above!
}
Both MyBrokenTest and MyWorkingTestThatIsNowBroken fail at Assert.NotNull, while MyWorkingTest passes even though it is identical to MyWorkingTestThatIsNowBroken. So not only does the AutoNSubstituteData attribute not work properly, but it is causing the downstream test to misbehave!
Update2: Changing the definition of AutoNSubstituteDataAttribute to public instead of internal fixes everything. xunit runner now discovers and passes all the tests as does TestDriven.Net. Any idea about this behavior? Is it expected?
Both xUnit visual studio runner and TestDriven.Net runner are causing these weird issues because the AutoNSubstituteDataAttribute class and constructor are internal. Changing these to public resolves all the issues. If the attribute is being ignored I would expect an error like this: System.InvalidOperationException : No data found for ...
This doesn't explain why the downstream tests are affected by the offending AutoNSubstituteData attribute from a totally different test. It seems like the unit test runners should be more robust in this case.
For completeness here is the working implementation of AutoNSubstituteDataAttribute:
public class AutoNSubstituteDataAttribute : AutoDataAttribute
{
public AutoNSubstituteDataAttribute()
: base(new Fixture().Customize(new AutoNSubstituteCustomization()))
{
}
}
I'm writing a gradle plugin that defines an upToDateWhen closure to skip the task when certain criteria is met. I'm having trouble figuring out how to wrap a test around this method. Currently it looks like:
class MyCoolTask extends DefaultTask {
MyCoolTask() {
outputs.upToDateWhen {
if (somecondition)
return true
else
return false
}
}
}
My test looks like this:
class MyCoolTaskTest {
#Test
void testUpToDateCheck() {
project = ProjectBuilder.builder().build()
project.apply plugin: 'myCoolPlugin'
project.myCoolTask.execute()
// But then how do you do a subsequent run and ensure that the task did not execute?
project.myCoolTask.execute() // running this a second time does not work.
project.myCoolTask.outputs.upToDateWhen() // Throws a syntax error
}
}
Any insight that could be offered would be great! Thanks!
ProjectBuilder is meant for low-level tests that configure the build but don't execute any tasks. You can either factor out the contents of outputs.upToDateWhen { ... } into a method/class and test that, and/or write an acceptance test that executes a real build using the Gradle tooling API.
I have a number of classes in my project. When I check Code Coverage result after running the unit test cases, it does not show all the classes. I am not sure what are the criteria on which the code coverage process the class files.
I read somewhere that if you have not created the test cases for a class file in your Test project it will not be covered in the code coverage. But for me it does not seem true as I can see the the class files even though those are not in the test project.
there are couple of ways to exclude files from code coverage, most popular being an attribute:
[ExcludeFromCodeCoverage]
A bit late in the day, but if you have a class that consists entirely of auto-properties, then this class won't be included in the code coverage stats.
Included
private int _seq;
public int InvoiceSequenceNumber
{
get
{
return _seq;
}
set
{
_seq = value;
}
}
Not included
public int InvoiceSequenceNumber { get; set; }
'here is output...'
Loading C:\TEMP\BankDemo_mstest\Test_BankDemo\bin\Debug\Test_BankDemo.dll...
Starting execution...
Results Top Level Tests
------- ---------------
Error Test.BankDemo.AccountTest.CreditTest
Error Test.BankDemo.AccountTest.DebitTest
Error Test.BankDemo.AccountTest.FreezeTest
0/3 test(s) Passed, 3 Error
Summary
-------
Test Run Error.
Error 3
--------
Total 3
This is the command I used
OpenCover\OpenCover.Console.exe -register:user
-output:"Codecoverage.xml"
-mergebyhash
-target:"C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe"
targetargs:"/testcontainer:
"C:\TEMP\BankDemo_mstest\Test_BankDemo\bin\Debug\Test_BankDemo.dll"
/noisolation"
-filter:"-[Bank.*]* +[Bank*]* +[Bank.Accounts*]* -[Test.BankDemo*]*"
ReportGenerator\bin\ReportGenerator.exe Codecoverage.xml Coverage HTML
(I have even tried regsvr32 to register the profile and I am using XP)
actually I am beginner to Nunit,mstest and opencoverage and I found sample Unit test case at http://www.nunit.org/index.php?p=quickStart&r=2.4 so
** Nunit test class is as below**
private TestContext testContextInstance;
public TestContext TestContext
{
get { return testContextInstance; }
set { testContextInstance = value; }
}
private int store;
[TestInitialize()]
public void TestFixtureSetUp()
{
store = 1;
}
the above class works fine with Nunit and Opencoverage also showing accurate data but same class after replacing mstest specific attributes didn't worked so after posting this questin I figured it that this method has to be static and added TestContext argument. so I made code changes(in bold) as below and above command worked fine.
MSTest class
private TestContext testContextInstance;
public TestContext TestContext
{
get { return testContextInstance; }
set { testContextInstance = value; }
}
[ClassInitialize()]
public **static** void ClassInit(**TestContext context**)
{
}
Your tests aren't failing -- they're erroring, meaning there appears to be a problem compiling the test project. It stands to reason that you'll get no coverage if the tests cannot be built and executed.
2 reasons could be for this however I suspect your filters are wrong, as described in the usage the filters are
(+/-)[assembly/module filter]namespace.typefilter
and exclusion filters take precedence over inclusion filters
So your -[Bank.*]* is excluding types before the +[Bank.Accounts*]* (and probably +[Bank*]*) can take effect. As the default filter +[*]* is only added if you have no other extra filters, other than the default ones, then you should only need to add filters for the modules you wish to profile i.e. +[Bank.*]*
If you open up the XML output then if a class is filtered out then a reason is supplied via the skippedDueTo attribute.
The other reason could be due to missing PDB files not in the folder of the assembly (some test harnesses copy assemblies to other folders - but I see you are using the /noisolation switch - so this shouldn't be it)
Please feel free to discuss or if you think there is a big raise the issue on the OpenCover GitHub site