I have a test with multiple DataRow(s). When executing test, it only runs first test
[TestClass]
public class UnitTest1
{
[DataTestMethod]
[DataRow(1, 2, 3)]
[DataRow(2, 3, 5)]
[DataRow(3, 5, 8)]
public void AdditionTest(int a, int b, int result)
{
Assert.AreEqual(result, a + b);
}
}
Any idea if I am missing anything?
I was having the same problem, but only in the ReSharper Test Sessions window. If I ran the same tests from the Test Explorer window, all of the DataRow tests would run. It had previously worked in ReSharper as well. Before this started I had recently updated to MSTest 2.2.7 and changed the test project to use NuGet package references.
The problem was resolved by updating ReSharper to 2021.2.2.
If you're like me and were attempting to run a DataTestMethod on arrays, I think the issue is that visual studio will not run multiple DataRows with identical values, and "new float[]" is treated as the same value regardless of what its initialised to.
See more here: https://github.com/microsoft/testfx/issues/1016
Related
Using Fact(Skip = "Manual Only") is not entirely satisfactory because, if I click directly on the test to run it, it's still ignored.
I want it to not appear in Test Explorer but I can still run it by clicking on it. Is this possible?
Nice trick from Jimmy Bogard is to use the fact that Skip is writable and react to something in the environment:
public class RunnableInDebugOnlyAttribute : FactAttribute
{
public RunnableInDebugOnlyAttribute()
{
if (!Debugger.IsAttached)
Skip = "Only running in interactive mode.";
}
}
(Aside from that, no xUnit does not have an [Interactive]; the closest thing is `[Trait("Interactive","True")] and use that to either use the trait grouping in test explorer to remove them.
Finally, a 'cheat' way is to use TestDriven.Net, which doesnt care if there is an attribute (along with lots of other facilities).
Need your help in the following scenario:
I have a solution with 2 projects with different unit tests
Those projects generate 2 different dll: *deployment.dll and *database.dll
I have a build on TFS that I want to use to run those tests, I'm using "Test Case Filter" to filter the categories of my tests
(TestCategory=TEST1|TestCategory=TEST2|TestCategory=TEST3|TestCategory=TEST4)
and in "Test Sources Spec" I'm filtering both dll (*deployment.dll;*database.dll)
*.deployment.dll has TEST2, TEST3, TEST4
*.database.dll has TEST1
This doesn't work, tests of *database.dll does not run. Test selected in Visual Studio Test Runner
Could you please help on that? If I make the build with only 1 dll, for example, *.database.dll, TEST1 runs well.
(UPDATE) SCENARIO 1
Test Case Filter: TestCategory=TEST1|TestCategory=TEST1|TestCategory=TEST2|TestCategory=TEST3|TestCategory=TEST4
Test Sources Spec: *database.dll;*deployment.dll
only runs TEST1
(UPDATE) SCENARIO 2
Test Case Filter: TestCategory=TEST1|TestCategory=TEST1|TestCategory=TEST2|TestCategory=TEST3|TestCategory=TEST4
Test Sources Spec: **\*deployment.dll;*database.dll
only runs TEST2,TEST3,TEST4
(UPDATE) Does not find tests in Database.dll
I've tested in TFS 2015.3, XAML build, but couldn't reproduce your issue. I'd like to share my steps here for your reference:
I have a solution with some projects, 2 of them are UnitTest projects (UnitTestProject1, UnitTestProject2).
In UnitTest1 project, I added TestCategory for two test case, check screenshot below:
[TestMethod()]
[TestCategory("Pro")]
public void M1Test()
{
//
}
[TestMethod()]
[TestCategory("Dev")]
public void M2Test()
{
//
}
Similar to Step2, in UnitTest2 project, I added TestCategory for two test case, check screenshot below:
[TestMethod()]
[TestCategory("Pro1")]
public void M3Test()
{
//
}
[TestMethod()]
[TestCategory("Dev1")]
public void M4Test()
{
//
}
Edit "Test Case Filter" and "Test Sources Spec" in build definition like below screenshot and queue a build:
The test result is as expected. Only M1Test and M2Test in UnitTestProject1, M3Test and M4Test in UnitTestProject2 are tested.
Finally, it's solved :)
So, what I have done to solve this problem, was, change Build Process Template.
There are one step in this process, calling: "FindMatchingFiles"
I change this value as the below image shows. (however, from now on, I must use "**\*" in all my filters that use this process template). This operation make sure that I have the assemblies with fullpath destination complete.
If you have different solutions, please post here :)
Special thank you to #Cece - MSFT for all support
I'm implementing tests using SpecFlow that have nothing to do with each other. Is there config option for SpecFlow that enables parallel test execution? I'm using VS10 and MSTest runner that supports running "up to 5 parallel unit tests" as they claim in documentation.
Thanks,
max.yz
I moved away from MSTest to MbUnit to achieve this. You can achieve parallelism at test fixture level with MbUnit using the ParallelizableAttribute. However as the test fixtures are generated from the .feature Gherkin files I had to grab the SpecFlow source code and modify the MbUnitTestGeneratorProvider class in the TechTalk.SpecFlow.Generator project to output the ParallelizableAttribute. So you end up with something like this:
public class MbUnitTestGeneratorProvider : IUnitTestGeneratorProvider
{
private const string TESTFIXTURE_ATTR = "MbUnit.Framework.TestFixtureAttribute";
private const string PARALLELIZABLE_ATTR = "MbUnit.Framework.ParallelizableAttribute";
private const string TEST_ATTR = "MbUnit.Framework.TestAttribute";
private const string ROWTEST_ATTR = "MbUnit.Framework.RowTestAttribute";
private const string ROW_ATTR = "MbUnit.Framework.RowAttribute";
private const string CATEGORY_ATTR = "MbUnit.Framework.CategoryAttribute";
private const string TESTSETUP_ATTR = "MbUnit.Framework.SetUpAttribute";
private const string TESTFIXTURESETUP_ATTR = "MbUnit.Framework.FixtureSetUpAttribute";
private const string TESTFIXTURETEARDOWN_ATTR = "MbUnit.Framework.FixtureTearDownAttribute";
private const string TESTTEARDOWN_ATTR = "MbUnit.Framework.TearDownAttribute";
private const string IGNORE_ATTR = "MbUnit.Framework.IgnoreAttribute";
private const string DESCRIPTION_ATTR = "MbUnit.Framework.DescriptionAttribute";
public bool SupportsRowTests { get { return true; } }
public void SetTestFixture(CodeTypeDeclaration typeDeclaration, string title, string description)
{
typeDeclaration.CustomAttributes.Add(
new CodeAttributeDeclaration(
new CodeTypeReference(TESTFIXTURE_ATTR)));
typeDeclaration.CustomAttributes.Add(
new CodeAttributeDeclaration(
new CodeTypeReference(PARALLELIZABLE_ATTR)));
SetDescription(typeDeclaration.CustomAttributes, title);
}
If you compile this up and use it you'll end up with parallelizable test fixtures:
[System.CodeDom.Compiler.GeneratedCodeAttribute("TechTalk.SpecFlow", "1.6.1.0")]
[System.Runtime.CompilerServices.CompilerGeneratedAttribute()]
[MbUnit.Framework.TestFixtureAttribute()]
[MbUnit.Framework.ParallelizableAttribute()]
[MbUnit.Framework.DescriptionAttribute("Test")]
public partial class TestFeature
{
The only problem with this as it stands is that you will need to make sure that test fixtures do not conflict with one another. That is to say, a test from one fixture adds or modifies a database row that breaks a test that is running at the same time as it. There are ways around this but that is probably out of scope of your original question.
Alex.
There is a new tool called SpecRun that was recently released by the makers of SpecFlow. SpecRun will allow you to run these tests in parallel. If you use the SpecRun.Nunit package with it, you can run NUnit tests in parallel. We use SpecRun on our CI server to run the tests in parallel, but the developers use whatever their test runner of choice.
Changing your test framework can be disruptive. Because all our tests were in NUnit to start, we simply added the new SpecRun runner and nothing else changed. Very simple and transparent to the developers. And since it's available on NuGet, it was very easy to install.
I created a solution that generates a nant build file, which uses nunit in a custom parallel nant task:
https://github.com/MartyIce/SpecflowParallelizer
Due to how my legacy tests were written, I get backend concurrency problems, so it hasn't been succesful for me (yet) but hopefully this will work for someone else.
There is an option in the MSTest .testsettings file for the test project.
By default the test runner will only run 1 test at a time, by changing the parallelTestCount attribute of the Execute node to 0 it will run on as many threads as are avalable (for some reason limited to a max of 5)
Just right click the .teststtings file and pick open with; choose an XML editor and off you go.
You must not run any Coded UI tests or configure any datacollectors for this to work.
For a more detailed explanation see this article
Is there a way to place a watch on variable and only have Visual Studio break when that value changes?
It would make it so much easier to find tricky state issues.
Can this be done?
Breakpoint conditions still need a breakpoint set, and I'd rather set a watch and let Visual Studio set the breakpoints at state changes.
In the Visual Studio 2005 menu:
Debug -> New Breakpoint -> New Data Breakpoint
Enter:
&myVariable
You can also choose to break explicitly in code:
// Assuming C#
if (condition)
{
System.Diagnostics.Debugger.Break();
}
From MSDN:
Debugger.Break:
If no debugger is attached, users are
asked if they want to attach a
debugger. If yes, the debugger is
started. If a debugger is attached,
the debugger is signaled with a user
breakpoint event, and the debugger
suspends execution of the process just
as if a debugger breakpoint had been
hit.
This is only a fallback, though. Setting a conditional breakpoint in Visual Studio, as described in other comments, is a better choice.
In Visual Studio 2015, you can place a breakpoint on the set accessor of an Auto-Implemented Property and the debugger will break when the property is updated
public bool IsUpdated
{
get;
set; //set breakpoint on this line
}
Update
Alternatively; #AbdulRaufMujahid has pointed out in the comments that if the auto implemented property is on a single line, you can position your cursor at the get; or set; and hit F9 and a breakpoint will be placed accordingly. Nice!
public bool IsUpdated { get; set; }
Imagine you have a class called A with the following declaration.
class A
{
public:
A();
private:
int m_value;
};
You want the program to stop when someone modifies the value of "m_value".
Go to the class definition and put a breakpoint in the constructor of A.
A::A()
{
... // set breakpoint here
}
Once we stopped the program:
Debug -> New Breakpoint -> New Data Breakpoint ...
Address: &(this->m_value)
Byte Count: 4 (Because int has 4 bytes)
Now, we can resume the program. The debugger will stop when the value is changed.
You can do the same with inherited classes or compound classes.
class B
{
private:
A m_a;
};
Address: &(this->m_a.m_value)
If you don't know the number of bytes of the variable you want to inspect, you can use the sizeof operator.
For example:
// to know the size of the word processor,
// if you want to inspect a pointer.
int wordTam = sizeof (void* );
If you look at the "Call stack" you can see the function that changed the value of the variable.
Change the variable into a property and add a breakpoint in the set method. Example:
private bool m_Var = false;
protected bool var
{
get {
return m_var;
}
set {
m_var = value;
}
}
Update in 2019:
This is now officially supported in Visual Studio 2019 Preview 2 for .Net Core 3.0 or higher. Of course, you may have to put some thoughts in potential risks of using a Preview version of IDE. I imagine in the near future this will be included in the official Visual Studio.
https://blogs.msdn.microsoft.com/visualstudio/2019/02/12/break-when-value-changes-data-breakpoints-for-net-core-in-visual-studio-2019/
Fortunately, data breakpoints are no longer a C++ exclusive because they are now available for .NET Core (3.0 or higher) in Visual Studio 2019 Preview 2!
If you are using WPF, there is an awesome tool : WPF Inspector.
It attaches itself to a WPF app and display the full tree of controls with the all properties, an it allows you (amongst other things) to break on any property change.
But sadly I didn't find any tool that would allow you to do the same with ANY property or variable.
I remember the way you described it using Visual Basic 6.0. In Visual Studio, the only way I have found so far is by specifying a breakpoint condition.
Right click on the breakpoint works fine for me (though mostly I am using it for conditional breakpoints on specific variable values. Even breaking on expressions involving a thread name works which is very useful if you're trying to spot threading issues).
As Peter Mortensen wrote:
In the Visual Studio 2005 menu:
Debug -> New Breakpoint -> New Data Breakpoint
Enter: &myVariable
Additional information:
Obviously, the system must know which address in memory to watch.
So
- set a normal breakpoint to the initialisation of myVariable (or myClass.m_Variable)
- run the system and wait till it stops at that breakpoint.
- Now the Menu entry is enabled, and you can watch the variable by entering &myVariable,
or the instance by entering &myClass.m_Variable. Now the addresses are well defined.
Sorry when I did things wrong by explaining an already given solution. But I could not add a comment, and there has been some comments regarding this.
You can use a memory watchpoint in unmanaged code. Not sure if these are available in managed code though.
You can probably make a clever use of the DebugBreak() function.
You can optionally overload the = operator for the variable and can put the breakpoint inside the overloaded function on specific condition.
What I would like to do is have VS2008, when I open a code file, collapse all members of the classes/interfaces in the file by default (including, crucially, any XML documentation and comments).
I do not want to use regions, at all.
I would also like to be able to use the ctrl+m, ctrl+l chord to toggle all member outlining (for example, if everything is collapsed, I would like it to expand all of the members, but not the comments or XML documentation).
Possible? How?
Yes to part 1.
Unsure about part 2.
To have VS2008 automatically open files in a Collapsed state you'll need to create an addin to run the "Edit.CollapsetoDefinition" when each document opens.
This isn't overly tricky - The difficult parts seems to be the that you have to run the code a few milliseconds after the document is actually opened so you need to use the threed pool to do that.
Create an Addin project for VS2008.
Add this code (see following) to the end of the OnConnection Method of the Connect class.
switch (connectMode)
{
case ext_ConnectMode.ext_cm_UISetup:
case ext_ConnectMode.ext_cm_Startup:
//Do nothing OnStartup will be called once IDE is initialised.
break;
case ext_ConnectMode.ext_cm_AfterStartup:
//The addin was started post startup so we need to call its initialisation manually
InitialiseHandlers();
break;
}
Add this method to the Connect class
private void InitialiseHandlers()
{
this._openHandler = new OnOpenHandler(_applicationObject);
}
Add a call to InitialiseHandlers() to the OnStartupComplete method of the Connect class.
public void OnStartupComplete(ref Array custom)
{
InitialiseHandlers();
}
Add this class to the project.
using System;
using System.Collections.Generic;
using System.Text;
using EnvDTE80;
using EnvDTE;
using System.Threading;
namespace Collapser
{
internal class OnOpenHandler
{
DTE2 _application = null;
EnvDTE.Events events = null;
EnvDTE.DocumentEvents docEvents = null;
internal OnOpenHandler(DTE2 application)
{
_application = application;
events = _application.Events;
docEvents = events.get_DocumentEvents(null);
docEvents.DocumentOpened +=new _dispDocumentEvents_DocumentOpenedEventHandler(OnOpenHandler_DocumentOpened);
}
void OnOpenHandler_DocumentOpened(EnvDTE.Document document)
{
if (_application.Debugger.CurrentMode != dbgDebugMode.dbgBreakMode)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(Collapse));
}
}
private void Collapse(object o)
{
System.Threading.Thread.Sleep(150);
_application.ExecuteCommand("Edit.CollapsetoDefinitions", "");
}
}
}
And now all opened files should be fully collapsed.
It would be much easier to use the Visual Studio Macros to do the same thing. Editing the "EnvironmentEvents" Macro file in MyMacros and adding a handler for DocumentEvents.DocumentOpened with :
DTE.ExecuteCommand("Edit.CollapsetoDefinitions")
A quick way to collapse all outlining to function-definitions is to press:
Contextmenu-button*(next to your right windows button)*, L, O
I use it all the time. If there is a real hotkey for this please tell me :)
I had tried working out some Visual Basic code for a macro myself, borrowing from different places, and couldn't get anything to work. So what did I do? Why, I asked a question on StackOverflow of course! It got answered, I added the suggested code to my EnvironmentEvents macro, and now when I open CS files, after about a second, all my definitions are collapsed. :)