I am seeing some very strange behavior for NUnit tests in VS2019, where the same solution works fine in VS2017. There are several NUnit test projects in my soultion.
In VS2017 with the NUnit Runner extension installed, I can see all of my tests in the Test Explorer window and the "run all" button will work and run all tests. Some developers in my organization use Resharper instead of the NUnit extension and this works too.
I have stopped using Resharper because as VS introduces more features, Resharper has made it so slow that VS is unusable.
In VS2019 the Test Explorer window will show all of my unit tests (even without the NUnit extension installed). If I click "run all" it will not run any tests and the Output window will say 0 tests discovered. Coworkers have said that Resharper will run all tests without issue. If I right click a single test project and run just those tests, some projects will run tests, but not all.
For some projects I have tried installing the NUnit3TestAdapater nuget package and this will let VS2019 run that project's test if only that project is selected. This does not work for all projects and it still does not work for "run all".
Does anyone know what might be causing this and what could fix it? I have updated to the latest version of NUnit (3.12) and latest TestAdapter (3.16) for all of these projects.
This has been a real pain for about a year, because I need to keep VS2017 and 2019 both installed, and I need to train new developers on how to work around this weird issue.
I stumbled upon the same problem. It seems to me that I have to install more and more with each release.
In my case (Visual Studio 2019 community edition, version 16.6.1), I now also had to obtain the Microsoft.NET.Test.Sdk via NuGet.
So in the end, I have installed three packages in my test project:
NUnit (3.12.0)
NUnit3TestAdapter (3.17.0)
Microsoft.NET.Test.Sdk (16.7.1)
Turning the automated test discovery on or off did not change anything for me though.
Hope this helps anyone.
The solution turned out to be a combination of two things.
On the top menu, navigating to Test > Options and disabling "Discover tests in real time from C# and Visual Basic .NET source files". This option appears to be incompatible with parameterized tests. The problem is that parameterized tests do not "exist" in source code but are generated by the test adapter at runtime. This also fixed an issue I've seen where the "base" test of the parameterized test shows up in the Test Explorer as a not-run test, even though only the specific test cases are real tests. (In my opinion, this should not be enabled by default because parameterized tests are extremely useful, whereas seeing new tests in the Test Explorer without compiling is a trivial convenience since you must compile to run them anyway.)
Using the NuGet package for the test adapter instead of using the VSIX extension. It seems that for this piece, all that is required is that at least one project in your solution references this. If at least one project references it, all test projects can be run. (This makes sense to me, as it is more compatible with build tools outside of Visual Studio.)
Hooray for breaking changes!
I had the same issue and solved by checking the logging.
In Tools->Options->Test->General setting the Logging Level to Diagnostic. This will produce additional output in the Tests Output Pane of the output window.
In my case the issue was linked to missing version of ".NET Core 3.1 Desktop Runtime"
You can download the last version in https://dotnet.microsoft.com/download/dotnet/thank-you/runtime-desktop-3.1.14-windows-x86-installer
In my case I had to install also the Microsoft.NET.Test.Sdk nuget pack.
After none of the above worked for me.
I just installed xunit.runner.visualstudio nuget and everything is working fine now.
Related
I have a specflow scenario, which is a scenario outline with two examples. In Test Explorer window of Visual Studio 2013 I can see it as two separate tests (which is ok), but "Open test" command from context menu doesn't work for them, while it does work for simple specflow scenarios (not "outline" scenarios). Is it somehow possible to quickly open code for such outline scenarios from Test Explorer window in Visual Studio 2013? Or does any other way exist to do that?
P.S. I use SpecFlow integration for Visual Studio 2013
The current version of specflow has some issues mapping code lines for scenario outlines. This pull request has a fix for this and was merged (via this PR) in the vNext branch. I have a version with this fix in it installed and I have just checked and when I choose 'Open Test' or double click on a test in the test explorer I get taken to the scenario line in the test explorer.
In order to use it you'll need to build a version of specflow yourself I thin from the 'v2' branch, at least until a beta version is published on nuget.
In order to use the v2 branch you need to checkout the code and build it. As a result of this you should end up with a nuget package in \SpecFlow\Installer\NuGetPackages\bin\. You will probably want to update the version number in the specflow.nuspec so that you get a proper version number. Once that's done you need to put the nuget package in a folder and then tell visual studio the that folder is a nuget source. Then you can install the nuget package (after uninstalling the old one) and you should be good to go. If it doesn't work then it might be because the extension needs to be updated. Searching the machine for the specflow dlls and replacing them with the newly built versions is probably the best way to go, as the ide integration is a separate package now.
Any other problems let me know, I'll do what I can to help.
I'm running Visual Studio 2013 (with latest updates) with ReSharper 8.1 (latest stable version). I'm trying to run my tests as usual, but they keep acting as if they're ignored. Both Run Unit Tests (ReSharper) and Run Tests (built-in test runner) act as if the tests are ignored.
I've tried: restarting Visual Studio, uninstalling and reinstalling ReSharper, clearing ReSharper caches, rebooting my Windows 7 machine, removing all [Ignore] annotations throughout the project... nothing works.
Even a very simple test like:
[TestMethod]
public void CrapTest()
{
Assert.IsFalse(false);
}
Will immediately go to an ignored state.
I've searched online but can't find any solutions. Help! This is driving me nuts.
Deleting this file fixed it:
Solution Items / TestSettings.testsettings
One of our devs was trying to set up distributed testing for our TFS / CI test runs using this file.
I know this is an old post, but identical symptoms can come from a different root cause. If a test case is marked Explicit then Resharper will ignore the test. In this case this is not an error, but rather works as designed.
<Test(), Explicit()>
Will cause tests to be ignored.
I wonder has anyone come across with this issue where the MSTest Unit Test doesn’t show up in the new Unit Test Explorer.
I’m running Windows 7 (32bit). I downloaded the VS11 Developer Preview from the link below. http://www.microsoft.com/download/en/details.aspx?id=27543
I created a sample C# Console App, and add the Test Library from the MSTest project template. Then I created a sample Unit Test, and rebuild the solution. When I open the Test Explorer (View->OtherWindows->UnitTest Explorer) I do not see any tests loaded.
I only see a message saying…
“No test discovered. Please build your project and ensure the appropriate test framework adapter is installed”.
I assume the MSTest adapter is automatically installed. If not I’m not even sure how to install an adapter.
I could be missing something here but I cannot figure it out. Has anyone experiencing this issue?
Found out that at this only works with Windows8. (At this stage)
More info
http://jerssoft.blogspot.com/2011/09/vs11-unit-test-explorer-acercamientos.html#more
(you need to translate to English)
"Of course, all this work the right way if we have properly configured and in the final version , since according to understand, if you want to try this plugin, you need to be working with Windows 8 :) "
http://geekswithblogs.net/lbugnion/archive/2011/09/24/running-unit-tests-in-visual-studio-2011-and-windows-8.aspx
If you are using VS Ultimate version of VS 11 developer preview, then it support MStest based unittesting for Windows7 style apps/projects if that is what you were trying out. It is possible that you are hitting a bug. Will it possible for you to share your repro steps?
Thanks,
Abhishek Agrawal
Program Manager, Visual Studio,
Microsoft Corp
This wasn't working for me using Visual Studio Express 2012 Web RC, but I got it going eventually.
I added the reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework which I found in C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PublicAssemblies
I made sure this was set to Copy Local but probably not required.
Then I tried cleaning and rebuilding but with no effect. I made sure I didn't have any testsettings selected (Test Menu > Test Settings > Untick the testrunconfig if selected), then changed my Test > Test Settings > Default Processor Architecture to x64 (my machine is x64).
Then I cleaned and rebuilt the solution and my Test Explorer was populated when I click Run All.
I use NUnit for Mono/C# on my Mac, but I need to use Microsoft.VisualStudio.TestTools.UnitTesting for unit testing.
How is Microsoft TestTools' UnitTesting different from NUnit? Can I just change the namespace and recompile to get the same result, or do I have to rewrite the test?
Can I copy some of the assemblies to my Mac to run tests for Microsoft.VisualStudio.TestTools.UnitTesting? If so, what files should be copied?
See Migrating from NUnit to MSTest.
As to why you might try convincing your team to migrate the other way, see NUnit vs. MsTest: NUnit wins for Unit Testing.
Ask yourself - why do you need the Ms tools if you already have a working set with NUnit.
Personally I've worked with both and unless working in legacy where the MS tool was in great use, I'd choose NUnit for this toolset due to the larger community support I get in cases of problems.
MSTest can be used without installing Visual Studio. You will need to install Visual Studio Test Agent, which is a free download from Microsoft: http://www.microsoft.com/download/en/details.aspx?id=1334.
I think this approach is better from a licensing perspective than manually copying MSTest.exe and its dependencies onto the build server.
My biggest problem with MSTest is that it requires the install of Visual Studio, even if you are trying to run a lean build server.
NUnit can install and run anywhere.
I realise that there are many older questions addressing the general question of NUnit v MSTest for versions of Visual Studio up to 2008 (such as this one).
Microsoft have a history of getting things right in their 3rd version. For MSTest, that is VS2010.
Have they done so with MSTest? Would you use it in a new project in preference to NUnit?
My specific concerns:
speed
running tests within CruiseControl.NET (either commandline or MSBuild task)
code coverage reports from CC.NET
can you run MSTest tests in debug mode
(We use ReSharper, so test-runners are not an issue for us. We have used NUnit for the last few years. We do not have TFS.)
List item speed is same, but MsTest may be a bit slower because it creates folder for test run every time
MSBuid and CC.Net is big pain. You can't run MSTest on computer without VS on it (not 100 sure about 2010, but with 2008 it is so)
not sure, sorry
yes you can, from visual studio
My recommendation is following: if NUnit satisfies you - use it, forget about MSTest
To correct some old information on the thread;
It IS possible to run 64 bit tests in 2010
From VS2008 forward it is not neccesary to have MSTEST create directories anc opy the binaries in, just disable deployment, in 2010 thats the default but you have to set it in 2008
2010 MSTEST is faster but as its a generalised test framework that also runs load/web/UI tests there are compromises in the design that will lead to it being slower. Jamie Cansdale appears to have managed to get perf increases with the lastest releases of TestDriven.net's support for MSTEST
I've mainly used NUnit, some xUnit and some MSTest. They seem functionality equivalent, but I don't like the MSTest test runner. It runs in visual studio so it either crowds the screen or is on another monitor getting in the way everytime I tab to visual studio. (I run NUnit on another monitor, but it doesn't cover everything on that monitor everytime I focus visual studio). It takes too many clicks to find out what test failed and why.
NUnit can run in the background until a test fails, at which point it shows you information about the breaking test. This seems like the ideal for keeping red/green/refactor going smoothly.
Nope. Same issues regarding appdomains and assembly resolving still exist. I would avoid unless you want the new goodness for other functional testing or integration with Team System.
I don't know much about CruseControl.net, but you can debug tests. We currently don't use TFS either, and the MSTest is working for us.
If you think you'll ever run your tests in 64 bit mode, use NUnit. MsTest is only x86.
One major difference between the two is that MSTest makes a copy of the current DLLs every time it runs a test. If you're doing TDD and running your tests frequently, this can eat up a lot of hard drive space.
If you're using MSTest, you can change this setting in Tools > Options > Test Tools > Test Execution. "Limit number of old Test Results to" is set to 25 by default in Visual Studio 2010. I usually change it to 1.
MSUnit runs your test cases under conditions that are different from the actual execution environment. Specifically, the deployed files differ from those that are deployed when you run your actual project. Nethertheless, there is the [DeploymentItem]-Attribute to specify which files shall be deployed by MSUnit. So if your application depends on any external
files, such as
database files
database configuration file
application configuration file
...
then MSUnit is not the right choice, because the MSUnit tests never cover what your file system is going to look like in execution environment. The Visual Studio Project File settings for deploying files (Copy always, Content, etc.) are ignored by the MSUnit runner. So those settings cannot be tested.