I recently installed XUnit and the XUnit VS runner (i.e. xunit.runner.visualstudio) on Visual Studio following the documentation. The installation process uses NuGet and on completion I can successfully see all tests in the MS Test Explorer.
However, if I start a new solution and reference all the required XUnit dll's, i.e., I want to use XUnit without using NuGet to install it, the tests never show up in MS Test Explorer (no problems seeing them in the ReSharper test explorer).
I think this has something to do with the XUnit test adapter that Visual Studio uses to find XUnit tests. It appears that NuGet somehow instructs Visual Studio to create the VisualStudioTestExplorerExtensions folder on each build putting the necessary XUnit test adapter dll's (e.g.xunit.runner.visualstudio.testadapter.dll) in that folder. The MS Test Explorer then uses this test adapter to find XUnit test (see SO Question).
When I attempt to use XUnit without NuGet, the VisualStudioTestExplorerExtensions folder never gets created on build. Adding the folder manually with all XUnit dll's in it also doesn't work. It's almost as if i need to instruct Visual Studio to look in that folder, but I can't see any obvious way of doing that.
How do I use/install XUnit without using NuGet? Or, how do I get VS to look in the `VisualStudioTestExplorerExtensions' when looking for tests?
If you want to install a test runner manually, copy all the necessary dlls to:
C:\Program Files (x86)\Microsoft Visual Studio {version}\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions
Replacing {version} with the appropriate number.
The drawback here is that this is machine wide (maybe desirable in your case), and that you can't quite as easily replicate this setup on a build server. You also will probably have to restart VS whenever you make changes to these files.
Related
I've got a Visual Studio Solution File conatining multiple projects. About half of them are Unit Test Projects.
Visual Studio Test Explorer reliably picks up all Test Projects and allows me to run all of them at once.
We also got a CI that should run these Tests. Is there a way to use the Test Explorer functionality from the command line? Currently I have to pass a list of dll's and executables to the vstest.console.exe Tool.
I searched google for solutions but only found solutions that pass dll files to vstest.console.exe. I would expect a possibility to run the tests from a solution also from the command line similar to how dotnet test allows me to do this.
Unfortunately I cannot use dotnet test to reach my goal, as dotnet does not yet support native projects (which are also conatined in my solution).
I have a specflow scenario, which is a scenario outline with two examples. In Test Explorer window of Visual Studio 2013 I can see it as two separate tests (which is ok), but "Open test" command from context menu doesn't work for them, while it does work for simple specflow scenarios (not "outline" scenarios). Is it somehow possible to quickly open code for such outline scenarios from Test Explorer window in Visual Studio 2013? Or does any other way exist to do that?
P.S. I use SpecFlow integration for Visual Studio 2013
The current version of specflow has some issues mapping code lines for scenario outlines. This pull request has a fix for this and was merged (via this PR) in the vNext branch. I have a version with this fix in it installed and I have just checked and when I choose 'Open Test' or double click on a test in the test explorer I get taken to the scenario line in the test explorer.
In order to use it you'll need to build a version of specflow yourself I thin from the 'v2' branch, at least until a beta version is published on nuget.
In order to use the v2 branch you need to checkout the code and build it. As a result of this you should end up with a nuget package in \SpecFlow\Installer\NuGetPackages\bin\. You will probably want to update the version number in the specflow.nuspec so that you get a proper version number. Once that's done you need to put the nuget package in a folder and then tell visual studio the that folder is a nuget source. Then you can install the nuget package (after uninstalling the old one) and you should be good to go. If it doesn't work then it might be because the extension needs to be updated. Searching the machine for the specflow dlls and replacing them with the newly built versions is probably the best way to go, as the ide integration is a separate package now.
Any other problems let me know, I'll do what I can to help.
I've setup my build as below using the build definition. I'm using XUnit and locally my tests are discovered and run. I've tested the glob **\*spec*.dll and it finds all my test dlls and the build log shows that those dlls are in fact built.
However in the build log I get
Run VS Test Runner
No test found. Make sure that installed test
discoverers & executors, platform & framework
version settings are appropriate and try again.
Which seems to suggest it is trying to use the MSTest test runner instead of the XUnit test runner. How do I tell the build for visual studio online to use the XUnit test runner and discoverer?
This might be out of date now, but this is how I have it setup and working - downvote and let me know if it's wrong and I'll delete this. I got it from a blog post/MSDN page, but I can't locate it any more.
First you need to create a TFVC Team Project (doesn't matter if you don't use it again).
Into $/MyTFVC/BuildProcessTemplate/CustomActivities/
Checking the following files from xunit.net:
Now in VS, click the BUILD, Manage Build Controllers... option. Select the "Hosted Build Controller (Hosted)" and click "Properties...".
Enter the path where you checked in the DLL's into the "Version control path to custom assemblies" field:
You should be good to go.
I've got a CI build set up to build my solution and run my unit tests. The solution is VS 2010. My test project targets the 4.0 framework. And the 10.0 version of the Unit Test Framework is properly referenced. If I run the tests locally in VS 2010, I have no issues. However, when I run my CI build on the TFS server (which is TFS 2010), I get an error stating:
File not found: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe
This is the MSTest for VS 2008, and it's not installed on my TFS server. I do have VS 2010 installed on my TFS server including the testing tools but TFS doesn't look for it.
My question: Why would TFS be attempting to use this version of MSTest to run the tests? I can't find this path specified anywhere in any project, solution, or configuration on the server.
The solution and related projects began life as VS 2008 projects and were migrated to VS 2010. These files were pulled into a fresh install of TFS 2010 and a new TFS project. They were not migrated from a 2008 TFS server.
Activity Log
Full Log
Thanks!
Just a follow-up on this because I eventually discovered what was causing my issue and it was not the answer given above. We were referencing a third party assertion utility kind of like "Should" called "SoftwareApproach.TestingExtensions" in this project. This dll was an older one and contained a reference to MSTest 9 (from VS 2008). Because this dll referenced MSTest 9, it was causing this error on the build server, however, the test would run locally. Frankly, not sure why it didn't complain locally as there was no binding redirect and VS 2008 is not installed.
However, updating to a build of the mentioned dll that instead referenced MSTest 10 resolved our issue on the build server.
From looking at the logs, the problem is that MSTest is running here using the wrong version because you have a legacy assembly reference.
Make sure your unit test project and any other projects being executed for test have the correct assembly reference to the new mstest assembly. I am guessing you either missed a reference or added in the new reference and forgot to remove the legacy one.
"I can't find this path specified anywhere in any project, solution, or configuration on the server."
Yeah, you wouldn't see the path anywhere- it will just exec based on what assembly is referenced into the unit test proj.
I use NUnit for Mono/C# on my Mac, but I need to use Microsoft.VisualStudio.TestTools.UnitTesting for unit testing.
How is Microsoft TestTools' UnitTesting different from NUnit? Can I just change the namespace and recompile to get the same result, or do I have to rewrite the test?
Can I copy some of the assemblies to my Mac to run tests for Microsoft.VisualStudio.TestTools.UnitTesting? If so, what files should be copied?
See Migrating from NUnit to MSTest.
As to why you might try convincing your team to migrate the other way, see NUnit vs. MsTest: NUnit wins for Unit Testing.
Ask yourself - why do you need the Ms tools if you already have a working set with NUnit.
Personally I've worked with both and unless working in legacy where the MS tool was in great use, I'd choose NUnit for this toolset due to the larger community support I get in cases of problems.
MSTest can be used without installing Visual Studio. You will need to install Visual Studio Test Agent, which is a free download from Microsoft: http://www.microsoft.com/download/en/details.aspx?id=1334.
I think this approach is better from a licensing perspective than manually copying MSTest.exe and its dependencies onto the build server.
My biggest problem with MSTest is that it requires the install of Visual Studio, even if you are trying to run a lean build server.
NUnit can install and run anywhere.