I’m having a "build partially succeeded" issue with this error “NUnit failed to load e:\Builds\184\CSTax\706USServices_Test\bin\Calc.Tests.dll”, but all of the unit test in this project ran and passed (all green).
Build environment
1) Using NUnit 2.6.4 and NUnit Test Adapter 2.0.0
2) TFS 2013 – upgraded from 2012
3) Visual studio 2012 and 2013 have been installed on the server
4) I did install the NUnit Test Adapter, using the Extensions and Updates in 2013 Visual Studio
5) Using build process templates from TFS 2012, even if I switch to the 2013 default build process templates I get the same error
6) I tried changing the project to copy the nunit references locally (including the test adapter), they did make it to the bin folder, but still fails
7) I used process monitor to see if I could find the failure, everything looked good, the dlls were copied from the packages folder to the bin folder and loaded.
I’m sure it has to be something in our build environment, but I do not know where to look, I need my builds to go green. Any ideas?
The issue is a bug in the NUnit 3.0 test adapter, I removed our E2E project off of the build server and all my builds went green. NUnit is looking into this bug.
You need to:
Download the NUnit test adapter: https://visualstudiogallery.msdn.microsoft.com/6ab922d0-21c0-4f06-ab5f-4ecd1fe7175d . Rename the NUnitVisualStudioTestAdapter-2.0.0.vsix to NUnitVisualStudioTestAdapter-2.0.0.zip. Unzip NUnitVisualStudioTestAdapter-2.0.0.zip.
Check in the following dlls: nunit.core.dll, nunit.core.interfaces.dll, nunit.util.dll and NUnit.VisualStudio.TestAdapter.dll into a folder in TFS Version Control.
Specify the build controller's path to custom assemblies:
Then queue a build, the NUnit test methods will be run.
Also have a check on the Enable a third-party unit framework part of this MSDN article for the details: https://msdn.microsoft.com/en-us/library/vstudio/ms253138.aspx
Related
I have created a continuous integration in dev.azure pipeline. I have created a build>>release>> added Agent as VSTS Test Platform Installer >> added Visual Studio Test runner which will use vstest.console.exe to execute webtests.
I have a solution>project>webtest scripts and supported files which i ran locally successfully in Visual Studio 2017. Now i am trying to move these web tests to pipeline such that i can run them as CI.
I have added everything as below in dev azure pipeline:
Added Visual Studio Test Platform Installer. vsTestVersion: 15.0
Configured assembly test files as ***test*.dll
***.WebTests
***.testsettings.
Search folder as: $(System.DefaultWorkingDirectory)/_Performance/drop/
Version as: Visual Studio 2017
Settings file as: $(System.DefaultWorkingDirectory)/_Performance/drop/Local.testsettings
Build platform: AnyCPU and rest settings as runInParallel: false
codeCoverageEnabled: true
testRunTitle: 'LoadTest_$(rev:r)'
platform: AnyCPU
configuration: Release
continueOnError: true
Now when i am saving my release and running it. It runs the whole solution and copy all files successfully in the virtual drive under D:\a\r1\a_Performance\drop
But as a next step once the test runs after VSTS Installer it shows.......................................
............
No test is available in D:\a\r1\a_Performance\drop\WebAndLoadTestProject1\bin\Release\Microsoft.VisualStudio.QualityTools.WebTestFramework.dll D:\a\r1\a_Performance\drop\WebAndLoadTestProject1\bin\Release\WebAndLoadTestProject1.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
2019-03-04T20:37:35.7477518Z
enter image description here
I figured out a quick solution. Convert the .webtest file to a code test file and checkin that code in git. Run the same solution now in azure DevOps release. It is running the same test successfully in the assigned vsts2017 VM. So as per my understanding it seems Visual Studio webtests cannot run directly in pipeline release.
To convert the webtests into code open the vsts webtest on the toolbar and there is a Generate Code button in bottle shape. Clicking this button prompts for a test name, and then generates code of the same webtest. Both the webtest and code are separate from each other. Changes in one will not added into other.
I recently installed XUnit and the XUnit VS runner (i.e. xunit.runner.visualstudio) on Visual Studio following the documentation. The installation process uses NuGet and on completion I can successfully see all tests in the MS Test Explorer.
However, if I start a new solution and reference all the required XUnit dll's, i.e., I want to use XUnit without using NuGet to install it, the tests never show up in MS Test Explorer (no problems seeing them in the ReSharper test explorer).
I think this has something to do with the XUnit test adapter that Visual Studio uses to find XUnit tests. It appears that NuGet somehow instructs Visual Studio to create the VisualStudioTestExplorerExtensions folder on each build putting the necessary XUnit test adapter dll's (e.g.xunit.runner.visualstudio.testadapter.dll) in that folder. The MS Test Explorer then uses this test adapter to find XUnit test (see SO Question).
When I attempt to use XUnit without NuGet, the VisualStudioTestExplorerExtensions folder never gets created on build. Adding the folder manually with all XUnit dll's in it also doesn't work. It's almost as if i need to instruct Visual Studio to look in that folder, but I can't see any obvious way of doing that.
How do I use/install XUnit without using NuGet? Or, how do I get VS to look in the `VisualStudioTestExplorerExtensions' when looking for tests?
If you want to install a test runner manually, copy all the necessary dlls to:
C:\Program Files (x86)\Microsoft Visual Studio {version}\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions
Replacing {version} with the appropriate number.
The drawback here is that this is machine wide (maybe desirable in your case), and that you can't quite as easily replicate this setup on a build server. You also will probably have to restart VS whenever you make changes to these files.
I've setup my build as below using the build definition. I'm using XUnit and locally my tests are discovered and run. I've tested the glob **\*spec*.dll and it finds all my test dlls and the build log shows that those dlls are in fact built.
However in the build log I get
Run VS Test Runner
No test found. Make sure that installed test
discoverers & executors, platform & framework
version settings are appropriate and try again.
Which seems to suggest it is trying to use the MSTest test runner instead of the XUnit test runner. How do I tell the build for visual studio online to use the XUnit test runner and discoverer?
This might be out of date now, but this is how I have it setup and working - downvote and let me know if it's wrong and I'll delete this. I got it from a blog post/MSDN page, but I can't locate it any more.
First you need to create a TFVC Team Project (doesn't matter if you don't use it again).
Into $/MyTFVC/BuildProcessTemplate/CustomActivities/
Checking the following files from xunit.net:
Now in VS, click the BUILD, Manage Build Controllers... option. Select the "Hosted Build Controller (Hosted)" and click "Properties...".
Enter the path where you checked in the DLL's into the "Version control path to custom assemblies" field:
You should be good to go.
I can see both test projects from the NUnit Gui (loaded separately) but I can not see both projects from the Visual Studio Test View. The Test View only shows the first/original project. Hitting reload doesn't do anything. How do I get the Test View to display the second/later project? It should show both at the same time/same view, yes?
If I can only see one project's test in the Test View by design, how do I get the Test View to change projects?
Details:
I created the second project by hand ie not "create a test" as a project library with the nunit.framework.dll referenced. I checked the assembly files of the two projects as well as the properties and they look the same.
I'm on VS 2010 Professional using NUnit 2.6.0.12051 with "Visual NUnit 2010" version 1.2.4 extension installed.
I have one NUnit test project that is working both in VS Test View and NUnit Gui. I added a second project with a [TestFixture] class and a [Test] method. All projects build sucessfully. Both tests refer to the nunit.framework.dll in a parent directory to both.
There is a tag in the project file describing a project as a test project.
In the PropertyGroup section of the csprj file, add this:
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
See more details here:
http://www.kindblad.com/2010/08/07/how-to-change-an-existing-visual-studio-project-into-a-test-project
There are many problems that can cause this. Here's what mine was:
I had 2 NUnit test projects.
"Test Explorer" -> "Run All" would only find tests from one project.
Explicitly right clicking on one of the tests that wasn't found and clicking "Run Test" resulted in "No tests found." in the Output window.
All projects on all configurations were set to Any CPU, but it didn't matter if X86 was chosen for everything either.
The problem was that my Visual Studio (2015, FWIW) was using Test Adapter 2.x, and one of my projects had pulled NUnit 3.0 using NuGet. Test Adapter 2.x doesn't run NUnit 3 tests, so my NUnit 3 test project was being ignored.
To solve the problem:
Uninstall all NUnit Test Adapters (from Visual Studio -> Tools -> Extensions and Updates and from each project's NuGet Package Manager window).
Update all test projects to NUnit 3.0 using each project's NuGet Package Manager Window.
Install the NUnit3 Test Adapter from Visual Studio -> Tools -> Extensions and Updates
Notes:
If you need to stick to NUnit 2, then just install the appropriate Test Adapter. The important thing is consistency.
If you were using test adapters installed by NuGet before, but switched to a Visual Studio Extension, then you may have problems loading resources/files from path strings. See this answer to solve that issue.
I've got a CI build set up to build my solution and run my unit tests. The solution is VS 2010. My test project targets the 4.0 framework. And the 10.0 version of the Unit Test Framework is properly referenced. If I run the tests locally in VS 2010, I have no issues. However, when I run my CI build on the TFS server (which is TFS 2010), I get an error stating:
File not found: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe
This is the MSTest for VS 2008, and it's not installed on my TFS server. I do have VS 2010 installed on my TFS server including the testing tools but TFS doesn't look for it.
My question: Why would TFS be attempting to use this version of MSTest to run the tests? I can't find this path specified anywhere in any project, solution, or configuration on the server.
The solution and related projects began life as VS 2008 projects and were migrated to VS 2010. These files were pulled into a fresh install of TFS 2010 and a new TFS project. They were not migrated from a 2008 TFS server.
Activity Log
Full Log
Thanks!
Just a follow-up on this because I eventually discovered what was causing my issue and it was not the answer given above. We were referencing a third party assertion utility kind of like "Should" called "SoftwareApproach.TestingExtensions" in this project. This dll was an older one and contained a reference to MSTest 9 (from VS 2008). Because this dll referenced MSTest 9, it was causing this error on the build server, however, the test would run locally. Frankly, not sure why it didn't complain locally as there was no binding redirect and VS 2008 is not installed.
However, updating to a build of the mentioned dll that instead referenced MSTest 10 resolved our issue on the build server.
From looking at the logs, the problem is that MSTest is running here using the wrong version because you have a legacy assembly reference.
Make sure your unit test project and any other projects being executed for test have the correct assembly reference to the new mstest assembly. I am guessing you either missed a reference or added in the new reference and forgot to remove the legacy one.
"I can't find this path specified anywhere in any project, solution, or configuration on the server."
Yeah, you wouldn't see the path anywhere- it will just exec based on what assembly is referenced into the unit test proj.