Hello I got a question regarding implementing Jasmine unit tests within a specific build of TFS. I found this Tutorial which explains exactly what I want.
But I do not want to use this because I do not want to check in phantomjs.exe, which is a file of +/- 46 MB, so what could I do next?
Does anyone of you know if there is a tutorial available on the web regarding installing it (instead on the TFS) directly on the build server?
It's now possible to reference Chutzpah as a NuGet package:
https://www.nuget.org/packages/Chutzpah
With that package referenced (and Visual Studio 2013 update 2 or later), the tests will be executed during the build. You will need to perform the other steps of adjusting the .runsettings and the test assembly specification to include your test files ...;*.test.js
Related
I am trying to execute NUnit tests on a legacy project in Visual Studio 2022 professional. Migrating from NUnit 2 to NUnit 3 is not yet an option, and part of the team works with Visual Studio 2019. The problem I am facing is that the tests are not executed by VS 2022, without any error message on the attempt.
The test explorer displays the following after executing the test (and the console output finishes with a successful build):
The NuGet Package manager had NUnit 2.6.1 already installed. I additionally installed the packets NUnit.Extension.NUnitV2Driver and NUnit.Extension.NUnitV2ResultWriter. A test adapter seems only available for NUnit3 (NUnit3TestAdapter, which I also installed). I enabled all the packages for the individual project as was commented in this question. Is there anything else to install/configure that I could be missing to make this work?
Just a few suggested approaches...
The NUnit3TestAdapter, as the name indicates, is designed for NUnit 3. While it is able to run NUnit3 tests by use of the NUnitV2Driver extension, I don't think that will work under Visual Studio just by installing the package. You could experiment by modifying the .addins file installed along with the the adapter, but I can't give you precise instructions as I haven't tried it.
BTW, the V2 driver extension produces output in V3 format, which is probably what you want for use under Visual Studio. The V2 result writer is only needed if you have a reason to want an output file in V2 format.
I think your best bet may be to try using the last release of the NUnitTestAdapter (i.e. without '3' in the name), which only works with NUnit V2. If it will load under VS2022, it should do exactly what you want. In that case, you should stop loading the two extensions, which the V2 runner doesn't support.
We are using VSTS to perform Continuous Integration utilizing a build template that was created to deploy a website to a Dev machine.
This worked fine but then we wanted to add in CodedUI tests to the build template in order to run the tests as part of the build step.
So I added a Visual Studio Test step to the build and added in the test by referencing the test plan and test suite it was associated with.
I made sure the codedUI project was built within the same solution as the website in order to make sure the binaries were built with it.
The problem I'm getting is that when the build runs it brings back errors stating that it is missing the relevant references for the code to work such as:
QualityTools.CodedUITestFramework
QualityTools.Common
TestTools.UITesting
etc.
However these reference are included in the project and "Copy Local" is set to true.
Does anyone have any idea what could be going wrong?
UPDATE
New error received:
MSTestAdapter failed to discover tests in class
'Email_Regression.CodedUITest1' of assembly
'C:\agent_work\4\s\Microservices\Email\Email
Regression\obj\Release\Email Regression.dll'. Reason Could not load
file or assembly
'Microsoft.VisualStudio.QualityTools.CodedUITestFramework,
Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
one of its dependencies. The system cannot find the file specified..
No test is available in C:\agent_work\4\s\Microservices\Email\Email
Regression\bin\Release\Email Regression.dll
C:\agent_work\4\s\Microservices\Email\Email
Regression\obj\Release\Email Regression.dll. Make sure that test
discoverer & executors are registered and platform & framework version
settings are appropriate and try again.
After talking to the Microsoft help team it appears that you need to have Visual Studio Enterprise edition installed on the machine you wish to run the codedUI tests on.
Update
After discussions with Microsoft and highlighting this being a rubbish solution they are now going to deliver the dll's as part of the new vsTestPlatform installer. I'm not saying this is down to my complaints, just coincidental timing.
I am using Visual Studio 2013, installed FsUnit 2.2.0, which requires NUnit 3.2.1 and FSharpCore 3.1. I created a separate test project and put a testfixture and test in there. My platform is x64 Win 10. The config is for 'AnyCPU' and 'Debug'. I've tried test settings for x86 and x64. When trying to build and create tests, I get:
------ Discover test started ------
NUnit Adapter 3.2.0.0: Test discovery starting
Exception NUnit.Engine.NUnitEngineException, Exception thrown discovering tests in C:\Users\Dad\Documents\Visual Studio 2013\Projects\...
Cannot run tests in process - a 32 bit process is required.
Exception NUnit.Engine.NUnitEngineException, Exception thrown discovering tests in C:\Users\Dad\Documents\Visual Studio 2013\Projects\...
Cannot run tests in process - a 32 bit process is required.
NUnit Adapter 3.2.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.1230077) ==========
If I change the config for both the test and target projects to 'x86' (instead of AnyCPU) then the error for the base project goes away, but the same thing happens for the test project.
No tests are ever discovered, help please ... and many thanks!
You will need to remove nunit from your solution and install it again with "NUnit3TestAdapter", version 3.0.10 works.
Uninstall any NUnit software in add/remove program and in your solutions. Now using Nuget package manager (tools > NuGet Package Manager > Manage NuGet Packages for solution...) remove any NUnit you have in a solution and install older version (e.g. 3.0.1). Find "NUnit3TestAdapter" and install in version 3.0.10.
I had the same problem with my .NET Core 2.0 project with NUnit 3.9 in Visual Studio 2017 and tests were not showing up in TestExplorer. Was stuck with this for quite some time. None of the solutions suggested in other related questions worked.
Then I figured out from this link that a class library with target .NET Standard does not work. The test project has to target .NET Core. Also, Microsoft.NET.Test.Sdk NuGet is required.
So, the steps are
Make sure that the test project targets .NET Core
Install latest NUnit NuGet (I used 3.9)
Install corresponding NUnitAdapter NuGet (I used NUnit3Adapter)
Install Microsoft.NET.Test.Sdk NuGet
Re-build and your tests will appear in Test Explorer in Visual Studio.
Note: Already added this answer to another .NET Core specific question. Adding here too, as the solution might be helpful in this scenario as well.
There is an error in NUnit 3.2.1 whereby the TestEngine assumes that it can't run a test requiring 32-bit execution in process. The assumption is always valid for NUnit's own runners, but not necessarily when the process is started by some other program. There's an issue filed about this.
Recently I came to know about the nuget based adapter/runner (http://xunit.github.io/docs/running-tests-in-vs.html)
I was trying to understand how these runners/adapters are recognized by Visual Studio and then used by the TestWindow.
For example, if I install "xunit.runner.visualstudio" nuget package all the xUnit based tests in my project are discovered and displayed in the Test Window.
I am trying to understand how does VS hook the Test Discovery using the package installed in packages folder?
Where/how do VS and nuget based adapter get hooked?
I tried finding this information but my google fu failed me. :(
I checked answer for the question (Custom test adapter installed via NuGet isn't discovering tests) and it does say that VS copies the packages to the path %TEMP%\VisualStudioTestExplorerExtensions but that's pretty much it.
Since this involves a lot of how VS works internally, I don't think you'll get a totally definitive answer. However, to solve the issue you linked to I looked at a bunch of disassembled code, so I have a pretty good idea. Here's how it seems to work:
You build a project that references a package named Sample.TestAdapter
VS copies Sample.TestAdapter directory from packages to %TEMP%\VisualStudioTestExplorerExtensions
Something triggers test discovery - rebuild always does, sometimes incremental builds do too. vstest.console.exe is useful here for debugging.
VS launches vstest.discovery.exe which looks in VisualStudioTestExplorerExtensions for an assembly that implements ITestDiscoverer
If an ITestDiscoverer is found, VS calls it with a list of assemblies that may contain tests
Tests that are discovered are sent back to VS by your test adapter
So, as far as I can tell it's a pretty simple reflection based plug-in architecture. Hope that helps.
I've been searching all morning for an answer online to this and I have tried a lot of the suggestions though all solutions seem to be for visual studio 2010 and I am running 2013 premium edition, I can't see anyone with the same problem.
I have premium VS 2013, I installed SpecFlow v1.9 through the extension manager and added the SpecFlow nuget package. I have created a CodedUI test project and added a feature file, some recorded codedui steps and a stepdef file. Nothing is complaining and it all looks like it should work. I have added :
<unitTestProvider name="MsTest"/>
to my App.config so I can run it from the test explorer in visual studio. I eventually want to run the tests via MTM but I will deal with that when I get this to work!
I have built and it is all happy so I go to test explorer and I can see my test so I right click, run the test. It fails with the following error, it does get to my Given step when I debug and falls over trying to open my application.
System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestTools.UITest.WindowsStoreUtility, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
I can not find the reference above when I have looked to add it to my project references, there is a similar one but it breaks some of my other references when I add it. I also had a look at my registry files and the path it is referring to does not exist for me to edit it. From what I have read codedui doesn't work nicely with specflow without a dll file. All references to this fix seem to be for vs 2010 and require that I copy a dll to my specflow source folder. I installed specflow from visual studio so I don't have a program files folder for it so I came to a bit of a sticking point with that solution.
Does anyone have specflow working with visual studio 2013 and a codedui test project? Am I missing a set up step? Or is this genuinely to do with my registry files / references. The error is confusing as I don't see why it is trying to find that reference especially when I can't find that reference anywhere when I try to find it manually. I'd be interested to know if you have had this error and managed to resolve it or if you could explain your set up steps so I can check I did not miss anything.
Thanks!
I believe you might be missing references to some of the required dlls. I just finished testing a test created using a Specflow, codedUI test project on my MS Visual Studio 2013 ultimate and it worked just fine.
Here are the steps which I followed:
Created a class library project.
Added reference to Specflow 1.9 from Extensions and Updates.
Added reference to Specflow CodedUI Attribute Generator (Install-Package SpecFlow.CodedUI -Version 1.0.0.23).
Added references to required libraries required for CodedUI and Test Tools to the project. See below the references I added.
Microsoft.VisualStudio.QualityTools.CodedUITestFramework
Microsoft.VisualStudio.QualityTools.UnitTestFramework
Microsoft.VisualStudio.TestTools.UITest.Common
Microsoft.VisualStudio.TestTools.UITest.Extension
Microsoft.VisualStudio.TestTools.UITest.ExtensionUtilities
Microsoft.VisualStudio.TestTools.UITesting
Added a separate CodedUI project to solution and added references of this project to the class library project.
Create a new Specflow feature file with some test and create step for it in step definition and reference to the function you created or recorded in the CodedUI project in the step definition.
Run the Specflow test from Test Explorer. It should work fine.
Note: The above steps should work just fine if you will create CodedUI project instead of a Class library project.
Mark the test class with the CodedUITest attribute.
I had this exact error message even though I had all the .dll's referenced correctly.
The solution for me was to regenerate the feature files.
(right click project and select "regenerate feature files")
Leaving this answer in case it helps someone in future.
I came across the same issue today, not with specflow but 2013 build controller and agent and codedui tests.
The solution was to install VS2013 Update 5 https://www.visualstudio.com/downloads/download-visual-studio-vs
I had same problem, found the following solution works well:
http://blog.majcica.com/2015/05/07/getting-started-with-specflow-and-codedui/