Trying to get SpecFlow running with a fresh VS2010 Professional install. Created a new console application and added references to NUnit and SpecFlow. Created a SpecFlow feature. The .feature with the default template code is created.
Now I try to run this test, but I don't understand how. When I right-click the project (at the top-level), there is no "Run test(s)" option in the mouse drop down menu. Didn't the SpecFlow install correctly, am I missing some references or some other tool I need to install?
If you want to be able to run your tests directly from Visual Studio 2010 without any additional tools or extensions than you should configure SpecFlow to use MsTest as its unit test framework.
This can be done in your application configuration file with the following:
<configSections>
<section
name="specFlow"
type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
</configSections>
<specFlow>
<unitTestProvider name="MsTest" />
<!--
Use this if you're running VS2010
<unitTestProvider name="MsTest.2010" />
-->
</specFlow>
The generated code-behind file will then contain MsTest tests that are recognisable by Visual Studio and can be run with the build-it test runner.
No need to use NUnit at all.
SpecFlow does not provide a runner itself.
SpecFlow generates fixtures for one of the common Unit-Test-Frameworks. In SpecFlow 1.3 NUnit (default), MSTest and xUnit.net are supported (configured in the App.config).
To run the fixtures you have to use a runner that is capable of running them.
ReSharper is a very good option for a test runner that is integratied in VisualStudio, but it is not free. ReSharper gives you the "Run Unit Tests" context menu in the solution explorer, you are referring to.
An alternative for VisualStudio integration is TestDriven.Net (also providing a context menu).
For NUnit you can also use the runners that come with NUnit itself (there is a GUI-Runner and a commandline runner).
For MSTest you can use the native VisualStudio integration for running tests (however I find that one a bit clumsy).
xUnit.net also comes with its runners, however I am not familiar with them.
Furthermore, you can use MSBuild tasks to run the fixtures ...
Just to update this questions, in the latest versions of specflow you should use, (use MsTest.2010)
<configSections>
<section
name="specFlow"
type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
</configSections>
<specFlow>
<unitTestProvider name="MsTest.2010" />
</specFlow>
see SpecFlow wrongly using NUnit
I have written a blog post on how to use a batch file as an external tool to automatically run SpecFlow features in NUnit and generate a html report. See: http://watirmelon.com/2011/02/18/c-sharp-atdd-on-a-shoestring/
Something that caught me out. I had to ensure that the Project Type was a Test Project. I had to unload the project and change the Project GUID to that of a Unit Test Project. Once I did that the SpecFlow (and any other) test in the project sprang to life
The Specflow tests are run using the NUnit (GUI), which needs to be invoked externally or alternatively TestDriven.net or Resharper can be installed to support running the tests from inside Visual Studio.
I have created a video demonstrating how to use Specflow with VS2010 here
You can also try Visual Nunit, an open source NUnit test runner plugin to Visual Studio 2008 and 2010. Get it using NuGet, for more info see http://www.bubblecloud.org/visualnunit
Related
For the life of me I cannot get SpecFlow configured to use NUnit as the test runner. NUnit is not listed in the Tools > Options > SpecFlow test runner dropdown, and the "auto" setting doesn't find NUnit.
I've tried reinstalling both NUnit and SpecFlow multiple times. Tried installing SpecFlow 1.8.1 and 1.9 to no avail. Tried using app.config. I've even tried re-installing Visual Studio 2010.
I just want SpecFlow to use NUnit (2.6.1). Can anyone help?
Visual Studio->Tools->Extensions and updates->Online->search "Test Adapter"->install.
http://visualstudiogallery.msdn.microsoft.com/6ab922d0-21c0-4f06-ab5f-4ecd1fe7175d
better late than never.
Try editing the config file of your unit test project:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />
</configSections>
<specFlow>
<unitTestProvider name="NUnit" />
</specFlow>
</configuration>
Check out the config documentation here.
SpecFlow itself does not provide a test runner and if you want to right click on the .feature and run it, you'll have to re-configure SpecFlow to use MSTest, which should use built-in VS runner.
If you want to use NUnit, then additional tools are required. I would recommend Resharper, but it's not free. This article should help you choose a way to run NUnit tests from VS.
Good luck
To get integration with Visual Studio, you need to install the SpecFlow Extension. SpecFlow comes in 2 parts.
The NuGet package contains the dlls necessary to author and run the tests.
The Visual Studio Extension which gives you Intellisense, syntax highlighting, code navigation, and the ability to run tests by right clicking and choosing "Run SpecFlow Scenarios".
I am guessing you have installed the Nuget package, but haven't installed the extension.
To install the extension:
Open Visual Studio
Choose Tools -> Extension Manager
Search for the SpecFlow extension
Click install
I have configured Specflow to target the MsTest framework (instead of NUnit) by specifying it like this in the app.config of my 'specs' class library:
<configSections>
<section name="specFlow"
type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
</configSections>
<specFlow>
<unitTestProvider name="MsTest.2010" />
</specFlow>
So once it's in place, I can see that my test fixtures are produced correctly by the Specflow custom tool, with correct TestClassAttribute() and methods, etc:
[System.CodeDom.Compiler.GeneratedCodeAttribute("TechTalk.SpecFlow", "1.3.3.0")]
[System.Runtime.CompilerServices.CompilerGeneratedAttribute()]
[Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute()]
...
The specs class builds, but now I cannot run the tests using the Test --> Run --> All Tests in Solution inside Visual Studio 2010 on my vista 64 box. Why doesn't VS recognize these as valid tests to run?
As per Dror Helper and Alex Duggleby you'll want to add the following line to your .csproj file:
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
Add it after the FileAlignment element, and reload the project. It should now be an MS Test project and you get the MS Test functionality in the context of this project. The Guids mean:
{3AC096D0-A1C2-E12C-1390-A8335801FDAB} - Test Project
{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} - C# Class Library
I had to recreate the project as a Test Project and not merely a Class Library -- because I had started development with NUnit and SpecFlow, I had created a vanilla class library to hold my specs that had the NUnit attributes decorated. I thought I could simply change the app.config of this existing project to point at the mstest framework and stop using NUnit, but VS2010 never recognized the tests, despite the correct creation of the stubs by specflow's custom tool.
So...I added a new Test Project to my solution, moved all of my spec code to that new project, then recompiled, and viola, VS2010 recognizes the tests. I'm sure there is a GUID it is looking for in the XML of the .csproj file or something that tells it to wire up the testing framework, but I didn't dig that far.
Hope this helps someone.
To change your class library project template into a test project, modify the .csproj and add the following line:
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
to the first property group element:
<PropertyGroup>
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
I'm beginning to get the grips of BDD and MSpec, but I'm still really bugged by the fact that I'm unable to debug my tests/specs, and that I have to leave the IDE to go to a html report file to see the results.
Currently, I have a post build event configured to run mspec.exe $(TargetFileName) --html “$(ProjectDir)Report.html”, but there must be some better way to do this.
Does anyone know any good add-ins, test runners or whatever that will let me
debug tests, instead of just running them
show the test results in a nice way inside Visual Studio
(Footnote: I'm running VS2010 Professional on Windows 7, if that matters.)
There are basically three options you have:
You can set up a custom tool in
Tools | External Tools to start
mspec.exe with the current project's
assembly to run the contexts and
generate the HTML report.
Install TestDriven.Net and
ensure that
Machine.Specifications.dll.tdnet
and
Machine.Specifications.TDNetRunner.dll
are in your project's copy of MSpec.
You can then run and debug your
contexts from the context menu: "Run
Tests", "Run With | Debugger" without further installation.
There's an example of what the MSpec folder
looks like for all of my projects.
If you use ReSharper 4.1, 4.5, 5.0
or the latest 5.1 EAP (== beta)
there are runners for each of these
versions.
The ZIP download
contains batch files that install
the runner for each respective
version of ReSharper. ReSharper's
unit test support is pretty
extensive in terms of UI
widgets/shortcut support, the reporting
tree view and debugging.
If you're
a dotTrace user you can also profile
right from within Visual Studio.
dotCover (another JetBrains product)
allows you to calcualate code
coverage results from your MSpec
runs.
On top of that, you get
all the nice coding and navigation features that ReSharper provides.
Be aware that only the first option will generate the HTML report as both the TestDriven.Net and ReSharper runners do not support HTML report generation. From my point of view this isn't an issue since the TD.Net and ReSharper runners offer fairly complete reporting mechanisms through the Visual Studio UI.
Another option that might work (I haven't used it myself) is to leverage the Gallio support that MSpec has. Gallio is a runner/framework for several testing frameworks; it might as well support debug runs with MSpec. Contact #smaclell if you have questions about Gallio support.
If I install Gallio 3.x will it also install a test runner plugin for Visual Studio?
Or must I use an additional plug-in like TestDriven.NET or Visual Nunit to run MbUnit test classes from within VS?
Install Gallio 3.1 on the dev machine. Then in VS2008, you'll have the option to create a "MbUnit v3 Test Project". This doesn't just include all of the Gallio dlls for you, it has a magic line in the project which identifies it to VS as a Test project.
You can now just use the in-built VS2008 Test runner.
If you have any existing projects with unit tests in, rather than making new projets, edit your existing project file and add the following line on line 9 (underneath the <ProjectGuid> on line 8):
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
(If you have a VB project, it has a different second GUID: <ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{F184B08F-C81C-45F6-A57F-5ABD9991F28F}</ProjectTypeGuids> You can find the correct values by creating a new MbUnit test project from the templates installed with Gallio and then looking at the project file (.csproj or .vbproj) in a text editor.)
Now when you reload the project, VS2008 will recognise it as a test project.
A distinct advantage that I found over using Icarus was that debugging is now far more straight forward with break points being hit as expected.
Good Luck, Lee
TestDriven.Net works really well. Gallio also supports the ReSharper unit test runner and Visual Studio test tools. We will be shipping a new release of Gallio this week with support for R# 5.0 and VS 2010.
These are instruction for running MBUnit tests in Visual Studio 2012 and above using a neat NUnit trick.
Firstly, install the NUnit Test Adapter extension (yes, NUnit)
Tools > Extension and Updates > Online > search for NUnit > install
NUnit Test Adapter.
You may need to restart the Visual Studio IDE.
Then, you simply need to add a new NUnit test attribute to your test methods. See example code here (notice the using statements at the top) ...
//C# example
using MbUnit.Framework;
using NuTest = NUnit.Framework.TestAttribute;
namespace MyTests
{
[TestFixture]
public class UnitTest1
{
[Test, NuTest]
public void myTest()
{
//this will pass
}
}
}
You can run and debug the test in visual studio as NUnit and Gallio Icarus GUI Test Runner will run them as MBUnit (enabling parallel runs for example). You will need to stop Gallio from running the NUnit tests by deleting the NUnit folder in the gallio install location i.e. C:\Program Files\Gallio\bin\NUnit
Hope this helps, this is a simple working method so please vote up, many thanks.
I'm trying to load a Visual Studio 2010 beta dll into the NUnit GUI. I get a popup error.
This assembly is built by a runtime newer than the currently loaded runtime and cannot be loaded. You may be attempting to load an assembly build with a later version of the CLR than the version under which NUnit is currently running.
How do I force an executable to run under .NET 4?
I've downloaded the NUnit 2.5 source and opened the VS2008 solution in the VS2010 beta. Once the conversion finished I opened all the projects and changed the target framework setting for all the projects to ".NET Framework 4.0". I then built the solution without any errors. I can now use the NUnit GUI app to run tests built for .NET 4.0. I've not done exhaustive testing of this build so there may be problems, but for my purposes it works fine.
Update: It is not necessary to rebuild NUnit. I discovered that if you add the following to the relevant NUnit config file you can run a test dll built for .NET 4.0.
Under <configuration> add:
<startup>
<supportedRuntime version="v4.0.30319" />
</startup>
and under <runtime> add:
<loadFromRemoteSources enabled="true" />
With .NET 4 being released, I used
<supportedRuntime version="v4.0.30319" />
in the NUnit 2.5.4 exe.config instead of requiredRuntime, and the loadFromRemoteResources tag as shown above and all worked well. Thanks!
You don't have to modify any file
just open this file and everything will work just fine
C:\Program Files (x86)\NUnit 2.5.10\bin\net-2.0\nunit-x86.exe
In NUnit 2.5.5 you can specify in the command line the option /framework=net-4.0 and it is compatible with the new assemblies generated with the release of the latest VS2010.
The proposed solution worked great for getting NUnit to run. Unfortunately, when I then got to my code coverage step, NCover started giving me:
Profiled process terminated. Profiler
connection not established.
The best solution I found to this was to just use the "-x86" version of NUnit with NCover:
NCover.Console.exe
nunit-console-x86.exe --additional params--
Works now.
To find your .net 4 version from the Visual [C# 2010 Express, or which ever] go to the Visual application's "About" under the menu's Help item. It should show up as the version under the Microsoft .NET Framework line on the top right hand side of the window.
I ran into the very same error message while running NUnit 2.4.8. As I had not upgraded in some time I installed the current NUnit (v2.5.9) and found that it now supports VS2010 assemblies. So if you have newly encountered this error check your NUnit version: as of December 2010 (or so) the only thing you need to do is upgrade NUnit.
As of NUnit 2.5.10 you can enable visual studio support in the GUI runner:
Tools-> Settings-> IDE Support
After that I was able to successfully attach to the nunit-agent.exe process which runs your assembly in a .NET 4.0 app domain
If you experience this issue after upgrading to nunit 2.5.5 then you will need to upgrade nant to the latest version for me it was .91 alpha.
I found usefull to start from NUnit Application Template. It support VS C# Express, allows debugging tests and contains precompiled NUnit for .NET 4.0. Thank to author new test project gets ready with one click.