A colleague and I are writing an application that we intend to go Open Source with and are trying to decide on a testing framework. Without integration of NUnit (one of us has resharper while the other does not) we are leaning toward MSTest. However, if the majority of people that work on opensource .net applications only have the express edition of Visual Studio, we may have to change our mind on that.
What version of VS do you use and would MSTest turn you off to an Open Source Project?
Personally, I'm using VS:TS, so no, test units in MSTest wouldn't turn anything off for me ;), on the other hand, most probably, you will receive different answers from the majority of people, that are not professional programmers as well (working for money doesn't mean you can not work with / on OS projects) (the question is, how many you will find here).
However, if I understand you correctly, you don't want to use nUnit, because you it's not integrated with VS08, you might then check out these projects:
1. http://sourceforge.net/projects/visualnunit/
2. http://sourceforge.net/projects/nunitaddin/
Integrating NUnit with VS08 doesn't seem to the biggest problem, on the other hand, getting VS with built in testing framework might be a bit harder, at least for some people.
If I truly felt strongly enough to participate in the Open Source project, MSTest alone wouldn't be enough to turn me away. I'd be willing to adapt to whatever the development tools/environment was.
...and for what it's worth, I use the full version of Visual Studio with Resharper.
Frankly I never heared of anyone using the Express editions. everybody I know or I talked with either use the fully fledged Visual Studio environment, or they use the compiler/libraries from the free SDK and some editor on top of it (CodeWarrior, CodeInsight etc) and rely on some variant of make to build.
As for the test framework, is a double edged sword. I'd recommend NUnit, but then... remember what happened with NDoc?
I use MonoDevelop and NUnit.
I use VS 2k8 pro & MSTest as much as possible.
Even Microsoft's open source code (Prism/Composite Wpf) doesn't use MSTest. They use NUnit for that project, I believe.
You may be able to use visual nunit to add an nunit test runner in VS. link I've never used it but it may be worth a shot.
If your problem is integration with Visual Studio, try Gallio. It includes MbUnit, but its runner can run tests from other frameworks, like NUnit, xUnit, etc.
It would turn me off. See MSBuild, NAnt, NUnit, MSTest, and frustration (not written by me!) - I suspect a number of people feel that way. MSBuild seems like a poorly-done copy of NAnt (we use both), and MSTest likewise looks like Microsoft's usual "not invented here" answer to open source. While I realize that there are good business reasons for Microsoft to develop their own tools, those reasons don't apply to their customers.
There are plug-ins to use NUnit inside Visual Studio without ReSharper, or you can always configure it to run from the Tools menu.
If you don't like NUnit, please consider one of the other open source testing alternatives. See NUnit vs. MbUnit vs. MSTest vs. xUnit.net
I think that people who don't use windows will be knocked out of the project by the choice to use MSTest. Most other c# testing libraries are available in mono, which is what mac and linux users would be using.
I don't know how many cross-platform developers you are trying to attract, but if it's an issue then you need to consider Linux/Mac devs.
Related
My company is upgrading our Visual Studio 2012 to 2013 Premium. In the midst of this, we are also looking to start automating our tests using Visual Studio Team Services
I have read a couple of MSTest vs nUnit posts and articles in the past but most of it compares the older version of MSTest. Also, nUnit had a lot of favourable reviews as compared to MSTest.
My question is, considering Microsoft's commitment towards the ALM, Agile practices and all the new stuff they've added into VS2013 Premium and Visual Studio Team Services to facilitate and encourage automated testing, how does MSTest compare to nUnit?
What other considerations should I take before making a decision on a testing framework to use?
MSTest hasn't changed much since it was originally introduced, so those old comparison posts are still valid. Microsoft's focus on ALM is mostly targeted at further tooling and server products, but the fundamental testing framework hasn't changed much.
It's also worth noticing that MSTest and their entire ALM strategy is targeted at many different styles of automated testing, including Integration Testing, System Testing, Performance Testing, etc., so while it attempts to be a one-size-fits-all, it's quite ill-suited for unit testing, because it's too heavy to work with.
While NUnit is better, I wouldn't recommend it either. It, too, hasn't changed much for years, and the extensibility model is frankly a mess.
Instead, I'd recommend xUnit.net. Although it's not perfect either, it's currently the best mainstream alternative on .NET. There are many reasons why xUnit.net is better than MSTest.
MSTest Vs NUnit:
MSTest is integrated with VS so it'll be easy to use. NUnit will require third-party tools (some are free, some are paid).
VS will give you Code Coverage in MSTest. NUnit requires DotCover (which is a paid tool).
MSTest has an option to execute your tests in parallel if they don't depend on each other. This isn't a feature that NUnit provides.
NUNit has TestCaseSourceAttribute which helps you to achieve parametrized test cases but in MSTest you'll need DataSourceAttribute which would be in XML file and will be difficult to manage when you have complex logic in the method.
NUnit is faster as compared to MSTest.
Overall both frameworks are good to use, but I'd suggest going for NUnit.
I currently develop C++ and C# applications with VS2008.
I have the need to have the IDE as well on our target demonstrator platform to debug and fix etc.
Could I use the Express editions for that task or are there any major drawbacks?
Any experience is welcome, also for VS2010.
You would only have to tweak things in order to compile for 64 bit (true only for 2008, that's easy in 2010), but no major functional drawback, except as Binary Worrier says it can be frustrating
However it is commonly accepted to have the full IDE on a target demo platform without buying another license, so why not deploying full IDE on it?
From About Licensing
Developer Tools – per user license
You must acquire a license for each
user you permit to access or use the
software. You may install any number
of copies on any number of devices for
access and use by one user to design,
develop, test and demonstrate
programs. Only licensed users may
access the software.
I normally don't like to just post links as answers, but this Wikipedia Article covers the differences in depth.
I've personally found coming down from the full- to the express-editions painful and limiting, you're forever "bumping your head" off some limitation or missing feature, very frustrating.
I use vs2008 at the buro and vs2008 c# excpess at home.
the limitations i come across were
c# express doesn-t like solution folders containing projects and cannot load other project types (ie setup-projects, mstest projects).
i had no painful and limiting experience yet because i use nunit instead of mstest and only ms-databases. express has advanced options (disabled by default) that give you the debug/release build feature.
by the way you donot need a visual studio to debug dotnet-applications. the dotnet-sdk has a gui debugger that can do the job but it is not as comfortable as vs. to rebuild you can use msbuild.
I am interested in improving my testing methodologies but am not sure if I prefer NUnit or VS 2008 test. I also have a book that uses NUnit in it's examples. Do the two frameworks coexist well with each other?
yes, check out this links NUnit/MSTest Dual Testing
and Strengthening Visual Studio Unit Tests
You would normally choose one or the other test framework rather than working with two. Although in fact, as Sergey mentions, NUnit and MSTest frameworks are similar enough that you can, with care, write common tests that can work with both frameworks. There are some subtle differences, however, so watch out and stick to the lowest common denominator!
If you want to run a suite of tests that are written for several frameworks, take a look at Gallio which supports this scenario.
If you use different frameworks in different classes, there is absolutely no problem, the can coexist without trouble (the project then references both VisualStudio test libraries and NUnit.Framework). You can also use hacks like the one Sergey posted to make them coexist on a same class but be aware that this has some limitation and you won't be able to use advanced functionalities of both frameworks and map them in this way (e.g. TestCase attribute for NUnit 2.5 etc.).
I'm curious about your thoughts on moving to a new IDE (specifically Eclipse). I have been hearing wonderful things about it from this community and I'm always on the lookout to try new things.
Currently I'm running Visual Studio 2005, with a bunch of external commands loaded (for compiling down to a binary, running lint, etc). We're developing C code for microcontrollers.
I've read over some of the other threads on here about the advantages and disadvantages to Eclipse and Visual Studio (specifically SO - best IDE thread and SO - best C IDE thread), but I'd like to hear your thoughts on using it for programming an embedded environment. I'd imagine that there is a simple way to use the external tools that Visual Studio currently uses (it simply calls various batch files that we've created).
Is it worth it to specifically switch over to Eclipse?
Answer to you question about way to call external tools: no problem - from eclispe you can anything: external program, batch file etc. Moreover, if you use custom build generator - you could use it transparent with eclipse.
I don't think I would switch from Visual Studio to Eclipse in hopes of getting a better IDE. Typically an embedded manufacturer makes plugins and toolchains that work specificially with Eclispe, that's what makes it worth using in the Embedded world. For example with the NetBurner plug-ins, when creating a new project you can just select New NetBurner Device Excecutable, or New NetBurner Library, all the default includes and libraries get set up for you and the proper cross platform tool chain is set up for you automatically. In the NetBurner case it also uses the Eclipse managed build process (as opposed to make files) which I find nice. There is also support for using make files if you prefer that option.
While I have a couple of tools set up to run as external tools (lint, an auto version updater, DOS prompt etc) most steps can be triggered from pre-build or post-build steps or there are many many add-ons for common needs like source code management, bug tracking, etc. There is great support for SVN, Trac and Mylyn for example. I use both VS2010 and Eclipse. I like them both but VS2010 is the better IDE. It's a little hard to compare because I do C# (and a little C++/CLI )in VS and C++ in Eclipse. That said, I wouldn't relish the though of trying set up VS to do my embedded tasks.
It seems like you should be able to configure your preferred unit testing framework as a plugin to Visual Studio/Team system like you can with SourceControl providers. Is anyone aware of an effort within microsoft to allow for swapping out MStest for nUnit, mbUnit or others. Obviosly nothing prevents me from using these other frameworks but they aren't integrated into the IDE.
If you have a look at TestDriven.NET and Gallio, they both have test runners that integrate to the IDE and allow the use of other testing frameworks.
If you are adverse to spending a little this is just on of the many benefits of ReSharper. (r#)
Not sure MS would want to encourage us to use another one unfortunately.
Kindness,
Dan
Visual Studio does provide the concept of custom test adapters that would allow custom test frameworks to execute under the MSTest platform. That is, tests for those frameworks would appear in the Test List Editor, etc.
The developers of the custom frameworks would have to develop a VSPackage that would find and provide tests to MSTest. Unfortunately, there's a lot of work involved.
TestDriven.NET, Resharper (and several others) provide their own VSPackage to provide their own Tool Window, but do not integrate directly with MSTest.
Looks like this is coming to Visual Studio 11