Is it possible to automatically run unit tests while you work without compiling or running them manually? I am aware that NDepend allows you to do so, but I would prefer to use the ReSharper suite.
This has been available since dotCover 10. See the dotCover documentation for details.
This adds a new panel "Continuous Testing Session" as well as a new status icon in the gutter.
Note that Visual Studio also has this feature, known as Live Unit Testing.
Not possible with Resharper at the moment, you will need something like NCrunch that runs your unit tests continuously in the background, highlighting code that breaks them as you write it and fails your tests.
Edit: At the time of my response it wasn't possible to do this with ReSharper but now in Version 10 it is, see Drew Noakes's answer. You could still give NCrunch a try as it continuously runs your tests in the background even without doing an explicit save.
Related
I have tried setting up a unit-tests project to cover front-end code in TypeScript with Jasmine and Chutzpah, but have a hard time figuring out what I'm doing wrong.
I have created a sample ASP.NET projet in which I have extracted and included the default jasmine tests. Pressing F5 opens a browser and makes the tests pass.
I have then converted all the tests to TypeScript and included its definitely typed definitions. Pressing F5 opens a browser and makes the tests pass.
I have finally installed Chutzpah with its Visual Studio extension, but I'm not able to either make the tests pass using the Visual Studio/Resharper Unit Tests window or the default chutzpah console. At this stage, pressing F5 still opens a browser and makes the tests pass.
That's the last step I'm struggling with. For clarity of the discussion, I have setup a sample projet on GitHub to reproduce my problem. I'm sure it must be something really simple but I just cannnot figure it out.
The project can be found at the following location:
https://github.com/springcomp/TypeScript.Jasmine.Chutzpah.Sample
This doesn't really answer the 'Why', I'm still trying to get my head round this myself. However it might get you a bit further forward..
I saw the same behavior when I pulled down your project. If I put the below reference paths in at the top of the PlayerSpec.ts :
///<reference path="../src/Player.ts"/>
///<reference path="../src/Song.ts"/>
///<reference path="../spec/SpecHelper.ts"/>
///<reference path="../Scripts/typings/jasmine/jasmine.d.ts"/>
After this Resharper runs the tests successfully with them passing.
I can't explain why - maybe resharper doesn't use the SpecRunner file for picking up references?
I'm currently using MSTest in VS2008 for unit tests, but I am looking at going to xUnit.net.
Question - What would be the easiest way to get my unit tests running continually locally (on my PC) and with some sort of visual alert of pass / # of fails.
i.e. similar to what we do in Ruby on Rails with autospec.
THanks
PS Intersested in hearing of the simplest open source option too.
You should also ask this on http://xunit.codeplex.com/Thread/List.aspx
There's an xUnit GUI runner and it has an auto-reload when DLLs get touched.
The missing bit is the auto-rerun of the last of executed tests when that happens. The minute we have that, you're just a Shift F6 (compile current project) away from waht you want.
The xunit.net guys are normally quite responsive, and avote on [my] http://xunit.codeplex.com/WorkItem/View.aspx?WorkItemId=8832 would help.
Regarding your other comment - I really believe xUnit.net is worth your effort using. You won't look back.
Resharper's test runner runs within VS, can run just a single test or a whole suite, and appears to support xUnit with a plugin.
I'm having a little annoying problem with resharper trying to run my unit tests, using mstest. I can select an individual unit test and run/debug it fine but when I select the parent node to run a bunch of tests, the tests don't run; it just displays pending for a few seconds and then the test goes grey. If I set breakpoints in the tests and try again with the debugger, I get the same result and no breakpoints are hit. Has anyone else experienced this before?
For anyone using VS2012 update 2 (Charlie obviously wasn't when he asked the question, but for anyone else...), then make sure you have R# version 7.1.3 or later. At the time of this writing, the official release is 7.1.2 so you'll have to follow this link to go download it:
http://download.jetbrains.com/resharper/ReSharperSetup.7.1.3000.1964.msi
(as referenced from http://youtrack.jetbrains.com/issue/RSRP-339987)
Try to rebuild your test project/entire solution.
I sometimes have similar problems where the resharper shows tests that I have removed, or doesn't execute tests for new methods in the SUT project. After a rebuild it gets back in sync.
Ran into this problem aswell with ReSharper 5.0.1659 and VS2010. Turns out that the problem lies somewhere in the testrun config files that VS creates when running test the regular VS way. Removed them from both disc as well as references in solution file which solved the problem and I was able to run the tests with ReSharper again.
Is there an API for running Visual Studio Unit Tests programmatically?
Running MSTests.exe with Process.Start() does not work in the current scenario. What I'm looking for is something like the NUnit SimpleTestRunner.
Any ideas?
/Erik
You're correct in that there's no public API for the mstest framework. I wrote a manual replacement for mstest one day to see how hard it was, and it's not as simple as it looks (particularly if you want to take advantage of more than one CPU core), so beware of going down this path.
Personally I've always just run mstest.exe programatically and then parsed the resulting .trx XML file. Are there any particular reasons why you can't use Process.Start to run it?
P.S. Some of the strange behaviour of mstest.exe are solved if you pass the /noisolation command line parameter - give that a go if you feel so inclined :-)
Update: Erik mentions he wants to run the test API in the current thread so he can set the thread culture for globalization issues.
If you run a unit test under the debugger, you'll notice that mstest creates a bunch of threads, and runs all your tests in different threads, so this isn't likely to work even if you could access the API.
What I'd suggest doing is this:
From your test "runner" application, set an environment variable
Run mstest pointing it at the specific tests
Add a [ClassInitialize] (or [TestInitialize]) method which reads this environment variable and sets the culture
Profit!
After taking a deep dive with reflector into MSTest.exe and further down into the Visual Studio Unit Test stack, I found that the API used by MSTest is sealed up and made private so that i cannot be used from the outside.
Why not using Reflector and seeing how NUnit SimpleTestRunner is running the tests... And then use this technique...
You can make use of the Microsoft REST API's for TFS to run ms tests. Please refer to the documentation here.
I've linked to "Call a Rest API" so that you can see how you'd go about calling one of the REST API's for TFS.
Note that if your tests are linked to the build, they should run automatically every time a build is queued.
Here is the link to Run Functional Tests.
I've also discovered an article on using the TFS SDK API to run tests. Here is that link as well: Link to API Article
Microsoft has a lot of stuff in there, but I'm wondering what features of Visual Studio Team System people really like and really use.
I'm specifically thinking about Team System as opposed to plain old Visual Studio.
What makes it worth the price?
I use the Development version of VSTS2005 and evaluating 2008. My top picks:
Profiler
Coding guidelines -- rules enforcement part
My favorite
Profiler
Integrated Testing Environment: I know a lot of people prefer other test frameworks but having the integration is just sweet.
FxCop
Some of the best features come from adding Team Foundation Server:
Continuous integration builds can be set up to run unit tests on every build
Code coverage figures can be gathered based on the unit test run
Reports of build success, unit test success, code coverage %, etc. can be produced daily
Code check-in can mark a work item (bug report) fixed, or can start the workflow to do so
It not only gives the developers a better idea what's going on with their code, and of how to fix it (unit tests, code coverage, code analysis), it also gives Management an overall picture of the same, without having to come around and bug the developers individually.
I like the line-by-line blame, profiler (as mentioned), but more importantly, I like the reports it produces, such as defect rates over time.
However, even though there are plenty of features that I like, I certainly don't think it provides good value for money.