I've got a Visual Studio Solution File conatining multiple projects. About half of them are Unit Test Projects.
Visual Studio Test Explorer reliably picks up all Test Projects and allows me to run all of them at once.
We also got a CI that should run these Tests. Is there a way to use the Test Explorer functionality from the command line? Currently I have to pass a list of dll's and executables to the vstest.console.exe Tool.
I searched google for solutions but only found solutions that pass dll files to vstest.console.exe. I would expect a possibility to run the tests from a solution also from the command line similar to how dotnet test allows me to do this.
Unfortunately I cannot use dotnet test to reach my goal, as dotnet does not yet support native projects (which are also conatined in my solution).
Hi I'm trying to run a build with code coverage, I am using Visual Stuido 2017 enterprise I have tried to enable the code coverage in VS but if i click on Edit build definition in team explorer it opens the build definition in TFS.
Apologies for the delay in posting this as an answer.
As a prerequisite to using Code Coverage, the first thing to do is to install Visual Studio Enterprise on the build agent (Which you have already done). Following this, you must then update your builds to specify that Code Coverage is/should be enabled.
To enable code coverage, ensure you have included a Visual Studio Test task (or any equivalent which supports Coverage) in your build definition.
Once you have added the Test task, you'll want to configure this task and set your test assemblies, for this you can use an absolute path to point to a DLL file, or use wildcards like I do here
$(build.sourcesDirectory)\Source\Tests**\Release*Test*.dll
This says that in my sources directory I have a folder called Source, and inside that is a Tests folder which contains all of my unit tests. Then in each Unit Tests folder I use a wildcard to say "any folder at all that contains a Release folder", look inside and take the DLL which has 'Test' in its name.
So now that the testing assemblies are gathered, you can tick the checkbox that says "Code Coverage Enabled", and you're good to go.
I will edit this post later today to include screenshots of my own personal VSTS builds.
I'd like to modify PATH (to include the path to an alternate Oracle client than the one my workstation usually uses) before unit tests run (or debug). How can this be done? Or is there a better approach?
I have noticed on occasion (not very frequently) that after I run my suite of MsTest unit tests in Visual Studio, I will get a folder or folders that are of the form (the guid varies):
EQT_6A69DAE9-1C34-4B96-8B96-698A94EA1ADF
The folder will only contain a copy of my .exe file for the application. In this case, the application is using WPF.
These folders always show up inside my Debug directory.
There is nowhere in my application that I am creating anything like this (and certainly not copying my own .exe file), so it appears to be a product of something in MsTest.
Why does MsTest sometimes create this folder with a copy of just my .exe file?
I believe these are the runs that would have code coverage enabled via your .testsettings file. Your .exe is being backed up here while an instrumented version is created for code coverage when running with .testsettings.
This will not happen when you run code coverage without any .testsettings using the "Analyze code coverage" option available in VS2012
I realise that there are many older questions addressing the general question of NUnit v MSTest for versions of Visual Studio up to 2008 (such as this one).
Microsoft have a history of getting things right in their 3rd version. For MSTest, that is VS2010.
Have they done so with MSTest? Would you use it in a new project in preference to NUnit?
My specific concerns:
speed
running tests within CruiseControl.NET (either commandline or MSBuild task)
code coverage reports from CC.NET
can you run MSTest tests in debug mode
(We use ReSharper, so test-runners are not an issue for us. We have used NUnit for the last few years. We do not have TFS.)
List item speed is same, but MsTest may be a bit slower because it creates folder for test run every time
MSBuid and CC.Net is big pain. You can't run MSTest on computer without VS on it (not 100 sure about 2010, but with 2008 it is so)
not sure, sorry
yes you can, from visual studio
My recommendation is following: if NUnit satisfies you - use it, forget about MSTest
To correct some old information on the thread;
It IS possible to run 64 bit tests in 2010
From VS2008 forward it is not neccesary to have MSTEST create directories anc opy the binaries in, just disable deployment, in 2010 thats the default but you have to set it in 2008
2010 MSTEST is faster but as its a generalised test framework that also runs load/web/UI tests there are compromises in the design that will lead to it being slower. Jamie Cansdale appears to have managed to get perf increases with the lastest releases of TestDriven.net's support for MSTEST
I've mainly used NUnit, some xUnit and some MSTest. They seem functionality equivalent, but I don't like the MSTest test runner. It runs in visual studio so it either crowds the screen or is on another monitor getting in the way everytime I tab to visual studio. (I run NUnit on another monitor, but it doesn't cover everything on that monitor everytime I focus visual studio). It takes too many clicks to find out what test failed and why.
NUnit can run in the background until a test fails, at which point it shows you information about the breaking test. This seems like the ideal for keeping red/green/refactor going smoothly.
Nope. Same issues regarding appdomains and assembly resolving still exist. I would avoid unless you want the new goodness for other functional testing or integration with Team System.
I don't know much about CruseControl.net, but you can debug tests. We currently don't use TFS either, and the MSTest is working for us.
If you think you'll ever run your tests in 64 bit mode, use NUnit. MsTest is only x86.
One major difference between the two is that MSTest makes a copy of the current DLLs every time it runs a test. If you're doing TDD and running your tests frequently, this can eat up a lot of hard drive space.
If you're using MSTest, you can change this setting in Tools > Options > Test Tools > Test Execution. "Limit number of old Test Results to" is set to 25 by default in Visual Studio 2010. I usually change it to 1.
MSUnit runs your test cases under conditions that are different from the actual execution environment. Specifically, the deployed files differ from those that are deployed when you run your actual project. Nethertheless, there is the [DeploymentItem]-Attribute to specify which files shall be deployed by MSUnit. So if your application depends on any external
files, such as
database files
database configuration file
application configuration file
...
then MSUnit is not the right choice, because the MSUnit tests never cover what your file system is going to look like in execution environment. The Visual Studio Project File settings for deploying files (Copy always, Content, etc.) are ignored by the MSUnit runner. So those settings cannot be tested.