I have the exact case as in this post:
How to set the test case sequence in XUnit
Which points me to this link:
http://www.bricelam.net/2012/04/xunitnet-extensibility.html
Followed the snippet of code in the post, the one using RunWithAttribute.
But as one of the comments in the post pointed out,
once I annotate my test class with [PrioritizedFixture] all of my test classes are no longer recognized by the xUnit Runner in Visual Studio.
Is there anyway to make this work?
*Using VS2013 and xUnit 1.9.2
Related
Since I upgraded to VS2012 and Resharper 7, my previously working MS Tests are not running anymore.
The tests are run in an ASP.NET environment. I use the following Attributes:
[TestMethod]
[HostType("ASP.NET")]
[AspNetDevelopmentServerHost("C:\\Projekte\\****\\Website", "/")]
[UrlToTest("http://localhost:7924/")]
Any idea how to fix this?
As odd as it is, using VS2012, using Resharper 8.0, using NUnit, I was receiving this error because of an entry in my app.config file. I added an EntityFramework connection string and this behavior started. Removing the entire connection strings section shows the test runner starts/works again. Viewing output shows the app.config is not valid - this was causing this specific behavior in the test runner - "Test wasn't run".
I had the testproject set to AnyCPU and the project set explicitly to x86 when this happened. Setting the testproject to x86 solved it for me.
I'm using VS2012 R#8 and nUnit
Try running the Unit Tests using the MSTest Test Explorer. You might find more details in the output window of the root cause.
For me, it was a referenced assembly that was using a more recent version of NUnit than the one that was referenced in the test project. Using the same up-to-date version fixed the problem.
System.IO.FileLoadException: Could not load file or assembly 'nunit.framework, Version=2.6.3.13283, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference.
I'd same problem I just..
Changed methods access modifier from private to public.
Removed static keyword from methods.
That's it. It worked for me. But that is for C#.
In my case it was the NUnitTestAdapter nuget that needed to be removed.
Check that any references that you have in the test project are set to Copy Local True.
I had the same problem. Couldn't get the Visual Studio test runner to work, so I tried debugging a test instead. This threw a ConfigurationErrorsException, which didn't have much of a stack trace but contained the phrase "ClientSettingProvider". I searched my solution and found that something had added a appSetting key for "ClientSettingProvider.ServiceUri" to my app.config. I deleted this (along with an empty connectionStrings element) and rebuild everything - fixed the problem!
Double check your app.config and try deleting any empty elements or anything that looks fishy.
Another (silly) problem it might be; I accidentally had the project set to not build. Go to Build/"Configuration Manager", and ensure the project is set to build.
Using VS2010 and ReSharper 9.1 the issue was that the file LocalTestRun.testrunconfig was missing but was referenced in my .vsmdi file.
The test without VS were running properly but I always had the "Test wasn't run" error in ReSharper test UI.
So I simply restored my old LocalTestRun.testrunconfig file and everything run perfectly.
I probably could have updated my .vsmdi file to not reference the missing file... I did not test that.
On VS2012, Test Project don't work on Shared Folders like \XXXXXX\XXX
I solved it copied the Test Project on local devices.
Good Luck
I found that the settings file for the unit test may need checking as to which one is being pointed to by ReSharper. I had the same thing happening and it was down to my unit tests for the RS Harness pointing to the wrong file.
I had the same issue because the test class name had the characters '<' and '>' in it (also '(' and ')' caused this issue).
Removing those symbols fixed the problem.
I could use symbols in identifiers thanks to F#'s Unicode support.
I had the exact same issue and nothing helped.
eventually I saw that I had a mismatch in my namespaces of the unit project and the unit test project.
The namespace of my unit project is unit.project and the test project was named unit.project.tests but the default namespace of the test was the same as the unit, both was unit.project.
Once I've updated the namespaces to be different (one namespace for each project) everything worked!
I had a similar problem with NUnit test, which would not be executed, but R# would only mark them as "Test wasn't run".
Executing them with the native NUnit runner revealed, that the app.config file had an error (actually, 2 ConnectionString sections). Fixing this also made the tests running in R# test runner.
I kept getting "Test wasn't run" in Resharper.. I tried all the recommendations but nothing worked. What solved it for me was running Visual Studio as Administrator. (VS2013 w/ Resharper 8.1)
Just an excerpt from MSDN regarding
Assert.Inconclusive:
The code generated by Visual Studio when creating unit tests includes
an Inconclusive statement as a placeholder.
It happens if something is wrong with the solution, the most often a misconfiguration, like wrong or mismatched namespaces, inconsistent build targets etc, what leads to the fact that UnitTestExplorer is unable to use provided unit tests properly. So the general solution is to check latest changes and fix errors.
I had the same problem in C#: Unit tests run by ReSharper all just stopped with "Test wasn't run". No other information.
It turned out to be due to my custom section in App.Config. Removing that and it worked.
Configuration: Visual Studio C# 12, ReSharper 8.2.3
It could be also that Your solution contains multiple versions of i.e. nUnit installed in different projects. In my case this was the reason of the problems. After unifying the nUnit version in the solution, the problem was gone.
Just to add to this, I had written over my app.config file with a new one that was missing some sections I needed. I added the sections back in, at which point I got this same error in resharper. Thanks to the comments above I compared it to an older version and found that I was missing the section names in the configSections.
I'm trying to debug into the tests after the setup method is called and the tests depend on the setup method being called.
I'm using Nunit 2.6.0.12051 testing a .Net 4.0 class library. The nunit Visual Studio project has a class marked with [SetUpFixture] and a method marked with [SetUp].
If I run the tests from the NUnit gui, I'm fairly certain the setup attrib'd class is called (because it isn't stopped at the setup class with a run-time error now) but I can't debug into it. If I try to change the settings to see Verbose Tracing, NUnit gui throws an unhandled excption.
If I run the tests from Visual Studio via Test View/Debug Selection, the break point at the setup method doesn't stop execution and the trace statements inside the method don't print in the debug window. So I'm certain the setup method isn't getting called.
While I could just change the setup class to be the base of all test classes, I only need the method run once.
Any help would be wonderful.
I just ran into this issue and eventually found this significant sentence from the NUnit SetUpFixture documentation:
"This is the attribute that marks a class that contains the one-time setup or teardown methods for all the test fixtures under a given namespace."
Turned out my SetUpFixture class was in an entirely different namespace than my tests, so it wasn't being run.
I just noticed the same when using the latest NUnit from NuGet (2.6). The [Setup] method is not run before the [Test] -methods.
I don't know why they changed this quite significant part of NUnit, but I fixed it for my purposes by going back to version 2.5.10 which does run [Setup] before [Test].
I had this issue too but installing the latest version of the test runner (TestDriven.NET in my case) fixed it. It wasn't an NUnit issue for me.
I just ran into a similar issue. My unit test was not calling setup either. After reading the NUnit doc. referred in the top answer, I figured my solution was extremely simple.
In very simple terms, I was missing the "Setup". Realised after reading the doc.
*Example from NUnit SetUpFixture doc mentioned in top answer.
[SetUp]
RunBeforeAnyTests()
{
//
}
----------------------------------------------------
// this was my fixed up code
// simply added [Setup] and boom, bob is your uncle.
[SetUp]
public void Setup()
{
readChoice.Reset();
}
MSTest + Visual Studio generates accessors which appear on the NCover coverage report. They usually have a low coverage in my case. I wonder if I use them in my tests if the method I'm testing will appear as covered in the Accessor but not in the original source code.
If this is the case, it is a problem and I need a solution.
The dev team was interested in this issue and created an MSTest private accessor project in VS 2010.
I ran coverage on it in NCover 3 with MSTest, and it appears that these accessors are reflecting the source code, not a generated version of it, at least in our test.
If you'd like to get the test project and try it out, just contact us at support#ncover.com.
Thanks!
If I understand your scenario, then any method called by your test will be shown as covered in the source code.
If the test adds accessors, those will also be shown as covered where they appear, but you can filter them out by excluding the get and set methods that are generated.
Here's a link to the syntax for the method exclude:
http://docs.ncover.com/ref/3-0/ncover-console/command-line/profiling-options#em
We've been successful here with the regex .*.get_.*, just as an example.
NCover Support
Is that possible to use Visual Studio Code Coverage without Unit Testing? I would like to make a coverage analysis within a normal program execution.
UPDATE:
There are no current solutions to use the Visual Studio Coverage tooling for what I want, although dotCover seems to be a nice third part solution for the problem.
Here's a more detailed answer rather than just a link:
To do this for already instrumented files with an IIS Express application:
Get the name of the site from C:\Users\<your user>\Documents\IISExpress\config\applicationhost.config
vsperfcmd /start:coverage /output:run.coverage
launch your app
run your manual tests
then to finish
vsperfcmd /shutdown
from related question https://stackoverflow.com/a/23791306/57883
for a full walk through, here's a link to the blog article I just posted on it:
http://imaginarydevelopment.blogspot.com/2015/02/get-code-coverage-from-vs-without.html
I think you may be misunderstanding what Code Coverage is. Code Coverage indicates how much of your code is exercised by your unit tests. If you have no unit tests, you have zero code coverage. Are you perhaps referring to code profiling (measuring how long it takes for code units to execute?)
UPDATE:
If you're looking for metrics on what code is executed during normal execution, you want to do profiling. There are several profilers out there (eg Red Gate ANTS) or you can use your own homegrown solution. If your app is ASP.NET/ASP.NET MVC, You can also check out the MVC Mini Profiler made by the StackExchange team: http://code.google.com/p/mvc-mini-profiler/
NEW UPDATE
I'm confused by the comments. Code coverage is about unit testing.
From Wikipedia:
In computer science, code coverage is a measure used to describe the
degree to which the source code of a program is tested by a particular
test suite. A program with high code coverage has been more thoroughly
tested and has a lower chance of containing software bugs than a
program with low code coverage.
The title of the question asks how to "use Visual Studio Code Coverage without Unit Testing." There is no point in measuring code coverage if there are no unit tests.
I noticed there's an attribute Microsoft.VisualStudio.TestTools.UnitTesting.WorkItemAttribute available in visual studio testing (I'm using VS 2010 Premium and work items with TFS 2010.)
Marking a test method with a work item number sounds handy, but does it actually do anything? I can't tell if there's any tool support for it at all. I set one up like this:
[WorkItem(25788)]
[TestMethod]
public void TestSomethingSpecificToABug()
{
...
But no magic - I thought maybe the context menu on the test in the Test Results window might offer to open the work item, or Team Explorer might have a feature to search for tests. The MSDN documentation is no help either. What is this attribute good for?
According to "Software Testing with Visual Studio® 2010" by Jeff Levinson (Addison-Wesley Professional, February 2011, ISBN-10: 0-321-73448-3):
This also means that one existing property should not be used anymore:
Associated Work Items. This value is not reported to the data
warehouse and therefore cannot be used for reporting. If you currently
use this property, consider associating your test with an actual Test
Case work item type.
So the answer is, don't use this with TFS 2010.
WorkItem Test Method attribute is not used for associating test methods to test cases. It is typically used to associate a test method with a bug.
A related C# example from Code Index - How to discover ignored tests:
When using MSTest to build your suite of unit tests, you can use the
attribute [Ignore] to tell the MSTest engine not to run a test instead
of commenting it. You may also use the attribute [WorkItem(id)] to
link the unit test to a bug database (such as TFS) item, so that you
can trace why a particular test has been marked as ignored:
[Ignore]
[WorkItem(12345)] // bug 12345 describes why this test was ignored
[TestMethod]
public void IgnoredButWithWorkItemTest()
{
//The actual code is not important;
}
It's to link the unit test back to a work item in TFS. I'd provide a link to more info but it seems it really is poorly documented.
I haven't used it myself but believe it can be used to generate reports on the status of work items.
This isn´t needed anymore: in VS 2013 via CodeLens
Find linked work items (Alt + 7)
Find linked code reviews (Alt + 8)
Find linked bugs (Alt + 9)
To review a test's definition, double-click the test.
Oh! for those that cherish Lync:
Contact the owner of an item (Shift + F10)
I really do remember, that I used this Attribute before and the test results have been attached to the respective WorkItem.
However, with Visual Studio 2012, it does not work anymore, or I forgot which mechanism was actually responsible for the magic. Could it be that this only works through the build server?
Pulling attributes from the test binary is really useful when you have a home grown test harness built for running Selenium UI unit tests.
After a test failure, I can pull the WorkItemAttribute value using System.Reflection.MemberInfo.CustomAttributes and then look up the ID with the TFS API. If the work item is a bug and it's still active, I can auto-resolve the failure to that bug. In this manner, I can run the failing test every day and automatically resolve the failure to reduce randomization.