MSTest + Visual Studio generates accessors which appear on the NCover coverage report. They usually have a low coverage in my case. I wonder if I use them in my tests if the method I'm testing will appear as covered in the Accessor but not in the original source code.
If this is the case, it is a problem and I need a solution.
The dev team was interested in this issue and created an MSTest private accessor project in VS 2010.
I ran coverage on it in NCover 3 with MSTest, and it appears that these accessors are reflecting the source code, not a generated version of it, at least in our test.
If you'd like to get the test project and try it out, just contact us at support#ncover.com.
Thanks!
If I understand your scenario, then any method called by your test will be shown as covered in the source code.
If the test adds accessors, those will also be shown as covered where they appear, but you can filter them out by excluding the get and set methods that are generated.
Here's a link to the syntax for the method exclude:
http://docs.ncover.com/ref/3-0/ncover-console/command-line/profiling-options#em
We've been successful here with the regex .*.get_.*, just as an example.
NCover Support
Related
I have the exact case as in this post:
How to set the test case sequence in XUnit
Which points me to this link:
http://www.bricelam.net/2012/04/xunitnet-extensibility.html
Followed the snippet of code in the post, the one using RunWithAttribute.
But as one of the comments in the post pointed out,
once I annotate my test class with [PrioritizedFixture] all of my test classes are no longer recognized by the xUnit Runner in Visual Studio.
Is there anyway to make this work?
*Using VS2013 and xUnit 1.9.2
According to this article, one can make a parameterized test in the GoogleTest framework with some code like this:
INSTANTIATE_TEST_CASE_P(InstantiationName,
MyStringTest,
::testing::Values("meek", "geek", "freek"));
TEST_P(MyStringTest, acceptsEekyWords)
{
ASSERT_TRUE(acceptName(GetParam()));
}
plus some scaffolding.
After going through the CxxTest User Guide, I couldn't help but notice the lack of any mention of parameterized tests. Are parameterized tests even possible with CxxTest?
This question seems to address something similar, but the answer is by no means trivial.
I'm new to C++ unit testing. Maybe parameterized tests aren't a big deal? Almost all of my tests were parameterized in my last C# NUnit project.
As I wrote in my answer to the other question you cited, there's no facility for generating tests at run time, which is what any parameter-based tester I've ever seen does. In CxxTest, the test list is determined at compile time by the Python-based C++ parser, cxxtestgen. You can generate suites dynamically, but you only have the option of generating zero or one copy of any suite.
Is that possible to use Visual Studio Code Coverage without Unit Testing? I would like to make a coverage analysis within a normal program execution.
UPDATE:
There are no current solutions to use the Visual Studio Coverage tooling for what I want, although dotCover seems to be a nice third part solution for the problem.
Here's a more detailed answer rather than just a link:
To do this for already instrumented files with an IIS Express application:
Get the name of the site from C:\Users\<your user>\Documents\IISExpress\config\applicationhost.config
vsperfcmd /start:coverage /output:run.coverage
launch your app
run your manual tests
then to finish
vsperfcmd /shutdown
from related question https://stackoverflow.com/a/23791306/57883
for a full walk through, here's a link to the blog article I just posted on it:
http://imaginarydevelopment.blogspot.com/2015/02/get-code-coverage-from-vs-without.html
I think you may be misunderstanding what Code Coverage is. Code Coverage indicates how much of your code is exercised by your unit tests. If you have no unit tests, you have zero code coverage. Are you perhaps referring to code profiling (measuring how long it takes for code units to execute?)
UPDATE:
If you're looking for metrics on what code is executed during normal execution, you want to do profiling. There are several profilers out there (eg Red Gate ANTS) or you can use your own homegrown solution. If your app is ASP.NET/ASP.NET MVC, You can also check out the MVC Mini Profiler made by the StackExchange team: http://code.google.com/p/mvc-mini-profiler/
NEW UPDATE
I'm confused by the comments. Code coverage is about unit testing.
From Wikipedia:
In computer science, code coverage is a measure used to describe the
degree to which the source code of a program is tested by a particular
test suite. A program with high code coverage has been more thoroughly
tested and has a lower chance of containing software bugs than a
program with low code coverage.
The title of the question asks how to "use Visual Studio Code Coverage without Unit Testing." There is no point in measuring code coverage if there are no unit tests.
I noticed there's an attribute Microsoft.VisualStudio.TestTools.UnitTesting.WorkItemAttribute available in visual studio testing (I'm using VS 2010 Premium and work items with TFS 2010.)
Marking a test method with a work item number sounds handy, but does it actually do anything? I can't tell if there's any tool support for it at all. I set one up like this:
[WorkItem(25788)]
[TestMethod]
public void TestSomethingSpecificToABug()
{
...
But no magic - I thought maybe the context menu on the test in the Test Results window might offer to open the work item, or Team Explorer might have a feature to search for tests. The MSDN documentation is no help either. What is this attribute good for?
According to "Software Testing with Visual Studio® 2010" by Jeff Levinson (Addison-Wesley Professional, February 2011, ISBN-10: 0-321-73448-3):
This also means that one existing property should not be used anymore:
Associated Work Items. This value is not reported to the data
warehouse and therefore cannot be used for reporting. If you currently
use this property, consider associating your test with an actual Test
Case work item type.
So the answer is, don't use this with TFS 2010.
WorkItem Test Method attribute is not used for associating test methods to test cases. It is typically used to associate a test method with a bug.
A related C# example from Code Index - How to discover ignored tests:
When using MSTest to build your suite of unit tests, you can use the
attribute [Ignore] to tell the MSTest engine not to run a test instead
of commenting it. You may also use the attribute [WorkItem(id)] to
link the unit test to a bug database (such as TFS) item, so that you
can trace why a particular test has been marked as ignored:
[Ignore]
[WorkItem(12345)] // bug 12345 describes why this test was ignored
[TestMethod]
public void IgnoredButWithWorkItemTest()
{
//The actual code is not important;
}
It's to link the unit test back to a work item in TFS. I'd provide a link to more info but it seems it really is poorly documented.
I haven't used it myself but believe it can be used to generate reports on the status of work items.
This isn´t needed anymore: in VS 2013 via CodeLens
Find linked work items (Alt + 7)
Find linked code reviews (Alt + 8)
Find linked bugs (Alt + 9)
To review a test's definition, double-click the test.
Oh! for those that cherish Lync:
Contact the owner of an item (Shift + F10)
I really do remember, that I used this Attribute before and the test results have been attached to the respective WorkItem.
However, with Visual Studio 2012, it does not work anymore, or I forgot which mechanism was actually responsible for the magic. Could it be that this only works through the build server?
Pulling attributes from the test binary is really useful when you have a home grown test harness built for running Selenium UI unit tests.
After a test failure, I can pull the WorkItemAttribute value using System.Reflection.MemberInfo.CustomAttributes and then look up the ID with the TFS API. If the work item is a bug and it's still active, I can auto-resolve the failure to that bug. In this manner, I can run the failing test every day and automatically resolve the failure to reduce randomization.
I have CLI/MFC application and I would like to begin to learn how to unit test with it. I have VS2008 Pro. If possible, I would like to use the built in unit testing. Thanks for your input!
I've had success with both CPPUnit and Google Test. For either you have to do a bit of work to get the test results to integrate back into Studio. The granularity of the results you want directly affects how much work. Do you want a pass/fail for the whole test set, or individual results? The former is a simple msbuild task, the latter requires outputting the result set to XML, massaging that with a transform, then pulling it back in.
We use Gallio & MbUnit to test our MFC & C++/CLI application. Simply write the tests in C++/CLI then you can test both managed and unmanged code in a single framework. We also use NMock2 for mocking the managed code.
If you have Team Test edition, you can use it to test C++/CLI applications and libraries. See here.