I noticed there's an attribute Microsoft.VisualStudio.TestTools.UnitTesting.WorkItemAttribute available in visual studio testing (I'm using VS 2010 Premium and work items with TFS 2010.)
Marking a test method with a work item number sounds handy, but does it actually do anything? I can't tell if there's any tool support for it at all. I set one up like this:
[WorkItem(25788)]
[TestMethod]
public void TestSomethingSpecificToABug()
{
...
But no magic - I thought maybe the context menu on the test in the Test Results window might offer to open the work item, or Team Explorer might have a feature to search for tests. The MSDN documentation is no help either. What is this attribute good for?
According to "Software Testing with Visual Studio® 2010" by Jeff Levinson (Addison-Wesley Professional, February 2011, ISBN-10: 0-321-73448-3):
This also means that one existing property should not be used anymore:
Associated Work Items. This value is not reported to the data
warehouse and therefore cannot be used for reporting. If you currently
use this property, consider associating your test with an actual Test
Case work item type.
So the answer is, don't use this with TFS 2010.
WorkItem Test Method attribute is not used for associating test methods to test cases. It is typically used to associate a test method with a bug.
A related C# example from Code Index - How to discover ignored tests:
When using MSTest to build your suite of unit tests, you can use the
attribute [Ignore] to tell the MSTest engine not to run a test instead
of commenting it. You may also use the attribute [WorkItem(id)] to
link the unit test to a bug database (such as TFS) item, so that you
can trace why a particular test has been marked as ignored:
[Ignore]
[WorkItem(12345)] // bug 12345 describes why this test was ignored
[TestMethod]
public void IgnoredButWithWorkItemTest()
{
//The actual code is not important;
}
It's to link the unit test back to a work item in TFS. I'd provide a link to more info but it seems it really is poorly documented.
I haven't used it myself but believe it can be used to generate reports on the status of work items.
This isn´t needed anymore: in VS 2013 via CodeLens
Find linked work items (Alt + 7)
Find linked code reviews (Alt + 8)
Find linked bugs (Alt + 9)
To review a test's definition, double-click the test.
Oh! for those that cherish Lync:
Contact the owner of an item (Shift + F10)
I really do remember, that I used this Attribute before and the test results have been attached to the respective WorkItem.
However, with Visual Studio 2012, it does not work anymore, or I forgot which mechanism was actually responsible for the magic. Could it be that this only works through the build server?
Pulling attributes from the test binary is really useful when you have a home grown test harness built for running Selenium UI unit tests.
After a test failure, I can pull the WorkItemAttribute value using System.Reflection.MemberInfo.CustomAttributes and then look up the ID with the TFS API. If the work item is a bug and it's still active, I can auto-resolve the failure to that bug. In this manner, I can run the failing test every day and automatically resolve the failure to reduce randomization.
Related
In my project I am using custom controls instead of normal ASP.NET controls. We have build an architecture on .NET and are using its controls.
Now I need to write a custom rule to check for if some of windows controls are being used. Reason being, my team needs to be limited to only my custom controls that were designed to replace the windows controls.
Example: I need to search and if they are using System.Windows.Controls.Textbox.....I need it to be an error.
Can anyone please help me out with code?
I hope the problem is clear ..... in case of any further clarifications needed please let me know.
The logic for this sort of rule is fairly simple:
Check method bodies, visiting each constructor invocation to see if
the target class inherits from the base Control class.
If it does, verify that the target class is in your namespace or assembly (or however you can best identify it as "yours").
This is relatively simple. A much bigger problem is that the relevant contructors will usually be invoked in designer-generated code, which most folks tend to prefer ignoring when executing FxCop. To get your rule to work, you will need to include designer-generated code in your analyses.
The tool NDepend let's write custom code rules on .NET code much more easily than with FxCop. Disclaimer: I am one of the developer of the tool
With this tool you can write custom code rules over LINQ queries (what is named CQLinq). For example, the query you are asking for can be written this way with CQLinq:
// <Name>Don't use system controls</Name>
warnif count > 0
let systemControls = ThirdParty.Types.Where(
t => t.DeriveFrom("System.Windows.Forms.Control".AllowNoMatch()))
where systemControls.Count() > 0
from t in systemControls
let methodsThatCreateT = t.TypesUsingMe.ChildMethods().Where(m => m.CreateA(t))
select new { t, methodsThatCreateT }
While editing such code rule, instantly a browsable result is shown (in 3 milliseconds here). Double clicking any type or method in this result, jumps to its declaration in source code in Visual Studio:
200 default code rules are proposed. The tool is 100% integrated in Visual Studio 2012, 2010, and 2008. Default or custom code rules can be validated inside Visual Studio, and/or at Build Process time, in a generated HTML+javascript report.
I am having a very frustrating problem in Visual Studio. I am using Resharper, and writing tests in Nunit.
If I set a breakpoint, it does get hit, however Step In/Over and Continue do not work, and the test never finishes. Even if I set two break points, continuing won't hit the second. If I debug a test without a breakpoint, it finishes fine.
Also, often and intermittently, when I try to debug a test by setting a break point, I can't evaluate the contents of variables, but instead see this message:
Function evaluation disabled because a previous function evaluation timed out. You must continue execution to reenable function evaluation.
I am using Visual Studio 2010.
Please let me know if you have any ideas of what to look at... I have scoured the web, but without any luck.
Happy to provide further information if needed.
EDIT - Example of method
Test:
[Test]
public void OneRowAddedToSourceData() {
//Factory just returns System.Data.DataTable with correct columns.
var sourceData = new DataTableContainerFactory().GetTargetTableContainer(DataTypeNames.EventSharedEnd);
//GetRow just returns a populated row.
var row = GetRow(sourceData, 123456, 123,60, 31);
sourceData.DataTable.Rows.Add(row);
Assert.AreEqual(1, sourceData.DataTable.Rows.Count);
}
When I set a break point on the Assert statement, and try to evaluate the data table I get this problem.
UPDATE 2!
So, I have narrowed this problem down to Resharper Debugging. I changed a test class to MSTest rather than NUnit, but still got both the above problems when debugging with Resharper. However, when I ran the tests with the built in VS MSTest test runner, ALL the problems had gone!
Now I much prefer using NUnit and Resharper, so I would be very happy if someone could point me at a setting that will allow Resharper to debug properly!
I have a possible solution, albeit with a caveat. Did some searching and ran across this blog item. It suggests turning off Enable property evaluation and other implicit function calls from Tools->Options->Debugging->General. I turned it off and I can now go into items that previously caused this problem.
The caveat is that all the properties you're used to seeing automatically evaluated are now not being shown. Instead it states Implicit function evaluation is turned off by user. All you need to do is hit the Refresh button on a property and you can see the value, but I haven't decided whether this is a trade-off I'm willing to make.
I've found de way to solve this problem!
Check the option DEBUG->OPTIONS AND SETTINGS->GENERAL->ENABLE PROPERTY EVALUATION AND OTHER IMPLICIT FUNCTION CALLS and uncheck the option Call string-conversion function on objects in variable windows
This worked to me!
MSTest + Visual Studio generates accessors which appear on the NCover coverage report. They usually have a low coverage in my case. I wonder if I use them in my tests if the method I'm testing will appear as covered in the Accessor but not in the original source code.
If this is the case, it is a problem and I need a solution.
The dev team was interested in this issue and created an MSTest private accessor project in VS 2010.
I ran coverage on it in NCover 3 with MSTest, and it appears that these accessors are reflecting the source code, not a generated version of it, at least in our test.
If you'd like to get the test project and try it out, just contact us at support#ncover.com.
Thanks!
If I understand your scenario, then any method called by your test will be shown as covered in the source code.
If the test adds accessors, those will also be shown as covered where they appear, but you can filter them out by excluding the get and set methods that are generated.
Here's a link to the syntax for the method exclude:
http://docs.ncover.com/ref/3-0/ncover-console/command-line/profiling-options#em
We've been successful here with the regex .*.get_.*, just as an example.
NCover Support
I'm currently dealing with a part-time (and annoying) bug with some extension methods in VB .NET (VS 2010 and VS 2008).
I have some client projects that use some extension methods from a "Framework" Project I created for general purpose.
However, sometime, when a try to start the solution, I get a long list of error saying the extension method "is not a member of" target object.
When this happen, I can rebuild the solution without getting any error but if I start the project again, the same error will be raised again.
So far I have 2 workaround
A) Go on the extension method definition, delete the tag, compile (with error), put the tag back.
B) Close Visual Studio et re-open it.
Is there any betterway to handle this problem?
There is an active bug report for this problem at Connect. There's a dire need for a repro, a project with specific steps to let them reproduce the issue in-house. Perhaps you can help, it's what it will take to solve it or recommend a workaround.
EDIT: one late addition to this answer, since I've posted this there have been additional questions that note a problem with the IntelliSense parser behaving different from the compiler. There's an issue with the multi-targeting feature, particularly in VS2010, triggered by the Client Profile option for .NET 4. It is the default for a new project. What goes wrong is adding an assembly reference that requires the full profile, commonly because of a dependency on System.Web. You'll get a warning for that, but that's quickly blown away with a large number of errors. The fix is to change the project's Framework Target from client to full.
I've recently encountered this same issue. My project would Build just fine but if I tried to use the Rebuilt option I would get the error about my extension methods not being found on the objects they were extending.
I have managed to fix this today by including an Import at the top of the classes using the Extension method.
So at the top of the file I actually have
Imports project_name.extension_module
I have no idea why this is required but it fixes the problem in my case. I hope this helps somebody else that has wasted as much time with this as I have!
Liam
A less time consuming work around than the two you listed is to "Rebuild Solution" instead of the normal Build Solution. This has always worked for me whenever I run into this problem.
We have something very similar. Call extensions merhod that extends type A. type is actually an instance of type B that implements interface C. interface C has a method with the same name as extension method.
Interface C declares method Foo
Class B inherits from A implements C
Static Class
Static method Foo (this A)
If A is C
(A as C).Foo
When debugging on the line if A is C it actually calls Foo multiple times and you can step at assembly level and see it indicates it is doing the comparison. The way we demonstrate that the call is happen Foo executes a sql statement and we can see sql profiler executing the code in Foo.
Change A as C to reflection and bug goes away. Or use debugger hidden/step through and bug is masked. Also if you micro-step the sql profiler shows the query is getting called multiple times (before Foo is actually called) for each assembly statement.
FxCops is something new to me, but as always I would like to get to know the new things..
From what I've read, FxCops is already included in VS2008. I guess it's the "Code Analysis" function. Whenever I try to run it though, it seems to start a rebuild and end in the "Finished Rebuilding" state.
I checked the output window and there are a bunch of warnings there. But if I'm not mistaking, there should be more of a GUI for this then the wall of text in my output window, right?
Am I missing a window that should have popped up? Can I open it somewhere? Or is there anything else I'm missing?
Yes, Code Analysis is the nice friendly name for FxCop. However, I'm not aware of a friendly window beyond the errors / warning list where they should appear, prefixed CA.
On the project properties screen there is a Code analysis tab where you can treat warnings as errors to enforce the rules you care about.
Just so everyone knows, because it took me a long time to figure this out.... Code Analysis / FxCop is only included in Team System and Team Suite versions of VS 2008, not in the Professional Edition.
You're not missing anything - there isn't a pop-up window.
The list of issues in the output window is pretty much all you'd get in FxCop. It's just that FxCop is a standalone application.
Here's a decent article on FxCop and Code Analysis:
Link
An alternative to FxCop would be to use the tool NDepend that lets write Code Rules over C# LINQ Queries (namely CQLinq). NDepend is integrated in VS 2012, 2010 and 2008. Disclaimer: I am one of the developers of the tool
More than 200 code rules are proposed by default. Customizing existing rules or creating your own rules is straightforward thanks to the well-known C# LINQ syntax.
NDepend code rules can be verified live in Visual Studio and at build process time, in a generated HTML+javascript report.
You seems concerned by the number of false-positive. To keep the number of false-positives low, CQLinq offers the unique capabilities to define what is the set JustMyCode through special code queries prefixed with notmycode. More explanations about this feature can be found here. Here are for example two notmycode default queries:
Discard generated and designer Methods from JustMyCode
Discard generated Types from JustMyCode
To keep the number of false-positives low, with CQLinq you can also focus rules result only on code added or code refactored, since a defined baseline in the past. See the following rule, that detect methods too complex added or refactored since the baseline:
warnif count > 0
from m in Methods
where m.CyclomaticComplexity > 20 &&
m.WasAdded() || m.CodeWasChanged()
select new { m, m.CyclomaticComplexity }