So our TFS 2013 builds do everything correctly except report code coverage. I've seen similar questions here, e.g. TFS 2013 - No Code Coverage Results, but we've already tried the suggested fixes with no results.
UPDATE 1 — we've taken more steps to try to resolve this; here's the complete list:
Ensured Visual Studio Ultimate was installed on the build server
Tried setting the build definition's "CodeCoverageEnabled" to true as well as the "Code coverage is enabled" setting
Ensured the build was using the Debug configuration and that PDBs were being generated for the DLLs to be tested
Added a .runsettings file with the correct ModulePath included under <CodeCoverage> (verified in build log that the module path was being interpreted correctly; it would produce errors if we intentionally malformed it)
Checked in .runsettings file
Set build definition to "Custom" and pointed to .runsettings file
The build process itself works fine. We can get code coverage results when we build the project locally in the IDE. On the build server, both MSTest and NUnit test projects run fine, and we see pass/fail results as expected. The "No Code Coverage Results" message still plagues us though.
Update 2 -
Here is what we see in the run log:
Somebody suggested homegrown code-coverage calculator in https://stackoverflow.com/a/16198120/141508, but it'd be a crime to spend $150 bazillion-thousand dollars on TFS 2013 & VS Ultimate 2013 with MSDN and still not have this one basic function working.
Add a run settings file to source control. Set the tests to custom and point to the run settings file. More info on using the .runsettings file can be found on msdn: http://msdn.microsoft.com/en-us/library/jj159530.aspx
I was experiencing the same problem. My issue was with the ModulePath. The MSDN examples suggest you can just use the name of a target binary. That was not working for me. However, when I made the name a regular expression, it worked. I am also dumping build output into one folder so that pdb and other reference files are found. Hope that helps.
<ModulePath>.*Administration\.dll.*</ModulePath>
I am using local build server with Visual Studio online with a .runsettings file and I had exactly the same issue.
None of the trickery above helped, so I tested the build script on the hosted build controller and it worked fine, so I decided the problem must be the build server itself.
I changed the Build Service account from "Network Service" to a regular windows user account in the TFS Configuration Tool and now code coverage is collected. Note that this user will need access to the TFS build directories.
I found this question because I saw something peculiar on this article. (Look for the "Delay" setting that defaults to 60).
d. Add a new argument ‘Delay’, enter details as mentioned below
Name – Delay, Direction – In, ArgumentType-Int32, Default Value – 60
This argument is required to delay coverage check, so that required build details are filled up by the build agent, this delay varies from
system to system, in some cases this might not be required at all.
http://www.prowareness.com/blog/failing-build-on-insufficient-code-coverageblock-coverage-part-3/
Maybe try putting in a "delay" work-flow item in the template you are using.......
Related
I'd like to ask for some help about setting up the DotTrace plugin for one of our TeamCity builds - so far I've followed the instructions here, and got a build that sometimes works:
https://blog.jetbrains.com/dotnet/2016/02/16/performance-profiling-in-continuous-integration-an-updated-dottrace-plugin-to-teamcity/
However, this is sometimes failing because the build produces two snapshots (snapshot.dtp and snapshot-2.dtp).
After much trial-and-error I've found out that sometimes the reason this fails is because my tests sometimes appear in the first .dtp file (which is the one used by DotTrace's Reporter.exe), and sometimes in the second file.
The other .dtp file appears to be associated with the nunit3-console.exe program, and so should be excluded, since I've added this program to "Process Filters" in the TeamCity build setup:
*nunit3-console.exe
(I've tried every combination here, including entering the fully qualified file path)
This has the effect of adding the following line to the dotTrace.project.xml file when the profiler is run:
<ProcessNameFilter>*nunit3-console.exe</ProcessNameFilter>
My understanding is that this should stop the named process from being profiled (so that the extra .dtp file isn't produced. However when the build is running, I see the following lines in the log file:
[16:01:04][Step 12/14] dotTrace Console Profiler 2018.3.2 build 777.0.20190124.112417. Copyright (C) 2019 JetBrains s.r.o.
[16:01:10][Step 12/14] Profiling in progress...
[16:01:10][Step 12/14] Executable='[file path]\nunit3-console.exe' (non-child) is being passed for filtering
[16:01:10][Step 12/14] Passed filter executable='[file path]\nunit3-console.exe'
This is showing that the nunit3-console.exe is not being excluded, as the filter would suggest.
The closest advice I can find is here, which appears to suggest what I'm doing, but is only for NUnit 2:
https://www.jetbrains.com/help/profiler/Profiling_Guidelines__Profiling_Unit_Tests.html
Does anyone know of a way to ensure nunit3-console.exe is excluded, so that it does not produce a snapshot file?
This plugin produces several snapshots by design and generates report based on all snapshots so filtering is not needed here.
I've created the issue
We have an established CI process which links 118 Ranorex test cases to MTM test cases and runs them on a dedicated automation VM, which is triggered by a timed build in Visual Studio.
The CI process is still working correctly, but I have just tried to add another test and found that due to changes in the layout of the test case work item page in VSO/MTM, the 'Associated Automation' function is no longer shown.
So you used to click on the ellipses and the list of test cases to be linked would appear: VS CodedUI MTM_VSO work item linking
As you can see in this image, the 'Associated Automation' tab has moved and the linking mechanism is no longer present: Missing MTM_VSO test case mechanism
I think it is tied into this recent Microsoft VSO update hxxps://www.visualstudio.com/en-us/news/2015-dec-10-vso (sorry for posting the link this way but I can't have more than 2 links) but I wondered if anyone else had encountered this issue and how to resolve it.
Sorry for the long post, but I wanted to make everything clear.
There is already a feedback submitted for this on Visual Studio page, refer to this link for details: https://connect.microsoft.com/VisualStudio/Feedback/Details/2137069
Currently, you can create the test case from your test assemblies via following steps first and then update the test cases.
Run "Developer Command Prompt" and navigate to the debug folder via "cd" command. For example: cd C:\xxx\xxx\solution\testproject\bin\debug
Create test case for the test assemblies via "tcm testcase" command: tcm testcase /collection:https:xxxxx/DefaultCollection /teamproject:yourproject /import /storage:xxx.dll
I have a problem which has received plenty of attention which, despite much googling, I have been unable to resolve. I have a Test Project attached to my Visual Studio 2010 MVC 2 Application. When I attempt to run my tests, I get:
"no tests were run because no tests are loaded or the selected tests are disabled."
Following this, I followed the instructions in these posts, to no avail:
MSTest: No tests are run because no tests are loaded or the selected tests are disabled = my GUIDs are right, and the assembly IS being built - I can see the file in explorer.
MSTest Not Finding New Tests = reopening the solution & rebuilding didn't result in any change
http://richallen.blogspot.com/2008/05/ms-test-re-enabling-ignored-tests.html = the tests do not even appear in the test list browser under "loaded tests"
Furthermore, the output console reports:
"Could not load file or assembly 'file://\shared\shared\IT\Development\TPS\TPS.Tests\bin\Debug\TPS.Tests.dll' or one of its dependencies. Operation is not supported. (Exception from HRESULT: 0x80131515)"
I have confirmed the build settings on the Test Project are the same as those for the main project (Any CPU targeted).
Therefore, is it a problem accessing the resource over network share? Otherwise, does anyone have a suggestion?
If you want to continue using a network share to host assemblies in .NET 4, you can change a Visual Studio configuration option to grant those assemblies full trust. You need to edit C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\devenv.exe.config and add the following line:
<loadFromRemoteSources enabled="true"/>
To the configuration/runtime element. This is described in (slightly) more detail at http://msdn.microsoft.com/en-us/library/dd409252%28VS.100%29.aspx. I would not recommend making this change without understanding the security implications of doing so, some of which are outlined in that MSDN article.
In general, though, I agree with the previous answer. Hosting Visual Studio projects on a network share is going to create a large number of problems for very little benefit.
Having your source code on a share is plain wrong (period) and will lead to all sorts of 'gremlins'.
Do yourself a favour, use Source Code Control and have a local copy of the source code. You will waste a lot less time, and as a bonus you will be able to track who changed what.
If you go with TFS, the Visual Studio TFS Branching Guide 2010 is a valuable resource.
If you use SubVersion, then the Red Bean Book is excellent.
I was running all local but still had the problem. I found that what caused it was removing an xml tag during manual editing.
add:
CodedUITest() inside of tag brackets
on the line immediately above your class where all your test methods are.
add:
TestMethod() inside of tag brackets
on the line immediately above your test methods you want to run.
build project and run.
I tried the following steps while encountering this problem and luckily the issue got resolved...
Close the solution and open the empty VS editor and Check out the testrunconfig file
Open the solution and under CodeCOverage column, uncheck and check the dll's available( these dll's would be having a warning symbol)
Rebuild the solution and now run the test cases.
Hope this resolves the issue... :)
I struggled with this for days, and didn't find the answer (for my situation) anywhere, so though I'd jot down my experience...
So I had the same problem, doing what I thought was local testing on a test project created locally.. (I'm a novice...) but returning the same error mentioned above : /
Anyway it seems that VS2010 had by default placed my project dir within the library folder, which was classified as network, subsequently all files within were 'unavailable offline'.
By moving my project dir to c:// my project files became index-able. (much to my relief!)
I am using Visual Studio Ultimate and I cannot get code coverage data. All of my unit tests pass, and I edited the Local.testsettings file to enable code coverage and configured it to reference the dll of my project. Also the active test setting is set to Local.testsettings and not TracAndTestImpact.testsettings.
I had the same problem. I configured my .testsettings to include code coverage as detailed here: http://msdn.microsoft.com/en-us/library/dd504821(VS.100).aspx
What I was doing wrong is that I was running the test by pressing F5 in Visual Studio.
What I needed to do was double-click the ".vsmdi" file, which will open the "Test List Editor". Then I clicked the "Run Tests in Current Context" - (Ctrl+R, T) in the Test Code command bar. Code Coverage worked after that.
Hope this helps.
Don't know if you still have this problem, but I had a similar issue and I've managed to fix it. I removed the testrunconfig file from my solution folder. Then added it back in and it came up with the settings dialog box, where I enabled the code coverage. Then I double clicked on code coverage and it came up with a list of dlls and I selected the ones that I needed. It saved the settings correctly and now when I run my tests, I can see the results in code coverage.
Hope this helps someone.
I had trouble getting this to work until I found out that the test project must use the 4.0 .NET Framework (or later).
I tried running the test from the test list rather than the testresults window and that fixed it for me.
I tried all the suggestions above, and in fact nothing worked until ... I restarted Visual Studio.
In Visual Studio, I just created a simple Unit Test to test a method I wrote. I wanted to check the code coverage, but the Code Coverage window only ever states that "Code Coverage is not enabled for this test run". Why? Note:
I've already followed the instructions on this page, i.e. the test configuration does have the relevant assemblies checked on the Code Coverage options page.
I've clicked "Run test", not "Debug test", which some sources claim causes this.
I've tried running the test with the current build configuration set to either "Debug" or "Release"; neither works.
I've also followed the suggestions on this page (re-select the testrunconfig file by selecting Test => Select Active Test Run Configuration), to no avail.
Yet I still get the message "Code Coverage is not enabled for this test run" every time I run the relevant test. How do I enable Code Coverage once and for all?
I used to struggle with MSTest, in a lot of areas (including code-coverage) - frankly, it* doesn't exactly go out of its way to make life easy - especially with the extra testrunconfig etc duplicating a lot of things already in the csproj.
Anyway; than I found that TestDriven.NET has this built in (as long as you have the right edition of Visual Studio to include MSTest and coverage):
(source: mutantdesign.co.uk)
This will use your existing test framework (including MSTest), but it will jump through all the usual hoops to get test coverage enabled, without you needing to mess with the configuration. It even works with the source-code colorization (red/blue untested/tested etc). Handy.
*=the tool itself, and the GUI integration
Take a look at this page and see if that helps:
Tips on Using Code Coverage in Visual Studio 2005
He had an issue even after setting the configuration file you mentioned and the issue was related with assemblies located on GAC.
If this is for a c++ project, ensure that you have Profiling enabled. Bring up your project properties -> Linking -> Advanced. Find Profile and set it to Enable Profiling information (/PROFILE).
Be sure to run your tests from the Test View or Test List window... not from the test results window. Re-running tests from within the test results window doesn't always honor your test run configuration settings.
In my case, when I challenged this problem, I had "no test run configuration available" under Test->Select Active Test Run Configuration menu.
I removed/added .testrunconfig file, and the code coverage came back.
(VS2005 Team Edition)