I'm using VS Code for a Go project. I have it configured to show the code coverage with gutter markings after I run the tests.
The issue I'm having is that if I make any changes to a file in the package, and then save it, VS Code then removes all of the coverage markings from all package files. I've gone through all of the Go settings and cannot find one that prevents this from happening.
There are options to re-run the tests on every save, which would then show the new coverage markings, but that's not feasible for this package because the tests take relatively long to run.
I'd like to find a way for it to keep the most recent test coverage markings when I save a file, instead of removing them all. Is this possible?
Related
We have a solution that contains a series of projects used for our tests. One of those projects contains a set of files that are needed for the tests. Those files are schemas that will get validated against every time an API route is called. Tests, naturally, call one or more API routes.
The solution has a .testsettings file. This file has deployment enabled, and it specifies that these schemas need to be deployed. In addition, every schema file is set to Copy Always. Also. the .testsettings file is in the solution, under Solution Items.
The problem is that the .testsettings file is only occasionally respected. Sometimes the files are copied; sometimes they are not. When they don't copy, we can do the following to fox it:
Go to the Test -> Test Settings menu and Choose Select Test Settings
Select the .testsettings file in the solution
Rebuild the solution
Rerun the tests
This usually works at least once. But inevitably, it stops working and the files aren't deployed again.
Note that when you go to the Test -> Test Settings menu, our current .testsettings file is always already checked. So choosing a new .testsettings file just means choosing the one that the UI already says is chosen.
We thought of going the DeploymentItem route, but this is impractical for a two reasons, surrounding code maintenance.
From what I can tell, DeploymentItem can only be placed on individual tests. With literally hundreds of tests, we'd be sprinkling it everywhere. It'd become a code maintenance nightmare. I thought of placing it on the global TestInitialize method, but that would just re-copy the files every time a test is run, which seems unnecessary. Not to mention that I'd have to put literally dozens of DeploymentItem attributes on the method and we'd need to keep that up-to-date every time a new schema is added.
Related to that, adding new schemas means altering existing tests where necessary. Again, we have hundreds of tests.
A far better solution would be to have the files copied over once, and then have the code look in the communal pool of schemas whenever it needs one.
I also looked at replacing .testsettings with .runsettings, but it doesn't seem to have a DeploymentEnabled node in the XML, and where that option exists it's again specific to DeploymentEnabled.
Does anyone have a solution to this, or does anyone know if it's a known bug? Schema validation happens behind the scenes -- individual test authors don't have to explicitly call it -- and it doesn't fail the test if it doesn't occur (we don't always have schemas available yet for every API call, so we don't want to fail the test if that's the case). As a result, it's often difficult to immediately ascertain whether or not validation happened. Because of that, we sometimes get false passes on tests where the schema actually broke, all because the .testsettings file didn't actually deploy our files like it's set to.
So I found out the problem: apparently this issue was fixed in Visual Studio 2015 Update 3. We were using Update 2. Once we got the new update, this problem went away.
I am following article for code coverage in VS 2013 http://msdn.microsoft.com/en-us/library/dd537628.aspx. It shows coverage of Test and not actual code.
I have selected unit test from Test explorer and click on Analyze Code Coverage -> Selected Test. Code coverage window is opened up at the bottom and It shows dll of unit test and its coverage and actual dll with code coverage is not shown.
Also, I looked into article http://msdn.microsoft.com/library/jj159523.aspx. Copied pdbs and dll into same location. In the code coverage window message was
Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings. For more information see http://go.microsoft.com/fwlink/?LinkID=253731"
Any idea about this?
I have tried steps given in the link.
http://blogs.msdn.com/b/allendm/archive/2012/09/08/empty-coverage-file-with-profiler-related-errors-in-the-event-logs.aspx
Slight change if you are using VS 2013.
a) Environment variable VS120COMNTOOLS is set to \common7\tools
b) Regkey HKLM\SOFTWARE\Microsoft\VisualStudio\12.0\InstallDir is set to your \Common7\IDE\
c) covrun32.dll and covrun64.dll exist in "\Team Tools\Dynamic Code Coverage"
Removed VS110COMNTOOLS environment variable. These changes worked for me.
I was able to resolve my issue. The reason why code coverage was not happening on my production DLL was: reason = "has_fixed_base". Once I changed this setting to: Yes (/DYNAMICBASE), the coverage started working.
It would help to check these steps:
Open Project Properties
go to Linker->Advanced page
In that, check Randomized Base Address
Change it to Yes (/DYNAMICBASE)
This should start code coverage.
Recently I migrated all my C# projects from VS2010 to VS2013. There were barely any code changes required but immediately after I checked in the projects, I noticed that CI started failing. On checking the issue, I realized that the coverage file generated after running unit tests was not being generated in the location specified in the .testsettings file. In the Test Results Names section of Test Settings in VS2010, I configured a user defined naming scheme and unchecked the Append date-time stamp checkbox. This would generate the coverage file in \\data.coverage. After migrating to VS2013, the coverage file is getting generated in a Test Settings folder which believe is the default location when the Default naming scheme is chosen.
On researching a bit, I found an MSDN post and a few other blogs posts suggesting doing away with the .testsettings file and adopting the new .runsettings file. The weird thing is Microsoft is recommending usage of the .runsettings file but don't seem to deliver a default one which I can build upon. In any case, I did create one based on the sample provided on the same MSDN post. It did work for me as far as creation of the coverage file is concerned but I again faced the same problem that it is getting generated in a Test Results folder. The bigger problem is that a sub-folder with some GUID type name is created into which the coverage file is dumped and it changes every time a test is run with code coverage ON.
I wonder if there's a way make my VS2010 settings work as expected in VS2013 or using the .runsettings file, can I provide the complete path where the coverage file should be created?
Go to Test->Test Settings->Select Test Settings File and select the .testsettings file associated with your test project. It's a hack but always works.
A little new to Javascript coding, so please bear with me.
I read through the following link
jasmine with jscoverage automated testing
However, since I don't have a Ruby project, it didn't seem to be what I wanted.
Here are the steps I executed:
Copy my js file to be tested to a directory 'input'
Run the following command 'jscoverage input output'
Copy my spec and src folders for Jasmine as well as SpecRunner.html into 'output'
Copy the instrumented source file into src
Open jscoverage.html in Chrome
Open SpecRunner.html in the 'Browser' tab
At this point, the browser page displays my Jasmine tests. However, the 'Summary' page shows 0 files tested and the 'Source' tab is grayed out.
I know I messed up somewhere but am not sure where. Please help me out. Thanks!
The correct steps to be followed are as follows:
Copy the jasmine files (spec and src folders for Jasmine as well as SpecRunner.html) to a directory 'input'
Run the following command 'jscoverage input output'
Open jscoverage.html in Firefox (Chrome will not show the individual files that were tested)
Open SpecRunner.html in the 'Browser' tab
This approach has the drawback that all the files including jasmine related js show up. But you can select your js file that's being tested and just look at the code coverage for it.
The standard jscoverage approach is to instrument the entire codebase you want to get coverage on, then run a suite a tests and generate a report. This approach is a bit heavy handed since a codebase only needs a coverage report on a set frequency. Unlike CI tests, how often does the dev really need to know the coverage percentage change, weekly?
The node.js jscoverage project uses the same 'instrumentation' approach as the larger jscoverage project, but it can be run from the node CLI on an individual file, or from code on one or more files, as they are called from the tests themselves. In place of separate step to 'instrument' a batch of files, the jscoverage (confusing name), 'instruments' at test runtime.
I have been working towards a pattern that allows jasmine tests and testing in either a browser or at the CLI using the same source code setup and test configuration. Still beta. The jasmine just-in-time 'instrumentation' is not done yet.
https://github.com/d1b1/jasmine-jscoverage
I've written a program that tells me whether all the dlls in a code coverage result file are above a specified code coverage percentage.
It works great except when a dll has 0% code coverage, because dlls with 0% code coverage aren't included in the visual studio code coverage result file at all.
I have access to a list of the dlls that were instrumented for this code coverage run, but I'd much rather just look at the code coverage file if I can.
Is there some way to get Visual Studio to add the code coverage results for dlls where the code coverage percentage = 0%?
Full disclosure: I am on the team that develops this feature.
Unfortunately, this information is not present in the code coverage file. When a binary is instrumented, we insert special probes to tell us that it exists and to detect when each code block is executed.
We do not save the list of DLLs in the target process, so we only know about instrumented DLLs that get executed (just being instrumented is not enough).
The easiest workaround is what you mentioned in your question.
Thanks for your feedback, though, we will take it into consideration. Sorry for the difficulty here.
The trick we're using is to add one Test Project with a special Test which calls a special public static method in every assembly that does nothing more than return true. That way the assembly is hit at least once and the one liner doesn't really impact the total coverage data.
I've used Reflection to do this in some projects, and now we're just adding the CodeCoverageTest project to every solution and the CodeCoverage.Ensure() method in every Assembly.