Code coverage in TFS build templates - visual-studio

I have a build template which was created with VS2010 or lower version of VS. Now, I have VS2013. I want to enable code coverage for that build, but I do not see such option as in this page:
http://www.codewrecks.com/blog/index.php/2013/06/08/code-coverage-during-tfs-2012-build/
If I select the default build template when creating a new build definition - then I see the option to enable code coverage.
Am I missing something?
Another question if I should re-create (with VS2013) all my custom build templates that were created with VS2010 or lower versions in order for everything to work fine and in order too see the features such as code coverage?

You should recreate your build in 2013 and make sure that you only use custom PowerShell. The new build system in TFS 2015 does not use xaml builds.
If you use PowerShell to customize you will minimize conversion cost.

Related

Can I Make Builds Break Based on ReSharper Rules in EditorConfig?

Lately I have tried to get all quality and style rules for my team into an .editorconfig file. As ReSharper still has a larger scope of style rules than .NET Analyzers and other such things, and as it has grown to embrace the EditorConfig approach, I have exported all of our ReSharper rules to the .editorconfig file, and deleted our team's DotSettings file. I've been attempting to make almost all rules have a minimum severity of warning, and to have warnings be treated as errors. This goes for ReSharper and the standard VS rules. My end goal is to have our builds break, both locally and on our Azure DevOps Pipelines, if any of the rules in the .editorconfig, from ReSharper or otherwise, are violated.
So far, I have been unable to get ReSharper-specific errors or warnings to break the build in any of the following:
Visual Studio build (MSBuild)
dotnet build (local)
ReSharper Build
dotnet build (Azure DevOps YAML Pipeline)
Now I'm starting to suspect that the ReSharper rules are really only for assistance at design time, but that there are no mechanisms to make them actually cause a build to fail. I'm hoping I'm wrong and that I just need to change a configuration or install some kind of add-on locally and/or on Azure DevOps.
As you already suspected, some of your criteria cannot be met ootb, namely:
Visual Studio build (MSBuild)
dotnet build (local)
dotnet build (Azure DevOps YAML Pipeline)
This is because R# is an IDE feature and low(er)-level tools like MSBuild or the .NET global tools are simply unaware of R#, its rules, etc.
I think one had to write a Roslyn analyzer to enable this because this is Microsoft's intended way of integrating custom code checks into the build pipeline.
Another option would be to use InspectCode out of the R# Command Line Tools. If you integrate this tool into your build pipeline, it will fail when specific R# rules are violated.
But you should be able to make builds fail using R# Builds when setting the severity of all your relevant rules inside .editorconfig to error.

How can I consistently automate, using TFS vNext build steps, building whatever solution files our development teams get working using Visual Studio?

Developers use the Visual Studio (VS) GUI to develop their solutions and get their projects all building using a solution file (.SLN). The build teams using vNext then have to automate those builds using MSBuild instead of devenv.exe (the Visual Studio executable file). This is a serious and chronic problem because not only is MSBuild incapable of building several project types, but the build order is defined in a completely different, and complex, way.
Some Microsoft advice (https://learn.microsoft.com/en-us/archive/blogs/msbuild/incorrect-solution-build-ordering-when-using-msbuild-exe) is to switch to explicit dependencies in each .*proj file and remove all dependency specifications in the .SLN file. This sounds like a person who has never worked in a relatively powerless build team trying to get development teams to:
do a lot of what they perceive as time-wasting extra work and to
change how they do things
What build teams need is a way to automate whatever VS allows dev teams to build. If VS is given a SLN to build, then a vNext build needs to be able to use that same SLN in the same way. Instead vNext currently only offers MSBuild as the build tool. MSBuild has many more options than devenv, so that would be great, IF it could be made to use the SLN to govern dependencies in the same way as VS, and would be upgraded to build all the same project types.
There have been prospective efforts, referenced by PatrickLu-MSFT at Build project using devenv.exe in TFS 15RC1 Build Server, to enhance a vNext build step to allow devenv to be used instead of MSBuild, but those efforts seem to have been dropped.
Maybe someone has developed a custom vNext build step to build using devenv?
Here is an existing extension you can reference, which provides a build task in your TFS/VSTS instance that will allow you to build your projects using DevEnv.com (Visual Studio):
https://marketplace.visualstudio.com/items?itemName=mmajcica.devenv-build
If you want to automatically use TFS/DevOps build whatever solution files our development teams get working using Visual Studio, you could set CI trigger in build pipeline, when the solution build successfully on local, you can check in/commit the changes, and trigger TFS/DevOps builds.

Run Code Analyzers in VSTS project build

In VSTS (hosted TFS) I have a build definition which uses MSBuild. What I would really like is to have pull requests annotated with any rule violation detections. However I can't get the analysers to work at all. What I've done so far;
Installed Microsoft.CodeAnalysis.FxCopAnalyzers in one of the projects that gets build
Verified that running "Analyze Code" in Visual Studio does output rule violations
All the *.ruleset files are not available on the self-hosted build agent, as it only has MSBuild installed (no full-fledged Visual Studio). For the same project as in point 1, I've also used a custom ruleset.
I have also build the project on a hosted build agent (which has VS2017 installed), but also to no avail.
With all of the above, the build log / build overview in VSTS doesn't list any rule violations. It's my understanding that the code analysis should be run automatically and the output should appear on the build overview page (i.e. https://MYSITE.visualstudio.com/MYPROJECT/_build/index?buildId=XXX&_a=summary&tab=details).
The Code Analysis setting is based on the configuration and platform, so you need to make sure you build the project with the same configuration and platform that enabled code analysis on build.
You also could specify it in MSBuild Arguments of MSBuild task:
/p:RunCodeAnalysis=true;CodeAnalysisRuleSet=..\RuleSets\MyMixedRecommendedRules.ruleset

TFS 2017 - how to run a build with code coverage?

Hi I'm trying to run a build with code coverage, I am using Visual Stuido 2017 enterprise I have tried to enable the code coverage in VS but if i click on Edit build definition in team explorer it opens the build definition in TFS.
Apologies for the delay in posting this as an answer.
As a prerequisite to using Code Coverage, the first thing to do is to install Visual Studio Enterprise on the build agent (Which you have already done). Following this, you must then update your builds to specify that Code Coverage is/should be enabled.
To enable code coverage, ensure you have included a Visual Studio Test task (or any equivalent which supports Coverage) in your build definition.
Once you have added the Test task, you'll want to configure this task and set your test assemblies, for this you can use an absolute path to point to a DLL file, or use wildcards like I do here
$(build.sourcesDirectory)\Source\Tests**\Release*Test*.dll
This says that in my sources directory I have a folder called Source, and inside that is a Tests folder which contains all of my unit tests. Then in each Unit Tests folder I use a wildcard to say "any folder at all that contains a Release folder", look inside and take the DLL which has 'Test' in its name.
So now that the testing assemblies are gathered, you can tick the checkbox that says "Code Coverage Enabled", and you're good to go.
I will edit this post later today to include screenshots of my own personal VSTS builds.

Integrating MSBuild into Visual Studio

I'm a solo developer running Visual Studio 2008 and looking into MSBuild to improve my build process.
Almost all of the tutorials I've found so far have plenty of information about writing a build file. However I'm having a lot of trouble finding out how to integrate MSBuild into Visual Studio. Maybe MSBuild is only used with something like CruiseControl but that's overkill for me as a single developer.
Where should the build file live in a Visual Studio project and how can I run it from within the IDE?
Visual Studio executes MSBuild automatically for projects it supports.
If you right click on a project and unload it, you can then edit it in Visual Studio. Reload (right click on project again), force a (re)build to test your changes. An alternative is to edit the project file in an external editor and Visual Studio will detect saves and offer to reload the project for you.
Sounds like you're on the right track, and if you are considering writing Targets or custom MSBuild Tasks, take the time to separate them from your current project so that you can re-use them. Don't re-invent the wheel though, the two main complementary MSBuild projects are MSBuild Community Tasks and MSBuild Extension Pack.
Update: Judging from your comment on Mitch's answer, you might also want to consider adding a new Configuration element or custom properties to a project. A new MSBuild Configuration (something other than the default Debug/Release) could run unit tests, build documentation, or whatever you want automated. A custom MSBuild property would allow you to use normal Debug/Release Configuration and extend it to automate more of your build process, just depends on what you want. Either approach could also be driven from the command line.
As others have noted, MSBuild is already available when you install Visual Studio.
If you want to integrate into VS2008: Running MSBuild from Visual Studio
MSBuild is the build engine used by Visual Studio to process the files included in a project.The Visual Studio project files themselves (**.csproj* for C#, and .vbproj for VB, for example) are in fact MSBuild scripts that are run every time you build a project.
Your .csproj file is a MSBuild file. So you are actually using it already.
You may of course wish to create a separate build file to have more control, especially within a continuous integration or nightly build say.
If you simply wish to edit your project build file then you can use the IDE to edit some settings such as pre and post build actions or edit the Xml itself by unloading project and right click and editing.
You can use your current .vcproj files to build your project with MSBuild. However, as MSBuild is not directly supported (at least for vc++) vcbuild is used instead (internally).
In VS2010 all project files are MSBuild based...
This is an older article about some simple extension points from the msbuild team
How To: Insert Custom Process at Specific Points During Build
Also, don't forget you can use the MSBuild SideKick for developing and debugging your (local) msbuilds, available for free at http://www.attrice.info/msbuild/
I'd suggest you call msbuild as a post build step. Then you can put your build script somewhere in your solution and call it.
<windowsdir>\Microsoft.NET\Framework\v3.5\MSBuild.exe c:\temp\MyProject\mybuildfile.proj
The easiest way is probably to invoke your custom build script using a post-build step. Right click project, choose "Build Events" and call msbuild with your custom msbuild file from there.
I use the msbuild template to intergrate with visual studio
http://msbuildtemplate.codeplex.com/

Resources