How can I make msbuild understand that a .csproj file depends on the availability of an exe built by another .csproj - visual-studio

I have a large VStudio solution whose .csproj files I'm porting over to the new .NET SDK format. I have integrated the github project with AppVeyor. When I work on the solution, my project is built by VStudio; AppVeyor uses msbuild to build my project.
All of the class library project files in the solution have been modified so that they specify <TargetFrameworks>netStandard2.0</TargetFrameworks>. For simplicity, let's represent the class library project files with classlib.csproj.
The solution also contains a few command-line tools, which need to be built in advance of some of the other projects, because those later projects need to run the command-line tools as a preprocessor before building themselves. These command-line projects are all specifying <TargetFrameworks>net472;netcoreapp2.0</TargetFrameworks>. Let's represent the tool project files with tool.csproj.
In order to build correctly, tool.csproj needs to be built before classlib.csproj. In the solution I've used the Build dependencies > Project dependencies command in the VStudio solution explorer to indicate this. This works fine when I'm working inside of VStudio.
However, when I push my changes to github, AppVeyor kicks off an msbuild process to build the projects. From the error spew, it's clear that the tool.csproj didn't get built in time. It seems msbuild doesn't understand the project dependency information in the .sln file.
Reading what's posted on the intertubes (e.g. https://devblogs.microsoft.com/visualstudio/incorrect-solution-build-ordering-when-using-msbuild-exe/), I attempted to add the following to classlib.csproj:
<ProjectReference Include="..\tool\tool.csproj">
<ReferenceOutputAssembly>false</ReferenceOutputAssembly>
</ProjectReference>
But when I do this I get the error:
Project '..\tool\tool.csproj' targets 'net472;netcoreapp2.0'. It cannot be referenced by a project that targets '.NETStandard,Version=v2.0'
which clearly makes sense.
Some other solutions I've read suggest editing the .sln file so that tool.csproj gets built first, and hope that building will be completed before classlib.csproj starts building. However, this is a race condition since msbuild is not aware of the dependency.
How, then, do I express that classlib.csproj depends on tool.csproj so that msbuild understands it?
EDIT
It turns out the issue was caused by an interaction between the target I'd written to start my tool and the globbing operation MSBuild does to resolve the <Compile> item collection using a pattern similar to **\*.cs. The tool generated the file after the globbing operation. Since the generated file isn't present when the globbing happens, it is missing from the <Compile> item collection, and the build failed. I mistakenly interpreted the error "foo.cs not found" as a failure to execute the tool, when in fact, it's a timing issue.

To make the build order works outside VS IDE, we need to use add reference
instead of BuildDependencies=>ProjectDependencies.
Right-click classlib project and choose add=>reference, in Project node choose the Tool Project you want like this:
Click ok to add it and save changes. After that, open the classlib.csproj you can find sentence like:
<ItemGroup>
<ProjectReference Include="..\xxx\Tools.csproj">
<Project>{6eaa430f-9793-4639-a84b-6ab767d57147}</Project>
<Name>Tools.csproj</Name>
</ProjectReference>
</ItemGroup>
This what the msbuild can understand. And I think that's what you want.
After that,you can use a single msbuild tool or Appveyor to check the
build order.
And in Appveyor, we can also disable the paralell builds(settings=>build=>msbuild options) to make sure it won't build classlib.csproj until the tools.csproj build ends.
The final build order in Appveyor will be what we defined in VS. Hope it helps.

Related

What is the easiest way to execute restore of nuget packages on build outside solution folder

Set up
I have a solution Commons.sln which contain common utils for other more specific projects. Let's take for future example one specific solution called Cars.sln just to make it easier to explore the case. All solutions have their own set of NuGet packages.
All projects are laying in the same VCS branch therefore folder structure is pretty strict.
What I want to achieve?
I would like to have references for common libraries in all specific projects. Accordingly, I have added projects from Commons.sln to my Cars.sln. However I met an issue with this setup because people from my team who never built Commons.sln do not have NuGet packages from this solution downloaded to their machines and when the try to build Cars.sln - they face an error saying that dlls for projects from Commons.sln are not found.
Long story short - I would like to execute nuget restore ../../Commons/Commons.sln before build, but do not know how to do it better on Cars.sln solution level on every build/rebuild in Visual Studio.
Specifics
I have done a little research on this and there are my bullet points:
I do not want to commit any of my NuGet packages in VCS
I so a solution with adding a .target file per solution but it works only when I build directly with msbuild.exe.
In other SO question. There is a recommendation to add one more project and resolve it by build order which seems to be hacky to me.
Specific projects are bound to CI server so this stucture should work there as well (currently I have solved by adding a build step that builds Commons.sln before everything else)
Any suggestions are appreciated.
Currently I have fixed it with adding a msbuild target to the project in Commons.sln which is a dependency to all projects including specific ones, like Cars.sln:
<Target Name="Test" BeforeTargets="BeforeBuild">
<Message Text="Restoring Common Nugets" Importance="high"></Message>
<Exec Command="nuget restore "$(ProjectDir)..\Avid.Common.sln"" />
</Target>
However if somebody has a better solution I would be glad to know about it.

Migrating from sonar-runner to MSBuild Runner. Where did the sonar-project.properties file go?

Scenario:
I am migrating our current VS Solution analysis setup from using the sonar-runner to using the MSBuild runner. However I am encountering a fairly significant problem.
In the old setup, we specified our project name, key and most importantly a long list of skipped projects (sonar.visualstudio.skippedProjectPattern) using the sonar-project.properties file.
This is because [WARNING: ugly legacy bad coding practice alert] we have six solutions that build dozens and dozens of projects, all out of the same git repo. A lot of the projects are common across several solutions and we don't want them analyzed more than once. So each solution has a set of projects that it "owns" and which are analyzed as part of it. Thus the sonar-project.properies file for each of the other solutions specifies that these projects are to be ignored.
The Problem: In the new MSBuild Runner approach, there does not appear to be MS solution level (also read as SonarQube Project level) configuration file or mechanism aside from passing arguments on the command line to the MSBuild runner's 'begin' phase. One either has the global configuration file, or the MSBuild *.*proj files, (that is, MS project level configuration files). This latter is clearly out of the question as whether a project gets excluded from analysis is based on which solution is being analyzed.
As noted, conceivably we could pass all this in on the command line but that is sub optimal. Our builds are done by scripts that are, to the extent possible, generic. Having the configuration in the sonar-project.properities file was a big help in keeping them that way and we are hoping we are missing something here that will let us keep using that file or a similar one. Are we?
There currently is no equivalent to the sonar-project.properties file in the MSBuild SonarQube Runner version 1.0. I've added a new ticket to the project's backlog to consider adding this feature in an upcoming release: http://jira.sonarsource.com/browse/SONARMSBRU-124
The v1.0 MSBuild SonarQube Runner supports a /s: command line argument that allows you to specify the global settings file to use. The settings file can contain any additional global settings that previous you would have put in the sonar-project.properties file.
If you don't specify a global setting file the MSBuild Runner will look for a default global settings file in the same location as the runner executable.
See the documentation repo for more information: https://github.com/SonarSource/sonar-.net-documentation/blob/master/doc/appendix-2.md
The properties now can be added via ItemGroups in each .csproj file, this way:
<ItemGroup>
<SonarQubeSetting Include="sonar.cpd.exclusions">
<Value>Models/**/*.cs</Value>
</SonarQubeSetting>
</ItemGroup>

TeamCity Using OctoPack - Isn't Excluding Superfluous Files

I'm just looking at streamlining the nuget packages that are coming out of my build system and I'm stuck on how to only package the files that are required.
I have several configurations sharing a Root VCS checkout. I have a configuration that runs a debug build with unit tests. I also have a release configuration that does a release build, this configuration then also uses the TeamCity OctoPack plugin to create the nuget packages.
What I want to achieve is the building of nuget packages that don't contain the *.pdb and *.xml documentation files as these aren't required for the release deployment.
I've looked through this page on the OD site:
http://docs.octopusdeploy.com/display/OD/Using+OctoPack
And according to this page OctoPack should only package up the required files by default. I'm not entirely clear on how or what needs to be done to get around this problem as it doesn't appear to be working as described.
It seems that one solution would be to provide a nuspec file for the projects I'm looking to deply but I'm also wondering if there is something I'm missing before I head off down that route.
I also have some MEF plugins that are copied in post build events and these aren't included in the nuget packages when in fact they are needed for the application to run. I think I need to get explicit with a nuspec file but would like to confirm this.
What is the simplest way of achieving what I need?
Assuming you're running the later versions of OctoPack, in your release build you can set a system parameter system.DebugType = None which will get passed to the OctoPack build scripts and prevent the PDB's being created.
This simply overrides the setting defined in your csproj msbuild file (assuming C#), so you can use it wherever you want to prevent PDB's being created at the build configuration level (not just OctoPack). I generally prefer this approach as it prevents side-effects in your build from changes by developers in the project file.
As for the xml files, I haven't actually tried this, but you can try a similar approach and create a system parameter system.DocumentationFile = "" to blank out the output.
If you really want to make sure that the files have been removed there are a couple of ways you can do this. Modify your deployment process to:
Execute your own custom PowerShell script in that removes the files
Include a script module from the Octopus Library to the same. Check out the File System - Clean Directory from the Octopus Library

Cannot automatically update a NuGet package to the latest version during build

We have two separate .NET solutions:
Running a build for the first solution produces our end product: a bunch of DLLs. These DLLs are delivered to our customers via a NuGet package.
The second solution serves as a product-test solution: the NuGet package is installed to it, and it is built and executed - thus it makes use of our product exactly the same way as our customers would do.
The challenge here is that there should be a way our latest NuGet package gets installed automatically to the product-test solution, preferably during the build of this product-test solution.
Based on the ideas from a similar question, I got this far with configuring the product-test solution:
First I enabled NuGet Package Restore. This lets me get rid of the "packages" directory completely from VCS since the package with the version defined in packages.config file would be downloaded automatically by NuGet before build.
Then I added the following pre-build event in Visual Studio: $(SolutionDir).nuget\nuget update -prerelease $(ProjectDir)packages.config. This lets me pull in the latest version of our NuGet package during build.
I currently use the above scenario to run local builds using Visual Studio and unattended builds using TeamCity. The solution seems to work for both scenarios on the first sight, but actually it does not produce the expected result: when the product-test solution is built, in the bin directory I don't get the latest version of the DLLs, only the latest-1 version.
The problem is that although the nuget update command updates everything as expected, including the packages.config and the .csproj file, their new content is not picked up by the build, therefore - as my guess goes - the HintPath settings from the .csproj file still reflect a "before build" state, therefore old DLLs are copied to the bin directory. I assume the .csproj file is processed only once: before the pre-build event is triggered, and the changes made by the pre-build event are ignored until the next build.
I considered the following solutions:
Apparently pre-build is not "pre" enough. If there was an even earlier point I could insert the nuget update command, my above solution would probably work.
I read that I could override the HintPath-s in the .csproj file by defining a ReferencePath. But I doubt I could easily figure out the right path or I could set it early enough so the build picks it up.
As a workaround I could run the builds twice: duplicate the build step for the product-test solution in TeamCity and I could always build the solution twice locally in Visual Studio.
Has someone figured out how to automatically update a NuGet package to the latest version during build?
Have a look at a blog post I just did regarding this process. Instead of configuring stuff on the server, I did it by augmenting the Nuget.Targets file that's put in place with the Nuget Package Restore option. Beauty of this approach is it executes locally AND on the server (so you get to see any possible side-effects before breaking the build)
Posted details here: http://netitude.bc3tech.net/2014/11/28/auto-update-your-nuget-packages-at-build-time/
I think that put automatic update to pre-build step it's not NuGet style. You can understand that you need this operation every time when you do it. Mostly because it can increase build time. For example when you use TDD and often rebuild project and run tests, you expect that it will be done quickly. Additionally it can update unexpected packages and break something, after that you can spend a lot of time to find the problem.
I suggest to do update as the separate step. On TeamCity you can use NuGet installer build step. For performing update just check two checkboxes in the bottom area of step configuration:
Additionally if you want keep result of update after successful build and passing test, you can add later build step which commits this changes to VCS (using cmd or PowerShell for example).
When you are working locally, I thing the better way run update packages once before you start working with project. You can use Package Manager Console for this, with command Update-Package -IncludePrerelease.
The MSBuild-solution I figured out is to override the BuildDependsOn property. Therefore, I created an UpdateNugetPackages.target which looks like this:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<BuildDependsOn>
UpdateNugetPackages;
$(BuildDependsOn);
</BuildDependsOn>
<UpdateCommand>"$(SolutionDir)pathToYourNugetExe.exe" update "$(SolutionDir)NameOfYourSolution.sln"</UpdateCommand>
</PropertyGroup>
<Target Name="UpdateNugetPackages">
<Exec Command="$(UpdateCommand)"></Exec>
</Target>
</Project>
The UpdateCommand defines where, and with which arguments the nuget.exe gets called. Feel free to adopt this for your own needs.
This target has then be referenced in your .csproj File. As simple as that:
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
...
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<!-- After the initial decleration of the BuildDependsOn-Property. -->
<Import Project="UpdateNugetPackages.target" Condition="Exists('UpdateNugetPackages.target')" />
Keep in mind, that the import order matters. You have to import your target-file (UpdateNugetPackages.targets), with overrides (or actually decorates) the BuildDependsOn property, after the target-file Microsoft.Common.targets which defines it. Otherwise, the property will be redefined and will remove your changes, as the initial definition in Microsoft.Common.targets does not include any existing value in BuildDependsOn. Microsoft.Common.targets is imported by Microsoft.CSharp.targets for a C# project. Thus, your import has to go after the import of Microsoft.CSharp.targets.

Validate C# project file against a template during build

I know that you can create a template project to ensure that certain required settings (for ex: code analysis, treat warnings as errors, platforms, etc.) are set in a new project.
I want to enforce this as a part of the CI process i.e. in the build I want to validate the C# project files against a template (or an XML file) and cause a build failure if the project files don't conform to it. I need suggestions and ideas on how this can be achieved.
I want to try this with TFS 2010 and VS 2010.
I am writing a custom task to search for all the C# project files and compare the Xml structure of the Template project file with the Actual project file and report any difference as error.
I usually tackle this by customizing my build process to run Code Analysis/StyleCop/etc against all projects regardless of project settings.
Likewise for Treat Warnings as Errors. I just override the proejct settings at the build-level so that no matter what people do to the projects I know that everything I want to run is being run at build time.

Resources