Cannot automatically update a NuGet package to the latest version during build - visual-studio

We have two separate .NET solutions:
Running a build for the first solution produces our end product: a bunch of DLLs. These DLLs are delivered to our customers via a NuGet package.
The second solution serves as a product-test solution: the NuGet package is installed to it, and it is built and executed - thus it makes use of our product exactly the same way as our customers would do.
The challenge here is that there should be a way our latest NuGet package gets installed automatically to the product-test solution, preferably during the build of this product-test solution.
Based on the ideas from a similar question, I got this far with configuring the product-test solution:
First I enabled NuGet Package Restore. This lets me get rid of the "packages" directory completely from VCS since the package with the version defined in packages.config file would be downloaded automatically by NuGet before build.
Then I added the following pre-build event in Visual Studio: $(SolutionDir).nuget\nuget update -prerelease $(ProjectDir)packages.config. This lets me pull in the latest version of our NuGet package during build.
I currently use the above scenario to run local builds using Visual Studio and unattended builds using TeamCity. The solution seems to work for both scenarios on the first sight, but actually it does not produce the expected result: when the product-test solution is built, in the bin directory I don't get the latest version of the DLLs, only the latest-1 version.
The problem is that although the nuget update command updates everything as expected, including the packages.config and the .csproj file, their new content is not picked up by the build, therefore - as my guess goes - the HintPath settings from the .csproj file still reflect a "before build" state, therefore old DLLs are copied to the bin directory. I assume the .csproj file is processed only once: before the pre-build event is triggered, and the changes made by the pre-build event are ignored until the next build.
I considered the following solutions:
Apparently pre-build is not "pre" enough. If there was an even earlier point I could insert the nuget update command, my above solution would probably work.
I read that I could override the HintPath-s in the .csproj file by defining a ReferencePath. But I doubt I could easily figure out the right path or I could set it early enough so the build picks it up.
As a workaround I could run the builds twice: duplicate the build step for the product-test solution in TeamCity and I could always build the solution twice locally in Visual Studio.
Has someone figured out how to automatically update a NuGet package to the latest version during build?

Have a look at a blog post I just did regarding this process. Instead of configuring stuff on the server, I did it by augmenting the Nuget.Targets file that's put in place with the Nuget Package Restore option. Beauty of this approach is it executes locally AND on the server (so you get to see any possible side-effects before breaking the build)
Posted details here: http://netitude.bc3tech.net/2014/11/28/auto-update-your-nuget-packages-at-build-time/

I think that put automatic update to pre-build step it's not NuGet style. You can understand that you need this operation every time when you do it. Mostly because it can increase build time. For example when you use TDD and often rebuild project and run tests, you expect that it will be done quickly. Additionally it can update unexpected packages and break something, after that you can spend a lot of time to find the problem.
I suggest to do update as the separate step. On TeamCity you can use NuGet installer build step. For performing update just check two checkboxes in the bottom area of step configuration:
Additionally if you want keep result of update after successful build and passing test, you can add later build step which commits this changes to VCS (using cmd or PowerShell for example).
When you are working locally, I thing the better way run update packages once before you start working with project. You can use Package Manager Console for this, with command Update-Package -IncludePrerelease.

The MSBuild-solution I figured out is to override the BuildDependsOn property. Therefore, I created an UpdateNugetPackages.target which looks like this:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<BuildDependsOn>
UpdateNugetPackages;
$(BuildDependsOn);
</BuildDependsOn>
<UpdateCommand>"$(SolutionDir)pathToYourNugetExe.exe" update "$(SolutionDir)NameOfYourSolution.sln"</UpdateCommand>
</PropertyGroup>
<Target Name="UpdateNugetPackages">
<Exec Command="$(UpdateCommand)"></Exec>
</Target>
</Project>
The UpdateCommand defines where, and with which arguments the nuget.exe gets called. Feel free to adopt this for your own needs.
This target has then be referenced in your .csproj File. As simple as that:
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
...
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<!-- After the initial decleration of the BuildDependsOn-Property. -->
<Import Project="UpdateNugetPackages.target" Condition="Exists('UpdateNugetPackages.target')" />
Keep in mind, that the import order matters. You have to import your target-file (UpdateNugetPackages.targets), with overrides (or actually decorates) the BuildDependsOn property, after the target-file Microsoft.Common.targets which defines it. Otherwise, the property will be redefined and will remove your changes, as the initial definition in Microsoft.Common.targets does not include any existing value in BuildDependsOn. Microsoft.Common.targets is imported by Microsoft.CSharp.targets for a C# project. Thus, your import has to go after the import of Microsoft.CSharp.targets.

Related

How can I make msbuild understand that a .csproj file depends on the availability of an exe built by another .csproj

I have a large VStudio solution whose .csproj files I'm porting over to the new .NET SDK format. I have integrated the github project with AppVeyor. When I work on the solution, my project is built by VStudio; AppVeyor uses msbuild to build my project.
All of the class library project files in the solution have been modified so that they specify <TargetFrameworks>netStandard2.0</TargetFrameworks>. For simplicity, let's represent the class library project files with classlib.csproj.
The solution also contains a few command-line tools, which need to be built in advance of some of the other projects, because those later projects need to run the command-line tools as a preprocessor before building themselves. These command-line projects are all specifying <TargetFrameworks>net472;netcoreapp2.0</TargetFrameworks>. Let's represent the tool project files with tool.csproj.
In order to build correctly, tool.csproj needs to be built before classlib.csproj. In the solution I've used the Build dependencies > Project dependencies command in the VStudio solution explorer to indicate this. This works fine when I'm working inside of VStudio.
However, when I push my changes to github, AppVeyor kicks off an msbuild process to build the projects. From the error spew, it's clear that the tool.csproj didn't get built in time. It seems msbuild doesn't understand the project dependency information in the .sln file.
Reading what's posted on the intertubes (e.g. https://devblogs.microsoft.com/visualstudio/incorrect-solution-build-ordering-when-using-msbuild-exe/), I attempted to add the following to classlib.csproj:
<ProjectReference Include="..\tool\tool.csproj">
<ReferenceOutputAssembly>false</ReferenceOutputAssembly>
</ProjectReference>
But when I do this I get the error:
Project '..\tool\tool.csproj' targets 'net472;netcoreapp2.0'. It cannot be referenced by a project that targets '.NETStandard,Version=v2.0'
which clearly makes sense.
Some other solutions I've read suggest editing the .sln file so that tool.csproj gets built first, and hope that building will be completed before classlib.csproj starts building. However, this is a race condition since msbuild is not aware of the dependency.
How, then, do I express that classlib.csproj depends on tool.csproj so that msbuild understands it?
EDIT
It turns out the issue was caused by an interaction between the target I'd written to start my tool and the globbing operation MSBuild does to resolve the <Compile> item collection using a pattern similar to **\*.cs. The tool generated the file after the globbing operation. Since the generated file isn't present when the globbing happens, it is missing from the <Compile> item collection, and the build failed. I mistakenly interpreted the error "foo.cs not found" as a failure to execute the tool, when in fact, it's a timing issue.
To make the build order works outside VS IDE, we need to use add reference
instead of BuildDependencies=>ProjectDependencies.
Right-click classlib project and choose add=>reference, in Project node choose the Tool Project you want like this:
Click ok to add it and save changes. After that, open the classlib.csproj you can find sentence like:
<ItemGroup>
<ProjectReference Include="..\xxx\Tools.csproj">
<Project>{6eaa430f-9793-4639-a84b-6ab767d57147}</Project>
<Name>Tools.csproj</Name>
</ProjectReference>
</ItemGroup>
This what the msbuild can understand. And I think that's what you want.
After that,you can use a single msbuild tool or Appveyor to check the
build order.
And in Appveyor, we can also disable the paralell builds(settings=>build=>msbuild options) to make sure it won't build classlib.csproj until the tools.csproj build ends.
The final build order in Appveyor will be what we defined in VS. Hope it helps.

What is the easiest way to execute restore of nuget packages on build outside solution folder

Set up
I have a solution Commons.sln which contain common utils for other more specific projects. Let's take for future example one specific solution called Cars.sln just to make it easier to explore the case. All solutions have their own set of NuGet packages.
All projects are laying in the same VCS branch therefore folder structure is pretty strict.
What I want to achieve?
I would like to have references for common libraries in all specific projects. Accordingly, I have added projects from Commons.sln to my Cars.sln. However I met an issue with this setup because people from my team who never built Commons.sln do not have NuGet packages from this solution downloaded to their machines and when the try to build Cars.sln - they face an error saying that dlls for projects from Commons.sln are not found.
Long story short - I would like to execute nuget restore ../../Commons/Commons.sln before build, but do not know how to do it better on Cars.sln solution level on every build/rebuild in Visual Studio.
Specifics
I have done a little research on this and there are my bullet points:
I do not want to commit any of my NuGet packages in VCS
I so a solution with adding a .target file per solution but it works only when I build directly with msbuild.exe.
In other SO question. There is a recommendation to add one more project and resolve it by build order which seems to be hacky to me.
Specific projects are bound to CI server so this stucture should work there as well (currently I have solved by adding a build step that builds Commons.sln before everything else)
Any suggestions are appreciated.
Currently I have fixed it with adding a msbuild target to the project in Commons.sln which is a dependency to all projects including specific ones, like Cars.sln:
<Target Name="Test" BeforeTargets="BeforeBuild">
<Message Text="Restoring Common Nugets" Importance="high"></Message>
<Exec Command="nuget restore "$(ProjectDir)..\Avid.Common.sln"" />
</Target>
However if somebody has a better solution I would be glad to know about it.

TeamCity Using OctoPack - Isn't Excluding Superfluous Files

I'm just looking at streamlining the nuget packages that are coming out of my build system and I'm stuck on how to only package the files that are required.
I have several configurations sharing a Root VCS checkout. I have a configuration that runs a debug build with unit tests. I also have a release configuration that does a release build, this configuration then also uses the TeamCity OctoPack plugin to create the nuget packages.
What I want to achieve is the building of nuget packages that don't contain the *.pdb and *.xml documentation files as these aren't required for the release deployment.
I've looked through this page on the OD site:
http://docs.octopusdeploy.com/display/OD/Using+OctoPack
And according to this page OctoPack should only package up the required files by default. I'm not entirely clear on how or what needs to be done to get around this problem as it doesn't appear to be working as described.
It seems that one solution would be to provide a nuspec file for the projects I'm looking to deply but I'm also wondering if there is something I'm missing before I head off down that route.
I also have some MEF plugins that are copied in post build events and these aren't included in the nuget packages when in fact they are needed for the application to run. I think I need to get explicit with a nuspec file but would like to confirm this.
What is the simplest way of achieving what I need?
Assuming you're running the later versions of OctoPack, in your release build you can set a system parameter system.DebugType = None which will get passed to the OctoPack build scripts and prevent the PDB's being created.
This simply overrides the setting defined in your csproj msbuild file (assuming C#), so you can use it wherever you want to prevent PDB's being created at the build configuration level (not just OctoPack). I generally prefer this approach as it prevents side-effects in your build from changes by developers in the project file.
As for the xml files, I haven't actually tried this, but you can try a similar approach and create a system parameter system.DocumentationFile = "" to blank out the output.
If you really want to make sure that the files have been removed there are a couple of ways you can do this. Modify your deployment process to:
Execute your own custom PowerShell script in that removes the files
Include a script module from the Octopus Library to the same. Check out the File System - Clean Directory from the Octopus Library

Consuming actively developed NuGet package

First, I will describe our setup (it is an extremely simplified version of the corporate development pipeline). We develop a .NET application, which is dependent on some shared components. The shared components typically develop together with the application. We want turn these shared component to NuGet packages.
So, I publish version 1.0.0.4 of package A and want to consume it in my application. My packages.config looks like this.
<package id="A" version="1.0.0.4"/>
So far so good. I probably commit packages.config to version control at this point to remember my dependencies. Now I run the next build of package A, which publishes the version 1.0.0.5. Now I want my application to automatically update package A to 1.0.0.5 before the build.
However, my packages config now contains an exact verion 1.0.0.4 and it's version-controlled. My questions are:
Can I somehow specify that I want version 1.0 or higher in packages config? In other words, can I avoid change of packages.config with every new build of every package?
Can I somehow update the dependencies before build to the the newest version? Can I do it automatically using a script?
I am quite new to NuGet and I come from the ant+ivy/maven world, where this feature is kinid of automatic, so I am still hoping I am missing something obvious in NuGet, although scanning through the discussions on stackoverflow doesn't sound too encouraging.
I've found Using Nuget in development environment - best practices / how to and How to automatically update NuGet packages to latest available version and NuGet issues with packages.config, project references and the solutionwide packages folder, which do not give clear answers.
You can use the allowedVersions attribute to automatically use newer package versions within a range, like this:
<packages>
<package id="SomePackage" version="2.1.0" allowedVersions="[2,3)" />
</packages>
In this example, any package from the 2.x series will be used, but not from the 3.x series. You can be more specific with the contraint e.g. `allowedVersions="[2.1,2.4)" if you wanted to take any package from 2.1, 2.2, and 2.3 but not 2.4.
See http://docs.nuget.org/docs/reference/versioning for more details
I don't think you can change the packages.config to implicitly allow a higher version. Also, the project files have references to the packages folder which include the package version in the path, so those need to be changed after a package update.
You can perform an update of the packages.config and the projects files by running nuget update.
You could automate this in the nightly builds and checking in the resulting changes as well. However, it's debatable whether that's a good idea. At the minimum, you should gate the update operations by running a build to make sure the new version doesn't cause a build break.

CC.NET: howto trigger an extended build when a subdirectory in svn has changed

We have a couple of projects configured in cc.net. Each of these projects has following items in it's working directory (svn):
source
lib
db scripts
SSIS package(s)
We would like to know if there is a way to find out if there are any modifications in the subdirectory containing the SSIS packages? This would allow us to do a full build (including execution of package). We don't want to do this with every build since package execution might take some time...
Our other option is to create a cc.net project that does the complete builds at night time.
Does anybody have a nice solution to this problem?
I would use the SvnVersion task from MSBuild Community Tasks to identify the latest revision on the SSIS package subdirectory. Then compare it to one you've stored in a file somewhere in your working directory or elsewhere.
If it's different, pass a property to your main MSBuild task with a flag instructing to build and execute the SSIS packages. Once that's done, update your revision file with the new new revision number for the SSIS subdir.

Resources