Is it possible to get NuGet.exe running disconnected from Visual Studio? - continuous-integration

I've been wrestling with NuGet for a few days now and I'm turning to StackOverflow in frustration - hopefully someone here can be kind enough to point me in the right direction.
I've used NuGet several times for simple one-man pet projects, but this is the first time I've used it for something I really care about and want to have fully continuous builds, etc. I'm trying to create a simple NAnt build script to get the source for Git, ensure the external dependencies have been brought down, compile, and run tests - vanilla CI.
I originally went down the path of trying to get solution restore working, but it just didn't work or I didn't how it worked. Visual Studio is not on the build server and will not be installed there - that is not an option. As an aside, I couldn't get solution restore to work just with two developers (one trying to bring down the source fresh and build cleanly). I'm assuming it's because "allow solution restore" must be turned on everywhere (and is not by default). I punted on that approach before I got to the bottom of it - frankly, having my package manager so tightly coupled to the IDE makes me uncomfortable and was hoping I could do it another way. The package managers I'm used to using are simple command line tools - the CI build script invokes it on build, and developers do it on demand. I've spent the last two hours trying to get this working with the last 30 minutes in the NuGet source code. I feel like I'm fighting the tool and need to reboot.
Does anyone have any examples of the best to use NuGet in a multi-developer + CI scenario? This is what I want:
Any and all developers can get the source and run the tests in 3 or
less clicks (preferably 1). If the binaries are not present locally, that will be JIT fetched. If they are there, they will be updated if necessary, etc. This would ideally not even require NuGet to be installed (i.e. NuGet.exe would need to be in my repo).
Do #1 via a CI server like Jenkins, TeamCity, etc. (preferably using the same script)
If its not overly fighting the tool, I would like to have all this disconnected from Visual Studio with a single packages.config file and all binaries dumped into a single Lib folder in the root of the repo.
Any pointers would be very much appreciated.

Below, how I think you can achieve each your requisites:
You need to "Enable NuGet Package Restore" in your solution: http://docs.nuget.org/docs/workflows/using-nuget-without-committing-packages
As #alexander-doroshenko mentioned for TeamCity you can use Nuget Installer: http://confluence.jetbrains.com/display/TCD7/NuGet+Installer, but if you want a script to run in Jenkins, try this (works at TC too, as a command line step) for each project:
nuget.exe install "[Project folder]/packages.config" -source "" -solutionDir "" -OutputDirectory "packages"
This requisite will be done by item 1 and 2.

TeamCity has a build step for that, called "NuGet Installer", it fetch required packages from .sln file and download the locally. It does not require Visual Studio to run.
Read more about it here: http://confluence.jetbrains.com/display/TCD7/NuGet+Installer

There are several different solutions for integrating NuGet into your build process depending on how much integration you require. In our case we wanted to use NuGet as package manager and allow developers to build their solutions even if they haven't got NuGet installed on their machine. For that to work we enabled package restore which adds the NuGet binaries to your solution folder and updates the project files. Note that NuGet doesn't always do the update of the project files correctly. In our case we found that some project files got updated but others didn't. To verify that the project was updated you will need to open the project file as XML file. To achieve this load the solution and right click the project in question and select unload project. Then right click the project again and select edit [PROJECT_NAME]. In the project file you should see
A RestorePackages property in the first propertygroup. This property should have the value true
An import statement at the very end of the project file. This import statement should point to the 'NuGet.targets file that accompanies the NuGet binary.
Below is an example of one of our project files (heavily edited)
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<SolutionDir Condition="'$(SolutionDir)' == '' or '$(SolutionDir)' == '*undefined*'">$(MSBuildProjectDirectory)\..</SolutionDir>
<ProjectGuid>{8B467882-7574-41B2-B3A8-2F34DA84BE82}</ProjectGuid>
<OutputType>Library</OutputType>
<RootNamespace>MyCompany.MyNamespace</RootNamespace>
<AssemblyName>MyCompany.MyNamespace</AssemblyName>
<!-- Allow NuGet to restore the packages if they are missing -->
<RestorePackages>true</RestorePackages>
</PropertyGroup>
<Import Project="$(SolutionDir)\BaseConfiguration.targets" />
<ItemGroup>
<Reference Include="System" />
<Reference Include="System.Core" />
</ItemGroup>
<ItemGroup>
<Compile Include="MyClass.cs" />
<!--
.... MANY MORE FILES HERE
-->
</ItemGroup>
<!-- Import the Nuget.targets file which integrates NuGet in the build process -->
<Import Project="$(SolutionDir)\.nuget\NuGet.targets" Condition="Exists('$(SolutionDir)\.nuget\NuGet.targets')" />
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
Other similar extension points exist, see Microsoft.Common.targets.
<Target Name="BeforeBuild">
</Target>
<Target Name="AfterBuild">
</Target>
-->
</Project>
The next step you'll need to take is to provide a solution level NuGet configuration file in which you'll indicate where the packages need to be 'installed' and what the URL of the package repository is. In our case the solution directory structure looks like:
(D) root
(D) build
(D) packages
(D) source
(D) .nuget
NuGet.config
NuGet.exe
NuGet.targets
(D) MyCoolProject
MyCoolProject.csproj
MyCoolProject.sln
(D) templates
NuGet.Config
Where (D) indicates a directory.
The NuGet.config file contains the following configuration settings.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageRestore>
<add key="enabled" value="True" />
</packageRestore>
<config>
<add key="repositorypath" value="packages" />
</config>
<packageSources>
<add key="OurPackageServer" value="PACKAGE_SERVER_ADDRESS" />
</packageSources>
<activePackageSource>
<add key="All" value="(Aggregate source)" />
</activePackageSource>
</configuration>
This configuration file indicates that package restore is enabled, that the repository path (where the packages are placed) is the packages directory and which package sources are active.
By placing a NuGet.config file in the root directory we can use the hierarchical configuration option with NuGet. This allows the individual solutions to override computer specific configurations. The other benefit is that this way we don't need to have NuGet installed on the build server (because the executable and the configurations are in the repository).
With this setup developers can build the solution from Visual Studio. The build should work fine on developers machines even if they don't have NuGet installed. Note however that they won't be able to add packages to a project without having NuGet installed in visual studio.
On the build server you can simply use MsBuild to build the solution which will automatically download the packages from your package repository. Visual Studio is not required to be installed on the build machine for that (just the .NET framework of your choice).

Related

Cannot copy folder with nuget package in .net core

I Create a NuGet package and install into another project. but don't copy a file into the location project. only my file reference to NuGet package and I change the code, The package also changes! I want to copy the package to the target project.
<?xml version="1.0"?>
<package>
<metadata minClientVersion="3.3.0">
<id>MyPackage</id>
<version>1.0.0</version>
<authors>Meysam</authors>
<owners>Meysam</owners>
<requireLicenseAcceptance>false</requireLicenseAcceptance>
<description>Package description</description>
<releaseNotes>Summary of changes made in this release of the package.</releaseNotes>
<copyright>Copyright 2018</copyright>
<tags>Tag1 Tag2</tags>
<contentFiles>
<files include="cs/*.*/**" buildAction="Compile" />
</contentFiles>
</metadata>
<files>
<file src="ConsoleApp1\CustomClass\CustomClass.cs" target="contentFiles\cs\any\CustomClass" />
</files>
</package>
Copying files into the project that uses a nuget package is only supported when using packages.config, which isn't supported by SDK-style projects, which are required for .NET Core projects. As your screenshots show, contentFiles in PackageReferences projects are included at build time. The little arrow icon in the bottom right corner of the C# and folder icons you pointed to in your screenshot are visual indicators that the file and folder are different to the other files and folders in your project. In fact, that little icon overlay is similar to what Windows adds for shortcuts, so if you understand a shortcut is a file that "points to" another file, it should make sense that these are shortcuts to files outside your project, but are included as part of your project.
Anyway, it's working as designed.
You'll need to find another way to do whatever you want, but you didn't describe why you're trying to include a file in the project that references your package, so I can't give direct advice. My best advice, if you are experienced with ASP.NET and ASP.NET Core is think about how ASP.NET used to read a lot of settings directly from web.config, but ASP.NET Core instead uses a builder pattern, so that users are not forced to store settings in web.config and can store settings anywhere they want. If what you're doing is similar, your package users will have a better experience if you provide them with a configuration builder that they can override, rather than having a file in their project that they must edit and gets overwritten every time they upgrade to a new version of your package.
Cannot copy folder with nuget package in .net core
Yes, just like zivkan said:
Copying files into the project that uses a nuget package is only
supported when using packages.config
But, we could use a workaround to resolve this issue. We could add a copy task in the xx.targets file, and set this file in the \build folder in the .nuspec file.
The content of mypackage.targets file:
<Target Name="CopyFile" AfterTargets="AfterBuild">
<ItemGroup>
<CopyFiles Include="$(NuGetPackageRoot)\mypackage\4.0.0\cs\*.*\**" />
</ItemGroup>
<Copy
SourceFiles="#(CopyFiles)"
DestinationFolder="$(ProjectDir)"
/>
</Target>
And the .nuspec file:
<file src="xxx\xxx\mypackage.targets" target="build" />
Hope this helps.

NuGet only imports my .targets file into the top project

I have multiple projects in my solution, they all have the same NuGet package installed. I installed this NuGet package via the 'Manage Nuget Packages for Solution' option in my menu.
Now this particular NuGet Package has a .targets file in my build folder that imports some files into the build.
When I rebuild my solution these files are only copied for the top-level project. The other projects don't get the files copied into their build folder. This is causing me headaches because all the projects need these files in their build folder.
When I look into my .csproject files I see the following difference:
Project 1
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<Import Project="..\packages\NuGetTester.1.0.1\build\NuGetTester.targets" Condition="Exists('..\packages\NuGetTester.1.0.1\build\NuGetTester.targets')" />
<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
<PropertyGroup>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
</PropertyGroup>
<Error Condition="!Exists('..\packages\NuGetTester.1.0.1\build\NuGetTester.targets')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\NuGetTester.1.0.1\build\NuGetTester.targets'))" />
</Target>
</Project>
Project 2
The 2nd project does not have these lines.
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
</Project>
When I include these lines in the 2nd project everything seem to work.
Can someone explain why this only happens for the first project? Can I somehow force that every project in the solution that installs this NuGet will also include these lines? I can't force my users to include this manually.
Edit, this is the .targets file.
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Content Include="$(MSBuildThisFileDirectory)\Versions\*.sql">
<Link>App_Data\Versions\%(FileName)%(Extension)</Link>
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</Content>
</ItemGroup>
</Project>
Thanks
Can someone explain why this only happens for the first project? Can I somehow force that every project in the solution that installs this NuGet will also include these lines?
Without install nuget package log, I could not give the reason that why this only happens for the first project. But you should make sure you have installed the nuget package to all projects when you installed the NuGet package via the 'Manage Nuget Packages for Solution' option:
If all check boxes are checked, you still have this issue, you should install individually that package to the problematic project, then check the install log.
Just in case, there is a workaround for this issue. You can create a MSBuild project file named "Directory.Build.props" in the same folder as your solution with you content in it.
It will be primitive imported into all projects in the directory hierarchy.
Check this document for more details.
Hope this helps.
It appears, to my knowledge, that the order of the projects was not the problem.
The problem only appears when I have no files in my target (net46) folder in the build folder.
Once I placed a random file into the .NET targetted folder the .targets file was imported into the project. Or if I removed all the .NET target folders from the build it works as usual.
So just to make a summary.
Build folder with no .NET framework target folders -> It works
Build folder with an empty .NET framework target folder -> Does not work
Build folder with a .NET framework target folder filled with one or many files -> It works

How to build VS 2015 solution with projects which import targets and props from nuget packages using msbuild 14?

The scenario
You add a nuget package, which in turn injects <Import .../> statement into your csproj file which references a targets or props file from the package itself.
This works fine when building the code in Visual Studio, but fails miserably when building the same solution with msbuild.
The root cause
There are several. First, the logic to restore the packages is executed by the VS itself outside of the build proper. We can solve it by importing Nuget.targets which would run the RestorePackage target before the build. Check.
But the second problem is more serious. The Import statements importing targets/props from the packages can only be resolved after the packages are restored. Meaning restoring the packages cannot be part of the build. It must happen before the msbuild is given the solution to build. Yes, Visual Studio does it already, but I do not use Visual Studio on my Gated Check-In or CI server. I need it to work with msbuild.
What one can do?
As far as I understand, I need to be able to run the same logic VS does only on the command line. I.e. identify the packages and restore them before running msbuild. But devil is in details. Cannot be I am the first one to face this problem.
How do you do it?
Found the answer in this blog post - http://sedodream.com/2013/06/06/HowToSimplifyShippingBuildUpdatesInANuGetPackage.aspx
Following this blog I have created before.MySolutionName.sln.targets file with the following content:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="Restore">
<Target Name="Restore" BeforeTargets="Build">
<Exec Command=".nuget\nuget.exe restore $(MSBuildProjectName)" LogStandardErrorAsError="true" />
</Target>
</Project>
Now all the packages in the solution are restored before the build.
Thank you, Sayed Ibrahim Hashimi.

How should I tell TeamCity's NuGet Installer build step to use both NuGet.org and the TeamCity provided packages source?

I'm having trouble with my NuGet Installer build step.
We're using both official NuGet.org packages and our own packages hosted on the TeamCity NuGet server. If I leave Packages Sources blank, then packages from nuget.org are found, but as soon as I specify %teamcity.nuget.feed.server% as the package source, then packages from nuget.org are not found.
I tried setting Packages Sources to include both, but it still isn't working for official nuget.org packages.
https://nuget.org/api/v2/
%teamcity.nuget.feed.server%
Is that not the right URL for the nuget.org package source? How do I tell it to use both sources?
I asked this on the JetBrains Developer discussion board, but haven't gotten any responses.
Had same problem, funny enough my Nuget sources were specified as
https://www.nuget.org/api/v2/
http://nugetserver/nuget
Adding a forward slash on the second url to make it http://mynugetserver/nuget/ fixed the problem.
Took me a while to figure out. Now my Nuget-installer build step is running fine.
Apparently the NuGet Installer build step is not even needed. I edited the .nuget/NuGet.targets file to include both paths and removed the NuGet Installer build step and it works now.
When originally setting up TeamCity for this solution, it didn't work without the NuGet Installer step, so I don't know what else I've done differently to make this work, but maybe the NuGet.targets file was the key all along.
The comment on this blog post pointed me in the right direction.
You can modify the NuGet.Config in AppData local folder for the user that TeamCity runs under and not modify each project's .targets file:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<config>
<!--<add key="repositoryPath" value="J:\TeamCity7\buildAgent\work\my_shared_packages_dir" />-->
</config>
<packageRestore>
<add key="enabled" value="True" />
</packageRestore>
<packageSources>
<add key="NuGet official package source" value="https://nuget.org/api/v2/" />
<add key="MMG TeamCity Nuget Server" value="http://myteamcityserver/guestAuth/app/nuget/v1/FeedService.svc" />
</packageSources>
<activePackageSource>
<add key="All" value="(Aggregate source)" />
</activePackageSource>
</configuration>
The NuGet.exe inside of .nuget folder in each project will respect the configs set here first, then apply any "overrides" done at the .targets file.
Same problem here. I am using TeamCity v10, Nuget step is required (no .targets file in my solution). However, I used another approach to add the "extra" package source:
c:\BuildAgent\tools\NuGet.CommandLine.2.7.0\tools\Nuget.exe sources Add -Name TeamCity-feed -Source http://myteamcityserver/guestAuth/app/nuget/v1/FeedService.svc/
After that, I added a Nuget installer step and did not specify anything in the package source box in TeamCity, now both packages from public feed nuget.org and my internal feed could be restored without problems.

How to configure where NuGet.exe Command Tool line looks for packages

We have successfully set up a couple of local package repositories using the NuGet.Server package and hosted them on a local IIS webserver. We are able to connect from the Package Manager and install no problem. So these are working fine.
In order for us not to have to check in our packages folder we have included the following command line in each project file that includes NuGet references. This works, if the NuGet.exe is in the path on the CI build agent.
However, I would like to move the source configuration form the command line in every project file and put it in just one place, preferably where other pesky developers can't change it ;)
<Target Name="BeforeBuild">
<Exec Command="nuget install $(ProjectDir)packages.config -s
http://domain:80/DataServices/Packages.svc/;
http://domain:81/DataServices/Packages.svc/
-o $(SolutionDir)packages" />
</Target>
Is there a better way?
Yes there is ;-)
Take a look at NuGetPowerTools. After running Install-Package NuGetPowerTools, it adds a .nuget folder to your $(SolutionDir) containing nuget.exe, nuget msbuild targets and settings (which you will need to check in).
After that, you simply run Enable-PackageRestore and it sets up msbuild targets into your visual studio project files which will make sure that packages will be fetched in a prebuild step, even on your build server, without checking in any packages. (don't forget to check in the .nuget folder though!).
This way, you simply manage the nuget package sources in a nuget msbuild settings file (in the .nuget folder) central to your solution, instead of in each project.
Cheers,
Xavier
I finally got NuGetPowerTools to install after the advice from digitaltrust on http://blog.davidebbo.com
Although NuGetPowerTools solved my problem, it was overkill for what I wanted. It requires that you check in to version control a .nuget folder that it creates in your solution root. The folder contains NuGet.exe, and a couple of target files. I don't like this as I think version control is for source code, not tools.
I came up with the following solution.
Save NuGet.exe to a folder on your local drive, both on dev and continuous integration machines. I chose C:\tools\nuget\
Add that filepath to the Path Environment Variable in all environments
On continuous integration machines, find %APPDATA%\NuGet\NuGet.Config and enter the following
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="LocalRepositoryName" value="http://Domain/DataServices/Packages.svc/" />
</packageSources>
You can add more than one entry to packageSources and NuGet will search them in the order that they appear
The after build code from my question can now be amended to the following.
<Target Name="BeforeBuild">
<Exec Command="nuget install $(ProjectDir)packages.config
-o $(SolutionDir)packages" />
</Target>
The end result of this is that whenever the approved repository location is changed, the config has to be changed in only one place rather than in every csproj file. Also, it is the continuous integration server administrators who determine that location, not the developers in their command line calls.

Resources