We are experimenting with using Octopus for CD using TeamCity. We have enabled OctoPack to create the Nuget Packages for use during the deployment. We also are experimenting with building libraries and using the integrated Nuget Server.
We were able to do both successfully. Both deployment to an environment, and using Nuget.Config to both install the library package and restore and build in TC.
Yesterday, the CD stopped working, the packages are being built but the Nuget Server is not making them accessible. We reset metadataBuilds, per TC instructions and we are still not getting new packages in the feed. We did confirm that the packages are still being built.
Any ideas?
I would say a starting point to get a solution to this depends on where the nuget packages are stored when your solution is built in TC.
If they are left on the TC Nuget feed, then you would want to watch to see if there are more than 100 packages in the Nuget store (TC Artifacts). We have found that once you go over 100 packages, the ones after those 100 do not show up in the feed when Octopus tries to pull from it.
If you are pushing to the native Octopus Nuget store, perhaps watch to sere that the space on that server hasn't filled to the point where it cannot push any more of them.
The build log in TC should tell you a lot about where and how these packages are being dealt with. They should also show up as build artifacts after a build, which would allow further verification that they are at least being built.
Although it may not be related, the Nuget feed in TC may take a while to pick up new packages after a build finishes. Particularly once you get a large number of packages. That may cause Octopus to fail if it is kicked off right afterwards (by a chained build).
What I've found works best is to push deployment packages directly to the Octopus internal nuget store, and keep shared (referenced in other projects) in TC or another nuget server. (NB you cannot use Octopus as a nuget server to retrieve packages). The push is done as an explicit step in the build that produces the packages.
Related
Just curious, per my title above.
I am leaning towards a nuget package-like style, where the dev team can continuously update libraries (dll's) and deploy them to a central location e.g. Nuget. Then, have project reference the dll depending on the version that's been deployed in a central location e.g. Nuget.
Deploying the dll's in a shared folder won't work.
Please advise, thanks.
Actually, nuget is the best way.
Pack
You can work with your teammates to maintain the same nuget project. When you are creating a nuget package or make some changes, you can just set different nuget versions for them to pack them as nuget package(.nupkg). These are different pack functions(non-sdk project uses nuget.exe, new-sdk projects use dotnet cli, msbuild.exe can work with both of them but note in non-sdk projects, it works with non-sdk projects with PackageReference).
Push
When you finish it, you could push the nuget package into a a shared folder, or push into an azure devops feed.
Use
You just send the full path of the shared folder or the feed url link(they should give your teammates sufficient permissions)
And anyone who wants to use it should set the link into nuget package sources, and then they can install it to use it.
Besides, if there is a subsequent update operation, as a nuget maintainer, you modify the lib project, after that, set a new version for it, any descriptions to tell others there is a new release updated version during pack process.
Then, push the new version into the feed to let others could install it.
I have been trying to find an "elegant" way to integrate non-NuGet package with my Bamboo builds.
There's a plethora of stacks on the topic of adding non-NuGet packages into NuGet bundles:
managing non-nuget dlls along with nuget packages
Creating NuGet package with reference to a non-NuGet reference
Trying to add non-.NET libraries to NuGet package
and the list goes on. There's also many a stack about using NuGet in Bamboo and that part works smoothly.
None of these deal with the situation of having an automated build environment, which may be sitting on some other remote server, running Bamboo.
Specifically, I'm trying to automate Xamarin.iOS deployments to HockeyApp.
The steps are:
Coding and local testing in VS2015 on Windows and with Mac for iPhoneSimulator
Merge into deployment branch and push to Bitbucket server
Bamboo picks up the push and kicks off build
Build checks out deployment branch
Runs NuGet downloads
Starts compile for Ad-Hoc/iPhone environment, creating IPA
Kicks off the HockeyApp deployment (there's a free addon for that)
Nearly all the steps are in place, except for the one where I have 2 dependencies which the commercial vendor (Syncfusion) has for unknown reasons decided to bundle into their "Studio" product, from where my Visual Studio project/solution has to refer to them by location outside of my project directory.
As a result, my Bamboo build fails with not-found DLLs, because they're missing as they would have to come in somewhere between 5. and 6. above.
I don't want to copy the binaries and then check/commit/push them into my repository, as that's considered a Bad Thing. My Bamboo Plan already successfully grabs NuGet packages before the actual build without having to drag binaries along.
Simply copying the DLLs on to the Bamboo build machine (i.e. where the remote agent is running) was one idea, but the problem is that the VS/MSBuild project file now has hard-wired directories - so, I'd have to install the whole Syncfusion Studio, or emulate their directory structure just for those 2 DLLs.
So I would need to adjust the .csproj references in an automated fashion. Not sure how I would do that, except with Yet Another Task and Script.
Apart from yelling at Syncfusion (which I've already done) about making all their DLLs available through NuGet (because some are, and those I'm successfully receiving in step 5. above), does anybody have a suggestion how to get this missing step to work?
For expediency sake, I have now added the libraries to the repository in a separate sub-directory.
It's not how I wanted to do it, but as the 2 libraries were a mere 200-300KB each and as there just didn't seem to be a simpler solution it solved the issue for now.
Specifically:
Leave .dll in .gitignore
Copy libraries you need into local sub-directory, e.g. LocalLibs/
add specific libraries with git add -f LocalLibs/speciallib.dll so that only these become a part of the repo
Change the project reference in Visual Studio to point to the local libraries, instead of their main install location
Verify that builds still work from within Visual Studio but also with MSBuild
I may revisit this and update if a better way comes along, especially if the libraries are significantly larger, such that you definitely would not want to add them to your repository.
So maybe I'm a little bit confused about how a nuget server works and the specifics of nugets in general.
I am setting up octopus deploy and TeamCity for my company and have run into a bit of a snag. I am trying to set up a deployment where I deploy a website and a service in the same release, however, Octopus can't seem to find the nuget package. It throws the following error:
Could not find any packages with ID 'PackageName' in the feed 'octopus://'
I am able to see the package when I test the feed in octopus, and I verified that I am using the correct ID in teamcity. Basically what I've been trying to research is how I can view all of my nugets in the feed. Is that a thing? I am using TeamCity as the Nuget server and I know with octopus you can view all packages that have been pushed to its repository. Am I able to view all packages in the TeamCity Nuget server? I want to verify that my package is there.
In order for Octopus to be able to publish the package, it needs it to be set to build an OctoPack.
To get this working, use the NuGet package manager for the project in question and add a reference to 'OctoPack'.
Bear in mind that if you have a Visual Studio solution containing several packages and you want several of these to be deployable packages you will have to add OctoPack to each project you want as a deployable package.
Include Octopack in the project to build the nuget package.
Use this parameter with MSBuild to automatically push the package to the destination nuget gallery after the build is complete.
/p:OctoPackPublishPackageToHttp=http://my-nuget-server/api/v2/package
I have just setup TeamCity to automate our builds, our current solution has both a dev and main branch. What I am trying to achieve is to have the development branch build and publish to a development NuGet feed on our ProGet installation, and then have the main branch publish to our Main NuGet feed on ProGet server.
We are using octopus deploy to deploy the packages, within TeamCity we have the octopus deploy plugin installed and if I tick the box to run OctoPack it builds the packages and they appear as artifacts when the build completes. If I try to use the NuGet Pack build step in TeamCity I get the following error for one of our projects:
[08:33:49] : [pack] Attempting to build package from 'xxx.csproj'.
[08:33:50]W: [pack] Unable to find 'xxx.exe'. Make sure the project has been built.
The project has been built and it works with OctoPack so why isn't it working with the NuGet Pack? Wwe have five projects being built and the first four run fine, one is a console app, one is an mvc website, and two are class libraries. The one that doesn't work is a windows service.
The end goal here is to publish these packages to a private feed on ProGet. I don't mind using OctoPack but in my head wanted to remove that dependency from TeamCity but I can live with it. However when I try to use the NuGet Publish runner type how do I select to publish any NuGet artifacts that have been created?
I have been googling like mad and I cannot find any helpful links that describe what you are supposed to enter, I would really appreciate any helpful comments/answers.
We are using version 8.15 of TeamCity.
Hopefully the following will help with at least part of your question; mainly the bit relating to how to publish packaged artifacts.
NuSpec Approach
When using the NuGet Pack build step, you can specify the Output Directory, which will determine the output location of the packages. You can specify this as a relative path to the checkout directory, probably best to define it as a build parameter, such as %system.PackageDeployOutput% as you'll be using it in the next step...:
Next, specify a NuGet Publish build step, fill in the Package Source / API key etc, and specify the Packages to upload as
%system.PackageDeployOutput%\*.nupkg
This will pick up the packages output in the previous step. I've used this quite effectively, and the parameterisation approach encourages conventions across all your builds.
OctoPack Support
If you're using the MsBuild build step with OctoPack, you can use a similar approach by declaring a system parameter called
system.OctoPackPublishPackageToFileShare = %teamcity.build.checkoutDir%\%system.PackageDeployOutput% (note the same parameter as above)
You can declare these as root project parameters, so you get the best of both worlds. My preferred approach to packaging is currently to use nuspec files for deployable endpoints. I've found OctoPack to be a bit of an overhead when it comes to more complex deployments (it's fine for basic MsBuild projects).
Our build plans for Sitecore (.Net) websites are currently using Bamboo as the build plan master for continuous integration process. Bamboo supports building a .Net project, and that .Net project contains references to several Sitecore binaries.
I have used NuGet as an artifact repository, mostly because that seemed to be the one most people were using.
So, now that I have a nuget repository, am hosting my own feed, and have added the Nuget package to my project (I just installed the package of binaries I created), what more do I need to do?
Should the build call out something special, or am I done? I guess I'm just nervous that bamboo isn't talking directly to my repository....
In the closest analogy I have, namely a Maven/Archiva combo, the maven plan specifically references the artifacts in archiva, pulling the correct version as needed. Does NuGet do this?
Since there have been some new developments in the NuGet package restore approach, I thought I'd post an update on this topic. We're using Visual Studio 2012. I wanted to be able to run MSBuild and make sure that it would first restore NuGet packages without setting "Package restore" to true in the config. Here's what I did (inspired by https://stackoverflow.com/a/23935892/414376):
Upgraded NuGet in my solution to a version later than 2.7 (2.8.3 in my case; this seems like an optional step)
Installed NuGet.exe on the build server
Added NuGet.exe to my remote agent so that it could be run as a command from Bamboo
Added the NuGet command to my build plan with argument restore (so that the command will be nuget restore); I've placed it right before my MSV Build task.
That was all I needed to get this to work properly according to the latest guidelines.
Nuget 1.4+ supports "Package restore" which embeds a call to an MsBuild task in the project file. When the packages are not available it will automatically restore them while building the project.
I'm sure this is old news, but my packages are in \packages. No amount of "dotnet restore" would work, until I explicitly mentioned the packages folder.
dotnet restore --packages .\packages
restored them!
VS2019. just in case it's version specific.