I understand that there is currently a feature request for something like this, but I'm hoping that there is some sort of workaround using the current version (1.6)
We build for our dev and test environments from a dev branch in TFS, and build for our QA and Production environments from a release branch in TFS. Since these produce distinct nuget packages, I can't use the same package feed. The deploy package step doesn't seem to have a environment scope option. Is there some other way to say "Deploy package X version Y for Dev/Test, and package A version B for QA/Production?"
You can use the same feed, with the following caveats:
The nugets built from the two branches obviously have to have different (non-colliding) versions. I add the 'dev' suffix to the package built from the 'dev' branch (eg 1.2.3.4-dev), and leave my 'stable' branch bare.
You have to be explicit when you 'create release', because by default Octopus will pick up the highest version nuget available, and that might not be the release you want (stable presumably lags dev). Pick the version of the package you want (and set the deployment version appropriately). If you are creating your release via TeamCity, ensure you use the --packageVersion argument as well as set the release number.
Since the above only works if there's only one package in the release, your deployment process does (unfortunately) have to produce one monolithic nuget, or you will get version-mismatches.
The advantage of this kind of arrangement, of course, is that at a pinch you could push your dev build to QA (or Prod) if the need ever arose.
All this assumes both branches build as the same package of course. You could build different packages between the dev and stable branches (but I don't think I'd recommend this because of duplicating all the Octopus config).
Update: apparently you can use Octo.exe to specify different version numbers across different packages - see https://github.com/OctopusDeploy/Octopus-Tools.
Related
I would like to integrate an automated versioning system in my ASP.NET project. For each release, it should have a version number based on the previous release. I am planning to integrate Gitversion https://gitversion.net/. Does anyone use it in your projects? For the CI/CD pipeline we have teamcity and octopus deploy.
What is the best practice for automated software release versions?
Thanks in advance
As one of the maintainers of GitVersion, I'm obviously biased, but since you're asking how to use GitVersion to implement a "best practice for automated software release", I'm going to unashamedly give you a textual description of a talk I've done on how I prefer to version and release software with GitVersion, TeamCity and Octopus Deploy.
Developer Workflow
The first thing you should figure out is what kind of developer workflow you want for your software. GitVersion supports Git Flow and many simplified variants of it, and GitHub Flow (as well as many other trunk based development flows). Which workflow you should choose depends on what type of software you are developing, your team and most importantly; your personal preference.
Once you have chosen your workflow, you can configure which mode GitVersion should operate under.
Version Source
GitVersion works by calculating a version number from your Git repository. This means that you should not commit a version number to Git in any shape or form. Not within a package.json, pom.xml, .csproj, or any other build- or project-related file that often mandates the existence of a version number.
Versioning
Instead, you should allow GitVersion to produce a version number based on the Git history, using the currently checked out commit as its starting point, and searching through the parents and their tags to calculate an appropriate version number for the current commit. This version number can then be used in any way you want in your build pipeline. You can, for instance, write GitVersion's FullSemVer variable to package.json by executing the following command:
npm version $GitVersion_FullSemVer
If you are developing on the .NET platform, it's also possible to use GitVersion to patch your AssemblyInfo.cs files so the calculated version number is compiled into your assemblies. With InformationalVersion containing the SHA of the Git commit being built and versioned, you'll be able to identify the exact origin of a compiled assembly.
Build
Once you have your workflow in order and GitVersion has a good source of information to use for its versioning, you can go ahead and create a build pipeline for your software. A typical build pipeline will look something like this:
git clone. (Ensure that the clone is complete and not shallow or a detached HEAD. See requirements for more information.)
GitVersion. Execute GitVersion by whichever means make most sense to your environment.
Patch. Patch the version number created by GitVersion in step 2 into every file in your repository that makes sense, such as AssemblyInfo.cs, package.json, etc.
Build. Perform the build of your software.
Test. Run tests that ensure the quality of the software.
Package. Create a package of your software, using the version number created by GitVersion in step 2.
Release. Release the package using the package management software of your choice, such as Octopus Deploy, npm, nuget, composer, or similar.
Test. Perform automatic tests of the released software, if possible. If successful, it's possible to automatically promote the released software to other environments, if applicable.
Through GitVersion's built-in support for build servers, the calculated version number will also be promoted to the build server to version the build itself. This will be done automatically on TeamCity, for instance. In TeamCity, I recommend that you run GitVersion as its own build configuration exposing the required variables which can then be used in dependent build configurations later on.
Release
Once you have a built artifact containing the version number generated by GitVersion, you can use the same version number to create a package, create a release and deploy the release in Octopus Deploy.
You told you want to integrate an automated versioning system? I would like to throw my hat in the ring.
I am author of Vernuntii, a simple semantic versioning library with git integration.
The answer of #Asbjørn told you already about the best practices for example choosing a workflow that is suited for your project.
The main part a versioning tool like GitVesion or Vernuntii is to generate a suitable NEXT VERSION based on (non-)existing git tags.
So at the end of the day it is a matter of taste what kind of complexity you want HOW the NEXT VERSION is calculated.
So when you want cross-branch versioning, then you are good to go with GitVersion, but if you don't need that kind of complexity then you can try a single-branch versioning like it is implemented in Vernuntii.
For more informations take a look at the README.md of Vernuntii.
Here, for giving you my impression of versioning tools and their complexity, let me give you a list (sorted from least complex to most complex):
MinVer
Verlite
Vernuntii
Nerdbank.GitVersioning
GitVersion
A fun fact beside: all libraries from top to Vernuntii also allows to calculate the next version from detached HEAD.
We have a Mainline build versioned "x.0.0.r" where we publish a web service using octopus. Our branches are versioned "x.y.z.r" for specific releases. We DO NOT want to publish this web service when building from a branch.
We are using TFS: 2013 xaml build def; msbuild args: "... /p:RunOctoPack=true ..."
Short of setting /p:RunOctoPack=false or removing the OctoPack arguments from the branch build definition, is there any other way to NOT run Octopus? Perhaps in the packages.config file, to say if the version is NOT ".0.0." set "\p:RunOctoPack=false"?
We have missed removing this parameter when branching and cloning the build def and have mistakenly published the web service from the branch a few times. Would like to avoid this going forward.
Thanks,
tom
The deployments you describe are a bit confusing. I think you are wanting to deploy all versions tagged with version 18.x.y.z and not deploy any other version numbers.
Using Channels in Octopus would allow you to achieve this, without adding logic into your build process.
In your deployment process in Octopus, you would have a step that is deployment of your application. This is the one that you only want run when you are deploying version 18.x.
In the Channels section, add a channel and create a version rule for the channel. The package step will be the deployment step created earlier, and the version range will be something like [18.0.0,18.999.999.999)
The Version Rule Design screen gives you the ability to add sample version numbers to test your version expression.
This means that all deployments of version 18 will run the package step, and all others will skip it.
You can also get additional help via the Support channels
I'm looking for advice on the best approach to setup TeamCity/Octopus.
Currently I have multiple branches in TFS2015 - dev, main and release (currently we create a release branch for each release).
Our procedure is to develop in dev and deploy to dev environment. When we are ready for testing we merge from dev to main and deploy to test from main. When happy we create a release branch and deploy to live from the release branch. This is a manual process.
Hotfixes are done on the release branch and deployed to live. We then merge back to main/dev.
I'm totally new to this and so far in a VM playground I've setup TFS2015, TeamCity and Octopus and can check-in to TFS, build/create package on TeamCity and deploy this pack from Octopus. But...
I'm unsure how I should setup TeamCity and Octopus to work with multiple branches? Multiple projects for each branch and generate different artifacts?
When I do this for real I have a TFS VM, I plan on installing TeamCity and Octopus on this along with the build agent. Is this a bad idea? Should I create a new VM just for TM and Octopus?
Any advice or best practice would be appreciated.
Although your question is good in scope, but a good answer must cover many details to be complete.
Let me try to point out the main areas that you will need to further investigate and configure.
TeamCity
A VCS root can be configured to listen to multiple branches via a branch specification
A VCS root can contain multiple projects/solutions and these can be built in multiple steps within TeamCity.
Given that Team City does not support conditional build steps, you will need a different strategy to allow you to vary build steps (and parameters) per release channels / environments.
My recommend approach is to split up the builds into a build definition per release channel (target environment).
Dev and Feature branches could share a single build definition.
Master and Hotfix branches can share a single build definition since they both publish to staging/production environments.
Release branches will need a separate build definition and publish to QA/Testing environment.
This gives you fine grained control over parameter and configurations of each release channel. build a debug version of your app from Dev branch for example at major version 3, while build a release version from Master branch with major version 2.
Every build definition will have a step to publish its artefacts/packages to Octopus Deploy, and specify the channel of which the artifacts belong to.
Octopus Deploy
In Octopus Deploy, define the channels to reflect the release channels, that also reflect your branching model.
Develop, Test, Release are my standard goto channels
Each channel can enforce a different Lifecycle to limit the environments that a channel can deploy to and how an application progresses through your overall ALM cycle.
I know this is not a complete answer. but it is a good start that I hope can help you refine your question to more specific technical details.
We're having somewhat similar CI setup requirements except TFS. In our case workflow for most projects is: GitHub -> TeamCity -> Octopus Deploy.
So I'm not sure about multi-branch setup with TFS, but in case with GitHub repos it's pretty easy to configure in TeamCity. You just have to specify branch-related settings in your VCS root (see Branch configuration). When you have configured that, TeamCity will let you run build's for every specified branch separately and will display build statuses for every branch nicely.
In Octopus we use Channels feature to split workflows of releases coming from different branches. That means we have channel-per-branch convention for the projects, so that TeamCity is pushing packaged releases from particular branch (in our case it's develop and master) into it's respective channel in Octopus (see Channels in Octopus).
Probably you can setup all the services on single machine but imho it's not the best practice to do performance-wise and scalability-wise.
Off course I don't know you code etc but I think you should step away from merging from dev to test and then creating a version from test. That way you essentially are building a different application compared to the one you were having on dev. Once you merge from test to production and build your application from there, you are releasing a build you haven't been testing.
You should strive for a flow in which you build once and deploy multiple times. So, build one package which you promote from dev to test to production.
Off course you can have a production branch on which you could fix bugs etc. The Channels feature in Octopus works great for scenario's like that.
So answering my own question (sorry), the approach I ended up taking was to simplify my branches and configure TeamCity/Octopus like so...
Branching Strategy
I've moved from
--dev
--main
--release
----release1
----release2
to
--master
--release
----release1
----release2
Master is where most of the devs do their work, when we are ready for a release we have a cut-off point and merge master into a new release branch.
The release branch is deployed to test and any fixes from testing are made on the release branch.
When testing is complete we deploy to live/production from this branch.
This means that the binaries we have tested are exactly the same as the ones we deploy to live/production.
Teamcity
In TeamCity master is automatically built each time a check-in occurs. Then the package is pushed to Octopus. Octopus acts as a repository in this case. TeamCity also creates the release on Octopus. So it should be checkin->build->create release->deploy.
To do this, I have one VCS Root and have a build configuration called Build-Master. This uses the checkout rules to ensure I'm only using the master branch. I use the Ocotpus packaging to build the package then use the OctopusDeploy runner in TeamCity to create a release automatically and deploy to the dev servers.
Release is different. I want to deploy to the test servers manually rather than each time a check-in occurs. When happy promote this to the live production servers.
Any fixes from test will be made to the release branch and deployed to test.
When testing is complete we promote to live and any hotfixes are made on the release branch. Obviously all fixes/hotfixes are merged down to master.
So, in TeamCity to achieve this I have a build configuration called Build-Release. Again, I use the checkout rules to ensure I'm dealing with the correct release branch.
The build creates a package using OctoPack, however this time it's not pushed to Octopus.
Octopus
Octopus has a project specifically for deploying master to our dev servers, for example projectnamehere-dev.
In Octopus, I have a separate project for Test/Prod. I've setup an external feed which points at TeamCity so I can pickup the package created in TeamCity. This is setup in Library->External Feeds.
So, to deploy to test. I create the release branch in TFS and give it a version number, 1,2,3 etc. I then change the Build-Release build config to point at this new branch. Change the version number.
Then in Octopus, I create a release, select this package and deploy to test. Any fixes from testing are made on this release branch. I just build the package again and create a new release and choose the new package.
When testing is complete, in Octopus I just promote the last release to the live production servers.
Channels in Octopus are used on the two projects because they have different life cycles. I also created two new life cycles, the default is dev/test/prod. I created just a dev and then test/prod.
Hope this helps.
In the version control settings -> vcs -> Branch Specifications: "*" ("This will do all branch, filter as needs be" e.g. +:refs/heads/master +:refs/heads/develop)
enter image description here
Octopus doesn't handle branches, it only deploys, you can however use their rest api, so for example, if test pass in develop then call the octopus rest api to create a new release and deploy.
If I have three branches ("Dev", "Main", "Release") and Main branch is built, deployed to QA for testing, does it make more sense to deploy the same binary files from QA to Staging/Production, or merge to Release branch first, build and then deploy to Staging/Production?
You want to keep using the same binaries across all systems, that way you can setup a deployment pipeline, so build once, create a Label or a Branch in case you ever need to service this version, then build from Main and release that. It's enough to create a label (and the build server can do that for you automatically).
That way, only in case you need to create an old service version, do you need to do extra work. Create the Branch from the label, fix the bug, clone the build definition from main, build, merge back to Main if needed, release.
In most cases you should be trying to keep Main stable, so that when you need to release a hotfix you can simply release the latest version from Main.
For extensive guidance check the ALM Rangers' Version Control Guidance.
Our development team has been fairly small and, until now, all working on a single Visual Studio 2012 solution. We are growing and wanting to create better separation with multiple solutions for different project teams.
However, there are occasions where the code in one solution will want to utilize code from another. We have decided using internal (i.e. private) NuGet packages will be a good way to manage these dependencies.
However, the question has come up on how to deal with multiple versions of the same package that are in different SDLC stages (e.g. Development, QA, Staging, Production, etc.)
Example: If we have these three solutions...
CoreStuff
CoolProject1
CoolProject2
If working in CoolProject1, and we need to utilize code from CoreStuff, we can add the NuGet package. Presumably this package will be the latest Production (stable) version of CoreStuff.
However, what if a developer working on CoolProject2 is aware of some changes in CoreStuff that are currently in Development and wants to utilize that version?
Not sure if the best approach is to create separate packages for each (seems to require changing your package references back and forth depending on what stage the solution is in) or somehow utilize multiple versions of the same package (not sure if that's easy to manage with NuGet).
Anyone tackle something like this?
The first thing to remember is that NuGet will not automatically update your package references, so if you have already 'linked' your solution to the latest stable package of CoreStuff (say 1.2.2) then there won't be any problems if a newer (unstable) version is provided (assuming that the package you're using doesn't disappear from the package repository). Obviously if you upgrade your package reference then you will get the unstable package.
So the simplest solution is to make sure that you 'link' your project to the stable package by getting it via the NuGet package manager before the other package is released. While the UI only allows you to get the latest version, the Package Manager Console can get any version of a package so you could use that to explicitly provide the version number, e.g.:
Install-Package CoreStuff -Version 1.2.2 -Project CoolProject1
If that is not a solution then there are several other options to tackle this problem:
Give the development version a different semantic version that indicates it is a unstable version, e.g. 1.2.3-alpha. In this case CoolProject1 could pull in package CoreStuff.1.2.2 (which should be latest stable version in your repository) and CoolProject2 could pull in CoreStuff.1.2.3-alpha (which would be the latest unstable version).
Have multiple repositories, e.g. one for stable (released) packages and one for unstable (development) versions. Then you can select your packages from the desired repositories. If you wanted to you could make it so that only your release process can push packages up to the stable repository and your CI build pushes up to the unstable one (so that you always have the latest packages available)
If the developer of CoolProject2 just wants to develop against the latest version (but will wait to release CoolProject2 until after CoreStuff v.next has been released) then he could potentially create a local package repository (i.e. a directory on his drive) and put the new package of core stuff there. That way other developers won't even see the package.
The most important thing will be to make sure that you don't get CoreStuff.1.2.2 and CoreStuff.v-next in the same repository if CoreStuff.v-next simply has a higher version number, because in that case the NuGet UI won't let you pick v1.2.2 (but the Package Manager Console does!).
If you would want to switch from one package type to another you'd have to do a manual update (which you always have to do when changing to the next package version anyway), but that's not a bad thing given that this forces a developer to at least check that the update of the package doesn't break anything.