We are using Perforce and Maven and we are in a situation where we have one branch A that hosts a mature project and another branch B, that we are starting to dev on that uses the code hosted in the first branch.
I anticipate that going forward, there could be cases where a check-in into Branch A will result in failed compilation in Branch B because the developers working on them are not aware of specific usages and cross-dependencies. And then devs will waste time updating B code and figure that its not compiling.
Branch/project A needs to remain as its own separate entity because it's code could be used in another future project.
Can someone advice me around handling and alleviating such problem? Any best-practices that you guys can suggest? Much appreciated.
You need to set up a build environment:
1) Set up a continuous integration (CI) server, such as Jenkins. When a developer commits to Branch A, it should detect the commit and trigger a build.
2) When Branch A completes its build, it deploys its artifacts to a shared repository. Highly recommended (buy not strictly necessary) is that the build for Branch A deploys to a Repository Server, such as Nexus (but a repository in source control or a FTP server does the job too).
3) Set up Branch B in your CI server, but with a build "trigger" that runs whenever a build for Branch A succeeds.
4) Configure both build configurations to email relevant team members (or everybody?) whenever a build fails.
Also, both branches should depend on your repository from #2 above. If you need the entire environment to be repeatable and self-contained, go the repository-in-scm route.
Related
Doing trunk based development to achieve continuous deployment. And this is our branching strategy.
master > whats live in production
release > test passed and release point created by CI server
dev > daily merges from the development team.
If we consider doing the pull request from release to master stage.
What are the pros and cons to that approach? How can we communicate this with the development team where they want to do PR at dev branch?
I don't really have an answer, but think the context of the question is worth further discussion.
If you're doing Continuous Deployment, then I'm not sure of the purpose of the Release branch. It seems to be duplicating the purpose of either:
'master': code being deployed/released
'develop' integrated feature
branches, but not ready release
Or, are you using it to group milestones or planned major releases (i.e. Release/1.0, Release/2.0), like a mini-master branch.
I would not consider this Continuous Delivery (deployment, maybe), but it is certainly a valid pattern if your project requires staged releases rather than Continuous Delivery.
It's also important to also consider your CI setup and how it integrates with your branches. It isn't source code that is deployed to Production, but the build artifacts from your CI system. Thinking about this might simplify your branching model.
If you want to rollback, you don't want to rebuild the application from source, you want to re-deploy the pre-built artifacts of the previous version. Likewise, if you have a build that has passed all your tests and is ready to ship - hopefully is already running on your pre-production environments - you don't want to merge it to a different branch, rebuild it and then deploy the new build - you use the build that was tested.
The next consideration is that every additional branch adds time & complexity for the developers. Merging, pull requests, waiting for CI to run, etc. are not free so reducing the complexity of the branching strategy to the minimum you require is a good goal.
To answer your question about where to PR to/from, do you consider "Develop" as the mainline and attempt to keep it stable and working at all times?
If so, then the PRs from feature-to-develop are the key integration. Develop is then built, tested, and deployed to your test environments.
Branching from develop at that point (i.e. to create a Release branch) is then known to be good.
Promoting that artifact from your test environment to production, without re-building, might remove the need for one of your branches.
I'm looking for advice on the best approach to setup TeamCity/Octopus.
Currently I have multiple branches in TFS2015 - dev, main and release (currently we create a release branch for each release).
Our procedure is to develop in dev and deploy to dev environment. When we are ready for testing we merge from dev to main and deploy to test from main. When happy we create a release branch and deploy to live from the release branch. This is a manual process.
Hotfixes are done on the release branch and deployed to live. We then merge back to main/dev.
I'm totally new to this and so far in a VM playground I've setup TFS2015, TeamCity and Octopus and can check-in to TFS, build/create package on TeamCity and deploy this pack from Octopus. But...
I'm unsure how I should setup TeamCity and Octopus to work with multiple branches? Multiple projects for each branch and generate different artifacts?
When I do this for real I have a TFS VM, I plan on installing TeamCity and Octopus on this along with the build agent. Is this a bad idea? Should I create a new VM just for TM and Octopus?
Any advice or best practice would be appreciated.
Although your question is good in scope, but a good answer must cover many details to be complete.
Let me try to point out the main areas that you will need to further investigate and configure.
TeamCity
A VCS root can be configured to listen to multiple branches via a branch specification
A VCS root can contain multiple projects/solutions and these can be built in multiple steps within TeamCity.
Given that Team City does not support conditional build steps, you will need a different strategy to allow you to vary build steps (and parameters) per release channels / environments.
My recommend approach is to split up the builds into a build definition per release channel (target environment).
Dev and Feature branches could share a single build definition.
Master and Hotfix branches can share a single build definition since they both publish to staging/production environments.
Release branches will need a separate build definition and publish to QA/Testing environment.
This gives you fine grained control over parameter and configurations of each release channel. build a debug version of your app from Dev branch for example at major version 3, while build a release version from Master branch with major version 2.
Every build definition will have a step to publish its artefacts/packages to Octopus Deploy, and specify the channel of which the artifacts belong to.
Octopus Deploy
In Octopus Deploy, define the channels to reflect the release channels, that also reflect your branching model.
Develop, Test, Release are my standard goto channels
Each channel can enforce a different Lifecycle to limit the environments that a channel can deploy to and how an application progresses through your overall ALM cycle.
I know this is not a complete answer. but it is a good start that I hope can help you refine your question to more specific technical details.
We're having somewhat similar CI setup requirements except TFS. In our case workflow for most projects is: GitHub -> TeamCity -> Octopus Deploy.
So I'm not sure about multi-branch setup with TFS, but in case with GitHub repos it's pretty easy to configure in TeamCity. You just have to specify branch-related settings in your VCS root (see Branch configuration). When you have configured that, TeamCity will let you run build's for every specified branch separately and will display build statuses for every branch nicely.
In Octopus we use Channels feature to split workflows of releases coming from different branches. That means we have channel-per-branch convention for the projects, so that TeamCity is pushing packaged releases from particular branch (in our case it's develop and master) into it's respective channel in Octopus (see Channels in Octopus).
Probably you can setup all the services on single machine but imho it's not the best practice to do performance-wise and scalability-wise.
Off course I don't know you code etc but I think you should step away from merging from dev to test and then creating a version from test. That way you essentially are building a different application compared to the one you were having on dev. Once you merge from test to production and build your application from there, you are releasing a build you haven't been testing.
You should strive for a flow in which you build once and deploy multiple times. So, build one package which you promote from dev to test to production.
Off course you can have a production branch on which you could fix bugs etc. The Channels feature in Octopus works great for scenario's like that.
So answering my own question (sorry), the approach I ended up taking was to simplify my branches and configure TeamCity/Octopus like so...
Branching Strategy
I've moved from
--dev
--main
--release
----release1
----release2
to
--master
--release
----release1
----release2
Master is where most of the devs do their work, when we are ready for a release we have a cut-off point and merge master into a new release branch.
The release branch is deployed to test and any fixes from testing are made on the release branch.
When testing is complete we deploy to live/production from this branch.
This means that the binaries we have tested are exactly the same as the ones we deploy to live/production.
Teamcity
In TeamCity master is automatically built each time a check-in occurs. Then the package is pushed to Octopus. Octopus acts as a repository in this case. TeamCity also creates the release on Octopus. So it should be checkin->build->create release->deploy.
To do this, I have one VCS Root and have a build configuration called Build-Master. This uses the checkout rules to ensure I'm only using the master branch. I use the Ocotpus packaging to build the package then use the OctopusDeploy runner in TeamCity to create a release automatically and deploy to the dev servers.
Release is different. I want to deploy to the test servers manually rather than each time a check-in occurs. When happy promote this to the live production servers.
Any fixes from test will be made to the release branch and deployed to test.
When testing is complete we promote to live and any hotfixes are made on the release branch. Obviously all fixes/hotfixes are merged down to master.
So, in TeamCity to achieve this I have a build configuration called Build-Release. Again, I use the checkout rules to ensure I'm dealing with the correct release branch.
The build creates a package using OctoPack, however this time it's not pushed to Octopus.
Octopus
Octopus has a project specifically for deploying master to our dev servers, for example projectnamehere-dev.
In Octopus, I have a separate project for Test/Prod. I've setup an external feed which points at TeamCity so I can pickup the package created in TeamCity. This is setup in Library->External Feeds.
So, to deploy to test. I create the release branch in TFS and give it a version number, 1,2,3 etc. I then change the Build-Release build config to point at this new branch. Change the version number.
Then in Octopus, I create a release, select this package and deploy to test. Any fixes from testing are made on this release branch. I just build the package again and create a new release and choose the new package.
When testing is complete, in Octopus I just promote the last release to the live production servers.
Channels in Octopus are used on the two projects because they have different life cycles. I also created two new life cycles, the default is dev/test/prod. I created just a dev and then test/prod.
Hope this helps.
In the version control settings -> vcs -> Branch Specifications: "*" ("This will do all branch, filter as needs be" e.g. +:refs/heads/master +:refs/heads/develop)
enter image description here
Octopus doesn't handle branches, it only deploys, you can however use their rest api, so for example, if test pass in develop then call the octopus rest api to create a new release and deploy.
Wondering how people manage their project artefacts through an environment lifecycle of say DEV - AQA - CQA - RELEASE and if there's some best practices to follow.
I use a Jenkins build server to build my projects (code checkout then maven build). My artefacts all have version 1.0.0-SNAPSHOT and are published to a local .m2 repo on the build server. There are also Jenkins jobs that rebuild the DEV system (on the same server) using those artefacts. The project build is automated whenever someone checks in code. The DEV build is automated on a nightly basis.
At some point, my lead developer determines that our project is fit to go to AQA (the first level of testing environment on a different server).
For this I need to mark the artefacts as version 1.0.0-1 and publish to a remote AQA repository (it's actually a Nexus repo).
The Maven deploy plugin sounds like the right approach, but how do I change the version number to be effectively 1.0.0-$release (where $release is just an incrementing number starting from 1)? Would Maven/Nexus be able to manage the value of $release, or would I need a simple properties file in my project to store/update the last used $release.
Furthermore, someone tests AQA and determines its fit to move on to CQA (second testing env). This is 'promote to AQA'. So my requirement is to copy the artefact from the AQA Nexus repo and publish to the CQA Nexus repo.
Likewise, after CQA, there'd be a 'promote to RELEASE' job too.
I think the version value remains unchanged during the 'promote' phases. I'd expect the AQA repo to see all versions 1-50, but CQA only 25 and 50, then RELEASE only 50, for example.
I can find loads of info about Maven plugins/goals/phases, but very little about a prescriptive method on how or where to use outside of the immediate development environment.
Any suggestions gratefully received.
Staging/promoting is out of scope for Maven. Once deployed/uploaded to a remote repository, that system is responsible for the completion of the release cycle. Read this chapter about staging: http://books.sonatype.com/nexus-book/reference/staging.html if you use Nexus.
Build numbers are just that build numbers. They are not promotion / staging numbers.
You should come up with another means of tracking your promotions, because otherwise one might get confused in "knowing" that build 10.1.4-2 is the same as 10.1.4-6. Certainly, all of the Maven related software will see those two builds as different builds.
In addition, if a person "grabs" the wrong copy of the build, the way you are managing staging within your build number will increase confusion. As if you don't kill all of the 10.1.4-2 builds, then someone might get a copy of that not realizing that the build has been promoted to 10.1.4-6. This means that for the "last" staging number to be the most likely one to be grabbed, you must do two things (which are impossible in combination)
Remove all the old staging numbers, updating them to the new ones.
Ensure that no copy of an old staging number escaped the update.
Since people generally can copy files without being tracked, or said files might not be reachable at time up "update", or timing between reaching all the files cannot be simultaneous, such a system is doomed to fail.
Instead, I recommend (if you must track by file), placing the same file in different "staging directories". This defines release gateways by whether the file exists in a certain directory, and makes it clear that it is the same file that is going through the entire process. In addition, it becomes easy to have various stages of verification poll their respective directories (and you can write Jenkins tasks to promote from one directory to another, if you really wish).
I am using teamcity 8.0 with github.
I have a continuous deployment build configuration set up to look at our master branch and when changes occur they are built and deployed.
When code is ready for deployment it is merged in from another branch into master, so master is always a snapshot of the last deployed code. ongoing development happens on other branches.
The problem we have is that when we deploy there is usually at least 20 commits and often many more from a mix of developers. Team city builds and deploys each commit separately which takes an age and leaves our live system unusable while this is occurring.
Is there a way to configure team city to just build the most recent code when it detects a change instead of every individual commit
Review your VCS trigger settings. "Per-checkin Triggering" should be turned off.
Also, check if you are using GitHub Service hook for "TeamCity" - it may also produce number of builds.
I have been tasked with updating our build process right now to be more efficient and I have spent a week reading best practices and strategies but I still have not found a solution for our current problem.
Background
Currently we have one monolithic build/application that really needs to be split apart into at least 4 applications with some shared libraries. We currently do not branch unless we absolutely have to. We have a teamcity build that builds on each check-in to TFS. When we are ready for a release we have a code freeze and only check-in fixes for bugs found in QA. Obviously this is a terrible practice and we have finally gotten approval to change it.
Proposed Solutions
The proposed solution will be to split up the application and have different release cycles for each application, move from ant to maven and branch per release.
Branching - Right now we just have a main trunk in source control. I think we want to branch off the trunk when we are ready for a release, and update the branch for bugs found in QA. When the build is ready to be released, merge the branch changes back into the trunk.
Here is how I was planning on setting up TFS.
+Apps
+App1
+Components
+Core
+Web
+Branches
+App2
+Components
+Core
+Web
+Branches
+Libraries
+Lib1
+Lib2
+Branches
Thinking about managing all of the POMs and versions in the POMs seems WAY too difficult right now. I've read up on the maven release plugin, but I'm not sure if it can branch in the way I'm thinking we want to.
Next problem is getting teamcity working. I was thinking of having 3 teamcity projects for each app. A dev project that always points at the trunks, a QA project for testing the QA build and a production project to build changes for hotfixes. Each time a new release comes to QA I would have to update the QA teamcity project to point at the new release branch and update the release build number in teamcity. When that release passes QA I would have to update the production teamcity project to point that the branch that just passed QA and update the build number to the build number that just passed QA.
Surely there is a better strategy that this.
Questions
Where should I be putting these branch folders?
Should QA builds be snapshots still until the build goes to pre-production?
How do you configure teamcity to pick up these branches without changing the sources path for every release?
Should there be parent POMs for each app that the developers use to make sure all of their dependencies are compiled and up to date?
I just want to question your thinking that your applications should be on different release cycles. Modularization is a good thing for code quality but if your modules are on separate release cycles you introduce a lot of overhead. In particular, version management becomes quite a burden and if you get it wrong you can introduce runtime bugs.
How do these separate applications relate to each other? Is there any dependency between them (maybe via a shared library)? Do they communicate with each other? Are they deployed together?
If it is not necessary that they be on separate release cycles then you're probably better off keeping them together.