I have referred the article: https://learn.microsoft.com/en-us/vsts/git/concepts/git-branching-guidance?view=vsts for knowing more about Branching concept. And if my understanding is correct, there should be a master branch, then a release branch then a support branch and a feature branch in common.
And the merging between the branches should be defined as below:
Create the master branch(with code added to it).
A release branch is then created from the master branch(else know as a topic branch).
Then create a support branch to fix bugs from the release branch and then merge them back into the release branch in a pull request.
Create a new feature branch off the master branch to port the changes. Cherry-pick the changes from the release branch to your new feature branch. And then merge the feature branch back into the master branch in a second pull request.
Coming to the question, assume that i have 4 environments, such as - Development, Test, Pre-Production & Production. Here i need to have a branching and merging mechanism and needs to setup cicd pipeline in VSTS.
How will i define the CICD pipeline for the above above case, if i use MS recommended branching and merging mechanism? Whether all the deployments will be done from the master branch only? (That's from Master branch, build & deploy to-> Dev -> Test -> Pre prod -> Prod environment?). Do i need to note of any other things in this?.
Or do i need to have a separate branching and merging mechanism, such that i need to have separate branches for each of the four environments and should define separate pipelines like the below screen?
For the standard branching model, you can refer A successful Git branching model. It's a wide used branching structure which is also applied for gitflow. And based on the branching model, you can CI/CD to dev environment (by develop branch) and production environment (by master branch). If you design to deploy four environments separately, you can adjust the branching model correspondingly.
And each environment is for a single branch only. Such as for Dev environment, the CI/CD aims to check the code from develop branch. Only when the code from develop branch has been qualified, you can merge it into master or create a release branch to prepare the next release for production.
Related
We are using codepipeline and codebuild with bitbucket cloud for source repositories.
We won't be releasing frequently and our releases are heavily regulated (Reviews by customers in a staging environment first) before releases etc.
We have separate branches for develop, staging, and production environment as the code pipeline doesn't support tag-based releases and feature branches will be frequently merged to the develop branch. The develop branch will be infrequently merged to staging the staging will be merged to master.
We also have a code pipeline per branch that gets triggered when there are updates in those branches.
We currently create pull requests manually from develop to staging and staging to master to trigger the respective pipelines.
I would like to understand how should the current set up be modified (minimally) to make it a true CI-CD but without a complete overhaul of the CI-CD pipeline.
How should I be making sure that the staging/production pipeline should not go through unless the develop pipeline had been triggered for the same code ( I was thinking of updating the list of versions in SSM when develop pipeline deploys so that if the version in staging/master is not found in SSM, the pipeline won't go through. Is there a better way of doing it? should I be doing this or taking a different approach?
P.S: Feature flags don't sound like a good option because we will end up with a lot of code with just feature flags since infrequent deployments. Yes, I know how trunk-based branching is what everyone keeps talking about but I don't see how is it sitting well for us with code pipeline (No tag-based triggers) and infrequent deployments to staging/prod?
It seems to me that your build and deployment process accurately reflects your release process.
I'm assuming in your company something like the following applies:
You need the production branch for hot-fixes to production. These changes need merged back to staging.
Upcoming releases (prod +1) sit in staging for a long time. You
continue to make changes on the staging branch that need merged back
to development and forward to the production branch.
Future releases (prod +2) are done on the development branch and this
code needs merged to the staging branch when ready.
You spend a lot of time managing code merges (and likely firefighting
the problems caused by code regressions)
The C in CI\CD stands for 'Continuous'. CI/CD is designed to accelerate frequent releases to production yet you explicitly say you release infrequently.
Your questions:
Closer to true CI would be to use trunk based development with short lived feature branches. Your pipeline would build a single deployment artefact and deploy this same artefact through develop, staging and production.
Feature flags or equivalent would be used to keep your code always release-able - there are no tag based triggers needed as every commit is potentially release-able.
You have indicated you understand this to some extent but don't feel it applies to your operating environment.
If you must have different pipelines running off different branches for different environments, to me this is a question of your code management process and not for code-pipeline. You should mainly handle this at source, which is in your code management tools and processes.
Your question 2 indicates to me you are not confident in this process internally at your company and are looking inside AWS to fix problems outside of AWS.
This is all a reflection of the unfortunate reality of having long lived branches for different environments.
This is exactly the scenario that that CI was developed as an antidote to but you are discounting two of the main techniques needed for the benefits of CI/CD ( Trunk based development and feature flags) and your release cycle does not support one of the main benefits, i.e frequent releases to production.
Every organisation has their own constraints and particular trade-offs and I of course don't have the full picture.
In my opinion the underlying issues in your question are a direct result of how you release and manage your code and you can't expect the benefits of CI/CD without the practices, which does mean a fundamental overhaul of how you work.
So i have master (Main Branch) and develop as long lived branches, i want to derive a feature from develop, this means that the project will merge with develop and not master.
Currently i am passing the arguments:
sonar.branch.name=feature-abc
sonar.branch.target=develop
This gets ignored and i get "feature-abc from master", is there a way to force a branch to derive from a branch?
Doing trunk based development to achieve continuous deployment. And this is our branching strategy.
master > whats live in production
release > test passed and release point created by CI server
dev > daily merges from the development team.
If we consider doing the pull request from release to master stage.
What are the pros and cons to that approach? How can we communicate this with the development team where they want to do PR at dev branch?
I don't really have an answer, but think the context of the question is worth further discussion.
If you're doing Continuous Deployment, then I'm not sure of the purpose of the Release branch. It seems to be duplicating the purpose of either:
'master': code being deployed/released
'develop' integrated feature
branches, but not ready release
Or, are you using it to group milestones or planned major releases (i.e. Release/1.0, Release/2.0), like a mini-master branch.
I would not consider this Continuous Delivery (deployment, maybe), but it is certainly a valid pattern if your project requires staged releases rather than Continuous Delivery.
It's also important to also consider your CI setup and how it integrates with your branches. It isn't source code that is deployed to Production, but the build artifacts from your CI system. Thinking about this might simplify your branching model.
If you want to rollback, you don't want to rebuild the application from source, you want to re-deploy the pre-built artifacts of the previous version. Likewise, if you have a build that has passed all your tests and is ready to ship - hopefully is already running on your pre-production environments - you don't want to merge it to a different branch, rebuild it and then deploy the new build - you use the build that was tested.
The next consideration is that every additional branch adds time & complexity for the developers. Merging, pull requests, waiting for CI to run, etc. are not free so reducing the complexity of the branching strategy to the minimum you require is a good goal.
To answer your question about where to PR to/from, do you consider "Develop" as the mainline and attempt to keep it stable and working at all times?
If so, then the PRs from feature-to-develop are the key integration. Develop is then built, tested, and deployed to your test environments.
Branching from develop at that point (i.e. to create a Release branch) is then known to be good.
Promoting that artifact from your test environment to production, without re-building, might remove the need for one of your branches.
I am having a bit of trouble with the Branch Updater feature of Bamboo.
If a new commit is pushed on a branch that is using the branch updater, Bamboo checks if there are new commits on the dependent branch(in my case the master branch) and if this is the case it merges the changes and runs the build.
However, if this build fails, it is hard to recognize, if the new code on the branch itself is broken or if it just could not be merged correctly with the master. Therefore I would like that in these cases the branch is build independently first and only if this build is successful, the changes on the master are merged in and the branch is build again.
Is this possible using the branch updater feature?
A workaround might be to use multiple build plans and use one to build all branches isolated and another plan to integrate everything using the branch updater.
The "Branch Updater" feature is to merge the code from master branch into feature branch after a successful build of the master branch.
However, currently there is NO straight forward option in Bamboo (except the workaround mentioned in your OP) to build the branch and auto merge only if the branch build is successful.
Typically, the developer runs the build in local and commit the changes to repository if the local build is successful. There are two things that can go wrong.
1) While running the local build, the latest code from feature branch is not available in local. Some of the repos like GIT would complain if the local doesn't match with HEAD revision, you need to update to HEAD before you commit the changes. In this case, you can run the build again to ensure there is no problem.
2) When the master code merges with feature branch, something could go wrong
I understand that still there is a gap which can be addressed to resolve the point 2.
You can create a ticket in Bamboo with detailed explanation of the use case to add this feature.
When I was using
mvn jgitflow:release-finish
I noticed that the release branch got merged into master branch.
Question: Is this the correct way?
Sorry my questions might be naive as I am new to this. I was thinking that the code from release branch will merge to develop and master and not like release --> master --> develop.
Question: What if I don't want this to happen and instead I should be in a position to rebase develop from master?
When I was using mvn jgitflow:release-finish I noticed that the release branch got merged into master branch. Is this the correct way?
This is the correct way according to the main philosophy behind, gitflow:
Release branches
May branch off from:develop
Must merge back into: develop and master
And according to the plugin documentation, the release-finish indeed merges back to the master and dev branch:
finishing a release - runs a maven build (deploy or install), merges the release branch, updates pom(s) with development versions
This makes sense, because (again back to gitflow):
When the state of the release branch is ready to become a real release, some actions need to be carried out. First, the release branch is merged into master (since every commit on master is a new release by definition, remember). Next, that commit on master must be tagged for easy future reference to this historical version. Finally, the changes made on the release branch need to be merged back into develop, so that future releases also contain these bug fixes.
I was thinking that the code from release branch will merge to develop and master and not like release --> master --> develop.
The order follows this flow (first master, then develop) because it's a release and as a release it must firstly go to master (which should always represent the released code base), then to develop (whish is the next potential release code base).
What if I don't want this to happen and instead I should be in a position to rebase develop from master.
You can use the noReleaseMerge option:
Whether to turn off merging changes from the release branch to master and develop
Default value is to false, hence by default merges are performed. However, the option covers the two merges, you cannot disable only one of them, it's either both (again, following gitflow philosophy) or none. This option may suit your needs but you would then perform additional actions via git commands.