I am currently setting up a CI system that will check for a passing deployment against a test environment as part of a pre-merge pull request check. This system is using Bamboo and Bitbucket, and will stop devs from merging their feature branches into the main branch if this validation fails. However, I am running into the (possibly common on my project) corner case of multiple pull requests being open at the same time, passing validation, and then being merged. In this scenario the PRs might all separately pass validation while all of them combined would break the build (I.E: PR#1 modifies a method name referenced by PR#2).
Is there a way to configure Bitbucket / Bamboo to rerun builds on pull requests if the target branch has been modified since the check last ran?
On git (bitbucket) level you could make sure to synchronize any feature or bugfix branch with an outgoing pull request by merging the latest common target branch (for example develop) immediately after a successful pull request merge to this target. This way you invalidate the 'latest' build result on feature or bugfix branches and they would be re-built because their git commit hash has changed. If any re-build of the feature branch fails, it won't be merged.
Ulrich
// Izymes - our mission is to eliminate boring from work. We build apps that turbo-charge team velocity through contextual automation.
I have defined a clean-up stage in Gitlab CI scipt that is used to automatically shut down the instance after the branch's testing, and I would like to have this stage being automatically triggered once I merged this branch into the master through the merge request in Gitlab, how should I implement this?
If the merge goes into develop you could use
cleanup:
stage: cleanup
only:
- develop
That way it'd be trigger by the merge rather than the merge request.
Anyhow, I'd say that what you aim to do should be done directly in the same pipeline and not wait until it is merged though
I am confused why the following 2 things happen:
When I push some commits to my feature_foo branch, 2 workflows (builds) are run: the primary workflow against the latest commit, and deploy workflow against my last PR, both on feature_foo. I expected only the primary workflow to be run as I haven't issued a PR yet
2 identical email notifications are sent to me from artifacts+\<my-bitrise-project-id\>#bitrise.io within the same minute. I understand that a PR can lead to two builds (as a PR is technically a push), but doubt that is the issue here as I've not created a PR yet.
Here is my current bitrise.yml trigger map:
trigger_map:
- push_branch: "*"
workflow: primary
- pull_request_source_branch: "*"
pull_request_target_branch: feature
workflow: deployment-staging
- tag: "v*.*.*"
workflow: deployment-production
By the way, this is my desired 3-workflow setup:
Run integration tests (primary workflow) on 2 occasions:
Code push to * (any branch)
Pull requests to feature branch (when the PR is created i.e. pre-merged state so contributors can preview the potential effect of their proposed changes)
Run deployment (deploy workflow) to staging when PRs from * to feature branch are merged
Run deployment (deploy workflow) to production when tags v*.*.* are pushed
What is the correct bitrise.yml config to achieve this? The docs do not indicate how we can differentiate PRs by state (issued vs merged). I want to deploy only after the code has been reviewed.
Thanks
If you open the PR will that trigger another build? Are you sure the PR isn't opened yet?
To answer
I want to deploy only after the code has been reviewed.
I guess you mean when the PR is reviewed and merged into the target branch e.g. into master.
For that you can use a config like this: https://devcenter.bitrise.io/builds/triggering-builds/trigger-map/#dont-start-two-builds-for-pull-requests-from-the-same-repository
trigger_map:
- push_branch: master
workflow: deploy
- pull_request_target_branch: "*"
workflow: primary
This will run a build using the workflow called primary when you open the PR and every time you update the PR. Typically you want to run some automated tests in this case, in the primary workflow (unit/ui tests, linters and/or doing maybe a test build).
Then when you merge the PR (in this case into the master branch) it'll trigger a build using the deploy workflow (as technically a merge generates a commit/push event).
I hope this helps, let me know if you have any questions!
Viktor's answer is sufficient, but I wanted to add some more findings that might be relevant to someone else:
When I push some commits to my feature_foo branch, 2 workflows (builds) are run: the primary workflow against the latest commit, and deploy workflow against my last PR, both on feature_foo
I believe this happened because I had an open PR and pushed additional commits to the source branch of that PR. Based on my trigger map (shared above on the OP) at that time, it would run a deploy-staging workflow. The trigger map shared by Viktor makes more sense for my use case
2 identical email notifications are sent to me from artifacts+\#bitrise.io within the same minute
Turns out Bitrise sends both a signed and an unsigned APK (for whatever) in two separate emails
We have 3 main branches: development, releases/xyz and master. This mirrors the flow from development through release candidates to production.
I want to trigger a job only when a merge request is made from releases/* to master. I need to block all other merge requests to master e.g. to prevent a merge directly from development to master.
Does Gitlab CI support a trigger ONLY when branch is master AND the originating branch matches releases/* ?
GitLab does not support triggering a pipeline for merge commits only so far. Therefore it does also not support triggering a pipeline only for some merge commits.
There are two ways you might differentiate merge commits from normal commits:
Merge commits have multiple parents.
Merge commits have a standardized commit message in regular cases.
GitLab does not support filtering based on number of parents nor on the commit message (except for skipping CI with [skip ci] or [ci skip]).
You could find more information about support filtering options in the documentation for only and except options of GitLab CI/CD Pipeline Configuration.
I have used Git for trying out but I'm pretty sure I don't use it correctly.
I would like to know how to use Git to become more professional.
I want to start from fresh so from a situation where no Git is installed.
Of course I have read the manuals and Googled a lot but I'm just not sure how to use it the correct way.
Right now we have a dedicated unix server for the website.
I use PhpStorm for webdevelopment on Windows and I have a college who make small 301-redirect changes in .htaccess.
And maybe in the future we will expand with another programmer.
I want to be able to track the changes made by him also, or is Git overkill for just adding some redirects to .htaccess?
What is the best way to use git in this situation?
Where should the master be, etc.
I'm very new to Git so please be specific with the explanation.
I also heard that it is possible that I have to review the commit from my college before it goes online, I would like to work that way.
Also do I need to update master with the new code or will it all be seperate branches, etc.
Let me explain, in simple and minimum words.
Git not necessarily used only for multi user environment, but it is same or more useful standalone for individual developer. When start we should first utilize all its function as a individual developer, than collaboratively manner.
Let’s we created the git repository on local system ( using git init ) , and developer working on ( either master or branch , will take up later.. ), during development cycle he/she should identify very small but complete unit of work which is working, i.e. button is added and its doing some basic functionality, and it should be staged ( git add ) and committed ( git commit ) with proper detailed. I as a developer used to do 5 to 10 commits daily in local repository.
Advantage of stage 2 is, we can go back in time anytime.. and in case of any issue within seconds ..make local repository as it was during commit no. 3 or commit no. 10 ( git checkout .. commit no. ) its extremely useful during debugging, and having clear picture for further planning . Even it will be very easy to catch any issue which raised after any specific commit (git diff will show diff between any 2 commits ). so we have very good control on our files through git..
Now instead of working in master , we can prefer to work in the branch. For example before each new functionally , a branch can be created ( git branch branch_xyz, git checkout branch_xyz ) , and we can make certain rules (not by git) like ( code review completed, unit testing, component testing ) is not completed, code will not be merged to master.. reviewer, tester, or interested project team, can pull the specific branch in their system.. for their piece of activity. Either its master or branch, commit logic remain same and all mentioned in point 2 will remain same. Few commits which we want to identify as milestone can be tagged i.e. ( git tag -a build_3.4 -m “build description ) . After all verification, code can be merged to master ( git checkout master, git merge branch_xyz )
All above mentioned was the minimum functionally can be used on local system itself, without any server. Now in multi user environment can choose to have repository on our own server, or we can use repository like github or bitbucket.. I use bitbucket for my code..Its not necessary to push only final changes to main repository ( don’t wait)... we can keep pushing our changes.. to main repository ( git push –all -u remotename ) as we were doing it locally, if locally it was part of branch, in main also it will be part of same branch, and if locally it was merged to master, than only it will be merged to master at server..
Other team members who want to work.. they can use git clone or git pull command to get copy from main repository.. and post that they can follow same steps mentioned in point 2 onward..
Git in Multi User Environment
Q - Where is the best place to create the master (with git) is it on the dedicated server where the website is also hosted or on a separate installed server in-house?
You need not to call it master, because git server and git node, they all will have master ( main code ) and branches ( WIP Code ) , it is explained below
Main Git server need not to be a dedicated server, also need not be on same machine where web site is hosted. It can be anywhere as long as you can
download and prepare build, which is being used for deployment. However git Main server/repository should not be used for development purpose, but not necessary.
Q - After creating the master I need to clone the master to my own computer and start developing. After that I need to commit the changes to the git-server?
Lets assume 3 member in your team 1) Ron - Developer 2) Gerard- Developer 3) Sytse - Tester , your current build is build_4_01 sitting on main git server in master branch, and as of today none of your team member have got code in their system.
to start with all these 3 guys have installed git on their local system, and clone it from main server. Now they have build_4_01 in local git repository under master branch
next Ron was asked to work on next functionality which will be named as build_4_02
Ron first will create a branch in his local git repository i.e. branch_build_4_02, and complete the code, under branch_build_4_02.
there were altogether 20 small functionalities under build build_4_02, and Ron completed it in 10 days.
after completion of each small functionality, Ron used to execute a commit on local repository
after each commit, or at the end of day, Ron will push code (git push ) to main server ( no worries, main server will not mess up with existing code. What Ron have pushed is inside the branch branch_build_4_02 , and main server keeping master and branch separate efficiently )
during these 10 days of his development time, many times Ron wanted Gerard's help to review the code, or help him technically
hence Gerard need all latest code from Ron. Every morning Gerard will execute git pull ( command ) , and it will sync up his local git repository with all latest changes Ron have done, Gerard can see them inside the branch_4_02
After completion of coding ( build_4_02) Sytse (Tester) was informed, and he also like Gerard executed a pull request in his local system , and got the latest code
Sytse deployed the code from git branch 4_02 on test server and completed the testing
during testing some defects were raised by him, but Ron have fixed them, and pushed code on git server, Sytse pulled them on local git for retest
after all was tested and test sign-off was received, a go ahead given for production deployment. Code was still sitting under branch 4_02
Ron now pointed his local git to master first time in 10 days ( git checkout master ) and merged branch to master ( git merge branch_4_02 ) , he also given a specific tag to last commit called build_4_02 so every one can identify by name
specific commit will look like 3003b9fe441dd6a2e3c1410880c3a86b496fcb27 but can be tagged with user friendly name i.e 4_01_code_completed, test_completed, 4_01_prod
Ron executed one more push ( git push .. ) so same changes will reflect on server too..
Gerard was responsible for Prod deployment, he executed a git pull on his local system, with latest code in Master branch, he perform prod deployment for specific commit ( Ron had done ) and named it tag build_4_02.
everyone in the team, executed a git pull request on their local git repository, to be in sync with prod.
Sytse sometime need to work on Production issue, and hence want to go back to build 401, he will just checkout that specific commit and will have build 4_01 code in his system
Q - After that I need to merge with the master on the git-server?
Merge will happen on local git after sign off will be received by Sytse, but will be pushed again to server as Ron did above after merge
Q - And how do I push the changes to the website from the git-server?
It will same as per your current process, only difference is person pushing the changes to website, will have git on his local PC, and before
pushing the changes to website, he will ensure he get latest code, and have correct commit ( tag build_4_02) , same as Gerard did above
Q & A
Q - So to be sure the git main server will be just for receiving and sending builds. I could simple use a separate pc as git main server, with only git and centos installed on it?
Yes a separate server can be used though not mandatory, however instead of calling its job to receive and send builds, it will be more appropriate to say, it will be centralized code repository, where team can get any version of code.
Q - When Ron did the git merge branch_4_02 the changes are merged with the local git and the git main server? And thanx to these merge the git main server will be always up-to-date?
No, merge happen only at local system, from one branch to another branch, or from child branch to master branch or vice versa. To send it main server, git push command should be executed on local server, its gives more control to developer that commits will be sent to server only when developer will ask for that. Further with push command, developer will have to mention remote server name as well.
Q - Is the best way of committing that the Developer gives it the tag 4_02_prod and the one who deploys the changes pull the code and gives it a tag 4_02_build?
Ideally code finished and tagged by developer (Ron) as 4_02_prod, should go as it is to Prod without changes, in this case no tag will require from deployer.
But in some case deployer ( Gerard ) making changes on top of Ron changes, he must tag it with good title i.e. 4_02_deployed. Advantage is during production issue, a simple diff ( git diff ) , will give a clear picture what was changed between 2 commits/tag to narrow down the issue, and further both commit/tag can be pulled by tester for further investigation.
Q- And when everything is done are there branches you should delete or do you keep everything?
Immediate after Ron merge branch 4_02 to master, and give it a tag 4_02_prod, and push it to Prod, branch_4_02 must be dropped. Because same content will be
available in master branch with tag 4_02_prod. Keeping the branch alive after this time, will have no advantage, but big risk if someone used it wrongly, it will be kind of mess.
To delete it, Ron will be able to see 2 branches 1) one at local system using command git branch 2) another at main server using command
git branch -r. Both must be deleted and followed by git push, so main server is also in sync.
Q -You mentioned that I need to delete the branch on two places. The one locally and the one on the main server. But when I delete the branch locally and on the main server can I still revert to old builds?
Yes it true, if you delete the branch from local git and from main server, you still have all version of code, because before branch was deleted, that branch itself was merged to master and was tagged as 4_02_prod.
This is the beauty of the git, by using just a single command git checkout developer can get any version of code as long as it was commited.
In current scenario Ron only need to execute a command git checkout tags/build_4_02_prod, he will have old code in his system. He can have latest
code by just executing a command git checkout master
Q - And you say first delete the builds and then do a push (from your master I guess), that all is done before deployment to the website/live code?
Yes after branch deletion I push again, so server is in sync. To be at safer side I do 2 push 1) after merge is completed to master 2) after branch deletion is completed. If somehow branch is deleted at one place but not other , it will keep giving error during subsequent push..
Normally master branch is default branch and always need to update it with all codes that are stable. Working Flow looks like -
One/more developer create and switch to a new branch from master. Then do changes and test the codes. After completing a task (& also the codes are stable), create a pull request and merge the codes with master branch.
Now, master has some new changes and all other developers take (pull) the latest codes of master into their different branches they are working on.
When need to release a version of the product, give a tag (e.g. v1.0.0) on a commit in master branch. Then create a new branch (e.g. release-v1.0.0) from that tag and release the product.