I'm using a webhook trigger and part of the configuration requires setting a default branch. This webhook invokes on pull requests so when the trigger runs it check-out that branch.
Everything works great except that in the Cloud Build history it obviously doesn't show the branch it ran on but the default branch set in the configuration, ie. 'master'
Is it possible to update the ref during build to be the actual branch it executed on so there is a bit more clarity when reviewing build history?
Referring to this documentation here, have I found the correct variable and would reassigning it work?
steps:
- id: 'Setup Credentials'
name: 'gcr.io/cloud-builders/git'
entrypoint: '/bin/bash'
args:
- '-c'
- |
# checkout 'feature/my-branch' branch
# do work on branch
$_REF_EVENT_NAME='feature/my-branch' # overwrite the configured default branch
If its possible then I'd want to update the commit reference as well as that's from the masters last commit not the branch.
Actually, you can't. This ref column is important when you use other type of trigger, but for webhook, the value is generic and not updatable.
Related
Using the Git API in a CI system (eg. GitHub Actions or Travis-CI), I want to gather information on all the branches of the repo.
Sadly, it appears that GitHub branches, unlike local branches, are isolated one from each other.
Let's say I have a repository with three branches (master and other two created from master):
If I run the following script:
#!/usr/bin/env bash
printf "\n$ git for-each-ref --format='%(refname)' \n"
printf "$(git for-each-ref)\n"
printf "__________________________________________\n"
printf "\n$ git branch -a\n"
printf "$(git branch -a)\n"
I can only see master, not the other two branches:
Is there any way to read all the GitHub branches with the Git API, or I'm forced to use the GitHub API?
I hoped to be able to read at least the branches generated from the branch I'm on (master, in this case). I'm starting to guess that GitHub keeps that information for itself, without disclosing it in any canonical Git way...
You have to perform a full repository checkout in your workflow job in order to get the full repository data (including all branches). When you do git clone... (without any switches like --depth or --single-branch) on your local machine you are getting all the repository data (tags, branches, commit history, etc.). This is why git branch -r command output contains all the remote branches.
In GitHub Actions (or mentioned Travis CI) the clone operation behaves slightly different. By default, to conserve bandwidth and disk storage space the CI system retrieves only the minimal required part of the repository in order to execute the workflow. This usually is only the current branch associated with the workflow and limited commit history. This is why your commands do not return all the expected branches. However, you can override the default checkout behaviour. If you look at the actions/checkout docs:
Only a single commit is fetched by default, for the ref/SHA that triggered the workflow. Set fetch-depth: 0 to fetch all history for all branches and tags.
Taking the above into consideration, to fetch all the branches, you have to adjust the actions/checkout#v2 step in your workflow according to the docs:
Fetch all history for all tags and branches
- uses: actions/checkout#v2
with:
fetch-depth: 0
Now, when you call in the next step sth. like:
- name: List all the remote branches
run: git branch -r
you should get the expected output.
Speaking about Travis CI. The documentation is also clear about this (see §Git Clone Depth). You have to adjust your travis.yml to disable --depth flag:
git:
depth: false
I'm having some trouble finding info that describes how to update the code base automatically in Gitlab
Scenario
Let's imagine a developer working on a project commits some code, but forgets to format it before committing. The .gitlab-ci.yml can have a job that will check the formatting, and display an error/warning if found. Is it possible to update the git commit with the code formatted automatically?
So the flow would be:
Developer commits unformatted code to gitlab
Git checks the code for formatting
If an issue is found, git will run tool X to automatically format the code, and commit it with a git message like "Automated Git commit -- formatting"
.gitlab-ci.yml continues as normal
Is this possible in Gitlab?
Using GitLab 13.09, the answer appears to be "no".
I used this .gitlab-ci.yml:
image: python-latest
format-job:
script:
- pip install black
- black src
As expected, black echos that my files were reformatted, yet when I check the repo they haven't been fixed.
My experience is consistent with this post.
A potential alternative is to use a pre-commit server hook.
I have written this yml file for GitLab CI/CD. There is a shared runner configured and running.
I am doing this first time and not sure where I am going wrong. The angular js project I am having
on the repo has a gulp build file and works perfectly on local machine. This code just has to trigger
that on the vm where my runner is present. On commit the pipeline does not show any job. Let me know what needs to be corrected!
image: docker:latest
cache:
paths:
- node_modules/
deploy_stage:
stage: build
only:
- master
environment: stage
script:
- rmdir -rf "build"
- mkdir "build"
- cd "build"
- git init
- git clone "my url"
- cd "path of cloned repository"
- gulp build
What branch are you commiting to? You pipeline is configured to run only for commit on master branch.
...
only:
- master
...
If you want to have triggered jobs for other branches as well then remove this restriction from .gitlab-ci.yml file.
Do not forget to Enable shared Runners (they may not be enabled by default), setting can be found on GitLab project page under Settings -> CI/CD -> Runners.
Update: Did your pipeline triggers ever work for your project?
If not then I would try configuring simple pipeline just to test if triggers work fine:
test_simple_job:
script:
- echo I should execute for any pipeline trigger.
I solved the problem by renaming the .gitlab-ci.yaml to .gitlab-ci.yml
I just wanted to add that I ran into a similar issue. I was committing my code and I was not seeing the pipeline trigger at all.There was also no error statement on gitlab nor in my vscode. It had ran perfectly before.My problem was because I had made some recent edits to my yaml that were invalid.I reverted the changes to a known valid yaml code, and it worked again and passed.
I also had this issue. I thought I would document the cause, in the hopes it may help someone (although this is not strictly an answer for the original question because my deploy script is more complex).
So in my case, the reason was that I had multiple jobs with the same job ID in my .gitlab-ci.yml. The latter one basically rendered the earlier one invisible.
# This job will never run:
deploy_my_stuff:
script:
- do something for job one
# This job overwrites the above.
deploy_my_stuff:
script:
- do something for job two
Totally obvious... after I discovered the mistake.
Our application uses VSTS for our CI flow and we have a requirement that the patch version should increase by one each time code is merged back into master.
We have created a shell script that will bump the version of the application and tag the repo, but now we are looking for the best place in the flow to inject it. We've thought of the following:
Master commit hook build - The problem with putting this here is that we have protected the master branch with certain policies (must resolve all comments, project must build, etc). Because of this, the build agent runs the script does not have access to push changes.
Pull Request build - This option would integrate the script into the build job that is used to verify the build. This actually works, but when the updates are pushed to the branch, this triggers an infinite build loop because the PR automatically rebuilds the branch. In general, this option seems more brittle.
I would love to have option 1 work out, but despite all attempts to grant the agent proper roles, it still does not have access to the repo. We're getting the following error:
TF402455: Pushes to this branch are not permitted; you must use a pull request to update this branch.
Is there a standard way to do this that I'm missing?
To update the file version and add tag for the new commit through CI build, you can add a PowerShell task. detail steps as below:
Set permission and configure in build definition
First go to Version control Tab, allow below items for Build Service:
Branch creation: Allow
Contribute: Allow
Read: Inherited allow
Tag creation: Inherited allow
Then in your build definition, add the variable system.prefergit as true in Variable Tab, and enable Allow script to access OAuth token in Options Tab.
More details, you can refer Run Git commands in a script.
Add powershell script
git -c http.extraheader="AUTHORIZATION: bearer %SYSTEM_ACCESSTOKEN%" fetch
git checkout master
# Get the line of the file that contain the version
# split the line, and only get the version number major.minor.patch.
# check the version numbers (such as if patch less than 9) and increase the version
#replace the old version with new version in the file
git add .
git commit -m 'change the version as $newVersion'
git tag $newVersion
git push origin master --tags
Note: Even the new version and tag can push to remote repo successful, the PowerShell task may failed. So you should set the task after PowerShell task with custom option as Even if a previous task has failed.
You can try to update your branch policy per documentation on https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops and https://learn.microsoft.com/en-us/azure/devops/organizations/security/permissions?view=azure-devops#git-repository-object-level
Try updating your branch policy with to "Allow" "Bypass policies when pushing"
using the AmadeusIT sonar-stash plugin...
After branching from main for feature/sprint we updated code locally and added, committed and pushed to BitBucket, creating a pull request. We'd like to run a scan and see the issues presently only for the code we just issued a PR for... I run sonar-scanner with this invocation:
sonar-scanner -Dsonar.analysis.mode=preview -Dsonar.stash.pullrequest.id=8 -
Dsonar.stash.repository=StaticAnalysisPOC -Dsonar.stash.login=myLogin -
Dsonar.stash.password=myPassword -Dsonar.login=sonarLogin -
Dsonar.password=sonarPword -
Dsonar.projectKey=com.company.static:StaticAnalysisPOC -
Dsonar.projectName=stat -Dsonar.projectVersion=1.0.3
output was:
INFO: Executing post-job org.sonar.plugins.stash.StashIssueReportingPostJob
INFO: org.sonar.plugins.stash.StashIssueReportingPostJob#43294e9b
not enabled, skipping
Tech Stack/Versions;
SonarQube 6.x - latest
BitBucket (on prem) 4.x - latest
Thanks!
According to the code of the plugin, you have to add parameter -Dsonar.stash.notification=true
My resolution to success was as follows:
Create feature branch off master
Run a clean, vanilla scan with following invocation on master (for baseline scan) as such: "$ sonar-scanner" - this should be called when you are attached to the master on your local machine i.e. "$ git branch" returns "master"
Issue a pull request for master to update your local master in local repo i.e. "$ git pull origin master"
Switch to feature branch on your local machine as such: "$ git checkout "featureBranchName"
5.In Eclipse, if you have the project already open, you can verify you are now attached to feature branch referenced above.
6.Now you may perform your code changes, fixes, etc. as per your desired work on the feature branch.
When work is complete, add, commit and push your changes as such:
"$ git add ."
"$ git commit -m "my commit comment"
"$ git push origin myBranchName"
Go to Bitbucket and create a pull request from your newly pushed changes in your feature branch
Take feature branch "pull request id" and append it to this invocation of sonar scanner:
$ sonar-scanner -Dsonar.analysis.mode=preview -Dsonar.stash.pullrequest.id=
<yourPullRequestIDFromAbove> -Dsonar.stash.repository=<YourStashRepo> -
Dsonar.stash.login=<StashLoginUser> -Dsonar.stash.password=<stashPassword> -
Dsonar.login=<SonarLogin> - Dsonar.password=<sonarPassword> -Dsonar.stash.notification=true -
Dsonar.projectKey=<ProjectKey> -Dsonar.projectName=<projectNameInSonar> -
Dsonar.stash.project=<StashProjectName> -Dsonar.projectVersion=
<projectVersion>
10.Review the issues as found in Bitbucket for your pull request ID