using the AmadeusIT sonar-stash plugin...
After branching from main for feature/sprint we updated code locally and added, committed and pushed to BitBucket, creating a pull request. We'd like to run a scan and see the issues presently only for the code we just issued a PR for... I run sonar-scanner with this invocation:
sonar-scanner -Dsonar.analysis.mode=preview -Dsonar.stash.pullrequest.id=8 -
Dsonar.stash.repository=StaticAnalysisPOC -Dsonar.stash.login=myLogin -
Dsonar.stash.password=myPassword -Dsonar.login=sonarLogin -
Dsonar.password=sonarPword -
Dsonar.projectKey=com.company.static:StaticAnalysisPOC -
Dsonar.projectName=stat -Dsonar.projectVersion=1.0.3
output was:
INFO: Executing post-job org.sonar.plugins.stash.StashIssueReportingPostJob
INFO: org.sonar.plugins.stash.StashIssueReportingPostJob#43294e9b
not enabled, skipping
Tech Stack/Versions;
SonarQube 6.x - latest
BitBucket (on prem) 4.x - latest
Thanks!
According to the code of the plugin, you have to add parameter -Dsonar.stash.notification=true
My resolution to success was as follows:
Create feature branch off master
Run a clean, vanilla scan with following invocation on master (for baseline scan) as such: "$ sonar-scanner" - this should be called when you are attached to the master on your local machine i.e. "$ git branch" returns "master"
Issue a pull request for master to update your local master in local repo i.e. "$ git pull origin master"
Switch to feature branch on your local machine as such: "$ git checkout "featureBranchName"
5.In Eclipse, if you have the project already open, you can verify you are now attached to feature branch referenced above.
6.Now you may perform your code changes, fixes, etc. as per your desired work on the feature branch.
When work is complete, add, commit and push your changes as such:
"$ git add ."
"$ git commit -m "my commit comment"
"$ git push origin myBranchName"
Go to Bitbucket and create a pull request from your newly pushed changes in your feature branch
Take feature branch "pull request id" and append it to this invocation of sonar scanner:
$ sonar-scanner -Dsonar.analysis.mode=preview -Dsonar.stash.pullrequest.id=
<yourPullRequestIDFromAbove> -Dsonar.stash.repository=<YourStashRepo> -
Dsonar.stash.login=<StashLoginUser> -Dsonar.stash.password=<stashPassword> -
Dsonar.login=<SonarLogin> - Dsonar.password=<sonarPassword> -Dsonar.stash.notification=true -
Dsonar.projectKey=<ProjectKey> -Dsonar.projectName=<projectNameInSonar> -
Dsonar.stash.project=<StashProjectName> -Dsonar.projectVersion=
<projectVersion>
10.Review the issues as found in Bitbucket for your pull request ID
Related
Using the Git API in a CI system (eg. GitHub Actions or Travis-CI), I want to gather information on all the branches of the repo.
Sadly, it appears that GitHub branches, unlike local branches, are isolated one from each other.
Let's say I have a repository with three branches (master and other two created from master):
If I run the following script:
#!/usr/bin/env bash
printf "\n$ git for-each-ref --format='%(refname)' \n"
printf "$(git for-each-ref)\n"
printf "__________________________________________\n"
printf "\n$ git branch -a\n"
printf "$(git branch -a)\n"
I can only see master, not the other two branches:
Is there any way to read all the GitHub branches with the Git API, or I'm forced to use the GitHub API?
I hoped to be able to read at least the branches generated from the branch I'm on (master, in this case). I'm starting to guess that GitHub keeps that information for itself, without disclosing it in any canonical Git way...
You have to perform a full repository checkout in your workflow job in order to get the full repository data (including all branches). When you do git clone... (without any switches like --depth or --single-branch) on your local machine you are getting all the repository data (tags, branches, commit history, etc.). This is why git branch -r command output contains all the remote branches.
In GitHub Actions (or mentioned Travis CI) the clone operation behaves slightly different. By default, to conserve bandwidth and disk storage space the CI system retrieves only the minimal required part of the repository in order to execute the workflow. This usually is only the current branch associated with the workflow and limited commit history. This is why your commands do not return all the expected branches. However, you can override the default checkout behaviour. If you look at the actions/checkout docs:
Only a single commit is fetched by default, for the ref/SHA that triggered the workflow. Set fetch-depth: 0 to fetch all history for all branches and tags.
Taking the above into consideration, to fetch all the branches, you have to adjust the actions/checkout#v2 step in your workflow according to the docs:
Fetch all history for all tags and branches
- uses: actions/checkout#v2
with:
fetch-depth: 0
Now, when you call in the next step sth. like:
- name: List all the remote branches
run: git branch -r
you should get the expected output.
Speaking about Travis CI. The documentation is also clear about this (see §Git Clone Depth). You have to adjust your travis.yml to disable --depth flag:
git:
depth: false
I have some projects I would like to upload on my bitbucket private profile. I performed the following steps but I get an error.
Convert my project to git with: git init
Next:
git add
git commit -m "some message"
I created a bitbucket repository and version control system is GIT.
I then type this (of course with real account name and reponame and repo owner):
git remote add origin https://<repo_owner>#bitbucket.org/<accountname>/<reponame>.git
Finally,
git push -u origin master
I did all of this and my terminal gives me this error:
To https://bitbucket.org/milosdev_me/lfs.git
! [rejected] master -> master (fetch first)
error: failed to push some refs to 'https://milosdev_me#bitbucket.org/milosdev_me/lfs.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
Try:
git push origin master --force
This works for me.
Your master branch has some code that you don't locally have, and to prevent you from conflicts, it doesn't let you to push further changes, before you have them locally. To resolve this, pull all the changes to your local repository (your project):
git pull origin master --allow-unrelated-histories
After that, you will have all the code that is available on your master branch.
NOTE: Be careful, pulling the code from remote branch might mess up all the changes locally. Make sure to save those changes somewhere, or create another branch locally where you will pull your origin master branch, so it doesn't mess up your changes.
Your issue is that you have different things between your local repository and your git repository (probably a readme file that you created automatically), so you have two options:
use git pull origin master, with this command, you will pull from your git repository, but it will cause a merge conflict, which you have to resolve using an IDE (recommended to beginners) or through cmd.
use git push origin master --force, this way, you will force your push from your local repository to your git repository, ignoring any merge conflict by overwriting data. I'm not sure, but I think it will overwrite all git repository data with you local repository data (which is what you want).
Adding --force option is a bad idea
Either rebase it or pull the code, merge it and push it.
git pull --rebase origin master
git push -u origin master
Your remote repository and local repository have differences, something new in remote repository. So for pushing you need pull changes, from remote, previously. Try do:
git pull
git push -u origin master
The issue is because your remote repository and local repository have differences.I had same issue and i tried all the above mentioned solution but nothing worked for me.
For me the scenario was :- I already had my branch in remote repository.If you have the same scenario then follow below steps:-
run command 'git pull'
delete branch from local repository
checkout that particular branch using "checkout as a new local branch" from
the Remote repository.
run command 'git branch -u <your_branch_name>'
run command 'git push or git push --force'
or you can try directly from step 4 first , if it does not work then follow entire steps.Hopefully it will help someone.
If you have your bitbucket account linked to jira.
(this answer will work for you, only if you have your jira account linked to bitbucket)
I was having the same problem trying to push my current branch with the origin.
for example:
my branch name was:
feature/PREFIX-000-new-name-branch.
previous commit:
git commit -m "Write your commit here"
so far it was not working.
you have to mentioned the ticket name in the commit.
if you have made the commit, make an --amend to rename your commit and retry it.
git commit --amend -m "PREFIX-000 Write your commit here"
try the push again.
If you are using BitBucket, this issue occured when I tried to push to a branch but that branch has writing disabled via the repository settings.
if you have already created a project locally and you want to upload it to git,
you will then need to do:
git status to see the changes you need to upload
git add . to add those changes to your repo
git commit -m "" to add a commit message
git push origin master
that way I solved the very same problem I
was having.
it might be a configuration issue
I fixed this issue after updating the global user.email value with my email
git config --global user.email my#email.com
Note: You need to delete the previous commit because it had the wrong email in the commit
I'm trying to do the following with Bitbucket Pipelines when i push to my test brand:
- git fetch
- git checkout master
- git pull origin test
- git push origin master
But i get the following message on git fetch: Permission denied (publickey). I was following this tutorial https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html so i already added an ssh key, but cannot understand what are the next steps that i need for the execution to have permissions to connect to the repository.
If “from same repository” really means that you want to access the repository in which the pipeline runs, the answer is: you don’t need that. When the pipeline runs, it starts the Docker image you defined in your YAML configuration and automatically checks out the commit you pushed. This means that at the moment when when your command (git fetch) is executed, the sources are already waiting for you in path /project.
BitBucket Pipelines automatically checks out the repository upon running. However, if you want to make changes to the repository (e.g. git tag or git push) you will need to add the SSH keys according to the post you have already found (https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html). That works for our environment.
Can you post the full bitbucket-pipelines.yml file?
Our application uses VSTS for our CI flow and we have a requirement that the patch version should increase by one each time code is merged back into master.
We have created a shell script that will bump the version of the application and tag the repo, but now we are looking for the best place in the flow to inject it. We've thought of the following:
Master commit hook build - The problem with putting this here is that we have protected the master branch with certain policies (must resolve all comments, project must build, etc). Because of this, the build agent runs the script does not have access to push changes.
Pull Request build - This option would integrate the script into the build job that is used to verify the build. This actually works, but when the updates are pushed to the branch, this triggers an infinite build loop because the PR automatically rebuilds the branch. In general, this option seems more brittle.
I would love to have option 1 work out, but despite all attempts to grant the agent proper roles, it still does not have access to the repo. We're getting the following error:
TF402455: Pushes to this branch are not permitted; you must use a pull request to update this branch.
Is there a standard way to do this that I'm missing?
To update the file version and add tag for the new commit through CI build, you can add a PowerShell task. detail steps as below:
Set permission and configure in build definition
First go to Version control Tab, allow below items for Build Service:
Branch creation: Allow
Contribute: Allow
Read: Inherited allow
Tag creation: Inherited allow
Then in your build definition, add the variable system.prefergit as true in Variable Tab, and enable Allow script to access OAuth token in Options Tab.
More details, you can refer Run Git commands in a script.
Add powershell script
git -c http.extraheader="AUTHORIZATION: bearer %SYSTEM_ACCESSTOKEN%" fetch
git checkout master
# Get the line of the file that contain the version
# split the line, and only get the version number major.minor.patch.
# check the version numbers (such as if patch less than 9) and increase the version
#replace the old version with new version in the file
git add .
git commit -m 'change the version as $newVersion'
git tag $newVersion
git push origin master --tags
Note: Even the new version and tag can push to remote repo successful, the PowerShell task may failed. So you should set the task after PowerShell task with custom option as Even if a previous task has failed.
You can try to update your branch policy per documentation on https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops and https://learn.microsoft.com/en-us/azure/devops/organizations/security/permissions?view=azure-devops#git-repository-object-level
Try updating your branch policy with to "Allow" "Bypass policies when pushing"
With my colleagues, we work on a C++ library that becomes more and more important each day. We already built continuous integration utilities through the gitlab-ci.yml file that let us:
Build & Test in Debug mode
Build & Test in Release mode
Perform safety checks like memory leaks using Valgrind and checking if there is any clear symbol in our library we don't want inside it
Generate documentation
All kind of stuff that made us choose GitLab !
We would like to profile our whole library and push the benchmarks in a separate project. We have already done something like for out documentation using the SSH key method but we would like to avoid this this time.
We tried a script like this:
test_ci_push:
tags:
- linux
- shell
- light
stage: profiling
allow_failure: false
only:
- new-benchmark-stage
script:
- git clone http://gitlab-ci-token:${CI_JOB_TOKEN}#gitlab.mycompany.home/developers/benchmarks.git &> /dev/null
- cd benchmarks
- touch test.dat
- echo "This is a test" > test.dat
- git config --global user.name "${GITLAB_USER_NAME}"
- git config --global user.email "${GITLAB_USER_EMAIL}"
- git add --all
- git commit -m "GitLab Runner Push"
- git push http://gitlab-ci-token:${CI_JOB_TOKEN}#gitlab.mycompany.home/developers/benchmarks.git HEAD:master
- cd ..
We also tried a basic git push origin master to push our updated files but each time we got the same answer:
remote: You are not allowed to upload code for this project.
fatal: unable to access 'http://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx#gitlab.mycompany.home/developers/benchmarks.git/': The requested URL returned error: 403
Both projects are under the same site and I have the rights to push in both. Where am I doing anything wrong here ?
The gitlab ci token is more like the deploy key in github.com, so it only has read access to the repository. To actually push you will need to generate a personal access token and use that instead.
First you need to generate the token as shown here in the gitlab documentation. Make sure you check both the read user and api scopes. Also this only works in GitLab 8.15 and above. If you are using an older version and do not wish to upgrade there's an alternative method I could show you but it is more complex and less secure.
In the end your gitlab-ci.yml should look something like this:
test_ci_push:
tags:
- linux
- shell
- light
stage: profiling
allow_failure: false
only:
- new-benchmark-stage
script:
- git clone http://gitlab-ci-token:${CI_JOB_TOKEN}#gitlab.mycompany.home/developers/benchmarks.git &> /dev/null
- cd benchmarks
- echo "This is a test" > test.dat
- git config --global user.name "${GITLAB_USER_NAME}"
- git config --global user.email "${GITLAB_USER_EMAIL}"
- git add --all
- git commit -m "GitLab Runner Push"
- git push http://${YOUR_USERNAME}:${PERSONAL_ACCESS_TOKEN}#gitlab.mycompany.home/developers/benchmarks.git HEAD:master
- cd ..
While the previous answers are more or less fine, there are some important gotya's.
before_script:
- git config --global user.name "${GITLAB_USER_NAME}"
- git config --global user.email "${GITLAB_USER_EMAIL}"
script:
- <do things>
- git push "https://${GITLAB_USER_LOGIN}:${CI_GIT_TOKEN}#${CI_REPOSITORY_URL#*#}" "HEAD:${CI_COMMIT_TAG}"
For one, we only need to set the username/email to please git.
Secondly having it in the before script, is not super crucial, but allows for easier reuse when doing 'extend'.
Finally, pushing https is 'fine' but since we're not using a stored ssh key, we should avoid anything that can reveal the token. For one, while gitlab won't print the token in this command, git will happily inform us that the new upstream is set to https://username:thetokeninplaintexthere#url
So there's your token in plain text, so don't use -u to set an upstream.
Also, it's not needed, we are only doing a single push.
Further more, when determining the URL, I found that using the exist CI_REPOSITORY_URL to be the most reliable solution (when moving repo's for example or whatnot). So we just replace the username/token in the URL string.
You could also provide user and password (user with write access) as secret variables and use them.
Example:
before_script:
- git remote set-url origin https://$GIT_CI_USER:$GIT_CI_PASS#$CI_SERVER_HOST/$CI_PROJECT_PATH.git
- git config --global user.email 'myuser#mydomain.com'
- git config --global user.name 'MyUser'
You have to define GIT_CI_USER and GIT_CI_PASS as secret variables (you could always create dedicated user for this purpose).
With this configuration you could normally work with git. I'm using this approach to push the tags after the release (with Axion Release Gradle Pluing - http://axion-release-plugin.readthedocs.io/en/latest/index.html)
Example release job:
release:
stage: release
script:
- git branch
- gradle release -Prelease.disableChecks -Prelease.pushTagsOnly
- git push --tags
only:
- master
I'm using the following GitLab job:
repo_pull_sync:
image: danger89/repo_mirror_pull:latest
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule"'
- if: $REMOTE_URL
- if: $REMOTE_BRANCH
- if: $ACCESS_TOKEN
before_script:
- git config --global user.name "${GITLAB_USER_NAME}"
- git config --global user.email "${GITLAB_USER_EMAIL}"
script:
- git checkout $CI_DEFAULT_BRANCH
- git pull
- git remote remove upstream || true
- git remote add upstream $REMOTE_URL
- git fetch upstream
- git merge upstream/$REMOTE_BRANCH
- git push "https://${GITLAB_USER_LOGIN}:${ACCESS_TOKEN}#${CI_REPOSITORY_URL#*#}" "HEAD:${CI_DEFAULT_BRANCH}"
I'm using my own danger89/repo_mirror_pull docker image based on alpine, check this GitHub repository for more info.
This GitLab job pull upstream changes from the predefined remote repository + branch (see variables below), and merge them locally in CI/CD and pushes them in GitLab again.
Basically I created a repository pull mirror (which is officially not available for free on GitLab CE, only a push mirror is supported in GitLab).
Create in GitLab a Project Access Token first in GitLab. Via: Settings->Access Tokens. Check 'api' as the scope.
Create a new schedule, via: CI/CD->Schedules->New schedule. With the following 3 variables:
REMOTE_URL (example: https://github.com/project/repo.git)
REMOTE_BRANCH (example: master)
ACCESS_TOKEN: (see Access Token in the first step! Example: gplat-234hcand9q289rba89dghqa892agbd89arg2854, )
Save pipeline schedule
Again, see also: https://github.com/danger89/repo_pull_sync_docker_image
Regarding the question, see the git push command above, which allows you to push changes back into GitLab using GitLab (project) access token.