I have a cloudbuild.yaml file which contains deployment steps (which is working nicely), however I do not want to execute these steps on a pull request, just a merge to a particular branch. How do I make cloudbuild distinguish between pull requests and merges?
I think I have found a somewhat hacky solution. In the cloudbuild.yaml I have added the following:
substitutions:
_DEPLOY: '0'
And in the cloudbuild console I have added the _DEPLOY substitution as '1`. Then in my build steps I can check the _DEPLOY flag like so:
- name: 'gcr.io/cloud-builders/gsutil'
entrypoint: 'bash'
args:
- '-c'
- |
if [ "${_DEPLOY}" -eq "1" ]; then echo 'hooray'; fi
The _DEPLOY flag is set when there is a push to my branch, but not during the github integration (e.g. PRs).
Related
I'm trying to commit a change that I make to a build file in my build process. This job runs on MacOS as a Bash task. All sorts of other scripts and tasks are working correctly, but trying to execute these git commands is giving back a 127 error.
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo $PATH
git add $(System.DefaultWorkingDirectory)/MyFilePath/my.file
git commit -m "update file [skip ci]"
git push origin
workingDirectory: '$(Build.SourcesDirectory)'
displayName: 'Git commit'
continueOnError: true
condition: and(succeeded(), eq(variables.ChangeFile, 'true'))
What I get back is the following error(s):
========================== Starting Command Output ===========================
##[debug]which '/bin/bash'
##[debug]found: '/bin/bash'
##[debug]/bin/bash arg: /Users/runner/work/_temp/7d35e773-5913-44d8-b4fd-0d22787e7680.sh
##[debug]exec tool: /bin/bash
##[debug]arguments:
##[debug] /Users/runner/work/_temp/7d35e773-5913-44d8-b4fd-0d22787e7680.sh
/bin/bash /Users/runner/work/_temp/7d35e773-5913-44d8-b4fd-0d22787e7680.sh
/Users/runner/hostedtoolcache/NuGet/6.2.1/x64:/usr/local/lib/ruby/gems/2.7.0/bin:/usr/local/opt/ruby#2.7/bin:/usr/local/opt/pipx_bin:/Users/runner/.cargo/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Users/runner/Library/Android/sdk/ndk-bundle:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools:/Users/runner/.ghcup/bin:/Users/runner/hostedtoolcache/stack/2.7.5/x64
git: 'add /Users/runner/work/1/s/MyFilePath/my.file' is not a git command. See 'git --help'.
/Users/runner/work/_temp/7d35e773-5913-44d8-b4fd-0d22787e7680.sh: line 3: git commit -m update file [skip ci]: command not found
/Users/runner/work/_temp/7d35e773-5913-44d8-b4fd-0d22787e7680.sh: line 4: git push origin: command not found
##[debug]Exit code 127 received from tool '/bin/bash'
##[debug]STDIO streams have closed for tool '/bin/bash'
##[error]Bash exited with code '127'.
Pretty vague, but here are some things I've tried:
- checkout: self
persistCredentials: true
clean: true
displayName: Checkout from git
- script: git checkout $(Build.SourceBranchName)
workingDirectory: $(Build.SourcesDirectory)
displayName: Checkout branch from git to receive commits on
failOnStderr: false
condition: and(succeeded(), eq(variables.ChangeFile, 'true'))
Making sure to checkout the code first. I've used some mostly
pre-defined Git steps for this, but that git checkout script works dandy.
Running a very similar add on the same file on my machine locally. The casing for that path needed to be fixed, but then it worked.
Using echo path to determine if Git is even in my path, as you can see before the commands. Looks like it's not, but then, wouldn't it be saying git itself is not a recognised command, rather than add, commit and push?
I've tried Googling error code 127 and using Git on a MacOS agent in Azure, but to no avail. People don't seem to be having quite the same problem as me.
Any help would be most pleasant!
To call git add, aren't a couple of things required?:
Doesn't the agent's current directory need to be a git repo folder, i.e. a folder containing a ".git" folder? You're not showing that your script ensures or verifies this.
AFAICT, git add's pathspec doesn't allow a leading slash like your "/Users/runner/work/1/s/MyFilePath/my.file" example. I have difficulty finding a good doc for the pathspec, but I can't find any examples using a leading slash. I'd suggest experimenting with git add on your local machine.
From the issue output, It seems that git treats the entire latter as a command parameter. I think the issue comes from the $(System.DefaultWorkingDirectory) in your OS type(You are based on Mac OS).
This should work:
trigger:
- none
pool:
vmImage: 'windows-latest'
steps:
- task: PythonScript#0
inputs:
scriptSource: 'inline'
script: |
from encodings import utf_8
import os
rootpath = "./MyFilePath"
dictpath = "./MyFilePath2"
for filename in os.listdir(rootpath):
with open(rootpath+'/'+filename, encoding='gb2312') as f:
with open(dictpath+'/'+filename, "w+", encoding='utf-8') as f1:
for line in f:
f1.write(line)
- task: Bash#3
inputs:
targetType: 'inline'
script: |
git config --global user.email "xxx#xxx.com"
git config --global user.name "Bowman Zhu(In Microsoft Agent)"
git add '$(System.DefaultWorkingDirectory)/MyFilePath2/my.file'
git commit -m 1
Success:
My repo structure:
I am trying to run a script with pre-commit hook. Here is my script:
build_script.sh
#! /bin/bash
echo "Stage 1: Preparing CMake configuration"
.pre-commit-config.yaml
fail_fast: false
- repo: local
hooks:
- id: Generate build
name: Generate build
entry: sh build_script.sh
language: system
always_run: true
pass_filenames: false
I can see when I run command git commit -m "message", the hook calls the script and Generate build will Pass. However, I do not see the echo on the terminal. I would like to see the message, "Stage 1: Preparing CMake configuration". What is wrong with this setup?
pre-commit takes a bit from unix philosophy here -- it is silent unless there is a problem (by default).
if your script fails then the output will be displayed
you can also use --verbose to show the full output (for debugging) and optionally set verbose: true on your hook itself. note: these are both debugging options and are not intended for use outside that -- we've found that when hooks are noisy contributors tend to ignore all of the output
disclaimer: I wrote pre-commit
I want to achieve the following build process:
decide the value of environment var depending on the build branch
persist this value through diff build steps
use this var to pass it as build-arg to docker build
Here is some of the cloudbuild config I've got:
- id: 'Get env from branch'
name: bash
args:
- '-c'
- |-
environment="dev"
if [[ "${BRANCH_NAME}" == "staging" ]]; then
environment="stg"
elif [[ "${BRANCH_NAME}" == "master" ]]; then
environment="prd"
fi
echo $environment > /workspace/environment.txt
- id: 'Build Docker image'
name: bash
dir: $_SERVICE_DIR
args:
- '-c'
- |-
environment=$(cat /workspace/environment.txt)
echo "===== ENV: $environment"
docker build --build-arg ENVIRONMENT=$environment -t gcr.io/${_GCR_PROJECT_ID}/${_SERVICE_NAME}/${COMMIT_SHA} .
The problem lies in the 2nd step. If I use bash step image, then I've got no docker executable in order to build my custom image.
And if I use gcr.io/cloud-builders/docker step image, then I can't execute bash scripts. In the args field I can only pass arguments for the docker executable. And this way I cannot extract the value of environment that I've persisted through the steps of the build.
The way I managed to accomplish both is to use my own, custom, pre-built image, which contains both bash and docker executables. I have that image in the container registry and I use it as the build step image. But this requires some custom work from my side. I was wondering if there is a better, more standardized way with built-in tools from cloudbuild.
Sources:
how to run inline bash scripts
how to persist values through build steps
You can change the default entrypoint by adding entrypoint: parameter
- name: 'gcr.io/cloud-builders/docker'
entrypoint: 'bash'
args:
- -c
- |
echo $PROJECT_ID
environment=$(cat /workspace/environment.txt)
echo "===== ENV: $environment"
docker build --build-arg ENVIRONMENT=$environment -t gcr.io/${_GCR_PROJECT_ID}/${_SERVICE_NAME}/${COMMIT_SHA} .
We have a requirement where collaborators with write access to a GitHub repository should not be allowed to create branches with certain names. They can create locally but can't push to remote and also can't create directly using the GitHub UI. I know we can do something like this using git hook and enable that for that repo on the server side (on github enterprise server), but struggling to figure out how. Is that even possible?
If so I would like a shell script or something similar that I can use as hook to reject creation/push of any branches that start with the string release. I tried the below by adding a pre-receive hook on the github enterprise server but it doesn't do any good. Created a repo with a branch_hook.sh file and configured that hook and enabled it on a repo to test.
#!/bin/bash
branch=`git rev-parse --abbrev-ref HEAD`
if [[ "$branch" == release* ]]; then
echo "Your branch starts with release and is not allowed to be
pushed. Please create one that doesn't start with the release"
exit 1
fi
It looks like GitHub server doesn't validate the branch name at all. If I execute it locally under a git repo, it works fine but git commit or push doesn't consider it at all.
Based on VonC's suggestions I tried this but doesn't seem to work.
Since I couldn't get the script at https://gist.github.com/caniszczyk/1327469
to work so tried something of my own.
#!/bin/bash
# Reject branch pushes that contain commits under those branches that have names starting with release
first_commit='0000000000000000000000000000000000000000'
while read -r oldrev newrev refname; do
[ "$newrev" = "$first_commit" ] && continue
[ "$oldrev" = "$first_commit" ] && range="$newrev" || range="$oldrev..$newrev"
for commit in $(git rev-list "$range" --not --all); do
if [[ "${refname#refs/heads/}" == release* ]]; then
echo "ERROR: Your push was rejected because the commit"
echo "ERROR: $commit in ${refname#refs/heads/}"
echo "ERROR: is not allowed as ${refname#refs/heads/} is not supported branch name"
echo "ERROR: Please fix your branch name or contact your repository admin."
exit 1
fi
done
done
This works great when you create branch locally (just created newly or created newly and some commits added) and try to push to remote however there is no way to prevent creating branches using GitHub UI as the hook doesn't work in that scenario as its not a push event. So I am wondering how to get the script at https://gist.github.com/caniszczyk/1327469 to work if that is a true solution that works in both the cases.
Since it is an on-premise GitHub Enterprise server, you can add n
A pre-receive hook script executes in a pre-receive hook environment on the GitHub Enterprise Server appliance.
When you create a pre-receive hook script, consider the available input, output, exit-status and environment variables.
For example, this hook will enforce a naming convention policy on any branch pushed, which means it will disallow said push if the branch name does not follow a certain convention.
You can try and fix this using a GitHub action such as this one:
name: Fail when incorrect branch names are used
on:
push:
branches:
- 'release*'
- 'release*/**'
jobs:
no_release_in_branch_names:
runs-on: ubuntu-latest
steps:
- name: If this is triggered, it should fail
run: |
__msg="::error::This branch ${GITHUB_REF_NAME} includes «release»; please don't use that in branch names
Please rename the local branch with
git checkout ${GITHUB_REF_NAME}
git branch -m name_without_release
echo "$__msg"
echo ::set-output name=status::failure
exit 1
That will prevent pushing to any branch created either locally or from the UI, by failing with the first commit that's done to that branch. Essentially, uses GitHub actions for a workflow that's triggered (and fails) just for the branch names that you don't want.
I have gitlab deployment activem and I want to get the deploy script to have some custom information about the deployment process (like $CI_PIPELINE_ID).
However, the script doesn't get the variables, instead it gets the "raw text".
the call performed by the script is: $ python deploy/deploy.py $CI_COMMIT_TAG $CI_ENVIRONMENT_URL $CI_PIPELINE_ID
How can i get it to use the variables?
My .gitlab-ci.yml:
image: python:2.7
before_script:
- whoami
- sudo apt-get --quiet update --yes
- sudo chmod +x deploy/deploy.py
deploy_production:
stage: deploy
environment: Production
only:
- tags
- trigger
except:
# - develop
- /^feature\/.*$/
- /^hotfix\/.*$/
- /^release\/.*$/
script:
- python deploy/deploy.py $CI_COMMIT_TAG $CI_ENVIRONMENT_URL $CI_PIPELINE_ID
It looks like potentially that you could be using a different environmental variable that you should be using.
bash/sh $variable
windows batch %variable%
PowerShell $env:variable
See using CI variables in your job script.
I don't get what you mean with "raw text", but you can declare the variables in your project settings. Also, have you configured you're runner?
Go to Settings->CI/CD->Secret Variables and just put them right there.
You can also find valuable information in the documentation.