I am trying to only trigger the pipeline when commit message has the conditional phrase. I know this has been asked a lot of times and there are helpful answers available. I have also checked gitlab ci documentation and it also provide the right ways to do it.
Still the stage is built no matter the required phrase is in commit message or not. Here is the .yml code.
before_script:
- export LC_ALL=en_US.UTF-8
- export LANG=en_US.UTF-8
- export BUILD_TIME=$(date '+%Y-%m-%d %H:%M:%S')
- echo $branch
stages:
- build
build_job:
stage: build
only:
variables:
- $branch
- $CI_COMMIT_MESSAGE =~ /\[ci build]/
script:
- bundle fastlane
- fastlane build
Anyone have any idea that what is wrong with it?
Maybe you can remove the variable $branch and use only: refs
here some example
before_script:
- export LC_ALL=en_US.UTF-8
- export LANG=en_US.UTF-8
- export BUILD_TIME=$(date '+%Y-%m-%d %H:%M:%S')
stages:
- build
build_job:
stage: build
script:
- bundle fastlane
- fastlane build
only:
variables:
- $CI_COMMIT_MESSAGE =~ /\[ci build]/
refs:
- /^develop*.*$/
you can use regex in refs , in my example meaning : when branch name contain develop and commit message contain [ci build] then run the stages
you can modify thats regex.
thats method used in my production.
consider the next solution:
before_script:
- export LC_ALL=en_US.UTF-8
- export LANG=en_US.UTF-8
- export BUILD_TIME=$(date '+%Y-%m-%d %H:%M:%S')
stages:
- build
build_job:
stage: build
rules:
- if: $CI_COMMIT_MESSAGE =~ /\[ci build]/
script:
- bundle fastlane
- fastlane build
Related
We have a gitlab pipeline which I am trying to configure to use a different Azure subscription per environment without much luck.
Basically what I need to be able to do is set the environment variables ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_SUBSCRIPTION_ID, ARM_TENANT_ID to different values depending on the environment being built.
In the cicd settings I have variables set for development_ARM_SUBSCRIPTION_ID, test_ARM_SUBSCRIPTION_ID etc with the idea being I assign the values from these variables to the ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_SUBSCRIPTION_ID, ARM_TENANT_ID variables in the pipeline.
This is what my pipeline looks like
stages:
- infrastructure-validate
- infrastructure-deploy
- infrastructure-destroy
variables:
DESTROY_INFRA: "false"
development_ARM_SUBSCRIPTION_ID: $development_ARM_SUBSCRIPTION_ID
development_ARM_TENANT_ID: $development_ARM_TENANT_ID
development_ARM_CLIENT_ID: $development_ARM_CLIENT_ID
development_ARM_CLIENT_SECRET: $development_ARM_CLIENT_SECRET
image:
name: hashicorp/terraform:light
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
before_script:
- rm -rf .terraform
- terraform --version
- terraform init
.terraform-validate:
script:
- export ARM_SUB_ID=${CI_ENVIRONMENT_NAME}_ARM_SUBSCRIPTION_ID
- export ARM_SUBSCRIPTION_ID=${!ARM_SUB_ID}
- export ARM_CLI_ID=${CI_ENVIRONMENT_NAME}_ARM_CLIENT_ID
- export ARM_CLIENT_ID=${!ARM_CLI_ID}
- export ARM_TEN=${CI_ENVIRONMENT_NAME}_ARM_TENANT_ID
- export ARM_TENANT_ID=${!ARM_TEN_ID}
- export ARM_CLI_SECRET=${CI_ENVIRONMENT_NAME}_ARM_CLIENT_SECRET
- export ARM_CLIENT_SECRET=${!ARM_CLI_SECRET")
- echo $development_ARM_SUBSCRIPTION_ID
- echo ${ARM_SUBSCRIPTION_ID}
- terraform workspace select ${CI_ENVIRONMENT_NAME}
- terraform validate
- terraform plan -out "terraform-plan-file"
only:
variables:
- $DESTROY_INFRA != "true"
development-validate-and-plan-terraform:
stage: infrastructure-validate
environment: development
extends: .terraform-validate
only:
refs:
- main
- develop
artifacts:
paths:
- terraform-plan-file
The variable substitution works fine when I test locally but in the pipeline it fails with
/bin/sh: eval: $ export ARM_SUBSCRIPTION_ID=${!ARM_SUB_ID}
line 139: syntax error: bad substitution
I think the problem is the terraform image does not have bash available, only sh but I can't for the life of me work out how I do the same substitution in sh. If anyone has any suggestions, or knows a better way of using different Azure subscriptions for different environments in the pipeline I would really appreciate it.
I would define different jobs for each environment that extend your main .terraform-validate job template, and define the environment variables on that job. This way you don't have to do the indirect substitution that seems to be giving you trouble. That would look something like this:
.terraform-validate:
stage: infrastructure-validate
script:
- echo ${ARM_SUBSCRIPTION_ID}
- terraform workspace select ${CI_ENVIRONMENT_NAME}
- terraform validate
- terraform plan -out "terraform-plan-file"
only:
variables:
- $DESTROY_INFRA != "true"
artifacts:
paths:
- terraform-plan-file
development-validate-and-plan-terraform:
extends: .terraform-validate
environment: development
only:
refs:
- main
- develop
variables:
ARM_SUBSCRIPTION_ID: $development_ARM_SUBSCRIPTION_ID
ARM_TENANT_ID: $development_ARM_TENANT_ID
ARM_CLIENT_ID: $development_ARM_CLIENT_ID
ARM_CLIENT_SECRET: $development_ARM_CLIENT_SECRET
production-validate-and-plan-terraform:
extends: .terraform-validate
environment: production
only:
refs:
- main
variables:
ARM_SUBSCRIPTION_ID: $production_ARM_SUBSCRIPTION_ID
ARM_TENANT_ID: $production_ARM_TENANT_ID
ARM_CLIENT_ID: $production_ARM_CLIENT_ID
ARM_CLIENT_SECRET: $production_ARM_CLIENT_SECRET
Then you define all the development_* and production_* vars in the GitLab CI/CD settings.
Note that I also moved the stage: infrastructure-validate and artifacts: ... directives to the template since I'd imagine they're the same for all environments.
On my gitlab repo I have to trigger pipeline only when a specific folder have changes AND when it's a Merge request (both condition). Especially on .zip file extension, i-e, add a new zip file in this folder, create a Merge request, then run the pipeline.
This is my initial pipeline yaml code:
trigger-ci-zip-file-only:
stage: prebuild
extends:
- .prebuild
- .preprod-tags
variables:
PROJECT_FOLDER: "my-specific-folder"
before_script:
- echo "Job currently run in $CI_JOB_STAGE"
- touch ${CI_PROJECT_DIR}/$PROJECT_FOLDER/prebuild.env
- cd $PROJECT_FOLDER
only:
refs:
- merge_requests
changes:
- ${PROJECT_FOLDER}/*.zip
artifacts:
reports:
dotenv: ${CI_PROJECT_DIR}/${PROJECT_FOLDER}/prebuild.env
allow_failure: false
As you can see, my pipeline have to be triggered only when there is a change in a specific folder on zip files and only on MR. But in this state, the pipeline is always running when MR is creatd or when I push on an existing MR, even if there is no changes or adding in the specific folder.
I also tried to make some changes on the pipeline yaml code like this:
only:
refs:
- merge_requests
changes:
- ${PROJECT_FOLDER}/**/*.zip
But the pipeline always running.
Also I tried this:
trigger-ci-zip-file-only:
stage: prebuild
extends:
- .prebuild
- .preprod-tags
variables:
PROJECT_FOLDER: "my-specific-folder"
before_script:
- echo "Job currently run in $CI_JOB_STAGE"
- touch ${CI_PROJECT_DIR}/$PROJECT_FOLDER/prebuild.env
- cd $PROJECT_FOLDER
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- changes:
- ${PROJECT_FOLDER}/**/*.zip
when: always
artifacts:
reports:
dotenv: ${CI_PROJECT_DIR}/${PROJECT_FOLDER}/prebuild.env
allow_failure: false
But the pipeline always running too.
How to make sure the pipeline only run on Merge Request AND only when a .zip file was added to the specific folder ?
I would like to save gradle-based project's version into a variable in gitlab-ci script. In my build.gradle I have:
tasks.register('version') {
doLast {
println(version)
}
}
It reads version from gradle.properties (let's say version=0.1) and returns it.
I execute it as gradlew version -q so I get only result, with no unnecessary output. When using unix-style variable creation of command result, that is: version=$(./gradlew version -q), the runner ends script. Is it possible to save the output into a variable for script?
My .gitlab-ci.yml:
image: gradle:jdk11
cache: &wrapper
paths:
- .gradle/wrapper
- .gradle/caches
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle
- chmod a+x gradlew
stages:
- prepare
- build
- deploy
wrapper:
stage: prepare
script:
- gradle wrapper
compile:
stage: build
script:
- ./gradlew assemble
artifacts:
paths:
- build/classes/**
- build/libs/*.jar
cache:
<<: *wrapper
policy: pull
properties:
stage: deploy
script:
- eval version=$(./gradlew version -q)
- echo $version # not even called
I also tried to omit eval to have version=$(./gradlew version -q) in script, but nothing changes.
CI output:
$ export GRADLE_USER_HOME=`pwd`/.gradle
$ chmod a+x gradlew
$ version=$(./gradlew version -q)
Cleaning up file based variables
Ok, I've found the solution. I mustn't use variable, simply. It's needed to directly pass evaluation to another command, using double quotes, like:
.gitlab-ci.yml (part):
properties:
stage: deploy
script:
- echo "$(./gradlew version -q)"
It started to work
I try to implement a conditional versioning depending on if the CI script runs for a tagged branch or not.
However the version var is not resolved. Instead it is printed as a string.
The relevant jobs of the GitLab CI script:
# build template
.build_base_template: &build_base_template
image: registry.gitlab.com/xxxxxxx/npm:latest
tags:
- docker
stage: LintBuildTest
script:
- export CUR_VERSION='$(cat ./version.txt)$BUILD_VERSION_SUFFIX'
- npm ci
- npm run build
artifacts:
expire_in: 1 week
paths:
- dist/
# default build job
build:
before_script:
- export BUILD_VERSION_SUFFIX='-$CI_COMMIT_REF_SLUG-SNAPSHOT-$CI_COMMIT_SHORT_SHA'
<<: *build_base_template
except:
refs:
- tags
only:
variables:
- $FEATURE_NAME == null
# specific build job for tagged versions
build_tag:
before_script:
- export BUILD_VERSION_SUFFIX=''
<<: *build_base_template
only:
refs:
- tags
Variables which are exported within before_script are visible within script.
before:
before_script:
- export HELLOWELT="hi martin"
script:
- echo $HELLOWELT # prints "hi martin"
in general you can't export variables from child to parent processes.
As workaround you can use text file to write/read variable value. Also maybe it's possible to pass variable via yaml template.
I'm building a workflow on circleci 2.0 and so far jobs are running until it gets to android job.
At the build step ./gradlew assembleRelease it fails stating that an ENV VAR is not set:
Unzipping /home/circleci/.gradle/wrapper/dists/gradle-2.14.1-all/8bnwg5hd3w55iofp58khbp6yv/gradle-2.14.1-all.zip to /home/circleci/.gradle/wrapper/dists/gradle-2.14.1-all/8bnwg5hd3w55iofp58khbp6yv
Set executable permissions for: /home/circleci/.gradle/wrapper/dists/gradle-2.14.1-all/8bnwg5hd3w55iofp58khbp6yv/gradle-2.14.1/bin/gradle
FAILURE: Build failed with an exception.
* What went wrong:
Could not open terminal for stdout: $TERM not set
What I did try according to this post is setting the $TERM variable is a run command prior to the gradle invocation. But the build still fails looking for this variable.
Question:
How can you resolve $TERM not set on gradlew ./assembleRelease on CIrcleCI?
I did verify that I'm using the correct docker image according to this SO post:
https://stackoverflow.com/a/45744987/1829251
Here is the config.yml gist of the android CI Job:
android:
working_directory: ~/repo/android
docker:
- image: circleci/android:api-25-node8-alpha
steps:
- checkout:
path: ~/repo
- restore_cache:
key: jars-{{ checksum "build.gradle" }}-{{ checksum "app/build.gradle" }}
- attach_workspace:
at: ~/repo
- run: ./gradlew androidDepedencies
- run: export TERM=xterm
- run: sudo chmod +x ./gradlew
- run: ./gradlew assembleRelease
- save_cache:
paths:
- ~/.gradle
key: jars-{{ checksum "build.gradle" }}-{{ checksum "app/build.gradle" }}
- store_test_results:
path: ~/repo/android/reports
disclaimer: Developer Evangelist at CircleCI
- run: export TERM=xterm
That line sets the variable $TERM only for that specific shell. Each run step starts a brand new shell.
Your solution of running gradlew in the same step is one possible solution:
- run: export TERM=xterm && ./gradlew androidDepedencies
Another would be to properly export $TERM so that all subsequent steps can see the variable. This would be done like this:
- run: echo 'export TERM=xterm' >> $BASH_ENV
$BASH_ENV contains the path to the Bash file that is sourced at the beginning of every CircleCI step. Here's where this came from: https://circleci.com/docs/2.0/env-vars/#setting-path
I was exporting the ENV VAR incorrectly, using the following fixed the missing $TERM not set erorr in the android build:
- run: export TERM=xterm && ./gradlew androidDepedencies