I am making a CICD pipeline with circle-ci-terraform
My dev branch names are dev/feature_name etc. Instead of running workflows by directly writing the branch name I want to use the prefix dev/ so that the same workflow runs for all branches starting with dev. Any idea how I can achieve that?
Here's my current workflow -
workflows:
dev_workflow:
jobs:
- build:
filters:
branches:
only:
- dev/cicd-integration
I already tried dev/*, it does not work.
CircleCI uses the Java variant of RegEx pattern matching (https://circleci.com/docs/workflows/#using-regular-expressions-to-filter-tags-and-branches).
So in your case, you need to specify the following RegEx pattern:
filters:
branches:
only:
- /^dev\/.*/
Related
I want to run Job only for merge_requests refs AND when changes for specified files, according to official doc i create this Job in .gitlab-ci.yml:
merge-request-test:
<<: *some_anchor
only:
refs:
- merge_requests
changes:
- "*.py"
- "**/*.py"
- postman/some-file.json
But this job starts even if i change any other file.
I understand that GitLab CI apply OR rule for refs and changes sections:
In the example below, the test job will not be created when any of the
following are true:
The pipeline runs for the master. There are changes to the README.md
file in the root directory of the repo.
test:
script: npm run test
except:
refs:
- master
changes:
- "README.md"
I want to start this Job only for MR and when changes specified files.
How i can achieve this behaviour?
If remove refs section, it works only when files changed, but official doc does not recommend do this, because i want to use it in pipelines:
If using only:changes with only allow merge requests to be merged if
the pipeline succeeds, undesired behavior could result if you do not
also use only:merge_requests.
For now i have this working solution:
merge-request-deploy:
<<: *deploy-app
rules:
- if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "develop"'
changes:
- "*.py"
- "**/*.py"
- postman/some-file.json
when: always
if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "develop"' - gives me MR branc
changes - gives me changed files
Rule's sections works with AND rule.
Using gitlab ci, I have a repo where all my templates are created.
For example, I have a sonar scanner job template named .sonar-scanner.yml:
sonar-analysis:
stage: quality
image:
name: sonar-scanner-ci:latest
entrypoint: [""]
script:
- sonar-scanner
-D"sonar.projectKey=${SONAR_PROJECT_NAME}"
-D"sonar.login=${SONAR_LOGIN}"
-D"sonar.host.url=${SONAR_SERVER}"
-D"sonar.projectVersion=${CI_COMMIT_SHORT_SHA}"
-D"sonar.projectBaseDir=${CI_PROJECT_DIR}"
I have include this template as a project like this in main gitlab ci file:
include:
- project: 'organization/group/ci-template'
ref: master
file: '.sonar-scanner.yml'
So as you can understand I have a repo named ci-templates where all my templates are created. And in another repo, I extends using include these templates.
Finally, in a repo, when a new merge request is created, my job for sonar is running under another file in my project test/quality.yml:
sonar:
stage: quality
extends:
- sonar-analysis
allow_failure: true
only:
refs:
- merge_requests
All is working well except the substitution or the overriding of my env var. Indeed of my template. I have many sonar server or project names. I would like to know how to override these variables SONAR_SERVER and SONAR_PROJECT_NAME when I extend the job from a template.
In my main .gitlab-ci.yml file I have a variables section declaration, and when I override these variables in, it works.
But it's not really what I want. Using many stages and many micro service it is possible to reuse the same extending job in a different way. That I really want to do is to override these variables directly in the file test/quality.yml.
This, for example does not work:
sonar:
stage: quality
extends:
- sonar-analysis
variables:
SONAR_PROJECT_NAME: foo
SONAR_SERVER: bar
allow_failure: true
only:
refs:
- merge_requests
This not work too:
variables:
SONAR_PROJECT_NAME: foo
SONAR_SERVER: bar
sonar:
stage: quality
extends:
- sonar-analysis
allow_failure: true
only:
refs:
- merge_requests
What is the best way to make this work ?
Since this question was asked in Feburary 2020, a new MR Use non-predefined variables inside CI include blocks has been merged into Gitlab 14.2, which resolves the issue for the overridden job.
The project that does the include can redefine the variables when extending a job:
include:
- project: 'organization/group/ci-template'
ref: master
file: '.sonar-scanner.yml'
sonar:
stage: quality
extends:
- sonar-analysis
variables:
SONAR_PROJECT_NAME: foo
SONAR_SERVER: bar
allow_failure: true
But in this case you probably want the job in the template to start with a dot .sonar-analysis instead of sonar-analysis to not create a real sonar-analysis job in the template (see hidden jobs).
Or you can also directly set the variables values (to redefine them) of an existing job in the project that does the include:
include:
- project: 'organization/group/ci-template'
ref: master
file: '.sonar-scanner.yml'
sonar-analysis:
variables:
SONAR_PROJECT_NAME: foo
SONAR_SERVER: bar
I verified this with a test project, which includes a template from a peer test project, and when it runs, results in two jobs. This is the job output for the overriding job:
$ echo sonar.projectKey=${SONAR_PROJECT_NAME}
sonar.projectKey=foo
$ echo sonar.login=${SONAR_LOGIN}
sonar.login=bob
$ echo sonar.host.url=${SONAR_SERVER}
sonar.host.url=bar
I use GitLab CI for my build action.
I need to build my production env only on master upstream branch and only by a tag.
Now I have something like this:
stages:
- build
- test
- deploy
build_project:
stage: build
script:
- cd ./some-dir
- build-script.sh
only:
- master
- tags
Does someone know how should I change my rules?
There's an example of how to do this in the Gitab docs: https://docs.gitlab.com/ce/ci/yaml/README.html#onlyexcept-basic
job:
only:
- branches#gitlab-org/gitlab
except:
- master#gitlab-org/gitlab
- /^release/.*$/#gitlab-org/gitlab
Im trying to create a very abstract .yaml-configuration which can be reused in different places.
Right now it looks like this:
.release: &release
stage: release
script:
- docker tag $IMAGE_TESTING $IMAGE_RELEASE
- docker push $IMAGE_RELEASE
only:
- master
when: manual
.amd64: &amd64
BASE_ARCH: 'amd64'
.debalike: &debalike
FLAVOUR: 'debalike'
release_debalike_amd64:
<<: *release
variables:
<< : [*amd64, *debalike]
Which correctly gets parsed into ...
.release:
stage: release
script:
- 'docker tag $IMAGE_TESTING $IMAGE_RELEASE'
- 'docker push $IMAGE_RELEASE'
only:
- master
when: manual
.amd64:
BASE_ARCH: amd64
.debalike:
FLAVOUR: debalike
release_debalike_amd64:
stage: release
script:
- 'docker tag $IMAGE_TESTING $IMAGE_RELEASE'
- 'docker push $IMAGE_RELEASE'
only:
- master
when: manual
variables:
BASE_ARCH: amd64
FLAVOUR: debalike
Which is the desired behaviour.
But would it be possible to avoid using the variables tag in release_debalike_amd64 and use include the anchors directly?
Something similar to this (which does not work):
.release: &release
stage: release
script:
- docker tag $IMAGE_TESTING $IMAGE_RELEASE
- docker push $IMAGE_RELEASE
only:
- master
when: manual
.amd64: &amd64
variables:
BASE_ARCH: 'amd64'
.debalike: &debalike
variables:
FLAVOUR: 'debalike'
release_debalike_amd64:
<<: *release
<<: [*amd64, *debalike]
Right now the yaml parser ignores the *debalike and just includes the values from *amd64.
Any way to achieve this? This is a .gitlab-ci.yml if it matters.
Unfortunately not. It is not possible to do deep merges using only YAML anchors and aliases.
GitLab EE introduced CI includes in 10.5, which was enhanced in 10.8 to do deep merges of CI jobs. I don't think it's going to help you in this particular case, but it's something you might be able to leverage in other ways depending on how you organize your CI files.
See include for more information about the include parameter.
In the circleci version 1 config, there was the option to specify owner as an option in a deployment. An example from the circleci docs ( https://circleci.com/docs/1.0/configuration/ ) with owner: circleci being the key line:
deployment:
master:
branch: master
owner: circleci
commands:
- ./deploy_master.sh
In version 2 of the config, there is the ability to use filters and tags to specify which branches are built, but I have yet to find (in the docs, or on the interwebs) anything that gives me the same capability.
What I'm trying to achieve is run build and test steps on forks, but only run the deploy steps if the repository owner is the main repo. Quite often people fork using the same branch name - in this case master - so having a build fail due to an inability to deploy is counter-intuitive, especially as I would like to use a protected branch in git and only merge commits based on a successful build in a pull request.
I realise we could move to only running builds based on tags being present, but nothing is stopping somebody with a fork also creating a tag in their fork, which puts us back at square one.
Is anybody aware of how to specify the owner of a repo in the version 2 config?
An example from the version 2 config document ( https://circleci.com/docs/2.0/workflows/ ) in case it helps jog somebodies memory:
workflows:
version: 2
un-tagged-build:
jobs:
- build:
filters:
tags:
ignore: /^v.*/
tagged-build:
jobs:
- build:
filters:
branches:
ignore: /.*/
tags:
only: /^v.*/
disclaimer: Developer Evangelist at CircleCI
That feature is not available on CircleCI 2.0. You can request it here.
As an alternative, you might be able to look for the branch name, say master, as well as the CIRCLE_PR_NUMBER environment variable. If that variable has any value, then it's a fork of master and you shouldn't deploy.