I want my Gitlab CI job to not run when the commit message starts with a particular string: [maven-scm]
So, I have the below configuration in my .gitlab-ci.yaml file:
image: maven:3.6.3-jdk-11-slim
stages:
- test
test:
stage: test
cache:
key: all
paths:
- ./.m2/repository
script:
- mvn clean checkstyle:check test spotbugs:check
rules:
- if: '$CI_COMMIT_MESSAGE !~ /^\[maven-scm\] .*$/'
My commit message is: [maven-scm] I hope the test job does not run
But the test job still runs to my frustration. I went over the GitLab documentation for rules but could not find the reason why the job still runs. I am not sure if I am missing something.
Would be great if someone can point me in the right direction.
Update:
I tried the only/except clause instead of the rules. I modified the yaml file to below:
image: maven:3.6.3-jdk-11-slim
stages:
- test
test:
stage: test
cache:
key: all
paths:
- ./.m2/repository
script:
- mvn clean checkstyle:check test spotbugs:check
except:
variables:
- $CI_COMMIT_MESSAGE =~ /^\[maven-scm\] .*$/
The job still runs when the commit message starts with [maven-scm].
This was a tricky problem, because the issue was not with the rules section. The problem is actually the regex. You only need to specify the desired pattern at the start of the commit message, i.e. don't need the following wildcard. The following works and has been tested:
test-rules:
stage: test
rules:
- if: '$CI_COMMIT_MESSAGE !~ /^\[maven-scm\] /'
script:
- echo "$CI_COMMIT_MESSAGE"
This has been tested with the following commit messages:
This commit message will run the job
This commit message [maven-scm] will run the job
[maven-scm] This commit message will NOT run the job
FYI GitLab documentation specifies that rules is preferred over only/except, so best to stick with rules: if. See onlyexcept-basic.
Related
I'm trying to build a bamboo specs yaml but I'm having some weird errors which have messages that are not helping much. I followed the documentation for it here but still not working.
So I have bamboo 7.2.4 and I'm trying to create a stage
version:2
stages:
- run tests:
jobs:
- Test
Test:
tasks:
- script:whatever
When running this I get
Bamboo YAML import failed: Document structure is incorrect: Tests: Property is required.
No clue what that means nor why it's happening
Bamboo YAML specs are very difficult to troubleshoot. This is one disadvantage of YAML specs when compared to Java specs. Looks like you are missing some key and essential tags in your example code. Can you re-format as below and see?
First, manually create a project (or use an existing project) but make sure to update the project-key by replacing <MYKEY> below,
---
version:2
plan:
project-key: <MYKEY>
key: MYPLN
name: My Plan
stages:
- run tests:
jobs:
- Test
Test:
key: JB1
tasks:
- script:
- echo 'My Plan'
I have many Gitlab project followed the same CI template. Whenever there is a small change in the CI script, I have to manually modify the CI script in each project. Is there a way you can store your CI script in a central location and have your project called that CI script with some environment variable substitution? For instance,
gitlab-ci.yml in each project
/bin/bash -c "$(curl -fsSL <link_to_the_central_location>.sh)"
gitlab-ci.yml in the central location
stages:
- build
- test
build-code-job:
stage: build
script:
- echo "Check the ruby version, then build some Ruby project files:"
- ruby -v
- rake
test-code-job1:
stage: test
script:
- echo "If the files are built successfully, test some files with one command:"
- rake test1
test-code-job2:
stage: test
script:
- echo "If the files are built successfully, test other files with a different command:"
- rake test2
You do not need curl, actually gitlab supports this via the include directive.
you need a repository, where you store your general yml files. (you can choose if it is a whole ci file, or just parts. For this example lets call this repository CI and assume your gitlab runs at example.com - so the project url would be example.com/ci. we create two files in there just to show the possibilities.
is a whole CI definition, ready to use - lets call the file ci.yml. This approach is not really flexible
stages:
- build
- test
build-code-job:
stage: build
script:
- echo "Check the ruby version, then build some Ruby project files:"
- ruby -v
- rake
test-code-job1:
stage: test
script:
- echo "If the files are built successfully, test some files with one command:"
- rake test1
test-code-job2:
stage: test
script:
- echo "If the files are built successfully, test other files with a different command:"
- rake test2
is a partly CI definition, which is more extendable. lets call the files includes.yml
.build:
stage: build
script:
- echo "Check the ruby version, then build some Ruby project files:"
- ruby -v
- rake
.test:
stage: test
script:
- echo "this script tag will be overwritten"
There is even the option to use template string from yaml. please reference the gitlab documentation but it is similar to 2.
we do have our project which wants to use such definitions. so either
For the whole CI file
include:
- project: 'ci'
ref: master # think about tagging if you need it
file: 'ci.yml'
as you can see now we are referencing one yml file, with all the cahnges.
with partial extends
include:
- project: 'ci'
ref: master # think about tagging if you need it
file: 'includes.yml'
stages:
- build
- test
build-code-job:
extends: .build
job1:
extends: .test
script:
- rake test1
job2:
extends: .test
script:
- rake test2
As you see, you can easily use the includes, to have a way more granular setup. Additionally you could define at job1 and job2 variables, eg for the test target, and move the script block into the includes.yml
Futhermore you can also use anchors for the script parts. Which looks like this
includes.yml
.build-scirpt: &build
- echo "Check the ruby version, then build some Ruby project files:"
- ruby -v
- rake
.build:
stage: build
script:
- *build
and you can use also the script anchor within your configuration
For a deeper explanation you can also take a look at https://docs.gitlab.com/ee/ci/yaml/includes.html
How to right write job in .gitlab-ci.yml when it run only in merge requests?
test_c:
stage: test
script:
- echo "This job tests something. It will only run when all jobs in the"
- echo "build stage are complete."
only:
- merge_requests
This job not run in merge request, but not run and in commint in master or develop.
Gitlab documentation recommends using 'rules' instead of 'only'. You can accomplish only merge_requests by doing the following:
test_c:
stage: test
script:
- echo "This job tests something. It will only run when all jobs in the"
- echo "build stage are complete."
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
https://docs.gitlab.com/ee/ci/yaml/#workflowrules
You can use the workflow to control pipeline creation. Define this keyword at the top level, with a single rules. This example shows that the pipeline will only be executed when a new merge request is created, the last when is set to never to prevent pipelines from executing when a new branch is pushed to the server or for any other type of event.
workflow:
rules:
- if: $CI_MERGE_REQUEST_ID
when: always
- when: never
Notice
As mentioned in the Gitlab documentation
These pipelines are labeled as detached in the UI, and they do not have access to protected variables. Otherwise, these pipelines are the same as other pipelines.
If your intention is just not to run the job on specific branches, like master or dev, you may simply exclude them with except:
test_c:
(...)
except:
- master
- dev
Your code is correct, commit it to master before test your merge_request pipelines please.
I want to run Job only for merge_requests refs AND when changes for specified files, according to official doc i create this Job in .gitlab-ci.yml:
merge-request-test:
<<: *some_anchor
only:
refs:
- merge_requests
changes:
- "*.py"
- "**/*.py"
- postman/some-file.json
But this job starts even if i change any other file.
I understand that GitLab CI apply OR rule for refs and changes sections:
In the example below, the test job will not be created when any of the
following are true:
The pipeline runs for the master. There are changes to the README.md
file in the root directory of the repo.
test:
script: npm run test
except:
refs:
- master
changes:
- "README.md"
I want to start this Job only for MR and when changes specified files.
How i can achieve this behaviour?
If remove refs section, it works only when files changed, but official doc does not recommend do this, because i want to use it in pipelines:
If using only:changes with only allow merge requests to be merged if
the pipeline succeeds, undesired behavior could result if you do not
also use only:merge_requests.
For now i have this working solution:
merge-request-deploy:
<<: *deploy-app
rules:
- if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "develop"'
changes:
- "*.py"
- "**/*.py"
- postman/some-file.json
when: always
if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "develop"' - gives me MR branc
changes - gives me changed files
Rule's sections works with AND rule.
I'm having a Gitlab CI/CD pipeline, and it works OK generally.
My problem is that my testing takes more than 10 minutes and it not stable (YET..) so occasionally randomly it fails on a minor test that I don't care for.
Generally, after retry, it works, but if I need an urgent deploy I need to wait another 10 minutes.
When we have an urgent bug, another 10 minutes is waaaay too much time, so I am looking for a way to force deploy even when the test failed.
I have the next pseudo ci yaml scenario that I'd failed to find a way to accomplish
stages:
- build
- test
- deploy
setup_and_build:
stage: build
script:
- build.sh
test_branch:
stage: test
script:
- test.sh
deploy:
stage: deploy
script:
- deploy.sh
only:
- master
I'm looking for a way to deploy manually if the test phase failed.
but if I add when: manual to the deploy, then deploy never happens automatically.
so a flag like when: auto_or_manual_on_previous_fail will be great.
currently, there is no such flag in Gitlab ci.
Do you have any idea for a workaround or a way to implement it?
Another approach would be to skip the test in case of an emergency release.
For that, follow "Skipping Tests in GitLab CI" from Andi Scharfstein, and:
add "skip test" in the commit message triggering that emergency release
check a variable on the test stage
That is:
.test-template: &test-template
stage: tests
except:
variables:
- $CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i
- $SKIP_TESTS
As you can see above, we also included the variable $SKIP_TESTS in the except block of the template.
This is helpful when triggering pipelines manually from GitLab’s web interface.
Here’s an example:
It's possible to control the job attribute of your deploy job by leveraging parent-child pipelines (gitlab 12.7 and above). This will let you decide if you want the job in the child pipeline to run as manual, or always
Essentially, you will need to have a .gitlab-ci.yml with:
stages:
- build
- test
- child-deploy
child-deploy stage will be used to run the child pipeline, in which the deploy job will run with the desired attribute.
Your test could generate as artifact the code for deploy section. For example, in the after_script section of your test, you can check the value of CI_JOB_STATUS builtin variable to decide if you want to write the child job to run as manual or always:
my_test:
stage: test
script:
- echo "testing, exit 0 on success, exit 1 on failure"
after_script:
- if [ "$CI_JOB_STATUS" == "success" ]; then WHEN=always; else WHEN=manual; fi
- |
cat << 'EOF' > deploy.yml
stages:
- deploy
my_deploy:
stage: deploy
rules:
- when: $WHEN
script:
- echo "deploying"
EOF
artifacts:
when: always
paths:
- deploy.yml
Note that variable expension is disabled in the heredoc section, by the use of single quoted 'EOF'. If you need variable expension, remember to escape the $ of $WHEN.
Finally, you can trigger the child pipeline with deploy.yml
gen_deploy:
stage: child-deploy
when: always
trigger:
include:
- artifact: deploy.yml
job: my_test
strategy: depend