I have a Bamboo pipeline that I want to improve build times on by making certain jobs run only if certain files were changed. I have seen similar functionality in Gitlab specs in the form of rules but uncertain if Bamboo has this feature.
I have reviewed the specs here: https://docs.atlassian.com/bamboo-specs-docs/7.2.3/specs.html?yaml#yaml for Bamboo and here: https://docs.gitlab.com/ee/ci/yaml/#rules for Gitlab and have seen the same question addressed specifically for gitlab: How to run CI job only if two files have been changed? [not one or another] but still unclear if it is possible to do this on Bamboo pipelines.
in the same yaml spec define multiple plans, each with their specific jobs. For each plan update following spec as needed (and specifically the file-filter-pattern):
repositories:
- myrepository:
type: git
url: https://bitbucket.org/myorg/myrepository.git
branch: main
shared-credentials: xyz
command-timeout-minutes: '180'
lfs: 'false'
verbose-logs: 'false'
use-shallow-clones: 'true'
cache-on-agents: 'false'
submodules: 'false'
ssh-key-applies-to-submodules: 'false'
fetch-all: 'false'
change-detection:
quiet-period:
quiet-period-seconds: '60'
max-retries: '5'
file-filter-type: include_only
file-filter-pattern: whatever/.*
Related
How to right write job in .gitlab-ci.yml when it run only in merge requests?
test_c:
stage: test
script:
- echo "This job tests something. It will only run when all jobs in the"
- echo "build stage are complete."
only:
- merge_requests
This job not run in merge request, but not run and in commint in master or develop.
Gitlab documentation recommends using 'rules' instead of 'only'. You can accomplish only merge_requests by doing the following:
test_c:
stage: test
script:
- echo "This job tests something. It will only run when all jobs in the"
- echo "build stage are complete."
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
https://docs.gitlab.com/ee/ci/yaml/#workflowrules
You can use the workflow to control pipeline creation. Define this keyword at the top level, with a single rules. This example shows that the pipeline will only be executed when a new merge request is created, the last when is set to never to prevent pipelines from executing when a new branch is pushed to the server or for any other type of event.
workflow:
rules:
- if: $CI_MERGE_REQUEST_ID
when: always
- when: never
Notice
As mentioned in the Gitlab documentation
These pipelines are labeled as detached in the UI, and they do not have access to protected variables. Otherwise, these pipelines are the same as other pipelines.
If your intention is just not to run the job on specific branches, like master or dev, you may simply exclude them with except:
test_c:
(...)
except:
- master
- dev
Your code is correct, commit it to master before test your merge_request pipelines please.
I'm having a Gitlab CI/CD pipeline, and it works OK generally.
My problem is that my testing takes more than 10 minutes and it not stable (YET..) so occasionally randomly it fails on a minor test that I don't care for.
Generally, after retry, it works, but if I need an urgent deploy I need to wait another 10 minutes.
When we have an urgent bug, another 10 minutes is waaaay too much time, so I am looking for a way to force deploy even when the test failed.
I have the next pseudo ci yaml scenario that I'd failed to find a way to accomplish
stages:
- build
- test
- deploy
setup_and_build:
stage: build
script:
- build.sh
test_branch:
stage: test
script:
- test.sh
deploy:
stage: deploy
script:
- deploy.sh
only:
- master
I'm looking for a way to deploy manually if the test phase failed.
but if I add when: manual to the deploy, then deploy never happens automatically.
so a flag like when: auto_or_manual_on_previous_fail will be great.
currently, there is no such flag in Gitlab ci.
Do you have any idea for a workaround or a way to implement it?
Another approach would be to skip the test in case of an emergency release.
For that, follow "Skipping Tests in GitLab CI" from Andi Scharfstein, and:
add "skip test" in the commit message triggering that emergency release
check a variable on the test stage
That is:
.test-template: &test-template
stage: tests
except:
variables:
- $CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i
- $SKIP_TESTS
As you can see above, we also included the variable $SKIP_TESTS in the except block of the template.
This is helpful when triggering pipelines manually from GitLab’s web interface.
Here’s an example:
It's possible to control the job attribute of your deploy job by leveraging parent-child pipelines (gitlab 12.7 and above). This will let you decide if you want the job in the child pipeline to run as manual, or always
Essentially, you will need to have a .gitlab-ci.yml with:
stages:
- build
- test
- child-deploy
child-deploy stage will be used to run the child pipeline, in which the deploy job will run with the desired attribute.
Your test could generate as artifact the code for deploy section. For example, in the after_script section of your test, you can check the value of CI_JOB_STATUS builtin variable to decide if you want to write the child job to run as manual or always:
my_test:
stage: test
script:
- echo "testing, exit 0 on success, exit 1 on failure"
after_script:
- if [ "$CI_JOB_STATUS" == "success" ]; then WHEN=always; else WHEN=manual; fi
- |
cat << 'EOF' > deploy.yml
stages:
- deploy
my_deploy:
stage: deploy
rules:
- when: $WHEN
script:
- echo "deploying"
EOF
artifacts:
when: always
paths:
- deploy.yml
Note that variable expension is disabled in the heredoc section, by the use of single quoted 'EOF'. If you need variable expension, remember to escape the $ of $WHEN.
Finally, you can trigger the child pipeline with deploy.yml
gen_deploy:
stage: child-deploy
when: always
trigger:
include:
- artifact: deploy.yml
job: my_test
strategy: depend
In the circleci version 1 config, there was the option to specify owner as an option in a deployment. An example from the circleci docs ( https://circleci.com/docs/1.0/configuration/ ) with owner: circleci being the key line:
deployment:
master:
branch: master
owner: circleci
commands:
- ./deploy_master.sh
In version 2 of the config, there is the ability to use filters and tags to specify which branches are built, but I have yet to find (in the docs, or on the interwebs) anything that gives me the same capability.
What I'm trying to achieve is run build and test steps on forks, but only run the deploy steps if the repository owner is the main repo. Quite often people fork using the same branch name - in this case master - so having a build fail due to an inability to deploy is counter-intuitive, especially as I would like to use a protected branch in git and only merge commits based on a successful build in a pull request.
I realise we could move to only running builds based on tags being present, but nothing is stopping somebody with a fork also creating a tag in their fork, which puts us back at square one.
Is anybody aware of how to specify the owner of a repo in the version 2 config?
An example from the version 2 config document ( https://circleci.com/docs/2.0/workflows/ ) in case it helps jog somebodies memory:
workflows:
version: 2
un-tagged-build:
jobs:
- build:
filters:
tags:
ignore: /^v.*/
tagged-build:
jobs:
- build:
filters:
branches:
ignore: /.*/
tags:
only: /^v.*/
disclaimer: Developer Evangelist at CircleCI
That feature is not available on CircleCI 2.0. You can request it here.
As an alternative, you might be able to look for the branch name, say master, as well as the CIRCLE_PR_NUMBER environment variable. If that variable has any value, then it's a fork of master and you shouldn't deploy.
I'd like to disable, or turn off, Travis CI builds for stretches of time. In my case it's because I'm using a private repo and don't want to use up free builds. Also, there are times down the road I'll want to disable Travis builds while experimenting with new features.
There are 2 methods to do this but neither is ideal:
Append[ci skip] or [skip ci] to a commit message - this would become tedious
Disable builds in the TravisCI.com interface - requires logging in to TravisCI to turn it on/off, or even see if it's on/off
What I'm looking for is a way to disable builds for a few days at a time, in my project, without completely deleting the .travis.yml file?
Is it possible to disable Travis CI builds via a config setting in .travis.yml?
Something like this:
enabled: false // Travis CI builds turned off
language: node_js
node_js:
- "6.11.2"
deploy:
provider: heroku
api_key:
secure: [KEY]
app: [APP-NAME]
If you do not create any job (or creates too many) in .travis.yml, the build request will be rejected.
For example:
language: ruby
rvm: '2.4'
matrix:
exclude:
rvm: '2.4'
This may be impractical depending on how complex your intended configuration is.
You'd probably want to use a CLI client https://github.com/travis-ci/travis.rb. See https://github.com/travis-ci/travis.rb#enable and https://github.com/travis-ci/travis.rb#disable.
We use a Gitlab Project in a team. Each developer has his own Kubernetes cluster in the cloud and an own branch within GitLab. We use GitLab-CI to automatically build new containers and deploy them to our Kubernetes clusters.
At the moment we have a .gitlab-ci.yml looks something like this:
variables:
USERNAME: USERNAME
CI_K8S_PROJECT: ${USERNAME_CI_K8S_PROJECT}
REGISTRY_JSON_KEY_FILE: ${USERNAME_REGISTRY_JSON_KEY_FILE}
[...]
stages:
- build
- deploy
- remove
build-zeppelin:
stage: build
image: docker:latest
variables:
image_name: "zeppelin"
only:
- ${USERNAME}#Gitlab-Repo
tags:
- cloudrunner
script:
- docker login -u _json_key -p "${REGISTRY_JSON_KEY_FILE?}" https://eu.gcr.io
- image_name_fqdn="eu.gcr.io/${CI_K8S_PROJECT?}/${image_name?}:latest"
- docker build -t ${image_name_fqdn?} .
- docker push ${image_name_fqdn?}
- echo "Your new image is '${image_name_fqdn?}'. Have fun!"
[...]
So in the beginning we reference the important information by using a USERNAME-prefix. This works quite well, but is problematic, since we need to correct them after every pull request from another user.
So we search for a way to keep the gitlab-ci file the same to every developer while still referencing some gitlab-variables different for every developer.
Things we thought about, that don't seem to work:
Use multiple yml files and import them into each other => not supported.
Try to combine Gitlab Environment variables as Prefix:
CI_K8S_PROJECT: ${${GITLAB_USER_ID}_CI_K8S_PROJECT}
or
INDIVIDUAL_CI_K8S_PROJECT: ${GITLAB_USER_ID}_CI_K8S_PROJECT
CI_K8S_PROJECT: ${INDIVIDUAL_CI_K8S_PROJECT}
We found a solution using indirect expansion (bash feature):
before_script:
- variableName=${GITLAB_USER_ID}_CI_K8S_PROJECT
- export wantedValue=${!variableName}
But we also recognised, that our setup was somehow stupid: It does not make sense to have multiple branches for each user and use prefixed variables, since this leads to problems such as the above and security concerns, since all variables are accessible to all users.
It is way easier if each user forks the root project and simply creates a merge request for new features. This way there is no renaming/prefixing of variables or branches necessary at all.
Solution from #nik will work only for bash. For sh will work:
before_script:
- variableName=...
- export wantedValue=$( eval echo \$$variableName )
Something like this works (on 15.0.5-ee):
variables:
IMAGE_NAME: "test-$CI_PROJECT_NAME"