I have a GitHub action workflow that tests, builds and deploys. Quite common.
I want the job test to run on both main and PRs (into main). But I want the build and deploy to only run on pushes to main. How can I safely protect this?
The summarized .github/workflows/ci.yml looks like:
name: CI
on:
push:
branches: ["main"]
pull_request:
branches: ["main"]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: "Lint, Test and Report"
run: echo "running linter, then tests then report on this"
build:
needs: test
runs-on: ubuntu-latest
steps:
- name: "Build"
run: echo "Building the artifacts"
deploy:
needs:
- test
- build
runs-on: ubuntu-latest
steps:
- name: "Deploy to Production"
run: echo "Drumroll...."
I don't see any ENV variable or github.x attribute that indicates that this is a PR. Maybe I'm missing something obvious?
Is it safe to match on branch-name instead? And e.g. use a
if: startsWith(github.ref, 'refs/heads/main')
to ensure we only ever run when the branch is main?
Yes, this is safe. However, change it to:
if: github.ref == 'refs/heads/main'
because otherwise somebody could push a branch mainfoo and you'd trigger the job as well.
An alternative would be to check for the event name, like:
if: github.event.name == 'push'
however I'd say this is less robust, since somebody could change the trigger above and remove the branches: .. part of the push trigger and suddenly you're deploying from PRs.
Related
I have workflow file that I want it to run on non-draft PRs and on every new commit to the PR.
So far, I have tried two ways:
Using if statement
name: Test
on:
pull_request:
branches:
- master
jobs:
test:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
This does not trigger workflow when PR converted to ready for review.
Using types statement
name: Test
on:
pull_request:
branches:
- master
types:
- ready_for_review
jobs:
test:
runs-on: ubuntu-latest
This does not trigger workflow when a new commit pushed to PR.
How can I add a condition so that my workflow runs on non-draft PRs and also all new commits?
This is similar to another question.
The answer in the other question suggests using
if: ! github.event.pull_request.draft
to check if the PR is a draft, since there might otherwise be some type confusion with booleans in GitHub Actions.
In your first code block you leave the types as the default (opened, synchronize & reopened).
In the second code block you only use the ready_for_review type.
You can just combine the two:
name: Test
on:
pull_request:
types:
- opened
- synchronize
- reopened
- ready_for_review
jobs:
test:
if: github.event.pull_request.draft == false
# You could have ths version too - note the single quotes:
# if: '! github.event.pull_request.draft'
name: Check PR is not a Draft
runs-on: ubuntu-latest
steps:
- run: |
echo "Non-draft Pull request change detected"
This is a question for the github workflow action dawidd6/action-download-artifact.
There is no discussion board in https://github.com/dawidd6/action-download-artifact, so asking this question in this forum.
This is how I wish to use this workflow in my GitHub repo:
A pull request is created.
This triggers an workflow – lets call it the “build workflow” - to build the entire repo and uploads the build artifacts.
Then another workflow – lets call it the “test workflow” - should start, that should download the build artifact using action-download-artifact and run some other actions.
Now if I put the trigger for the “test workflow” as pull_request, then how can I make it wait for the corresponding “build workflow” to complete? Do I specify the run_id ?
For now I am using “workflow_run” as the trigger for the run WF. But then when a PR is created, it does not show the “test workflow” as one of the checks for the PR. Can you help me figure out the correct way of using the download-artifact action that would help for my purpose?
You could write two workflows where the first builds when the pull request is opened or edited, and the second executes the test when the pull request is closed and merged. The HEAD commit SHA could be used to identify the artifact name between the two workflows.
I'm going to reword your requirements slightly.
Build everything and upload the artifacts when a pull request is opened or edited (e.g. new commits added).
Download the artifact and test it when a pull request is closed and merged.
Here are two sample workflows that would accomplish that. You will need to create a token to share the artifacts between workflows (see secrets.GITHUB_TOKEN below).
Build.yml
name: Build
on:
pull_request:
jobs:
Build:
steps:
- name: Environment Variables
shell: bash
run: |
ARTIFACTS_SHA=$(git rev-parse HEAD)
BUILD_ARTIFACTS=BuildArtifacts_${ARTIFACTS_SHA}
echo "ARTIFACTS_DIR=$ARTIFACTS_DIR" >> $GITHUB_ENV
- name: Build
run: make
- name: Capture Artifacts
uses: actions/upload-artifact#2
with:
name: Artifacts_${{ env.ARTIFACTS_SHA }}
path: path/to/artifact/
Test.yml
name: Test
on:
pull_request:
types: [closed]
jobs:
Test:
steps:
- name: Environment Variables
shell: bash
run: |
ARTIFACTS_SHA=$(git rev-parse HEAD)
BUILD_ARTIFACTS=BuildArtifacts_${ARTIFACTS_SHA}
echo "ARTIFACTS_DIR=$ARTIFACTS_DIR" >> $GITHUB_ENV
- name: Download Artifacts
uses: dawidd6/action-download-artifact#v2
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
workflow: Build.yml
name: ${{ env.BUILD_ARTIFACTS }}
On my gitlab repo I have to trigger pipeline only when a specific folder have changes AND when it's a Merge request (both condition). Especially on .zip file extension, i-e, add a new zip file in this folder, create a Merge request, then run the pipeline.
This is my initial pipeline yaml code:
trigger-ci-zip-file-only:
stage: prebuild
extends:
- .prebuild
- .preprod-tags
variables:
PROJECT_FOLDER: "my-specific-folder"
before_script:
- echo "Job currently run in $CI_JOB_STAGE"
- touch ${CI_PROJECT_DIR}/$PROJECT_FOLDER/prebuild.env
- cd $PROJECT_FOLDER
only:
refs:
- merge_requests
changes:
- ${PROJECT_FOLDER}/*.zip
artifacts:
reports:
dotenv: ${CI_PROJECT_DIR}/${PROJECT_FOLDER}/prebuild.env
allow_failure: false
As you can see, my pipeline have to be triggered only when there is a change in a specific folder on zip files and only on MR. But in this state, the pipeline is always running when MR is creatd or when I push on an existing MR, even if there is no changes or adding in the specific folder.
I also tried to make some changes on the pipeline yaml code like this:
only:
refs:
- merge_requests
changes:
- ${PROJECT_FOLDER}/**/*.zip
But the pipeline always running.
Also I tried this:
trigger-ci-zip-file-only:
stage: prebuild
extends:
- .prebuild
- .preprod-tags
variables:
PROJECT_FOLDER: "my-specific-folder"
before_script:
- echo "Job currently run in $CI_JOB_STAGE"
- touch ${CI_PROJECT_DIR}/$PROJECT_FOLDER/prebuild.env
- cd $PROJECT_FOLDER
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- changes:
- ${PROJECT_FOLDER}/**/*.zip
when: always
artifacts:
reports:
dotenv: ${CI_PROJECT_DIR}/${PROJECT_FOLDER}/prebuild.env
allow_failure: false
But the pipeline always running too.
How to make sure the pipeline only run on Merge Request AND only when a .zip file was added to the specific folder ?
I would like to avoid repeating a strategy matrix across jobs:
jobs:
build-sdk:
runs-on: macOS-latest
strategy:
fail-fast: false
matrix:
qt-version: ['5.15.1']
ios-deployment-architecture: ['arm64', 'x86_64']
ios-deployment-target: '12.0'
steps:
…
create-release:
needs: build-sdk
runs-on: macOS-latest
steps:
…
publish-sdk:
needs: [build-sdk, create-release]
runs-on: macOS-latest
strategy:
fail-fast: false
matrix: ?????
steps:
…
Is this possible (without creating a job to create the matrix as JSON itself)?
There's an action that allows uploading multiple assets to the same release from a matrix build that's triggered on push to a tag. Someone filed an issue about this specific use-case, and the action's author responded with
Assets are uploaded for the GitHub release associated with the same tag so as long as the this action is run in a workflow run for the same tag all assets should get added to the same GitHub release.
This suggests that a workflow like this would probably meet your needs:
on:
push:
tags:
- 'v*' # Push events to matching v*, i.e. v1.0, v20.15.10
jobs:
release:
runs-on: macOS-latest
strategy:
fail-fast: false
matrix:
qt-version: ['5.15.1']
ios-deployment-architecture: ['arm64', 'x86_64']
ios-deployment-target: '12.0'
steps:
- name: build SDK
run: ...
- name: Create Release
uses: softprops/action-gh-release#v1
with:
files: |
- "SDK_file1" # created in previous build step
- "SDK_file2"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: username/reponame
- name: publish SDK
run: ...
I've simplified what you would need to do, but I'm guessing you might be wanting to upload assets with names reflecting their applicable matrix options. For that detail, I recommend adding an explicit step in your job to create the asset's filename and add it to the job environment, somewhat similar to what I've done here:
- name: Name asset
run: |
BINARY_NAME=sdk-qt${{matrix.qt-version}}-iOS${{matrix.ios-deployment-target}}-${{matrix.ios-deployment-architecture}}
echo "BINARY_NAME=$BINARY_NAME" >> $GITHUB_ENV
Then, when your build step generates your assets, you can name them with the filename in ${{env.BINARY_NAME}}, and pass that same name to the release creation step like I've done in my asset release step here.
I'm having a Gitlab CI/CD pipeline, and it works OK generally.
My problem is that my testing takes more than 10 minutes and it not stable (YET..) so occasionally randomly it fails on a minor test that I don't care for.
Generally, after retry, it works, but if I need an urgent deploy I need to wait another 10 minutes.
When we have an urgent bug, another 10 minutes is waaaay too much time, so I am looking for a way to force deploy even when the test failed.
I have the next pseudo ci yaml scenario that I'd failed to find a way to accomplish
stages:
- build
- test
- deploy
setup_and_build:
stage: build
script:
- build.sh
test_branch:
stage: test
script:
- test.sh
deploy:
stage: deploy
script:
- deploy.sh
only:
- master
I'm looking for a way to deploy manually if the test phase failed.
but if I add when: manual to the deploy, then deploy never happens automatically.
so a flag like when: auto_or_manual_on_previous_fail will be great.
currently, there is no such flag in Gitlab ci.
Do you have any idea for a workaround or a way to implement it?
Another approach would be to skip the test in case of an emergency release.
For that, follow "Skipping Tests in GitLab CI" from Andi Scharfstein, and:
add "skip test" in the commit message triggering that emergency release
check a variable on the test stage
That is:
.test-template: &test-template
stage: tests
except:
variables:
- $CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i
- $SKIP_TESTS
As you can see above, we also included the variable $SKIP_TESTS in the except block of the template.
This is helpful when triggering pipelines manually from GitLab’s web interface.
Here’s an example:
It's possible to control the job attribute of your deploy job by leveraging parent-child pipelines (gitlab 12.7 and above). This will let you decide if you want the job in the child pipeline to run as manual, or always
Essentially, you will need to have a .gitlab-ci.yml with:
stages:
- build
- test
- child-deploy
child-deploy stage will be used to run the child pipeline, in which the deploy job will run with the desired attribute.
Your test could generate as artifact the code for deploy section. For example, in the after_script section of your test, you can check the value of CI_JOB_STATUS builtin variable to decide if you want to write the child job to run as manual or always:
my_test:
stage: test
script:
- echo "testing, exit 0 on success, exit 1 on failure"
after_script:
- if [ "$CI_JOB_STATUS" == "success" ]; then WHEN=always; else WHEN=manual; fi
- |
cat << 'EOF' > deploy.yml
stages:
- deploy
my_deploy:
stage: deploy
rules:
- when: $WHEN
script:
- echo "deploying"
EOF
artifacts:
when: always
paths:
- deploy.yml
Note that variable expension is disabled in the heredoc section, by the use of single quoted 'EOF'. If you need variable expension, remember to escape the $ of $WHEN.
Finally, you can trigger the child pipeline with deploy.yml
gen_deploy:
stage: child-deploy
when: always
trigger:
include:
- artifact: deploy.yml
job: my_test
strategy: depend