Using an array of values to repeat a step in GitHub Actions workflow - yaml

I am trying to create a GitHub Actions workflow which would collect specific paths changed in last commit and run a step for each of collected paths, if any.
Currently, in my workflow I'm creating an array of paths, but I'm not sure how to proceed with my array:
name: Test
on:
push
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
# This step will create "an array" of strings, e.g. "path1 path2 path3"
- name: array
id: arr
run: |
arr=()
for i in "$(git diff-tree --no-commit-id --name-only -r ${{ github.sha }})"
do
if [[ $i == *"path1"* ]]; then
arr+=("path1")
fi
if [[ $i == *"path2"* ]]; then
arr+=("path2")
fi
done
echo ::set-output name=arr::${arr[#]}
# How to run this step by iterating the `${{ steps.arr.outputs.arr }}`?
- name: reviewdog-lint
uses: reviewdog/action-eslint#v1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
reporter: github-pr-review
eslint_flags: 'my_project/some_folder/${{ SINGLE_PATH }}/' # `SINGLE_PATH` would be a path from the array
Is something like this even possible in the first place? If not, what would be recommended way to loop through some values and use them as arguments in other workflow steps?

There is some support for this in Github Actions. There is a very good tutorial here that explains how to do it in detail, but essentially what you'll do is split the steps into two jobs. The first job will output a JSON object that will serve as the input to the matrix of the second job.
Here's a simple example:
jobs:
setup:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.matrix.outputs.value }}
steps:
- id: matrix
run: |
echo '::set-output name=value::[\"a\", \"b\", \"c\"]'
build:
needs: [ setup ]
runs-on: ubuntu-latest
strategy:
matrix:
value: ${{fromJson(needs.setup.outputs.matrix)}}
steps:
- run: |
echo "${{ matrix.value }}"

Difficult to say without running it, but I would say you need to use the output in the second step by assigning it to a variable, something like:
env:
OUTPUT: ${{ steps.id.outputs.arr }}
Then you use $OUTPUT as an environment variable inside the action.
The problem with that particular action is that it takes one commit at a time. But you can check out the code, it's a shell script. You can fork it from line 15 and make it split input and run a loop over it, applying eslint to every one of them.

Related

Github Workflows: Changing or specifying environment variables in matrix jobs

How do I specify or change an environment variable as part of a job matrix?
For example, here I want my job to echo "Two" and "Three" instead of "One", but github completely ignores the definition or change of the environment variable in the matrix:
name: test-test-test
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main, develop ]
env:
MY_VAR: One
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- env:
MY_VAR: Two
- env:
MY_VAR: Three
steps:
- run: echo "$MY_VAR2"
yields echo "One" and echo "One" in both jobs.
Use this context to access information about environment variables that have been set in a workflow, step, or job. Note that you cannot use the env context in the value of the id and uses keys within a step. An example is env.env_var.
GitHub provides extensive documentation about contexts at
https://docs.github.com/en/actions/learn-github-actions/contexts.
The syntax you used won't work for matrix. As stated in the official documentation:
The variables that you define become properties in the matrix context, and you can reference the property in other areas of your workflow file.
In your case, to access the matrix env variable you set in the include field list, you would need to use ${{ matrix.env.MY_VAR }}.
Something like this:
env:
MY_VAR: One
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- env:
MY_VAR: Two
- env:
MY_VAR: Three
steps:
- run: |
echo "$MY_VAR" # is equal to ${{ env.MY_VAR }}
echo ${{ matrix.env.MY_VAR }}
This will generate 2 test jobs: test (Two) and test (Three).
I made a test here if you want to have a look.
Conclusion:
Basically, the env you set at the workflow level isn't the same env set in the matrix strategy, so you can't substitute the value there.
Alternative
What you could do eventually (as #Adam stated in the other answer) is to set the env value at the workflow level for it to work, in the environment context. (Reference in the documentation + another workflow example).
Example:
env:
WORKFLOW_VARIABLE: WORKFLOW
jobs:
test:
runs-on: ubuntu-latest
env:
JOB_VARIABLE: JOB
steps:
- name: Example
env:
STEP_VARIABLE: STEP
run: |
echo "Hello World"
echo "This is the $WORKFLOW_VARIABLE environment variable"
echo "This is the $JOB_VARIABLE environment variable"
echo "This is the $STEP_VARIABLE environment variable"
But depending on what you plan to achieve, a matrix strategy you be more efficient.

Cannot list changed files using github actions

I am trying to list all the changed files on file push. However my List all changed files command does not return anything even though I change the file and commit
name: test on: push:
branches: [ main ]
jobs: build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
with:
fetch-depth: 0 # OR "2" -> To retrieve the preceding commit.
- name: Get changed files using defaults
id: changed-files
uses: actions/checkout#v2
- name: List all added files
run: |
for file in ${{ steps.changed-files.outputs.added_files }}; do
echo "$file was added"
done
- name: List all changed files
run: |
for file in ${{ steps.changed-files.outputs.modified_files }}; do
echo "$file was modified"
done
Try using the one available in marketplace https://github.com/jitterbit/get-changed-files#get-all-changed-files

how can I make a job in github action run randomly between 1 and 50 times a day?

how can I make a job in github action run randomly between 1 and 50 times a day?
Here is my cron job.
cron: '0 0 * * *'
This will run once a day.
But what I want is to run randomly 1-50 times a day.
How do I make it work randomly from 1 to 50?
below is my git action's yml setting file as workflows
#1. Repository Fork
# 2. Modify the files A and B according to the procedure
# 3. After committing the modifications, push & Enjoy!
name: planting-grass
# A. Comment lines 8-11
# on:
# push:
# branches:
# - unknown
# B. Uncomment lines 14-16
on:
schedule:
- cron: '0 0 * * *'
jobs:
task:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set current date
id: date
run: echo "::set-output name=date::$(date +'%Y-%m-%d')"
- name: Execute commands
run: bash ./task.sh ${{ steps.date.outputs.date }}
- name: Commit files
run: |
git config --global user.name "$(git --no-pager log --format=format:'%an' -n 1)"
git config --global user.email "$(git --no-pager log --format=format:'%ae' -n 1)"
git add date.txt
git commit -m ${{ steps.date.outputs.date }}
- name: Push changes
uses: ad-m/github-push-action#master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ github.ref }}
Cron jobs and random times, within given hours The method through this post does not work.
Best Regards!
You can do this by modifying your Bash script to loop a random number of times. For example, this Bash script loops between 1 and 50 times.
#!/usr/bin/env bash
loops=$(( ( RANDOM % 50 ) + 1 ))
echo "$loops"
for i in $(seq 1 $loops); do
echo foo
done
The one downside of this approach is that you need to take the steps being done in the 'Commit files' step and merge them into the script in the 'Execute commands' step, or they won't be repeated.

How can I run PNPM workspace projects as parallel jobs on GitHub Actions?

Given a repository structure with two packages like this:
$ tree
.
└── packages
├── foo
└── bar
$ cat pnpm-workspace.yaml
packages:
- 'packages/**'
$ pnpm -s m ls --depth -1
monorepo /monorepo
#mono/foo#0.0.0 /monorepo/packages/foo
#mono/baz#0.0.0 /monorepo/packages/bar
I'd like to run GitHub Actions CI such that it automatically runs each project as separate job. Here I've set up a job that manually does that parallelization:
name: CI
on:
push:
jobs:
build:
strategy:
matrix:
package: ["#mono/foo", "#mono/bar" ]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: pnpm/action-setup#v2
with:
version: 6.9.1
- run: pnpm run --filter ${{ matrix.package }} test
That runs fast because each project is run as parallel jobs, but I don't want to manually maintain that matrix.package list. How can I utilize pnpm to provide a list of workspace projects that gets fed into GitHub Actions CI?
Hmmm... I hit my head on this a bit more and I've found a solution.
I first made a package.json script to turn pnpm output into a json array-fragment:
$ cat package.json
{
"scripts": {
"list-packages": "echo [$(pnpm -s m ls --depth -1 | tr \" \" \"\n\" | grep -o \"#.*#\" | rev | cut -c 2- | rev | sed -e 's/\\(.*\\)/\"\\1\"/' | paste -sd, - )]",
}
}
$ pnpm -s list-packages
["#mono/foo","#mono/bar"]
(I'm not good enough with shell to know if there's a much easier way to express this transformation so I'd be happy to learn!)
I then followed GitHub documentation on dynamically setting matrix variables and created this workflow:
name: CI
on:
push:
workflow_dispatch:
jobs:
packages:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- uses: actions/checkout#v2
- uses: pnpm/action-setup#v2
with:
version: 6.9.1
- run: $(echo pnpm -s list-packages2)
- id: set-matrix
run: echo "::set-output name=matrix::{\"packages\":$(pnpm -s list-packages)}"
build:
needs: packages
strategy:
matrix: ${{ fromJson(needs.packages.outputs.matrix) }}
runs-on: ubuntu-latest
steps:
- run: echo ${{ matrix.package }}
The packages job now takes the output of $(pnpm -s list-packages) and puts it into the matrix variable, and that makes GitHub Actions run them all in parallel:

GitHub Actions pass list of variables to shell script

Using GitHub Actions, I would like to invoke a shell script with a list of directories.
(Essentially equivalent to passing an Ansible vars list to the shell script)
I don't really know how, is this even possible? Here's what I have until now, how could one improve this?
name: CI
on:
push:
branches:
- master
tags:
- v*
pull_request:
jobs:
run-script:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout#v2
- name: Run script on targets
run: ./.github/workflows/script.sh {{ targets }}
env:
targets:
- FolderA/SubfolderA/
- FolderB/SubfolderB/
Today I was able to do this with the following YAML (truncated):
...
with:
targets: |
FolderA/SubfolderA
FolderB/SubfolderB
The actual GitHub Action passes this as an argument like the following:
runs:
using: docker
image: Dockerfile
args:
- "${{ inputs.targets }}"
What this does is simply sends the parameters as a string with the newline characters embedded, which can then be iterated over similar to an array in a POSIX-compliant manner via the following shell code:
#!/bin/sh -l
targets="${1}"
for target in $targets
do
echo "Proof that this code works: $target"
done
Which should be capable of accomplishing your desired task, if I understand the question correctly. You can always run something like sh ./script.sh $target in the loop if your use case requires it.

Resources