Github Action Cache question with containers - caching

I got a quick question and I can't find docs on it (maybe it is impossible)
How can I retrieve what has been built from a container (like this) :
prod-dependencies:
name: Production Dependencies
runs-on: ubuntu-20.04
container: elixir:1.11-alpine
env:
MIX_ENV: prod
steps:
- name: Checkout
uses: actions/checkout#v2
with:
fetch-depth: 0
- name: Retrieve Cached Dependencies
uses: actions/cache#v2
id: mix-prod-cache
with:
path: |
${{ github.workspace }}/deps
${{ github.workspace }}/_build
key: PROD-${{ hashFiles('mix.lock') }}
- name: Install Dependencies
if: steps.mix-prod-cache.outputs.cache-hit != 'true'
run: |
apk add build-base rust cargo
mix do local.hex --force, local.rebar --force, deps.get --only prod, deps.compile
I need to build on Alpine since I am deploying on Alpine and some C libs are different for the Erlang VM to work.
So here it is the place where I build dependencies and I want to put them on cache for the subsequent jobs but the cache is never populated. According to doc GitHub mount a volume with containers to retrieve artifacts but I must miss use it.
Thanks a lot for your help.

Here is a sample Github workflow for an Elixir app with a Postgres database that utilizes the action/cache#v2. Note that it defines 2 separate steps for restoring the deps and _build directories. In this workflow, the mix deps.get and mix compile steps are always run: they should execute very quickly if the cache is restored.
Adapted from this workflow
name: Test
on:
pull_request:
branches:
- develop
paths-ignore:
- 'docs/**'
- '*.md'
jobs:
test:
name: Lint and test
runs-on: ubuntu-18.04
env:
MIX_ENV: test
services:
db:
image: postgres:11
ports: ['5432:5432']
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout#v2
- uses: erlef/setup-elixir#v1
with:
otp-version: '23.1.2'
elixir-version: '1.11.2'
- name: Configure SSH for private Repos
uses: webfactory/ssh-agent#v0.4.1
with:
ssh-private-key: ${{ secrets.HEADLESS_PRIV }}
- name: Restore the deps cache
uses: actions/cache#v2
id: deps-cache-restore
with:
path: deps
key: ${{ runner.os }}-deps-mixlockhash-${{ hashFiles(format('{0}{1}', github.workspace, '/mix.lock')) }}
restore-keys: |
${{ runner.os }}-deps-
- name: Restore the _build cache
uses: actions/cache#v2
id: build-cache-restore
with:
path: _build
key: ${{ runner.os }}-build-mixlockhash-${{ hashFiles(format('{0}{1}', github.workspace, '/mix.lock')) }}
restore-keys: |
${{ runner.os }}-build-
- name: Get/update Mix dependencies
run: |
mix local.hex --force
mix local.rebar
mix deps.get
- name: Compile
run: mix compile
- name: Create database
run: mix ecto.create
- name: Migrate database
run: mix ecto.migrate
- name: Test
run: mix test

Related

Github action yml keeps failing: Invalid workflow file: .github/workflows/AzureARMDeploy.yml#L13

I have crossed check the syntax below and everything seem to be in order but it keeps failing, could someone please look through this syntax?
on:
# Triggers the workflow on push or pull request events but only for the "main" branch
push:
branches: master
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checout code
uses: actions/checkout#v2
- name: Set up .NET Core
uses: actions/setup-dotnet#v1
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
- name: Set up dependency caching for faster builds
uses: actions/cache#v2
with:
path: ~/.nuget/packages
key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }}
restore-keys: |
${{ runner.os }}-nuget-
- name: Build with dotnet
run: dotnet build --configuration Release
- name: Test
run: dotnet test --no-restore --verbosity normal
- name: dotnet publish
run: dotnet publish -c Release -o ${{env.DOTNET_ROOT}}/myapp
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v3
with:
name: .net-app
path: ${{env.DOTNET_ROOT}}/myapp
- name: Login to Aure
uses: azure/login#v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v3
with:
name: .net-app
- name: Deploy to Azure
uses: azure/CLI#v1
with:
inlineScript: |
...
The deploy job should be an item of the jobs table, you have wrong indentation
try
on:
# Triggers the workflow on push or pull request events but only for the "main" branch
push:
branches: master
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checout code
uses: actions/checkout#v2
- name: Set up .NET Core
uses: actions/setup-dotnet#v1
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
- name: Set up dependency caching for faster builds
uses: actions/cache#v2
with:
path: ~/.nuget/packages
key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }}
restore-keys: |
${{ runner.os }}-nuget-
- name: Build with dotnet
run: dotnet build --configuration Release
- name: Test
run: dotnet test --no-restore --verbosity normal
- name: dotnet publish
run: dotnet publish -c Release -o ${{env.DOTNET_ROOT}}/myapp
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v3
with:
name: .net-app
path: ${{env.DOTNET_ROOT}}/myapp
- name: Login to Aure
uses: azure/login#v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v3
with:
name: .net-app
- name: Deploy to Azure
uses: azure/CLI#v1
with:
inlineScript: |
...

How to workaround "The command line is too long" with GraalVM and Windows?

I'm trying to make a JHipster 7 app build with GraalVM on Windows. I've successfully got it to work with macOS and Linux. With Windows, I get an error about the command line being too long.
[INFO] Executing: C:\Users\runneradmin\.graalvm\graalvm-ce-java17-22.0.0.2\bin\native-image.cmd -cp ...;D:\a\auth0-full-stack-java-example\auth0-full-stack-java-example\target\flickr-2-0.0.1-SNAPSHOT.jar --no-fallback --verbose -J-Xmx10g -H:Name=native-executable
The command line is too long.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
I've copied and pasted the command that GitHub Actions runs and it's just over 12,000 characters long. According to this document, Windows only supports 8191 characters.
I tried shortening up the classpath using this suggestion, but it doesn't help. I even tried C:\r instead of C:\repo.
People on Twitter suggested using PowerShell and echoing the commands into a file, and then running that. However, I haven't been able to find any examples of how to extract the native-image.cmd command and its arguments from Maven.
You can find the GitHub action I'm using to configure things here. I've also pasted it below for convenience.
name: Publish
on:
release:
types: [published]
env:
graalvm_version: '22.0.0.2'
java_version: '17'
branch: 'spring-native'
jobs:
build:
name: GraalVM - ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 90
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
steps:
- uses: actions/checkout#v2
with:
ref: '${{ env.branch }}'
- name: Set up GraalVM (Java ${{ env.java_version }})
uses: graalvm/setup-graalvm#v1
with:
version: '${{ env.graalvm_version }}'
java-version: '${{ env.java_version }}'
components: 'native-image'
- name: Cache Maven dependencies
uses: actions/cache#v2
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-maven
- name: Cache npm dependencies
uses: actions/cache#v2
with:
path: |
~/.npm
~/.cache/Cypress/
key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
- name: Set up swap space
if: runner.os == 'Linux'
uses: pierotofy/set-swap-space#v1.0
with:
swap-size-gb: 10
- name: Build native images
run: ./mvnw -B -ntp package -Pnative,prod -DskipTests
- name: Archive binary
uses: actions/upload-artifact#v2
with:
name: flickr2-${{ matrix.os }}-x86_64
path: target/native-executable
- name: Get release version
run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
- name: Rename binary
run: mv target/native-executable target/flickr2-${{ runner.os }}-${{ env.RELEASE_VERSION }}-x86_64
- name: Upload release
uses: alexellis/upload-assets#0.3.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
asset_paths: '["target/flickr2-${{ runner.os }}*"]'
build-windows:
name: GraalVM - ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
os: [windows-latest]
steps:
- uses: actions/checkout#v2
with:
ref: '${{ env.branch }}'
- uses: ilammy/msvc-dev-cmd#v1
- uses: microsoft/setup-msbuild#v1
- name: Set up GraalVM (Java ${{ env.java_version }})
uses: graalvm/setup-graalvm#v1
with:
version: '${{ env.graalvm_version }}'
java-version: '${{ env.java_version }}'
components: 'native-image'
- name: Cache Maven dependencies
uses: actions/cache#v2
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-maven
- name: Cache npm dependencies
uses: actions/cache#v2
with:
path: |
~/.npm
~/.cache/Cypress/
key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
- name: Configure pagefile
uses: al-cheb/configure-pagefile-action#v1.2
- name: Set up pagefile
run: |
(Get-CimInstance Win32_PageFileUsage).AllocatedBaseSize
- name: mvnw --version
run: mvnw --version
shell: cmd
- name: Maven resolve
run: mvnw -B -ntp dependency:resolve-plugins
shell: cmd
- name: Build native images
run: |
mklink /J C:\r C:\Users\runneradmin\.m2\repository
mvnw -B -ntp package -Pnative,prod -DskipTests -Dmaven.repo.local=C:\r
shell: cmd
- name: Archive binary
uses: actions/upload-artifact#v2
with:
name: flickr-${{ matrix.os }}-x86_64.exe
path: target/native-executable.exe
- name: Get release version
run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
- name: Rename binary
run: move target/native-executable.exe target/flickr2-${{ runner.os }}-${{ env.RELEASE_VERSION }}-x86_64.exe
- name: Upload release
uses: alexellis/upload-assets#0.3.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
asset_paths: '["target/flickr2-${{ runner.os }}*"]'
Upgrading to native-build-tools v0.9.10 fixes this issue on Windows. More details at https://github.com/graalvm/native-build-tools/issues/214 and https://github.com/graalvm/setup-graalvm/issues/6#issuecomment-1054582083.

"The cypress npm package is installed, but the Cypress binary is missing." in GitHub Actions

I'm receiving the following error when running a Cypress e2e test runner on GitHub Actions:
The cypress npm package is installed, but the Cypress binary is missing.
We expected the binary to be installed here: /home/runner/.cache/Cypress/8.5.0/Cypress/Cypress
Reasons it may be missing:
- You're caching 'node_modules' but are not caching this path: /home/runner/.cache/Cypress
- You ran 'npm install' at an earlier build step but did not persist: /home/runner/.cache/Cypress
My .github/workflow/tests.yml is set up as follows:
name: celestia/tests
on:
pull_request:
branches:
- main
- master
jobs:
unit:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
node: [14]
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout ๐Ÿ›Ž
uses: actions/checkout#master
# Setup .npmrc file to publish to GitHub Packages
- name: Setup node env ๐Ÿ— and .npmrc file to publish to GitHub Packages
uses: actions/setup-node#v2.1.2
with:
node-version: ${{ matrix.node }}
registry-url: 'https://npm.pkg.github.com'
# Defaults to the user or organization that owns the workflow file:
scope: '#observerly'
- name: Cache node_modules ๐Ÿ“ฆ
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
- name: Install project dependencies ๐Ÿ‘จ๐Ÿปโ€๐Ÿ’ป
run: npm ci --ignore-scripts
- name: Run jest unit tests ๐Ÿงช
run: npm run test:unit
e2e:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
node: [14]
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout ๐Ÿ›Ž
uses: actions/checkout#master
# Setup .npmrc file to publish to GitHub Packages
- name: Setup node env ๐Ÿ— and .npmrc file to publish to GitHub Packages
uses: actions/setup-node#v2.1.2
with:
node-version: ${{ matrix.node }}
registry-url: 'https://npm.pkg.github.com'
# Defaults to the user or organization that owns the workflow file:
scope: '#observerly'
- name: Install project dependencies ๐Ÿ‘จ๐Ÿปโ€๐Ÿ’ป
run: npm ci --ignore-scripts
- name: Run e2e cypress tests ๐Ÿงช
run: npm run test:e2e:headless
How can I side-step this particular issue?

Set up github actions to run cypress tests on localhost from 2 different repositories

I have my front-end (repUI) and cypress tests (repTest) in separate repositories., I am trying to run the tests by checkout both from repTest. Once the repUI is set up (install and build), the localhost:8000 is ready for tests to run.
So far the following workflow is designed:
name: Cypress E2E Tests on Dev
on:
workflow_dispatch:
push:
branches-ignore: [ main ]
schedule:
- cron: '0 0 * * 1-5'
jobs:
setup:
runs-on: ubuntu-20.04
strategy:
matrix:
node-version: [14.x]
steps:
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
# Checkout UI repo
- name: Checkout voy-frontend repo
uses: actions/checkout#v2
with:
repository: orgName/xyz-frontend
path: xyz-frontend
ssh-key: ${{ secrets.GIT_SSH_KEY }}
# Install voy-f dependancies
- name: Install dependancies for application
run: |
cd voy-frontend
pwd
npm install
- name: Get npm cache directory
id: npm-cache-dir
run: |
echo "::set-output name=dir::$(npm config get cache)"
- uses: actions/cache#v2
id: npm-cache # use this to check for `cache-hit` ==> if: steps.npm-cache.outputs.cache-hit != 'true'
with:
path: ${{ steps.npm-cache-dir.outputs.dir }}
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
# Run application
- name: Run application
run: |
pwd
cd voy-frontend
npm run serve
cypress-test:
name: Test
needs: setup
runs-on: ubuntu-latest
steps:
# Checkout test automation repo
- name: git checkout
uses: actions/checkout#v2
with:
ref: main
# Install cypress dependancies
- name: Cypress install
uses: cypress-io/github-action#v2
# Run cypress tests
- name: Cypress run - localhost
run: npm run cy:r -- --env TENANT=${{ secrets.TENANT }},CLIENT_ID=${{ secrets.CLIENT_ID }},CLIENT_SECRET=${{ secrets.CLIENT_SECRET }},ORG_SCOPE=${{ secrets.ORG_SCOPE }},G_SCOPE=${{ secrets.G_SCOPE }} --spec "cypress/integration/specs/Search.feature" --headless --browser chrome
- uses: actions/upload-artifact#v2
if: failure()
with:
name: cypress-videos
path: |
cypress/reports
cypress/videos
cypress/screenshots
retention-days: 1
The problem is if I run the build command npm run serve for repUI, the applications successfully getting started but it is not proceeding further instead it just waits. If I comment out the npm run serve, all other jobs are running as it should.
Here is the screenshot:
For building, it only took ~60 seconds, and then it waits until I cancel it. Same case if I trigger the job again and always.
What needs to be done in order to proceed to the cypress-test job once the build is ready.?

Github action not caching node_modules across jobs

I have a Github action on pull_request, and am attempting to cache node_modules between jobs. I can only see cache working on a job re-run, but not when opening new PRs, even when there have been no changes to my node_modules.
Is this by design?
Each time it runs in a new PR, I see this output in the job:
Run actions/cache#v2
with:
path: ~/.npm
key: Linux-build-cache-node-modules-83fc6365b85dd061416fccd5993d48c2003c388bb2184fd57f28d1041d9d261e
restore-keys: Linux-build-cache-node-modules-
Linux-build-
Linux-
env:
cache-name: cache-node-modules
Cache not found for input keys: Linux-build-cache-node-modules-83fc6365b85dd061416fccd5993d48c2003c388bb2184fd57f28d1041d9d261e, Linux-build-cache-node-modules-, Linux-build-, Linux-
Yet, re-running the job will enjoy the cache hit:
Run actions/cache#v2
Received 54525952 of 109978126 (49.6%), 51.8 MBs/sec
Received 109978126 of 109978126 (100.0%), 67.8 MBs/sec
Cache Size: ~105 MB (109978126 B)
/usr/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/0bdf436d-9d4d-4397-b8ed-5910ff503949/cache.tzst -P -C /home/runner/work/argutopia/argutopia
Cache restored successfully
Cache restored from key: Linux-build-cache-node-modules-83fc6365b85dd061416fccd5993d48c2003c388bb2184fd57f28d1041d9d261e
Here is my action yml for reference:
name: PR CI
on:
pull_request:
branches: [main, develop]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x]
steps:
- uses: actions/checkout#v2
with:
fetch-depth: 0
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: Cache node modules
uses: actions/cache#v2
id: cache
env:
cache-name: cache-node-modules
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Install Dependencies
if: steps.cache.outputs.cache-hit != 'true'
run: |
echo '::debug::deps cache miss - installing deps'
npm install
Caching ~/.npm wont allow you to skip running npm install. It will save the global npm cache on this machine so that running npm install doesn't incur as much download time, but you'll then still need to build the local node_modules for your project.
If you want to be able to skip your install, you can cache the project node_modules folder as well and skip install on cache hits there, though this could present some issues down the road.
Use advanced alternative caching technique from this article
- name: Cache dependencies
id: cache
uses: actions/cache#v3
with:
path: ./node_modules
key: modules-${{ hashFiles('package-lock.json') }}
- name: Install dependencies
if: steps.cache.outputs.cache-hit != 'true'
run: npm ci --ignore-scripts

Resources