I'm trying to download the latest artifacts of a Cirrus CI build on a Github repo, and according to the docs, it is
https://api.cirrus-ci.com/v1/artifact/github/<USER OR ORGANIZATION>/<REPOSITORY>/<TASK NAME OR ALIAS>/<ARTIFACTS_NAME>/<PATH>.
Applying it to https://github.com/SDP-Rock-Paper-Scissors/RockPaperScissors, I get https://api.cirrus-ci.com/v1/artifact/github/SDP-Rock-Paper-Scissors/RockPaperScissors/check_android/jacoco_coverage/app/build/reports/jacoco/jacocoTestReport/html/index.html but that doesn't seem to work, am I doing something wrong ?
Add Alias to your task and download it by alias.
wheel_macos_arm_task:
only_if: $CIRRUS_BRANCH == 'master'
name: macosx • Apple Silicon
alias: wheel_macos_arm
macos_instance:
image: ghcr.io/cirruslabs/macos-monterey-xcode
env:
PATH: /opt/homebrew/opt/python#3.10/bin:$PATH
install_pre_requirements_script:
- brew install python#3.10
- ln -s python3 /opt/homebrew/opt/python#3.10/bin/python
install_cibuildwheel_script:
- python -m pip install cibuildwheel==2.11.4
run_cibuildwheel_script:
- cibuildwheel
wheels_artifacts:
path: "wheelhouse/*"
https://api.cirrus-ci.com/v1/artifact/github/GITHUB_USERNAME/PROJECT_NAME/wheel_macos_arm/wheels.zip
Related
I've released MooseX::Extended to the CPAN (github repository here).
I'm trying to set up github actions and the linux tests run just fine. However, (Windows is failing with this error:
Configuring true-v1.0.2 ... OK
==> Found dependencies: Function::Parameters
--> Working on Function::Parameters
Fetching http://www.cpan.org/authors/id/M/MA/MAUKE/Function-Parameters-2.001003.tar.gz ... OK
Configuring Function-Parameters-2.001003 ... OK
Building Function-Parameters-2.001003 ... OK
Successfully installed Function-Parameters-2.001003
! Installing true failed. See C:\Users\RUNNER~1\.cpanm\work\1653412748.5640\build.log for details. Retry with --force to force install it.
Building true-v1.0.2 ... FAIL
Of course, I can't see that C:\Users\RUNNER~1\.cpanm\work\1653412748.5640\build.log to understand what happened.
The true module passes its CPAN testers tests on Windows, so I don't know why it's failing in Github Actions.
My workflow looks like this:
# Hacked from https://github.com/skaji/perl-github-actions-sample/blob/master/.github/workflows/windows.yml
# See also: https://perlmaven.com/github-actions-running-on-3-operating-systems
name: windows
on:
push:
branches:
- '*'
tags-ignore:
- '*'
pull_request:
jobs:
perl:
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
perl-version:
- '5.20'
- '5.22'
- '5.24'
- '5.26'
- '5.28'
- '5.30'
- '5.32'
- '5.34'
- 'latest'
steps:
- uses: actions/checkout#v2
- name: Set up Perl
run: |
choco install strawberryperl
echo "C:\strawberry\c\bin;C:\strawberry\perl\site\bin;C:\strawberry\perl\bin" >> $GITHUB_PATH
- name: perl -V
run: perl -V
- name: Install Dependencies
run: curl -sL https://git.io/cpm | perl - install -g --show-build-log-on-failure Dist::Zilla
- name: Run Tests
run: |
dzil authordeps --missing | cpanm --notest
dzil listdeps --author --missing | cpanm --notest
dzil test --author --release
This is the PR to which the actions are attached.
I don't have access to a Windows box. Does anyone know what I missed?
Since GitHub Actions/Workflows uses a container for Windows that already has a version of Strawberry Perl pre-installed, it will not allow you to install any other version. You cannot remove the version of Perl that's pre-installed, and removing/installing a new one via Chocolatey is also next to impossible. If you re-install the version from Chocolatey that's already on the container, it seems to allow this, but it's basically a NOOP for you as a test setup.
The container also has MinGW installed; this can be bad for us as well. Having MinGW installed separately prevents XS modules from building (whether they be a dependency or if your own module is an XS module). Granted, this only happens if MinGW appears in the PATH ahead of your Perl install, but when you remove one Perl and add another, you're going to hit this problem.
To get around this, the best course of action is to remove the currently installed version of Perl from the PATH environment variable, along with their currently installed version of MinGW. Once both are safely out of the PATH, you can install a Portable[1] Strawberry Perl, put that Perl's paths in your PATH and begin testing with a fresh install of Strawberry Perl. GitHub recently broke our ability to do this directly in an Action YAML file.
That all sounds like a big headache, but it's really not. There's an Action available to us for this very purpose: actions-setup-perl. With this action you can easily test using any version of Perl you like. So, if you're hearing someone report a bug on Perl v5.26 on Windows, you can now add that to your matrix and test easily without the need for any back-and-forth from the user:
name: windows
on:
push:
branches:
- '*'
tags-ignore:
- '*'
pull_request:
jobs:
perl:
runs-on: windows-latest
strategy:
fail-fast: true
matrix:
perl-version:
- '5.30'
# - '5.28'
# - '5.26'
# - '5.24'
# - '5.22'
# - '5.20'
# - '5.18'
# - '5.16'
- '5.14'
steps:
- name: Setup perl
uses: shogo82148/actions-setup-perl#v1
with:
perl-version: ${{ matrix.perl-version }}
distribution: strawberry
- name: Set git to use LF
run: |
git config --global core.autocrlf false
git config --global core.eol lf
- uses: actions/checkout#v2
- name: perl -V
run: perl -V
- name: Ensure we have a working toolchain
run: cpanm ExtUtils::Manifest App::cpanminus
- name: Install Dependencies
run: cpanm -n --installdeps .
- name: Run Tests
run: cpanm --test-only -v .
[1] Portable versions of Strawberry Perl are zipped up, already compiled versions of Perl that do not require you to run an installer on Windows. This means that no heightened privileges are required, etc. You just unzip the archive in the directory you want to run Perl from, then add the relevant paths to Perl in your $env:PATH variable. It takes away any annoyances of build irregularities, etc. I've found it to be the most sane way to test on Windows.
I have created a custom brew formula for go binary. so when I update the main code repo, I need to manually update the homebrew-X formula to change the version or say update the shasum 256 of that.
can someone please help how can I update the shasum 256 and version auto?
You can automate it with goreleaser help - see brew
here is a configuration example of how it can be used with goreleaser (note it's referring to the private repository).
brews:
- name: app-cli
homepage: 'https://github.com/xendit/app-cli'
description: 'app-cli binary distribution using homebrew.'
folder: Formula
download_strategy: GitHubPrivateRepositoryReleaseDownloadStrategy
custom_require: "lib/private_strategy"
commit_author:
name: goreleaserbot
email: goreleaser#xendit.co
tap:
owner: username
name: app-cli
install: |
bin.install "app-cli"
Goreleaser itself can be automated with any ci system.
I have a wordpress plugin where i'm using composer to define my dependent libaries and github-actions to build the installable package. I plan to publish the vendors folder to a 'build' branch in github so the whole application can be installed.
My composer.json file has this content and works locally
{
"name" : "emeraldjava/bhaa_wordpress_plugin",
"description" : "bhaa_wordpress_plugin",
"type" : "wordpress-plugin",
"require": {
"scribu/scb-framework": "dev-master",
"scribu/lib-posts-to-posts": "dev-master",
"mustache/mustache": "2.12.0",
"league/csv": "^9.1",
"michelf/php-markdown": "^1.8"
},
and my github-actions build.yml file uses 'MilesChou/composer-action' to install the composer env in the docker container
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v1
- name: Composer install
uses: MilesChou/composer-action/7.3#master
with:
args: install --no-dev
- uses: docker://php:7.3-alpine
- uses: docker://alpine/git:latest
From the build log, I can see that the zip files for these composer artifacts have been downloaded to the cache
36/38: https://codeload.github.com/scribu/wp-scb-framework/legacy.zip/95b23ac342fce16bf5eb8d939ac5a361b94b104b
37/38: https://codeload.github.com/sebastianbergmann/phpunit/legacy.zip/a7834993ddbf4b0ed2c3b2dc1f3b1d093ef910a9
38/38: https://codeload.github.com/scribu/wp-lib-posts-to-posts/legacy.zip/a695438e455587fa228e993d05b4431cde99af1b
Finished: success: 38, skipped: 0, failure: 0, total: 38
The build then failed with this 'sh: git: not found' error
Package operations: 5 installs, 0 updates, 0 removals
- Installing scribu/scb-framework (dev-master 95b23ac): Cloning 95b23ac342
Failed to download scribu/scb-framework from source: Failed to clone https://github.com/scribu/wp-scb-framework.git, git was not found, check that it is installed and in your PATH env.
sh: git: not found
Now trying to download from dist
- Installing scribu/scb-framework (dev-master 95b23ac): Loading from cache
- Installing scribu/lib-posts-to-posts (dev-master a695438): Cloning a695438e45
Failed to download scribu/lib-posts-to-posts from source: Failed to clone https://github.com/scribu/wp-lib-posts-to-posts.git, git was not found, check that it is installed and in your PATH env.
sh: git: not found
Now trying to download from dist
- Installing scribu/lib-posts-to-posts (dev-master a695438): Loading from cache
- Installing mustache/mustache (v2.12.0): Loading from cache
- Installing michelf/php-markdown (1.8.0): Loading from cache
- Installing league/csv (9.4.1): Loading from cache
I'm assuming i need to ensure the docker container had git installed, but it seems odd that composer can access the legacy.zip file, so why it git needed at this stage?
EDIT 1
I guess the quick fix here is a duplicate of this issue, and as the answer below states.
For the sake of completeness, lets assume i can't call 'composer --prefer-dist' how could i ensure the docker container has git available to it?
By default Composer uses dist (zip files) for tagged releases and source (git clone) for branches. Since you're targeting master branch for your dependencies, Composer tries to clone repositories first. You can override this behavior by using --prefer-dist switch:
with:
args: install --prefer-dist --no-dev
--prefer-dist: Reverse of --prefer-source, Composer will install from dist if possible. This can speed up installs substantially on build
servers and other use cases where you typically do not run updates of
the vendors. It is also a way to circumvent problems with git if you
do not have a proper setup.
https://getcomposer.org/doc/03-cli.md#install-i
I'm currently using GitLab in combination with CI runners to run unit tests of my project, to speed up the process of bootstrapping the tests I'm using the built-in cache functionality, however this doesn't seem to work.
Each time someone commits to master, my runner does a git fetch and proceeds to remove all cached files, which means I have to stare at my screen for around 10 minutes to wait for a test to complete while the runner re-downloads all dependencies (NPM and PIP being the biggest time killers).
Output of the CI runner:
Fetching changes...
Removing bower_modules/jquery/ --+-- Shouldn't happen!
Removing bower_modules/tether/ |
Removing node_modules/ |
Removing vendor/ --'
HEAD is now at 7c513dd Update .gitlab-ci.yml
Currently my .gitlab-ci.yml
image: python:latest
services:
- redis:latest
- node:latest
cache:
key: "$CI_BUILD_REF_NAME"
untracked: true
paths:
- ~/.cache/pip/
- vendor/
- node_modules/
- bower_components/
before_script:
- python -V
# Still gets executed even though node is listed as a service??
- '(which nodejs && which npm) || (apt-get update -q && apt-get -o dir::cache::archives="vendor/apt/" install nodejs npm -yqq)'
- npm install -g bower gulp
# Following statements ignore cache!
- pip install -r requirements.txt
- npm install --only=dev
- bower install --allow-root
- gulp build
test:
variables:
DEBUG: "1"
script:
- python -m unittest myproject
I've tried reading the following articles for help however none of them seem to fix my problem:
http://docs.gitlab.com/ce/ci/yaml/README.html#cache
https://fleschenberg.net/gitlab-pip-cache/
https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/336
Turns out that I was doing some things wrong:
Your script can't cache files outside of your project scope, creating a virtual environment instead and caching that allows you to cache your pip modules.
Most important of all: Your test must succeed in order for it to cache the files.
After using the following config I got a -3 minute time difference:
Currently my configuration looks like follows and works for me.
# Official framework image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/python
image: python:latest
# Pick zero or more services to be used on all builds.
# Only needed when using a docker container to run your tests in.
# Check out: http://docs.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-service
services:
- mysql:latest
- redis:latest
cache:
untracked: true
key: "$CI_BUILD_REF_NAME"
paths:
- venv/
- node_modules/
- bower_components/
# This is a basic example for a gem or script which doesn't use
# services such as redis or postgres
before_script:
# Check python installation
- python -V
# Install NodeJS (Gulp & Bower)
# Default repository is outdated, this is the latest version
- 'curl -sL https://deb.nodesource.com/setup_8.x | bash -'
- apt-get install -y nodejs
- npm install -g bower gulp
# Install dependencie
- pip install -U pip setuptools
- pip install virtualenv
test:
# Indicate to the framework that it's being unit tested
variables:
DEBUG: "1"
# Test script
script:
# Set up virtual environment
- virtualenv venv -ppython3
- source venv/bin/activate
- pip install coverage
- pip install -r requirements.txt
# Install NodeJS & Bower + Compile JS
- npm install --only=dev
- bower install --allow-root
- gulp build
# Run all unit tests
- coverage run -m unittest project.tests
- coverage report -m project/**/*.py
Which resulted in the following output:
Fetching changes...
Removing .coverage --+-- Don't worry about this
Removing bower_components/ |
Removing node_modules/ |
Removing venv/ --`
HEAD is now at 24e7618 Fix for issue #16
From https://git.example.com/repo
85f2f9b..42ba753 master -> origin/master
Checking out 42ba7537 as master...
Skipping Git submodules setup
Checking cache for master... --+-- The files are back now :)
Successfully extracted cache --`
...
project/module/script.py 157 9 94% 182, 231-244
---------------------------------------------------------------------------
TOTAL 1084 328 70%
Creating cache master...
Created cache
Uploading artifacts...
venv/: found 9859 matching files
node_modules/: found 7070 matching files
bower_components/: found 982 matching files
Trying to load /builds/repo.tmp/CI_SERVER_TLS_CA_FILE ...
Dialing: tcp git.example.com:443 ...
Uploading artifacts to coordinator... ok id=127 responseStatus=201 Created token=XXXXXX
Job succeeded
For the coverage report, I used the following regular expression:
^TOTAL\s+(?:\d+\s+){2}(\d{1,3}%)$
Is it possible to separate the install deps and caching from the build of the source code?
I have:
sudo: required
language: cpp
matrix:
include:
- env: GCC_VERSION="4.9"
os: linux
dist: trusty
compiler: gcc
cache:
directories:
- /usr/local/include
- /usr/local/lib
- /usr/local/share
addons:
apt:
packages:
- gcc-4.9
- g++-4.9
sources:
- ubuntu-toolchain-r-test
# Install dependencies
install:
- export BUILD_DEPS="OFF"
- export BUILD_GRSF="ON"
- export CHECKOUT_PATH=`pwd`;
- chmod +x $CHECKOUT_PATH/travis/install_${TRAVIS_OS_NAME}.sh
- . $CHECKOUT_PATH/travis/install_${TRAVIS_OS_NAME}.sh
script:
- chmod +x $CHECKOUT_PATH/travis/build.sh
- . $CHECKOUT_PATH/travis/build.sh
notifications:
email: false
Because my build takes too long (more than 50 minutes with building dependencies and the source code) I proceed in the following way:
I set
BUILD_DEPS="ON" # build only deps
BUILD_GRSF="OFF"
which only builds the dependencies and caches them, afterwards I set
BUILD_DEPS="OFF"
BUILD_GRSF="ON" # build only source
in the .travis.yaml file which then builds only the source code.
This seems to work but is cumbersome? Is there a better solution to this? Maybe directly on travis modifying the .travis.yaml and make a new commit "travis cached, build source now". which will then trigger another travis build (which now builds the source)
Your dependency install script could look for a marker file your script leaves after successful installation in a cached dir and only if that's not found you would re-run the build.
That way you don't need any modifications to the travis spec at least.
It seems travis can only cache in $HOME:
https://github.com/travis-ci/travis-ci/issues/6115#issuecomment-222817367