I started using github actions and I am trying to let a few lines of shell script running before I create a python package. But I can't get my code to run. It always stops on the line run: | - which is odd because I have been using this exact line before before (see further down). Does anyone know what I am doing wrong?
name: Python packaging
on: [push]
jobs:
job1:
name: Update version
runs-on: ubuntu-latest
steps:
- name: checkout repo
uses: actions/checkout#v3
# ----- This is where the error occurs. Error message:
# Check failure on line 11 in .github/workflows/main.yml
# GitHub Actions / .github/workflows/main.yml
# Invalid workflow file
# You have an error in your yaml syntax on line 11
- name: Increase version
run: |
old_version=$(grep -oP '(?<=0.0.)[0-99]+' contrib/_version.py)
new_version=$(($old_version + 1))
str="__version__=\"0.0."
out="$str$new_version\""
sed -i '1s/.*/'$out'/' contrib/_version.py
job2:
name: Build and upload
needs: job1
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9"]
steps:
- name: checkout repo
uses: actions/checkout#v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python#v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install setuptools wheel
python -m pip install --upgrade build
python -m pip install --upgrade twine
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Create package
run: |
python3 -m build
- name: Upload to Test PyPI
env:
TWINE_USERNAME: "__token__"
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
TWINE_REPOSITORY_URL: "https://test.pypi.org/legacy/"
run: |
twine check dist/*
twine upload --verbose --skip-existing dist/*
The file is in YAML syntax which highly depends on indentation.
Based on your example:
job1: and job2: must have the same indentation.
Under steps:, name: and uses: must have the same indentation.
...
name: Python packaging
on: [push]
jobs:
job1:
name: Update version
runs-on: ubuntu-latest
steps:
- name: checkout repo
uses: actions/checkout#v3
- name: Increase version
run: |
old_version=$(grep -oP '(?<=0.0.)[0-99]+' contrib/_version.py)
new_version=$(($old_version + 1))
str="__version__=\"0.0."
out="$str$new_version\""
sed -i '1s/.*/'$out'/' contrib/_version.py
job2:
name: Build and upload
# ...
Related
I have added Machine Learning Model and preceding with CI pipeline in Github, however while executing workflow, I am getting error for Missing required root key on. Below is my yml file code.
ci_pipeline:
on:
push:
branches:
- main
steps:
- uses: actions/checkout#v1
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python#v1
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Format
run: |
black app.py
- name: Lint
run: |
pylint --disable=R,C app.py
- name: Test
run: |
python -m pytest -vv test.py
I'm trying to run a local server for my project CI/CD Pipeline. When I start the server I got a "Connection refused" on it.
My project is a fastAPI application and I'm trying to run a integration tests on PR to validate the app before merge the code. I tried to start my app directly (gunicorn), building a docker image and starting it... I tried a lot of things. Then, I tried to run a simple server instead of my app and... got the same error!
This is my simple server workflow:
on:
push:
branches:
- "develop"
jobs:
mylocalserver:
name: Run server
runs-on: ubuntu-latest
steps:
- name: Setup python
uses: actions/setup-python#v3
with:
python-version: 3.9
- name: Run server in background
run: |
python -V
python -m http.server 3000 &> /dev/null &
sudo lsof -i -P -n | grep LISTEN
curl http://localhost:3000
- name: Run server with console
run: |
python -m http.server 3000
Output:
If I run my app with console (no daemon mode in gunicorn), the server start and log to console in workflow with success:
But this way I cannot run nothing after this (and I have to cancel workflow). Some idea? Thank you!
Maybe not the best answer, but for now runnig the job into a container works (only add a container label in question example). Example for my fastAPI app:
on:
pull_request:
branches:
- 'main'
- 'develop'
jobs:
run-on-pr:
runs-on: ubuntu-latest
container: ubuntu
services:
mongodb:
image: mongo
ports:
- 27017:27017
steps:
- name: Setup git
run: |
apt-get update; apt-get install -y git
- name: Git checkout
uses: actions/checkout#v3
with:
path: api
- name: Setup python
uses: actions/setup-python#v4
with:
python-version: 3.9
- name: Install pip
run: |
apt-get update; apt-get install -y python3-pip
- name: Build and ENV
run: |
cd api
cp .env_example_docker .env
pip3 install -r requirements.txt
- name: Run fastAPI
run: |
cd api
gunicorn -D -k uvicorn.workers.UvicornWorker -c ./gunicorn_conf.py app.main:app
env:
MONGO_URL: "mongodb://mongodb:27017"
- name: Install curl
run: |
apt-get update; apt-get install -y curl
- name: Run curl
run: |
curl http://localhost:3000
This works, but I have to install all in container (git, pip). I will try a solution without using the container label and if I found anything I can post here.
I'm using https://github.com/dawidd6/action-ansible-playbook in my pipeline and I need to somehow install the dependency before even running Ansible on the Github Runner. And that does not seem to work for some reason. Any idea what this could be?
deploy:
name: Deploy
runs-on: ubuntu-latest
needs: build-docker
steps:
- name: Check out the codebase.
uses: actions/checkout#v2
- name: Uses Python 3.11
uses: actions/setup-python#v3
with:
python-version: '3.11.0-alpha.1'
- name: Install Jmespath Dependency
run: |
pip3 install jmespath
pip3 freeze # just to see what is installed
sudo apt install -y python3-jmespath
- name: Run playbook
uses: dawidd6/action-ansible-playbook#v2
with:
playbook: provision_vps.yml
directory: ./ansible
key: ${{secrets.ANSIBLE_PRIVATE_KEY}}
vault_password: ${{secrets.ANSIBLE_VAULT_PASS}}
options: |
--inventory hosts.inventory
--verbose
Note that the indentation above is not correct here, it is fine in my editor. Everything else works fine, it's just that dependency that will not be installed on my GitHub Runner so the Ansible controller can run using this task:
- name: Get all file_sd files from scrape_configs
set_fact:
file_sd_files: "{{ prometheus_scrape_configs | json_query('[*][].file_sd_configs[*][].files[]') }}"
The error message I get is:
fatal: [cache.hugin.chat]: FAILED! => {"msg": "You need to install \"jmespath\" prior to running json_query filter"}
I'm trying to change the path of a executable depending on the OS. This is what I have tried:
name: PyInstaller
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
workflow_dispatch:
jobs:
PyInstaller:
strategy:
matrix:
os: [ macos-latest, windows-latest ]
include:
- os: macos-latest
release_suffix: mac
- os: windows-latest
release_suffix: windows
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.10
uses: actions/setup-python#v2
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Run PyInstaller
run: |
pyinstaller "Qr Rechnung.spec"
- name: Upload artifact
uses: actions/upload-artifact#v3
with:
name: "Qr Rechnung"
if ${{ matrix.os }} == "macos-latest"
path: "dist/Qr Rechnung.app"
elif ${{ matrix.os }} == "windows-latest"
path: "dist/Qr Rechnung.exe"
But the last part does not work. How can I change the path key depending on the OS? The if does not work as I would expect, it says path is redefined
I was trying to understand the CI/CD flow from this tutorial by Indian Pythonista and after following the tutorial and setting the Heroku app name and token in the Github Secrets I ran the yaml file below.
name: Python application
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- run: |
git fetch --prune --unshallow
- name: Set up Python 3.8
uses: actions/setup-python#v1
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pip install pytest
export PYTHONPATH=src
pytest
- name: Deploy to Heroku
env:
HEROKU_API_TOKEN: ${{ secrets.HEROKU_API_TOKEN }}
HEROKU_APP_NAME: ${{ secrets.HEROKU_APP_LINKNAME }}
if: github.ref == 'refs/heads/master' && job.status == 'success'
run: |
git remote add heroku https://heroku:$HEROKU_API_TOKEN#git.heroku.com/$HEROKU_APP_NAME.git
git push heroku HEAD:master -f
The build looked like the screenshot below:
I do not exactly know what should I call this error as well, since I am new to the topic. Due to this, the flask app doesn't get updated on the Heroku app link as well.