I have added Machine Learning Model and preceding with CI pipeline in Github, however while executing workflow, I am getting error for Missing required root key on. Below is my yml file code.
ci_pipeline:
on:
push:
branches:
- main
steps:
- uses: actions/checkout#v1
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python#v1
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Format
run: |
black app.py
- name: Lint
run: |
pylint --disable=R,C app.py
- name: Test
run: |
python -m pytest -vv test.py
Related
I'm trying to run a local server for my project CI/CD Pipeline. When I start the server I got a "Connection refused" on it.
My project is a fastAPI application and I'm trying to run a integration tests on PR to validate the app before merge the code. I tried to start my app directly (gunicorn), building a docker image and starting it... I tried a lot of things. Then, I tried to run a simple server instead of my app and... got the same error!
This is my simple server workflow:
on:
push:
branches:
- "develop"
jobs:
mylocalserver:
name: Run server
runs-on: ubuntu-latest
steps:
- name: Setup python
uses: actions/setup-python#v3
with:
python-version: 3.9
- name: Run server in background
run: |
python -V
python -m http.server 3000 &> /dev/null &
sudo lsof -i -P -n | grep LISTEN
curl http://localhost:3000
- name: Run server with console
run: |
python -m http.server 3000
Output:
If I run my app with console (no daemon mode in gunicorn), the server start and log to console in workflow with success:
But this way I cannot run nothing after this (and I have to cancel workflow). Some idea? Thank you!
Maybe not the best answer, but for now runnig the job into a container works (only add a container label in question example). Example for my fastAPI app:
on:
pull_request:
branches:
- 'main'
- 'develop'
jobs:
run-on-pr:
runs-on: ubuntu-latest
container: ubuntu
services:
mongodb:
image: mongo
ports:
- 27017:27017
steps:
- name: Setup git
run: |
apt-get update; apt-get install -y git
- name: Git checkout
uses: actions/checkout#v3
with:
path: api
- name: Setup python
uses: actions/setup-python#v4
with:
python-version: 3.9
- name: Install pip
run: |
apt-get update; apt-get install -y python3-pip
- name: Build and ENV
run: |
cd api
cp .env_example_docker .env
pip3 install -r requirements.txt
- name: Run fastAPI
run: |
cd api
gunicorn -D -k uvicorn.workers.UvicornWorker -c ./gunicorn_conf.py app.main:app
env:
MONGO_URL: "mongodb://mongodb:27017"
- name: Install curl
run: |
apt-get update; apt-get install -y curl
- name: Run curl
run: |
curl http://localhost:3000
This works, but I have to install all in container (git, pip). I will try a solution without using the container label and if I found anything I can post here.
I'm using https://github.com/dawidd6/action-ansible-playbook in my pipeline and I need to somehow install the dependency before even running Ansible on the Github Runner. And that does not seem to work for some reason. Any idea what this could be?
deploy:
name: Deploy
runs-on: ubuntu-latest
needs: build-docker
steps:
- name: Check out the codebase.
uses: actions/checkout#v2
- name: Uses Python 3.11
uses: actions/setup-python#v3
with:
python-version: '3.11.0-alpha.1'
- name: Install Jmespath Dependency
run: |
pip3 install jmespath
pip3 freeze # just to see what is installed
sudo apt install -y python3-jmespath
- name: Run playbook
uses: dawidd6/action-ansible-playbook#v2
with:
playbook: provision_vps.yml
directory: ./ansible
key: ${{secrets.ANSIBLE_PRIVATE_KEY}}
vault_password: ${{secrets.ANSIBLE_VAULT_PASS}}
options: |
--inventory hosts.inventory
--verbose
Note that the indentation above is not correct here, it is fine in my editor. Everything else works fine, it's just that dependency that will not be installed on my GitHub Runner so the Ansible controller can run using this task:
- name: Get all file_sd files from scrape_configs
set_fact:
file_sd_files: "{{ prometheus_scrape_configs | json_query('[*][].file_sd_configs[*][].files[]') }}"
The error message I get is:
fatal: [cache.hugin.chat]: FAILED! => {"msg": "You need to install \"jmespath\" prior to running json_query filter"}
I started using github actions and I am trying to let a few lines of shell script running before I create a python package. But I can't get my code to run. It always stops on the line run: | - which is odd because I have been using this exact line before before (see further down). Does anyone know what I am doing wrong?
name: Python packaging
on: [push]
jobs:
job1:
name: Update version
runs-on: ubuntu-latest
steps:
- name: checkout repo
uses: actions/checkout#v3
# ----- This is where the error occurs. Error message:
# Check failure on line 11 in .github/workflows/main.yml
# GitHub Actions / .github/workflows/main.yml
# Invalid workflow file
# You have an error in your yaml syntax on line 11
- name: Increase version
run: |
old_version=$(grep -oP '(?<=0.0.)[0-99]+' contrib/_version.py)
new_version=$(($old_version + 1))
str="__version__=\"0.0."
out="$str$new_version\""
sed -i '1s/.*/'$out'/' contrib/_version.py
job2:
name: Build and upload
needs: job1
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9"]
steps:
- name: checkout repo
uses: actions/checkout#v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python#v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install setuptools wheel
python -m pip install --upgrade build
python -m pip install --upgrade twine
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Create package
run: |
python3 -m build
- name: Upload to Test PyPI
env:
TWINE_USERNAME: "__token__"
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
TWINE_REPOSITORY_URL: "https://test.pypi.org/legacy/"
run: |
twine check dist/*
twine upload --verbose --skip-existing dist/*
The file is in YAML syntax which highly depends on indentation.
Based on your example:
job1: and job2: must have the same indentation.
Under steps:, name: and uses: must have the same indentation.
...
name: Python packaging
on: [push]
jobs:
job1:
name: Update version
runs-on: ubuntu-latest
steps:
- name: checkout repo
uses: actions/checkout#v3
- name: Increase version
run: |
old_version=$(grep -oP '(?<=0.0.)[0-99]+' contrib/_version.py)
new_version=$(($old_version + 1))
str="__version__=\"0.0."
out="$str$new_version\""
sed -i '1s/.*/'$out'/' contrib/_version.py
job2:
name: Build and upload
# ...
I want to test my Python package not only when built locally, but also when it is installed via pip. For this, I basically want to do a pip3 install git+https://github.com/…, like this:
name: CI test
on: [push, pull_request]
jobs:
tests:
name: Pip test
runs-on: ubuntu-20.04
steps:
- name: Setup basic environment
run: |
sudo apt-get update
sudo apt-get install --no-install-recommends python3-dev python3-pip
- name: Install my package via pip
run: |
pip3 install git+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY#$GITHUB_SHA
- name: Run tests
run: |
python3 -m pytest -s --pyargs mypkg
This works nicely for pushed commits, but the Pull Request actions fail claiming that the GITHUB_SHA is "not a tree".
How can I change this so that it works on both commits and pull requests?
I was trying to understand the CI/CD flow from this tutorial by Indian Pythonista and after following the tutorial and setting the Heroku app name and token in the Github Secrets I ran the yaml file below.
name: Python application
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- run: |
git fetch --prune --unshallow
- name: Set up Python 3.8
uses: actions/setup-python#v1
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pip install pytest
export PYTHONPATH=src
pytest
- name: Deploy to Heroku
env:
HEROKU_API_TOKEN: ${{ secrets.HEROKU_API_TOKEN }}
HEROKU_APP_NAME: ${{ secrets.HEROKU_APP_LINKNAME }}
if: github.ref == 'refs/heads/master' && job.status == 'success'
run: |
git remote add heroku https://heroku:$HEROKU_API_TOKEN#git.heroku.com/$HEROKU_APP_NAME.git
git push heroku HEAD:master -f
The build looked like the screenshot below:
I do not exactly know what should I call this error as well, since I am new to the topic. Due to this, the flask app doesn't get updated on the Heroku app link as well.