This is my simple Action on my GitHub repo:
name: CI
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Get /my_account/my_infra
run: |
sudo mkdir /my_account
sudo chmod -R 777 /my_account
cd /my_account
git clone https://github.com/my_account/my_infra
- name: Get /my_account/my_repo
run: |
cd /my_account
git clone https://github.com/my_account/my_repo
- name: Run my build script
run: |
cd /my_account/my_infra
./build.sh /my_account/my_repo
Since GitHub does not provide a way to reuse actions across multiple similar repos, I came up with the idea of creating a base repo, then download that base alongside the current repo, then run a custom shell script from that base repo, passing my current repo as a parameter.
This works perfect. This way I can reuse my base repo across many similar repositories. And I can reuse near 500 lines of build script instead of repeating myself for 50 repositors (which means 25000 lines of CI/CD code).
However, now I need to access some resources (like login into my docker hub account) to pull and push stuff.
Is it possible to use GitHub secrects in my build.sh?
When you set env in your workflow, doc here, they are set as environment variables in your containerised workflow.
This means that if you set a secret in your repository, can be found under settings=> secrets and then assign it to an env in your workflow, they can then be accessed in your build.sh
example:
name: CI
on:
push:
branches: [ main ]
env:
super_secret: ${{ secrets.my_secret }}
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Get /my_account/my_infra
run: |
sudo mkdir /my_account
sudo chmod -R 777 /my_account
cd /my_account
git clone https://github.com/my_account/my_infra
- name: Get /my_account/my_repo
run: |
cd /my_account
git clone https://github.com/my_account/my_repo
- name: Run my build script
run: |
cd /my_account/my_infra
./build.sh /my_account/my_repo
In this case your build.sh can do something like:
#!/bin/bash
npm run build $super_secret
Yes, you just need to assign them to a variable, like
env:
ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}
run: build.sh
Then you can refer to ACCESS_TOKEN variable in the shell script.
Related
I have 2 branches, main and yamltesting. In main I have a test.py file and my yaml file is triggered on push. The yaml file should build an ubuntu linux OS checkout my repo and run a git script to extract the test.py file from "main" branch.
name: CI for Hello World
on:
push:
branches:
- yamlTesting
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-python#v2
with:
python-version: 3.11
- name: Install Libraries
run: |
pip install flake8 pytest pytest-cov
- name: Checkout own repo
uses: actions/checkout#v2
with:
fetch-depth: "0"
- name: git
run: |
# setup the username and email. I tend to use 'GitHub Actions Bot' with no email by default
git --version
git config user.name "Username"
git config user.email "email#gmail.com"
git status
git tag
git checkout main -- test.py
- name: list our contents
run: |
ls
My issue is that when the action runs this error appears
error: pathspec 'main' did not match any file(s) known to git
error: pathspec 'test.py' did not match any file(s) known to git
Error: Process completed with exit code 1.
I tried the following variations I found.
git checkout "main" -- test.py
git checkout main test.py
Extract file from remote branch needs to use remotes/origin/<branchName> instead of just the branch name
I have a dockerised fastapi app whith depends on mysql and redis which are all configured in docker-compose.yml. Want to implement a CI/CD using github actions and AWS EC2 instance. My EC2 instance has docker and docker-compose installed. Here are my questions.
What do I do to run the tests that depends on the test db?
How do I implement CD from github actions and AWS EC2 instance?
I might not be clear so please ask some questions for clarification. Thank you.
name: backend-api
on:
push:
branches: ["main"]
pull_request:
branches: ["main"]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Set up Python 3.10
uses: actions/setup-python#v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Create .env file for configuration settings.
uses: SpicyPizza/create-envfile#v1.3
with:
envkey_APP_ENV: ${{secrets.APP_ENV}}
envkey_APP_HOST: ${{secrets.APP_HOST}}
envkey_MYSQL_USER: ${{secrets.APP_ENV}}
envkey_PROD_BASE_URL: ${{secrets.MYSQL_USER}}
envkey_DEV_BASE_URL: ${{secrets.APP_ENV}}
envkey_MYSQL_ROOT_PASSWORD: ${{secrets.MYSQL_ROOT_PASSWORD}}
envkey_MYSQL_DATABASE: ${{secrets.MYSQL_DATABASE}}
envkey_PRODUCTION_DB_URI: ${{secrets.PRODUCTION_DB_URI}}
envkey_TEST_DB_URI: ${{secrets.TEST_DB_URI}}
envkey_BASE_URL: ${{secrets.BASE_URL}}
envkey_WALLET_PROVIDER_ACCESS_TOKEN: ${{secrets.WALLET_PROVIDER_ACCESS_TOKEN}}
envkey_S3_BUCKET_NAME: ${{secrets.S3_BUCKET_NAME}}
envkey_S3_ACCESS_SECRET: ${{secrets.S3_ACCESS_SECRET}}
envkey_S3_ACCESS_KEY: ${{secrets.S3_ACCESS_KEY}}
envkey_S3_BUCKET_REGION: ${{secrets.S3_BUCKET_REGION}}
envkey_JWT_SECRET_KEY: ${{secrets.JWT_SECRET_KEY}}
envkey_ETHERSCAN_API_URL: ${{secrets.ETHERSCAN_API_URL}}
envkey_BLOCKCHAIN_API_URL: ${{secrets.BLOCKCHAIN_API_URL}}
envkey_WALLET_PROVIDER_BASE_URL: ${{secrets.WALLET_PROVIDER_BASE_URL}}
envkey_STRATEGY_PROVIDER_BASE_URL: ${{secrets.STRATEGY_PROVIDER_BASE_URL}}
envkey_INDEX_PROVIDER_BASE_URL: ${{secrets.INDEX_PROVIDER_BASE_URL}}
- name: Running Tests with pytest
run: |
pytest
Deploy:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Git pull
env:
AWS_EC2_PEM: ${{ secrets.AWS_EC2_PEM }}
AWS_EC2_PUBLIC_IP: ${{ secrets.AWS_EC2_PUBLIC_IP }}
AWS_EC2_USERNAME: ${{ secrets.AWS_EC2_USERNAME }}
run: |
pwd
echo "$AWS_EC2_PEM" > private_key && chmod 600 private_key
ssh -o StrictHostKeyChecking=no -i private_key ${AWS_EC2_USERNAME}#${AWS_EC2_PUBLIC_IP}
git checkout main &&
git fetch --all &&
git reset --hard origin/main &&
git pull origin main &&
touch .env
docker-compose up -d --build
I'm currently working on a Django project. I have create a GitHub action to run python manage.py test command everytime I push the code to main branch.
The problem here is, I have many environment variables in my project. I can't set env variables as GitHub secrets for each variables.
I have a env.dev file in my repository. What I need to do is, everytime I push the code, it needs to assign environment variables by reading it from the env.dev file.
Is there any way to do this?
This is my django.yml file used for GitHub actions.
name: Django CI
on:
pull_request:
branches: [ "main"]
jobs:
build:
runs-on: ubuntu-latest
strategy:
max-parallel: 4
matrix:
python-version: [3.9]
steps:
- uses: actions/checkout#v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python#v3
with:
python-version: ${{ matrix.python-version }}
- name: Install Dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Run Tests
run: |
python manage.py test
As noted in "How do I use an env file with GitHub Actions?", you do have actions like Dotenv Action which do read a .env file from the root of this repo and provides environment variables to build steps.
But those would not be GitHub secret though.
There is an API to create secrets, which means, if your .env content does not change too often, you can script the secret creation step.
I want to do a simple CD/CD to do automatic deployments for my Laravel project but turns out my .env file is always replaced. How do I make sure it's not always replaced
Here is my action file
name: Laravel
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
laravel-tests:
runs-on: self-hosted
steps:
- uses: shivammathur/setup-php#b7d1d9c9a92d8d8463ce36d7f60da34d461724f8
with:
php-version: '7.4'
- uses: actions/checkout#v2
- name: Copy .env
run: php -r "file_exists('.env') || copy('.env.example', '.env');"
- name: Install Dependencies
run: composer install -q --no-ansi --no-interaction --no-scripts --no-progress --prefer-dist
- name: Generate key
run: php artisan key:generate
- name: Directory Permissions
run: chmod -R 777 storage bootstrap/cache
- name: Create Database
run: |
mkdir -p database
touch database/database.sqlite
- name: Execute tests (Unit and Feature tests) via PHPUnit
env:
DB_CONNECTION: sqlite
DB_DATABASE: database/database.sqlite
run: vendor/bin/phpunit
Each time your workflow run you will get a new machine. Thus there is no files you created them on previous run. As this if you need to have some file created at runtime you need to repeat this step each time.
Please check it here to understand better github ations basics.
You don't have much options to share this file accros run, as keeping secrets in artifacts is bad choice. So you need to recreate this file each time you need it based on the secret which you may keep in secrets context. Please check this link:
- name: Create env file
run: |
cat << EOF >> .env
API_ENDPOINT="https://xxx.execute-api.us-west-2.amazonaws.com"
API_KEY=${{ secrets.API_KEY }}
EOF
The github action works on the files on your git repo.
Your .env is not and should not be in the repo, because it contains your credentials/secrets.
You may have it on your local computer, but it's included in the gitignore so git (an github as a consequence) doesn't track it.
As a consequence no, you don't have the .env file at each action run. The most straightforward way to do it is:
create a .env.production file that is gitted and committed. Place there your main .env variables that aren't secrets/sensitive, such as:
APP_NAME=YourAppName
APP_ENV=production
APP_DEBUG=false
APP_LOG_LEVEL=warning
APP_URL=https://your-url.com
CACHE_DRIVER=redis
SESSION_DRIVER=file
QUEUE_CONNECTION=redis
... etc ...
DO NOT INCLUDE KEYS, PASSWORDS OR SECRETS THOUGH..
Now copy that file as your default .env file in one of your steps:
- name: Copy .env
run: php -r "file_exists('.env') || copy('.env.production', '.env');"
Now it's time to handle your secrets. You should add them as github secret of your repo, and included using the env directive of your github actions:
env:
APP_KEY: ${{ secrets.APP_KEY }}
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
....
This last step may need to vary a bit depending on how/where you do the actual deploy, but that should be the gist of it: you pull them from the github secrets space and you add them to the production environment
Try to use GitHub Actions but getting error when trying to run executable build. I can find executable when doing ls and also changing mod of file executable but didn't got anything. Attached log/output of GitHub action.
Here is my github action workflow file
on:
push:
branches:
- master
name: Build For platforms
jobs:
build:
# We want to define a strategy for our job
strategy:
# this will contain a matrix of all of the combinations
# we wish to test again:
matrix:
go-version: [1.14.x]
platform: [ubuntu-latest]
# Defines the platform for each test run
runs-on: ${{ matrix.platform }}
# the steps that will be run through for each version and platform
# combination
steps:
# sets up go based on the version
- name: Install Go
uses: actions/setup-go#v2
with:
go-version: ${{ matrix.go-version }}
# checks out our code locally so we can work with the files
- name: Checkout code
uses: actions/checkout#v2
- name: Download modules
run: go mod download
- name: Build
run: go build -o executable main.go
- name: Chmod
run: chmod +x executable
- name: List Files
run: ls && pwd
- name: Run
run: ./executable