How to assign environment variables from a file in GitHub actions? - bash

I'm currently working on a Django project. I have create a GitHub action to run python manage.py test command everytime I push the code to main branch.
The problem here is, I have many environment variables in my project. I can't set env variables as GitHub secrets for each variables.
I have a env.dev file in my repository. What I need to do is, everytime I push the code, it needs to assign environment variables by reading it from the env.dev file.
Is there any way to do this?
This is my django.yml file used for GitHub actions.
name: Django CI
on:
pull_request:
branches: [ "main"]
jobs:
build:
runs-on: ubuntu-latest
strategy:
max-parallel: 4
matrix:
python-version: [3.9]
steps:
- uses: actions/checkout#v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python#v3
with:
python-version: ${{ matrix.python-version }}
- name: Install Dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Run Tests
run: |
python manage.py test

As noted in "How do I use an env file with GitHub Actions?", you do have actions like Dotenv Action which do read a .env file from the root of this repo and provides environment variables to build steps.
But those would not be GitHub secret though.
There is an API to create secrets, which means, if your .env content does not change too often, you can script the secret creation step.

Related

Is that possible to deploy Laravel web application to shared hosting using GitHub Action & GitHub FTP Deploy?

Is that possible to deploy the Laravel web application to shared hosting using GitHub Action & GitHub FTP Deploy? If possible how should I change the.github\workflows\master.yml?
on:
push:
branches:
- master
name: 🚀 Deploy website on push
jobs:
web-deploy:
name: 🎉 Deploy
runs-on: ubuntu-latest
steps:
- name: 🚚 Get latest code
uses: actions/checkout#v2
- name: 📂 Sync files
uses: SamKirkland/FTP-Deploy-Action#4.2.0
with:
server: ${{ secrets.ftp_server }}
username: ${{ secrets.ftp_username }}
password: ${{ secrets.ftp_password }}
server-dir: /
Looks like you're very close but are missing 2 important steps: set up a temporary PHP environment, and use that environment to install your dependencies (Composer).
GitHub Actions Setup
This guide assumes you have a working Laravel installation, a GitHub account, and a shared hosting account that you can access via FTP using a username/password.
I found this video https://www.youtube.com/watch?v=UNWIXYSZfZY helpful to get a basic understanding of how to deploy a simple application. To make this answer helpful to a wider range of people, I'll give a quick outline of my setup. There really aren't any Laravel specific steps.
Workflow directory set up
Create the directories .github\workflows at the root of your project. In the workflows directory, create a yml file named after the branch you want to push to your shared hosting account. Ex. master.yml, staging.yml, development.yml etc. If you only have a single branch then just create one file. The name is important and should match the name of the branch.
Design your workflow
This is very dependent on your project but assuming you have a basic Laravel application without the need for additional components such as Node, then this is a basic GitHub Action that works for me on a variety of projects.
A basic action file consists of 2 sections, the workflow, and the jobs. A workflow triggers the jobs.
Workflow
Lines 1-4 say this will run each time we push to the master branch.
on:
push:
branches:
- master
Line 5 is the name of this workflow and will show up on your Actions page. Set this to something descriptive.
name: 🚀 Deploy website on push (Master)
Setting up jobs
In this action, there are 5 jobs. Some take parameters, others don't. I'm not going to explain all the details here but have linked to the corresponding repositories if you need details.
Checkout your code so the workflow has access to it,
https://github.com/actions/checkout
name: 🚚 Get latest code
uses: actions/checkout#v2
Sets up a temporary PHP environment so you can run things like
Composer, https://github.com/shivammathur/setup-php. Make sure to set your PHP version here otherwise you could run into issues when installing Composer packages with an unexpected PHP version.
name: Setup PHP
uses: shivammathur/setup-php#v2
with:
php-version: 7.2
Caches your dependencies for faster deploys,
https://github.com/actions/cache
name: Cache Composer packages
id: composer-cache
uses: actions/cache#v2
with:
path: vendor
key: ${{ runner.os }}-php-${{ hashFiles('**/composer.lock') }}
restore-keys: |
${{ runner.os }}-php-
Install your dependencies from composer.json and composer.lock
files.
name: Install dependencies
run: composer install --prefer-dist --no-progress
Deploys your code to your remote shared hosting site,
https://github.com/SamKirkland/FTP-Deploy-Action. Note the use of ${{ secrets.ftp_username }} and ${{ secrets.ftp_password }}. These are set up in your repository's secrets section. See https://docs.github.com/en/actions/security-guides/encrypted-secrets
name: 📂 Sync files
uses: SamKirkland/FTP-Deploy-Action#4.0.0
with:
server: name_of_server.com
username: ${{ secrets.ftp_username }}
password: ${{ secrets.ftp_password }}
server-dir: public_html/
Final file
on:
push:
branches:
- master
name: 🚀 Deploy website on push (Master)
jobs:
web-deploy:
name: 🎉 Deploy
runs-on: ubuntu-latest
steps:
- name: 🚚 Get latest code
uses: actions/checkout#v2
- name: Setup PHP
uses: shivammathur/setup-php#v2
with:
php-version: 7.2
- name: Cache Composer packages
id: composer-cache
uses: actions/cache#v2
with:
path: vendor
key: ${{ runner.os }}-php-${{ hashFiles('**/composer.lock') }}
restore-keys: |
${{ runner.os }}-php-
- name: Install dependencies
run: composer install --prefer-dist --no-progress
- name: 📂 Sync files
uses: SamKirkland/FTP-Deploy-Action#4.0.0
with:
server: name_of_server.com
username: ${{ secrets.ftp_username }}
password: ${{ secrets.ftp_password }}
server-dir: public_html/
Running the workflow
Check-in .github\workflows\master.yml, and others if appropriate,
into your GitHub repository. Without these files checked in nothing
will happen when you push a change to the branch.
Go to your Actions tab and ensure the workflow shows up there.
Push a change to your branch and watch the Actions tab. Click into
the running action to see details about the run.
Fix any errors that show up in the console.
Finally, you mentioned in a comment something about NPM. If you have Node as a component in your project you can simply run two extra steps that will bundle your assets and will get deployed along with the rest of the code.
Good luck!

Using github secrets in another non-workflow yaml file

Is it possible to access a github secret in a yaml file that's not a workflow or an action yaml file?
For example, I've saved in github the environment secret INFURA_RINKEBY_WSS and I attempt to access it in the following yaml config file for my program.
type: EndpointList
endpoints:
- type: RPCEndpoint
chain_id: 1
network: rinkeby
provider: Infura
url: ${{ secrets.INFURA_RINKEBY_WSS}}
explorer: https://etherscan.io
However, the INFURA_RINKEBY_WSS environment variable I've set in github isn't accessed yet by my yaml config file.
The following is my main.yaml github workflow:
name: Report to eth/usd on rinkeby w/ pytelliot
on: push
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.9"]
steps:
- uses: actions/checkout#v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python#v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install telliot-feed-examples
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Move pre-made pytelliot configs to home directory
run: |
cp -r ./config ~/
- name: report :)
run: telliot-examples --legacy-id 1 report --submit-once
env:
PK: ${{ secrets.PK }}
INFURA_RINKEBY_WSS: ${{ secrets.INFURA_RINKEBY_WSS }}
Thanks!

Is it possible to use GitHub secrets inside my shell file?

This is my simple Action on my GitHub repo:
name: CI
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Get /my_account/my_infra
run: |
sudo mkdir /my_account
sudo chmod -R 777 /my_account
cd /my_account
git clone https://github.com/my_account/my_infra
- name: Get /my_account/my_repo
run: |
cd /my_account
git clone https://github.com/my_account/my_repo
- name: Run my build script
run: |
cd /my_account/my_infra
./build.sh /my_account/my_repo
Since GitHub does not provide a way to reuse actions across multiple similar repos, I came up with the idea of creating a base repo, then download that base alongside the current repo, then run a custom shell script from that base repo, passing my current repo as a parameter.
This works perfect. This way I can reuse my base repo across many similar repositories. And I can reuse near 500 lines of build script instead of repeating myself for 50 repositors (which means 25000 lines of CI/CD code).
However, now I need to access some resources (like login into my docker hub account) to pull and push stuff.
Is it possible to use GitHub secrects in my build.sh?
When you set env in your workflow, doc here, they are set as environment variables in your containerised workflow.
This means that if you set a secret in your repository, can be found under settings=> secrets and then assign it to an env in your workflow, they can then be accessed in your build.sh
example:
name: CI
on:
push:
branches: [ main ]
env:
super_secret: ${{ secrets.my_secret }}
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Get /my_account/my_infra
run: |
sudo mkdir /my_account
sudo chmod -R 777 /my_account
cd /my_account
git clone https://github.com/my_account/my_infra
- name: Get /my_account/my_repo
run: |
cd /my_account
git clone https://github.com/my_account/my_repo
- name: Run my build script
run: |
cd /my_account/my_infra
./build.sh /my_account/my_repo
In this case your build.sh can do something like:
#!/bin/bash
npm run build $super_secret
Yes, you just need to assign them to a variable, like
env:
ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}
run: build.sh
Then you can refer to ACCESS_TOKEN variable in the shell script.

Installing private Github Package using yarn on Github Actions is Unauthorized with yarn.lock

There are a lot of similar issues already floating around:
Install private github package from package.json on Github Actions
Download private module from Github Package Registry via Yarn within a Github Action? Publishing works, but installing is met with '401 Unauthorized'
Installing private package from Github Package registry using Yarn fails with not authorized
However, our issue seems different, because:
yarn install runs fine on a local machine
the issue is only when using Github Actions
yarn install succeeds on GH Actions if we delete yarn.lock
Has anyone run into this before? Specifically with it not working with a yarn.lock file?
In case it matters, here's the setup:
build.yml:
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
with:
node-version: '10.x'
registry-url: 'https://npm.pkg.github.com'
- name: Install
run: yarn install
env:
# GITHUB_TOKEN can't access packages hosted in private repos,
# even within the same organisation
NODE_AUTH_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
- name: Build
run: yarn build
- name: Test
run: yarn test --forbid-only
We also have a .npmrc file for local installs:
#<org>:registry=https://npm.pkg.github.com
But no .yarnrc file.
I'm create a file .npmrc and .yarnrc.
Type:
name: Test
on: push
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x]
steps:
- uses: actions/checkout#v2
- name: Node ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: Create NPMRC
run: |
echo "//npm.pkg.github.com/:_authToken=${{ secrets.PACKAGES_TOKEN }}" >> ~/.npmrc
echo "#you-scope:registry=https://npm.pkg.github.com" >> ~/.npmrc
echo 'registry "https://registry.yarnpkg.com"' >> ~/.yarnrc
- run: yarn install
Replace #you-scope for you user of github or of your org in github in LowerCase.
Create a PACKAGES_TOKEN secrete token of your github access for this repository.
We managed to solve this by explicitly duplicating the .npmrc config in the build.yml config:
- uses: actions/setup-node#v1
with:
node-version: '10.x'
registry-url: 'https://npm.pkg.github.com'
# These following two lines are the key:
always-auth: true
scope: '#reedsy'

Github Actions, Python Coverage and Sonar Qube

I want to create a Github workflow that does the following:
test my code with pytest
trigger Sonar Qube Cloud to analyze to the code and show my test coverage!
As far as I understand, SonarQ needs a file coverage.xml to display the code coverage. This can be generated with
pytest --cov=./ --cov-report=xml --doctest-modules
According to this article coverage.xml should be available under /github/workspace/coverage.xml.
Thus, I specify my sonar-project.properties in the root folder of the project:
sonar.organization=pokemate
sonar.projectKey=PokeMate_name-generator
sonar.sources=.
sonar.python.coverage.reportPath=/github/workspace/coverage.xml
my actions file build.yml:
on:
push:
branches:
- master
- develop
- sonar-qube-setup
jobs:
build:
runs-on:
- ubuntu-latest
steps:
# Checkout repo
- uses: actions/checkout#v2
# Dependencies
- name: Set up Python 3.7
uses: actions/setup-python#v1
with:
python-version: 3.7
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
# Test
- name: Test with pytest
run: |
pytest --cov=./ --cov-report=xml --doctest-modules
# Sonar Qube
- name: SonarCloud Scan
uses: sonarsource/sonarcloud-github-action#master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
However, on SonarQ it still shows 0% test coverage, which is probably because it cannot find the coverage.xml. Any idea how to make this work?
The error came from the missing s in reportPaths in the sonar-project.properties file.

Resources