Using github secrets in another non-workflow yaml file - yaml

Is it possible to access a github secret in a yaml file that's not a workflow or an action yaml file?
For example, I've saved in github the environment secret INFURA_RINKEBY_WSS and I attempt to access it in the following yaml config file for my program.
type: EndpointList
endpoints:
- type: RPCEndpoint
chain_id: 1
network: rinkeby
provider: Infura
url: ${{ secrets.INFURA_RINKEBY_WSS}}
explorer: https://etherscan.io
However, the INFURA_RINKEBY_WSS environment variable I've set in github isn't accessed yet by my yaml config file.
The following is my main.yaml github workflow:
name: Report to eth/usd on rinkeby w/ pytelliot
on: push
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.9"]
steps:
- uses: actions/checkout#v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python#v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install telliot-feed-examples
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Move pre-made pytelliot configs to home directory
run: |
cp -r ./config ~/
- name: report :)
run: telliot-examples --legacy-id 1 report --submit-once
env:
PK: ${{ secrets.PK }}
INFURA_RINKEBY_WSS: ${{ secrets.INFURA_RINKEBY_WSS }}
Thanks!

Related

CI/CD using Github Actions and AWS EC2 instance

I have a dockerised fastapi app whith depends on mysql and redis which are all configured in docker-compose.yml. Want to implement a CI/CD using github actions and AWS EC2 instance. My EC2 instance has docker and docker-compose installed. Here are my questions.
What do I do to run the tests that depends on the test db?
How do I implement CD from github actions and AWS EC2 instance?
I might not be clear so please ask some questions for clarification. Thank you.
name: backend-api
on:
push:
branches: ["main"]
pull_request:
branches: ["main"]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Set up Python 3.10
uses: actions/setup-python#v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Create .env file for configuration settings.
uses: SpicyPizza/create-envfile#v1.3
with:
envkey_APP_ENV: ${{secrets.APP_ENV}}
envkey_APP_HOST: ${{secrets.APP_HOST}}
envkey_MYSQL_USER: ${{secrets.APP_ENV}}
envkey_PROD_BASE_URL: ${{secrets.MYSQL_USER}}
envkey_DEV_BASE_URL: ${{secrets.APP_ENV}}
envkey_MYSQL_ROOT_PASSWORD: ${{secrets.MYSQL_ROOT_PASSWORD}}
envkey_MYSQL_DATABASE: ${{secrets.MYSQL_DATABASE}}
envkey_PRODUCTION_DB_URI: ${{secrets.PRODUCTION_DB_URI}}
envkey_TEST_DB_URI: ${{secrets.TEST_DB_URI}}
envkey_BASE_URL: ${{secrets.BASE_URL}}
envkey_WALLET_PROVIDER_ACCESS_TOKEN: ${{secrets.WALLET_PROVIDER_ACCESS_TOKEN}}
envkey_S3_BUCKET_NAME: ${{secrets.S3_BUCKET_NAME}}
envkey_S3_ACCESS_SECRET: ${{secrets.S3_ACCESS_SECRET}}
envkey_S3_ACCESS_KEY: ${{secrets.S3_ACCESS_KEY}}
envkey_S3_BUCKET_REGION: ${{secrets.S3_BUCKET_REGION}}
envkey_JWT_SECRET_KEY: ${{secrets.JWT_SECRET_KEY}}
envkey_ETHERSCAN_API_URL: ${{secrets.ETHERSCAN_API_URL}}
envkey_BLOCKCHAIN_API_URL: ${{secrets.BLOCKCHAIN_API_URL}}
envkey_WALLET_PROVIDER_BASE_URL: ${{secrets.WALLET_PROVIDER_BASE_URL}}
envkey_STRATEGY_PROVIDER_BASE_URL: ${{secrets.STRATEGY_PROVIDER_BASE_URL}}
envkey_INDEX_PROVIDER_BASE_URL: ${{secrets.INDEX_PROVIDER_BASE_URL}}
- name: Running Tests with pytest
run: |
pytest
Deploy:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Git pull
env:
AWS_EC2_PEM: ${{ secrets.AWS_EC2_PEM }}
AWS_EC2_PUBLIC_IP: ${{ secrets.AWS_EC2_PUBLIC_IP }}
AWS_EC2_USERNAME: ${{ secrets.AWS_EC2_USERNAME }}
run: |
pwd
echo "$AWS_EC2_PEM" > private_key && chmod 600 private_key
ssh -o StrictHostKeyChecking=no -i private_key ${AWS_EC2_USERNAME}#${AWS_EC2_PUBLIC_IP}
git checkout main &&
git fetch --all &&
git reset --hard origin/main &&
git pull origin main &&
touch .env
docker-compose up -d --build

Env variables set in GitHub Actions workflow not accessible in AWS lambda project

I have a Chalice (AWS lambda Python framework) project the following CI/CD GitHub Action workflow:
name: Production Workflow
on:
push:
branches:
- "main"
env:
REPO: ${{ github.repository }}
GITHUB_REF_NAME: ${{ github.ref_name }}
GITHUB_SHA: ${{ github.sha }}
jobs:
production:
name: Deploy production
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Set up Python
uses: actions/setup-python#v1
with:
python-version: "3.9"
- name: Install requirements
run: pip3 install -r requirements.txt
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials#v1
with:
aws-region: ${{ secrets.AWS_REGION }}
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: Check branch
run: echo "${{ env.GITHUB_REF_NAME }}"
- name: Check branch
run: echo "${{ env.GITHUB_SHA }}"
- name: Run tests
run: python3 -m unittest discover -s tests
- name: Deploy with Chalice
run: chalice deploy --stage=production
However, from inside the project, the env variables REPO, GITHUB_REF_NAME and GITHUB_SHA are not accessible (i.e. os.environ.get("GITHUB_REF_NAME", None)). Why?
I also tried setting the env variables not globally, but in the "Deploy with Chalice" step only, with the same result. Also, I can successfully see the branch and commit ID written in GitHub Actions by the "Check branch" and "Check branch" steps.
Other env variables that are set in the Chalice config file .chalice/config.json are accessible.
You need to set up the environment and explicitly list all ENV variables you want to use, like this:
- name: Deploy with Chalice
run: chalice deploy --stage=production
env:
GITHUB_REF_NAME: ${{ secrets.GITHUB_REF_NAME }}
GITHUB_SHA: ${{ secrets.GITHUB_SHA }}

Run build process before copying files

I have the following deploy.yml
name: Deploy
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Copy repository contents via scp
uses: appleboy/scp-action#master
env:
HOST: ${{ secrets.HOST }}
USERNAME: ${{ secrets.USERNAME }}
PORT: ${{ secrets.PORT }}
PASSWORD: ${{ secrets.PASSWORD }}
with:
source: "."
target: "/var/www/html/cnaiapp"
rm: true
- name: Executing remote command
uses: appleboy/ssh-action#develop
with:
host: ${{ secrets.HOST }}
USERNAME: ${{ secrets.USERNAME }}
PORT: ${{ secrets.PORT }}
PASSWORD: ${{ secrets.PASSWORD }}
script: cd /var/www/html/cnaiapp && npm run deploy
However, the master branch has unminified and testing code, that I don't want to have in my VPS. Do you know how could I achieve this? BTW, in order to remove this unnecessary code, I'd need to run the npm run build command.
PS: The npm run deploy command just builds the code and starts the server.
Just add a new step with the run command after checkout:
โ€ฆ
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: build
run: npm run build
- name: Copy repository contents via scp
โ€ฆ

"The cypress npm package is installed, but the Cypress binary is missing." in GitHub Actions

I'm receiving the following error when running a Cypress e2e test runner on GitHub Actions:
The cypress npm package is installed, but the Cypress binary is missing.
We expected the binary to be installed here: /home/runner/.cache/Cypress/8.5.0/Cypress/Cypress
Reasons it may be missing:
- You're caching 'node_modules' but are not caching this path: /home/runner/.cache/Cypress
- You ran 'npm install' at an earlier build step but did not persist: /home/runner/.cache/Cypress
My .github/workflow/tests.yml is set up as follows:
name: celestia/tests
on:
pull_request:
branches:
- main
- master
jobs:
unit:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
node: [14]
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout ๐Ÿ›Ž
uses: actions/checkout#master
# Setup .npmrc file to publish to GitHub Packages
- name: Setup node env ๐Ÿ— and .npmrc file to publish to GitHub Packages
uses: actions/setup-node#v2.1.2
with:
node-version: ${{ matrix.node }}
registry-url: 'https://npm.pkg.github.com'
# Defaults to the user or organization that owns the workflow file:
scope: '#observerly'
- name: Cache node_modules ๐Ÿ“ฆ
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
- name: Install project dependencies ๐Ÿ‘จ๐Ÿปโ€๐Ÿ’ป
run: npm ci --ignore-scripts
- name: Run jest unit tests ๐Ÿงช
run: npm run test:unit
e2e:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
node: [14]
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout ๐Ÿ›Ž
uses: actions/checkout#master
# Setup .npmrc file to publish to GitHub Packages
- name: Setup node env ๐Ÿ— and .npmrc file to publish to GitHub Packages
uses: actions/setup-node#v2.1.2
with:
node-version: ${{ matrix.node }}
registry-url: 'https://npm.pkg.github.com'
# Defaults to the user or organization that owns the workflow file:
scope: '#observerly'
- name: Install project dependencies ๐Ÿ‘จ๐Ÿปโ€๐Ÿ’ป
run: npm ci --ignore-scripts
- name: Run e2e cypress tests ๐Ÿงช
run: npm run test:e2e:headless
How can I side-step this particular issue?

Installing private Github Package using yarn on Github Actions is Unauthorized with yarn.lock

There are a lot of similar issues already floating around:
Install private github package from package.json on Github Actions
Download private module from Github Package Registry via Yarn within a Github Action? Publishing works, but installing is met with '401 Unauthorized'
Installing private package from Github Package registry using Yarn fails with not authorized
However, our issue seems different, because:
yarn install runs fine on a local machine
the issue is only when using Github Actions
yarn install succeeds on GH Actions if we delete yarn.lock
Has anyone run into this before? Specifically with it not working with a yarn.lock file?
In case it matters, here's the setup:
build.yml:
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
with:
node-version: '10.x'
registry-url: 'https://npm.pkg.github.com'
- name: Install
run: yarn install
env:
# GITHUB_TOKEN can't access packages hosted in private repos,
# even within the same organisation
NODE_AUTH_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
- name: Build
run: yarn build
- name: Test
run: yarn test --forbid-only
We also have a .npmrc file for local installs:
#<org>:registry=https://npm.pkg.github.com
But no .yarnrc file.
I'm create a file .npmrc and .yarnrc.
Type:
name: Test
on: push
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x]
steps:
- uses: actions/checkout#v2
- name: Node ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: Create NPMRC
run: |
echo "//npm.pkg.github.com/:_authToken=${{ secrets.PACKAGES_TOKEN }}" >> ~/.npmrc
echo "#you-scope:registry=https://npm.pkg.github.com" >> ~/.npmrc
echo 'registry "https://registry.yarnpkg.com"' >> ~/.yarnrc
- run: yarn install
Replace #you-scope for you user of github or of your org in github in LowerCase.
Create a PACKAGES_TOKEN secrete token of your github access for this repository.
We managed to solve this by explicitly duplicating the .npmrc config in the build.yml config:
- uses: actions/setup-node#v1
with:
node-version: '10.x'
registry-url: 'https://npm.pkg.github.com'
# These following two lines are the key:
always-auth: true
scope: '#reedsy'

Resources