Under my Git Repo Settings > Environments : I define an env name PPM_DEV, under this env PPM_DEV I define an environment variable named HOSTNAME and give it a value in the git configurations page.
Now what is the YAML syntax under GitActions WF to read the value of this variable ?
basically I may have 2 Env's defined PPM_DEV , PPM_TEST
but where do I set the Env context to pull the variable HOSTNAME from the PPM_DEV env ?
In the example below , I am trying to populate a variable VARHOSTNAME with the value of the Env variable HOSTNAME that is pre-defined against the Env named PPM_DEV
However it fails with
The workflow is not valid. .github/workflows/Ext_Conn_v2.yml (Line: 10, Col: 7): Unexpected value 'VARHOSTNAME'
name: Ext_Conn_v2
on:
workflow_dispatch:
jobs:
RunonVM:
runs-on: ubuntu-latest
environment:
name: ppm_dev
VARHOSTNAME: ${{ env.HOSTNAME }}
steps:
- name: Run a command
run: |
echo "This workflow was manually triggered."
echo "value of the variable HOSTNAME: " ${VARHOSTNAME}
pwd
echo "end of run"
You need to use jobs.<job_id>.steps[*].env to specify the environment variables or secrets using vars or secrets contexts:
Here's an example with the secrets context:
jobs:
job:
runs-on: ubuntu-latest
environment: ppm_dev
steps:
- name: Command
env:
HOSTNAME: ${{ secrets.HOSTNAME }}
run: |
echo "HOSTNAME: $HOSTNAME"
See a linted example here.
Related
How do I specify or change an environment variable as part of a job matrix?
For example, here I want my job to echo "Two" and "Three" instead of "One", but github completely ignores the definition or change of the environment variable in the matrix:
name: test-test-test
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main, develop ]
env:
MY_VAR: One
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- env:
MY_VAR: Two
- env:
MY_VAR: Three
steps:
- run: echo "$MY_VAR2"
yields echo "One" and echo "One" in both jobs.
Use this context to access information about environment variables that have been set in a workflow, step, or job. Note that you cannot use the env context in the value of the id and uses keys within a step. An example is env.env_var.
GitHub provides extensive documentation about contexts at
https://docs.github.com/en/actions/learn-github-actions/contexts.
The syntax you used won't work for matrix. As stated in the official documentation:
The variables that you define become properties in the matrix context, and you can reference the property in other areas of your workflow file.
In your case, to access the matrix env variable you set in the include field list, you would need to use ${{ matrix.env.MY_VAR }}.
Something like this:
env:
MY_VAR: One
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- env:
MY_VAR: Two
- env:
MY_VAR: Three
steps:
- run: |
echo "$MY_VAR" # is equal to ${{ env.MY_VAR }}
echo ${{ matrix.env.MY_VAR }}
This will generate 2 test jobs: test (Two) and test (Three).
I made a test here if you want to have a look.
Conclusion:
Basically, the env you set at the workflow level isn't the same env set in the matrix strategy, so you can't substitute the value there.
Alternative
What you could do eventually (as #Adam stated in the other answer) is to set the env value at the workflow level for it to work, in the environment context. (Reference in the documentation + another workflow example).
Example:
env:
WORKFLOW_VARIABLE: WORKFLOW
jobs:
test:
runs-on: ubuntu-latest
env:
JOB_VARIABLE: JOB
steps:
- name: Example
env:
STEP_VARIABLE: STEP
run: |
echo "Hello World"
echo "This is the $WORKFLOW_VARIABLE environment variable"
echo "This is the $JOB_VARIABLE environment variable"
echo "This is the $STEP_VARIABLE environment variable"
But depending on what you plan to achieve, a matrix strategy you be more efficient.
So I have two secrets: DEV_SERVER_IP and MASTER_SERVER_IP.
in main.yml I need something like this
run: echo "::set-env name=BRANCH_NAME::$(echo ${GITHUB_REF#refs/heads/} | sed 's/\//_/g')"
run: ssh-keyscan -H ${{ secrets.BRANCH_NAME_SERVER_IP }} >> ~/.ssh/known_hosts
but am getting error
env:
BRANCH_NAME: dev
Error: Input required and not supplied: key
I need here something like this ssh-keyscan -H ${{ secrets.${BRANCH_NAME}_SERVER_IP }}
how can I fix this?
You're trying to use shell style logic inside a Github context
expansion (${{ ... }}) which won't work. Just move all your logic
into your shell script instead:
name: Example
on:
push:
jobs:
example:
runs-on: ubuntu-latest
steps:
- name: get target ip
env:
DEV_SERVER_IP: ${{ secrets.DEV_SERVER_IP }}
MAIN_SERVER_IP: ${{ secrets.MAIN_SERVER_IP }}
run: |
branch_name=$(sed 's|/|_|g' <<< ${GITHUB_REF#refs/heads/})
target="${branch_name^^}_SERVER_IP"
mkdir -p ~/.ssh
ssh-keyscan -H ${!target} >> ~/.ssh/known_hosts
cat ~/.ssh/known_hosts
In the above workflow, the expression ${branch_name^^} is a bash expression that returns the value of $branch_name in uppercase, and ${!target} is a bash expression that returns the value of the variable who name is stored in $target.
Note that I'm not using your "set the BRANCH_NAME environment variable"
task because the ::set-env command is disabled by default for
security reasons.
(Added bash and terminal tags since I'm unsure if my issue is specific to Github actions specifically or if instead is a misunderstanding on how env vars work more generally)
I'm working on a workflow.yml and in a step "Env substitue in sql script" am trying to set some env vars:
on: [push]
env:
GAME: "FunGame"
TRAIN_HORIZON: 7
jobs:
ssql:
runs-on: ubuntu-latest
name: Get data
steps:
- name: Checkout cum-rev repo
uses: actions/checkout#v2 # Defaults to current repo - check out current repo
- name: Checkout ds-ssql-gh-action
uses: actions/checkout#v2
with:
repository: ourorg/ds-ssql-gh-action
token: ${{ secrets.cumrev_workflow_token }}
ref: main
path: './ds-ssql-gh-action'
- name: Env substitue in sql script
run: |
INSTALL_DATE=$(date -d "`date +%Y%m01` -12 month" +%Y-%m-%d)
echo "Here is install date $INSTALL_DATE"
IOS_GAME="${{ env.GAME }}_IOS_PROD"
ANDROID_GAME="${{ env.GAME }}_ANDROID_PROD"
envsubst < get-data/training-data.sql
cat get-data/training-data.sql
printenv
After pushing this the job attempts to run. I printenv at the bottom and when I see the env variables, I don't see any of INSTALL_DATE, IOS_GAME or ANDROID_GAME.
Why are those env variables not being set with the lines:
INSTALL_DATE=$(date -d "`date +%Y%m01` -12 month" +%Y-%m-%d)
echo "Here is install date $INSTALL_DATE"
IOS_GAME="${{ env.GAME }}_IOS_PROD"
ANDROID_GAME="${{ env.GAME }}_ANDROID_PROD"
Note line echo "Here is install date $INSTALL_DATE" does indeed print out the correct value as expected. But it's not showing when I run printenv?
You have to export the variables you want to see in the environment:
export INSTALL_DATE=$(date -d "`date +%Y%m01` -12 month" +%Y-%m-%d)
...
We're using Ansible playbook with GitLab CI in this project, where we'd pass some variables from ENV_FILE through Ansible playbook, then rendering JJ2 template with them.
Now the problem occurs when some variable has $ in its value, which seems interpreted as shell variable at some point, and the final value is rendered incorrect.
For example, in ENV_FILE
(set via GitLab CI Settings > CI/CD > Variables menu):
export FIRST_VAR=...
export SOME_VAR='123$abc#xyz'
export SOME_OTHER_VAR=...
And the final result in docker-compose.yaml becomes 123#xyz
EDIT: We just tried changing to export SOME_VAR='123''$''abc#xyz', the final result becomes 123abc#xyz, still missing the $.
gitlab-ci.yaml
deploy:
stage: deploy
environment:
name: dev
script:
- source $ENV_FILE
- cd ansible && ansible-playbook -i inventory/dev.ini runapp.yaml --vault-password-file=${ANSIBLE_VAULT_FILE}
runapp.yaml
- hosts: app
become: yes
roles:
- { role: some_app }
vars:
SOME_VAR: "{{ lookup('env', 'SOME_VAR') }}"
Task File:
- name: "Templating docker-compose file"
become: yes
template:
src: app-docker-compose.yaml.j2
dest: /opt/someapp/docker-compose.yaml
app-docker-compose.yaml.j2
someapp-svc:
image: someapp:version
restart: always
ports:
- ####:####
environment:
SOME_VAR: {{ SOME_VAR }}
Any hint about this?
Thanks!
I can reproduce that behavior when setting a CI/CD variable containing $; the docs kind of hint at it, although the docs are written as if the problem only applies when setting variables inside .gitlab-ci.yml which is demonstrably false
If you want a CI/CD variable to contain a literal $, it needs to be doubled, so SOME_VAR would need to be written as 123$$abc#xyz in the CI/CD configuration page in order for it to materialize as 123$abc#xyz inside the pipeline (although as the comments correctly point out, one will want to be exceedingly careful about the use of source to avoid further interpolation)
I'm not able to access environment variables defined at the top-level of a GitHub Action configuration file from within a script run by the action.
For example, given the following config file:
name: x-pull-request
on: pull_request
env:
FOO: bar
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: does a thing
run: ./scripts/do-a-thing.sh
... and the following script:
X=${FOO:default}
echo "X: $X" # X: default
The FOO environment variable defined in the config file is not available to the script and the default value is being used.
So, how can I access the environment variable from a Bash script run by the build step? Am I missing a prefix or something? (I know values defined in the input hash require you to use an INPUT_ prefix when referencing them.)
You can use env at any level also in jobs and steps.
I wrote a test action and a test script to validate it:
The action file:
name: Env tests
on: push
env:
FOO_ROOT: bar on root
jobs:
test:
runs-on: ubuntu-latest
env:
FOO_JOB: bar on job
steps:
- uses: actions/checkout#v1
- name: Test envs
run: ./env-test.sh
env:
FOO_STEP: bar on step
The script file:
#!/usr/bin/env bash
echo "FOO_ROOT: $FOO_ROOT"
echo "FOO_JOB: $FOO_JOB"
echo "FOO_STEP: $FOO_STEP"
echo " "
printenv
The results:
FOO_ROOT: bar on root
FOO_JOB: bar on job
FOO_STEP: bar on step
LEIN_HOME=/usr/local/lib/lein
M2_HOME=/usr/share/apache-maven-3.6.3
...
Check my results and, in fact, I don't know why it didn't work on your side because it must work.
If some one else is searching for a solution you have to explicitly pass the environment variables to the bash script. Otherwise they are not available:
name: x-pull-request
on: pull_request
env:
FOO: bar
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: does a thing
run: ./scripts/do-a-thing.sh $FOO
and in the script you have to map the parameter to a variable.
X=$1
echo "X: $X" # X: default