So I have two secrets: DEV_SERVER_IP and MASTER_SERVER_IP.
in main.yml I need something like this
run: echo "::set-env name=BRANCH_NAME::$(echo ${GITHUB_REF#refs/heads/} | sed 's/\//_/g')"
run: ssh-keyscan -H ${{ secrets.BRANCH_NAME_SERVER_IP }} >> ~/.ssh/known_hosts
but am getting error
env:
BRANCH_NAME: dev
Error: Input required and not supplied: key
I need here something like this ssh-keyscan -H ${{ secrets.${BRANCH_NAME}_SERVER_IP }}
how can I fix this?
You're trying to use shell style logic inside a Github context
expansion (${{ ... }}) which won't work. Just move all your logic
into your shell script instead:
name: Example
on:
push:
jobs:
example:
runs-on: ubuntu-latest
steps:
- name: get target ip
env:
DEV_SERVER_IP: ${{ secrets.DEV_SERVER_IP }}
MAIN_SERVER_IP: ${{ secrets.MAIN_SERVER_IP }}
run: |
branch_name=$(sed 's|/|_|g' <<< ${GITHUB_REF#refs/heads/})
target="${branch_name^^}_SERVER_IP"
mkdir -p ~/.ssh
ssh-keyscan -H ${!target} >> ~/.ssh/known_hosts
cat ~/.ssh/known_hosts
In the above workflow, the expression ${branch_name^^} is a bash expression that returns the value of $branch_name in uppercase, and ${!target} is a bash expression that returns the value of the variable who name is stored in $target.
Note that I'm not using your "set the BRANCH_NAME environment variable"
task because the ::set-env command is disabled by default for
security reasons.
Related
Under my Git Repo Settings > Environments : I define an env name PPM_DEV, under this env PPM_DEV I define an environment variable named HOSTNAME and give it a value in the git configurations page.
Now what is the YAML syntax under GitActions WF to read the value of this variable ?
basically I may have 2 Env's defined PPM_DEV , PPM_TEST
but where do I set the Env context to pull the variable HOSTNAME from the PPM_DEV env ?
In the example below , I am trying to populate a variable VARHOSTNAME with the value of the Env variable HOSTNAME that is pre-defined against the Env named PPM_DEV
However it fails with
The workflow is not valid. .github/workflows/Ext_Conn_v2.yml (Line: 10, Col: 7): Unexpected value 'VARHOSTNAME'
name: Ext_Conn_v2
on:
workflow_dispatch:
jobs:
RunonVM:
runs-on: ubuntu-latest
environment:
name: ppm_dev
VARHOSTNAME: ${{ env.HOSTNAME }}
steps:
- name: Run a command
run: |
echo "This workflow was manually triggered."
echo "value of the variable HOSTNAME: " ${VARHOSTNAME}
pwd
echo "end of run"
You need to use jobs.<job_id>.steps[*].env to specify the environment variables or secrets using vars or secrets contexts:
Here's an example with the secrets context:
jobs:
job:
runs-on: ubuntu-latest
environment: ppm_dev
steps:
- name: Command
env:
HOSTNAME: ${{ secrets.HOSTNAME }}
run: |
echo "HOSTNAME: $HOSTNAME"
See a linted example here.
(Added bash and terminal tags since I'm unsure if my issue is specific to Github actions specifically or if instead is a misunderstanding on how env vars work more generally)
I'm working on a workflow.yml and in a step "Env substitue in sql script" am trying to set some env vars:
on: [push]
env:
GAME: "FunGame"
TRAIN_HORIZON: 7
jobs:
ssql:
runs-on: ubuntu-latest
name: Get data
steps:
- name: Checkout cum-rev repo
uses: actions/checkout#v2 # Defaults to current repo - check out current repo
- name: Checkout ds-ssql-gh-action
uses: actions/checkout#v2
with:
repository: ourorg/ds-ssql-gh-action
token: ${{ secrets.cumrev_workflow_token }}
ref: main
path: './ds-ssql-gh-action'
- name: Env substitue in sql script
run: |
INSTALL_DATE=$(date -d "`date +%Y%m01` -12 month" +%Y-%m-%d)
echo "Here is install date $INSTALL_DATE"
IOS_GAME="${{ env.GAME }}_IOS_PROD"
ANDROID_GAME="${{ env.GAME }}_ANDROID_PROD"
envsubst < get-data/training-data.sql
cat get-data/training-data.sql
printenv
After pushing this the job attempts to run. I printenv at the bottom and when I see the env variables, I don't see any of INSTALL_DATE, IOS_GAME or ANDROID_GAME.
Why are those env variables not being set with the lines:
INSTALL_DATE=$(date -d "`date +%Y%m01` -12 month" +%Y-%m-%d)
echo "Here is install date $INSTALL_DATE"
IOS_GAME="${{ env.GAME }}_IOS_PROD"
ANDROID_GAME="${{ env.GAME }}_ANDROID_PROD"
Note line echo "Here is install date $INSTALL_DATE" does indeed print out the correct value as expected. But it's not showing when I run printenv?
You have to export the variables you want to see in the environment:
export INSTALL_DATE=$(date -d "`date +%Y%m01` -12 month" +%Y-%m-%d)
...
I have a cross-platform project which is to be built on 2 platforms: mac and linux(ubuntu).
My pipeline contains 3 jobs:
prepare docker image with all nessesary too to build the project.
build on ubuntu in prepared docker container, depends on step 1
build on MacOS, needs nothing
Steps for linux and macos are definitely the same. But matrixes differs much, and linux build is
run inside container.
Is there a way to share steps between two different jobs?
I tried YAML anchors but GitHub does not support them.
Full workflow
on:
push:
branches: [ main, support/1.2.x ]
pull_request:
branches: [ main, support/1.2.x ]
jobs:
Docker-iroha-builder:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout#v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action#v1
-
name: Cache Docker layers
uses: actions/cache#v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
-
name: Login to DockerHub
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push
uses: docker/build-push-action#v2
with:
file: docker/develop/Dockerfile.builder
# context: .
push: true
tags: ${{ secrets.DOCKERHUB_ORG }}/iroha:builder
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new
-
# Temp fix
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
build-iroha-ubuntu:
needs: Docker-iroha-builder
runs-on: ubuntu-latest
container: ikyb/iroha:builder
strategy:
fail-fast: false
matrix:
cc: [ gcc-9, gcc-10, clang ] ##todo g++-10
USE_BURROW: [ -DUSE_BURROW=OFF ]
debrel: [ Debug ] #,Release, RelWithDebInfo
steps:
- ## Takes 22 seconds with default github runner
name: Homebrew
run: brew install cmake ninja coreutils
if: ${{ runner.os == 'MacOS' }}
-
name: Checkout
uses: actions/checkout#v2
-
name: Cache vcpkg
uses: actions/cache#v2
with:
path: |
build-vcpkg
build/vcpkg_installed
$HOME/.cache/vcpkg
key: ${{ runner.os }}-vcpkg-${{ github.sha }}
restore-keys: ${{ runner.os }}-vcpkg-
-
name: Build Iroha vcpkg dependancies
run: ./vcpkg/build_iroha_deps.sh $PWD/build-vcpkg
-
name: CMake configure
run: |
export CC=${{ matrix.cc }} CXX=$(echo ${{ matrix.cc }} | sed -es,gcc,g++, -es,clang,clang++,)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=$PWD/build-vcpkg/scripts/buildsystems/vcpkg.cmake \
${{ matrix.USE_BURROW }} -GNinja #-DCMAKE_VERBOSE_MAKEFILE=ON
-
name: CMake build
run: cmake --build build --config ${{ matrix.debrel }}
build-iroha-macos:
runs-on: macos-latest
strategy:
fail-fast: false
matrix:
USE_BURROW: [ -DUSE_BURROW=OFF ]
debrel: [ Debug,Release ]
steps:
- ## Takes 22 seconds with default github runner
name: Homebrew
run: brew install cmake ninja coreutils
if: ${{ runner.os == 'MacOS' }}
-
name: Checkout
uses: actions/checkout#v2
-
name: Cache vcpkg
uses: actions/cache#v2
with:
path: |
build-vcpkg
build/vcpkg_installed
$HOME/.cache/vcpkg
key: ${{ runner.os }}-vcpkg-${{ github.sha }}
restore-keys: ${{ runner.os }}-vcpkg-
-
name: Build Iroha vcpkg dependancies
run: ./vcpkg/build_iroha_deps.sh $PWD/build-vcpkg
-
name: CMake configure
run: |
export CC=${{ matrix.cc }} CXX=$(echo ${{ matrix.cc }} | sed -es,gcc,g++, -es,clang,clang++,)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=$PWD/build-vcpkg/scripts/buildsystems/vcpkg.cmake \
${{ matrix.USE_BURROW }} -GNinja #-DCMAKE_VERBOSE_MAKEFILE=ON
-
name: CMake build
run: cmake --build build --config ${{ matrix.debrel }}
TL;DR
I solved my problem with shell tool yq
yq eval 'explode(.)' file.yml
The repository with example usage and detailed description https://github.com/kuvaldini/make-workflows.sh may be helpful to easy start. It was make from this answer. Pay attention to Actions tab.
Long answer
GitHub Workflow description in YAML does not support anchors.
There are several workarounds => anyway they come to building-editing workflow yaml from source.
So I suggest yet another one make-workflows.sh based on YAML tool yq.
USAGE
Move your workflows to .github/*.src.yml
Put make-workflows.sh to directory .github/
(optional) Copy or link pre-commit.sh to .git/hooks/pre-commit
Like ln -s ../../.github/pre-commit.sh .git/hooks/pre-commit
File make-workflows.sh
#!/usr/bin/env bash
set -euo pipefail
## The script expands '*.src.yml' from $1(default: script's directory)
## to $2 (default:subdirectory 'workflows') with corresponding name '*.yml'
## Main goal is to dereference YAML anchors.
## Deals only with Git cached/indexed files
## Set -x to debug
script_dir=$(dirname $(realpath "$0"))
dir_from=${1:-${script_dir}}
dir_to=${2:-workflows}
cd $dir_from
edited=
for f in $(git status -s -- \*.src.yml | sed 's,^.. ,,') ;do
readonly out=$(echo $f | sed s,.src.yml\$,.yml,)
readonly wout=$dir_to/$out
readonly tempout=$(mktemp)
trap "rm -f $tempout" EXIT
echo >>$tempout "## DO NOT EDIT"
echo >>$tempout "## Generated from $f with $(basename $0)"
echo >>$tempout ""
yq eval 'explode(.)' $f >>$tempout
if ! diff -q $wout $tempout &>/dev/null ;then
mv $tempout $wout
edited+="'$out' "
fi
done
if [[ -n "$edited" ]]
then echo >&2 "make-workflows: these files were edited: $edited"
else echo >&2 "make-workflows: everything is up to date"
fi
File pre-commit.sh
#!/usr/bin/env bash
set -euo pipefail
gitroot=$(git rev-parse --show-toplevel)
cd $gitroot
./.github/make-workflows.sh
git add .github/workflows
Links
ready to use solution with detailed description https://github.com/kuvaldini/make-workflows.sh
Share same steps for different GitHub Actions jobs
https://github.community/t/support-for-yaml-anchors/16128/60
https://github.com/mithro/actions-includes
https://github.com/allejo/gha-workflows
While github actions does not support YAML anchors directly, one can expand those e.g. by converting from YAML to JSON and then back to YAML. I am doing this here (Makefile in .github/workflows): https://github.com/agda/agda/blob/557681d04aae2100ccde2e045a8afcf30528c3a5/.github/workflows/Makefile
srcpath=../../src/github/workflows
sources=$(wildcard $(srcpath)/*.yml $(srcpath)/*.yaml)
targets=$(sort $(notdir $(sources)))
all : $(targets)
# Normalize YAML files by going via JSON.
# This expands anchors which are not understood by github workflows.
% : $(srcpath)/%
yaml2json $< | json2yaml - > $#
An example for a workflow file with anchors is here: https://github.com/agda/agda/blob/557681d04aae2100ccde2e045a8afcf30528c3a5/src/github/workflows/test.yml
jobs:
build:
runs-on: &runs_on ubuntu-22.04
steps:
- &checkout
uses: actions/checkout#v3
- &haskell_setup
uses: haskell/actions/setup#v2
with:
ghc-version: ${{ env.GHC_VER }}
...
test:
needs: build
runs-on: *runs_on
steps:
- *checkout
- *haskell_setup
...
I have a workflow where after a push to master I want to create a release and upload an asset to it.
I'm using actions/create-release#v1 and actions/upload-release-asset#v1.
I would like to pass the outputs of a bash commands to the action parameters. However I found out the syntax of "$(command)" does not work.
How can I pass the output of a bash command to an action's parameter.
For example I'd like to do something like this:
- name: Create Release
id: create_release
uses: actions/create-release#v1
env:
GITHUB_TOKEN: ${{ secrets.token }}
with:
tag_name: $(cat projectFile | grep -Po '(?<=Version>).*(?=</Version>)')
Update: set-output is being deprecated as well "Starting 1st June 2023 workflows using save-state or set-output commands via stdout will fail with an error."
Now that set-env is deprecated and set-output is soon to be deprecated, you can use GITHUB_OUTPUT environment files: to accomplish the same thing in this answer
- name: Retrieve version
run: |
echo "TAG_NAME=$(cat projectFile | grep -Po '(?<=Version>).*(?=</Version>)')" >> $GITHUB_OUTPUT
id: version
- name: Create Release
id: create_release
uses: actions/create-release#v1
env:
GITHUB_TOKEN: ${{ secrets.token }}
with:
tag_name: ${{ steps.version.outputs.TAG_NAME }}
References:
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#environment-files
How to save the output of a bash command to output parameter in github actions
Use environment files
steps:
- name: Set the value
id: step_one
run: |
echo "FOO=$(git status)" >> $GITHUB_ENV
- name: Use the value
id: step_two
run: |
echo "${{ env.FOO }}"
UPDATE: This answer will not work as GitHub as disabled this syntax for security reasons. You should use environment files instead.
I would create an environment variable based of your command output:
- name: Retrieve version
run: |
echo ::set-env name=TAG_NAME::$(cat projectFile | grep -Po '(?<=Version>).*(?=</Version>)')
And then access it like the following:
- name: Create Release
id: create_release
uses: actions/create-release#v1
env:
GITHUB_TOKEN: ${{ secrets.token }}
with:
tag_name: ${{ env.TAG_NAME }}
I'm trying to define an ansible playbook to checkout several sources from CVS, just similarly to what I do with the git module (http://docs.ansible.com/ansible/git_module.html). Unfortunately there's no cvs module as far as I can tell.
Is there any recommended way to go about it?
If you could use SSH for login this is more or less an example to get you started:
- name: Remove cvs mirror host key
shell: ssh-keygen -R {{ openbsd_cvs_mirror }}
- name: Add cvs mirror host key
shell: ssh-keyscan -H {{ openbsd_cvs_mirror }} >> /root/.ssh/known_hosts
- name: Get OpenBSD cvs src
shell: cvs -qd anoncvs#{{ openbsd_cvs_mirror }}:/cvs checkout -P src
args:
chdir: /usr/cvs_current
creates: /usr/cvs_current/src