I am facing issue while sending notification to microsoft teams using below github action workflow YAML. As you can see in first job i am using right "ls -lrt" command and when this job1 succeeded then i got success notification in teams but to get failed notification, i purposefully removed hypen (-) from "ls lrt" command so that second job can fail and i can get fail notification. Overall idea is any job fail or success, i must get notification. But this is not happening for me actually. Any guidance and help would be appreciated.
name: msteams
on: push
jobs:
job1:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: test run
run: ls -lrt
- name: "testing_ms"
if: always()
uses: ./.github/actions
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- uses: actions/checkout#v2
- name: test run
run: ls lrt
- name: "testing ms"
if: always()
uses: ./.github/actions
As in above YAML you can see i am using uses: ./.github/actions so i kept below mentioned code in another YAML file and kept in .github/actions folder parallel to my above main github action workflow YAML.
name: 'MS Notification'
description: 'Notification to MS Teams'
runs:
using: "composite"
steps:
- id: notify
shell: bash
run: |
echo "This is for testing"
# step logic
# Specific to this workflow variables set
PIPELINE_PUBLISHING_NAME="GitHub Actions Workflows Library"
BRANCH_NAME="${GITHUB_REF#refs/*/}"
PIPELINE_TEAMS_WEBHOOK=${{ secrets.MSTEAMS_WEBHOOK }}
# Common logic for notifications
TIME_STAMP=$(date '+%A %d %B %Y, %r - %Z')
GITHUBREPO_OWNER=$(echo ${GITHUB_REPOSITORY} | cut -d / -f 1)
GITHUBREPO_NAME=${GITHUB_REPOSITORY}
GITHUBREPO_URL=${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}
SHA=${GITHUB_SHA}
SHORT_SHA=${SHA::7}
RUN_ID=${GITHUB_RUN_ID}
RUN_NUM=${GITHUB_RUN_NUMBER}
AUTHOR_AVATAR_URL="${{github.event.sender.avatar_url}}"
AUTHOR_HTML_URL="${{github.event.sender.url}}"
AUTHOR_LOGIN="${{github.event.sender.login}}"
COMMIT_HTML_URL="${GITHUBREPO_URL}/commit/${SHA}"
COMMIT_AUTHOR_NAME="${{github.event.sender.login}}"
case ${{ job.status }} in
failure )
NOTIFICATION_COLOR="dc3545"
NOTIFICATION_ICON="❌"
NOTIFICATION_STATUS="FAILURE"
;;
success )
NOTIFICATION_COLOR="28a745"
NOTIFICATION_ICON="✅"
NOTIFICATION_STATUS="SUCCESS"
;;
cancelled )
NOTIFICATION_COLOR="17a2b8"
NOTIFICATION_ICON="✖"
NOTIFICATION_STATUS="CANCELLED"
;;
*)
NOTIFICATION_COLOR="778899"
NOTIFICATION_ICON=❔""
NOTIFICATION_STATUS="UNKOWN"
;;
esac
# set pipeline version information if available
if [[ '${{ env.CICD_PIPELINE_VERSION}}' != '' ]];then
PIPELINE_VERSION="(v. ${{ env.CICD_PIPELINE_VERSION}})"
else
PIPELINE_VERSION=""
fi
NOTIFICATION_SUMARY="${NOTIFICATION_ICON} ${NOTIFICATION_STATUS} - ${PIPELINE_PUBLISHING_NAME} [ ${BRANCH_NAME} branch ] >> ${{ github.workflow }} ${PIPELINE_VERSION} "
TEAMS_WEBHOOK_URL="${PIPELINE_TEAMS_WEBHOOK}"
# addtional sections can be added to specify additional, specific to its workflow, information
message-card_json_payload() {
cat <<EOF
{
"#type": "MessageCard",
"#context": "https://schema.org/extensions",
"summary": "${NOTIFICATION_SUMARY}",
"themeColor": "${NOTIFICATION_COLOR}",
"title": "${NOTIFICATION_SUMARY}",
"sections": [
{
"activityTitle": "**CI #${RUN_NUM} (commit [${SHORT_SHA}](COMMIT_HTML_URL))** on [${GITHUBREPO_NAME}](${GITHUBREPO_URL})",
"activitySubtitle": "by ${COMMIT_AUTHOR_NAME} [${AUTHOR_LOGIN}](${AUTHOR_HTML_URL}) on ${TIME_STAMP}",
"activityImage": "${AUTHOR_AVATAR_URL}",
"markdown": true
}
],
"potentialAction": [
{
"#type": "OpenUri",
"name": "View Workflow Run",
"targets": [{
"os": "default",
"uri": "${GITHUBREPO_URL}/actions/runs/${RUN_ID}"
}]
},
{
"#type": "OpenUri",
"name": "View Commit Changes",
"targets": [{
"os": "default",
"uri": "${COMMIT_HTML_URL}"
}]
}
]
}
EOF
}
echo "NOTIFICATION_SUMARY ${NOTIFICATION_SUMARY}"
echo "------------------------------------------------"
echo "MessageCard payload"
echo "$(message-card_json_payload)"
echo "------------------------------------------------"
HTTP_RESPONSE=$(curl -s -H "Content-Type: application/json" \
--write-out "HTTPSTATUS:%{http_code}" \
--url "${TEAMS_WEBHOOK_URL}" \
-d "$(message-card_json_payload)"
)
echo "------------------------------------------------"
echo "HTTP_RESPONSE $HTTP_RESPONSE"
echo "------------------------------------------------"
# extract the body
HTTP_BODY=$(echo $HTTP_RESPONSE | sed -e 's/HTTPSTATUS\:.*//g')
# extract the status
HTTP_STATUS=$(echo $HTTP_RESPONSE | tr -d '\n' | sed -e 's/.*HTTPSTATUS://')
if [ ! $HTTP_STATUS -eq 200 ]; then
echo "::error::Error sending MS Teams message card request [HTTP status: $HTTP_STATUS]"
# print the body
echo "$HTTP_BODY"
exit 1
fi
I don't know the entire answer for you, but right off I see the composite action trying to read secrets, which composite actions don't support. Try setting input params to the composite actions to pass in what you need.
Related
I am creating a git-hub workflow using workflow_dispatch for my inputs data
workflow_dispatch:
inputs:
Browser:
description: 'Select what browsers to use'
required: true
default: 'Chrome'
And I got my setup job, where I take my workflow_dispatch data and trasfer them to json, so I can use it in my matrix
jobs:
setup:
runs-on: ubuntu-latest
outputs:
matrix-browser: ${{ steps.set-matrix-browser.outputs.matrix-browser }}
steps:
- uses: actions/checkout#v2
- id: set-matrix-browser
run: |
testBro="${{ github.event.inputs.Browser }}"
echo "::set-output name=matrix-browser::[\"${testBro}\"]"
echo "[\"${testBro}\"]"
So, my question is:
If I got two or more browsers in my github.event.inputs.Browser = "Chrome, Safari, Edge", who I can to split them, for each browser to be a separate string.
I want my output to look like this,
["Chrome", "Safari, "Edge"]
but instead I got this
["Chrome, Safari, Edge"]
Can you please suggest how I need to change this line of code?
echo "::set-output name=matrix-browser::[\"${testBro}\"]"
I've tried something like this:
echo "::set-output name=matrix-browser::[\"${testBro | tr "," "\n"}\"]"
Assuming testBro="Chrome,Safari,Edge"
echo [\"$testBro\"] |sed 's/,/\", \"/g' or for the full line echo "::set-output name=matrix-browser:: $(echo [\"$testBro\"] |sed 's/,/\", \"/g')"
gives the output
::set-output name=matrix-browser:: ["Chrome", "Safari", "Edge"]
I figured it
echo "::set-output name=matrix-browser::[\"${testBro//', '/\",\"}\"]"
Thanks everybody
I'm attempting to pass an array of dynamically fetched data from one GitHub Action job to the actual job doing the build. This array will be used as part of a matrix to build for multiple versions. However, I'm encountering an issue when the bash variable storing the array is evaluated.
jobs:
setup:
runs-on: ubuntu-latest
outputs:
versions: ${{ steps.matrix.outputs.value }}
steps:
- id: matrix
run: |
sudo apt-get install -y jq && \
MAINNET=$(curl https://api.mainnet-beta.solana.com -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id":1, "method":"getVersion"}' | jq '.result["solana-core"]') && \
TESTNET=$(curl https://api.testnet.solana.com -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id":1, "method":"getVersion"}' | jq '.result["solana-core"]') && \
VERSIONS=($MAINNET $TESTNET) && \
echo "${VERSIONS[#]}" && \
VERSION_JSON=$(echo "${VERSIONS[#]}" | jq -s) && \
echo $VERSION_JSON && \
echo '::set-output name=value::$VERSION_JSON'
shell: bash
- id: debug
run: |
echo "Result: ${{ steps.matrix.outputs.value }}"
changes:
needs: setup
runs-on: ubuntu-latest
# Set job outputs to values from filter step
outputs:
core: ${{ steps.filter.outputs.core }}
package: ${{ steps.filter.outputs.package }}
strategy:
matrix:
TEST: [buy, cancel, create_auction_house, delegate, deposit, execute_sale, sell, update_auction_house, withdraw_from_fee, withdraw_from_treasury, withdraw]
SOLANA_VERSION: ${{fromJson(needs.setup.outputs.versions)}}
steps:
- uses: actions/checkout#v2
# For pull requests it's not necessary to checkout the code
- uses: dorny/paths-filter#v2
id: filter
with:
filters: |
core:
- 'core/**'
package:
- 'auction-house/**'
- name: debug
id: debug
working-directory: ./auction-house/program
run: echo ${{ needs.setup.outputs.versions }}
In the setup job above, the two versions are evaluated to a bash array (in VERSIONS) and converted into a JSON array to be passed to the next job (in VERSION_JSON). The last echo in the matrix step results in a print of [ "1.10.31", "1.11.1" ], but the debug step prints out this:
Run echo "Result: "$VERSION_JSON""
echo "Result: "$VERSION_JSON""
shell: /usr/bin/bash -e {0}
env:
CARGO_TERM_COLOR: always
RUST_TOOLCHAIN: stable
Result:
The changes job also results in an error:
Error when evaluating 'strategy' for job 'changes'.
.github/workflows/program-auction-house.yml (Line: 44, Col: 25): Unexpected type of value '$VERSION_JSON', expected type: Sequence.
It definitely seems like the $VERSION_JSON variable isn't actually being evaluated properly, but I can't figure out where the evaluation is going wrong.
For echo '::set-output name=value::$VERSION_JSON' you need to use double quotes or bash would not expand $VERSION_JSON.
set-output is not happy with multi-lined data. For your case, you can use jq -s -c so the output will be one line.
I am running my Github Action on a self-hosted machine. The action should send a curl command:
name: Build and Publish
on:
push:
branches:
- master
jobs:
trigger:
runs-on: self-hosted
steps:
- uses: actions/checkout#v2
- name: curl command
working-directory: /home/ubuntu/actions-runner/_work/${{ github.repository }}
run: |
DATA= '{"data": { "repository" : "${{ github.repository }}", "package_version" : "0.1.0-${{ github.sha }}", "branch" : "${{ github.sha }}" }}'
printf -v content "$DATA"
curl --request 'POST' --data "$content" 'http://<my-url>/actions'
The output seems strange as it is looking for a certain path with an .sh script and fails on "no such file or directory"
Run DATA= '{"data": { "repository" : "iz/<my-repo>", "package_version" : "0.1.0-41b1005d15069bbb515b52416721b84", "branch" : "41b1005d15069bbb515b52416721b84" }}'
DATA= '{"data": { "repository" : "iz/<my-repo>", "package_version" : "0.1.0-41b1005d15069bbb515b52416721b84", "branch" : "41b1005d15069bbb515b52416721b84" }}'
printf -v content "$DATA"
curl --request 'POST' --data "$content" 'http://<my-url>/actions'
shell: /usr/bin/bash -e {0}
/home/ubuntu/actions-runner/_work/_temp/5fee3c-a409-4db-48f-cb1d16d15d.sh: line 1: {"data": { "repository" : "iz/<my-repo>", "package_version" : "0.1.0-41b1005d15069bbb515b52416721b84", "branch" : "41b1005d15069bbb515b52416721b84" }}:
No such file or directory
Error: Process completed with exit code 127.
Try removing the space after DATA=. It thinks you are temporarily setting DATA to empty for executing the command '{"data": { "repository" :... instead of setting DATA to that value permanently (i.e. non-temporarily).
Set X to Y (until overwritten):
X=Y
Set X to Y temporarily during execution of CMD:
X=Y CMD
I have job in my gitlab-cicd.yml file:
unit_test:
stage: test
image: $MAVEN_IMAGE
script:
- *tests_variables_export
- mvn ${MAVEN_CLI_OPTS} clean test
- cat ${CI_PROJECT_DIR}/rest-service/target/site/jacoco/index.html
- cat ${CI_PROJECT_DIR}/soap-service/target/site/jacoco/index.html
artifacts:
expose_as: 'code coverage'
paths:
- ${CI_PROJECT_DIR}/soap-service/target/surefire-reports/
- ${CI_PROJECT_DIR}/rest-service/target/surefire-reports/
- ${CI_PROJECT_DIR}/soap-service/target/site/jacoco/index.html
- ${CI_PROJECT_DIR}/rest-service/target/site/jacoco/index.html
And I want to change it to this one:
unit_test:
stage: test
image: $MAVEN_IMAGE
script:
- *tests_variables_export
- mvn ${MAVEN_CLI_OPTS} clean test
- cat ${CI_PROJECT_DIR}/rest-service/target/site/jacoco/index.html
- cat ${CI_PROJECT_DIR}/soap-service/target/site/jacoco/index.html
artifacts:
expose_as: 'code coverage'
paths:
- *resolve_paths
I try to use this bash script:
.resolve_paths: &resolve_paths |-
if [ "${MODULE_FIRST}" != "UNKNOWN" ]; then
echo "- ${CI_PROJECT_DIR}/${MODULE_FIRST}/target/surefire-reports/"
echo "- ${CI_PROJECT_DIR}/${MODULE_FIRST}/target/site/jacoco/index.html"
fi
if [ "${MODULE_SECOND}" != "UNKNOWN" ]; then
echo "- ${CI_PROJECT_DIR}/${MODULE_SECOND}/target/surefire-reports/"
echo "- ${CI_PROJECT_DIR}/${MODULE_SECOND}/target/site/jacoco/index.html"
fi
And right now I'm getting this error in pipeline:
WARNING: if [ "rest-service" != "UNKNOWN" ]; then\n echo "- /builds/minv/common/testcommons/taf-api-support/rest-service/target/surefire-reports/"\n echo "- /builds/minv/common/testcommons/taf-api-support/rest-service/target/site/jacoco/index.html"\nfi\nif [ "soap-service" != "UNKNOWN" ]; then\n echo "- /builds/minv/common/testcommons/taf-api-support/soap-service/target/surefire-reports/"\n echo "- /builds/minv/common/testcommons/taf-api-support/soap-service/target/site/jacoco/index.html"\nfi: no matching files ERROR: No files to upload
Can I execute [sic] paths using bash script like this?
No, scripts cannot alter the current YAML, particularly not if you specify the script (which is just a string) in a place where it is interpreted as a path.
You could trigger a dynamically generated YAML:
generate:
stage: build
script:
- |
exec > generated.yml
echo ".resolve_paths: &resolve_paths"
for module in "${MODULE_FIRST}" "${MODULE_SECOND}"; do
[[ "$module" = UNKNOWN ]] && continue
echo "- ${CI_PROJECT_DIR}/${module}/target/surefire-reports/"
echo "- ${CI_PROJECT_DIR}/${module}/target/site/jacoco/index.html"
done
sed '1,/^\.\.\. *$/d' "${CI_CONFIG_PATH}"
artifacts:
paths:
- generated.yml
run:
stage: deploy
trigger:
include:
- artifact: generated.yml
job: generate
...
# Start of actual CI. When this runs, there will be an
# auto-generated job `.resolve_paths: &resolve_paths`.
# Put the rest of your CI (e.g. `unit_test:`) here.
But there are so many extensions in GitLab's YAML that you likely will find a tremendously better solution, which depends on what you plan to do with .resolve_paths. Maybe have a look at
artifacts:exclude
additional jobs with rules:
Background:
I'm using an AWS CodeBuild buildspec.yml to iterate through directories from a GitHub repo. Before looping through the directory path $TF_ROOT_DIR, I'm using a bash if statement to check if the GitHub branch name $BRANCH_NAME is within an env variable $LIVE_BRANCHES. As you can see in the error screenshot below, the bash if statement outputs the error: syntax error: bad substitution. When I reproduce the if statement within a local bash script, the if statement works as it's supposed to.
Here's the env variables defined in the CodeBuild project:
Here's a relevant snippet from the buildspec.yml:
version: 0.2
env:
shell: bash
phases:
build:
commands:
- |
if [[ " ${LIVE_BRANCHES[*]} " == *"$BRANCH_NAME"* ]]; then
# Iterate only through BRANCH_NAME directory
TF_ROOT_DIR=${TF_ROOT_DIR}/*/${BRANCH_NAME}/
else
# Iterate through both dev and prod directories
TF_ROOT_DIR=${TF_ROOT_DIR}/*/
fi
- echo $TF_ROOT_DIR
Here's the build log that shows the syntax error:
Here's the AWS CodeBuild project JSON to reproduce the CodeBuild project:
{
"projects": [
{
"name": "terraform_validate_plan",
"arn": "arn:aws:codebuild:us-west-2:xxxxx:project/terraform_validate_plan",
"description": "Perform terraform plan and terraform validator",
"source": {
"type": "GITHUB",
"location": "https://github.com/marshall7m/sparkify_end_to_end.git",
"gitCloneDepth": 1,
"gitSubmodulesConfig": {
"fetchSubmodules": false
},
"buildspec": "deployment/CI/dev/cfg/buildspec_terraform_validate_plan.yml",
"reportBuildStatus": false,
"insecureSsl": false
},
"secondarySources": [],
"secondarySourceVersions": [],
"artifacts": {
"type": "NO_ARTIFACTS",
"overrideArtifactName": false
},
"cache": {
"type": "NO_CACHE"
},
"environment": {
"type": "LINUX_CONTAINER",
"image": "hashicorp/terraform:0.12.28",
"computeType": "BUILD_GENERAL1_SMALL",
"environmentVariables": [
{
"name": "TF_ROOT_DIR",
"value": "deployment",
"type": "PLAINTEXT"
},
{
"name": "LIVE_BRANCHES",
"value": "(dev, prod)",
"type": "PLAINTEXT"
}
Here's the associated buildspec file content: (buildspec_terraform_validate_plan.yml)
version: 0.2
env:
shell: bash
parameter-store:
AWS_ACCESS_KEY_ID_PARAM: TF_AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY_PARAM: TF_AWS_SECRET_ACCESS_KEY_ID
phases:
install:
commands:
# install/incorporate terraform validator?
pre_build:
commands:
# CodeBuild environment variables
# BRANCH_NAME -- GitHub branch that triggered the CodeBuild project
# TF_ROOT_DIR -- Directory within branch ($BRANCH_NAME) that will be iterated through for terraform planning and testing
# LIVE_BRANCHES -- Branches that represent a live cloud environment
- export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID_PARAM
- export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY_PARAM
- bash -version || echo "${BASH_VERSION}" || bash --version
- |
if [[ -z "${BRANCH_NAME}" ]]; then
# extract branch from github webhook
BRANCH_NAME=$(echo $CODEBUILD_WEBHOOK_HEAD_REF | cut -d'/' -f 3)
fi
- "echo Triggered Branch: $BRANCH_NAME"
- |
if [[ " ${LIVE_BRANCHES[*]} " == *"$BRANCH_NAME"* ]]; then
# Iterate only through BRANCH_NAME directory
TF_ROOT_DIR=${TF_ROOT_DIR}/*/${BRANCH_NAME}/
else
# Iterate through both dev and prod directories
TF_ROOT_DIR=${TF_ROOT_DIR}/*/
fi
- "echo Terraform root directory: $TF_ROOT_DIR"
build:
commands:
- |
for dir in $TF_ROOT_DIR; do
#get list of non-hidden directories within $dir/
service_dir_list=$(find "${dir}" -type d | grep -v '/\.')
for sub_dir in $service_dir_list; do
#if $sub_dir contains .tf or .tfvars files
if (ls ${sub_dir}/*.tf) > /dev/null 2>&1 || (ls ${sub_dir}/*.tfvars) > /dev/null 2>&1; then
cd $sub_dir
echo ""
echo "*************** terraform init ******************"
echo "******* At directory: ${sub_dir} ********"
echo "*************************************************"
terraform init
echo ""
echo "*************** terraform plan ******************"
echo "******* At directory: ${sub_dir} ********"
echo "*************************************************"
terraform plan
cd - > /dev/null
fi
done
done
Given this is just a side project, all files that could be relevant to this problem are within a public repo here.
UPDATES
Tried adding #!/bin/bash shebang line but resulted in the CodeBuild error:
Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: #!/bin/bash
version: 0.2
env:
shell: bash
phases:
build:
commands:
- |
#!/bin/bash
if [[ " ${LIVE_BRANCHES[*]} " == *"$BRANCH_NAME"* ]]; then
# Iterate only through BRANCH_NAME directory
TF_ROOT_DIR=${TF_ROOT_DIR}/*/${BRANCH_NAME}/
else
# Iterate through both dev and prod directories
TF_ROOT_DIR=${TF_ROOT_DIR}/*/
fi
- echo $TF_ROOT_DIR
Solution
As mentioned by #Marcin, I used an AWS managed image within Codebuild (aws/codebuild/standard:4.0) and downloaded Terraform within the install phase.
phases:
install:
commands:
- wget https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip -q
- unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip && mv terraform /usr/local/bin/
I tried to reproduce your issue, but it all works fine for me.
The only thing I've noticed is that you are using $BRANCH_NAME but its not defined anywhere. But even with missing $BRANCH_NAME the buildspec.yml you've posted runs fine.
Update using hashicorp/terraform:0.12.28 image