Trailing newline included in variable assignment after grep, and sent to GitHub actions output - bash

Question

I've got a bash script in a GitHub action that greps for image tags, then replaces them in a docker-compose.yml.
# docker-compose.yml from a JSON payload
docker_compose='"services:\n api:\n proxy: ghcr.io/org/api:main-4095094301\n ports:\n - 0:80\n restart: always\n\n proxy:\n image: ghcr.io/org/proxy:main-4095124301\n depends_on:\n - api\n environment:\n - \"ENVIRONMENT=dev\"\n - \"PORT=8000\"\n ports:\n - 0:3000\n restart: always\n\n ..."'
# Extract the tag of each image, send to GitHub actions output
echo "proxy_tag=$(echo -n $docker_compose | grep -oP 'proxy:\K[-\w]+')" >> $GITHUB_OUTPUT
If I remove the >> operator to echo the output, everything looks OK:
proxy_tag=main-4095094301
But when I feed it into sed later in a different step of the pipeline, an extra newline character seems to come from nowhere:
echo "running s/PROXY_TAG/$proxy_tag/"
sed -i "s/PROXY_TAG/$proxy_tag/" docker-compose.yml
# running s/PROXY_TAG/main-4095094301
# /
# sed: -e expression #1, char 18: unterminated `s' command
# Error: Process completed with exit code 1.
I've tried some common suggestions eg. piping output through tr -d '\n':
echo "proxy_tag=$(echo -n $docker_compose | grep -oP 'proxy:\K[-\w]+' | tr -d '\n')" >> $GITHUB_OUTPUT
Is there something missing I don't understand about bash vars or Github actions?
See below a more complete context of how where these commands are being used.
name: Update Docker Compose
on:
workflow_dispatch:
inputs:
proxy_tag:
description: proxy_tag
type: string
required: false
# api_tag:
# site_tag:
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Fetch docker-compose
id: get-tags
run: |
docker_compose=$(curl ...)
echo "proxy_tag=$(echo -n $docker_compose | grep -oP 'proxy:\K[-\w]+')" >> $GITHUB_OUTPUT
echo "api_tag=$(echo $docker_compose | grep -oP 'api:\K[-\w]+')" >> $GITHUB_OUTPUT
echo "site_tag=$(echo $docker_compose | grep -oP 'site:\K[-\w]+')" >> $GITHUB_OUTPUT
- name: Replace tags docker-compose
env:
proxy_tag: >
${{ github.event_name == 'workflow_dispatch'
&& github.event.inputs.proxy_tag
|| steps.get-tags.outputs.proxy_tag
|| 'latest'
}}
# api_tag: >
# site_tag: >
run: |
echo "Setting [$proxy_tag]"
sed -i "s/PROXY_TAG/$proxy_tag/" docker-compose.yml
sed -i "s/API_TAG/$api_tag/" docker-compose.yml
sed -i "s/SITE_TAG/$site_tag/" docker-compose.yml
# Outputs:
# Setting [main-4095094301 # <--- notice the newline
# ]
#
# sed: -e expression #1, char 18: unterminated `s' command
# Error: Process completed with exit code 1.

Answer 1

I suspect that the newline character being added is from the YAML's folding using > while setting your environment variables under env. Try >- to not add (strip) the last newline character.
This (https://yaml-multiline.info/) might be helpful for live experimentation with YAML.

Answer 2

Worst case, if you're using bash,
$: proxy_tag="main-4095094301
" # embedded newline for example
$: echo "[$proxy_tag]" # show with embedded newline
[main-4095094301
]
$: echo "[${proxy_tag%$'\n'}]" # show with embedded newline removed
[main-4095094301]
$: echo PROXY_TAG | sed "s/PROXY_TAG/$proxy_tag/" # syntax error with embedded newline
sed: -e expression #1, char 27: unterminated `s' command
$: echo PROXY_TAG | sed "s/PROXY_TAG/${proxy_tag%$'\n'}/" # removed, syntax ok
main-4095094301
That seems error-prone to me, though, and I prefer to do most of my basic string processing right in the interpreter unless it's huge and I need it faster, so I'd probably do something based on this -
$: docker_compose=$'services:\n api:\n proxy: ghcr.io/org/api:main-4095094301\n ports:\n - 0:80\n restart: always\n\n proxy:\n image: ghcr.io/org/proxy:main-4095124301\n depends_on:\n - api\n environment:\n - "ENVIRONMENT=dev"\n - "PORT=8000"\n ports:\n - 0:3000\n restart: always\n\n ...' # note $'...' for clean quotes and newlines
$: echo "[$docker_compose]"
[services:
api:
proxy: ghcr.io/org/api:main-4095094301
ports:
- 0:80
restart: always
proxy:
image: ghcr.io/org/proxy:main-4095124301
depends_on:
- api
environment:
- "ENVIRONMENT=dev"
- "PORT=8000"
ports:
- 0:3000
restart: always
...]
$: shopt -s extglob # enable extended globbing
$: tmp="${docker_compose##* proxy: *([^:]):}" # trim front - *(...) is "0 or more"
$: proxy_tag="${tmp%%$'\n'*}" # trim after, from 1st newline on
$: echo "[$proxy_tag]" # show that it's clean
[main-4095094301]