How to enclose a multiline command within an until-do loop in YAML - shell

I need to use an until-do loop in the "script" section of a YAML file, and the command in the until clause visually benefits from being broken into multiple lines:
- >
helm upgrade $(...) /helm/charts/ -f .(...)
--install
--set host=(...)
--set tag=(...)
--set image=(...)
--namespace (...)
I've tried using this formatting to enclose the helm upgrade command within an until-do loop:
- >
until helm upgrade $FOO helm/charts/ -f ./(...)
--install
--set imagePullSecret=(...)
--set host=(...) --set tag=(...)
--set image=(...)
--namespace (...); do
helm uninstall $FOO
done
...which results in the errors:
(...): line 161: --install: command not found
(...): line 162: --set: command not found
(...): line 163: --set: command not found
(...): line 164: --set: command not found
(...): line 165: --namespace: command not found
Error: uninstall: Release not loaded: (...): release: not found
However, if I abandon efforts to make this more readable:
- >
until helm upgrade foo helm/charts/ -f ./(...) --install --set imagePullSecret=(...) --set host=(...) --set tag=(...) --set image=(...) --namespace (...); do
helm uninstall foo
done
...the command runs as expected:
Release "bar" has been upgraded. Happy Helming!
What is the correct way to enclose a multiline command within an until-do loop in YAML?

A literal block scalar (started with >) in YAML will fold line endings into spaces between consecutive non-empty lines… unless one of the lines is more indented than the base indentation of the scalar.
This has been designed so that something like
>
lorem ipsum
dolor sit amet
* one
* two
will be parsed as
"lorem ipsum dolor sit amet\n\n *one\n *two\n"
since the * symbols are more indented than the lorem ipsum line.
In your case, this means that if you want to break the command into multiple lines but want it parsed as a single line, you mustn't indent the continuation lines:
- >
until helm upgrade $FOO helm/charts/ -f ./(...)
--install
--set imagePullSecret=(...)
--set host=(...) --set tag=(...)
--set image=(...)
--namespace (...); do
helm uninstall $FOO
done
You can indent the helm uninstall since it should be executed as separate command.
If you dislike this and want to indent the continuation lines, you need to escape the resulting newlines:
- >
until helm upgrade $FOO helm/charts/ -f ./(...) \
--install \
--set imagePullSecret=(...) \
--set host=(...) --set tag=(...) \
--set image=(...) \
--namespace (...); do
helm uninstall $FOO
done

Related

bash command substituation comes up blank

I have the following Dockerfile
FROM public.ecr.aws/docker/library/python:3.7-slim-buster as base
WORKDIR /code
RUN apt-get update -y && \
pip install --upgrade pip && \
mkdir ~/.aws \
&& touch ~/.aws/credentials
RUN apt-get install -y nodejs npm awscli jq
Which is base from slim-buster.
I am trying to do a one command aws stscommand from this recommended AWS sts assume role in one command solution
However the the eval solution is coming up "blank" or the parenthesis is not escaped properly within a make command:
sts-qa:
eval $(aws sts assume-role --role-arn arn:aws:iam::1234556778:role/aws-role --role-session-name pebble | jq -r '.Credentials | "export AWS_ACCESS_KEY_ID=\(.AccessKeyId)\n export AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey)\nexport AWS_SESSION_TOKEN=\(.SessionToken)\n"')
the error I get is:
root#b79fd0ebf8cf:/code# make sts-qa
eval \n export AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey)\n export AWS_SESSION_TOKEN=\(.SessionToken)\n"')
/bin/sh: 1: Syntax error: ")" unexpected
I have tried:
escaping all () from the command. i get the following error: /bin/sh: 2: Syntax error: Unterminated quoted string
assigning the content of the
$(...) into a variable. what I get back as an output is eval (the
entire bash command is blank
I have ran out of ideas
$(...) is Makefile syntax for expanding a variable name, so the ( is closed by the first unescaped ) in the command you want to execute. You need to double the $ to have it be treated literally.
sts-qa:
eval $$(aws sts assume-role --role-arn arn:aws:iam::1234556778:role/aws-role --role-session-name pebble | jq -r '.Credentials | "export AWS_ACCESS_KEY_ID=\(.AccessKeyId)\n export AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey)\nexport AWS_SESSION_TOKEN=\(.SessionToken)\n"')

Jenkins - Assign variable inside sh script

I would like to create a variable name as POD inside script to assign kubectl output and then pass this variable while running kubectl port-forward pods..
But I received below error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 151: illegal string body character after dollar sign;
solution: either escape a literal dollar sign "\$5" or bracket the value expression "${5}" # line 151, column 80.
e-context ${KUBE_CLUSTER_STAGE}
Here is my script.
environment {
POD = ''
}
steps {
script {
withCredentials([file(credentialsId: 'mbtkubeconfig', variable: 'config')]){
try {
// Expose PostreSQL
sh '''#!/bin/sh
chmod ug+w ${config}
export KUBECONFIG=\${config}
kubectl config use-context ${KUBE_CLUSTER_STAGE}
kubectl config set-context --current --namespace=database
POD = `$(kubectl get po -n database --selector='role==master' -o jsonpath="{.items[0].metadata.name}")`
kubectl port-forward pods/$POD 5432:64000 & echo \$! > filename.txt
'''
When I tried without variable there is no any error.Here is the script running without any error.
sh """#!/bin/sh
chmod ug+w ${config}
export KUBECONFIG=\${config}
kubectl config use-context ${KUBE_CLUSTER_STAGE}
kubectl config set-context --current --namespace=database
kubectl get pods -n database
kubectl port-forward pods/my-postgres-postgresql-helm-0 5432:64000 & echo \$! > filename.txt
"""
When you run commands with sh make sure you are using " not '. Groovy variables will only be resolved when using "${config}".
By the way, it is considered best practice to mark variables with env. although not needed to resolve the variable. For instance, try to mark your cluster stage with ${env.KUBE_CLUSTER_STAGE}

How to save command in variable and print translated command before execution without bash -x

Using bash script function with a following command, looking for a way to print command for before execution for logging purposes without using bash -x, is that possible ? Problematic part is a last command.
helm upgrade $2 chart --install --wait --timeout $3 $debug "${#:4}"
Would like to print a following:
helm upgrade mypod chart --install --wait --timeout 600s --set key=value
Why not just turning debug on and off?
set -x
helm upgrade $2 chart --install --wait --timeout $3 $debug "${#:4}"
set +x
other commands...

Getting "bash: syntax error near unexpected token `|'" when trying to install tensorflow serving

I was trying to install TensorFlow-Serving by following this offical documentation :
I copied and pasted this command into my Ubuntu 18.04 terminal:
echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | sudo tee /etc/apt/sources.list.d/tensorflow-serving.list && \
curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | sudo apt-key add -
But I get this error:
bash: syntax error near unexpected token `|'
Why am I getting this error?
Problem: You copied html tags like <a href=""> which shouldn't be part of the command.
Quick Fix
Remove the html tags from the command:
echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | sudo tee /etc/apt/sources.list.d/tensorflow-serving.list && \
curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | sudo apt-key add -
Details
Angle brackets are used for redirection in bash.
cmd < file reads file and passes its contents to cmd.
cmd > file writes the output of cmd into file.
In the first command you were lucky: all < and > where quoted with ", therefore the command executed "normally" (it didn't do what you expected, but at least it didn't fail).
The second command does some stuff and ends with > |. You can reproduce this error with echo > | cat. Bash expects a file name after > but only found the start of another command (| ...).

Use sed in docker to replace path

I know it may be a stupid question ...
I want to do a docker from a docker file. The thing is that I do not have the Command-line completion when I'm going in the docker using docker run -it docker_name /bin/bash.
So I did this https://docs.docker.com/machine/completion/ (still not working) and want to do this https://linuxconfig.org/tab-does-not-automatically-complete-docker-commands-solution
The thing is that I'm not able to replace the few line from the docker file. I wanted to use the command sed like this : RUN sed -i 's+#if ! shopt -oq posix; then+if ! shopt -oq posix; then+g' /etc/bash.bashrc but it does not work to replace line containing a path (I'm getting the error sed: -e expression #1, char 75: unterminated `s' command).
I also tryed RUN sed -i 's+\# if [ -f \/usr\/share\/bash-compl+ if [ -f \/usr\/share\/bash-compl+g' /etc/bash.bashrc but got the same result.
What's the good way the use sed in the dockerfile, or what's the good way to update a file in the future docker from the dockerfile ?
------------- EDIT ------------
Sorry I forgot to share my dockerfile.
the first lines are the following :
FROM nvidia/cuda:8.0-devel-ubuntu14.04
RUN apt-get update && apt-get install --yes --force-yes \
software-properties-common build-essential wget curl git pkg-config cmake-curses-gui vim nano doxygen libboost-all-dev libeigen3-dev libflann-dev freeglut3-dev libglew-dev
# to get cmake3.0
RUN add-apt-repository ppa:george-edison55/cmake-3.x
RUN apt-get update && apt-get install --yes --force-yes \
cmake
RUN apt-get install --yes --force-yes \
g++ python
# Intall some basic GUI tools
RUN apt-get install --yes --force-yes \
cmake-qt-gui gnome-terminal
I want to edit the file /etc/bash.bashrc in the docker so I did :
RUN sed -i 's+#if ! shopt -oq posix; then+if ! shopt -oq posix; then+g' /etc/bash.bashrc #this works
RUN sed -i 's+\# if [ -f \/usr\/share\/bash-compl+ if [ -f \/usr\/share\/bash-compl+g' /etc/bash.bashrc #this does not work and provide the previous error
's+# if \[ -f /usr/share/bash-compl+ if \[ -f /usr/share/bash-compl+g' should parse.
You need to escape the square brackets [ to \[. Square brackets allow grouping characters in sed, so they will not match the literal brackets unless escaped.
You do not need to escape the forward slashes if you continue to use s+++ rather than s///.

Resources