I was trying out gomplate and encountered an error.
For context I've defined a template file, test.tmplt, and a datasource file, dev.yaml.
test.tmplt has the following content :
localAPIEndpoint:
advertiseAddress: {{ (datasource "k8s").api-advertise-ip }}
while dev.yaml contains the following :
api-advertise-ip: 192.168.0.1
If I try to fill in the content of test.tmplt using gomplate like so :
gomplate -d k8s=./dev.yaml -f ./test.tmplt -o test.conf
I get the following error :
09:42:44 FTL error="template: ./test.tmplt:2: bad character U+002D '-'"
Seems to me that it does not like the '-' symbol in the template file. Any workaround?Is it the intended behaviour?
Edit 1:
Thanks #icza for the answer which works correctly for the example above. Yet if I modify the yaml file to have nested fields it seems to break down.
For example
dev.yaml :
kubernetes:
api-advertise-ip: 192.168.0.0
test.tmplt :
localAPIEndpoint:
advertiseAddress: {{ index (datasource "k8s") "kubernetes.api-advertise-ip" }}
In this case the output of :
gomplate -d k8s=./dev.yaml -f ./test.tmplt -o test.conf
is :
localAPIEndpoint:
advertiseAddress: <no value>
Your "k8s" data source is a YAML config, and you want to access the api-advertise-ip property of it.
Since api-advertise-ip contains dashes, you can't use the name as-is in the template, because that's a syntax error: the template engine tries to use api as the property name, and the dash after that is a syntax error.
You have to put the property name in quotes that contains dashes: "api-advertise-ip", but this using the . selector is also invalid syntax.
Use the builtin index function to index the YAML datasource with this key:
localAPIEndpoint:
advertiseAddress: {{ index (datasource "k8s") "api-advertise-ip" }}
gomplate uses text/template under the hood, see a working example on the Go Playground.
When using index and you have multiple nested levels, provide each key as an additional parameter to index.
For example:
localAPIEndpoint:
advertiseAddress: {{ index (datasource "k8s") "kubernetes" "api-advertise-ip" }}
Try this one on the Go Playground.
For those landing here for accessing values from Kubernetes secret (with kubectl) whose keys have hyphen - in them, the solution as suggested by #icza is:
kubectl -n <namespace> \
get secret <secret-name> \
-ogo-template='{{ index .data "key-with-hyphens" | base64decode }}'
Related
I have a yml file, from which I am obtaining a key with yq and I am storing it in a variable in bash, I am trying to do another query with the value of the variable but it does not give the expected result
file.yml
version: '3'
services:
task_auth:
environment:
AWS_API_VERSION: "2016-04-19"
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: /creds
PORT: "8000"
SES_AWS_ACCESS_KEY: xxxxx
SES_AWS_SECRET_KEY: xxxxx
image: xxxxxxx
ports:
- "8000:8000"
yq e '(.services | keys)[]' file.yml
Result:
task_auth
Storing the query in a variable TASK_NAME
TASK_NAME=$(yq e '(.services | keys)[]' file.yml)
Calling the variable to see the previously stored value
$TASK_NAME
bash: task_auth: command not found
yq eval '.services.$TASK_NAME.environment.PORT' file.yml
Result
null
if I execute the command echo $TASK_NAME I get the correct value task_auth but to make queries from yq no
Now when I need this other query I need to use the initial variable, how can I do the query using that value of $TASK_NAME in yq ?
Thanks in advance
Provide the Bash variable as environment variable, and inside mikefarah/yq use env to retrieve it:
TASK_NAME="$TASK_NAME" yq e '.services[env(TASK_NAME)].environment.PORT' file.yml
8000
See the manual section Read string environment variable
I have Terraform variables defined like
variable "location" {
type = string
default = "eastus"
description = "Desired Azure Region"
}
variable "resource_group" {
type = string
default = "my-rg"
description = "Desired Azure Resource Group Name"
}
and potentially / partially overwritten in terraform.tfvars file
location = "westeurope"
and then defined variables as outputs e.g. a file outputs.tf:
output "resource_group" {
value = var.resource_group
}
output "location" {
value = var.location
}
How can I "source" the effective variable values in a bash script to work with these values?
One way is to use Terraform output values as JSON and then an utility like jq to convert and source as variables:
source <(terraform output --json | jq -r 'keys[] as $k | "\($k|ascii_upcase)=\(.[$k] | .value)"')
note that output is only available after executing terraform plan, terraform apply or even a terraform refresh
If jq is not available or not desired, sed can be used to convert Terraform HCL output into variables, even with upper case variable names:
source <(terraform output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
or using -chdir argument if Terraform templates / modules are in another folder:
source <(terraform -chdir=$TARGET_INFRA_FOLDER output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
Then these variables are available in bash script:
LOCATION="westeurope"
RESOURCE_GROUP="my-rg"
and can be addressed as $LOCATION and $RESOURCE_GROUP.
I have pillar that is supposed to have value of "-" (without the quotation).
I tried:
my_pillar: -
my_pillar: '-'
my_pillar: "-"
my_pillar: {% raw %}"-"{% endraw %}
{% raw %}my_pillar: "-"{% endraw %}
None of those worked and I am getting error: "block sequence entries are not allowed in this context".
I understand what the problem here is, Salt thinks that it is part of YAML definition of values, not the value itself, but how do I work around that?
As OrangeDog mentions, any type of quoting will work:
docker run --rm -it saltstack/salt /bin/sh
mkdir -p /srv/pillar
cat <<EOF>/srv/pillar/top.sls
base:
'*':
- stuff
EOF
echo "my_pillar: -" >/srv/pillar/stuff.sls
salt-call --local pillar.items -ldebug
# Error happens ^^^^^^^^^^^^^^^^^^^^^^
echo "my_pillar: '-'" >/srv/pillar/stuff.sls
salt-call pillar.items
The output for me:
local:
----------
my_pillar:
-
apiVersion: v1
kind: Secret
metadata:
name: {{ include "backstage.fullname" . }}-backend
type: Opaque
stringData:
GH_APP_PRIVATEKEY_BASE:|-
{{ .Values.auth.ghApp.privateKey | quote | b64dec | indent 2 }}
Getting error converting YAML to JSON: yaml: line 22: could not find expected ':' as the result when
trying to store a base64 encoded string to GH_APP_PRIVATEKEY_BASE
My application (backstage) is using helm charts to map the env secret.
I keep having trouble with storing/passing multi-line RSA. private key,
Currently trying to base64 encoded private key into one-liner, but still failed at validating the secret file. Would love to know other approach like passing a file with key written on it?
BTW, I use GITHUB_PRVATE_KEY=$(echo -n $GITHUB_PRVATE_KEY | base64 -w 0) and
helm_overrides="$helm_overrides --set auth.ghApp.clientSecret=$GITHUB_PRVATE_KEY"
at GitHub action to encoded the private key.
Try increase the indent to 4:
...
stringData:
GH_APP_PRIVATEKEY_BASE: |-
{{ .Values.auth.ghApp.privateKey | quote | b64dec | indent 4 }}
GH_APP_PRIVATEKEY_BASE:|-
You need a space in there GH_APP_PRIVATEKEY_BASE: |-
Also not sure why you have a b64dec in there but I don't think that's the immediate problem.
I have something like this dictionary:
env: qat
target_host: >
{%if env in ['prd'] %}one
{%elif env in ['qat','stg'] %}two
{%else%}three
{%endif%}
when I print it I get:
ok: [localhost] => {
"var": {
"target_host": "two "
} }
So it is converting the \n at the end of the line to a space. Which is exactly what it is supposed to do. However in this case I am just trying to spread out the lines to make the structure of the if/else more readable and I don't want the extra space. It works as expected if I put it all on one line without the > but I would like to be able to make it multiline just so its easier to read.
I found this question
Is there a way to represent a long string that doesnt have any whitespace on multiple lines in a YAML document?
So I could do:
env: qat
target_host: "{%if env in ['prd'] %}one\
{%elif env in ['qat','stg'] %}two\
{%else%}three\
{%endif%}"
And that gives the desired result.
Is there anyway to accomplish this without cluttering it up even more?
In Jinja* you can strip whitespaces/newlines by adding a minus sign to the start/end of a block. This should do the trick:
env: qat
target_host: >
{%if env in ['prd'] -%}one
{%- elif env in ['qat','stg'] -%}two
{%- else -%}three
{%- endif%}
* Jinja 2 is the templating engine used by Ansible.
Maybe what you need is the | literal?
env: qat
target_host: |
{%if env in ['prd'] %}one
{%elif env in ['qat','stg'] %}two
{%else%}three
{%endif%}
This will not 'fold' newlines into spaces, as oposed to >