I am using yq v4.27.2 and want to achieve this using the environment variable as the key:
yq -n '.key1.key2="value"'
key1:
key2: value
However, I get the error as:
export KEY=key1.key2; yq -n '.env(KEY)="value"'
Error: 1:6: invalid input text "KEY)=env(VALUE)"
With this, the whole key is treated as one node,
export KEY="key1.key2"; yq -n '.[env(KEY)]="value"'
key1.key2: value
What is the right syntax? Thanks
Found the syntax after reading the source code here https://github.com/mikefarah/yq/blob/master/pkg/yqlib/operator_env_test.go#L61
export KEY=.key1.key2; ./yq -n 'eval(strenv(KEY))="value"'
key1:
key2: value
Related
I'm trying to do a replace where I first escape all double quotes and then I want to use the result of this replace to updated a value. But in the last replace the backslashes are removed, why is this and how do I avoid that?
Example in bash:
>TEST_OBJECT='{"val1": "a", "val2": "b"}'
>ESCAPED_OBJ="$(echo $TEST_OBJECT | sed 's/"/\\"/g')"
>echo $ESCAPED_OBJ
{\"val1\": \"a\", \"val2\": \"b\"}
>echo 'value: "_REPLACE_ME"' | sed "s#_REPLACE_ME#$ESCAPED_OBJ#g"
value: "{"val1": "a", "val2": "b"}"
I'm expecting this on the last row:
value: "{\"val1\": \"a\", \"val2\": \"b\"}"
EDIT
I realize I presented the issue wrong, the reason why I do it in 2 steps is because the first replace happens in one step and then the second replace happens in a later step. This is part of a github workflow and the last replace actually replaces a string in a different yaml file.
sed "s#_REPLACE_ME#$ESCAPED_OBJ#g" > ${{ github.action_path }}/config/job.yml
So I don't think I can do the replace in one step, first I need to update the string and then replace a value in another file.
job.yml
...
env:
- name: CUSTOM_DATA_OBJECT
value: "_REPLACE_ME"
I need the value to be escaped so that it doesn't break the yaml.
Using sed
$ sed "s#_REPLACE_ME#${TEST_OBJECT//\"/\\\\\"}#" input_file
...
env:
- name: CUSTOM_DATA_OBJECT
value: "{\"val1\": \"a\", \"val2\": \"b\"}"
Maybe letting yq take care of the correct escaping for YAML?
TEST_OBJECT='{"val1": "a", "val2": "b"}' \
yq eval '.env[0].value = strenv(TEST_OBJECT)' file.yaml
env:
- name: CUSTOM_DATA_OBJECT
value: "{\"val1\": \"a\", \"val2\": \"b\"}"
Given yaml config file that looks like this:
key1:
key11:value1
key12:value2
key2:
key21:value3
How can I convert it in a bash script (preferable with yq) to env vars prefixed with a string? Desired output for env:
TF_VAR_key11=value1
TF_VAR_key12=value2
TF_VAR_key21=value3
Assuming the missing spaces in the input's subelements between key and value are intentional, so we are just dealing with an array of string values containing :, and separated by whitespace.
yq '.[] | split(" ") | .[] | split(":") | "TF_VAR_" + .[0] + "=" + .[1]' file.yaml
Which implementation of yq are you using? This works for mikefarah/yq. To be used with kislyuk/yq, add the -r option.
You'd first load the YAML configuration into memory.
from yaml import loads
with open("config.yaml") as f:
config = loads(f.read())
Then, iterate over the dictionary values, which appear to also be dictionaries. For each of these dictionaries, write the key=val pair to the new file.
env_str = ""
for inner_dict in config.values():
for key, val in inner_dict.items():
env_str = f"TF_VAR_{key}={val}\n"
Using python as suggested is easy and readable, if you need to do everything in bash, you can check this thread which has various solutions:
How can I parse a YAML file from a Linux shell script?
I am trying to pass bash variable in yq
test.yml
configuration:
Properties:
corporate-url: https://stackoverflow.com/
temp = '.configuration.Properties.corporate-url'
export $temp
Value1=$(yq '.[env($temp)]' test.yml)
expected output:
https://stackoverflow.com/
but I am getting this error(Actual output)
Error: Value for env variable '$variable1' not provided in env()
Please note:
I am trying to fetch corporate-url value, using a bash variable, constraint is that I cannot pass string directly in yq as the value of temp changes as this snippet is running inside a for loop which changes value of temp every time so cannot hard code for a particular value.
Reference YQ Documentation:
https://mikefarah.gitbook.io/yq/operators/env-variable-operators
ApisDraft folder contains multiple yml files
ApisDraft=$(find drafts/* -maxdepth 1 -type f)
for ApiFixOrgsTags in $ApisDraft
do
my_var=$(yq '.securityDefinitions.[].tokenUrl' $ApiFixOrgsTags)
ConfigProper='.configuration.Properties.'
CatelogProper='.configuration.catalogs.[].Properties.'
variable1=$ConfigProper${my_var}
variable2=$CatelogProper${my_var}
# to remove white all spaces
variable1= echo $variable1 | sed -E 's/(\.) */\1/g'
variable2= echo $variable2 | sed -E 's/(\.) */\1/g'
export $variable1
export $variable2
Value1=$(yq "$variable1" $ApiFixOrgsTags)
Value2=$(yq '.[env($variable2)]' $ApiFixOrgsTags)
done
In this case, you don't need to put it in the environment. Let the shell expand it so yq just sees the value of the variable:
yq "$temp" test.yml # => https://stackoverflow.com/
Similar to e.g. this: Parse out key=value pairs into variables
.env
key1=value1
key2=value2
config.yml
some:
key1: ${key1}
key2: ${key2}
How can I replace the values in config.yml with the values from .env?
So far I got this:
awk 'FNR==NR {n=index($1,"=");if(n){vars[substr($i, 1, n - 1)]=substr($i, n + 1)}; next } { for (i in vars) gsub("${"i"}", vars[i]) }1' .env file.yml
^- Can't figure out how to replace $
I can make it work with {key1} but not with ${key1}. How can I achieve this?
The simplest would be:
. .env
export key1 key2
envsubst '$key1 $key2' < config.yml
Can't figure out how to replace $
Seems you have to escape $ and { and }, like:
gsub("\\$\\{"i"\\}", vars[i])
Without using awk:
(set -a; source .env; envsubst < config.yml)
some:
key1: value1
key2: value2
(...): Runs command line in a sub-shell to avoid polluting current shell's environment
set -a: Exports all declaring variables
source .env: Source in .env file
envsubst < config.yml: Substitutes env variables in config.yml
I am trying to use jq to construct a hash in which a key name comes from a variable. Something like this:
jq --null-input --arg key foobar '{$key: "value"}'
This doesn't work, however, and gives the following error:
error: syntax error, unexpected '$'
{$key: "value"} 1 compile error
Use parentheses to evaluate $key early as in:
jq --null-input --arg key foobar '{($key): "value"}'
See also: Parentheses in JQ for .key
You can also use String interpolation in jq which is of the form "\(..)". Inside the string, you can put an expression inside parens after a backslash. Whatever the expression returns will be interpolated into the string.
You can do below. The contents of the variable key is expanded and returned as a string by the interpolation sequence.
jq --null-input --arg key foobar '{ "\($key)": "value"}'