I have a yml file, from which I am obtaining a key with yq and I am storing it in a variable in bash, I am trying to do another query with the value of the variable but it does not give the expected result
file.yml
version: '3'
services:
task_auth:
environment:
AWS_API_VERSION: "2016-04-19"
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: /creds
PORT: "8000"
SES_AWS_ACCESS_KEY: xxxxx
SES_AWS_SECRET_KEY: xxxxx
image: xxxxxxx
ports:
- "8000:8000"
yq e '(.services | keys)[]' file.yml
Result:
task_auth
Storing the query in a variable TASK_NAME
TASK_NAME=$(yq e '(.services | keys)[]' file.yml)
Calling the variable to see the previously stored value
$TASK_NAME
bash: task_auth: command not found
yq eval '.services.$TASK_NAME.environment.PORT' file.yml
Result
null
if I execute the command echo $TASK_NAME I get the correct value task_auth but to make queries from yq no
Now when I need this other query I need to use the initial variable, how can I do the query using that value of $TASK_NAME in yq ?
Thanks in advance
Provide the Bash variable as environment variable, and inside mikefarah/yq use env to retrieve it:
TASK_NAME="$TASK_NAME" yq e '.services[env(TASK_NAME)].environment.PORT' file.yml
8000
See the manual section Read string environment variable
Related
I'm trying to use variables inside variables in azure pipelines.
Below is an example of the bash script:
#!/bin/bash
customer=google
environment=preprod
android_google_preprod_account_activation_url=preprod.google.com
echo "Customer is $customer"
echo "Environment is $environment"
var1=android_${customer}_${environment}_account_activation_url
echo "variable is $var1"
echo "original value is ${!var1}"
I get the expected output for the above bash script when I run it on my Ubuntu server, with NO errors:
Customer is google
Environment is preprod
variable is android_google_preprod_account_activation_url
original value is preprod.google.com
The yml code for azure pipelines is:
parameters:
- name: customer
displayName: 'select customer'
type: string
values:
- google
- name: environment
displayName: 'select environment'
type: string
values:
- preprod
variables:
- group: android-${{ parameters.customer }}-${{ parameters.environment }}
- name: var1
value: android-${{ parameters.customer }}-${{ parameters.environment }}-account-activation-url
script: |
echo "Customer is $(customer)"
echo "Environment is $(environment)"
echo "variable is $(var1)"
echo "original value is $(!var1)"
displayName: 'echo variables'
The value of android-google-preprod-account-activation-url is being taken from variable groups inside library.
It gives me an error for the 4th line:
invalid indirect expansion
The first 3 lines output is as expected.
Expected output is:
Customer is google
Environment is preprod
variable is android_google_preprod_account_activation_url
original value is preprod.google.com
Is there a different syntax that needs to be followed in azure pipelines?
I`m not a bash expert ))) however... you're trying to use the parameters expansion What is indirect expansion? What does ${!var*} mean?
But it refers to the bash variables.... when you define variables in the devops pipeline, you have to use them as environment variables or through the macro.
or something like that:
android_google_preprod_account_activation_url=preprod.google.com
echo "Customer is $(customer)"
echo "Environment is $(environment)"
var1=android_$(customer)_$(environment)_account_activation_url
echo "variable is $var1"
echo "original value is ${!var1}"
The macro syntax "$(varName)" is a proprietary syntax in Azure Pipelines to interpolate variable values. It is processed during runtime and different with the syntax "${varName}" in Bash scripts.
For your case, you can try to use the compile time syntax "${{ variables.varName }}" to get the value in the pipeline.
echo "original value is $(${{ variables.var1 }})"
With above change, after you triggered the pipeline:
At the compile time, the expression "${{ variables.var1 }}" will be replaced with the actual value "android_google_preprod_account_activation_url". So, the expression "$(${{ variables.var1 }})" will be changed to "$(android_google_preprod_account_activation_url)".
Then at the runtime, the expression will be parsed as the correct value "preprod.google.com".
Below is an example I have tested on my side.
YAML
variables:
android_google_preprod_account_activation_url: 'preprod.google.com'
var1: 'android_google_preprod_account_activation_url'
jobs:
- job: A
displayName: 'Job A'
pool:
vmImage: ubuntu-latest
steps:
- checkout: none
- task: Bash#3
displayName: 'Print variables'
inputs:
targetType: inline
script: |
echo "android_google_preprod_account_activation_url = $(android_google_preprod_account_activation_url)"
echo "var1 = $(var1)"
echo "original value = $(${{ variables.var1 }})"
Result
For more details, you can reference the related document "Understand variable syntax".
I have Terraform variables defined like
variable "location" {
type = string
default = "eastus"
description = "Desired Azure Region"
}
variable "resource_group" {
type = string
default = "my-rg"
description = "Desired Azure Resource Group Name"
}
and potentially / partially overwritten in terraform.tfvars file
location = "westeurope"
and then defined variables as outputs e.g. a file outputs.tf:
output "resource_group" {
value = var.resource_group
}
output "location" {
value = var.location
}
How can I "source" the effective variable values in a bash script to work with these values?
One way is to use Terraform output values as JSON and then an utility like jq to convert and source as variables:
source <(terraform output --json | jq -r 'keys[] as $k | "\($k|ascii_upcase)=\(.[$k] | .value)"')
note that output is only available after executing terraform plan, terraform apply or even a terraform refresh
If jq is not available or not desired, sed can be used to convert Terraform HCL output into variables, even with upper case variable names:
source <(terraform output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
or using -chdir argument if Terraform templates / modules are in another folder:
source <(terraform -chdir=$TARGET_INFRA_FOLDER output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
Then these variables are available in bash script:
LOCATION="westeurope"
RESOURCE_GROUP="my-rg"
and can be addressed as $LOCATION and $RESOURCE_GROUP.
I have a set of properties, and I want to create env variables in bash. The format of the properties is like this:
CONFIG=$(curl -s "endpoint")
echo $CONFIG
property1: value1 property2: value2 property3: value3...
Then I apply a replace to get '='
CONFIG2=${CONFIG//: /=}
echo $CONFIG2
property1=value1 property2=value2 property3=value3...
And finally, applying a for loop I want to add those properties
for property in $CONFIG2; do env "$property"; done
I need to use env since my properties contains . characters, export doesn't work for me. For example: property.1.thisisanexample=value1
While executing the 'for', I can see that in each iteration it is adding the current property, but after it finish and I run env, they aren't there. Any idea of what is happening and how can this be done?
#1
Assuming your CONFIG variable has or gets the following string value:
CONFIG='property.1.thisisanexample: value1 property.2.thisisanexample: value2 property.3.thisisanexample: value:3'
Then you may create a CONFIG2 parameter like in below:
Note: This will substitute the colon+ space : characters with a =
CONFIG2=$(echo ${CONFIG//:\ /=} )
#!/bin/bash
CONFIG='property.1.thisisanexample: value1 property.2.thisisanexample: value2 property.3.thisisanexample: value:3'
CONFIG2=$(echo ${CONFIG//:\ /=} )
echo "${CONFIG2}"
Output:
property.1.thisisanexample=value1 property.2.thisisanexample=value2 property.3.thisisanexample=value:3
#2
Now if you want to use env with the properties you may do the following:
for property in ${CONFIG2} ; do
env ${property}
done
Messing around with a simple aws cli query to check for the existence of a Lambda function and echo the associated role if it exists:
#!/bin/bash
fname=$1
role=$(aws lambda list-functions --query 'Functions[?FunctionName == `$fname`].Role' --output text)
echo "$fname role: $role"
However, $fname appears to be resolving to an empty string in the aws command. I've tried escaping the back ticks, swapping ` to ' and a miriad of other thrashing edits (and yes, I'm passing a string on the cl when invoking the script :)
How do I properly pass a variable into JMESPath query inside a bash script?
Because the whole JMESPath expression is enclosed in single quotes, bash is not expanding the $fname variable. To fix this you can surround the value with double quotes and then use single quotes (raw string literals) for the $fname var:
aws lambda list-functions --query "Functions[?FunctionName == '$fname'].Role" --output text
Swapping the backticks to single quotes, didn't work for me... :(
But escaping the backticks works :)
Here are my outputs:
aws elbv2 describe-listeners --load-balancer-arn $ELB_ARN --query "Listeners[?Port == '$PORT'].DefaultActions[].TargetGroupArn | [0]"
null
aws elbv2 describe-listeners --load-balancer-arn $ELB_ARN --query "Listeners[?Port == \`$PORT\`].DefaultActions[].TargetGroupArn | [0]"
"arn:aws:elasticloadbalancing:ap-southeast-2:1234567:targetgroup/xxx"
I have a TARGET variable that can be set to dev, test or prod.
I defined the following lists:
dev=(server1 user1 target1)
test=(server2 user2 target2)
prod=(server3 user3 target3)
Depending on the value of TARGET, I'd like to dynamically associate the variable CONFIG to one of the list.
Let's say TARGET=dev. I then have
eval CONFIG=\$$TARGET # I expect CONFIG to be a list containing (server1 user1 target1)
echo ${CONFIG[*]} # OK, it gives (server1 user1 target1)
echo ${CONFIG[1]} # I would expect to have "server1" but it returns "1", seems like CONFIG is not seen as a list
Any idea ?
eval CONFIG=\$$TARGET sets CONFIG to the string $TARGET. When an array is expanded in a string context, the result is the concatenation of the values in the array, with the first character of IFS inserted as a separator. Thus after the assignment the value of CONFIG is the string server1 user1 target1.
You need to assign to CONFIG as an array. Since you're working in zsh, you don't need to use eval to obtain the value of a variable whose name is in a variable. Use the P parameter expansion flag.
CONFIG=(${(P)TARGET})