I try to format 4 digits in Azure pipeline (YAML) but its not working, someone help?
i have this variable >> serialNumber: 2
I used the following yaml syntax:
formatnumber: $[format('{0:D4}', variables.serialNumber)]
formatnumber: $[format('{0:0000}', variables.serialNumber)]
formatnumber: $[format('{0:####}', variables.serialNumber)]
i want to convert the number 2 to 0002 with format but i get the following error:
An error occurred while loading the YAML build pipeline. The format specifiers 'D4' are not valid for objects of type 'String'
The above formater doesnot work is because the type of the value returned by variables.serialNumber is string not int type.
There is a workaround to achieve this. You can add a powershell task to format the number. You can check below example.
trigger:
- master
variables:
number: 2
pool:
vmImage: 'ubuntu-latest'
steps:
- powershell: |
$number = $(number)
$format = "{0:0000}" -f $number
echo "##vso[task.setvariable variable=formatnumber]$format"
- powershell: echo "$(formatnumber)"
Above script in powershell task formats the variable number, and set the formatted number to variable 'formatnumber'. Then the following task can refer to the formatted number by $(formatnumber)
Related
I'm trying to use variables inside variables in azure pipelines.
Below is an example of the bash script:
#!/bin/bash
customer=google
environment=preprod
android_google_preprod_account_activation_url=preprod.google.com
echo "Customer is $customer"
echo "Environment is $environment"
var1=android_${customer}_${environment}_account_activation_url
echo "variable is $var1"
echo "original value is ${!var1}"
I get the expected output for the above bash script when I run it on my Ubuntu server, with NO errors:
Customer is google
Environment is preprod
variable is android_google_preprod_account_activation_url
original value is preprod.google.com
The yml code for azure pipelines is:
parameters:
- name: customer
displayName: 'select customer'
type: string
values:
- google
- name: environment
displayName: 'select environment'
type: string
values:
- preprod
variables:
- group: android-${{ parameters.customer }}-${{ parameters.environment }}
- name: var1
value: android-${{ parameters.customer }}-${{ parameters.environment }}-account-activation-url
script: |
echo "Customer is $(customer)"
echo "Environment is $(environment)"
echo "variable is $(var1)"
echo "original value is $(!var1)"
displayName: 'echo variables'
The value of android-google-preprod-account-activation-url is being taken from variable groups inside library.
It gives me an error for the 4th line:
invalid indirect expansion
The first 3 lines output is as expected.
Expected output is:
Customer is google
Environment is preprod
variable is android_google_preprod_account_activation_url
original value is preprod.google.com
Is there a different syntax that needs to be followed in azure pipelines?
I`m not a bash expert ))) however... you're trying to use the parameters expansion What is indirect expansion? What does ${!var*} mean?
But it refers to the bash variables.... when you define variables in the devops pipeline, you have to use them as environment variables or through the macro.
or something like that:
android_google_preprod_account_activation_url=preprod.google.com
echo "Customer is $(customer)"
echo "Environment is $(environment)"
var1=android_$(customer)_$(environment)_account_activation_url
echo "variable is $var1"
echo "original value is ${!var1}"
The macro syntax "$(varName)" is a proprietary syntax in Azure Pipelines to interpolate variable values. It is processed during runtime and different with the syntax "${varName}" in Bash scripts.
For your case, you can try to use the compile time syntax "${{ variables.varName }}" to get the value in the pipeline.
echo "original value is $(${{ variables.var1 }})"
With above change, after you triggered the pipeline:
At the compile time, the expression "${{ variables.var1 }}" will be replaced with the actual value "android_google_preprod_account_activation_url". So, the expression "$(${{ variables.var1 }})" will be changed to "$(android_google_preprod_account_activation_url)".
Then at the runtime, the expression will be parsed as the correct value "preprod.google.com".
Below is an example I have tested on my side.
YAML
variables:
android_google_preprod_account_activation_url: 'preprod.google.com'
var1: 'android_google_preprod_account_activation_url'
jobs:
- job: A
displayName: 'Job A'
pool:
vmImage: ubuntu-latest
steps:
- checkout: none
- task: Bash#3
displayName: 'Print variables'
inputs:
targetType: inline
script: |
echo "android_google_preprod_account_activation_url = $(android_google_preprod_account_activation_url)"
echo "var1 = $(var1)"
echo "original value = $(${{ variables.var1 }})"
Result
For more details, you can reference the related document "Understand variable syntax".
I have a yml file, from which I am obtaining a key with yq and I am storing it in a variable in bash, I am trying to do another query with the value of the variable but it does not give the expected result
file.yml
version: '3'
services:
task_auth:
environment:
AWS_API_VERSION: "2016-04-19"
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: /creds
PORT: "8000"
SES_AWS_ACCESS_KEY: xxxxx
SES_AWS_SECRET_KEY: xxxxx
image: xxxxxxx
ports:
- "8000:8000"
yq e '(.services | keys)[]' file.yml
Result:
task_auth
Storing the query in a variable TASK_NAME
TASK_NAME=$(yq e '(.services | keys)[]' file.yml)
Calling the variable to see the previously stored value
$TASK_NAME
bash: task_auth: command not found
yq eval '.services.$TASK_NAME.environment.PORT' file.yml
Result
null
if I execute the command echo $TASK_NAME I get the correct value task_auth but to make queries from yq no
Now when I need this other query I need to use the initial variable, how can I do the query using that value of $TASK_NAME in yq ?
Thanks in advance
Provide the Bash variable as environment variable, and inside mikefarah/yq use env to retrieve it:
TASK_NAME="$TASK_NAME" yq e '.services[env(TASK_NAME)].environment.PORT' file.yml
8000
See the manual section Read string environment variable
I have Terraform variables defined like
variable "location" {
type = string
default = "eastus"
description = "Desired Azure Region"
}
variable "resource_group" {
type = string
default = "my-rg"
description = "Desired Azure Resource Group Name"
}
and potentially / partially overwritten in terraform.tfvars file
location = "westeurope"
and then defined variables as outputs e.g. a file outputs.tf:
output "resource_group" {
value = var.resource_group
}
output "location" {
value = var.location
}
How can I "source" the effective variable values in a bash script to work with these values?
One way is to use Terraform output values as JSON and then an utility like jq to convert and source as variables:
source <(terraform output --json | jq -r 'keys[] as $k | "\($k|ascii_upcase)=\(.[$k] | .value)"')
note that output is only available after executing terraform plan, terraform apply or even a terraform refresh
If jq is not available or not desired, sed can be used to convert Terraform HCL output into variables, even with upper case variable names:
source <(terraform output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
or using -chdir argument if Terraform templates / modules are in another folder:
source <(terraform -chdir=$TARGET_INFRA_FOLDER output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
Then these variables are available in bash script:
LOCATION="westeurope"
RESOURCE_GROUP="my-rg"
and can be addressed as $LOCATION and $RESOURCE_GROUP.
I have test$001 as a value in Jenkins secret text credentials. Later in pipeline script i'm accessing that value and writing it to yaml file like mentioned below, which is used as K8S configmap.
Problem is with the Dollar sign in the value.
environment {
TEST_CRED=credentials('TEST_CRED')
}
script.sh
cat << EOF > test.yaml
...
data:
TEST: ${TEST_CRED}
EOF
Expected: test$001
Printed: test$$001 (Note extra dollar sign being inserted automatically)
I tried all possibilities to escape this dollar sign, nothing worked.
TEST_01: '${TEST_CRED}'
TEST_02: ${TEST_CRED}
TEST_03: '$${TEST_CRED}'
TEST_04: $${TEST_CRED}
TEST_05: "$${TEST_CRED}"
TEST_08: $TEST_CRED
When storing value in Jenkins secret text credentials, escape the dollar sign. So, test$001 should actually be stored as test\$001.
Following works for me:
pipeline {
agent any
environment {
MYTEST_CRED=credentials('TEST_CRED')
}
stages {
stage('Special Char') {
steps {
sh """
cat << EOF > test.yaml
Name: test-config
Namespace: default
data:
TEST: ${MYTEST_CRED}
EOF
"""
}
}
}
}
Output:
This is an example when I'm passing a not escaped string to the Jenkins job via parameters. And things are not going my way.
// Original and expected value. Works fine with pure groovy
echo env.SECRET_VALUE
test#U$3r
// But this variable in shell is getting messed up
// sh("\$ENV") and sh('$ENV') are using value of shell env variale
sh("echo \$SECRET_VALUE")
test#U$$3r
sh('echo $SECRET_VALUE')
test#U$$3r
// sh("$ENV") and sh("${ENV}") are using value of groovy variables passed to the shell
sh("echo $SECRET_VALUE")
test#Ur
sh("echo ${SECRET_VALUE}")
test#Ur
Let's try to fix it
env.ESCAPED_SECRET_VALUE = env.SECRET_VALUE.replaceAll(/(!|"|#|#|\$|%|&|\\/|\(|\)|=|\?)/, /\\$0/)
// groovy variable is becoming a bit broken
echo env.ESCAPED_SECRET_VALUE
test\#U\$3r
// shell env variable is still broken
sh("echo \$ESCAPED_SECRET_VALUE")
test\#U\$$3r
sh('echo $ESCAPED_SECRET_VALUE')
test\#U\$$3r
// But, if we will pass groovy env variable to the shell - it looks good
sh("echo $ESCAPED_SECRET_VALUE")
test#U$3r
sh("echo ${ESCAPED_SECRET_VALUE}")
test#U$3r
If You are using command straight in the sh(script:""), then just pass groovy ESCAPED variable. If You need to invoke shell script file, then try to pass value of this groovy ESCAPED variable as input argument into it
Example:
sh("./my_super_script.sh $ESCAPED_SECRET_VALUE")
# my_super_script.sh
#!/bin/bash
SECRET_VALUE=$1
echo $SECRET_VALUE
I did a setup as per your requirement and got the desired results.
The setup is shown below with the screenshots,
Setup Jenkins secret text credential
Setup Binding in the Jenkins job
Configuring the build to create the test.yaml
Content of test.yaml
$ cat test.yaml
...
data:
TEST: test$001
I have a playbook that queries a server for it's SystemID which can be converted to a model number using a vendor-provided table that maps the id to a model. The server returns a decimal value but the table uses the hexadecimal equivalent.
What I want to do is to convert the decimal string to an hexadecimal string that can be matched with an entry in the vendor-provided table.
Example:
Server returns: SystemID = 1792
Matching entry in vendor table: 0x0700
I've searched in the Ansible documentation and Web searched for either a native Ansible command or jinja2 expression to do the conversion.
I've only found the int(value, base=x) jinja2 function that does the opposite of what I am trying to do.
The native python hex() command can do it. But I'd like to avoid that if possible.
Here is the playbook task that parses the servers stdout to get systemid value:
set_fact:
server_model: "{{ ( server_model_results.stdout_lines | select('match','SystemID' ) | list| first ).split('=')[1] | regex_replace('^\ |\ /$', '' ) }}"
Environment:
Ansible 2.9.7
Python 3.8.0
macOS 10.15.4
You can use a python format with the % operator inside a jinja2 template string:
$ ansible localhost -m debug -a msg="{{ '%#x' % 1792 }}"
localhost | SUCCESS => {
"msg": "0x700"
}
You will probably still have to deal with the leading 0 that is present in your file (i.e. 0x0700).
If all your values are padded to 4 hexa digits in your file (i.e. after the 0x prefix) a quick and dirty solution could be:
$ ansible localhost -m debug -a msg="0x{{ '%04x' % 1792 }}"
localhost | SUCCESS => {
"msg": "0x0700"
}
If not, you will have to implement some kind of dynamic 0 padding to the next odd number of chars yourself.
You might want to switch the 'x' type specifier to 'X' (see doc link above) if hexa digits above nine are uppercase in your vendor table
$ ansible localhost -m debug -a msg="0x{{ '%04x' % 2569 }}"
localhost | SUCCESS => {
"msg": "0x0a09"
}
$ ansible localhost -m debug -a msg="0x{{ '%04X' % 2569 }}"
localhost | SUCCESS => {
"msg": "0x0A09"
}