I'm having some issues with a Jenkins Job Builder YAML file which contains an attribute (message-content) with "{}" characters in it:
- job-template:
id: senderjob
name: '{job-prefix}{id}'
command: 'echo "executed with $PARAMETER"'
type: freestyle
properties:
- ownership:
enabled: true
owner: user1
- build-discarder:
num-to-keep: 100
parameters:
- string:
name: PARAMETER
default: 'param'
description: 'default parameter for message.'
# template settings
builders:
- shell: '{command}'
publishers:
- ci-publisher:
override-topic: VirtualTopic.abcd.message
message-type: 'Custom'
message-properties: |
release-name=$PARAMETER
PARAMETER=$PARAMETER
message-content: '{"release-name" : "1.0"}'
The error reported is:
jenkins_jobs.errors.JenkinsJobsException: release-name parameter missing to format {release-name : 1.0}
So it looks like it's trying to expand "release-name" with a parameter.
So I have added it as parameter:
- string:
name: release-name
default: '1.0'
description: 'default parameter for release.'
It still reports the same error. Should I include the parameter somewhere else or escape it ? I couldn't find any YAML escape for "{}" characters though.
Any idea?
You should add the following to the configuration file
[job_builder]
allow_empty_variables = True
Link: https://jenkins-job-builder.readthedocs.io/en/latest/definition.html#job-template-1
Related
I'm trying to use variables inside variables in azure pipelines.
Below is an example of the bash script:
#!/bin/bash
customer=google
environment=preprod
android_google_preprod_account_activation_url=preprod.google.com
echo "Customer is $customer"
echo "Environment is $environment"
var1=android_${customer}_${environment}_account_activation_url
echo "variable is $var1"
echo "original value is ${!var1}"
I get the expected output for the above bash script when I run it on my Ubuntu server, with NO errors:
Customer is google
Environment is preprod
variable is android_google_preprod_account_activation_url
original value is preprod.google.com
The yml code for azure pipelines is:
parameters:
- name: customer
displayName: 'select customer'
type: string
values:
- google
- name: environment
displayName: 'select environment'
type: string
values:
- preprod
variables:
- group: android-${{ parameters.customer }}-${{ parameters.environment }}
- name: var1
value: android-${{ parameters.customer }}-${{ parameters.environment }}-account-activation-url
script: |
echo "Customer is $(customer)"
echo "Environment is $(environment)"
echo "variable is $(var1)"
echo "original value is $(!var1)"
displayName: 'echo variables'
The value of android-google-preprod-account-activation-url is being taken from variable groups inside library.
It gives me an error for the 4th line:
invalid indirect expansion
The first 3 lines output is as expected.
Expected output is:
Customer is google
Environment is preprod
variable is android_google_preprod_account_activation_url
original value is preprod.google.com
Is there a different syntax that needs to be followed in azure pipelines?
I`m not a bash expert ))) however... you're trying to use the parameters expansion What is indirect expansion? What does ${!var*} mean?
But it refers to the bash variables.... when you define variables in the devops pipeline, you have to use them as environment variables or through the macro.
or something like that:
android_google_preprod_account_activation_url=preprod.google.com
echo "Customer is $(customer)"
echo "Environment is $(environment)"
var1=android_$(customer)_$(environment)_account_activation_url
echo "variable is $var1"
echo "original value is ${!var1}"
The macro syntax "$(varName)" is a proprietary syntax in Azure Pipelines to interpolate variable values. It is processed during runtime and different with the syntax "${varName}" in Bash scripts.
For your case, you can try to use the compile time syntax "${{ variables.varName }}" to get the value in the pipeline.
echo "original value is $(${{ variables.var1 }})"
With above change, after you triggered the pipeline:
At the compile time, the expression "${{ variables.var1 }}" will be replaced with the actual value "android_google_preprod_account_activation_url". So, the expression "$(${{ variables.var1 }})" will be changed to "$(android_google_preprod_account_activation_url)".
Then at the runtime, the expression will be parsed as the correct value "preprod.google.com".
Below is an example I have tested on my side.
YAML
variables:
android_google_preprod_account_activation_url: 'preprod.google.com'
var1: 'android_google_preprod_account_activation_url'
jobs:
- job: A
displayName: 'Job A'
pool:
vmImage: ubuntu-latest
steps:
- checkout: none
- task: Bash#3
displayName: 'Print variables'
inputs:
targetType: inline
script: |
echo "android_google_preprod_account_activation_url = $(android_google_preprod_account_activation_url)"
echo "var1 = $(var1)"
echo "original value = $(${{ variables.var1 }})"
Result
For more details, you can reference the related document "Understand variable syntax".
I want to build a pipeline function that replaces a value in a yaml file. For that I want to make both the
pattern and the replacement value variable. I have seen the env-variables-operators article in the yq docs, however I cannot find the relevant section.
I have a yaml file with the following content:
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "1.0.0"
I now want to build a pipeline function that will replace the value of the value key in the yaml.
I can do so with:
$ yq '.spec.source.helm.parameters[0].value = "2.0.0"' myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "2.0.0"
Now I want to make this command customizable.
What works:
$ VALUE=3.0.0
$ replacement=$VALUE yq '.spec.source.helm.parameters[0].value = env(replacement)' myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "3.0.0"
What doesn't work
$ VALUE=3.0.0
$ PATTERN=.spec.source.helm.parameters[0].value
$ replacement=$VALUE pattern=$PATTERN yq 'env(pattern) = env(replacement)'
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "1.0.0"
I have also tried to use strenv and wrapping the replacement pattern in quotes, but it is not working.
Can anyone help me with the correct syntax?
You can import data with env but not code. You could inject it (note the changes in the quoting), but this is bad practice as it makes your script very vulnerable:
VALUE='3.0.0'
PATTERN='.spec.source.helm.parameters[0].value'
replacement="$VALUE" yq "${PATTERN} = env(replacement)" myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "3.0.0"
Better practice would be to import the path in a form that is interpretable by yq, e.g. as an array and using setpath:
VALUE='3.0.0'
PATTERN='["spec","source","helm","parameters",0,"value"]'
replacement="$VALUE" pattern="$PATTERN" yq 'setpath(env(pattern); env(replacement))' myyaml.yml
I don't understand how lineinfile works, I use:
- name: "configure"
lineinfile:
path: /etc/apt/apt.conf.d/50unattended-upgrades
regexp: '^\/\/ "\${distro_id}:\${distro_codename}-updates";'
line: ' "${distro_id}:${distro_codename}-updates";'
state: present
And I expect that line in file will be uncommented:
// "${distro_id}:${distro_codename}-updates";
instead of this, this line stay as it is, but at the end of file,
I see:
"${distro_id}:${distro_codename}-updates";
so "match" happens, but why there is new line at the end of file instead of replace?
Your regexp does not escape all of the special symbols and therefore there is no match which causes the addition of the new line instead of the replacement. Curly braces should be escaped, according to Python's re module documentation.
This task works as expected on my Ubuntu 18.04 server.
- name: "configure"
lineinfile:
path: /etc/apt/apt.conf.d/50unattended-upgrades
regexp: '^//\s+"\$\{distro_id\}:\$\{distro_codename\}-updates";'
line: ' "${distro_id}:${distro_codename}-updates";'
state: present
Diff
-// "${distro_id}:${distro_codename}-updates";
+ "${distro_id}:${distro_codename}-updates";
There must be other problem. The code works as expexted
shell> diff 50unattended-upgrades 50unattended-upgrades.orig
1c1
< "${distro_id}:${distro_codename}-updates";
---
> // "${distro_id}:${distro_codename}-updates";
I am trying to put a together Puppet bolt plan in YAML format.
I got it working in .pp file and here is the plan
plan profiles::chg123456(
TargetSpec $nodes,
) {
apply($nodes) {
logrotate::rule {'proftpd':
path => ['/var/log/proftpd/*.log', '/var/log/xferlog', '/var/log/proftpd.system.log', '/var/log/sftp.log', '/var/log/sftp-xferlog',],
maxsize => '100m',
rotate_every => 'week',
compress => true,
ifempty => true,
missingok => true,
sharedscripts => true,
postrotate => 'test -f /var/lock/subsys/proftpd && /usr/bin/killall -HUP proftpd || :'
}
}
}
It worked and created /etc/logrotate.d/proftpd with all the correct settings.
Now I want to convert to YAML format but no idea how to do that.
Here is what I guessed but bolt plan show keep saying
$ bolt plan show
Parse error in step "chg123456":
No valid action detected (file: C:/Users/puppet/msys64/home/puppet/.puppetlabs/bolt/modules/profiles/plans/chg123456.yaml)
My YAML plan looks like follows
parameters:
nodes:
type: TargetSpec
steps:
- name: chg123456
target: $nodes
logrotate::rules:
proftpd:
path:
- '/var/log/proftpd/*.log'
- '/var/log/xferlog'
- '/var/log/proftpd.system.log'
- '/var/log/sftp.log'
- '/var/log/sftp-xferlog'
maxsize: '100m'
compress: true
ifempty: true
missingok: true
sharedscripts: true
postrotate: 'test -f /var/lock/subsys/proftpd && /usr/bin/killall -HUP proftpd || :'
return: $chg123456
What am I doing wrong?
Thanks
You'll want to use a resources step, and list the resources you want to use in yaml (documentation):
parameters:
nodes:
type: TargetSpec
steps:
- name: chg123456
target: $nodes
resources:
- logrotate::rules: proftpd
parameters:
path:
- '/var/log/proftpd/*.log'
- '/var/log/xferlog'
- '/var/log/proftpd.system.log'
- '/var/log/sftp.log'
- '/var/log/sftp-xferlog'
maxsize: '100m'
compress: true
ifempty: true
missingok: true
sharedscripts: true
postrotate: 'test -f /var/lock/subsys/proftpd && /usr/bin/killall -HUP proftpd || :'
return: $chg123456
In response to one comment, bolt plan convert is only used to convert yaml plans into Puppet plans, not the other way around.
In a bash script in a Heat template, is it possible to use a parameter value from that template?
Yes, according to the Heat Orchestration Template specification, you can accomplish this with the str_replace function. They give an example that uses str_replace, together with get_param, to use a parameter value DBRootPassword in a bash script:
parameters:
DBRootPassword:
type: string
label: Database Password
description: Root password for MySQL
hidden: true
resources:
my_instance:
type: OS::Nova::Server
properties:
# general properties ...
user_data:
str_replace:
template: |
#!/bin/bash
echo "Hello world"
echo "Setting MySQL root password"
mysqladmin -u root password $db_rootpassword
# do more things ...
params:
$db_rootpassword: { get_param: DBRootPassword }
Each key in params is replaced in template with its value. Since $db_rootpassword's value is set to the result of get_param, that means the parameter is passed into the bash script wherever $db_rootpassword is used.