I dont have any detail knowledge on yq.
template1.yaml
spec:
template:
temp:
vars:
- name: first
env: []
template2.yaml
env:
-name: "first"
value: 1
-name: "two"
value: 2
I want to add env array of template2.yaml to template1.yaml's env array using yq. How can we do this ??
Which tool called yq are you using?
Using mikefarah/yq (tested with v4.20.2):
yq '
.spec.template.temp.vars[].env += load("template2.yaml").env
' template1.yaml
Using kislyuk/yq (tested with v3.0.2):
yq -y '
.spec.template.temp.vars[].env += input.env
' template1.yaml template2.yaml
Output:
spec:
template:
temp:
vars:
- name: first
env:
- name: "first"
value: 1
- name: "two"
value: 2
Note: This assumes, your template2.yaml looks more like this:
env:
- name: "first"
value: 1
- name: "two"
value: 2
Related
spec:
templates:
- name: some_step1
script:
image: some_image
imagePullPolicy: IfNotPresent
- name: some_step2
activeDeadlineSeconds: 300
script:
image: some_image
imagePullPolicy: IfNotPresent
env:
- name: HOST
value: "HOST"
- name: CONTEXT
value: "CONTEXT"
I would like to update the value for CONTEXT. The following command updates the value however it does not return the whole yaml:
yq '.spec.templates | map(select(.name == "some_step2")).[].script.env | map(select(.name == "CONTEXT").value = "CONTEXT1")' test.yaml
The output of the command above:
- name: HOST
value: "HOST"
- name: CONTEXT
value: "CONTEXT1"
How can I make changes in the yq command above to update the value for CONTEXT and return the whole yaml?
I am using mikefarah/yq version v4.30.6.
Use parentheses around the LHS of the assignment to retain the context, i.e. (…) = ….
( .spec.templates[]
| select(.name == "some_step2").script.env[]
| select(.name == "CONTEXT").value
) = "CONTEXT1"
spec:
templates:
- name: some_step1
script:
image: some_image
imagePullPolicy: IfNotPresent
- name: some_step2
activeDeadlineSeconds: 300
script:
image: some_image
imagePullPolicy: IfNotPresent
env:
- name: HOST
value: "HOST"
- name: CONTEXT
value: "CONTEXT1"
I have a YAML document like this
services:
- name: newlogd
image: NEWLOGD_TAG
cgroupsPath: /eve/services/newlogd
oomScoreAdj: -999
- name: edgeview
image: EDGEVIEW_TAG
cgroupsPath: /eve/services/eve-edgeview
oomScoreAdj: -800
- name: debug
image: DEBUG_TAG
cgroupsPath: /eve/services/debug
oomScoreAdj: -999
- name: wwan
image: WWAN_TAG
cgroupsPath: /eve/services/wwan
oomScoreAdj: -999
I need to insert a new object AFTER given element e.g. with name == "edgeview". so the output looks like this
services:
- name: newlogd
image: NEWLOGD_TAG
cgroupsPath: /eve/services/newlogd
oomScoreAdj: -999
- name: edgeview
image: EDGEVIEW_TAG
cgroupsPath: /eve/services/eve-edgeview
oomScoreAdj: -800
- name: new_element_name
image: new_element_image
- name: debug
image: DEBUG_TAG
cgroupsPath: /eve/services/debug
oomScoreAdj: -999
- name: wwan
image: WWAN_TAG
cgroupsPath: /eve/services/wwan
oomScoreAdj: -999
I couldn't find anything about it in YQ documentation. Is it even possible using YQ?
UPDATE: I'm using YQ https://github.com/mikefarah/yq version 4.28.1. I was not aware that there several tools with the same name.
Using the YAML processor yq:
yq --arg insertAfter edgeview -y '
limit(1; .services | to_entries[] | select(.value.name == $insertAfter) | .key + 1) as $idx
| .services |= .[0:$idx] +
[{name: "new_element_name", image: "new_element_image"}] +
.[$idx:]'
In the first line, the index of the element to be inserted after is determined and stored into $idx.
If you have several elements with the same name, only the first match is used (limit).
In the following filter step, $idx is used to split the array and insert the new element at the desired position.
Output
services:
- name: newlogd
image: NEWLOGD_TAG
cgroupsPath: /eve/services/newlogd
oomScoreAdj: -999
- name: edgeview
image: EDGEVIEW_TAG
cgroupsPath: /eve/services/eve-edgeview
oomScoreAdj: -800
- name: new_element_name
image: new_element_image
- name: debug
image: DEBUG_TAG
cgroupsPath: /eve/services/debug
oomScoreAdj: -999
- name: wwan
image: WWAN_TAG
cgroupsPath: /eve/services/wwan
oomScoreAdj: -999
In the mikefarah/yq (v.4.30+) you can do:
yq '.services |= (
(.[] | select(.name == "edgeview") | key + 1) as $pos |
.[:$pos] +
[{"name": "new_element_name", "image": "new_element_image"}] +
.[$pos:])' examples/data1.yaml
Explanation:
You want to update the 'services' array, so we have '.services |= '
Find the position to update the array in, which is one after then one with name "edgeview". Splat the array, find the matching entry, get its index and add one (.[] | select(.name == "edgeview") | key + 1)
Assign the value of that position to $pos.
Now you can use the array slice operator to reconstruct the array with the new element.
Add the start of the array, [:$pos] with the new element {"name": "new_element_name"} to the end of the array [$pos:]
Disclaimer: I wrote yq
I have a yaml configuration as follows:
parameters:
group: '$(group)'
acl:
certificateFile: AclCertificates.p12
provisioningProfileFile: AmericashDisProfile.mobileprovision
keystore: 'acl.jks'
sail:
certificateFile: AclCertificates.p12
provisioningProfileFile: AmericashDisProfile.mobileprovision
keystore: 'acl.jks'
steps:
- bash: |
echo ${{ parameters[$(group)]['certificateFile'] }}
I want to access the object value using the dynamic key. Here group: '$(group)' is a dynamic value which is coming from another var file.
I have tried a way of access the object value like ${{ parameters[$(group)]['certificateFile'] }} But its not working. I'm not able to figure out that how should i pass the parameter group in the echo ${{ parameters[$(group)]['certificateFile'] }} in order to get specific object's value.
For example, you have a YAML pipeline A:
parameters:
- name: test
type: object
default:
- name: Name1
path: Path1
- name: Name2
path: Path2
variables:
sth: ${{ join(';',parameters.test.*.name) }}
And then you can use YAML pipeline B to get the object value:
variables:
- template: azure-pipelines-2.yml # Template reference
steps:
- task: CmdLine#2
inputs:
script: 'echo "${{variables.sth}}"'
This question already has an answer here:
Need help on bask awk to update Yaml file by finding the pattern in the file
(1 answer)
Closed 1 year ago.
I am new to bash.. looking some advise on one issue mentioned below.
I have config file below.
impulse.yaml
- job_name: orch
value: CPST
- group: indalco
value1: wr
- monitor:
- name: quid
cnt: 2
- name: kwid
cnt: 3
- name: knid
cnt: 4
- interval: 3m
- static_configs:
- targets: targets1
labels:
group: BSA
gid: geo
dc: lba
if i run the bash script, it should update like below
need to be updated impulse.yaml
- job_name: orch
value: CPST
- group: indalco
value1: wr
- monitor:
- name: quid
cnt: 2
- name: kwid
cnt: 3
- name: knid
cnt: 4
- name: orch_vm1
- name: orch_vm2
- interval: 3m
- static_configs:
- targets: targets1
labels:
group: BSA
gid: geo
dc: lba
------------------bash script---------
getline() {
awk '
BEGIN { ln=1; find_monitor=0; }
(find_monitor==1 && $0~/^[a-z]/) { exit }
($0~/^monitor:/) { find_monitor = 1 ;ln = NR }
END { print ln }' ${1}
}
word="monitor" # no use of this variable
echo $line
filename="impulse.yaml"
for vm_name in orch_vm1 orch_vm2;
do
line=`getline $filename $word`
sed -i -e ${line}"a\ - name: \"${vm_name}\" " $filename
the code right now is updating at the begning of the monitor section of yaml file like below..but it needs to be updated at the end of the monitor section before interval section. Please advise what pattern matching technic can be applied.
- job_name: orch
value: CPST
- group: indalco
value1: wr
- monitor:
- name: orch_vm1
- name: orch_vm2
- name: quid
cnt: 2
- name: kwid
cnt: 3
- name: knid
cnt: 4
- interval: 3m
- static_configs:
- targets: targets1
labels:
group: BSA
gid: geo
dc: lba
I agree with #LéaGris’ comment. Structured data like YAML should be interpreted via its defined syntax. Traditional command line tools can't do this. yq is the closest analogue
I am passing in a parameter to the pipeline as shown below -
parameters:
- name: ListofValues
displayName: values for test
type: object
default:
- test:
name: "test"
desired: '"abc","xyz"'
When I am passing these values in bash step it is coming as
"abc,xyz" and I am expecting it to be "abc","xyz" is there a way I can fix this ?
values may look similar, kindly look for " after end of value 1.
You can use ${{ convertToJson() }} to keep the double quota.
My sample yaml(please notice the structure of test):
parameters:
- name: ListofValues
displayName: values for test
type: object
default:
test:
name: "test"
desired: '"abc","xyz"'
trigger: none
pool:
vmImage: ubuntu-latest
steps:
- script: |
echo "${{ convertToJson(parameters.listOfValues.test.desired) }}"
Output value is bash task:
Or you can add \ before quota for the value:
My sample yaml:
parameters:
- name: ListofValues
displayName: values for test
type: object
default:
test:
name: "test"
desired: '\"abc\",\"xyz\"'
trigger: none
pool:
vmImage: ubuntu-latest
steps:
- script: |
echo ${{ parameters.ListofValues.test.desired }}
var="abc,xyz"
IFS="," read -a array <<< $var
var=""
for word in "${array[#]}"; do
var="$var\"$word\","
done
var="${var%,}"
echo "${var}"
# outputs:
# "abc","xyz"
Read the elements of var delimited by , into an array named array.
Then loop over the elements in the array and concatenate them into var while adding the required ".
At the end remove the extra trailing ,.
This solution works with any number of elements.