I've this YAML file:
developers:
bob:
softwares:
- yq:
version: "1.2.3"
- visual-studio-code:
version: "1.2.3"
john:
softwares:
- xcode:
version: "1.2.3"
- jq:
version: "1.2.3"
and I'm trying to modify it with yq to get this result:
developers:
bob:
softwares:
- yq:
version: "1.2.3"
license-owner: bob
- visual-studio-code:
version: "1.2.3"
license-owner: bob
john:
softwares:
- xcode:
version: "1.2.3"
license-owner: john
- jq:
version: "1.2.3"
license-owner: john
Please note I want to use the 2nd level key as value for license-owner.
By using the following formula I got a result similar to what I want, however the context has been changed and only the updated segment is returned.
yq '.developers.* | .softwares[].*.license-owner = (. | key)' test.yml
produces:
softwares:
- yq:
version: "1.2.3"
license-owner: bob
- visual-studio-code:
version: "1.2.3"
license-owner: bob
softwares:
- xcode:
version: "1.2.3"
license-owner: john
- jq:
version: "1.2.3"
license-owner: john
Any idea how to get the expected result?
Use |= to update while keeping the context:
yq '.developers[] |= .softwares[][].license-owner = key' test.yaml
developers:
bob:
softwares:
- yq:
version: "1.2.3"
license-owner: bob
- visual-studio-code:
version: "1.2.3"
license-owner: bob
john:
softwares:
- xcode:
version: "1.2.3"
license-owner: john
- jq:
version: "1.2.3"
license-owner: john
Tested with mikefarah/yq version 4.27.2
Another, imo more readable option would be to use with_entries, which then would work in both mikefarah/yq and kislyuk/yq (note that the latter would also require license-owner to be quoted as it contains "special characters"):
.developers |= with_entries(.value.softwares[][]."license-owner" = .key)
Related
I have a YAML document like this
services:
- name: newlogd
image: NEWLOGD_TAG
cgroupsPath: /eve/services/newlogd
oomScoreAdj: -999
- name: edgeview
image: EDGEVIEW_TAG
cgroupsPath: /eve/services/eve-edgeview
oomScoreAdj: -800
- name: debug
image: DEBUG_TAG
cgroupsPath: /eve/services/debug
oomScoreAdj: -999
- name: wwan
image: WWAN_TAG
cgroupsPath: /eve/services/wwan
oomScoreAdj: -999
I need to insert a new object AFTER given element e.g. with name == "edgeview". so the output looks like this
services:
- name: newlogd
image: NEWLOGD_TAG
cgroupsPath: /eve/services/newlogd
oomScoreAdj: -999
- name: edgeview
image: EDGEVIEW_TAG
cgroupsPath: /eve/services/eve-edgeview
oomScoreAdj: -800
- name: new_element_name
image: new_element_image
- name: debug
image: DEBUG_TAG
cgroupsPath: /eve/services/debug
oomScoreAdj: -999
- name: wwan
image: WWAN_TAG
cgroupsPath: /eve/services/wwan
oomScoreAdj: -999
I couldn't find anything about it in YQ documentation. Is it even possible using YQ?
UPDATE: I'm using YQ https://github.com/mikefarah/yq version 4.28.1. I was not aware that there several tools with the same name.
Using the YAML processor yq:
yq --arg insertAfter edgeview -y '
limit(1; .services | to_entries[] | select(.value.name == $insertAfter) | .key + 1) as $idx
| .services |= .[0:$idx] +
[{name: "new_element_name", image: "new_element_image"}] +
.[$idx:]'
In the first line, the index of the element to be inserted after is determined and stored into $idx.
If you have several elements with the same name, only the first match is used (limit).
In the following filter step, $idx is used to split the array and insert the new element at the desired position.
Output
services:
- name: newlogd
image: NEWLOGD_TAG
cgroupsPath: /eve/services/newlogd
oomScoreAdj: -999
- name: edgeview
image: EDGEVIEW_TAG
cgroupsPath: /eve/services/eve-edgeview
oomScoreAdj: -800
- name: new_element_name
image: new_element_image
- name: debug
image: DEBUG_TAG
cgroupsPath: /eve/services/debug
oomScoreAdj: -999
- name: wwan
image: WWAN_TAG
cgroupsPath: /eve/services/wwan
oomScoreAdj: -999
In the mikefarah/yq (v.4.30+) you can do:
yq '.services |= (
(.[] | select(.name == "edgeview") | key + 1) as $pos |
.[:$pos] +
[{"name": "new_element_name", "image": "new_element_image"}] +
.[$pos:])' examples/data1.yaml
Explanation:
You want to update the 'services' array, so we have '.services |= '
Find the position to update the array in, which is one after then one with name "edgeview". Splat the array, find the matching entry, get its index and add one (.[] | select(.name == "edgeview") | key + 1)
Assign the value of that position to $pos.
Now you can use the array slice operator to reconstruct the array with the new element.
Add the start of the array, [:$pos] with the new element {"name": "new_element_name"} to the end of the array [$pos:]
Disclaimer: I wrote yq
I dont have any detail knowledge on yq.
template1.yaml
spec:
template:
temp:
vars:
- name: first
env: []
template2.yaml
env:
-name: "first"
value: 1
-name: "two"
value: 2
I want to add env array of template2.yaml to template1.yaml's env array using yq. How can we do this ??
Which tool called yq are you using?
Using mikefarah/yq (tested with v4.20.2):
yq '
.spec.template.temp.vars[].env += load("template2.yaml").env
' template1.yaml
Using kislyuk/yq (tested with v3.0.2):
yq -y '
.spec.template.temp.vars[].env += input.env
' template1.yaml template2.yaml
Output:
spec:
template:
temp:
vars:
- name: first
env:
- name: "first"
value: 1
- name: "two"
value: 2
Note: This assumes, your template2.yaml looks more like this:
env:
- name: "first"
value: 1
- name: "two"
value: 2
I'm working on a concourse pipeline and I need to duplicate a lot of code in my YAML so I'm trying to refactor it so it is easily maintainable and I don't end up with thousands of duplicates lines/blocks.
I have achieve the following yaml file after what seems to be the way to go but it doesn't fullfill all my needs.
add-rotm-points: &add-rotm-points
task: add-rotm-points
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ((registre))/polygone/concourse/cf-cli-python3
tag: 0.0.1
insecure_registries: [ ((registre)) ]
run:
path: source-pipeline/commun/rotm/trigger-rotm.sh
args: [ "source-pipeline", "source-code-x" ]
inputs:
- name: source-pipeline
- name: source-code-x
jobs:
- name: test-a
plan:
- in_parallel:
- get: source-pipeline
- get: source-code-a
trigger: true
- <<: *add-rotm-points
- name: test-b
plan:
- in_parallel:
- get: source-pipeline
- get: source-code-b
trigger: true
- <<: *add-rotm-points
My problem is that both my jobs uses the generic task defined at the top. But in the generic task I need to change source-code-x to the -a or -b version I use in my jobs.
I cannot find a way to achieve this without duplicating my anchor in every jobs and that seems to be counter productive. But i may not have full understood yaml anchors/merges.
All you need to do is map inputs on individual tasks, like this:
add-rotm-points: &add-rotm-points
task: add-rotm-points
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ((registre))/polygone/concourse/cf-cli-python3
tag: 0.0.1
insecure_registries: [ ((registre)) ]
run:
path: source-pipeline/commun/rotm/trigger-rotm.sh
args: [ "source-pipeline", "source-code-x" ]
inputs:
- name: source-pipeline
- name: source-code-x
jobs:
- name: test-a
plan:
- in_parallel:
- get: source-pipeline
- get: source-code-a
trigger: true
- <<: *add-rotm-points
input_mapping:
source-code-x: source-code-a
- name: test-b
plan:
- in_parallel:
- get: source-pipeline
- get: source-code-b
trigger: true
- <<: *add-rotm-points
input_mapping:
source-code-x: source-code-b
See Example Three in this blog: https://blog.concourse-ci.org/introduction-to-task-inputs-and-outputs/
I have the following yaml file:
name: myapp
app1:
version: 1.2
replicas: 1
app2:
version: 2.3
replicas: 1
app3:
version: 4.0
app4:
version: 2.4
I'd like to use yq to increment the values of each replicas field so it results in:
name: myapp
app1:
version: 1.2
replicas: 2
app2:
version: 2.3
replicas: 2
app3:
version: 4.0
app4:
version: 2.4
Any ideas ?
Use select with has to check if the field exists, and += to update it:
Using mikefarah/yq:
yq '(.[] | select(has("replicas")).replicas) += 1'
Using kislyuk/yq:
yq -y '(.[] | select(has("replicas")?).replicas) += 1'
I have a list of objects
List1:
- name: Rule1
description: Description1
sources:
ip_addresses:
- any
- name: Rule2
description: Description2
sources:
ip_addresses:
- any
- name: Rule3
description: Description3
sources:
ip_addresses:
- any
- name: Rule4
description: Description4
sources:
ip_addresses:
- any
I wanted to create a new list by omitting rules with certain descriptions, I was able to do it for single description by using:
List2: "{{
List1 | difference(
List1
| selectattr('description')
| selectattr('description', 'match', Description1)
| list) }}"
I am not able to figure out how to do so for multiple description, for example if we need to omit both Rule1 and Rule2. One way I was able to do so was creating List2 after omitting Rule1 and then creating a List3 where by using a similar filter on List2.
Create a list of the descriptions you want to omit. For example
omit_desc:
- Description1
- Description2
Then the task below does the job
- set_fact:
List2: "{{ List1|rejectattr('description', 'in', omit_desc)|list }}"
- debug:
var: List2
gives
List2:
- description: Description3
name: Rule3
sources:
ip_addresses:
- any
- description: Description4
name: Rule4
sources:
ip_addresses:
- any
See filter rejectattr
See test in