I would like to delete all keys that have a scalar value of false, using https://github.com/mikefarah/yq:
Input
project: false_deletion
fields:
name:
numeric: false
value: John
age:
value: 25
numeric: true
address:
value: 123 Main St
numeric: false
Desired output
project: false_deletion
fields:
name:
value: John
age:
value: 25
numeric: true
address:
value: 123 Main St
yq e 'del(.. | select(. == false))' input
Will del() all those keys where the value is false.
Output:
project: false_deletion
fields:
name:
value: John
age:
value: 25
numeric: true
address:
value: 123 Main St
Related
In Ansible, is it possible to output a list of dictionaries in pretty print YAML with flow style mapping?
Example of source list of dict:
my_servers:
- hostname: myhostname1
id: 101
reverse: 'true'
type: red
- hostname: myhostname2
id: 102
reverse: 'false'
type: white
- hostname: myhostname3
id: 103
reverse: 'true'
type: blue
- hostname: myhostname4
id: 104
reverse: 'false'
type: green
I'd like to output to be like this:
my_servers:
- {hostname: myhostname1, id: 101, type: red, reverse: "true"}
- {hostname: myhostname2, id: 102, type: white, reverse: "false"}
- {hostname: myhostname3, id: 103, type: blue, reverse: "true"}
- {hostname: myhostname4, id: 104, type: green, reverse: "false"}
The only way I got it to look like this was through custom Jinja templating. Is there another way someone can share?
#β.εηοιτ.βε above led to dig further and found the answer.
The key is to set default_flow_style=none in the to_yaml filter.
- name: Test
hosts: "localhost"
gather_facts: no
serial: 1
vars:
my_servers:
- hostname: myhostname1
id: 101
reverse: 'true'
type: red
- hostname: myhostname2
id: 102
reverse: 'false'
type: white
- hostname: myhostname3
id: 103
reverse: 'true'
type: blue
- hostname: myhostname4
id: 104
reverse: 'false'
type: green
tasks:
- debug:
msg: |
my_servers:
{{ my_servers | to_yaml(default_flow_style=none) }}
Gives this:
ok: [localhost] => {}
MSG:
my_servers:
- {hostname: myhostname1, id: 101, reverse: 'true', type: red}
- {hostname: myhostname2, id: 102, reverse: 'false', type: white}
- {hostname: myhostname3, id: 103, reverse: 'true', type: blue}
- {hostname: myhostname4, id: 104, reverse: 'false', type: green}
The problem now is to get proper indentation.
I dont have any detail knowledge on yq.
template1.yaml
spec:
template:
temp:
vars:
- name: first
env: []
template2.yaml
env:
-name: "first"
value: 1
-name: "two"
value: 2
I want to add env array of template2.yaml to template1.yaml's env array using yq. How can we do this ??
Which tool called yq are you using?
Using mikefarah/yq (tested with v4.20.2):
yq '
.spec.template.temp.vars[].env += load("template2.yaml").env
' template1.yaml
Using kislyuk/yq (tested with v3.0.2):
yq -y '
.spec.template.temp.vars[].env += input.env
' template1.yaml template2.yaml
Output:
spec:
template:
temp:
vars:
- name: first
env:
- name: "first"
value: 1
- name: "two"
value: 2
Note: This assumes, your template2.yaml looks more like this:
env:
- name: "first"
value: 1
- name: "two"
value: 2
I have a yaml file that looks like this:
---
component1:
tests:
- name: test1
type: manual
- name: test2
type: automated
- name: test3
type: manual
- name: test4
type: automated
- name: test5
type: manual
- name: test6
type: automated
component2:
tests:
- name: test1
type: manual
- name: test2
type: automated
- name: test3
type: manual
- name: test4
type: automated
- name: test5
type: manual
- name: test6
type: automated
I am trying to list only the automated tests, but keep the information about which component the tests are in.
I would like the output to be similar to this:
component1:
tests:
- name: test2
type: automated
- name: test4
type: automated
- name: test6
type: automated
component2:
tests:
- name: test2
type: automated
- name: test4
type: automated
- name: test6
type: automated
But I would also take any answer that allowed me to group the automated tests by component.
I have tried yq e '.[].tests | map(select(.type == "automated"))' file.yaml -P, but that just gives me the following output and the component information is lost and I just get the following output:
- name: test2
type: automated
- name: test4
type: automated
- name: test6
type: automated
- name: test2
type: automated
- name: test4
type: automated
- name: test6
type: automated
Assuming you are using mikefarah/yq, you can use the update select operator with |= to update the results back to the tests list
yq e '.[].tests |= map(select(.type == "automated"))' file.yaml -P
Note that, if you are using yq version above 4.18.1, the eval action e is the default one and can be skipped altogether.
Trying to deploy BOSH director integrated with LDAP and running into problems. Here are my steps.
Clone latest repository:
git clone https://github.com/cloudfoundry/bosh-deployment
Create vars file and populate it.
Modify opsfile for ldap
File: ldap.yml
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.enabled?
value: "true"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.profile_type
value: "search-and-bind"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.url
value: "ldap://XXXX.XXXXXXX.XXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.userDNPatternDelimiter
value: ";"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.userDN
value: "XXXXX\\XXXXXXXXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.userPassword
value: "XXXXXXXXXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.searchBase
value: "dc=XXX,dc=XXXXXXX,dc=XXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.searchFilter
value: "cn={0}"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.passwordAttributeName
value: "userPassword"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.passwordEncoder
value: "org.cloudfoundry.identity.uaa.ldap.DynamicPasswordComparator"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.sslCertificate
value: ""
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.ssl.skipVerification?
value: "true"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.ssl.tls
value: "none"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.mailAttributeName
value: "mail"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.mailSubstitute
value: ""
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.mailSubstituteOverrridesLdap?
value: "false"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.referral
value: "follow"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.groups.profile_type
value: "groups-map-to-scopes"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.groups.searchBase
value: "dc=XXX,dc=XXXXXXXXX,dc=XXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.groups.groupRoleAttribute
value: "member"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.groupSearchFilter
value: "member={0}"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.searchSubtree?
value: "true"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.maxSearchDepth
value: "1"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.emailDomain
value: ""
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.attributeMappings
value:
user.attribute.name-of-attribute-in-uaa-id-token: name-of-attribute-in-ldap-record
user.attribute.name-of-other-attribute-in-uaa-id-token: name-of-other-attribute-in-ldap-record
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.storeCustomAttributes?
value: "true"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.externalGroupWhitelist?
value: ""
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.add_shadow_user_on_login?
value: "true"
Run a deployment script:
#!/bin/bash
bosh create-env bosh-deployment/bosh.yml \
--state=state.json \
--vars-store=creds.yml \
--vars-file=vars.yml \
-o bosh-deployment/vsphere/cpi.yml \
-o bosh-deployment/uaa.yml \
-o bosh-deployment/ldap.yml \
-o bosh-deployment/misc/dns.yml \
-o bosh-deployment/misc/ntp.yml \
-o bosh-deployment/jumpbox-user.yml
Getting error:
Deployment manifest: '/Users/user1/workarea/bosh-1/bosh-deployment/bosh.yml'
Deployment state: 'state.json'
Started validating
Failed validating (00:00:00)
Parsing release set manifest '/Users/user1/workarea/bosh-1/bosh-deployment/bosh.yml':
Evaluating manifest:
Expected to find a map at path '/instance_groups/name=bosh/jobs/uaa' but found '[]interface {}'
Exit code 1
All works well if I remove ops file for LDAP integration (bosh-deployment/ldap.yml) - BOSH, UAA and etc get deployed. I don't understand why would it complain. UAA is creating a job in BOSH and I am passing ops file after uaa.yml file. It should have been already be able to create UAA job in BOSH deployment.
Any hints ?
Thanks so much!!
Answering my own question. I had to put in ?/- in the path so new element under ldap could be created. Once I put that in, I was able to deploy the environment with new OpsFile.
- type: replace
path: /instance_groups/name=bosh/jobs/name=uaa/properties/ldap?/-
value:
enabled: true
profile_type: "search-and-bind"
url: "ldap://ldap.localnet.com"
userDN: 'XXXX\\XXXXXXX'
userPassword: 'XXXXXXXXXX'
searchBase: "dc=local,dc=localnet,dc=com"
searchFilter: "cn={0}"
passwordAttributeName: "userPassword"
passwordEncoder: "org.cloudfoundry.identity.DynamicPasswordComparator"
sslCertificate: ""
ssl:
skipVerification: "true"
tls: "none"
mailAttributeName: "mail"
mailSubstitute: ""
mailSubstituteOverrridesLdap: "false"
referral: "follow"
groups:
profile_type: "groups-map-to-scopes"
searchBase: "dc=local,dc=localnet,dc=com"
groupRoleAttribute: "member"
groupSearchFilter: "member={0}"
searchSubtree: "true"
maxSearchDepth: "10"
emailDomain: ""
storeCustomAttributes: "true"
externalGroupWhitelist: ""
add_shadow_user_on_login: "true"
I have dict defined with a lot of sub elements:
kernel:
vm:
swappiness: 0
dirty_background_ratio: 123
dirty_ratio: 123
dirty_expire_centisecs: 123
sys:
fs:
file_max: 123456
net:
ipv4:
tcp_max_syn_backlog: 123456
tcp_slow_start_after_idle: 0
tcp_abort_on_overflow: 1
core:
somaxconn: 123456
netdev_max_backlog: 123456
rmem_max: 123456
wmem_max: 123456
rmem_default: 123456
rmem_max: 123456
And I have a task (not working) below as an example of what I'm trying to do to set the kernel parameters.
- name: "Tune kernel parameters"
sysctl:
name: "{{ item.key }}"
value: "{{ item.value[item.key] }}"
state: present
sysctl_set: yes
with_subelements:
- kernel
How can each item the dict be set without having to name each key name?
Iterating over dictionaries
d = {'x': 1, 'y': 2, 'z': 3}
for key, value in d.iteritems():
print key,value