a doubt on how to iterate a certain dictionary
---
- name: main playbook
hosts: localhost
tasks:
- name: Set fact
var:
tower_objects_organizations:
security:
org: ORG_I-SSO-ES-ADM-SSMM
teams:
- I-SSO-ES-ADM-SSMM
- I-SSO-ES-ADM-SSMM2
monit:
org: ORG_I-SSO-ES-TS-MONIT
teams:
- I-SSO-ES-TS-MONIT
- name: "[{{ productname }}][parallel set organization] Set organizations fact"
debug:
msg: "XXXXX"
with_items: "{{ tower_objects_organizations| dict2items | subelements('value.teams') }}"
Does anyone know how could I iterate through the teams in each subdirectionary (security, monit ...) and to fill a dictionary with the team as key and its org as value, i.e. I-SSO-ES-ADM-SSMM: ORG_I-SSO-ES-ADM-SSMM, I-SSO-ES-ADM-SSMM2: ORG_I-SSO-ES-ADM-SSMM, I-SSO-ES-TS-MONIT:ORG_I-SSO-ES-TS-MONIT
Using subelements I am capable of reaching the teams list but the output is not the expected one
A solution with and without custom filter:
when i have to do some complex actions on dict i prefer to use a custom filter:
you create a folder filter_plugins in your playbook folder (i have named the file myfilters.py and the filter customfilter)
#!/usr/bin/python
class FilterModule(object):
def filters(self):
return {
'customfilter': self.customfilter
}
def customfilter(self, obj):
result = {}
for k in obj:
org = obj[k]['org']
for l in obj[k]['teams']:
result[l] = org
return result
playbook sample:
tasks:
- name: Set fact
set_fact:
result: "{{ tower_objects_organizations | customfilter}}"
vars:
tower_objects_organizations:
security:
org: ORG_I-SSO-ES-ADM-SSMM
teams:
- I-SSO-ES-ADM-SSMM
- I-SSO-ES-ADM-SSMM2
monit:
org: ORG_I-SSO-ES-TS-MONIT
teams:
- I-SSO-ES-TS-MONIT
others:
org: ORG_I-SSO-ES-TS-OTHER
teams:
- I-SSO-ES-TS-OTHER0
- I-SSO-ES-TS-OTHER1
- name: display result
debug:
var: result
result:
ok: [localhost] =>
result:
I-SSO-ES-ADM-SSMM: ORG_I-SSO-ES-ADM-SSMM
I-SSO-ES-ADM-SSMM2: ORG_I-SSO-ES-ADM-SSMM
I-SSO-ES-TS-MONIT: ORG_I-SSO-ES-TS-MONIT
I-SSO-ES-TS-OTHER0: ORG_I-SSO-ES-TS-OTHER
I-SSO-ES-TS-OTHER1: ORG_I-SSO-ES-TS-OTHER
EDIT
another solution without custom filter but less readable i think:
- set_fact:
result: "{{ result | d({}) | combine(dict(item.value.teams|product([item.value.org]))) }}"
loop: "{{ tower_objects_organizations|dict2items }}"
- name: debug result
debug:
var: result
Related
I have a json file like so
{
"service_accounts": {
"sql-automation": "Automation Service account for Cloud SQL Instances",
"gke-proxy": "GKE proxy account",
"gke-node": "Default GKE nodes service account",
"syncdata": "Data sync between environments",
}
}
Then I have another json file with the default access the accounts need (this is a template file)
{
"key_gen_auths": {
"worker": [{
"role": "roles/iam.serviceAccountKeyAdmin",
"members": [
"serviceAccount:dataprovisioning#{{ project_id }}.iam.gserviceaccount.com"
]
}],
"gke-node": [{
"role": "roles/iam.serviceAccountUser",
"members": [
"serviceAccount:deployment-automation#{{ project_id }}.iam.gserviceaccount.com"
]
}],
"sql-automation": [{
"role": "roles/iam.serviceAccountUser",
"members": [
"serviceAccount:deployment-automation#{{ project_id }}.iam.gserviceaccount.com"
]
}]
}
}
Then in the playbook, i would like to say:
If the service account exists in the service_accounts file, extract that object from key_gen_auths if it exists there (if it doesnt exist in key_gen_auths ignore). Ive looked at several options, like selectattr, combine, difference, but im no further than when i started really, and wonder if im just better with a when statement if i can find the right syntax, any help would be appreciated.
- name: Play for reading json
hosts: localhost
vars:
project_id: 123
tasks:
- name: acct_list
set_fact:
acct_list: "{{ lookup('template', 'acct_list.json') }}"
- name: key_auth_default
set_fact:
keyauth_default: "{{ lookup('template', 'key_auths.json') }}"
- name:
set_fact:
access_file: "{{ item }}"
with_items: "{{ keyauth_default.key_gen_auths }}"
when: item in acct_list.service_accounts
- debug:
var: access_file
You can read the files without lookup
vars_files:
- acct_list.json
- key_auths.json
vars:
project_id: 123
give
service_accounts:
gke-node: Default GKE nodes service account
gke-proxy: GKE proxy account
sql-automation: Automation Service account for Cloud SQL Instances
syncdata: Data sync between environments
key_gen_auths:
gke-node:
- members:
- serviceAccount:deployment-automation#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountUser
sql-automation:
- members:
- serviceAccount:deployment-automation#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountUser
worker:
- members:
- serviceAccount:dataprovisioning#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountKeyAdmin
Q: "If the service account exists in service_accounts extract that object from key_gen_auths."
A: Get the lists of the keys from the dictionaries. Then the intersect of these lists gives the existing service accounts
service_accounts_list: "{{ service_accounts.keys()|list }}"
key_gen_auths_list: "{{ key_gen_auths.keys()|list }}"
service_account_exists: "{{ service_accounts_list|intersect(key_gen_auths_list) }}"
give
service_accounts_list: [sql-automation, gke-proxy, gke-node, syncdata]
key_gen_auths_list: [worker, gke-node, sql-automation]
service_account_exists: [sql-automation, gke-node]
Use the list of existing accounts to extract the objects and create a dictionary
access_file: "{{ dict(service_account_exists|
zip(service_account_exists|map('extract', key_gen_auths))) }}"
give
access_file:
gke-node:
- members:
- serviceAccount:deployment-automation#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountUser
sql-automation:
- members:
- serviceAccount:deployment-automation#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountUser
Example of the complete playbook for testing
- hosts: localhost
vars_files:
- acct_list.json
- key_auths.json
vars:
project_id: 123
service_accounts_list: "{{ service_accounts.keys()|list }}"
key_gen_auths_list: "{{ key_gen_auths.keys()|list }}"
service_account_exists: "{{ service_accounts_list|intersect(key_gen_auths_list) }}"
access_file: "{{ dict(service_account_exists|
zip(service_account_exists|map('extract', key_gen_auths))) }}"
tasks:
- debug:
var: service_accounts
- debug:
var: key_gen_auths
# If the service account exists in service_accounts
# extract those objects from key_gen_auths
- debug:
var: service_accounts_list|to_yaml
- debug:
var: key_gen_auths_list|to_yaml
- debug:
var: service_account_exists|to_yaml
- debug:
var: access_file
Well, I think this shows the data you want, in debug, and you can use that with set_fact as you see fit:
- name: Play for reading json
hosts: localhost
vars_files:
- acct_list.json
- key_auths.json
vars:
project_id: 123
tasks:
- debug:
msg: "{{ key_gen_auths[item] | default('Not Found') }}"
loop: "{{ service_accounts.keys() }}"
ignore_errors: yes
I've got a dict of lists like this:
packages:
server:
- foo
- bar
- baz
client:
- spam
- ham
- eggs
runtime:
- corge
- grault
- garply
and I want to generate a list by defining keys to select in another dict, so e.g.:
enable:
server: true
client: false
runtime: true
would result in:
output:
- foo
- bar
- baz
- corge
- grault
- garply
Any ideas how to do this in ansible/jinja?? Ideally without using the loop construct but I could live with that. I suspect it'll need an intermediate variable and so far all I've got is something to extract a list of the keys from enable which have a true value:
- set_fact:
enabled: "{{ (enable | dict2items | selectattr('value') | list | items2dict).keys() }}"
If this looks a bit convoluted its because both packages and enable are existing role variables which I'd prefer not to change. But open to suggestions (especially on packages) if reshaping them makes this much easier.
Let's create the list of enabled hosts first. For example
- set_fact:
my_hosts: "{{ enable|dict2items|
selectattr('value')|
map(attribute='key')|list }}"
- debug:
var: my_hosts
give
my_hosts:
- server
- runtime
Then use this list to extract the packages. For example
- set_fact:
my_list: "{{ my_hosts|map('extract', packages)|list|flatten }}"
- debug:
var: my_list
give
my_list:
- foo
- bar
- baz
- corge
- grault
- garply
The single task below comprises both steps
- set_fact:
my_list: "{{ my_hosts|map('extract', packages)|list|flatten }}"
vars:
my_hosts: "{{ enable|dict2items|
selectattr('value')|
map(attribute='key')|list }}"
I have a var file named prop.yml and contains:
var1:
- 'a'
- 'b'
var2:
- 'blah'
- 'blab'
Now, my playbook looks like:
task:
- name: including a variety file
include_vars:
file: prop.yml
name: property
- set_fact:
project: "{{ lookup ('vars', 'property') }}"
- debug:
msg: "{{ project }}"
Now, my output is
var1[
"a"
"b"]
var2[ "blah" , "blab"]
What I want as output is
["a", "b", "blah", "blab"]
Simply add the lists.
- debug:
msg: "{{ project.var1 + project.var2 }}"
The following should get you going:
task:
- name: including a variety file
include_vars:
file: prop.yml
name: property
- set_fact:
project: "{{ property.var1 + property.var2 }}"
- debug:
msg: "{{ project }}"
Rather than using the vars statically we can concatenate all lists in the included vars file by replacing the set_fact task with the following one.
This will only work if all declared vars in the file are all lists. Note the use of the default filter to make sure our result var is always defined. This also uses a loop over a dict
- name: Iteratively concatenate our lists
set_fact:
project: "{{ project | default([]) + item.value }}"
loop: "{{ property | dict2items }}"
I'm trying to get a list of grants based on database role to re-use in one of the modules. For that the result needs to be a list
name: adding permissions
module:
role: database_role
permissions:
- "schema:USAGE/table1:SELECT/table2:SELECT,UPDATE"
- "another_schema:USAGE/ALL:ALL"
My permissions are defined as variables as follows:
db_roles:
- name: role1
grants:
- schema: schema
permissions:
- table1:SELECT
- table2:SELECT,UPDATE
- schema: another_schema:
permissions:
- ALL:ALL
I have more roles defined as well. This definition of roles means I can add a new permission per row, making it more readable.
Now I'm trying to format this variable to receive something like this:
permissions:
- role1:
- "schema:USAGE/table1:SELECT/table2:SELECT,UPDATE"
- "another_schema:USAGE/ALL:ALL"
- role2:
- "schema:USAGE/ALL:ALL"
But I have no idea how to get to this result.
What I've tried
So far the furthest I got is this but I'm not sure if it's possible to retrieve the values from the dictionary somehow
ok: [localhost] => {
"permissions": {
"role1": {
"schema": "schema:USAGE/table1:SELECT/table2:SELECT",
"another_schema": "another_schema:USAGE/ALL:ALL"
}
}
}
The code that got me there is:
- name: Create privs for users
set_fact:
permissions: "{{ permissions|default( {item.0.name:{}} ) | combine( {item.0.name:{item.1.schema: item.1.schema ~ ':USAGE/' ~ item.1.permissions | join('/')}}, recursive=True) }}"
with_subelements:
- "{{ db_roles }}"
- grants
tough one :) hope i got it right.
picking up where you left off, i added this task, to further process the variable you prepared:
- name: Create privs for users - step 2
set_fact:
permissions_final: "{{ permissions_final|default([]) + [{ item : permissions[item] | dict2items | map(attribute='value') | list }] }}"
with_items:
- "{{ permissions.keys() | list }}"
full code and sample output:
---
- hosts: localhost
gather_facts: false
vars:
db_roles:
- name: role1
grants:
- schema: schema
permissions:
- table1:SELECT
- table2:SELECT,UPDATE
- schema: another_schema
permissions:
- ALL:ALL
- name: role2
grants:
- schema: schema3
permissions:
- table1:SELECT
- table2:SELECT,UPDATE
- schema: another_schema4
permissions:
- ALL:ALL
tasks:
- name: Create privs for users
set_fact:
permissions: "{{ permissions|default( {item.0.name:{}} ) | combine( {item.0.name:{item.1.schema: item.1.schema ~ ':USAGE/' ~ item.1.permissions | join('/')}}, recursive=True) }}"
with_subelements:
- "{{ db_roles }}"
- grants
# - name: print results
# debug:
# var: permissions
- name: Create privs for users - step 2
set_fact:
permissions_final: "{{ permissions_final|default([]) + [{ item : permissions[item] | dict2items | map(attribute='value') | list }] }}"
with_items:
- "{{ permissions.keys() | list }}"
- name: print results
debug:
var: permissions_final
variable produced:
TASK [print results] ***************************************************************************************************************************************************************************************************
ok: [localhost] => {
"permissions_final": [
{
"role1": [
"schema:USAGE/table1:SELECT/table2:SELECT,UPDATE",
"another_schema:USAGE/ALL:ALL"
]
},
{
"role2": [
"schema3:USAGE/table1:SELECT/table2:SELECT,UPDATE",
"another_schema4:USAGE/ALL:ALL"
]
}
]
}
hope this helps!
Splitting dictionary not working in ansible.
Ansible- 2.5.15
Could anyone please help with any solution.
I was trying to fetch the values from dictionary but unable to fetch the values.
Tried code:
- hosts: localhost
connection: local
tasks:
- set_fact:
some_module: "{{ item.split(': ')[1] }}"
with_items:
- git: true
- gradle: false
Getting below error:
The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'split'
Expected Results are like below:
[true, false]
You can handle it as a hashmap and get the key or the value:
- hosts: localhost
connection: local
tasks:
- set_fact:
some_module: "{{ item.values }}"
with_items:
- {git: true}
- {gradle: false}
(Updated for Ansible 2.9 and later)
Given the list
l:
- git: true
- gradle: false
the task
- set_fact:
out: "{{ l|map('dict2items')|
flatten|
map(attribute='value')|
list }}"
gives the required result
out:
- true
- false
If the data were a dictionary, e.g.
d:
git: true
gradle: false
the solution would have been much simpler, e.g. the task below gives the same result
- set_fact:
out: "{{ d.values()|list }}"
Notes
Your data is not a dictionary. It is a list
- git: true
- gradle: false
Dictionary is below
git: true
gradle: false
Let's create from the data a dictionary first and then use the dict2items filter.
The play below
- hosts: localhost
vars:
data1:
- {git: true}
- {gradle: false}
data2: {}
tasks:
- set_fact:
data2: "{{ data2|combine(item) }}"
loop: "{{ data1 }}"
- debug:
msg: "{{ data2|dict2items|json_query('[].value') }}"
gives:
"msg": [
true,
false
]
dict2items is available since Ansible 2.6. In older versions use simple filter_plugin hash_utils.py
$ cat filter_plugins/hash_utils.py
def hash_to_tuples(h):
return h.items()
def hash_keys(h):
return h.keys()
def hash_values(h):
return h.values()
class FilterModule(object):
''' utility filters for operating on hashes '''
def filters(self):
return {
'hash_to_tuples' : hash_to_tuples
,'hash_keys' : hash_keys
,'hash_values' : hash_values
}
The task below
- debug:
msg: "{{ data2|hash_values }}"
gives the same result as the construct with dict2items above. You might want to try other filters and review the details about filter_plugin.