I have a situation where i am trying to remove the item from a list. but i am not getting expected result. Please help me what I am doing wrong here?
here is the list :
"get_ec2_id.instances[0].tags": {
"Name": "test-db-system-2",
"aws:cloudformation:logical-id": "DBInstance",
"aws:cloudformation:stack-id": "arn:aws:cloudformation:us-east-1:123456789012:stack/test-db-system-2/0115v0a0-5d44-17e8-a024-503ama4a5qd1",
"aws:cloudformation:stack-name": "test-db-system-2",
"dbsystem:stack": "test-db-system-2",
"dbsystem:type": "db"
}
}
I am trying to remove the all "aws:cloudformation" tags from a list using below filter:
"{{ get_ec2_id.instances[0].tags | reject('search','aws:') | list }}"
I am getting the below result:
ok: [10.52.8.101] => {
"instances_tags": [
"dbsystem:type",
"dbsystem:stack",
"Name"
]
}
but I am expected below result :
"instances_tags": [
"dbsystem:stack": "test-db-system-2",
"dbsystem:type": "db"
"Name" : "test-db-system-2",
]
}
Help me to solve the issue.
More generic solution where input is a dict and blacklist is a list:
---
- set_fact:
blacklist:
- bad1
- bad2
- set_fact:
output: {}
- name: remove blacklisted items from input
set_fact:
output: "{{ output | combine({item.key: item.value}) }}"
when: item.key not in blacklist
loop: "{{ input | dict2items }}"
Given the data
get_ec2_id:
instances:
- tags:
Name: test-db-system-2
aws:cloudformation:logical-id: DBInstance
aws:cloudformation:stack-id: arn:aws:cloudformation:us-east-1:123456789012:stack/test-db-system-2/0115v0a0-5d44-17e8-a024-503ama4a5qd1
aws:cloudformation:stack-name: test-db-system-2
dbsystem:stack: test-db-system-2
dbsystem:type: db
Use rejectattr. For example
dict2: "{{ get_ec2_id.instances.0.tags|
dict2items|
rejectattr('key', 'search', 'aws:')|
items2dict }}"
gives
dict2:
Name: test-db-system-2
dbsystem:stack: test-db-system-2
dbsystem:type: db
Then, convert the dictionary into a list of dictionaries
instances_tags: "{{ dict2|
dict2items|
json_query('[].[[key, value]]')|
map('community.general.dict')|
list }}"
gives
instances_tags:
- Name: test-db-system-2
- dbsystem:stack: test-db-system-2
- dbsystem:type: db
Use this:
---
- name: dictionary
hosts: localhost
gather_facts: False
connection: local
vars:
get_ec2_id:
instances:
tags:
Name: "test-db-system-2"
"aws:cloudformation:logical-id": "DBInstance"
"aws:cloudformation:stack-id": "arn:aws:cloudformation:us-east-1:123456789012:stack/test-db-system-2/0115v0a0-5d44-17e8-a024-503ama4a5qd1"
"aws:cloudformation:stack-name": "test-db-system-2"
"dbsystem:stack": "test-db-system-2"
"dbsystem:type": "db"
dict2: {}
tasks:
- name: Fact1
set_fact:
dict: "{{ get_ec2_id.instances.tags }}"
- name: Debug1
debug:
var: dict
- name: Fact2
set_fact:
dict2: "{{ dict2 | combine({item.key: item.value}) }}"
when: "{{ item.key.find('aws:') }}"
with_dict: "{{ dict }}"
- name: Debug2
debug:
var: dict2
Output:
TASK [Debug2] ******************************************************************
ok: [localhost] => {
"dict2": {
"Name": "test-db-system-2",
"dbsystem:stack": "test-db-system-2",
"dbsystem:type": "db"
}
}
Related
I would like to create a dictionary and populated from variables both the keys and values. The values will be a list of IPs.
If the key is already present in the dictionary, then, I need to add the IP to the existing list.
Here is the input data, which is a list of list.
parse1.json:
[
[
"/RAD33G4099/mobile_app/pl_crad.la-dc.vivo.com_80",
"properties",
"members",
"/Common/10.50.100.25:80",
"session"
],
[
"/RAD33G4099/mobile_app/pl_crad.la-dc.vivo.com_80",
"properties",
"members",
"/Common/10.50.100.39:80",
"session"
],
[
"/RAD33G4099/mobile_app/pl_crad.la-dc.vivo.com_80",
"properties",
"members",
"/Common/10.50.100.49:80",
"session"
],
[
"/RAD33G4099/mobile_app/pl_crad.la-dc.vivo.com_443",
"properties",
"members",
"/Common/10.50.100.18:80",
"session"
],
[
"/RAD33G4099/mobile_app/pl_crad.la-dc.vivo.com_443",
"properties",
"members",
"/Common/10.50.100.28:80",
"session"
],
[
"/RAD33G4099/mobile_app/pl_crad.la-dc.vivo.com_443",
"properties",
"members",
"/Common/10.50.100.48:80",
"session"
]
]
Here is my trial at solving it:
- set_fact:
my_dic: {}
disabled_host: "{{ lookup('file', 'parse1.json') }}"
- name: add new key / value pairs to dict
set_fact:
my_dict_var: "{{ my_dict_var + [ my_dic | combine ( { name + '_' + app_name : host } , recursive=True) ] }}"
loop: "{{ disabled_host }}"
vars:
name: "{{ item[0] | regex_replace('^\/.*\/(.*)$', '\\1') }}"
app_name: "{{ item[0] | regex_replace('^\/.*\/(.*)\/.*$', '\\1') }}"
host: "{{ item[3] | regex_replace('^\/.*\/(.*):.*$', '\\1') }}"
The actual output is
ok: [localhost] => {
"msg": [
{
"pl_crad.la-dc.vivo.com_80_mobile_app": "10.50.100.25"
},
{
"pl_crad.la-dc.vivo.com_80_mobile_app": "10.50.100.39"
},
{
"pl_crad.la-dc.vivo.com_80_mobile_app": "10.50.100.49"
},
{
"pl_crad.la-dc.vivo.com_443_mobile_app": "10.50.100.18"
},
{
"pl_crad.la-dc.vivo.com_443_mobile_app": "10.50.100.28"
},
{
"pl_crad.la-dc.vivo.com_443_mobile_app": "10.50.100.48"
}
]
}
But, I would like to create a list of IPs if the key is same.
My desired output is
{
"pl_crad.la-dc.vivo.com_80_mobile_app": [
"10.50.100.25",
"10.50.100.39",
"10.50.100.49"
],
"pl_crad.la-dc.vivo.com_443_mobile_app": [
"10.50.100.18",
"10.50.100.28",
"10.50.100.48"
]
}
You don't have to go the hard way and use regex, you can just split those strings, and / or, use basename and dirname.
Using dirname and basename:
- set_fact:
disabled_hosts_ips: >-
{{
disabled_hosts_ips | default({})
| combine({
name: (disabled_hosts_ips | default({}))[name] | default([])
+ [ip]
})
}}
loop: "{{ disabled_hosts }}"
vars:
name: "{{ item[0] | basename ~ '_' ~ item[0] | dirname | basename }}"
ip: "{{ (item[3] | basename).split(':')[0] }}"
Using split:
- set_fact:
disabled_hosts_ips: >-
{{
disabled_hosts_ips | default({})
| combine({
name: (disabled_hosts_ips | default({}))[name] | default([])
+ [ip]
})
}}
loop: "{{ disabled_hosts }}"
vars:
name: "{{ item[0].split('/')[-2:] | reverse | join('_') }}"
ip: "{{ item[3].split('/')[-1].split(':')[0] }}"
Those both give:
disabled_hosts_ips:
pl_crad.la-dc.vivo.com_443_mobile_app:
- 10.50.100.18
- 10.50.100.28
- 10.50.100.48
pl_crad.la-dc.vivo.com_80_mobile_app:
- 10.50.100.25
- 10.50.100.39
- 10.50.100.49
For example, using the variable my_dict_var
my_dict_groups: "{{ dict(my_dict_keys|zip(my_dict_vals)) }}"
my_dict_keys: "{{ dh_groups|map('first')|list }}"
my_dict_vals: "{{ dh_groups|map('last')|map('map', attribute='value')|list }}"
dh_groups: "{{ my_dict_var|map('dict2items')|flatten|groupby('key') }}"
gives
my_dict_groups:
pl_crad.la-dc.vivo.com_443_mobile_app:
- 10.50.100.18
- 10.50.100.28
- 10.50.100.48
pl_crad.la-dc.vivo.com_80_mobile_app:
- 10.50.100.25
- 10.50.100.39
- 10.50.100.49
The complete set of the declarations is below
my_dict_groups: "{{ dict(my_dict_keys|zip(my_dict_vals)) }}"
my_dict_keys: "{{ dh_groups|map('first')|list }}"
my_dict_vals: "{{ dh_groups|map('last')|map('map', attribute='value')|list }}"
dh_groups: "{{ dh_keys|zip(dh_vals)|map('combine')|groupby('key') }}"
dh_vals: "{{ disabled_host|
map(attribute=3)|
map('split', '/')|map('last')|
map('split', ':')|map('first')|
map('community.general.dict_kv', 'value')|
list }}"
dh_keys: "{{ disabled_host|
map(attribute=0)|
map('regex_replace', dh_regex, dh_replace)|
map('community.general.dict_kv', 'key')|
list }}"
dh_regex: '^/.*?/(.*?)/(.*)$'
dh_replace: '\2_\1'
disabled_host: "{{ lookup('file', 'parse1.json') }}"
Details
In both cases use the filter groupby and create the list
dh_groups:
- - pl_crad.la-dc.vivo.com_443_mobile_app
- - key: pl_crad.la-dc.vivo.com_443_mobile_app
value: 10.50.100.18
- key: pl_crad.la-dc.vivo.com_443_mobile_app
value: 10.50.100.28
- key: pl_crad.la-dc.vivo.com_443_mobile_app
value: 10.50.100.48
- - pl_crad.la-dc.vivo.com_80_mobile_app
- - key: pl_crad.la-dc.vivo.com_80_mobile_app
value: 10.50.100.25
- key: pl_crad.la-dc.vivo.com_80_mobile_app
value: 10.50.100.39
- key: pl_crad.la-dc.vivo.com_80_mobile_app
value: 10.50.100.49
Create the lists of the keys my_dict_keys and values my_dict_vals, and create the dictionary my_dict_groups
my_dict_keys:
- pl_crad.la-dc.vivo.com_443_mobile_app
- pl_crad.la-dc.vivo.com_80_mobile_app
my_dict_vals:
- - 10.50.100.18
- 10.50.100.28
- 10.50.100.48
- - 10.50.100.25
- 10.50.100.39
- 10.50.100.49
For example, put the declarations into the playbook vars
- hosts: localhost
vars:
my_dict_groups: "{{ dict(my_dict_keys|zip(my_dict_vals)) }}"
my_dict_keys: "{{ dh_groups|map('first')|list }}"
my_dict_vals: "{{ dh_groups|map('last')|map('map', attribute='value')|list }}"
# dh_groups: "{{ my_dict_var|map('dict2items')|flatten|groupby('key') }}"
dh_groups: "{{ dh_keys|zip(dh_vals)|map('combine')|groupby('key') }}"
dh_vals: "{{ disabled_host|
map(attribute=3)|
map('split', '/')|
map('last')|map('split', ':')|map('first')|
map('community.general.dict_kv', 'value')|
list }}"
dh_keys: "{{ disabled_host|
map(attribute=0)|
map('regex_replace', dh_regex, dh_replace)|
map('community.general.dict_kv', 'key')|
list }}"
dh_regex: '^/.*?/(.*?)/(.*)$'
dh_replace: '\2_\1'
disabled_host: "{{ lookup('file', 'parse1.json') }}"
# my_dict_var:
# - pl_crad.la-dc.vivo.com_80_mobile_app: 10.50.100.25
# - pl_crad.la-dc.vivo.com_80_mobile_app: 10.50.100.39
# - pl_crad.la-dc.vivo.com_80_mobile_app: 10.50.100.49
# - pl_crad.la-dc.vivo.com_443_mobile_app: 10.50.100.18
# - pl_crad.la-dc.vivo.com_443_mobile_app: 10.50.100.28
# - pl_crad.la-dc.vivo.com_443_mobile_app: 10.50.100.48
tasks:
- debug:
var: my_dict_groups
I have a list of lists of hosts:
[['host-0', 'host-1'], ['host-2', 'host-3'], ['host-4', 'host-5', 'host-6']]
How can I add a port number, e.g., 8000, to each host using ansible/ jinja2 to get:
[['host-0:8000', 'host-1:8000'], ['host-2:8000', 'host-3:8000'], ['host-4:8000', 'host-5:8000', 'host-6:8000']]
this task shall do it:
- name: convert items in list
set_fact:
my_new_list: "{{ my_new_list | default([])+ [ my_var ] }}"
vars:
my_var: "{{ item | map('regex_replace', '$', ':8000') | list }}"
with_items:
- "{{ my_list }}"
full playbook to run as demo:
---
- hosts: localhost
gather_facts: false
vars:
my_list:
- ['host-0', 'host-1']
- ['host-2', 'host-3']
- ['host-4', 'host-5', 'host-6']
tasks:
- name: print original variable
debug:
var: my_list
- name: convert items in list
set_fact:
my_new_list: "{{ my_new_list | default([])+ [ my_var ] }}"
vars:
my_var: "{{ item | map('regex_replace', '$', ':8000') | list }}"
with_items:
- "{{ my_list }}"
- name: print new variable
debug:
var: my_new_list
result:
TASK [print new variable] **********************************************************************************************************************************************************************************************
ok: [localhost] => {
"my_new_list": [
[
"host-0:8000",
"host-1:8000"
],
[
"host-2:8000",
"host-3:8000"
],
[
"host-4:8000",
"host-5:8000",
"host-6:8000"
]
]
}
PLAY RECAP
Use map + regex_replace.
- debug:
msg: "{{ foo | map('map', 'regex_replace', '$', ':8000') }}"
vars:
foo: [['host-0', 'host-1'], ['host-2', 'host-3'], ['host-4', 'host-5', 'host-6']]
"msg": [
[
"host-0:8000",
"host-1:8000"
],
[
"host-2:8000",
"host-3:8000"
],
[
"host-4:8000",
"host-5:8000",
"host-6:8000"
]
]
I have got multiple yaml files on remote machine. I would like to parse those files in order to get information about names for each kind (Deployment, Configmap, Secret) of object, For example:
...
kind: Deployment
metadata:
name: pr1-dep
...
kind: Secret
metadata:
name: pr1
...
....
kind: ConfigMap
metadata:
name: cm-pr1
....
Ecpected result:
3 variables:
deployments = [pr1-dep]
secrets = [pr1]
configmaps = [cm-pr1]
I started with:
- shell: cat "{{ item.desc }}"
with_items:
- "{{ templating_register.results }}"
register: objs
but i have no idea how to correctly parse item.stdout from objs
Ansible has a from_yaml filter that takes YAML text as input and outputs an Ansible data structure. So for example you can write something like this:
- hosts: localhost
gather_facts: false
tasks:
- name: Read objects
command: "cat {{ item }}"
register: objs
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
- debug:
msg:
- "kind: {{ obj.kind }}"
- "name: {{ obj.metadata.name }}"
vars:
obj: "{{ item.stdout | from_yaml }}"
loop: "{{ objs.results }}"
loop_control:
label: "{{ item.item }}"
Given your example files, this playbook would output:
PLAY [localhost] ***************************************************************
TASK [Read objects] ************************************************************
changed: [localhost] => (item=deployment.yaml)
changed: [localhost] => (item=configmap.yaml)
changed: [localhost] => (item=secret.yaml)
TASK [debug] *******************************************************************
ok: [localhost] => (item=deployment.yaml) => {
"msg": [
"kind: Deployment",
"name: pr1-dep"
]
}
ok: [localhost] => (item=configmap.yaml) => {
"msg": [
"kind: ConfigMap",
"name: pr1-cm"
]
}
ok: [localhost] => (item=secret.yaml) => {
"msg": [
"kind: Secret",
"name: pr1"
]
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Creating the variables you've asked for is a little trickier. Here's
one option:
- hosts: localhost
gather_facts: false
tasks:
- name: Read objects
command: "cat {{ item }}"
register: objs
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
- name: Create variables
set_fact:
names: >-
{{
names|combine({
obj.kind.lower(): [obj.metadata.name]
}, list_merge='append')
}}
vars:
names: {}
obj: "{{ item.stdout | from_yaml }}"
loop: "{{ objs.results }}"
loop_control:
label: "{{ item.item }}"
- debug:
var: names
This creates a single variable named names that at the end of the
playbook will contain:
{
"configmap": [
"pr1-cm"
],
"deployment": [
"pr1-dep"
],
"secret": [
"pr1"
]
}
The key to the above playbook is our use of the combine filter, which can be used to merge dictionaries and, when we add list_merge='append', handles keys that resolve to lists by appending to the existing list, rather than replacing the existing key.
Include the dictionaries from the files into the new variables, e.g.
- include_vars:
file: "{{ item }}"
name: "objs_{{ item|splitext|first }}"
register: result
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
This will create dictionaries objs_deployment, objs_configmap, and objs_secret. Next, you can either use the dictionaries
- set_fact:
objs: "{{ objs|d({})|combine({_key: _val}) }}"
loop: "{{ query('varnames', 'objs_') }}"
vars:
_obj: "{{ lookup('vars', item) }}"
_key: "{{ _obj.kind }}"
_val: "{{ _obj.metadata.name }}"
, or the registered data
- set_fact:
objs: "{{ dict(_keys|zip(_vals)) }}"
vars:
_query1: '[].ansible_facts.*.kind'
_keys: "{{ result.results|json_query(_query1)|flatten }}"
_query2: '[].ansible_facts.*.metadata[].name'
_vals: "{{ result.results|json_query(_query2)|flatten }}"
Both options give
objs:
ConfigMap: cm-pr1
Deployment: pr1-dep
Secret: pr1
I have a text file that needs to be parsed so i can use it via a REST API with ansible.
10.0.0.0/16 Building-A
10.1.0.0/16 Building-A
10.2.0.0/16 Building-B
10.3.0.0/16 Building-B
I need to convert this text to something like this:
{
"parsed":[
{
"Building-A":[
"10.0.0.0/16",
"10.1.0.0/16"
]
},
{
"Building-B":[
"10.2.0.0/16",
"10.3.0.0/16"
]
}
]
}
Currently i run the following playbook to test the textparsing, without success. The list created is not unique.
- name: Test
hosts: localhost
tasks:
- name: Combine
ansible.builtin.set_fact:
parsed: "{{ (parsed | default([])) | union( [{item.split()[1]: item.split()[0] }] ) }}"
loop: "{{ lookup('file','hostgroups.txt').strip().splitlines() }}"
- name: Debug
ansible.builtin.debug:
var: parsed
ok: [localhost] => {
"parsed": [
{
"Building-A": "10.0.0.0/16"
},
{
"Building-A": "10.1.0.0/16"
},
{
"Building-B": "10.2.0.0/16"
},
{
"Building-B": "10.3.0.0/16"
}
]
}
Thanks for the tip with the helper variables and the groupby filter. Here is the final playbook:
- name: Test
hosts: localhost
# strategy: free
tasks:
- name: Get List
ansible.builtin.set_fact:
parsed_list: "{{ parsed_list | default([]) + [_item] }}"
loop: "{{ lookup('file','hostgroups.txt').strip().splitlines() }}"
vars:
_list: "{{ item.split() }}"
_item: "{{ {'Name': _list[1], 'Subnet': _list[0] } }}"
- name: Debug parsed_list
ansible.builtin.debug:
var: parsed_list
- name: Group parsed_list
ansible.builtin.set_fact.set_fact:
parsed_group: "{{ parsed_group | default([]) + [{_key: _value}] }}"
loop: "{{ parsed_list | groupby('Name') }}"
vars:
_key: "{{ item.0 }}"
_value: "{{ item.1 | map(attribute='Subnet') | list }}"
- name: Debug parsed_group
ansible.builtin.debug:
var: parsed_group
Parse the content, e.g.
- set_fact:
parsed_list: "{{ parsed_list|d([]) + [_item] }}"
loop: "{{ lookup('file','hostgroups.txt').splitlines() }}"
vars:
_array: "{{ item.split() }}"
_item: "{{ {'building': _array.1, 'ip': _array.0} }}"
gives
parsed_list:
- building: Building-A
ip: 10.0.0.0/16
- building: Building-A
ip: 10.1.0.0/16
- building: Building-B
ip: 10.2.0.0/16
- building: Building-B
ip: 10.3.0.0/16
Then, use filter groupby and create the list
- set_fact:
parsed: "{{ parsed|d([]) + [{_key: _val}] }}"
loop: "{{ parsed_list|groupby('building') }}"
vars:
_key: "{{ item.0 }}"
_val: "{{ item.1|map(attribute='ip')|list }}"
gives
parsed:
- Building-A:
- 10.0.0.0/16
- 10.1.0.0/16
- Building-B:
- 10.2.0.0/16
- 10.3.0.0/16
In some cases, a dictionary might be a better structure, e.g.
- set_fact:
parsed_dict: "{{ parsed_dict|d({})|combine({_key: _val}) }}"
loop: "{{ parsed_list|groupby('building') }}"
vars:
_key: "{{ item.0 }}"
_val: "{{ item.1|map(attribute='ip')|list }}"
gives
parsed_dict:
Building-A:
- 10.0.0.0/16
- 10.1.0.0/16
Building-B:
- 10.2.0.0/16
- 10.3.0.0/16
How do I convert a dictionary like below
{
"host1": ["tag1", "tag2"],
"host2": ["tag1"]
}
to below in ansible:
{
"tag1": ["host1","host2"],
"tag2": ["host1"]
}
I have been trying to do this but got stuck due to the value being a list.
The playbook
- hosts: localhost
vars:
dict1:
host1: [tag1, tag2]
host2: [tag1]
tasks:
- set_fact:
dict2: "{{ dict2|default({})|combine({item: list_of_hosts}) }}"
loop: "{{ dict1.values()|flatten|unique }}"
vars:
list_of_hosts: "{{ dict1|
dict2items|
selectattr('value', 'contains', item)|
map(attribute='key')|
list }}"
- debug:
var: dict2
gives
dict2:
tag1: [host1, host2]
tag2: [host1]
Optionally, use json_query
vars:
list_of_hosts: "{{ dict1|dict2items|json_query(_query) }}"
_query: '[?value.contains(#, `{{ item }}`)].key'