Imagine I have the following inventory
[myservers]
127.0.0.1
192.168.0.6
And in my vars file, the following:
interesting_things:
- name: alice
port: 8080
- name: bob
port: 8181
How can I produce the following result?
127.0.0.1[8080],127.0.0.1[8181],192.168.0.6[8080],192.168.0.6[8181]
I tried getting the cartesian product, but came unstuck:
debug: msg="{{ lookup('cartesian', groups['myservers'], interesting_things | map(attribute='port')) }}"
This gave me the following, but I can't work out how to go any further.
{
"msg": [
[
"127.0.0.1",
8080
],
[
"127.0.0.1",
8181
],
[
"192.168.0.6",
8080
],
[
"192.168.0.6",
8181
]
]
}
Building on what you already have:
- name:
set_fact:
my_list: "{{ my_list|default([]) + [my_element] }}"
vars:
my_element: "{{ item[0] }}[{{ item[1] }}]"
with_items:
- "{{ lookup('cartesian', groups['myservers'], interesting_things | map(attribute='port')) }}"
- debug:
msg: "{{ my_list|join(',') }}"
Result:
"msg": "127.0.0.1[8080],127.0.0.1[8181],192.168.0.6[8080],192.168.0.6[8181]"
Related
I have a list of lists of hosts:
[['host-0', 'host-1'], ['host-2', 'host-3'], ['host-4', 'host-5', 'host-6']]
How can I add a port number, e.g., 8000, to each host using ansible/ jinja2 to get:
[['host-0:8000', 'host-1:8000'], ['host-2:8000', 'host-3:8000'], ['host-4:8000', 'host-5:8000', 'host-6:8000']]
this task shall do it:
- name: convert items in list
set_fact:
my_new_list: "{{ my_new_list | default([])+ [ my_var ] }}"
vars:
my_var: "{{ item | map('regex_replace', '$', ':8000') | list }}"
with_items:
- "{{ my_list }}"
full playbook to run as demo:
---
- hosts: localhost
gather_facts: false
vars:
my_list:
- ['host-0', 'host-1']
- ['host-2', 'host-3']
- ['host-4', 'host-5', 'host-6']
tasks:
- name: print original variable
debug:
var: my_list
- name: convert items in list
set_fact:
my_new_list: "{{ my_new_list | default([])+ [ my_var ] }}"
vars:
my_var: "{{ item | map('regex_replace', '$', ':8000') | list }}"
with_items:
- "{{ my_list }}"
- name: print new variable
debug:
var: my_new_list
result:
TASK [print new variable] **********************************************************************************************************************************************************************************************
ok: [localhost] => {
"my_new_list": [
[
"host-0:8000",
"host-1:8000"
],
[
"host-2:8000",
"host-3:8000"
],
[
"host-4:8000",
"host-5:8000",
"host-6:8000"
]
]
}
PLAY RECAP
Use map + regex_replace.
- debug:
msg: "{{ foo | map('map', 'regex_replace', '$', ':8000') }}"
vars:
foo: [['host-0', 'host-1'], ['host-2', 'host-3'], ['host-4', 'host-5', 'host-6']]
"msg": [
[
"host-0:8000",
"host-1:8000"
],
[
"host-2:8000",
"host-3:8000"
],
[
"host-4:8000",
"host-5:8000",
"host-6:8000"
]
]
I have the following two variables:
target_groups:
- name: http
port: 80
- name: https
port: 443
targets:
- 1.2.3.4
- 2.3.4.5
For an elb_target_group task I need a list of hashes as targets parameter. So what I'd like to have is a structure like the following:
target_groups:
- name: http
port: 80
targets:
- Id: 1.2.3.4
Port: 80
- Id: 2.3.4.5
Port: 80
- name: https
port: 443
targets:
- Id: 1.2.3.4
Port: 443
- Id: 2.3.4.5
Port: 443
So the targets entry of each target_groups element must be composed of the port of the element and all IPs of the targets list.
I have twisted my head around all map, combine... whatever filter I could find but couldn't come up with a solution.
Actually I don't even need that targets element in the list, as long as I can generate a suitable list of hashes on the fly, I'd be happy to do that. My task would look like that:
- name: update target groups
elb_target_group:
name: "{{ item.name }}"
protocol: tcp
port: "{{ item.port }}"
state: present
vpc_id: "{{ vpc_id }}"
targets: <<NEEDHELPHERE>>
with_items:
- { name: http, port: 80 }
- { name: https, port: 443 }
Is this even possible? Thanks in advance.
There are more options.
Iterate the list and combine the dictionaries. For example,
- set_fact:
tg2: "{{ tg2|d([]) + [item|combine({'targets':_targets})] }}"
loop: "{{ target_groups }}"
vars:
_targets: "{{ dict(targets|product([item.port]))|
dict2items(key_name='Id', value_name='Port') }}"
gives the updated list of dictionaries
tg2:
- name: http
port: 80
targets:
- Id: 1.2.3.4
Port: 80
- Id: 2.3.4.5
Port: 80
- name: https
port: 443
targets:
- Id: 1.2.3.4
Port: 443
- Id: 2.3.4.5
Port: 443
The next option is putting the code into the vars. For example, the expression below gives the same result
tg2: "{{ target_groups|
json_query('[].[port]')|
map('product', targets)|
map('map', 'zip', ['Port', 'Id'])|
map('map', 'map', 'reverse')|
map('map', 'community.general.dict')|
map('community.general.dict_kv', 'targets')|
zip(target_groups)|
map('combine')|
list }}"
Example of a complete playbook
- hosts: localhost
vars:
target_groups:
- name: http
port: 80
- name: https
port: 443
targets:
- 1.2.3.4
- 2.3.4.5
tg2: "{{ target_groups|
json_query('[].[port]')|
map('product', targets)|
map('map', 'zip', ['Port', 'Id'])|
map('map', 'map', 'reverse')|
map('map', 'community.general.dict')|
map('community.general.dict_kv', 'targets')|
zip(target_groups)|
map('combine')|
list }}"
tasks:
- debug:
var: tg2
Create the structure in Jinja if you want to. For example, the expressions below give the same result too
_tg2: |-
{% for i in target_groups %}
-
{% for k, v in i.items() %}
{{ k }}: {{ v }}
{% endfor %}
targets:
{% for ip in targets %}
- Id: {{ ip }}
Port: {{ i.port }}
{% endfor %}
{% endfor %}
tg2: "{{ _tg2|from_yaml }}"
Still rather ugly, but a slightly more readable solution is to build YAML and use the to_yaml filter. I've not found a way to avoid doing this in two steps so far, but this is an example of what I mean:
--- # test.yml
- name: test
hosts: localhost
# user: root
vars:
bar:
- apple
- banana
- carrot
foo: |-
{% for x in bar %}
- greet: "hello {{ x }}"
farewell: "hello {{ x }}"
{% endfor %}
tasks:
- name: test
debug:
msg: "{{ item }}"
loop: "{{ foo | from_yaml }}"
Running:
ansible-playbook test.yml
Gives:
PLAY [test] ******************************************************************************************
TASK [Gathering Facts] *******************************************************************************
ok: [localhost]
TASK [test] ******************************************************************************************
ok: [localhost] => (item={'greet': 'hello apple', 'farewell': 'hello apple'}) => {
"msg": {
"farewell": "hello apple",
"greet": "hello apple"
}
}
ok: [localhost] => (item={'greet': 'hello banana', 'farewell': 'hello banana'}) => {
"msg": {
"farewell": "hello banana",
"greet": "hello banana"
}
}
ok: [localhost] => (item={'greet': 'hello carrot', 'farewell': 'hello carrot'}) => {
"msg": {
"farewell": "hello carrot",
"greet": "hello carrot"
}
}
I need to role which adds records to my zones (bind9)
In hostvars I created vars like as below:
zones:
zone.name1:
- [ type: A, name: mike, ip: 192.168.1.10 ]
- [ type: A, name: bob, ip: 192.168.1.11 ]
zone.name2:
- [ type: A, name: alice, ip: 192.168.1.12 ]
- [ type: A, name: joanne, ip: 192.168.1.13 ]
In role/tasks/main.yaml
- lineinfile:
path: "/etc/bind/zones/{{ item }}.zone"
line: '# IN "{{ item.value.type }}" "{{ item.value.name }}" "{{ item.value.i }}"'
with_items: "{{ zones | dict2items }}"
How to get a result which adds a new record to the zone file?
There is couple of things wrong here.
You are using list notation for dict ([ type: A, name: mike, ip: 192.168.1.10 ] should be { type: A, name: mike, ip: 192.168.1.10 })
Your data structure requires two loops which you cannot do directly in the playbook.
You probably also want to have the freedom to remove records when they are not needed which doesn't work just like that when using lineinfile.
The following solution fixes all the above problems:
# main.yaml
---
- hosts: all
gather_facts: no
connection: local
vars:
zones:
zone.name1:
- { type: A, name: mike, ip: 192.168.1.10 }
# Remove this record
- { type: A, name: bob, ip: 192.168.1.11, state: absent }
zone.name2:
- { type: A, name: alice, ip: 192.168.1.12 }
- { type: A, name: joanne, ip: 192.168.1.13 }
tasks:
- include_tasks: lines.yaml
loop: "{{ zones | dict2items }}"
loop_control:
loop_var: records
Another task file which we loop through:
# lines.yaml
---
- lineinfile:
path: /tmp/{{ records.key }}.zone
line: >-
# IN "{{ item.type }}" "{{ item.name }}" "{{ item.ip }}"
regexp: >-
^#\s+IN\s+"{{ item.type }}"\s+"{{ item.name }}"\s+"{{ item.ip }}"$
state: >-
{{ 'present' if 'state' not in item or item.state == 'present' else 'absent' }}
loop: "{{ records.value }}"
Execute it with this command:
ansible-playbook -i localhost, --diff main.yaml
I'm creating some ec2 instances from a specific image, then trying to get a list of disks attached to these instances.
The problem is when I try to loop over the registered variable from the create instance task, I got an error
I have tried the solution from this post but with no luck
ansible get aws ebs volume id which already exist
- name: create instance
ec2:
region: us-east-1
key_name: xxxxxxx
group: xxxxxx
instance_type: "{{ instance_type }}"
image: "{{ instance_ami }}"
wait: yes
wait_timeout: 500
instance_tags:
Name: "{{ item.name }}"
vpc_subnet_id: "{{ item.subnet }}"
register: ec2
loop: "{{ nodes }}"
- name: show attached volumes Ids
debug:
msg: "{{ item.block_device_mapping | map(attribute='volume_id') }}"
loop: "{{ ec2.results[0].instances }}"
while printing only msg: "{{ item.block_device_mapping }}" I get:
"msg": {
"/dev/sda1": {
"delete_on_termination": true,
"status": "attached",
"volume_id": "vol-xxxxxxx"
},
"/dev/xvdb": {
"delete_on_termination": false,
"status": "attached",
"volume_id": "vol-xxxxxx"
},
"/dev/xvdc": {
"delete_on_termination": false,
"status": "attached",
"volume_id": "vol-xxxxxx"
}
}
but when I use
msg: "{{ item.block_device_mapping | map(attribute='volume_id') }}"
I get this error:
"msg": "[AnsibleUndefined, AnsibleUndefined, AnsibleUndefined]"
The task below
- debug:
msg: "{{ item }}: {{ block_device_mapping[item].volume_id }}"
loop: "{{ block_device_mapping.keys() }}"
gives the {device: volume_id} tuples (grep msg):
"msg": "/dev/xvdb: vol-xxxxxx"
"msg": "/dev/xvdc: vol-xxxxxx"
"msg": "/dev/sda1: vol-xxxxxxx"
To iterate instances use json_query. The task below
- debug:
msg: "{{ item.block_device_mapping|json_query('*.volume_id') }}"
loop: "{{ ec2.results[0].instances }}"
gives:
"msg": [
"vol-xxxxxx",
"vol-xxxxxx",
"vol-xxxxxxx"
]
and the task below with zip
- debug:
msg: "{{ item.block_device_mapping.keys()|zip(
item.block_device_mapping|json_query('*.volume_id'))|list }}"
loop: "{{ ec2.results[0].instances }}"
gives the list of lists:
"msg": [
[
"/dev/xvdb",
"vol-xxxxxx"
],
[
"/dev/xvdc",
"vol-xxxxxx"
],
[
"/dev/sda1",
"vol-xxxxxxx"
]
]
and the task below with dict
- debug:
msg: "{{ dict (item.block_device_mapping.keys()|zip(
item.block_device_mapping|json_query('*.volume_id'))) }}"
loop: "{{ ec2.results[0].instances }}"
gives the tuples
"msg": {
"/dev/sda1": "vol-xxxxxxx",
"/dev/xvdb": "vol-xxxxxx",
"/dev/xvdc": "vol-xxxxxx"
}
The mistake:
So the main mistake you made was thinking of item.block_device_mapping as if it was the map you wanted to work with instead of a map within a map. That is, the keys that you have to first find would, according to the msg that you printed /dev/sda, /dev/xvdb and /dev/xvdc.
So first you'd have to make an array with the keys of the parent map. In the question you can see the necessary code to make Jinja get you the necessary strings:
# The necessary filter to get that array should be something along these lines
item['block_device_mapping'] | list() | join(', ')
You should register that to then loop over,giving you the keys you need to access those elements' attributes.
I have a situation where i am trying to remove the item from a list. but i am not getting expected result. Please help me what I am doing wrong here?
here is the list :
"get_ec2_id.instances[0].tags": {
"Name": "test-db-system-2",
"aws:cloudformation:logical-id": "DBInstance",
"aws:cloudformation:stack-id": "arn:aws:cloudformation:us-east-1:123456789012:stack/test-db-system-2/0115v0a0-5d44-17e8-a024-503ama4a5qd1",
"aws:cloudformation:stack-name": "test-db-system-2",
"dbsystem:stack": "test-db-system-2",
"dbsystem:type": "db"
}
}
I am trying to remove the all "aws:cloudformation" tags from a list using below filter:
"{{ get_ec2_id.instances[0].tags | reject('search','aws:') | list }}"
I am getting the below result:
ok: [10.52.8.101] => {
"instances_tags": [
"dbsystem:type",
"dbsystem:stack",
"Name"
]
}
but I am expected below result :
"instances_tags": [
"dbsystem:stack": "test-db-system-2",
"dbsystem:type": "db"
"Name" : "test-db-system-2",
]
}
Help me to solve the issue.
More generic solution where input is a dict and blacklist is a list:
---
- set_fact:
blacklist:
- bad1
- bad2
- set_fact:
output: {}
- name: remove blacklisted items from input
set_fact:
output: "{{ output | combine({item.key: item.value}) }}"
when: item.key not in blacklist
loop: "{{ input | dict2items }}"
Given the data
get_ec2_id:
instances:
- tags:
Name: test-db-system-2
aws:cloudformation:logical-id: DBInstance
aws:cloudformation:stack-id: arn:aws:cloudformation:us-east-1:123456789012:stack/test-db-system-2/0115v0a0-5d44-17e8-a024-503ama4a5qd1
aws:cloudformation:stack-name: test-db-system-2
dbsystem:stack: test-db-system-2
dbsystem:type: db
Use rejectattr. For example
dict2: "{{ get_ec2_id.instances.0.tags|
dict2items|
rejectattr('key', 'search', 'aws:')|
items2dict }}"
gives
dict2:
Name: test-db-system-2
dbsystem:stack: test-db-system-2
dbsystem:type: db
Then, convert the dictionary into a list of dictionaries
instances_tags: "{{ dict2|
dict2items|
json_query('[].[[key, value]]')|
map('community.general.dict')|
list }}"
gives
instances_tags:
- Name: test-db-system-2
- dbsystem:stack: test-db-system-2
- dbsystem:type: db
Use this:
---
- name: dictionary
hosts: localhost
gather_facts: False
connection: local
vars:
get_ec2_id:
instances:
tags:
Name: "test-db-system-2"
"aws:cloudformation:logical-id": "DBInstance"
"aws:cloudformation:stack-id": "arn:aws:cloudformation:us-east-1:123456789012:stack/test-db-system-2/0115v0a0-5d44-17e8-a024-503ama4a5qd1"
"aws:cloudformation:stack-name": "test-db-system-2"
"dbsystem:stack": "test-db-system-2"
"dbsystem:type": "db"
dict2: {}
tasks:
- name: Fact1
set_fact:
dict: "{{ get_ec2_id.instances.tags }}"
- name: Debug1
debug:
var: dict
- name: Fact2
set_fact:
dict2: "{{ dict2 | combine({item.key: item.value}) }}"
when: "{{ item.key.find('aws:') }}"
with_dict: "{{ dict }}"
- name: Debug2
debug:
var: dict2
Output:
TASK [Debug2] ******************************************************************
ok: [localhost] => {
"dict2": {
"Name": "test-db-system-2",
"dbsystem:stack": "test-db-system-2",
"dbsystem:type": "db"
}
}