Ansible iterate over array and use filter - ansible

I'm kinda new to ansible seeking help with below scenario. what I'm trying to do is iterate over the array 'access_key_ids' and run regex_search filter. for the regex_search filter argument is 'item' which is the variable from with_items. It does not work this way below is what I'm trying.
name: Set Fatcs
block:
# extract access key ids from get event response
- set_fact:
event_response_access_key_ids: "{{event_response_access_key_ids}} + [{{event_response.content | regex_search(item)}}]"
with_items: "{{access_key_ids}}"
# check if the response contains access key id for the license
- set_fact:
scwx_output: "{{ (event_response_access_key_ids | length > 0 ) | ternary(event_response, 'License Key does not match with available sensors')}}"
when: event_response.json is undefined
It gives event_response_access_key_ids as empty. but when I hard code a value instead of 'item' it works
Thanks.

I've been testing this solution:
---
- name: Test
hosts: local
gather_facts: False
vars:
event_response:
content: "hi1"
access_key_ids:
- "1"
- "h"
- "3"
tasks:
- name: Fact
set_fact:
event_response_access_key_ids: "{{ event_response_access_key_ids|default([]) + [ event_response.content | regex_search( item ) ] }}"
with_items: "{{ access_key_ids }}"
And It gets the vars properly:
ok: [localhost] => (item=1) => {
"ansible_facts": {
"event_response_access_key_ids": [
"1"
]
},
"changed": false,
"item": "1"
}
ok: [localhost] => (item=h) => {
"ansible_facts": {
"event_response_access_key_ids": [
"1",
"h"
]
},
"changed": false,
"item": "h"
}
ok: [localhost] => (item=3) => {
"ansible_facts": {
"event_response_access_key_ids": [
"1",
"h",
null
]
},
"changed": false,
"item": "3"
}

Related

Ansible include task and loop register different result

I'm trying to use a module as an included task so I can loop several items and register the credentials in a variable.
Files:
main.yml:
---
- hosts: localhost
gather_facts: False
tasks:
- include: tasks/myvault/get-vault.yml
with_items:
- demo
register: output
- debug:
msg: "{{output}}"
get-vault.yml:
- name: Retrieve secret from Vault
community.hashi_vault.vault_kv2_get:
url: https://myvaul
path: "{{ item }}"
auth_method: token
token: '{{ mytoken }}'
namespace: MyNamespace
validate_certs: no
output:
ok: [localhost] => {
"msg": {
"changed": false,
"msg": "All items completed",
"results": [
{
"ansible_loop_var": "item",
"include": "tasks/myvault/get-vault.yml",
"include_args": {},
"item": "demo"
}
],
"skipped": false
}
}
The output doesn't display the "real" result of the task.
If I remove:
register: output and put it in get-vault.yml, here is the result:
ok: [localhost] => {
"msg": {
"changed": false,
"data": {
"data": {
"password": "Password",
"username": "Username"
}, ...
How can I get this result returned when I register the output from my included task?

Select specific cluster with prefix in VMware cluster from Ansible

I'm using the below playbook to list all the VMware cluster belonging to my datacenter, but I need to select a specific cluster containing LAB in the cluster name.
- hosts: localhost
vars_files: 1credentials.yml
tasks:
- name: Gather cluster info from given datacenter
community.vmware.vmware_cluster_info:
hostname: '{{ vcenter_hostname }}'
username: '{{ vcenter_username }}'
password: '{{ vcenter_password }}'
datacenter: SDx-CoE-T7
validate_certs: no
delegate_to: localhost
register: cluster_info
- debug:
msg: "{{ cluster_info }}"
- debug:
msg: "{{ cluster_info.clusters.keys() | list | to_yaml }}"
I'm trying to use selectattr(), but, I am not able to get the exact syntax. Can you please suggest the best option to filter the cluster name containing LAB.
Below is the playbook output:
PLAY [localhost] *******************************************************************************************************************************************************
TASK [Gathering Facts] *************************************************************************************************************************************************
ok: [localhost]
TASK [Gather cluster info from given datacenter] ***********************************************************************************************************************
ok: [localhost]
TASK [debug] ***********************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"changed": false,
"clusters": {
"TEST_LAB": {
"datacenter": "TEST_CoE-T7",
"drs_default_vm_behavior": "fullyAutomated",
"drs_enable_vm_behavior_overrides": true,
"drs_vmotion_rate": 3,
"enable_ha": false,
"enabled_drs": false,
"enabled_vsan": false,
"ha_admission_control_enabled": true,
"ha_failover_level": 1,
"ha_host_monitoring": "enabled",
"ha_restart_priority": [
"medium"
],
"ha_vm_failure_interval": [
30
],
"ha_vm_max_failure_window": [
-1
],
"ha_vm_max_failures": [
3
],
"ha_vm_min_up_time": [
120
],
"ha_vm_monitoring": "vmMonitoringDisabled",
"ha_vm_tools_monitoring": [
"vmMonitoringDisabled"
],
"hosts": [],
"moid": "domain-c28615",
"resource_summary": {
"cpuCapacityMHz": 0,
"cpuUsedMHz": 0,
"memCapacityMB": 0,
"memUsedMB": 0,
"pMemAvailableMB": 0,
"pMemCapacityMB": 0,
"storageCapacityMB": 0,
"storageUsedMB": 0
},
"tags": [],
"vsan_auto_claim_storage": false
},
"TEST_LaaS": {
"datacenter": "TEST_CoE-T7",
"drs_default_vm_behavior": "fullyAutomated",
"drs_enable_vm_behavior_overrides": true,
"drs_vmotion_rate": 3,
"enable_ha": true,
"enabled_drs": true,
"enabled_vsan": false,
"ha_admission_control_enabled": true,
"ha_failover_level": 1,
"ha_host_monitoring": "enabled",
"ha_restart_priority": [
"medium"
],
"ha_vm_failure_interval": [
30
],
"ha_vm_max_failure_window": [
-1
],
"ha_vm_max_failures": [
3
],
"ha_vm_min_up_time": [
120
],
"ha_vm_monitoring": "vmMonitoringDisabled",
"ha_vm_tools_monitoring": [
"vmMonitoringDisabled"
],
"hosts": [
{
"folder": "/TEST_CoE-T7/host/TEST_LaaS",
"name": "172.17.65.84"
},
{
"folder": "/TEST_CoE-T7/host/TEST_LaaS",
"name": "172.17.65.85"
},
{
"folder": "/TEST_CoE-T7/host/TEST_LaaS",
"name": "172.17.168.202"
}
],
"moid": "domain-c861",
"resource_summary": {
"cpuCapacityMHz": 158040,
"cpuUsedMHz": 17494,
"memCapacityMB": 786073,
"memUsedMB": 361592,
"pMemAvailableMB": 0,
"pMemCapacityMB": 0,
"storageCapacityMB": 22707456,
"storageUsedMB": 14326118
},
"tags": [],
"vsan_auto_claim_storage": false
}
},
"failed": false
}
}
TASK [debug] ***********************************************************************************************************************************************************
ok: [localhost] => {
"msg": "[TEST_Automation, TEST_LAB, TEST_LaaS,]\n"
}
If I use ansible version 2.9.27, I'm getting below error:
fatal: [localhost]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{ cluster_info.clusters | dict2items | selectattr('key','contains','Common')| items2dict | list }}): items2dict requires a list, got <class 'generator'> instead."}
I tried with below one removing items2dict:
- debug:
msg: "{{ cluster_info.clusters | dict2items | selectattr('key','contains','TEST') | list }}"
It is giving the below output but I need only the key: TEST_Lab
ok: [localhost] => {
"msg": [
{
"key": "TEST_LAB",
"value": {
"drs_default_vm_behavior": "partiallyAutomated",
"drs_enable_vm_behavior_overrides": true,
"drs_vmotion_rate": 3,
"enable_ha": false,
"enabled_drs": true,
"enabled_vsan": false,
"ha_admission_control_enabled": true,
"ha_failover_level": 1,
"ha_host_monitoring": "enabled",
"ha_restart_priority": [
"medium"
],
"ha_vm_failure_interval": [
30
],
"ha_vm_max_failure_window": [
-1
],
"ha_vm_max_failures": [
3
],
"ha_vm_min_up_time": [
120
],
"ha_vm_monitoring": "vmMonitoringDisabled",
"ha_vm_tools_monitoring": [
"vmMonitoringDisabled"
],
"tags": [],
"vsan_auto_claim_storage": false
}
}
]
}
In order to use selectattr, you need to have a value to filter on, not a key. But transforming a dictionary into a list, containing the key and value is as simple as using the dict2items filter, in Ansible.
Then, to come back to your dictionary, just use the "antonym" filter, items2dict.
So, your debug task ends up being:
- debug:
var: >-
cluster_info.clusters
| dict2items
| selectattr('key','contains','LAB')
| items2dict
Given the task, with a reduced dictionary for readability:
- debug:
var: |-
cluster_info.clusters
| dict2items
| selectattr('key','contains','LAB')
| items2dict
vars:
cluster_info:
clusters:
TEST_LAB:
datacenter: TEST_CoE-T7
drs_default_vm_behavior: fullyAutomated
TEST_LaaS:
datacenter: TEST_CoE-T7
drs_default_vm_behavior: fullyAutomated
This will yield:
ok: [localhost] =>
? |-
cluster_info.clusters
| dict2items
| selectattr('key','contains','LAB')
| items2dict
: TEST_LAB:
datacenter: TEST_CoE-T7
drs_default_vm_behavior: fullyAutomated

Ansible set_fact from dictionary based on item type

I am reading a dictionary from a Kubernetes config map (cm_config below) and am using it to replace variables set in defaults/main.yml like this:
- name: 'Overwrite defaults'
set_fact: "{{ item.key }}={{ item.value }}"
with_dict: "{{ cm_config }}"
This works fine as long as the items are simple variables. But as soon as an item is another dictionary, I'd like to combine the values.
How can I integrate this into the above task? I thought about running the loop twice, with some kind of type check. Not sure how this would work. Additionally, I believe there might be a better way?
One solution below to achieve your requirement in a single task whit just a bit of jinja2 templating and a vars lookup to get existing dict content. The key is to calculate the value based on the variable type.
Note that this does not take into account the situations when the var is a list which will be replaced as all other type of values. This will not either deal with type mismatch between existing vars and config map. e.g. if your existing var is string and the corresponding one in config map a dict it will break.
The following playbook:
---
- hosts: localhost
gather_facts: false
vars:
cm_config:
label1: toto
label2:
a_value: 1
other_value: 2
label3:
a_value: 3
other_value: 4
label4: tata
label1: I am set in play
label3:
some_value: I'm a poor lonesome cowboy
tasks:
- name: show initial state
debug:
var: "{{ item.key }}"
with_dict: "{{ cm_config }}"
- name: Process values from config map
vars:
my_value: >-
{% if item.value is mapping %}
{{ lookup('vars', item.key, default={}) | combine(item.value) }}
{% else %}
{{ item.value }}
{% endif %}
set_fact:
"{{ item.key }}": "{{ my_value }}"
with_dict: "{{ cm_config }}"
- name: Show the result after processing config map
debug:
var: "{{ item.key }}"
with_dict: "{{ cm_config }}"
gives the following result:
PLAY [localhost] ****************************************************************************************************************************************************************************************************************************
TASK [show initial state] *******************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=label1) => {
"ansible_loop_var": "item",
"item": {
"key": "label1",
"value": "toto"
},
"label1": "I am set in play"
}
ok: [localhost] => (item=label2) => {
"ansible_loop_var": "item",
"item": {
"key": "label2",
"value": {
"a_value": 1,
"other_value": 2
}
},
"label2": "VARIABLE IS NOT DEFINED!"
}
ok: [localhost] => (item=label3) => {
"ansible_loop_var": "item",
"item": {
"key": "label3",
"value": {
"a_value": 3,
"other_value": 4
}
},
"label3": {
"some_value": "I'm a poor lonesome cowboy"
}
}
ok: [localhost] => (item=label4) => {
"ansible_loop_var": "item",
"item": {
"key": "label4",
"value": "tata"
},
"label4": "VARIABLE IS NOT DEFINED!"
}
TASK [Process values from config map] *******************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'key': 'label1', 'value': 'toto'})
ok: [localhost] => (item={'key': 'label2', 'value': {'a_value': 1, 'other_value': 2}})
ok: [localhost] => (item={'key': 'label3', 'value': {'a_value': 3, 'other_value': 4}})
ok: [localhost] => (item={'key': 'label4', 'value': 'tata'})
TASK [Show the result after processing config map] ******************************************************************************************************************************************************************************************
ok: [localhost] => (item=label1) => {
"ansible_loop_var": "item",
"item": {
"key": "label1",
"value": "toto"
},
"label1": " toto "
}
ok: [localhost] => (item=label2) => {
"ansible_loop_var": "item",
"item": {
"key": "label2",
"value": {
"a_value": 1,
"other_value": 2
}
},
"label2": " {'a_value': 1, 'other_value': 2} "
}
ok: [localhost] => (item=label3) => {
"ansible_loop_var": "item",
"item": {
"key": "label3",
"value": {
"a_value": 3,
"other_value": 4
}
},
"label3": " {'some_value': \"I'm a poor lonesome cowboy\", 'a_value': 3, 'other_value': 4} "
}
ok: [localhost] => (item=label4) => {
"ansible_loop_var": "item",
"item": {
"key": "label4",
"value": "tata"
},
"label4": " tata "
}
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

How to loop over this dictionary in Ansible?

Say I have this dictionary
war_files:
server1:
- file1.war
- file2.war
server2:
- file1.war
- file2.war
- file3.war
and for now I just want to loop over each item (key), and then over each item in the key (value). I did this
- name: Loop over the dictionary
debug: msg="Key={{ item.key }} value={{ item.value }}"
with_dict: "{{ war_files }}"
And I get this. It is of course correct, but is NOT what I want.
ok: [localhost] => (item={'value': [u'file1.war', u'file2.war'], 'key': u'server1'}) => {
"item": {
"key": "server1",
"value": [
"file1.war",
"file2.war"
]
},
"msg": "Server=server1, WAR=[u'file1.war', u'file2.war']"
}
ok: [localhost] => (item={'value': [u'file1.war', u'file2.war', u'file3.war'], 'key': u'server2'}) => {
"item": {
"key": "server2",
"value": [
"file1.war",
"file2.war",
"file3.war"
]
},
"msg": "Server=server2, WAR=[u'file1.war', u'file2.war', u'file3.war']"
}
I want to get an output that says
"msg": "Server=server1, WAR=file1.war"
"msg": "Server=server1, WAR=file2.war"
"msg": "Server=server2, WAR=file1.war"
"msg": "Server=server2, WAR=file2.war"
"msg": "Server=server2, WAR=file3.war"
IOW, how can I write a task to iterates over the dictionary so it goes through each key, and then the items within each key? In essence, I have a nested array and want to iterate over it?
Hows this
- hosts: localhost
vars:
war_files:
server1:
- file1.war
- file2.war
server2:
- file1.war
- file2.war
- file3.war
tasks:
- name: Loop over subelements of the dictionary
debug:
msg: "Key={{ item.0.key }} value={{ item.1 }}"
loop: "{{ war_files | dict2items | subelements('value') }}"
dict2items, subelements filters are coming in Ansible 2.6.
FYI, if a filter for your objective doesn't exist, you can write your own in python without having to resort to jinja2 hacks. Ansible is easily extendable; filters in filter_plugins/*.py are searched by default adjacent to your plays/roles and are automatically included - see Developing Plugins for details.
Now Ansible allows this
- name: add several users
user:
name: "{{ item.name }}"
state: present
groups: "{{ item.groups }}"
with_items:
- { name: 'testuser1', groups: 'wheel' }
- { name: 'testuser2', groups: 'root' }
EDIT: At the time of writing this answer, Ansible 2.6 wasn't out. Please read the answer provided by #tmoschou, as it is much better.
Well, I couldn't find a very easy way to do it, however, with a little bit of jinja2, we can achieve something of this sort:
/tmp ❯❯❯ cat example.yml
---
- hosts: 127.0.0.1
vars:
war_files:
server1:
- file1.war
- file2.war
server2:
- file1.war
- file2.war
- file3.war
tasks:
- set_fact:
war_files_list_of_dicts: |
{% set res = [] -%}
{% for key in war_files.keys() -%}
{% for value in war_files[key] -%}
{% set ignored = res.extend([{'Server': key, 'WAR':value}]) -%}
{%- endfor %}
{%- endfor %}
{{ res }}
- name: let's debug the crap out of this
debug: var=war_files_list_of_dicts
- name: Servers and their WARs!!!
debug:
msg: "Server={{ item.Server }}, WAR={{ item.WAR }}"
with_items: "{{ war_files_list_of_dicts }}"
And, when the playbook is run:
/tmp ❯❯❯ ansible-playbook example.yml
[WARNING]: provided hosts list is empty, only localhost is available
PLAY [127.0.0.1] ***************************************************************
TASK [setup] *******************************************************************
ok: [127.0.0.1]
TASK [set_fact] ****************************************************************
ok: [127.0.0.1]
TASK [let's debug the crap out of this] ****************************************
ok: [127.0.0.1] => {
"war_files_list_of_dicts": [
{
"Server": "server1",
"WAR": "file1.war"
},
{
"Server": "server1",
"WAR": "file2.war"
},
{
"Server": "server2",
"WAR": "file1.war"
},
{
"Server": "server2",
"WAR": "file2.war"
},
{
"Server": "server2",
"WAR": "file3.war"
}
]
}
TASK [Servers and their WARs!!!] ***********************************************
ok: [127.0.0.1] => (item={'WAR': u'file1.war', 'Server': u'server1'}) => {
"item": {
"Server": "server1",
"WAR": "file1.war"
},
"msg": "Server=server1, WAR=file1.war"
}
ok: [127.0.0.1] => (item={'WAR': u'file2.war', 'Server': u'server1'}) => {
"item": {
"Server": "server1",
"WAR": "file2.war"
},
"msg": "Server=server1, WAR=file2.war"
}
ok: [127.0.0.1] => (item={'WAR': u'file1.war', 'Server': u'server2'}) => {
"item": {
"Server": "server2",
"WAR": "file1.war"
},
"msg": "Server=server2, WAR=file1.war"
}
ok: [127.0.0.1] => (item={'WAR': u'file2.war', 'Server': u'server2'}) => {
"item": {
"Server": "server2",
"WAR": "file2.war"
},
"msg": "Server=server2, WAR=file2.war"
}
ok: [127.0.0.1] => (item={'WAR': u'file3.war', 'Server': u'server2'}) => {
"item": {
"Server": "server2",
"WAR": "file3.war"
},
"msg": "Server=server2, WAR=file3.war"
}
PLAY RECAP *********************************************************************
127.0.0.1 : ok=4 changed=0 unreachable=0 failed=0
Here is my preferred way to loop over dictionaries:
input_data.yml contains the following:
----
input_data:
item_1:
id: 1
info: "Info field number 1"
item_2:
id: 2
info: "Info field number 2"
I then use a data structure like the above in a play using the keys() function and iterate over the data using with_items:
---
- hosts: localhost
gather_facts: false
connection: local
tasks:
- name: Include dictionary data
include_vars:
file: data.yml
- name: Show info field from data.yml
debug:
msg: "Id: {{ input_data[item]['id'] }} - info: {{ input_data[item]['info'] }}"
with_items: "{{ input_data.keys() | list }}"
The above playbook produces the following output:
PLAY [localhost] ***********************************************************
TASK [Include dictionary data] *********************************************
ok: [localhost]
TASK [Show info field from data.yml] ***************************************
ok: [localhost] => (item=item_2) => {
"msg": "Id: 2 - info: Info field item 2"
}
ok: [localhost] => (item=item_3) => {
"msg": "Id: 3 - info: Info field item 3"
}
ok: [localhost] => (item=item_1) => {
"msg": "Id: 1 - info: Info field item 1"
}
PLAY RECAP *****************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0
dict2items
I found myself wanting to iterate over a heterogeneous set of keys and their associated values and use the key-value pair in a task. The dict2items filter is the least painful way I've found. You can find dict2items in Ansible 2.6
Example Dict
systemsetup:
remotelogin: "On"
timezone: "Europe/Oslo"
usingnetworktime: "On"
sleep: 0
computersleep: 0
displaysleep: 0
harddisksleep: 0
allowpowerbuttontosleepcomputer: "Off"
wakeonnetworkaccess: "On"
restartfreeze: "On"
restartpowerfailure: "On"
Example Task
---
- debug:
msg: "KEY: {{ item.key }}, VALUE: {{ item.value }}"
loop: "{{ systemsetup | dict2items }}"
One way of doing it that worked for me was using with_dict. Note the dict should not be named. Just the key value pairs.
- name: ssh config
lineinfile:
dest: /etc/ssh/sshd_config
regexp: '^#?\s*{{item.key}}\s'
line: '{{item.key}} {{item.value}}'
state: present
with_dict:
LoginGraceTime: "1m"
PermitRootLogin: "yes"
PubkeyAuthentication: "yes"
PasswordAuthentication: "no"
PermitEmptyPasswords: "no"
IgnoreRhosts: "yes"
Protocol: 2

Ansible shows error: "One or more undefined variables: 'item' is undefined" when using 'with_items'

I am trying to count the instances inside an elb. This is my Ansible playbook:
- name: Get elb facts
local_action:
module: ec2_elb_facts
name: "{{elb}}"
region: "{{ansible_ec2_placement_region}}"
environment: creds
register: elb_facts
- debug:
var: elb_facts
verbosity: 2
- debug:
msg: "Instance: {{ item.instances }}"
with_items: "{{ elb_facts.elbs }}"
and my output (sensitive data removed):
TASK: [debug ] ****************************************************************
ok: [10.0.0.0] => {
"elb_facts": {
"changed": false,
"elbs": [
{
"availability_zones": [
"ap-southeast-2b",
"ap-southeast-2a"
],
"dns_name": "elbname123.ap-southeast-2.elb.amazonaws.com",
"health_check": {
"healthy_threshold": 2,
"interval": 10,
"target": "TCP:0000",
"timeout": 5,
"unhealthy_threshold": 2
},
"instances": [
{
"id": "i-000000000000000",
"state": null
}
],
"name": "accessgateway",
"scheme": "internal",
"security_groups": [
"sg-00000000"
],
"subnet": [
"subnet-0000000",
"subnet-1111111"
],
"vpc_id": "vpc-000000"
}
],
"invocation": {
"module_args": "",
"module_name": "ec2_elb_facts"
}
}
}
TASK: [debug ] ****************************************************************
fatal: [10.0.0.0] => One or more undefined variables: 'item' is undefined
FATAL: all hosts have already failed -- aborting
So what im trying to do is just loop through and print everything inside the elb_facts, instances variable. From what I can tell it's a hash, containing a list of hashes.
I am using http://docs.ansible.com/ansible/playbooks_loops.html#looping-over-subelements as a reference. I cannot for the life of mine figure out why this is not working.
with_items (and the whole family of with_ loops) is a dictionary key defined in a task, not as a parameter to the action.
Fix the indentation:
- debug:
msg: "Instance: {{ item.instances }}"
with_items: "{{ elb_facts.elbs }}"

Resources