I am using ansible to set up a distributed application. i'm installing nodes, and then creating virtual interfaces, and cannot have more virtual interfaces than nodes. therefore, if i install on X nodes, and Y nodes fail, I need to check there are no more that (X-Y) virtual interfaces.
Is there a way to get, for a specific task, a numerical value of how many nodes succeeded/failed, so i can later use it to check the number of virtual interfaces?
Use ansible-runner. See Runner Artifact Job Events and "stats" in particular. For example ansible-runner and the playbook
shell> cat private3/project/test.yml
- hosts: test_01:test_02
gather_facts: false
tasks:
- debug:
var: inventory_hostname
- fail:
msg: Fail test_02
when: inventory_hostname == 'test_02'
shell> ansible-runner -p test.yml -i ID01 run private3
...
ASK [fail] ********************************************************************
skipping: [test_01]
fatal: [test_02]: FAILED! => {"changed": false, "msg": "Fail test_02"}
...
created records in the directory private3/artifacts/ID01/job_events/. I'm not aware of any publicly available tool to analyze the events. I've created a playbook that displays failed tasks
shell> cat pb.yml
- hosts: localhost
gather_facts: false
vars:
events_dir: private3/artifacts/ID01/job_events
tasks:
- find:
paths: "{{ events_dir }}"
register: result
- include_vars:
file: "{{ item }}"
name: "{{ 'my_var_' ~ my_idx }}"
loop: "{{ result.files|json_query('[].path') }}"
loop_control:
index_var: my_idx
label: "{{ my_idx }}"
- set_fact:
my_events: "{{ my_events|default({})|
combine({my_key: lookup('vars', my_key)}) }}"
loop: "{{ range(0, result.matched)|list }}"
loop_control:
index_var: my_idx
vars:
my_key: "{{ 'my_var_' ~ my_idx }}"
- set_fact:
my_list: "{{ my_events|json_query('*.{counter: counter,
event: event,
task: event_data.task_action,
host: event_data.host}') }}"
- debug:
var: item
loop: "{{ my_list|sort(attribute='counter') }}"
loop_control:
label: "{{ item.counter }}"
when: item.event == 'runner_on_failed'
gives
shell> ansible-playbook pb.yml
...
skipping: [localhost] => (item=11)
ok: [localhost] => (item=12) => {
"ansible_loop_var": "item",
"item": {
"counter": 12,
"event": "runner_on_failed",
"host": "test_02",
"task": "fail"
}
}
skipping: [localhost] => (item=13)
...
Feel free to fit the playbook to your needs.
Related
I'm trying to make a list with IP addresses of various hosts and then use this list in another task. My question is, how can I pick an IP (I need the public IP) from the output of each host and add it to a list? I need the IPs that do not start with 10..
Later, I need to use this list in another task.
I extract this information by running this playbook:
- hosts: facts
become: true
gather_facts: True
tasks:
- debug:
msg: "The ip: {{ item }}"
with_items: "{{ ansible_all_ipv4_addresses }}"
Later, I need to use this list in another task:
- wait_for:
host: "{{ item[0] }}"
port: "{{ item[1] }}"
state: started
delay: 0
timeout: 2
delegate_to: localhost
become: false
ignore_errors: no
ignore_unreachable: yes
register: result
failed_when: not result.failed
with_nested:
- [ IP LIST HERE]
- [443,80,9200,9300,22,5432,6432]
You can access those values from the hostvars right away, then use a reject filter with a match test in order to reject what you don't want to test for.
Which, in a debug task would gives:
# note: ports list reduced for brevity
- debug:
msg: "I should wait for interface {{ item.0 }}:{{ item.1 }}"
loop: >-
{{
hostvars
| dict2items
| selectattr('key', 'in', ansible_play_hosts)
| map(attribute='value.ansible_all_ipv4_addresses', default=[])
| flatten
| reject('match', '10\..*')
| product(_ports)
}}
loop_control:
label: "{{ item.0 }}"
run_once: true
delegate_to: localhost
vars:
_ports:
- 22
- 80
In my lab, this give:
ok: [ansible-node-1 -> localhost] => (item=172.18.0.3) =>
msg: I should wait for interface 172.18.0.3:22
ok: [ansible-node-1 -> localhost] => (item=172.18.0.3) =>
msg: I should wait for interface 172.18.0.3:80
ok: [ansible-node-1 -> localhost] => (item=172.18.0.4) =>
msg: I should wait for interface 172.18.0.4:22
ok: [ansible-node-1 -> localhost] => (item=172.18.0.4) =>
msg: I should wait for interface 172.18.0.4:80
Try the example below
shell> cat pb.yml
- hosts: all
vars:
ip_list: "{{ ansible_play_hosts|
map('extract', hostvars, 'ansible_all_ipv4_addresses')|
map('first')|list }}"
ip_list_reject: "{{ ip_list|reject('match', '10\\.')|list }}"
tasks:
- setup:
gather_subset: network
- block:
- debug:
var: ip_list
- debug:
var: ip_list_reject
- wait_for:
host: "{{ item.0 }}"
port: "{{ item.1 }}"
state: started
delay: 0
timeout: 2
delegate_to: localhost
register: result
with_nested:
- "{{ ip_list_reject }}"
- [443, 80, 9200, 9300, 22, 5432, 6432]
run_once: true
Here's the scenario:
a playbook that calls a role to create users in multiple servers, including a VM Scale Set (where ansible_hostnames can't be predicted) - inventory is already being dynamically generated and works fine and not the issue
a users dict variable will provide the user list as well as a series of attributes for each
one of these attributes is a server list named target_servers - this variable's attribute is the actual issue
target_servers is used by the playbook to decide if the user will be present/absent on that particular server - it complements ansible's inventory
target_servers might include only the starting name of a particular target host, a sub-string, like "vmss" as a "vmss*" wildcard, but also fixed hostnames server12345, server12346, etc.
so, dynamic inventory tells ansible which servers to connect to, but the variable tells it whether the user should be created or removed from that particular servers (i.e. servers have different users)
Objective(s):
Have a conditional that checks if a target_server list element content matches the ansible_hostname (i.e. if the substring found in the target_servers list (from the users dict) matches, then we provision the user; additionally, off course, if the list provides the entire hostname, it should match and the users also be provisioned)
Here's the code:
---
- hosts: all
become: yes
vars:
users:
user1:
is_sudo: no
is_chrooted: yes
auth_method: hvault
sa_homedir: firstname1lastname1
state: present
target_servers:
- vmss
- ubuntu
user2:
is_sudo: no
is_chrooted: yes
auth_method: hvault
sa_homedir: firstname2lastname2
state: present
target_servers:
- vmss
- ubuntu18
tasks:
- debug:
msg: "{{ ansible_hostname }}"
- debug:
msg: "{{ item.value.target_servers }}"
loop: "{{ lookup('dict', users|default({})) }}"
# This is just to exemplify what I'm trying to achieve as it is not supposed to work
- debug:
msg: "ansible_hostname is in target_servers of {{ item.key }}"
loop: "{{ lookup('dict', users|default({})) }}"
when: ansible_hostname is match(item.value.target_servers)
Here's the output showing that the match string test cannot be applied to a list (as expected):
TASK [debug] ************************************************************************************************************************************************
ok: [ubuntu18] =>
msg: ubuntu18
TASK [debug] ************************************************************************************************************************************************
ok: [ubuntu18] => (item={'key': 'user1', 'value': {'is_sudo': False, 'is_chrooted': True, 'auth_method': 'hvault', 'sa_homedir': 'firstname1lastname1', 'state': 'present', 'target_servers': ['vmss', 'ubuntu']}}) =>
msg:
- vmss
- ubuntu
ok: [ubuntu18] => (item={'key': 'user2', 'value': {'is_sudo': False, 'is_chrooted': True, 'auth_method': 'hvault', 'sa_homedir': 'firstname2lastname2', 'state': 'present', 'target_servers': ['vmss', 'ubuntu18']}}) =>
msg:
- vmss
- ubuntu18
TASK [debug] ************************************************************************************************************************************************
fatal: [ubuntu18]: FAILED! =>
msg: |-
The conditional check 'ansible_hostname is match(item.value.target_servers)' failed. The error was: Unexpected templating type error occurred on ({% if ansible_hostname is match(item.value.target_servers) %} True {% else %} False {% endif %}): unhashable type: 'list'
The error appears to be in 'test-play-users-core.yml': line 32, column 5, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- debug:
^ here
Already tried researching about selectattr, json_query and subelements but I currently lack the understanding on how to make them work to match a substring inside a dict attribute that is a list.
In the example above, by changing from is match() to in, exact hostnames work fine, but that is not the goal. I need to match both exact hostnames and sub-strings for these hostnames.
Any help on how to accomplish this or suggestions about alternate methods will be greatly appreciated.
The example here might work if I could find a way to run it against a list (target_servers) after having already looped through the entire dictionary (are nested loops possible?): https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html#testing-strings
I guess I've just found what I needed: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/subelements_lookup.html
Will try and provide an update soon.
Update: yes, subelements work! Here's the code needed:
- name: test 1
debug:
msg: "{{ item.1 }} matches {{ ansible_hostname }}"
with_subelements:
- "{{ users }}"
- target_servers
when: >
ansible_hostname is match(item.1)
You can use the select filter to apply the in test to all the elements of your users' target_servers list.
This would be your debug task:
- debug:
msg: "hostname is in target_servers of {{ item.key }}"
loop: "{{ users | dict2items }}"
loop_control:
label: "{{ item.key }}"
when: >-
item.value.target_servers
| select('in', inventory_hostname)
| length > 0
Given the playbook:
- hosts: all
gather_facts: false
vars:
_hostname: ubuntu18
users:
user1:
target_servers:
- vmss
- ubuntu
user2:
target_servers:
- vmss
- ubuntu18
tasks:
- debug:
msg: "hostname is in target_servers of {{ item.key }}"
loop: "{{ users | dict2items }}"
loop_control:
label: "{{ item.key }}"
when: >-
item.value.target_servers
| select('in', inventory_hostname)
| length > 0
This yields:
ok: [ubuntu18] => (item=user1) =>
msg: hostname is in target_servers of user1
ok: [ubuntu18] => (item=user2) =>
msg: hostname is in target_servers of user2
Doing it with subelements instead:
- hosts: all
gather_facts: false
vars:
_hostname: ubuntu18
users:
user1:
target_servers:
- vmss
- ubuntu
user2:
target_servers:
- vmss
- ubuntu18
tasks:
- debug:
msg: "hostname is in target_servers of {{ item.0.key }}"
loop: "{{ users | dict2items | subelements('value.target_servers') }}"
loop_control:
label: "{{ item.0.key }}"
when: item.1 in inventory_hostname
Will yield:
skipping: [ubuntu18] => (item=user1)
ok: [ubuntu18] => (item=user1) =>
msg: hostname is in target_servers of user1
skipping: [ubuntu18] => (item=user2)
ok: [ubuntu18] => (item=user2) =>
msg: hostname is in target_servers of user2
I m trying to iterate over some variables in an ansible role. However, I want to ignore if the var is empty ex:ns3 from below code? I m trying when item length is greater than 0 but it seems not working? any ideas on how to do it?
---
- hosts: localhost
gather_facts: no
vars:
ns1: adm_analytics
ns2: adm_snap
ns3: ""
ns4: adm_eck
tasks:
- name: print namespace loop
with_items:
- "{{ ns1 }}"
- "{{ ns2 }}"
- "{{ ns3 }}"
- "{{ ns4 }}"
include_role:
name: verify_pod_status
vars:
NAMESPACE: "{{ item }}"
when: "{{ item | lenght > 0 }}"
You may have a look into the documentation about Conditionals.
There are some typos in your when clause.
---
- hosts: localhost
gather_facts: false
vars:
ns1: adm_analytics
ns2: adm_snap
ns3: ""
ns4: adm_eck
tasks:
- name: Print namespace loop
debug:
msg: "{{ item }}"
when: item | length > 0
with_items:
- "{{ ns1 }}"
- "{{ ns2 }}"
- "{{ ns3 }}"
- "{{ ns4 }}"
result into an output of
TASK [Print namespace loop] **************
ok: [localhost] => (item=adm_analytics) =>
msg: adm_analytics
ok: [localhost] => (item=adm_snap) =>
msg: adm_snap
ok: [localhost] => (item=adm_eck) =>
msg: adm_eck
when: condition is expanded by default. Fix the syntax
when: item|length > 0
Make your life easier and put the ns* variables into a dictionary. Then you can simply reference the values in the loop instead of listing the variables again. For example, the playbook
shell> cat playbook.yml
- hosts: localhost
gather_facts: no
vars:
ns:
ns1: adm_analytics
ns2: adm_snap
ns3: ""
ns4: adm_eck
tasks:
- name: print namespace loop
include_role:
name: verify_pod_status
loop: "{{ ns.values()|list }}"
vars:
NAMESPACE: "{{ item }}"
when: item|length > 0
and the role
shell> cat roles/verify_pod_status/tasks/main.yml
- debug:
var: NAMESPACE
give (abridged)
TASK [verify_pod_status : debug] ***********************************
skipping: [localhost] => (item=)
TASK [verify_pod_status : debug] ***********************************
ok: [localhost] =>
NAMESPACE: adm_analytics
TASK [verify_pod_status : debug] ***********************************
ok: [localhost] =>
NAMESPACE: adm_snap
TASK [verify_pod_status : debug] ***********************************
ok: [localhost] =>
NAMESPACE: adm_eck
I have got multiple yaml files on remote machine. I would like to parse those files in order to get information about names for each kind (Deployment, Configmap, Secret) of object, For example:
...
kind: Deployment
metadata:
name: pr1-dep
...
kind: Secret
metadata:
name: pr1
...
....
kind: ConfigMap
metadata:
name: cm-pr1
....
Ecpected result:
3 variables:
deployments = [pr1-dep]
secrets = [pr1]
configmaps = [cm-pr1]
I started with:
- shell: cat "{{ item.desc }}"
with_items:
- "{{ templating_register.results }}"
register: objs
but i have no idea how to correctly parse item.stdout from objs
Ansible has a from_yaml filter that takes YAML text as input and outputs an Ansible data structure. So for example you can write something like this:
- hosts: localhost
gather_facts: false
tasks:
- name: Read objects
command: "cat {{ item }}"
register: objs
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
- debug:
msg:
- "kind: {{ obj.kind }}"
- "name: {{ obj.metadata.name }}"
vars:
obj: "{{ item.stdout | from_yaml }}"
loop: "{{ objs.results }}"
loop_control:
label: "{{ item.item }}"
Given your example files, this playbook would output:
PLAY [localhost] ***************************************************************
TASK [Read objects] ************************************************************
changed: [localhost] => (item=deployment.yaml)
changed: [localhost] => (item=configmap.yaml)
changed: [localhost] => (item=secret.yaml)
TASK [debug] *******************************************************************
ok: [localhost] => (item=deployment.yaml) => {
"msg": [
"kind: Deployment",
"name: pr1-dep"
]
}
ok: [localhost] => (item=configmap.yaml) => {
"msg": [
"kind: ConfigMap",
"name: pr1-cm"
]
}
ok: [localhost] => (item=secret.yaml) => {
"msg": [
"kind: Secret",
"name: pr1"
]
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Creating the variables you've asked for is a little trickier. Here's
one option:
- hosts: localhost
gather_facts: false
tasks:
- name: Read objects
command: "cat {{ item }}"
register: objs
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
- name: Create variables
set_fact:
names: >-
{{
names|combine({
obj.kind.lower(): [obj.metadata.name]
}, list_merge='append')
}}
vars:
names: {}
obj: "{{ item.stdout | from_yaml }}"
loop: "{{ objs.results }}"
loop_control:
label: "{{ item.item }}"
- debug:
var: names
This creates a single variable named names that at the end of the
playbook will contain:
{
"configmap": [
"pr1-cm"
],
"deployment": [
"pr1-dep"
],
"secret": [
"pr1"
]
}
The key to the above playbook is our use of the combine filter, which can be used to merge dictionaries and, when we add list_merge='append', handles keys that resolve to lists by appending to the existing list, rather than replacing the existing key.
Include the dictionaries from the files into the new variables, e.g.
- include_vars:
file: "{{ item }}"
name: "objs_{{ item|splitext|first }}"
register: result
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
This will create dictionaries objs_deployment, objs_configmap, and objs_secret. Next, you can either use the dictionaries
- set_fact:
objs: "{{ objs|d({})|combine({_key: _val}) }}"
loop: "{{ query('varnames', 'objs_') }}"
vars:
_obj: "{{ lookup('vars', item) }}"
_key: "{{ _obj.kind }}"
_val: "{{ _obj.metadata.name }}"
, or the registered data
- set_fact:
objs: "{{ dict(_keys|zip(_vals)) }}"
vars:
_query1: '[].ansible_facts.*.kind'
_keys: "{{ result.results|json_query(_query1)|flatten }}"
_query2: '[].ansible_facts.*.metadata[].name'
_vals: "{{ result.results|json_query(_query2)|flatten }}"
Both options give
objs:
ConfigMap: cm-pr1
Deployment: pr1-dep
Secret: pr1
Sorry if there are many posts about variables inside variable my use case is different.
Trying to access an element from a variable list "efs_list" based on the index-number of the current host. There are three hosts in the inventory
vars:
efs_list:
- efs1
- efs2
- efs3
sdb_index: "{{ groups['all'].index(inventory_hostname) }}"
The values should be as follows
host1- efs1
host2- efs2
host3- efs3
Tried accessing it through efs_list.{{ sdb_index }}
for - debug: var=efs_list.{{ sdb_index }} the output is as intended
ok: [10.251.0.174] => {
"efs_list.0": "efs1"
}
ok: [10.251.0.207] => {
"efs_list.1": "efs2"
}
ok: [10.251.0.151] => {
"efs_list.2": "efs3"
}
But for
- debug:
msg: "{{ efs_list.{{ sdb_index }} }}"
fatal: [10.251.0.174]: FAILED! => {"msg": "template error while templating string: expected name or number. String: {{ efs_list.{{ sdb_index }} }}"}
---
- name: SDB Snapshots Creation
hosts: all
remote_user: "centos"
become: yes
vars:
efs_list:
- efs1
- efs2
- efs3
sdb_index: "{{ groups['all'].index(inventory_hostname) }}"
tasks:
- debug: var=efs_list.{{ sdb_index }}
- debug:
msg: "{{ efs_list.{{ sdb_index }} }}"
- name: Get Filesystem ID
become: false
local_action: command aws efs describe-file-systems --creation-token "{{ efs_list.{{ sdb_index }} }}"
--region us-east-1 --query FileSystems[*].FileSystemId --output text
register: fs_id
It should attribute the element of list to current indexenter code here
extract filter will do the job. The input of the filter must be a list of indices and a container (array in this case). The tasks below
- set_fact:
sdb_index: "{{ [] + [ groups['all'].index(inventory_hostname) ] }}"
- debug:
msg: "{{ sdb_index|map('extract', efs_list)|list }}"
give
ok: [host1] => {
"msg": [
"efs1"
]
}
ok: [host2] => {
"msg": [
"efs2"
]
}
ok: [host3] => {
"msg": [
"efs3"
]
}
If the hosts are not sorted in the inventory it's necessary to sort them in the play
- set_fact:
my_hosts: "{{ groups['all']|sort }}"
- set_fact:
sdb_index: "{{ [] + [ my_hosts.index(inventory_hostname) ] }}"
- debug:
msg: "{{ sdb_index|map('extract', efs_list)|list }}"