ansible expression to use with with_item - ansible

I have a variable, more specifically it's the result (registered in a variable with register:) of a set_fact task using with_item, let's named it myvar, and to simply, I only display the relevant part of it.
{
"msg": "All items completed",
"results": [
{
"item": "item1",
"stdout_lines": [
[
"result line 1",
"result line 2"
]
]
},
{
"item": "item2",
"stdout_lines": [
[
"result line 1"
]
]
}
]
}
I need to run a task for each "result line X" that will include in the command the value of "item" and the "result line X" value
I'm not sure how to do this.
if I do
- debug:
msg: "{{ item.item }} , {{ item | json_query('stdout_lines[0][*]') | list }}"
with_items: "{{ myvar | json_query('results[*]') }}"
The following would output
item1 ; [u'result line 1',u'result line 2']
item2 ; [u'result line 1']
I have both all stdout_lines entries on the same line. so the split of the stdout_lines should happen in the with_item expression
But here I don't find how I could "fetch" all the stdout_lines of all results and at the same time keep the "item" value.

If you'd have flat stdout_lines, you could just do with_subelements.
In your case, you need to spice it with JMESPath a bit:
---
- hosts: localhost
gather_facts: no
vars:
myvar:
"results": [
{
"item": "item1",
"stdout_lines": [
[
"result line 1",
"result line 2"
]
]
},
{
"item": "item2",
"stdout_lines": [
[
"result line 1"
]
]
}
]
tasks:
- command: echo {{ item.0.name }} {{ item.1 }}
with_subelements:
- "{{ myvar.results | json_query('[].{name:item,lines:stdout_lines[]}') }}"
- lines

Related

Create dictionary from list of strings using delimiter with Ansible

I have a nested list of strings, which I am trying to convert into a dictionary. The list seems to be in a reasonable format, but my dictionary is getting overwritten each time I append to it.
Initial list:
TASK [test first_list] **********************************************************************************************************************************************************************************************************************************************************************************************************
ok: [node11] => {
"first_list": [
[
"DEVNAME: /dev/sanstorage/node11-data103-2fd31b74e1ef5a8006c9ce9000b10399f",
"UUID: 2751719f-d1a2-49b0-9e42-3d686c90d2e6",
"TYPE: xfs"
],
[
"DEVNAME: /dev/sanstorage/node11-data104-2e81f1279171dce656c9ce9000b10399f",
"UUID: 6a6265d6-b103-471e-9e25-e8cc5f5585a8",
"TYPE: xfs"
],
[
"DEVNAME: /dev/sanstorage/node11-data102-2e385a327788d866e6c9ce9000b10399f",
"UUID: 8c52d974-8584-4aa6-89b8-f1e1db016118",
"TYPE: xfs"
],
[
"DEVNAME: /dev/sanstorage/node11-data101-241afe5ab93b5c0ee6c9ce9000b10399f",
"UUID: 94b56164-6717-4b82-8f11-86fd94a39672",
"TYPE: xfs"
],
[
"DEVNAME: /dev/sanstorage/node11-data100-25626d38d4c4239456c9ce9000b10399f",
"UUID: 55a1a388-fe0a-4dd0-980a-a10c5317952e",
"TYPE: xfs"
],
[
"DEVNAME: /dev/sanstorage/node11-data114-231d2661b7ab88f8f6c9ce9000b10399f",
"UUID: f87ad708-1d12-41a5-9441-d32a97b5318c",
"TYPE: ext4"
],
[
"DEVNAME: /dev/sanstorage/node11-data115-2d3a824975e90550f6c9ce9000b10399f",
"UUID: b8b79886-9710-4205-900b-7b9d7d4ad933",
"TYPE: ext4"
],
[
"DEVNAME: /dev/sanstorage/oneview-FA-data8165-284392eae1ad17d846c9ce9000b10399f",
"UUID: c5a43057-676e-49b7-b2a9-b53a40f7010b",
"TYPE: xfs"
],
[
"DEVNAME: /dev/sanstorage/oneview-FA-data3420-2e262703236f9b7046c9ce9000b10399f",
"UUID: 1a70c187-9364-4f48-92f8-f9b9dec9824f",
"TYPE: xfs"
],
[
"DEVNAME: /dev/sanstorage/node11-data112-2464954fd324508e66c9ce9000b10399f",
"UUID: 2238a12e-2ca1-466e-8617-11051e1d612e",
"TYPE: ext4"
],
[
"DEVNAME: /dev/sanstorage/node11-data111-25bb14827531149456c9ce9000b10399f",
"UUID: db37479f-80b0-46b2-85e0-aef679bea164",
"TYPE: ext4"
],
[
"DEVNAME: /dev/sanstorage/node11-data105-28ac6825aa80520d46c9ce9000b10399f",
"UUID: e987fc7c-9a4f-46ea-91f0-4d5f37d3421e",
"TYPE: xfs"
],
[
"DEVNAME: /dev/sanstorage/node11-data113-2177c50c4dd5063506c9ce9000b10399f",
"UUID: 36c1194d-ac4f-4cee-8688-353974cb6be0",
"TYPE: ext4"
]
]
}
From here, I try to form a dictionary, but only the last list entry is stored at the end:
TASK [test final_list] **********************************************************************************************************************************************************************************************************************************************************************************************************
ok: [node11] => {
"final_list": {
"DEVNAME": "/dev/sanstorage/node11-data113-2177c50c4dd5063506c9ce9000b10399f",
"TYPE": "ext4",
"UUID": "36c1194d-ac4f-4cee-8688-353974cb6be0"
}
}
Ideally, it would look something like this, I'm guessing I need to nest the dictionary within a list?
"final list": [
{
"DEVNAME: /dev/sanstorage/node11-data103-2fd31b74e1ef5a8006c9ce9000b10399f",
"UUID: 2751719f-d1a2-49b0-9e42-3d686c90d2e6",
"TYPE: xfs"
},
{
"DEVNAME: /dev/sanstorage/node11-data104-2e81f1279171dce656c9ce9000b10399f",
"UUID: 6a6265d6-b103-471e-9e25-e8cc5f5585a8",
"TYPE: xfs"
},
{
"DEVNAME: /dev/sanstorage/node11-data102-2e385a327788d866e6c9ce9000b10399f",
"UUID: 8c52d974-8584-4aa6-89b8-f1e1db016118",
"TYPE: xfs"
},
... snip ...
Below is the important part of the playbook:
- set_fact:
first_list: "{{ first_list | default([]) + [disk_data.split('\n')] }}"
vars:
disk_data: '{{ item.stdout }}'
with_items: "{{ disk_check.results }}"
- name: test first_list
debug:
var: first_list
- set_fact: final_list:{}
- set_fact:
final_list: "{{ final_list | default([]) | combine(dict([ item.partition(':')[::2]|map('trim')])) }}"
with_items: "{{ first_list }}"
- name: test final_list
debug:
var: final_list
Thoughts?
TL;DR
- debug:
msg: "{{ disk_check.results | map(attribute='stdout') | map('from_yaml') | list }}"
You did not provide the initial task which returns disk_check. From your usage in your example, I'll take for granted it is a shell or command task used in a loop.
Meanwhile you still provided enough information so we can avoid falling into an x/y problem. Your question is basically "how can I split a string on delimiters and use the result to create a dict?" where what you really want is more "How can I parse a string representing a yaml dict into an actual dict?"
To start with
- set_fact:
first_list: "{{ first_list | default([]) + [disk_data.split('\n')] }}"
vars:
disk_data: '{{ item.stdout }}'
with_items: "{{ disk_check.results }}"
This is going through all your command results, spliting stdout on a new line and adding that list to a top level list.
Since the output of shell/command contains a stdout_lines attribute and that we can extract attributes from a list of dicts with the jinja2 map filter, this could be replaced in one go without having to run a task at all with the following jinja expression:
"{{ disk_check.results | map(attribute='stdout_lines') | list }}"
But we would still be walking the wrong path.
Let's have a look at one of your (reconstructed) individual stdout from results
"stdout": "DEVNAME: /dev/sanstorage/node11-data102-2e385a327788d866e6c9ce9000b10399f\nUUID: 8c52d974-8584-4aa6-89b8-f1e1db016118\nTYPE: xfs"
This is a string representation of a yaml dict. Ansible has a from_yaml filter. And we can use the map filter to apply a filter to each element in a list.
The below playbook tries to reproduce your original data to display it in one go
---
- name: Parse a yaml dict in each element of a list
hosts: localhost
gather_facts: false
vars:
example_disks: 3
tasks:
- name: Faking your disk check supposed command
shell: |-
cat <<EOF
DEVNAME: /dev/sanstorage/node{{ (item | string)*2 }}-data{{ (item | string)*3 }}-$(uuidgen)
UUID: $(uuidgen)
TYPE: xfs
EOF
loop: "{{ range(1, example_disks+1) }}"
register: disk_check
- name: Show the original data (use -v to trigger)
debug:
var: disk_check
verbosity: 1
- name: Display a list of dicts from the above result
debug:
msg: "{{ disk_check.results | map(attribute='stdout') | map('from_yaml') | list }}"
and gives (run with -v to show the intermediate debug):
PLAY [Parse a yaml dict in each element of a list] *************************************************************************************************************************************************************************************
TASK [Faking your disk check supposed command] *****************************************************************************************************************************************************************************************
changed: [localhost] => (item=1)
changed: [localhost] => (item=2)
changed: [localhost] => (item=3)
TASK [Show the original data (use -v to trigger)] **********************************************************************************************************************************************************************************************************
skipping: [localhost]
TASK [Display a list of dicts from the above result] ***********************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"DEVNAME": "/dev/sanstorage/node11-data111-10e7a318-62af-46db-895e-5d94b0c2cf88",
"TYPE": "xfs",
"UUID": "5a87ae1c-c312-4325-88ac-b9cc1edfa69b"
},
{
"DEVNAME": "/dev/sanstorage/node22-data222-18247d1d-87b2-4c7d-8d48-cd333d7530f9",
"TYPE": "xfs",
"UUID": "bc0d7f1a-16e4-4694-b3e2-904e69cc208d"
},
{
"DEVNAME": "/dev/sanstorage/node33-data333-21bc7fde-645f-4cf2-9e18-72d004700085",
"TYPE": "xfs",
"UUID": "58985028-3eb1-4f34-90e9-f0e4c8257607"
}
]
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
Let's simplify the data, e.g.
first_list:
- - a: 1
- b: 1
- - a: 2
- b: 2
The expected result is
final_list:
- a: 1
b: 1
- a: 2
b: 2
The task below does the job
- set_fact:
final_list: "{{ final_list|default([]) + [_dict|from_yaml] }}"
loop: "{{ first_list }}"
vars:
_dict: |
{% for i in item %}
{{ (i|to_yaml)[1:-2] }}
{% endfor %}
There are other options on how to format the item, e.g.
{{ i.keys()|first }}: {{ i.values()|first }}
Write a filter if you use such conversions frequently, e.g.
shell> cat filter_plugins/list_filters.py
def list2dict(l):
out = []
for i in l:
item = {}
for j in range(0, len(i)):
item.update(i[j])
out.append(item)
return out
class FilterModule(object):
''' List filters. '''
def filters(self):
return {
'list2dict': list2dict,
}
Then the simple filter below gives the same result
- set_fact:
final_list: "{{ first_list|list2dict }}"

Looping over registered variable from multiple items per inventory host

I have the following sample inventory (trimmed for brevity):
all:
hosts:
foo1.example.com:
disks:
- disk: /dev/sdaa
- disk: /dev/sdab
- disk: /dev/sdac
- disk: /dev/sdad
foo2.example.com:
disks:
- disk: /dev/sdaa
- disk: /dev/sdab
- disk: /dev/sdac
- disk: /dev/sdad
I have a play where I loop over each disk and perform a shell command on each one and register the output. Basically stdout will be 'true' or 'false'.
- name: "Checking disk health"
shell: /sbin/smartctl -H -j {{ item.disk }} | jq '.smart_status.passed'
register: passed
with_items:
- "{{ hostvars[inventory_hostname].disks }}"
loop_control:
label: "/sbin/smartctl -H -j {{ item.disk }}"
For some reason I can't figure out how to loop over this in an assert statement. I know I've done something like this before years ago but can't find my notes or any examples.
For each host, I need to loop over each disk's output and assert that passed == true.
When I execute the following debug play:
- debug:
var: "{{ item }}"
with_items:
- passed
I get this output (trimmed for brevity):
ok: [foo1.example.com] => (item=passed) => {
"ansible_loop_var": "item",
"item": "passed",
"passed": {
"changed": true,
"msg": "All items completed",
"results": [
{
"ansible_loop_var": "item",
"changed": true,
"cmd": "/sbin/smartctl -H -j /dev/sdaa | jq '.smart_status.passed'",
"stdout": "true",
"stdout_lines": [
"true"
]
},
{
"ansible_loop_var": "item",
"changed": true,
"cmd": "/sbin/smartctl -H -j /dev/sdab | jq '.smart_status.passed'",
"stdout": "true",
"stdout_lines": [
"true"
]
},
{
"ansible_loop_var": "item",
"changed": true,
"cmd": "/sbin/smartctl -H -j /dev/sdac | jq '.smart_status.passed'",
"stdout": "true",
"stdout_lines": [
"true"
]
},
{
"ansible_loop_var": "item",
"changed": true,
"cmd": "/sbin/smartctl -H -j /dev/sdad | jq '.smart_status.passed'",
"stdout": "true",
"stdout_lines": [
"true"
]
}
]
}
}
Side preliminary notes:
{{ hostvars[inventory_hostname].disks }} can be shortened to {{ disks }}
{{ disks }} is a list so you should not pass it as a list of list to you loop (even though with_items is flattening transparently for you...)
Now in the following task (fixed as per my above remarks):
- name: "Checking disk health"
shell: /sbin/smartctl -H -j {{ item.disk }} | jq '.smart_status.passed'
register: passed
with_items: "{{ disks }}"
loop_control:
label: "/sbin/smartctl -H -j {{ item.disk }}"
you are registering a result from a looped task in the variable passed
As explained in the documentation on loops and in the documentation on variables, registers from loop are modified to include a top results element which is a list of all individual results in the loop.
If I understand well you want to test in these results all element have "true" in their stdout attribute. You don't need to loop for that you can check that in a single run.
What I would do is demonstrated in the example below: extract all stdout attributes from the results list, map each element with the bool filter to convert to a real boolean value, and use the all test to make sure all values are True.
- name: Assert all disks passed the smartctl test
assert:
that: (passed.results | map(attribute='stdout') | map('bool') | list) is all
msg: One or more disks failed the test

What is the best way to do case statement in Ansible?

I am trying to convert my scripts into Ansible for automation. I am stuck in understanding loops or the "with_items" use case.
The original bash script:
for i in apple banana orange; do
case $i in
apple) export var="var.1:apple1,var.2:apple2" ;;
banana) export var="var.1:banana1,var.2:banana2,var.3:banana3" ;;
orange) export var="var.1:orange1" ;;
esac
echo "$i"
What I have tried so far:
VARS file:
fruits:
- name: apple
var: "{{ item }}"
with_items:
- apple1
- apple2
- name: banana
var: "{{ item }}"
with_items:
- banana1
- banana2
- banana3
- name: orange
var: "{{ item }}"
with_items:
- orange1
TASKS file:
- include_vars: vars.yml
- debug:
msg: "{{ fruits }}"
- name: output in shell using echo
shell: |
echo "{{ fruits.name }}" ;
echo "{{ fruits.var }}"
loop: "{{ fruits }}"
Outputs:
The output from include_vars task:
{
"ansible_included_var_files": [
"/etc/ansible/roles/openssl/tasks/vars.yml"
],
"ansible_facts": {
"fruits": [
{
"var": "{{ item }}",
"name": "apple",
"with_items": [
"apple1",
"apple2"
]
},
{
"var": "{{ item }}",
"name": "banana",
"with_items": [
"banana1",
"banana2",
"banana3"
]
},
{
"var": "{{ item }}",
"name": "orange",
"with_items": [
"orange1"
]
}
]
},
"_ansible_no_log": false,
"changed": false
}
Debug
debug task failed
{
"msg": "The task includes an option with an undefined variable. The error was: 'item' is undefined\n\nThe error appears to be in '/etc/ansible/roles/openssl/tasks/main.yml': line 262, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- debug:\n ^ here\n",
"_ansible_no_log": false
}
I know that Yaml are space sensitive and formatting is important. I am a novice at writing playbooks and any pointers would be helpful.
With the dictionary
fruits:
apple:
- apple1
- apple2
banana:
- banana1
- banana2
- banana3
orange:
- orange1
the loop
- hosts: localhost
tasks:
- include_vars:
vars.yml
- debug:
msg: "{{ item.key }} {{ item.value }}"
loop: "{{ fruits|dict2items }}"
gives (abridged)
"msg": "orange [u'orange1']"
"msg": "apple [u'apple1', u'apple2']"
"msg": "banana [u'banana1', u'banana2', u'banana3']"
It's possible to reference the items in the dictionary. For example
- debug:
var: fruits.banana
- debug:
var: fruits.apple.1
gives
"fruits.banana": [
"banana1",
"banana2",
"banana3"
]
"fruits.apple.1": "apple2"

Ansible Nested Loops from JSON

I need to map objects in JSON in the following way. Note that objects can have bespokeVals or not and the keys are dynamic.
[{
"obj": "obj1",
"bespokeVals": [
{"key1": "val1"},
{"key2": "val2"}
]
},
{
"obj": "obj2",
"bespokeVals": [
{"key1": "val3"},
{"key2": "val4"},
{"key3": "val3"}
]
},
{
"obj": "obj3",
"bespokeVals": [
{"randomKey": "vdsk"}
]
}]
What I want to do, is print key=value when obj2 == something in Ansible
This is as close as I've got so far. This is working but printing out bespokeVals as a:
- name: "Print Bespoke Vals"
debug:
msg: "{{ item.bespokeVals }}"
loop: "{{ objectList }}"
when: item.obj == something
If it helps, I have full control over the JSON so can change the format if needed.
Is this probably what you're looking for? The play below
tasks:
- name: "Print Bespoke Vals"
debug:
msg: "{{ item|dict2items|map(attribute='key')|join() }} =
{{ item|dict2items|map(attribute='value')|join() }}"
loop: "{{ objectList|json_query('[?obj == `obj2`].bespokeVals')|flatten }}"
gives:
"msg": "key1 = val3"
"msg": "key2 = val4"
"msg": "key3 = val3"

Ansible register result of multiple commands

I was given a task to verify some routing entries for all Linux server and here is how I did it using an Ansible playbook
---
- hosts: Linux
serial: 1
tasks:
- name: Check first
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result
changed_when: false
- debug: msg="{{result.stdout}}"
- name: Check second
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result
changed_when: false
- debug: msg="{{result.stdout}}"
You can see I have to repeat same task for each routing entry and I believe I should be able to avoid this. I tried use with_items loop but got following error message
One or more undefined variables: 'dict object' has no attribute 'stdout'
is there a way to register variable for each command and loop over them one by one ?
Starting in Ansible 1.6.1, the results registered with multiple items are stored in result.results as an array. So you can use result.results[0].stdout and so on.
Testing playbook:
---
- hosts: localhost
gather_facts: no
tasks:
- command: "echo {{item}}"
register: result
with_items: [1, 2]
- debug:
var: result
Result:
$ ansible-playbook -i localhost, test.yml
PLAY [localhost] **************************************************************
TASK: [command echo {{item}}] *************************************************
changed: [localhost] => (item=1)
changed: [localhost] => (item=2)
TASK: [debug ] ****************************************************************
ok: [localhost] => {
"var": {
"result": {
"changed": true,
"msg": "All items completed",
"results": [
{
"changed": true,
"cmd": [
"echo",
"1"
],
"delta": "0:00:00.002502",
"end": "2015-08-07 16:44:08.901313",
"invocation": {
"module_args": "echo 1",
"module_name": "command"
},
"item": 1,
"rc": 0,
"start": "2015-08-07 16:44:08.898811",
"stderr": "",
"stdout": "1",
"stdout_lines": [
"1"
],
"warnings": []
},
{
"changed": true,
"cmd": [
"echo",
"2"
],
"delta": "0:00:00.002516",
"end": "2015-08-07 16:44:09.038458",
"invocation": {
"module_args": "echo 2",
"module_name": "command"
},
"item": 2,
"rc": 0,
"start": "2015-08-07 16:44:09.035942",
"stderr": "",
"stdout": "2",
"stdout_lines": [
"2"
],
"warnings": []
}
]
}
}
}
PLAY RECAP ********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0
A slightly different situation, which took a while to figure out. If you want to use the results of multiple items, but for changed_when, then the register variable will not have a var.results! Instead, changed_when, is evaluated for each item, and you can just directly use the register var.
Simple example, which will result in changed: false:
- action: command echo {{item}}
register: out
changed_when: "'z' in out.stdout"
with_items:
- hello
- foo
- bye
Another example:
- name: Create fulltext index for faster text searches.
mysql_db: name={{SO_database}} state=import target=/tmp/fulltext-{{item.tableName}}-{{item.columnName}}.sql
with_items:
- {tableName: Posts, columnName: Title}
- {tableName: Posts, columnName: Body}
- {tableName: Posts, columnName: Tags}
- {tableName: Comments, columnName: Text}
register: createfulltextcmd
changed_when: createindexcmd.msg.find('already exists') == -1
Finally, when you do want to loop through results in other contexts, it does seem a bit tricky to programmatically access the index as that is not exposed. I did find this one example that might be promising:
- name: add hosts to known_hosts
shell: 'ssh-keyscan -H {{item.host}}>> /home/testuser/known_hosts'
with_items:
- { index: 0, host: testhost1.test.dom }
- { index: 1, host: testhost2.test.dom }
- { index: 2, host: 192.168.202.100 }
when: ssh_known_hosts.results[{{item.index}}].rc == 1
Posting because I can't comment yet
Relating to gameweld's answer, since Ansible 2.5 there's another way to accessing the iteration index.
From the docs:
Tracking progress through a loop with index_var
New in version 2.5.
To keep track of where you are in a loop, use the index_var directive
with loop_control. This directive specifies a variable name to contain
the current loop index:
- name: count our fruit
debug:
msg: "{{ item }} with index {{ my_idx }}"
loop:
- apple
- banana
- pear
loop_control:
index_var: my_idx
This also allows you to gather results from an array and act later to the same array, taking into account the previous results
- name: Ensure directories exist
file:
path: "{{ item }}"
state: directory
loop:
- "mouse"
- "lizard"
register: reg
- name: Do something only if directory is new
debug:
msg: "New dir created with name '{{ item }}'"
loop:
- "mouse"
- "lizard"
loop_control:
index_var: index
when: reg.results[index].changed
Please note that the "mouse lizard" array should be exactly the same
If what you need is to register the output of two commands separately, use different variable names.
---
- hosts: Linux
serial: 1
tasks:
- name: Check first
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result0
changed_when: false
- debug: msg="{{result0.stdout}}"
- name: Check second
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result1
changed_when: false
- debug: msg="{{result1.stdout}}"

Resources