I'm using the archive module to zip some folders and archives.
I need to list all first level folders inside my zip file.
Is there any Jinja or python function to do this using the registered output of the archive module?
My test, so far are:
- name: zip folder
archive:
path:
- /tmp/test/*
dest: /tmp/zipfile.zip
format: zip
mode: "0755"
force_archive: true
register: module_result
Printing module_result, I get:
ok: [host1] =>
msg:
ansible_facts:
discovered_interpreter_python: /usr/bin/python
archived:
- /tmp/test/cfg/1.txt
- /tmp/test/cfg/2.txt
- /tmp/test/cfg/3.txt
- /tmp/test/folder1/folder2/4.txt
- /tmp/test/folder1/folder2/5.txt
Using another task to show my attempts:
- name: list folders
debug:
msg: "{{ item.lstrip(module_result.arcroot).split('/')[0] }}"
# msg: "{{ item }}"
with_items: "{{module_result.archived}}"
I'm trying to cut the first part of the url, using module_result.arcroot on every item that module outputs me and then, a split of the rest of the path, but I get a wrong result. And the worst part is that I'm getting too many times the same folder name.
And the result:
cfg
cfg
cfg
folder1
folder1
My expected result would be:
cfg
folder1
Is this possible?
If your goal is to achieve an unicity on the folders of first level, then the unique Jinja filter is what you are looking for.
Since this filter acts on lists, you would then need to, first, register your results as a list, via the module set_fact.
- name: Making a list out of the files first folder
set_fact:
folders: "{{ folders | default([]) + [(item.lstrip(faked_module_result_arcroot).split('/')[0])] }}"
with_items: "{{ faked_module_result_archived }}"
Then just use the Jinja filter:
- debug:
msg: "{{ folders | unique }}"
Fully working playbook:
---
- hosts: localhost
connection: local
vars:
faked_module_result_archived:
- /tmp/test/cfg/1.txt
- /tmp/test/cfg/2.txt
- /tmp/test/cfg/3.txt
- /tmp/test/folder1/folder2/4.txt
- /tmp/test/folder1/folder2/5.txt
faked_module_result_arcroot: /tmp/test/
tasks:
- name: Making an list out of the files first folder
set_fact:
folders: "{{ folders | default([]) + [(item.lstrip(faked_module_result_arcroot).split('/')[0])] }}"
with_items: "{{ faked_module_result_archived }}"
- debug:
msg: "{{ folders | unique }}"
Would output
PLAY [localhost] ************************************************************************************************************************************************************
TASK [Gathering Facts] ******************************************************************************************************************************************************
ok: [localhost]
TASK [Making an list out of the files first folder] ************************************************************************************************************************
ok: [localhost] => (item=/tmp/test/cfg/1.txt)
ok: [localhost] => (item=/tmp/test/cfg/2.txt)
ok: [localhost] => (item=/tmp/test/cfg/3.txt)
ok: [localhost] => (item=/tmp/test/folder1/folder2/4.txt)
ok: [localhost] => (item=/tmp/test/folder1/folder2/5.txt)
TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"cfg",
"folder1"
]
}
PLAY RECAP ******************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Your example works great! I could get some tips and tricks to customize mi script, but for some reason, I'm am getting a strange lstrip behavior when I run the adaptation.
Here is my code:
- name: zip a folder
archive:
path:
- /tmp/test/*
dest: /tmp/{{ path_example | basename }}-files.zip
exclude_path:
- /tmp/test/another_folder
format: zip
mode: "0755"
force_archive: true
register: module_result
- name: First, lstrip result
set_fact:
lstrip_result: "{{ lstrip_result | default([]) [(item.lstrip(module_result.arcroot))] }}"
with_items: "{{ module_result.archived }}"
After run "Archive" task:
ok: [desa01] => changed=false
ansible_facts:
discovered_interpreter_python: /usr/bin/python
archived:
- /tmp/test/cfg/1.txt
- /tmp/test/cfg/2.txt
- /tmp/test/cfg/3.txt
- /tmp/test/folder1/folder2/4.txt
- /tmp/test/folder1/folder2/5.txt
arcroot: /tmp/test/
dest: /tmp/lelo/test-files.zip
expanded_exclude_paths:
- /tmp/test/another_folder
After run "First, lstrip result" task:
TASK [task2: First remove arcroot using lstrip ] ********************************************************************************************************
ok: [desa01] => (item=/tmp/test/cfg/1.txt) => changed=false
ansible_facts:
lstrip_result:
- cfg/1.txt
ansible_loop_var: item
item: /tmp/test/cfg/1.txt
ok: [desa01] => (item=/tmp/test/cfg/2.txt) => changed=false
ansible_facts:
lstrip_result:
- cfg/1.txt
- cfg/2.txt
ansible_loop_var: item
item: /tmp/test/cfg/2.txt
ok: [desa01] => (item=/tmp/test/cfg/3.txt) => changed=false
ansible_facts:
lstrip_result:
- cfg/1.txt
- cfg/2.txt
- cfg/3.txt
ansible_loop_var: item
item: /tmp/test/cfg/3.txt
ok: [desa01] => (item=/tmp/test/folder1/folder2/4.txt) => changed=false
ansible_facts:
lstrip_result:
- cfg/1.txt
- cfg/2.txt
- cfg/3.txt
- st/folder1/folder2/4.txt
ansible_loop_var: item
item: /tmp/test/folder1/folder2/4.txt
ok: [desa01] => (item=/tmp/test/folder1/folder2/5.txt) => changed=false
ansible_facts:
lstrip_result:
- cfg/1.txt
- cfg/2.txt
- cfg/3.txt
- st/folder1/folder2/4.txt
- st/folder1/folder2/4.txt
ansible_loop_var: item
item: /tmp/test/folder1/folder2/5.txt
Here is my problem, with folder cfg everything it is ok, but with folder test, lstrip function is like striping 2 chars more than expected and the result is:
st/folder...
instead of:
test/folder...
Someone gets an idea of this strange lstrip out??
EDIT
lstrip doc says:
Definition and Usage
The lstrip() method removes any leading characters (space is the default leading character to remove)
That's why my output was bad striped, becouse some chars where repited in the string.
I replace lstrip with replace
- name: First, lstrip result
set_fact:
lstrip_result: "{{ lstrip_result | default([]) [(item.lstrip(module_result.arcroot))] }}"
with_items: "{{ module_result.archived }}"
Related
Finally, I found a solution on how to create nested directory structure which is an equivalent of the Bash command
mkdir -p jenkins/cache/{war,tmp,workspace}
My playbook
---
- name: Create directories
hosts: localhost
vars:
dirs: [
jenkins, # for one elemnt it can be string
[ cache ],
[ war, tmp, workspace ]
]
base_directory: ~/tmp
tasks:
- debug:
msg="{{base_directory}}/{{item.0}}/{{item.1}}/{{item.2}}"
with_nested: "{{ dirs | list }}"
It generates:
PLAY [Create directories] **********************************************************************************************************************************
TASK [debug] ***********************************************************************************************************************************************
ok: [localhost] => (item=['jenkins', 'cache', 'war']) => {
"msg": "~/tmp/jenkins/cache/war"
}
ok: [localhost] => (item=['jenkins', 'cache', 'tmp']) => {
"msg": "~/tmp/jenkins/cache/tmp"
}
ok: [localhost] => (item=['jenkins', 'cache', 'workspace']) => {
"msg": "~/tmp/jenkins/cache/workspace"
}
PLAY RECAP *************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
But there is an issue if I want to use more (or less) levels of nesting. I need more (or less) item.X.
How to do it dynamically in Ansible?
I know there is community module filetree, which can be helpfully but I don't want or need to use it.
You can definitely use a with_nested and join the second level list with a /.
So, a task like this would do:
- debug:
msg: >-
{{ base_directory }}/
{{- item | join('/') }}
with_nested: "{{ dirs }}"
Given the task:
- debug:
msg: >-
{{ base_directory }}/
{{- item | join('/') }}
with_nested: "{{ dirs }}"
vars:
dirs:
- jenkins
- - cache
- - war
- tmp
- workspace
base_directory: ~/tmp
This yields:
ok: [localhost] => (item=['jenkins', 'cache', 'war']) =>
msg: ~/tmp/jenkins/cache/war
ok: [localhost] => (item=['jenkins', 'cache', 'tmp']) =>
msg: ~/tmp/jenkins/cache/tmp
ok: [localhost] => (item=['jenkins', 'cache', 'workspace']) =>
msg: ~/tmp/jenkins/cache/workspace
The filter community.general.path_join takes the items of a list and creates a path. For example, the simplified task below
- debug:
msg: "{{ base_directory }}/{{ item|community.general.path_join }}"
with_nested: "{{ dirs }}"
vars:
dirs:
- jenkins
- cache
- [war, tmp, workspace]
base_directory: /tmp
gives
msg: /tmp/jenkins/cache/war
msg: /tmp/jenkins/cache/tmp
msg: /tmp/jenkins/cache/workspace
I'm having trouble grasping data manipulation in Ansible. Here's the setup:
- hosts: emirr01
gather_facts: no
vars:
setup:
- emirr01: { label: "label1" }
- emirr02: { label: "label2" }
- emirr03: { label: "label3" }
lookup: [ "emirr01", "emirr02"]
use_labels: [ ]
tasks:
- debug: msg="setup={{ setup }}"
- debug: msg="lookup={{ lookup }}"
- debug: msg="item0={{ item.0 }} item1={{ item.1 }}"
when: inventory_hostname == item.1
with_nested:
- "{{ setup }}"
- "{{ lookup }}"
- set_fact:
use_labels: "{{ use_labels + [ item.1.label ] }}"
when: item.0 == item.1.label
with_nested:
- "{{ setup }}"
- "{{ lookup }}"
- debug: msg="use_labels={{ use_labels }}"
What I need is to set a fact use_labels which is a list of labels as defined in setup list for each host found in lookup list. What I expect is a list like this:
[ "label1", "label2" ]
My problem is not being able to "reach" label in a list I get in item.1 which is (example):
item1=[{'emirr02': {'label': 'label2'}}
Here's is the error:
$ ansible-playbook test.yml -l emirr01
PLAY [emirr01] ****************************************************************************************************************************************************************************************************************************************************************************************************************************
TASK [debug] ******************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [emirr01] =>
msg: 'setup=[{''emirr01'': {''label'': ''label1''}}, {''emirr02'': {''label'': ''label2''}}, {''emirr03'': {''label'': ''label3''}}]'
TASK [debug] ******************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [emirr01] =>
msg: lookup=['emirr01', 'emirr02']
TASK [debug] ******************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [emirr01] => (item=[{'emirr01': {'label': 'label1'}}, 'emirr01']) =>
msg: 'item0={''emirr01'': {''label'': ''label1''}} item1=emirr01'
skipping: [emirr01] => (item=[{'emirr01': {'label': 'label1'}}, 'emirr02'])
ok: [emirr01] => (item=[{'emirr02': {'label': 'label2'}}, 'emirr01']) =>
msg: 'item0={''emirr02'': {''label'': ''label2''}} item1=emirr01'
skipping: [emirr01] => (item=[{'emirr02': {'label': 'label2'}}, 'emirr02'])
ok: [emirr01] => (item=[{'emirr03': {'label': 'label3'}}, 'emirr01']) =>
msg: 'item0={''emirr03'': {''label'': ''label3''}} item1=emirr01'
skipping: [emirr01] => (item=[{'emirr03': {'label': 'label3'}}, 'emirr02'])
TASK [set_fact] ***************************************************************************************************************************************************************************************************************************************************************************************************************************
fatal: [emirr01]: FAILED! =>
msg: |-
The conditional check 'item.0 == item.1.label' failed. The error was: error while evaluating conditional (item.0 == item.1.label): 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'label'
The error appears to be in '/home/ansible/test.yml': line 21, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- set_fact:
^ here
It all boils down to "digging" for a value from a dictionary in a list. Does anyone have an idea of how to reach item.1.label? It's a little bit frustrating since this is trivial in Python or other programming languages but in Ansible... Or maybe my mistake is to use a wrong kind of with_* loop for this purpose?
Q: "Expect a list like this [label1, label2]"
setup:
- emirr01: { label: "label1" }
- emirr02: { label: "label2" }
- emirr03: { label: "label3" }
lookup: [ "emirr01", "emirr02"]
A: This is a typical example of a wrong data structure for this use-case. A list of dictionaries can't be used in mapping the filter extract. Moreover, the keys in the dictionaries don't have to be unique, because they are hidden in the items of the list. The solution is simple with the data in the dictionary
setup:
emirr01: { label: "label1" }
emirr02: { label: "label2" }
emirr03: { label: "label3" }
- set_fact:
use_labels: "{{ lookup|map('extract', setup, 'label')|list }}"
gives
use_labels:
- label1
- label2
One of the options is creating a dictionary from the list first. For example
- set_fact:
setup2: "{{ setup2|default({})|combine(item) }}"
loop: "{{ setup }}"
Then the dictionary setup2 can be used to get the same result
It's possible to filter the list directly. For example, the task below gives the same result too
- set_fact:
use_labels: "{{ setup|map('dict2items')|map('first')|
selectattr('key', 'in', lookup )|
map(attribute='value.label')|
list }}"
Your issue with the with_nested is actually coming from how those "two-lists" loops are working.
When you are using loops like with_nested, the item.0 is the item of the first list, while the item.1 is the item of the second list.
Because lookup is a list and not a list of dictionaries like
lookup:
- label: "emirr01"
- label: "emirr02"
You can use the value of item.1 right away.
So the way to have you issue fixed is to use it this way:
- set_fact:
use_labels: "{{ use_labels + [ item.0[item.1].label ] }}"
when: item.0[item.1].label is defined
with_nested:
- "{{ setup }}"
- "{{ lookup }}"
Here is a playbook demonstrating this:
- hosts: localhost
gather_facts: no
tasks:
- set_fact:
use_labels: "{{ use_labels | default([]) + [ item.0[item.1].label ] }}"
when: item.0[item.1].label is defined
with_nested:
- "{{ setup }}"
- "{{ lookup }}"
vars:
setup:
- emirr01:
label: "label1"
- emirr02:
label: "label2"
- emirr03:
label: "label3"
lookup:
- "emirr01"
- "emirr02"
- debug:
msg: "{{ use_labels }}"
Wich gives:
PLAY [localhost] ***************************************************************************************************
TASK [set_fact] ****************************************************************************************************
ok: [localhost] => (item=[{'emirr01': {'label': 'label1'}}, 'emirr01'])
skipping: [localhost] => (item=[{'emirr01': {'label': 'label1'}}, 'emirr02'])
skipping: [localhost] => (item=[{'emirr02': {'label': 'label2'}}, 'emirr01'])
ok: [localhost] => (item=[{'emirr02': {'label': 'label2'}}, 'emirr02'])
skipping: [localhost] => (item=[{'emirr03': {'label': 'label3'}}, 'emirr01'])
skipping: [localhost] => (item=[{'emirr03': {'label': 'label3'}}, 'emirr02'])
TASK [debug] *******************************************************************************************************
ok: [localhost] => {
"msg": [
"label1",
"label2"
]
}
PLAY RECAP *********************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
how can I add a condition to ansible task that is based on a loop, when the task itself also based on a loop?
For example, here's my code:
- hosts: all
gather_facts: False
vars:
current_version: 826
versions:
- 805
- 821
- 824
- 826
tasks:
- name: First Task
find:
paths: /Users/tomer/projects/personal/ansible/test
patterns: snapshot*
register: files
when:
- current_version == item
loop: "{{versions}}"
- name: Second task
set_fact:
test_work: "{{ true if item > 0 else false}}"
loop:
- "{{ files | json_query('results[*].matched') }}"
So far, this is working as expected.
The first task is looking for any file with the name snapshot if the current_version is matching one of the versions in the list.
The second task iterates over the dictionary result from the first task and based on each item it is setting the fact. (In my case, only one item has this attribute).
I wanted to run the second task, only when the first task did run, however, the changed status is always false, so this condition is not useful.
I wanted to add the same condition of current_version == item but I can't use item twice here.
Any idea how to achieve that?
The find command is not really going to change anything, it just queries the file system without doing any modification, so it will indeed always give you a false.
On the other hand, you can definitely use the skipped field of the item.
This said, I would simplify the loop on your set_fact, because there is no real need to use json_query here.
This task would do the job perfectly fine:
- set_fact:
test_work: "{{ item.matched > 0 }}"
loop: "{{ files.results }}"
when: item is not skipped
Another extra tip is to not do things like
true if condition_that_evaluates_to_true else false
But rather do right away
condition_that_evaluates_to_true
Here would be a made up example playbook
- hosts: all
gather_facts: no
tasks:
- find:
path: /tmp
pattern: dummy*
when: item == current_version
register: files
loop: "{{ versions }}"
vars:
current_version: 826
versions:
- 805
- 821
- 824
- 826
- debug:
msg: "{{ item.matched > 0 }}"
loop: "{{ files.results }}"
when: item is not skipped
loop_control:
label: "{{ item.item }}"
This would yield the result
PLAY [all] *******************************************************************************************************
TASK [find] ******************************************************************************************************
skipping: [localhost] => (item=805)
skipping: [localhost] => (item=821)
skipping: [localhost] => (item=824)
ok: [localhost] => (item=826)
TASK [debug] *****************************************************************************************************
skipping: [localhost] => (item=805)
skipping: [localhost] => (item=821)
skipping: [localhost] => (item=824)
ok: [localhost] => (item=826) => {
"msg": true
}
PLAY RECAP *******************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
I have some variables defined like this:
x_php_versions_installed:
php70:
- php70-curl
- php70-xml
- php70-xmlrpc
- php70-zip
- pecl-memcached
php71:
- php71-curl
- php71-xml
- php71-xmlrpc
- php71-zip
php72:
- php72-curl
- php72-xml
- php72-xmlrpc
- php72-zip
- pecl-memcached
And I would like to check all of the vars (php70, php71, php72 and so on) has this variable: pecl-memcached and if one has, then run a command. My playbook looks like this:
- name: memcached pecl install
pear:
executable: '/usr/local/{{ item }}/bin/pecl'
name: 'pecl/memcached'
state: 'latest'
with_items: '{{ x_php_versions_installed | list }}'
when: 'item.pecl-memcached is defined'
this should call the /usr/local/php70/bin/pecl and /usr/local/php72/bin/pecl binary to install memcached. As soon as I remove the when condition, it works very well, but it will call every variable inside x_php_versions_installed not only where pecl-memcached is defined. So I need to fix the when condition in this case, but all of my tries are gives me an error.
If you want your when to check a list properly, you'll have to use the test operator in of Jinja:
- name: memcached pecl install
pear:
executable: '/usr/local/{{ item }}/bin/pecl'
name: 'pecl/memcached'
state: 'latest'
with_items: '{{ x_php_versions_installed | list }}'
when: "'pecl-memcached' in x_php_versions_installed[item]"
Given the playbook:
- hosts: localhost
gather_facts: no
vars:
x_php_versions_installed:
php70:
- php70-curl
- php70-xml
- php70-xmlrpc
- php70-zip
- pecl-memcached
php71:
- php71-curl
- php71-xml
- php71-xmlrpc
- php71-zip
php72:
- php72-curl
- php72-xml
- php72-xmlrpc
- php72-zip
- pecl-memcached
tasks:
- debug:
msg: "{{ item }}"
with_items: "{{ x_php_versions_installed | list }}"
when: "'pecl-memcached' in x_php_versions_installed[item]"
The recap would be:
TASK [debug] *******************************************************************
ok: [localhost] => (item=php70) => {
"msg": "php70"
}
skipping: [localhost] => (item=php71)
ok: [localhost] => (item=php72) => {
"msg": "php72"
}
Your subelement is a list not a dictionary, so accessing an element key like your are trying here, i.e.:
when: "x_php_versions_installed[item]['pecl-memcached'] is defined"
would work on a dictionary like this one:
x_php_versions_installed:
php70:
php70-curl:
php70-xml:
php70-xmlrpc:
php70-zip:
pecl-memcached:
# same goes for the other versions of PHP
Given the playbook:
- hosts: localhost
gather_facts: no
vars:
x_php_versions_installed:
php70:
php70-curl:
php70-xml:
php70-xmlrpc:
php70-zip:
pecl-memcached:
php71:
php71-curl:
php71-xml:
php71-xmlrpc:
php71-zip:
php72:
php72-curl:
php72-xml:
php72-xmlrpc:
php72-zip:
pecl-memcached:
tasks:
- debug:
msg: "{{ item }}"
with_items: "{{ x_php_versions_installed | list }}"
when: "x_php_versions_installed[item]['pecl-memcached'] is defined"
The recap would be:
TASK [debug] *******************************************************************
ok: [localhost] => (item=php70) => {
"msg": "php70"
}
skipping: [localhost] => (item=php71)
ok: [localhost] => (item=php72) => {
"msg": "php72"
}
I have a list of devices, each of which has a varying list of attributes that must be created on the devices, one at a time. Is there a way to build a nested set of loops to do this? The with_nested construct applies every attribute to every device; I need only a single attribute per device per call.
This playbook demonstrates what I have tried (Ansible 2.7.1, Python 2.7.1):
- name: Test nested list traversal
hosts: localhost
connection: local
vars:
Stuff:
- Name: DeviceA
Info: AInfo
Values:
- ValueA1
- ValueA2
- Name: DeviceB
Info: BInfo
Values:
- ValueB1
- ValueB2
- ValueB3
tasks:
- name: Loop test
debug:
msg: "{{ item.Name }},{{ item.Info }}, {{ item.Values }}"
with_items:
- "{{ Stuff }}"
What I get: (One call per device, containing all attributes at once)
ok: [localhost] => (item={u'Info': u'AInfo', u'Values': [u'ValueA1', u'ValueA2'], u'Name': u'DeviceA'}) =>
msg: DeviceA,AInfo, [u'ValueA1', u'ValueA2']
ok: [localhost] => (item={u'Info': u'BInfo', u'Values': [u'ValueB1', u'ValueB2', u'ValueB3'], u'Name': u'DeviceB'}) =>
msg: DeviceB,BInfo, [u'ValueB1', u'ValueB2', u'ValueB3']
What I want (each msg represents a separate operation to be performed on the device with just that one attribute)
msg: DeviceA, AInfo, ValueA1
msg: DeviceA, AInfo, ValueA2
msg: DeviceB, BInfo, ValueB1
msg: DeviceB, BInfo, ValueB2
msg: DeviceB, BInfo, ValueB3
You can get what you want using the subelements filter:
---
- hosts: localhost
gather_facts: false
vars:
Stuff:
- Name: DeviceA
Info: AInfo
Values:
- ValueA1
- ValueA2
- Name: DeviceB
Info: BInfo
Values:
- ValueB1
- ValueB2
- ValueB3
tasks:
- debug:
msg: "{{ item.0.Name }}, {{ item.0.Info }}, {{ item.1 }}"
loop: "{{ Stuff|subelements('Values') }}"
loop_control:
label: "{{ item.0.Name }}"
Running the above playbook gets you:
PLAY [localhost] ******************************************************************************************************************************************************************************
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => (item=DeviceA) => {
"msg": "DeviceA, AInfo, ValueA1"
}
ok: [localhost] => (item=DeviceA) => {
"msg": "DeviceA, AInfo, ValueA2"
}
ok: [localhost] => (item=DeviceB) => {
"msg": "DeviceB, BInfo, ValueB1"
}
ok: [localhost] => (item=DeviceB) => {
"msg": "DeviceB, BInfo, ValueB2"
}
ok: [localhost] => (item=DeviceB) => {
"msg": "DeviceB, BInfo, ValueB3"
}
PLAY RECAP ************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0