Ansible - Looking for files and compare their hash - ansible

Practice:
I have the file files.yml with a list of files and their respective md5_sum hash, like:
files:
- name: /opt/file_compare1.tar
hash: 9cd599a3523898e6a12e13ec787da50a /opt/file_compare1.tar
- name: /opt/file_compare2tar.gz
hash: d41d8cd98f00b204e9800998ecf8427e /opt/file_compare2.tar.gz
I need to create a playbook to check this list of files if the current hash is the same or if it was changed, the playbook should have a debug message like below:
---
- hosts: localhost
connection: local
vars_files:
- files.yml
tasks:
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
with_items:
- "{{ files }}"
- name: Debug files - Different
debug:
msg: |
"Hash changed: {{ item.name }}"
when:
- item.hash != hash_check
with_items:
- "{{ files }}"
- name: Debug files - Equal
debug:
msg: |
"Hash NOT changed: {{ item.name }}"
when:
- item.hash == hash_check
with_items:
- "{{ files }}"
- debug:
msg: |
- "{{ hash_check }} {{ item.name }}"
with_items:
- "{{ files }}"

For example, given the files
files:
- name: /scratch/file_compare1.tar
hash: 4f8805b4b64dcc575547ec1c63793aec /scratch/file_compare1.tar
- name: /scratch/file_compare2.tar.gz
hash: 2dc4f1e9ca4081cc49d25195627982ef /scratch/file_compare2.tar.gz
the tasks below
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- name: Debug files - Different
debug:
msg: |
Hash NOT changed: {{ item.0.name }}
{{ item.0.hash.split()|first }}
{{ item.1 }}
with_together:
- "{{ files }}"
- "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
when: item.0.hash.split()|first == item.1
give
msg: |-
Hash NOT changed: /scratch/file_compare1.tar
4f8805b4b64dcc575547ec1c63793aec
4f8805b4b64dcc575547ec1c63793aec
msg: |-
Hash NOT changed: /scratch/file_compare2.tar.gz
2dc4f1e9ca4081cc49d25195627982ef
2dc4f1e9ca4081cc49d25195627982ef
A more robust option would be to create a dictionary with the calculated hashes
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- set_fact:
path_hash: "{{ dict(_path|zip(_hash)) }}"
vars:
_path: "{{ hash_check.results|map(attribute='stat.path')|list }}"
_hash: "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
gives
path_hash:
/scratch/file_compare1.tar: 4f8805b4b64dcc575547ec1c63793aec
/scratch/file_compare2.tar.gz: 2dc4f1e9ca4081cc49d25195627982ef
Then use this dictionary to compare the hashes. For example, the task below gives the same results
- name: Debug files - Different
debug:
msg: |
Hash NOT changed: {{ item.name }}
{{ item.hash.split()|first }}
{{ path_hash[item.name] }}
loop: "{{ files }}"
when: item.hash.split()|first == path_hash[item.name]
The next option is to create a dictionary with the original hashes and both lists of original and calculated hashes
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- set_fact:
hash_name: "{{ dict(_hash|zip(_name)) }}"
hash_orig: "{{ _hash }}"
hash_stat: "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
vars:
_hash: "{{ files|map(attribute='hash')|map('split')|map('first')|list }}"
_name: "{{ files|map(attribute='name')|list }}"
gives
hash_name:
2dc4f1e9ca4081cc49d25195627982ef: /scratch/file_compare2.tar.gz
4f8805b4b64dcc575547ec1c63793aec: /scratch/file_compare1.tar
hash_orig:
- 4f8805b4b64dcc575547ec1c63793aec
- 2dc4f1e9ca4081cc49d25195627982ef
hash_stat:
- 4f8805b4b64dcc575547ec1c63793aec
- 2dc4f1e9ca4081cc49d25195627982ef
Then calculate the difference of the lists and use it to extract both lists of changed and unchanged files
- set_fact:
files_diff: "{{ _diff|map('extract', hash_name)|list }}"
files_orig: "{{ _orig|map('extract', hash_name)|list }}"
vars:
_diff: "{{ hash_orig|difference(hash_stat) }}"
_orig: "{{ hash_orig|difference(_diff) }}"
- name: Debug files changed
debug:
var: files_diff
- name: Debug files NOT changed
debug:
var: files_orig
gives
files_diff: []
files_orig:
- /scratch/file_compare1.tar
- /scratch/file_compare2.tar.gz

I used your suggestion to complement the playbook, it's working now.
The idea is to get a list of files, read each one and compare with both hash, file, and current hash.
---
- hosts: localhost
connection: local
gather_facts: false
vars_files:
- files3.yml
tasks:
- stat:
path: "{{ item.file }}"
checksum_algorithm: md5
loop: "{{ files }}"
register: stat_results
- name: NOT changed files
debug:
msg: "NOT changed: {{ item.stat.path }}"
when: item.stat.checksum == item.item.checksum.split()|first
loop: "{{ stat_results.results }}"
loop_control:
label: "{{ item.stat.path }}"
- name: Changed files
debug:
msg: "CHANGED: {{ item.stat.path }}"
when: item.stat.checksum != item.item.checksum.split()|first
loop: "{{ stat_results.results }}"
loop_control:
label: "{{ item.stat.path }}"
Result:
>> ansible-playbook playbooks/check-file3.yml
PLAY [localhost] ********************************************************************************************************************************************************************************************************************
TASK [stat] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'file': '/opt/file_compare1.tar', 'checksum': '9cd599a3523898e6a12e13ec787da50a /opt/file_compare1.tar'})
ok: [localhost] => (item={'file': '/opt/file_compare2.tar.gz', 'checksum': 'd41d8cd98f00b204e9800998ecf8427e /opt/file_compare2.tar.gz'})
TASK [NOT changed files] ************************************************************************************************************************************************************************************************************
skipping: [localhost] => (item=/opt/file_compare1.tar)
ok: [localhost] => (item=/opt/file_compare2.tar.gz) => {
"msg": "NOT changed: /opt/file_compare2.tar.gz"
}
TASK [Changed files] ****************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=/opt/file_compare1.tar) => {
"msg": "CHANGED: /opt/file_compare1.tar"
}
skipping: [localhost] => (item=/opt/file_compare2.tar.gz)
PLAY RECAP **************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Related

Ansible Executing Nested Variable Loops on Multiple Files

I want to perform the following command using an Ansible playbook:
sed -i.bak 's/^mirror/#mirror/;s/#baseurl/baseurl/;s/mirror.centos/vault.centos/' /etc/yum.repos.d/CentOS-*.repo
A basic working Ansible playbook looks like this:
---
- name: Resolve any CentOS issues
hosts: "{{ nodes }}"
tasks:
- shell: sed -i.bak 's/^mirror/#mirror/;s/#baseurl/baseurl/;s/mirror.centos/vault.centos/' /opt/yum.repos.d/CentOS-*.repo
However, I would like to create a more advanced playbook, which *might *look something like this:
---
- name: Modify CentOS Reps
hosts: localhost
vars:
filelist: []
reg:
- ^mirrorlist
- ^#baseurl
- mirror.centos
rep:
- #mirror
- baseurl
- vault.centos
tasks:
- name: Find matching repo files and register to a variable
find:
paths: /etc/yum.repos.d
file_type: file
recurse: no
patterns: CentOS-*.repo
register: output
- name: Adding filelist to the LIST
no_log: true
set_fact:
filelist: "{{ filelist + [item.path]}}"
with_items: "{{ output.files }}"
- debug: var=filelist
- name: Change the lines
replace:
path: "{{ item.0 }}"
regexp: "{{ item.1.regexp }}"
replace: "{{ item.1.replace }}"
backup: true
with_nested:
- "{{ filelist }}"
- '{ regexp: "{{ reg }}", replace: "{{ rep }}" }'
The above gives the following attribute error:
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'regexp'\n\nThe error appears to be in '/etc/ansible/playbooks/centos-correct-repo301.yml': line 33, column 6, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Change the lines\n ^ here\n"}
Any thoughts?
Thanks
There are more options. For example,
Put the regex/replace logic into a dictionary
reg_rep:
- {reg: '^mirror', rep: '#mirror'}
- {reg: '^#baseurl', rep: 'baseurl'}
- {reg: 'mirror.centos', rep: 'vault.centos'}
Declare the list of the files
filelist: "{{ output.files|map(attribute='path')|list }}"
and find the files (no changes to your code in this task)
- name: Find matching repo files and register to a variable
find:
paths: /etc/yum.repos.d
file_type: file
recurse: no
patterns: CentOS-*.repo
register: output
Update the configuration
- name: Change the lines
replace:
path: "{{ item.0 }}"
regexp: "{{ item.1.reg }}"
replace: "{{ item.1.rep }}"
backup: true
with_nested:
- "{{ filelist }}"
- "{{ reg_rep }}"
loop_control:
label: "{{ item.0|basename }}"
Example of a complete playbook for testing
shell> cat pb.yml
- hosts: test_24
vars:
reg_rep:
- {reg: '^mirror', rep: '#mirror'}
- {reg: '^#baseurl', rep: 'baseurl'}
- {reg: 'mirror.centos', rep: 'vault.centos'}
filelist: "{{ output.files|map(attribute='path')|list }}"
tasks:
- name: Find matching repo files and register to a variable
find:
paths: /etc/yum.repos.d
file_type: file
recurse: no
patterns: CentOS-*.repo
register: output
- debug:
var: filelist
- debug:
msg: |
path: "{{ item.0 }}"
regexp: "{{ item.1.reg }}"
replace: "{{ item.1.rep }}"
backup: true
with_nested:
- "{{ filelist }}"
- "{{ reg_rep }}"
loop_control:
label: "{{ item.0|basename }}"
when: debug|d(false)|bool
- name: Change the lines
replace:
path: "{{ item.0 }}"
regexp: "{{ item.1.reg }}"
replace: "{{ item.1.rep }}"
backup: true
with_nested:
- "{{ filelist }}"
- "{{ reg_rep }}"
loop_control:
label: "{{ item.0|basename }}"
gives abridged, running in --check --diff mode
shell> ansible-playbook pb.yml -CD
PLAY [test_24] *******************************************************************************
TASK [Find matching repo files and register to a variable] ***********************************
ok: [test_24]
TASK [debug] *********************************************************************************
ok: [test_24] =>
filelist:
- /etc/yum.repos.d/CentOS-Linux-ContinuousRelease.repo
- /etc/yum.repos.d/CentOS-Linux-Debuginfo.repo
- /etc/yum.repos.d/CentOS-Linux-Devel.repo
- /etc/yum.repos.d/CentOS-Linux-Extras.repo
- /etc/yum.repos.d/CentOS-Linux-FastTrack.repo
- /etc/yum.repos.d/CentOS-Linux-HighAvailability.repo
- /etc/yum.repos.d/CentOS-Linux-Media.repo
- /etc/yum.repos.d/CentOS-Linux-Plus.repo
- /etc/yum.repos.d/CentOS-Linux-PowerTools.repo
- /etc/yum.repos.d/CentOS-Linux-Sources.repo
- /etc/yum.repos.d/CentOS-Linux-BaseOS.repo
- /etc/yum.repos.d/CentOS-Linux-AppStream.repo
TASK [debug] *********************************************************************************
skipping: [test_24] => (item=CentOS-Linux-ContinuousRelease.repo)
skipping: [test_24] => (item=CentOS-Linux-ContinuousRelease.repo)
...
skipping: [test_24] => (item=CentOS-Linux-AppStream.repo)
skipping: [test_24]
TASK [Change the lines] **********************************************************************
--- before: /etc/yum.repos.d/CentOS-Linux-ContinuousRelease.repo
+++ after: /etc/yum.repos.d/CentOS-Linux-ContinuousRelease.repo
## -17,7 +17,7 ##
[cr]
name=CentOS Linux $releasever - ContinuousRelease
-mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=cr&infra=$infra
+#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=cr&infra=$infra
#baseurl=http://mirror.centos.org/$contentdir/$releasever/cr/$basearch/os/
gpgcheck=1
enabled=0
changed: [test_24] => (item=CentOS-Linux-ContinuousRelease.repo)
--- before: /etc/yum.repos.d/CentOS-Linux-ContinuousRelease.repo
+++ after: /etc/yum.repos.d/CentOS-Linux-ContinuousRelease.repo
## -18,7 +18,7 ##
[cr]
name=CentOS Linux $releasever - ContinuousRelease
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=cr&infra=$infra
-#baseurl=http://mirror.centos.org/$contentdir/$releasever/cr/$basearch/os/
+baseurl=http://mirror.centos.org/$contentdir/$releasever/cr/$basearch/os/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
changed: [test_24] => (item=CentOS-Linux-ContinuousRelease.repo)
...
--- before: /etc/yum.repos.d/CentOS-Linux-AppStream.repo
+++ after: /etc/yum.repos.d/CentOS-Linux-AppStream.repo
## -11,7 +11,7 ##
[appstream]
name=CentOS Linux $releasever - AppStream
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=AppStream&infra=$infra
-#baseurl=http://mirror.centos.org/$contentdir/$releasever/AppStream/$basearch/os/
+#baseurl=http://vault.centos.org/$contentdir/$releasever/AppStream/$basearch/os/
baseurl=http://vault.centos.org/$contentdir/$releasever/AppStream/$basearch/os/
gpgcheck=1
enabled=0
changed: [test_24] => (item=CentOS-Linux-AppStream.repo)
PLAY RECAP ***********************************************************************************
test_24: ok=3 changed=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0

How to include variables with include_vars with the same name without overwriting previous

I am having this let's call it include.yaml
#- name: "Playing with Ansible and Include files"
- hosts: localhost
connection: local
tasks:
- find: paths="./" recurse=yes patterns="test.yaml"
register: file_to_exclude
- debug: var=file_to_exclude.stdout_lines
- name: shell
shell: "find \"$(pwd)\" -name 'test.yaml'"
register: files_from_dirs
- debug: var=files_from_dirs.stdout_lines
- name: Include variable files
include_vars: "{{ item }}"
with_items:
- "{{ files_from_dirs.stdout_lines }}"
- debug: var=files
and 2 ore more test files
./dir1/test.yaml
that contains
files:
- file1
- file2
./dir2/test.yaml
that contains
files:
- file3
- file4
the result is
TASK [Include variable files] ******************************************************************************************
ok: [localhost] => (item=/mnt/c/Users/GFlorinescu/ansible_scripts/ansible/1st/test.yaml)
ok: [localhost] => (item=/mnt/c/Users/GFlorinescu/ansible_scripts/ansible/2nd/test.yaml)
TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
"files": [
"file3",
"file4"
]
}
How can I get all the values in files, at the moment the last included files variable from last file overrides the files from the previous files? Of course without changing the variables names in files test.yaml?
In other words I want files to be:
ok: [localhost] => {
"files": [
"file1",
"file2",
"file3",
"file4"
]
}
To be more specific, I ask for any kind of solution or module, even not official or some github module, I don't want a specific include_vars module solution.
Put the included variables into the dictionaries with unique names. For example, create the names from the index of the loop. Then, iterate the names and concatenate the lists
- command: "find {{ playbook_dir }} -name test.yaml"
register: files_from_dirs
- include_vars:
file: "{{ item }}"
name: "{{ name }}"
loop: "{{ files_from_dirs.stdout_lines }}"
loop_control:
extended: true
vars:
name: "files_{{ ansible_loop.index }}"
- set_fact:
files: "{{ files|d([]) + lookup('vars', item).files }}"
with_varnames: "files_[0-9]+"
- debug:
var: files
give
files:
- file1
- file2
- file3
- file4
Notes:
You have to provide either a path relative to the home directory or an absolute path. See the example below
- command: "echo $PWD"
register: out
- debug:
var: out.stdout
give
out.stdout: /home/admin
For example, when you want to find the files relative to the directory of the playbook
- command: "find {{ playbook_dir }} -name test.yaml"
register: files_from_dirs
- debug:
var: files_from_dirs.stdout_lines
give
files_from_dirs.stdout_lines:
- /export/scratch/tmp8/test-987/dir1/test.yaml
- /export/scratch/tmp8/test-987/dir2/test.yaml
The same is valid for the module find. For example,
- find:
paths: "{{ playbook_dir }}"
recurse: true
patterns: test.yaml
register: files_from_dirs
- debug:
var: files_from_dirs.files|map(attribute='path')|list
give the same result
files_from_dirs.files|map(attribute='path')|list:
- /export/scratch/tmp8/test-987/dir1/test.yaml
- /export/scratch/tmp8/test-987/dir2/test.yaml
Simplify the code and put the declaration of files into the vars. For example, the below declaration gives the same result
files: "{{ query('varnames', 'files_[0-9]+')|
map('extract', hostvars.localhost, 'files')|
flatten }}"
Example of a complete playbook for testing
- hosts: localhost
vars:
files: "{{ query('varnames', 'files_[0-9]+')|
map('extract', hostvars.localhost, 'files')|
flatten }}"
tasks:
- find:
paths: "{{ playbook_dir }}"
recurse: true
patterns: test.yaml
register: files_from_dirs
- include_vars:
file: "{{ item }}"
name: "{{ name }}"
loop: "{{ files_from_dirs.files|map(attribute='path')|list }}"
loop_control:
extended: true
vars:
name: "files_{{ ansible_loop.index }}"
- debug:
var: files
(maybe off-topic, see comments)
Q: "Is there a way to write the path where it was found?"
A: Yes, it is. See the self-explaining example below. Given the inventory
shell> cat hosts
host_1 file_1=alice
host_2 file_2=bob
host_3
the playbook
- hosts: host_1,host_2,host_3
vars:
file_1_list: "{{ hostvars|json_query('*.file_1') }}"
file_2_list: "{{ hostvars|json_query('*.file_2') }}"
file_1_dict: "{{ dict(hostvars|dict2items|
selectattr('value.file_1', 'defined')|
json_query('[].[key, value.file_1]')) }}"
file_1_lis2: "{{ hostvars|dict2items|
selectattr('value.file_1', 'defined')|
json_query('[].{key: key, file_1: value.file_1}') }}"
tasks:
- debug:
msg: |-
file_1_list: {{ file_1_list }}
file_2_list: {{ file_2_list }}
file_1_dict:
{{ file_1_dict|to_nice_yaml|indent(2) }}
file_1_lis2:
{{ file_1_lis2|to_nice_yaml|indent(2) }}
run_once: true
gives
msg: |-
file_1_list: ['alice']
file_2_list: ['bob']
file_1_dict:
host_1: alice
file_1_lis2:
- file_1: alice
key: host_1

Nested loop with user and folder in Ansible

I have the following task:
- name: Create required folders.
become: true
ansible.builtin.file:
owner: "{{ item.key }}"
group: ftp
mode: '0755'
path: '/data/{{ item.key }}/in'
state: directory
loop: "{{ query('dict', ftp) | list }}"
when: "'state' not in item.value or item.value.state == 'present'"
And the following host variables with different users:
ftp:
test:
ssh_public_key: "XXXX"
password: "XXX"
home: /data/test
test2:
ssh_public_key: "XXXX"
password: "XXX"
home: /data/test2
What I want is to create two directories for every user :
path: '/data/{{ user }}/in' # item.key, in the code above
path: '/data/{{ user }}/out' # item.key, in the code above
But I already need the loop for iterating over the users itself:
loop: "{{ query('dict', ftp) | list }}"
How can I handle this, for example, with nested loop?
Use a product filter to generate every possible combination of user/folder.
loop: "{{ ftp.keys() | product(['in', 'out']) }}"
Then, respectively,
item.0 contains the users dictionary keys
item.1 contains the folders
It is not fully clear what your condition when does actually, in order to adapt it too, but I guess that you do have an absent or present state in those use dictionaries.
So, the resulting task should be something along the lines of
- name: Create required folders
ansible.builtin.file:
owner: "{{ item.0 }}"
group: ftp
mode: '0755'
path: "/data/{{ item.0 }}/{{ item.1 }}"
state: directory
loop: "{{ ftp.keys() | product(['in', 'out']) }}"
loop_control:
label: "/data/{{ item.0 }}/{{ item.1 }}"
when: "ftp[item.0].state | default('absent') == 'present'"
become: true
Given the task above, when run on those data:
ftp:
test:
state: present
test1:
test2:
state: present
It will yield:
TASK [Create required folders] ***************************************
ok: [localhost] => (item=/data/test/in)
ok: [localhost] => (item=/data/test/out)
skipping: [localhost] => (item=/data/test1/in)
skipping: [localhost] => (item=/data/test1/out)
ok: [localhost] => (item=/data/test2/in)
ok: [localhost] => (item=/data/test2/out)
Test it first, for example
- debug:
msg: "Create /data/{{ item.0.key }}/{{ item.1 }}"
with_nested:
- "{{ ftp|dict2items }}"
- [in, out]
when: item.0.value.state|d('present') == 'present'
gives (abridged)
msg: Create /data/test/in
msg: Create /data/test/out
msg: Create /data/test2/in
msg: Create /data/test2/out
Then try to create the dictionaries
- file:
owner: "{{ item.0.key }}"
group: ftp
mode: '0755'
path: "/data/{{ item.0.key }}/{{ item.1 }}"
state: directory
with_nested:
- "{{ ftp|dict2items }}"
- [in, out]
when: item.0.value.state|d('present') == 'present'
(not tested)

Parse yaml files in Ansible

I have got multiple yaml files on remote machine. I would like to parse those files in order to get information about names for each kind (Deployment, Configmap, Secret) of object, For example:
...
kind: Deployment
metadata:
name: pr1-dep
...
kind: Secret
metadata:
name: pr1
...
....
kind: ConfigMap
metadata:
name: cm-pr1
....
Ecpected result:
3 variables:
deployments = [pr1-dep]
secrets = [pr1]
configmaps = [cm-pr1]
I started with:
- shell: cat "{{ item.desc }}"
with_items:
- "{{ templating_register.results }}"
register: objs
but i have no idea how to correctly parse item.stdout from objs
Ansible has a from_yaml filter that takes YAML text as input and outputs an Ansible data structure. So for example you can write something like this:
- hosts: localhost
gather_facts: false
tasks:
- name: Read objects
command: "cat {{ item }}"
register: objs
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
- debug:
msg:
- "kind: {{ obj.kind }}"
- "name: {{ obj.metadata.name }}"
vars:
obj: "{{ item.stdout | from_yaml }}"
loop: "{{ objs.results }}"
loop_control:
label: "{{ item.item }}"
Given your example files, this playbook would output:
PLAY [localhost] ***************************************************************
TASK [Read objects] ************************************************************
changed: [localhost] => (item=deployment.yaml)
changed: [localhost] => (item=configmap.yaml)
changed: [localhost] => (item=secret.yaml)
TASK [debug] *******************************************************************
ok: [localhost] => (item=deployment.yaml) => {
"msg": [
"kind: Deployment",
"name: pr1-dep"
]
}
ok: [localhost] => (item=configmap.yaml) => {
"msg": [
"kind: ConfigMap",
"name: pr1-cm"
]
}
ok: [localhost] => (item=secret.yaml) => {
"msg": [
"kind: Secret",
"name: pr1"
]
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Creating the variables you've asked for is a little trickier. Here's
one option:
- hosts: localhost
gather_facts: false
tasks:
- name: Read objects
command: "cat {{ item }}"
register: objs
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
- name: Create variables
set_fact:
names: >-
{{
names|combine({
obj.kind.lower(): [obj.metadata.name]
}, list_merge='append')
}}
vars:
names: {}
obj: "{{ item.stdout | from_yaml }}"
loop: "{{ objs.results }}"
loop_control:
label: "{{ item.item }}"
- debug:
var: names
This creates a single variable named names that at the end of the
playbook will contain:
{
"configmap": [
"pr1-cm"
],
"deployment": [
"pr1-dep"
],
"secret": [
"pr1"
]
}
The key to the above playbook is our use of the combine filter, which can be used to merge dictionaries and, when we add list_merge='append', handles keys that resolve to lists by appending to the existing list, rather than replacing the existing key.
Include the dictionaries from the files into the new variables, e.g.
- include_vars:
file: "{{ item }}"
name: "objs_{{ item|splitext|first }}"
register: result
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
This will create dictionaries objs_deployment, objs_configmap, and objs_secret. Next, you can either use the dictionaries
- set_fact:
objs: "{{ objs|d({})|combine({_key: _val}) }}"
loop: "{{ query('varnames', 'objs_') }}"
vars:
_obj: "{{ lookup('vars', item) }}"
_key: "{{ _obj.kind }}"
_val: "{{ _obj.metadata.name }}"
, or the registered data
- set_fact:
objs: "{{ dict(_keys|zip(_vals)) }}"
vars:
_query1: '[].ansible_facts.*.kind'
_keys: "{{ result.results|json_query(_query1)|flatten }}"
_query2: '[].ansible_facts.*.metadata[].name'
_vals: "{{ result.results|json_query(_query2)|flatten }}"
Both options give
objs:
ConfigMap: cm-pr1
Deployment: pr1-dep
Secret: pr1

Ansible: How to filter dict2items and run playbook only for the matched values

I have a dict playbook which looks like this:
x_php_versions_installed:
ea-php71:
- ea-php71-php-bcmath
- ea-php71-php-xmlrpc
- ea-php71-php-zip
- pecl-memcached
- pecl-imagick
ea-php72:
- ea-php72-php-cli
- ea-php72-php-common
- ea-php72-php-curl
- pecl-imagick
I would like to filter them, to write me each item.value which contains 'ea' string but not everything else. My task looks like this:
- name: Write out only the ea packages
debug:
msg: '{{ item.value }}'
when: item.value | selectattr(item.value, 'contains', 'ea')
loop: '{{ x_php_versions_installed | dict2items }}
But it does not work, because it will list all of the packages, not only the ea ones. The expected answer should look like this:
...
"msg": [
"ea-php71-php-bcmath",
"ea-php71-php-xmlrpc",
"ea-php71-php-zip"
]
...
"msg": [
"ea-php72-php-cli",
"ea-php72-php-common",
"ea-php72-php-curl"
]
...
Another possibility is to filter out the 'pecl' string, it will gave me the same result and it also works fine.
Q: "Filter item.value which contains ea string."
A: The task below does the job
- debug:
msg: "{{ item.value|select('match','^ea-(.*)$')|list }}"
loop: "{{ x_php_versions_installed|dict2items }}"
gives (abridged)
msg:
- ea-php71-php-bcmath
- ea-php71-php-xmlrpc
- ea-php71-php-zip
msg:
- ea-php72-php-cli
- ea-php72-php-common
- ea-php72-php-curl
Note: The test match by default "succeeds if it finds the pattern at the beginning of the string". The task below gives the same result
- debug:
msg: "{{ item.value|select('match', 'ea-')|list }}"
loop: "{{ x_php_versions_installed|dict2items }}"
Q: "Filter out the pecl string."
A: Change the filter to reject and fit the regex. For example, the task below gives the same result
- debug:
msg: "{{ item.value|reject('match','^pecl-(.*)$')|list }}"
loop: "{{ x_php_versions_installed|dict2items }}"
Notes:
Select the lists without iteration. Declare the variables
x_php_versions_installed_keys: "{{ x_php_versions_installed.keys()|list }}"
x_php_versions_installed_ea_vals: "{{ x_php_versions_installed|dict2items|
map(attribute='value')|
map('select', 'match', 'ea-')|list }}"
x_php_versions_installed_ea: "{{ dict(x_php_versions_installed_keys|
zip(x_php_versions_installed_ea_vals)) }}"
gives
x_php_versions_installed_ea:
ea-php71:
- ea-php71-php-bcmath
- ea-php71-php-xmlrpc
- ea-php71-php-zip
ea-php72:
- ea-php72-php-cli
- ea-php72-php-common
- ea-php72-php-curl
Example of a complete playbook for testing
- hosts: localhost
vars:
x_php_versions_installed:
ea-php71:
- ea-php71-php-bcmath
- ea-php71-php-xmlrpc
- ea-php71-php-zip
- pecl-memcached
- pecl-imagick
ea-php72:
- ea-php72-php-cli
- ea-php72-php-common
- ea-php72-php-curl
- pecl-imagick
x_php_versions_installed_keys: "{{ x_php_versions_installed.keys()|list }}"
x_php_versions_installed_ea_vals: "{{ x_php_versions_installed|dict2items|
map(attribute='value')|
map('select', 'match', 'ea-')|list }}"
x_php_versions_installed_ea: "{{ dict(x_php_versions_installed_keys|
zip(x_php_versions_installed_ea_vals)) }}"
tasks:
- debug:
msg: "{{ item.value|select('match','^ea-(.*)$')|list }}"
loop: "{{ x_php_versions_installed|dict2items }}"
- debug:
msg: "{{ item.value|select('match', 'ea-')|list }}"
loop: "{{ x_php_versions_installed|dict2items }}"
- debug:
msg: "{{ item.value|reject('match','^pecl-(.*)$')|list }}"
loop: "{{ x_php_versions_installed|dict2items }}"
- debug:
msg: "{{ item.value|reject('match','pecl-')|list }}"
loop: "{{ x_php_versions_installed|dict2items }}"
- debug:
var: x_php_versions_installed_ea

Resources