Format output to list - ansible

I'm working on an ansible playbook. I'm checking for new package patch versions of software, these versions are part of a list.
My code looks like this atm, credits for building the list goes to #Zeitounator
- name: get list of all supported version packages
shell: |
set -o pipefail
repoquery --cache --showduplicates --qf "%{VERSION}" --enablerepo xyz abc \
| grep -E -- "13." \
| sort --unique --version-sort
changed_when: false
register: versions
- name: get the major versions
set_fact:
major_versions: >-
{{
versions.stdout_lines
| map('regex_replace', '^(\d*\.\d*)\.\d*$', '\g<1>')
| unique
| sort
}}
- name: Create a consolidated list per major version
set_fact:
consolidated_versions: >-
{{
consolidated_versions | default([])
+
[{'major_version': item, 'patch_versions': versions.stdout_lines | select('contains', item) | list }]
}}
loop: "{{ major_versions }}"
My first task was to check the patch levels only through their corresponding major versions.
For example:
Check version 13.0.[1-10] only in folder 13.0, 13.1.[1-5] only in folder 13.1 etc.
This works as expected.
What I want to do now is to download only the latest package version of a major version, so I only need a list with the patch version numbers, like this:
13.0.10
13.1.5
13.2.2
I tried it with another set_fact
Like this:
- name: get latest patchversion of supported version
set_fact:
patch_version: "{{ consolidated_versions | json_query('[*].patch_versions[-1]') }}"
- name: output the last versions
debug:
var: patch_version
This gives me an output like this:
ok: [localhost] => {
"patch_version": [
"13.0.10",
"13.1.5",
"13.2.2"
]
}
This is indeed the data I need.
I need to use these 3 elements in a loop to download the packages like this:
- name: Download the files
get_url:
url: https://packages.xyz.com/abc/def/packages/el/{{ ansible_distribution_major_version }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm/download.rpm
dest: /var/www/html/abc/{{ date }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm
loop: "{{ patch_version }}"
Problem is now that ansible does not handle the fact as a list, so it replaces {{ patch_version }}with this: ['13.0.10', '13.1.5', '13.2.2']
Of course, this won't work.
How do I transform this output into a loopable list?
I already tried to make it a list, but then I got the whole output as a string as one element. How do I split this into a list like this:
- 13.0.10
- 13.1.5
- 13.2.2
?
Thanks in advance, I'm so confused.

I couldn't reproduce your problem because fact is handled as list in my ansible version (ansible 2.9.6, python 3.7.3).
Nevertheless, if fact is a string try to pass it through from_json filter:
loop: "{{ patch_version | from_json }}"
FWIW, you can check variable type piping it into type_debug:
- debug:
msg: "{{ patch_version | type_debug }}"

Fixed it by myself, not sure what the bug was, maybe just my mess ;).
For testing purposes I tried some things with with_nested and with loop.
So my code looked like this for a while:
- name: Download the files
get_url:
url: https://packages.xyz.com/abc/def/packages/el/{{ ansible_distribution_major_version }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm/download.rpm
dest: /var/www/html/abc/{{ date }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm
loop:
- "{{ patch_version }}"
# - commented out loop
# - another commented out loop
At this case the list seems to be read as a string.
So ansible replaces the variable with the all values as a string into my code.
After I changed the code to:
- name: Download the files
get_url:
url: https://packages.xyz.com/abc/def/packages/el/{{ ansible_distribution_major_version }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm/download.rpm
dest: /var/www/html/abc/{{ date }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm
loop: "{{ patch_version }}"
It worked as expected. Thanks for the hints, maybe my brain's a little bit confused after the whole day working on that playbook ;)

Related

Iterate through an array of values of yaml data

I have the output of a command:
---
data:
versions:
- alt-php53
- ea-php56
- ea-php72
metadata:
command: php_get_installed_versions
reason: OK
result: 1
version: 1
And I need to iterate through the values of the versions key.
I have this to assign the output to a variable:
- name: Get PHP versions
shell: "/usr/sbin/whmapi1 php_get_installed_versions"
register: eaphp_versions
But I do not know how exactly to iterate through those values. I tried various methods with dict2items, from_yaml, from_yaml_all using loop, I tried sub_elements but none worked.
The closest I got is the below:
- name: 'Apply stuff'
shell: "echo the item is {{ item }} >> report.txt"
loop: "{{ eaphp_versions.stdout | from_yaml_all | list }}"
But judging from the output it iterates through the whole output as one item:
the item is {udata: {uversions: [ualt-php53, uea-php56, uea-php72]}, umetadata: {ureason: uOK, uversion: 1, ucommand: uphp_get_installed_versions, uresult: 1}}
Please help.
Thanks in advance.
Assume the data structure at top are the contents from eaphp_versions, then you can:
- any-your-module:
any-module-params: use of {{ item }}
with_items: "{{ eaphp_versions.data.versions }}"
The {{ item }} is one of the 3 versions.
I found a solution to this myself. Here it is below how I did it.
- name: Set fact
set_fact:
versions: "{{ versions|default({})|combine(item) }}"
loop: "{{ eaphp_versions.stdout | from_yaml_all|list }}"
- name: 'Set PHP memory_limit to 512M (all PHP versions)'
#40
shell: "/usr/sbin/whmapi1 php_ini_set_directives directive-1=memory_limit%3A512M version={{ item }}"
with_items: "{{ versions.data.versions }}"

Sorting list of version numbers in Jinja2 [duplicate]

I'm building an Ansible playbook in which I want to make a backup of a database in case I need to upgrade the software. For this I want to compare the highest version number that is available to the version number that is installed. In case the latest version is hight than the installed version I'll back up the database.
The problem however is that I cannot find a good way to sort version numbers in Ansible. The standard sort filter sorts on strings instead of numbers/versions.
This is what I'm doing right now:
- name: Get package version
yum:
list: package
register: software_version
- name: Read which version is installed and available
set_fact:
software_version_installed: "{{ software_version | json_query(\"results[?yumstate=='installed'].version\") | sort | last }}"
software_version_available: "{{ software_version | json_query(\"results[?yumstate=='available'].version\") | sort | last }}"
- name: Backup old database file on remote host
copy:
src: "{{ software.database_path }}"
dest: "{{ software.database_path }}_{{ ansible_date_time.date }}_v{{ software_version_installed }}"
remote_src: yes
when: software_version_installed is version(software_version_available, "<")
The playbook above works, as long as version numbers stay underneath the number 10 (e.g. 1.2.3, but not 1.10.1) since sorting is performed like a string. When the version number has to sort e.g. 1.2.3 and 1.10.1, it will take 1.2.3 as latest version.
To show the issue:
- name: Read which version is installed and available
set_fact:
software_versions: [ "2.5.0", "2.9.0", "2.10.0", "2.11.0" ]
- name: Debug
debug:
var: software_versions | sort
TASK [grafana : Debug]**********************************
ok: [localhost] => {
"software_versions | sort": [
"2.10.0",
"2.11.0",
"2.5.0",
"2.9.0"
]
}
Does anyone know a good way to sort version numbers in Ansible?
Q: Does anyone know a good way to sort version numbers in Ansible?
A: Use filter_plugin. For example the filter
shell> cat filter_plugins/version_sort.py
from distutils.version import LooseVersion
def version_sort(l):
return sorted(l, key=LooseVersion)
class FilterModule(object):
def filters(self):
return {
'version_sort' : version_sort
}
with the playbook
shell> cat test-versions.yml
- name: Sort versions
hosts: localhost
vars:
versions:
- "0.1.0"
- "0.1.5"
- "0.11.11"
- "0.9.11"
- "0.9.3"
- "0.10.2"
- "0.6.1"
- "0.6.0"
- "0.11.0"
- "0.6.5"
tasks:
- debug:
msg: "{{ versions|version_sort }}"
gives
"msg": [
"0.1.0",
"0.1.5",
"0.6.0",
"0.6.1",
"0.6.5",
"0.9.3",
"0.9.11",
"0.10.2",
"0.11.0",
"0.11.11"
]
For your convenience, the filter is available at Github ansible-plugins.
Version comparison does the job to iterate the list and compare items. See the example below
shell> cat test-versions.yml
- hosts: localhost
vars:
version_installed: "1.10.1"
versions:
- "1.1.3"
- "1.2.3"
- "1.7.5"
- "1.10.7"
tasks:
- debug: msg="{{ item }} is newer than {{ version_installed }}"
loop: "{{ versions }}"
when: item is version(version_installed, '>')
shell> ansible-playbook test-versions.yml | grep msg
"msg": "1.10.7 is newer than 1.10.1"
It's now solved in another way. Instead of sorting the versions I compared the current version to all available versions.
I've started by setting an update variable to false
Next I compared the installed version to every available version
If installed version > current version, set the update variable to true
The task performing the backup will only be performed when the update variable is true.
- name: Get package version
yum:
list: package
register: software_version
- name: Read which version is installed and available
set_fact:
software_update: false
software_version_installed: "{{ software_version | json_query(\"results[?yumstate=='installed'].version\") | last }}"
software_version_available: "{{ software_version | json_query(\"results[?yumstate=='available'].version\") }}"
- name: Check if upgrade is needed
set_fact:
software_update: true
when: software_version_installed is version(item, "<")
with_items: "{{ software_version_available }}"
- name: Backup old database file on remote host
copy:
src: "{{ software.database_path }}"
dest: "{{ software.database_path }}_{{ ansible_date_time.date }}_v{{ software_version_installed }}"
remote_src: yes
when: software_update
All the answers here provide a custom way of sorting versions, I'd like to point out that stock ansible can do this already:
- name: Sort list by version number
debug:
var: ansible_versions | community.general.version_sort
vars:
ansible_versions:
- '2.8.0'
- '2.11.0'
- '2.7.0'
- '2.10.0'
- '2.9.0'
https://docs.ansible.com/ansible/latest/collections/community/general/docsite/filter_guide_working_with_versions.html
The current accepted answer offers a very nice solution, using a filter_plugin. Unfortunately, the distutils Python package that is uses appears to be deprecated. Some googling led me to the packaging package, which offers a similar Version class. Here's an updated filter_plugin that doesn't use distutils:
from packaging.version import Version
def version_sort(l):
return sorted(l, key=Version)
class FilterModule(object):
def filters(self):
return {
'version_sort': version_sort
}
It's working well for us, but I don't want to promise that the behavior will be exactly the same in every situation.

Is there any method to map multiple attributes in ansible

I have an output from yum list module. The thing is I want to display the output without those number (epoch attribute). The problem is I couldn't find any solutions of mapping 2 attributes (name and version). All the solutions I found are connected to only 1 attribute ( envra) in my case.
- name: check packages
become: true
yum:
list: installed
register: output
- name: add lines to files
lineinfile:
dest: "./file.txt"
line: "{{ inventory_hostname }} {{ item }}"
with_items:
- "{{ output.results | map(attribute='envra') |list }}"
delegate_to: localhost
This is the output without any mapping. As you can see there are multiple attributes. I would like to display only name and version of the package.
10.112.65.15 {u'envra': u'0:GeoIP-1.5.0-14.el7.x86_64', u'name': u'GeoIP', u'repo': u'installed', u'epoch': u'0', u'version': u'1.5.0', u'release': u'14.el7', u'yumstate': u'installed', u'arch': u'x86_64'}
The closest to expected values is envra attribute, but still has those epoch number inside...
10.112.65.15 0:GeoIP-1.5.0-14.el7.x86_64
As I mentioned at the begging I would like to get output of something like that
10.112.65.15 GeoIP 1.5.0
or at least without epoch attribute.
I've also change approach and tried this method
- name: add lines to files
lineinfile:
dest: "./file.txt"
line: "{{ inventory_hostname }} {{ item }} "
with_items:
- "{{ output | json_query(my_query) | list }}"
delegate_to: localhost
vars:
my_query: "results[].[name, version]"
but received result was with '[]' and u' which I'd like to delete but don't exactly know how.
10.112.65.15 [u'GeoIP', u'1.5.0']
Why do you extract the attribute or use json query ? Simply use the hash and print out the needed fields. The following should work out of the box.
- name: add lines to files
lineinfile:
dest: "./file.txt"
line: "{{ inventory_hostname }} {{ item.name }} {{ item.version }}"
with_items:
- "{{ output.results }}"
delegate_to: localhost

How to sort version numbers in Ansible

I'm building an Ansible playbook in which I want to make a backup of a database in case I need to upgrade the software. For this I want to compare the highest version number that is available to the version number that is installed. In case the latest version is hight than the installed version I'll back up the database.
The problem however is that I cannot find a good way to sort version numbers in Ansible. The standard sort filter sorts on strings instead of numbers/versions.
This is what I'm doing right now:
- name: Get package version
yum:
list: package
register: software_version
- name: Read which version is installed and available
set_fact:
software_version_installed: "{{ software_version | json_query(\"results[?yumstate=='installed'].version\") | sort | last }}"
software_version_available: "{{ software_version | json_query(\"results[?yumstate=='available'].version\") | sort | last }}"
- name: Backup old database file on remote host
copy:
src: "{{ software.database_path }}"
dest: "{{ software.database_path }}_{{ ansible_date_time.date }}_v{{ software_version_installed }}"
remote_src: yes
when: software_version_installed is version(software_version_available, "<")
The playbook above works, as long as version numbers stay underneath the number 10 (e.g. 1.2.3, but not 1.10.1) since sorting is performed like a string. When the version number has to sort e.g. 1.2.3 and 1.10.1, it will take 1.2.3 as latest version.
To show the issue:
- name: Read which version is installed and available
set_fact:
software_versions: [ "2.5.0", "2.9.0", "2.10.0", "2.11.0" ]
- name: Debug
debug:
var: software_versions | sort
TASK [grafana : Debug]**********************************
ok: [localhost] => {
"software_versions | sort": [
"2.10.0",
"2.11.0",
"2.5.0",
"2.9.0"
]
}
Does anyone know a good way to sort version numbers in Ansible?
Q: Does anyone know a good way to sort version numbers in Ansible?
A: Use filter_plugin. For example the filter
shell> cat filter_plugins/version_sort.py
from distutils.version import LooseVersion
def version_sort(l):
return sorted(l, key=LooseVersion)
class FilterModule(object):
def filters(self):
return {
'version_sort' : version_sort
}
with the playbook
shell> cat test-versions.yml
- name: Sort versions
hosts: localhost
vars:
versions:
- "0.1.0"
- "0.1.5"
- "0.11.11"
- "0.9.11"
- "0.9.3"
- "0.10.2"
- "0.6.1"
- "0.6.0"
- "0.11.0"
- "0.6.5"
tasks:
- debug:
msg: "{{ versions|version_sort }}"
gives
"msg": [
"0.1.0",
"0.1.5",
"0.6.0",
"0.6.1",
"0.6.5",
"0.9.3",
"0.9.11",
"0.10.2",
"0.11.0",
"0.11.11"
]
For your convenience, the filter is available at Github ansible-plugins.
Version comparison does the job to iterate the list and compare items. See the example below
shell> cat test-versions.yml
- hosts: localhost
vars:
version_installed: "1.10.1"
versions:
- "1.1.3"
- "1.2.3"
- "1.7.5"
- "1.10.7"
tasks:
- debug: msg="{{ item }} is newer than {{ version_installed }}"
loop: "{{ versions }}"
when: item is version(version_installed, '>')
shell> ansible-playbook test-versions.yml | grep msg
"msg": "1.10.7 is newer than 1.10.1"
It's now solved in another way. Instead of sorting the versions I compared the current version to all available versions.
I've started by setting an update variable to false
Next I compared the installed version to every available version
If installed version > current version, set the update variable to true
The task performing the backup will only be performed when the update variable is true.
- name: Get package version
yum:
list: package
register: software_version
- name: Read which version is installed and available
set_fact:
software_update: false
software_version_installed: "{{ software_version | json_query(\"results[?yumstate=='installed'].version\") | last }}"
software_version_available: "{{ software_version | json_query(\"results[?yumstate=='available'].version\") }}"
- name: Check if upgrade is needed
set_fact:
software_update: true
when: software_version_installed is version(item, "<")
with_items: "{{ software_version_available }}"
- name: Backup old database file on remote host
copy:
src: "{{ software.database_path }}"
dest: "{{ software.database_path }}_{{ ansible_date_time.date }}_v{{ software_version_installed }}"
remote_src: yes
when: software_update
All the answers here provide a custom way of sorting versions, I'd like to point out that stock ansible can do this already:
- name: Sort list by version number
debug:
var: ansible_versions | community.general.version_sort
vars:
ansible_versions:
- '2.8.0'
- '2.11.0'
- '2.7.0'
- '2.10.0'
- '2.9.0'
https://docs.ansible.com/ansible/latest/collections/community/general/docsite/filter_guide_working_with_versions.html
The current accepted answer offers a very nice solution, using a filter_plugin. Unfortunately, the distutils Python package that is uses appears to be deprecated. Some googling led me to the packaging package, which offers a similar Version class. Here's an updated filter_plugin that doesn't use distutils:
from packaging.version import Version
def version_sort(l):
return sorted(l, key=Version)
class FilterModule(object):
def filters(self):
return {
'version_sort': version_sort
}
It's working well for us, but I don't want to promise that the behavior will be exactly the same in every situation.

Ansible with_items if item is defined

Ansible 1.9.4.
The script should execute some task only on hosts where some variable is defined. It works fine normally, but it doesn't work with the with_items statement.
- debug: var=symlinks
when: symlinks is defined
- name: Create other symlinks
file: src={{ item.src }} dest={{ item.dest }} state=link
with_items: "{{ symlinks }}"
when: symlinks is defined
But I get:
TASK: [app/symlinks | debug var=symlinks] *********************
skipping: [another-host-yet]
TASK: [app/symlinks | Create other symlinks] ******************
fatal: [another-host-yet] => with_items expects a list or a set
Maybe I am doing something wrong?
with_items: "{{ symlinks | default([]) }}"
The reason for this behavior is conditions work differently inside loops. If a loop was defined the condition is evaluated for every item while iterating over the items. But the loop itself requires a valid list.
This is also mentioned in the docs:
Note that when combining when with with_items (see Loops), be aware that the when statement is processed separately for each item. This is by design:
tasks:
- command: echo {{ item }}
with_items: [ 0, 2, 4, 6, 8, 10 ]
when: item > 5
I think this is a bad design choice and for this functionality they better should have introduced something like with_when.
As you have already figured out yourself, you can default to an empty list.
with_items: "{{ symlinks | default([]) }}"
Finally if the list is dynamically loaded from a var, say x, use:
with_items: "{{ symlinks[x|default('')] | default([])}}"
This will default to an empty list when 'x' is undefined
Accordingly, fall back to an empty dict with default({}):
# service_facts skips, then dict2items fails?
with_dict: "{{ ansible_facts.services|default({})|dict2items|selectattr('key', 'match', '[^#]+#.+\\.service')|list|items2dict }}"

Resources