How to sort version numbers in Ansible - sorting

I'm building an Ansible playbook in which I want to make a backup of a database in case I need to upgrade the software. For this I want to compare the highest version number that is available to the version number that is installed. In case the latest version is hight than the installed version I'll back up the database.
The problem however is that I cannot find a good way to sort version numbers in Ansible. The standard sort filter sorts on strings instead of numbers/versions.
This is what I'm doing right now:
- name: Get package version
yum:
list: package
register: software_version
- name: Read which version is installed and available
set_fact:
software_version_installed: "{{ software_version | json_query(\"results[?yumstate=='installed'].version\") | sort | last }}"
software_version_available: "{{ software_version | json_query(\"results[?yumstate=='available'].version\") | sort | last }}"
- name: Backup old database file on remote host
copy:
src: "{{ software.database_path }}"
dest: "{{ software.database_path }}_{{ ansible_date_time.date }}_v{{ software_version_installed }}"
remote_src: yes
when: software_version_installed is version(software_version_available, "<")
The playbook above works, as long as version numbers stay underneath the number 10 (e.g. 1.2.3, but not 1.10.1) since sorting is performed like a string. When the version number has to sort e.g. 1.2.3 and 1.10.1, it will take 1.2.3 as latest version.
To show the issue:
- name: Read which version is installed and available
set_fact:
software_versions: [ "2.5.0", "2.9.0", "2.10.0", "2.11.0" ]
- name: Debug
debug:
var: software_versions | sort
TASK [grafana : Debug]**********************************
ok: [localhost] => {
"software_versions | sort": [
"2.10.0",
"2.11.0",
"2.5.0",
"2.9.0"
]
}
Does anyone know a good way to sort version numbers in Ansible?

Q: Does anyone know a good way to sort version numbers in Ansible?
A: Use filter_plugin. For example the filter
shell> cat filter_plugins/version_sort.py
from distutils.version import LooseVersion
def version_sort(l):
return sorted(l, key=LooseVersion)
class FilterModule(object):
def filters(self):
return {
'version_sort' : version_sort
}
with the playbook
shell> cat test-versions.yml
- name: Sort versions
hosts: localhost
vars:
versions:
- "0.1.0"
- "0.1.5"
- "0.11.11"
- "0.9.11"
- "0.9.3"
- "0.10.2"
- "0.6.1"
- "0.6.0"
- "0.11.0"
- "0.6.5"
tasks:
- debug:
msg: "{{ versions|version_sort }}"
gives
"msg": [
"0.1.0",
"0.1.5",
"0.6.0",
"0.6.1",
"0.6.5",
"0.9.3",
"0.9.11",
"0.10.2",
"0.11.0",
"0.11.11"
]
For your convenience, the filter is available at Github ansible-plugins.
Version comparison does the job to iterate the list and compare items. See the example below
shell> cat test-versions.yml
- hosts: localhost
vars:
version_installed: "1.10.1"
versions:
- "1.1.3"
- "1.2.3"
- "1.7.5"
- "1.10.7"
tasks:
- debug: msg="{{ item }} is newer than {{ version_installed }}"
loop: "{{ versions }}"
when: item is version(version_installed, '>')
shell> ansible-playbook test-versions.yml | grep msg
"msg": "1.10.7 is newer than 1.10.1"

It's now solved in another way. Instead of sorting the versions I compared the current version to all available versions.
I've started by setting an update variable to false
Next I compared the installed version to every available version
If installed version > current version, set the update variable to true
The task performing the backup will only be performed when the update variable is true.
- name: Get package version
yum:
list: package
register: software_version
- name: Read which version is installed and available
set_fact:
software_update: false
software_version_installed: "{{ software_version | json_query(\"results[?yumstate=='installed'].version\") | last }}"
software_version_available: "{{ software_version | json_query(\"results[?yumstate=='available'].version\") }}"
- name: Check if upgrade is needed
set_fact:
software_update: true
when: software_version_installed is version(item, "<")
with_items: "{{ software_version_available }}"
- name: Backup old database file on remote host
copy:
src: "{{ software.database_path }}"
dest: "{{ software.database_path }}_{{ ansible_date_time.date }}_v{{ software_version_installed }}"
remote_src: yes
when: software_update

All the answers here provide a custom way of sorting versions, I'd like to point out that stock ansible can do this already:
- name: Sort list by version number
debug:
var: ansible_versions | community.general.version_sort
vars:
ansible_versions:
- '2.8.0'
- '2.11.0'
- '2.7.0'
- '2.10.0'
- '2.9.0'
https://docs.ansible.com/ansible/latest/collections/community/general/docsite/filter_guide_working_with_versions.html

The current accepted answer offers a very nice solution, using a filter_plugin. Unfortunately, the distutils Python package that is uses appears to be deprecated. Some googling led me to the packaging package, which offers a similar Version class. Here's an updated filter_plugin that doesn't use distutils:
from packaging.version import Version
def version_sort(l):
return sorted(l, key=Version)
class FilterModule(object):
def filters(self):
return {
'version_sort': version_sort
}
It's working well for us, but I don't want to promise that the behavior will be exactly the same in every situation.

Related

selectattr returns generator rows and cannot use results as a dict

SOLUTION:
I Don't know what is the exact difference between python -m pip install ansible or apt install ansible but when I installed python -m pip ansible-core==2.10.4 it works fine.
I have CSV file which looks like:
id;env;credentials;path
1;tst;userA;/tmpA
2;dev;userB;/tmpB
3;dev;userB;/tmpC
4;acc;userB;/tmpD
5;prd;userC;/tmpE
I read this file using read_csv module and then I'm filtering using selectattr:
- name: Read CSV
read_csv:
path: "/tmp/example.csv"
delimiter: ';'
register: csv_output
- name: Filter rows
set_fact:
new_fact: "{{ csv_output.list | selectattr('env', 'equalto', tst) }}"
In the past I was able to just use these results as a dict so for example:
- debug:
msg: "{{ new_fact }}"
ok: [ansible_main] => {
"msg": [
{
"id": "1",
"env": "tst",
"credentials": "userA",
"path": "/tmpA"
}
]
}
but when I try to print new_fact on my local machine I see only generator:
ok: [ansible_main] => {
"msg": "<generator object select_or_reject at 0x7f2e4e8847b0>"
}
and I cannot use new_fact.credentials variable... Do you know how can I fix it? I know I can add | list at the end of my filter but then I also cannot use new_fact.credentials
Details of my installation:
ansible 2.9.6
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/userA/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]
Regarding
In the past I was able to just use these results as a dict ...
was it a much newer version of Ansible than v2.9.6?
I know I can add | list at the end of my filter but then I also cannot user new_fact.credentials then
Since you will get a list too, it will be necessary to specify the element
- name: Filter rows
set_fact:
new_fact: "{{ csv_output.list | selectattr('env', 'contains', 'tst') | list }}"
- debug:
msg: "{{ new_fact[0].credentials }}"
to loop over the result, or pickup one element only.
- name: Filter rows
set_fact:
new_fact: "{{ csv_output.list | selectattr('env', 'contains', 'tst') | first }}"
- debug:
msg: "{{ new_fact.credentials }}"

Sorting list of version numbers in Jinja2 [duplicate]

I'm building an Ansible playbook in which I want to make a backup of a database in case I need to upgrade the software. For this I want to compare the highest version number that is available to the version number that is installed. In case the latest version is hight than the installed version I'll back up the database.
The problem however is that I cannot find a good way to sort version numbers in Ansible. The standard sort filter sorts on strings instead of numbers/versions.
This is what I'm doing right now:
- name: Get package version
yum:
list: package
register: software_version
- name: Read which version is installed and available
set_fact:
software_version_installed: "{{ software_version | json_query(\"results[?yumstate=='installed'].version\") | sort | last }}"
software_version_available: "{{ software_version | json_query(\"results[?yumstate=='available'].version\") | sort | last }}"
- name: Backup old database file on remote host
copy:
src: "{{ software.database_path }}"
dest: "{{ software.database_path }}_{{ ansible_date_time.date }}_v{{ software_version_installed }}"
remote_src: yes
when: software_version_installed is version(software_version_available, "<")
The playbook above works, as long as version numbers stay underneath the number 10 (e.g. 1.2.3, but not 1.10.1) since sorting is performed like a string. When the version number has to sort e.g. 1.2.3 and 1.10.1, it will take 1.2.3 as latest version.
To show the issue:
- name: Read which version is installed and available
set_fact:
software_versions: [ "2.5.0", "2.9.0", "2.10.0", "2.11.0" ]
- name: Debug
debug:
var: software_versions | sort
TASK [grafana : Debug]**********************************
ok: [localhost] => {
"software_versions | sort": [
"2.10.0",
"2.11.0",
"2.5.0",
"2.9.0"
]
}
Does anyone know a good way to sort version numbers in Ansible?
Q: Does anyone know a good way to sort version numbers in Ansible?
A: Use filter_plugin. For example the filter
shell> cat filter_plugins/version_sort.py
from distutils.version import LooseVersion
def version_sort(l):
return sorted(l, key=LooseVersion)
class FilterModule(object):
def filters(self):
return {
'version_sort' : version_sort
}
with the playbook
shell> cat test-versions.yml
- name: Sort versions
hosts: localhost
vars:
versions:
- "0.1.0"
- "0.1.5"
- "0.11.11"
- "0.9.11"
- "0.9.3"
- "0.10.2"
- "0.6.1"
- "0.6.0"
- "0.11.0"
- "0.6.5"
tasks:
- debug:
msg: "{{ versions|version_sort }}"
gives
"msg": [
"0.1.0",
"0.1.5",
"0.6.0",
"0.6.1",
"0.6.5",
"0.9.3",
"0.9.11",
"0.10.2",
"0.11.0",
"0.11.11"
]
For your convenience, the filter is available at Github ansible-plugins.
Version comparison does the job to iterate the list and compare items. See the example below
shell> cat test-versions.yml
- hosts: localhost
vars:
version_installed: "1.10.1"
versions:
- "1.1.3"
- "1.2.3"
- "1.7.5"
- "1.10.7"
tasks:
- debug: msg="{{ item }} is newer than {{ version_installed }}"
loop: "{{ versions }}"
when: item is version(version_installed, '>')
shell> ansible-playbook test-versions.yml | grep msg
"msg": "1.10.7 is newer than 1.10.1"
It's now solved in another way. Instead of sorting the versions I compared the current version to all available versions.
I've started by setting an update variable to false
Next I compared the installed version to every available version
If installed version > current version, set the update variable to true
The task performing the backup will only be performed when the update variable is true.
- name: Get package version
yum:
list: package
register: software_version
- name: Read which version is installed and available
set_fact:
software_update: false
software_version_installed: "{{ software_version | json_query(\"results[?yumstate=='installed'].version\") | last }}"
software_version_available: "{{ software_version | json_query(\"results[?yumstate=='available'].version\") }}"
- name: Check if upgrade is needed
set_fact:
software_update: true
when: software_version_installed is version(item, "<")
with_items: "{{ software_version_available }}"
- name: Backup old database file on remote host
copy:
src: "{{ software.database_path }}"
dest: "{{ software.database_path }}_{{ ansible_date_time.date }}_v{{ software_version_installed }}"
remote_src: yes
when: software_update
All the answers here provide a custom way of sorting versions, I'd like to point out that stock ansible can do this already:
- name: Sort list by version number
debug:
var: ansible_versions | community.general.version_sort
vars:
ansible_versions:
- '2.8.0'
- '2.11.0'
- '2.7.0'
- '2.10.0'
- '2.9.0'
https://docs.ansible.com/ansible/latest/collections/community/general/docsite/filter_guide_working_with_versions.html
The current accepted answer offers a very nice solution, using a filter_plugin. Unfortunately, the distutils Python package that is uses appears to be deprecated. Some googling led me to the packaging package, which offers a similar Version class. Here's an updated filter_plugin that doesn't use distutils:
from packaging.version import Version
def version_sort(l):
return sorted(l, key=Version)
class FilterModule(object):
def filters(self):
return {
'version_sort': version_sort
}
It's working well for us, but I don't want to promise that the behavior will be exactly the same in every situation.

Format output to list

I'm working on an ansible playbook. I'm checking for new package patch versions of software, these versions are part of a list.
My code looks like this atm, credits for building the list goes to #Zeitounator
- name: get list of all supported version packages
shell: |
set -o pipefail
repoquery --cache --showduplicates --qf "%{VERSION}" --enablerepo xyz abc \
| grep -E -- "13." \
| sort --unique --version-sort
changed_when: false
register: versions
- name: get the major versions
set_fact:
major_versions: >-
{{
versions.stdout_lines
| map('regex_replace', '^(\d*\.\d*)\.\d*$', '\g<1>')
| unique
| sort
}}
- name: Create a consolidated list per major version
set_fact:
consolidated_versions: >-
{{
consolidated_versions | default([])
+
[{'major_version': item, 'patch_versions': versions.stdout_lines | select('contains', item) | list }]
}}
loop: "{{ major_versions }}"
My first task was to check the patch levels only through their corresponding major versions.
For example:
Check version 13.0.[1-10] only in folder 13.0, 13.1.[1-5] only in folder 13.1 etc.
This works as expected.
What I want to do now is to download only the latest package version of a major version, so I only need a list with the patch version numbers, like this:
13.0.10
13.1.5
13.2.2
I tried it with another set_fact
Like this:
- name: get latest patchversion of supported version
set_fact:
patch_version: "{{ consolidated_versions | json_query('[*].patch_versions[-1]') }}"
- name: output the last versions
debug:
var: patch_version
This gives me an output like this:
ok: [localhost] => {
"patch_version": [
"13.0.10",
"13.1.5",
"13.2.2"
]
}
This is indeed the data I need.
I need to use these 3 elements in a loop to download the packages like this:
- name: Download the files
get_url:
url: https://packages.xyz.com/abc/def/packages/el/{{ ansible_distribution_major_version }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm/download.rpm
dest: /var/www/html/abc/{{ date }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm
loop: "{{ patch_version }}"
Problem is now that ansible does not handle the fact as a list, so it replaces {{ patch_version }}with this: ['13.0.10', '13.1.5', '13.2.2']
Of course, this won't work.
How do I transform this output into a loopable list?
I already tried to make it a list, but then I got the whole output as a string as one element. How do I split this into a list like this:
- 13.0.10
- 13.1.5
- 13.2.2
?
Thanks in advance, I'm so confused.
I couldn't reproduce your problem because fact is handled as list in my ansible version (ansible 2.9.6, python 3.7.3).
Nevertheless, if fact is a string try to pass it through from_json filter:
loop: "{{ patch_version | from_json }}"
FWIW, you can check variable type piping it into type_debug:
- debug:
msg: "{{ patch_version | type_debug }}"
Fixed it by myself, not sure what the bug was, maybe just my mess ;).
For testing purposes I tried some things with with_nested and with loop.
So my code looked like this for a while:
- name: Download the files
get_url:
url: https://packages.xyz.com/abc/def/packages/el/{{ ansible_distribution_major_version }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm/download.rpm
dest: /var/www/html/abc/{{ date }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm
loop:
- "{{ patch_version }}"
# - commented out loop
# - another commented out loop
At this case the list seems to be read as a string.
So ansible replaces the variable with the all values as a string into my code.
After I changed the code to:
- name: Download the files
get_url:
url: https://packages.xyz.com/abc/def/packages/el/{{ ansible_distribution_major_version }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm/download.rpm
dest: /var/www/html/abc/{{ date }}/abc-def-{{ item }}-ee.0.el{{ ansible_distribution_major_version }}.x86_64.rpm
loop: "{{ patch_version }}"
It worked as expected. Thanks for the hints, maybe my brain's a little bit confused after the whole day working on that playbook ;)

Ansible: How to append string to each value of list

I'm trying to append a string to each value of a list in ansible, so basically am trying to install multiple pip modules offline using .whl files.
I have two files in /opt/tmp/ path
vars:
DIR: /opt/
pymongo_modules:
- pip-19.1.1-py2.py3-none-any.whl
- pymongo-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl
- name: Install the latest pymongo package
pip:
name: "{{DIR}}/tmp/{{ pymongo_modules | join(' ') }}"
executable: "{{pip_path}}"
The above is not working because it's formating like below
"name": ["/opt/tmp/pip-19.1.1-py2.py3-none-any.whl pymongo-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl"]
I can achieve the same with below syntax but I'm getting deprecation warning
- name: Install the latest pymongo package
pip:
name: "{{DIR}}/tmp/{{ module }}"
executable: "{{pip_path}}"
with_items:
- "{{ pymongo_modules }}"
loop_control:
loop_var: module
Expecting value:
"name": ["/opt/tmp/pip-19.1.1-py2.py3-none-any.whl", "/opt/tmp/pymongo-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl"]
Use product filter like below. BTW, DIR variable already ends with a / so do not need an additional / before tmp.
- debug:
msg: "{{ item }}"
loop: "{{ [(DIR + 'tmp')] | product(pymongo_modules) | map('join', '/') | list }}"
Gives:
ok: [localhost] => (item=/opt/tmp/pip-19.1.1-py2.py3-none-any.whl) => {
"msg": "/opt/tmp/pip-19.1.1-py2.py3-none-any.whl"
}
ok: [localhost] => (item=/opt/tmp/pymongo-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl) => {
"msg": "/opt/tmp/pymongo-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl"
}
I've just answered a similar question over here. In summary, you can achieve this with regex_replace and map. It's quite a tidy method and is also in the official Ansible docs, so it seems to be recommended too.

How to sort complex version numbers in Ansible

I'm building an Ansible playbook in which I want to retrieve the latest version of a software. For this I used "sort" filter in Ansible. That, however, becomes a bit harder, when using version numbers, that are more complex and are not really numbers, e.g. 0.2.1, 0.10.1.
This is what I'm doing right now:
- name: Set version to compare
set_fact:
versions:
- "0.1.0"
- "0.1.5"
- "0.11.11"
- "0.9.11"
- "0.9.3"
- "0.10.2"
- "0.6.1"
- "0.6.0"
- "0.11.0"
- "0.6.5"
- name: Sorted list
debug:
msg: "{{ versions | sort }}"
- name: Show the latest element
debug:
msg: "{{ versions | sort | last }}"
The playbook above works, as long as version numbers stay underneath the number 10 (e.g. 0.9.3, but not 0.10.2).
To show the issue:
TASK [Set version to compare] ***************************************************************************************************************
ok: [localhost]
TASK [Sorted list] **************************************************************************************************************************
ok: [localhost] => {
"msg": [
"0.1.0",
"0.1.5",
"0.10.2",
"0.11.0",
"0.11.11",
"0.6.0",
"0.6.1",
"0.6.5",
"0.9.11",
"0.9.3"
]
}
TASK [Show the latest element] **************************************************************************************************************
ok: [localhost] => {
"msg": "0.9.3"
}
In this example the value the desired value is 0.11.11
Does anyone know a good way to sort complex version numbers in Ansible?
Any help would be appreciated. Thanks.
An option would be to write a filter plugin, e.g.
shell> cat filter_plugins/sort_versions.py
from distutils.version import LooseVersion
def filter_sort_versions(value):
return sorted(value, key=LooseVersion)
class FilterModule(object):
filter_sort = {
'sort_versions': filter_sort_versions,
}
def filters(self):
return self.filter_sort
Then the task below
- debug:
msg: "{{ versions | sort_versions }}"
gives
msg:
- 0.1.0
- 0.1.5
- 0.6.0
- 0.6.1
- 0.6.5
- 0.9.3
- 0.9.11
- 0.10.2
- 0.11.0
- 0.11.11
You don't have to write the filter if you can install the collection community.general. Use the filter community.general.version_sort, e.g. the task below gives the same result
- debug:
msg: "{{ versions | community.general.version_sort }}"
You can use Jinja2 compare version instead of installing a filter plugin
- name: test
set_fact:
max_number: "{{ item }}"
when: max_number |default('0') is version(item, '<')
loop: "{{ master_version }}"
just follow Playbook Test Comparing versions.
Part of what you are looking for is the version test, however, it is built on the idea that a user just wants one comparison. So, you'll need to do some glue to find the "latest" one:
- set_fact:
max_version: >-
{%- set vmax = {} -%}
{%- for v_1 in versions -%}
{%- for v_2 in versions -%}
{%- if v_1 is version(v_2, ">") and v_1 is version(vmax.get("max", "0.0.0"), ">") -%}
{%- set _ = vmax.update({"max": v_1}) -%}
{%- endif -%}
{%- endfor -%}
{%- endfor -%}
{{ vmax.max }}
(I don't pretend that's the optimal solution, as it is very likely comparing versions against each other more than once, but it should work fine for small version lists)
I had a similar use case where I had a version and needed to obtain the previous version from a list of versions.
---
- name: previous_version_filter
hosts: localhost
gather_facts: false
vars:
versions:
- "21.7.1"
- "21.13.0"
- "21.7.2"
- "21.13.1"
- "21.8.0"
- "21.7.0"
version: "21.13.0"
newer_versions: []
tasks:
- name: Create newer_versions list
set_fact:
newer_versions: "{{ newer_versions + [item] }}"
when: item is version(version, '>')
loop: "{{ versions }}"
- name: Create previous_versions list
set_fact:
previous_versions: "{{ versions | difference(newer_versions) | difference([ version ]) }}"
- name: Obtain previous_version
set_fact:
previous_version: "{{ item }}"
when: previous_version | default('0') is version(item, '<')
loop: "{{ previous_versions }}"
- debug:
msg: "{{ previous_version }}"

Resources