Find Remote File Using Wildcard In Ansible - ansible

I run a java command using the shell module which creates a file with a random string suffixed to this.
I need to subsequently POST this file using the uri module.
I am trying to find the file using the stat module and a wildcard but it isn't finding it.
- stat:
path: "{{ my_dir }}/info-*"
register: info
- debug:
msg: "info isn't defined (path doesn't exist)"
when: info.stat.exists == False
How else can I find the filename?

The stat module requires a full path. Use the find module instead. Quoting:
paths List of paths of directories to search.
patterns One or more (shell or regex) patterns, which type is controlled by use_regex option.
The patterns restrict the list of files to be returned to those whose basenames match at least one of the patterns specified. Multiple patterns can be specified using a list. This parameter expects a list, ...
For example, find the info-* files in the directory /tmp/test and display the list of files
- find:
paths: /tmp/test
patterns:
- "info-*"
register: info
- debug:
var: info.files
Q: "I run a java command using the shell module which creates a file with a random string suffixed to this. I need to subsequently POST this file using the uri module."
A: It is possible to use the first file from the list
my_file: "{{ info.files.0.path }}"
, but there might be more files matching the pattern info-*. A robust solution would be to make the java command ... which creates a file with a random string suffixed to return the filename. Or, it might be possible to use the tempfile module instead.
Update
There is the creation time attribute ctime in each of the files from the list info.files. You can sort the list by ctime and take the last one created
my_files: "{{ info.files|sort(attribute='ctime')|map(attribute='path') }}"
my_file: "{{ my_files|last }}"
Be careful because this creates a race condition. Other processes may create newer matching files.
Given the tree
shell> tree /tmp/test
/tmp/test
├── info-1
├── info-2
└── info-3
and the example of a complete playbook for testing
- hosts: localhost
vars:
my_files: "{{ info.files|sort(attribute='ctime')|map(attribute='path') }}"
my_file: "{{ my_files|last }}"
tasks:
- find:
paths: /tmp/test
patterns:
- "info-*"
register: info
- debug:
var: my_files
- debug:
var: my_file
gives
PLAY [localhost] *****************************************************************************
TASK [find] **********************************************************************************
ok: [localhost]
TASK [debug] *********************************************************************************
ok: [localhost] =>
my_files:
- /tmp/test/info-1
- /tmp/test/info-2
- /tmp/test/info-3
TASK [debug] *********************************************************************************
ok: [localhost] =>
my_file: /tmp/test/info-3
PLAY RECAP ***********************************************************************************
localhost: ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Related

How to loop multiple tasks in Ansible? [duplicate]

I need to check if a file named deploy.db exists. If it does not exist, I need to perform a set of tasks for which I am using a block.
Below is how I run the playbook
ansible-playbook test.yml \
-e Layer=APP \
-e BASEPATH="/logs" \
-e Filenames="file1,file2,file3"
Here is the playbook test.yml:
---
- name: "Play 1"
hosts: localhost
gather_facts: false
tasks:
- name: Construct
debug:
msg: "Run"
- block:
- stat: path="{{ BASEPATH }}/deploy.db"
register: currdb
- file: path="{{ BASEPATH }}/deploy.db" state=touch recurse=no
when: currdb.stat.exists == False
- shell: "echo done>>{{ BASEPATH }}/deploy.db"
when: currdb.stat.exists == False
when: Layer == 'APP'
with_items:
- "{{ Filenames.split(',') }}"
I am getting the below error running the playbook:
ERROR! 'with_items' is not a valid attribute for a Block
The error appears to be in '/app/test.yml': line 9, column 6, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- block:
^ here
After researching a bit, I understand that neither with_items nor loop is supported by a block and the solution is to include a tasks file.
I am, however, not sure how to get that to work. Can you suggest what tweaks I need in order to make my playbook work?
Considering I am on the latest version of Ansible, are there other solutions?
TL;DR
'with_items' is not a valid attribute for a Block
The error message says it all: you cannot loop over a block.
If you need to loop over a set of tasks, put them in a separate file and use include_tasks
Implementation (and some good practice...)
Below is an implementation based on your example illustrating the solution.
Since your question and code lacks some precision and since I pointed out some bad practices, please note that:
I fixed the looped code to effectively use the filenames you loop on (I inferred it was supposed to the deploy.db file). Note the use of loop_control to disambiguate the variable name in the included file (i.e. db_filename).
I made the code idempotent as much as possible by using the ansible module copy in place of shell and dropped the touch phase.
I transformed the var names to all lowercase and underscore separator.
To make sure the copy task works on all occasion, I replaced the removed tasks with a single making sure the basepath dir exists.
I added a unique filter after filenames.split(',') as well as a trim filter on each value to remove possible duplicates and eventual spaces added by error in the coma separated list.
I used not keyword and bool filter (for extra security) rather than a bare compare to a boolean False value.
Here is the included file create_db_each.yml
---
- name: Check if file exists
stat:
path: "{{ basepath }}/{{ db_filename }}"
register: currdb
- name: Create the file with "done" line if not present
copy:
content: "done"
dest: "{{ basepath }}/{{ db_filename }}"
when: not currdb.stat.exists | bool
used in the following create_db.yml playbook
---
- name: "Create my dbs"
hosts: localhost
gather_facts: false
tasks:
- name: Make sure the base directory exists
file:
path: "{{ basepath }}"
state: directory
- name: load each db
include_tasks: "create_db_each.yml"
when: layer == 'APP'
loop: "{{ filenames.split(',') | unique | map('trim') }}"
loop_control:
loop_var: db_filename
which gives
notes:
first run only, run it again on your side to witness it reports OK everywhere
see the filenames parameter value to illustrate the use of unique and trim
$ ansible-playbook -e basepath=/tmp/my/base/path -e "filenames='a.bla, b.toto, c , z.txt,a.bla'" -e layer=APP create_db.yml
PLAY [Create my dbs] ************************************************
TASK [Make sure the base directory exists] **************************
changed: [localhost]
TASK [load each db] *************************************************
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=a.bla)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=b.toto)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=c)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=z.txt)
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
PLAY RECAP **********************************************************
localhost: ok=13 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ tree /tmp/my/base/path/
/tmp/my/base/path/
├── a.bla
├── b.toto
├── c
└── z.txt
$ for f in /tmp/my/base/path/*; do cat $f; echo; done
done
done
done
done

a task is not mentioned as skipped if it is skipped [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
In a folder I have a playbook:
---
- hosts: cent8
gather_facts: true
become: yes
roles:
- Ansible-RHEL8-CIS-Benchmarks
The tasks are in "tasks" folder in the "Ansible-RHEL8-CIS-Benchmarks" roles folder (current folder, where above playbook resides in). The roles path is being configued in ansible.cfg file:
[defaults]
roles_path = ../
I execute tasks from "tasks/level-1/1.1.18.yml" (tasks tagged with "1.1.18" in this file)
---
- name: 1.1.18 - Ensure sticky bit is set on all world-writable directories - changed_when false
shell: df --local -P|awk 'NR>1{print $6}'|xargs -I {} find {} -xdev -type d \( -perm -0002 -a ! -perm -1000 \)
register: shell_output
changed_when: shell_output.stdout_lines | length > 0
tags:
- "1.1.18"
- name: 1.1.18 - Checking or Setting permissions on the world-writable directories
file:
path: "{{ item }}"
mode: a+t
with_items:
- "{{ shell_output.stdout_lines }}"
changed_when: shell_output.stdout_lines | length > 0
tags:
- "1.1.18"
Everything works fine but I don't understand why there is no note: "skipping: [192.168.170.222]
" beneath the last task "1.1.18 - Checking or Setting permissions on the world-writable directories]", which is being skipped (it is only mentioned in summary as skipped=1):
$ ansible-playbook playbook.yml -t 1.1.18
PLAY [cent8] ********************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************
ok: [192.168.170.222]
TASK [Ansible-RHEL8-CIS-Benchmarks : Preflight - Fail if host is not suitable for this benchmark] *******************************************************************************************************************************************
ok: [192.168.170.222]
TASK [Ansible-RHEL8-CIS-Benchmarks : 1.1.18 - Ensure sticky bit is set on all world-writable directories - changed_when false] **************************************************************************************************************
ok: [192.168.170.222]
TASK [Ansible-RHEL8-CIS-Benchmarks : 1.1.18 - Checking or Setting permissions on the world-writable directories] ****************************************************************************************************************************
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
192.168.170.222 : ok=3 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
The source of the tasks can be found on https://github.com/HarryHarcourt/Ansible-RHEL8-CIS-Benchmarks
I use the latest Ansible version ansible [core 2.11.3] installed with command:
python3.8 -m pip install --user ansible
Empiric testing
I'll let you dig in the source code if you want to understand why (I did not). As far as I am concerned the final result is strictly the same: the task isn't played.
From what I could test, when the argument to loop/with_items/with_list is a jinja2 templated empty list, ansible does not even bother to enter the loop and simply displays the task title.
If you pass a static empty list then the skipped info you are looking for is displayed. Note that passing a list of lists to with_items automatically flattens the first level and you then drop in the first scenario again, even with static arguments. See the item lookup documentation page for more info
IMO, the first behavior is the closest to what is really happening: I have no elements in my loop so I'm not even skipping any iteration since there are none.
Here is my ansible version:
$ ansible --version
ansible [core 2.11.3]
config file = None
configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0]
jinja version = 2.11.3
libyaml = True
This is the demo playbook I tested with:
- hosts: localhost
gather_facts: false
tasks:
- name: Loop on a templated empty list
debug:
msg: "I'm doing nothing and I won't be called"
with_items: "{{ [] }}"
# You get the same behavior with
# loop: "{{ [] }}"
# with_list: "{{ [] }}"
- name: Loop on a statically passed empty list
debug:
msg: "I'm lazy as well so no one calls me either"
with_items: []
# You get the same behavior with
# loop: []
# with_list: []
- name: Demonstrating automagic flatenning on item lookup with list of lists
(same behavior as templated empty list)
debug:
msg: "Please leave me alone, don't call me!"
with_items:
- []
- []
Which gives:
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Loop on a templated empty list] **************************************************************************************************************************************************************************************************
TASK [Loop on a statically passed empty list] ******************************************************************************************************************************************************************************************
skipping: [localhost]
TASK [Demonstrating automagic flatenning on item lookup with list of lists (same behavior as templated empty list)] *******************************************************************************************************************
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=0 changed=0 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0

Execute all yaml files from different directory

I have a directory I have created with several sub-tasks but I'm having trouble in making Ansible run all tasks from inside the specified directory.
The script looks like this:
---
- hosts: localhost
connection: local
tasks:
# tasks file for desktop
- name: "LOADING ALL TASKS FROM THE 'SUB_TASKS' DIRECTORY"
include_vars:
dir: sub_tasks
extensions:
- 'yml'
And this is the output:
plbchk main.yml --check
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not
match 'all'
PLAY [localhost] ****************************************************************************************************
TASK [Gathering Facts] **********************************************************************************************
ok: [localhost]
TASK [LOADING ALL TASKS FROM THE 'SUB_TASKS' DIRECTORY] *************************************************************
fatal: [localhost]: FAILED! => {"ansible_facts": {}, "ansible_included_var_files": [], "changed": false, "message": "/home/user/Documents/ansible-roles/desktop/tasks/sub_tasks/gnome_tweaks.yml must be stored as a dictionary/hash"}
PLAY RECAP **********************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
I've tried all sorts of ways to make it run the sub-tasks but to no avail.
I'd like to do it this way instead of creating one big file containing all the tasks. Is this possible?
include_vars is not for including tasks , this is to include vars (as its name suggest). Also If you check the error message it says "must be stored as a dictionary/hash.
fatal: [localhost]: FAILED! => {"ansible_facts": {}, "ansible_included_var_files": [], "changed": false, "message": "/home/user/Documents/ansible-roles/desktop/tasks/sub_tasks/gnome_tweaks.yml must be stored as a dictionary/hash"}
Solution:
You need to use include_taskfor what you are trying. Check out here.
Here is a complete/minimal working example, here we are making a list of yaml or yml files present in a provided directory and then running include_tasks over loop for all the files.
---
- name: Sample playbook
connection: local
gather_facts: false
hosts: localhost
tasks:
- name: Find all the yaml files in the directory
find:
paths: /home/user/Documents/ansible-roles/desktop/tasks
patterns: '*.yaml,*.yml'
recurse: yes
register: file_list
- name: show the yaml files present
debug: msg="{{ item }}"
loop: "{{ file_list.files | map(attribute='path') | list }}"
- name: Include task list in play
include_tasks: "{{ item }}"
loop: "{{ file_list.files | map(attribute='path') | list }}"

Ansible: Save host names and results of linux command in dictionary format

I want to save the hosts name and results of linux command in the dictionary format. The problem is I cannot successfully get the dictionary format, and the new line stored in the results.txt file will replace the previous lines.
---
- hosts: "{{variable_host | default('lsbxdmss001')}}"
tasks:
- name: Check Redhat version for selected servers
shell:
cmd: rpm --query redhat-release-server
warn: False
register: myshell_output
- debug: var=myshell_output
- name: set fact
set_fact: output = "{{item.0}}:{{item.1}}"
with_together:
- groups['{{variable_host}}']
- "{{myshell_output.stdout}}"
register: output
- debug: var=output
- name: copy the output to results.txt
copy:
content: "{{output}}"
dest: results.txt
delegate_to: localhost
Looks like you have four issues:
Your dictionary creation seems odd, a dictionary in python is enclosed in curly bracers { ... }
So you line dictionary
"{{item.0}}:{{item.1}}"
Should rather be
"{{ {item.0: item.1} }}"
Adding to a file, in Ansible, is done via the module lineinfile rather than the copy one.
Your loop doesn't really makes sense, as Ansible is already executing each task on all hosts, so the reason you have only one information is also coming from this: you keep on overriding the output fact, in your trial to do it.
For the same reason as above, instead of your with_together loop, you should use the special variable inventory_hostname, as this is
The inventory name for the ‘current’ host being iterated over in the play
Source: https://docs.ansible.com/ansible/latest/reference_appendices/special_variables.html#special-variables
So after your shell command, here should be the next task:
- lineinfile:
path: results.txt
line: "{{ {inventory_hostname: myshell_output.stdout} | string }}"
create: yes
delegate_to: localhost
Mind that the string filter was added in order to convert the dictionary into a string to write it into the file without Ansible issuing a warning.
An example of playbook would be:
- hosts: all
gather_facts: no
tasks:
- name: Creating a fake shell result
shell:
cmd: echo 'Linux Vlersion XYZ {{ inventory_hostname }}'
register: shell_output
- lineinfile:
path: results.txt
line: "{{ {inventory_hostname: shell_output.stdout} | string }}"
create: yes
delegate_to: localhost
Which gives the recap:
PLAY [all] ******************************************************************************************************************************************
TASK [Creating a fake shell result] *****************************************************************************************************************
changed: [host2]
changed: [host1]
TASK [lineinfile] ***********************************************************************************************************************************
changed: [host1 -> localhost]
changed: [host2 -> localhost]
PLAY RECAP ******************************************************************************************************************************************
host1 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host2 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
And fill the results.txt file with:
{'host1': 'Linux Vlersion XYZ host1'}
{'host2': 'Linux Vlersion XYZ host2'}

Issue looping on block containing a set of tasks in Ansible

I need to check if a file named deploy.db exists. If it does not exist, I need to perform a set of tasks for which I am using a block.
Below is how I run the playbook
ansible-playbook test.yml \
-e Layer=APP \
-e BASEPATH="/logs" \
-e Filenames="file1,file2,file3"
Here is the playbook test.yml:
---
- name: "Play 1"
hosts: localhost
gather_facts: false
tasks:
- name: Construct
debug:
msg: "Run"
- block:
- stat: path="{{ BASEPATH }}/deploy.db"
register: currdb
- file: path="{{ BASEPATH }}/deploy.db" state=touch recurse=no
when: currdb.stat.exists == False
- shell: "echo done>>{{ BASEPATH }}/deploy.db"
when: currdb.stat.exists == False
when: Layer == 'APP'
with_items:
- "{{ Filenames.split(',') }}"
I am getting the below error running the playbook:
ERROR! 'with_items' is not a valid attribute for a Block
The error appears to be in '/app/test.yml': line 9, column 6, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- block:
^ here
After researching a bit, I understand that neither with_items nor loop is supported by a block and the solution is to include a tasks file.
I am, however, not sure how to get that to work. Can you suggest what tweaks I need in order to make my playbook work?
Considering I am on the latest version of Ansible, are there other solutions?
TL;DR
'with_items' is not a valid attribute for a Block
The error message says it all: you cannot loop over a block.
If you need to loop over a set of tasks, put them in a separate file and use include_tasks
Implementation (and some good practice...)
Below is an implementation based on your example illustrating the solution.
Since your question and code lacks some precision and since I pointed out some bad practices, please note that:
I fixed the looped code to effectively use the filenames you loop on (I inferred it was supposed to the deploy.db file). Note the use of loop_control to disambiguate the variable name in the included file (i.e. db_filename).
I made the code idempotent as much as possible by using the ansible module copy in place of shell and dropped the touch phase.
I transformed the var names to all lowercase and underscore separator.
To make sure the copy task works on all occasion, I replaced the removed tasks with a single making sure the basepath dir exists.
I added a unique filter after filenames.split(',') as well as a trim filter on each value to remove possible duplicates and eventual spaces added by error in the coma separated list.
I used not keyword and bool filter (for extra security) rather than a bare compare to a boolean False value.
Here is the included file create_db_each.yml
---
- name: Check if file exists
stat:
path: "{{ basepath }}/{{ db_filename }}"
register: currdb
- name: Create the file with "done" line if not present
copy:
content: "done"
dest: "{{ basepath }}/{{ db_filename }}"
when: not currdb.stat.exists | bool
used in the following create_db.yml playbook
---
- name: "Create my dbs"
hosts: localhost
gather_facts: false
tasks:
- name: Make sure the base directory exists
file:
path: "{{ basepath }}"
state: directory
- name: load each db
include_tasks: "create_db_each.yml"
when: layer == 'APP'
loop: "{{ filenames.split(',') | unique | map('trim') }}"
loop_control:
loop_var: db_filename
which gives
notes:
first run only, run it again on your side to witness it reports OK everywhere
see the filenames parameter value to illustrate the use of unique and trim
$ ansible-playbook -e basepath=/tmp/my/base/path -e "filenames='a.bla, b.toto, c , z.txt,a.bla'" -e layer=APP create_db.yml
PLAY [Create my dbs] ************************************************
TASK [Make sure the base directory exists] **************************
changed: [localhost]
TASK [load each db] *************************************************
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=a.bla)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=b.toto)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=c)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=z.txt)
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
PLAY RECAP **********************************************************
localhost: ok=13 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ tree /tmp/my/base/path/
/tmp/my/base/path/
├── a.bla
├── b.toto
├── c
└── z.txt
$ for f in /tmp/my/base/path/*; do cat $f; echo; done
done
done
done
done

Resources