Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
In a folder I have a playbook:
---
- hosts: cent8
gather_facts: true
become: yes
roles:
- Ansible-RHEL8-CIS-Benchmarks
The tasks are in "tasks" folder in the "Ansible-RHEL8-CIS-Benchmarks" roles folder (current folder, where above playbook resides in). The roles path is being configued in ansible.cfg file:
[defaults]
roles_path = ../
I execute tasks from "tasks/level-1/1.1.18.yml" (tasks tagged with "1.1.18" in this file)
---
- name: 1.1.18 - Ensure sticky bit is set on all world-writable directories - changed_when false
shell: df --local -P|awk 'NR>1{print $6}'|xargs -I {} find {} -xdev -type d \( -perm -0002 -a ! -perm -1000 \)
register: shell_output
changed_when: shell_output.stdout_lines | length > 0
tags:
- "1.1.18"
- name: 1.1.18 - Checking or Setting permissions on the world-writable directories
file:
path: "{{ item }}"
mode: a+t
with_items:
- "{{ shell_output.stdout_lines }}"
changed_when: shell_output.stdout_lines | length > 0
tags:
- "1.1.18"
Everything works fine but I don't understand why there is no note: "skipping: [192.168.170.222]
" beneath the last task "1.1.18 - Checking or Setting permissions on the world-writable directories]", which is being skipped (it is only mentioned in summary as skipped=1):
$ ansible-playbook playbook.yml -t 1.1.18
PLAY [cent8] ********************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************
ok: [192.168.170.222]
TASK [Ansible-RHEL8-CIS-Benchmarks : Preflight - Fail if host is not suitable for this benchmark] *******************************************************************************************************************************************
ok: [192.168.170.222]
TASK [Ansible-RHEL8-CIS-Benchmarks : 1.1.18 - Ensure sticky bit is set on all world-writable directories - changed_when false] **************************************************************************************************************
ok: [192.168.170.222]
TASK [Ansible-RHEL8-CIS-Benchmarks : 1.1.18 - Checking or Setting permissions on the world-writable directories] ****************************************************************************************************************************
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
192.168.170.222 : ok=3 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
The source of the tasks can be found on https://github.com/HarryHarcourt/Ansible-RHEL8-CIS-Benchmarks
I use the latest Ansible version ansible [core 2.11.3] installed with command:
python3.8 -m pip install --user ansible
Empiric testing
I'll let you dig in the source code if you want to understand why (I did not). As far as I am concerned the final result is strictly the same: the task isn't played.
From what I could test, when the argument to loop/with_items/with_list is a jinja2 templated empty list, ansible does not even bother to enter the loop and simply displays the task title.
If you pass a static empty list then the skipped info you are looking for is displayed. Note that passing a list of lists to with_items automatically flattens the first level and you then drop in the first scenario again, even with static arguments. See the item lookup documentation page for more info
IMO, the first behavior is the closest to what is really happening: I have no elements in my loop so I'm not even skipping any iteration since there are none.
Here is my ansible version:
$ ansible --version
ansible [core 2.11.3]
config file = None
configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0]
jinja version = 2.11.3
libyaml = True
This is the demo playbook I tested with:
- hosts: localhost
gather_facts: false
tasks:
- name: Loop on a templated empty list
debug:
msg: "I'm doing nothing and I won't be called"
with_items: "{{ [] }}"
# You get the same behavior with
# loop: "{{ [] }}"
# with_list: "{{ [] }}"
- name: Loop on a statically passed empty list
debug:
msg: "I'm lazy as well so no one calls me either"
with_items: []
# You get the same behavior with
# loop: []
# with_list: []
- name: Demonstrating automagic flatenning on item lookup with list of lists
(same behavior as templated empty list)
debug:
msg: "Please leave me alone, don't call me!"
with_items:
- []
- []
Which gives:
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Loop on a templated empty list] **************************************************************************************************************************************************************************************************
TASK [Loop on a statically passed empty list] ******************************************************************************************************************************************************************************************
skipping: [localhost]
TASK [Demonstrating automagic flatenning on item lookup with list of lists (same behavior as templated empty list)] *******************************************************************************************************************
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=0 changed=0 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
Related
I need to check if a file named deploy.db exists. If it does not exist, I need to perform a set of tasks for which I am using a block.
Below is how I run the playbook
ansible-playbook test.yml \
-e Layer=APP \
-e BASEPATH="/logs" \
-e Filenames="file1,file2,file3"
Here is the playbook test.yml:
---
- name: "Play 1"
hosts: localhost
gather_facts: false
tasks:
- name: Construct
debug:
msg: "Run"
- block:
- stat: path="{{ BASEPATH }}/deploy.db"
register: currdb
- file: path="{{ BASEPATH }}/deploy.db" state=touch recurse=no
when: currdb.stat.exists == False
- shell: "echo done>>{{ BASEPATH }}/deploy.db"
when: currdb.stat.exists == False
when: Layer == 'APP'
with_items:
- "{{ Filenames.split(',') }}"
I am getting the below error running the playbook:
ERROR! 'with_items' is not a valid attribute for a Block
The error appears to be in '/app/test.yml': line 9, column 6, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- block:
^ here
After researching a bit, I understand that neither with_items nor loop is supported by a block and the solution is to include a tasks file.
I am, however, not sure how to get that to work. Can you suggest what tweaks I need in order to make my playbook work?
Considering I am on the latest version of Ansible, are there other solutions?
TL;DR
'with_items' is not a valid attribute for a Block
The error message says it all: you cannot loop over a block.
If you need to loop over a set of tasks, put them in a separate file and use include_tasks
Implementation (and some good practice...)
Below is an implementation based on your example illustrating the solution.
Since your question and code lacks some precision and since I pointed out some bad practices, please note that:
I fixed the looped code to effectively use the filenames you loop on (I inferred it was supposed to the deploy.db file). Note the use of loop_control to disambiguate the variable name in the included file (i.e. db_filename).
I made the code idempotent as much as possible by using the ansible module copy in place of shell and dropped the touch phase.
I transformed the var names to all lowercase and underscore separator.
To make sure the copy task works on all occasion, I replaced the removed tasks with a single making sure the basepath dir exists.
I added a unique filter after filenames.split(',') as well as a trim filter on each value to remove possible duplicates and eventual spaces added by error in the coma separated list.
I used not keyword and bool filter (for extra security) rather than a bare compare to a boolean False value.
Here is the included file create_db_each.yml
---
- name: Check if file exists
stat:
path: "{{ basepath }}/{{ db_filename }}"
register: currdb
- name: Create the file with "done" line if not present
copy:
content: "done"
dest: "{{ basepath }}/{{ db_filename }}"
when: not currdb.stat.exists | bool
used in the following create_db.yml playbook
---
- name: "Create my dbs"
hosts: localhost
gather_facts: false
tasks:
- name: Make sure the base directory exists
file:
path: "{{ basepath }}"
state: directory
- name: load each db
include_tasks: "create_db_each.yml"
when: layer == 'APP'
loop: "{{ filenames.split(',') | unique | map('trim') }}"
loop_control:
loop_var: db_filename
which gives
notes:
first run only, run it again on your side to witness it reports OK everywhere
see the filenames parameter value to illustrate the use of unique and trim
$ ansible-playbook -e basepath=/tmp/my/base/path -e "filenames='a.bla, b.toto, c , z.txt,a.bla'" -e layer=APP create_db.yml
PLAY [Create my dbs] ************************************************
TASK [Make sure the base directory exists] **************************
changed: [localhost]
TASK [load each db] *************************************************
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=a.bla)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=b.toto)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=c)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=z.txt)
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
PLAY RECAP **********************************************************
localhost: ok=13 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ tree /tmp/my/base/path/
/tmp/my/base/path/
├── a.bla
├── b.toto
├── c
└── z.txt
$ for f in /tmp/my/base/path/*; do cat $f; echo; done
done
done
done
done
I want to save the hosts name and results of linux command in the dictionary format. The problem is I cannot successfully get the dictionary format, and the new line stored in the results.txt file will replace the previous lines.
---
- hosts: "{{variable_host | default('lsbxdmss001')}}"
tasks:
- name: Check Redhat version for selected servers
shell:
cmd: rpm --query redhat-release-server
warn: False
register: myshell_output
- debug: var=myshell_output
- name: set fact
set_fact: output = "{{item.0}}:{{item.1}}"
with_together:
- groups['{{variable_host}}']
- "{{myshell_output.stdout}}"
register: output
- debug: var=output
- name: copy the output to results.txt
copy:
content: "{{output}}"
dest: results.txt
delegate_to: localhost
Looks like you have four issues:
Your dictionary creation seems odd, a dictionary in python is enclosed in curly bracers { ... }
So you line dictionary
"{{item.0}}:{{item.1}}"
Should rather be
"{{ {item.0: item.1} }}"
Adding to a file, in Ansible, is done via the module lineinfile rather than the copy one.
Your loop doesn't really makes sense, as Ansible is already executing each task on all hosts, so the reason you have only one information is also coming from this: you keep on overriding the output fact, in your trial to do it.
For the same reason as above, instead of your with_together loop, you should use the special variable inventory_hostname, as this is
The inventory name for the ‘current’ host being iterated over in the play
Source: https://docs.ansible.com/ansible/latest/reference_appendices/special_variables.html#special-variables
So after your shell command, here should be the next task:
- lineinfile:
path: results.txt
line: "{{ {inventory_hostname: myshell_output.stdout} | string }}"
create: yes
delegate_to: localhost
Mind that the string filter was added in order to convert the dictionary into a string to write it into the file without Ansible issuing a warning.
An example of playbook would be:
- hosts: all
gather_facts: no
tasks:
- name: Creating a fake shell result
shell:
cmd: echo 'Linux Vlersion XYZ {{ inventory_hostname }}'
register: shell_output
- lineinfile:
path: results.txt
line: "{{ {inventory_hostname: shell_output.stdout} | string }}"
create: yes
delegate_to: localhost
Which gives the recap:
PLAY [all] ******************************************************************************************************************************************
TASK [Creating a fake shell result] *****************************************************************************************************************
changed: [host2]
changed: [host1]
TASK [lineinfile] ***********************************************************************************************************************************
changed: [host1 -> localhost]
changed: [host2 -> localhost]
PLAY RECAP ******************************************************************************************************************************************
host1 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host2 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
And fill the results.txt file with:
{'host1': 'Linux Vlersion XYZ host1'}
{'host2': 'Linux Vlersion XYZ host2'}
I need to check if a file named deploy.db exists. If it does not exist, I need to perform a set of tasks for which I am using a block.
Below is how I run the playbook
ansible-playbook test.yml \
-e Layer=APP \
-e BASEPATH="/logs" \
-e Filenames="file1,file2,file3"
Here is the playbook test.yml:
---
- name: "Play 1"
hosts: localhost
gather_facts: false
tasks:
- name: Construct
debug:
msg: "Run"
- block:
- stat: path="{{ BASEPATH }}/deploy.db"
register: currdb
- file: path="{{ BASEPATH }}/deploy.db" state=touch recurse=no
when: currdb.stat.exists == False
- shell: "echo done>>{{ BASEPATH }}/deploy.db"
when: currdb.stat.exists == False
when: Layer == 'APP'
with_items:
- "{{ Filenames.split(',') }}"
I am getting the below error running the playbook:
ERROR! 'with_items' is not a valid attribute for a Block
The error appears to be in '/app/test.yml': line 9, column 6, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- block:
^ here
After researching a bit, I understand that neither with_items nor loop is supported by a block and the solution is to include a tasks file.
I am, however, not sure how to get that to work. Can you suggest what tweaks I need in order to make my playbook work?
Considering I am on the latest version of Ansible, are there other solutions?
TL;DR
'with_items' is not a valid attribute for a Block
The error message says it all: you cannot loop over a block.
If you need to loop over a set of tasks, put them in a separate file and use include_tasks
Implementation (and some good practice...)
Below is an implementation based on your example illustrating the solution.
Since your question and code lacks some precision and since I pointed out some bad practices, please note that:
I fixed the looped code to effectively use the filenames you loop on (I inferred it was supposed to the deploy.db file). Note the use of loop_control to disambiguate the variable name in the included file (i.e. db_filename).
I made the code idempotent as much as possible by using the ansible module copy in place of shell and dropped the touch phase.
I transformed the var names to all lowercase and underscore separator.
To make sure the copy task works on all occasion, I replaced the removed tasks with a single making sure the basepath dir exists.
I added a unique filter after filenames.split(',') as well as a trim filter on each value to remove possible duplicates and eventual spaces added by error in the coma separated list.
I used not keyword and bool filter (for extra security) rather than a bare compare to a boolean False value.
Here is the included file create_db_each.yml
---
- name: Check if file exists
stat:
path: "{{ basepath }}/{{ db_filename }}"
register: currdb
- name: Create the file with "done" line if not present
copy:
content: "done"
dest: "{{ basepath }}/{{ db_filename }}"
when: not currdb.stat.exists | bool
used in the following create_db.yml playbook
---
- name: "Create my dbs"
hosts: localhost
gather_facts: false
tasks:
- name: Make sure the base directory exists
file:
path: "{{ basepath }}"
state: directory
- name: load each db
include_tasks: "create_db_each.yml"
when: layer == 'APP'
loop: "{{ filenames.split(',') | unique | map('trim') }}"
loop_control:
loop_var: db_filename
which gives
notes:
first run only, run it again on your side to witness it reports OK everywhere
see the filenames parameter value to illustrate the use of unique and trim
$ ansible-playbook -e basepath=/tmp/my/base/path -e "filenames='a.bla, b.toto, c , z.txt,a.bla'" -e layer=APP create_db.yml
PLAY [Create my dbs] ************************************************
TASK [Make sure the base directory exists] **************************
changed: [localhost]
TASK [load each db] *************************************************
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=a.bla)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=b.toto)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=c)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=z.txt)
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
PLAY RECAP **********************************************************
localhost: ok=13 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ tree /tmp/my/base/path/
/tmp/my/base/path/
├── a.bla
├── b.toto
├── c
└── z.txt
$ for f in /tmp/my/base/path/*; do cat $f; echo; done
done
done
done
done
I need to store in variable & print the output of "ls -ltr" for a set of files on remote host. This variable with be used by another play in the same yml.
I tried the following however, it just prints the filename and not the complete results of "ls -ltr" command.
My lstest.yml looks like below:
- name: Play 4- Configure nodes
hosts: remotehost
tasks:
- name: "Collecting APP file information"
command: ls -ltr /app/{{ vars[item.split('.')[1]] }}/{{ item | basename }}
register: fdetails_APP
when: Layer == 'APP'
with_fileglob:
- "{{ playbook_dir }}/tmpfiles/*"
- debug: var=fdetails_APP.stdout_lines
- set_fact: fdet_APP={{ fdetails_APP.stdout_lines }}
- name: Printing fpath
debug:
var: fdet_APP
Output:
TASK [Collecting APP file information]
************************************************************************************ changed: [localhost] =>
(item=/app/Ansible/playbook/tmpfiles/filename1.exe) changed:
[localhost] => (item=/app/Ansible/playbook/tmpfiles/33211.sql)
changed: [localhost] =>
(item=/app/Ansible/playbook/tmpfiles/file1.mrt) changed: [localhost]
=> (item=/app/Ansible/playbook/tmpfiles/filename1.src)
PLAY RECAP
************************************************************************************************************************************************** localhost : ok=3 changed=1 unreachable=0
failed=0 skipped=0 rescued=0 ignored=0
Can you please suggest.
Note. In near future I would also like to add the checksum of the files stored in the same variable.
As per fileglob documentation- "Matching is against local system files on the Ansible controller. To iterate a list of files on a remote node, use the find module." For remote host fileglob cannot be used.
I run a java command using the shell module which creates a file with a random string suffixed to this.
I need to subsequently POST this file using the uri module.
I am trying to find the file using the stat module and a wildcard but it isn't finding it.
- stat:
path: "{{ my_dir }}/info-*"
register: info
- debug:
msg: "info isn't defined (path doesn't exist)"
when: info.stat.exists == False
How else can I find the filename?
The stat module requires a full path. Use the find module instead. Quoting:
paths List of paths of directories to search.
patterns One or more (shell or regex) patterns, which type is controlled by use_regex option.
The patterns restrict the list of files to be returned to those whose basenames match at least one of the patterns specified. Multiple patterns can be specified using a list. This parameter expects a list, ...
For example, find the info-* files in the directory /tmp/test and display the list of files
- find:
paths: /tmp/test
patterns:
- "info-*"
register: info
- debug:
var: info.files
Q: "I run a java command using the shell module which creates a file with a random string suffixed to this. I need to subsequently POST this file using the uri module."
A: It is possible to use the first file from the list
my_file: "{{ info.files.0.path }}"
, but there might be more files matching the pattern info-*. A robust solution would be to make the java command ... which creates a file with a random string suffixed to return the filename. Or, it might be possible to use the tempfile module instead.
Update
There is the creation time attribute ctime in each of the files from the list info.files. You can sort the list by ctime and take the last one created
my_files: "{{ info.files|sort(attribute='ctime')|map(attribute='path') }}"
my_file: "{{ my_files|last }}"
Be careful because this creates a race condition. Other processes may create newer matching files.
Given the tree
shell> tree /tmp/test
/tmp/test
├── info-1
├── info-2
└── info-3
and the example of a complete playbook for testing
- hosts: localhost
vars:
my_files: "{{ info.files|sort(attribute='ctime')|map(attribute='path') }}"
my_file: "{{ my_files|last }}"
tasks:
- find:
paths: /tmp/test
patterns:
- "info-*"
register: info
- debug:
var: my_files
- debug:
var: my_file
gives
PLAY [localhost] *****************************************************************************
TASK [find] **********************************************************************************
ok: [localhost]
TASK [debug] *********************************************************************************
ok: [localhost] =>
my_files:
- /tmp/test/info-1
- /tmp/test/info-2
- /tmp/test/info-3
TASK [debug] *********************************************************************************
ok: [localhost] =>
my_file: /tmp/test/info-3
PLAY RECAP ***********************************************************************************
localhost: ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0