How to loop multiple tasks in Ansible? [duplicate] - ansible

I need to check if a file named deploy.db exists. If it does not exist, I need to perform a set of tasks for which I am using a block.
Below is how I run the playbook
ansible-playbook test.yml \
-e Layer=APP \
-e BASEPATH="/logs" \
-e Filenames="file1,file2,file3"
Here is the playbook test.yml:
---
- name: "Play 1"
hosts: localhost
gather_facts: false
tasks:
- name: Construct
debug:
msg: "Run"
- block:
- stat: path="{{ BASEPATH }}/deploy.db"
register: currdb
- file: path="{{ BASEPATH }}/deploy.db" state=touch recurse=no
when: currdb.stat.exists == False
- shell: "echo done>>{{ BASEPATH }}/deploy.db"
when: currdb.stat.exists == False
when: Layer == 'APP'
with_items:
- "{{ Filenames.split(',') }}"
I am getting the below error running the playbook:
ERROR! 'with_items' is not a valid attribute for a Block
The error appears to be in '/app/test.yml': line 9, column 6, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- block:
^ here
After researching a bit, I understand that neither with_items nor loop is supported by a block and the solution is to include a tasks file.
I am, however, not sure how to get that to work. Can you suggest what tweaks I need in order to make my playbook work?
Considering I am on the latest version of Ansible, are there other solutions?

TL;DR
'with_items' is not a valid attribute for a Block
The error message says it all: you cannot loop over a block.
If you need to loop over a set of tasks, put them in a separate file and use include_tasks
Implementation (and some good practice...)
Below is an implementation based on your example illustrating the solution.
Since your question and code lacks some precision and since I pointed out some bad practices, please note that:
I fixed the looped code to effectively use the filenames you loop on (I inferred it was supposed to the deploy.db file). Note the use of loop_control to disambiguate the variable name in the included file (i.e. db_filename).
I made the code idempotent as much as possible by using the ansible module copy in place of shell and dropped the touch phase.
I transformed the var names to all lowercase and underscore separator.
To make sure the copy task works on all occasion, I replaced the removed tasks with a single making sure the basepath dir exists.
I added a unique filter after filenames.split(',') as well as a trim filter on each value to remove possible duplicates and eventual spaces added by error in the coma separated list.
I used not keyword and bool filter (for extra security) rather than a bare compare to a boolean False value.
Here is the included file create_db_each.yml
---
- name: Check if file exists
stat:
path: "{{ basepath }}/{{ db_filename }}"
register: currdb
- name: Create the file with "done" line if not present
copy:
content: "done"
dest: "{{ basepath }}/{{ db_filename }}"
when: not currdb.stat.exists | bool
used in the following create_db.yml playbook
---
- name: "Create my dbs"
hosts: localhost
gather_facts: false
tasks:
- name: Make sure the base directory exists
file:
path: "{{ basepath }}"
state: directory
- name: load each db
include_tasks: "create_db_each.yml"
when: layer == 'APP'
loop: "{{ filenames.split(',') | unique | map('trim') }}"
loop_control:
loop_var: db_filename
which gives
notes:
first run only, run it again on your side to witness it reports OK everywhere
see the filenames parameter value to illustrate the use of unique and trim
$ ansible-playbook -e basepath=/tmp/my/base/path -e "filenames='a.bla, b.toto, c , z.txt,a.bla'" -e layer=APP create_db.yml
PLAY [Create my dbs] ************************************************
TASK [Make sure the base directory exists] **************************
changed: [localhost]
TASK [load each db] *************************************************
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=a.bla)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=b.toto)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=c)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=z.txt)
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
PLAY RECAP **********************************************************
localhost: ok=13 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ tree /tmp/my/base/path/
/tmp/my/base/path/
├── a.bla
├── b.toto
├── c
└── z.txt
$ for f in /tmp/my/base/path/*; do cat $f; echo; done
done
done
done
done

Related

Ansible: What is the variable order/precedence in case of multiple 'extra_vars' files?

In this older question about "Can extra_vars receive multiple files?", the original poster answered the question, saying that multiple vars files could be accomplished by just using multiple --extra-vars parameters.
The followup question that I have is that, in such a case, where the ansible-playbook command line has two --extra-vars parameters, each pointing to a different file, what is the order or precedence of those files?
Also, what happens if both files have the same var name (e.g., my_host) in them?
For example, say I have 2 files, extraVars1.yml and extraVars2.yml and in the ansible-playbook command line I have:
ansible-playbook... --extra-vars "#extraVars1.yml" --extra-vars "#extraVars2.yml"
and the extraVars1.yml file has:
my_host: 1.2.3.4
and the extraVars2.yml file has:
my_host: 5.6.7.8
What will the value of the my_host var be when the playbook is run?
Thanks!
Jim
According the Ansible documentation about Using Variables and Understanding variable precedence
extra vars (for example, -e "user=my_user") (always win precedence)
In general, Ansible gives precedence to variables that were defined more recently ...
This means the last defined wins.
Lets have a short test here with a vars.yml playbook.
---
- hosts: localhost
become: false
gather_facts: false
vars:
my_host: 9.0.0.0
tasks:
- name: Show value
debug:
msg: "{{ my_host }}"
The execution of ansible-playbook vars.yml will result into an output of
TASK [Show value] ***
ok: [localhost] =>
msg: 9.0.0.0
The execution of ansible-playbook -e "#extraVars1.yml" vars.yml will result into an output of
TASK [Show value] ***
ok: [localhost] =>
msg: 1.2.3.4
The execution of ansible-playbook -e "#extraVars1.yml" -e "#extraVars2.yml" vars.yml will result into an output of
TASK [Show value] ***
ok: [localhost] =>
msg: 5.6.7.8
The execution of ansible-playbook -e "#extraVars2.yml" -e "#extraVars1.yml" vars.yml will result into an output of
TASK [Show value] ***
ok: [localhost] =>
msg: 1.2.3.4

Execute all yaml files from different directory

I have a directory I have created with several sub-tasks but I'm having trouble in making Ansible run all tasks from inside the specified directory.
The script looks like this:
---
- hosts: localhost
connection: local
tasks:
# tasks file for desktop
- name: "LOADING ALL TASKS FROM THE 'SUB_TASKS' DIRECTORY"
include_vars:
dir: sub_tasks
extensions:
- 'yml'
And this is the output:
plbchk main.yml --check
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not
match 'all'
PLAY [localhost] ****************************************************************************************************
TASK [Gathering Facts] **********************************************************************************************
ok: [localhost]
TASK [LOADING ALL TASKS FROM THE 'SUB_TASKS' DIRECTORY] *************************************************************
fatal: [localhost]: FAILED! => {"ansible_facts": {}, "ansible_included_var_files": [], "changed": false, "message": "/home/user/Documents/ansible-roles/desktop/tasks/sub_tasks/gnome_tweaks.yml must be stored as a dictionary/hash"}
PLAY RECAP **********************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
I've tried all sorts of ways to make it run the sub-tasks but to no avail.
I'd like to do it this way instead of creating one big file containing all the tasks. Is this possible?
include_vars is not for including tasks , this is to include vars (as its name suggest). Also If you check the error message it says "must be stored as a dictionary/hash.
fatal: [localhost]: FAILED! => {"ansible_facts": {}, "ansible_included_var_files": [], "changed": false, "message": "/home/user/Documents/ansible-roles/desktop/tasks/sub_tasks/gnome_tweaks.yml must be stored as a dictionary/hash"}
Solution:
You need to use include_taskfor what you are trying. Check out here.
Here is a complete/minimal working example, here we are making a list of yaml or yml files present in a provided directory and then running include_tasks over loop for all the files.
---
- name: Sample playbook
connection: local
gather_facts: false
hosts: localhost
tasks:
- name: Find all the yaml files in the directory
find:
paths: /home/user/Documents/ansible-roles/desktop/tasks
patterns: '*.yaml,*.yml'
recurse: yes
register: file_list
- name: show the yaml files present
debug: msg="{{ item }}"
loop: "{{ file_list.files | map(attribute='path') | list }}"
- name: Include task list in play
include_tasks: "{{ item }}"
loop: "{{ file_list.files | map(attribute='path') | list }}"

issue with ansible fetch module

I am using the below Ansible code to get the file system details (NAME,MOUNTPOINT,FSTYPE,SIZE) from node servers to control server. I am not getting any issues while running the playbook. But the CSV file is not copied to control machine.
Can anyone please help me on this?
tasks:
- name: Fsdetails
shell: |
lsblk -o NAME,MOUNTPOINT,FSTYPE,SIZE > $(hostname).csv
register: fsdetails_files_to_copy
- name: Fetch the fsdetails
fetch:
src: "{{ item }}"
dest: /data3/deployments/remediation
flat: yes
with_items: "{{ fsdetails_files_to_copy.stdout_lines }}"
Output:
PLAY [all] ************************************************************************************************
TASK [Gathering Facts] ************************************************************************************
ok: [10.xxx.xxx.xx]
TASK [Fsdetails] ******************************************************************************************
changed: [10.xxx.xxx.xx]
TASK [Fetch the fsdetails] ********************************************************************************
PLAY RECAP ************************************************************************************************
10.xxx.xxx.xx : ok=2 changed=1 unreachable=0 failed=0
Your shell command is not returning anything, since it is writing the output to the CSV file. Because of this, your fetch task has nothing to loop on (stdout_lines is an empty list).
What you could do is make your shell task echo the CSV name $(hostname):
- name: Fsdetails
shell: |
lsblk -o NAME,MOUNTPOINT,FSTYPE,SIZE > $(hostname).csv && echo $(hostname).csv
register: fsdetails_files_to_copy
This way, your fetch task will pick the correct filename to download.

Issue looping on block containing a set of tasks in Ansible

I need to check if a file named deploy.db exists. If it does not exist, I need to perform a set of tasks for which I am using a block.
Below is how I run the playbook
ansible-playbook test.yml \
-e Layer=APP \
-e BASEPATH="/logs" \
-e Filenames="file1,file2,file3"
Here is the playbook test.yml:
---
- name: "Play 1"
hosts: localhost
gather_facts: false
tasks:
- name: Construct
debug:
msg: "Run"
- block:
- stat: path="{{ BASEPATH }}/deploy.db"
register: currdb
- file: path="{{ BASEPATH }}/deploy.db" state=touch recurse=no
when: currdb.stat.exists == False
- shell: "echo done>>{{ BASEPATH }}/deploy.db"
when: currdb.stat.exists == False
when: Layer == 'APP'
with_items:
- "{{ Filenames.split(',') }}"
I am getting the below error running the playbook:
ERROR! 'with_items' is not a valid attribute for a Block
The error appears to be in '/app/test.yml': line 9, column 6, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- block:
^ here
After researching a bit, I understand that neither with_items nor loop is supported by a block and the solution is to include a tasks file.
I am, however, not sure how to get that to work. Can you suggest what tweaks I need in order to make my playbook work?
Considering I am on the latest version of Ansible, are there other solutions?
TL;DR
'with_items' is not a valid attribute for a Block
The error message says it all: you cannot loop over a block.
If you need to loop over a set of tasks, put them in a separate file and use include_tasks
Implementation (and some good practice...)
Below is an implementation based on your example illustrating the solution.
Since your question and code lacks some precision and since I pointed out some bad practices, please note that:
I fixed the looped code to effectively use the filenames you loop on (I inferred it was supposed to the deploy.db file). Note the use of loop_control to disambiguate the variable name in the included file (i.e. db_filename).
I made the code idempotent as much as possible by using the ansible module copy in place of shell and dropped the touch phase.
I transformed the var names to all lowercase and underscore separator.
To make sure the copy task works on all occasion, I replaced the removed tasks with a single making sure the basepath dir exists.
I added a unique filter after filenames.split(',') as well as a trim filter on each value to remove possible duplicates and eventual spaces added by error in the coma separated list.
I used not keyword and bool filter (for extra security) rather than a bare compare to a boolean False value.
Here is the included file create_db_each.yml
---
- name: Check if file exists
stat:
path: "{{ basepath }}/{{ db_filename }}"
register: currdb
- name: Create the file with "done" line if not present
copy:
content: "done"
dest: "{{ basepath }}/{{ db_filename }}"
when: not currdb.stat.exists | bool
used in the following create_db.yml playbook
---
- name: "Create my dbs"
hosts: localhost
gather_facts: false
tasks:
- name: Make sure the base directory exists
file:
path: "{{ basepath }}"
state: directory
- name: load each db
include_tasks: "create_db_each.yml"
when: layer == 'APP'
loop: "{{ filenames.split(',') | unique | map('trim') }}"
loop_control:
loop_var: db_filename
which gives
notes:
first run only, run it again on your side to witness it reports OK everywhere
see the filenames parameter value to illustrate the use of unique and trim
$ ansible-playbook -e basepath=/tmp/my/base/path -e "filenames='a.bla, b.toto, c , z.txt,a.bla'" -e layer=APP create_db.yml
PLAY [Create my dbs] ************************************************
TASK [Make sure the base directory exists] **************************
changed: [localhost]
TASK [load each db] *************************************************
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=a.bla)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=b.toto)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=c)
included: /home/olcla/Sources/ZZ_tests/ansitests/create_db_each.yml for localhost => (item=z.txt)
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
TASK [Check if file exists] *****************************************
ok: [localhost]
TASK [Create the file with "done" line if not present] **************
changed: [localhost]
PLAY RECAP **********************************************************
localhost: ok=13 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ tree /tmp/my/base/path/
/tmp/my/base/path/
├── a.bla
├── b.toto
├── c
└── z.txt
$ for f in /tmp/my/base/path/*; do cat $f; echo; done
done
done
done
done

Find Remote File Using Wildcard In Ansible

I run a java command using the shell module which creates a file with a random string suffixed to this.
I need to subsequently POST this file using the uri module.
I am trying to find the file using the stat module and a wildcard but it isn't finding it.
- stat:
path: "{{ my_dir }}/info-*"
register: info
- debug:
msg: "info isn't defined (path doesn't exist)"
when: info.stat.exists == False
How else can I find the filename?
The stat module requires a full path. Use the find module instead. Quoting:
paths List of paths of directories to search.
patterns One or more (shell or regex) patterns, which type is controlled by use_regex option.
The patterns restrict the list of files to be returned to those whose basenames match at least one of the patterns specified. Multiple patterns can be specified using a list. This parameter expects a list, ...
For example, find the info-* files in the directory /tmp/test and display the list of files
- find:
paths: /tmp/test
patterns:
- "info-*"
register: info
- debug:
var: info.files
Q: "I run a java command using the shell module which creates a file with a random string suffixed to this. I need to subsequently POST this file using the uri module."
A: It is possible to use the first file from the list
my_file: "{{ info.files.0.path }}"
, but there might be more files matching the pattern info-*. A robust solution would be to make the java command ... which creates a file with a random string suffixed to return the filename. Or, it might be possible to use the tempfile module instead.
Update
There is the creation time attribute ctime in each of the files from the list info.files. You can sort the list by ctime and take the last one created
my_files: "{{ info.files|sort(attribute='ctime')|map(attribute='path') }}"
my_file: "{{ my_files|last }}"
Be careful because this creates a race condition. Other processes may create newer matching files.
Given the tree
shell> tree /tmp/test
/tmp/test
├── info-1
├── info-2
└── info-3
and the example of a complete playbook for testing
- hosts: localhost
vars:
my_files: "{{ info.files|sort(attribute='ctime')|map(attribute='path') }}"
my_file: "{{ my_files|last }}"
tasks:
- find:
paths: /tmp/test
patterns:
- "info-*"
register: info
- debug:
var: my_files
- debug:
var: my_file
gives
PLAY [localhost] *****************************************************************************
TASK [find] **********************************************************************************
ok: [localhost]
TASK [debug] *********************************************************************************
ok: [localhost] =>
my_files:
- /tmp/test/info-1
- /tmp/test/info-2
- /tmp/test/info-3
TASK [debug] *********************************************************************************
ok: [localhost] =>
my_file: /tmp/test/info-3
PLAY RECAP ***********************************************************************************
localhost: ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Resources