Ansible: variable in variable / loop or iterating through item - ansible

Here is a simple vars file we have to debug
./roles/test/vars/{{ ansible_distribution|lower }}/apt-packages.yml
packages:
required:
- htop
# - aptitude
package:
htop:
allow_unauthenticated: no
autoclean: no
autoremove: no
cache_valid_time: 0
# default_release:
force: no
force_apt_get: no
install_recommends: yes
only_upgrade: no
purge: no
state: latest
update_cache: yes
upgrade: no
Here is a simple task to debug
./roles/test/tasks/main.yml
- name: "Register variable"
include_vars:
#dir: vars/ubuntu
file: "vars/{{ ansible_distribution|lower }}/apt-packages.yml"
name: apt_install
- name: "This a test"
apt:
name: "{{item}}"
cache_valid_time: "{{ apt_install.package[item].cache_valid_time }}"
state: "{{ apt_install.package[item].state }}"
update_cache: "{{ apt_install.package[item].update_cache }}"
with_items: "{{ apt_install.packages.required }}"
./roles/test-playbook.yml
- name: "playbook test"
hosts: localhost
roles:
- role: test
become: true
become_user: root
become_method: sudo
using following answer stackoverflow.com/questions/29276198 we are trying to loop through items and get item related values.
Tasks is looping well through items but it is impossible to retrieve related variable with [item] syntax or any other one we have tested.
We have always the same error
fatal: [localhost]: FAILED! => {
"msg": "The task includes an option with an undefined variable. The error was: dict object has no element [u'htop']
But calling directly the variable works
- name: "echo variable test"
debug:
msg: "{{ apt_install.package.htop.allow_unauthenticated }}"
What is the right syntax to get current loop value of a variable and use it inside another variable to retrieve related value ... (inside the same task) ?
So far it's we who are going around in circles without end !
Kind Regards

This is fully working using a loop / loop_control instead of with_items
- name: "This a test"
apt:
name: "{{item}}"
cache_valid_time: "{{ apt_install.package[item].cache_valid_time }}"
state: "{{ apt_install.package[item].state }}"
update_cache: "{{ apt_install.package[item].update_cache }}"
loop: "{{ apt_install.packages.required|flatten(levels=1) }}"
loop_control:
index_var: index
So as it we can define different settings for each package and for each distrib.
Now I can export too settings for each package in different var files.
Learning is hard :)

Related

In Ansible loop, test existence of files from registered results

I have several files that I need to backup in different directories. I have tried the code below and not working for me.
vars:
file_vars:
- {name: /file1}
- {name: /etc/file2}
- {name: /etc/file/file3}
tasks:
- name: "Checking if config files exists"
stat:
path: "{{ item.name }}"
with_items: "{{ file_vars }}"
register: stat_result
- name: Backup Files
copy: src={{ item.name }} dest={{ item.name }}{{ ansible_date_time.date }}.bak
with_items: "{{ file_vars }}"
remote_src: yes
when: stat_result.stat.exists == True
The problem is the condition
when: stat_result.stat.exists == True
There is no attribute stat_result.stat. Instead, the attribute stat_result.results is a list of the results from the loop. It's possible to create a dictionary of files and their statuses. For example
- set_fact:
files_stats: "{{ dict(my_files|zip(my_stats)) }}"
vars:
my_files: "{{ stat_result.results|json_query('[].item.name') }}"
my_stats: "{{ stat_result.results|json_query('[].stat.exists') }}"
Then simply use this dictionary in the condition
when: files_stats[item.name]
Below is a shorter version which creates the dictionary more efficiently
- set_fact:
files_stats: "{{ dict(stat_result.results|
json_query('[].[item.name, stat.exists]')) }}"
Please try using below worked for me:
---
- name: Copy files
hosts: localhost
become: yes
become_user: root
vars_files:
- files.yml
tasks:
- name: "Checking if config files exists"
stat:
path: "{{ item }}"
with_items: "{{ files }}"
register: stat_result
- name: Ansible
debug:
msg: "{{ stat_result }}"
- name: Backup Files
copy:
src: "{{ item }}"
dest: "{{ item.bak }}"
with_items: "{{ files }}"
when: stat_result == "True"
and files.yml will look like:
---
files:
- /tmp/file1
- /tmp/file2
you can check you playbook syntax using below command:
ansible-playbook copy.yml --syntax-check
Also you do dry run your playbook before actual execution.
ansible-playbook -i localhost copy.yml --check

ansible - Incorrect type. Expected "object"

site.yaml
---
- name: someapp deployment playbook
hosts: localhost
connection: local
gather_facts: no
vars_files:
- secrets.yml
environment:
AWS_DEFAULT_REGION: "{{ lookup('env', 'AWS_DEFAULT_VERSION') | default('ca-central-1', true) }}"
tasks:
- include: tasks/create_stack.yml
- include: tasks/deploy_app.yml
create_stack.yml
---
- name: task to create/update stack
cloudformation:
stack_name: someapp
state: present
template: templates/stack.yml
template_format: yaml
template_parameters:
VpcId: "{{ vpc_id }}"
SubnetId: "{{ subnet_id }}"
KeyPair: "{{ ec2_keypair }}"
InstanceCount: "{{ instance_count | default(1) }}"
DbSubnets: "{{ db_subnets | join(',') }}"
DbAvailabilityZone: "{{ db_availability_zone }}"
DbUsername: "{{ db_username }}"
DbPassword: "{{ db_password }}"
tags:
Environment: test
register: cf_stack
- name: task to output stack output
debug: msg={{ cf_stack }}
when: debug is defined
Error at line debug: msg={{ cf_stack }} saying:
This module prints statements during execution and can be useful for debugging variables or expressions without necessarily halting the playbook. Useful for debugging together with the 'when:' directive.
This module is also supported for Windows targets.
Incorrect type. Expected "object".
Ansible documentation allows the above syntax, as shown here
$ ansible --version
ansible 2.5.1
....
How to resolve this error?
You still need to remember quotes for lines starting with a {, even when using short-hand notation:
- debug: msg="{{ cf_stack }}"
This would be more obvious using full YAML notation:
- debug:
msg: "{{ cf_stack }}"
Also, given this is a variable, you could just do:
- debug:
var: cf_stack

Ansible rollback: run a group of tasks over list of hosts even when one of hosts failed

I have a playbook with multiple roles, hosts and groups. I am trying to develop a rollback functionality, that would run over all hosts. My current obstacle is that I see no way to delegate role, block or set of tasks to group of hosts
I tried looking up delegation to group without loops, so it would work on a block.
import_role doesn't accept loops
include_role doesn't accept delegate_to
same with import_tasks/include_tasks
here is what I have now as a playbook file (shortened version)
- hosts: all
any_errors_fatal: true
vars_prompt:
- name: "remote_user_p"
prompt: "Remote user running the playbook"
default: "root"
private: no
- name: "service_user_p"
prompt: "Specify user to run non-root tasks"
default: "user"
private: no
tasks:
- set_fact:
playbook_type: "upgrade"
- import_role:
name: 0_pre_check
run_once: true
remote_user: "{{ remote_user_p }}"
become_user: "{{ service_user_p }}"
become_method: su
become: yes
- block:
- import_role:
name: 1_os
- import_role:
name: 2_mysql
when: inventory_hostname in groups['mysql'] | default("")
- import_role:
name: 3_web
when: inventory_hostname in groups['web'] | default("")
...
rescue:
- block:
- name: run rollback
import_tasks: ../common/roles/5_rollback/tasks/rollback.yml
remote_user: "{{ remote_user }}"
become_user: "{{ service_user }}"
become_method: su
become: yes
This is some example code from rollback.yml:
- block:
- name: rollback symlinks to config dir
file:
src: "{{ current_config_path }}"
dest: "{{ install_dir }}/static/cfg"
owner: "{{ service_user }}"
group: "{{ service_user_primary_group }}"
state: link
when: current_new_configs | default("N") == "Y"
delegate_to: "{{ item }}"
with_items:
- "{{ ansible_play_hosts }}"
- block:
- name: return config files
shell: test -f '{{ item.1.current_ver_file_path }}' && cp -p {{ item.1.current_ver_file_path }} {{ item.1.old_config_location }}
args:
warn: false
register: return_config_files
failed_when: return_config_files.rc >= 2
when:
- roolback_moved_cfg | default('N') == "Y"
- inventory_hostname in groups[item.0.group]
- item.1.old_config_location != ""
- item.1.current_ver_file_path != ""
with_subelements:
- "{{ config_files }}"
- files
become_user: root
become_method: sudo
become: yes
- name: systemctl daemon-reload
shell: systemctl daemon-reload
failed_when: false
when: root_rights == "Y"
args:
warn: false
delegate_to: "{{ item }}"
with_items:
- "{{ ansible_play_hosts }}"
when: root_rights == "Y"
become_user: root
become_method: sudo
become: yes
- fail:
msg: "Upgrade failed. Symbolic links were set to the previous version. Fix the issues and try again. If you wish to cancel the upgrade, restore the database backup manually."
As you can see, now I use lame workaround by introducing
delegate_to: "{{ item }}"
with_items:
- "{{ ansible_play_hosts }}"
after every task.
There are two problems here:
1. I can't use same approach after task return config files, because it already uses one loop
2. This is generally lame duplication of code and I hate it
Why I need it at all: if playbook execution fails somewhere in mysql role, for example, the rescue block will be executed only over the hosts in that mysql role (and btw, execution of tasks from next role will continue while running rescue block - same amount of tasks, despite all efforts), while I would like it to run over all hosts instead.
I finally was able to solve this with an ugly-ugly hack. Used plays instead of just roles - now there are more than 10 plays. Don't judge me, I spent lots of effort trying to make it nice ):
Example play followed by a check - same as for every other.
- hosts: mysql
any_errors_fatal: true
tasks:
- block:
- import_role:
name: 2_mysql
when: not rollback | default(false)
rescue:
- block:
- name: set fact for rollback
set_fact:
rollback: "yes"
delegate_to: "{{ item }}"
delegate_facts: true
with_items: "{{ groups['all'] }}"
- hosts: all
any_errors_fatal: true
tasks:
- name: run rollback
import_tasks: ../common/roles/5_rollback/tasks/rollback.yml
when: rollback | default(false)
include_role doesn't accept delegate_to
Actually, it does.
With ansible-core 2.8:
- name: "call my/role with host '{{ansible_hostname}}' for hosts in '{{ansible_play_hosts}}'"
include_role:
name: my/role
apply:
delegate_to: "{{current_host}}"
with_items: "{{ansible_play_hosts}}"
loop_control:
loop_var: current_host
With ansible-core 2.5 to 2.7, see "2.5: delegate_to, include_role with loops" from George Shuklin, mentioned in ansible/ansible issue 35398
- name: "call my/role with host '{{ansible_hostname}}' for items in '{{ansible_play_hosts}}'"
include_tasks: loop.yml
with_items: "{{ansible_play_hosts}}"
loop_control:
loop_var: current_host
With loop.yml another tasks in its own file:
- name: "Import my/role for '{{current_host}}'"
import_role: name=my/role
delegate_to: "{{current_host}}"
So in two files (with ansible-core 2.7) or one file (2.8), you can make a all role and its tasks run on a delegated server.

Iterate with Ansible with_dict over list of a dictionaries

I am stuck in iterating over the list of a dictionary. Sample vars.yml and the minimal playbook is bellow.
---
- hosts: localhost
connection: local
gather_facts: false
become: false
vars:
csvfile: "{{ lookup('file', 'vars/users.csv') }}"
tasks:
- name: Convert CSV to YAML
template:
src: "./users_csv.j2"
dest: "vars/users.yml"
run_once: true
- name: Include users from users.yml to users variable
include_vars:
file: vars/users.yml
name: users
- debug:
msg: "{{ users.value }}"
with_dict:
- "{{ users }}"
My Jinja2 template produces a list of dictionaries in YAML format as below:
--
users:
- username: Auser1
group: Admin
- username: Auser2
group: Admin
- username: Auser3
group: User
Anyhow, when I am iterating the dictionary, I am not able to get for example a username or group.
Most far I got is getting a fatal error message saying:
fatal: [localhost]: FAILED! => {"msg": "with_dict expects a dict"}
I know how to iterate over the list, but I don't have an idea why it fails here.
The users is not a dictionary, its a list variable of dictionaries.
if you want to parse this variable in a loop, you can use:
- debug:
msg: "username: {{ item.username }}, group: {{ item.group }}"
with_items:
- "{{ users.users }}"
hope it helps
UPDATE
i noticed now that when including the var file, you pass the name: users instruction as well. this cause all the variables of the file to be placed under the users variable. So to refer to the users list which is defined in the variable file, you need to use users.users.
updated the with_items to:
with_items:
- "{{ users.users }}"

Adding field to dict items

Consider the following play. What I am trying to do is add a field, tmp_path which is basically the key and revision appended together to each element in the scripts dict.
---
- hosts: localhost
connection: local
gather_facts: no
vars:
scripts:
a.pl:
revision: 123
b.pl:
revision: 456
tasks:
- with_dict: "{{ scripts }}"
debug:
msg: "{{ item.key }}_{{ item.value.revision }}"
# - with_items: "{{ scripts }}"
# set_fact: {{item.value.tmp_path}}="{{item.key}}_{{item.value.revision}}"
# - with_items: "{{ scripts }}"
# debug:
# msg: "{{ item.value.tmp_path }}"
...
Obviously the commented code doesn't work, any idea how I can get this working? Is it possible to alter the scripts dict directly, or should I somehow be creating a new dict to reference instead?
By the way welcome to correct the terminology for what I am trying to do.
OK, I think I got a solution (below), at least to let me move forwards with this. Disadvantages are it has removed the structure of my dict and also seems a bit redundant having to redefine all the fields and use a new variable, If anyone can provide a better solution I will accept that instead.
---
- hosts: localhost
connection: local
gather_facts: no
vars:
scripts:
a.pl:
revision: 123
b.pl:
revision: 456
tasks:
- with_dict: "{{ scripts }}"
debug:
msg: "{{ item.key }}_{{ item.value.revision }}"
- with_dict: "{{ scripts }}"
set_fact:
new_scripts: "{{ (new_scripts | default([])) + [ {'name': item.key, 'revision': item.value.revision, 'tmp_path': item.key ~ '_' ~ item.value.revision}] }}"
# - debug:
# var: x
# - with_dict: "{{ scripts }}"
- with_items: "{{ new_scripts }}"
debug:
msg: "{{ item.tmp_path }}"
...
BTW credit to the following question which pointed me in the right direction:
Using Ansible set_fact to create a dictionary from register results

Resources