The scenario is: I have several services running on several hosts. There is one special service - the reverseproxy/loadbalancer. Any service needs to configure that special service on the host, that runs the rp/lp service. During installation/update/deletion of a random service with an Ansible role, I need to call the ReverseProxy role on the specific host to configure the corresponding vhost.
At the moment I call a specific task file in the reverse proxy role to add or remove a vhost by the service with include_role and set some vars (very easy example without service and inventory specific vars).
- name: "Configure ReverseProxy"
include_role:
name: reverseproxy
tasks_from: vhost_add
apply:
delegate_to: "{{ groups['reverseproxy'][0] }}"
vars:
reverse_proxy_url: "http://{{ ansible_fqdn }}:{{ service_port }}/"
reverse_proxy_domain: "sub.domain.tld"
I have three problems.
I know, it's not a good idea to build such dependencies between roles and different hosts. I don't know a better way, especially if you think about the situation, where you need to do some extra stuff after creating the vhost (f.e. configure the service via REST API, which needs the external fqdn). In case of two separate playbooks with "backend"-service and "reverseproxy"-service - then I need a third playbook for configuring "hanging" services. Also I'm not sure, if I can retrieve the correct backend URL in the reverse proxy role (only think about the HTTP scheme or paths). That sounds not easy, or?
Earlier I had separate roles for adding/removing vhosts to a reverseproxy. This roles didn't have dependencies, but I needed to duplicate several defaults and templates and vars etc. which isn't nice too. Then I've changed that to a single role. Of course - in my opinion, this isn't really that, what a "role" should be. A role is something like "webserver" or "reverseproxy" (a state). But not something like "add_vhost_to_reverseproxy" (a verb). This would be something like a playbook - but is calling a parameterized playbook via a role a good idea/possible? The main problem is, that the state of reverseproxy is the sum of all services in the inventory.
In case of that single included role, including it, starts also all dependent roles (configure custom, firewall, etc.). Nevertheless in that case I found out, that the delegation did not use the facts of the delegated host.
I tested that with the following example - the inventory:
all:
hosts:
server1:
my_var: a
server2:
my_var: b
children:
service:
hosts:
server1:
reverseproxy:
hosts:
server2:
And playbook which assigns a role-a to the group webserver. The role-a has a task like:
- block:
- setup:
- name: "Include role b on delegated {{ groups['reverseproxy'][0] }}"
include_role:
name: role-b
delegate_to: "{{ groups['reverseproxy'][0] }}"
delegate_facts: true # or false or omit - it has no effect on Ansible 2.9 and 2.10
And in role-b only outputing the my_var of the inventory will output
TASK [role-b : My_Var on server1] *******************
ok: [server1 -> <ip-of-server2>] =>
my_var: a
Which says me, that role-b that should be run on server2 has the facts of server1. So - configuring the "reverseproxy" service is done in context of the "backend"-service. Which would have several other issues - when you think about firewall-dependencies etc. I can avoid that, by using tags - but then I need to run the playbook not just with the tag of the service, but also with all tags I want to configure, and I cannot use include_tasks with args-apply-tags anymore inside a role that also includes other roles (the tags will applied to all subtasks...). I miss something like include_role but only that specific tags or ignore dependencies. This isn't a bug, but has possible side effects in case of delegate_to.
I'm not really sure, what is the question? The question is - what is a good way to handle dependencies between hosts and roles in Ansible - especially when they are not on the same host?
I am sure I do not fully understand your exact problem, but when I was dealing with load balancers I used a template. So this was my disable_workers playbook:
---
- hosts: "{{ ip_list | default( 'jboss' ) }}"
tasks:
- name: Tag JBoss service as 'disabled'
ec2_tag:
resource: "{{ ec2_id }}"
region: "{{ region }}"
state: present
tags:
State: 'disabled'
delegate_to: localhost
- action: setup
- hosts: httpd
become: yes
become_user: root
vars:
uriworkermap_file: "{{ httpd_conf_dir }}/uriworkermap.properties"
tasks:
- name: Refresh inventory cache
ec2_remote_facts:
region: "{{ region }}"
delegate_to: localhost
- name: Update uriworkermap.properties
template:
backup: yes
dest: "{{ uriworkermap_file }}"
mode: 0644
src: ./J2/uriworkermap.properties.j2
Do not expect this to work as-is. It was v1.8 on AWS hosts, and things may have changed.
But the point is to set user-defined facts, on each host, for that host's desired state (enabled, disabled, stopped), reload the facts, and then run the Jinja template that uses those facts.
Related
The situation
I have a playbook whose execution I sometimes want to limit to specific hosts: web servers and application servers, e.g., --limit mywebserver, appserver1, appserver2. I have two groups:
webservers contain web servers including mywebserver
appservers contain application servers including appserver1 and appserver2
The playbook calls a play that I call for each of my webservers:
- name: Install and prepare Webservers
hosts: webservers
roles:
- role: webserver_setup
In that play, I execute tasks, e.g., setting up a website for each application server that is stored in a group variable:
- name: Configure website for all application servers
notify: Reload webserver
loop: "{{ groups['webserver_' + inventory_hostname] }}"
ansible.builtin.template:
src: ./templates/website.y2
dest: "/etc/nginx/sites-available/{{ hostvars[item]['app_server'] }}.conf"
mode: '0640'
owner: webserver
group: webserver
That works fine if I run the script without --limit. However, if I specify a host for --limit, then the loop in the play is still executed for all application servers, because it doesn't take the limit option into account. (No surprise here.)
My question
My problem is that I have a lot of application servers, and that I would like to limit the play's loop to only the hosts specified by --limit. How'd I do that?
Thoughts on solutions
There are special variables provided by Ansible, ansible_play_hosts, ansible_play_batch, but they only provide the hosts limited by --limit for the current play. In my case, these variables would contain only mywebserver, because the play is called for hosts: webservers.
I've thought about refactoring the play, such that it is called for each application server, instead, and use delegate_to to actually execute the task above on each associated webserver. However, I think this would have some drawbacks (ssh log in overhead for each app server, reload webserver config overhead for each app server, possible conflicts due to parallel task execution on the web server).
ansible_limit provides the limit that was passed on the command line, and can be expanded into a list of hostnames using the inventory_hostnames lookup. You could just use this in a condition; it's not entirely clear from the question what your actual structure is, but one of the following is probably right:
when: item in query('inventory_hostnames', ansible_limit | default('all'))
# OR
when: hostvars[item]['app_server'] in query('inventory_hostnames', ansible_limit | default('all'))
You might also be able to eliminate items from the loop entirely, though that makes the code a bit more complex and it's helpful to use intermediate variables to make it more readable:
- name: Configure website for all application servers
ansible.builtin.template:
src: website.y2 # Relative paths in template actions use `templates/` automatically, so you shouldn't specify it.
dest: /etc/nginx/sites-available/{{ hostvars[item]['app_server'] }}.conf
mode: "0640"
owner: webserver
group: webserver
notify: Reload webserver
loop: "{{ webserver_group | intersect(limit_hosts) }}"
vars:
webserver_group: "{{ groups['webserver_' ~ inventory_hostname] }}"
limit_hosts: "{{ query('inventory_hostnames', ansible_limit | default('all')) }}"
New to Ansible here. I've been adapting an Ansible playbook to automate the setup of a Raspberry Pi to use Pi-hole as described on this blog and as per this repo.
One change I've made is to use Bitwarden instead of 1Password as the secrets manager (see details below).
But after having done it, and thinking of offering a PR to the repo in question, I realized that my use of Bitwarden is just as hard-coded as the use of 1Password in the original (hardly an improvement).
Is there a way to make the choice of secret manager configurable in an Ansible playbook or setup? Or "call a different function" based on the value of a variable?
A desirable setup would be something along the lines of:
vars:
secret_manager: bitwarden
# one of {'1password', 'bitwarden', 'lastpass', 'ansible-vault',
# 'aws-secrets', 'azure-key-vault', ...}
tasks:
- name: ...
...
value: "{{ lookup(secret_manager, item_name, field=field_name) }}"
Details
Here is an example of using bitwarden instead of 1password in a playbook.yml:
New:
roles:
- role: ansible-modules-bitwarden
tasks:
- name: "test: get 'username' from the secrets manager"
debug:
- msg: "{{ lookup('bitwarden', 'pi-hole', field='username') }}"
(and note: if the field is a custom one, one must add custom_field=true whereas with
community.general.onepassword this is not needed).
Instead of:
tasks:
- name: "test: get 'username' from the secrets manager"
debug:
- msg: "{{ lookup('community.general.onepassword', 'pi-hole', field='username') }}"
Side note and for sake of completeness, in order to make this work I needed to install Bitwarden CLI as well as (one of) the ansible-modules-bitwarden, and then of course log in and get a session token:
export BW_SESSION="$(bw unlock --raw)"
This all works fine, but I am not quite happy: now the use of Bitwarden is hard-coded instead of "1Password". I would like to make this more generic and customizable. What if someone would like to use LastPass or AWS Secrets Manager instead?
Things I've tried
using if-else:
- hosts: all
vars:
use_bitwarden: true
secret_item: pi-hole
roles:
- role: ansible-modules-bitwarden
when: use_bitwarden
tasks:
- name: get username from the secrets manager
debug:
msg: "{{ lookup('bitwarden', secret_item, field='username') if use_bitwarden \
else lookup('community.general.onepassword', secret_item, field='username') }}"
but this bloats the playbook, makes it less readable and less DRY. Also, it isn't very extensible to the many other secrets managers out there.
define which manager to use in a variable:
- hosts: all
vars:
secret_manager: bitwarden
# secret_manager: community.general.onepassword
secret_item: pi-hole
roles:
- role: ansible-modules-bitwarden
when: secret_manager == 'bitwarden'
tasks:
- name: get username from the secrets manager
debug:
msg: "{{ lookup(secret_manager, secret_item, field='username') }}"
but this doesn't work as custom fields are to be queried with custom_field=true with the Bitwarden plugin, but not with 1Password.
Other ideas (not tried)
define a new generic plugin that uses other existing ones, based on a config variable.
use a task to get all the needed variables from whatever password manager at once and insert them, encrypted, into Ansible's vault. That way the task that does that can be isolated and have conditional execution with when: secret_manager == 'that'. But it makes maintenance of the playbook more difficult.
Maybe there is a much simpler and concise way of accomplishing this?
I am writing a playbook where i need to select host which will be a part of group which starts with name "hadoop". The host will be supplied as an extra variable in term of parent group. The task is about upgrading the java on all machines with repo but there are certain servers which dont have repo configured or are in dmz and can only use there local repo... i need to enable local_rpm:true so that when the playbook execute the server which belong to hadoop group have this fact enabled.
I tried like below :
- hosts: '{{ target }}'
gather_facts: no
become: true
tasks:
- name: enable local rpm
set_fact:
local_rpm: true
when: "'hadoop' in group_names"
tags: always
and then importing my role based on tag
It's probably better to use group_vars in this case.
https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#group-variables
I am asking for help with a problem of deploying multiple versions (different variables) of the app with the same role run from one playbook.
We have an app with multiple product families, which are different code versions. Each version has separate uWSGI vassal config and Nginx virtualhost config(/api/v2, /api/v3, ...).
The desired state would be to run playbook and configure the server with all versions specified.
Sadly, ansible's import_role/import_tasks can't be used with with_items, so include_role/include_tasks must be used (pitty because they do not honor role tags).
The include_role method would not be the biggest problem, but we use handlers to notify uWSGI touch to reload - on a code change, link change, virtualenv change, app_config change, ...).
But when using loop (with_items), the variables passed from the loop does not correctly propagate to handlers.
I tried this scenarios
playbook.yml - with_items loop inside the playbook
PROBLEM: Handler is run only for the first iteration of the loop.
#!/usr/bin/env ansible-playbook
# HAndler is run only once, from first notifier
- hosts: localhost
gather_facts: no
vars:
app_root: "/tmp/test_ansible"
app_versions:
- app_product_family: 1
app_release: "v1.0.2"
- app_product_family: 3
app_release: "v4.0.7"
tasks:
- name: Deploy multiple versions of app
include_role:
name: app
with_items: "{{ app_versions }}"
loop_control:
loop_var: app_version
vars:
app_product_family: "{{ app_version.app_product_family }}"
app_release: "{{ app_version.app_release }}"
tags:
- app
- app_debug
playbook_v2.yml - with_items loop inside role task
PROBLEM: Handler is run with the default value from "Defaults"
#!/usr/bin/env ansible-playbook
- hosts: localhost
gather_facts: no
roles:
- app_v2
vars:
app_v2_root: "/tmp/test_ansible_v2"
app_v2_versions:
- app_v2_product_family: 1
app_v2_release: "v1.0.2"
- app_v2_product_family: 3
app_v2_release: "v4.0.7"
Tasks roles/app_v2/main.yml
---
# Workaround because import_tasks can't be run with_items
- include_tasks: deploy.yml
when: app_v2_versions
with_items: "{{ app_v2_versions }}"
loop_control:
loop_var: app_v2
vars:
app_v2_product_family: "{{ app_v2.app_v2_product_family }}"
app_v2_release: "{{ app_v2.app_v2_release }}"
tags:
- app_v2
- app_v2_deploy
...
One idea was about writing a separate role for each product family, but they share nginx and uWSGI, so it will be lots of copy-pasting and sharing tasks (so tags would not work properly).
For now, I solved it with shell script wrapper, but this is not an ideal solution and does not work from Ansible tower.
Sample repo with tasks to reproduce problem (tested with ansible 2.4, 2.5, 2.6)
Any ideas & recommendations are very welcome.
The order of overrides for variables is broken for includes in Ansible. F.e. even set_fact in the included role will be shadowed by role defaults.
See this bug: https://github.com/ansible/ansible/issues/22025
It's closed but not fixed. My advice: use include and variables really carefully.
In practice I never use role includes with loop. If you need loop, include a tasklist in this loop (and that tasklist, in turn, may import_role).
Ok, It is a bug as #George Shuklin posted.
I will use my shell wrapper, which reads group_vars yaml and then runs the playbook multiple times according to the variable list length.
Sadly I hit multiple annoying bugs in ansible in last few weeks, kinda losing my trust in it ):
And probably everybody is using microservices and kubernetes, so need to speed up our migration (:
I have a task file that looks like this:
- name: Drop schemas
mysql_db: state=import name=mysql target={{ role_path }}/files/schemas/drop-imdb-perf.sql login_user={{ MYSQL_ROOT_USER }} login_password={{ MYSQL_ROOT_PWD }} login_host={{ inventory_hostname }}
I'm invoking it from a playbook that looks like this:
- name: Drop mySQL data
gather_facts: no
hosts: imdb
connection: local
tags:
- mysql-data-drop
tasks:
- include: ../roles/mysql/tasks/drop-perf.yml
I'm using Ansible version 1.9.4 so I'm thinking role_path should be a valid variable.
But when I run the playbook, I get this output:
TASK: [Drop schemas] **************************************************
fatal: [imdb] => One or more undefined variables: 'role_path' is undefined
I can't figure out why role_path is undefined. According to the ansible docs, it seems like for versions 1.8 and above it should be populated with the directory of the role in question, but I'm clearly mistaken about something or other.
I don't see you using any roles. Without looking into the Ansible code it seems obvious that role_path is defined within a role. Including a file of a role does not make it run in context of a role though.
If your include is by intend, role_path won't be defined. You could try to set it yourself together with the include like so:
tasks:
- include: ../roles/mysql/tasks/drop-perf.yml
role_path: ../roles/mysql
That might work or not, since role_path still is a magic variable and therefore might not be manually changed.
If you actually meant to include the role, then you need to define your playbook like this:
- name: Drop mySQL data
gather_facts: no
hosts: imdb
connection: local
tags:
- mysql-data-drop
roles:
- role: ../roles/mysql
But my guess is, you're trying to only run a single tasks file of that role and not the whole role. But what you're trying to do there seem to be against best practice. My recommendation would be to move the tag mysql-data-drop into the tasks of the file drop-perf.yml, because that's what tags are for: to trigger a limited set of tasks of roles or playbooks.