I'm using ansible to deploy several sites to the same server. Each site is a separate 'host' in the ansible hosts inventory, which works really well.
However, there are only two databases: production and testing.
How can I make sure my database-migration task only runs once per database?
I've read into the group_by, run_once and delegate_to features, but I'm not sure how to combine those.
The hosts look something like:
[production]
site1.example.com ansible_ssh_host=webserver.example.com
site2.example.com ansible_ssh_host=webserver.example.com
[beta]
beta-site1.example.com ansible_ssh_host=webserver.example.com
beta-site2.example.com ansible_ssh_host=webserver.example.com
[all:children]
production
beta
The current playbook looks like this:
---
- hosts: all
- tasks:
# ...
- name: "postgres: Create PostgreSQL database"
sudo: yes
sudo_user: postgres
postgresql_db: db="{{ DATABASES.default.NAME }}" state=present template=template0 encoding='UTF-8' lc_collate='en_US.UTF-8' lc_ctype='en_US.UTF-8'
tags: postgres
register: createdb
delegate_to: "{{ DATABASES.default.HOST|default(inventory_hostname) }}"
# ...
- name: "django-post: Create Django database tables (migrate)"
django_manage: command=migrate app_path={{ src_dir }} settings={{ item.settings }} virtualenv={{ venv_dir }}
with_items: django_projects
#run_once: true
tags:
- django-post
- django-db
- migrate
The best way I found was to restrict the execution of a task to the first host of a group. Therefore you need to add the groupname and the databases to a group_vars file like:
group_vars/production
---
dbtype=production
django_projects:
- name: project_1
settings: ...
- name: project_n
settings: ...
group_vars/beta
---
dbtype=beta
django_projects:
- name: project_1
settings: ...
- name: project_n
settings: ...
hosts
[production]
site1.example.com ansible_ssh_host=localhost ansible_connection=local
site2.example.com ansible_ssh_host=localhost ansible_connection=local
[beta]
beta-site1.example.com ansible_ssh_host=localhost ansible_connection=local
beta-site2.example.com ansible_ssh_host=localhost ansible_connection=local
[all:children]
production
beta
and limit the task execution to the first host that matches that group:
- name: "django-post: Create Django database tables (migrate)"
django_manage: command=migrate app_path={{ src_dir }} settings={{ item.settings }} virtualenv={{ venv_dir }}
with_items: django_projects
when: groups[dbtype][0] == inventory_hostname
tags:
- django-post
- django-db
- migrate
So, the below will illustrate why I say "once per group" is generally not supported. Naturally, I would love to see a cleaner out-of-the-box way, and do let me know if "once per group" isn't what you're after.
Hosts:
[production]
site1.example.com ansible_ssh_host=localhost ansible_connection=local
site2.example.com ansible_ssh_host=localhost ansible_connection=local
[beta]
beta-site1.example.com ansible_ssh_host=localhost ansible_connection=local
beta-site2.example.com ansible_ssh_host=localhost ansible_connection=local
[beta:vars]
dbhost=beta-site1.example.com
[production:vars]
dbhost=site1.example.com
[all:children]
production
beta
Example playbook which will do something on the dbhost once per group (the two real groups):
---
- hosts: all
tasks:
- name: "do this once per group"
sudo: yes
delegate_to: localhost
debug:
msg: "do something on {{hostvars[groups[item.key].0]['dbhost']}} for {{item}}"
register: create_db
run_once: yes
with_dict: groups
when: item.key not in ['all', 'ungrouped']
Related
I'm trying to install a website with ansible 2.4.0.0. When I do:
./architecture/scripts/provision.sh lxc
I get this result:
/usr/local/lib/python2.7/dist-packages/cryptography/__init__.py:39: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
CryptographyDeprecationWarning,
/usr/local/lib/python2.7/dist-packages/cryptography/__init__.py:39: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
CryptographyDeprecationWarning,
[WARNING]: - ansible-genericservice was NOT installed successfully: - command git clone git#git.smile.fr:ansible/ansible-genericservice.git ansible-genericservice failed in directory /tmp/tmpmZfdUa (rc=128)
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.
/usr/local/lib/python2.7/dist-packages/cryptography/__init__.py:39: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
CryptographyDeprecationWarning,
[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static inclusions or 'include_tasks' for dynamic inclusions. This feature will be removed in a future release.
Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain more about this rationale.. This feature will be removed in a
future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
PLAY [dbservers,cacheservers,searchservers,webservers] ************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
Warning: the ECDSA host key for 'webstore.lxc' differs from the key for the IP address '10.124.193.20'
Offending key for IP in /home/jredor/.ssh/known_hosts:19
Matching host key in /home/jredor/.ssh/known_hosts:21
Are you sure you want to continue connecting (yes/no)? yes
ok: [lxc-server]
TASK [include_vars] ***********************************************************************************************************************************************************************************************
fatal: [lxc-server]: FAILED! => {"failed": true, "msg": "No file was found when using with_first_found. Use the 'skip: true' option to allow this task to be skipped if no files are found"}
to retry, use: --limit #/home/jredor/projets/webstore/architecture/provisioning/provision.retry
PLAY RECAP ********************************************************************************************************************************************************************************************************
lxc-server : ok=1 changed=0 unreachable=0 failed=1
I can't find how to pass the skip option to true. My ansible file look like that:
cd "$( dirname "${BASH_SOURCE[0]}" )"
cd ..
source scripts/_environment-with-lxc.sh
source scripts/_ansible.sh
ansible-galaxy install -r provisioning/requirements.yml -p provisioning/roles -n -f
ansible-playbook --ssh-extra-args="${ANSIBLE_SSH_ARGS}" provisioning/provision.yml -i provisioning/inventory/$inventory
exit $?
It's the first time that I use ansible so I'm lost.
Thanks
Edit:
So here is the code in provision.yml
---
# load variables for each servers
- hosts:
- dbservers
- cacheservers
- searchservers
- webservers
vars:
ansible_user: "root"
tasks:
- include: includes/include-vars.yml
# Prepare the delivery authorized keys
- hosts:
- dbservers
- cacheservers
- searchservers
- webservers
connection: local
vars:
ansible_user: "root"
tmp_delivery_users:
- name: "{{ magento_project_user }}"
group: "{{ magento_webserver_group }}"
authorized_keys: "{{ delivery_authorized_keys }}"
tasks:
- name: "Prepare the list of the authorized keys for delivery - Extra Keys"
set_fact: delivery_authorized_keys="{{ delivery_authorized_extra_keys }}"
- name: "Prepare the delivery_users object"
set_fact: delivery_users="{{ tmp_delivery_users }}"
# add hosts alias on Servers
- hosts:
- dbservers
- cacheservers
- searchservers
- webservers
vars:
ansible_user: "root"
tasks:
- include: includes/init-hosts.yml
with_items: "{{ specific_hosts|default([]) }}"
# add magento hosts alias on WebServers
- hosts:
- webservers
vars:
ansible_user: "root"
tasks:
- include: includes/init-hosts.yml
with_items: "{{ magento_server_alias|default([]) }}"
# Generic behaviors on all servers
- hosts:
- dbservers
- cacheservers
- searchservers
- webservers
vars:
ansible_user: "root"
roles:
- role: ansible-basicserver
# Generic usage of the ansible roles - DB Server
- hosts: dbservers
vars:
ansible_user: "root"
roles:
- role: ansible-mysql-server
# Generic usage of the ansible roles - Cache Server
- hosts: cacheservers
vars:
ansible_user: "root"
roles:
- {
role: ansible-redis,
redis_instance_name: "magento_cache",
redis_setting_port: "{{ magento_cache_port }}",
redis_setting_save: "{{ redis_setting_save_cache }}"
}
- {
role: ansible-redis,
redis_instance_name: "magento_session",
redis_setting_port: "{{ magento_cache_session_port }}",
redis_setting_save: "{{ redis_setting_save_session }}"
}
# Generic usage of the ansible roles - Search Server
- hosts: searchservers
vars:
ansible_user: "root"
roles:
- role: ansible-elasticsearch
# Prepare php parameters
- hosts: webservers
vars:
ansible_user: "root"
tasks:
- include: includes/prepare-php-parameters.yml
# Generic usage of the ansible roles - Webserver Server
- hosts: webservers
vars:
ansible_user: "root"
roles:
- role: ansible-php
- role: ansible-apache
- role: ansible-varnish
- role: ansible-nginx
# Specific usage of the ansible roles - Webserver Server - Dev Tools
- hosts: webservers
vars:
ansible_user: "root"
roles:
- { role: ansible-npm, when: magento_install_maildev or magento_install_grunt }
- { role: ansible-maildev, when: magento_install_maildev }
tasks:
- name: "Install NPM package: grunt-cli"
npm: name="grunt-cli" global=yes
when: magento_install_grunt
- name: "Add delivery user in groups"
user:
name: "{{ magento_project_user }}"
groups: "{{ magento_delivery_groups }}"
- name: "Create {{ magento_source_path }} folder"
file:
path: "{{ magento_source_path }}"
state: directory
owner: "{{ magento_project_user }}"
group: "{{ magento_project_group }}"
mode: "u=rwX,g=rX,o=rX"
# Specific task for Magento 2
- name: "Check if Magento app/etc/env.php exists"
stat:
path: "{{ magento_source_path }}/app/etc/env.php"
register: magento_app_etc_env
- name: "Update app/etc/env.php configuration file"
template:
src: "templates/magento/env.php.j2"
dest: "{{ magento_source_path }}/app/etc/env.php"
owner: "{{ magento_project_user }}"
group: "{{ magento_webserver_group }}"
mode: "u=rw,g=rw,o=r"
vars:
magento_cache_database: "{{ magento_cache_database_for_run }}"
when: magento_app_etc_env.stat.exists
# Update permissions
- include: includes/permissions-tasks-full.yml
It was coded by a fellow coworker and I'm just trying to use it to deplay the website on my computer. So I guess I have to put here the "skip:true" but I really don't know where, sorry. Thanks for your help!
You can specify the skip option as the argument in the failing lookup, https://docs.ansible.com/ansible/latest/plugins/lookup/first_found.html#parameter-skip
I have a Ansible playbook which does multiple things as below -
Download artifacts fron nexus into local server (Ansible Master).
Copy those artifacts onto multiple remote machines let's say server1/2/3 etc..
And I have used roles in my playbook and the role (repodownload) which downloads the artifacts I want to run it only once because why would i want to download the same thing again. I have tried to use run_once: true but i guess that won't work because that only works for one playbook run but my playbook is running multiple times for multiple hosts.
---
- name: Deploy my Application to tomcat nodes
hosts: '{{ target_env }}'
serial: 1
roles:
- role: repodownload
tags:
- repodownload
- role: copyrepo
tags:
- copyrepo
- role: stoptomcat
tags:
- stoptomcat
- role: deploy
tags:
- deploy
Here target_env is being passed from the command line and it's the remote host group.
Any help is appreciated.
Below is the code from main.yml from repodownload role -
- connection: local
name: Downloading files from Nexus to local server
get_url: url="{{ nexus_url }}/{{item}}/{{ myvm_release_version }}/{{item}}-{{ release_ver }}.war" dest={{ local_server_location }}
with_items:
- "{{ temps }}"
This is a really simple one that I battled with too.
Try this:
- connection: local
name: Downloading files from Nexus to local server
get_url:
url: "{{ nexus_url }}/{{item}}/{{ myvm_release_version }}/{{item}}-{{ release_ver }}.war"
dest: "{{ local_server_location }}"
with_items:
- "{{ temps }}"
run_once: true
Just something else, unrelated to your main question;
When you run a module that has really long args, like in your example above, rather break the params into their own lines nested under the module. It makes for easier reading, and it makes it easier to spot any potential typos or syntax errors early.
Okay extending from your converstation with Zeitounator. The following workaround will work without changing your vars files. Just remember that this is a workaround, might not be the most efficient way to do the job.
---
- name: Download my repo to localhost
# Executes only for first host in target_env and has visibility to group vars of target_env
hosts: '{{ target_env }}[0]'
serial: 1
roles:
- role: repodownload
tags:
- repodownload
- name: Deploy my Application to tomcat nodes
# Executes for all hosts in target_env
hosts: '{{ target_env }}'
serial: 1
roles:
- role: copyrepo
tags:
- copyrepo
- role: stoptomcat
tags:
- stoptomcat
- role: deploy
tags:
- deploy
How can I add a host to a group using tower_group or tower_host modules?
The following code creates a host and a group, but they are unrelated to each other:
---
- hosts: localhost
connection: local
gather_facts: false
tasks:
- tower_inventory:
name: My Inventory
organization: Default
state: present
tower_config_file: "~/tower_cli.cfg"
- tower_host:
name: myhost
inventory: My Inventory
state: present
tower_config_file: "~/tower_cli.cfg"
- tower_group:
name: mygroup
inventory: My Inventory
state: present
tower_config_file: "~/tower_cli.cfg"
Docs mention instance_filters parameter ("Comma-separated list of filter expressions for matching hosts."), however do not provide any usage example.
Adding instance_filters: myhost to the tower_group task has no effect.
I solved it using Ansible shell module and tower-cli. I Know that create a ansible module is better than it, but to a fast solution...
- hosts: awx
vars:
tasks:
- name: Create Inventory
tower_inventory:
name: "Foo Inventory"
description: "Our Foo Cloud Servers"
organization: "Default"
state: present
- name: Create Group
tower_group:
inventory: "Foo Inventory"
name: Testes
register: fs_group
- name: Create Host
tower_host:
inventory: "Foo Inventory"
name: "host"
register: fs_host
- name: Associate host group
shell: tower-cli host associate --host "{{fs_host.id}}" --group "> {{fs_group.id}}"
This isn't natively available in the modules included with Tower, which are older and use the deprecated tower-cli package.
But it is available in the newer AWX collection, which uses the awx CLI, as long as you have a recent enough Ansible (2.9 should be fine).
In essence, install the awx collection through a requirements file, or directly like
ansible-galaxy collection install awx.awx -p ./collections
Add the awx.awx collection to your playbook
collections:
- awx.awx
and then use the hosts: option to tower_group:.
- tower_group:
name: mygroup
inventory: My Inventory
hosts:
- myhost
state: present
You can see a demo playbook here.
Be aware though that you may need preserve_existing_hosts: True if your group already contains other hosts. Unfortunately there does not seem to be an easy way to remove a single host from a group.
In terms of your example this would probably work:
---
- hosts: localhost
connection: local
gather_facts: false
collections:
- awx.awx
tasks:
- tower_inventory:
name: My Inventory
organization: Default
state: present
tower_config_file: "~/tower_cli.cfg"
- tower_host:
name: myhost
inventory: My Inventory
state: present
tower_config_file: "~/tower_cli.cfg"
- tower_group:
name: mygroup
inventory: My Inventory
state: present
tower_config_file: "~/tower_cli.cfg"
hosts:
- myhost
My folder structure:
First I'll give you this so you can see how this is laid out and reference it when reading below:
/environments
/development
hosts // Inventory file
/group_vars
proxies.yml
/custom_tasks
firewall_rules.yml // File I'm trying to bring in
playbook.yml // Root playbook, just brings in the plays
rev-proxy.yml // Reverse-proxy playbook, included by playbook.yml
playbook.yml:
---
- include: webserver.yml
- include: rev-proxy.yml
proxies.yml just contains firewall_custom_include_file: custom_tasks/firewall_rules.yml
firewall_rules.yml:
tasks:
- name: "Allowing traffic from webservers on 80"
ufw: src=10.10.10.3, port=80, direction=in, rule=allow
- name: "Allowing traffic all on 443"
ufw: port=443, rule=allow
and finally rev-proxy.yml play:
---
- hosts: proxies
become: yes
roles:
- { role: firewall }
- { role: geerlingguy.nginx }
pre_tasks:
# jessie-backports for nginx-extras 1.10
- name: "Adding jessie-backports repo"
copy: content="deb http://ftp.debian.org/debian jessie-backports main" dest="/etc/apt/sources.list.d/jessie-backports.list"
- name: Updating apt-cache.
apt: update_cache="yes"
- name: "Installing htop"
apt:
name: htop
state: present
- name: "Coopying SSL certificates"
copy: src=/vagrant/ansible/files/ssl/ dest=/etc/ssl/certs force=no
tasks:
- name: "Including custom firewall rules."
include: "{{ inventory_dir }}/{{ firewall_custom_include_file }}.yml"
when: firewall_custom_include_file is defined
vars_files:
- ./vars/nginx/common.yml
- ./vars/nginx/proxy.yml
What I'm trying to do:
Using Ansible 2.2.1.0
I'm trying to include a list of tasks that will be run if a variable firewall_custom_include_file is set. The list is included relative to the inventory directory by doing "{{ inventory_dir }}/{{ firewall_custom_include_file }}.yml" - in this case that works out to /vagrant/ansible/environments/development/custom_tasks/firewall_rules.yml
Essentially the idea here is that I need to have different firewall rules be executed based on what environment I'm in, and what hosts are being provisioned.
To give a simple example: I might want to whitelist a database server IP on the production webserver, but not on the reverse proxy, and also not on my development box.
The problem:
Whenever I include firewall_rules.yml like above, it tells me:
TASK [Including custom firewall rules.] ****************************************
fatal: [proxy-1]: FAILED! => {"failed": true, "reason": "included task files must contain a list of tasks"}
I'm not sure what it's expecting, I tried taking out the tasks: at the beginning of the file, making it:
- name: "Allowing traffic from webservers on 80"
ufw: src=10.10.10.3, port=80, direction=in, rule=allow
- name: "Allowing traffic all on 443"
ufw: port=443, rule=allow
But then it gives me the error:
root#ansible-control:/vagrant/ansible# ansible-playbook -i environments/development playbook.yml
ERROR! Attempted to execute "/vagrant/ansible/environments/development/custom_tasks/firewall_rules.yml" as inventory script: problem running /vagrant/ansible/environments/development/custom_tasks/firewall_rules.yml --list ([Errno 8] Exec format error)
Attempted to read "/vagrant/ansible/environments/development/custom_tasks/firewall_rules.yml" as YAML: 'AnsibleSequence' object has no attribute 'keys'
Attempted to read "/vagrant/ansible/environments/development/custom_tasks/firewall_rules.yml" as ini file: /vagrant/ansible/environments/development/custom_tasks/firewall_rules.yml:2: Expected key=value host variable assignment, got: name:
At this point I'm not really sure what it's looking for in the included file, and I can't seem to really find clear documentation on this, or other people having this issue.
Try to execute with -i environments/development/hosts instead of directory.
But I bet that storing tasks file inside inventory is far from best practices.
You may want to define list of custom rules as inventory variable, e.g.:
custom_rules:
- src: 10.10.10.3
port: 80
direction: in
rule: allow
- port: 443
rule: allow
And instead of include task, make something like this:
- ufw:
port: "{{ item.port | default(omit) }}"
rule: "{{ item.rule | default(omit) }}"
direction: "{{ item.direction | default(omit) }}"
src: "{{ item.src | default(omit) }}"
with_items: "{{ custom_rules }}"
my inventory file's contents -
[webservers]
x.x.x.x ansible_ssh_user=ubuntu
[dbservers]
x.x.x.x ansible_ssh_user=ubuntu
in my tasks file which is in common role i.e. it will run on both hosts but I want to run a following task on host webservers not in dbservers which is defined in inventory file
- name: Install required packages
apt: name={{ item }} state=present
with_items:
- '{{ programs }}'
become: yes
tags: programs
is when module helpful or there is any other way? How could I do this ?
If you want to run your role on all hosts but only a single task limited to the webservers group, then - like you already suggested - when is your friend.
You could define a condition like:
when: inventory_hostname in groups['webservers']
Thank you, this helps me too.
hosts file:
[production]
host1.dns.name
[internal]
host2.dns.name
requirements.yml file:
- name: install the sphinx-search rpm from a remote repo on x86_64 - internal host
when: inventory_hostname in groups['internal']
yum:
name: http://sphinxsearch.com/files/sphinx-2.2.11-1.rhel7.x86_64.rpm
state: present
- name: install the sphinx-search rpm from a remote repo on i386 - Production
when: inventory_hostname in groups['production']
yum:
name: http://sphinxsearch.com/files/sphinx-2.2.11-2.rhel6.i386.rpm
state: present
An alternative to consider in some scenarios is -
delegate_to: hostname
There is also this example form the ansible docs, to loop over a group. https://docs.ansible.com/ansible/latest/user_guide/playbooks_delegation.html -
- hosts: app_servers
tasks:
- name: gather facts from db servers
setup:
delegate_to: "{{item}}"
delegate_facts: True
loop: "{{groups['dbservers']}}"