In my Ansible playbook I've been running into intermittent failures with apt tasks due to failures to obtain a lock.
I'm thinking of using some retry logic as described in this answer. e.g.
- name: install packages
apt:
name:
- nginx
- git
register: result
until: result is not failed
retries: 7
delay: 9
The playbook includes multiple apt tasks so I'm wondering if there's a way that I can avoid repeating the register, retries, until for each apt task i.e. some way to make these defaults, or to define my own apt-with-retries?
My playbook is for provisioning DigitalOcean droplets. I've got several apt tasks due to:
Breaking down the playbook into logic steps i.e. apt install a few things, followed by a few other steps, followed by apt installing another group of packages...
I've extracted a couple of roles into their own self-contained files e.g. a postgres_server role.
The goals here are the usual reasons for wanting to avoid duplication/repetition. e.g. avoid having to make changes in multiple places, avoid having to remember to copy and paste when adding another apt task in the future.
I looked into using module_defaults, but these properties are on the task itself rather than the apt module so I don't think they can be used for this scenario.
Newer version of apt module has support for retries at module level:
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_module.html#parameter-lock_timeout
Default is 60.
Related
I'm new to ansible and I would like to run some tasks only once. Not once per every run but only once and then never. In puppet I used file placeholders for that, similar to ansible "creates".
What is the best practice for my use case?
If I should use the "creates" method, which folder should be safe to use?
For example I would like to install letsencrypt on host where the first step is to update snap. I don't want to refresh snap every run so I would like to run it only before letsencrypt install.
- name: Update snap
command: snap install core && snap refresh core
args:
creates: /usr/local/ansible/placeholders/.update-snap
become: true
Most of the ansible modules are made to be idempotent, which means that they will perform action only when needed.
shell, script and command are the exception as ansible cannot know if action needs to be performed, and that's why there is the creates parameter to add idempotency.
That's why before writing a shell task you should check if there is not already an ansible module that could perform your task. There are almost 3000 base modules and with new versions of ansible you can get new community-provided modules through collections in Ansible Galaxy.
In your specific case, the snap module (https://docs.ansible.com/ansible/2.10/collections/community/general/snap_module.html) may be enough:
- name: Update snap
snap:
name: core
become: true
It will install the core snap if it's not already installed, and do nothing otherwise.
If it's not enough, you can implement yourself idempotency by creating a task querying snap to check the presence of core with it's version (add a changed_when: False as it's a task not doing any changes) and then launching the install/update depending on the query result (with a when condition).
If there is no way to do that, then you can fallback to using the creates. If the command executed already creates a specific file that presence can be used to know if it was already executed, better to reference this one. Else you need to create your own marker file in the shell. As it's a last option solution, there is no convention of where this marker file should be, so up to you to create your own (the one you propose looks good, even if the file name could be more specific and include the snap name in case you should use the same trick for other snaps).
In the Ansible role that I'm creating, I'm covering both an install and an uninstall scenario:
foo-install.yml is called from main.yml when the install flag is set to true.
foo-uninstall.yml is called from main.yml when the install flag is set to false.
While the install covers installing an RPM package, copying config files, and starting a system service, the uninstall steps basically reverse the installation: Stop the system service, uninstall the RPM package, remove the application folder.
As a good citizen, I have created a test for the role using Molecule, which runs the role in a CentOS Vagrant box. This works fine for the install scenario, where I use Python tests (using testinfra) to validate the RPM is installed, the service is started, etc.
How can I use Molecule to now test the uninstall scenario as well? Is there a way to change the steps in Molecule that it does something like this (simplified)?
create
converge (run the install parts of the role)
idempotence (for the install part)
verify (the install steps)
converge (run the uninstall parts of the role)
idempotence (for the uninstall part)
verify (the uninstall steps)
destroy
Maybe I'm missing something, but I have not found an obvious way (or examples) on how to do something like this.
Is there a way to cover scenarios like this? Or am I better off with just testing the install scenario?
Recommended Solution
The recommended way to address this is using multiple Molecule Scenarios. You could use your install scenario as the default, and then add a second uninstall scenario that just runs and tests the uninstall steps.
When setting this up, simply create a second scenario directory in your role's molecule folder (copy the default one), and then make some changes:
(Edit: this step was necessary for molecule < 3.0. scenario.name was removed in later versions) In the molecule.yml file change the scenario.name attribute to uninstall.
In the same file, use the default scenario's playbook.yml file as the playbook for the prepare step:
provisioner:
name: ansible
playbooks:
prepare: ../default/playbook.yml
converge: playbook.yml
Adjust the tests for the uninstall scenario to verify the uninstall steps.
This will ensure that the same steps are used for installing the software as in the install/default scenario, and you can focus on the uninstall steps.
To run the scenarios you can either run all of them or a single one:
# Run all scenarios
molecule test --all
# Run only the uninstall scenario
molecule test --scenario-name uninstall
This should get you pretty close to what you want to do, without replicating any code.
If you want to try some other things, here are a couple of other thoughts:
Alternatives
I would keep a scenario for the install only that would play all the needed tests (lint, idempotency, check, verify....) and create an install_uninstall specific scenario.
Playing install_uninstall will never be idempotent. So this scenario should disable the idempotency tests that will never pass. You might as well want to disable the check test that is played in your other scenario, lint... This can be done in molecule.yml by adjusting the parameters for scenario.test_sequence:
scenario:
name: install_uninstall
test_sequence:
- destroy
- create
- prepare
- converge
- verify
- destroy
Of course you can adjust to your real needs (like droping verify also if you have no testinfra tests for this case).
Once this is done you only have to add two plays in your scenario playbook:
---
- name: install
hosts: all
roles:
- role: my_role
install: true
- name: uninstall
hosts: all
roles:
- role: my_role
install: false
And you should be ready to test with:
molecule test -s install_uninstall
Edit:
An other option would be to keep only your current install scenario but launch the individual molecule commands rather than a full test. Assuming your current working scenario is in default
# Check and lint my files
molecule lint
# Make sure no box/container is on the way
molecule destroy
# Create my box/container for tests
molecule create
# Play my default playbook
molecule converge
# Idempotency check
molecule idempotence
# Verify we can correctly use check mode
molecule check
# Play testinfra tests
molecule verify
# Now play the uninstall
molecule converge -- -e install=false
## add more tests you can think off ##
# and finally cleanup
molecule destroy.
Unfortunately, unless this feature was very recently added to molecule, it is not possible to call idempotency and check with extra variables
I've got a monolithic playbook I'm refactoring into a series of smaller playbooks that get imported. Which is all well and good when importing them, but when testing them individually with e.g. vagrant the apt-cache often is stale and half my packages are missing.
I could of course put a task to update the apt-cache in every single one of them, but that seems wasteful.
Is there any conditional check for if a playbook is running inside another playbook or directly from the command line? Or check that the cache is in fact up-to-date in a globally-scoped var and condition based on that?
Or check that the cache is in fact up-to-date in a globally-scoped var and condition based on that
You can very easily declare you are running in vagrant through the Vagrantfile (if you are using the ansible "provisioner"), through a static fact in /etc/ansible/facts.d, by expending the energy to declare an extra-var on the command line, by checking for the NIC vendor that corresponds to the VM (so VBox in my case), or the virtualization method, or or or.
Separately, you are also welcome to actually declare a "staleness" threshold, and run apt-get update if the mtime for /var/lib/apt/lists/partial is older than some number of seconds/minutes/etc, which actually sounds much closer to what you really want versus having things behave one way in Vagrant and a separate way for real.
Following up on Matthew L. Daniel's excellent answer which I've accepted, here's the actual tasks:
tasks:
- name: Get the apt-cache stat
stat: path=/var/lib/apt/lists/partial
register: cachage
- name: Update apt cache
apt: update_cache=yes
when: '{{ (ansible_date_time.epoch|float - cachage.stat.mtime) > 1800.0 }}'
I've specified dependency for my role by declaring it inside meta/main.yml.
---
dependencies:
- role: angstwad.docker_ubuntu
src: https://github.com/angstwad/docker.ubuntu
scm: git
version: v2.3.0
However when I try to execute Ansible playbook I'm seeing error message:
ERROR! the role 'angstwad.docker_ubuntu' was not found in /home/.../ansible/roles:/etc/ansible/roles:/home/.../ansible/roles:/home/.../ansible
The error appears to have been in '/home/.../roles/docker-node/meta/main.yml': line 5, column 5, but may be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- role: angstwad.docker_ubuntu
^ here
From the error message I conclude that external role angstwad.docker_ubuntu hasn't be imported, even if it has been explicitly mentioned in docker-node/meta/main.yml.
Can I specify external dependencies from Ansible Galaxy even if my role itself has not been uploaded to Ansible Galaxy?
Or do I need to explicitly import them using ansible-galaxy?
From Ansible documentation:
Roles can also be dependent on other roles, and when you install a
role that has dependencies, those dependenices will automatically be
installed.
You specify role dependencies in the meta/main.yml file by providing a
list of roles. If the source of a role is Galaxy, you can simply
specify the role in the format username.role_name. The more complex
format used in requirements.yml is also supported, allowing you to
provide src, scm, version and name.
By following suggestions from another SO question (How to automatically install Ansible Galaxy roles?) it sounds like downloading external dependencies needs to be done explicitly either manually or with workaround.
If the role that has this dependency is itself being installed via ansible-galaxy, then revise meta/main.yml to look like this:
---
dependencies:
- src: https://github.com/angstwad/docker.ubuntu.git
scm: git
version: v2.3.0
Thus, when installing the primary role the docker.ubuntu role will automatically be installed in ~/.ansible/roles/docker.ubuntu and it will run when the playbook is executed.
If the role that has this dependency will NOT be installed via ansible-galaxy, that is, if the dependency is the only thing being installed via the ansible-galaxy command, then docker.ubuntu will have to be installed separately, e.g.
ansible-galaxy install git+https://github.com/angstwad/docker.ubuntu.git
or
ansible-galaxy install git+https://github.com/angstwad/docker.ubuntu.git -p /path_to/my_playbook/roles
This will place the role in your search path (again, ~/.ansible/roles/docker.ubuntu if not specified via -p). In this scenario, revise meta/main.yml to this:
---
dependencies:
- role: docker.ubuntu
Note that when pulling your role from github, it lands as docker.ubuntu vs docker_ubuntu. See https://galaxy.ansible.com/docs/using/installing.html#dependencies for more. Hope this helps.
As per documentation you linked, you see:
When dependencies are encountered by ansible-galaxy, it will automatically install each dependency to the roles_path.
Downloading of dependencies is handled by ansible-galaxy only, the src meta information doesn't affect ansible-playbook, which just looks inside your role directory and if it don't see a corresponding folder - it fails. As in answer in question you linked, you can indeed create a first task, or include in you playbook to run ansible galaxy before each start.
I'm working with Ansible using ansible-pull (runs on cron).
Can I install Ansible role from Ansible Galaxy without login in to all computers (just by adding a command to my Ansible playbook)?
If I understand you correctly, you're trying to download and install roles from Ansible Galaxy from the command line, in a hands-off manner, possibly repeatedly (via cron). If this is the case, here's how you can do it.
# download the roles
ansible-galaxy install --ignore-errors f500.elasticsearch groover.packerio
# run ansible-playbook to install the roles downloaded from Ansible Galaxy
ansible-playbook -i localhost, -c local <(echo -e '- hosts: localhost\n roles:\n - { role: f500.elasticsearch, elasticsearch_cluster_name: "my elasticsearch cluster" }\n - { role: groover.packerio, packerio_version: 0.6.1 }\n')
Explanation / FYI:
To download roles from Ansible Galaxy, use ansible-galaxy, not ansible-pull. For details, see the manual. You can download multiple roles at once.
If the role had been downloaded previously, repeated attempts at downloading using ansible-galaxy install will result in an error. If you wish to call this command repeatedly (e.g. from cron), use --ignore-errors (skip the role and move on to the next item) or --force (force overwrite) to work around this.
When running ansible-playbook, we can avoid having to create an inventory file using -i localhost, (the comma at the end signals that we're providing a list, not a file).
-c local (same as --connection=local) means that we won't be connecting remotely but will execute commands on the localhost.
<() functionality is process substitution. The output of the command appears as a file, so we can feed a "virtual playbook file" into the ansible-playbook command without saving the playbook to the disk first (e.g., playbookname.yml).
As shown, it's possible to embed role variables, such as packerio_version: 0.6.1 and apply multiple roles in a single command.
Note that whitespace is significant in playbooks (they are YAML files). Just as in Python code, be careful about indentation. It's easy to make typos in long lines with echo -e and \n (newlines).
You can run updates of roles from Ansible Galaxy and ansible-playbook separately.
With a bit of magic, you don't have to create inventory files or playbooks (this can be useful sometimes). The solution to install Galaxy roles remotely via push is less hacky / cleaner but if you prefer to use cron and pulling then this can help.
I usually add roles from galaxy as submodules in my own repository; that way I have control over when I update them, and ansible-pull will automatically fetch them - removing the need to run ansible-galaxy.
E.g.:
mkdir roles
git submodule add https://github.com/groover/ansible-role-packerio roles/groover.packerio
Yes you can.
# install Ansible Galaxy requirements via the pull playbook itself
- hosts: localhost
tasks:
- command: ansible-galaxy install -r requirements.yml