In the Ansible role that I'm creating, I'm covering both an install and an uninstall scenario:
foo-install.yml is called from main.yml when the install flag is set to true.
foo-uninstall.yml is called from main.yml when the install flag is set to false.
While the install covers installing an RPM package, copying config files, and starting a system service, the uninstall steps basically reverse the installation: Stop the system service, uninstall the RPM package, remove the application folder.
As a good citizen, I have created a test for the role using Molecule, which runs the role in a CentOS Vagrant box. This works fine for the install scenario, where I use Python tests (using testinfra) to validate the RPM is installed, the service is started, etc.
How can I use Molecule to now test the uninstall scenario as well? Is there a way to change the steps in Molecule that it does something like this (simplified)?
create
converge (run the install parts of the role)
idempotence (for the install part)
verify (the install steps)
converge (run the uninstall parts of the role)
idempotence (for the uninstall part)
verify (the uninstall steps)
destroy
Maybe I'm missing something, but I have not found an obvious way (or examples) on how to do something like this.
Is there a way to cover scenarios like this? Or am I better off with just testing the install scenario?
Recommended Solution
The recommended way to address this is using multiple Molecule Scenarios. You could use your install scenario as the default, and then add a second uninstall scenario that just runs and tests the uninstall steps.
When setting this up, simply create a second scenario directory in your role's molecule folder (copy the default one), and then make some changes:
(Edit: this step was necessary for molecule < 3.0. scenario.name was removed in later versions) In the molecule.yml file change the scenario.name attribute to uninstall.
In the same file, use the default scenario's playbook.yml file as the playbook for the prepare step:
provisioner:
name: ansible
playbooks:
prepare: ../default/playbook.yml
converge: playbook.yml
Adjust the tests for the uninstall scenario to verify the uninstall steps.
This will ensure that the same steps are used for installing the software as in the install/default scenario, and you can focus on the uninstall steps.
To run the scenarios you can either run all of them or a single one:
# Run all scenarios
molecule test --all
# Run only the uninstall scenario
molecule test --scenario-name uninstall
This should get you pretty close to what you want to do, without replicating any code.
If you want to try some other things, here are a couple of other thoughts:
Alternatives
I would keep a scenario for the install only that would play all the needed tests (lint, idempotency, check, verify....) and create an install_uninstall specific scenario.
Playing install_uninstall will never be idempotent. So this scenario should disable the idempotency tests that will never pass. You might as well want to disable the check test that is played in your other scenario, lint... This can be done in molecule.yml by adjusting the parameters for scenario.test_sequence:
scenario:
name: install_uninstall
test_sequence:
- destroy
- create
- prepare
- converge
- verify
- destroy
Of course you can adjust to your real needs (like droping verify also if you have no testinfra tests for this case).
Once this is done you only have to add two plays in your scenario playbook:
---
- name: install
hosts: all
roles:
- role: my_role
install: true
- name: uninstall
hosts: all
roles:
- role: my_role
install: false
And you should be ready to test with:
molecule test -s install_uninstall
Edit:
An other option would be to keep only your current install scenario but launch the individual molecule commands rather than a full test. Assuming your current working scenario is in default
# Check and lint my files
molecule lint
# Make sure no box/container is on the way
molecule destroy
# Create my box/container for tests
molecule create
# Play my default playbook
molecule converge
# Idempotency check
molecule idempotence
# Verify we can correctly use check mode
molecule check
# Play testinfra tests
molecule verify
# Now play the uninstall
molecule converge -- -e install=false
## add more tests you can think off ##
# and finally cleanup
molecule destroy.
Unfortunately, unless this feature was very recently added to molecule, it is not possible to call idempotency and check with extra variables
Related
In my Ansible playbook I've been running into intermittent failures with apt tasks due to failures to obtain a lock.
I'm thinking of using some retry logic as described in this answer. e.g.
- name: install packages
apt:
name:
- nginx
- git
register: result
until: result is not failed
retries: 7
delay: 9
The playbook includes multiple apt tasks so I'm wondering if there's a way that I can avoid repeating the register, retries, until for each apt task i.e. some way to make these defaults, or to define my own apt-with-retries?
My playbook is for provisioning DigitalOcean droplets. I've got several apt tasks due to:
Breaking down the playbook into logic steps i.e. apt install a few things, followed by a few other steps, followed by apt installing another group of packages...
I've extracted a couple of roles into their own self-contained files e.g. a postgres_server role.
The goals here are the usual reasons for wanting to avoid duplication/repetition. e.g. avoid having to make changes in multiple places, avoid having to remember to copy and paste when adding another apt task in the future.
I looked into using module_defaults, but these properties are on the task itself rather than the apt module so I don't think they can be used for this scenario.
Newer version of apt module has support for retries at module level:
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_module.html#parameter-lock_timeout
Default is 60.
The ansible-galaxy tool can create a skeleton of an Ansible role. (ansible-galaxy role init myrolename) Inside is a tests directory, but the Ansible documentation does not mention what it's for. Presumably tests of some sort, but who runs them, and how?
Travis CI.
The tests directory seems to have been added in 2015, in PR #13489, and its description makes it clear that it's for Travis. It also adds .travis.yml, which looks like it does nothing by default. Snippet:
script:
# Basic role syntax check
- ansible-playbook tests/test.yml -i tests/inventory --syntax-check
I am not aware of any other tools using it.
I'm new to ansible and I would like to run some tasks only once. Not once per every run but only once and then never. In puppet I used file placeholders for that, similar to ansible "creates".
What is the best practice for my use case?
If I should use the "creates" method, which folder should be safe to use?
For example I would like to install letsencrypt on host where the first step is to update snap. I don't want to refresh snap every run so I would like to run it only before letsencrypt install.
- name: Update snap
command: snap install core && snap refresh core
args:
creates: /usr/local/ansible/placeholders/.update-snap
become: true
Most of the ansible modules are made to be idempotent, which means that they will perform action only when needed.
shell, script and command are the exception as ansible cannot know if action needs to be performed, and that's why there is the creates parameter to add idempotency.
That's why before writing a shell task you should check if there is not already an ansible module that could perform your task. There are almost 3000 base modules and with new versions of ansible you can get new community-provided modules through collections in Ansible Galaxy.
In your specific case, the snap module (https://docs.ansible.com/ansible/2.10/collections/community/general/snap_module.html) may be enough:
- name: Update snap
snap:
name: core
become: true
It will install the core snap if it's not already installed, and do nothing otherwise.
If it's not enough, you can implement yourself idempotency by creating a task querying snap to check the presence of core with it's version (add a changed_when: False as it's a task not doing any changes) and then launching the install/update depending on the query result (with a when condition).
If there is no way to do that, then you can fallback to using the creates. If the command executed already creates a specific file that presence can be used to know if it was already executed, better to reference this one. Else you need to create your own marker file in the shell. As it's a last option solution, there is no convention of where this marker file should be, so up to you to create your own (the one you propose looks good, even if the file name could be more specific and include the snap name in case you should use the same trick for other snaps).
I want to be able to guarantee that after a long running shell command is executed successfully on a given host, it is not executed in subsequent playbook runs on that host.
I was happy to learn of the creates option to the Ansible shell task (see http://docs.ansible.com/ansible/shell_module.html); using this I can create a file after a shell command succeeds and not have that command execute in future runs:
- name: Install Jupiter theme
shell: wp theme install ~/downloads/jupiter-theme.zip --activate
&& touch ~/ansible-flags/install-jupiter-theme
args:
creates: ~/ansible-flags/install-jupiter-theme
As you can see, I have a directory ansible-flags for these control files in the home directory of my active user (deploy).
However, this is quite a kludge. Is there a better way to do this?
Also, the situation gets a lot more complicated when I need to run a step as a different user; in that case, I don't have access to the deploy home directory and its subdirectories. I could grant access, of course, but that's adding even more complexity. How would you recommend I deal with this?
Your playbook should be idempotent!
So either task should be safe to be executed multiple times (e.g. echo ok) or your should check whether you should execute it at all.
Your task may look like this:
- name: Install Jupiter theme
shell: wp theme is-installed jupiter || wp theme install ~/downloads/jupiter-theme.zip --activate
It will check if the jupiter theme is installed and will run wp theme install only if theme is not installed.
But this will mark the result of task as changed in any case.
To make it nice, you can do this:
- name: Install Jupiter theme
shell: wp theme is-installed jupiter && echo ThemeAlreadyInstalled || wp theme install ~/downloads/jupiter-theme.zip --activate
register: cmd_result
changed_when: cmd_result.stdout != "ThemeAlreadyInstalled"
First, it will check if the jupiter theme is already installed.
If this is the case, it will output ThemeAlreadyInstalled and exit.
Otherwise it will call wp theme install.
The task's result is registered into cmd_result.
We user changed_when parameter to state the fact that if ThemeAlreadyInstalled ansible shouldn't consider the task as changed, so it remains green in your playbook output.
If you do some preparation tasks before wp theme install (e.g. download theme), you may want to run the test as separate task to register the result and use when clauses in subsequent tasks:
- name: Check Jupiter theme is installed
shell: wp theme is-installed jupiter && echo Present || echo Absent
register: wp_theme
changed_when: false
- name: Download theme
get_url: url=...
when: wp_theme.stdout == "Absent"
- name: Install Jupiter theme
shell: wp theme install ~/downloads/jupiter-theme.zip --activate
when: wp_theme.stdout == "Absent"
Ansible itself is stateless, so whatever technique you use needs to be implement by yourself. There is no easier way like telling Ansible to only ever run a task once.
I think what you have already is pretty straight forward. Maybe use another path to check. I'm pretty sure the wp theme install will actually extract that zip to some location, which you then can use together with the creates option.
There are alternatives but nothing that makes it easier:
Create a script in any language you like, that checks for the theme and if it is not installed, install it. Instead of the shell task then you would execute the script with the script module.
Install a local fact in /etc/ansible/facts.d which checks if the theme in already installed. Then you could simply apply a condition based on that fact to the shell task.
Set up fact caching, then register the result of your shell task and store it as a fact. With fact caching enabled you can access the stored result on later playbook runs.
It only gets more complicated. The creates option is perfect for this scenario and if you use the location where the wp command extracted the zip to, it also is pretty clean.
Of course you need to grant access to that file, if you have a need to access it from another user account. Storing stuff in a users home directory with the permissions set to only be readable by the owner indeed is no ideal solution.
I'm looking for some best practices on how to increase my productivity when writing new puppet modules. My workflow looks like this right now:
vagrant up
Make changes/fixes
vagrant provision
Find mistakes/errors, GOTO 2
After I get through all the mistakes/errors I do:
vagrant destroy
vagrant up
Make sure everything is working
commit my changes
This is too slow... how can i make this workflow faster?
I am in denial about writing tests for puppet. What are my other options?
cache your apt/yum repository on your host with the vagrant-cachier plugin
use profile –evaltrace to find where you loose time on full provisioning
use package base distribution :
eg: rvm install ruby-2.0.0 vs a pre-compiled ruby package created with fpm
avoid a "wget the internet and compile" approach
this will probably make your provisioning more reproducible and speedier.
don't code modules
try reusing some from the forge/github/...
note that it can be against my previous advice
if this is an option, upgrade your puppet/ruby version
iterate and prevent full provisioning
vagrant up
vagrant provision
modify manifest/modules
vagrant provision
modify manifest/modules
vagrant provision
vagrant destroy
vagrant up
launch server-spec
minimize typed command
launch command as you modify your files
you can perhaps setup guard to launch lint/test/spec/provision as you save
you can also send notifications from guest to host machine with vagrant-notify
test without actually provisioning in vagrant
rspec puppet (ideal when refactoring modules)
test your provisioning instead of manual checking
stop vagrant ssh-ing checking if service is running or a config has a given value
launch server-spec
take a look at Beaker
delegate running the test to your preferred ci server (jenkins, travis-ci,...)
if you are a bit fustrated by puppet... take a look at ansible
easy to setup (no ruby to install/compile)
you can select portion of stuff you want to run with tags
you can share the playbooks via synched folders and run ansible in the vagrant box locally (no librairian-puppet to launch)
update : after discussion with #garethr, take a look at his last presentation about guard.
I recommand using language-puppet. It comes with a command line tool (puppetresources) that can compute catalogs on your computer and let you examine them. It has a few useful features that can't be found in Puppet :
It is really fast (6 times faster on a single catalog, something like 50 times on many catalogs)
It tracks where each resource was defined, and what was the "class stack" at that point, which is really handy when you have duplicate resources
It automatically checks that the files you refer to exist
It is stricter than Puppet (breaks on undefined variables for example)
It let you print to standard output the content of any file, which is useful for developing complex templates
The only caveat is that it only works with "modern" Puppet practices. For example, require is not implemented. It also only works on Linux.