I'm using AWX as a task runner to run a variety of Ansible modules. Some of the Ansible modules are third-party modules whose parameters I can't control without forking the module, which is undesirable for a variety of reasons.
AWX supplies ansible_user as one variable that is used by some of the modules I'm using, and I'm trying to allow a user to some hosts by setting another variable, user_override.
I first thought to simply add the line ansible_user: "{{ user_override | default(ansible_user) }}" to the task's parameters, which would work... but the modules in question don't accept credentials via parameters. My next thought was to add a vars: entry to the playbook and supply the override there via the same markup as above. This unfortunately results in the error recursive loop detected in template string, which has been the bane of my existence while working through this problem.
I've also tried using the if/else syntax and intermediate variables, but neither appear to solve this problem.
How can I achieve this override functionality without forking AWX or the module in question?
Mods: This is distinct from the pile of questions asking about simple variable defaulting because the existing questions aren't in the context of AWX or can be solved by simply using default() or default(lookup()).
At this point, I'm pretty sure that the thing I (the asker) was trying to do is intended to be done by just sometimes setting a value, but some challenging-to-anonymize constraints in our software make that somewhere between challenging and not an option.
A sideways solution to the particular problem of overriding the user is that AWX sets the username in remote_user as well as ansible_user, which is then used by networkcli. So you can use the line `remote_user: "{{ override_user | default(ansible_user) }}". This doesn't help generally, but does answer the question.
Related
I was unable to find documentation or Q&A covering this specific topic. I'm posting it here with the answer in the hopes that someone finds it useful. I am a member of a small group of automation engineers. One task is to provide automation in Tower for other engineers and admins.
The automation team is happy with our current setup which allows us to run plays from the commandline without locking our accounts, without typing a password every time, and with no passwords stored in plain text:
We use an ansible vault stored in each admin's profile to store encrypted login credentials, along with a gpg armored key. Each vault uses teh same name, similar to ~/.ansible/vault.yml
A script extracts the key and unlocks the vault.
The script is defined in ansible.cfg, [defaults], vault_identity_list.
the playbooks load the vault with vars_files
Tower in job isolation mode cannot access home directories. And we do not want vault+key outside the admin's home folder, subject to random prying. Tower has its own vault system that we use when using tower. We want to maintain our current method of commandline runs, but be able to use the same playbook in Tower and Engine.
I tried:
Forcing tower to read the vault. (no joy)
Playbook with vault commented out. (This worked in tower, but I had to toggle the commenting to run from commandline. Put a pin in this as a last resort.)
Using conditional to only load vars_files when ansible_user is not awx. (Well guess what, it still runs as the user who triggers the job. Put a pin in this to find another variable that is consistent and indicates tower is the platform.)
Using tags and skip-tags within tower to skip vars_files (no joy. Tags don't work on vars_files:)
What I found that worked:
skip-tags does exactly what I needed to do
learned about include_vars (this is a task module that can be tagged)
learned about pre_tasks (since we're including the become credentials in the vault, regular tasks would never be reached because 'no SUDO credentials' would prevent tasks from being run)
so:
pre_tasks:
- include_vars: ~/.ansible/vault.yml
tags: engine
and, in Tower, set the job template to skip-tags: engine
Now the same play works in or out of Tower. With minimal authentication. Without plaintext passwords.
I'm trying to use the aws_ec2 inventory plugin for ansible. As I understand this is supposed to be the recommended method over the ec2.py inventory script. I got that gold nugget of information from ansible's own blog post and some random articles that mention it in the passing. However, as a novice to ansible, I find the documentation lacking.
3 problems I'm wandering about are:
A good way to pass secrets to the inventory definition file (I called it hosts.aws_ec2.yml). Jinja2 style of:
aws_secret_key: "{{ aws_secret }}"
produces a parse error.
As I understand this can be mitigating by setting env variables that will be picked up the plugin, but I'd like to get them from my ansible-vault encrypted file.
how do structure my host vars with this approach. So far, with a "static" inventory, I followed the structure outlined in this blog post -- one dir per environment. Any advice on how to deal with it using this plugin provided dynamic inventory?
is there a way I could specify ansible user and key for different inventory groups in a dynamic setting like this? Right now I set the globally, but I'd rather have the freedom to define them explicitly in the inventory.
Any advice is appreciated. Even a RTFM, as long as it's followed by some good reference links.
Assume that a normal deployment script does a lot of things and many of them are related to preparing the OS itself.
These tasks are taking a LOT of time to run even if there is nothing new to do and I want to prevent running them more often than, let's say once a day.
I know that I can use tags to filter what I am running but that's not the point: I need to make ansible aware that these "sections" executed successfully one hour ago and due to this, it would skip the entire block now.
I was expecting that caching of facts was supposed to do this but somehow I wasnt able to see any read case.
You need to figure out how to determine what "executed successfully" means. Is is just that a given playbook ran to completion? Certain roles ran to completion? Certain indicators exist that allow you determine success?
As you mention, I don't think fact caching is going to help you here unless you want to introduce custom facts into each host (http://docs.ansible.com/ansible/playbooks_variables.html#local-facts-facts-d)
I typically come up with a set of variables/facts that indicate a role has already been run. Sometimes this involves making shell calls and registering vars, looking at gathered facts and determining if certain files exist. My typical pattern for a role looks something like this
roles/my_role/main.yml
- name: load facts
include: facts.yml
- name: run config tasks if needed
include: config.yml
when: fact_var1 and fact_vars2 and inv_var1 and reg_var1
You could also dynamically write a yaml variable file that get's included in your playbooks and contains variables about the configured state of your environment. This is a more global option and doesn't really work if you need to look at the configured status of individual machines. An extension of this would be to write status variables to host_vars or group_vars inventory variable files. These would get loaded automatically on a host by host basis.
Unfortunately, as far as I know, fact caching only caches host based facts such as those created by the setup module so wouldn't allow you to use set_fact to register a fact that, for example, a role had been completed and then conditionally check for that at the start of the role.
Instead you might want to consider using Ansible with another product to orchestrate it in a more complex fashion.
My current Ansible project is setup like so:
backup-gitlab.yml
roles/
aws_backups/
tasks/
main.yml
backup-vm.yml
gitlab/
tasks/
main.yml
start.yml
stop.yml
backup-gitlab.yml needs to do the following:
Invoke stop.yml on the gitlab host.
Invoke backup-gitlab.yml on a different host.
Invoke start.yml on the gitlab host.
The problem I'm running into is Ansible doesn't seem to support a way of choosing which task files to run within the same role in the same playbook. Before I was using tags to control what Ansible would do, but in this case tagging the include statements for start.yml and stop.yml doesn't work because Ansible doesn't appear to have a way to dynamically change the applied tags that are run once they are set through the command line.
I can't come up with an elegant way to achieve this.
Some options are:
Have each task file be contained within its own role. This is annoying because I will end up with a million roles that are not grouped in any way. It's essentially abandoning the whole 'role' concept.
Use include with hard coded paths. This is prone to error as things move around. Also, since Ansible deprecated combining with_items with include (or using any sort of dynamic looping with include), I can no longer quickly change up the task files being run. Any minor change in my workflow requires lots of coding changes. I would really like to stick with using tags from the command line to control exactly what Ansible does.
Use shell scripts to invoke separate Ansible playbooks.
Use conditionals (when clause) on every single Ansible action, and control what gets run by setting variables. While several people have recommended this on SO, it sounds awful. I will have to add the conditional to hundreds of actions and every time I run a playbook the output will be cluttered by hundred's of 'skip' statements.
Leverage Jinja templates and ansible's local_connection to dynamically build static main.yml files with all the required task files included in the proper order (using computed relative paths). Then invoke that computed main.yml file. This is dangerous and convoluted.
Use top level Ansible plays to invoke lower level plays. Seems messy, also this brings in problems when I need to pass variables between plays. Using Ansible's Python Api may help this.
Ansible strives to bring VMs into idempotent states but this isn't very helpful and is a dated way of thinking in my opinion (I would have stuck with Chef if that is all I wanted). I want to leverage Ansible to actually do things such as: actively change configuration states, kick off processes, monitor events, react to events, etc. Essentially I want it to automate as much of my job as possible. The current 'role' structure (with static configurations) that Ansible recommends doesn't fit this paradigm very well even though their usage of remote command execution via SSH gets us so close to the dream.
Just use a playbook for these types of management tasks.
Granted the skip statements do somewhat clutter the output. If you wish to fix that you can further breakdown the roles into something like aws_backups-setup and aws_backups-managment.
Also the roles documentation has some information on how you can run pre_tasks and post_tasks on roles.
In Ansible 1.7, I can use --tags from the command-line to only run a subset of that playbooks tasks.
But I'm wanting to bake into my playbook to run a set of roles with only tasks that match tags. That is, I don't want to have to pass this in via the command-line since it will be the same every time.
At first I thought it was this command, but this does the opposite: tagging tasks with these tags instead of filtering them out based on this.
roles:
- { role: webserver, port: 5000, tags: [ 'web', 'foo' ] }
I can imagine implementing this using conditionals but tags would be a much more elegant way of achieving this.
You only have the following options with the current version of Ansible:
Specify the tags on the command line
Use a variable instead of a tag to conditionally run tasks
Split your webserver role into multiple roles and use role dependencies for the common tasks
This feature request has come up on the mailing list a few times and I haven't seen any indication from the dev team that it will be added as a new feature.
The use of conditionals can meet your needs.
You can see more information in my answer to another person's similar question here.