I've been reading the ansible documentation on how to create a dynamic inventory. From what I understand I have to provide a json that is capable of outputing host_vars and group_vars.
With that in mind, how would I go about extending the group_vars and host_vars concepts to include the definition of service ?
In essence, my "end goal" would be to have something that allows me to define:
Host A has services A B C that would then turn into the corresponding host and group vars.
What is the best way to approach this?
I have been thinking about maybe a database but I'm not quite sure on how to propperly abstract the service concept.
Thanks in advance for any help
I cannot give you all your answers, I just started using Ansible four weeks ago. However, I have successfully integrated dynamic inventories. Here's what I can share: (extrapolate for your setup, I'm in a RHEL shop, using 6.9 and 7.4)
By default, ansible looks for your inventory in a file found at /etc/ansible/hosts The default format for that file is (I believe) INI format.
[servers]
server-1
server-2
[labhosts]
labhost-1
labhost-2
[localhost]
127.0.0.1
/etc/ansible/ansible.cfg will allow you relocate your inventory file/directory if needed. For now, my comments will assume no changes from the default
The example above is static inventory. You can move your /etc/ansible/hosts file aside, then do: mkdir /etc/ansible/hosts/
mv your hosts file, into /etc/ansible/hosts/hosts It's OK, to have static inventory files inside your dynamic directory (for now) So the beauty is, you can still use static inventory, it just now lives in /etc/ansible/etc/ <-- directory There is nothing special about the static filename. It can be any name, however some chars are not valid as part of the static file names.
To use dynamic inventory, you now only need to put into the /etc/ansible/hosts/ directory, executable scripts that pull your hostnames from some external database. AND, this is the KEY part, the output (the stdout) of that script MUST output in JSON format.
When ansible looks for your inventory files, it will "see" that /etc/ansible/hosts/ is a dir and then look in there for scripts. When you run a play or playbook, it will execute the script, and use the JSON output as your host targets of your play.
Now, I'm no JSON expert, but here's what works for me. The syntax of the JSON is like this: {"GROUPNAME":["HOST1","HOST2","HOST3",]}
So the entire string is bounded by left and right curly braces. The first field is the quoted groupname, separated by a colon, then the comma delimited list of quoted hosts, bounded by left and right square brackets.
In my environment, we have a perl script, and based on switch parameters, pulls lists of hostnames. We recently modified the perl script, using print statements to generate the JSON output. There is a JSON: perl module, but we didn't find it necessary to use, as formatting the output using print was sufficient. As for the groupname, we also "built" that groupname from the switch settings on the perl script.
So using my INI inventory example above, the JSON output would be something like this: {"servers":["server-1","server-2",]}
Note1: One quirk that I've learned, if you only have ONE host, it must be terminated with a comma. There's a reason, I'm not sure I can explain it. When we are generating our JSON output, we add a comma, regardless of the number of hosts, and it just works.
Note2: I realize this is not real JSON output, but it's working for our needs.
In your playbooks, you would put - hosts: all or - hosts: your_group_name
I usually just put - hosts: all, then limit using -i option and/or "--limit=hostname"
"-i", narrows your inventory to just the static or dynamic generated list
--limit=hostname where "hostname" is one of the subset of -i output.
Consider this command: ansible all -m ping
This will ping all hosts in your entire inventory. Both static and dynamic
ansible all -m ping -i servers
This will ping all hosts in your servers group
ansible all -m ping -i server --limit=server-1
This will ping just the one host, "server-1"
Using --limit= is great for testing plays or playbooks
When moving on to playbooks, you specify the hostlist, in the playbook.
Then you only need to add limits as needed, on the command line.
Good luck!
Related
I have a playbook with the following task that must copy the 2 Gb file from local to remote servers and extract files:
- name: Copy archived file to target server and extract
unarchive:
src: /path_to_source_dir/file.tar.gz
dest: /path_to_dest_dir
This task fails because ansible copies file to /home mount point on the target server and there's not enough space there:
sftp> put /path_to_source_dir/file.tar.gz /home/my_user_name/.ansible/tmp/ansible-tmp-1551129648.53-14181330218552/source
scp: /home/my_user_name/.ansible/tmp/ansible-tmp-1551129648.53-14181330218552/source: No space left on device
The reason for that is because ansible.cfg has a default parameter:
remote_tmp = ~/.ansible/tmp
How to overwrite this parameter from the playbook (if possible) and make ansible to copy file to the same destination directory specified in the task? So it would be like this:
remote_tmp = /path_to_dest_dir/.ansible/tmp
And the destination path is going to be different each time for a different target server!
Cleaning /home is not an option for me.
The answer here unfortunately is not very clear to me.
There are a few different ways to achieve what you are looking to do. Which one is a matter of preference and your use case.
You found the first way, setting an environment variable before running the playbook. Great for a quick on-the-fly job. Remembering to do that every time you run a certain playbook is indeed annoying. A slight variation of that is to use the environment keyword to set that variable for the play. You can also set environment variable in a role, block or a single task. https://docs.ansible.com/ansible/devel/reference_appendices/playbooks_keywords.html?highlight=environment%20directive. Here is an example of it in use: https://docs.ansible.com/ansible/devel/reference_appendices/faq.html?highlight=environment.
Using the environment keyword in a play et al works well for a specific application of automation, but what if you want Ansible to always use a different remote tmp path for specific servers? Like all variables, the remote_tmp can be sourced from inventory host and group variables not just the config file or environment variables. You need to mind you variable precedence if it is being set in different places. With this you could set remote_tmp in your inventory for that host or a group of hosts. Ansible will always use that path for that host or hosts in that group without having to define it in every play or roles. If you need to change that path, you change it in your inventory and it changes the behavior for all playbook runs without any additional edits. Here is an example of it being used as a host variable in static inventory: https://docs.ansible.com/ansible/devel/reference_appendices/faq.html?highlight=remote_tmp#running-on-solaris
So while the specific issue of "dynamically" setting the remote tmp directory on a host is not a best practice topic per se, it does become an example of the best practice of making the most of variables in Ansible.
For reference, remote temporary directories are handled by the shell plugins. While many parameters are shared, there are others that are specific to the shell Ansible using. Ansible uses sh by default. https://docs.ansible.com/ansible/latest/plugins/shell/sh.html
Hope that helps. Happy automating.
Is it possible within an Ansible host file to refer to hosts within a group like my provided sample or is there any other way of doing it directly in the host file? (I would prefer not to change existing playbooks or use limit flags)
# Handled by terraform with company policies (can't change this)
[web]
direct-15-67-156-6.bdb.company.com
direct-12-67-116-124.lia.company.com
[lb]
direct-12-68-117-13.osp.company.com
# BEGIN ANSIBLE MANAGED BLOCK
[mywebsite]
web[0]
[gatling]
web[1]
You can't do anything like you suggest with a static hosts file, however the static hosts file can be replaced with a script, and then you are free to use whatever logic you wish to build out your groups and hosts. Perhaps you can persuade Terraform to produce the data in a JSON file (or similar) that your inventory script could then consume?
Docs are here.
We are using vagrant and ansible to create standard development environments.
The ansible playbooks, vagrant files, etc. are in a git repository.
I've using variable file separation to refer to variable files in the developer's home directory for some senstitive and/or user-specific information (e.g. email address).
We use the variables by doing a vars_file: as part of the playbook, but have to do it for every play.
I don't want to put it in the group_vars/all file because it would then be in the repository and not specific to the user.
I would rather not have a file in the repository that is ignored because people still manage to include it and it screw everybody else up.
Is there a way of doing an equivalent of groups/all which can contain tasks and/or variable definitions that will automatically run whenever a playbook is run?
We use the variables by doing a vars_file: as part of the playbook, but have to do it for every play.
Nope, you can do it on playbook level. (But this might be a new thing, could have been impossible back then, I did not check.)
Is there a way of doing an equivalent of groups/all which can contain tasks and/or variable definitions that will automatically run whenever a playbook is run?
Automatically run/included when?! I don't think this is possible as there would be a lot of open questions like:
Should this be specified on the target machine or the ansible server?
How do you specify for which user should this happen on which host?
If there are tasks: do you want this to be executed on each playbook
when it is run using the given user? What about tasks which specifies
that they run as root (become)? What about tasks that specify a
given user to be executed as? What about tasks that are run as root
but creates a file with the owner matching the given user?
As there are no user scopes with variables and we don't really have a "user context" outlined (see the last questions) we are currently stuck with inclusion of variable files explicitly. Hence the below options:
You can keep using vars_file and specify a first found list.
vars_file:
- - ~/ansible_config/vars.yml
- <default vars file somewhere on the machine>
This way the ansible executor user can redefine values...
You can use the --extra-vars #<filepath> syntax to include all variables from a file, and you can have more than one of these.
A similar thing I do is that I include every variable from every yml file within my GLOBAL_INPUT_DIR (which is an environment variable that can be defined before running the bash script executing ansible-playbook or in a your bash profile or something).
EXTRA_ARGS=`{
{
find "${GLOBAL_INPUT_DIR}" -iname "*.yml";
}\
| while read line; do echo "--extra-vars #${line} "; done \
| tr -d "\n"
}`
ansible-playbook $# ${EXTRA_ARGS}
I usually include something like this in my doings to provide an easy way of redifining variables...
BUT: be aware that this will redefine ALL occurances of a variable name within the playbook (but this was also true with vars_file).
I have the requirement to skip some steps in my scripts when I run a deployment against production.
When a playbook is started, it always requires an environment (-i option), so there would be information I could query to distinguish which steps to take.
This leads me to ask:
How can I query the environment I am running a playbook in?
As an alternative, I could provide an extra variable as a parameter like -e "env=prod". But this would be redundant, since I have specified the environment already with -i...
Another option would be to set up a group environment, put all hosts of this environment in there, and define a group_var called env: prod. But putting all hosts in this group is overkill.
Bottom line: can I query the environment? Is there another option I'm not considering?
From Magic Variables in the Ansible documentation:
Also available, inventory_dir is the pathname of the directory holding Ansible’s inventory host file, inventory_file is the pathname and the filename pointing to the Ansible’s inventory host file.
Use string manipulation to extract the information you want from the above variable (e.g., the last segment from the path).
A filter exists to extract the last part of a pathname/filename :
managing-file-names-and-path-names
So you can use inventory_file | basename
I'm using Ansible is a mixed environment of AWS and non-AWS machines. I'd like to avoid passing hosts on the command line. How do I combine multiple host files in Ansible and make it the default? The current recommendation on the Ansible site is to override /etc/ansible/hosts with ec2.py. which prevents me from adding additional hosts. Thanks.
You can mix dynamic and static inventory files by creating a directory and dropping ec2.py in it plus your ini formatted inventory list as a separate file.
It is mentioned briefly in the docs here.
for example:
./inventory/ec2.py
./inventory/additional-hosts
ansible-playbook ... -i inventory/
Note that any file with the executable bit set will be treated as a dynamic inventory so make sure you files have the correct permissions.