I'm using aws_rds plugin to create an inventory with my rds cluster/instances and create groups_vars based on the environment tags.
plugin: aws_rds
regions:
- xx-xxxx-x
include_clusters: true
keyed_groups:
- key: tags.Environment
prefix: "tag_Environment_"
separator: ""
https://docs.ansible.com/ansible/2.9/plugins/inventory/aws_rds.html
The problem is that this plugin doesn't allow to take the Endpoint name so I was wondering if it's possible to add the last part of this cluster-xxxxxxxxxx.xx-xxxx-x.rds.amazonaws.com on each group_var created, like ansible_host= ${ansible_host}.cluster-xxxxxxxxxx.xx-xxxx-x.rds.amazonaws.com or something like that.
Those AWS inventory plugin are gathering a lot of information from the API, I had quite a similar requirement lately on aws_ec2 inventory plugin and here is how I proceeded to find the information I needed under the compose parameter of the inventory's configuration.
To fetch what I needed to put under the compose parameter, I first configured the inventory, like you did.
Then I displayed all the collected hosts, using
ansible-inventory --graph
This yields something like
#all:
|--#nodes:
| |--node1
| |--node2
| |--node3
|--#ungrouped:
With this, I targeted one node, let's say node1, here, and I displayed all the information Ansible had on them, doing:
ansible -m setup node1
and
ansible -m debug -a "var=vars" node1
In all this, in searched for the information needed.
In your case that could be achieved by doing:
ansible -m setup <one-of-the-hosts> | grep "cluster-" | grep ".rds.amazonaws.com"
and
ansible -m debug -a <one-of-the-hosts> | grep "cluster-" | grep ".rds.amazonaws.com"
Last but not least, configured the variable I found out in the compose parameter of the inventory's configuration
compose:
ansible_host: <variable-name-found-at-step-4>
e.g., in my EC2 configuration, it ended up being public_dns_address
plugin: aws_ec2
hostnames:
- tag:Name
# ^-- Given that you indeed have a tag named "Name" on your EC2 instances
compose:
ansible_host: public_dns_address
In the end, I achieved what I wanted by defining this variable in groups_vars (so I could define one for each environment).
ansible_host: "{{ inventory_hostname }}.cluster-xxxxxxxx.xx-xxxx-x.rds.amazonaws.com"
I have a very complex Ansible setup with thousands of servers and hundreds of groups various servers are member of (dynamic inventory file).
Is there any way to easily display all groups that a specific host is member of?
I know how to list all groups and their members:
ansible localhost -m debug -a 'var=groups'
But I want to do this not for ALL hosts, but only for a single one.
Create a playbook called 'showgroups' (executable file) containing:
#!/usr/bin/env ansible-playbook
- hosts: all
gather_facts: no
tasks:
- name: show the groups the host(s) are in
debug:
msg: "{{group_names}}"
You can run it like this to show the groups of one particular host (-l) in your inventory (-i):
./showgroups -i develop -l jessie.fritz.box
There is group_names magic variable:
Additionally, group_names is a list (array) of all the groups the current host is in. This can be used in templates using Jinja2 syntax to make template source files that vary based on the group membership (or role) of the host
cat /etc/ansible/hosts | grep -e [[] && ansible all --list-hosts
The source of my host inventory is from an internal tool that outputs pairs of values, example, here are six observations, I currently have 160 observations:
servername1 processname1
servername1 processname2
servername1 processname3
servername2 processname1
servername3 processname1
servername4 processname1
So column 1 is my target host list (my inventory). Column2 are unique processname values, assigned specifically to the value of the servername. Often the same server will occur. Some servers have only one processname, others may have 2 to N. Meaning my target host may repeat for a unique list of processnames. I want to use both dynamic inventory from this output list of pairs, and I need both values on each observation to be associated and assigned to variables. I'm not absolutely required to use dynamic inventory, I just need a solution. I also need to pass to the target host and the value in {{ processname# }}, via the command: or shell: modules. (This is unique, there are no modules related to this need)
If required, I have a way to filter this data and output it in JSON format or YAML, making a separate YML file for each host. While I'd prefer to process these dynamically; pre-processing the list is acceptable.
Because ansible-playbook, requires some known host inventory list, I'm getting stuck understanding how I can create this list, from my dynamic output, at the time I start the play.
What I've done so far: I've tried reading up and trying to set these pairs as in the /etc/ansible/hosts/host_vars/servername#.yml files. This is extremely ugly, as I have to pre-process the output of the data, into YML format. But it does not give me a host list to reference in my playbook. So while it seems that hostvar is the logical choice, I cannot get my head around it.
What I need:
- The suggested format of the data? JSON? YAML? Other? (if I cannot read it in dynamically.
- Is putting this in host_vars correct?
- Last night I saw another answer using set_fact, would that help?
Thank you for any insight. I've now been using Ansible for 3.5 weeks! I've done pretty good using static and dynamic inventories, but this stumps me as the inventory list is not obvious, give the format of the matched pairs.
Note: MANY have suggested using host_vars, but that seems to me, to be reserved to hostnames, and related port and proxy values. I could be wrong.
===================================================================
UPDATE: Thanks for the help in the right direction.
I have updated our inventory script to output host list in JSON.
The first new option is to output the hosts in JSON.
Example:
{"my_host":["servername1","servername2",]}
Calling this as a dynamic inventory script, works great!
ansible all -m ping
servername1 | SUCCESS => {
"changed": false,
"ping": "pong"
}
servername2 | SUCCESS => {
"changed": false,
"ping": "pong"
}
Next: The second new option to the inventory script was to add a new switch to input a hostname. This part is still confusing me. Here is the output:
showInv --host=servername1
{"servername1":["processname1","processname2","processname3",]}
The final part that I am missing is how I call the inventory script with a specific "--host={{ my_host }} , from within my playbook.
It seems that I need to find the variable for the existing hostname and pass that back to the inventory script as the switch option "--host= "
You say that you are OK with dynamic inventories. Make your own.
Here is the docs.
You need to make a script that will do two things:
when executed with --list, processes your file and prints this JSON to stdout:
{ "myhosts": ["servername1", "servername2", "servername3"] }
when executed with --host servername1, prints this JSON to stdout:
{ "myprocesses": ["processname1", "processname2"] }
So with --list you should provide uniq list of hosts. In my example they belong to myhosts group.
And with --host <hostname> you should provide list dict of host vars for that host (<hostname>). In my example there is a list variable myprocesses that contains all processes for that host.
Then just call ansible-playbook -i my_inv_script myplaybook.yml.
Example playbook:
---
- hosts: myhosts
tasks:
- debug:
msg: "Process name is {{ item }}"
with_items: "{{ myprocesses }}"
This playbook will go trough all hosts in your dynamic inventory and print all processes for each host.
You will need to develop a dynamic inventory script, that takes the first column as the hostname, and the second column as variables for that host.
Please, find below the link to my dynamic inventory written in php
https://github.com/walden-it/ansible-ij/blob/master/inventory.php
take a look at the functions get_vars() and get_hosts() to see how is the array being populated.
And in case you need it, here is the dump for the database this script is looking at:
https://github.com/walden-it/ansible-ij/blob/master/ansible.sql
Then you just specify it with -i inventory in the ansible run, or add it as inventory_file to the ansible.cfg
Closing this out. With the help of Konstantin's suggestions, I now have a working play. What is not immediately apparent, is that Ansible is doing some "magic" behind the scenes. I had to modify my inventory script, that generates my dynamic inventory to accept the "--list" switch option, and the "--host hostname" option.
Once this was done, I could run the playbook with the -i listInv and Ansible internally calls this script as listInv --list, which produces my dynamic inventory list. Then it loops, through to the with_items, and internally calls the script as, listInv --host {{ items }} and outputs the matching processnames.
Additionally, the JSON output generated by my script, had to make the "group" (first) field, "myprocess". Initially, I had it as "my_process", and this failed. Removing the underscore, fixed, that error.
All working now. This is a great example for learning, but it's still magic.
The playbook looks like this:
- hosts: all
gather_facts: no
connection: local
tasks:
- debug:
msg: "Process name is {{ item }}"
with_items: "{{ myprocess }}"
I am relatively new to ansible.
I have a parameters.yml with something like this:
parameters:
stuff: bullshit
otherstuff: otherbullshit
domain_locale: domain.locale.test
domain_differentlocale1: domain.differentlocale1.test
domain_differentlocale2: domain.differentlocale2.test
domain_differentlocale3: domain.differentlocale3.test
uninteresting_stuff: uninteresting_bullshit
This file is in another directory, but I could also symlink it to be in defaults/main.yml. Doesn't matter.
What I would like to accomplish is grep somehow within an ansible task these domain.*.test values and use it as vars in another task with ansible. This other task would be to write it directly after 127.0.0.1 localhost in the same line in the /etc/hosts.
I am not even sure which of the modules to use in ansible. include_vars has no additional grep as I see it. set_fact does not read from a different file(can I combine it with a shell command? how?). lineinfile does not have a source and dest, so it is also limited to one file.
I also thought this post would help me: Ansible read multiple variables with same name from vars_file
but it does not need the grep part, I need.
Here you go:
---
- hosts: localhost
gather_facts: no
vars:
parameters:
stuff: bullshit
otherstuff: otherbullshit
domain_locale: domain.locale.test
domain_differentlocale1: domain.differentlocale1.test
domain_differentlocale2: domain.differentlocale2.test
domain_differentlocale3: domain.differentlocale3.test
uninteresting_stuff: uninteresting_bullshit
tasks:
- lineinfile:
dest: /tmp/test.txt
regexp: ^127.0.0.1
line: "127.0.0.1 localhost {{ parameters.values() | select('match','domain.*test') | list | join(' ') }}"
I set up group of test VMs (with pure Debian) to play with, all of them are on test network so it is self-contained group of VM. These VMs be destroyed soon so I don't want to add its names to DNS while I need to have them communicate oven hostnames.
Is there any way I can form /etc/hosts files on these VMs based on hosts IPs from Ansible hosts file?
I can set hostnames like that
192.168.0.5 myhostname=host01.local
192.168.0.6 myhostname=host02.local
so I think I can somehow form /etc/hosts like that:
{{ ip }} {{ myhostname }}
but can archive that. I think I can iterate over group['all'] and use lineinfile to add lines to /etc/hosts, but it won't work for me so far.
You can use the inventory_file variable to get the path filename of the hosts file:
Also available, inventory_dir is the pathname of the directory holding Ansible’s inventory host file, inventory_file is the pathname and the filename pointing to the Ansible’s inventory host file.
Reference
You can read its variables with something like:
tasks:
- local_action: shell cat {{ inventory_file }}
register: result
As recommended here. More on local actions here.
You can then use jinja filters and loops to use the variable.