I have the following test setup and result:
$ cat hosts
[test]
abc-1
abc-ecs
abc-x
ecs-1
ecs-y
$ cat test-regex.yaml
- name: test
hosts:
- test
tasks:
- name: set var
set_fact:
myvar: "this is a test"
$ cat hosts | sed "s: ::g" | ggrep -P '^(?!ecs).*'
[test]
abc-1
abc-ecs
abc-x
$ ansible-playbook -i hosts -l ~'^(?!ecs).*' test-regex.yaml -D -C --list-hosts --list-tasks
playbook: test-regex.yaml
play #1 (test): test TAGS: []
pattern: ['test']
hosts (5):
abc-ecs
abc-x
ecs-1
ecs-y
abc-1
tasks:
set var TAGS: []
As shown with ggrep, ^(?!ecs).* does not match ecs-1 and ecs-y, but Ansible says it matches, can you explain?
I would like to apply to hosts starting with ecs only. I tested with Ansnile Core 2.13.3.
Also for testing purpose, can I maintain one file instead of two as shown here?
The question of the original post asks why -l ~'^(?!ecs).*' matches every host in the test group:
[test]
abc-1
abc-ecs
abc-x
ecs-1
ecs-y
but not those three abc-* hosts only as intended. This is because -l ~'^(?!ecs).*' not only matches hosts, but groups as well, and ansible creates two default groups: all and ungrouped. In this case, the regular expression matches:
test group which contains all 5 hosts listed above
all group which contains all 5 hosts listed above
ungrouped group which contains 0 hosts
abc-1 host
abc-x host
abc-ecs host
The union of all above consists the 5 hosts as shown in the output in the original post, so this is an expected behavior. To get what I intended, which is every hosts not starting with ecs, I should exclude the test and all groups also. The regular expression and the intended result are copied below.
$ ansible-playbook -i hosts -l ~'^(?!ecs|test|all).*' test-regex.yaml -D -C --list-hosts --list-tasks
playbook: test-regex.yaml
play #1 (test): regexcheck TAGS: []
pattern: ['test']
hosts (3):
abc-1
abc-x
abc-ecs
tasks:
set var TAGS: []
The "mystery" resolved.
Related
I am trying to run a playbook to create disks on VMs which are on different Vcenters.
so when I put them in an inventory file something like this:
**inv.yml:
vcenter1 datacenter=dc1 datastore=ds1 name=vm1
vcenter1 datacenter=dc1 datastore=ds1 name=vm2
vcenter1 datacenter=dc1 datastore=ds1 name=vm3
vcenter2 datacenter=dc2 datastore=ds2 name=vm4
vcenter2 datacenter=dc2 datastore=ds2 name=vm5
vcenter3 datacenter=dc3 datastore=ds3 name=vm6
vcenter3 datacenter=dc3 datastore=ds3 name=vm6**
When I run this it reads only once for each vcenter and ignores other entries.
Any way I can make Ansible read same hostname again and again.
I am able to do this with a variable file but I am trying to do this with an inventory file.
Q: "Any way I can make Ansible read same hostname again and again?"
A: No. There isn't any. Change the data instead. For example
shell> cat inv.yml
vcenter1 datacenter=dc1 datastore=ds1 name=[vm1,vm2,vm3]
vcenter2 datacenter=dc2 datastore=ds2 name=[vm4,vm5]
vcenter3 datacenter=dc3 datastore=ds3 name=[vm6]
shell> cat test.yml
- hosts: all
tasks:
- debug:
var: name
shell> ansible-playbook -i inv.yml test.yml
ok: [vcenter1] =>
name: '[vm1,vm2,vm3]'
ok: [vcenter2] =>
name: '[vm4,vm5]'
ok: [vcenter3] =>
name: '[vm6]'
I have the following groups defined in my inventory:
[webservers]
[server_storage]
[server_ws1]
[webservers:children]
server_storage
server_ws1
During my play, I'm in need of getting the names of the children groups of group['webservers'] ONLY (I think of theese as 'subgroups')
So, let's say I would need to set_fact a variable that contains in this case, the list of strings:
- server_storage
- server_ws1
This would have to be dynamic, so if I add group 'server_ws2' to group['webservers'], this should return
- server_storage
- server_ws1
- server_ws2
I've been playing with the use of group_names, group['webservers'] (which doesn't return subgroups, but hostnames)
Basically, I need a simple way of getting a the list of subgroups of a specific group. Is this possible without the use of black magic?
UPDATE: The idea, is that the hosts could belong to more groups, but I need only the subgroups or children of webservers group. It's constant, no matter the host, the output should allways be the same.
By the way, this one didn't work How can I get a list of child groups in Ansible?, because it retuns all groups for the current host, I need only subgroups of the specified group.
Q: "Get the names of the children groups of group['webservers']"
A: For example, the inventory
shell> cat hosts
[webservers]
[server_storage]
host_01
[server_ws1]
host_02
[webservers:children]
server_storage
server_ws1
and the playbook
shell> cat playbook.yml
- hosts: webservers
tasks:
- set_fact:
subgroups: "{{ subgroups|default([]) + hostvars[item].group_names }}"
loop: "{{ groups.webservers }}"
run_once: true
- debug:
msg: "{{ subgroups|unique|difference(['webservers']) }}"
run_once: true
give (abridged)
shell> ansible-playbook -i hosts playbook.yml
msg:
- server_storage
- server_ws1
As a workaround, it's possible to use ansible-inventory to list the structure of the inventory, store the output in a file, and use this file as needed, e.g. include_vars in a playbook. See the example below
shell> cat hosts
host_01
host_02
[webservers]
[server_storage]
[server_ws1]
[webservers:children]
server_storage
server_ws1
shell> ansible-inventory -i hosts --list --output hosts.json
shell> cat hosts.json
{
"_meta": {
"hostvars": {}
},
"all": {
"children": [
"ungrouped",
"webservers"
]
},
"ungrouped": {
"hosts": [
"host_01",
"host_02"
]
},
"webservers": {
"children": [
"server_storage",
"server_ws1"
]
}
I have a hosts file myEnv with:
[myEnv:children]
app0
app1
app2
And a group file myEnv with:
env: "myEnv"
In the following playbook task I'm attempting to pass environment (value of env key) and list of apps in that environment (myEnv:children) to a shell script as parameters. Only the environment ('myEnv') is getting passed to the script. I'm unable to figure out correct jinja2 syntax to pass the list of apps associated with this key which is in my hosts file.
- name: Run createFacts.sh in bin directory
command: ./createFacts.sh {{ env }} {{ hostvars[env] }}
register: createPuppetFacts
args:
chdir: "{{binHome}}"
What jinja2 syntax do I require for this? I've scoured ansible docs and stack overflow and just not finding right format...other than errors in syntax the best I can do is get an empty string back!
Much appreciate help on this.
Q: "Pass environment (value of env key) and list of apps in that environment (myEnv:children) to a shell script as parameters."
A: Given the inventory
$ cat hosts
test_01
test_02
test_03
[app0]
test_01
test_02
test_03
[app1]
test_01
test_02
[app2]
test_02
test_03
[myEnv:children]
app0
app1
app2
and the group_vars
$ cat group_vars/myEnv
env: "myEnv"
The playbook
$ cat playbook.yml
- hosts: test_01
tasks:
- command: "/home/admin/createFacts.sh {{ env }}"
register: result
- debug:
var: result.stdout
- command: "/home/admin/createFacts.sh {{ groups[env] }}"
register: result
- debug:
var: result.stdout
with the script
$ cat /home/admin/createFacts.sh
#!/bin/sh
echo $#
exit 0
gives
ok: [test_01] => {
"result.stdout": "myEnv"
}
ok: [test_01] => {
"result.stdout": "[utest_01, utest_02, utest_03]"
}
Notes
myEnv, app0, app1, app2 are groups. See Inheriting variable values: group variables for groups of groups
The value of env is myEnv. This is the name of a group. It's posible to list the members of the group groups[env]. But hostvars[env] must fail, because hostvars expects the name of a host as a parameter.
Ansible does not provide the runtime with the information that the group myEnv is parent of the groups app0, app1, app2.
There is a special variable group_names: "List of groups the current host is part of"
The task below
- debug:
var: group_names
gives
ok: [test_01] => {
"group_names": [
"app0",
"app1",
"myEnv"
]
}
Our inventory in INI style looks like this:
foo-host ansible_host=1.2.3.4 some_var=bla
bar-host ansible_host=5.6.7.8 some_var=blup
I can limit a playbook run to a single host by using the host alias:
$ ansible-playbook playbook.yml --limit foo-host
But I can't limit the run by mentioning the host's IP address from the ansible_host variable:
$ ansible-playbook playbook.yml --limit 1.2.3.4
ERROR! Specified hosts and/or --limit does not match any hosts
The reason I want to do that is because Ansible is triggered by an external system that only knows the IP address, but not the alias.
Is there a way to make this work? Mangling the IP address (e.g. ip_1_2_3_4) would be acceptable.
Things I've considered:
Turn it on its head and identify all hosts by IP address:
1.2.3.4 some_var=bla
5.6.7.8 some_var=blup
But now we can't use the nice host aliases anymore, and the inventory file is less readable too.
Write a custom inventory script that is run after the regular inventory, and creates a group like ip_1_2_3_4 containing only that single host, so we can use --limit ip_1_2_3_4. But there's no way to access previously loaded inventory from inventory scripts, so I don't know which groups to create.
Create the new groups dynamically using the group_by module. But because this is a task, it is run only after --limit has already decided that there are no hosts matching the pattern, and at that point Ansible just gives up and doesn't run the group_by task anymore.
Better solutions still welcome, but currently I'm doing it with a small inventory plugin, which (as opposed to an inventory script) does have access to previously added inventory:
plugins/inventory/ip_based_groups.py
import os.path
import re
from ansible.plugins.inventory import BaseInventoryPlugin
from ansible.inventory.group import Group
PATH_PLACEHOLDER = 'IP_BASED_GROUPS'
IP_RE = re.compile('^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$')
class InventoryModule(BaseInventoryPlugin):
'''
This inventory plugin does not create any hosts, but just adds groups based
on IP addresses. For each host whose ansible_host looks like an IPv4
address (e.g. 1.2.3.4), a corresponding group is created by prefixing the
IP address with 'ip_' and replacing dots by underscores (e.g. ip_1_2_3_4).
Use it by putting the literal string IP_BASED_GROUPS at the end of the list
of inventory sources.
'''
NAME = 'ip_based_groups'
def verify_file(self, path):
return self._is_path_placeholder(path)
def parse(self, inventory, loader, path, cache=True):
if not self._is_path_placeholder(path):
return
for host_name, host in inventory.hosts.items():
ansible_host = host.vars.get('ansible_host', '')
if self._is_ip_address(ansible_host):
group = 'ip_' + ansible_host.replace('.', '_')
inventory.add_group(group)
inventory.add_host(host_name, group)
def _is_path_placeholder(self, path):
return os.path.basename(path) == PATH_PLACEHOLDER
def _is_ip_address(self, s):
return bool(IP_RE.match(s))
ansible.cfg
[defaults]
# Load plugins from these directories.
inventory_plugins = plugins/inventory
# Directory that contains all inventory files, and placeholder to create
# IP-based groups.
inventory = inventory/,IP_BASED_GROUPS
[inventory]
# Enable our custom inventory plugin.
enable_plugins = ip_based_groups, host_list, script, auto, yaml, ini, toml
Q: Playbook running a script running a playbook... it would work, but it's a bit hacky
A: FWIW. It's possible to use json_query and avoid the script. For example
- hosts: all
gather_facts: false
tasks:
- set_fact:
my_host: "{{ hostvars|dict2items|json_query(query)|first }}"
vars:
query: "[?value.ansible_host == '{{ my_host_ip }}' ].key"
run_once: true
- add_host:
hostname: "{{ my_host }}"
groups: my_group
run_once: true
- hosts: my_group
gather_facts: false
tasks:
- debug:
var: inventory_hostname
Q: "Unfortunately I'm running ansible-playbook from AWX, so no wrapper scripts allowed."
A: It is possible to run the script from the playbook. For example the playbook below
- hosts: all
gather_facts: false
tasks:
- set_fact:
my_host: "{{ lookup('pipe', playbook_dir ~ '/script.sh ' ~ my_host_ip) }}"
delegate_to: localhost
run_once: true
- add_host:
hostname: "{{ my_host }}"
groups: my_group
run_once: true
- hosts: my_group
gather_facts: false
tasks:
- debug:
var: inventory_hostname
gives
$ ansible-playbook -e 'my_host_ip=10.1.0.53' play.yml
PLAY [all] ********************
TASK [set_fact] ***************
ok: [test_01 -> localhost]
TASK [add_host] ***************
changed: [test_01]
PLAY [my_group] ***************
TASK [debug] ************
ok: [test_03] => {
"inventory_hostname": "test_03"
}
(Fit the script to print the hostname only.)
Q: I can limit a playbook run to a single host by using the host alias:
$ ansible-playbook playbook.yml --limit foo-host
But I can't limit the run by mentioning the host's IP address from the ansible_host variable
$ ansible-playbook playbook.yml --limit 1.2.3.4
A: ansible-inventory and jq are able to resolve the host. For example the script
#!/bin/bash
my_host="$(ansible-inventory --list | jq '._meta.hostvars | to_entries[] | select (.value.ansible_host=="'"$1"'") | .key')"
my_host="$(echo $my_host | sed -e 's/^"//' -e 's/"$//')"
echo host: $my_host
ansible-playbook -l $my_host play.yml
with the inventory
test_01 ansible_host=10.1.0.51
test_02 ansible_host=10.1.0.52
test_03 ansible_host=10.1.0.53
and with the playbook.yml
- hosts: all
gather_facts: false
tasks:
- debug:
var: inventory_hostname
gives
$ ./script.bash 10.1.0.53
host: test_03
PLAY [all] **********
TASK [debug] ************
ok: [test_03] => {
"inventory_hostname": "test_03"
}
As the topic says, my question is if its possible to add tags to the hosts described inside the inventory?
My goal is to be able to run the ansible-playbook on specific host/group of hosts which has that specific tag e.g only on servers with tag 'Env=test and Type=test'
So for example when I run the playbook:
ansible-playbook -i hosts test.yml --extra-vars "Env=${test} Type=${test}"
I will pass the tags in the command and it will run only on the filtered hosts.
Thanks a lot!
Update:
Alternatively maybe doing something like in dynamic inventory? https://docs.ansible.com/ansible/latest/dev_guide/developing_inventory.html#developing-inventory
[tag_Name_staging_foo]
[tag_Name_staging_bar]
[staging:children]
tag_Name_staging_foo
tag_Name_staging_bar
To answer your question
Is it possible to add tags to hosts inside inventory to run the ansible-playbook on a specific host/group of hosts?
No, tags apply ONLY to the tasks
When you apply tags: attributes to structures other than tasks, Ansible processes the tag attribute to apply ONLY to the tasks they contain. Applying tags anywhere other than tasks is just a convenience so you don’t have to tag tasks individually.
Hosts don't have ansible "tags"; tasks have tags and they'e used to conditionally execute the tasks, not conditionally target hosts.
There's a few ways to conditionally target hosts, and the best way in my experience is ansible groups. Put the hosts you want to target in a group; then either target this group directly in a play:
- hosts: my_host_group
tasks: [ ... ]
Or limit the playook to a subset of the hosts targeted in a play:
ansible-playbook -l my_limited_hosts_group playbook.yaml
As other's pointed out, tags are not applicable to hosts. But...
I've found that you can use variables to filter hosts.
You can also put your hosts into multiple groups. Whether you do it dynamically, up to you.
I was searching for similar thing.
Anyway, given that as input:
all:
hosts:
test1:
tags:
- bar
test2:
host_var: value
tags:
- foo
test3:
tags:
- zap
test4: {}
vars:
group_all_var: value
children:
other_group:
children:
group_x:
hosts: test5
group_y:
hosts:
test6: {}
vars:
g2_var2: value3
hosts:
test4:
ansible_host: 127.0.0.1
last_group:
tags:
- foo
hosts: test3
vars:
group_last_var: value
You could just preprocess that hosts-config using a dynamic inventory or
use jq to filter by tag:
(I use yq to convert first to JSON; yq itself has not the full functionality of jq; best probably would be to just use oq):
yq -ojson . test.yaml \
| jq -re '{} as $n|path(..) as $p|getpath($p)|objects|select(.tags|IN(["foo"]))|del(.tags)|. as $o|$n|setpath($p;$o)' \
| jq -n 'reduce inputs as $item ({}; . *= $item)'
OUTPUT
{
"all": {
"hosts": {
"test2": {
"host_var": "value",
"tags": [
"foo"
]
}
},
"children": {
"last_group": {
"tags": [
"foo"
],
"hosts": "test3",
"vars": {
"group_last_var": "value"
}
}
}
}
}
And then you could just pack that into a shell-script called ansible-filterhosts and just do…
ansible-filterhosts -t foo hosts.yaml | ansible-playbook -i /dev/stdin myplaybook.yaml
…or…
ansible-playbook -i <(ansible-filterhosts -t foo hosts.yaml) myplaybook.yaml
…or develop a plugin for ansible itself ;)