I try to build elasticsearch cluster with ansible and I have a problem with jinja2.
How can I set in jinja2 IP addresses of other hosts?
I have in my inventory.ini:
[elasticsearch]
192.168.0.1
192.168.0.2
192.168.0.3
and I want in jinja2 template to pass two addresses 192.168.0.2,192.168.0.3 when there is more hosts than 1, to look something like that:
- discovery.seeds_hosts=192.168.0.2,192.168.0.3
but when is one host in inventory.ini
it should look like this:
- discovery.seeds_hosts=192.168.0.1
I tried with something like this(hostnames):
{% for host in groups['elasticsearch'] %}{% if host == ansible_host %}{% else %}{%if loop.index0 > 0 %}{% endif %}elasticsearch-{{ loop.index }}{% if not loop.last %},{% endif %}{% endif %}{% endfor %}
and it works only for more than 1 host in elasticsearch group. If in the inventory.ini is only one host, the variable discovery.seeds_hosts will be empty.
Ansible version:
ansible [core 2.14.1]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.10/site-packages/ansible
ansible collection location = /home/ansible/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.10.9 (main, Dec 8 2022, 01:46:27) [GCC 10.2.1 20210110] (/usr/local/bin/python)
jinja version = 3.1.2
libyaml = True
You could use a conditional that checks the number of hosts:
{% if groups.elasticsearch|length > 1 %}
- discovery.seeds_hosts={{ groups.elasticsearch[1:]|join(',') }}
{% else %}
- discovery.seeds_hosts={{ groups.elasticsearch.0 }}
{% endif %}
I've tested this locally, and with multiple hosts in the inventory:
[elasticsearch]
192.168.0.1
192.168.0.2
192.168.0.3
This produces:
- discovery.seeds_hosts=192.168.0.2,192.168.0.3
With a single host in the inventory:
[elasticsearch]
192.168.0.1
This produces:
- discovery.seeds_hosts=192.168.0.1
Related
I have a string containing a jinja expression. How can I parse/evaluate/expand/ that string so that the expression is replaced the host variable running the role/task?
Detailed explanation:
I do have a proxmox cluster with multiple hosts. I want to manage their iptables with a dedicated role, but with separate data sources. The role should collect all variables that start with a specific prefix, collect their rules for filter and nat tables, sort them and write them into /etc/iptables/rules.v4via ansible.builtin.template.
Later I want to define multiple lxc/ and VMs inside group_vars, e.g. inventory/group_vars/proxmox/lxc_nginx.yml: ```
nginx_reverse_proxy:
host: '<some_cluster_node>'
network:
ip: 10.1.0.3
cidr: 24
iprules_nginx_reverse_proxy:
nat:
weight: 20
entries: |
-A PREROUTING -i {{ network_external_ip.external_device }} -p tcp --dport 80 -j DNAT --to-destination 10.1.0.3
with network_external_ip beeing defined for each node, in inventory/host_vars/some_cluster_node.yml:
network_external_ip:
ip: 1.1.1.1
gateway: 0.0.0.0
mask: 255.255.252.0
cidr: 22
external_device: ens3 # <<< this differs for each cluster node
now I collect all iprules, which might have been defined in multiple files, within my role and try to get them all converted into a sorted dict:
- name: basics/iptables | collect all rules
set_fact:
helper: |
{%- set filter = [] %}
{%- set nat = [] %}
{%- for entry in vars.items() %}
{%- if entry[0].startswith('iprules_') %}
{%- if entry[1].filter is defined %}
{%- set _ = filter.append(entry[1].filter) %}
{%- endif %}
{%- if entry[1].nat is defined %}
{%- set _ = nat.append(entry[1].nat) %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- set filter = filter | sort(attribute='weight') | map(attribute='entries') | join('\\n') %}
{%- set nat = nat | sort(attribute='weight') | map(attribute='entries') | join('\\n') -%}
filter: "{{ filter }}"
nat: "{{ nat }}"
- name: basics/iptables | create iptables dict
set_fact:
iptables_rules: "{{ helper | from_yaml }}"
- name: debug
debug:
var: iptables_rules
- name: basics/iptables | write ip rules
ansible.builtin.template:
src: rules.v4.j2
dest: /etc/iptables/rules.v4
which results in:
TASK [basics/iptables : basics/iptables | collect all rules] *****************************************************************************************************************************************************************************************************
ok: [<some_cluster_node>]
TASK [basics/iptables : basics/iptables | create iptables dict] **************************************************************************************************************************************************************************************************
ok: [<some_cluster_node>]
TASK [basics/iptables : debug] ***********************************************************************************************************************************************************************************************************************************
ok: [<some_cluster_node>] =>
iptables_rules:
filter: ''
nat: '-A PREROUTING -i {{ network_external_ip.external_device }} -p tcp --dport 80 -j DNAT --to-destination 10.1.0.3 '
Meaning, {{ network_external_ip.external_device }} never gets evaluated to "ens3". It stays the same, as in the inventory. How can I get this expression being parsed?
Information: The shown dicts here are WIP and do not fulfill certain criteria (like I would write all iprules_* into all hosts, regardless if they even host the lxc), but without the basics working this is no a concern right now.
PS: ansible version used
~ $ ansible --version
ansible [core 2.13.5]
config file = /home/apit/.ansible.cfg
configured module search path = ['/home/apit/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.10/site-packages/ansible
ansible collection location = /home/apit/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/lib/python-exec/python3.10/ansible
python version = 3.10.8 (main, Nov 27 2022, 20:55:16) [GCC 11.3.0]
jinja version = 3.1.2
libyaml = True
I have an Ansible template file which I'm applying correctly with the 'template' directive, but it's showing up on the remote machine with no substitutions:
- name: "buildAgent.properties for {{ agent_name }}"
template:
src: buildAgent.properties.j2
dest: "{{ config_path }}/buildAgent.properties"
The template file looks something like this:
serverUrl={{ teamcity_url }}
name={{ agent_name }}
{% if teamcity_agent_variables %}
{% for variable in teamcity_agent_variables %}
{{ variable }}={{ teamcity_agent_variables[variable] }}
{% endfor %}
{% else %}
# no teamcity_agent_variables from ansible
{% endif %}
and when it arrived on the remote machine, without errors from ansible, it looked exactly the same - even though when I displayed the variables in the step before the template step, they existed
Update: version stuff.
% ansible --version
ansible [core 2.13.3]
config file = /Users/timb/git/mre-ansible/ansible.cfg
configured module search path = ['/Users/timb/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/Cellar/ansible/6.3.0/libexec/lib/python3.10/site-packages/ansible
ansible collection location = /Users/timb/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.10.6 (main, Aug 30 2022, 05:12:36) [Clang 13.1.6 (clang-1316.0.21.2.5)]
jinja version = 3.1.2
libyaml = True
% grep error_on /Users/timb/git/mre-ansible/ansible.cfg
#error_on_missing_handler = True
error_on_undefined_vars = True
It turned out that the teamcity_agent_variables had been over-ridden in one file by a list instead of a hash, so the attempts to deference it failed.
I have no idea why Ansible didn't print a useful error when jinja failed - nor indeed when the type of the variable changed - instead of simply copying the template.
I have role for install & configure elasticsearchm and here is snippet from my elasticsearch.yml file ( to get cluster ip addresses )
discovery.seed_hosts: "{% set comma = joiner(", ") %}{% for host in groups['elasticsearch-tashkent'] -%}{{ comma() }}{{ hostvars[host]['ansible_eth0']['ipv4']['address']}}{%- endfor -%}"
and here is my inventory file
[elasticsearch-tashkent]
elasticsearch-01 ansible_host=178.154.241.134
and all works perfectly! but what should I do if, for example, a second group of hosts with a different name is added to my inventory file? how to make this block more flexible?
"{% set comma = joiner(", ") %}{% for host in groups['elasticsearch-tashkent'] -%}{{ comma() }}{{ hostvars[host]['ansible_eth0']['ipv4']['address']}}{%- endfor -%}"
I've spent most of the day trying to solve this problem and have thus far failed. I am building some playbooks to automate functions in Splunk, and am attempting to convert a list of hosts from an inventory group E.G.
[search_head]
1.2.3.4
5.6.7.8
My expected (desired) result from the debug output of the play should be:
https://1.2.3.4:8089, https://5.6.7.8:8089
I am attempting to complete this by running the following playbook against a running host:
---
- name: Build search head list to initialize the captain
hosts: search_head
remote_user: ansible
vars:
inventory_file: ./inventory-ec2-single-site
search_head_uri: "{{ lookup('template', './bootstrap-sh-deployer.j2') }}"
pre_tasks:
- include_vars:
dir: 'group_vars'
extensions:
- yml
- yaml
tasks:
- name: dump array
debug:
msg: "{{ search_head_uri }}"`
With the template bootstrap-sh-deployer.j2:
{%- set search_head_uri = [] %}
{% for host in groups['search_head'] %}
{%- if search_head_uri.append("https://{{ host }}:8089") %}
{%- endif %}
{%- if not loop.last %}, {% endif -%}
{%- endfor %}
However, the current play returns search_head_uri: ", " which tells me that the loop is running, but {{ host }} is not resolving.
Once you open a Jinja2 expression or a statement you should use Jinja2 syntax. You cannot nest them (i.e. you can't use {{ }} inside {% %}).
{%- if search_head_uri.append("https://" + host + ":8089") %}
This worked - Combination of the answer above to fix jinja formatting and using hostvars to get to the ansible_nodename.
{%- set search_head_uri = [] %}
{% for host in groups['search_head'] %}
{{ "https://" + hostvars[host]['ansible_nodename'] + ":8089" }}
{%- if not loop.last %}, {% endif -%}
{%- endfor %}
I use Vagrant with Ansible. In my playbook I have the following variable:
seeds: '192.168.56.11,192.168.56.12'
192.168.56.11 and 192.168.56.12 here are IP addresses of multi-machine Vagrant configuration.
Can I do my configuration more flexible using Ansible i.e. can Ansible compose this string programmatically for me?
You can use Jinja2 to template out your variable from other variables.
So if we have a list of things like this:
seeds:
- 192.168.56.11
- 192.168.56.12
We can turn that into a comma delimited string by looping through it with something like this:
seeds_string: '{% for seed in seeds %} {{ seed }}{% if not loop.last %},{% endif %}{% endfor %}'
As for getting the IP addresses of hosts in your inventory we can access facts about other hosts than the one being configured by using the groups and hostvars magic variables.
So to get the IP addresses of all hosts in the inventory we could use something like:
{% for host in groups['all'] %}
{{ hostvars[host]['ansible_eth0']['ipv4']['address'] }}
{% endfor %}
Combining this together we can then do something like this:
seeds: '{% for host in groups['all'] %} {{ hostvars[host]['ansible_eth0']['ipv4']['address'] }}{% if not loop.last %},{% endif %}{% endfor %}'