I have an Excel spreadsheet that I've saved as a CSV file (comma deliminated) containing the IP addresses of multiple interfaces for a list of servers. There's been an interface configured on these servers (initially) so I have connectivity. I'd like to go through this file row-by-row and grab the necessary values to build the ifcfg file locally, copy to the server and then restart the network.
This is not the actual file; rather, a sample outlining the idea that there's a location with multiple IP addresses provided.
New Orleans, 192.168.10.42, 13, 192.168.3.10
Atlanta, 192.168.31.100, 18, 192.168.10.10
Detroit, 172.16.31.8, 43, 172.16.10.27
The goal is to parse this file and create the network interface files as follows
ifcfg-eth0
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
NM_CONTROLLED=no
IPADDR=192.168.10
ifcfg-ens3
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
NM_CONTROLLED=no
IPADDR=192.168.3.10
This looks to get me partway there
- debug:
loop: "{{ lookup('file', './file.one').splitlines() }}"
register: val
And returns
ok: [localhost] => (item=New Orleans, 192.168.10.42, 13, 192.168.3.10) => {
"msg": "Hello world!"
}
ok: [localhost] => (item=Atlanta, 192.168.31.100, 18, 192.168.10.10) => {
"msg": "Hello world!"
}
ok: [localhost] => (item=Detroit, 172.16.31.8, 43, 172.16.10.27) => {
"msg": "Hello world!"
}
I did find a snippet on StackOverflow using the_dict I used but everything is returned as a list and not a scalar :(
tasks:
- debug: var=the_dict
vars:
the_dict: >-
{%- set row = lookup("file", "~/Documents/Book1.csv").split("\n") | list -%}
{%- for i in row -%}
{%- set v = i.split(",") -%}
{%- set A_Location = v.0 -%}
{%- set B_Interface = v.1 -%}
...
...
...
to read csv with ansible, read module CSV
an example to use template
the template file conf.j2 (to put in templates directory)
{% for record in records.list %}
ifcfg-eth0
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
NM_CONTROLLED=no
IPADDR={{ record.ip1 }}
ifcfg-ens3
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
NM_CONTROLLED=no
IPADDR=1{{ record.ip2 }}
{% endfor %}
- name: playbook2.0
hosts: localhost
gather_facts: False
tasks:
- name: Read CSV file and return a list
community.general.read_csv:
path: file.csv
fieldnames: location,ip1,data,ip2
delimiter: ','
register: records
delegate_to: localhost
- name: generate fileconf.txt from conf.j2
template:
src: conf.j2
dest: fileconf.txt
you have the new file conf generated in the file fileconf.txt
you could adapt the file.j2 as you want with jinja2 syntax
Related
In my inventory I have 3 servers in a group. I want to be able to increase that size in the future so I can add more nodes into the network and generate the template with Jinja2.
- name: Gathering API results
shell:
cmd: "curl {{ groups['nodes'][node_index] }}/whatever/api/result "
loop: "{{ groups['nodes'] }}"
loop_control:
index_var: node_index
register: api_value
If I run some debug tasks hardcoding which list I want to use everyhing works fine
- debug: "msg={{ api_value.results.0.stdout }}"
- debug: "msg={{ api_value.results.1.stdout }}"
- debug: "msg={{ api_value.results.2.stdout }}"
output:
ok: [server-1] => {
"msg": "random-value-a"
ok: [server-2] => {
"msg": "random-value-b"
ok: [server-3] => {
"msg": "random-value-c"
The problem is when I try to increase the list number in Jinja template. I tried several for loops combination, nested for loops and many other things but nothing seems to be working.
For example I want my Jinja template look similar like this:
{% for vm in groups['nodes'] %}
NODE_{{ loop.index }}={{ api_value.results.{loop.index}.stdout }}
{% endfor %}
This way I want to achieve this output:
NODE_0=random-value-a
NODE_1=random-value-b
NODE_2=random-value-c
Is there any other way to workaround this? Or maybe is something I could do better in the "Gathering API results" task?
Given the inventory
shell> cat hosts
[nodes]
server-1
server-2
server-3
Either run it in the loop at a single host, e.g.
- hosts: localhost
gather_facts: false
vars:
whatever_api_result:
server-1: random-value-a
server-2: random-value-b
server-3: random-value-c
tasks:
- command: "echo {{ whatever_api_result[item] }}"
register: api_value
loop: "{{ groups.nodes }}"
- debug:
msg: "{{ api_value.results|json_query('[].[item, stdout]') }}"
gives
msg:
- - server-1
- random-value-a
- - server-2
- random-value-b
- - server-3
- random-value-c
Then, in the Jinja template, fix the index variable
- debug:
msg: |-
{% for vm in groups.nodes %}
NODE_{{ loop.index0 }}={{ api_value.results[loop.index0].stdout }}
{% endfor %}
gives what you want
msg: |-
NODE_0=random-value-a
NODE_1=random-value-b
NODE_2=random-value-c
Optionally, iterate api_value.results. This gives the same result
- debug:
msg: |-
{% for v in api_value.results %}
NODE_{{ loop.index0 }}={{ v.stdout }}
{% endfor %}
Or run it in the group, e.g.
- hosts: nodes
gather_facts: false
vars:
whatever_api_result:
server-1: random-value-a
server-2: random-value-b
server-3: random-value-c
tasks:
- command: "echo {{ whatever_api_result[inventory_hostname] }}"
register: api_value
delegate_to: localhost
- debug:
msg: "{{ api_value.stdout }}"
(delegate to localhost for testing)
gives
ok: [server-1] =>
msg: random-value-a
ok: [server-2] =>
msg: random-value-b
ok: [server-3] =>
msg: random-value-c
Then, in the Jinja template, use hostvars
- debug:
msg: |-
{% for vm in groups.nodes %}
NODE_{{ loop.index0 }}={{ hostvars[vm].api_value.stdout }}
{% endfor %}
run_once: true
gives also what you want
msg: |-
NODE_0=random-value-a
NODE_1=random-value-b
NODE_2=random-value-c
Optionally, iterate hostvars. This gives the same result
- debug:
msg: |-
{% for k,v in hostvars.items() %}
NODE_{{ loop.index0 }}={{ v.api_value.stdout }}
{% endfor %}
run_once: true
I want to define a dictionary variable that various host groups can add their own keys to in group_vars (not using set_fact). E.g. something like this:
group_vars\ftp_servers.yml:
important_ports:
ftp: 21
group_vars\web_servers.yml:
important_ports:
http: 80
so that when run on a server with both of these roles the dictionary is combined, i.e. important_ports =
{
ftp: 21,
http: 80
}
This is exactly what hash_behaviour = merge does, but it's deprecated and will be removed in Ansible 2.13. How do I achieve the same thing without that?
The only solution I've seen recommended is to use the combine filter:
set_fact:
important_ports: "{{ important_ports | combine({ http: 80 }) }}"
This works in a set_fact task, but fails in group_vars with "recursive loop detected in template string: {{ important_ports | combine({ http: 80 }) }}"
I even tried initialising the variable to empty dictionary (important_ports: {}) in group_vars/all, which is supposed to be evaluated before other group_vars, but it still gives the same error.
A (not so clean...) possible solution to this specific problem. This is based on convention where you use the group name in the var name for specific group ports.
Note that if you redefine the same port name with a different value, the latest loaded group will win.
Given the following all-in-one inventory placed in inventories/mergegroups/hosts.yml
---
all:
vars:
ansible_connection: local
important_ports: >-
{%- set result={} -%}
{%- for group in groups -%}
{{ result.update(lookup('vars', group + '_important_ports', default={})) }}
{%- endfor -%}
{{ result }}
ftp_servers:
vars:
ftp_servers_important_ports:
ftp: 21
hosts:
a:
b:
web_servers:
vars:
web_servers_important_ports:
http: 80
hosts:
a:
other_group:
vars:
other_group_important_ports:
mysql: 3306
hosts:
a:
b:
no_port_group:
# as you can see no port definition here.
hosts:
a:
b:
c:
We get the following results for the 3 different host:
$ ansible -i inventories/mergegroups/ a -m debug -a msg="{{ important_ports }}"
a | SUCCESS => {
"msg": {
"ftp": 21,
"http": 80,
"mysql": 3306
}
}
$ ansible -i inventories/mergegroups/ b -m debug -a msg="{{ important_ports }}"
b | SUCCESS => {
"msg": {
"ftp": 21,
"mysql": 3306
}
}
$ ansible -i inventories/mergegroups/ c -m debug -a msg="{{ important_ports }}"
c | SUCCESS => {
"msg": {}
}
I am trying to figure out a complex variable structure with jinja2 template in Ansible. I have tried different solutions with dictsort and "if" loop inside "for" loop, but I do not see any progress. Any help would be appreciated.
I am trying to print the virtual_ro_id based on the ansible_hostname. And hostnames are server1.dc1.com & server2.dc1.com, same for dc2 . The var file is given below.
datacenters:
dc1:
server1:
- virtual_ro_id: "60"
server2:
- virtual_ro_id: "60"
dc2:
server1:
- virtual_ro_id: "61"
server2:
- virtual_ro_id: "61"
This is what my template syntax looks like:
{% for dc in lookup('dict', datacenters) %}
{% set dc_name=ansible_fqdn.split(.)[1] %}
{% if 'dc' == dc_name %}
ID: {{ dc.ansible_hostname.virtual_ro_id }}
{% endif %}
{% endfor %}
I usually get syntax error or no value gets by the template. Thanks in advance.
Given the inventory
shell> cat hosts
server1.dc1.com
server2.dc1.com
server1.dc2.com
server2.dc2.com
the task
- debug:
var: datacenters[mydomain][myhost][0]['virtual_ro_id']
vars:
myhost: "{{ inventory_hostname.split('.').0 }}"
mydomain: "{{ inventory_hostname.split('.').1 }}"
gives
ok: [server1.dc1.com] => {
"datacenters[mydomain][myhost][0]['virtual_ro_id']": "60"
}
ok: [server2.dc1.com] => {
"datacenters[mydomain][myhost][0]['virtual_ro_id']": "60"
}
ok: [server1.dc2.com] => {
"datacenters[mydomain][myhost][0]['virtual_ro_id']": "61"
}
ok: [server2.dc2.com] => {
"datacenters[mydomain][myhost][0]['virtual_ro_id']": "61"
}
Is this probably what you're looking for?
I'm super fresh to ansible and creating a playbook that in one of the tasks should copy templated file and replace values in 2 lines. First line should have current hostname, and in second semicolon separated list of all other hosts (used in the play) - it will be different group
First line is super easy, as it's just:
localnode={{ inventory_hostname }}
but I'm having problem with exclusion in the second line. I'd like something similar to:
{% for host in groups.nodes -%} # but without inventory_hostname
othernodes={{ host }}{% if not loop.last %};{% endif %}
{%- endfor %}
Given the inventory of:
nodes:
hosts:
hosta:
hostb:
hostc:
hostd:
I'd like to get following output (example for hostd):
localnode=hostd
othernodes=hosta,hostb,hostc
I'll be very grateful for all hints on possible solution
Create the list of hosts without inventory_hostname and use it in the template
- set_fact:
list_other_hosts: "{{ groups.nodes|difference([inventory_hostname]) }}"
Simplify the template
othernodes={{ list_other_hosts|join(';') }}
As an example, the inventory
shell> cat hosts
test_jails:
hosts:
test_01:
test_02:
test_03:
and the play
- hosts: test_jails
tasks:
- set_fact:
list_other_hosts: "{{ groups.test_jails|
difference([inventory_hostname]) }}"
- debug:
msg: "{{ msg.split('\n') }}"
vars:
msg: |-
localnode={{ inventory_hostname }}
othernodes={{ list_other_hosts|join(';') }}
give
TASK [debug] ********************************************************
ok: [test_01] => {
"msg": [
"localnode=test_01",
"othernodes=test_02;test_03"
]
}
ok: [test_02] => {
"msg": [
"localnode=test_02",
"othernodes=test_01;test_03"
]
}
ok: [test_03] => {
"msg": [
"localnode=test_03",
"othernodes=test_01;test_02"
]
}
In a playbook I got the following code:
---
- hosts: db
vars:
postgresql_ext_install_contrib: yes
postgresql_pg_hba_passwd_hosts: ['10.129.181.241/32']
...
I would like to replace the value of postgresql_pg_hba_passwd_hosts with all of my webservers private ips. I understand I can get the values like this in a template:
{% for host in groups['web'] %}
{{ hostvars[host]['ansible_eth1']['ipv4']['address'] }}
{% endfor %}
What is the simplest/easiest way to assign the result of this loop to a variable in a playbook? Or is there a better way to collect this information in the first place? Should I put this loop in a template?
Additional challenge: I'd have to add /32 to every entry.
You can assign a list to variable by set_fact and ansible filter plugin.
Put custom filter plugin to filter_plugins directory like this:
(ansible top directory)
site.yml
hosts
filter_plugins/
to_group_vars.py
to_group_vars.py convert hostvars into list that selected by group.
from ansible import errors, runner
import json
def to_group_vars(host_vars, groups, target = 'all'):
if type(host_vars) != runner.HostVars:
raise errors.AnsibleFilterError("|failed expects a HostVars")
if type(groups) != dict:
raise errors.AnsibleFilterError("|failed expects a Dictionary")
data = []
for host in groups[target]:
data.append(host_vars[host])
return data
class FilterModule (object):
def filters(self):
return {"to_group_vars": to_group_vars}
Use like this:
---
- hosts: all
tasks:
- set_fact:
web_ips: "{{hostvars|to_group_vars(groups, 'web')|map(attribute='ansible_eth0.ipv4.address')|list }}"
- debug:
msg: "web ip is {{item}}/32"
with_items: web_ips
in playbook:
vars:
- arrayname:
- name: itemname
value1: itemvalue1
value2: itemvalue2
- name: otheritem
value1: itemvalue3
value2: itemvalue4
in template: (example is of type ini file, with sections, keys and values):
{% for item in arrayname %}
[{{ item.name }}]
key1 = {{ item.value1 }}
key2 = {{ item.value2 }}
{% endfor %}
This should render the template as:
[itemname]
key1 = itemvalue1
key2 = itemvalue2
[otheritem]
key1 = itemvalue3
key2 = itemvalue4
Variables can be represented as standard YAML structures so you can assign a list value to a key like this:
---
- hosts: db
vars:
postgresql_ext_install_contrib: yes
postgresql_pg_hba_passwd_hosts:
- '10.129.181.241/32'
- '1.2.3.0/8'
You can use jinja2 filters:
{{ groups['nodes']|map('extract', hostvars, ['ansible_eth1','ipv4', 'address']) |list }}
will return a list of ip addresses. i.e.
---
- hosts: db
vars:
postgresql_ext_install_contrib: yes
postgresql_pg_hba_passwd_hosts: {{ groups['nodes']|map('extract', hostvars, ['ansible_eth1','ipv4', 'address']) |list }}
...
Does not include the challange (appending /32). But it should also be possible somehow with jinja2 filters.
Reqiures ansible version >= 2.1
To add '/32' to the address, you can use the Ansible ipaddr filter (converting to CIDR notation).
{{ ip_addresses|ipaddr('host') }}