Started to experiment with Ansible and using playbooks to automate some routine tasks on network devices. I was able to get some basic stuff working and learn in the process but I know my knowledge is limited so when I see this playbook and how much stuff seems redundant I have to assume there are better ways to eliminate some of the redundancy and make things cleaner and more efficient.
Example I want to try to use and explain in order to get some ideas on is around configuring a new vlan on a group of devices.
Typically a new vlan first needs to be configured on the two distribution switches and then there are specific interfaces on those two switches that we have to add the vlan to.
So, for this first part I have the two hosts in a group called "dist" in my hosts file:
[dist]
DIST01 ansible_host=10.10.1.1
DIST02 ansible_host=10.10.1.2
Then I created the following in my playbook:
- name: Add Heartbeat VLAN to DIST
hosts: dist
connection: local
gather_facts: no
tasks:
- name: Include Login Credentials
include_vars: secrets.yml
- name: Define Provider
set_fact:
provider:
host: "{{ ansible_host }}"
username: "{{ creds['username'] }}"
password: "{{ creds['password'] }}"
tasks:
- name: Ensure VLAN Exists
provider: "{{ provider }}"
nxos_vlan: vlan_id="2600" state=present host={{ ansible_host }}
- name: Ensure VLAN Name Configured
provider: "{{ provider }}"
nxos_vlan: vlan_id={{ item.vid }} name={{ item.name }} host={{ ansible_host }} state=present
with_items:
- { vid: 2600, name: Ansible Heartbeat VLAN }
- name: ASSIGN VLAN TO TRUNK PORTS
nxos_switchport:
interface: "{{ item.interface }}"
mode: trunk
trunk_vlans: "{{ item.vlan }}"
provider: "{{ provider }}"
with_items:
- { interface: po850, vlan: 2600 }
- { interface: po860, vlan: 2600 }
- { interface: po865, vlan: 2600 }
- { interface: po868, vlan: 2600 }
- { interface: po871, vlan: 2600 }
- { interface: po872, vlan: 2600 }
- { interface: po875, vlan: 2600 }
- { interface: po877, vlan: 2600 }
- { interface: po884, vlan: 2600 }
So, for each host in that group it iterates through a list of interfaces / ports and adds the vlan specified.
Question #1.
First thing that stands out as being "inefficient" in my mind is I don't believe its very wise to have to specify the "vlan: 2600" every where.
I would think I should just set the vlan as a variable some where (in the playbook? in some other file that gets called?) to be used in each case where it is needed.
Next set of tasks:
After the previous task the next requires us to connect to each access switch that needs the vlan to be deployed on and configure the new vlan there.
The issue I run into here is that the port-channel on each of these switches is a different interface #. So I can't apply the same config by just iterating through a list of devices.
For instance what I have to do is something like this:
host: ACCESS01 interface: po850 vlan: 2600
host: ACCESS02 interface: po860 vlan: 2600
host: ACCESS03 interface: po870 vlan: 2600
So for each host/switch you add the vlan to the interface associated with that switch.
I just created a new task for each device that specifies the interface to configure for that switch.
Example:
- name: Add Heartbeat VLAN to ACCESS01
hosts: ACCESS01
connection: local
gather_facts: no
tasks:
- name: Include Login Credentials
include_vars: secrets.yml
- name: Define Provider
set_fact:
provider:
host: "{{ ansible_host }}"
username: "{{ creds['username'] }}"
password: "{{ creds['password'] }}"
tasks:
- name: Ensure VLAN Exists
provider: "{{ provider }}"
nxos_vlan: vlan_id="2600" state=present host={{ ansible_host }}
- name: Ensure VLAN Name Configured
provider: "{{ provider }}"
nxos_vlan: vlan_id={{ item.vid }} name={{ item.name }} host={{ ansible_host }} state=present
with_items:
- { vid: 2600, name: Ansible Heartbeat VLAN }
- name: ASSIGN VLAN TO PORTS
nxos_switchport:
interface: "{{ item.interface }}"
mode: trunk
trunk_vlans: "{{ item.vlan }}"
provider: "{{ provider }}"
with_items:
- { interface: po850, vlan: 2600 }
- name: Add Heartbeat VLAN to ACCESS02
hosts: ACCESS02
connection: local
gather_facts: no
tasks:
- name: Include Login Credentials
include_vars: secrets.yml
- name: Define Provider
set_fact:
provider:
host: "{{ ansible_host }}"
username: "{{ creds['username'] }}"
password: "{{ creds['password'] }}"
tasks:
- name: Ensure VLAN Exists
provider: "{{ provider }}"
nxos_vlan: vlan_id="2600" state=present host={{ ansible_host }}
- name: Ensure VLAN Name Configured
provider: "{{ provider }}"
nxos_vlan: vlan_id={{ item.vid }} name={{ item.name }} host={{ ansible_host }} state=present
with_items:
- { vid: 2600, name: Ansible Heartbeat VLAN }
- name: ASSIGN VLAN TO PORTS
nxos_switchport:
interface: "{{ item.interface }}"
mode: trunk
trunk_vlans: "{{ item.vlan }}"
provider: "{{ provider }}"
with_items:
- { interface: po860, vlan: 2600 }
- name: Add Heartbeat VLAN to ACCESS03
hosts: ACCESS03
connection: local
gather_facts: no
tasks:
- name: Include Login Credentials
include_vars: secrets.yml
- name: Define Provider
set_fact:
provider:
host: "{{ ansible_host }}"
username: "{{ creds['username'] }}"
password: "{{ creds['password'] }}"
tasks:
- name: Ensure VLAN Exists
provider: "{{ provider }}"
nxos_vlan: vlan_id="2600" state=present host={{ ansible_host }}
- name: Ensure VLAN Name Configured
provider: "{{ provider }}"
nxos_vlan: vlan_id={{ item.vid }} name={{ item.name }} host={{ ansible_host }} state=present
with_items:
- { vid: 2600, name: Ansible Heartbeat VLAN }
- name: ASSIGN VLAN TO PORTS
nxos_switchport:
interface: "{{ item.interface }}"
mode: trunk
trunk_vlans: "{{ item.vlan }}"
provider: "{{ provider }}"
with_items:
- { interface: po870, vlan: 2600 }
And so you see... I know when I see things almost identical repeated over and over again I have to assume there is a better way and I just don't know enough yet to solve on my own.
Question #2. I suspect there is a better way to handle repeating the following for each task in the playbook:
tasks:
- name: Include Login Credentials
include_vars: secrets.yml
- name: Define Provider
set_fact:
provider:
host: "{{ ansible_host }}"
username: "{{ creds['username'] }}"
password: "{{ creds['password'] }}"
Question #3, Could I possibly just list this data some where, either in the playbook or another file maybe and then create a task that could iterate through the data to determine what port needs to be configured?
host: ACCESS01 interface: po850 vlan: 2600
host: ACCESS02 interface: po860 vlan: 2600
host: ACCESS03 interface: po870 vlan: 2600
Some sort of logic to this in my mind would be something like, if "host" equals "ACCESS01" then interface equals po850.
So the task could just be referencing variables that are populated depending on the host its currently working on?
Any thoughts and advice on improving both the playbook and my knowledge of things is greatly appreciated. I guess I'm look for the most "ansiblistic" way to accomplish this. That's not a word huh?
For Question#1, you can use like this:
- name: ASSIGN VLAN TO TRUNK PORTS
nxos_switchport:
interface: "{{ item.interface }}"
mode: trunk
trunk_vlans: "{{ item.vlan | default('2600') }}"
provider: "{{ provider }}"
with_items:
- interface: po850
- interface: po860
- interface: po865
- interface: po868
- interface: po871
- interface: po872
- interface: po875
- interface: po884
If you want to assign different vlan to one or more interface(s), then you can use like this:
- { interface: po850, vlan: 2700 }
Hope that help you.
Related
I am pretty new to Ansible as a network engineer and have found it breaking my brain. I've used basic loops in some Ansible playbooks. Now I'm trying something a bit more complex and I'm sure I'm missing something because it feels like it should be simple.
I want to take these variables in a playbook:
vars:
smb_tcp_ports:
- '139'
- '445'
smb_udp_ports:
- '137'
- '138'
smb_ips:
- '172.16.13.130'
- '172.16.13.0/26'
- '200X:8X0:fX4a:X053::/64'
- '200X:8X0:fX4a:X050::130'
and loop through them so I build a new variable like this:
vars:
smb_allowed_ips_tcp:
- { ip: "172.16.13.130", port: ['139','445'] }
- { ip: "172.16.13.0/26", port: ['139','445'] }
- { ip: "X001:8X0:fX4a:X053::/64", port: ['139','445'] }
- { ip: "X001:8X0:fX4a:X050::130", port: ['139','445'] }
smb_allowed_ips_udp:
- { ip: "172.16.13.130", port: ['137','138'] }
- { ip: "172.16.13.0/26", port: ['137','138'] }
- { ip: "200X:8X0:fX4a:X053::/64", port: ['137','138'] }
- { ip: "200X:8X0:fX4a:X050::130", port: ['137','138'] }
^^^ the above bit that I want to generate is the bit that I'm struggling with ^^^
Then I can send it to this:
- name: Allow SMB TCP
ufw:
rule: allow
src: '{{ item.0.ip }}'
port: '{{ item.1 }}'
proto: tcp
with_subelements:
- "{{ smb_allowed_ips_tcp }}"
- port
when: "'smbserver' in group_names"
- name: Allow SMB UDP
ufw:
rule: allow
src: '{{ item.0.ip }}'
port: '{{ item.1 }}'
proto: udp
with_subelements:
- "{{ smb_allowed_ips_udp }}"
- port
when: "'smbserver' in group_names"
The question used to have a lot of words here. I deleted it. Thanks Larsks. I hope this is clearer?
I tried set_facts, but there is loads of stuff I don't understand in examples i see, like adding | symbols and writing list, product etc, and I always end up breaking. It also doesnt seem to add as an array, it overwrites.
Answered here: using https://ansibledaily.com/process-complex-variables-with-set_fact-and-with_items/
---
- hosts: myhosts
gather_facts: true
become: true
vars:
smb_tcp_ports:
- '139'
- '445'
smb_udp_ports:
- '137'
- '138'
smb_ips:
- '172.16.13.130'
- '172.16.13.0/26'
- '200X:8X0:fX4a:X053::/64'
- '200X:8X0:fX4a:X050::130'
smb_ips_tcp: {}
tasks:
- name: Populate IPs in dict
set_fact:
smb_ips_tcp: "{{ smb_ips_tcp | combine({'ip': item}) }}"
with_items:
- "{{ smb_ips }}"
register: smbout
- name: Populate ports in dict
set_fact:
smb_ips_tcp: "{{ item | combine({'port': smb_tcp_ports}) }}"
with_items:
- "{{ smbout.results | map(attribute='ansible_facts.smb_ips_tcp') | list }}"
register: smbout
- name: smbout results
set_fact:
smb_ips_tcp: "{{ smbout.results | map(attribute='ansible_facts.smb_ips_tcp') | list }}"
- name: Allow SMB TCP
ufw:
rule: allow
src: '{{ item.0.ip }}'
port: '{{ item.1 }}'
proto: tcp
with_subelements:
- "{{ smb_ips_tcp }}"
- port
I think I had missed the register bit. So when I tried the register facts bit before it kept leaving me with one key value pair. Which was useless. The register though is allowing me to keep all the key values and use them again.
Unsure if this is a duplicate question now.
It sounds like you may have resolved your question, but I thought you might be interested in an alternative implementation. I would probably solve this using the product filter, which produces the cartesian product of two lists. For example, to produce smb_allowed_ips_tcp, I would write:
- name: create smb_allowed_ips_tcp
set_fact:
smb_allowed_ips_tcp: "{{ smb_allowed_ips_tcp + [{'ip': item.0, 'port': item.1}] }}"
loop: "{{ smb_ips|product(smb_tcp_ports)|list }}"
vars:
smb_allowed_ips_tcp: []
This produces a data structure that looks like:
TASK [debug] ******************************************************************************************
ok: [localhost] => {
"smb_allowed_ips_tcp": [
{
"ip": "172.16.13.130",
"port": "139"
},
{
"ip": "172.16.13.130",
"port": "445"
},
{
"ip": "172.16.13.0/26",
"port": "139"
},
{
"ip": "172.16.13.0/26",
"port": "445"
},
{
"ip": "200X:8X0:fX4a:X053::/64",
"port": "139"
},
{
"ip": "200X:8X0:fX4a:X053::/64",
"port": "445"
},
{
"ip": "200X:8X0:fX4a:X050::130",
"port": "139"
},
{
"ip": "200X:8X0:fX4a:X050::130",
"port": "445"
}
]
}
We can feed that to the ufw module like this:
- name: Allow SMB TCP
ufw:
rule: allow
src: '{{ item.ip }}'
port: '{{ item.port }}'
proto: tcp
loop: "{{ smb_allowed_ips_tcp }}"
There are fewer tasks required for this solution, and I think the logic is a little easier to follow.
I have a role which uses with_items:
- name: Create backup config files
template:
src: "config.yml.j2"
dest: "/tmp/{{ project }}_{{ env }}_{{ item.type }}.yml"
with_items:
- "{{ backups }}"
I can access the item.type, as usual, but not project or env which are defined outside the collection:
deploy/main.yml
- hosts: ...
vars:
project: ...
rails_env: qa
roles:
- role: ../../../roles/deploy/dolly
project: "{{ project }}"
env: "{{ rails_env }}"
backups:
- type: mysql
username: ...
password: ...
The error I get is:
Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while templating '{{ project }}'
The template, config.j2.yml, is:
type: {{ item.type }}
project: {{ project }}
env: {{ env }}
database:
username: {{ item.username }}
password: {{ item.password }}
It turns out for can't redefine a var with the same name as an existing var, so project: {{ project }} will always fail with an error.
Instead project can be omitted and the existing definition, in vars, can be used.
- hosts: ...
vars:
project: ... # <- already defined here
roles:
- role: ../../../roles/deploy/dolly
backups:
- type: mysql
username: ...
password: ...
If the var is not defined in vars can be defined in the role:
- hosts: ...
vars:
name: ...
roles:
- role: ../../../roles/deploy/dolly
project: "{{ name }}" # <- define here
backups:
- type: mysql
username: ...
password: ...
I need to get JMX metrics from the Hazelcast product. I have created a Logstash process that connects to the JMX port. This process has to read a json where is the information of the hostname, port, cluster, environment, etc of Hazelcast JMX. I need to deploy on the Logstash machines the json file for each Hazelcast machine / port. In this case there are three Hazelcast machines and a total of 6 processes with different ports.
Example data:
Hazelcast Hostnames: hazelcast01, hazelcast02, hazelcast03
Hazelcast Ports: 6661, 6662, 6663, 6664, 6665
Logstash Hostnames: logstash01, logstash02, logstash03
Dictionary of Hazelcast information in Ansible:
logstash_hazelcast_jmx:
- hazelcast_pre:
name: hazelcast_pre
port: 15554
cluster: PRE
- hazelcast_dev:
name: hazelcast_dev
port: 15555
cluster: DEV
Example of task in Ansible:
- name: Deploy HAZELCAST JMX config
template:
src: "hazelcast_jmx.json.j2"
dest: "{{ logstash_directory_jmx_hazelcast }}/hazelcast_jmx_{{ item }}_{{ item.value.cluster }}.json"
owner: "{{ logstash_system_user }}"
group: "{{ logstash_system_group }}"
mode: 0640
with_dict:
- "{{ groups['HAZELCAST'] }}"
- logstash_hazelcast_jmx
The final result should be as follows:
/opt/logstash/jmx/hazelcast/hazelcast_jmx_hazelcast01_DEV.json
/opt/logstash/jmx/hazelcast/hazelcast_jmx_hazelcast01_PRE.json
/opt/logstash/jmx/hazelcast/hazelcast_jmx_hazelcast02_DEV.json
...
Here is an example of the json content:
{
"host" : "{{ hostname of groups['HAZELCAST' }}",
"port" : {{ item.value.port }},
"alias" : "{{ hostname of groups['HAZELCAST' }}_{{ item.value.cluster }}",
"queries" : [
{
"object_name" : "com.hazelcast:instance=_hz_{{ item.value.cluster }},type=XXX,name=YYY",
"attributes" : [ "size", "localHits" ],
"object_alias" : "Hazelcast_map"
} ,{
"object_name" : "com.hazelcast:instance=_hz_{{ item.value.cluster }},type=IMap,name=user",
"attributes" : [ "size", "localHits" ],
"object_alias" : "Hazelcast_map"
}
]
}
I think the problem I have is that the with_dict option does not allow using a listing of inventory hosts and a dictionary.
How can I get this generation of json files for each machine / port?
If you run your playbook against logstash hosts, you can use with_nested:
---
- hosts: logstash_hosts
tasks:
- name: Deploy HAZELCAST JMX config
template:
src: "hazelcast_jmx.json.j2"
dest: "{{ logstash_directory_jmx_hazelcast }}/hazelcast_jmx_{{ helper_host }}_{{ helper_cluster }}.json"
owner: "{{ logstash_system_user }}"
group: "{{ logstash_system_group }}"
mode: 0640
with_nested:
- "{{ groups['HAZELCAST'] }}"
- "{{ logstash_hazelcast_jmx }}"
vars:
helper_host: "{{ item.0 }}"
helper_cluster: "{{ item.1.cluster }}"
helper_port: "{{ item.1.port }}"
I also used helper variables with more meaningful names. You should also modify your template with either helper vars or item.0, item.1 – where item.0 is a host from HAZELCAST group, and item.1 is an item from logstash_hazelcast_jmx list.
Is it possible to call a role multiple times in a loop like this:
vars:
my_array:
- foo
- bar
- baz
roles:
- role: foobar
with_items: my_array
How can we do this?
Now supported as of Ansible 2.3.0:
- name: myrole
with_items:
- "aone"
- "atwo"
include_role:
name: myrole
vars:
thing: "{{ item }}"
There's no way to loop over a role currently but as mentioned in that Google Group discussion you can pass a list or dict to the role and then loop through that internally.
So instead you could do something like:
# loop_role/tasks/main.yml
- name: debug item
debug: var="{{ item }}"
with_items: my_array
And then use it like this:
- hosts: all
vars:
my_array:
- foo
- bar
- baz
roles:
- { role: loop_role, my_array: "{{ my_array }}" }
I used something like below on Ansible version 2.8
tasks:
- name: looping role to create multiple filesystem
include_role:
name: /opt/ansible/playbook/app_filesystem
vars:
vgname: "{{ item.vgname }}"
lvname: "{{ item.lvname }}"
lvsize: "{{ item.lvsize }}"
mountpoint: "{{ item.mountpoint }}"
loop:
- { vgname: 'vgapp', lvname: 'lvapp', lvsize: '30g', mountpoint: '/app' }
- { vgname: 'vgapp', lvname: 'lvappzk', lvsize: '64g', mountpoint: '/app/z' }
- { vgname: 'vgapp', lvname: 'lvappdatazk', lvsize: '+100%FREE', mountpoint: '/app/data/zookeeper' }
tasks:
- name: looping role to create multiple filesystem
include_role:
name: /opt/ansible/playbook/app_filesystem
vars:
vgname: "{{ item.vgname }}"
lvname: "{{ item.lvname }}"
lvsize: "{{ item.lvsize }}"
mountpoint: "{{ item.mountpoint }}"
loop:
- { vgname: 'vgapp', lvname: 'lvapp', lvsize: '30g', mountpoint: '/app' }
- { vgname: 'vgapp', lvname: 'lvappzk', lvsize: '64g', mountpoint: '/app/zookeeper' }
- { vgname: 'vgapp', lvname: 'lvappdatazk', lvsize: '+100%FREE', mountpoint: '/app/data/zookeeper' }
You can do so using the include_role module. See docs
According to the docs it was introduced in Ansible 2.2 already (not in 2.3 as others have stated).
The code would then look like
- name: Use role in loop
ansible.builtin.include_role:
name: my-role
vars:
some_role_variable: '{{ loop_var }}'
loop:
- '{{ roleinput1 }}'
- '{{ roleinput2 }}'
loop_control:
loop_var: loop_var
Here is a code sample for using include_role looping on my_array:
- name: Use role in loop
include_role:
name: myrole
loop: "{{ my_array }}"
I am creating an ansible playbook using version 2.0.0-0.3.beta1
I'd like to get the subnet id after i create a subnet. I'm refering the official ansible docs : http://docs.ansible.com/ansible/ec2_vpc_route_table_module.html
- name: Create VPC Public Subnet
ec2_vpc_subnet:
state: present
resource_tags: '{"Name":"{{ prefix }}_subnet_public_0"}'
vpc_id: "{{ vpc.vpc_id }}"
az: "{{ az0 }}"
cidr: 172.16.0.0/24
register: public
- name: Create Public Subnet Route Table
ec2_vpc_route_table:
vpc_id: "{{ vpc.vpc_id }}"
region: "{{ region }}"
tags:
Name: Public
subnets:
- "{{ public.subnet_id }}"
routes:
- dest: 0.0.0.0/0
gateway_id: "{{ igw.gateway_id }}"
after running the playbook i received following error:
fatal: [localhost]: FAILED! => {"failed": true, "msg": "ERROR! 'dict object' has no attribute 'subnet_id'"}
Try using: public.subnet.id instead of public.subnet-id
Its useful to debug by running this task:
- debug: msg="{{ public }}"