Ansible Update multiple firewall rules on Vyos - ansible

I've been trying to create a playbook that I can run periodically to go through all my Vyos firewall rules and ensure the "log enabled" command is present, just in case someone forgets to add logging to a firewall rule. I've found the vyos.vyos.vyos_firewall_rules module which I think will be perfect for what I need to do.
The only problem is, is that this module requires you input the rule set name and rule number of each firewall rule that you want to update. However in my case, I want this to be done automatically and Ansible to go through each firewall rule set and associated rules that are present, and ensure logging is enabled on each rule. Something like this is what I need:
- name: Enable logging for each firewall rule
vyos.vyos.vyos_firewall_rules:
config:
- afi: ipv4
rule_sets:
- name: *all rules sets*
rules:
- number: *all numbers*
log: enabled
I've used vyos.vyos.vyos_firewall_rules to gather a dump of all rule sets and associated rules and have filtered this down to list each rule set name along with each associated rule number :
- name: Get rulesets
vyos.vyos.vyos_firewall_rules:
config:
state: gathered
register: output
- name: Filter output and populate the list of rule set names
debug:
msg: "Rule set name: {{ item.0.name }}, rule number: {{ item.1.number }}"
loop: "{{ output.gathered[0]['rule_sets'] | subelements('rules') }}"
This produces output like this:
"Rule set name: ruleset-1, rule number: 1"
"Rule set name: ruleset-1, rule number: 2"
"Rule set name: ruleset-1, rule number: 15"
"Rule set name: ruleset-1, rule number: 20"
"Rule set name: ruleset-2, rule number: 1"
"Rule set name: ruleset-2, rule number: 2"
I'm a bit stuck on where to go from here. I feel like I need the info filtered into a nested list like I have below, and then somehow loop the vyos.vyos.vyos_firewall_rules module to update each rule set name and rule number.
firewall_rules:
ruleset-1:
1
2
15
20
ruleset-2
1
2
I haven't been able to figure out how to create a nested list, or if I even need one in the first place.
I'm relatively new to Ansible so if anyone could point me in the right direction I would appreciate it.

You can create the dictionary first
- set_fact:
firewall_rules: "{{ dict(rsets|zip(rules)) }}"
vars:
rsets: "{{ output.gathered.0.rule_sets|
map(attribute='name')|list }}"
rules: "{{ output.gathered.0.rule_sets|
map(attribute='rules')|
map('map', attribute='number')|list }}"
gives
firewall_rules:
ruleset-1:
- 1
- 2
- 15
- 20
ruleset-2:
- 1
- 2
Then use Jinja to create the structure, e.g.
- debug:
var: _config|from_yaml
vars:
_config: |-
- afi: ipv4
rule_sets:
{% for set, rules in firewall_rules.items() %}
- name: {{ set }}
rules:
{% for rule in rules %}
- number: {{ rule }}
{% endfor %}
{% endfor %}
gives
_config|from_yaml:
- afi: ipv4
rule_sets:
- name: ruleset-1
rules:
- number: 1
- number: 2
- number: 15
- number: 20
- name: ruleset-2
rules:
- number: 1
- number: 2
Fit the structure to your needs and use it in the module, e.g.
- vyos.vyos.vyos_firewall_rules:
config: "{{ _config|from_yaml }}"
vars:
_config: |-
...
Q: "Output of {{ _config|from_yaml }} is not preserving the hyphens."
A: The format you see depends on the callback plugin. If you want to see YAML set DEFAULT_STDOUT_CALLBACK to yaml
shell> ANSIBLE_STDOUT_CALLBACK=yaml ansible-playbook test-753.yml
or copy the content to a file
- copy:
dest: test-753-out.yml
content: |
{{ _config }}
vars:
_config: |-
- afi: ipv4
rule_sets:
{% for set, rules in firewall_rules.items() %}
- name: {{ set }}
rules:
{% for rule in rules %}
- number: {{ rule }}
{% endfor %}
{% endfor %}
gives
shell> cat test-753-out.yml
- afi: ipv4
rule_sets:
- name: ruleset-1
rules:
- number: 1
- number: 2
- number: 15
- number: 20
- name: ruleset-2
rules:
- number: 1
- number: 2

Related

Ansible format list variable

I'm working on setting up some automation for an F5 BigIP load balancer. When creating the virtual server, I have a variable containing the various profiles I want to include and also a few that should always be included.
I want to use this variable:
domains:
- foo.example.com
- bar.example.com
- baz.example.com
In the following module under profiles. Note, I can't do a loop because that would replace the value each time. I want all list items from domains to be expanded in the single execution of this task. I've tried using a Jinja for loop but it just hangs when I try to execute.
- name: Configure virtual server
f5networks.f5_modules.bigip_virtual_server:
state: present
partition: Common
name: insite-ssl
destination: 10.10.10.10
port: 443
pool: example-pool
snat: Automap
description: Testing VIP
profiles: |
{% for domain in domains %}
- name: {{ domain }}
context: client-side
{% endfor %}
- http-SSL-XForwarded
- name: example.com_wildcard
context: client-side
provider: "{{ f5_conn }}"
tags:
- vip
What is the right way to solve this?
By using this construct, you are actually creating a string, not a list.
This can be tested doing:
- set_fact:
what_am_I: |
{% for domain in domains %}
- name: {{ domain }}
context: client-side
{% endfor %}
- http-SSL-XForwarded
- name: example.com_wildcard
context: client-side
- debug:
var: what_am_I is string
Which gives:
TASK [set_fact] ******************************************************************
ok: [localhost]
TASK [debug] *********************************************************************
ok: [localhost] =>
what_am_I is string: true
You could use json_query in order to create a list of dictionaries out of your domains list, something like:
- name: Configure virtual server
f5networks.f5_modules.bigip_virtual_server:
state: present
partition: Common
name: insite-ssl
destination: 10.10.10.10
port: 443
pool: example-pool
snat: Automap
description: Testing VIP
profiles: |
{{ domains | json_query("[].{name: #, context: 'client-side'}") + extra_domains }}
provider: "{{ f5_conn }}"
tags:
- vip
vars:
extra_domains:
- http-SSL-XForwarded
- name: example.com_wildcard
context: client-side

How can I loop over a list of dicts and their content lists

I have the following var
---
- hosts: all
vars:
new_service:
name: test
Unit:
- option: Description
value: "Testname"
Service:
- option: ExecStart
value: "/usr/bin/python3 -m http.server 8080"
- option: WorkingDirectory
value: /home/test/testservice/html
I want to be able to use the ini_file module to create a service template so that the above var is converted into the following ini file
[Unit]
Description=Testname
[Service]
ExecStart=/usr/bin/python3 -m http.server 8080
WorkingDirectory=/home/test/testservice/html
I cannot figure out how to loop over this. I was thinking to use the product() so as to loop over nested lists, maybe something like this?
- name: "Create new unit section of service file"
ini_file:
path: "~{{ USERNAME }}/.config/systemd/user/{{ new_service[name] }}"
section: "{{ item.0 }}"
option: "{{ item.1.option }}"
value: "{{ item.1.value }}"
loop: "{{ ['unit', 'service'] | product({{ new_service[item.0] }})"
But I do not believe item is defined in the loop definition itself
(The reason I'm going with ini_file rather than template is because I want the service file creation to be able to handle any number of fields on demand)
You can still use a template to have a variable number of sections and options. Using loop with ini_file here is not efficient IMO. The only real use case would be if you need to keep the original contents of the file only adding new ones. But performance will be dramatically lower than a single template, especially if your have a lot of elements.
The only difficulty I see is that you have a name attribute in your dict which is not a section title. But it can be easily ruled out.
template.j2
{% for section in new_service %}
{% if section != 'name' %}
[{{ section }}]
{% for option in new_service[section] %}
{{ option.option }}={{ option.value }}
{% endfor %}
{% endif %}
{% endfor %}
Back to original question
If you really want to go through the loop route, it is still possible but will require quite a bit of effort with your actual data structure (loop/set_fact/... to finally get a single loopable structure).
If possible, I would change it to the following:
new_service:
name: test
sections:
- section: Unit
options:
- option: Description
value: "Testname"
- section: Service
options:
- option: ExecStart
value: "/usr/bin/python3 -m http.server 8080"
- option: WorkingDirectory
value: /home/test/testservice/html
And you can then directly loop through this structure using the subelements lookup. Note that "name" (on top level) is not a var but a string identifier for your service name value and should be used as such (fixed in my below example):
- name: "Create new unit section of service file"
ini_file:
path: "~{{ USERNAME }}/.config/systemd/user/{{ new_service.name }}"
section: "{{ item.0.section }}"
option: "{{ item.1.option }}"
value: "{{ item.1.value }}"
loop: "{{ lookup('subelements', new_service.sections, 'options') }}"
You can easily adapt my first example template to this new data structure as well if needed.

How to get all the node list IP addresses?

In my case, there are four nodes which runs ansible. I want to get each and every node ip address. Therefore i tried these.
In my playbook.yml
- name: Ansible
hosts: all
gather_facts: true
vars:
ansible_ec2_local_ipv4: "{{ ansible_default_ipv4.address }}"
roles:
- role: "ansible-mongo/roles/mongo"
- role: "ansible-mongo/roles/replication"
In my main.yml
- name: ensure file exists
copy:
content: ""
dest: /tmp/myconfig.cfg
force: no
group: "{{ mongodb_group }}"
owner: "{{ mongodb_user }}"
mode: 0555
- name: Create List of nodes to be added into Cluster
set_fact: nodelist={%for host in groups['all']%}"{{hostvars[host].ansible_eth0.ipv4.address}}"{% if not loop.last %},{% endif %}{% endfor %}
- debug: msg=[{{nodelist}}]
- name: Set Cluster node list in config file
lineinfile:
path: "/tmp/myconfig.cfg"
line: "hosts: [{{ nodelist }}]"
But as a result when i tried to view /tmp/myconfig.cfg file. I only get one IP.
cat /tmp/myconfig.cfg
hosts: ["10.1.49.149"]
Any idea on this?
Your set_fact loop is overwriting the value of 'nodelist' on each pass, effectively meaning you only ever end up with the last element in the loop. Try this:
- set_fact:
nodelist: "{{ ( nodelist | default([]) ) + [ hostvars[item].ansible_eth0.ipv4.address ] }}"
loop: "{{ groups['all'] }}"
- debug:
var: nodelist | join(',')
(nodelist | default([])) outputs the current value of 'nodelist' or an empty list if it is not set (first pass)
+ [] merges the existing list with a new list, containing a single element - the IP of the host
So 'nodelist' ultimately ends up containing a list of IP's. You can then use | join(',') to turn that into a CSV.

Ansible Jinja Template config and stdout comparison

My pseudocode:
1. Get the ntp server config from "sh run"
2. Store that to a list
3. Jinja template generates the required config. I am passing the ntp_server IPs via -e (extra variables).
4. Add the config from 3, compare 3 and 4 and remove the rest.
I am struggling on step 4 [comparison part]. How do I compare the current config with the config generated from the jinja template? I am using roles.
Please advise.
# Jinja Template
{% for ntp_srv in ntp_servers %}
ntp server {{ ntp_srv }}
{% endfor %}
# tasks file for ansible-ios-ntp
---
- name: Current Edge servers before
ios_command:
commands:
- sh run | include ntp server
register: runconfser
- debug:
var: runconfser
# NTP SECTION - START
- name: Set NTP servers
ios_config:
src: ntprequired.j2
notify: Save Config
- name: Remove the rest NTP Servers
with_items: "{{ runconfser.stdout_lines[0] }}"
when: (item not in {src: 'ntprequired.j2'} and (item!=""))
ios_config:
lines:
- "no {{ item }}"
If I understand your question correctly, I believe you'd want to extract the current IPs from the registered output, then capture the ones which are not in the ntp_servers list:
- set_fact:
need_ips: |
{{ ntp_servers | difference(stdout_lines | join(" ") | regex_findall('[0-9.]+')) }}
Or you can obtain the "extra" ones by inverting the order of the difference:
- set_fact:
extra_ips: |
{{ stdout_lines | join(" ") | regex_findall('[0-9.]+') | difference(ntp_servers) }}
I cheated by just searching for [0-9.]+ but you can, of course, make that expression less tolerant by being more specific (aka [1-9](.[0-9.]){3})

How to use template module with different set of variables?

My use case is the following :
I have a template file, and I would like to create 2 different files from that template, with the variables being filled by a different set of variables for each file.
For example, lets say I want to template the file containing the line:
mkdir -p {{myTemplateVariable}}
I would like to find a proper way to get this variable filled by "File1" and "File2". Something like :
- name: template test 1
template:
src=myTemplateFile
dest=result1
- name: template test 2
template:
src=myTemplateFile
dest=result2
where I could specify for the first templating that the variable to use is a = "File1" and for the second, b = "File2".
With Ansible 2.x you can use vars: with tasks.
Template test.j2:
mkdir -p {{myTemplateVariable}}
Playbook:
- template: src=test.j2 dest=/tmp/File1
vars:
myTemplateVariable: myDirName
- template: src=test.j2 dest=/tmp/File2
vars:
myTemplateVariable: myOtherDir
This will pass different myTemplateVariable values into test.j2.
For Ansible 2.x:
- name: template test
template:
src: myTemplateFile
dest: result1
vars:
myTemplateVariable: File1
- name: template test
template:
src: myTemplateFile
dest: result2
vars:
myTemplateVariable: File2
For Ansible 1.x:
Unfortunately the template module does not support passing variables to it, which can be used inside the template. There was a feature request but it was rejected.
I can think of two workarounds:
1. Include
The include statement supports passing variables. So you could have your template task inside an extra file and include it twice with appropriate parameters:
my_include.yml:
- name: template test
template:
src=myTemplateFile
dest=destination
main.yml:
- include: my_include.yml destination=result1 myTemplateVariable=File1
- include: my_include.yml destination=result2 myTemplateVariable=File2
2. Re-define myTemplateVariable
Another way would be to simply re-define myTemplateVariable right before every template task.
- set_fact:
myTemplateVariable: File1
- name: template test 1
template:
src=myTemplateFile
dest=result1
- set_fact:
myTemplateVariable: File2
- name: template test 2
template:
src=myTemplateFile
dest=result2
You can do this very easy, look my Supervisor recipe:
- name: Setup Supervisor jobs files
template:
src: job.conf.j2
dest: "/etc/supervisor/conf.d/{{ item.job }}.conf"
owner: root
group: root
force: yes
mode: 0644
with_items:
- { job: bender, arguments: "-m 64", instances: 3 }
- { job: mailer, arguments: "-m 1024", instances: 2 }
notify: Ensure Supervisor is restarted
job.conf.j2:
[program:{{ item.job }}]
user=vagrant
command=/usr/share/nginx/vhosts/parclick.com/app/console rabbitmq:consumer {{ item.arguments }} {{ item.job }} -e prod
process_name=%(program_name)s_%(process_num)02d
numprocs={{ item.instances }}
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/{{ item.job }}.stderr.log
stdout_logfile=/var/log/supervisor/{{ item.job }}.stdout.log
Output:
TASK [Supervisor : Setup Supervisor jobs files] ********************************
changed: [loc.parclick.com] => (item={u'instances': 3, u'job': u'bender', u'arguments': u'-m 64'})
changed: [loc.parclick.com] => (item={u'instances': 2, u'job': u'mailer', u'arguments': u'-m 1024'})
Enjoy!
This is a solution/hack I'm using:
tasks/main.yml:
- name: parametrized template - a
template:
src: test.j2
dest: /tmp/templateA
with_items: var_a
- name: parametrized template - b
template:
src: test.j2
dest: /tmp/templateB
with_items: var_b
vars/main.yml
var_a:
- 'this is var_a'
var_b:
- 'this is var_b'
templates/test.j2:
{{ item }}
After running this, you get this is var_a in /tmp/templateA and this is var_b in /tmp/templateB.
Basically you abuse with_items to render the template with each item in the one-item list. This works because you can control what the list is when using with_items.
The downside of this is that you have to use item as the variable name in you template.
If you want to pass more than one variable this way, you can dicts as your list items like this:
var_a:
-
var_1: 'this is var_a1'
var_2: 'this is var_a2'
var_b:
-
var_1: 'this is var_b1'
var_2: 'this is var_b2'
and then refer to them in your template like this:
{{ item.var_1 }}
{{ item.var_2 }}
I did it in this way.
In tasks/main.yml
- name: template test
template:
src=myTemplateFile.j2
dest={{item}}
with_dict: some_dict
and in vars/main.yml
some_dict:
/path/to/dest1:
var1: 1
var2: 2
/path/to/dest2:
var1: 3
var2: 4
and in templates/myTemplateFile.j2
some_var = {{ item.value.var1 }}
some_other_var = {{ item.value.var2 }}
Hope this solves your problem.
I had a similar problem to solve, here is a simple solution of how to pass variables to template files, the trick is to write the template file taking advantage of the variable. You need to create a dictionary (list is also possible), which holds the set of variables corresponding to each of the file. Then within the template file access them.
see below:
the template file: test_file.j2
# {{ ansible_managed }} created by xbalaji#gmail.com
{% set dkey = (item | splitext)[0] %}
{% set fname = test_vars[dkey].name %}
{% set fip = test_vars[dkey].ip %}
{% set fport = test_vars[dkey].port %}
filename: {{ fname }}
ip address: {{ fip }}
port: {{ fport }}
the playbook
---
#
# file: template_test.yml
# author: xbalaji#gmail.com
#
# description: playbook to demonstrate passing variables to template files
#
# this playbook will create 3 files from a single template, with different
# variables passed for each of the invocation
#
# usage:
# ansible-playbook -i "localhost," template_test.yml
- name: template variables testing
hosts: all
gather_facts: false
vars:
ansible_connection: local
dest_dir: "/tmp/ansible_template_test/"
test_files:
- file_01.txt
- file_02.txt
- file_03.txt
test_vars:
file_01:
name: file_01.txt
ip: 10.0.0.1
port: 8001
file_02:
name: file_02.txt
ip: 10.0.0.2
port: 8002
file_03:
name: file_03.txt
ip: 10.0.0.3
port: 8003
tasks:
- name: copy the files
template:
src: test_file.j2
dest: "{{ dest_dir }}/{{ item }}"
with_items:
- "{{ test_files }}"
- name: copy vhosts
template: src=site-vhost.conf dest=/etc/apache2/sites-enabled/{{ item }}.conf
with_items:
- somehost.local
- otherhost.local
notify: restart apache
IMPORTANT: Note that an item does not have to be just a string, it can be an object with as many properties as you like, so that way you can pass any number of variables.
In the template I have:
<VirtualHost *:80>
ServerAdmin me#example.org
ServerName {{ item }}
DocumentRoot /vagrant/public
ErrorLog ${APACHE_LOG_DIR}/error-{{ item }}.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
Another real world example using a list
an extract for a template for php.ini
{% if 'cli/php.ini' in item.d %}
max_execution_time = 0
memory_limit = 1024M
{% else %}
max_execution_time = 300
memory_limit = 512M
{% endif %}
This is the var
php_templates:
- { s: 'php.ini.j2', d: "/etc/php/{{php_version}}/apache2/php.ini" }
- { s: 'php.ini.j2', d: "/etc/php/{{php_version}}/cli/php.ini" }
Then i deploy with this
- name: push templated files
template:
src: "{{item.s}}"
dest: "{{item.d}}"
mode: "{{item.m | default(0644) }}"
owner: "{{item.o | default('root') }}"
group: "{{item.g | default('root') }}"
backup: yes
with_items: "{{php_templates}}"

Resources