So, the scenario is I have producers and consumers in ratio 7:1, and I want to have a consistent and deterministic multilple mapping b/w the producers and consumers in my service. List of consumers is provided in the config to each of the producer, which is done via ansible. So, I try to implement the mapping logic in the ansible itself, rather than passing the entire list of consumers, and doing it inside producer service. So, I thought of using a custom filter to filter out from the list of consumers, and assign it to producer. Below is the custom filter I wrote:
#!/usr/bin/python
class FilterModule(object):
def filters(self):
return { 'map_producer_to_consumer': self.map_producer_to_consumer }
# consumer_servers: complete list of consumers servers
# producer_id: provided to each producer for mapping purpose
# producer_count: total no. of producers
# map_consumer_count: no. of consumers need to be mapped to each producer
# consumer_count: total no. of consumers
def map_producer_to_consumer(self, consumer_servers, producer_id, producer_count, map_consumer_count):
consumer_count = len(consumer_servers)
index_offset = 0 if producer_count%consumer_count else 1
rotation_count = (producer_id/consumer_count) % (map_consumer_count-1) # used for left rotation of mapped servers
map_consumer_indexes = [ (producer_count*i + producer_id + index_offset*i) % consumer_count for i in xrange(map_consumer_count)]
mapped_consumer_servers = [consumer_servers[map_consumer_indexes[0]]]
for i in xrange(1, map_consumer_count):
index = (i + rotation_count) % map_consumer_count
if i + rotation_count >= map_consumer_count:
mapped_consumer_servers.append( consumer_servers[map_consumer_indexes[index] + 1] )
else:
mapped_consumer_servers.append( consumer_servers[map_consumer_indexes[index]] )
return (',').join(mapped_consumer_servers)
This filter is working as expected, when using with static arguements like this:
"{{ tsdb_boxes | map_producer_to_consumer(2,3,3) }}"
but I want to make it use dynamic arguments via jinja2 templating, something like:
"{{ groups['producers'] | map_producer_to_consumer ({{ consumer_servers }}, {{ producer_id }}, {{ producer_count }}, {{ map_consumer_count }}) }}"
but its resulting in errors due to nesting of variables, which is not allowed in Jinja2. If I try something like this:
"{{ groups['producers'] }} | map_producer_to_consumer ({{ consumer_servers }}, {{ producer_id }}, {{ producer_count }}, {{ map_consumer_count }})"
it results in printing out the string like this:
['ip-1', 'ip-2'...] | map_producer_to_consumer (1000, 10, 150, 3)
Can someone please suggest what should be the best ansible way to achieve this. Should I use script module and convert the logic in bash, or it will be better to keep this inside the service only.
Answer from comments:
Why not try {{ groups['producers'] | map_producer_to_consumer(consumer_servers, producer_id, producer_count, map_consumer_count) }}
And link from #techraf about nesting.
Related
Is there a way to generate a list of IP addresses between two arbitrary IPs (not from a subnet/range) with Ansible (v2.9)?
I've searched and the ipaddr filter looks like a good candidate, but from the documentation I couldn't figure out if it supports this.
I'm looking for a solution that allows me to get a list like
[ '10.0.0.123', '10.0.0.124', ... , '10.0.1.23' ]
from a task like
- name: generate IP list
set_fact:
ip_list: "{{ '10.0.0.123' | ipaddr_something('10.0.1.23') }}"
Create a filter plugin. For example
shell> cat filter_plugins/netaddr.py
import netaddr
def netaddr_iter_iprange(ip_start, ip_end):
return [str(ip) for ip in netaddr.iter_iprange(ip_start, ip_end)]
class FilterModule(object):
''' Ansible filters. Interface to netaddr methods.
https://pypi.org/project/netaddr/
'''
def filters(self):
return {
'netaddr_iter_iprange' : netaddr_iter_iprange,
}
Then, the task below shall create the list
- set_fact:
ip_list: "{{ '10.0.0.123'|netaddr_iter_iprange('10.0.1.23') }}"
I have an ansible inventory with groups as follows:
+hosts/
+all/
+group_vars/
- linux.yml
- webserver.yml
- dbserver.yml
...
And I have a playbook that sets monitoring for hosts; and the kind of monitoring is done by plugins. So in each group y set a list monitoring_plugins that contain the plugin to be able t monitor each service.
Inside each group yml I try to "append" to the list:
monitoring_plugins: {{ monitoring_plugins|default([]) + [ 'whatever_plugin_correspond_to_group' ] }}
But it doesn't work as expected, being expected that if a host belongs to more than one group, it should have the plugins corresponding to those groups.
Is there a way to acommplish this?
Thanks in advance.
What you're describing should work as expected from within a task, but you cannot have executable code in a vars or group_vars yaml or json file -- those are static
So you will need to set a distinct name at the low level and then roll them up at the top level:
group_vars/
dbserver.yml # sets monitoring_plugins_dbserver: ["a", "b"]
linux.yml # sets monitoring_plugins_linux: ["c", "d"]
and then in your tasks, something like this (but be forewarned I haven't tested this specific example):
- set_fact:
monitoring_plugins: >-
{% set results = [] %}
{% for g in groups.keys() %}
{% set _ = results.extend(vars['monitoring_plugins_'+g]|d([])) %}
{% endif %}
{{ results }}
I'm trying to develop a playbook were I have the following variable.
disk_vars:
- { Unit: C, Size: 50 }
- { Unit: D, Size: 50 }
With the variables defined on the playbook there is no problem but when I try to use a texarea survey on Ansible Tower I cannot manage to parse them as list of dictionaries.
I tried adding to the survey the following two lines which are already on yaml format.
- { Unit: C, Size: 50 }
- { Unit: D, Size: 50 }
And on my vars section I use test_var: "{{ test_var1.split('\n') }} which converts the output into a two line string. Without the split is just a single line string.
I could make my playbook work with a simple dictionary like
dict1: {{ Unit: C, Size: 50 }}
but I'm having issues parsing it as well.
EDIT
Changing it to the following as suggested by mdaniels works.
- set_fact:
test_var: "{{ test_var1 | from_yaml }}"
- name: test
debug: msg=" hostname is {{ item.Unit }} and {{ item.Size }}"
with_items:
- "{{ test_var }}"
I'm trying to figure a way to clear-up the data input as asking users to respect the format is not a very good idea.
tried changing the input date to the following but I could not figure out how to format that into a list of dictionaries.
disk_vars:
Unit: C, Size: 50
Unit: D, Size: 50
I tried with the following piece of code
- set_fact:
db_list: >-
{{ test_var1.split("\n") | select |
map("regex_replace", "^", "- {") |
map("regex_replace", "$", "}") |
join("\n") }}
But is putting it all on a single line.
"db_list": "- {dbid: 1, dbname: abc\ndbid: 2, dbname: xyz} "
I have tried to play with it but could not manage to make it work.
I believe you were very close; instead of "{{ test_var1.split('\n') }}" I believe you can just feed it to the from_yaml filter:
- set_fact:
test_var1: '{{ test_var1 | from_yaml }}'
# this is just to simulate the **str** that you will receive from the textarea
vars:
test_var1: "- { Unit: C, Size: 50 }\n- { Unit: D, Size: 50 }\n"
- debug:
msg: and now test_var1[0].Unit is {{ test_var1[0].Unit }}
I faced a similar dilemma, i.e. that I was bound to the survey format(s) available, and I was forced to use mdaniels suggested solution above with sending the data as text and then later parse it from YAML . Problem was however that controlling the format of the input (i.e. a YAML-string inside the text) would probably cause a lot of headache/errors, just like you describe.
Maybe you really need to use the Survey, but in my case I was more interested of calling the Job Template using the Tower REST API. For some reason I thought I then had to have a survey with all parameters defined. But it turned out I was wrong, when having a survey I was not able to provide dictionaries as input data (in the extra_vars). However, when removing the Survey, and also (not sure if required or not) enabling "Extra Variables -> prompt on launch", then things started to work!! Now I can provide lists / dictionaries as input to my Templates when calling them using REST API POST calls, see example below:
{
"extra_vars": {
"p_db_name": "MYSUPERDB",
"p_appl_id": "MYD32",
"p_admin_user": "myadmin",
"p_admin_pass": "mysuperpwd",
"p_db_state": "present",
"p_tablespaces": [
{
"name": "tomahawk",
"size": "10M",
"bigfile": true,
"autoextend": true,
"next": "1M",
"maxsize": "20M",
"content": "permanent",
"state": "present"
}
],
"p_users": [
{
"schema": "myschema",
"password": "Mypass123456#",
"default_tablespace": "tomahawk",
"state": "present",
"grants": "'create session', 'create any table'"
}
]
}
}
I am using ansible to template a jinja2 file.
IP:{{ ansible_eth0.ipv4.address }}
IP:{{ ansible_docker0.ipv4.address }}
IP:{{ ansible_{{ ka_interface }}.ipv4.address }}
there is a var named ka_interface for network adapter.
but you will get error in 3rd var
(IP:{{ ansible_{{ ka_interface }}.ipv4.address }} )
It seems that var in jinja2 template can be nested.
It's not possible to construct a dynamic variable with Jinja2 syntax.
However, you can access any play-bound variables via the builit-in vars hash object:
{{ vars['ansible_' + ka_interface]['ipv4']['address] }}
Edit: Fixed hash syntax
follow Chris Lam 's advice,
It works
- name: test
shell: echo {{ vars['ansible_' + ka_interface]['ipv4']['address'] }}
tags: test
I write Ansible module my_module that need to set some facts.
I define in module the below code
....
response = {
"hello": "world",
"ansible_facts" : {
"my_data": "xjfdks"
}
}
module.exit_json(changed=False, meta=response)
Now in playbook after execution my_module I want access to new facts, but it's not define
- my_module
- debug: msg="My new fact {{ my_data }}"
What is the correct way to do it?
You should set ansible_facts directly in module's output, not inside meta.
To return all response's keys from your example:
module.exit_json(changed=False, **response)
Or only for ansible_facts:
module.exit_json(changed=False, ansible_facts=response['ansible_facts'])