Append in host_vars to list from group_vars/defaults - ansible

Suppose I have a list configured in the role defaults (under roles/myrole/defaults/main.yml):
the_list:
- one
- two
And suppose that for a particular host I need to add also three to the list. Is it possible?
By default, the list is overriden, rather than concatenated. E.g. if I put into host_vars:
the_list:
- three
... then the resulting list will include just three, the other two elements will be lost.
Any way to merge the lists? Maybe with some kind of yaml / jinja magic...?
Thanks!

There has been a number of issues and feature requests raised around this on the Ansible GitHub; see this pull request for example. In summary, there isn't a good way to do this yet, hopefully there will be soon.
A common workaround for the time being is to define a list values in one place and a second list extra_values elsewhere then merge them before use.

Related

Am I misunderstanding something about using the List and Fetch combination?

I am trying to understand the combination of List and Fetch processors.
I have a directory with three JSON files and I get the ListAzureDataLakeStorage to list them. But when I connect a FetchAzureDataLakeStorage with which I intend to take only one of the files, the Fetch takes the same file three times. In summary, it takes the file whose azure.filename matches with the value that I put in the File Name property, but as many times as there are files in the listed directory.
I really want to use a single List and connect three Fetches to it, each one to take a different file, and thus use them for different streams.
In each Fetch I put in the "File Name" property the name of the file that I want to take. For example:
File Name: fileName1.json
I have also tried putting in "File Name" with Expression Language the following:
FileName: $ {azure.filename: equals ('fileName1.json')}. But this option causes a 404 empty body error.
But there is no way. Am I misunderstanding something about using the List and Fetch combination?
If you are statically entering file names and you want to respond to each one differently, then the ListX processors aren't very beneficial to your flow.
The easier option would be to use a GenerateFlowFile processor with the appropriate schedule to trigger a corresponding FetchX processor.
If you're only doing this for 3 files, it's not too much manual overhead. You could also achieve something similar using RouteOnContent/Attribute.

Alternative ansible syntax

I've seen a playbook which looks something like this:
- hosts:
- foo
- bar
roles:
- role: whatever
It works, but from the documentation I would have expected that:
a. Hosts would be given as a single space separated line e.g.:
- hosts: foo bar
rather than a list.
b. The value for the "roles" key in the play would be a list, e.g.:
roles:
- whatever
rather than a key:value pair.
Can someone explain what I'm missing either in yaml which makes these alternatives equivalent once parsed, or where in the ansible docs it explains these alternative definitions?
TL;DR
For hosts use the syntax that you and the other people working with this are most comfortable with.
For roles, you need the role: <name> syntax only in cases where you want to also set other attributes for the role.
Longer answer
I have wondering about this occasionally as well.
In the docs section Intro to Playbooks, Basics, it says:
The hosts line is a list of one or more groups or host patterns, separated by colons, as described in the Working with Patterns documentation.
It does, however, not mention explicitly that this list, could also be a space separated string.
As far as the roles attribute of a play is concerned, I think the alternate syntax variant is straight forward. If you just pass a name (a single string), then this is obviously the name of the role.
If you want to pass additional arguments, like variables, then you need to create a dictionary. See an example of the two syntaxes used together here in the docs (search for "Roles can accept other keywords").
The definite answer to both questions is in the source code:
Here is the part that parses the hosts list in a play:
https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/play.py#L104-L116
Here is the part that does it for a role in roles:
https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/role/definition.py#L68-L135
There is another hint in the playbook/base.py#preprocess_data:
infrequently used method to do some pre-processing of legacy terms
The Play class for example inherits / overrides this method, directly below the snippet I linked to above.

Which module to use to edit files - Ansible

I want to edit the configuration file of telegraf(system metrics collecting agent).
Telegraf comes in with a default config file which can be edited. There are many input and output plugins defined in there, which are commented out and can be added by removing the comments and also be customized.
I want to edit only some of the plugins defined there, not all of them. For example, consider this is the file,
[global]
interval='10s'
[outputs.influxdb]
host=['http://localhost:8086']
#[outputs.elasticsearch]
# host=['http://localhost:9200']
[inputs.netstat]
interface='eth0'
Now, I want to edit the 3 blocks, global, outputs.influxdb and inputs.netstat. I don't want to edit outputs.elasticsearch but also want that the block outputs.elasticsearch should remain in the file.
When Using Ansible, I firstly used Template module, but if I use that, then the commented data would be lost.
Then I used the ini_file module, instead of editing the already present block, it adds a new block even if it is already present, and results in something like this,
[outputs.influxdb]
host=[http://localhost:8086]
[outputs.influxdb]
host=[http://xx.xx.xx.xx:8086]
Which module is ideal for my scenario ?
There are several options, depending on your purpose.
The lineinfile - module is the best option, if you just want to add, replace or remove one line.
The replace - module is best, if you want to add, replace or delete several lines.
The blockinfile - module can add several lines, surrounded by markers.
If you only want to change two or three lines, you could use as many calls of lineinfile. To change a whole config file, I would recommend, like the commenters suggest, use the template - module.
Ok, if you really really want to avoid using templates, you could try to use replace and a regex like this:
- hosts: local
tasks:
- replace:
path: testfile
regexp: '^\[{{ item.category }}\]\s(.*)host(.*)$'
replace: '[{{ item.category }}]\n host=[{{ item.host }}]'
with_items:
- { category: 'outputs.influxdb', host: 'http://cake.com:8080' }
This, in its current form, would not necessarily handle more than one option under each category, but the regex can be modified to handle multiple lines.
As required, it will not touch the # commented lines. However, if you decide to enable some of the previously inactive sections, you might end up with a slightly messier configuration file that would include the instructions both commented and uncommented (shouldn't impact functionality, only 'looks'). You will also need to account for options that look like the example below (interleaved commented/uncommented values) and create regexes specially for those use-cases:
[section]
option1=['value']
# option2=['value']
option3=['value']
It highly depends on your use-case, but my recommendation remains that templates are to be used instead, as they are a more robust approach, with less chances of things going wrong.

How to loop over playbook include?

(I'm currently running Ansible 2.1)
I have a playbook that gathers a list of elements and I have another playbook (that calls different hosts and whatnot) using said element as the basis for most operations. Therefore, whenever I use with_items over the playbook, it causes an error.
The loop control section of the docs say that "In 2.0 you are again able to use with_ loops and task includes (but not playbook includes) ". Is there a workaround? I really need to be able to call multiple hosts in an included playbook that runs over a set of entries. Any workarounds, ideas for such or anything are greatly appreciated!
P.S. I could technically command: ansible-playbook but I dont want to go down that rabbit hole if necessary
I think I faced same issues, and by the way, migrating to shows more than in 'item' already in use.
refering to http://docs.ansible.com/ansible/playbooks_best_practices.html , you should have an inventory (that contains all your hosts), and a master playbook (even if theorical).
A good way, instead of including playbooks, is to design roles, even if empty. Try to find a "common" role for everything that could be applied to most of your hosts.Then, include additional roles depending of usage, this will permit you to trigg on correct hosts.
You can also have roles that do nothing (meaning, nothing in 'tasks'), but that contain set of variables that can be common for two roles (you avoid then duplicate entries).

Improved way of scaling in saltstack

I have a problem about the Jinja2 template and that problem is breaking a one line string over multiple lines when it comes to writing a state or anything in salt [my exact case refers to trying to write a list of machines one after the other,in a list,instead of just in a really long line].
What I am trying to say is that I want to achieve this:
nodegroups:
- group: 'L#adsdasdadas' +
'dasdasdasdas'
.............->imagine 10.000 names coming here
'adsasdasddsa'
Compared to the approach that I have to do now:
nodegroups:
- group: 'L#adsdasdadas,dasdsadasdsa,dasdsadasdsa,......,asdqwe'
Is there a better way to do it?Is there a better way to handle thousands of machines?
You could say grains,and I thought about it but I was wondering if there's a better and elegant way of doing it.
Any thoughts or opinions would help me a lot
[Edit1]:
I wrote a script that takes a list of hostnames and adds them to the master config file in the nodegroups section.For now it might work
Choice of data source
I would recommend targeting with pillars because they are managed centrally from Master = convenient, rather than static custom grains (which are configured distributively on each Minion) = inconvenient - see comparison summary here.
Limitations of configuration files
The nodegroups are specified in Salt configuration file /etc/salt/master which is not a Jinja template (it has pure YAML format). So, you don't have an option to use Jinja to join external input with list of strings.
Possible solution
Why joining is even mentioned? You can turn the problem of "breaking a one line string over multiple lines" into solution of using lists right away - no need to break (and if you need "one line string" somewhere, joining list items is easy).
In other words, you could define nodegroups via pillar (avoiding long strings as in your example). Pillars, in turn, are rendered by Jinja. Therefore, using the same list of Minions defined somewhere, you could generate derived product in pillars through Jinja (be it joined string of them or list as is). There is a trick which allows reusing the same external data in multiple pillars files.
First of all I would like to thank uvsmtid for the wonderful idea.Sorry for the confusion created too
So,what I did was create a pillar with the name of each minion[which happens to be it's id] and then in a state what I did was compared the value from that list to the actual id of the minion
{%for item in salt['pillar.get']('info') %}
{%if grains['id'] == item %}
something:
cmd.run:
- name: touch something
{%endif%}
{%endfor%}
I hope this solution will help someone the same way it helped me

Resources