I'm new to Ansible so be patient. I was trying to create a dynamic inventory using Ansible's dynamic inventory pluging. In particular I'm using hcloud plugin to interface with Hetzner's API.
What I'd like to do is create 3 groups : databases, nfs, k8s.
All servers have already been tag as follows :
app=mysql for mySQL database servers
app=nfs for NFS servers
app=k8s for k8s cluster servers (workers and masters)
To say the docs are lacking is using an euphemism. I've tried using label_selector as follows with no success:
plugin: hcloud
token: hehe
groups:
databases:
label_selector: app=mysql
nfs:
label_selector: app=nfs
k8s:
label_selector: app=k8s
This indeed results in the creation of 3+ groups :
all
databases
nfs
k8s
ungrouped
hcloud
yet all groups contain all hosts, regardless of the label.
So my questions are :
Assuming I'm using something like keyed_groups, which "Add hosts to group based on the values of a variable.", where can I find accepted variables?In the example they use 'location', 'image_os_flavor' and 'status' but I haven't found a list of accepted variable names that I could use.
How could I implement a dynamic Inventory that reaches the previously explained goal?
Is it possible to add group variables in dynamically generated inventories?
Thanks a lot to everyone, let me know how to improve my question as well.
You can do:
groups:
databases: labels.app == 'mysql'
nfs: labels.app == 'nfs'
k8s: labels.app == 'k8s'
The documentation is here:
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/constructed_inventory.html
Related
I want to create a dynamic inventory, and create a group to put the nodes where I scrape data from.I'm using the gcp_compute plugin in my dynamic inventory.
What I want to achieve:
I want to retrieve multiple hostnames/ip and put them in the same group. I want it to be quite modular, so I can switch nodenames easily.
This how it looks so far:
plugin: gcp_compute
auth_kind: serviceaccount
cache: true
gather_facts: false
retrieve_image_info: true
service_account_file: servicefile.json
groups:
node_exporter: "'db-vm' in name"
I want to use something like: node_exporter: "'db-vm' OR 'prometheus-vm' in name" or something similar. How can I achieve this.
This is the command I use too see if it works or not: ansible-inventory -i gce_inventory.gcp.yml --graph. All my nodes gets ungrouped.
I solved it very easily...
I believe I made some mistakes before when doing the conditional, but now accidentally solved it.
To be able to retrieve multiple, here how:
node_exporter: "'db-vm' in name or 'prometheus-vm' in name"
I am trying to filter EC2 instances in prometheus.yml. Suppose that the following is part of the yml file. How can I use regex in the values to return instance that starts with, lets say, prod or qa or other labels? Is this possible without configuring relabeling?
ec2_sd_configs:
- region: us-east-1
access_key: access
secret_key: mysecret
profile: profile
filters:
- name: tag:Name
values:
- prod.*
- qa.*
It seems that Prometheus does not support regex in filtering API right now but it would be a nice feature if they can add it in future releases. What can be done in this situation is that you can add a separate tag on EC2 instances so you can filter based on those tag. Filtering at early stage is extremely helpful if you have large number of instances. Otherwise, you'll get a huge list and you need to go through a drop/keep phase via relabeling which still keeps a long list in service discovery panel and make it difficult to read.
In the next step, you can use relabeling to replace the address of each discovered instance from private IP to public IP. As a final touch, you can replace the instance name with the tag name so, for example, all instances of QA are labeled as QA.
ec2_sd_configs:
- region: value
access_key: value
secret_key: value
port: value
filters:
- name: tag:Name
values:
- qa
- prod
- some other types
relabel_configs:
- source_labels: [__meta_ec2_public_ip]
replacement: '${1}:your port number'
target_label: __address__
- source_labels: [__meta_ec2_tag_Name]
target_label: instance
I don't have any experience with AWS, but I believe its API does not support regular expressions in filtering API.
In general relabelling is the preferred way to do filtering. An example of how to achieve this would be (for consul, but that does not matter much): Prometheus: how to drop a target based on Consul tags
List of ec2 meta labels available is in prometheus docs at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#ec2_sd_config
I see that Prometheus docs recommend using filters over relabelling for efficiency reasons when you have potentially thousands of instances. Using separate tag (for example "Env") that has values of "qa", "prod" etc. so they can be matched exactly (without regex/wildcard) would be an elegant solution here I'd guess?
I need to update a yaml inventory with multiple managed servers and apply different variables to each of them. The current yaml structure has only one admin server per domain.
---
all:
hosts:
localhost:
ansible_connection: local
children:
targets:
hosts:
Domain1:
ansible_host: "www.example1.com"
admin_url: "t3://www.example1.com"
Domain2:
ansible_host: "www.example2.com"
admin_url: "t3://www.example2.com"
so the ansible_host under domain 1 and 2 are admin servers. Now i will like to add multiple managed servers under each domain and still have the flexibility to assign each server (both admin and managed) different variable values if needed.
You should have a deep look at the yaml inventory documentation to understand how they are structured. IMHO they are a little more complex to understand for starters than the historic ini inventories. You should have a good understanding of all the inventory concepts before starting anyway
How you should read your current inventory file:
The generic group all declares:
one direct child host named localhost
one children group named targets with two hosts named Domain1 and Domain2
From your question, I understand that you would like to treat Domain1 and Domain2 as groups (as their names suggests), not as hosts.
You probably have to rewrite your inventory like the following.
all:
children:
targets:
vars:
admin_url: "t3://{{ inventory_hostname }}"
children:
Domain1:
hosts:
www.example1.com:
www.example2.com:
www.yetanother.com:
Domain2:
hosts:
host1.domain2.com:
host2.domain2.com:
host3.domain2.com:
Things to notice:
I dropped the reference to localhost. It is always available by default.
I kept your top level group targets. Since all admin addresses have the same format, I declared a var for that group which will be used for every host using its inventory_hostname (see ansible magic variables)
Your targets group now has two children groups: Domain1 and Domain2 both declaring their hosts.
Since we now declare hosts with their real names, there is no need to set ansible_host to a different value, so no hosts vars are needed so far.
Hope this will help you getting started.
I have an Ansible role to handle creation of an RDS instance and the databases on that instance. The role allows a security group to be specified for the database. I want to be able to add a rule to the security group at the beginning of the role that allows access from the current host so that Ansible can run some database creation/maintenance tasks. I then want to remove this rule from the security group while maintaining the existing groups.
What I've done so far is used the ec2_group_facts module to get information about the given security group which I save in the security_group variable. I then add a rule with a task similar to the following:
- name: Add hole to security group
local_action:
module: ec2_group
name: "{{ security_group.group_name }}"
purge_rules: no
rules:
- proto: tcp
from_port: "{{ db_port }}"
to_port: "{{ db_port }}"
cidr_ip: 0.0.0.0/0
This all works properly. The issue is that at the end of the role, when I want to restore the existing rules, the format of the rules returned by ec2_group_facts is not accepted by the ec2_group module. The information saved about security_group is in the following format:
{
"group_id": "sg-1234abcd",
"group_name": "security-group",
"ip_permissions": [
{
"from_port": 1234,
"ip_protocol": "tcp",
"ip_ranges": [
{
"cidr_ip": "0.0.0.0/0"
}
],
"ipv6_ranges": [],
"prefix_list_ids": [],
"to_port": 1234,
"user_id_group_pairs": []
}
],
"ip_permissions_egress": [],
"owner_id": "123456789012",
"tags": {
"Name": ""
},
"vpc_id": "vpc-1234abcd"
}
The rules argument of the ec2_group module needs a list of objects with proto, from_port, to_port, and cidr_ip attributes, so how would I map the data above to the required format?
Edit: I guess one solution would be to add a temporary security group that allows access from the current host. If my understanding of EC2 security groups is correct, the most permissive rule of the security groups associated with an instance is applied so this would achieve what I want. However this would require editing the security groups attached to an existing RDS instance, so I would prefer to edit the rules of an existing security group if possible.
Edit 2: Travis CI publishes the IP addresses used to run builds. I could just add these to the security group permanently, although I'm not sure what the security implications of this are.
ec2_group docs
ec2_group_facts docs
When running playbooks you want a consistent state and from the sounds of things you don't have a consistent state throughout your play.
I would suggest that the additional task that you would like to perform on the database could be run from some other instance that is more trusted (perhaps the place you're running ansible from?).
Consider what will happen if a playbook is run twice at the same time. Perhaps this isn't something your workflow allows for but you should still consider this case.
If this isn't an option or you would rather not change your implementation then your edit's suggestion sounds more suitable. Apply your standard rules and when required add to your rules (or create a new security group for this purpose) and then destroy or modify when no longer required.
(This is a cross-post from the mailing list, since I haven't gotten any answers there.)
I've recently set up fail2ban for ssh on one of my servers using https://galaxy.ansible.com/tersmitten/fail2ban/ . This was very easy, and I'm loving galaxy.
Now, I'm wanting to extend fail2ban to also block IPs that have failed basic auth checks in Apache too many times. Since all my hosts run ssh, but not all run Apache, I've created a new role for this (cleverly named 'apache', while the first is 'common'). However, as shown in the fail2ban role's README, configuration of individual services is done with an array, fail2ban_services, and redefining that in my apache role overwrites what was defined in common (or, due to timing, vice versa).
I've seen that I can change the hash behavior, which would seem to do what I want. I'm wary of this, however, since it's a global change.
There is also the combine filter for jinja, but reports indicate I can't do what I want, which would be
fail2ban_services: "{{ fail2ban_services | combine({...}) }}"
Is there another method to do this combination? Or perhaps am I going about this the wrong way, and should be architecting this differently?
I'm using ansible 2.0.1.0.
hash_behavior: merge and the combine filter would not be helpful, because those are for hashes. If I get the docs of fail2ban right, the fail2ban_services variable though is a list of hashes:
fail2ban_services:
- name: serviceA
...
- name: serviceB
...
- name: serviceC
...
You can easily merge lists:
listA: "{{ listB + listC }}"
... but the same limitations apply as for the combine filter. You can not override a variable which depends on itself, Ansible will complain about some "recursive loop detected" or so. No clue why this is such a big problem for Ansible, there should nothing be recursive in doing a = a + b. But unfortunately that's how it is.
If I understood your description, you define the fail2ban_services list in your role common. Then later you want to re-define it in the role apache. How is actually the fail2ban role implemented? Do you have 3 roles then in your playbook like so?
roles:
- common
- apache
- fail2ban
And the goal is, the first roles define what the last role then is using?
Given the mentioned limitations, my only idea then would be to rename your variables. The role common will define fail2ban_services_common, the role apache will define fail2ban_services_apache and then you pass the combined value to fail2ban:
roles:
- common
- apache
- role: fail2ban
fail2ban_services: "{{ fail2ban_services_common + fail2ban_services_apache}}"