I am relatively new to Configuration Management tools for big infrastructure. The company will be using Salt for Linux and Windows, but I guess the question does not relate to specific tool.
The thing I don't get is, let's imagine we have 30 machines in cloud and couple of custom services installed for each of them. Versions can be different, depending on customer subscription. And each service has a config file that can have machine specific data. How you update config files for those services using Salt, or Puppet, Chef, Ansible kind of tools.
I don't know about Salt but in general in Configuration Management tools there is groups of hosts for several machines with the same configuration, templates and variables for customization for each machine
for example you can specify variable port=8888 for host1 and port=9999 for host2 but nginx-template will be something like this:
server {
listen {{port}};
}
for the both servers.
The same idea with machine specific data example(ansible):
- name: start container with mon
docker:
name: test-mongo
state: reloaded
image: mongo
when: master_ip == ansible_default_ipv4.address
where master_ip is my variable and ansible_default_ipv4.address is ip which ansible was connected to this host.
Hope this helps.
With Chef you would use an Erb template for the config and interpolate based on node attributes in most cases. Those might be set directly on the node itself, or you could make a role per customer and set customer-specific attribute data in there.
Related
You might not understand what I am trying to say in the question. Before stating the problem, I would give an example to understand even more.
Let's take terraform for the example.
Imagine we use terraform to spin up an EC2 instance in the aws. So, we are currently storing the state locally. Let's say we need to add a tag to the ec2 instance.
So what we do is, we add the this small block to achieve that.
tags = {
Name = "HelloWorld"
}
Nothing complex, just simple as it is. So, as we add a tag, the terraform will look for any changes in the state and as it found a tag has been added, it runs and only add the tag without recreating the whole instance. (There can be other scenarios where it needs to recreate the instance but let's leave those for now)
So, as I mentioned, terraform doesn't recreate the instance for adding a tag. If we want to add another tag, it will add the tag without recreating the instance. Just as simple as that.
doesn't matter how many times we run the terraform apply it doesn't do any changes to existing resources unless we made any in the terraform files.
So, let's now come to the real question.
Let's say we want to install a httpd using ansible. So I write ansible playbook. And it will use the yum package and install and then start and enable the service.
Okay it is simple like that.
Let's say we we run the same playbook for the second time, now will try to execute the same commands from scratch without checking first whether it has executed this playbook before?
I have couple of experiences where I try to install some services and when I try to execute the same playbook again, it fails.
Is it possible to preserve the state in ansible just like we do in terraform so it always only execute the newer changes other than executing from the start on the second run?
You seem to be describing idempotence:
Idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application.
It's up to you to make sure your playbooks are idempotent in ansible. You could do this for your example using the package and service modules.
- name: Setup Apache
hosts: webservers
tasks:
- name: Install Apache
package:
name: httpd
state: present
- name: Start and enable Apache
service:
name: httpd
state: started
enabled: yes
Ansible doesn't achieve idempotence by retaining state on the the controller. Each module should check the state of the host it's operating on to determine what changes it needs to make to achieve the state specified by the playbook. If it determines that it doesn't need to make any changes then that's what it should report back to the controller.
I have 2 VPCs, each of which has a VPN instance, whose configuration I would like to share. Both instances have a class tag, whose value is now vpn_dev, however, they should differ by the environment tags - one would have environment: default, and the other environment: london.
I also have 2 playbooks, one for each VPC. I would like to run my vpn role on exactly one of the instances based on the combination of the class and environment tags, i.e. select the instance that has class: vpn_dev and environment: london. E.g.
- name: Deploy developer VPN in AWS
hosts:
- tag_class_vpn_dev
- tag_environment_london
roles:
- vpn
However, this would of course install the role on all instances that have class: vpn_dev (regardless of environment), and on all instances that have environment: london (regardless of class).
Is there a way to achieve this? Currently it seems like the only way is to have a single tag that uniquely identifies an instance.
The lists of hosts in a playbook is an OR operation by default. You can use AND over to host groups with this syntax:
- hosts:
- tag_class_vpn_dev:&tag_environment_london
More about host patterns here
It is possible to apply some ansible role to one host for each tag combination, although I would not recommend it, because this approach IMHO is not idempotent.
One of the possible risk with this scenario is that when you re-run this playbook, selected host for some tag combination can be different from previous run, and therefore you will end up with two instances per this unique tag combination group.
My recommended approach would be to use some tag to mark host which is supposed to be used as VPN role target, and then use ansible to apply VPN role only to hosts marked by this tag.
I'm writing an Ansible playbook, that sets up systems, where in certain configurations, some systems might be provided by another organization. A very simple example:
Inventory:
An app server
A db server
Playbook:
Setup up both servers, and add application.properties file to app server with IP, port and user/pass of the db server.
Works so far, but then a requirement comes, that in some deployments the DB server is provided by another organization, so I can't include it in inventory, as the setup step fails, yet I still want to generate the properties file for the app server (with db server info I get from other people).
What would be the least painful solution that would cover both scenarios (my own db server and provided db server), considering there's 6 such server types, not just 2 (so not just 2 different scenarios, there are many permutations of which server is provided for me and which is mine).
Edit:
To be more specific, the problem I have is that if I use vars when a system is not mine and facts when it is mine, then I have problems writing application.properties.j2 template, since facts and vars are referenced differently. How would use var in a template, but use a fact if var isn't defined?
I assume you have an inventory file per deployment. So an inventory file might look like this:
[app]
an-app.server
[db]
a-db.server
[deploymentA:children]
app
db
[deploymentA:vars]
db_ip=foo
dp_port=foo
db_user=foo
db_pass=foo
The deploymentA:vars you probably would store as group_vars/deploymentA but for simplicity I just put it into the inventory.
Now the playbook:
- hosts: db
roles:
- setup-db-server
- hosts: app
roles:
- setup-app-server
In a deployment where the db server is not managed by yourself, you simply would not have a db server defined in your inventory. The group db would be empty. In that case the first play will simply print "no hosts matched" or something like that and proceed to the next play.
In a deployment where the db server is managed by you, the role setup-db-server will be run and you can use the deploymentA:vars for configuration.
In your app play you have the deploymentA:vars no matter if the host is yours or not and can roll out the application.properties file.
My setup contains "pairs of servers" in the sense that every "instance" of my service consists of two servers (say database and web) on (usually) different hosts. That is, I don't have a single database server that many web servers talk to -- instead, each web server talks to its own database server. I want to manage multiple such pairs, but I don't know the best way to model the pairs using Ansible.
The main host organization tools in Ansible seem to be groups and roles. However, these only group together hosts of the same kind. I can use that to, for example, set up the necessary software on my database and web servers. However, I don't think I can use groups or roles to tell every web server to use exactly one of the database servers.
It also makes little sense to use an Ansible group to model one server pair, since I then cannot distinguish between the two different roles in these groups.
Although my setup consists of server pairs I'd also be interested in modeling multiple instances of n-to-m host relations (say n database servers and m web servers) in general.
You could just use ansible variables to provide the necessary links. E.g., in your inventory:
[webservers]
web0 dbserver=db0
web1 dbserver=db1
[dbservers]
db0
db1
Now, when running tasks on your web servers, you can simply refer to the dbserver variable to find the appropriate server. Assuming a standard naming scheme, you could automate this through the use of the set_fact module.
Update
You don't need to specify the links in both directions; you could use
something like this to find the webservers which should have access to
a given database server:
- name: allow access to webserver
command: >
iptables -A INPUT
-s {{hostvars[item].ansible_default_ipv4.address}}
with_items: "{{groups.webservers}}"
when: "hostvars[item].dbserver == {{inventory_hostname}}"
I ended up using the following approach: Each pair of servers gets their own group, and group variables are used to define which server runs on which host:
[test]
test-host
[test:vars]
web_server = test-host
db_server = test-host
[production]
web
db
[production:vars]
web_server = web
db_server = db
The tasks for web and database servers are then put into separate roles (web_server and db_server). I then switch between those roles based on the group variables:
- hosts: all
roles:
- { role: web_server, when: "web_server == inventory_hostname" }
- { role: db_server, when: "db_server == inventory_hostname" }
This setup allows me to easily define shared variables for roles and pairs, and it also allows me to easily target just one pair (via --limit).
I know this is kind of a newbie question but: How should I define my hosts file? if I want a NTP server and several clients and still have the ability to do common things to all the hosts?
I have, let's say, 10 servers. One of them will be the NTP server and the rest will sync against it. I could define my hosts file like:
[ntp-server]
hostA
[ntp-slaves]
hostB
hostC
hostD
...
[my_servers]
ntp-server
ntp-slaves
So, I can apply common config (ssh, iptables) to all of them. I can have other classifications like webservers, loadbalancers or any other. So, far I can't figure out how to solve this the "elegant" way.
Also, related to this, should I have two roles? (ntp_server, ntp_client) or a single one with differente behaviour if it's inside the client or server group?
Thank You
You're on the right track, but the correct syntax for my_servers is :
[my_servers:children]
ntp-server
ntp-slaves
Yes, Ansible needs a little help when building groups from other groups.
Note that you can combine [my_servers:children] (to add groups) and [my_servers] (to add hosts) in the same inventory file.