Consider the following Puppet-provisioned Vagrant project directory structure:
environments/
dev/
manifests/
site.pp <- Some standard Puppet stuff in here.
prod/
manifests/
site.pp <- Very similar to dev/manifests/site.pp, but with some prod differences
manifests/
default.pp <- Empty, but a place I'd like to keep common parts of the site.pp files
modules/
Vagrantfile <- See below.
Puppetfile
and the following (simplified) Vagrantfile:
# -*- mode: ruby -*-
# vi: set ft=ruby :
envs = ["local", "dev"]
Vagrant.configure(2) do |config|
config.vm.box = "puppetlabs/centos-7.0-64-puppet"
envs.each do |env|
config.vm.define env do |node|
node.vm.provision "puppet" do |puppet|
puppet.environment_path = "environments"
puppet.module_path = "modules"
puppet.environment = env
end
end
end
end
What options are available to me if I'd like to factor out the common parts of the two site.pp files into a single file?
I have tried the following strategies to no avail:
1 - One Manifest Per Environment Which "Inherits" From a Single Project-Wide Manifest
After some reading I'm not even sure this is possible. There used to be the import command which I feel like could have been what I was looking for, but that has been deprecated in newer versions of Puppet.
2 - A Single Project-Wide Manifest and One Hiera File Per Environment
I consider this the least-attractive of the two options since it means the only way to distinguish between environments is via data, and some things are just better expressed with differences in manifests.
However I can't even get this to work. Given the following environment.conf files in the dev/ and prod/ folders:
manifest = ../../manifests/default.pp
gives me an error about not being able to find the default.pp file.
Basically I need to a way to stop duplication between environments. My project is actually more complicated than that above, given that the manifests are getting rather large and unwieldy, and I have more than two environments. I also feel like the fact I'm using Vagrant here (which has it's own ways of passing options to Puppet) is complicating things further.
Any help at all would be appreciated here.
So, here's what I went with. Not what I originally wanted but I'm still new to Puppet and not even convinced what I wanted is possible anyway.
I settled on using hiera data to distinguish between environments, and am now using a single manifest (default.pp). The two site.pp files are completely empty, and are only there as placeholders so Git keeps the dev and prod folders. Rather than have environment.conf files pointing towards the manifest, I simply added the following to the Vagrantfile:
puppet.manifest_file = "default.pp"
I also have started putting things into their own modules, rather than having a huge monolithic manifest.
Related
Vagrant Multi-Machine functionality seems pretty cool, however one thing that bothers me (or is isn't immediately apparent) is that there doesn't seem to be a good way to share configuration options between a "parent" Vagrantfile and "children" Vagrantfiles. Is there a way to effectively and maintainably share configuration options between the parent and it's children? An example might make this more clear.
Let's assume I've got a platform which is comprised of 3 apps/services: API, Web, and Worker.
Let's presume a directory structure of the following:
/some_platform
/api
# app code...
Vagrantfile
/web
# app code...
Vagrantfile
/worker
# app code...
Vagrantfile
Vagrantfile
Let's say /some_platform/api/Vagrantfile looks like:
Vagrant.configure("2") do |config|
config.vm.box = "debian/jessie64"
end
Presumably the web and worker Vagrantfiles look similar.
Now, using the wonders of Multi-Machine I rely on Vagrant coordinate these VMs, and /some_platform/Vagrantfile looks like:
Vagrant.configure("2") do |config|
config.vm.define "web" do |api|
api.vm.box = "debian/jessie64"
end
config.vm.define "web" do |web|
web.vm.box = "debian/jessie64"
end
config.vm.define "web" do |worker|
worker.vm.box = "debian/jessie64"
end
end
I realize this example is contrived, but it's easy to see how once you get more and more complex config declarations, it's annoying and hazardous to have that config duplicated in two places.
You might be wondering "Why does each project have it's own Vagrantfile?" Doing so provides a single source of truth for how the server that app runs on should be setup. I realize there are provisioners you can use (and I will use them), but you still have to declare a few other things outside of that and I want to keep that DRY so that I can either bring up a cluster of apps via Multi-Machine, or I can work on a single app and change it's VM/server setup.
What I'd really love is a way to merge other Vagrantfiles into a "parent" file.
Is that possible? Or am I crazy for trying? Any clever ideas on how to achieve this? I've mucked about with some yaml files and POROs to skate around this issue, but none of the hacks feel very satisfying.
Good news!
You can in fact apply DRY principles in a Vagrantfile.
First: Create a file /some_platform/DRY_vagrant/Vagrantfile.sensible to hold some sensible defaults :
Vagrant.configure("2") do |config|
# With the setting below, any vagrantfile VM without a 'config.vm.box' will
# automatically inherit "debian/jessie64"
config.vm.box = "debian/jessie64"
end
Second: Create a file /some_platform/DRY_vagrant/Vagrantfile.worker for the 'worker' virtual machine :
Vagrant.configure("2") do |config|
config.vm.define "worker" do |worker|
# This 'worker' VM will not inherit "debian/jessie64".
# Instead, this VM will explicitly use "debian/stretch64"
worker.vm.box = "debian/stretch64"
end
end
Finally: Create a file /some_platform/Vagrantfile to tie it all together :
# Load Sensible Defaults
sensible_defaults_vagrantfile = '/some_platform/DRY_vagrant/Vagrantfile.sensible'
load sensible_defaults_vagrantfile if File.exists?(sensible_defaults_vagrantfile)
# Define the 'api' VM within the main Vagrantfile
Vagrant.configure("2") do |config|
config.vm.define "api" do |api|
# This 'api' VM will automatically inherit the "debian/jessie64" which we
# configured in Vagrantfile.sensible
# Make customizations to the 'api' VM
api.vm.hostname = "vm-debian-jessie64-api"
end
end
# Load the 'worker' VM
worker_vm_vagrantfile = '/some_platform/DRY_vagrant/Vagrantfile.worker'
load worker_vm_vagrantfile if File.exists?(worker_vm_vagrantfile)
This approach can be used for almost any other vagrantfile config options. It is not limited to just the "config.vm.box" setting.
Hope this helped!
There are 2 things you can look at (probably more, but those two comes to my mind)
look at How to template Vagrantfile using Ruby? its an example how you can read the content of another file, Vagrantfile is just a ruby script so you can use all the power of ruby.
vagrant has a concept of loading and merging, see from doc so if you wanted to do something anytime you run a vagrant command, you could create a Vagrantfile under your ~/.vagrant.d/ folder and it will always run
one drawback (or at least to pay attention) : the Vagrantfile is a ruby script that is evaluated each (and every time) a vagrant command is executed (up, status, halt ....)
I am having an absolute nightmare getting Puppet to load a group of modules that will be shared between multiple environments.
The modules in puppet/environments/development/modules get loaded fine BUT none of the dependencies in puppet/modules can be found.
The folder structure for my project is:
And the project is up on bitbucket:
https://bitbucket.org/andrew_hancox/vagrantmoodle
What I do usually to manage the modules dependencies is to have a shell script that will install the modules directly, this way it downloads the necessary dependency as well as pushing to the right place.
I will have in my Vagrantfile
node_config.vm.provision "shell", path: "puppet/script/install-puppet-modules-app.sh"
node_config.vm.provision :puppet do |puppet|
puppet.environment = "production"
puppet.environment_path = "puppet/environments"
puppet.manifests_path = "puppet/environments/production/manifests"
puppet.manifest_file = "base-app.pp"
#puppet.options = "--verbose --trace"
end
The script shell is something like
#!/bin/bash
puppet module install puppet-nginx --version 0.4.0
here you will have your apache, mysql module etc
the environment.conf file will locate the default place for the installed module
# environment configuration used by Puppet4
modulepath = /etc/puppetlabs/code/environments/production/modules:$basemodulepath
I got the whole thing working properly thanks to #michael-mulqueen
The way he fixed it was by setting the module path in the vagrant file:
puppet.module_path = ["puppet/modules", "puppet/environments/development/modules"]
You can see this in the repo referenced in the question.
Not experienced with puppet and vagrant. We used to have all the puppet settings in a big puppet repository and our vagrant instance worked fine. Recently, to have things better isolated so they are not rolled out accidentally, we have a certain path from the big puppet repos separated into a new puppet repos. But I still need all the facters living within the old big repos while the server specific setting from the new repos. Unfortunately, I cannot just specify the big old repos as a module path as my server specific settings will come from the old directory in the old repos, but if I just specify the new repos as the module path, then I miss all the facts. And I have been googling crazily to find a way to specify the facter path for vagrant in vain :(
old_repos -> dir -> my_server_setting
-> module1 -> facter
-> module2 -> facter
....
new_repos -> my_server_setting
can anyone please give me some hints? many thanks
The module_path can be a vector
Vagrant::Config.run do |config|
...
config.vm.provision :puppet do |puppet|
puppet.manifests_path = "manifests"
puppet.module_path = ["old_repos","new_repos"]
puppet.manifest_file = "base.pp"
end
end
I never tried but I guess the hiera_config_path can specify a vector of location too so if you have hiera defined in both the old and the new repos you should be able to point the 2.
I have a projekt with a setup for a multi machine environment for Vagrant. I had to fix some problems, which were initially caused by the redirect issue to https, but solving these lead into other errors, which I fixed in all projects except this one now, which uses the multi machine feature of Vagrant.
So I have this folder structure:
/Vagrantfile
/puppet/box_1/puppetfile
/puppet/box_1/manifests/site.pp
This is my code snippet, where I define my provision directories:
config.vm.provision :puppet do |puppet|
puppet.manifests_path = "puppet/box_1/manifests"
puppet.manifest_file = "site.pp"
end
My puppetfile looks like this:
forge "https://forgeapi.puppetlabs.com"
mod 'tPl0ch/composer'
mod 'puppetlabs/apt'
mod 'puppetlabs/apache'
mod 'puppetlabs/firewall'
In my site.pp I try to include apt, but I get this error message:
Error: Evaluation Error: Error while evaluating a Function Call, Could not find class ::apt for project.local at /tmp/vagrant-puppet/manifests-f2b1fd0ac42b51938ed6ae7e6917367e/site.pp:1:1 on node project.local
When I rearange my puppet files like this:
/Vagrantfile
/puppet/puppetfile
/puppet/manifests/site.pp
like this is the common way of setting this up, it works without that problem, but as I mentioned, there are other boxes, which use different puppetfiles and site.pp files, so this folder structure makes some kinda sense. It seems, that it doesn'even matter, if I delete the config for the other boxes, and setup my Vagrantfile, as if it would be only one box, so I am just confused, how the location, of these files influence the scope of certain classes.
So my questions is here: Is there a way, to keep this folder structure and still have these modules defined in puppetfile available in my site.pp? Or is this generally some kinda bad practice to organize it this way? I was searching for some examples for this, but couldn't find any for some reason...
EDIT: It seems, on provision the puppetfile isnt even used anymore, when its not located in /puppet/ So maybe I just have to tell Vagrant how to use it?
define where librarian should find the puppet file
Vagrant.configure("2") do |config|
config.librarian_puppet.puppetfile_dir = "puppet/box1"
config.vm.provision :puppet do |puppet|
puppet.manifests_path = "puppet/box_1/manifests"
puppet.manifest_file = "site.pp"
end
I'd like to specify directly in the vagrantfile which provider to use by default for each VM.
For example, given this vagrantfile:
# Vagrantfile
[...]
config.vm.define 'dev_vm' do |machine|
machine.vm.provider :libvirt do |os|
[...]
end
# machine.default_provider = :libvirt
end
config.vm.define 'production_vm' do |machine|
machine.vm.provider :openstack do |os|
[...]
end
# machine.default_provider = :openstack
end
To boot up the following to VMs, I have to issue two commands currently:
vagrant up --provider=libvirt dev_vm
vagrant up --provider=openstack production_vm
I'd like to bring up both with a single vagrant up, especially because I'm running quite a few more machines. Some configuration like the commented machine.default_provider = :openstack would be fantastic to have.
Is there a way to do so?
I don't think there is any easy way to do it. Vagrant will currently use the same provider during the whole run so it could possibly be quite big code change to support this.
Maybe wrapper scripts are the easiest solution now.
Another workaround would be to use separate Vagrantfiles for the VMs and set VAGRANT_DEFAULT_PROVIDER in each. If there is a lot of common config, you could extract it to e.g. Vagrantfile.common, which is included by the others. Something like:
# Vagrantfile 1
ENV['VAGRANT_DEFAULT_PROVIDER'] = 'libvirt'
# assume the common config is in parent directory
load File.expand_path('../../Vagrantfile.common', __FILE__)
Vagrant.configure('2') do |config|
# ...
end