I am experimenting with Puppet using Vagrant. I'm new to Puppet.
I'm installing modules in my Puppet manifest using the approach suggested at: Can I install puppet modules through puppet manifest?
My default.pp contains something like:
$dsesterojava = 'dsestero-java'
exec { 'dsestero-java':
command => "puppet module install ${dsesterojava}",
unless => "puppet module list | grep ${dsesterojava}",
path => ['/usr/bin', '/bin']
}
include java::java_7
I'm trying to import a module and then immediately use the classes defined in it.
Currently, I get:
Error: Could not find class java::java_7
If I comment out the include line and re-run it. The module installs. If I then removed the comment and run the provisioning again then it works.
There is some kind of "chicken and egg" situation here. Can I use a module in the same Puppet manifest that installs it?
How should I solve it?
No, you cannot do this. When your catalog is compiled, Puppet will search in the appropriate directories for all of the required code and data. Since the java module does not exist until catalog application, the compilation of a catalog (occurs prior to application) depending upon it will fail. You are absolutely dealing with a "chicken and egg" situation here. I highly recommend against using Puppet code to install Puppet code.
Alternatively, the recommended approach to install and manage your Puppet modules is to use one of these solutions:
librarian-puppet: http://librarian-puppet.com/
r10k: https://github.com/puppetlabs/r10k
code-manager (PE only): https://puppet.com/docs/pe/2017.3/code_management/code_mgr.html
These will also solve the problem for you within the Vagrant if you are using the agent provisioner and subscribing the Vagrant instance to a Puppet Master.
If you are using the apply provisioner inside of Vagrant, then you will need to go a different route. The simplest solution is to use the shell provisioner to install Puppet modules via module install after the Puppet installation (unless you are using a Vagrant box with Puppet baked in, in which case you are probably not installing Puppet on it). Alternatively, you could share a directory with the host where your modules are installed, or install the librarian-puppet or r10k gems onto the Vagrant box and then use them to install into the appropriate path. I can go into more detail on these upon request.
So I am tasked with packaging a Ruby application. The idea is to eliminate the need to apt-get any packages or gem install anything. the package is all-inclusive.
I've gotten as far as finding Omnibus (there is also fmp-cookery but it doesnt seem to be as well known)
I've figured out the general structure of omnibus projects but now getting problems with specifics:
1) I have a bunch of recipies that run with Chef AT INSTALL TIME. These setup the DB and nginx configs. Whats the best way to run through these on target machine?
2) I added a bundle install line in my build scripts, but this now dubplicates my requirements. For example nokogiri gets installed twice. Once as my actual project Gem and another time as another requirement of omnibus. So then I end up with 2 binaries one in /#{instal_dir}/embedded/bin and one in ${install_dir}/embedded/lib/ruby/.../gems any way to prevent this?
Building an installer is bit more involved than creating a chef cookbook.
User needs to write a DSL for each module (this has instructions on gathering the artifacts and installing it on specific OS).
After creating the omnibus project, refer to the README file to get started with basics.
Refer to this github project for details on how to build your own DSLs. Gitlab omnibus project has DSLs for some modules like postgresql etc.
https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master
Example DSL for embedding nginx module in your installer looks like this:
name "nginx"
default_version "1.9.10"
dependency "pcre"
dependency "openssl"
source url: "http://nginx.org/download/nginx-#{version}.tar.gz",
md5: "64cc970988356a5e0fc4fcd1ab84fe57"
relative_path "nginx-#{version}"
build do
command ["./configure",
"--prefix=#{install_dir}/embedded",
"--with-http_ssl_module",
"--with-http_stub_status_module",
"--with-http_gzip_static_module",
"--with-http_v2_module",
"--with-ipv6",
"--with-debug",
"--with-ld-opt=-L#{install_dir}/embedded/lib",
"--with-cc-opt=\"-L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include\""].join(" ")
command "make -j #{workers}", :env => {"LD_RUN_PATH" => "#{install_dir}/embedded/lib"}
command "make install"
end
I've been looking for a puppet package website such as https://packagist.org/ but with no luck, are there any websites out there where you can search for a mongodb/php or elascticsearch puppet package?
I've looked at puphpet but it doesn't have all the packages that I'm looking for.
Thanks.
When I am searching for puppet modules, the first place I go to is the official puppetlabs forge:
https://forge.puppetlabs.com/
You can install puppet modules directly from the puppetforge using the following command line:
puppet module install puppetlabs-apache --version 0.0.2
(http://docs.puppetlabs.com/puppet/latest/reference/modules_installing.html)
I would like to minimize the setup for users of my vagrant project.
I already think that installing Virtualbox and Vagrant are too steps too many for users of my project. See here for more info on that.
Because my project depends on a few plugins, rather than require the user to use vagrant plugin install ..., I would like to automate installation of the plugins.
Question 1: Is it an ok practise to include the plugin source in my project tree, and include the plugin using Vagrant.require_plugin "plugin-dir/plugin-name"?
Question 2: Will this approach work?
The plugins I depend on are available on github so I can add them as git submodules as described here
Edit
The approach above did NOT work for the vagrant-vbguest plugin
I had a similar need and the method I went with was to add the following at the top of my Vagrantfile:
unless Vagrant.has_plugin?("vagrant-omnibus") || ARGV[0] == 'plugin'
origargs = ARGV.join(" ");
puts "Plugins not found"
exec "vagrant plugin install vagrant-omnibus;vagrant #{origargs}"
end
It doesn't really answer your questions I guess but it works for me, I found that I also needed to add sudo in front of the vagrant plugin command for my ubuntu setup, but as you don't state what OS you are running that might not be relevant.
Supposed execution of a Vagrantfile requires a specific Vagrant plugin to be installed. So, basically what you need to do is
$ vagrant plugin install foobar-plugin
$ vagrant up
If you skip the first step, vagrant up fails.
Is there an option in Vagrant to make it install the plugin automatically? In other words: Is it possible to specify within a Vagrantfile which plugins to install automatically before creating and booting up the machine?
UPDATE Aug 31, 2018: See #Starx's fix below for later versions of Vagrant (1.8 and above)
Here is version based on Louis St. Amour's solution together with Rob Kinyon's comment about re-exec if a new plugin was installeed, I use it successfully in my own setup:
required_plugins = %w(vagrant-share vagrant-vbguest...)
plugins_to_install = required_plugins.select { |plugin| not Vagrant.has_plugin? plugin }
if not plugins_to_install.empty?
puts "Installing plugins: #{plugins_to_install.join(' ')}"
if system "vagrant plugin install #{plugins_to_install.join(' ')}"
exec "vagrant #{ARGV.join(' ')}"
else
abort "Installation of one or more plugins has failed. Aborting."
end
end
Since I'm a Ruby dev, and Bindler is no longer being maintained, I found it most natural to just write some code at the top of my Vagrantfile to install required plugins if missing (e.g. before Vagrant.configure line)
The following works for me:
required_plugins = %w( vagrant-hostmanager vagrant-someotherplugin )
required_plugins.each do |plugin|
system "vagrant plugin install #{plugin}" unless Vagrant.has_plugin? plugin
end
system, unlike using backticks, will echo the command to stdout, just as running the command yourself would. And this way I don't need yet another strangely named plugin or system to keep track of required plugins which can be updated by Vagrant anyway.
As I pointed out on my answer to your other question, you can use bindler for installing a set of plugins specific to a project using a single command.
If bindler is installed and the required plugin is not, bindler will error out and will abort the process. There is also an open issue related to automatically installing plugins on vagrant ups but so far no one signed up for it yet.
If you don't want to use bindler, you can make use of Vagrant.has_plugin? (available on 1.3.0+) at the top of your Vagrantfile and error out if the required plugin is not installed.
Something like:
unless Vagrant.has_plugin?("vagrant-some-plugin")
raise 'some-plugin is not installed!'
end
Vagrant.configure("2") do |config|
config.vm.box = "box-name"
end
UPDATE: Bindler is no longer supported and no equivalent funcionality has been provided by Vagrant core as of May 11th, 2015
2019 Update: Vagrant now has functionality to require plugins through the Vagrantfile via:
Vagrant.configure("2") do |config|
config.vagrant.plugins = "vagrant-some-plugin"
# or as array:
config.vagrant.plugins = ["vagrant-some-plugin", "vagrant-some-other-plugin"]
# or as hash
config.vagrant.plugins = {"vagrant-some-plugin" => {"version" => "1.0.0"}}
end
If Vagrant detects there are plugins not already installed it will prompt the user to install them itself:
$ vagrant up
Vagrant has detected project local plugins configured for this
project which are not installed.
vagrant-some-plugin
Install local plugins (Y/N) [N]: y
Installing the 'vagrant-some-plugin' plugin. This can take a few minutes...
Fetching vagrant-some-plugin-1.0.0.gem
Installed the plugin 'vagrant-some-plugin (1.0.0)'!
Vagrant has completed installing local plugins for the current Vagrant
project directory. Please run the requested command again.
See https://www.vagrantup.com/docs/vagrantfile/vagrant_settings.html
Please note that as of Vagrant 1.5, you can specify your dependencies in your Gemfile. Per the blog post on the update:
Now, Vagrant 1.5 will automatically load any gems in the "plugins"
group in your Gemfile. As an example, here is the Gemfile for a
"vagrant-bar" plugin:
source "https://rubygems.org"
group :development do
gem "vagrant",
git: "https://github.com/mitchellh/vagrant.git"
end
group :plugins do
gem "vagrant-foo",
gem "vagrant-bar", path: "."
end
Couldn't add a comment to Louis St-Amour's answer, but I wanted to post this just in case anyone needed help extending his solution.
# Check for missing plugins
required_plugins = %w(vagrant-list)
plugin_installed = false
required_plugins.each do |plugin|
unless Vagrant.has_plugin?(plugin)
system "vagrant plugin install #{plugin}"
plugin_installed = true
end
end
# If new plugins installed, restart Vagrant process
if plugin_installed === true
exec "vagrant #{ARGV.join' '}"
end
On the new version of Vagrant, answer by #Amos Shapira gets stuck in an infinite loop. The reason for that is each call to vagrant plugin install also processes the Vagrantfile and when processed executes the code relating to installing the plugin again and again and so on.
Here is my solution which avoids the loop.
# Plugins
#
# Check if the first argument to the vagrant
# command is plugin or not to avoid the loop
if ARGV[0] != 'plugin'
# Define the plugins in an array format
required_plugins = [
'vagrant-vbguest', 'vagrant-hostmanager',
'vagrant-disksize'
]
plugins_to_install = required_plugins.select { |plugin| not Vagrant.has_plugin? plugin }
if not plugins_to_install.empty?
puts "Installing plugins: #{plugins_to_install.join(' ')}"
if system "vagrant plugin install #{plugins_to_install.join(' ')}"
exec "vagrant #{ARGV.join(' ')}"
else
abort "Installation of one or more plugins has failed. Aborting."
end
end
end
I just noticed here http://docs.vagrantup.com/v2/plugins/packaging.html an instruction
Vagrant.require_plugin "vagrant-aws"
which does exactly the same thing as what descibed fgrehm: raising quickly an error if the plugin is not installed.
As far as I know, there are stil no way to auto-install plugins
My answer is very close to Louis St-Amour's answer, but instead of installing plugins automatically, it just raises an error message, so that the user has to install the plugin manually.
I would rather users be aware of any plugins that get installed, because they apply globally to all Vagrant instances, not just to the current Vagrantfile.
Put at the top of Vagrantfile one line like this for each plugin, in this example, vagrant-vbguest:
raise "vagrant-vbguest plugin must be installed" unless Vagrant.has_plugin? "vagrant-vbguest"
You could use this project (I am the author): https://github.com/DevNIX/Vagrant-dependency-manager
It's based on similar answers but trying to be more complete and stable. The big advantage of this idea is, you can distribute your Vagrantfile and it will run on every computer regardless of the installed plugins on that environment.
It is easy to use:
Copy dependency_manager.rb next to your Vagrantfile
Include it and call check_plugins passing your dependencies as an array
Example:
# -*- mode: ruby -*-
# vi: set ft=ruby :
require File.dirname(__FILE__)+"./dependency_manager"
check_plugins ["vagrant-exec", "vagrant-hostsupdater", "vagrant-cachier", "vagrant-triggers"]
Vagrant.configure(2) do |config|
config.vm.box = "base"
end
???
Profit!
I would love to merge pull requests, fix any issue you could have, and to get ideas of new features. Currently I'm thinking about updating the dependency manager itself, and requiring specific plugin versions :D
Regards!
I got a problem with new install of Vagrant, where .vagrant.d directory is not created yet. I was able to make the snippet from Luis St. Amour working by catching the exception.
required_plugins = %w(vagrant-share vagrant-vbguest)
begin
plugins_to_install = required_plugins.select { |plugin| not Vagrant.has_plugin? plugin }
if not plugins_to_install.empty?
puts "Installing plugins: #{plugins_to_install.join(' ')}"
if system "vagrant plugin install #{plugins_to_install.join(' ')}"
exec "vagrant #{ARGV.join(' ')}"
else
abort "Installation of one or more plugins has failed. Aborting."
end
end
rescue
exec "vagrant #{ARGV.join(' ')}"
end
Here's what I am using on Vagrant 1.8 and it is working fine. Put this somewhere before the configure block in your Vagrantfile.
# install required plugins if necessary
if ARGV[0] == 'up'
# add required plugins here
required_plugins = %w( plugin1 plugin2 plugin3 )
missing_plugins = []
required_plugins.each do |plugin|
missing_plugins.push(plugin) unless Vagrant.has_plugin? plugin
end
if ! missing_plugins.empty?
install_these = missing_plugins.join(' ')
puts "Found missing plugins: #{install_these}. Installing using sudo..."
exec "sudo vagrant plugin install #{install_these}; vagrant up"
end
end