I would like to have a fully configured ruby unix development environment using Vagrant configuration and provisioning. Ideally it would refer to a simple base box (e.g., precise32) and build the environment up through provisioning in such a way thait it will easily be repeatable for other team members, can be posted to github, and can be upgraded as new versions of the different technologies are available by just changing the provisioning. I have not found any full examples of this searching the web although [Rails Dev Box][1] has some useful ideas. Most of the dev environment examples (like Rails Dev Box) do not set up the guest dev environment because they assume dev will be done on the host using a shared file strategy - or if they do the configuration by hand and then save the box rather than provisioning it.
This also needs to work both in behind a proxy as well as with no proxy.
Here are the steps I am thinking will be required:
On the host:
install virtualbox, vagrant, vagrant proxyconf
On the guest, via Vagrantfile/provisioning:
use a base unix box (e.g., precise32)
optionally set proxy variables (if proxyconf plugin is installed and http_proxy env var is set)
provision everything else (puppet, chef, or shell script)
install various unix tools (apt-get install git, etc.expo ...)
set up bash environment
set up vim environment (pathogen plugin, ruby plugins, etc.)
install rvm
install ruby 1.9, 2.0, JRuby, Rubinius
installs and configures tmux
Ideally I could push this into github, it could be cloned, then cd to new directoy, and vagrant up to have a fully configured dev environment ...
Does anyone have any examples of doing this?
My preference for doing a task like this would be to use puppet as the provisioning step in your Vagrantfile.
With something like this, you can always get something thrown together quick and dirty by just doing all the steps in a shell provisioner... but I prefer the puppet and modules approach as I've found it easier to maintain, extend and to share with the team.
I've experimented with a couple of different ways of doing the provisioning with Ruby and rvm as you mentioned;
Theres the rvm puppet module by maestrodev which allows you to configure many of rvms core features: ruby versions, gemsets, gems and rvm wrappers. Typically to manage which puppet modules are included with a project I use the librarian-puppet gem which allows you to use a Puppetfile to specify the module and the version you require, much like bundler. This handles dependencies such as the stdlib and concat modules. This scenario requires external internet access to have been configured before provisioning so as to be able to download ruby and rubygems.
Offline installation of rvm - I made the relevant files (rvm itself, ruby and rubygems) accessible to the vagrant machine using a shared folder and turned the offline rvm instructions into a (not very good) puppet module and used that. One particular gotcha to pay attention to here is the naming of the ruby source that gets installed; the extension has to be .tar.bz2, its described in the list.
Additionally for your other provisioning steps you can build up puppet modules yourself for your additional requirements: vim / tmux etc and keep this versioned separately in git. You can get pretty far with modules with just the 'puppet trifecta':
class vim {
package { 'vim':
ensure => installed,
}
file { '.vimrc':
ensure => file,
...
}
}
Additionally check out the puppet forge for modules which might have already been written to do what you want.
So heres an example of what you could check in:
/ Puppetfile
/ README.md
/ Vagrantfile
/ puppet
/manifests
site.pp
And the vagrant provisioner would be
Vagrant.configure("2") do |config|
config.vm.provision "puppet" do |puppet|
puppet.manifests_path = "puppet/manifests"
puppet.module_path = "puppet/modules"
puppet.manifest_file = "site.pp"
end
end
I've used a rake task before to use librarian-puppet to pull in puppet dependencies from git / puppet forge and any additional steps you might need to do before vagranting up. This way the code as configuration is all you check in.
Finally, with puppet you can use the facter and hiera tools which are very useful for keeping data out of your modules and worth having a look at as a means of refactoring once you have your initial setup working.
Related
I am experimenting with Puppet using Vagrant. I'm new to Puppet.
I'm installing modules in my Puppet manifest using the approach suggested at: Can I install puppet modules through puppet manifest?
My default.pp contains something like:
$dsesterojava = 'dsestero-java'
exec { 'dsestero-java':
command => "puppet module install ${dsesterojava}",
unless => "puppet module list | grep ${dsesterojava}",
path => ['/usr/bin', '/bin']
}
include java::java_7
I'm trying to import a module and then immediately use the classes defined in it.
Currently, I get:
Error: Could not find class java::java_7
If I comment out the include line and re-run it. The module installs. If I then removed the comment and run the provisioning again then it works.
There is some kind of "chicken and egg" situation here. Can I use a module in the same Puppet manifest that installs it?
How should I solve it?
No, you cannot do this. When your catalog is compiled, Puppet will search in the appropriate directories for all of the required code and data. Since the java module does not exist until catalog application, the compilation of a catalog (occurs prior to application) depending upon it will fail. You are absolutely dealing with a "chicken and egg" situation here. I highly recommend against using Puppet code to install Puppet code.
Alternatively, the recommended approach to install and manage your Puppet modules is to use one of these solutions:
librarian-puppet: http://librarian-puppet.com/
r10k: https://github.com/puppetlabs/r10k
code-manager (PE only): https://puppet.com/docs/pe/2017.3/code_management/code_mgr.html
These will also solve the problem for you within the Vagrant if you are using the agent provisioner and subscribing the Vagrant instance to a Puppet Master.
If you are using the apply provisioner inside of Vagrant, then you will need to go a different route. The simplest solution is to use the shell provisioner to install Puppet modules via module install after the Puppet installation (unless you are using a Vagrant box with Puppet baked in, in which case you are probably not installing Puppet on it). Alternatively, you could share a directory with the host where your modules are installed, or install the librarian-puppet or r10k gems onto the Vagrant box and then use them to install into the appropriate path. I can go into more detail on these upon request.
I come from a Python and JavaScript background.
When developing a JavaScript project, dependencies are installed in a node_modules directory in the project root.
When developing Python project, typically virtualenvwrapper is used. In this case dependencies are installed in a virtual environment, which is located in ~/.virtualenvs/<project_name> by default.
Now I need to use a ruby tool for a project. The tool that appears to be the most promising for a similar setup as described above, is bundler.
However, the default installation location for bundler is system-wide. I consider this to be harmful.
For one of my systems, it will prompt for a password, at which point I can still abort.
However, for my other system I can write into the global ruby installation. I'm using a homebrew installed ruby here. Bundle will just install dependencies globally.
I know I can specify the installation location by adding --path, but this is easy to forget.
One way to enforce an installation path is by committing .bundle/config. It would just have to contain this:
---
BUNDLE_PATH: "."
However, some googling around shows that it's not adviced to commit this file.
What is the recommended way to prevent accidental global installations using bundler?
Who's to say it will be accidental? It really depends on what context you're talking about here. I have my Ruby set up so that bundle install works without requiring sudo, it's all done through rbenv automatically. The same is true with rvm if done as a user-level install.
When it comes to deploying apps and you want to make sure it's deployed correctly, that's where tools like Capistrano come into play: Create a deployment script that will apply the correct procedure every time.
Checking in a .bundle/config is really rude from a dev perspective, just like checking in any other user-specific preferences you might have. It causes no end of conflict with other team members.
So I am tasked with packaging a Ruby application. The idea is to eliminate the need to apt-get any packages or gem install anything. the package is all-inclusive.
I've gotten as far as finding Omnibus (there is also fmp-cookery but it doesnt seem to be as well known)
I've figured out the general structure of omnibus projects but now getting problems with specifics:
1) I have a bunch of recipies that run with Chef AT INSTALL TIME. These setup the DB and nginx configs. Whats the best way to run through these on target machine?
2) I added a bundle install line in my build scripts, but this now dubplicates my requirements. For example nokogiri gets installed twice. Once as my actual project Gem and another time as another requirement of omnibus. So then I end up with 2 binaries one in /#{instal_dir}/embedded/bin and one in ${install_dir}/embedded/lib/ruby/.../gems any way to prevent this?
Building an installer is bit more involved than creating a chef cookbook.
User needs to write a DSL for each module (this has instructions on gathering the artifacts and installing it on specific OS).
After creating the omnibus project, refer to the README file to get started with basics.
Refer to this github project for details on how to build your own DSLs. Gitlab omnibus project has DSLs for some modules like postgresql etc.
https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master
Example DSL for embedding nginx module in your installer looks like this:
name "nginx"
default_version "1.9.10"
dependency "pcre"
dependency "openssl"
source url: "http://nginx.org/download/nginx-#{version}.tar.gz",
md5: "64cc970988356a5e0fc4fcd1ab84fe57"
relative_path "nginx-#{version}"
build do
command ["./configure",
"--prefix=#{install_dir}/embedded",
"--with-http_ssl_module",
"--with-http_stub_status_module",
"--with-http_gzip_static_module",
"--with-http_v2_module",
"--with-ipv6",
"--with-debug",
"--with-ld-opt=-L#{install_dir}/embedded/lib",
"--with-cc-opt=\"-L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include\""].join(" ")
command "make -j #{workers}", :env => {"LD_RUN_PATH" => "#{install_dir}/embedded/lib"}
command "make install"
end
I would like to minimize the setup for users of my vagrant project.
I already think that installing Virtualbox and Vagrant are too steps too many for users of my project. See here for more info on that.
Because my project depends on a few plugins, rather than require the user to use vagrant plugin install ..., I would like to automate installation of the plugins.
Question 1: Is it an ok practise to include the plugin source in my project tree, and include the plugin using Vagrant.require_plugin "plugin-dir/plugin-name"?
Question 2: Will this approach work?
The plugins I depend on are available on github so I can add them as git submodules as described here
Edit
The approach above did NOT work for the vagrant-vbguest plugin
I had a similar need and the method I went with was to add the following at the top of my Vagrantfile:
unless Vagrant.has_plugin?("vagrant-omnibus") || ARGV[0] == 'plugin'
origargs = ARGV.join(" ");
puts "Plugins not found"
exec "vagrant plugin install vagrant-omnibus;vagrant #{origargs}"
end
It doesn't really answer your questions I guess but it works for me, I found that I also needed to add sudo in front of the vagrant plugin command for my ubuntu setup, but as you don't state what OS you are running that might not be relevant.
I'm working with someone else's chef recipe and it consists of these references to the process of installing 1.9.3p0 on my server:
package 'ruby1.9.3'
package 'ruby1.9.1-dev'
# set ruby 1.9 to be default
execute 'update-alternatives --set ruby /usr/bin/ruby1.9.1'
execute 'update-alternatives --set gem /usr/bin/gem1.9.1'
ohai "reload" do
action :reload
end
I'm new to chef so I'm not sure where these packages reside, but seeing no other reference them to them in the repo of recipes, I'm guessing it's referring to a central repo. In that case, how could I modify this recipe to get chef (solo) to prepare my servers with a different patch level?
The documentation may clear things up a little here:
package tells the chef-client to use one of sixteen different
providers during the chef-client run, where the provider that is used
by chef-client depends on the platform of the machine on which the
chef-client run is taking place
So on Debian-based systems like the one that recipe was written for, Chef will automatically resolve the package resource to an apt_package resource, which will call apt-get to install ruby1.9.3.
Now, given none of the mainstream Linux distros or FreeBSD package up multiple patchlevels of Ruby (and, in some cases, stated patchlevels are not what they seem), you probably don't want to use package to get Ruby. Most likely you'll end up wanting to build it from source using something like the bash resource.
package will still be useful for installing the Ruby prerequisites, which you can use from your vendor's package repository without issue.