Using Vagrant to manage development and production environments? - vagrant

How are people handling simple automation (with puppet) for dev / prod environments with vagrant (ideally from the same vagrantfile)?
Use case I'm trying to solve
I would love to spin up the the production machine with vagrant if it isn't created.
I would love to reload nginx or apache confs on production with vagrant if they were tweaked in the puppet files for my dev environment.
The Problem
When you call vagrant up with a provider like AWS or Digital Ocean, it becomes the active provider and you can't switch. You get this error:
An active machine was found with a different provider. Vagrant
currently allows each machine to be brought up with only a single
provider at a time. A future version will remove this limitation.
Until then, please destroy the existing machine to up with a new
provider.
It seems the answer it to destroy, but I just need to switch. I don't want to destroy.
I would love to be able to say
vagrant up prod
or
vagrant reload prod
and then a simple vagrant up would fall back to the default machine.
This syntax is similar to how multiple machines work, but I don't want to spin up a dev and production environment when I just call vagrant up (which is the default behavior).
Should I be looking at packer as part of the workflow? I watched the whole talk at puppetconf 2013 on Mitchell's talk on Multi-Provider http://puppetlabs.com/presentations/multi-provider-vagrant-aws-vmware-and-more
I'm still not seeing a solution for my problem.
UPDATE 9/27/13
In case anybody else is fighting this idea, this article cleared up a lot of questions I had.
http://pretengineer.com/post/packer-vagrant-infra

As for workaround, you should define config.vm.define (as suggested here), in order to support multiple providers.
Please find the following configuration posted by #kzap as example:
Vagrant.configure("2") do |config|
# Store the current version of Vagrant for use in conditionals when dealing
# with possible backward compatible issues.
vagrant_version = Vagrant::VERSION.sub(/^v/, '')
# Configuration options for the VirtualBox provider.
def configure_vbox_provider(config, name, ip, memory = 2048, cpus = 1)
config.vm.provider :virtualbox do |v, override|
# override box url
override.vm.box = "ubuntu/trusty64"
# configure host-only network
override.vm.hostname = "#{name}.dev"
override.vm.network :private_network, id: "vvv_primary", ip: ip
v.customize ["modifyvm", :id,
"--memory", memory,
"--cpus", cpus,
"--name", name,
"--natdnshostresolver1", "on",
"--natdnsproxy1", "on"
]
end
end
default_provider = "virtualbox"
supported_providers = %w(virtualbox rackspace aws managed)
active_provider = ENV['VAGRANT_ACTIVE_PROVIDER'] # it'd be better to get this from the CLI --provider option
supported_providers.each do |provider|
next unless (active_provider.nil? && provider == default_provider) || active_provider == provider
#
# VM per provider
#
config.vm.define :"sample-#{provider}" do | sample_web_config |
case provider
when "virtualbox"
configure_vbox_provider(sample_web_config, "examine-web", "192.168.50.1")
when "aws"
configure_aws_provider(sample_web_config)
when "managed"
configure_managed_provider(sample_web_config, "1.2.3.4")
when "rackspace"
configure_rackspace_provider(sample_web_config)
end
end
end
Or the following example posted at gist by #maxlinc:
# -*- mode: ruby -*-
# vi: set ft=ruby :
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "dummy"
config.vm.provider :rackspace do |rs|
rs.username = ENV['RAX_USERNAME']
rs.api_key = ENV['RAX_API_KEY']
rs.rackspace_region = :ord
end
supported_providers = %w(virtualbox rackspace)
active_provider = ENV['VAGRANT_ACTIVE_PROVIDER'] # it'd be better to get this from the CLI --provider option
supported_providers.each do |provider|
next unless active_provider.nil? || active_provider == provider
config.vm.define "exact_name_#{provider}" do |box|
box.vm.provider :rackspace do |rs|
rs.flavor = '1 GB Performance'
rs.image = 'Ubuntu 14.04 LTS (Trusty Tahr) (PVHVM)'
end
end
config.vm.define "regex_#{provider}" do |box|
box.vm.provider :rackspace do |rs|
rs.flavor = /1\s+GB\s+Performance/
rs.image = /Ubuntu.*Trusty Tahr.*(PVHVM)/
end
end
config.vm.define "id_#{provider}" do |box|
box.vm.provider :rackspace do |rs|
rs.flavor = 'performance1-1'
rs.image = 'bb02b1a3-bc77-4d17-ab5b-421d89850fca'
end
end
config.vm.define "unlisted_#{provider}" do |box|
box.vm.provider :rackspace do |rs|
rs.flavor = 'performance1-1'
rs.image = '547a46bd-d913-4bf7-ac35-2f24f25f1b7a'
end
end
end
end

Not an ideal solution, but what about using git branches? My thinking is that it could be conceptually similar to using heroku, where you might have a master, staging, and production versions (since they're usually different remotes).
In this case you start off the prod branch with the small edit to the Vagrantfile to name the VM a little differently. Then you should be able to merge all changes from dev with the prod branch as they occur. So your workflow would look like:
$ git checkout prod
$ vagrant up
$ git checkout master
... make changes to puppet ...
$ git checkout prod
$ git merge master
$ vagrant reload
$ git checkout master
You could script and alias these so you end up with
$ start_production
$ reload_production

Here is a simple way of dynamically changing the 'default' machine name depending on the specified --provider from the command line, so they won't conflict between the different providers:
require 'getoptlong'
opts = GetoptLong.new(
[ '--provider', GetoptLong::OPTIONAL_ARGUMENT ],
[ '--vm-name', GetoptLong::OPTIONAL_ARGUMENT ]
)
provider=ENV['PROVIDER'] || 'virtualbox'
vm_name=ENV['VM_NAME'] || 'default'
opts.each do |opt, arg|
case opt
when '--provider'
provider=arg
when '--vm-name'
vm_name=arg
end
end
Vagrant.configure(2) do |config|
# HERE you are dynamically changing the machine name to prevent conflict.
config.vm.define "mt-#{provider}-#{vm_name}"
# Below sections are just examples, not relevant.
config.vm.provider "virtualbox" do |vm|
vm.name = "test.local"
vm.network "private_network", ip: "192.168.22.22"
vm.customize ['modifyvm', :id, '--natdnshostresolver1', 'on']
config.vm.box = "ubuntu/wily64"
end
config.vm.provider :aws do |aws, override|
aws.aws_profile = "testing"
aws.instance_type = "m3.medium"
aws.ami = "ami-7747d01e"
config.vm.box = "testing"
end
end
Example usage:
VM_NAME=dev PROVIDER=virtualbox vagrant up --provider=virtualbox
VM_NAME=uat PROVIDER=aws vagrant up --provider=aws
VM_NAME=test PROVIDER=aws vagrant up --provider=aws
VM_NAME=prod PROVIDER=aws vagrant up --provider=aws
VM_NAME=uat PROVIDER=aws vagrant destroy -f
VM_NAME=test PROVIDER=aws vagrant status
See also: Multiple provisioners in a single vagrant file?

what I came up with to work with this scenario is to manage 2 distincts .vagrant folder.
Note: most of the other answers deal with setting up multi-provider assuming you will run dev and prod on different provider, in most cases this might be true but you can definitely have cases where you have same provider for dev and prod. Lets say you're using aws and you want to use dev and prod as ec2 instance it will be the same provider.
Say you want to manage dev and prod instances, potentially using different providers (but could also very well be on the same provider) so you'll do:
set up dev instance with normal vagrant up --provider <dev_provider>.
This will create a dev VM that you can manage
back up the .vagrant folder created in your project directory and rename it like .vagrant.dev
set up prod instance with your provider of choice vagrant up --provider <prod_provider>. This now creates your prod VM
back up the newly .vagrant folder created in your project directory and rename it like .vagrant.prod
now, depending if you want to work on dev or prod, you'll rename the .vagrant.dev or .vagrant.prod directory as .vagrant and vagrant will operate the right VM.
I did not come up with a script as mainly the most of the time I work with dev and very few times I need to switch to the other provider. but I dont think it will be too hard to read the parameter from CLI and make the renaming more dynamic.

Related

Multi-machine Vagrant with Ubuntu - Sinatra & PostgreSQL

I'm stuck again. I need to provision a multi-machine Environment - one VM for a Sinatra app and a second for its PostgreSQL DB.
So far, I've managed to get the Sinatra app up and running in the ubuntu/xenial64 box but the provisioning "breaks" when it hits the configuration for the DB
Vagrant.configure("2") do |config|
config.vm.define "app" do |app|
# Use ubuntu/xenial64 as the virtual machine
app.vm.box = "ubuntu/xenial64"
# Use a private network to connect the VM to the local machine via an IP with an alias
app.vm.network "private_network", ip: "192.168.10.100"
app.hostsupdater.aliases = ["development.local"]
# sync the 'app' directory in the local directory to '/app' on the VM
app.vm.synced_folder "app", "/app"
# Use the provisioning script in envirnonment to provision the VM for a Sinatra environment
app.vm.provision "shell", path: "environment/app/provision.sh"
app.vm.provision "shell", inline: set_env({ DATABASE_URL: "postgresql://myapp:dbpass#localhost:15432/myapp" })
end
config.vm.define "db" do |db|
db.vm.box = "ubuntu/trusty64"
db.vm.host_name = "postgresql"
db.vm.network "private_network", ip: "10.0.2.15"
# db.vm.forward_port 8000, 8000
db.hostsupdater.aliases = ["database.local"]
# db.vm.share_folder "home", "/home/vagrant", ".", :create => true
db.vm.provision "shell", path: "environment/db/provision.sh", privileged: false
end
end
As you've probably guessed, I'm running an external provisioning script for the PG setup. The odd thing is I'm using the script recommended from Postgres' own site here.
In a separate location, I've git cloned that repo and followed the instructions and it works absolutely fine, creating a properly provisioned VM with PG installed.
However, I want to run a single vagrant up command and provisioning both the app and db correctly and have them speak to each other.
I'm (quite clearly) new to provisioning and DevOps as a whole, so would really appreciate some help.
I've uploaded my hilariously broken code here for you kind souls to look over if you feel so inclined.
Vagrant documentation on Multi-machines is quite thin and Google isn't being much help
Thanks!

How to use aliases in Vagrant

I'm trying to create aliases that I can use in Vagrant any time I run the VM. I've found several sources on the web about it, but can't get it working. I tried making a .bash_profile in my synced folder, but that didn't work. I noticed if I run the command alias name="command" this will work, but only for the current session. Anyone know how to do this? I'm using macOS. Thanks for your help!
Here is my Vagrantfile:
# -*- mode: ruby -*-
# vi: set ft=ruby :
unless Vagrant.has_plugin?("vagrant-vbguest")
warn "\nWARNING: The vagrant-vbguest plugin should be installed or your shared folders might not mount properly!"
warn "You can do this by running the command 'vagrant plugin install vagrant-vbguest'.\n\n"
end
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "pype_vm"
config.vm.box_url = "https://.../pype_vm.json"
config.vm.network "private_network", ip: ""
config.vm.boot_timeout = 600
config.vm.provider "virtualbox" do |v|
# This forces VirtualBox to use the host's DNS resolver instead of
# VirtualBox's
v.customize ["modifyvm", :id, "", "on"]
# This enables the PAE/NX option, which resolved at least one user's
# issues with the VM hanging on boot
v.customize ["modifyvm", :id, "--pae", "on"]
# The RHEL VM was created with 2GB of memory to facilitate provisioning,
# but this is causing issues with certain workstations. This reduces
# the amount of memory allocated to the VM but should not impact development
# performance. The number is in MB and can be increased if desired.
v.memory = 1024
end
# Share an additional folder to the guest VM.
config.vm.synced_folder File.dirname(__FILE__), "/pype"
end
The details depend on the specific of the guest being run, but some notes:
Assuming the default user account is active for vagrant ssh, ensure that any dotfiles you wish to override are copied to /home/vagrant.
If overriding .bashrc, ensure that the remote shell is started with the interactive flag (if this is true, echo $- will include i).
If overriding .bash_profile, ensure that the remote shell is started as a login shell (if this is true, echo $- will include l).

Vagrant: Best method for Multiple Identical VMs

I would like to provision multiple VMs with Vagrant. I would like to be able to increase the number of VMs quickly and easily.
Based on my current understanding of Vagrant. The following method detailed here https://www.edureka.co/blog/10-steps-to-create-multiple-vms-using-vagrant/ seems to be the best way:
# This defines the version of vagrant
Vagrant.configure(2) do |config|
# Specifying the box we wish to use
config.vm.box = "chef/centos-6.5"
# Adding Bridged Network Adapter
config.vm.network "public_network"
# Iterating the loop for three times
(1..3).each do |i|
# Defining VM properties
config.vm.define "edureka_vm#{i}" do |node|
# Specifying the provider as VirtualBox and naming the VM's
config.vm.provider "virtualbox" do |node|
# The VM will be named as edureka_vm{i}
node.name = "edureka_vm#{i}"
end
end
end
end
So if I want to increase from 3 to 4, I just change the loop range.
Is there another method? When I was reading the docs I was hoping there would be some version of the vagrant up command that would spin up a new unique instance.
Environment Variables of Vagrant is your best bet. Define something like this in your Vagrantfile.
myVMs = ENV['NODES'] || 1
Where NODES is an environment variable. You can use this in the following way.
NODES=4 vagrant up
This will update the myVMs variable in your Vagrantfile and spawn 4 VMs. You do not need to open your Vagrantfile and update it. But note, this will be for this session alone.If you want to save the value of the NODES, then you might have to add it to your ~/.profile in Linux or environment variables in windows.
Hope this helps.

Vagrant workflow

Hi i was trying to understand what exactly is done when running
vagrant up
my reason for that is that in my case we need to install a lot of utilities.
i.e Version control tools, build tools, ide, etc...
which takes a lot of time.
so actually i wanted a 'box' with all those tools.
After i have clean environment and got all tools, i would like to make CI for our product.
If i will reinstall all utilities it should take a lot of time. so what i am actually need is just installing and testing our product.
How should i handle that ??
create my own box? does the command reinstall all utilities when we make CI ??
what i actually need are 2 processes :
1.installing utilities for my vm. (once a month)
2.test our product (each commit\push to version control)
how can i achieve that ?
For the first time, vagrant up will create a new VM for you, pulling the box image if needed, and it will provision it with what you configured in the Vagrantfile. In the provision configuration, you can tell Chef or Puppet to install all the utilities and tools that you need.
When you suspend or halt the VM, the next time you do a vagrant up it will only bring that VM back up. It will not install or try to provision it again.
You can force it with vagrant up --provision or just vagrant provision.
This usually works well in a development environment.
In a CI environment, it may not be possible to have the VM already provision, forcing you to run the provisioning step every time. You can achieve what you need packaging your own box with the tools already installed, essentially creating a golden or base image.
Just be extra careful so that the CI environment don't differ for what you have in production.
All depend on the setting in Vagrantfile
you have modules folder to put all puppet modules, manifests folder with site.pp and Vagrantfile as below under same place.
Give you a sample of Vagrantfile I used mostly.
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "precise64"
config.vm.box_url = "https://cloud-images.ubuntu.com/vagrant/precise/current/precise-server-cloudimg-amd64-vagrant-disk1.box "
config.vm.provision :puppet do |puppet|
puppet.module_path = "modules"
puppet.manifests_path = "manifests"
puppet.manifest_file = "site.pp"
end
config.vm.define :www do |config|
config.vm.host_name = "www.example.com"
config.vm.network :private_network, ip: "192.168.1.2"
end
config.vm.define :db1 do |config|
config.vm.host_name = "db1.example.com"
config.vm.network :private_network, ip: "192.168.1.4"
end
end
So after run vagrant up www or vagrant up db1 it will start the box with puppet apply directly. You can understand this way as puppet masterless
Vagrant just make the box up running from linux image defined in config.vm.box_url as a simple fresh linux box, and mount your local folder to new box's /vagrant folder, then hand over to puppet. How the server to be provisioned, will be depended on site.pp (define applications on each node) and puppet modules. The command is similar as
puppet apply --modulepath /vagrant/module /vagrant/manifests/site.pp
So if your puppet modules are fine, your new box will automatically have all utilities and products installed. Then you run vagrant ssh www or vagrant ssh db1, you can login it and start working.
You can put your local folder with moduels, manifests folders and Vagrantfile to version control (such as git). So developers can clone the git repository to their own computer easily.

vagrant primary box defined but commands still run against all boxes

I am trying to set my "dev" VM as primary so most commands such as vagrant up, vagrant halt, etc operate on the "dev" VM and ignore the "stage" VM unless the "stage" VM name is explicitly listed on the command line. Here's my Vagrantfile, but when I run vagrant up both VMs are brought up instead of just "dev".
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"
def box_setup(config, ip, playbook, inventory)
config.vm.network :private_network, ip: ip
config.vm.provision "ansible" do |ansible|
ansible.playbook = playbook
ansible.inventory_path = inventory
# ansible.verbose = "vvv"
end
end
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# common settings shared by all vagrant boxes for this project
config.vm.box = "debian-7-amd64"
config.vm.box_url = "https://dl.dropboxusercontent.com/s/xymcvez85i29lym/vagrant-debian-wheezy64.box"
config.ssh.forward_agent = true
# development box intended for ongoing development
config.vm.define "dev", primary: true do |dev|
box_setup dev, \
"10.9.8.30", "deploy/playbook_dev.yml", "deploy/hosts/vagrant_dev.yml"
end
# stage box intended for configuration closely matching production
config.vm.define "stage" do |stage|
box_setup stage, \
"10.9.8.31", "deploy/playbook_full_stack.yml", "deploy/hosts/vagrant_stage.yml"
end
end
The primary flag seems to only work for vagrant ssh for me.
In the past I have used the following method to hack around the issue.
# stage box intended for configuration closely matching production
if ARGV[1] == 'stage'
config.vm.define "stage" do |stage|
box_setup stage, \
"10.9.8.31", "deploy/playbook_full_stack.yml", "deploy/hosts/vagrant_stage.yml"
end
end

Resources