How do I provision a personal database during first vagrant up? - vagrant

We've got a multi-machine Vagrantfile in our project repo which sets up a LAMP stack and creates some common MySQL databases. Now, each of us also have a personal database that we'd like Vagrant to create as part of the initial "vagrant up" provisioning. I've placed my personal provisioning code in ~/.vagrant.d/Vagrantfile, but according to the load order of Vagrantfiles, that code runs before the provisioning code of the common Vagrantfile. My personal provisioning code is unable to add any MySQL database because MySQL simply hasn't been installed yet, as that is the responsibility of the common Vagrantfile.
How can I have it so that the common Vagrantfile installs MySQL and the personal Vagrantfile adds a database once MySQL is in place?
Update
Here is what I ended up doing.
Vagrantfile (shared with other developers):
Vagrant.configure("2") do |config|
config.vm.define "dev14" do |dev14|
dev14.vm.box = "ubuntu/trusty64"
dev14.vm.provision "file", source: "common.sql", destination: "common.sql"
dev14.vm.provision "shell", inline: <<-SHELL
sudo apt-get update
sudo apt-get install -y apache2
sudo apt-get install -y php5
sudo apt-get install -y mysql-server
sudo apt-get install -y mysql-client
mysql -uroot -proot -e "CREATE DATABASE IF NOT EXISTS common"
mysql -uroot -proot common < common.sql
SHELL
# User specific provisioning
$dev14 = dev14
load './provision.dev14.rb' if File.exists?('./provision.dev14.rb')
end
end
provision.dev14.rb (my personal provisioning file, Git ignored):
$dev14.vm.provision "file", source: "personal.sql", destination: "personal.sql"
$dev14.vm.provision "shell", inline: <<-SHELL
mysql -uroot -proot -e "CREATE DATABASE IF NOT EXISTS personal"
mysql -uroot -proot personal < personal.sql
SHELL
These are of course sanitised and simplified excerpts.

We found the easiest solution for this was to specify multiple provision scripts and have your 'create the database' Vagrant file script detect whether or not it needs to do anything.
Also, can you parameterise the personalization stage e.g. base database creation on ENV['DB_USERNAME'] etc.? This would allow you to benefit from trying each other's setup.
Beyond this, you can write a plugin to have finer control over which Vagrant action is running and hook in to pre/post action events.

Related

The safety of running vagrant images

I have external vagrant running scripts - is it safe to run something like that locally (without risking doing anything externally in production)? I've been using vagrant a long time ago, but it's supposed to be used locally in a dev environment, right?
I have the following Vagrantfile:
config.vm.define "vmmachine" do |vmmachine|
vmmachine.vm.hostname = "machine-dev"
vmmachine.vm.network :private_network, ip: "192.168.10.12"
vmmachine.vm.provision :shell, :path => "vagrant/machine/init.sh"
end
vagrant/machine/init.sh is:
#!/usr/bin/env bash
export DEBIAN_FRONTEND=noninteractive
# Add repo
echo "deb http://www.somewebsite.com testing main" > /etc/apt/sources.list.d/machine.list
# Add key
wget --quiet -O - https://www.somewebsite.com.key.asc | apt-key add -
apt-get update
apt-get install -q -y screen htop vim curl wget
apt-get install -q -y machine-server
service machine start
apt-get clean
Also, why is it 192.168.x.x instead of 127.0.0.x? What is the benefit of that?
Yes Vagrant can definitely be used in local development. I'm not entirely sure what scripts you're trying to run but it looks to me like running it locally would definitely be an option. I used Vagrant as my local dev server for awhile and know of quite a few other devs who do so currently.
The reason Vagrant doesn't use 127.0.0.x is because your site is running in a VM on your computer and not locally on the computer itself. You basically connect to the VM through a local IP address Vagrant gives a small explanation about the IP address in their docs:
While you can choose any IP you would like, you should use an IP from the reserved private address space. These IPs are guaranteed to never be publicly routable, and most routers actually block traffic from going to them from the outside world.
So I would say they "officially" recommend using 192.168.x.x though they say you can use pretty much any IP you want.

Reload VM in Vagrant script

Introduction
I'm using Vagrant and want to create a box that fits my needs. I'm currently building my provisioning script but I have a problem which would require me to relog into the box.
What I'm trying to achieve
I want to set my locales to the German language
What I'm doing
After logging into my vagrant box with vagrant ssh I'm running the following commands
sudo apt-get update
sudo cp /var/www/projectfantasy/www/vagrant_ressources/locale.gen /etc/
sudo locale-gen de_DE.UTF-8
These steps are done with the help of Debian's wiki. The last step is
To use the new settings with your programs, log out and back in.
And now I am where I need help. How would I relog while being in the vagrant provisioning script? When I don't relog I'm getting the following warnings when installing extra packages.
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = "de_DE.UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
I mean, these are only warnings so that shouldn't be a problem, right? But I don't want to have warnings and would like to know how I could fix this issue.
When I relog and try to install those packages again it works without any problems.
unfortunately, this is not so simple - if you issue a reboot command or shutdown -r now in your provisioning script the VM will restart (indeed) but then the provisioning will not continue.
fortunately, some people wrote plugin (https://github.com/exratione/vagrant-provision-reboot and https://github.com/aidanns/vagrant-reload) I experienced with the second and it works
make sure to install the plugin
$ vagrant plugin install vagrant-reload
in your vagrantfile, your provisioning will look like
config.vm.provision "shell", path: "vagrant_ressources/preparations.sh"
config.vm.provision :reload
config.vm.provision "shell", path: "vagrant_ressources/bootstrap.sh"
so preparations.sh will execute, then the VM will reload and bootstrap.sh will execute
in your preparations.sh, after you generate locale, make sure variables are set in the locale file :
sudo locale-gen
echo -e 'LANG=de_DE.UTF-8\nLC_ALL=de_DE.UTF-8' > /etc/default/locale
sudo timedatectl set-timezone Europe/Berlin
PS : one note on your usage of the shell provisioning. By default provisioning is run with root privilege so there is no need for all the sudo ; if you want to run the provisioning as vagrant user run it as config.vm.provision :shell, privileged: false ... and you will need the sudo
I couldn't reproduce your warning, but I find this will allow you to set LC_ALL:
sudo update-locale LC_ALL=de_DE.UTF-8
(from https://askubuntu.com/questions/114759/warning-setlocale-lc-all-cannot-change-locale, but works for debian jessie as well)

Create an apt-cache on host for multiple vagrant installations

When you startup a vagrant box using a bootstrap.sh, you have to download all packages with apt-get inside each box again.
How can I set up an external folder on my host, that is used for all boxes as cache?
Maybe somehow adding the apt-cache folder in the Vagrantfile lilke this?
config.vm.synced_folder "/var/cache/apt/archives/", "/var/cache/apt/archives/"
You cannot mount the same folder as on your host mashine, but you can create a special folder for your vagrant boxes.
Add these lines in my Vagrantfile:
# add the host apt cache so packages don't have to be downloaded each time
# you need to create this before you start vagrant up:
#sudo mkdir -p /var/tmp/vagrant/apt-archives/
#sudo mkdir -p /var/tmp/vagrant/apt-lists/
#sudo chmod 777 /var/tmp/vagrant/apt-archives/ /var/tmp/vagrant/apt-lists/
config.vm.synced_folder "/var/tmp/vagrant/apt-archives/", "/var/cache/apt/archives/", type: "nfs"
config.vm.synced_folder "/var/tmp/vagrant/apt-lists/", "/var/lib/apt/lists", type: "nfs"
you must create those folders before the first run of vagrant up yourself. Then everything works fine without having to download anything you already have again.
Note: I am not sure, but it could be, that this gets conflicts, if you use different distributions in different vagrant mashines

Using Vagrant on cloud CI services

Are there any cloud CI services that allow Vagrant VMs to run using VirtualBox as a provider?
Early investigation shows this seems not to be possible with Travis CI or Circle CI, although the vagrant-aws plugin allows for the use of AWS servers as a Vagrant provider. Is this correct?
Update January 2021: GitHub Actions also supports Vagrant - and Vagrant/VirtualBox are both installed out-of-the-box in the MacOS environment (not on Linux or Windows currently!). See the possible environments here. Therefore I created a fully comprehensible example project at: https://github.com/jonashackt/vagrant-github-actions
1.: Create a Vagrantfile (and you're not limited to libvirt as with Travis, you have a full VirtualBox environment with nested virtualization working on GitHub Actions!) like this:
Vagrant.configure("2") do |config|
config.vm.box = "generic/ubuntu1804"
config.vm.define 'ubuntu'
# Prevent SharedFoldersEnableSymlinksCreate errors
config.vm.synced_folder ".", "/vagrant", disabled: true
end
2.: Create a GitHub Actions workflow like this vagrant-up.yml inside the .github/workflows directory in your repository:
name: vagrant-up
on: [push]
jobs:
vagrant-up:
runs-on: macos-10.15
steps:
- uses: actions/checkout#v2
- name: Run vagrant up
run: vagrant up
- name: ssh into box after boot
run: vagrant ssh -c "echo 'hello world!'"
You can even add caching for the Vagran boxes, this will safe you some seconds :)
Early 2020:
TravisCI is now able to run Vagrant finally! Thanks to this GitHub issue I learned about libvirt and KVM, which could be used together with the vagrant-libvirt Plugin to run Vagrant boxes on TravisCI.
An example TravisCI .travis.yml should look somehow like that:
---
dist: bionic
language: python
install:
# Install libvrt & KVM
- sudo apt-get update && sudo apt-get install -y bridge-utils dnsmasq-base ebtables libvirt-bin libvirt-dev qemu-kvm qemu-utils ruby-dev
# Download Vagrant & Install Vagrant package
- sudo wget -nv https://releases.hashicorp.com/vagrant/2.2.7/vagrant_2.2.7_x86_64.deb
- sudo dpkg -i vagrant_2.2.7_x86_64.deb
# Vagrant correctly installed?
- vagrant --version
# Install vagrant-libvirt Vagrant plugin
- sudo vagrant plugin install vagrant-libvirt
script:
- sudo vagrant up --provider=libvirt
- sudo vagrant ssh -c "echo 'hello world!'"
With the help of the generic Vagrant Box images from Vagrant Cloud you can also establish a workflow of using Vagrant + libvirt + KVM on Travis and Vagrant + VirtualBox on your local machine, if you like:
I created a fully working and 100% comprehensible example project here: https://github.com/jonashackt/vagrant-travisci-libvrt
Many CI services are not allowing to run Vagrant via LXC or Virtualbox as it will require nested virtualization (running VM in VM) or a pure bare metal server provisioned for you.
Current 2021 (updated) situation:
Github Actions can do it.
Travis was able to run Vagrant with some workarounds.
AppVeyor allows running VirtualBox (non-free plans).
You can't under CodeShip
You can't under CircleCI
Don't know about other CI services, will investigate further.
I hope during the time we'll see CI services allowing to run Vagrant with Virtualbox or LXC, but for now Docker (with its limitations) is the only option.
Personally, I would be happy to use it for integration tests against different platforms/linux distros via Test-Kitchen CI or similar.
AppVeyor runs Vagrant using VirtualBox as a provider. Furthermore, you can use other providers like libvirt or Hyper-v.
Updated:
Github Actions is another option. Only Mac OS environment has nested virtualization enabled. An example is here and here.

Understanding vagrant with a docker provisioner

So my goal here is to have Vagrant spin me up a VM that I can use as a rails web application development environment. I am very, very new to Vagrant, Docker, etc. and wanted to check if this even makes sense:
Should I be creating one Docker image, or two? My thought was that one image would be for rvm, ruby, and rails, and the other image would be for Apache.
The goal would be something that could be distributed to a few people, who would then get the exact same environment set up easily.
Here is my current Vagrantfile:
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "phusion/ubuntu-12.04-amd64"
config.vm.network "forwarded_port", :guest => 80, :host => 5000
# Share the docker configs
config.vm.synced_folder "./docker", "/docker"
config.vm.provision "docker" do |d|
d.build_image "-t me/rvm-dev /docker/rvm-dev"
d.run "me/rvm-dev"
end
config.vm.host_name = "coursera-rail.dev"
config.vm.network :forwarded_port, guest:80, host:8080
end
And here is my current (probably wrong) Dockerfile
(as an aside, I am trying to find some good guidance on creating a "docker" user account so I don't run everything as root)
## Dockerfile
FROM ubuntu:latest
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y curl
RUN mkdir -p /tmp/downloads
## Install RVM, Ruby, Rails
RUN cd /tmp/downloads
RUN \curl -L https://get.rvm.io | bash -s stable --ruby --rails
Lastly, the above works, and if I use vagrant ssh I can head in to my VM, but obviously I am unable to use RVM without doing docker run -i -t me/rvm-dev and getting a shell with docker. Is this supposed to be the workflow?
In my humble opinion, the best flow would be to
1) separate your app and db stack into vagrant boxes
or
2) run docker on the host (the physical machine) and your DB instance (mysql, etc..) inside a vagrant box.
Option one is best because you are "sandboxing" both layers of the stack (DB and APP) inside Vagrant thus leaving the OS on your local machine (windows, Linux, Mac OSX) instact. Of course, you could install/run your IDE on the physical machine.

Resources