I just ran vagrant provision in a futile attempt at getting my customized synced_folders directive to work and now my whole guest box is wiped out.
Is this normal? I don't see any references to Vagrant docs about this behavior.
As per the doc:
Provisioners in Vagrant allow you to automatically install software, alter configurations, and more on the machine as part of the vagrant up process.
The only thing I have in my config provision shell commands are installation commands. Nothing about wiping anything out.
I do have app.vm.provision for puppet that sets fqdn, user name and box name (along with the normal module_path, manifests_path and manifests_file). Maybe this caused things to be reset?
The Answer
Is Vagrant Provision suppose to wipe out all your data?
No. Vagrant should never harm your "data" (i.e., websites, code, etc.).
...now my whole guest box is wiped out. Is this normal?
Yes. Your Vagrant environment (in other words, the guest operating system created in a virtual environment by Vagrant) is volatile, and you should be able to destroy and recreate it at will without having any impact on your working files (because those should be kept in your local, or host, file system).
Explanation
On Vagrant's website, the very first thing they tell you is this:
Create and configure lightweight, reproducible, and portable development environments.
Your development environment allows you to work. You work on your data, in your development work environment. When you are done with your "development work environment," you should be able to delete it freely without affecting your data in the least.
Further, you should be able to send a collaborating developer your Vagrantfile so that they can create the exact same development environment you used to create your data (i.e., write your program, build your website, and so forth). Then, when you provide them with your code, they can use it in an environment identical to the one that your code was created in without having to reconfigure their own setup.
For more details about how your data files (code, working files, etc.) are kept safely in your computer while making them accessible to your guest system created by Vagrant, see my answer to this question.
So what appears to have happened was that when I set up a synced folder, it wiped out everything because there was nothing on my host machine in that synced folder. Unless there is a way to recover the lost data, there should be an unmistakable WARNING in their docs that this can happen.
I setup the synced_folder to be on my whole home directory. When I created a new machine, I cloned the one project I had saved and decided to just sync my individual projects instead of my whole user directory this time. When I reloaded, the project directory was empty since it was empty on my host machine.
So I guess, make sure the directories on your host machine are already setup with the data before configuring your Vagrantfile with synced_folder information.
Related
I have tried installing latest versions of VirtualBox and Vagrant, but I keep receiving the error message below when trying to run vagrant. How can I find the problem and correct it?
No usable default provider could be found for your system.
Vagrant relies on interactions with 3rd party systems, known as
"providers", to provide Vagrant with resources to run development
environments. Examples are VirtualBox, VMware, Hyper-V.
The easiest solution to this message is to install VirtualBox, which
is available for free on all major platforms.
If you believe you already have a provider available, make sure it
is properly installed and configured. You can see more details about
why a particular provider isn't working by forcing usage with
`vagrant up --provider=PROVIDER`, which should give you a more specific
error message for that particular provider.
Be sure to install VirtualBox first, since Vagrant depends on it.
-Download and install VirtualBox
-Download and install Vagrant
-Adjust VirtualBox and Vagrant
The next step is to make some tweaks to VirtualBox and Vagrant. We need to do this because, by default, both tools store data in the same drive where they were installed (tipically the C drive). The thing with virtual machines is they can take up a lot of space, so they can eat up your main drive’s storage very quickly.
First we’ll change the storage path in Virtual Box, since it’s pretty straightforward. You now should have a program called Oracle VM VirtualBox installed. Open it and then go to File > Preferences. A
new dialog will open where you can change the folder next to the option that reads Default Machine Folder. Click the dropdown and choose Other…. Here you can choose the new folder for VirtualBox to store data.
Now we’ll do the same for Vagrant.
By default the path where data related to virtual machines is
stored is C:\Users\YourUser\.vagrant.d so I recommend changing it to a different drive. This can be done with environment variables. We need to create a variable called VAGRANT_HOME and point it to our desired location.
Open the control panel and search for the word environment. From the results choose the one that says edit environment variables for your account.
On the new dialog that appears, click the New… button.
Here you will create the new variable. Enter VAGRANT_HOME as the value for the field named Variable name.
For the field named Variable value enter the path of your choice. In my case I used C:\VM\Vagrant
I currently have a Vagrant vm with a typical web setup (apache, php, etc). The actual web repo is checked out onto the my local machine, and accessed through synced folders and forwarded apache ports (like http://127.0.0.1:4567/web). The question is, how do I make this work for a team, as opposed to just me? Since you have to sync folders, how is it possible for multiple devs to connect to one VM and sync folders?
I initially thought sharing the Vagrantfile would be all that is needed, but the Vagrantfile is agnostic to all apt-get commands and apache/php/node etc setup. I don't want devs to have to worry about running those.
This is what Vagrant provisioning is for.
Vagrant allows you to automatically alter configuration of a box or install additional software packages as a part of the first vagrant up process. Moreover, you can provision your system with various configuration management systems: Chef, Puppet, Ansible, CFEngine. Or you can simply use shell scripts.
Sitepoint offers a good tutorial on provisioning Ubuntu 14.04 box with bash scripts to install nginx, PHP-FPM and MySQL. Read it. Later you might want to move to configuration management systems.
Take a look at Phansible project which generates Ansible provisionings for PHP-based projects. Puppet provisioning for PHP-projects can be generated via PuPHPet. They will give you an idea how to use configuration management with Vagrant and might be used as s template for your Vagrantfile.
I always try to craft my Vagrantfiles carefully (provisioning, synced folder, for example, maps host's ./src/public onto /var/www/html/ on guest machine, port forwards and so on) so I can put them into project's root under version control. Later when my teammate clones project's repo on his machine he can issue vagrant up right away and get a fully functional development environment.
The only problem I haven't solved yet is updating existing VMs when new dependencies are added to projects. Now we update provisioning scripts and if developers encounter errors they manually reprovision their VMs by using vagrant reload --provision.
P.S. I asked a question regarding this problem
I have a Vagrant set up with 3 virtual machines. Each machine has its own shell script for provisioning.
Now I would like to share the exact same status of my set up with somebody else. Since the provisioning procedure takes really (!!) long for each machine, I hope there is another solution.
Ideally I would be able to save each machine as it is in one file, which the other person then could import into Virtualbox. Is there a way to do that?
If I understand you correctly you would like to make a Vagrant base box from provisioned by Vagrant VMs. This is not recommended way to go. How you can approach this is:
Create new VM manually with required OS in the VBox.
Adjust it so Vagrant can connect to it as described here and here.
Provision it using your shell scripts.
Install all the things you would find useful to have on this VM.
Use Vagrant to package it as a base box as described here.
After packaging it with Vagrant you will get a Vagrant base box file with .box extension. You can then pass this to your team mates (usb, network share, ftp etc.) and they can add it to their Vagrant installation and use it. Whenever they will do Vagrant up they will get fully provisioned VM in VBox with all the stuff you have packaged to it. Vagrant also gives you versioning capabilities. If properly configured whenever you will create new version of base box everybody who is using it will be notified and would be able to download and use new version of your box.
Hope I understood your problem correctly and this will help to solve it.
Ok. I am playing around with different tools to prepare dev environment. Docker is nice option. I created the whole dev environment in docker and can build a project in it.
The source code for this project lives outside of docker container (on the host). This way you can use your IDE to edit it and use docker just to build it.
However, there is one problem
a) Docker on OS X uses VM (VirtualBox VM)
b) File sharing is reasonably slow (way slower than file IO on host)
c) The project has something like a gazzilion files (which exaggerate problems #a an #b).
If I move source code in the docker, I will have the same problem in IDE (it will have to access shared files and it will be slow).
I heard about some workaround to make it fast. However, I can't seem to find any information on this subject.
Update 1
I used Docker file sharing feature (meaning I run)
docker run -P -i -v <VMDIR>:<DOCKERDIR> -t <imageName> /bin/bash
However, sharing between VM and Docker isn't a problem. It's fast.
The bottle neck is sharing between host and VM.
The workaround I use is not to use boot2docker but instead have a vagrant VM provisioned with docker. No such big penalty for mounting folders host->vagrant->docker.
On the downside, I have to pre-map folders to vagrant (basically my whole work directory) and pre-expose a range of ports from the vagrant box to the host to have access to the docker services directly from there.
On the plus side, when I want to clean unused docker garbage (images, volumes, etc.) I simply destroy the vagrant vm and re-create it again :)
Elaboration
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "trusty-docker"
config.vm.box_url = "https://oss-binaries.phusionpassenger.com/vagrant/boxes/latest/ubuntu-14.04-amd64-vbox.box"
config.vm.provision "docker"
#by default we'll claim ports 9080-9090 on the host system
for i in 9080..9090
config.vm.network :forwarded_port, guest: i, host: i
end
#NB: this folder mapping will not have the boot2docker issue of slow sync
config.vm.synced_folder "~/work", "/home/vagrant/work"
end
Having that:
host$ vagrant up && vagrant ssh
vagrant$ docker run -it --rm -v $(pwd)/work:/work ubuntu:12.04 find /work
This is unfortunately a typical problem Windows and OS X users are currently struggling with that cannot be solved trivially, especially in the case of Windows users. The main culprit is VirtualBox's vboxfs which is used for file sharing which, despite being incredibly useful, results in poor filesystem I/O.
There are numerous situations by which developing the project sources inside the guest VM are brought to a crawl, the main two being scores of 3rd party sources introduce by package managers and Git repositories with a sizable history.
The obvious approach is to move as much of the project-related files outside of vboxfs somewhere else into the guest. For instance, symlinking the package manager directory into the project's vboxfs tree, with something like:
mkdir /var/cache/node_modules && ln -s /var/cache/node_modules /myproject/node_modules
This alone improved the startup time from ~28 seconds down to ~4 seconds for a Node.js application with a few dozen dependencies running on my SSD.
Unfortunately, this is not applicable to managing Git repositories, short of splatting/truncating your history and committing to data loss, unless the Git repository itself is provisioned within the guest, which forces you to have two repositories: one to clone the environment for inflating the guest and another containing the actual sources, where consolidating the two worlds becomes an absolute pain.
The best way to approach the situation is to either:
drop vboxfs in favor of a shared transport mechanism that results in better I/O in the guest, such as the Network File System. Unfortunately, for Windows users, the only way to get NFS service support is to run the enterprise edition of Windows (which I believe will still be true for Windows 10).
revert to mounting raw disk partitions into the guest, noting the related risks of giving your hypervisor raw disk access
If your developer audience is wholly compromised of Linux and OS X users, option 1 might be viable. Create a Vagrant machine and configure NFS shares between your host and guest and profit. If you do have Windows users, then, short of buying them an enterprise license, it would be best to simply ask them to repartition their disks and work inside a guest VM.
I personally use a Windows host and have a 64 GB partition on my SSD that I mount directly into my Arch Linux guest and operate from there. I also switched to GPT and UEFI and have an option to boot directly into Arch Linux in case I want to circumvent the overhead of the virtualized hardware, giving me the best of both worlds with little compromise.
Two steps can improve your performance quite well:
Switch to NFS. You can use this script to do this.
Switch your VBox NIC driver to FAST III. You can do this on your default machine by running:
VBoxManage modifyvm default --nictype1 Am79C973
VBoxManage modifyvm default --nictype2 Am79C973
I run a simple watch script that kicks off an rsync inside my container(s) from the shared source-code volume to a container-only volume whenever anything changes. My entry-point only reads from the container-only volume so that avoids the performance issues, and rsync works really well, especially if you set it up correctly to avoid things like .git folders and libraries that don't change frequently.
Couple of pieces of info which I found
I started to use Vagrant to work with VirtualBox and install Docker in it. It gives more flexibility
Default sharing in Vagrant VirtualBox provisioning is very slow
NFS sharing is much faster. However, it could be reasonably slow it (especially if your build process will create files which needs to be written back to this share).
Vagrant 1.5+ have a rsync option (to use rsync to copy files from host to VM). It's faster because it doesn't have to write back any changes.
This rsync option has autosync (to continiously sync it).
This rsync option consumes a lot of CPU and people came up with a gem to overcome it (https://github.com/smerrill/vagrant-gatling-rsync)
So, Vagrant + VirtualBox + Rsync shared folder + auto rsync + vagrant gatling looks like a good option for my case (still researching it).
I tried vagrant gatling. However, it results in non deterministic behavior. I never know whether new files were copied into VM or not. It wouldn't be a problem if it would take 1 second. However, it make take 20 seconds which is too much (a user can switch a window and start build when new files weren't synced yet).
Now, I am thinking about some way to copy over ONLY files which changed. I am still in research phase.The idea would be to use FSEvent to listen for file changes and send over only changed files. It looks like there are some tools around which do that.
BTW. Gatling internally using FSEvent. The only problem that it triggers full rsync (which goes and start comparing date/times and sizes for 40k files)
I mounted a vagrant machine several time ago.I have modified some configurations to the VM since the installation and now my co-workers need to use exactly the same VM.
How can I do it ? I would like keep the database i used for my co-workers too if it's possible.
Look into VagrantCloud: https://vagrantcloud.com/
I'm using that service to host a customized box for my team. Started from a vanilla box, customized it to match our environment, then used the "vagrant package" command to create a new *.box file. You just need to host that file somewhere online, register it with the VagrantCloud service, and you can manage box releases and deployments for your co-workers. It even notifies them if you update the box. Pretty nifty.
As far as the DB, if it is in the VM it will get packaged up too. Might wat to look into providing updated DB files for later since they will get out of date fast, I would imagine.