I'm trying to automate our CI process for a couple of .NET apps, and in a perfect world I'd like to spin up a Windows EC2 instance for each, bootstrap the instance to install Chef Solo and then execute a Chef recipe to install some dependencies and the packaged software itself.
However - I'm a novice and have no idea even if that is feasible let alone where to start :)
I'm fairly well versed with the command line tools for AWS so can spin up an AMI ok, but beyond that point I'm pretty stuck. I would like to avoid building a custom AMI with chef pre-installed as that takes a lot of the advantages away.
I think this is essentially what I need to do - but is (unsurprisingly) focused on Linux:
http://www.opinionatedprogrammer.com/2011/06/chef-solo-tutorial-managing-a-single-server-with-chef/
Does anyone have a link to someone who has done this or similar before? Or a better way of achieving what I'd like to do?
Any help appreciated.
Okay, this requires that you have Chef preinstalled on your AMI:
http://scottwb.com/blog/2012/12/13/provision-and-bootstrap-windows-ec2-instances-with-chef/
But this is a strategy for installing Puppet to a stock Windows AMI, which could easily be modified for Chef:
http://dansrandombits.blogspot.com/2012/06/bootstrapping-custom-windows-ec2.html
I can't say I've done this yet, but I've both in my bookmarks bar since they was posted and have been planning on giving it a shot in at least our dev environment at some point. It seems like as long as there's a solid silent install for Chef, you could pull this off.
I realize this post is a bit old, but for those that still may come across this. I'm provisioning servers using Chef-Solo. Essentially I configure the User-Data of the instance to download and install Chef, download the cookbooks/recipes, and then launch Chef-Solo.
Here's a blog post I've made to demonstrate the steps: http://thesysadminswatercooler.blogspot.com/2015/11/aws-bootstrap-windows-ec2-instance-with.html
Related
Using Vagrant to have a number of developers share the same environment I see two approaches that people are using.
Create a box file with entire basic environment installed, and share it on Vagrant cloud?
Create a setup script that will "apt-get install" and "git clone" everything that is needed?
With 1st approach I don't like the fact that you have to host large files, also changes to environment would be harder to do (uploading the subsequent version of the entire box?).
Its difficult to make an answer for this, I can only share my experience, I started with #1 and provided a full box to my team, the box was 13 GB but it worked fine (although was not too easy to share), then when we had to upgrade any 3rd party software its was more difficult (it was also before the time vagrant update was available) but still I like option #2 much more flexible now. The provisioning takes 2-3 hours but we can run it at night, repeat the process anytime and at the end, its much easier to provision a new instance for a new member.
I'm trying to automate the provisioning of our developer workspaces which are based on Linux running in a Virtualbox. We also run the IDE inside the Virtualbox. The host system is Windows 7.
The new solution will install the required dependencies (C libraries) and install + pre-configure the IDE (plugins, code formater).
I'm wondering if Vagrant would be the tool of choice for this task. I am irritated since the Vagrant philosophy seems to be: easy provisioning of a dev/test server but running the IDE on the host - not inside the guest. The latter is not an option for us.
What tooling would you recommend for our purposes?
This is the answer to my question. I eventually went for Vagrant.
This simple switch let the normally hidden Virtualbox GUI appear:
v.gui = true
More details here: https://docs.vagrantup.com/v2/virtualbox/configuration.html
My experience with the solution was generally ok. I got my recipes version controlled as wanted. However, the feedback loop was quite long. To build the box with all my customization it took me about 7 minutes which was usually wasted waiting time. When changing a little bit I could try to execute just the changed scripts but to see the whole thing I had to rebuild the box - another 7 minutes wasted waiting.
I am currently trying to understand how in the world of infrastructure as code you still get acceptable feedback loop (let's say below one minute). I assume this is where you enter the world of configuration management with tools like Ansible or Puppet.
I have a network with 10+ Windows boxes. I need to be able to automatically reconfigure them all in the certain way:
Install Java SE Development Kit 7u79
Set up startup script .bat on all clients (Win boxes).
Set up script .bat to perform each 6 hours or at 03:00 every day.
Install FF v.30-40 in folder C:/firefox/official/ff-v.X, where X is the version
I have Ubuntu 14.04 as a server: 192.168.122.100. Win boxes are 192.168.122.150-160. All required exe, msi will be located at 192.168.122.100/files/win/
The question: Is opscode chef the right solution to achieve the result or should I consider other ways?
PS: I don't have an experience as administrator and I don't have time to investigate all of the approaches myself. Currently I'm working to set up this system myself, but I still not sure if this is right investment of time in my case.
Well, Chef can do all of the task you've listed above.
For installing JDK 7 just use windows_package resource (important: remember to download cookbook windows from Chef Supermarket. It looks like this:
windows_package name do
source src
action :install
installer_type :custom
end
And remember, that - on Windows systems - variable name should be actual name of package. You can check names in Control Panel (for example, JDK 7.79 has name Java SE Development Kit 7 Update 79).
Firefox can be installed in the same manner.
Setting batch scripts can be done in multiple ways - creating them with template resource and just execute them with another resource (powershell_script or execute). You can also implement built-in batch files as batch resource.
Always try to find implemented cookbook for your tasks - in Supermarket. If there aren't any you are looking for, you have to write your own cookbook. Since you are not asking for this, I won't write here any long tutorial, but instead will just redirect to Chef Docs.
Chef can download resources from server, so this is not a problem.
In the Internet you have also many tutorials of setting server-client Chef, but I would recommend you testing your cookbook with chef-client in --local mode first -in this mode you do not need to have any server running.
Learning Chef isn't easy nor hard - it all depends on quality of tutorial and - to be honest - many of them are just... Hard-to-understand-written. Setting environment with automatic tool is always nice idea, but you have to think if you really need this - if writting cookbook will give you many free hours in reinstalling Windows, it is for sure worth it.
Let me know if you need more informations.
I tried to adopt Vagrant in our team. I created a Vagrantfile and make provisioning in some way. Everything works as charm, but ...
It's unclear for me how I can automate some routine tasks like:
running django(I use django, but it's framework agnostic problem) dev server on 0.0.0.0
running grunt watcher
providing a separate console for django-specific commands
It is looks like vagrant not intended to help with this kind of automation and I look for some community adopted way to do that. I goggled and found nothing.
I see a few way to that:
bootstrap.sh script but messy and hard to mantain
something like tmuxinator -- requires tmux on host machine and now it's impossible to put tmuxconfig in project repo
etc
What is the 'canonical' way to resolve this problem?
P.S.: Please, think about designers, manual testers and other guys which like to use tools as is
In general you are best off using a provisioner. To be honest, a bootstrap.sh file is a good place to start unless you want to learn the ins and outs of something like chef / ansible / salt / puppet. If you do you might want to start at salt (SaltStack) because it is written in python which I'm guessing you use given the django angle.
For your specific questions:
Part of the point of vagrant is it lets you develop against real stacks and real web servers so you can avoid the "oh, that don't quite work the same on apache" moment that often comes in projects. So for your first question I would look at how to provision the app behind apache / nginx or whatever you are using for the production web servers.
Because of the shared file systems users can just run grunt locally on the host machine. This also lets grunt do things like hook into OSX notifications.
I'm not familiar with tmuxinator so I'm not sure how to start here. But if it is a service that the server really runs then you should figure out a way to package the install and deploy it to the provisioned VM. As for configuration, is it possible to get a dev config in the repo?
Same as #Wyatt, I recommend use Vagrant with provision tools, such as puppet, saltstack, chef, anisble, etc. These tools are created for the requirements you ask for, and most are open source. Choice is no wrong, you can start learning from any one, they are similar.
With that, you can quickly and easily run several VM servers with all applications installed automatically. With the customised Puppet codes or chef cookbooks, you can update them any time and provision to VM easily, you can re-use them for your PROD environment as well.
Take some times to learn one of these automation tools first, you will get benefit to save a lot of time.
I use Puppet, and recommend the best puppet book PRO PUPPET to you. It has all you need.
Ok, so I'm a bit late jumping onto the Vagrant band-wagon, but figured it's about time I did.
Brief background: I've been a freelance developer for quite some time now developing solutions based on Magento and Drupal, and have finally gathered enough demand to warrant the need to build up a team. Previously, whenever I started development on any new project, I use to clone a preconfigured base VM in Virtualbox, and use that. Of course there were still configurations to do on it until I could start with actual development. Every project's web files therefore all resided inside /var/www/projectname on an Ubuntu VM.
Now I've read up on why I should be Vagrant, especially considering that I now have a team of 4 developers working with me, but I would appreciate any feedback on the following questions I have:
Moderator note: I know this isn't exactly asking a programming question, so please advise if this could be turned into a wiki, as I'm sure that feedback into this will help someone just like me.
I am still reading through the Vagrant docs, so please be kind...noob questions ahead!
I now work on a Mac. Does it matter if I use Parallels, and another developer uses VirtualBox on Windows if we need to share or collaborate on projects?
When I issue the command, vagrant up for an existing project, will it start the VM up as I would in VirtualBox or will it recreate the VM?
Is the command vagrant halt the same issuing sudo poweroff in Ubuntu, for example?
I currently use PhpStorm and its SFTP feature for project files synchronization with the option to exclude certain files on the remote server (VM) from being imported and sync'ed...will I be able to specify the same using Vagrant folder sharing?
Could I easily zip or archive a Vagrant VM, move it to a file server, and then "re-import" when and if needed? (example bug fixes, or new feature enhancements)
What do we use to easily provision VMs for common projects? Should we being using Puppet, Chef, Puphpet or Salt? I've seen that Puphpet provides a nice GUI to create a vagrantfile which I'm sure once generated, we could customize for future projects. At a very basic level, we need to ensure that certain applications are installed onto the server (zip, phpmyadmin, OpenSSL, etc.), certain PHP settings, PHP and PEAR modules, and Apache settings. I already have base VMs set up as I'd like them for both Magento projects as well as Drupal projects.
EDIT: I should also add that I use to enable Host Adapter in VirtualBox (on Windows), configure the VHost inside Ubuntu, and then update my host machine's hosts file with something like 192.168.56.3 drupalsite1.dev. So I'm unsure if Port Forwarding would be better to use? I'm not very clued up on that I must admit.
Like i said - noob questions! However, I would really appreciate any feedback on these questions. My deepest thanks!
Most of what you are asking is subjective so common sense and experience are the best tools.
I recommend all team members use the same provider (parallels isn't officially supported) and virtualbox is readily available. The base boxes, by provider, could have slight variances, you never know.
Vagrant will start the vm similarly but vagrant also does other things like configuration the network, hostname, shared folders, etc. Not quite the same. The big power lies in the capability to be able to teardown the environment and bring it back in a cleanly provisioned state.
Basically, yes.
Yes, your vagrant VMs are just like your own mini cloud. You would interact the servers similar to the way you'd interact with external boxes.
Yes, the simple answer is that it's called packaging and you can share the resultant .box. However, it's good practice to keep the base box and provisioning scripts under CM so you can rebuild and modify as needed.
For provisioners, I think it is dependent upon your experience and your familiarity with the provisioner language and how much you want to invest in learning them. Look through the provisioner support and see what fits your need and budget. Chef has a very steep learning curve, in my experience, but also has a lot of thought built in. Most provisioners have wide libraries of available installation "scripts".
The host adapter can be handled identically in vagrant.
Learn by doing, I recommend going down the table of contents (navbar) of the vagrant docs and trying each step where it makes sense. Then make your decisions.
That is my 2 cents. Hope this helps!