I am looking for some modern Capistrano 2.x / Deprec recipes for setting up toosl ike MongoDB, Xapian and Xapian-fu, and other tools (Redis, Memcache/d, etc)
Anyone know if Deprec is still active enough to have some recipes for some of these modern tools
Thanks
Charles
I would suggest looking into using Chef or Puppet to manage the tool/system configuration and continue to use Capistrano for app deployment.
There are modules to integrate Capistrano with either Chef or Puppet, eg:
capistrano-chef
capistrano-puppet
The Chef community is growing quickly but Puppet has been around for longer. The community contributed Puppet recipes tend to have broader O/S support, but Chef is catching up.
Either could be a great choice but discussing the merits of each for this question would lead to extended debate :).
A few perspectives:
Puppet vs Chef - Fight!
Why We Chose Chef
deprec.org and the deprec-users discussion group both seem rather dormant at the moment.
Related
I have been learning some new technologies for automatization of workflow. On some projects I saw people using Rakefiles. I have learned that they are used to automate the deployment/workflow etc.
On others, I saw people using Ansible.
Maybe I am misunderstanding one of those technologies, but how do they differ?
Rake is a tool most often used for administrative tasks that can be scripted for an application whether that be database migrations, data loading, export/import of data, clearing caches, etc. Ansible is a provisioning tool it can configure a server, and do application deployment. You could conceivably use rake to do some of what Ansible does and vice versa, but if you were to try to recreate Ansible in a rake utility you'd be re-inventing the wheel. The closer equivalent to Anisble in the ruby/rails world is Capistrano. However, Ansible is a more generically applicable tool and Capistrano is more a specialty tool for rails applications (IMO).
Question
Can anyone explain why it would be better to choose the puppet or chef vagrant provisioners, rather than the shell provisioner?
Background
I'm in the process of getting started with Vagrant. One of the things I'm having trouble with is deciding which provisioner to use. So far, I've had some success using the shell provisioner, but it has been more work than I expected to get it to run reliably.
At the moment, I'm not familar with ruby, puppet or chef, but I'm happy to learn any or all of them if I have to. My early experience playing with puppet and chef is that if someone else has a recipe that does exactly what you want, it works really well, but doing something non-standard means falling back coding up solution in ruby.
I'm aware of articles comparing puppet and chef, and I'm less worried about which of them to use, rather than knowing when and why I should use them at all.
Full disclosure: I'm a Puppet Labs employee. But I chose Puppet as a product over 2 years before joining them.
I would recommend that you use Puppet or Chef over shell if your configurations are going to a) have any degree of complexity and b) going to change over time - or you expect your installation environment itself to change in a way that might alter the way your deployment performs. Your scripts may be very good, but ultimately, unless you are following terrific programming practices around them, testing and QA'ing them, etc they are going to fail at some point.
There's an entire body of literate around DevOps discussing this notion, but it comes down to the principle of "technical debt" - we tend to do things the easy way now, and thus perceive them as simpler, at the cost of increasing complexity and difficulty later.
One of Puppet's strengths is its deterministic nature - the manifest you write must be able to be programmatically transformed by Puppet into a model of the server you are building. This is perceived by people as being more "difficult" but I would argue that the difficulty is lessened if you average it out along the curve of your technology's lifecycle. In other words, Puppet forces you to do your thinking now, but then deploy to scale with ease, rather than thinking later and re-engineering as you go. Pay in cash now, rather than by credit, with interest, later.
If you're purely pulling down other peoples' manifests, you're going to run into trouble at some point - although we would like it not to be so, working with Puppet today that's certainly the case, because they are writing them to address the general case, and not your particular system. Many general-purpose manifests become useful only when you reach a better understanding of Puppet.
So rather than start there, I'd work my way through the excellent Learning Puppet guide to start to grasp the basics. Puppet's learning curve is steep, but it levels off after a short while.
There are other reasons to use other provisioners or tools, but I'd surely argue that you are better with Puppet or Chef than trying to ensure that your shell scripts are doing exactly what you think they are supposed to do, for as long as you need to spawn new environments.
Ah, with the freedom of choice comes the complication of choosing what is right for you.
Chef Solo - Chef solo is most ideal if you’re just getting started with chef or a chef server is simply too heavy for your situation. Chef solo allows you to embed all your cookbooks within your project as well, which is nice for projects which want to keep track of their cookbooks within the same repository. Chef solo runs standalone – it requires no chef server or any other server to talk to; it simply runs by itself on the VM.
Chef Server - Chef server is useful for companies or individuals which manage many projects, since it allows you to share cookbooks across multiple projects. The cookbooks themselves are stored on the server, and the client downloads the cookbooks upon running.
Puppet - The Puppet provisioner runs stand-alone Puppet manifests that are stored on the server and downloaded to the client VM when it is created. The provisioner does not require a Puppet server and runs on the VM itself.
Puppet Server - The Puppet Server provisioner connects to a Puppet server and configures your client VM using node configuration on that server.
Other tools, shell scripts, etc. - Do you use something other than that which is built into Vagrant?
Provisioners are simply subclasses of Vagrant::Provisioners::Base, meaning you can easily build your own, should the need arise.
You can also check out the documentation, docs.vagrantup.com/v2
I would choose the Shell provisioner, then let the shell script clone your puppet/chef repository from github or bitbucket. The script can setup a ssh key to allow automated git clone. The benefits are most cloud providers support this as well so you can use the same script.This blog is explains git, puppet and vagrant well, one man and the cloud blog
RVM is great for developing on your local machine. But is it safe on a production machine?
I built RVM for production and added the developer 'niceties' later on.
If you would like more information read the documentation on the website and come talk to me in #rvm on irc.freenode.net sometime during the day EDT most days.
Since RVM is just a fancy way of downloading, isolating and switching between existing Ruby implementations, I'd say that it's as production ready as whatever ruby implementation you're currently running it with.
Essentially, all RVM does is point your path at a specific Ruby implementation. This is exactly what happens when you use your *nix distribution's Ruby implementation. The only real difference is that your path will be re-written so that when you run ruby -v it will run a ruby from your current user's .rvm directory instead of a global system directory like /usr/local/bin.
I'd go even further and say that using RVM is a better solution than using what generally gets installed in a *nix distro because it makes it easy to sandbox the specific ruby implementation on a per-user basis. RVM also makes it possible to attempt switching rubies (ie; from 1.8.7 to 1.9.2) on your production app while keeping a solid rollback strategy in place if something doesn't work quite right. It also makes it easier to keep old applications running on one version of Ruby, while switching new apps to more current versions.
I disagree, especially if you're using any kind of automated production process (puppet, chef, fog, etc) and you have more than one or two machines.
We've had issues where version X of RVM worked in a completely different way to version Y of RVM (different default Rubygems versions, different default gemset configs, complete changeup of how system wide install works), breaking our automated provisioning process.
Not an issue if you're developing and on hand to tune things, a killer if you have an unattended scripted / puppet install. We worked around these issues by locking to a particular RVM version, but I remember having a conversation with Wayne where he discouraged this. If we kept using RVM in prod, we were going to actually package it into a series of .debs (one for the install, one for each Ruby).
The way that .rvmrc prompts by default and can only be overriden in the homedir ~/.rvmrc (and not the system-wide one) was also unhelpful.
I actually like the way that RVM will change up and do things this way in development - nothing sucks more than being held back by backward compatibility. This approach, however, cost us some time (and pulled hairs) in production/staging/uat/test.
RVM is apparently a reasonable production tool
You know, I once made a similar rvm is a development tool comment and was informed that rvm was originally a production tool.
So, RVM will make your production environment more complex, which is bad, but it makes it more isolated and compartmentalized, what the language people would call modular, and that's good.
In the end, as long as you test your deployments, I don't see how a static configuration of any kind could be, all by itself, "unsafe".
It all depends on how you are installing RVM , single-user or multi-user . installing RVM system wide can cause lots of mess whole switching between different rubies. Better you opt for single user , minus that RVM does a good job for what it's meant to do .
I guess there's two parts to this question:
Is RVM intended to be for production machines, as opposed to development machines?
Is RVM reliable enough software to be used on production machines?
For (1), Wayne E. Seguin has stated that it's intended to be used on production machines. There's no point in disputing his intent.
For (2), I'm not so sure. Is it appropriate to use software that has a new version number every couple of days on a production machine? Also, RVM once deleted my entire ~/ruby directory. To Wayne's credit, when I told him about it, he fixed it that night, but that doesn't exactly say "production ready" to me.
Edit: I've just read about bumblebee's deletion of /usr, and I'll just say - it could have been worse! LOL.
I've been using RVM on a production webserver for over a year now with zero problems. I've kept it pretty up-to-date, running rvm get head frequently. Zero issues, ever. :)
Yes, I've used rvm on production machines and also set up puppet modules to install rvm as the default system ruby along with gemsets, etc.
If you run multiple apps on a single server, rvm can help you keep all your apps gemsets (and ruby versions) totally separate. However, if you are running only a single app on a server, there may not be as much benefit to having rvm installed.
I've pretty much used RVM on all my production servers running rails apps!. RVM has not let me down.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
Are there any tools for windows like that *nix world has? I am looking for something like Chef or Puppet.
I have found cfEngine but it still looks very *nix centric. Ideally it would be open source, and command line driven.
The idea is to put together an automated infrastructure with windows based servers. Our current IT department does not allow non-windows servers.
Chef is supported on Windows by Opscode. While we don't run Windows for any of our infrastructure, we do have developers who are continually improving our Windows support. We also get community contributions, and most of the early phase Windows functionality for Chef was contributed by the community.
Important: Opscode now provides an MSI installer for Chef on Windows. This makes it easier than ever to get Chef and Ruby installed on Windows.
While we have a lot of Unix/Linux background across our teams, our intention is that Windows is treated as a first class citizen. 2012 will be a big year for Chef and Windows. Keep an eye on the Opscode blog for announcements.
The following Chef Resources work on Windows:
Environment Resource: sets windows environment variables
User
Group
Mount
File
Gem Package
Remote File
Cookbook File
Template
Service
Ruby Block
Execute
That is, these are resources included in Chef itself. As Chef is extensible with cookbooks, many more resources are added through a variety of Windows specific cookbooks. Read on for more information.
You can get started with using Chef and Windows here:
http://wiki.opscode.com/display/chef/Fast+Start+Guide+for+Windows
Originally, Doug MacEchern wrote some cookbooks to do a number of things to automate windows, too.
https://github.com/dougm/site-cookbooks/tree/master/windows
This information and more available on the Chef Wiki:
http://wiki.opscode.com/display/chef/Installation+on+Windows
Update
The following cookbook adds new resources to Chef to manage Windows:
http://community.opscode.com/cookbooks/windows
It is an update/rewrite of Doug's fine resources from his repository linked above. Documentation is available on the Chef Wiki.
The following cookbook deploys PowerShell and provides a resource to run PowerShell commands/scripts directly in Chef Recipes:
http://community.opscode.com/cookbooks/powershell
Documentation is available in the README.md included in the cookbook tarball.
Additional cookbooks for installing 7-zip, managing IIS and SQL Server have been added. Our "database" cookbook has been extended with a resource/provider for managing SQL Server databases and users (with the tds rubygem).
The knife-windows plugin for knife adds functionality for interacting with Windows systems to provision them with Chef installation.
Update: We have now added File ACL support for Windows to Chef, for all the usual file/directory suspects.
Cfengine Nova has had stable native support for Windows for years (i.e. no Cygwin is needed).
Everything that is possible on Unix (e.g. run commands, file copy, file edit, etc.) can also be done on Windows, with addition to Windows-specific features like
Registry management
File ACL permissions
Windows service management
Event log support
Advanced OS discovery (version, arch, service pack, network, domain, etc.)
For a short introduction, look in Section 11 of the Nova supplement manual: http://cfengine.com/files/Nova_Supplement.pdf
I have been looking for exactly the same thing.
I ran across Windows PowerShell Desired State Configuration which purports to do many of the CM tasks you'd want to do with Chef/Puppet.
I am not going to give any opinion on it because I am still early in evaluating it - but if you're still looking for something, give this a shot.
Having had a deal of experience here i think that it really depends what you need to manage on a given box. Find your most common use case and ask a more specific question in terms of: whats the best tool for managing that. Things like IIS can be tricky (on older windows builds at least) and finding modules for CM tools to manage things like IIS settings and app pools can be hit and miss - especially when it comes to advanced settings (worker thready recycling springs to mind)
However Puppet has forge modules for more basic IIS settings which largely worked for me - and i really like the declarative format here - especially if you are managing large numbers of IIS servers
Also one of the major issues with Windows and configuration management is the lack of a proper package management solution - what actually comes in really handy here is something called Chocolatey (see http://chocolatey.org/) there is a plugin to use this as a provider in puppet - making the whole management process particularly easy. However i imagine that this can be integrated with other CM tools aswell - but does a great deal to help with installing software automatically in a windows environment
Though it is worth stating that whatever you end up using you will have to spend a while figuring out some odd bugs - windows and scripting/automation is not the best.
Based on my experience, I am glad to use such toolset:
Use powershell to manage the windows configuration. If only windows server, you can remotely execute the ps script.
If you like, suggest to use Puppet or Chef to distribute your powershell script into client.
If you like, suggest to use cygwin + (windows sshd) + crond to manage the shell-likely scripts which run in windows.
I think its better if you can use Microsoft System Center.
It a solution from the Microsoft to manage Microsoft base products form desktops to servers and all. Should give a try ...
How to Deploy Applications in Configuration Manager
Deploying Software and Operating Systems in System Center 2012
I don't recommend Chef unless you are a software developer who knows ruby. Chef is designed for software developers who want to manage IT infrastructure. Not for SysOps people who are the ones that actually end up managing it. It tries to treat IT infrastructure like code which is counter intuitive to how it actually works which is more like just data.
Puppet is more friendly for SysOps people imho but still can be complicated.
The good news is that there are other options as well. Less so for windows but projects like Puppet do support it reasonably well.
What are the relative pros and cons of:
http://rubyforge.org/projects/daemons
http://github.com/kennethkalmer/daemon-kit
Which is more robust?
Are there any other effective Ruby daemon management tools?
DISCLAIMER: I maintain daemon-kit, so this might appear bias but I'm trying my best to be honest.
daemon-kit grew as a set of wrappers around the daemons gem, then about a year ago (with 0.1.7.3) I ripped all traces of the daemons gem from the project and handled everything myself, which resolved the issues you mentioned above, as well as several other.
Instead of acclaiming my own project (not that it needs it), I'll highlight some shortcomings that I plan to address in the future:
Daemons are not easily embedded into Rails applications
Project layout enforced on developers might be to rigid
Biased towards capistrano-based deployments of daemons
Testing daemons is difficult, but on inconceivable
I've got a separate branch where I'm toying with a total rewrite that hopes to make the project more flexible, but it is by no means a pain at the moment. It is currently in production use at quite a few companies, from ISP infrastructure management to telecoms, twitter polling & processing, and just about everything in between.
Movement on the project has been slow in the last few months, purely because it works well. The low version number is very deceptive, it should in fact be way past a 1.x release by now...
Hope this helps!