I have written a ruby service that I want to package and distribute internally to a specific environment (a standardized linux host). After digging around a bit for the best ways to create a distribution, I've come across a lot of blogs and answers here that recommend bundler and gem packaging as well as a lot of binary distribution options (e.g. traveling-ruby), but these all seem like a bit too much for a relatively simple service that will get deployed to a known environment. I want to create a distribution that doesn't require dependencies to be resolved at deploy time (e.g. bundle install --deployment is not the approach I want).
So with this in mind, is there an available framework that is commonly in use by ruby apps that would create a redistributable package with all dependencies included in it? I am currently doing this in 2 steps and wondering if there's a "better" ruby-way of doing it - something along the lines of gem build ... that creates dependency-inclusive archive?
# Assuming `bundle install` was run on developer workstation and there's a `Gemfile.lock`
$ bundle install --deployment
$ tar zcf ../my-app.tar.gz ./
my-app.tar.gz can now be distributed and if I have an executable to run it with, I can do so with bundle exec bin/run from within the directory after it's extracted. This a good approach?
I've seen a gem called crate that might be able to work....
Related
Background: Not the Question
I have a project that is running in both development and production in a chruby environment, where ruby-install was installed as root and rubies are stored in /opt/rubies. I have a really hard time (as many people do) getting nokogiri to compile its native extensions as part of a bundle, but it compiles fine as any chruby user outside of the project directory and so long as there are no binstubs. That's just background context; it's not really what my question is about.
My Current Hack
I find that when I have committed binstubs to my project, all kinds of badness happens. So far, my best solution seems to be:
# skip any bundled/binstub version of bundler
rm -rf ./bin
`which bundle` install --binstubs --without="development test"
which seems to work just fine. Nokogiri is apparently not a production dependency (yay!) and the ./bin directory gets trimmed down to just the gems needed for production.
Question: Would the Bundler Deployment Flag Fix Anything?
So, here's the question: What's the difference between what I'm doing here, and bundle install --deploy with or without binstubs? I know it points to vendor/bundle instead of bin by default, but the docs don't really explain the pros and cons of this approach (at least not in a way that I'm understanding).
Aside from knowledge, which is valuable for its own sake, I'd really like to vendor in production gems in a way that it's likely to work across systems (e.g. RHEL6 and RHEL7) without having to rebuild native extensions or strip out development/test gems on production machines.
Is the deployment flag the answer? Or is there a better way to vendor gems with native extensions for cross-distro projects?
I come from a Python and JavaScript background.
When developing a JavaScript project, dependencies are installed in a node_modules directory in the project root.
When developing Python project, typically virtualenvwrapper is used. In this case dependencies are installed in a virtual environment, which is located in ~/.virtualenvs/<project_name> by default.
Now I need to use a ruby tool for a project. The tool that appears to be the most promising for a similar setup as described above, is bundler.
However, the default installation location for bundler is system-wide. I consider this to be harmful.
For one of my systems, it will prompt for a password, at which point I can still abort.
However, for my other system I can write into the global ruby installation. I'm using a homebrew installed ruby here. Bundle will just install dependencies globally.
I know I can specify the installation location by adding --path, but this is easy to forget.
One way to enforce an installation path is by committing .bundle/config. It would just have to contain this:
---
BUNDLE_PATH: "."
However, some googling around shows that it's not adviced to commit this file.
What is the recommended way to prevent accidental global installations using bundler?
Who's to say it will be accidental? It really depends on what context you're talking about here. I have my Ruby set up so that bundle install works without requiring sudo, it's all done through rbenv automatically. The same is true with rvm if done as a user-level install.
When it comes to deploying apps and you want to make sure it's deployed correctly, that's where tools like Capistrano come into play: Create a deployment script that will apply the correct procedure every time.
Checking in a .bundle/config is really rude from a dev perspective, just like checking in any other user-specific preferences you might have. It causes no end of conflict with other team members.
I need to install a number of ruby gems (all with C extensions) on a production server which does not have any dev tools installed. I'd like to build gems on a dev server first and then repackage and install resulting native gems on production server.
However, there seems to be no standard methods to package gem with native extensions for redistribution. I am aware of rake-compiler, but none of the gems in concern works with it out of the box. Specifically, I am working with json-1.7.5, rb-inotify-0.8.8 and ffi-1.2.1 gems.
Any pointers on how to do this task or documentations on the subject are appreciated.
Using Jordan Sissel's fpm you can take various input archives (including gems) and compile and package them as (among others) DEBs or RPMs.
An example to compile the json gem into a deb package follows:
cd /tmp
fpm -s gem -t deb json
This will download the latest version of the json gem and create a rubygem-json-1.5.7-1.amd64.deb archive in /tmp which you can install on your server. Note that the compile box and the final server need to be rather identical. At least the distribution and bitness, the ruby version and its file layout, and the available loadable libraries should be the same. Basically all the constraints your upstream distribution deals with...
That said, in the long term I found it much easier to just install a compiler on the target servers and use rbenv or rvm on the server. For most small and mid-size installations, it's much easier to handle as you don't need to pre-compile and ship everything to your servers.
Hi You could do it with: gem-compiler
You need to tell RubyGems the filename of the gem you want to compile:
$ gem compile yajl-ruby-1.1.0.gem
The above command will unpack, compile any existing extensions found and repackage everything as a binary gem:
Unpacking gem: 'yajl-ruby-1.1.0' in temporary directory...
Building native extensions. This could take a while...
Successfully built RubyGem
Name: yajl-ruby
Version: 1.1.0
File: yajl-ruby-1.1.0-x86-mingw32.gem
It have well written documentations here:
https://github.com/luislavena/gem-compiler.
I am using it as well as we have own gem server. Just have to be careful with distribution because some gems compiled on wheezy won't work on jessie and so on.
You're going to have to build them on a system that's almost exactly the same for this to work. If you're linking against shared libraries that are in a different location or have slightly different versions it may not work at all. Sometimes you have some slack, where it will work with a range of versions, but this cannot be assured.
There's no way to package with native extensions for this very reason, there's just too many possible combinations of libraries.
You'll also need to make sure you're using the same architecture, including 32-bit or 64-bit as required.
Sometimes you'll get lucky and there's a package for your OS that can install these for you, but these won't work with rvm or rbenv.
[Ruby Noob]
I have a small (command line) utility written in Ruby, which requires a few gems. Is there a way to create a self contained bundle of my program such that I can run it on another machine that has Ruby installed (but not necessarily the gems)?
FWIW, the target machine runs Linux/Ubuntu.
You can use the gem bundle http://gembundler.com/
With bundle you create a Gemfile in your project root - a text that contains all your dependencies, very similar to Maven concept
In order to fetch all your dependencies simply tun
bundle install
The only issue is that you need to have the bundle gem itself installed, so you are back with the chicken-or-Egg problem :-)
I've used:
http://www.erikveen.dds.nl/rubyscript2exe/
before, but it was a while ago. Seemed to work okay for simple programs.
You can download it here:
http://rubyforge.org/projects/rubyscript2exe/
I want to automate things like:
Creating a new Ruby on Rails application with pre-selected database, Git initialize it, create a Heroku project, commit all files, etc.
Upload all files in folder to another computer through SSH, but do not overwrite files.
Upgrade Ubuntu, install all basic packages through apt-get.
From what I understand, tools for this are Rake and Thor, however, which one should I use?
Rake seems to me more de-facto and popular. I have heard people recommending Thor.
How do these stand to each other in a rundown?
Rake and Thor serve different purposes.
Rake is a general build script tool that is project-specific. In other words, you put your rakefile into your project folder and in your project's source control, and you can create, build and do other automation tasks that are specific to your project in that rakefile. Rake requires a rakefile to run.
Thor is a general purpose command line scripting tool that makes it very easy to re-use scripts across many projects and to do project setup, etc., like you are suggesting. Thor allows you to "install" an executable script that you can call from anywhere on your system, similar to calling "ruby", "gem" or "rake" command lines. However, Thor's scripts are more suited to general purpose, cross-application automation because the Thor script does not rely on a file sitting in your project-specific folder. A Thor script is the entire script, packed and installed for re-use anywhere.
Based on your stated needs, you are better off using Thor because you will be able to install your script in one location and have it work anywhere on your system. You will not be bound to where a Rake file is sitting or anything like that.
By the way, Rails 3 uses Thor for pretty much everything that is not project specific. You still have a Rake file and you still run things like "rake db:migrate" or "rake test:units". Thor is used for things like "rails new ...", "rails server" and "rails generate ..." The use of Thor AND Rake in Rails 3 is the perfect illustration of where each of these tools is best suited.
For setting up Ubuntu chores, Chef might be a better option.
From their web site:
Chef is an open source systems integration framework, built to bring the benefits of server configuration management to your entire infrastructure.
It's written in Ruby and there are tons of Chef recipes/cookbooks. Chef will handle setting up Ubuntu and installing packages, servers, etc.
I don't know if you are working with virtual machines, but Vagrant will set up a virtual machine and then use Chef to configure it.
There is something important to mention here.
http://guides.rubyonrails.org/generators.html in its section 8 Application Templates.
You can execute git commands, select gems, capify project.
And you could also execute system commands to satisfy your last point: Upgrade Ubuntu, install all basic packages through apt-get.
I would go with puppet.
By the way, maybe vagrant is useful to you?