I am building a framework that loads user provided ruby code. It is basically a plugin mechanism. I want the user provide ruby code to be able to require gems of its own. I intend to have the "plugin" package include a vendor directory with the gems.
How can I load gems that are required by the plugin without having them conflict with my framework's gems? For example, if my framework uses treetop version 1.3.0, and a plugin uses treetop 1.4.2 I want each to work with their specified version.
Likewise, is there a way to prevent plugins from conflicting with each other?
I have looked at gem_plugin, _why's sandbox, and some other tools. But I don't see any library that specifically handles this case - I assume its been done before.
I have also looked at the internals of Bundler to see how it manages gem versions. I am prepared to do some pretty complex stuff if need be. But I am still uncertain of how to go about it.
I also have a lot of freedom in how I implement this. So if you think I am barking up the wrong tree, please say so.
Thanks for any advice.
SIDE NOTE: It occurred to me while writing this that I need something similar to the Classloaders in a Java servlet container. A WAR file can include jar files, and the web application's class loader will prefer those over the jars that are on the global classpath. Is there any way in ruby to segment the ruby "classpath" (i.e. load_path, require, etc)?
To be blunt, you can't have two versions of the same gem loaded at the same time.
Bundler does a good (ish) job of looking through all of the required gems and finding a solution to the various overlapping dependencies, but even so it is limited to only loading one version of a gem at a time.
This leads to plugin developers constantly having to update to support any changes that are made in dependent gems in order to avoid just the situation you describe.
(Don't get me started on the screw up that results from the various competing JSON implementations and the pain you have to go through when you have several gem dependencies all requiring different ones.)
Respectfully disagree with the answer above. Here is how I do it:
ruby -S gem list my_gem
`*** LOCAL GEMS ***
my_gem (1.0.1, 1.0.0, 0.0.2)
`
ruby -S gem lock my_gem-1.0.0 > locklist.rb
which generates list of dependencies for a specific version into locklist
require 'rubygems'
gem 'my_gem', '= 1.0.0'
gem 'gem_base', '= 1.0.0'
gem 'rest-client', '= 1.7.2'
gem 'savon', '= 1.1.0'
gem 'addressable', '= 2.3.6'
gem 'mime-types', '= 1.25.1'
gem 'netrc', '= 0.11.0'
now you can do load('locklist.rb') which will load a specific version of a gem along with its dependencies. Look ma, no Bundler.
Related
I am working on learning Ruby and one thing that I have seen in several instances and cannot understand are scripts that start with the keyword gem.
An example can been seen in the Sensu code.
gem "amqp", "1.3.0"
require "amqp"
require File.join(File.dirname(__FILE__), "base")
I understand the require statement for accessing another gem, but what does the exact gem "amqp", "1.3.0" mean?
This is the gem method that Rubygems adds to Kernel (Rubygems is required by default in current Ruby versions). It activates a specific version of a gem (version 1.3.0 of the amqp gem in this case), meaning that the gems lib dir (or whatever dirs the gem specifies) is added to your LOAD_PATH, as are the lib dirs of any dependent gems it has.
It also checks that there are no version incompatibilities with any already activated gems.
All gems are activated when you use them. This normally happens when you call require. Calling gem activates the gem but doesn’t require any files from it, hence the line require "amqp" below (note the difference between amqp the gem, which the gem method refers to, and amqp the file, which is contained in the amqp gem and is what the require method is refering to).
This method is used to ensure you are using a specific version of a gem, without needing to use Bundler (or something similar). Bundler also has a gem method used in Gemfiles, but this is a different (but similar) method.
It’s also used to specify that you want to use the gem version of a library that is also part of the standard library (say if you want to use a more recent version). For example the Yaml library distributed with Ruby is Psych which is also available as a gem.
I think I have a basic understanding of what require/include statements at the top of a ruby script are doing, like
require 'rspec'
These statements are easy to google and find relevant results. But sometimes I have seen a gem statement like
gem 'rspec'
What does this line do?
In ruby code, gem(gem_name, *requirements) defined in Kernel tells Ruby to load a specific version of gem_name. That's useful when you have installed more than one version of the same gem.
For example, if you have installed two versions of rspec, say 2.12.0 and 2.13.0, you can call gem before require to use specific version. Note that gem should come before the require call.
gem 'rspec', '=2.12.0'
require 'rspec'
A gem 'gem_name' without version uses the latest version on your machine, and that's unnecessary. You can call require without gem to get the same behavior.
And besides, in Bundler::Dsl, gem is used to tell bundler to prepare/install specific version of ruby gems. You'll see that in Gemfile
The original behaviour of require, before Rubygems, was to search all the directories listed in the $LOAD_FILES variable for the file, and to load the first one it finds that matches. If no matching file was found, require would raise a LoadError.
Rubygems changes this process. With Rubygems, require will search the existing $LOAD_PATH as before, but if there is no matching file found then Rubygems will search the installed gems on your machine for a match. If a gem is found that contains a matching file, that gem is activated, and then the $LOAD_PATH search is repeated. The main effect of activating a gem is that the gems lib directory is added to your load path. In this way the second search of the load path will find the file being required.
Normally this will mean that the latest version of a gem that you have installed gets activated. Sometimes you will want to use a different version of a gem, and to do that you can use the gem method. The gem method activates a gem, and you can specify the version you want, but doesn’t require any files. When you later require the files you want, you’ll get them from the gem version you specified.
In Ruby, gems are packages with functionality that can be used out of the box (as libraries in other Programming languages).
The gems that you use with your Ruby Project can easily be managed with a tool called "bundler", just google it. The snippet of code you posted is part of the spec file that bundler uses to install and update all the libraries that you specify for your project.
If you are developing a Ruby on Rails, using gems an managing them with bundler is very common and so to say best practice.
Gems are just great because there are so many useful libraries that extend default functionality, eg of rails, and that you can use out of the box!
For a list of gems, visit rubygems.org
It seems like a strange feature that rubygems wants to know what my development dependencies are, so far I never saw it used to actually load these when running the gem tests.
If someone decides to hack your gem for whatever reason, e.g. they fork it on Github to add a feature to it (which they might want to contribute), it helps if they know what development dependencies your gem needs (e.g. testing frameworks, mocking tools etc.).
If you're using bundler with the gemspec command, it will hook into your gemspec dependencies and install the development dependencies along with the runtime ones when you run
bundle install
This saves you having to install these gems manually.
The gem command can also list all the dependencies of a gem including the development ones:
gem dependency my_gem
Gem my_gem-0.1.3
activerecord (~> 3.0.0)
json (~> 1.4.3, development)
rake (>= 0, development)
rspec (~> 2.5.0, development)
ruby-openid (~> 2.1.0)
Once again this is probably more for other people rather than for yourself.
My gems have normally unit-test. This test requires sometimes gems, which are not needed to use the gem. Or perhaps you need additional gems to generate parts of the gem.
Once I misused development dependencies to define 'optional dependecies' (dependecies were necessary for some specific features of my gem, but not needed for the 'normal' usage).
Example:
My application offered the posibility to export to a file as text or pdf. The pdf-generation uses prawn. So prawn is a dependecy - but it is not necessary to use the application, only a specific feature needs it.
So I didn't add prawn to gem dependencies (it is not necessary), but to the development dependencies (it is usefull for the gem).
The documented answer to this question at RubyGems is as follows:
Development dependencies are useful for when someone wants to make
modifications to your gem. When you specify development dependencies,
another developer can run gem install --dev your_gem and RubyGems will
grab both sets of dependencies (runtime and development). Typical
development dependencies include test frameworks and build systems.
For example, I may specify spec.add_development_dependency 'minitest'
Adding the --dev flag to the gem install command would then pull in the minitest gem automatically. This feature saves another developer from manually having to identify and install the gems necessary for development.
If I'm developing a gem using Bundler and RSpec for testing. Where do I do my Bundler.setup? Let's assume my gem is called fancy-gem and my directory setup is similar to the following:
Gemfile
Gemfile.lock
lib/
fancy-gem.rb
Rakefile
README
spec/
...
Should I execute Bundler.setup in my 'fancy-gem.rb' or does this cause problems with other gems which might use bundler? I'm thinking, when I'm not doing this, then there is no way to guarantee that the right version of the third party libraries I'm requiring is loaded.
I already asked, if I need to add Bundler itself to the Gemfile. The answer was no, but now I'm not so sure, because if I do execute Bundler.setup somewhere then Bundler actually is a dependency of my Gem and should be installed along with my Gem when it is downloaded from rubygems.org
In my opinion you should neither depend on bundler, nor use it in your gem. The way I'd do it is simply require your gem's dependencies in lib/fancy-gem.rb (almost every gem only has a handful of runtime dependencies, so this should not be too much of a hassle) and I'd call Bundler.setup only in the development files (like spec_helper.rb or Rakefile). This way you don't screw with applications that use your gem and still get all the convenience of automatic dependency management when developing your gem.
To create a new Ruby gem, should I use Jeweler or should I use Bundler's built-in gem skeleton to create a base gem? What are the differences that matter?
Use Bundler
From the command line:
bundle gem your_new_gem
This will create a directory called your_new_gem with just a basic set of files and directory structure that are now considered best-practice. It's quick, easy, and a great place to start.
Creating a Gem isn't that difficult and I would advise to try building a gem from scratch, without any tools. After you know what's involved (creating a gemspec, building and pushing it to rubygems.org), you can use tools to speed up the process. My guess is you won't because making a gem is hardly the trouble at all.
I would go with Jeweler. The Bundler skeleton is only going to give you the basics. Jeweler has alot more options to work with and many helpful rake tasks for versioning, pushing to github, creating the gemspec, building and installing.
If you are working with Rails 3 engines, I have a Jeweler fork (definitely a work-in-progress) that will generate the app skelaton and include the engine file. You just have to run the jeweler command with --rails3-engine as an option. Here is the fork if you are interested:
https://github.com/johnmcaliley/jeweler
I would recommend using the built-in bundler command.
bundle gem your_gem_name
There are some rules that you should follow when creating a gem. Such as naming conventions and versioning rules.
I recently wrote a post on creating gems in netguru's blog. I think you'll find what you need in there.
https://netguru.co/blog/posts/creating-a-gem-a-step-by-step-tutorial
Hope this helps.
Here's an alternative that's worth looking at: ore
Bundler gives you a single template for ruby gems, whereas ore has multiple built in templates, plus the ability to create your own. It also supports Git, SVN (urgh) and Mercurial.
You can build a gem in RubyMine too. File > New Project > New Gem. It is that easy. But I want to make some notes about this approach:
For debugging, RubyMine will use the Fast Debugger gem, ruby-debug-ide. I know that most people now are using Pry with Byebug, but ruby-debug-ide is an interface which glues ruby-debug to IDEs like Eclipse (RDT), NetBeans and RubyMine.
Under Run > Edit Configurations > + > Ruby, I add a new debug configuration, according to the documentation here: https://www.jetbrains.com/help/ruby/run-debug-configuration-gem-command.html#1
Under Configuration, under 'Ruby Script', I add the path to the ruby gem file under lib: lib/my_gem.rb
Under Configuration, under 'Ruby SDK', I specify an RVM gemset I am using.
Under Bundler section, I check 'Run the script in context of bundler'. This would use bundle exec, which will read the dependencies in my Gemfile in my project's root. Now for gems, the Gemfile contains a method call "gemspec", which in turn reads the dependencies in dependencies in my_gem.gemspec. There, I have dependencies passed to the Gem::Specification.new block:
spec.add_development_dependency "bundler", "~> 1.7"
spec.add_development_dependency "rake", "~> 10.0"