I need to ensure some of my gems are installed from our own gem repository rather than rubygems, while the rest are installed from rubygems. Can I set this up in the Gemfile without worrying about a naming conflict with a identically named gem in Rubygems? How Can I determine where the gem is downloaded from?
eg
Gemfile:
source :rubygems
gem 'gemfromrubygems1'
gem 'gemfromrubygems2'
source "http://our.own.gem.repo.com/the/path/to/it"
gem 'gemfromourrepo'
Bundler 1.7 has a new feature that allows you to select the source for specific gems by nesting them in a block:
source "https://rubygems.org"
gem 'gemfromrubygems1'
gem 'gemfromrubygems2'
source "http://our.own.gem.repo.com/the/path/to/it" do
gem 'gemfromourrepo'
end
or specifying it as an option:
source "https://rubygems.org"
gem 'gemfromrubygems1'
gem 'gemfromrubygems2'
gem 'gemfromourrepo', source: "http://our.own.gem.repo.com/the/path/to/it"
See http://bundler.io/v1.7/gemfile.html for details.
According to the Source Priority section in the Gemfile manpage sources are searched from last entered to first entered.
Based on what you said, it sounds like you want to always prefer your gem over rubygems.org. As long as you don't need to vary your preference (ie. some dups from rubygems.org and some dups from your private repo) then your problem is solved simply with the following Gemfile:
source 'https://rubygems.org'
source 'http://our.own.gem.repo.com/the/path/to/it'
gem 'gemfromrubygems1'
gem 'gemfromrubygems2'
gem 'gemfromourrepo'
The only way I found seems like a horrible hack.
Bundler will search for the best version of your gem starting at the source listed last and then searching all the sources listed previously. It doesn't matter where the source lines are relative to the gem lines, only relative to each other.
I tried to make it work using :git and :path, but neither of those work for gemservers. That leaves matching the best version.
If you set the version of your gem to something like 2.mine.1 and push that to your server, you can constrain the version in your Gemfile.
source :rubygems
source 'http://myrepo'
gem 'gemfromourrepo', '~> 2.ourrepo'
Then the best matching version should be one from your server. There's a chance someone could push their own gem of the same name with 2.ourrepo.2 to rubygems, but that is unlikely if it is unique.
The path command might be able to help. It allows you to set gem specific sources
gem "foo", "1.0", :path => "bar"
Source:http://bundler.io/v1.3/man/gemfile.5.html
Related
My company has a private Gem-in-a-box server where multiple teams can share internally created gems. Recently, a gem has been added to this server which I want to use. It turns out, this gem has a dependency on net-ssh and net-scp, which are available from ruby-gems.org, and are not stored on the Gem-in-a-box server. When I add the new gem to my Bundler Gemfile and run an install, I get the following error:
C:\jruby-1.7.18\bin\jruby.exe --1.9 C:\jruby-1.7.18\bin/bundle install
Fetching source index from http://my.server.org/geminabox/
Fetching gem metadata from http://rubygems.org/.
Fetching source index from http://my.server.org/geminabox/
Fetching gem metadata from http://rubygems.org/.
Fetching additional metadata from http://rubygems.org/........
Resolving dependencies...
Could not find gem '["net-ssh", "net-scp"] (>= 0) java', which is required by
gem 'gem_dependent_on_ssh (>= 0) java', in any of the sources.
Process finished with exit code 6
Here is a snippet from my Gemfile:
source 'http://rubygems.org' do
gem 'net-scp'
gem 'net-sftp'
end
source 'http://my.server.org/geminabox/' do
gem 'gem_dependent_on_ssh'
end
It looks like it is only looking for gem dependencies on the same server as the gem being loaded in... Is there something I can add to my Gemfile to get around this? Or, can I go to the team that created the gem and have them add something to tell it where to look for dependencies? Or, is the only solution to add the net-ssh and net-scp gems to the Gem-in-a-box server so it can find them as a local dependency?
Thanks in advance!
It looks like the docs say no (very bottom):
SOURCE PRIORITY
When attempting to locate a gem to satisfy a gem
requirement, bundler uses the following priority order:
The source explicitly attached to the gem (using :source, :path, or
:git)
For implicit gems (dependencies of explicit gems), any source,
git, or path repository declared on the parent. This results in
bundler prioritizing the ActiveSupport gem from the Rails git
repository over ones from rubygems.org
The sources specified via
global source lines, searching each source in your Gemfile from last
added to first added.
The relevant one being #2 here. It's a little vague but I think this other part of the docs helps clarify:
Bundler will search for child dependencies of this gem by first looking in the source selected for the parent, but if they are not found there, it will fall back on global sources using the ordering described in SOURCE PRIORITY.
So it sounds like you need to provide rubygems.org as the fallback global source if you want that to be the case, i.e. using the source 'http://rubygems.org' line on its own as you have done in your answer. Otherwise when looking for child gems (dependencies) it will look only on the source provided for the parent.
That said, I also have a company Gem-in-a-box server and tried to reproduce your issue but could not, so I'm not positive I'm interpreting those docs correctly.
After consulting with a few other sources, it appears that if you remove the blocks and simply declare the sources, it manages to install all gems:
source 'http://rubygems.org'
source 'http://my.server.org/geminabox/'
gem 'net-scp'
gem 'net-sftp'
gem 'gem_dependent_on_ssh'
While this does fix this specific issue, I can still imagine a scenario where you would want to specify the source for a specific gem, and yet use it to resolve a dependency in another source. So while our issue is resolved, the question still stands.
In my own gem, I have a Gemfile that looks basically like this:
source 'https://my.gemserver.com'
source 'https://rubygems.org'
gemspec
My .gemspec has all dependencies listed as add_dependency and add_development_dependency.
As of Bundler 1.8, I get the warning:
Warning: this Gemfile contains multiple primary sources. Using `source` more than
once without a block is a security risk, and may result in installing unexpected gems.
To resolve this warning, use a block to indicate which gems should come from the
secondary source. To upgrade this warning to an error,
run `bundle config disable_multisource true`.
Is there a way to resolve this warning (without muting via bundle config)? I cannot find anything about a source option in the Rubygems specification.
No, you'll either need to mute the warning or add the source block to your Gemfile with the specific gems you want to come from your private server. There isn't a need to duplicate the ones that come from rubygems.org (or you could do it the other way around, if you depend on more private gems than public ones, and your private gems do not themselves depend on public ones).
The problem is that the gemspec format has no support for specifying the source for each gem, so without duplicating them into the Gemfile, there is no way to specify which gems come from each source.
Kind of sad, but one has to move it out to Gemfile :-(
Gemfile:
source 'https://my.gemserver.com' do
your_gem1
your_gem2
#...
end
source 'https://rubygems.org'
gemspec
but then, if some of your gems should be included in :development or :test group, following could be used
Gemfile:
your_gem1, :source => 'https://my.gemserver.com'
#...
group :development do
your_gem2, :source => 'https://my.gemserver.com'
#...
end
source 'https://rubygems.org'
gemspec
To elaborate on the discussion on the bundler issue, as previous answers have stated, you must include the gem in you Gemfile. However, you only need to specify the version of the gem in your .gemspec. If you change versions more often than private dependencies this isn't a terrible solution.
Reference the gem without version in Gemfile:
# Gemfile
source 'https://rubygems.org'
source 'https://xxx#gem.fury.io/me/' do
gem 'my-private-dependency'
end
gemspec
Reference the gem with version specification in the .gemspec:
# my-gem.gemspec
lib = File.expand_path('../lib', __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
Gem::Specification.new do |spec|
spec.add_dependency 'my-private-dependency', '~> 0.1.5'
end
I am running my own gem server with geminabox. This gem server hosts private forks of several public gems, as well as my own gems. I have a Gemfile with the sources listed like so:
source https://rubygems.org
source http://my/gem/server
When Bundler installs a bundle, I would like it to always use the version available on my gem server if it exists and satisfies the version requirements. I can't get this to work. Instead, bundler seems to pull from rubygems unless it can't find a gem with the same name there. This may be because the versions on rubygems are higher. However, bundler's documentation makes no mention of version numbers. This is how it describes source priority:
SOURCE PRIORITY
When attempting to locate a gem to satisfy a gem requirement, bundler uses the following priority order:
The source explicitly attached to the gem (using :path or :git)
For implicit gems (dependencies of explicit gems), any git or path repository otherwise declared. This results in bundler prioritizing the ActiveSupport gem from the Rails git repository over ones from rubygems.org
The sources specified via source, searching each source in your Gemfile from last added to first added.
Also, I know it is possible to explicitly list a source for each gem in the Gemfile. I would prefer to avoid this, since I have many of my own gems, and it is annoying to make explicit entries in the Gemfile for dependencies of dependencies (which you have to do, because bundler will not read the Gemfile of dependencies).
How can I achieve my goal?
I have a gem that is not public and not on a publicly accessible repo (it is on the local filesystem) that I wish to use in a Heroku hosted app.
Bundler does not even allow this, for example:
group :production do
gem 'mygem', :git => #giturl
end
group :development do
gem "mygem", :require => "mygem", :path => "/gem_dev/mygem"
end
$ bundle install
...
You cannot specify the same gem twice coming from different sources. You specified that mygem (>= 0) should come from source at vendor/cache and source at ...
I've used bundle install --path vendor and bundle package to try and get it to use the cache, but since the gem is a local path Bundler tells me (helpfully) that it won't cache it. To get around this I copied the .gem to vendor/cache and had the line in Gemfile:
gem 'mygem', :path => 'vendor/cache'
but I get this error from Bundler:
Could not find gem 'mygem (>= 0) ruby' in source at vendor/cache.
Source does not contain any versions of 'mygem (>= 0) ruby'
Heroku needs a valid path. Any ideas how I can get this to work for me?
Any help is much appreciated.
This can't be done with the current version (1.0.x). From http://gembundler.com/man/bundle-package.1.html
GIT AND PATH GEMS In Bundler 1.0, the bundle package command only
packages .gem files, not gems specified using the :git or :path
options. This will likely change in the future.
What follows is my opinion:
Why not? That surely wasn't a technical decision so I'm... aggrieved... Bundler is supposed to solve problems, and since it's written in Ruby by a couple of well known rubyists you'd expect (or I would) that they'd have taken the route that Ruby core has - we're adults, let us choose what we really want to do, regardless of whether the computer believes otherwise.
If I'm using Bundler and want to install a thousand different versions of a gem then that should be my business. A warning would've done. Let's hope the next version doesn't have this strange decision included in the code.
I'm helping on the development of a series of interrelated gems. As such, I don't want them to have a hard dependency on each other, but I do want them to run tests in development that use each other. Simple right? Just use add_development_dependency in the gemspec, right? Well, there is one little wrinkle - the git repository contains all the gems, and so I want the Gemfile to point to the local copy of the gem. This works with a hard dependency. In the gemspec, I have this line for my hard dependency:
s.add_dependency "mygem-core"
And then in the Gemfile, I have this line:
gem "mygem-core", :path => "../mygem-core"
This works PERFECT. The dependency exists for when I push this package out, and when I'm testing, it will use the local copy of mygem-core. The problem is that when I put THIS in the gemspec:
s.add_development_dependency "mygem-runtime"
And then this in the Gemfile:
gem "mygem-runtime", :path => "../mygem-runtime"
Then I get an error when I run bundle:
You cannot specify the same gem twice coming from different sources. You specified that mygem-packager (>= 0) should come from source at ../mygem-packager and
Yes, that's not a typo at the end. There is literally a blank, empty space at the end for the second 'location'. Is there any smart ways to work around this? I want to add this as a development dependency, and use the local source. What am I missing?
It's probably better that you leave that gem out of the .gemspec manifest and put it in the Gemfile under the :development group.
# Gemfile
source :rubygems
gemspec
gem "mygem-runtime", :path => '../mygem-runtime', :group => :development
If your using Gemfile to specificy a local path to a gem you will need to remove it from gemspec. Bundler will parse gemspec and add the dependencies those bundler is installing, so its like having the gem specified twice.