Ruby gems repository - ruby

I'm trying to set a gem repository on one of our local servers. Here are the steps I'm running, that I've followed from several guides.
1) I create the BASEDIR folder /var/www/html/gems
2) sudo cp -r /usr/lib/ruby/gems/1.8/gems/someGem /var/www/html/gems
3) sudo gem generate_index -d /var/www/html/gems
However, when I run this, I get the following output:
Loading 0 gems from /var/www/html/gems
Loaded all gems
Generating quick index gemspecs for 0 gems
Complete
Generating specs index
Generating latest specs index
Generating quick index
Generating latest index
Generating Marshal master index
Generating YAML master index for 0 gems (this may take a while)
Complete
Compressing indicies
It's not loading the gem for some reason. I did see a guide that recommended making the BASEDIR as /var/www/html/rubygems/ and then further make a gems/ directory within the BASEDIR and copy the desired gems to this gems/ directory. I also tried this, but was getting the same results.
Our server had the unfortunate luck of having the same configuration as mentioned in this post (RHEL5, ruby 1.8.5, /var and /tmp on separate partitions), but we upgraded as suggested to ruby 1.8.6, but it still won't load the gem.
Has anyone come across this? Found a solution?

I stumbled upon my old post and realized I'd actually solved this some time ago, so I figured I'd post my answer.
The problem was my step 2: copy gems action.
In the OP, I had tried
2) sudo cp -r /usr/lib/ruby/gems/1.8/gems/someGem /var/www/html/gems
What I found was that I needed to copy the actual .gem files, which lived at .../gems/1.8/gems/cache . So what I really needed to do for step 2 is:
sudo cp .../gems/1.8/gems/cache/*.gem /var/www/html/gems
After copying the gems correctly, the indexing worked as expected, and we could then use our server by adding it as a gem source on various machines. Installing our custom gems then worked smoothly.

We internally use Artifactory for managing our in-house rubygems - some of them are proprietary and some are publicly released. we can enforce security between our different groups (dev, qa - prerelease, release, ...)
Also, rubygems.org is proxied and cached locally, which helps us gain better performance and avoid remote downtimes.
Eventually, developers are using a single source url, aggregating both remote and local repositories transparently.

You should generate the index not on the gems subdirectory. But on the basedir one.
sudo gem generate_index -d /var/www/html
It'll take automatically the gems in the "gems" subdirectory.
And generate the index at in the basedir.

The same thing happened to me so I started to browse the code for Gem::Indexer and wrote a few lines of code to do the indexing from irb launched from my host destination (the one above ./gems):
require 'rubygems'
require 'rubygems/indexer'
i=Gem::Indexer.new '.'
i.generate_index
And if you want to update the index later:
require 'rubygems'
require 'rubygems/indexer'
i=Gem::Indexer.new '.'
i.update_index
I suspect the issue with the gem command must exist somewhere higher at the command parsing level but I have not looked into it (I just wanted to create my repository and move on).

Related

The 'gems' configuration option has been renamed to 'plugins'.?

When I use the latest jekyll and start its server, a certain warning below shows up:
$ jekyll serve Configuration file:
/Users/ishida/git/gipcompany.github.io/_config.yml
Deprecation: The 'gems' configuration option has been renamed to 'plugins'. Please update your config file accordingly.
I guess the following gems should be replaced with plugins, and then the warning won't show up, but I couldn't find any official documentation. Is my guess correct?
$ cat _config.yml
(omit)
gems:
- jekyll-feed
Your guess is correct, gems key is now plugins.
In jekyll/lib/jekyll/configuration.rb you can find:
def backwards_compatibilize
...
renamed_key "gems", "plugins", config
It is part of this Pull Request: plugins config key as replacement for gems (#5130)
And it was the result of this issue:
What do you think about renaming the gems config directive to plugins,
with full backwards compatibility?
The idea being, if you're not from the Ruby world, "gems" isn't a very
intuitive term (even though that's what they are). Further, users
shouldn't have to care about (or see) the behind-the-scenes plumbing,
in terms of how things are distributed.
Instead, let's call the key "plugins", a term which has a broader
understanding outside the Ruby ecosystem, and one that should be
familiar to any user coming from a more traditional CMS like WordPress
or Drupal.
In terms of practicality, it'd just be a matter of combining the gems
and plugins arrays, if they exist when we load the config, and
updating the docs in the next minor release.
You're right, in order to fix that, follow these steps:
Open your _config.yml file
Find the line beginning with gems:
Replace gems with plugins

How to copy Ruby gem from one server to another

Is there a way for a particular ruby gem (mysql gem) to be copied from one server to another (both are similar servers)?
In my case, the other server is a production server and gem install mysql is failing to compile there as rubymysqllib is missing and can't install build dependencies there.
Basically my question is: how can I copy a gem from one server to another server?
I couldn't figure out if it is a good practice but I did this way sometimes and worked.
To copy all gems I just copied and past the folder C:\Ruby187\lib\ruby\gems\ (adapt this in accordance with your case) from one machine to another.
To copy a single gem I went inside the gemsfolder and copied the files with same name (of the gem) inside of the respective folders.
C:\Ruby187\lib\ruby\gems\1.8\cache
C:\Ruby187\lib\ruby\gems\1.8\doc*
C:\Ruby187\lib\ruby\gems\1.8\gems*
C:\Ruby187\lib\ruby\gems\1.8\specifications
#*doc and gems are optional
Another way I found, but I never did is like this.
CAUTION
When you copy the gems like this you are not verifying the dependencies and it may cause bad behavior of your application.

can you get the Gem object to return a list of available gems in the path?

I am having issues getting my local gems to work on my web server. I keep getting no such file to load errors even though I have added my local gem dir to my .gemrc paths. I am hoping to find a way to at least see what gems I DO have access to.
I have tried adding my local gem path a couple ways including
Gem.path.push "/myHome/usrName/ruby/gems
with no luck. How do I do something like
Gem.available_gems.each do |g|
puts g
end
?
Try using Bundler. It manages your gems for you and lets your project know where to find them when employed properly. On production servers it can install a local copy of the gem so that you don't need to worry about setting up your path vars correctly.

Generating RDOCs for locally installed gems

I am trying to contribute to a gem I recently took interest in - Nesta. The developer has done a great job in creating one of the lightest, thinest CMSs you can find and I want to document it. I have read through the code and commented on a few methods to the best of my knowledge.
However, I seek to test this out locally by calling gem server and seeing the changes on my machine before pushing it online.
Things I have tried:
Manual edit.
Documented the file.
Fired up gem server.
Using the gem tool.
Documented the file.
Ran gem rdoc nesta --rdoc
Restarted gem server
All to no avail. Please help.
Thank you.
You can preview generated html pages without installing a modified version of the gem on your machine. Add this to nesta's Rakefile:
require 'rake/rdoctask'
Rake::RDocTask.new('doc') do |i|
i.rdoc_files = FileList['lib/**/*']
end
and type rake doc. Then view generated html/index.html file.
Are you sure you've installed the version that you've modified, not the original version?
If you've installed the modified version, but have forgotten to install the rdoc, see Can you install documentation for existing gems?

Should Gemfile.lock be included in .gitignore?

I'm sort of new to bundler and the files it generates. I have a copy of a git repo from GitHub that is being contributed to by many people so I was surprised to find that bundler created a file that didn't exist in the repo and wasn't in the .gitignore list.
Since I have forked it, I know adding it to the repo won't break anything for the main repo, but if I do a pull request, will it cause a problem?
Should Gemfile.lock be included in the repository?
Update for 2022 from TrinitronX
Fast-forward to 2021 and now Bundler docs [web archive] now say to commit the Gemfile.lock inside a gem... ¯_(ツ)_/¯ I guess it makes sense for developers and ease of use when starting on a project. However, now CI jobs need to be sure to remove any stray Gemfile.lock files to test against other versions.
Legacy answer ~2010
Assuming you're not writing a rubygem, Gemfile.lock should be in your repository. It's used as a snapshot of all your required gems and their dependencies. This way bundler doesn't have to recalculate all the gem dependencies each time you deploy, etc.
From cowboycoded's comment below:
If you are working on a gem, then DO NOT check in your Gemfile.lock. If you are working on a Rails app, then DO check in your Gemfile.lock.
Here's a nice article explaining what the lock file is.
The real problem happens when you are working on an open-source Rails app that needs to have a configurable database adapter. I'm developing the Rails 3 branch of Fat Free CRM.
My preference is postgres, but we want the default database to be mysql2.
In this case, Gemfile.lock still needs be checked in with the default set of gems, but I need to ignore changes that I have made to it on my machine. To accomplish this, I run:
git update-index --assume-unchanged Gemfile.lock
and to reverse:
git update-index --no-assume-unchanged Gemfile.lock
It is also useful to include something like the following code in your Gemfile. This loads the appropriate database adapter gem, based on your database.yml.
# Loads the database adapter gem based on config/database.yml (Default: mysql2)
# -----------------------------------------------------------------------------
db_gems = {"mysql2" => ["mysql2", ">= 0.2.6"],
"postgresql" => ["pg", ">= 0.9.0"],
"sqlite3" => ["sqlite3"]}
adapter = if File.exists?(db_config = File.join(File.dirname(__FILE__),"config","database.yml"))
db = YAML.load_file(db_config)
# Fetch the first configured adapter from config/database.yml
(db["production"] || db["development"] || db["test"])["adapter"]
else
"mysql2"
end
gem *db_gems[adapter]
# -----------------------------------------------------------------------------
I can't say if this is an established best practice or not, but it works well for me.
My workmates and I have different Gemfile.lock, because we use different platforms, windows and mac, and our server is linux.
We decide to remove Gemfile.lock in repo and create Gemfile.lock.server in git repo, just like database.yml. Then before deploy it on server, we copy Gemfile.lock.server to Gemfile.lock on server using cap deploy hook
Agreeing with r-dub, keep it in source control, but to me, the real benefit is this:
collaboration in identical environments (disregarding the windohs and linux/mac stuff). Before Gemfile.lock, the next dude to install the project might see all kinds of confusing errors, blaming himself, but he was just that lucky guy getting the next version of super gem, breaking existing dependencies.
Worse, this happened on the servers, getting untested version unless being disciplined and install exact version. Gemfile.lock makes this explicit, and it will explicitly tell you that your versions are different.
Note: remember to group stuff, as :development and :test
Simple answer in the year 2021:
Gemfile.lock should be in the version control also for Rubygems. The accepted answer is now 11 years old.
Some reasoning here (cherry-picked from comments):
#josevalim https://github.com/heartcombo/devise/pull/3147#issuecomment-52193788
The Gemfile.lock should stay in the repository because contributors and developers should be able to fork the project and run it using versions that are guaranteed to work.
#rafaelfranca https://github.com/rails/rails/pull/18951#issuecomment-74888396
I don't think it is a good idea to ignore the lock file even for plugins.
This mean that a "git clone; bundle; rake test" sequence is not guarantee to be passing because one of yours dozens of dependencies were upgraded and made your code break. Also, as #chancancode said, it make a lot harder to bisect.
Also Rails has Gemfile.lock in git:
https://github.com/rails/rails/commit/0ad6d27643057f2eccfe8351409a75a6d1bbb9d0
The Bundler docs address this question as well:
ORIGINAL: http://gembundler.com/v1.3/rationale.html
EDIT: http://web.archive.org/web/20160309170442/http://bundler.io/v1.3/rationale.html
See the section called "Checking Your Code into Version Control":
After developing your application for a while, check in the
application together with the Gemfile and Gemfile.lock snapshot. Now,
your repository has a record of the exact versions of all of the gems
that you used the last time you know for sure that the application
worked. Keep in mind that while your Gemfile lists only three gems
(with varying degrees of version strictness), your application depends
on dozens of gems, once you take into consideration all of the
implicit requirements of the gems you depend on.
This is important: the Gemfile.lock makes your application a single
package of both your own code and the third-party code it ran the last
time you know for sure that everything worked. Specifying exact
versions of the third-party code you depend on in your Gemfile would
not provide the same guarantee, because gems usually declare a range
of versions for their dependencies.
The next time you run bundle install on the same machine, bundler will
see that it already has all of the dependencies you need, and skip the
installation process.
Do not check in the .bundle directory, or any of the files inside it.
Those files are specific to each particular machine, and are used to
persist installation options between runs of the bundle install
command.
If you have run bundle pack, the gems (although not the git gems)
required by your bundle will be downloaded into vendor/cache. Bundler
can run without connecting to the internet (or the RubyGems server) if
all the gems you need are present in that folder and checked in to
your source control. This is an optional step, and not recommended,
due to the increase in size of your source control repository.
No Gemfile.lock means:
new contributors cannot run tests because weird things fail, so they won't contribute or get failing PRs ... bad first experience.
you cannot go back to a x year old project and fix a bug without having to update/rewrite the project if you lost your local Gemfile.lock
-> Always check in Gemfile.lock, make travis delete it if you want to be extra thorough https://grosser.it/2015/08/14/check-in-your-gemfile-lock/
A little late to the party, but answers still took me time and foreign reads to understand this problem. So I want to summarize what I have find out about the Gemfile.lock.
When you are building a Rails App, you are using certain versions of gems in your local machine. If you want to avoid errors in the production mode and other branches, you have to use that one Gemfile.lock file everywhere and tell bundler to bundle for rebuilding gems every time it changes.
If Gemfile.lock has changed on your production machine and Git doesn't let you git pull, you should write git reset --hard to avoid that file change and write git pull again.
The other answers here are correct: Yes, your Ruby app (not your Ruby gem) should include Gemfile.lock in the repo. To expand on why it should do this, read on:
I was under the mistaken notion that each env (development, test, staging, prod...) each did a bundle install to build their own Gemfile.lock. My assumption was based on the fact that Gemfile.lock does not contain any grouping data, such as :test, :prod, etc. This assumption was wrong, as I found out in a painful local problem.
Upon closer investigation, I was confused why my Jenkins build showed fetching a particular gem (ffaker, FWIW) successfully, but when the app loaded and required ffaker, it said file not found. WTF?
A little more investigation and experimenting showed what the two files do:
First it uses Gemfile.lock to go fetch all the gems, even those that won't be used in this particular env. Then it uses Gemfile to choose which of those fetched gems to actually use in this env.
So, even though it fetched the gem in the first step based on Gemfile.lock, it did NOT include in my :test environment, based on the groups in Gemfile.
The fix (in my case) was to move gem 'ffaker' from the :development group to the main group, so all env's could use it. (Or, add it only to :development, :test, as appropriate)

Resources