TL;TR
We have two projects (minitest project, and a page objects project). The Minitest project uses the page object project. When Jenkins runs the tests, we use a remote git path to the page objects project. When we run it locally we change the gem file in the minitest project to use a local path\or local git repo. This results in us needing to continuously change the gem file. We don't wan't to have to do this. How can we set this up so that we don't need to keep changing the gem file?
The long version
I have two Ruby projects, which are tightly coupled. Both projects are used to support testing with Capybara. One project contains Minitest tests. The other project contains page objects. The minitest project uses the page object project. The page object project is published as a gem so that other teams can use it. Quite often we need to edit both projects at once.
I am using Bundler to manage the gems. These projects used to be in two different git repos.
The tests run in Jenkins, and when we work on them we run them locally. When the tests run in Jenkins the minitest project can reference the page objects project either as a published gem or as a remote git repo.
The problem we had with this is that we kept needing to change the gem file of the minitest project. We would change it from pointing to a remote git repo, to a checked out (local) path. Or we would point to a local git repo. In this case we would still need to update the minitest project's gem file with the local branch name. We needed to edit the gem file with either approach, then we had to remember to change this back before checking in and merging.
We then moved both projects into a single repo. Now the gem file in the minitest project only uses the relative path e.g. path => .... This means that we don't need to constantly update the gem file in the minitest project. However, now when we run bundle install or bundle update, only the gem file from the minitest project is used (at least this is what I assume is happening).
How do I use a different gem path\git repo on my dev machine to what is used in Jenkins, without continuously changing the gem file?
I would like to know what approach should I be using in this case? Can avoid making changes to the minitest project's gem file?
Consider using bundler's groups (in Gemfile):
group :project1 do
gem 'rspec'
end
group :project2 do
gem 'minitest'
end
Then specify which group to use in application.rb, for example
Bundler.require(:project1) # or :project2 depending on the project
More info here: http://bundler.io/groups.html
I'm fairly new to ruby and jekyll and I wanna start using jekyll to meu personal webpage. Among the files in my repo there is a'Gemfile' and 'Gemfile.lock' files.
The Gemfile is used to manage dependencis and Gemfile.lock is generated through
bundle install
Is any of them required to be in the repo, or does gh-pages overrides any definitions contained in those files?
Other contained folder is '.sass-cache'. From this question I get the impression that it's used for caching issues. Once again, is it required, or gh-pages generates those files itself?
You can version your Gemfile. This can be useful for a reinstall. gemfile.lock and .sass-cache are not necessary to push to Github. They are used locally.
If your Gemfile contains something other than:
source 'https://rubygems.org'
gem 'github-pages'
That certainly means that you use plugins that are not supported by Github (see supported plugins list here). You then need to generate your site locally and push the result to your Github.
I've described a workflow here and you can also have a look at Octopress rake file that can help for such a setup (need small hacks to be used on a basic Jekyll install).
From my interpretation of the docs, github simply uses their own github-pages gem to pull in dependencies and then runs jekyll serve on your repo.
One consequence of that is that your gemfile (and thus any gems not part of github-pages) is ignored. For example, I use jekyll-less to generate my CSS but since this is not part of github-pages, making CSS changes is a multi-step process for me:
Update .less files
Build
Copy compiled CSS out of _site
Commit .css changes
Push
Another consequence is that any run-time or build-time stuff you use is ignored by github, so your .sass-cache directory does not need to be added to the repo.
TL;DR:
Don't run bundle within an existing git repository. Weird things will happen without any error messages.
Original Question:
I've built a gem by adapting the steps in this tutorial:
http://net.tutsplus.com/tutorials/ruby/gem-creation-with-bundler/
As a final step, I've run gem build .gemspec
This succeeds, but when I install the gem I find the critical file, the one which contains my code, isn't in the gem. Another file in the same (lib) directory, "version.rb", does exist in the gem.
I don't know how to start debugging this...how does bundler/gem build decide which files to include in the gem?
Edit:
My workflow is:
gem build <project_name>.gemspec
gem unpack <project_name>
=> confirm file does not exist in <unpacked>/lib/
gem install <project name>
=> confirm file structure in ~/home/stefan/.rvm/... contains gem, but does not contain desired file
Edit 2 / Resolution:
I was finally able to get this working by committing all my code to a remote repository, creating a clean clone, and building the gem. The new gem included all the required files.
A bit of history...I originally created the code and committed it before thinking about making a gem (this is my first gem). I then used bundle inside the original repository, which didn't complain, but was probably the reason for the weirdness.
One of the things bundler did for you is start a local git repo to version-manage your gem code. Check that you have added the file in git
git add lib/gem_name/missing_file.rb
Bundler generated gems use git internally to track "membership" of source files for the gem. You can see this in the .gemspec where it uses backticks to call out to git and generate a file list:
gem.files = `git ls-files`.split($/)
Note this also means you should pay attention to what you list in .gitignore
I'm trying to create a gem with Bundler, following this guide: http://rakeroutes.com/blog/lets-write-a-gem-part-one/. In it, it says:
I incorrectly thought after taking my first look through the gemspec
that I would need to add more require statements as I developed my
gem. That isn’t the case: the files just need to be in git.
I am trying to clean up some of my old gems to use this convention, but when I install my gem, the classes from the other files are not available. I have a couple directories nested under my /lib dir, but I wouldnt think that would be an issue. Is there anything simple to overlook that would prevent my other files from being required? Any help would be appreciated.
In the link, when he says he doesn't need to add a lot of "require" statements, he must mean adding files to the s.files, s.executables, and s.test_files arrays--these determine what files get packaged up into the gem and what files get ignored. As you can see from the gem spec, whatever's tracked by git in certain directories is going to be included in the packaged gem.
Ruby's require is a different story. Standard require rules still apply.
Ruby's gem system works by adding a bunch of different places for Ruby to look for "foo.rb" when you run require "foo". If "lib" is your only require path for your gem, when you require "my_gem" Ruby is only going to run the code in lib/my_gem.rb. If lib/my_gem.rb doesn't require any other files in your gem, then Ruby hasn't seen them and so you'll get undefined constant errors when you try to use the classes from those files.
For examples, you might take a look at two simple gems I've written; both were started with bundle gem: HashToHiddenFields and SimpleStats. In both gems, main Ruby file in lib/ requires everything that needs to be loaded for the gem to work correctly. For example, hash_to_hidden_fields.rb requires action_view/helpers/hash_to_hidden_fields so that the ActionView::Helpers::HashToHiddenFields constant+module exists so we can include it into ActionView::Base.
Hope that answers the question. I know Ruby requiring was pretty fuzzy to me for a while.
I'm sort of new to bundler and the files it generates. I have a copy of a git repo from GitHub that is being contributed to by many people so I was surprised to find that bundler created a file that didn't exist in the repo and wasn't in the .gitignore list.
Since I have forked it, I know adding it to the repo won't break anything for the main repo, but if I do a pull request, will it cause a problem?
Should Gemfile.lock be included in the repository?
Update for 2022 from TrinitronX
Fast-forward to 2021 and now Bundler docs [web archive] now say to commit the Gemfile.lock inside a gem... ¯_(ツ)_/¯ I guess it makes sense for developers and ease of use when starting on a project. However, now CI jobs need to be sure to remove any stray Gemfile.lock files to test against other versions.
Legacy answer ~2010
Assuming you're not writing a rubygem, Gemfile.lock should be in your repository. It's used as a snapshot of all your required gems and their dependencies. This way bundler doesn't have to recalculate all the gem dependencies each time you deploy, etc.
From cowboycoded's comment below:
If you are working on a gem, then DO NOT check in your Gemfile.lock. If you are working on a Rails app, then DO check in your Gemfile.lock.
Here's a nice article explaining what the lock file is.
The real problem happens when you are working on an open-source Rails app that needs to have a configurable database adapter. I'm developing the Rails 3 branch of Fat Free CRM.
My preference is postgres, but we want the default database to be mysql2.
In this case, Gemfile.lock still needs be checked in with the default set of gems, but I need to ignore changes that I have made to it on my machine. To accomplish this, I run:
git update-index --assume-unchanged Gemfile.lock
and to reverse:
git update-index --no-assume-unchanged Gemfile.lock
It is also useful to include something like the following code in your Gemfile. This loads the appropriate database adapter gem, based on your database.yml.
# Loads the database adapter gem based on config/database.yml (Default: mysql2)
# -----------------------------------------------------------------------------
db_gems = {"mysql2" => ["mysql2", ">= 0.2.6"],
"postgresql" => ["pg", ">= 0.9.0"],
"sqlite3" => ["sqlite3"]}
adapter = if File.exists?(db_config = File.join(File.dirname(__FILE__),"config","database.yml"))
db = YAML.load_file(db_config)
# Fetch the first configured adapter from config/database.yml
(db["production"] || db["development"] || db["test"])["adapter"]
else
"mysql2"
end
gem *db_gems[adapter]
# -----------------------------------------------------------------------------
I can't say if this is an established best practice or not, but it works well for me.
My workmates and I have different Gemfile.lock, because we use different platforms, windows and mac, and our server is linux.
We decide to remove Gemfile.lock in repo and create Gemfile.lock.server in git repo, just like database.yml. Then before deploy it on server, we copy Gemfile.lock.server to Gemfile.lock on server using cap deploy hook
Agreeing with r-dub, keep it in source control, but to me, the real benefit is this:
collaboration in identical environments (disregarding the windohs and linux/mac stuff). Before Gemfile.lock, the next dude to install the project might see all kinds of confusing errors, blaming himself, but he was just that lucky guy getting the next version of super gem, breaking existing dependencies.
Worse, this happened on the servers, getting untested version unless being disciplined and install exact version. Gemfile.lock makes this explicit, and it will explicitly tell you that your versions are different.
Note: remember to group stuff, as :development and :test
Simple answer in the year 2021:
Gemfile.lock should be in the version control also for Rubygems. The accepted answer is now 11 years old.
Some reasoning here (cherry-picked from comments):
#josevalim https://github.com/heartcombo/devise/pull/3147#issuecomment-52193788
The Gemfile.lock should stay in the repository because contributors and developers should be able to fork the project and run it using versions that are guaranteed to work.
#rafaelfranca https://github.com/rails/rails/pull/18951#issuecomment-74888396
I don't think it is a good idea to ignore the lock file even for plugins.
This mean that a "git clone; bundle; rake test" sequence is not guarantee to be passing because one of yours dozens of dependencies were upgraded and made your code break. Also, as #chancancode said, it make a lot harder to bisect.
Also Rails has Gemfile.lock in git:
https://github.com/rails/rails/commit/0ad6d27643057f2eccfe8351409a75a6d1bbb9d0
The Bundler docs address this question as well:
ORIGINAL: http://gembundler.com/v1.3/rationale.html
EDIT: http://web.archive.org/web/20160309170442/http://bundler.io/v1.3/rationale.html
See the section called "Checking Your Code into Version Control":
After developing your application for a while, check in the
application together with the Gemfile and Gemfile.lock snapshot. Now,
your repository has a record of the exact versions of all of the gems
that you used the last time you know for sure that the application
worked. Keep in mind that while your Gemfile lists only three gems
(with varying degrees of version strictness), your application depends
on dozens of gems, once you take into consideration all of the
implicit requirements of the gems you depend on.
This is important: the Gemfile.lock makes your application a single
package of both your own code and the third-party code it ran the last
time you know for sure that everything worked. Specifying exact
versions of the third-party code you depend on in your Gemfile would
not provide the same guarantee, because gems usually declare a range
of versions for their dependencies.
The next time you run bundle install on the same machine, bundler will
see that it already has all of the dependencies you need, and skip the
installation process.
Do not check in the .bundle directory, or any of the files inside it.
Those files are specific to each particular machine, and are used to
persist installation options between runs of the bundle install
command.
If you have run bundle pack, the gems (although not the git gems)
required by your bundle will be downloaded into vendor/cache. Bundler
can run without connecting to the internet (or the RubyGems server) if
all the gems you need are present in that folder and checked in to
your source control. This is an optional step, and not recommended,
due to the increase in size of your source control repository.
No Gemfile.lock means:
new contributors cannot run tests because weird things fail, so they won't contribute or get failing PRs ... bad first experience.
you cannot go back to a x year old project and fix a bug without having to update/rewrite the project if you lost your local Gemfile.lock
-> Always check in Gemfile.lock, make travis delete it if you want to be extra thorough https://grosser.it/2015/08/14/check-in-your-gemfile-lock/
A little late to the party, but answers still took me time and foreign reads to understand this problem. So I want to summarize what I have find out about the Gemfile.lock.
When you are building a Rails App, you are using certain versions of gems in your local machine. If you want to avoid errors in the production mode and other branches, you have to use that one Gemfile.lock file everywhere and tell bundler to bundle for rebuilding gems every time it changes.
If Gemfile.lock has changed on your production machine and Git doesn't let you git pull, you should write git reset --hard to avoid that file change and write git pull again.
The other answers here are correct: Yes, your Ruby app (not your Ruby gem) should include Gemfile.lock in the repo. To expand on why it should do this, read on:
I was under the mistaken notion that each env (development, test, staging, prod...) each did a bundle install to build their own Gemfile.lock. My assumption was based on the fact that Gemfile.lock does not contain any grouping data, such as :test, :prod, etc. This assumption was wrong, as I found out in a painful local problem.
Upon closer investigation, I was confused why my Jenkins build showed fetching a particular gem (ffaker, FWIW) successfully, but when the app loaded and required ffaker, it said file not found. WTF?
A little more investigation and experimenting showed what the two files do:
First it uses Gemfile.lock to go fetch all the gems, even those that won't be used in this particular env. Then it uses Gemfile to choose which of those fetched gems to actually use in this env.
So, even though it fetched the gem in the first step based on Gemfile.lock, it did NOT include in my :test environment, based on the groups in Gemfile.
The fix (in my case) was to move gem 'ffaker' from the :development group to the main group, so all env's could use it. (Or, add it only to :development, :test, as appropriate)