Bundler, when attempting to update or install, will hang forever - ruby

When attempting to run bundle install or bundle update, bundler will perpetually hang, and not complete its function. The only time that it will finish is when I specify a gem to update.
For example:
bundle update
Will hang forever unless I use it like this:
bundle update activerecord
Then it will complete as normal.
Any assistance would be appreciated.

This problem is due to a missing dependency, or worse a dependency-of-a-dependency. It's common when you don't use rubygems.org as your gemserver (an enterprise environment).
Common patterns:
You don't have that gem installed
You don't have the dependencies of that gem installed
You don't have that gem installed without its dependencies
Easiest Technique
create a new gemset, and re-bundle. This fixes the problem many times.
If you can't do that for production reasons, and you don't have an app history from which to reflect on when the problem gem was added, then:
Easier Technique
Having learned a bit since writing this answer, I thought I'd pass on this excellent article for how to run bundler with verbose debug output
export DEBUG_RESOLVER=1
bundle 2> debug_output.txt 1> stdio.txt &
This will dump all the debugging (err) output to debug_output.txt and the normal screen stuff to stdio.txt.
You'll want to dump 1> as well because every time bundler dumps a line to 2(stderr) it will put a crlf into 1. So Either dump 1 or background the job. I do both so I can work in the same terminal.
I usually follow it up with:
tail stdio.txt
to be sure things have started, then:
tail -n 10000 -f debug_output.txt
To search with /FAIL] through the file for failed attempts to install a dependency. Find a couple of them that are the same and you've generally located your culprit. The stderr works for bundle install or bundle update.
Debug Your Private Gemserver Technique
I needed to use this process-of-elimination method to determine that my (enterprise) gemserver index had become corrupted
comment out all gems
run bundle update to confirm the empty-set works
un-comment one gem and bundle update
GOTO 3 until you hit the problem
research its dependencies
When you are done
Unset the ENV var with
unset DEBUG_RESOLVER

You can try the --verbose flag too to get more output, but it's really archaic and not very helpful. Really the only way to do this from what I can see is:
Uncomment one gem at a time until it stops working, then try to figure out what the heck is going.
Remove version enforcement one gem at a time (if any) to see if that fixes it.
Remove the offending gem if you can (use a different gem) or lock in versions that do work.
For me, after running:
sudo bundle update --verbose
it kept hanging here:
Query Gemcutter Dependency Endpoint API: tenderlove-frex
Fetching from: http://rubygems.org/api/v1/dependencies?gems=tenderlove-frex
HTTP Success
Query List: []
Not at all helpful. I ended up figuring out that the conflict was between two gems. The following would hang forever:
source 'http://rubygems.org'
gem 'rails', '~> 3'
gem 'airbrake'
I removed the rails version:
source 'http://rubygems.org'
gem 'rails'
gem 'airbrake'
Then it worked, but I noticed in my Gemfile.lock that it was using Rails 2.3.X. So airbrake seemed to be dependent on Rails 2, but I wanted 3. I couldn't find anywhere that airbrake had this Rails 2.x dependency so not sure why it ended up like that. Why bundler can't output something more meaningful is beyond me.
This worked though:
source 'http://rubygems.org'
gem 'rails', '~> 3'
gem 'airbrake', '~> 3'
I really think there is something wrong with Bundler, but not sure.

Bundler may not be hanging. I just experienced a 7 minute time to bundle a relatively small Gemfile, and that is on the fastest possible SSD Macbook Pro with a 50 Gb download connection.

When deploying onto AWS instances, be sure your security group allows outbound HTTP and/or HTTPS connections - I encountered this problem because I had restricted the security group too much.

Related

Ruby: Unable to start worker - check_for_activated_spec

I am unable to run some of my jobs in sidekiq. And it seems to be somehow related to Bundler.. Maybe.
when I run my puma server with pumactl start in the logs I am getting:
[156149] ! Unable to start worker
[156149] /home/todd/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/bundler-2.3.25/lib/bundler/runtime.rb:309:in `check_for_activated_spec!'
Currently I'm on sidekiq 6.5.6 and Bundler 2.3.25
Anyone know of any issues with those two versions or anything else that may be causing this?
EDIT:
The interesting thing is when I start puma with bundle exec pumactl start I get a totally different error:
[ActionDispatch::HostAuthorization::DefaultResponseApp] Blocked host:
But my host is defined in my development.rb file.. in fact I've added the following, so NO hosts should be blocked:
config.hosts << /.*\.ngrok\.io/
config.hosts.clear
Then finally if I just start puma with a rails s all is fine, just my sidekiq worker won't run correctly.
Let's say that you want to start using Rails and one day you follow the general installation instructions which say that you should run this command:
gem install rails
And you get this output:
...
Successfully installed rails-7.0.1
You also start working with puma and sidekiq and install those gems for the convenience of running pumactl start and sidekiq:
gem install puma
...
Successfully installed puma-5.6.2
gem install sidekiq
...
Successfully installed sidekiq-6.4.2
Then after a day or a week or a month of tinkering you create a new Rails app:
rails new app
And since you want to use Sidekiq you add that to your Gemfile, which looks something like this:
# frozen_string_literal: true
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gem "puma", "~> 5.6.2"
gem "rails", "~> 7.0.1"
gem "sidekiq", "~> 6.0"
But you know that there are newer versions of those gems so you update your Gemfile to look like this:
# frozen_string_literal: true
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gem "puma", "~> 6.0.0"
gem "rails", "~> 7.0.4"
gem "sidekiq", "~> 7.0"
And then you run bundle install and the gems update. Or maybe you don't change the versions, but some day run bundle update which uses the ~> versioning operator and updates the gems to newer versions.
Here's where you're going to start running into compatibility problems.
First problem:
When you installed the sidekiq and puma and rails gems to run their scripts like pumactl they were installed using gem install ... which installed them globally and with a specific version.
When you added them to your Rails app and updated the versions they were installed separately by bundler with specific versions that are noted in Gemfile.lock.
Now your global version of puma is 5.6.2 and your bundled version of puma is 6.0.0.
Trying to manage puma using an old version of the CLI with a new version of the gem isn't guaranteed to work and can introduce hard-to-pinpoint problems. The same is true of the rails and sidekiq gems and any gem with a CLI.
Second problem:
When you run scripts like pumactl they aren't necessarily going to look at your application's Gemfile.lock and they aren't guaranteed to see or respect bundler's configuration for your Rails app when it loads it.
When you run scripts prefixed with bundle exec (like bundle exec sidekiq) it uses bundler to look at your bundled environment and ensure that all the dependencies are properly loaded.
Trying to run a bundled application without bundle exec can introduce hard-to-pinpoint problems. The same is true for any gems that have CLI tools.
Short answer
Always use bundle exec ... to run gem CLIs in your app, whether it's bundle exec rails server or bundle exec puma or bundle exec sidekiq. This will ensure that your app is started or managed using the bundled gem rather than the global version.
If you see errors when starting your app using bundle exec ... then pay attention to them because they are indicative of actual problems that need to be addressed. Likewise, if you do see errors with bundle exec but don't see errors when starting your app using globally installed gems then pay attention to them because it means your app is not portable -- it's likely that it is papering over bugs to make the app run and that your app will not run on another computer.
Extended answer
pumactl start gives you an error -- probably because you aren't using bundle exec.
bundle exec pumactl start gives you a different error -- possibly because you're bypassing the standard way to start Rails; pumactl will read configu.ru and config/puma.rb and decide how it wants to start Rails. Use bundle exec rails server instead.
rails s doesn't load your sidekiq worker -- because you aren't using bundle exec rails s it likely can't see the things it's supposed to see to start properly, because it isn't using your bundled app config
Because the errors that you're reporting are due to a misconfiguration of your system and app I can't give you any more detailed answers. You need to fix your configuration first and determine which of the three different errors you're experiencing is valid. It's a lot of work to try to work through all three questions. A standard "vote to close" reason for questions is:
Needs more focus
This question currently includes multiple questions in one. It should focus on one problem only.
I'm not voting to close your question but I'm mentioning it in case it does get closed later.
I recommend that after you fix the misconfiguration you create a new post about that specific error with a minimal reproducible example.

Bundler using binstub generated for a different gem using Middleman

I am using Middleman to build a project. I receive this message any time I run a Middleman command:
Bundler is using a binstub that was created for a different gem.
This is deprecated, in future versions you may need to `bundle binstub middleman-core` to work around a system/bundle conflict.
When I run bundle binstub middleman-core, I get this:
middleman-core has no executables, but you may want one from a gem it depends on.
bundler has: bundle, bundler
rack has: rackup
tilt has: tilt
erubis has: erubis
listen has: listen
sass has: sass, sass-convert, scss
Don't really know where to go and what to do from that message.
It is not causing the anything to fail and the server runs, but I feel like this could be a bigger issue if I leave it unfixed. This ended up happening when I was playing with s3_sync to push this up to s3 bucket and I gem installed middleman-sync_s3.
I have tried research and others led me through the path of deleting the bin/* file multiple times. I've tried updating the bin also and neither helps.
Any help is appreciated.
So I was hopping around the Gemfile trying to figure out what was going on. I had built a few previous projects in middleman and decided to look them up. I saw that I was using a previous version of Middleman 3.1.0 where as with this current project I was using Middleman 4.0.0
I reverted back to 3.1.0 and ran a bundle update. Tried running a Middleman command and the binstubs message no longer appears.
Ultimately, I think it had something with the way bundler plays with middleman-core.
gem install middleman-cli seems to help in case someone else is looking for a solution to this.

Bundler including gem in ignored group

When I Bundle.require(:default) I'm getting an error saying that one of the gems in my test group isn't found. I installed this bundle --without test so this makes sense, but shouldn't all the gems in the :test group be ignored here? Isn't that the point?
Thoughts on what might be wrong with this environment?
EDIT:
It may have something to do with my setup for this projects. I'm caching the gems in vendor/cache with bundle package --all. My cache has gems from git, file, and traditional sources. Bundler is reporting
some gems appear to be missing from your vendor/cache -> could not
find gem factory_girl which is required by
try editing the gemfile so that it fg is like this:
group :development do
gem 'factory_girl'
end
Then it won't look for it in the testing group. Or, I can't tell which group you actually want it in, but my point is limit it down to the group you want.
Also, try running Bundler.require(:default, Rails.env) instead of just (:default). That would put it back to the standard Rails requirements.
Finally, I figure you're trying to install the gems for development(or production?) have you tried running bundle install --without test production to make sure that nothing is coming in from the other group you aren't focused on?
Edit:
Last try before I'm out of ideas. From the documentation it seems like you can specify the load path as well as requiring them. Try setting Bundler.setup as well? I don't have a rails environment on this machine so I can't test.
Bundle.setup(:production)
Bundle.require(:production)
or
Bundle.setup(:production, :default)
Bundle.require(:production, :default)

bundler ruby process burns at 99% CPU on Mac when upgrading from Rails 3.0.9 to 3.1.0

I have tried this a couple of times now. I use rvm and the ruby I'm using is ree 1.8.7. Running "bundle update" after changing my Gemfile hangs the processor at almost 100% CPU. It has been running for over an hour. Is there something special I need to do?
I figured out how to debug this and was thus able to resolve my issues.
Short version (based on my admittedly superficial knowledge of bundler):
bundle update or bundle install both look at your Gemfile and then try to resolve dependencies for the specified gems. This is the step that's causing your CPU to burn, most likely (it should be after it prints Fetching source index for http://rubygems.org/)
What I've run into is that sometimes Bundler gets stuck in an infinite loop (or, at least longer than I've waited) trying to resolve conflicting requirements. In my case, it was that two different gems both required a third gem with differing version requirements.
For some reason, bundler was getting caught in an unending loop (or in some very, very long loop) trying to resolve dependencies.
I basically found this issue on github: https://github.com/carlhuda/bundler/issues/1450
which led me to try this command:
DEBUG_RESOLVER=1 bundle install
Running that produced enough output for me to identify the gem dependency that was confusing bundler. In my case, it was two different gems requiring different versions of the builder gem.
I fixed it by specifying the version of builder that would work for both gems:
gem 'builder', '~> 3.0.0'
That sorted it out, and the next time I ran install or update, it completed in a reasonable time.
I hope that helps you figure out your issue.

modifying a gem changes won't take effect

I am trying to modify the albino gem slightly but any change I make to it does not take effect including ones that should break the gem like dropping everything in the python script https://github.com/austinbv/albino/blob/master/vendor/multipygmentize to one line.
The gem was installed with bundler, and I am using rvm. The gem is stored in ~/.rvm/gems/ruby_version#project I tried editing what I wanted directly, and I tried using
EDITOR=vim bundle open albino and and editing what I wanted. Both of these did not work.
I cannot make any changes to the gem or even break it.
How do you modify a gem so that the changes will take effect.
rails reload only application's classes, not gems dependencies. if you use passenger in development, use "touch tmp/always_restart.txt", else restart application every time you do any changes in gem
The only answer I could find is to uninstall and reinstall the gem I modified it. Not an idea solution but worked for the time being.

Resources