FreeBSD Could not find gem in any of the sources - ruby

I am trying to bundle on FreeBSD and it cannot find the grpc gem. I installed the grpc 1.30.2 version using a FreeBSD package but it cannot find the system gem.
I'm hoping there is a bundle config setting to find and use the system gem which is already built and intalled.
The error I get is:
An error occurred while installing grpc (1.30.2), and Bundler cannot continue.
Make sure that `gem install grpc -v '1.30.2' --source 'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
grpc
* bundle install
Warning: the running version of Bundler (2.0.2) is older than the version that created the lockfile (2.2.11). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.
/usr/local/lib/ruby/site_ruby/2.7/rubygems/package.rb:509: warning: Using the last argument as keyword parameters is deprecated
/usr/local/lib/ruby/site_ruby/2.7/rubygems/package.rb:509: warning: Using the last argument as keyword parameters is deprecated
/usr/local/lib/ruby/site_ruby/2.7/rubygems/package.rb:509: warning: Using the last argument as keyword parameters is deprecated
/usr/local/lib/ruby/site_ruby/2.7/rubygems/package.rb:509: warning: Using the last argument as keyword parameters is deprecated
Some gems seem to be missing from your vendor/cache directory.
Could not find grpc-1.30.2 in any of the sources
* Visruleparser
A extras/visruleparser.pl
Export complete.
* precompile assets
Could not find grpc-1.30.2 in any of the sources
Run `bundle install` to install missing gems.
My Gemfile has these lines
source 'https://rubygems.org'
gem 'grpc', '1.30.2'
gem 'grpc-tools', '1.30.2'
The Gemfile.lock has these
grpc (= 1.30.2)
grpc-tools (= 1.30.2)
These FreeBSD packages are installed
>pkg info -x grpc
grpc-1.22.0_4,2
rubygem-grpc-1.30.2
>gem list grpc
*** LOCAL GEMS ***
grpc (1.30.2)
Listing remote gems shows nothing on FreeBSD
>gem list --remote -all gprc
*** LOCAL GEMS ***
But on macOS I get these (among others)
>gem list --remote --all grpc
grpc (1.37.0 ruby universal-darwin x64-mingw32 x86-linux x86-mingw32 x86_64-linux,
1.36.0 ruby universal-darwin x64-mingw32 x86-linux x86-mingw32 x86_64-linux,
...
1.30.2 ruby universal-darwin x64-mingw32 x86-linux x86-mingw32 x86_64-linux,
...
The gem is installed in the directory /usr/local/lib/ruby/gems/2.7/gems/grpc-1.30.2 with a src subdirectory and a ruby subdirectory below that.
>ls /usr/local/lib/ruby/gems/2.7/gems/grpc-1.30.2/src/ruby
bin ext lib pb spec
Environment
FreeBSD 11.4
Ruby 2.7.2p137
RubyGems 3.0.8
Bundler 2.1.4

Looks like this may be my fault. The gem packages purposefully remove the cache dir from the gem. This was originally added here:
https://cgit.freebsd.org/ports/commit/Mk/bsd.ruby.mk?id=8d77480356f58d6d8d1c69884d1ba9cc98ed383a
and lives on here:
https://cgit.freebsd.org/ports/tree/Mk/Uses/gem.mk#n111
${RM} -r ${STAGEDIR}${PREFIX}/${CACHE_DIR} 2> /dev/null || ${TRUE}
This was added because so many gems include large amounts of stuff that is never used in the cache dir, including complete copies of source, intermediate build output, etc. that shouldn't really be packaged.
If you're building your own packages you can try the line from gem.mk that removes the cache dir. Otherwise, we may need to revisit this upstream. Regardless, you should probably open a bug upstream at:
https://bugs.freebsd.org/bugzilla/
But there are tradeoffs and I'm not sure about the use case of bundling gems installed from system packages. (I haven't been using Ruby as much recently.)

Related

Rails bundle install doesn't work 'cause json version error

I just wanted to convert from ImageMagick v7 to ImageMagick v6.
while doing that, this error was happened.
bundle install doesn't work correctly.
It seems like json version is something wrong.
How do I fix this error?
Environment below
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-darwin17]
Rails -v (couldn't find gem) (becuase bundle install can't work)
json list / json (default: 1.8.3)
multi_json (1.13.1, 1.11.2, 1.11.0)
Bundler version 1.16.4
Mac Mojave 10.14
when I do bundle install on terminal, then I got this error
Fetching gem metadata from https://rubygems.org/........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies.......
Using rake 12.3.2
Using concurrent-ruby 1.1.4
Using i18n 0.9.5
Fetching json 1.8.6
Installing json 1.8.6 with native extensions
Errno::EPERM: Operation not permitted # chmod_internal -
/Users/***/projects/***/vendor/bundle/ruby/2.3.0/gems/json-1.8.6/tests/test_json.rb
An error occurred while installing json (1.8.6), and Bundler cannot continue.
Make sure that `gem install json -v '1.8.6' --source
'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
rails was resolved to 4.2.6, which depends on
actionmailer was resolved to 4.2.6, which depends on
actionpack was resolved to 4.2.6, which depends on
actionview was resolved to 4.2.6, which depends on
rails-dom-testing was resolved to 1.0.9, which depends on
rails-deprecated_sanitizer was resolved to 1.0.3, which depends on
activesupport was resolved to 4.2.6, which depends on
json
I tried to do like this because error statement says Make sure that
gem install json -v '1.8.6' --source 'https://rubygems.org/' succeeds before bundling
but result shows like this
ERROR: While executing gem ... (Errno::EPERM)
Operation not permitted # chmod_internal - /Users/***/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/json-1.8.6/tests/test_json.rb
I can't understand this error statement
Errno::EPERM: Operation not permitted # chmod_internal -
Also, I goggled a lot, then I update commandlinetool follow this
https://howchoo.com/g/m2u0mmuwzda/macos-mojave-fix-invalid-active-developer-path
then, I did this command again,
sudo gem install json -v '1.8.6' --source 'https://rubygems.org/'
then, it's completely succeded like this.
Building native extensions. This could take a while...
Successfully installed json-1.8.6
Parsing documentation for json-1.8.6
Installing ri documentation for json-1.8.6
Done installing documentation for json after 1 seconds
1 gem installed
But, if I do bundle install, still doesn't work: they show same error.
My recommendation is if possible to start fresh. Maybe your rbenv was not installed correctly, try reinstalling it, I suggest using brew install rbenv https://github.com/rbenv/rbenv#homebrew-on-macos
Make sure to add the eval "$(rbenv init -)" to your ~/.bash_profile then open a new terminal.
Navigate to the project directory and install the Ruby version you need: rbenv install 2.3.1
You can make sure you are using that version by issuing rbenv use 2.3.1 and ruby --version.
Now install bundler for that Ruby version, I suggest 1.17.3 for now (the latest before 2.0.1) gem install bundler -v '1.17.3'.
You should be ready to bundle install.
The most common scenario I see for a message Installing ... with native extensions to result in errors is usually due to the lack of binaries, header files and C related code to build that native extension, in your case the issue is due to permissions, hence why I am suggesting some fresh installation.
I second #Danilo Cabello's recommendation to start fresh if you can. I just have a few other trouble-shooting ideas:
The fact that your bundle is installing gems in /vendor/bundle means that at some point, you must have specified the path with bundle install --path vendor/bundle as #mogbee alludes to. That path flag will load files associated with your gems into vendor/bundle instead of your system gem location. You might have done that if you're trying to keep the project's gems separate from any other project, but if not, you will need to update your bundler's gem path.
To do this, first, check for any issues by running bundle doctor. If no issues are found, check your bundle configuration with bundle env. Make sure that your RubyGems Gem Home and Gem Path are routed through .rbenv, so they should match and look like /Users/***/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0. If they don't match, run the rbenv-doctor command below which should make sure your rbenv installation exported the path properly.
If the output of bundle env tells you that you're actually running an older version of bundler (older than 1.16.4), I would definitely update, and would recommend version 1.17.3 as #Danilo Cabello did.
Second, I would run this rbenv-doctor curl command to check the status of your rbenv install: curl -fsSL https://github.com/rbenv/rbenv-installer/raw/master/bin/rbenv-doctor | bash (https://github.com/rbenv/rbenv-installer#rbenv-doctor).
Third, depending on the output of ls -l within your project folder, you may also want to recursively change owner/group for your ~/.rbenv folder to make sure everything is owned by you and not root (https://superuser.com/questions/260925/how-can-i-make-chown-work-recursively/260939#260939)

bundle install using wrong ruby with rvm when building native extension?

I'm using bundle install to install rest-client. It's failing with the following:
$ bundle
Fetching gem metadata from https://rubygems.org/..........
Fetching version metadata from https://rubygems.org/..
Resolving dependencies...
Installing unf_ext 0.0.7.4 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/roger/.rvm/gems/ruby-2.3.4#junk-drawer/gems/unf_ext-0.0.7.4/ext/unf_ext
/usr/bin/ruby2.3 -r ./siteconf20180115-9722-1iow75n.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h
extconf failed, exit code 1
Gem files will remain installed in /home/roger/.rvm/gems/ruby-2.3.4#junk-drawer/gems/unf_ext-0.0.7.4 for inspection.
Results logged to /home/roger/.rvm/gems/ruby-2.3.4#junk-drawer/extensions/x86_64-linux/2.3.0/unf_ext-0.0.7.4/gem_make.out
Installing mime-types-data 3.2016.0521
Installing netrc 0.11.0
Using bundler 1.11.2
An error occurred while installing unf_ext (0.0.7.4), and Bundler cannot continue.
Make sure that `gem install unf_ext -v '0.0.7.4'` succeeds before bundling.
I'm using rvm, which appears to be working correctly, if I'm reading the output from rvm info right.
which ruby reports /home/roger/.rvm/rubies/ruby-2.3.4/bin/ruby, which looks right.
I found Install ruby headers with rvm, but I have Ruby headers in (apparently) the correct place:
$ ruby -rmkmf -e 'print RbConfig::CONFIG["rubyhdrdir"]'
/home/roger/.rvm/rubies/ruby-2.3.4/include/ruby-2.3.0
$ ls /home/roger/.rvm/rubies/ruby-2.3.4/include/ruby-2.3.0
ruby/ ruby.h x86_64-linux/
In the output from bundle, I see /usr/bin/ruby2.3, which looks suspicious.
It's looking in the wrong place for Ruby header files. How do I get it to use the RVM-installed ones?
I was using the wrong bundle.
$ which bundle
/usr/bin/bundle
Either the one included in Ubuntu 16.04 is old, or using the system-installed bundler doesn't work with rvm-installed Ruby.
$ gem install bundler
Fetching: bundler-1.16.1.gem (100%)
...
1 gem installed
$ which bundle
/home/roger/.rvm/gems/ruby-2.3.4#junk-drawer/bin/bundle
That looks better.
$ bundle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using bundler 1.16.1
...
Bundle complete! 1 Gemfile dependency, 9 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Lesson learned: when using rvm, remember to gem install bundler as well.

Bundler installing gem that's already installed

I believe I'm misunderstanding the way bundler works, but from the bundle install documentation it seems to indicate bundler will use locally installed system gems.
...
--system: Install to the system location ($BUNDLE_PATH or $GEM_HOME) even if the bundle was previously installed somewhere else for this application
...
The --system option is the default. Pass it to switch back after using the --path option as described below.
I'm not using rbenv/rvm or any other Ruby version manager. I'm using ChefDK as my primary development environment, which ships with Ruby and a bunch of preinstalled gems.
The full contents of the Gemfile, there is no Gemfile.lock yet.
source 'https://rubygems.org'
gem 'nokogiri', '1.6.3.1'
Local nokogiri installed
$ gem list --local | grep nokogiri
nokogiri (1.6.6.2, 1.6.3.1, 1.5.5)
System Gem location has nokogiri 1.6.3.1 installed
$ echo $GEM_HOME
/Users/arthur/.chefdk/gem/ruby/2.1.0
$ find /Users/arthur/.chefdk/gem/ruby/2.1.0 | grep nokogiri | grep 1.6.3.1
/Users/arthur/.chefdk/gem/ruby/2.1.0/cache/nokogiri-1.6.3.1.gem
/Users/arthur/.chefdk/gem/ruby/2.1.0/extensions/x86_64-darwin-12/2.1.0/nokogiri-1.6.3.1
/Users/arthur/.chefdk/gem/ruby/2.1.0/extensions/x86_64-darwin-12/2.1.0/nokogiri-1.6.3.1/mkmf.log
/Users/arthur/.chefdk/gem/ruby/2.1.0/gems/nokogiri-1.6.3.1
/Users/arthur/.chefdk/gem/ruby/2.1.0/gems/nokogiri-1.6.3.1/.autotest
/Users/arthur/.chefdk/gem/ruby/2.1.0/gems/nokogiri-1.6.3.1/.editorconfig
...
However, when I run a bundle install, it tries to install and compile libxml2 for nokogiri.
$ bundle install
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using mini_portile 0.6.0
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
- 0001-Fix-parser-local-buffers-size-problems.patch
- 0002-Fix-entities-local-buffers-size-problems.patch
- 0003-Fix-an-error-in-previous-commit.patch
- 0004-Fix-potential-out-of-bound-access.patch
- 0005-Detect-excessive-entities-expansion-upon-replacement.patch
- 0006-Do-not-fetch-external-parsed-entities.patch
- 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
- 0008-Improve-handling-of-xmlStopParser.patch
- 0009-Fix-a-couple-of-return-without-value.patch
- 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
- 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT! Nokogiri builds and uses a packaged version of libxml2.
...
What am I missing? How can I force bundler to use the already installed nokogiri 1.6.3.1 (that ships with ChefDK)? I'm trying to avoid having nokogiri compile libxml2 because that fails consistently on many different developer/operations workstations and has caused no end of grief. Thanks.
Edit
Thanks to Tim Moore, using bundle env I noticed in the output that bundler had shared gems disabled.
$ bundle env
Bundler 1.7.12
Ruby 2.1.4 (2014-10-27 patchlevel 265) [x86_64-darwin12.0]
Rubygems 2.4.4
GEM_HOME /Users/arthur/.chefdk/gem/ruby/2.1.0
GEM_PATH /Users/arthur/.chefdk/gem/ruby/2.1.0:/opt/chefdk/embedded/lib/ruby/gems/2.1.0
Bundler settings
disable_shared_gems
Set for the current user (/Users/arthur/.bundle/config): "1"
Gemfile
source 'https://rubygems.org'
...
Looking at the ~/.bundle/config, sure enough the global config was set.
---
BUNDLE_DISABLE_SHARED_GEMS: '1'
Once removed, Bundler resolves nokogiri 1.6.3.1 correctly and doesn't try reinstalling it. This setting should not be there by default, by default bundler installs with --system. I must have set this setting many months back and forgot I did.
Try running bundle env to verify that the install location is what you expect.
If not, check whether there is a .bundle/config or ~/.bundle/config file overriding the install path. The output of bundle env will tell you what configuration it is using and how it was determined (i.e., which file it was in or whether it was picked up from an environment variable).
Try removing all contents of gemfile.lock file. Save the file. Run bundle install again.
There are a couple of methods. In gemfile you can specify the path which will force bundle to use from there.
gem "my_gem", :path => "path to gem"
As i see the issue is with the default paths here. Try doing this.
ChefDK doesn't install gems globally, it installs them under /opt/chefdk so they won't interfere with "your" global gems. I suggest you leave ChefDK gems isolated as they should be.
You need to use the proper bundler and gem. If you're using ChefDK, then it includes its own bundler and gem executables. They should be inside the /opt/chefdk directory, I believe under /opt/chefdk/embedded (I don't use chefdk, so I can be 100% sure of that).
To work 100% inside that ruby install, you need to ensure that the chefdk binaries are in your path before the other ruby related binaries. You can verify that with which ruby which gem and which bundle.
All that said, you really SHOULDN'T be messing with the ruby install for chefdk. It's embedded for a reason, so that you don't accidentally mess it up. I'd suggest you stick with the system ruby for your own work, and let Chef handle its ruby.
From Bundler docs:
--path: Specify a different path than the system default ($BUNDLE_PATH or $GEM_HOME). Bundler will remember this value for future installs on
this machine
Your bundler may have cached a --path specified install command.
Try:
bundle install --system
This will tell bundler to use the system installed gems as opposed to downloading new gem copies to a folder specific gem collection.

How do you precompile the native extensions for a ruby gem for linux?

We have a ruby application that depends on a gem with native extensions (in this specific case Nokogiri). However, for various reasons we cannot install the build prerequisites (such as build-essential, libxslt-dev, ruby-dev, etc) for that gem onto our production host.
Is there a (standard?) way to repackage the gem with the native extensions pre-built?
It should be possible (it seems to be fairly standard to do this for Windows), but I can't find any documentation on the subject.
Note that we only need to support a single platform, with known versions of all system libraries (Ubuntu 9.04 Server 64 bit, Ruby 1.8.7).
UPDATE:
We're using Bundler, so we want to still have a gem to install at the end of the day, not a debian package.
Finally found a way to do this for gems that use rake-compiler for building their C extensions (which is most of them).
You need to do the following on a machine that is identical to the one you want to deploy to, or it simply won't work:
Install the build prerequisites for building C extensions:
# apt-get install build-essentials ruby-dev # ... etc
# gem install rake-compiler
Unpack the gem you want to rebuild:
$ gem unpack nokogiri
Build your shiny new precompiled gem:
$ rake native gem
You can now install the native gem on a machine without any build tools installed:
$ gem install pkg/nokogiri-1.4.3.1-x86-linux.gem
Successfully installed nokogiri-1.4.3.1-x86-linux
1 gem installed
Build machine
To have the correctly platformed version for all of your gems already cached,
package all gems in vendor/cache on an identical machine:
$ bundle package --all
Prduction machine
Install all gem dependecies already cached that requires to have the correctly
platformed version using the gem cache:
$ bundle install --local --deployment

sqlite3-ruby gem: Failed to build gem native extension

Update: Check out this follow-up question: Gem Update on Windows - is it broken?
On Windows, when I do this:
gem install sqlite3-ruby
I get the following error:
Building native extensions. This could take a while...
ERROR: Error installing sqlite3-ruby:
ERROR: Failed to build gem native extension.
c:/ruby/bin/ruby.exe extconf.rb install sqlite3-ruby --platform Win32
checking for fdatasync() in rt.lib... no
checking for sqlite3.h... no
nmake
'nmake' is not recognized as an internal or external command,
operable program or batch file.
Gem files will remain installed in c:/ruby/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.4 for inspection.
Results logged to c:/ruby/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.4/ext/sqlite3_api/gem_make.out
Same thing happens with the hpricot gem. I seem to remember these gems installed just fine on < 1.0 gems, but now I'm on 1.2.0, things have gone screwy.
I have also tried this:
gem install sqlite3-ruby --platform Win32
Needless to say, this doesn't work either (same error)
Does anyone know what is going on here and how to fix this?
Update: Check out this follow-up question: Gem Update on Windows - is it broken?
As Nathan suggests, this does appear to be related to the fact that the latest versions of the sqlite3-ruby and hpricot gems don't appear to have Windows versions. Here's what to do when faced with this situation (note, the name of the gem is automatically wildcarded, so you can type just sql and get a list of all gems beginning with sql):
$ gem list --remote --all sqlite
*** REMOTE GEMS ***
sqlite (2.0.1, 2.0.0, 1.3.1, 1.3.0, 1.2.9.1, 1.2.0, 1.1.3, 1.1.2, 1.1.1, 1.1)
sqlite-ruby (2.2.3, 2.2.2, 2.2.1, 2.2.0, 2.1.0, 2.0.3, 2.0.2)
sqlite3-ruby (1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0, 1.1.0, 1.0.1, 1.0.0, 0.9.0, 0.6.0, 0.5.0)
Then you can choose the version you would like to install:
gem install sqlite3-ruby -v 1.2.3
To successfully install hpricot, I did this:
gem install hpricot -v 0.6
Annoyingly, doing a gem update tries to update the gems to their latest, broken-on-Windows, versions. When the update routine encounters an error, it ditches you out of the whole process. There's a (hacky) solution to this problem here.
So, is this issue a bug in gems? Should gems not automatically detect the platform and install the latest compatible version?
I had the same problem on Ubuntu, this solved the problem for me:
http://newsgroups.derkeiler.com/Archive/Comp/comp.lang.ruby/2008-08/msg00339.html
first from sqlite.org(http://www.sqlite.org/download.html)
download ->
Precompiled Binaries: sqlite-dll-win32-x86-3071700.zip
and Source Code: sqlite-autoconf-3071700.tar.gz
then extract as:
-include
--sqlite3.h
--sqlite3ext.h
-lib
--shell.c
--sqlite3.c
--sqlite3.def
--sqlite3.dll
last install gem like:
gem install sqlite3 --platform=ruby -- --with-sqlite3-include=path\to\include
--with-sqlite3-lib=path\to\lib --no-ri --no-rdoc
Good luck!
Is it possible that the newest version has not yet been ported to Win32 yet? Since this particular gem does have bindings to compiled code, it would require a platform-specific gem. If I force installation of version 1.2.3 rather than the current 1.2.4, the platform-specific version does install, but when I allow gem to try to install the current version, I get the generic 1.2.4 version (in the gems library folder, it lacks the -x86-mswin32 suffix that the other sqlite3-ruby folders have in their names.
Maybe someone else can answer how gem handles platform specific gems. Are separate gems uploaded for each platform and the gem software selects which one to pull down?
I also ran into this problem. It's worth knowing that the difference between 1.2.3 and 1.2.4 is not significant. Here are the 1.2.4. release notes:
Release Name: 1.2.4
Notes: This release only updates the
generated C file to reflect the
compatibility changes that were made
to the SWIG file. Binary builds (e.g.,
Windows) are not affected, and need no
update. In general, you will not need
this update unless you are using a
version of Ruby prior to 1.8.6.
(source: 1.2.4. release notes)
Hope that helps others!
I had the same problem on Windows and I have installe MinGW
http://sourceforge.net/projects/mingw/files/Automated%20MinGW%20Installer/MinGW%205.1.6/MinGW-5.1.6.exe/download
and the problem has gone :-)
C:>gem install hpricot
Successfully installed hpricot-0.8.2-x86-mswin32
1 gem installed
Installing ri documentation for hpricot-0.8.2-x86-mswin32...
Installing RDoc documentation for hpricot-0.8.2-x86-mswin32...
C:>gem install ruby-postgres
Successfully installed ruby-postgres-0.7.1.2006.04.06-x86-mswin32
1 gem installed
Installing ri documentation for ruby-postgres-0.7.1.2006.04.06-x86-mswin32...
Installing RDoc documentation for ruby-postgres-0.7.1.2006.04.06-x86-mswin32...

Resources