I have a ruby gem that adds some default rake tasks. I also have some basic integration tests checking the output of rake -T and rake -AT for each of these tasks.
In order to farm out these tasks from some Ruby source code, I chose to use PTY.spawn and captured the stdout and stderr output to streams.
As you might see here on the Travis build:
Failure/Error: Paint.unpaint(out.read)
Errno::EIO:
Input/output error # io_fread - /dev/pts/1
# ./spec/spec_helper.rb:184:in `read'
# ./spec/spec_helper.rb:184:in `run_command'
# ./spec/rake/test_spec.rb:4:in `block (2 levels) in <top (required)>'
This is one of the first times I've used the Travis CI runner, so I tried manually requiring it to install libreadline too. I've updated bundler, rubygems, and have run this on multiple versions of ruby with the exact same error.
Per the constraints of one of my dependencies, I tested against MRI 2.2, 2.3, and 2.4. All of them failed the same way on travis, but also tried them on my local machine (MacOS) using chruby with no problem.
What am I missing?
Related
We found interesting problem. Our environment is configured by using ansible, which in turn installs gems.
Some of the gems, we want version that is newer than something. For example, aws-sdk-core version >= 3.104.
This ansible tasks runs:
gem install -v '>= 3.104' aws-sdk-core
Then, we have a cronjob that every 5 minutes (but across couple of thousand servers) runs a script that does 'require aws-sdk-core'.
And, every so often, it breaks with:
/var/lib/gems/2.5.0/gems/aws-sdk-core-3.166.0/lib/seahorse.rb:3:in `require_relative': cannot load such file -- /var/lib/gems/2.5.0/gems/aws-sdk-core-3.166.0/lib/seahorse/util (LoadError)
...
I made trivial script that shows the problem on another, much smaller gem:
#!/usr/bin/env ruby
# frozen_string_literal: true
require 'progressbar'
puts 1
If you'll save it as z.rb, and then run in shell: while true; do ./z.rb; done, and then in another shell: while true; do gem install -v '>= 1.0.0' progressbar; done, eventually (after a minute or two) you will get, in the shell that runs z.rb:
1
1
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- progressbar (LoadError)
from <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'
from ./z.rb:3:in `<main>'
1
1
1
Is there any way to avoid this problem, other than begin/rescue and retry after 1 second sleep (which I can do, but it's OH SO UGLY)?
The problem, for us, is that we need to install with at least some specific version (if we'd provide version = SOMETHING, ansible avoids calling gem install altogether, but we want new releases installed too), and while the window for race condition is small, with many thousand servers, and cronjob that runs every 5 minutes, (ansible runs every 4 hours), we get ~ dozen mails per day with cronjob fails.
Not an answer maybe, but still... :) I think there is no better way than to synchronize the tasks somehow.
Rubygems installer seems to remove existing gem files before installing new ones (if the version to install exists I guess).
This is easy to confirm. For example, having 2 pry versions installed -- 0.14.0 and 0.14 1 - I do the following:
Run while true; do gem install --no-document -f -v '0.14.1' pry; done
In another shell run while true; do gem which pry; done
The result of 2 looks like:
...
/path/to/ruby/lib/ruby/gems/2.7.0/gems/pry-0.14.1/lib/pry.rb
/path/to/ruby/lib/ruby/gems/2.7.0/gems/pry-0.14.1/lib/pry.rb
/path/to/ruby/lib/ruby/gems/2.7.0/gems/pry-0.14.0/lib/pry.rb <=== Notice this one
/path/to/ruby/lib/ruby/gems/2.7.0/gems/pry-0.14.1/lib/pry.rb
/path/to/ruby/lib/ruby/gems/2.7.0/gems/pry-0.14.1/lib/pry.rb
...
Traceback (most recent call last):
9: from /path/to/ruby/bin/gem:9:in `<main>'
8: from /path/to/ruby/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
7: from /path/to/ruby/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
6: from /path/to/ruby/lib/ruby/2.7.0/rubygems/gem_runner.rb:86:in `<top (required)>'
5: from /path/to/ruby/lib/ruby/2.7.0/rubygems.rb:1131:in `load_plugins'
4: from /path/to/ruby/lib/ruby/2.7.0/rubygems.rb:538:in `find_latest_files'
3: from /path/to/ruby/lib/ruby/2.7.0/rubygems/specification.rb:1086:in `latest_specs'
2: from /path/to/ruby/lib/ruby/2.7.0/rubygems/specification.rb:1093:in `_latest_specs'
1: from /path/to/ruby/lib/ruby/2.7.0/rubygems/specification.rb:1093:in `reverse_each'
/path/to/ruby/lib/ruby/2.7.0/rubygems/specification.rb:1096:in `block in _latest_specs': undefined method `platform' for nil:NilClass (NoMethodError)
...
/path/to/ruby/lib/ruby/gems/2.7.0/gems/pry-0.14.1/lib/pry.rb
/path/to/ruby/lib/ruby/gems/2.7.0/gems/pry-0.14.1/lib/pry.rb
...
ERROR: While executing gem ... (Errno::ENOENT)
No such file or directory # rb_sysopen - /path/to/ruby/lib/ruby/gems/2.7.0/specifications/pry-0.14.1.gemspec
...
so pretty much as one would expect - all sorts of surprises (from resolving the gem to a previous version to several exceptions happening because we caught the gem installation process in the intermediate incomplete state).
Besides retrying that you mentioned one could also do smth like:
maintain some external lock that would be raised when gem installation starts and released when it is finished; the corn jobs check it and shut down gracefully if the installation is in progress (seems unnecessarily complicated for the case though, but still)
check the gem state from the cron job using Gem API, and shut down if the required gem doesn't exist
rescue the LoadError and shut down gracefully
etc etc etc.
While talking with others we figured a solution. It seems that adding these two options: --conservative --minimal-deps to gem install solves the problem.
I would like to do the testfirst.org exercises although can't seem to get to the starting line due to a version issue/lack of knowledge on my part as well..
At the begining you have to run the test $rake
i got following error..
(in /home/arno/learn_ruby)
rake aborted!
Gem::MissingSpecError: Could not find 'rspec' (~> 2) among 94 total
gem(s)
Checked in
'GEM_PATH=/home/arno/.gem/ruby/2.4.0:/home/arno/.rbenv/
versions/2.4.0/lib/ruby/gems/2.4.0', execute `gem env` for more
information
/home/arno/learn_ruby/Rakefile:2:in `<top (required)>'
(See full trace by running task with --trace)
I then tried to uninstall rspec. I then tried to install a version less than 3. That also did not work. Now I have version 2.9 and 3.2 installed but the same error persisted..
I then did bundle install and bundle update...something changed although still not working..
Here is the latest error when i run the test $rake..
(in /home/arno/learn_ruby)
rake aborted!
NoMethodError: undefined method `last_comment' for #
<Rake::Application:0x00564ff13bf4b0>
/home/arno/learn_ruby/Rakefile:8:in `new'
/home/arno/learn_ruby/Rakefile:8:in `<top (required)>'
(See full trace by running task with --trace)
Here is the rakefile..
# This Rakefile has all the right settings to run the tests inside
each lab
gem 'rspec', '~>2'
require 'rspec/core/rake_task'
task :default => :spec
desc "run tests for this lab"
RSpec::Core::RakeTask.new do |task|
lab = Rake.application.original_dir
task.pattern = "#{lab}/*_spec.rb"
task.rspec_opts = [ "-I#{lab}", "-I#{lab}/solution", '-f
documentation', '-r ./rspec_config']
task.verbose = false
end
After 2 days of searching i found a solution from The Odin Project.
They have a Test First - Rspec 3 edition..
If your having the same issue go to their repo https://github.com/TheOdinProject/learn_ruby.
Read through the readme file, fork that repo, clone your version of the repo, cd into it and bundle install..
All works fine now..
I'm creating a gem (see https://github.com/hamchapman/rewinder) that has ruby-git (https://github.com/schacon/ruby-git) as a dependency.
Part of my gem has an executable https://github.com/hamchapman/rewinder/blob/master/bin/rewinder
which basically just requires the main lib file https://github.com/hamchapman/rewinder/blob/master/lib/rewinder.rb
and then runs the method there (all messy at the moment - ignore that for now).
When I install the gem locally and use it in another repo I get the following error:
/Users/Hami/.rvm/gems/ruby-2.1.2/gems/git-1.2.7/lib/git/lib.rb:764:in `command': git checkout '6eef72baf24bed761f753267cce16402e4a947f8' 2>&1:Note: checking out '6eef72baf24bed761f753267cce16402e4a947f8'. (Git::GitExecuteError)
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
HEAD is now at 6eef72b... Trying another bg-color change for homeboy
/Users/Hami/.rvm/gems/ruby-2.1.2#global/gems/bundler-1.6.2/lib/bundler/rubygems_integration.rb:252:in `block in replace_gem': hookup is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
from /Users/Hami/.rvm/gems/ruby-2.1.2/bin/hookup:22:in `<main>'
from /Users/Hami/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `eval'
from /Users/Hami/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `<main>'
from /Users/Hami/.rvm/gems/ruby-2.1.2/gems/git-1.2.7/lib/git/lib.rb:528:in `checkout'
from /Users/Hami/.rvm/gems/ruby-2.1.2/gems/git-1.2.7/lib/git/base.rb:325:in `checkout'
from /Users/Hami/.rvm/gems/ruby-2.1.2/gems/rewinder-0.0.1/lib/rewinder.rb:11:in `block in heloo'
from /Users/Hami/.rvm/gems/ruby-2.1.2/gems/rewinder-0.0.1/lib/rewinder.rb:10:in `each'
from /Users/Hami/.rvm/gems/ruby-2.1.2/gems/rewinder-0.0.1/lib/rewinder.rb:10:in `each_with_index'
from /Users/Hami/.rvm/gems/ruby-2.1.2/gems/rewinder-0.0.1/lib/rewinder.rb:10:in `heloo'
from /Users/Hami/.rvm/gems/ruby-2.1.2/gems/rewinder-0.0.1/bin/rewinder:4:in `<top (required)>'
from /Users/Hami/.rvm/gems/ruby-2.1.2/bin/rewinder:23:in `load'
from /Users/Hami/.rvm/gems/ruby-2.1.2/bin/rewinder:23:in `<main>'
from /Users/Hami/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `eval'
from /Users/Hami/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `<main>'
I don't really understand the error seeing as neither my gem, nor the ruby-git gem have hookup as a dependency. If I load up the file /Users/Hami/.rvm/gems/ruby-2.1.2/bin/hookup
then comment out the bottom two lines like this:
#!/usr/bin/env ruby_executable_hooks
#
# This file was generated by RubyGems.
#
# The application 'hookup' is installed as part of a gem, and
# this file is here to facilitate running it.
#
require 'rubygems'
version = ">= 0"
if ARGV.first
str = ARGV.first
str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding
if str =~ /\A_(.*)_\z/ and Gem::Version.correct?($1) then
version = $1
ARGV.shift
end
end
#gem 'hookup', version
#load Gem.bin_path('hookup', 'hookup', version)
then the error doesn't occur.
From what I've read it looks like it could be something to do with how oh-my-zsh wraps certain commands - maybe it's wrapping a git one?
Any suggestions?
From the hookup documentation:
Each time your current HEAD changes, hookup checks to see if your Gemfile, Gemfile.lock, or gem spec has changed. If so, it runs bundle check, and if that indicates any dependencies are unsatisfied, it runs bundle install.
From this I gather that the gem listens to git checkout events. Since you do that in your code, the hookup hooks come to life, and get confused (regardless of whether it is part of your ruby application or not).
Try uninstalling the gem and trying again.
If this resolves your problem, you might consider opening an issue to the author of the gem to fix this.
This works:
[rails31]$ ruby -S rspec ./spec/models/domain_spec.rb
*.
Pending:
Domain add some examples to (or delete) /home/keith/Code/elements2/spec/models/domain_spec.rb
# Not Yet Implemented
# ./spec/models/domain_spec.rb:4
Finished in 0.04241 seconds
2 examples, 0 failures, 1 pending
However I'm trying to run from guard, which executes the following command:
[rails31]$ /home/keith/.rbenv/versions/1.9.2-p290/bin/ruby -S rspec ./spec/models/domain_spec.rb
/home/keith/.rbenv/versions/1.9.2-p290/bin/ruby: no Ruby script found in input (LoadError)
Breaking that down enough to execute I get this:
[rails31]$ /home/keith/.rbenv/versions/1.9.2-p290/bin/ruby ./spec/models/domain_spec.rb
/home/keith/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/rspec-core-2.8.0.rc1/lib/rspec/core/configuration.rb:335:in `rescue in debug=': (RuntimeError)
**************************************************
no such file to load -- ruby-debug
If you have it installed as a ruby gem, then you need to either require
'rubygems' or configure the RUBYOPT environment variable with the value
'rubygems'.
...
This is actually where I started in the first place - I know rspec respects the "dont require rubygems" rule, so maybe I need to run rake. The big problem I have with this error is that "ruby-debug" does NOT exist for ruby1.9 - it should be ruby-debug19 - so whats happening here?
So anyway, I tried with Rake:
[rails31]$ rake spec ./spec/models/domain_spec.rb
(in /home/keith/Code/elements2)
rake aborted!
uninitialized constant Rake::DSL
...
I've tried Googling the problem but nothing obvious is coming up, so really I'm stumped
UPDATE:
Well after reading this post I've managed to resolve the rake issue, however now when I run rake I get this:
[rails31]$ rake spec
(in /home/keith/Code/elements2)
/home/keith/.rbenv/versions/1.9.2-p290/bin/ruby -S rspec ./spec/models/domain_spec.rb
/home/keith/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/rspec-core-2.8.0.rc1/lib/rspec/core/configuration.rb:335:in `rescue in debug=': (RuntimeError)
...
Have a strange problem in that I have a Rake task that seems to be using multiple versions of Ruby. When one fails, it seems to try another one.
Details
MacBook running 10.6.5
rvm 1.1.0
Rubies: 1.8.7-p302, ree-1.8.7-2010.02, ruby-1.9.2-p0
Rake 0.8.7
Gem 1.3.7
Veewee (provisioning Virtual Machines using Opcode.com, Vagrant and Chef)
I'm not entirely sure the specific details of the error matter, but since it might be an issue with Veewee itself. So, what I'm trying to do is build a new box base on a veewee definition. The command fails with an error about a missing method- but what's interesting is how it fails.
Errors
I managed to figure out that if I only have one Ruby installed with RVM, it just fails. If I have more than one Ruby install, it fails at the same place, but execution seems to continue in another interpreter.
Here are two different clipped console outputs. I've clipped them for size. The full outputs of each error are available as a gist.
One Ruby version installed
Here is the command run when I only have a single version of Ruby (1.8.7) available in RVM
boudica:veewee john$ rvm rake build['mettabox'] --trace
rvm 1.1.0 by Wayne E. Seguin (wayneeseguin#gmail.com) [http://rvm.beginrescueend.com/]
(in /Users/john/Work/veewee)
** Invoke build (first_time)
** Execute build
…
creating new harddrive
rake aborted!
undefined method `max_vdi_size' for #<VirtualBox::SystemProperties:0x102d6af80>
/Users/john/.rvm/gems/ruby-1.8.7-p302/gems/virtualbox-0.8.3/lib/virtualbox/abstract_model/dirty.rb:172:in `method_missing'
<------ stacktraces cut ---------->
/Users/john/.rvm/gems/ruby-1.8.7-p302/gems/rake-0.8.7/bin/rake:31
/Users/john/.rvm/gems/ruby-1.8.7-p302#global/bin/rake:19:in `load'
/Users/john/.rvm/gems/ruby-1.8.7-p302#global/bin/rake:19
Multiple Ruby Versions
Here is the same command run with three versions of Ruby available in RVM. Prior to doing this, I used "rvm use 1.8.7." Again, I don't know how important the details of the specific errors are- what's interesting to me is that there are three separate errors- each with it's own stacktrace- and each in a different Ruby interpreter. Look at the bottom of each stacktrace and you'll see that they are all sourced from different interpreter locations- First ree-1.8.7, then ruby-1.8.7, then ruby-1.9.2:
boudica:veewee john$ rvm rake build['mettabox'] --trace
rvm 1.1.0 by Wayne E. Seguin (wayneeseguin#gmail.com) [http://rvm.beginrescueend.com/]
(in /Users/john/Work/veewee)
** Invoke build (first_time)
** Execute build
…
creating new harddrive
rake aborted!
undefined method `max_vdi_size' for #<VirtualBox::SystemProperties:0x1059dd608>
/Users/john/.rvm/gems/ree-1.8.7-2010.02/gems/virtualbox-0.8.3/lib/virtualbox/abstract_model/dirty.rb:172:in `method_missing'
…
/Users/john/.rvm/gems/ree-1.8.7-2010.02/gems/rake-0.8.7/bin/rake:31
/Users/john/.rvm/gems/ree-1.8.7-2010.02#global/bin/rake:19:in `load'
/Users/john/.rvm/gems/ree-1.8.7-2010.02#global/bin/rake:19
(in /Users/john/Work/veewee)
** Invoke build (first_time)
** Execute build
isofile ubuntu-10.04.1-server-amd64.iso is available
["a1b857f92eecaf9f0a31ecfc39dee906", "30b5c6fdddbfe7b397fe506400be698d"]
[]
Last good state: -1
Current step: 0
last good state -1
destroying machine+disks
(re-)executing step 0-initial-a1b857f92eecaf9f0a31ecfc39dee906
VBoxManage: error: Machine settings file '/Users/john/VirtualBox VMs/mettabox/mettabox.vbox' already exists
VBoxManage: error: Details: code VBOX_E_FILE_ERROR (0x80bb0004), component Machine, interface IMachine, callee nsISupports
Context: "CreateMachine(bstrSettingsFile.raw(), name.raw(), osTypeId.raw(), Guid(id).toUtf16().raw(), FALSE , machine.asOutParam())" at line 247 of file VBoxManageMisc.cpp
rake aborted!
undefined method `memory_size=' for nil:NilClass
/Users/john/Work/veewee/lib/veewee/session.rb:303:in `create_vm'
/Users/john/Work/veewee/lib/veewee/session.rb:166:in `build'
/Users/john/Work/veewee/lib/veewee/session.rb:560:in `transaction'
/Users/john/Work/veewee/lib/veewee/session.rb:163:in `build'
/Users/john/Work/veewee/Rakefile:87
/Users/john/.rvm/gems/ruby-1.8.7-p302/gems/rake-0.8.7/lib/rake.rb:636:in `call'
/Users/john/.rvm/gems/ruby-1.8.7-p302/gems/rake-0.8.7/lib/rake.rb:636:in `execute'
/Users/john/.rvm/gems/ruby-1.8.7-p302/gems/rake-0.8.7/lib/rake.rb:631:in `each'
…
/Users/john/.rvm/gems/ruby-1.8.7-p302/gems/rake-0.8.7/bin/rake:31
/Users/john/.rvm/gems/ruby-1.8.7-p302#global/bin/rake:19:in `load'
/Users/john/.rvm/gems/ruby-1.8.7-p302#global/bin/rake:19
(in /Users/john/Work/veewee)
** Invoke build (first_time)
** Execute build
isofile ubuntu-10.04.1-server-amd64.iso is available
["a9c4ab3257e1da3479c984eae9905c2a", "30b5c6fdddbfe7b397fe506400be698d"]
[]
Last good state: -1
Current step: 0
last good state -1
(re-)executing step 0-initial-a9c4ab3257e1da3479c984eae9905c2a
VBoxManage: error: Machine settings file '/Users/john/VirtualBox VMs/mettabox/mettabox.vbox' already exists
VBoxManage: error: Details: code VBOX_E_FILE_ERROR (0x80bb0004), component Machine, interface IMachine, callee nsISupports
Context: "CreateMachine(bstrSettingsFile.raw(), name.raw(), osTypeId.raw(), Guid(id).toUtf16().raw(), FALSE , machine.asOutParam())" at line 247 of file VBoxManageMisc.cpp
rake aborted!
undefined method `memory_size=' for nil:NilClass
/Users/john/Work/veewee/lib/veewee/session.rb:303:in `create_vm'
/Users/john/Work/veewee/lib/veewee/session.rb:166:in `block in build'
/Users/john/Work/veewee/lib/veewee/session.rb:560:in `transaction'
/Users/john/Work/veewee/lib/veewee/session.rb:163:in `build'
/Users/john/Work/veewee/Rakefile:87:in `block in <top (required)>'
/Users/john/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/rake.rb:634:in `call'
/Users/john/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/rake.rb:634:in `block in execute'
…
/Users/john/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/rake.rb:2013:in `top_level'
/Users/john/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/rake.rb:1992:in `run'
/Users/john/.rvm/rubies/ruby-1.9.2-p0/bin/rake:35:in `<main>'
It isn't until we reach the last installed version of Ruby that execution halts.
Discussion
Does anyone have any idea what's going on here? Has anyone seen this "failover"-like behavior before? It seems strange to me that the first exception would not halt execution as it did with one interpreter, but I wonder if there are things happening when RVM is installed that we Ruby developers are not considering.
Since rvm rake is SUPPOSED to run the rake task against all versions of Ruby under RVM's control I think it's doing what is expected: rvm help shows:
rake - runs a rake task against specified and/or all rubies
Specifically requesting help for rake shows:
$ rvm help rake
Rake
RVM allows you to run rake tasks across multiple ruby versions, for example:
∴ rvm 1.8.6,1.9.1 rake spec
JSON Summary
Adding a --json flag prior to the word 'rake' and a JSON summary will be printed out at the end of the run.
YAML Summary
Adding a --yaml flag prior to the word 'rake' and a YAML summary will be printed out at the end of the run.
If you are trying to run a rake task, you do not have to involve RVM. Set your desired Ruby version with RVM first, then run the task:
rvm 1.9.2
rake some_task
the error you are mentioning is not a problem of different ruby vms.
Could it be you have upgrade to Virtualbox 4.X because the new vagrant needs it?
Some functions of the virtualbox on which veewee is dependent are (not yet) ported to Virtualbox 4.x
I'm working around this now by execing the virtualbox commands directly. Funny that while implementing the update, I hit the exact same error.
Patrick (veewee inspirator)