I am working on a gem related utility and I have observed strange results using Gem.latest_version_for method. Here are some observations under irb:
irb(main):001:0> Gem.latest_version_for('rails').to_s
=> "5.2.2"
irb(main):002:0> Gem.latest_version_for('gosu').to_s
=> "0.7.38"
Note how the first line, gets the correct version of rails, 5.2.2 as I write this and checking with rubygems.org confirms this. The query for the gosu gem returns 0.7.38 which is wildly wrong. The correct answer should be 0.14.4
I am at a loss to explain what is happening here.
I can confirm that my host is https://rubygems.org and
C:\Sites\mysh
8 mysh>ruby --version
ruby 2.3.3p222 (2016-11-21 revision 56859) [i386-mingw32]
C:\Sites\mysh
9 mysh>gem --version
2.5.2
The latest version available for i386-mingw32 platform is 0.7.38. You'll note this comports with what your ruby version is reported as.
https://rubygems.org/gems/gosu/versions
latest_version_for calls latest_spec_for, which calls Gem::SpecFetcher.spec_for_dependency with only the name of the gem as an argument. spec_for_dependency takes another argument, matching_platform, which defaults to true.
It looks like latest_version_for is scoped to your current platform thru that chain, with the matching_platform default. The gem install command might treat i386/x386 as the same/equivalent and allow them.
spec_for_dependency
if matching_platform is false, gems for all platforms are returned
You should be able to mirror the latest_spec_for method and pass in the multi_platform argument to override. Something like
dependency = Gem::Dependency.new name
fetcher = Gem::SpecFetcher.fetcher
spec_tuples, _ = fetcher.spec_for_dependency dependency, true # true added here
With the excellent help of Jay Dorsey, I think I have made some progress here. What I need to say is too large to fit in a comment and is the actual answer to the question about the odd behavior. Well at least I am pretty sure that it is.
As mentioned above: latest_version_for calls latest_spec_for, which calls Gem::SpecFetcher.spec_for_dependency.
The key is that that method then calls Gem::SpecFetcher.search_for_dependency. This is a long rambling method. I want to focus of one line that occurs after the specs have be obtained:
tuples = tuples.sort_by { |x| x[0] }
This sorts the tuples which are an array of [spec, source] arrays. It sorts them in ascending version/platform (as far as I can tell)
Now we return to the Gem class method latest_spec_for(name) and in particular the line:
spec, = spec_tuples.first
This grabs the first sub-array and keeps the spec and discards the source.
Note that it grabs the first element. The one with the lowest version number. This is normally not a problem because for the vast majority of gems, there will be only one spec present. Not so for the gosu gem. Here there are three due to the fact that gosu contains platform specific code. It seems to grab specs for the two Gem platforms ("ruby" and "x86-mingw32") and also the ruby platform (i386-mingw32).
To test my idea, I created the file glmp.rb (get last monkey patch) Here it is:
# The latest_spec_for(name) monkey patch.
module Gem
# Originally in File rubygems.rb at line 816
def self.latest_spec_for(name)
dependency = Gem::Dependency.new name
fetcher = Gem::SpecFetcher.fetcher
spec_tuples, = fetcher.spec_for_dependency dependency
spec_tuples[-1][0]
end
end
Now I know monkey patching is frowned upon, but for now this is just to test the idea. Here are my results:
36 mysh>=Gem.latest_version_for('gosu')
Gem::Version.new("0.7.38")
C:\Sites\ideas\gem_usage
37 mysh>ls
gem_latest.rb gem_usage.rb glmp.rb
C:\Sites\ideas\gem_usage
39 mysh>=require './glmp'
true
C:\Sites\ideas\gem_usage
40 mysh>=Gem.latest_version_for('gosu')
Gem::Version.new("0.14.4")
While I can use this hack to solve my problem for now, I think I will raise an issue with rubygems bringing up this matter.
Related
A recent change in Ruby's YAML Library (Psych 4) causes "unsafe" YAMLs to fail if they contain aliases, or try to instantiate unspecified classes. This is discussed in multiple places, like this StackOverflow question.
I am trying to figure out how to tell the derivative yaml/store library to allow loading unsafe YAMLs, or to provide it with my list of allowed classes.
The documentation is scarce as far as I could find, and after reading it, this is the only logical attempt I could come up with:
require 'date'
require 'yaml/store'
# 1. These options work perfectly with YAML.load_file, but not with YAML::Store
# 2. These options are not needed in Psych < 4.0
yaml_opts = { aliases: true, permitted_classes: [Time, Date, Symbol] }
store = YAML::Store.new 'log.yml', yaml_opts
data = store.transaction { store[:entries] }
p data
using this YAML file:
# log.yml
:entries:
- :timestamp: 2018-07-09 00:00:00.000000000 +03:00
:action: Comment
:comment: Started logging
This fails with Psych 4, and succeeds with Psych 3.
# Gemfile
source "https://rubygems.org"
gem 'psych', '>= 4.0' # fail
# gem 'psych', '< 4.0' # pass
As a related anecdote, the example demonstrated in the docs, also fails as-is when trying to load it with store.transaction { store["people"] }
Although this is not the proper way of doing things, until there is a better answer, I found that adding the below code fixes the problem.
module YAML
class << self
alias_method :load, :unsafe_load
end
end
This simply restores the underlying YAML::load method to its 3.x behavior of unsafe_load instead of safe_load.
In cases where my YAMLs come from a trusted source (100% of my use cases), I do not see any benefit in the new Psych 4 behavior, and feel it is ok (although awkward) to revert it.
The relevant source code reference is the 3.3.2 → 4.0.0 diff
For the sake of security internal Ruby gems in the Gemfile should always be referenced inside a source block so it never tries to fetch them from rubygems.org. I'd like to automate finding where people fail to do this, so would like to parse the Gemfile, find any gems that match our internal names, and check that rubygems.org isn't in their possible sources list.
source 'https://rubygems.org'
gem 'rails'
gem 'my-private-gem1' # this should be in the source block below
source PRIVATE_GEM_REPO do
gem 'my-private-gem2'
end
I've seen you can parse the Gemfile
Bundler::Definition.build('Gemfile', '', {})
But can't find anything in the returned data structure that shows me the available / allowed sources per gem
If I include the Gemfile.lock I see more source info, but it doesn't seem right because every gem lists all my sources regardless of if they're in a source block
Bundler::Definition.build('Gemfile', 'Gemfile.lock', {}).
locked_gems.
specs.
map {|g| [g.full_name, g.source.remotes.map(&:hostname).join(', ')]}
=> ["rails-6.0.3.4", "my.private.gemserver, rubygems.org"],
["my-private-gem1-1.0.0", "my.private.gemserver, rubygems.org"],
["my-private-gem2-1.0.0", "my.private.gemserver, rubygems.org"]]
Any thoughts on how to parse the Gemfile to find that my-private-gem1 is outside a source block?
Figured it out finally, just took awhile digging through the Bundler methods - and a coworker's help.
Bundler::Definition.
build('Gemfile', '', nil).
dependencies.
map {|dep| [dep.name, dep.source&.remotes&.map(&:hostname)&.join(', ')]}
=>
[["rails", nil],
["my-private-gem1", nil],
["my-private-gem2", "my.private.gemserver"]]
Now I can easily search that resulting data structure for any private gems that aren't locked down to my private gem server.
Preface
While I was writing this answer, the OP found a Bundler-specific answer. However, I offer a more generalizable solution below. This solution also offers user feedback that may make it easier to fix the file.
Finding Candidate Gems by Column Alignment, with Whitelisting
If you can safely assume that your Gemfile is always properly indented, the KISS solution may be to simply identify the gems that aren't indented within a group definition. For example:
# example Gemfile to test against
GEMFILE = <<~'EOF'
source 'https://rubygems.org'
gem 'rails'
gem 'my-private-gem1' # this should be in the source block below
source PRIVATE_GEM_REPO do
gem 'my-private-gem2'
end
EOF
# gems that are acceptable in a non-group context
whitelist = Regexp.new %w[rails sass-rails webpacker].join(?|)
UngroupedGem = Struct.new :line_no, :line_txt, :gem_name
ungrouped_gems = []
GEMFILE.lines.each_with_index do |line_txt, line_no|
next if line_txt =~ whitelist or line_txt !~ /^\s*gem/
gem_name = line_txt.match(/(?<=(['"]))(.*?)(?=\1)/)[0]
ungrouped_gems.append(
UngroupedGem.new line_no.succ, line_txt, gem_name
).compact!
end
# tell the user what actions to take
if ungrouped_gems.any?
puts "Line No.\tGem Name"
ungrouped_gems.each { printf "%d\t\t%s\n", _1.line_no, _1.gem_name }
else
puts "No gems need to be moved."
end
With this example, it will print:
Line No. Gem Name
3 my-private-gem1
5 my-private-gem2
which will give you a solid idea of what lines in the Gemfile need to be moved, and which specific gems are involved.
I'm trying to write a plugin for ohai. It seems like a pretty straightforward task:
Ohai.plugin(:Uname) do
provides 'uname'
depends 'kernel'
collect_data do
uname Mash.new
uname[:message] = `uname -a`
end
end
To me this looks like the online examples provided by Opscode, O'Reilly and others. But here's what happens when I try to test it:
% irb -rohai
irb(main):001:0> Ohai::Config[:plugin_path] << "."
=> ["/home/ll0359/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/ohai-8.3.0/lib/ohai/plugins", "."]
irb(main):002:0> o = Ohai::System.new
=> #<Ohai::System:0x007fed82e43078 #plugin_path="", #data={}, #provides_map=#<Ohai::ProvidesMap:0x007fed82e42fd8 #map={}>, #v6_dependency_solver={}, #d82e42f38 #controller=#<Ohai::System:0x007fed82e43078 ...>, #v6_plugin_classes=[], #v7_plugin_classes=[]>, #runner=#<Ohai::Runner:0x007fed82e42ec0 #prp:0x007fed82e42fd8 #map={}>, #safe_run=true>>
irb(main):003:0> o.all_plugins
And here's where the fun begins. I get this output, over and over and over:
[2015-05-20T03:13:09+00:00] WARN: Plugin Definition Error: <./ohai_uname.rb>: collect_data already defined on platform default
[2015-05-20T03:13:09+00:00] WARN: [DEPRECATION] Plugin at ./test_ohai.rb is a version 6 plugin. Version 6 plugins will not be supported in future releases....
your plugin to version 7 plugin syntax. For more information visit here: docs.chef.io/ohai_custom.html
(the text on my second line was clipped by my screen but you get the idea)
I've tried running this code with and without the 'depends' line. Same result.
I've tried running this code with and without the Mash line, substituing 'uname uname -a' for the assignment line. Same result.
I've tried running with and without passing ":linux" as a parameter to collect_data. The only difference is I get a warning about collect_data(:linux) already being defined instead of :default.
I've tried renaming the plugin to a random 8 character identifier just in case it was tripping over being called :Uname. Same result.
I've tried passing "uname" (capital and lower) as a parameter to o.all_plugins. Same result.
So my questions are:
Why does ohai (8.3, running under Ruby 2.2.1) think this is a version 6 plugin? I can't see anything in it that would make it look like it's not version 7.
How can I get this working?
Thanks
Note to self: Next time you do this, don't try to test from the directory your plugin is in and add "." to your plugin_path. Moving to a different directory and adding the absolute path to the plugin solved the problem.
I plan to leave this up in case someone else has this happen to them.
I am having a perplexing problem. I want to call one ruby script from another.
With this in mind, I create a testscript.rb and executed it, it contains this code,
require './paypal.rb'
puts paypal['L_AMT0']
This code returns a number, which is my paypal balance. It relies on a paypal.rb file which uses the ruby-paypal gem. When I do ruby testscript.rb I get the output of my paypal balance, which means it is working properly. This tells me that my method for calling one RB from another is okay, since, in the above scenario, testscript.rb is getting a variable that is returned from paypal.rb.
Using this same logic, I inserted the above code into another program which is called SiriProxy. It is a ruby program. I need it to get my paypal balance.
So Inside that program, I did a require paypal.rb, it failed because it could not find it, so I set an absolute path in require which fixed it.
However, when SiriProxy (the other ruby rb giving me an issue) trys to run puts paypal['L_AMT0'] it results in an error and ends the program.
This is the error,
[Info - Plugin Manager] Say: Checking Paypal Balance
/home/siriproxy/.rvm/gems/ruby-1.9.3-p194#SiriProxy/gems/siriproxy-0.3.0/plugins/siriproxy-example/lib/siriproxy-example.rb:47:in `block in <class:Example>': undefined local variable or method `paypal' for #<SiriProxy::Plugin::Example:0x931a228> (NameError)
from /home/siriproxy/.rvm/gems/ruby-1.9.3-p194#SiriProxy/bundler/gems/cora-1edcfb9073d5/lib/cora/plugin.rb:47:in `instance_exec'
from /home/siriproxy/.rvm/gems/ruby-1.9.3-p194#SiriProxy/bundler/gems/cora-1edcfb9073d5/lib/cora/plugin.rb:47:in `block (2 levels) in process'
In the above output it appears the issue is it does not understand "paypal", as seen here:
undefined local variable or method `paypal'
However, I do not understand why, since in testscript.rb I do the exact same thing and it works.
Can anyone help? Thank you.
Seems like #CodeGnome is right.
From the Kernel documentation:
require(name) → true or false
Loads the given name, returning true if successful and false if the
feature is already loaded.
require_relative(string) → true or false
Ruby tries to load the library named string relative to the requiring
file’s path. If the file’s path cannot be determined a LoadError is
raised. If a file is loaded true is returned and false otherwise.
Loading Files from the Current Directory
I don't know anything about your library or its internals, but it looks like your require statement may be wrong. If you want to load a file from the current directory in Ruby 1.9.3, you should use:
require_relative 'paypal'
Bundler and Gems
If it's a gem that you've installed as part of a bundle (e.g. the gem is defined in a Gemfile), then providing a path in your require statement is wrong. Instead, you need to require the bundled gem as follows:
require 'bundler/setup'
require 'paypal'
Over in this question, an answer was given to analyze the Bundler dependency list. That works great, but it doesn't give you the list of packages and versions actually being used, because of ">=" dependencies. Is there a way to get the list of packages and versions actually being used rather than just what the dependencies are?
Looks like the way to do this is similar to what was posted in the other question:
Rails.logger.debug "Type is " + Bundler.environment.specs.class.to_s
Rails.logger.debug "Value is " + Bundler.environment.specs.to_hash.to_s
Produces:
Type is Bundler::SpecSet
Value is {"activemodel"=>[#<Gem::Specification name=activemodel version=3.1.3>],
"actionpack"=>[#<Gem::Specification name=actionpack version=3.1.3>],
"actionmailer"=>[#<Gem::Specification name=actionmailer version=3.1.3>]}
This code will print out all of the gems and versions being used in your current environment. One thing to note about the answer in that other question is that it will return all of the dependencies, even those that aren't in your current rails environment (for example, the ones that are in your "test" gem grouping).
This code was extracted from the Bundler codebase and will do the exact same thing as bundle list from within a Rails console.
Bundler.load.specs.sort_by(&:name).each{|s| puts " * #{s.name} (#{s.version}#{s.git_version})"}; nil
If you just want an array of the dependencies, this will also suffice.
Bundler.load.specs.map{|s| "#{s.name} (#{s.version}#{s.git_version})"}