Here My Problem: Warbler doesn't support gems which are included via path in the gemfile. nevertheless i need to do that for my work. it is important that the included gem is packed and handled as a simple rubygem in the war archive. until now i have tried to manipulate the bundler so when the spec arrives in the warbler/traits/bundler.rb (that's where the specs are packed to the archive) it already has 'Bundler::Source::Rubygems' as source. the problem is that it sill needs to be build and installed from the path, but i can't handle it to pass the path anywhere in the spec or in the source. it already works that the gem is built, installed and packed into the archive as a rubygem and listed under GEM in the Lockfile but only with bad coding (it is all with reference to a specific gem and the path is typed in clearly)
Here my code:
warbler/lib/warbler/traits/bunlder.rb line: 60
case spec.source
when ::Bundler::Source::Git
config.bundler[:git_specs] ||= []
config.bundler[:git_specs] << spec
when ::Bundler::Source::Path
$stderr.puts("warning: Bundler `path' components are not currently supported.",
"The `#{spec.full_name}' component was not bundled.",
"Your application may fail to boot!")
else
##################################################################### MINE
if spec.name == "charla_common"
path = ::Bundler::GemHelper.new("../../common/trunk", spec.name).install_gem
end
##################################################################### MINE END
config.gems << spec
end
This is where the gem is installed path
Bundler/lib/bundler/dsl.rb line: 120
def source(source, options = {})
############################################################### MINE
if source.class == Bundler::Source::Path
options[:path] = source.options["path"]
source = "https://rubygems.org"
end
############################################################### MINE END
case source
when :gemcutter, :rubygems, :rubyforge then
Bundler.ui.warn "The source :#{source} is deprecated because HTTP " \
"requests are insecure.\nPlease change your source to 'https://" \
"rubygems.org' if possible, or 'http://rubygems.org' if not."
#rubygems_source.add_remote "http://rubygems.org"
return
when String
# ensures that the source in the lockfile is shown only once
unless options[:prepend]
#rubygems_source.add_remiote source
end
return
else
#source = source
if options[:prepend]
#sources = [#source] | #sources
else
#sources = #sources | [#source]
end
yield if block_given?
return #source
end
ensure
#source = nil
end
I'm not sure if this is an option for you, but in order to work around this problem, I create symbolic links in vendor/cache to the path with the included gem.
E.g. vendor/cache/gem_name -> ../../../gem_name.
Related
Cannot find type `Sinatra::Base`
ruby file
class StaticApp < Sinatra::Base
end
rbs file
class StaticApp < Sinatra::Base
end
run
bundle exec steep check --log-level=fatal
result
[error] Cannot find type `Sinatra::Base`
Diagnostic ID: RBS::UnknownTypeName
I use steep gem. It seems it is needed to require some files. But
library 'sinatra'
doesn't work.
#<RBS::EnvironmentLoader::UnknownLibraryError: Cannot find type definitions for library: sinatra ([nil])>
What do I wrong?
Thanks.
But library 'sinatra' doesn't work ..
RBS::EnvironmentLoader::UnknownLibraryError
Short answer
# Steepfile
target :app do
repo_path 'vendor/rbs'
Create a directory structure like vendor/rbs/sinatra/0/sinatra.rbs.
How you write or generate the contents of sinatra.rbs is out of the scope of this answer, but check out:
rbs collection
gem_rbs_collection
rbs_rails
Long answer
Steep is using an RBS::EnvironmentLoader.
# steep-0.47.0/lib/steep/project/target.rb:54
loader = RBS::EnvironmentLoader.new(core_root: core_root_path, repository: repo)
options.libraries.each do |lib|
name, version = lib.split(/:/, 2)
loader.add(library: name, version: version)
end
The EnvironmentLoader can find a library in either gem_sig_path (I think this would be a sig folder distributed in the gem), or in a repository.
# rbs-1.7.1/lib/rbs/environment_loader.rb:68
def has_library?(library:, version:)
if self.class.gem_sig_path(library, version) || repository.lookup(library, version)
true
else
false
end
end
In a Steepfile, repositories are configured via repo_path.
# Steepfile
repo_path 'vendor/rbs'
For the directory structure of a repository, I took https://github.com/ruby/rbs/tree/master/stdlib as an example.
ls -R vendor/rbs
activesupport
vendor/rbs/activesupport:
0
vendor/rbs/activesupport/0:
activesupport.rbs
For the sake of security internal Ruby gems in the Gemfile should always be referenced inside a source block so it never tries to fetch them from rubygems.org. I'd like to automate finding where people fail to do this, so would like to parse the Gemfile, find any gems that match our internal names, and check that rubygems.org isn't in their possible sources list.
source 'https://rubygems.org'
gem 'rails'
gem 'my-private-gem1' # this should be in the source block below
source PRIVATE_GEM_REPO do
gem 'my-private-gem2'
end
I've seen you can parse the Gemfile
Bundler::Definition.build('Gemfile', '', {})
But can't find anything in the returned data structure that shows me the available / allowed sources per gem
If I include the Gemfile.lock I see more source info, but it doesn't seem right because every gem lists all my sources regardless of if they're in a source block
Bundler::Definition.build('Gemfile', 'Gemfile.lock', {}).
locked_gems.
specs.
map {|g| [g.full_name, g.source.remotes.map(&:hostname).join(', ')]}
=> ["rails-6.0.3.4", "my.private.gemserver, rubygems.org"],
["my-private-gem1-1.0.0", "my.private.gemserver, rubygems.org"],
["my-private-gem2-1.0.0", "my.private.gemserver, rubygems.org"]]
Any thoughts on how to parse the Gemfile to find that my-private-gem1 is outside a source block?
Figured it out finally, just took awhile digging through the Bundler methods - and a coworker's help.
Bundler::Definition.
build('Gemfile', '', nil).
dependencies.
map {|dep| [dep.name, dep.source&.remotes&.map(&:hostname)&.join(', ')]}
=>
[["rails", nil],
["my-private-gem1", nil],
["my-private-gem2", "my.private.gemserver"]]
Now I can easily search that resulting data structure for any private gems that aren't locked down to my private gem server.
Preface
While I was writing this answer, the OP found a Bundler-specific answer. However, I offer a more generalizable solution below. This solution also offers user feedback that may make it easier to fix the file.
Finding Candidate Gems by Column Alignment, with Whitelisting
If you can safely assume that your Gemfile is always properly indented, the KISS solution may be to simply identify the gems that aren't indented within a group definition. For example:
# example Gemfile to test against
GEMFILE = <<~'EOF'
source 'https://rubygems.org'
gem 'rails'
gem 'my-private-gem1' # this should be in the source block below
source PRIVATE_GEM_REPO do
gem 'my-private-gem2'
end
EOF
# gems that are acceptable in a non-group context
whitelist = Regexp.new %w[rails sass-rails webpacker].join(?|)
UngroupedGem = Struct.new :line_no, :line_txt, :gem_name
ungrouped_gems = []
GEMFILE.lines.each_with_index do |line_txt, line_no|
next if line_txt =~ whitelist or line_txt !~ /^\s*gem/
gem_name = line_txt.match(/(?<=(['"]))(.*?)(?=\1)/)[0]
ungrouped_gems.append(
UngroupedGem.new line_no.succ, line_txt, gem_name
).compact!
end
# tell the user what actions to take
if ungrouped_gems.any?
puts "Line No.\tGem Name"
ungrouped_gems.each { printf "%d\t\t%s\n", _1.line_no, _1.gem_name }
else
puts "No gems need to be moved."
end
With this example, it will print:
Line No. Gem Name
3 my-private-gem1
5 my-private-gem2
which will give you a solid idea of what lines in the Gemfile need to be moved, and which specific gems are involved.
I'm building a Ruby gem file that has various 3rd party executable requirements (some non-ruby tools that the gem will then use through system calls).
The requirement list is specified using the gemspec informative requirements list, and I was tasked with creating a set up where that requirement list is verified during installation - essentially converting the requirements list from being informative to being authoritative.
My initial approach is to use extensions, like in the extconf.rb pattern, to run some ruby code during installation that will verify the list of requirements and fail the installation if the requirements are not met.
So my current implementation is using a ext/Rakefile, that reads gemspec file using a hard-coded path, parses the requirements list using a custom (but straight-forward and simple) syntax and run some simple tests on the required executables.
I would really want to factor out the dependency on knowing the path to the gemspec file, so I can reuse the code in a modular way on other projects.
For completion (and in the hopes that it will be useful for someone), here's my code - which uses the Gem::Ext::Builder "Rakefile mode" to just run some arbitrary code.
require 'rubygems'
require 'mkmf'
def verify_requirement(reqspec)
exe, version_spec, version_regex = reqspec.split(/\s*,\s*/,3)
exe = $1 if exe =~ /\[([^\]]+)\]/ # support human readable alias for executable name
abort "Missing required executable #{exe}!" unless find_executable(exe)
return unless version_spec # no version is specified, so executable existing is good enough
version = %x|#{exe} --version 2>&1|.chomp
version = %x|#{exe} -version 2>&1|.chomp unless $?.success?
abort "Error checking for version of #{exe}: (#{$?.exitstatus}) #{version}" unless $?.success?
if version_regex
abort "Version test '#{version_regex}' failed to match '#{version}'!" unless version =~ /#{version_regex}/m
abort "Version test '#{version_regex}' failed to generate a version number!" unless $1
version = $1
end
abort "Failed to locate a valid version number for #{exe}, found '#{version}'" unless version =~ /^[0-9\.]+/
abort "Insufficient version '#{version}' for #{exe} - requires #{version_spec}" unless Gem::Dependency.new(exe, version_spec).match? exe, version
end
task :default do
gemspec = File.absolute_path('../my.gemspec')
Gem::Specification::load(gemspec).requirements.each do |req|
verify_requirement(req)
end
end
This code requires that the requirements uses either of these syntaxes:
exe-file[, <semantic-version-spec>[, <version-garabbing-regex>]]
alias '[' exe-file ']'[, <semantic-version-spec>[, <version-garabbing-regex>]]
For example:
spec.requirements << 'sox'
spec.requirements << 'ffmpeg, >= 2.5, ffmpeg version ([\d\.]+)'
spec.requirements << 'ImageMagick [convert], >= 6.7.5, ImageMagick ([\d\.]+)'
The Gem::Specification class can be used to load and parse the gemspec. I'm using the following code in ext/Rakefile to perform tasks based on the gemspec file:
task :default do
gemspec = File.absolute_path('../gemname.gemspec')
Gem::Specification::load(gemspec).requirements.each do |req|
verify_requirement(req)
end
end
I have a Buildr extension that I'm packaging as a gem. I have a collection of scripts that I want to add to a package. Currently, I have these scripts stored as a big text block that I'm writing to file. I would prefer to have individual files that I can either copy directly or read/write back out. I would like these files to be packaged into the gem. I don't have a problem packaging them in (just stick them in the file system before rake install) but I can't figure out how to access them. Is there a Gem Resources bundle type thing?
There are basically two ways,
1) You can load resources relative to a Ruby file in your gem using __FILE__:
def path_to_resources
File.join(File.dirname(File.expand_path(__FILE__)), '../path/to/resources')
end
2) You can add arbitrary paths from your Gem to the $LOAD_PATH variable and then walk the $LOAD_PATH to find resources, e.g.,
Gem::Specification.new do |spec|
spec.name = 'the-name-of-your-gem'
spec.version ='0.0.1'
# this is important - it specifies which files to include in the gem.
spec.files = Dir.glob("lib/**/*") + %w{History.txt Manifest.txt} +
Dir.glob("path/to/resources/**/*")
# If you have resources in other directories than 'lib'
spec.require_paths << 'path/to/resources'
# optional, but useful to your users
spec.summary = "A more longwinded description of your gem"
spec.author = 'Your Name'
spec.email = 'you#yourdomain.com'
spec.homepage = 'http://www.yourpage.com'
# you did document with RDoc, right?
spec.has_rdoc = true
# if you have any dependencies on other gems, list them thusly
spec.add_dependency('hpricot')
spec.add_dependency('log4r', '>= 1.0.5')
end
and then,
$LOAD_PATH.each { |dir| ... look for resources relative to dir ... }
I've been learning Ruby recently, and I've not gotten into the dirty recesses of learning Rake yet. I've been playing around with NetBeans, and I made a little Ruby project with a file that simply prints "Hello, World!". I was looking at the Rakefile that NetBeans generates, and I noticed that it had commented out the s.executables line, so I uncommented, and tried to build it. Of course it failed with:
Don't know how to build task 'bin/your_executable_here'
What I'm trying to do, is figure out how to make that work. I've googled around, and I can't find any information on how to correctly generate an executable. Here is the Rakefile generated by NetBeans:
require 'rubygems'
require 'rake'
require 'rake/clean'
require 'rake/gempackagetask'
require 'rake/rdoctask'
require 'rake/testtask'
spec = Gem::Specification.new do |s|
s.name = 'Learning'
s.version = '0.0.1'
s.has_rdoc = true
s.extra_rdoc_files = ['README', 'LICENSE']
s.summary = 'Your summary here'
s.description = s.summary
s.author = ''
s.email = ''
s.executables = ['your_executable_here']
s.files = %w(LICENSE README Rakefile) + Dir.glob("{bin,lib,spec}/**/*")
s.require_path = "lib"
s.bindir = "bin"
end
Rake::GemPackageTask.new(spec) do |p|
p.gem_spec = spec
p.need_tar = true
p.need_zip = true
end
Rake::RDocTask.new do |rdoc|
files =['README', 'LICENSE', 'lib/**/*.rb']
rdoc.rdoc_files.add(files)
rdoc.main = "README" # page to start on
rdoc.title = "Learning Docs"
rdoc.rdoc_dir = 'doc/rdoc' # rdoc output folder
rdoc.options
I'm sorry if this is a stupid question, I honestly did try to find the information myself.
EDIT: I was unaware that there had to be an executable file by the same name as the default one you specify in ./bin in your project. I figured it all out.
The s.executables array must contain the names of the executables in the bin directory of your gem
s.executables = %w( my_awesome_commandline_churner )
This code is used to make a gem file. Gems are ruby's package management devices. Some gems come with executable script files to be run from the command line. They are placed in the ./bin directory when the gem is built, and hen it is deployed they will be copied into the same folder as the ruby executable.
To make a file executable you will need to add a shabang (#!/user/local/bin/ruby) to the first line and change the file permission to allow execution.