Specific many gems from the same github account - ruby

like with path and source I'd like to do this:
git 'https://github.com/my_company', branch: 'rm_2422-rails5-upgrade' do
gem 'foo'
gem 'bar'
# many more gems...
end
The idea being this would fetch the foo gem with the URL's https://github.com/my_company/foo.git using the rm_2422-rails5-upgrade branch.
I can see from the bundler docs that this is not how it works and I know I can do:
git 'foo', git: 'https://github.com/my_company/foo.git', branch: 'rm_2422-rails5-upgrade'
But I have lots of gems which need to be pulled from said branch.
I also looked at git_source but this does not seem to work for this case either.

You may define a new git_source in your Gemfile:
git_source(:your_source_name) do |repo_name|
repo_name = "company/#{repo_name}"
"https://github.com/#{repo_name}.git"
end
And then use it:
gem "gem_name", your_company: "gem_repo_name"
This was very simple example, but you may pass more options to the block. We use this approach to redefine original github source to be able to pass auth token for private repositories:
git_source(:github) do |(repo_name, auth_token)|
repo_name = "#{repo_name}/#{repo_name}" unless repo_name.include?("/")
auth_data = !!auth_token ? "#{auth_token}:x-oauth-basic#" : ""
"https://#{auth_data}github.com/#{repo_name}.git"
end
UPDATE: I found a way to do what you need, but this is quite a dirty hack IMHO. Since Gemfile is a pure ruby file you may define your own functions there:
def my_gem(name, *args)
options = args.last.is_a?(Hash) ? args.pop.dup : {}
version = args || [">= 0"]
options[:branch] = "develop"
gem(name, version, options)
end
And then use it, instead of original gem method:
my_gem "gem_name"

So one solution I used this:
['foo', 'bar'].each do |gem_name|
gem gem_name, git: "https://github.com/my_company/#{gem_name}.git", branch: 'rm_2422-rails5-upgrade'
end

Related

How to configure Steep to find RBS files for a gem outside the stdlib?

Cannot find type `Sinatra::Base`
ruby file
class StaticApp < Sinatra::Base
end
rbs file
class StaticApp < Sinatra::Base
end
run
bundle exec steep check --log-level=fatal
result
[error] Cannot find type `Sinatra::Base`
Diagnostic ID: RBS::UnknownTypeName
I use steep gem. It seems it is needed to require some files. But
library 'sinatra'
doesn't work.
#<RBS::EnvironmentLoader::UnknownLibraryError: Cannot find type definitions for library: sinatra ([nil])>
What do I wrong?
Thanks.
But library 'sinatra' doesn't work ..
RBS::EnvironmentLoader::UnknownLibraryError
Short answer
# Steepfile
target :app do
repo_path 'vendor/rbs'
Create a directory structure like vendor/rbs/sinatra/0/sinatra.rbs.
How you write or generate the contents of sinatra.rbs is out of the scope of this answer, but check out:
rbs collection
gem_rbs_collection
rbs_rails
Long answer
Steep is using an RBS::EnvironmentLoader.
# steep-0.47.0/lib/steep/project/target.rb:54
loader = RBS::EnvironmentLoader.new(core_root: core_root_path, repository: repo)
options.libraries.each do |lib|
name, version = lib.split(/:/, 2)
loader.add(library: name, version: version)
end
The EnvironmentLoader can find a library in either gem_sig_path (I think this would be a sig folder distributed in the gem), or in a repository.
# rbs-1.7.1/lib/rbs/environment_loader.rb:68
def has_library?(library:, version:)
if self.class.gem_sig_path(library, version) || repository.lookup(library, version)
true
else
false
end
end
In a Steepfile, repositories are configured via repo_path.
# Steepfile
repo_path 'vendor/rbs'
For the directory structure of a repository, I took https://github.com/ruby/rbs/tree/master/stdlib as an example.
ls -R vendor/rbs
activesupport
vendor/rbs/activesupport:
0
vendor/rbs/activesupport/0:
activesupport.rbs

How do I parse the Gemfile to find internal gems not inside a source block?

For the sake of security internal Ruby gems in the Gemfile should always be referenced inside a source block so it never tries to fetch them from rubygems.org. I'd like to automate finding where people fail to do this, so would like to parse the Gemfile, find any gems that match our internal names, and check that rubygems.org isn't in their possible sources list.
source 'https://rubygems.org'
gem 'rails'
gem 'my-private-gem1' # this should be in the source block below
source PRIVATE_GEM_REPO do
gem 'my-private-gem2'
end
I've seen you can parse the Gemfile
Bundler::Definition.build('Gemfile', '', {})
But can't find anything in the returned data structure that shows me the available / allowed sources per gem
If I include the Gemfile.lock I see more source info, but it doesn't seem right because every gem lists all my sources regardless of if they're in a source block
Bundler::Definition.build('Gemfile', 'Gemfile.lock', {}).
locked_gems.
specs.
map {|g| [g.full_name, g.source.remotes.map(&:hostname).join(', ')]}
=> ["rails-6.0.3.4", "my.private.gemserver, rubygems.org"],
["my-private-gem1-1.0.0", "my.private.gemserver, rubygems.org"],
["my-private-gem2-1.0.0", "my.private.gemserver, rubygems.org"]]
Any thoughts on how to parse the Gemfile to find that my-private-gem1 is outside a source block?
Figured it out finally, just took awhile digging through the Bundler methods - and a coworker's help.
Bundler::Definition.
build('Gemfile', '', nil).
dependencies.
map {|dep| [dep.name, dep.source&.remotes&.map(&:hostname)&.join(', ')]}
=>
[["rails", nil],
["my-private-gem1", nil],
["my-private-gem2", "my.private.gemserver"]]
Now I can easily search that resulting data structure for any private gems that aren't locked down to my private gem server.
Preface
While I was writing this answer, the OP found a Bundler-specific answer. However, I offer a more generalizable solution below. This solution also offers user feedback that may make it easier to fix the file.
Finding Candidate Gems by Column Alignment, with Whitelisting
If you can safely assume that your Gemfile is always properly indented, the KISS solution may be to simply identify the gems that aren't indented within a group definition. For example:
# example Gemfile to test against
GEMFILE = <<~'EOF'
source 'https://rubygems.org'
gem 'rails'
gem 'my-private-gem1' # this should be in the source block below
source PRIVATE_GEM_REPO do
gem 'my-private-gem2'
end
EOF
# gems that are acceptable in a non-group context
whitelist = Regexp.new %w[rails sass-rails webpacker].join(?|)
UngroupedGem = Struct.new :line_no, :line_txt, :gem_name
ungrouped_gems = []
GEMFILE.lines.each_with_index do |line_txt, line_no|
next if line_txt =~ whitelist or line_txt !~ /^\s*gem/
gem_name = line_txt.match(/(?<=(['"]))(.*?)(?=\1)/)[0]
ungrouped_gems.append(
UngroupedGem.new line_no.succ, line_txt, gem_name
).compact!
end
# tell the user what actions to take
if ungrouped_gems.any?
puts "Line No.\tGem Name"
ungrouped_gems.each { printf "%d\t\t%s\n", _1.line_no, _1.gem_name }
else
puts "No gems need to be moved."
end
With this example, it will print:
Line No. Gem Name
3 my-private-gem1
5 my-private-gem2
which will give you a solid idea of what lines in the Gemfile need to be moved, and which specific gems are involved.

Warbler: No Support for Path Gems

Here My Problem: Warbler doesn't support gems which are included via path in the gemfile. nevertheless i need to do that for my work. it is important that the included gem is packed and handled as a simple rubygem in the war archive. until now i have tried to manipulate the bundler so when the spec arrives in the warbler/traits/bundler.rb (that's where the specs are packed to the archive) it already has 'Bundler::Source::Rubygems' as source. the problem is that it sill needs to be build and installed from the path, but i can't handle it to pass the path anywhere in the spec or in the source. it already works that the gem is built, installed and packed into the archive as a rubygem and listed under GEM in the Lockfile but only with bad coding (it is all with reference to a specific gem and the path is typed in clearly)
Here my code:
warbler/lib/warbler/traits/bunlder.rb line: 60
case spec.source
when ::Bundler::Source::Git
config.bundler[:git_specs] ||= []
config.bundler[:git_specs] << spec
when ::Bundler::Source::Path
$stderr.puts("warning: Bundler `path' components are not currently supported.",
"The `#{spec.full_name}' component was not bundled.",
"Your application may fail to boot!")
else
##################################################################### MINE
if spec.name == "charla_common"
path = ::Bundler::GemHelper.new("../../common/trunk", spec.name).install_gem
end
##################################################################### MINE END
config.gems << spec
end
This is where the gem is installed path
Bundler/lib/bundler/dsl.rb line: 120
def source(source, options = {})
############################################################### MINE
if source.class == Bundler::Source::Path
options[:path] = source.options["path"]
source = "https://rubygems.org"
end
############################################################### MINE END
case source
when :gemcutter, :rubygems, :rubyforge then
Bundler.ui.warn "The source :#{source} is deprecated because HTTP " \
"requests are insecure.\nPlease change your source to 'https://" \
"rubygems.org' if possible, or 'http://rubygems.org' if not."
#rubygems_source.add_remote "http://rubygems.org"
return
when String
# ensures that the source in the lockfile is shown only once
unless options[:prepend]
#rubygems_source.add_remiote source
end
return
else
#source = source
if options[:prepend]
#sources = [#source] | #sources
else
#sources = #sources | [#source]
end
yield if block_given?
return #source
end
ensure
#source = nil
end
I'm not sure if this is an option for you, but in order to work around this problem, I create symbolic links in vendor/cache to the path with the included gem.
E.g. vendor/cache/gem_name -> ../../../gem_name.

ArgumentError in YAML::dump if (what looks like a) date is invalid

Is there a way to make Psych a little dumber? If a string matches certain regular expression it tries to convert it to date and fails miserably. So far I switched to Syck for serializing but it's no longer maintained and I'm looking for better solution.
Here's an example of what's going on:
s = "2222-33-44" # => "2222-33-44"}
require 'yaml' # => true
YAML::dump s # ArgumentError: invalid date
YAML::ENGINE.yamler = 'syck' # => "syck"
YAML::dump s # => "--- \"2222-33-44\"\n"
I have found that it is a known bug. It has been fixed but looks like the version in RubyGems is not containing the fix.
Also as the git repository doesn't have a proper .gemspec is not easy way to install from the repo.
You can fork the repo and add a proper .gemspec and use a Gemfile that points to your fork repo.
Also you can ask kindly to Aaron to release the fix.

Accessing files packaged into a Ruby Gem

I have a Buildr extension that I'm packaging as a gem. I have a collection of scripts that I want to add to a package. Currently, I have these scripts stored as a big text block that I'm writing to file. I would prefer to have individual files that I can either copy directly or read/write back out. I would like these files to be packaged into the gem. I don't have a problem packaging them in (just stick them in the file system before rake install) but I can't figure out how to access them. Is there a Gem Resources bundle type thing?
There are basically two ways,
1) You can load resources relative to a Ruby file in your gem using __FILE__:
def path_to_resources
File.join(File.dirname(File.expand_path(__FILE__)), '../path/to/resources')
end
2) You can add arbitrary paths from your Gem to the $LOAD_PATH variable and then walk the $LOAD_PATH to find resources, e.g.,
Gem::Specification.new do |spec|
spec.name = 'the-name-of-your-gem'
spec.version ='0.0.1'
# this is important - it specifies which files to include in the gem.
spec.files = Dir.glob("lib/**/*") + %w{History.txt Manifest.txt} +
Dir.glob("path/to/resources/**/*")
# If you have resources in other directories than 'lib'
spec.require_paths << 'path/to/resources'
# optional, but useful to your users
spec.summary = "A more longwinded description of your gem"
spec.author = 'Your Name'
spec.email = 'you#yourdomain.com'
spec.homepage = 'http://www.yourpage.com'
# you did document with RDoc, right?
spec.has_rdoc = true
# if you have any dependencies on other gems, list them thusly
spec.add_dependency('hpricot')
spec.add_dependency('log4r', '>= 1.0.5')
end
and then,
$LOAD_PATH.each { |dir| ... look for resources relative to dir ... }

Resources