Is there anyway to get Ruby's require statement to download a file from somewhere like github rather than just the local file system?
Update: Sorry I should have made the question clearer. I want to download a file that contains Ruby module and import it into my script rather than just downloading an image or some other arbitrary file within my script.
In other words something like this
require 'http:\\github.com\myrepo\snippet.rb'
puts 'hi'
By default, this is not possible. Also, it's not a good idea for security reasons.
You have a couple of alternatives. If the file you want to include is a Gem and Git repository, then you can use Bundler to download and package the dependency in your project. Then you'll be able to require the file directly in your source code.
This is the best and safest way to include an external dependency.
If you trust the source and you really know what you are doing, you can download the file using Net::HTTP (or any other HTTP library) and eval the body directly in your Ruby code.
You can package everything in a custom require_remote function.
You could download and eval it
require "open-uri"
alias :require_old :require
def require(path)
return false if $".include?(path)
unless path=~ /\Ahttp:\/\/
return require_old(path)
end
eval(open(path).read)
$"<< path
true
end
Be aware, this code has no error checking for network outages nonexisting files, ... . I also believe it is in general not a good idea to require libraries this way, there are security and reliability problems in this approach. But maybe you have a valid usecase for this.
you can include a remote gem from within Gemfiles then it will download when you run bundle install
After reading this question and answers I wanted something a little more bullet proof and verbose that used a paradigm of creating a local file from a repo and then requiring it, only if it didn't already exist locally already. The request for the repo version is explicit via the method repo_require. Used on files you control, this approach improves security IMO.
# try local load
def local_require(filename, relative_path)
relative_flname = File.join(relative_path, filename)
require_relative(relative_flname)
end
# try loading locally first, try repo version on load error
# caution: only use with files you control access to!
def repo_require(raw_repo_prefix, filename, relative_path = '')
local_require(filename, relative_path)
rescue LoadError => e
puts e.message
require 'open-uri'
tempdir = Dir.mktmpdir("repo_require-")
temp_flname = File.join(tempdir, File.basename(filename))
return false if $LOADED_FEATURES.include?(temp_flname)
remote_flname = File.join(raw_repo_prefix, filename)
puts "file not found locally, checking repo: #{remote_flname}"
begin
File.open(temp_flname, 'w') do |f|
f.write(URI.parse(remote_flname).read)
end
rescue OpenURI::HTTPError => e
raise "Error: Can't load #{filename} from repo: #{e.message} - #{remote_flname}"
end
require(temp_flname)
FileUtils.remove_entry(tempdir)
end
Then you could call repo_require like this:
repo_require('https://raw.githubusercontent.com/username/reponame/branch',
'filename', 'relative_path')
The relative_path would the the relative path you would use for the file if the repo was locally installed. For example, you may have something like require_relative '../lib/utils.rb'. In this example filename='lib/utils.rb' and relative_path='..'. This information allows the repo url to be constructed correctly as it does not use the relative path portion.
Related
What is the difference between require_relative and require in Ruby?
Just look at the docs:
require_relative complements the builtin method require by allowing you to load a file that is relative to the file containing the require_relative statement.
For example, if you have unit test classes in the "test" directory, and data for them under the test "test/data" directory, then you might use a line like this in a test case:
require_relative "data/customer_data_1"
require_relative is a convenient subset of require
require_relative('path')
equals:
require(File.expand_path('path', File.dirname(__FILE__)))
if __FILE__ is defined, or it raises LoadError otherwise.
This implies that:
require_relative 'a' and require_relative './a' require relative to the current file (__FILE__).
This is what you want to use when requiring inside your library, since you don't want the result to depend on the current directory of the caller.
eval('require_relative("a.rb")') raises LoadError because __FILE__ is not defined inside eval.
This is why you can't use require_relative in RSpec tests, which get evaled.
The following operations are only possible with require:
require './a.rb' requires relative to the current directory
require 'a.rb' uses the search path ($LOAD_PATH) to require. It does not find files relative to current directory or path.
This is not possible with require_relative because the docs say that path search only happens when "the filename does not resolve to an absolute path" (i.e. starts with / or ./ or ../), which is always the case for File.expand_path.
The following operation is possible with both, but you will want to use require as it is shorter and more efficient:
require '/a.rb' and require_relative '/a.rb' both require the absolute path.
Reading the source
When the docs are not clear, I recommend that you take a look at the sources (toggle source in the docs). In some cases, it helps to understand what is going on.
require:
VALUE rb_f_require(VALUE obj, VALUE fname) {
return rb_require_safe(fname, rb_safe_level());
}
require_relative:
VALUE rb_f_require_relative(VALUE obj, VALUE fname) {
VALUE base = rb_current_realfilepath();
if (NIL_P(base)) {
rb_loaderror("cannot infer basepath");
}
base = rb_file_dirname(base);
return rb_require_safe(rb_file_absolute_path(fname, base), rb_safe_level());
}
This allows us to conclude that
require_relative('path')
is the same as:
require(File.expand_path('path', File.dirname(__FILE__)))
because:
rb_file_absolute_path =~ File.expand_path
rb_file_dirname1 =~ File.dirname
rb_current_realfilepath =~ __FILE__
Summary
Use require for installed gems
Use require_relative for local files
require uses your $LOAD_PATH to find the files.
require_relative uses the current location of the file using the statement
require
Require relies on you having installed (e.g. gem install [package]) a package somewhere on your system for that functionality.
When using require you can use the "./" format for a file in the current directory, e.g. require "./my_file" but that is not a common or recommended practice and you should use require_relative instead.
require_relative
This simply means include the file 'relative to the location of the file with the require_relative statement'. I generally recommend that files should be "within" the current directory tree as opposed to "up", e.g. don't use
require_relative '../../../filename'
(up 3 directory levels) within the file system because that tends to create unnecessary and brittle dependencies. However in some cases if you are already 'deep' within a directory tree then "up and down" another directory tree branch may be necessary. More simply perhaps, don't use require_relative for files outside of this repository (assuming you are using git which is largely a de-facto standard at this point, late 2018).
Note that require_relative uses the current directory of the file with the require_relative statement (so not necessarily your current directory that you are using the command from). This keeps the require_relative path "stable" as it always be relative to the file requiring it in the same way.
From Ruby API:
require_relative complements the
builtin method require by allowing you
to load a file that is relative to the
file containing the require_relative
statement.
When you use require to load a file,
you are usually accessing
functionality that has been properly
installed, and made accessible, in
your system. require does not offer a
good solution for loading files within
the project’s code. This may be useful
during a development phase, for
accessing test data, or even for
accessing files that are "locked" away
inside a project, not intended for
outside use.
For example, if you have unit test
classes in the "test" directory, and
data for them under the test
"test/data" directory, then you might
use a line like this in a test case:
require_relative "data/customer_data_1"
Since neither
"test" nor "test/data" are likely to
be in Ruby’s library path (and for
good reason), a normal require won’t
find them. require_relative is a good
solution for this particular problem.
You may include or omit the extension
(.rb or .so) of the file you are
loading.
path must respond to to_str.
You can find the documentation at http://extensions.rubyforge.org/rdoc/classes/Kernel.html
The top answers are correct, but deeply technical. For those newer to Ruby:
require_relative will most likely be used to bring in code from another file that you wrote.
for example, what if you have data in ~/my-project/data.rb and you want to include that in ~/my-project/solution.rb? in solution.rb you would add require_relative 'data'.
it is important to note these files do not need to be in the same directory. require_relative '../../folder1/folder2/data' is also valid.
require will most likely be used to bring in code from a library someone else wrote.
for example, what if you want to use one of the helper functions provided in the active_support library? you'll need to install the gem with gem install activesupport and then in the file require 'active_support'.
require 'active_support/all'
"FooBar".underscore
Said differently--
require_relative requires a file specifically pointed to relative to the file that calls it.
require requires a file included in the $LOAD_PATH.
I just saw the RSpec's code has some comment on require_relative being O(1) constant and require being O(N) linear. So probably the difference is that require_relative is the preferred one than require.
I want to add that when using Windows you can use require './1.rb' if the script is run local or from a mapped network drive but when run from an UNC \\servername\sharename\folder path you need to use require_relative './1.rb'.
I don't mingle in the discussion which to use for other reasons.
absolute path
require './app/example_file.rb'
shortened name
require_relative 'example_file'
In this blog post he gives this example of a Ruby config file.
config do
allow ['server.com', `hostname`.strip]
vhost 'api.server.com' do
path ‘/usr/local/api’
end
vhost 'www.server.com' do
path '/usr/local/web'
end
%w{wiki blog support}.each do |host|
vhost "#{host}.server.com" do
path "/usr/local/#{host}"
end
end
end
I think of a hash after a config file have been loaded, but maybe that is not how this type of configs are intended for...
Update
If I execute it, I get
$ ruby config.rb
config.rb:2:in `<main>': undefined method `config' for main:Object (NoMethodError)
Question
What Ruby code is needed to parse and dump the content of this config file?
That config example is not directly loadable and, if I understand the blog post author correctly, it's not meant to be either so there's no easy way of loading/parsing that example.
The key part is in the blog post where he states "build simple DSLs to design semantically robust config files without the underlying ruby being conspicuous" (my emphasis). The 'underlying ruby' I take to mean the code that enables the DSL elements you're seeing such as 'config' and 'vhost'.
Your original question was, however, what code is required to load that config - below is a sample of something would work, full implementation is up to you and tbh I'm pretty sure there are cleaner, "better" ways of doing the same.
class AppConfig
attr_accessor :hosts
def allow(hosts)
#hosts = hosts
end
def vhost(hostname)
end
def process_config(&block)
instance_eval(&block)
end
end
def config(&block)
config = AppConfig.new
config.process_config &block
puts "Hosts are: #{config.hosts}"
end
load 'config.rb'
I am trying to create an LWRP that will call the resource that is defined within itself. My cookbook's structure is as follows:
In the machine cookbook's provider, I have a code snippet as follows:
require 'chef/provisioning' # driver for creating machines
require '::File'
def get_environment_json
##environment_template = JSON.parse(File::read(new_resource.template_path + "environment.json"))
return ##environment_template
end
The code is only trying to read a json file and I am using File::read for it.
I keep getting an error as follows:
LoadError
cannot load such file -- ::File
Does anyone know how I can use File::read inside my LWRP's provider?
OK, so the prior two answers are both half right. You have two problems.
First, you can't require ::File as it's already part of Ruby. This is the cause of your error.
Second, if you call File.read you will grab Chef's File not ruby's. You need to do a ::File.read to use Ruby's File class.
require '::File'
Is incorrect and is causing the LoadError. Delete this line. You don't need it. File is part of the Ruby core and doesn't need to be required.
To further explain, the string argument to require represents the file name of the library you want to load. So, it should look like require "file", or require "rack/utils".
It happens becuase Chef already has a file resource. We have to use the Ruby File class in a recipe.We use ::File to use the Ruby File class to fix this issue. For example:
execute 'apt-get-update' do
command 'apt-get update'
ignore_failure true
only_if { apt_installed? }
not_if { ::File.exist?('/var/lib/apt/periodic/update-success-stamp') }
end
Source: https://docs.chef.io/ruby.html#ruby-class
I'm currently working on a script (command line tool) for work to help me manage expose consoles.
At first I was passing three arguments to the script each time I used it to login into the consoles, for example:
$ nexose-magic.rb -u user -p password -i 192.168.1.2 --display-scans
It's not very efficient, so I created a config.yml file that stores console information in a hash.
$ nexpose-magic.rb -c console --display-scans
I believe the tool will be useful to admins out there, so I'd like to share it in a gem. I can't figure out how to get my config.yml file to work with a gem install..It can't find the config.yml file! It's easy to point it at the relative path in my development directory, but once I create a gem that relative path isn't so relative anymore. How do I point nexpose-magic.rb at the config.yml file?
Is there a better way to handle something like this?
You can create a gem that include a configure class. This class has a load method that will take a directory as an argument. Then, you can pass the directory where you are currently working.
A nice way of preparing your gem is to create a Configuration singleton class in your gem:
require 'singleton'
class Configuration
include Singleton
attr_accessor :config, :app_path
def load(app_path)
#app_path = app_path
#load the config file and store the data
#config = YAML.load_file( File.join(#app_path,'config','config.yml'))
end
end
In your main class :
module MyFancyGem
class << self
#define a class method that configure the gem
def configure(app_path)
# Call load method with given path
config.load(app_path)
end
# MyFancyGem.config will refer to the singleton Configuration class
def config
MyFancyGem::Configuration.instance
end
end
end
Usage:
-Working directory
- my_new_script
- Gemfile
- config/
- config.yml
In my_new_script :
require 'bundler'
Bundler.setup(:default)
require 'my_fancy_gem'
MyFancyGem.configure(File.join(File.dirname(__FILE__),"./")) #here, you define the path
MyFancyGem.hello_world
I hope that's clear enough. I was actually about to write a blog post to explain this particular point (I hope in a more complete version of it). Let me know if you're interested !
I am trying to use the main gem for making command line utilities. This was presented in a recent Ruby Rogues podcast.
If I put all the code in one file and require that file, then rspec gives me an error, as the main dsl regards rpsec as a command line invocation of the main utility.
I can break out a method into a new file and have rspec require that file. Suppose you have this program, but want to put the do_something method in a separate file to test with rspec:
require 'main'
def do_something(foo)
puts "foo is #{foo}"
end
Main {
argument('foo'){
required # this is the default
cast :int # value cast to Fixnum
validate{|foo| foo == 42} # raises error in failure case
description 'the foo param' # shown in --help
}
do_something(arguments['foo'].value)
}
What is the convenient way to distribute/deploy a ruby command line program with multiple files? Maybe create a gem?
You are on the right track for testing - basically you want your "logic" in separate files so you can unit test them. You can then use something like Aruba to do an integration test.
With multiple files, your best bet is to distribute it as a RubyGem. There's lots of resources out there, but the gist of it is:
Put your executable in bin
Put your files in lib/YOUR_APP/whatever.rb where "YOUR_APP" is the name of your app. I'd also recommend namespacing your classes with modules named for your app
In your executable, require the files in lib as if lib were in the load path
In your gemspec, make sure to indicate what your bin files are and what your lib files are (if you generate it with bundle gem and are using git, you should be good to go)
This way, your app will have access to the files in lib at runtime, when installed with RubyGems. In development, you will need to either do bundle exec bin/my_app or RUBYLIB=lib bin/my_app. Point is, RubyGems takes care of the load path at runtime, but not at development time.