I am trying to execute rspec from ruby, and get the status or number of failures from a method or something like that. Actually I am running something like this:
system("rspec 'myfilepath'")
but I only can get the string returned by the function. Is there any way to do this directly using objects?
I think the best way would be using RSpec's configuration and Formatter. This would not involve parsing the IO stream, also gives much richer result customisation programmatically.
RSpec 2:
require 'rspec'
config = RSpec.configuration
# optionally set the console output to colourful
# equivalent to set --color in .rspec file
config.color = true
# using the output to create a formatter
# documentation formatter is one of the default rspec formatter options
json_formatter = RSpec::Core::Formatters::JsonFormatter.new(config.output)
# set up the reporter with this formatter
reporter = RSpec::Core::Reporter.new(json_formatter)
config.instance_variable_set(:#reporter, reporter)
# run the test with rspec runner
# 'my_spec.rb' is the location of the spec file
RSpec::Core::Runner.run(['my_spec.rb'])
Now you can use the json_formatter object to get result and summary of a spec test.
# gets an array of examples executed in this test run
json_formatter.output_hash
An example of output_hash value can be found here:
RSpec 3
require 'rspec'
require 'rspec/core/formatters/json_formatter'
config = RSpec.configuration
formatter = RSpec::Core::Formatters::JsonFormatter.new(config.output_stream)
# create reporter with json formatter
reporter = RSpec::Core::Reporter.new(config)
config.instance_variable_set(:#reporter, reporter)
# internal hack
# api may not be stable, make sure lock down Rspec version
loader = config.send(:formatter_loader)
notifications = loader.send(:notifications_for, RSpec::Core::Formatters::JsonFormatter)
reporter.register_listener(formatter, *notifications)
RSpec::Core::Runner.run(['spec.rb'])
# here's your json hash
p formatter.output_hash
Other Resources
Detailed work through
Gist example
I suggest you to take a look into rspec source code to find out the answer. I think you can start with example_group_runner
Edit: Ok here is the way:
RSpec::Core::Runner::run(options, err, out)
Options - array of directories, err & out - streams. For example
RSpec::Core::Runner.run(['spec', 'another_specs'], $stderr, $stdout)
Your problem is that you're using the Kernel#system method to execute your command, which only returns true or false based on whether or not it can find the command and run it successfully. Instead you want to capture the output of the rspec command. Essentially you want to capture everything that rspec outputs to STDOUT. You can then iterate through the output to find and parse the line which will tell you how many examples were run and how many failures there were.
Something along the following lines:
require 'open3'
stdin, stdout, stderr = Open3.popen3('rspec spec/models/my_crazy_spec.rb')
total_examples = 0
total_failures = 0
stdout.readlines.each do |line|
if line =~ /(\d*) examples, (\d*) failures/
total_examples = $1
total_failures = $2
end
end
puts total_examples
puts total_failures
This should output the number of total examples and number of failures - adapt as needed.
This one prints to console and at the same time captures the message. The formatter.stop is just a stub function, I don't know what it is for normally, I had to include it to use DocumentationFormatter. Also the formatter output contains console coloring codes.
formatter = RSpec::Core::Formatters::DocumentationFormatter.new(StringIO.new)
def formatter.stop(arg1)
end
RSpec.configuration.reporter.register_listener(formatter, :message, :dump_summary, :dump_profile, :stop, :seed, :close, :start, :example_group_started)
RSpec::Core::Runner.run(['test.rb','-fdocumentation'])
puts formatter.output.string
Related
I am learning ruby and trying to write a unit test with rspec for the following method:
def get()
options = {}
OptionParser.new do |opt|
opt.banner = 'Usage: validate-gitlab-ci [options]'
opt.on('-f', '--yaml YAML-PATH', 'Path to .gitlab-ci.yml') { |o| options[:yamlFile] = o }
opt.on('-l', '--base-url GitLab url', 'GitLab API url') { |o| options[:baseUrl] = o + API_PATH }
opt.on('-t', '--timeout[TIMEOUT]', Integer, 'Api timeout in seconds') { |o| options[:timeout] = o || 10 }
opt.on('-v', '--version', 'Program version') { |o| options[:version] = o }
end.parse!
validateUrl!(options[:baseUrl])
validateYamlFile!(options[:yamlFile])
#baseUrl = options[:baseUrl]
#pathToYamlFile = options[:yamlFile]
end
The code for my unit test so far is:
RSpec.describe Gitlab::Lint::Client::Args do
describe "#get" do
context "when arguments are valid" do
it "sets baseUrl and pathToYamlFile" do
io = StringIO.new
io.puts "glab-lint --base-url=https://example.com --yaml=valid.ym\n"
io.rewind
$stdin = io
args = Gitlab::Lint::Client::Args.new
args.get()
expect(args.baseUrl).to.eq("https://example.com")
end
end
end
end
I am trying to mock STDIN for OptionParser. However, upon executing the test the following error is displayed:
OptionParser::InvalidOption:
invalid option: --pattern
This is raised by the end.parse! line in the get() method
Has anyone managed to test OptionsParser with stdin mocked?
Update
I think what is happening is that some RSpec options, e.g. --pattern?? are being captured in STDIN and passed to script??? Or .... RSpec is consuming the stdin options??
Reading this post seems to suggest that the desired functionality is not possible with RSpec....if this is indeed true then I will migrate over to using alternative test frameworks in future for CLI projects that use ARGV. There is a workaround suggested here but that suggests using environment variables for capturing commmand line arguments. In this case that would require further refactoring of the software under test, purely to suit the capabilities of the RSpec test framework!!
If I add a puts statement to display the contents of ARGV in the test script it confirms this is the case, with this output:
--pattern
spec/**{,/*/**}/*_spec.rb
[--base-url=https://gitlab.com --yaml=valid.ym]
So.....as a complete newbie to RSpec.....my options are:
Update the signature of the get method to accept an args array:
def get(args)
options = {}
OptionParser.new do |opt|
...
end.parse!(args)
end
This delays the issue with testing the code that reads from ARGV further up the call hierarchy
Modify ARGV shifting the first two arguments out of the array and then after the test has completed restore ARGV to original state. Looks like something similar has already been tried here without success.
Some other configuration that I am not aware of as a newbie to RSpec
Investigate alternative options, e.g. minitest, that maybe do not modify the ARGV array??
Further information regarding options 3 and 4 appreciated....
You can use RSpec to mock STDIN. For example:
STDIN.should_receive(:read).and_return("glab-lint --base-url=https://example.com --yaml=valid.yml")
Alternatively, you can invoke your actual command line program using backticks or system, and assert on the response.
I would like to know whether I can get source code a method on the fly, and whether I can get which file is this method in.
like
A.new.method(:a).SOURCE_CODE
A.new.method(:a).FILE
Use source_location:
class A
def foo
end
end
file, line = A.instance_method(:foo).source_location
# or
file, line = A.new.method(:foo).source_location
puts "Method foo is defined in #{file}, line #{line}"
# => "Method foo is defined in temp.rb, line 2"
Note that for builtin methods, source_location returns nil. If want to check out the C source code (have fun!), you'll have to look for the right C file (they're more or less organized by class) and find the rb_define_method for the method (towards the end of the file).
In Ruby 1.8 this method does not exist, but you can use this gem.
None of the answers so far show how to display the source code of a method on the fly...
It's actually very easy if you use the awesome 'method_source' gem by John Mair (the maker of Pry):
The method has to be implemented in Ruby (not C), and has to be loaded from a file (not irb).
Here's an example displaying the method source code in the Rails console with method_source:
$ rails console
> require 'method_source'
> I18n::Backend::Simple.instance_method(:lookup).source.display
def lookup(locale, key, scope = [], options = {})
init_translations unless initialized?
keys = I18n.normalize_keys(locale, key, scope, options[:separator])
keys.inject(translations) do |result, _key|
_key = _key.to_sym
return nil unless result.is_a?(Hash) && result.has_key?(_key)
result = result[_key]
result = resolve(locale, _key, result, options.merge(:scope => nil)) if result.is_a?(Symbol)
result
end
end
=> nil
See also:
https://rubygems.org/gems/method_source
https://github.com/banister/method_source
http://banisterfiend.wordpress.com/
Here is how to print out the source code from ruby:
puts File.read(OBJECT_TO_GET.method(:METHOD_FROM).source_location[0])
Without dependencies
method = SomeConstant.method(:some_method_name)
file_path, line = method.source_location
# puts 10 lines start from the method define
IO.readlines(file_path)[line-1, 10]
If you want use this more conveniently, your can open the Method class:
# ~/.irbrc
class Method
def source(limit=10)
file, line = source_location
if file && line
IO.readlines(file)[line-1,limit]
else
nil
end
end
end
And then just call method.source
With Pry you can use the show-method to view a method source, and you can even see some ruby c source code with pry-doc installed, according pry's doc in codde-browing
Note that we can also view C methods (from Ruby Core) using the
pry-doc plugin; we also show off the alternate syntax for show-method:
pry(main)> show-method Array#select
From: array.c in Ruby Core (C Method):
Number of lines: 15
static VALUE
rb_ary_select(VALUE ary)
{
VALUE result;
long i;
RETURN_ENUMERATOR(ary, 0, 0);
result = rb_ary_new2(RARRAY_LEN(ary));
for (i = 0; i < RARRAY_LEN(ary); i++) {
if (RTEST(rb_yield(RARRAY_PTR(ary)[i]))) {
rb_ary_push(result, rb_ary_elt(ary, i));
}
}
return result;
}
I created the "ri_for" gem for this purpose
>> require 'ri_for'
>> A.ri_for :foo
... outputs the source (and location, if you're on 1.9).
GL.
-r
Internal methods don't have source or source location (e.g. Integer#to_s)
require 'method_source'
User.method(:last).source
User.method(:last).source_location
I had to implement a similar feature (grab the source of a block) as part of Wrong and you can see how (and maybe even reuse the code) in chunk.rb (which relies on Ryan Davis' RubyParser as well as some pretty funny source file glomming code). You'd have to modify it to use Method#source_location and maybe tweak some other things so it does or doesn't include the def.
BTW I think Rubinius has this feature built in. For some reason it's been left out of MRI (the standard Ruby implementation), hence this hack.
Oooh, I like some of the stuff in method_source! Like using eval to tell if an expression is valid (and keep glomming source lines until you stop getting parse errors, like Chunk does)...
I have the following Ruby code:
def report_deviation(departure)
deviation = departure.fetch('Dev')
trip = departure.fetch('Trip')
run_id = trip.fetch('RunId')
headsign = trip.fetch('InternetServiceDesc')
timestamp = Time.now.strftime '%l:%M %P'
FileUtils.mkdir 'log' unless File.directory? 'log'
File.open DAILY_LOG_FILE, 'a' do |file|
file.puts "#{timestamp}, #{name}: Run #{run_id} (#{headsign}), deviation #{deviation}"
end
end
Tested by the following RSpec code:
describe 'report_deviation' do
let(:departure) { double }
let(:trip) { double }
let(:file) { double }
it 'appends to a log file with the correct entry format' do
expect(departure).to receive(:fetch).with('Trip').and_return trip
expect(departure).to receive(:fetch).with('Dev').and_return 'DEVIATION'
expect(trip).to receive(:fetch).with('RunId')
.and_return 'RUN'
expect(trip).to receive(:fetch).with('InternetServiceDesc')
.and_return 'HEADSIGN'
stub_const 'DeviationValidator::DAILY_LOG_FILE', :log_file
expect(File).to receive(:open).with(:log_file, 'a').and_yield file
timestamp = '12:00 pm: Run RUN (HEADSIGN), deviation DEVIATION'
expect(file).to receive(:puts).with timestamp
Timecop.freeze(Time.new 2017, 7, 31, 12) { report_deviation(departure) }
end
end
But when I run I receive the failure message:
`name` is not available from within an example (e.g. an `it` block) or from constructs that run in the scope of an example (e.g. `before`, `let`, etc). It is only available on an example group (e.g. a `describe` or `context` block).
The word name isn't written anywhere in here, and if I remove the final line of the test (which invokes the actual code) I get the test failures I would expect for unsatisfied exceptions. I normally would boil my code down to the pieces that are causing the error, but I have no idea what's causing the error.
For what it's worth, the specific line number mentioned in the backtrace is the file.puts within the File.open block - but I don't understand why that should cause a failure. I've set up test doubles such that those objects are nothing special - File receives open and yields file, whose only job is to listen for receiving puts with the string I expect. So what piece of code is calling what happens to be a keyword RSpec method name?
The problem is from rspec gem, if you are using Rails 6 you need to use gem 'rspec-rails', '~> 4.1.0'
name is not a keyword RSpec method, it's a method that report_deviation is trying to call
file.puts "#{timestamp}, #{name}: Run #{run_id} (#{headsign}), deviation #{deviation}"
but the method is not defined.
You need to define the name method in the class where report_deviation is defined. Or, if report_deviation is defined and used in the spec file, add a simple variable called name:
describe 'report_deviation' do
let(:departure) { double }
let(:trip) { double }
let(:file) { double }
let(:name) { "simple name" }
...
`name` is not available from within an example (e.g. an `it` block) [...]
I had a similar problem today. The final solution for the issue for now with a monkeypatch to go back to using method_name.
Create config/initializers/monkeypatches.rb file and fill inside with the following lines.
# config/initializers/monkeypatches.rb
#
# This fixes what seems to be a bug introduced by
# https://github.com/rails/rails/pull/37770
# "Modify ActiveRecord::TestFixtures to not rely on AS::TestCase:"
#
module ActiveRecord::TestFixtures
def run_in_transaction?
use_transactional_tests &&
!self.class.uses_transaction?(method_name) # this monkeypatch changes `name` to `method_name`
end
end
Credits: https://github.com/graphql-devise/graphql_devise/issues/42
I've just started using OptionParser for Ruby and I wanted to use flags that would use more than just one argument.
For instance, I would like to be able to run:
script --move src dst
Note how src and dst are not separated using a coma.
My initial idea was:
opts.on("-m src dst", "--move src dst ", "move file from SRCto DST") do |src|
# do something
end
But this is not working. I assume that this is not the right approach. But how could this be done?
The example under the "Complete Example" section of the OptionParser details how a list of items can be accepted.
Here is a sample program based on that example. The third parameter Array in opts.on indicates that input src, dst should be used to create an array. To run this sample, you need to do gem install trollop.
# test.rb
require 'optparse'
options = {}
OptionParser.new do |opt|
opt.on("-m src, dst", "--move src, dst", Array, "Move from src to dst") do |list|
options[:src] = list[0]
options[:dst] = list[1]
end
end.parse!
puts options # It's a hash of parsed options
Sample run:
> ruby test.rb -m from,to
{:src=>"src", :dst=>"dst"}
>ruby test.rb -h
Usage: test [options]
-m, --move src, dst Move from src to dst
The above script forces one to separate the options using comma.
As indicated by "Really Cheap Command-Line Option Parsing in Ruby", there seems to be a gem, trollop, that can be quite easy to use for command-line parsing.
A sample program based on Trollop is given below, which allows usage of spaces for specifying options with multiple values
# test.rb
require "trollop"
opts = Trollop::options do
banner "Command line parsing using Trollop"
opt :move, "--move src dst', Move from src to dst", :short => "-m", :long => "--move", :type => :strings
end
# An array of option values
p opts.move
Sample run:
>ruby test.rb -m hello world
["hello", "world"]
>ruby test.rb -h
Command line parsing using Trollop
-m, --move=<s+> '--move src dst', Move from src to dst
-h, --help Show this message
There is a subtle difference between the help output between the two approaches. Trollop produces help text where --move=<s+> does not indicate clearly that it needs accepts two values, so I had to repeat the command syntax description.
OptionParser doesn't support that; It could be patched to do so, but I'm not sure it's worth the trouble.
Consider this code:
require 'optparse'
options = {}
OptionParser.new do |opt|
opt.on('-m', '--move') { |o| options[:move] = o }
end.parse!
from_name, to_name = ARGV
puts "Should move: #{ options.key?(:move) }"
puts "From: #{ from_name }"
puts "To: #{ to_name }"
Saving it and running it with various combinations of the parameters returns:
> ruby test.rb --move from to
Should move: true
From: from
To: to
> ruby test.rb from to
Should move: false
From:
To:
If the code is supposed to move files by default then don't bother with the --move flag, simply use:
test.rb from to
and consider removing the OptionParser block entirely.
If the code is supposed to normally copy with the option to move, then --move becomes more sensible to act as a flag that moving is desired.
ruby test.rb --move from to
I'd have code that tests for options[:move] and run the code to move instead of copy at that point.
In either case, the filenames shouldn't be tied to the flag, they should be supplied separately and retrieved from ARGV after OptionParser has finished parsing the command-line and removing entries it's handled.
I'm trying to test a little gem that makes downloads from youtube using 'youtube-dl'.
I want to test the output from the command youtube-dl [url] --get-title but I dont know how I do that.
This is my code:
module Youruby
class Youtube
YT_DL = File.join(File.expand_path(File.dirname(__FILE__)), "../bin/youtube-dl")
def initialize(id)
#id = id
end
def get_title
system(YT_DL, '--get-title', get_url)
end
end
end
And this is my test:
require "spec_helper"
require "youruby"
describe Youruby do
it "get video title" do
video = Youruby::Youtube.new('uaEJvYWc2ag')
video.get_title.should == "FFmpeg-slowmotion.1"
end
end
When I run the tests I get this error:
Failure/Error: video.get_title.should == "FFmpeg-slowmotion.1"
expected: "FFmpeg-slowmotion.1"
got: true (using ==)
Diff:
## -1,2 +1,2 ##
-"FFmpeg-slowmotion.1"
+true
How do I do that?
Seems like your test is OK, and the implementation is failing (so, is OK for the test to report the fail)
On the implementation, Instead of using system method (which return true/false according the return code of the command), use backtick (which return the string with the output of the command)
def get_title
`#{YT_DL} --get-file #{get_url}`
end
ALso, as additional note, is not good for your implementation to depend on external commands (from Unit testing point of view), maybe you want to mock external system command execution (or not, you maybe know what strategy is better for your particular case)