According to this piece of documentation, it is possible to achieve the following output format by selecting the format documentation:
something
does something that passes
does something that fails (FAILED - 1)
does something that is pending (PENDING: Not Yet Implemented)
Is it possible to slightly edit this so it outputs as:
something
does something (SUCCESS)
does something (FAIL)
does something (PENDING)
Basically I would like the result of the test displayed no matter what - instead of just logging explicitly the failures and the pending ones.
I was able to do this by subclassing RSpec::Core::Formatters::DocumentationFormatter. Create the following file as spec/formatters/custom_formatter.rb:
class CustomFormatter < RSpec::Core::Formatters::DocumentationFormatter
RSpec::Core::Formatters.register self
private
def passed_output(example)
format_output(example, 'SUCCESS', :success)
end
def pending_output(example, _message)
format_output(example, 'PENDING', :pending)
end
def failure_output(example)
format_output(example, 'FAILED', :failure)
end
def format_output(example, status_text, code_or_symbol)
RSpec::Core::Formatters::ConsoleCodes.wrap(
"#{current_indentation}#{example.description.strip} (#{status_text})",
code_or_symbol
)
end
end
Then run the specs using this:
rspec --require formatters/custom_formatter --format CustomFormatter
Instead of --require formatters/custom_formatter, you can also require the formatter on spec/spec_helper.rb, e.g.
require_relative 'formatters/custom_formatter'
Then you only need to run this:
rspec --format CustomFormatter
If you want CustomFormatter to be the default formatter, you can add the command line options to .rspec configuration file at your project root. Here's how it should look like:
--require spec_helper
--require /path/to/custom_formatter.rb
--format CustomFormatter
With that, you no longer need to specify any command line arguments to use CustomFormatter.
Documentations and references:
https://relishapp.com/rspec/rspec-core/v/3-6/docs/formatters/custom-formatters
http://www.rubydoc.info/gems/rspec-core/RSpec/Core/Formatters
https://github.com/rspec/rspec-core/blob/v3.7.0/lib/rspec/core/formatters/documentation_formatter.rb
You can not change the existing RSpec formatters, but you can create your own
When RSpec's built-in output formatters don't, however, give you everything
you need, you can write your own custom formatter and tell RSpec to use that
one instead. The simplest way is to subclass RSpec's BaseTextFormatter, and
then override just the methods that you want to modify.
Related
I want output line, which was failed during rspec comparasion inside example, but I don't know how to do it best.
For example I have test like this:
require 'rspec'
describe 'My behaviour' do
it 'should do something' do
test_string = 'test'
expect(test_string).to eq('failed_test')
end
after :each do |example|
puts example.metadata[:expect_line]
end
end
And I want to outputed line in after :each be
"expect(test_string).to eq('failed_test')"
I know, I have acces to example.metadata[:location] which return something like "./spec/test_spec.rb:4" and I can parse it and extract line, but is there already something like I need hided in whole example structure?
Update:
I just understand. that example.metadata[:location] return not failed line, but actually line in whitch it started, so it have no use for me :(
So question still exist - how to get failed line?
This information isn't hidden anywhere in the example structure that I'm aware of. RSpec's default output shows the line that failed:
Failures:
1) My Description should fail
Failure/Error: expect(1).to eq(2)
expected: 2
got: 1
If we look at how rspec itself gets this in rspec/core/formatters/exception_presenter.rb and rspec/core/formatters/snippet_extractor.rb, it appears that they go through the backtrace of the example exception to find the spec file and extract the line (similar to what you mentioned). If there was an easier way to pull that out of the example, I would think RSpec itself would use that :)
This is the contents of my .rspec_parallel file. I am using parallel_tests gem to run tests in multiple browser instances. To my knowledge, the gem uses the same formatter options available in RSpec.
--format html --out results<%= ENV['TEST_ENV_NUMBER'] %>.html
This works fantastic and I'm able to get the HTML output I normally see from RSpec. However, all of the 'puts' messages and basic standard output is logged to my console window, and not to the HTML files.
How can I get this output into each individual HTML file that I have set up?
puts will output to $stdout where as output is actually an instance variable of the RSpec::Core::Formatters::BaseFormatter class. output is defaulted to $stdout but when you pass in a string it determines that it should create a new StringIO and then output this to the given file name. Thus puts will not append to the #output variable.
You could do something ugly like create a runner file like
File.open('some_file_name.html','w+') do |file|
file << `rspec spec --format html`
end
Then this file will have the output from $stdoutbut your puts code will not be html formatted in this case. Other than that you could try building your own Custom Formatter but it will probably take quite a bit of source searching to make sure you can capture everything appropriately.
That being said it does seem the reporter was exposed for adding custom messages but I am uncertain of how to use this appropriately See Pull Request 1866
Seems it would be something like
it "has a name" do |ex|
ex.reporter.message("Custom Message Here")
#actual test
end
but the html formatter seems to ignore this. I can see the output in $stdout but not in the html file itself.
Best of luck.
I'm using an external API that takes a key string, and would like to pass this key string to the test suite. Something like:
rake test [key=api_key]
The code together with the tests will be open sourced, but I'm not allowed to distribute my key string to other users, so I cannot put it in the test file. Can I pass it as a parameter?
You have two options. Pass it as an environment variable:
API_KEY='key' rake test
You can then access this through the ENV object in your test:
key = ENV['API_KEY']
Second option is to put this key in a file (e.g. key.txt) and you read it from that. To ensure that you don't distribute that file with your code, add it to your .gitignore file (or whatever is the ignore file used by your SCM)
Thank you very much!
I actually was thinking of putting it into a file and gitignoring it, but ended up passing a parameter to rake. May be, I will combine both (it's a long key).
Modify the Rakefile code for the :test task, such as adding a
parameter to it.
task :test, :key do |t, k|
result = system("ruby -Ilib -Itest -e 'ARGV.each { |f| load(f) if File.exists?(f)}' test/unit/* '#{k[:key]}'")
exit(result ? 0 : 1)
end
Call rake test['blah-blah']
It may take more then one key if needed.
I have some test on Rspec, which looks like this:
describe "description" do
before :each do
do_before()
end
it "something_1" do
...
end
it "something_2" do
...
end
end
I know that I can get name of current test ("something_1") by using
example.description
Is there any way to get array of all descriptions in before :each area?
rspec -f d --color --dry-run filename
Works for me in rspec 3.5.2, lists all tests without running them
There used to be a way to do this using the tag --dry-run, but that has been removed and no longer works.
You can use the -fd which is for format = documentation. This will show you a long list of all the specs that you've done and what they look like. It does, however, still run the test and show you any errors you have in the process. That said it's still a great way to list of all of your tests.
With Test::Unit, I can run:
ruby path/to/test.rb --name=test_name_that_i_want_to_run
Thus far, I have not been able to figure out how to do this with test/spec specifications. I am wondering if the way that specifications are automatically named does not allow me to do something like this.
Take the following spec for example:
require 'rubygems'
require 'spec'
describe 'tests' do
it 'should be true' do
1.should == 1
end
it 'should be false' do
1.should_not == 2
end
end
You can execute a single spec by using the -e flag and providing the portion specified by the it block. e.g. ruby my_spec.rb -e 'should be false'
After contacting the gem maintainer, Christian Neukirchen, I found out how to do this, so I am documenting it here for future reference.
specrb path/to/test.rb --name ".*should behave this way.*"
I needed to use the specrb test runner, an extended version Test::Unit's test runner, rather than just the ruby command.
You can also do this with the ruby command:
ruby path/to/test.rb -n "/should behave this way/"