For integrating our Rubocop reports into Jenkins, I'd like to generate both Checkstyles -- XML output and HTML output -- in one go.
I can achieve this when running Rubocop directly:
$ rubocop -r rubocop/formatter/checkstyle_formatter \
--format html --out reports/html/index.html \
--format RuboCop::Formatter::CheckstyleFormatter --out reports/checkstyle-result.xml
However, I can't figure out how to achieve the same result with RuboCop::RakeTask and the formatters array.
Here's what I have tried:
Using multiple --out entries in options
RuboCop::RakeTask.new(:rubocop_ci) do |t|
t.formatters = ['html', 'RuboCop::Formatter::CheckstyleFormatter']
t.options = ['--out', 'reports/checkstyle-result.xml',
'--out', 'reports/html/index.html']
end
This generates reports/checkstyle-result.xml, but prints the HTML output to STDOUT (presumably, Rubocop ignores the second --out parameter).
Put everything into the options, and ignore formatters completely
RuboCop::RakeTask.new(:rubocop_ci2) do |t|
t.options = ['--format', 'RuboCop::Formatter::CheckstyleFormatter',
'--out', 'reports/checkstyle-result.xml',
'--format', 'html',
'--out', 'reports/html/index.html']
end
This works, but is ugly; is it possible to use multiple entries in formatters with different output files?
Related
According to this piece of documentation, it is possible to achieve the following output format by selecting the format documentation:
something
does something that passes
does something that fails (FAILED - 1)
does something that is pending (PENDING: Not Yet Implemented)
Is it possible to slightly edit this so it outputs as:
something
does something (SUCCESS)
does something (FAIL)
does something (PENDING)
Basically I would like the result of the test displayed no matter what - instead of just logging explicitly the failures and the pending ones.
I was able to do this by subclassing RSpec::Core::Formatters::DocumentationFormatter. Create the following file as spec/formatters/custom_formatter.rb:
class CustomFormatter < RSpec::Core::Formatters::DocumentationFormatter
RSpec::Core::Formatters.register self
private
def passed_output(example)
format_output(example, 'SUCCESS', :success)
end
def pending_output(example, _message)
format_output(example, 'PENDING', :pending)
end
def failure_output(example)
format_output(example, 'FAILED', :failure)
end
def format_output(example, status_text, code_or_symbol)
RSpec::Core::Formatters::ConsoleCodes.wrap(
"#{current_indentation}#{example.description.strip} (#{status_text})",
code_or_symbol
)
end
end
Then run the specs using this:
rspec --require formatters/custom_formatter --format CustomFormatter
Instead of --require formatters/custom_formatter, you can also require the formatter on spec/spec_helper.rb, e.g.
require_relative 'formatters/custom_formatter'
Then you only need to run this:
rspec --format CustomFormatter
If you want CustomFormatter to be the default formatter, you can add the command line options to .rspec configuration file at your project root. Here's how it should look like:
--require spec_helper
--require /path/to/custom_formatter.rb
--format CustomFormatter
With that, you no longer need to specify any command line arguments to use CustomFormatter.
Documentations and references:
https://relishapp.com/rspec/rspec-core/v/3-6/docs/formatters/custom-formatters
http://www.rubydoc.info/gems/rspec-core/RSpec/Core/Formatters
https://github.com/rspec/rspec-core/blob/v3.7.0/lib/rspec/core/formatters/documentation_formatter.rb
You can not change the existing RSpec formatters, but you can create your own
When RSpec's built-in output formatters don't, however, give you everything
you need, you can write your own custom formatter and tell RSpec to use that
one instead. The simplest way is to subclass RSpec's BaseTextFormatter, and
then override just the methods that you want to modify.
This is the contents of my .rspec_parallel file. I am using parallel_tests gem to run tests in multiple browser instances. To my knowledge, the gem uses the same formatter options available in RSpec.
--format html --out results<%= ENV['TEST_ENV_NUMBER'] %>.html
This works fantastic and I'm able to get the HTML output I normally see from RSpec. However, all of the 'puts' messages and basic standard output is logged to my console window, and not to the HTML files.
How can I get this output into each individual HTML file that I have set up?
puts will output to $stdout where as output is actually an instance variable of the RSpec::Core::Formatters::BaseFormatter class. output is defaulted to $stdout but when you pass in a string it determines that it should create a new StringIO and then output this to the given file name. Thus puts will not append to the #output variable.
You could do something ugly like create a runner file like
File.open('some_file_name.html','w+') do |file|
file << `rspec spec --format html`
end
Then this file will have the output from $stdoutbut your puts code will not be html formatted in this case. Other than that you could try building your own Custom Formatter but it will probably take quite a bit of source searching to make sure you can capture everything appropriately.
That being said it does seem the reporter was exposed for adding custom messages but I am uncertain of how to use this appropriately See Pull Request 1866
Seems it would be something like
it "has a name" do |ex|
ex.reporter.message("Custom Message Here")
#actual test
end
but the html formatter seems to ignore this. I can see the output in $stdout but not in the html file itself.
Best of luck.
I'm looking to have reports output from my Cucumber tests. So far I can put the following:
cucumber features/feature_name.feature -f json -o C:/Reports
This will only produce a json report at the end of the test and you can't see the results as they're happening as you normally would with cucumber.
Can I get it to do both so that I will see the green/red steps within cmd and get the json report at the end?
Thank you
You can add as many report formatters as you want.
To include the default "pretty" formatter and the "json" formatter:
cucumber features/feature_name.feature -f pretty -f json -o C:/Reports/results.txt
It's now recommended to use -p json:C:/Reports/results.json.
I'm running a bunch of rspec tests in parallel using the parallel_tests framework. Before I parallelized the tests, I was outputting the results from the tests into an html file like so:
rspec --format html --out tmp/index.html <pattern>
Now it looks more like this:
parallel:spec --format html --out tmp/index.html <pattern>
However, now that the tests are running in parallel, each test is generating its own html file, and since they all use the same path (tmp/index.html), the last test to finish overwrites the output html file and I'm left with a report of only that one test. How can I generate either a single html file which contains the aggregated results of all of my tests (this would be ideal)? And if that's not possible, how can I output each test to its own output html file so they don't all overwrite each other?
I tried using the built-in loggers in the parallel_test project (ParallelTests::RSpec::RuntimeLogger, ParallelTests::RSpec::SummaryLogger, and ParallelTests::RSpec::FailuresLogger) but those all just generate simple text files instead of the nice html files like rspec does. I also saw this question here but I'm not using cucumber, so this doesn't really apply to me. I tried putting --format html --out tmp/report<%= ENV['TEST_ENV_NUMBER'] %>.html in my .rspec_parallel file, but that didn't have any effect.
I had to write my own formatter, here's the code in case anyone else runs into this problem:
require 'fileutils'
RSpec::Support.require_rspec_core "formatters"
RSpec::Support.require_rspec_core "formatters/helpers"
RSpec::Support.require_rspec_core "formatters/base_text_formatter"
RSpec::Support.require_rspec_core "formatters/html_printer"
RSpec::Support.require_rspec_core "formatters/html_formatter"
# Overrides functionality from base class to generate separate html files for each test suite
# https://github.com/rspec/rspec-core/blob/master/lib/rspec/core/formatters/html_formatter.rb
class ParallelFormatter < RSpec::Core::Formatters::HtmlFormatter
RSpec::Core::Formatters.register self, :start, :example_group_started, :start_dump,
:example_started, :example_passed, :example_failed,
:example_pending, :dump_summary
# TEST_ENV_NUMBER will be empty for the first one, then start at 2 (continues up by 1 from there)
def initialize(param=nil)
output_dir = ENV['OUTPUT_DIR']
FileUtils.mkpath(output_dir) unless File.directory?(output_dir)
raise "Invalid output directory: #{output_dir}" unless File.directory?(output_dir)
id = (ENV['TEST_ENV_NUMBER'].empty?) ? 1 : ENV['TEST_ENV_NUMBER'] # defaults to 1
output_file = File.join(output_dir, "result#{id}.html")
opened_file = File.open(output_file, 'w+')
super(opened_file)
end
end
I have a Sinatra app which needs to provide downloadable reports in Microsoft Word format. My approach to creating the reports is to generate the content using ERB, and then convert the resulting HTML into docx. Pandoc seems to be the best tool for accomplishing this, but my implementation involves generating some temporary files which feels kludgy.
Is there a more direct way to generate the docx file and send it to the user?
I know that PandocRuby exists, but I couldn't quite get it working for my purposes. Here is an example of my current implementation:
#setting up the docx mime type
configure do
mime_type :docx, 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'
end
# route to generate the report
get '/report/:name' do
content_type :docx
input = erb :report, :layout=>false #get the HTML content for the input file
now = Time.now.to_i.to_s #create a unique file name
input_path = File.join('tmp', now+'.txt')
f = File.new(input_path, "w+")
f.write(input.to_s) #write HTML to the input to the file
f.close()
output_path = File.join('tmp', now+'.docx') # create a unique output file
system "pandoc -f html -t docx -o #{output_path} #{input_path}" # convert the input file to docs
send_file output_path
end
A recent update to pandoc-ruby added support for piping binary output to standard output. Does that solve your problem?
I don't have any experience with Sinatra, and I have not tried to use pandoc-ruby to pipe binary output, but something like
puts PandocRuby.convert(input, :from => :html, :to => :docx)
might do the trick.