Given a Rakefile that executes some RSpec tests by way of a Spec::Rake::SpecTask, how can I programmatically determine the number of tests that passed, failed, and are pending once the task that ran the tests is finished?
SpecTask is just a nice wrapper around sh ruby -S spec ..., so you have no access to the formatter's meta data. You could parse the result, or do the evaluation somewhere else than your Rakefile, or not use SpecTask at all.
Related
I have many test spec files with describe() and it(). Needs to run only some cases (it()) say that is sanity cases of each spec file. How to run all sanity cases of each describe() of all test spec files?
I am using Webdriverio and javascript.
There are two ways of doing it.
Create separate files for each type of tests and run them as per your needs.
You can utilize grep flag of mocha to tell mocha which test case to pick.
I would prefer second one as it is more extensible. Here is what you have to do:
Update summary of it blocks to include a pattern e.g. #sanity#regression etc
At the tile of running tests from command line, pass grep flag as
mocha -g "#sanity"
Mocha will check for the text passed in the command in each of the tests and will execute only the matching ones.
I have a script that runs my Lua Unit Test. Each test has its own output of a summary. However, I want to count and see which test fail after all of the test are ran.
The script loops through the test like so:
# Loop over all the UTs and run them
for utLuaScript in `ls ut*.lua` ; do
echo "LAUNCH TEST: ${utLuaScript}"
lua ./${utLuaScript} -v
echo
done
What is the solution here? Have the number of successes and failure saved to file, then once outside of this loop, go through the file and summarize all of the test. Can the script spit out a variable? What is best practice?
The usual way this is done is with a test runner script. In this approach, the unit tests are not executable by themselves (i.e. lua ut_foo.lua doesn't do anything), but must be run through the test runner. For example, lua test_runner.lua might run all the tests, and lua test_runner.lua ut_foo.lua might run just the "ut_foo" tests. The test runner script takes care of formatting and displaying the test results.
There are quite a few test runners already available for Lua; see the overview on the Lua-users wiki. Perhaps one of those will meet your needs, or can be adapted to do so.
I have the "foo.feature" file and I want to execute the commands in this file via Rake:
rake feature['foo.feature',100000]
The first argument (or what is it) is completely clear, but what about the other one? Is it an amount of scripts executions or something like this?
Unfortunately I am unable to find this in the docs.
If memory serves, that is the syntax used to call rake feature and deliver two arguments to the feature task, namely 'foo.feature' and 100000.
You'll want to look into your rakefile and see what is expected for the task in question.
For what it's worth, it's an unwieldily and error-prone syntax. There are better libraries to build scripts that accept arguments and options such as OptionParse or Thor. Even better, use rake as a replacement for make, and use something else to develop shell-based commands.
I'm using a helper script to execute rspec tests.
command = "rake spec #{path} #{scope}"
output = `#{command}`
puts output
This works fine, except that I lose all the colors from the rake rspec output. The appropriate ANSI codes do not appear to be contained within the output string.
How can I execute a process so that it returns output which includes the text color?
Kernel.exec() gets me the solution I want (colored rspec output), but it does so by replacing my ruby script process with the rspec process. That means I can't do anything with the output or run anything after the rspec call.
That's acceptable in my particular situation, but less than ideal as a general solution. So I'd like a better answer if available.
RSpec will disable colour if it is not writing to a tty (i.e. the console).
In case of RSpec you can force colouring by rspec --tty or via rake by rake spec SPEC_OPTS=' --tty'.
See also Ruby popen3 and ANSI colour
However this solution is still specific to Rspec. I'd be interested to hear a general one.
Turns out it's possible to run commands in a pseudo terminal via the PTY module in order to preserve a user facing terminal-like behaviour. Credits go to the creator of the tty-command gem (see this issue) who implemented this behaviour in his gem:
require 'tty-command'
cmd = TTY::Command.new(pty: true)
cmd.run('rake', 'rspec')
Keep in mind that using a pseudo terminal may have unwanted side effects, such as certain git commands using a pager which will essentially cause commands to hang. So introducing the functionality might be a breaking change.
If you don't want to replace your ruby process with that command, use Kernel.system() or Kernel.spawn() instead of a Kernel.exec(). Both of them execute your command in a subshell, system waits for the subprocess to finish, spawn returns its pid and you have to wait by yourself using Process.wait pid.
command = "rake spec #{path} #{scope}"
system(command)
or
command = "rake spec #{path} #{scope}"
pid = spawn(command)
# Some other stuff here
Process.wait pid
I have a set of test cases under ruby test framework 1.8.7
Lets say i have a ruby file named check.rb which contains differents tests like
test_a_check, test_b_check and test_c_check.
Now when i run the file ruby check.rb, all the test cases will be executed.
My part of the question is,
I want to pass a new parameter to the script while running, say ruby check.rb --sunset
based on the sunset parameter i want my script to execute only test_a_check and test_b_check and not the test_c_check.
By default, if i run the script all the tests should be excuted but when the --sunset parameter is passed only two of three tests should be executed.
is there are way i can achieve this?
If you are using minitest you can specify the method via
ruby check --name test_method_name
If it's a common testing framework, then look into it's manual, but
If it's your personal testing script, then just look in ARGV:
test_a_check
test_b_check
test_c_check if ARGV[0] != '--sunset'