I'm writing some tests for a webpage that I'd like to run in several environments. The idea is that the test will run in one, then repeat in the next. The two environments are preview and uat.
I've written an Around hook to set the environment variables. Below:
Around do |scenario, block|
def test_envs
chosen_env = ENV['test_env'] || 'preview'
chosen_env.split(',').map(&:strip)
end
test_envs.each do |test_env|
$base_url = "https://#{test_env}.webpage.com"
end
block.call
end
I have then written a method to execute the navigation step:
def navigate_to(path)
visit $base_url + path
end
My Scenario step_definition is:
navigate_to '/login'
The tests will work in either environment, Preview by default or UAT if I set test_env=uat
However, I was aiming to set test_env=preview,uat and have them run consecutively in both environments.
Is there something obvious that I've missed here?
Thanks
If I'm understanding you correctly, it's the 'parallel' aspect that you're asking about.
Rspec can be used with parallel tests (the parallel_tests gem) but I wouldn't be so sure that calling something like 3.times { blk.call } in an around hook will run each block in parallel.
An alternative may be do so some metaprogramming with your example definitions, i.e.
test_envs.each do |env_name|
it "does something in #{env_name}" do
# do something with the specific environment
end
end
Now, I haven't actually used this gem and I don't know for sure it would work. I think the simplest solution may be to just write a wrapper script to call the tests
# run_tests.rb
environments = ENV["TEST_ENV"]&.split(",") || []\
filename = ENV["filename"]
environments.each do |env_name|
Thread.new do
system <<-SH
env TEST_ENV=#{env_name} bundle exec rspec #{filename}
SH
end
end
Running it like env TEST_ENV=foo,bar ruby run_tests.rb would call the following commands in their own threads:
env TEST_ENV=foo bundle exec rspec
env TEST_ENV=bar bundle exec rspec
I like this approach because it means you don't have to touch your existing test code.
Related
Is there a Ruby Cucumber test hook for at_start? I tried at_start and it didn't work.
I have something like this in support/hooks.rb and I want to print a single global message before any of the tests start:
Before do
print '.'
end
at_exit do
puts ''
puts 'All Cucumber tests finished.'
end
It seems like if they have an at_exit hook, they should have a before-start hook as well right?
There is some documentation for "global hooks" at https://github.com/cucumber/cucumber/wiki/Hooks
You don't need to wrap it in any special method such as Before or at_exit. You just execute the code at the root level in any file contained in the features/support directory, such as env.rb. To copy and paste the example they've given:
# these following lines are executed at the root scope,
# accomplishing the same thing that an "at_start" block might.
my_heavy_object = HeavyObject.new
my_heavy_object.do_it
# other hooks can be defined in the same file
at_exit do
my_heavy_object.undo_it
end
They also give an example of how to write a Before block that gets executed only once. Basically you have this block exit if some global variable is defined. The first time the block is run, the global variable is defined which prevents it from being executed multiple times. See the "Running a Before hook only once" section on that page I linked.
I am using a AfterConfiguration hook to run some setup config before my tests start, however the issue that I am faced with is that when I run my methods one of them will run a set of feature files using backticks in a Ruby method, which in turn seems to re-initialize cucumber and repeat the process, so I am stuck in a loop
AfterConfiguration do
EnvironmentSetup::TestUsers.create_test_users
end
module EnvironmentSetup
class TestUsers
def self.create_test_users
# other logic here
`cucumber "#{path_to_feature}"` # Use backticks to run cucumber scripts in a subshell
end
end
end
So when this is executed it goes back to the beginning and runs it all my other logic again
Is there a way to only run this once, or ignore the AfterConfiguration on the second loop? declare a global variable?
I have also tried
AfterConfiguration do
if defined? $a == nil
EnvironmentSetup::RedisUsers.check_redis_users
EnvironmentSetup::TestUsers.create_test_users
end
end
module EnvironmentSetup
class TestUsers
def self.create_test_users
# other logic here
$a = true
`cucumber "#{path_to_feature}"` # Use backticks to run cucumber scripts in a subshell
end
end
end
but I'm guessing that the variable set is not being carried across when re-initializing?
Try setting Environment variable:
AfterConfiguration do
return if ENV['CUCUMBER_CONFIGURED'] == 'yes'
EnvironmentSetup::TestUsers.create_test_users
ENV['CUCUMBER_CONFIGURED'] = 'yes'
end
And run cucumber something like this:
CUCUMBER_CONFIGURED='no'; cucumber ...
I have to maintain a Ruby script, which requires some libs I don't have locally and which won't work in my environment. Nevertheless I want to spec some methods in this script so that I can change them easily.
Is there an option to stub some of the require statements in the script I want to test so that it can be loaded by rspec and the spec can be executed within my environment?
Example (old_script.rb):
require "incompatible_lib"
class Script
def some_other_stuff
...
end
def add(a,b)
a+b
end
end
How can I write a test to check the add function without splitting the "old_Script.rb" file and without providing the incompatible_lib I don't have?
Instead of stubbing require which is "inherited" from Kernel, you could do this:
Create a dummy incompatible_lib.rb file somewhere that is not in your $LOAD_PATH. I.e., if this is a Ruby application (not Rails), don't put it in lib/ nor spec/.
You can do this a number of ways, but I'll tell you one method: in your spec file which tests Script, modify $LOAD_PATH to include the parent directory of your dummy incompatible_lib.rb.
Ordering is very important -- next you will include script.rb (the file which defines Script).
This will get you around the issue and allow you test test the add method.
Once you've successfully tested Script, I would highly recommend refactoring it so that you don't have to do this technique, which is a hack, IMHO.
Thanks, I also thought about the option of adding the files, but finally hacked the require itself within the test case:
module Kernel
alias :old_require :require
def require(path)
old_require(path) unless LIBS_TO_SKIP.include?(path)
end
end
I know that this is an ugly hack but as this is legacy code executed on a modified ruby compiler I can't easily get these libs running and it's sufficient to let me test my modifications...
I'd like to test class and gem loading. Have a look at the following stupid test case:
require 'rubygems'
require 'shoulda'
class SimpleTest < Test::Unit::TestCase
context 'This test' do
should 'first load something' do
require 'bundler'
assert Object.const_defined? :Bundler
end
should 'second have it reset again' do
assert !Object.const_defined?(:Bundler)
end
teardown do
# This works, but is tedious and unclean
#Object.send :remove_const, :Bundler rescue nil
# Instead I want something like this ;)
magic_reset
end
end
end
How about creating a subclass of Test::Unit::TestCase which runs the test method in a forked process?
class ForkingTestCase < Test::Unit::TestCase
def run(...)
fork do
super.run(...)
# somehow communicate the result back to the parent process
# this is the hard part
end
end
end
If this can be implemented, it should then just be a matter of changing the base class of your test case.
AFAIK, you cannot unload a file that you have loaded. You need to start a separate Ruby process for every test. (Or a separate Ruby instance if you are running on a Ruby implementation which supports multiple instances in the same process.)
Try using Kernel#load with wrap set to true:
load(filename, wrap=false) → true
Loads and executes the Ruby program in the file filename. If the
filename does not resolve to an absolute path, the file is searched
for in the library directories listed in $:. If the optional wrap
parameter is true, the loaded script will be executed under an
anonymous module, protecting the calling program’s global namespace.
In no circumstance will any local variables in the loaded file be
propagated to the loading environment.
each time you want to do a test of bundler, load it into a new anonymous module, do your tests on the bundler class within that module, and then go onto your next test.
If your code refers to the Bundler constant, then you'd have to set and unset that constant, though.
I haven't tried it myself, but can't see why it wouldn't work.
Try keeping track of what constants were defined before your tests started, and what constants are defined after your test finished, and remove the constants that were defined during the test.
I guess this isn't so much telling you how to call magic_reset as how to implement magic_reset.
With Test::Unit, I can run:
ruby path/to/test.rb --name=test_name_that_i_want_to_run
Thus far, I have not been able to figure out how to do this with test/spec specifications. I am wondering if the way that specifications are automatically named does not allow me to do something like this.
Take the following spec for example:
require 'rubygems'
require 'spec'
describe 'tests' do
it 'should be true' do
1.should == 1
end
it 'should be false' do
1.should_not == 2
end
end
You can execute a single spec by using the -e flag and providing the portion specified by the it block. e.g. ruby my_spec.rb -e 'should be false'
After contacting the gem maintainer, Christian Neukirchen, I found out how to do this, so I am documenting it here for future reference.
specrb path/to/test.rb --name ".*should behave this way.*"
I needed to use the specrb test runner, an extended version Test::Unit's test runner, rather than just the ruby command.
You can also do this with the ruby command:
ruby path/to/test.rb -n "/should behave this way/"