Background:
We have a monorepo of services, and each service has unit tests and runs their own tests using rspec, for example:
repo/appA/spec/*_spec.rb
repo/appB/spec/*_spec.rb
repo/appC/subserviceD/spec/*_spec.rb
repo/appC/subserviceE/spec/*_spec.rb
My goal:
To preface, I'm new to the Ruby ecosystem, but I would like to setup a separate gem (an isolated repo from the monorepo) which is responsible for running some code after specs finish, basically an rspec afterall hook, or any time after the tests are done.
For anyone familiar with the Python/pytest ecosystem, this is basically analogous to writing a "pytest plugin," where you can add extra functionality to existing hooks.
Question:
Is it possible to create an afterall RSpec hook (or something behaviour) in a separate gem, and any code that imports this gem automatically has the hook applied after their tests run?
Thank you in advance!
Related
Problem to solve: We want to be able to run multiple ruby cucumber tests in parallel with different users. Since we have user collision in the app, we are not able to use the same user simultaneously.
We tried looking into parallel_test gem to use parallel_cucumber but did not find any way to pass a different user for each process. One option I read online was to have user info in the DB and make a call to get a free user before each test. This was not feasible for us.
Does anyone know any way to make parallel_cucumber or any other ruby gem work to run parallel cucumber tests with a different user for each process
If you are running parallel tests you should be able to use a separate db for each stream. This should avoid the issue of user collisions.
I am trying to use RSpec to functional test my REST apis.
The way I would LIKE it to work is using a CI build to build and deploy my application to a test server somewhere in the cloud and then have it kick off automated functional tests. But in order to properly do that, I need to be able to pass in the base url/domain for where the app was deployed. It will not be the same.
Everything I've found so far makes it seem like RSpec can't do this. Is there another way to do it if I can't pass in parameters on the command line? Or is RSpec not the right choice for this?
One way would be to bypass the call to rspec with something that accepts command line arguments and then initiate rspec in code. If you do not want to write your own binary for that, rake is capable of that too.
Look here for how to run rspec from code.
Another way would be setting an ENV variable when calling your test and preferably making it optional in the specs.
$> SPEC_URL=http://anotherhost:666 rspec
in code:
url = ENV['SPEC_URL'] || "http://localhost:4000"
I would suggest method two as it's the cleaner and easier approach in my opinion.
We use DelayedJob to run some of our long running processes and would like to test with Cucumber/Webrat.
Currently, we are calling Delayed::Job.work_off in a Ruby thread to get work done in the background, but are looking for a more robust solution
What is the best approach for this?
Thanks.
The main problem I see with the Delayed:Job.work_off approach is that you are making explicit in your Cucumber scenarios something that belongs to the internals of your system. Mixing both concerns is against the spirit of functional testing:
When I click some link # Some operation is launched in the background
And Jobs are dispatched # Delayed:Job.work_off invoked here
Then I should see the results...
Another problem is that you populate your Cucumber scenarios with repetitive steps for dispatching jobs when needed.
The approach I am currently using is launching delayed_job in the background while cucumber scenarios are being executed. You can check the Cucumber hooks I am using in that link.
In our development environment, we run a Continuous Integration service (TeamCity) which responds to code checkins by running build/test jobs and reporting the results. While the job is in progress, we can easily see how many unit tests have executed so far, how many have failed, etc.
My automated testing team is delivering UI tests developed in Rational Functional Tester. Extracting those tests from the source control system, compiling them, and executing them from the command line all seem to be pretty straight forward exercises.
What I haven't been able to find is a way to report the test results automatically - there don't appear to be any hooks for listeners, for example, or any way to customize the messages that are emitted.
From my research thus far, I've come to the conclusion that my only option is to (a) wait until the tests finish, then (b) parse the HTML report that RFT generates.
Does anybody have a better answer than that?
Here is the workaround I've used for the similar purpose:
Write a helper super class that overwrite the onTerminate callback method, implement your log parsing logics there.
Change the helper super class of your test scripts to the helper super class create in step1.
Use RFT CLI invoke your scripts in your Continous Integration code.
Expanding on #eric2323223, in your onTerminate override, you can use TeamCity's build script interaction functionality to have your RFT pass/fail status rolled up to TeamCity. You just need these TeamCity specific messages emitted to the command line, so that TeamCity picks them up.
##teamcity[testStarted name='test1']
##teamcity[testFailed name='test1' message='failure message' details='message and stack trace']
##teamcity[testFinished name='test1']
##teamcity[testStarted name='test2']
##teamcity[testFailed type='comparisonFailure' name='test2' message='failure message' details='message and stack trace' expected='expected value' actual='actual value']
##teamcity[testFinished name='test2']
I have a Rakefile that defines the spec task as
task :spec => [:check_dependencies, :load_backends]
And then runs the actual rspec tests. During the load_backends task, it loads a class called Story, but in the first spec test, defined?(Story) returns false.
I'm assuming that it is intended behavior of Rake to start with a fresh environment at the beginning of each task, but is there a way to override this? Or do I need to re-architect loading the backends into each task?
RSpec's spec task fires up a new Ruby process (mainly to not screw with your Rake process, I think), therefore classes defined in a rake task (even the spec task) are not available in your specs. Consider moving this logic to your spec helper or don't use RSpec's spec task.