Running same Cucumber Features on different machines concurrently using parallel_tests gem
I'm trying to figure out how to use the parallel_tests gem to run the EXACT SAME Cucumber Features on a variety of different machines concurrently.
Currently I have them running on different machines but the SAME features do not execute, it just splits up the features and runs different features on different machines.
More clarification on what I want to do:
MACHINE 1 (Win 7) - EXECUTE SAME FEATURES CONCURRENTLY
MACHINE 2 (Win 8) - EXECUTE SAME FEATURES CONCURRENTLY
MACHINE 3 (Mac OS X) - EXECUTE SAME FEATURES CONCURRENTLY
...
Also my architecture is:
Cucumber -> Capybara -> Selenium Webdriver -> Selenium Grid
parallel_tests gem: https://github.com/grosser/parallel_tests
The parallel_tests gem is really purposed to parallellism, not executing multiple platforms.
You may find the following useful:
Sauce Gem 30 rake tasks
"The Sauce integration is currently targeted at RSpec and Cucumber, running on a local server you can spin up multiple copies of. It runs a copy of each test for each platform, and divides them up across all the concurrency available to your Sauce Labs account by default"
Hope it helps, I haven't used it myself yet.
Thanks for the clarification. To accomplish testing your app for all the supported environments (OSs/browsers) you can use something as Selenium Grid here is an example Selenium RC: Run tests in multiple browsers automatically.
TestingBot is a service that use this: http://blog.testingbot.com/2012/02/19/selenium-cucumber-capybara
Other approach is described here: http://altoros.github.io/2013/running-capybara-tests-in-remote-browsers/
http://paauspaani.blogspot.mx/2013/05/running-tests-remotely-using-selenium.html
There is also a presentation: http://www.slideshare.net/martinkleppmann/crossbrowser-testing-in-the-real-world
Also this could help you Has anyone figured out a way to run the same cucumber scenario on multiple browsers/web drivers?
Hopefully this will give you some clues.
I am confused by this statement of yours:
Currently I have them running on different machines but the SAME
features do not execute, it just splits up the features and runs
different features on different machines.
parallel_tests is used for splitting up tests and running them on multiple cores, not on a distributed environment of multiple machines. It can group tests to be run on a distributed environment using the --only-group flag, but it doesn't actually run tests on a distributed environment. So you have to do some lifting yourself to get the tests running on a distributed environment. You could run all of your tests on multiple machines like so:
Use net-ssh to ssh into each one of your boxes.
Use each ssh session to run your parallel_tests script (which should break the tests up and run them in parallel on the current box -- just don't use the --only-group option as described here)
Related
How can I run several rspecs in parallel without launching several processes? More precisely, my requirement is that the RSpec.configure blocks should be run only once for the whole test suite. However, solutions such as parallel_tests seem to launch several processes and each process runs the Rspec.configure blocks.
The tests are not CPU intensive. Basically, an RSpec.configure block launches a bunch of machines on Amazon and then each tests runs commands on a subset of these machines and checks the output.
I have a Cucumber test (feature files) in the RubyMine IDE and lately I have a need to execute one of the feature repeatedly on a scheduled time.
I haven't found a way to do so. Any idea or thoughts on scheduling that feature file?
You can create a cron job which will execute a rake.
The software utility Cron is a time-based job scheduler in Unix-like
computer operating systems. People who set up and maintain software
environments use cron to schedule jobs (commands or shell scripts) to
run periodically at fixed times, dates, or intervals.
These links might help
How to create a cron job using Bash
how to create a cron job to run a ruby script?
http://rake.rubyforge.org/
I solved the problem by simply installing Jenkins on my machine from its official site,https://jenkins-ci.org/. I configured master and slave nodes on my own machine because I just needed to run one feature file(it has the script I want to run on daily basis).
Although, it is recommended to configure slave on a different machine if we have multiple jobs to run and our jobs are resource intensive.
There is a very good illustration on installing, configuring and running jobs in this link http://yakiloo.com/setup-jenkins-and-windows/
Is there any possibility to run written CUIT tests without necessity of having active browser window?
Basically, what I would like to have is just to run a bunch of tests on local machine. And I need to wait until all the tests are executed (each test requires browser window to be active during test run). But during this execution I can't use my machine for any other work - otherwise tests fall due to loosing control on the window. So is there any ability to run tests locally and work on the machine without any limitations simultaneously?
Coded UI Test Needs Active Browser while Execution.
To match your necessity you should try running test cases into virtual machine. That will not affect your work in local machine.
You might consider using WatiN and run them by "normal" MSTest/NUnit/whatever instead of using CodedUiTests. It does open up a browser window too, but allows you to interact with your desktop nevertheless.
That of course means rewriting your tests but as these tests are much more readable IMHO this might be worth it.
what are the two types of execution that can be done in jmeter?
This was asked in a interview to me. i was asked what are the ways u execute the script i.e GUI or non -GUI. I am working on jmeter but I have never come across these two words
GUI means "graphical user interface", like you run in microsoft windows.
We can run Jmeter in both GUI or Non-GUI (From command line).
I strongly suggest you do some reading about Jmeter on below website
http://jmeter.apache.org/usermanual/index.html
http://developer.amd.com/community/blog/2009/03/31/using-apache-jmeter-in-non-gui-mode/
hope this will help.
It is absolutely not recommended to use JMeter GUI (graphical user interface) for anything apart from developing or debugging test as AWT Event Thread will kill both your test and JMeter in case of more or less high load.
There is a variety of approaches on how to run a JMeter test in non-GUI mode including:
command-line mode
Apache Ant task
Apache Maven plugin
Jenkins plugin
Execution pre-defined .jmx from Java code or creating a test on-the-fly
Plugins for IDEs like Eclipse or Idea
See 5 Ways To Launch a JMeter Test without Using the JMeter GUI for more details on above.
I have to wait quite long time (comparing to my friends machines) for executing scripts on jRuby, especially when I'm running rake tasks or tests. I've tried jRuby version 1.3.1 and 1.4.1 with and without ruby-debug gem and the same problem occurred in each configuration.
The question is simple:
Is there any way to improve jRuby load process?
..or have I something wrong with my jvm configuration or jRuby installation (I'm using jRuby via rvm - ruby version manager)?
There are a couple of things you could try:
use the very latest and greatest version of JRuby (due to the extensive testsuites, even the bleeding edge git master branch is usually pretty stable), they are constantly working on startup time
choose your JVM wisely, Oracle JRockit for example is geared towards servers and thus startup performance is not a concern (those apps are only restarted once every couple of years anyway), Sun has mainly neglected the desktop for the last ten years or so, but has gotten consistently better since 1.6u12 (try the recently released 1.6u18) and also in 1.7. IBM's J9 is also said to be pretty lightweight.
try nailgun, which is a project that keeps a JVM running as a daemon in the background (there is builtin support in JRuby, try running your scripts with jruby --ng)
just don't use JRuby for unit tests and rake tasks: the ThoughtWorks Mingle team, for example uses MRI for unit tests, rake tasks and development and JRuby for integration tests, regression tests and production. (This obviously only works if you don't use any Java libraries in your rake tasks and tests.)
However, tests and scripts are the worst case scenario for JRuby. The JRuby runtime alone is already pretty heavy, much heavier than MRI. Just loading the entire beast from disk into RAM can already take longer than running the same script in MRI. And we haven't even added the startup time for the JVM yet!
Also, make sure that you run JVM in client mode (assuming that you're using Sun's JVM), since this mode provides faster startup and better overall performance for things like test suites. JRuby by default should use JVM in client mode, but this depends on the system and your JVM settings, etc. To verify that you're using client JVM, invoke jruby -v and you should see something like this
Java HotSpot(TM) *Client* VM 1.6.0_18
Update: Take a look at Charles' blog post with tips to improve startup: http://blog.headius.com/2010/03/jruby-startup-time-tips.html
JRuby now has a --dev flag which combines many speedy options. I ran my model tests on Rails 5 and JRuby 9.1.7.0 with over 80% improvement!
$ time rspec spec/models
Finished in 2.85 seconds (files took 10.63 seconds to load)
86 examples, 0 failures
rspec spec/models 57.79s user 1.14s system 288% cpu 20.425 total
$ time JRUBY_OPTS=--dev rspec spec/models
Finished in 1.4 seconds (files took 4.15 seconds to load)
86 examples, 0 failures
JRUBY_OPTS=--dev rspec spec/models 11.51s user 0.48s system 139% cpu 8.600 total
Don't want to type all of this? Create a Makefile! You can add -G to include bundle exec
# Makefile
tests:
JRUBY_OPTS='--dev -G' rspec
Then simply run
$ make tests
source: https://github.com/jruby/jruby/wiki/Improving-startup-time
spork might help, if its unit tests you want to improve time on.