We are considering a move to jruby, and as part of this have been researching testing approaches and frameworks. I've been testing rspec on my local development machine and am finding that the total time taken to run 2 tests for a single class with 2 very simple methods is 7-8 seconds. By simple, I mean one setter and one that returns true.
The rspec output shows that the tests run in roughly 2 seconds, so 5-6 seconds of the total time is spent loading and initializing rspec. I'm running from the command line using
C:\rubycode\rspec_tutorial>rspec --profile user_spec.rb
..
Top 2 slowest examples:
User User should NOT be in any roles not assigned to it
0.023 seconds ./user_spec.rb:15
User User should be in any roles assigned to it
0.006 seconds ./user_spec.rb:10
Finished in 2 seconds
2 examples, 0 failures
I'm running jruby 1.6.5 and rspec 2.7.1
I've read this post, Faster RSpec with JRuby
but it's over 1.5 years old, and the answer relates to running suites of tests as opposed to short bursts of a small number of tests locally to aid TDD, which is how we want to develop. Down the line we'll incorporate a CI server.
My question is, is this the expected execution time? Is there any way to speed up the running of rspec tests on the local development machine uner JRUBY?
EDIT:
Biggest performance gains were switching from 64 bit "server" JVM to 32 bit "client" mode. I saw about a 40% reduction in the time taken to run a simple test. I got Nailgun up and running as well, but performance varied. The link provided below by banzaiman was most helpful
It is not loading RSpec, but it is JVM startup time you are feeling.
See https://github.com/jruby/jruby/wiki/Improving-startup-time for more information.
Related
We are using Visual Studio Enterprise 2017 and Self Provisioned Rig for load generation.
Are there any limits for the test run duration?
Can the test run for 2 weeks continuous duration with load test agents in Self provisioned Rig in case of Soak Test scenario for an example?
I believe that it can run for two weeks. But I must caveat that by saying lots of data is stored in the computer's RAM as the test runs and lots more is written to the SQL database. I have no idea what the limits on their storage would be. Also running a test of that duration risks losing lots of data should something unexpected happen during the run.
See here where a 672 hour test failed with an integer overflow of a value holding milliseconds. Some quick sums suggest the integer would overflow at about 600 hours which is about 25 days.
I have a set of Watin GUI tests that run on Teamcity. They each run in about 300 seconds locally. They previously ran in about the same time on a TeamCity agent (PC specs are about the same). For some reason they now run incredibly slow. The tests have a timeout of 1500 seconds and this is being triggered to cause them to fail.
I have checked the tests again locally and they run in the same time as expected (about 300 seconds). When I remote view into the Team City agent I can see that the tests are keying in text at a very slow pace. The actual speed at which tests operate is slower. It almost seems as if the tests are set to run at a quarter speed if such a thing existed.
These tests are only run about once every two weeks so its possible something on the agent has changed to effect their functionality.
I am at a loss as to what could be wrong. I've never encountered this before and on paper everything is correct but it is just going slow.
Does anyone have any ideas?
When trying to run an intensive Ruby method, I noticed it's only using 25% of the CPU resources while 70% sits idle. Is there any way to configure this to use more? I'm on Windows 7, ruby 2.0.0
You probably have 4 CPU cores. You're running 1 Ruby process. 1 Ruby process = 1 thread = can use max 1 CPU core. The MRI (default) implementation of Ruby currently cannot run more than 1 thread in parallel. For that, you may want to try JRuby or some other implementation like Rubinius that allows parallel threads. I'm guessing you'll need to learn a bit about multi-threading to understand this wholly, start by reading some basic tutorials and then questions like "Does ruby have real multithreading?".
When the process is running, go to the task manager, right click on the program, click "go to process," right click on the process, go to select priority, and check off "high."
Important: never set application to "realtime" it may cause several problems.
references:
http://www.tomshardware.com/forum/57576-63-maximum-capacity-application
After running some tests, I'm convinced there has to be something wrong with my setup (windows, rubymine and latest ruby versions). My times right now are:
Finished tests in 14.289817s, 0.0700 tests/s, 0.3499 assertions/s.
1 tests, 5 assertions, 0 failures, 0 errors, 0 skips
Process finished with exit code 0
With 5 VERY easy tests (just checking if validation on empty fields works). The total time for these 5 unit tests is 160 seconds, over 2 minutes.
What could I do to improve this speed?
Here are the tests:
require 'test_helper'
class ItemTest < ActiveSupport::TestCase
test 'item attributes must not be empty' do
item = Item.new
assert item.invalid?
assert item.errors[:name].any?
assert item.errors[:description].any?
assert item.errors[:image_url].any?
assert item.errors[:rating].any?
end
end
Your problem is Windows. We use JRuby on Windows and it actually runs faster than RubyInstaller(mingw) ruby on Windows but we do see very slow results when running test suites or starting a rails server. About 1 minute for a single test run due to the loading of the Rails environment.
You have a few options:
Switch to linux / osx
Use spork to keep a couple rails environments pre-loaded for your tests. Note that this isn't perfect but it will reduce your times substantially. With this option you'll probably want to use minitest or rspec, I had trouble getting spork to work on Windows with testunit. With spork you should be able to get your single test run time down to about 10 seconds.
Write as many of your tests to run outside of Rails, in other words to not require the Rails stack. This will be very fast, you should be able to run a test in only a few seconds but as you could guess, it's hard to test a lot of things (controllers, views) outside of rails. Works perfectly though for functions you've broken out into modules that already do not require anything from rails.
Good luck!
What's the rest of your gem stack? Sometimes third-party gems are initialized by rails and will try to phone home (New Relic, Airbrake) which can inflate your test times (though probably not by this much). If something isn't strictly required for your test suite, you should try to pull it into the proper env group, or set require :false via bundler:
group :production do
gem 'newrelic_rpm'
end
Startup time appears to be killing you, so you and I are probably in the same boat. Between Jodell and Dark Castle a lot of this is covered already, but here's my nearly-whole list of things that helped, in descending order of efficacy.
Get a patched 1.9.3 with the 2.0 filesystem improvements backported. The first gets 2x better numbers but I'm using the second because I felt the first was unstable
https://groups.google.com/forum/?fromgroups=#!topic/thecodeshop/WVA4N2lJoM4
https://raw.github.com/gist/3242245/5cdb932c0a544cfbb51fdee5759942d4d2c263f5/1-complete.diff
Set your GC options
http://snaprails.tumblr.com/post/241746095/rubys-gc-configuration (I'm using the last one)
Turn off collecting coverage data (My IDE kept volunteering this)
Run Spork, and set SPEC_OPTS=--drb
Turn off virus scanner (don't actually do this, it's only worth 10% for me anyway)
Double-check your Gems, delaying the loading of gems with require: false
6 didn't actually buy me very much. The biggest thing we had wrong was loading Thin unconditionally (which pulls in Eventmachine, which 21 megabytes installed), but the next 3 on the stack were actually being used by RSpec. I have not looked at network traffic, but Jodell is probably on to something there.
I have to wait quite long time (comparing to my friends machines) for executing scripts on jRuby, especially when I'm running rake tasks or tests. I've tried jRuby version 1.3.1 and 1.4.1 with and without ruby-debug gem and the same problem occurred in each configuration.
The question is simple:
Is there any way to improve jRuby load process?
..or have I something wrong with my jvm configuration or jRuby installation (I'm using jRuby via rvm - ruby version manager)?
There are a couple of things you could try:
use the very latest and greatest version of JRuby (due to the extensive testsuites, even the bleeding edge git master branch is usually pretty stable), they are constantly working on startup time
choose your JVM wisely, Oracle JRockit for example is geared towards servers and thus startup performance is not a concern (those apps are only restarted once every couple of years anyway), Sun has mainly neglected the desktop for the last ten years or so, but has gotten consistently better since 1.6u12 (try the recently released 1.6u18) and also in 1.7. IBM's J9 is also said to be pretty lightweight.
try nailgun, which is a project that keeps a JVM running as a daemon in the background (there is builtin support in JRuby, try running your scripts with jruby --ng)
just don't use JRuby for unit tests and rake tasks: the ThoughtWorks Mingle team, for example uses MRI for unit tests, rake tasks and development and JRuby for integration tests, regression tests and production. (This obviously only works if you don't use any Java libraries in your rake tasks and tests.)
However, tests and scripts are the worst case scenario for JRuby. The JRuby runtime alone is already pretty heavy, much heavier than MRI. Just loading the entire beast from disk into RAM can already take longer than running the same script in MRI. And we haven't even added the startup time for the JVM yet!
Also, make sure that you run JVM in client mode (assuming that you're using Sun's JVM), since this mode provides faster startup and better overall performance for things like test suites. JRuby by default should use JVM in client mode, but this depends on the system and your JVM settings, etc. To verify that you're using client JVM, invoke jruby -v and you should see something like this
Java HotSpot(TM) *Client* VM 1.6.0_18
Update: Take a look at Charles' blog post with tips to improve startup: http://blog.headius.com/2010/03/jruby-startup-time-tips.html
JRuby now has a --dev flag which combines many speedy options. I ran my model tests on Rails 5 and JRuby 9.1.7.0 with over 80% improvement!
$ time rspec spec/models
Finished in 2.85 seconds (files took 10.63 seconds to load)
86 examples, 0 failures
rspec spec/models 57.79s user 1.14s system 288% cpu 20.425 total
$ time JRUBY_OPTS=--dev rspec spec/models
Finished in 1.4 seconds (files took 4.15 seconds to load)
86 examples, 0 failures
JRUBY_OPTS=--dev rspec spec/models 11.51s user 0.48s system 139% cpu 8.600 total
Don't want to type all of this? Create a Makefile! You can add -G to include bundle exec
# Makefile
tests:
JRUBY_OPTS='--dev -G' rspec
Then simply run
$ make tests
source: https://github.com/jruby/jruby/wiki/Improving-startup-time
spork might help, if its unit tests you want to improve time on.