I was wondering this and couldn't find anything except this
"Thread scheduler bug fixes and performance improvements. Threading on Ruby Enterprise Edition can be more than 10 times faster than official Ruby 1.8"
REE is derived from MRI 1.8.7. As such, it only used green threads. REE changes some parts of 1.8.7 (esp. in the areas memory management and garbage collection). But it still widely follows the design of the upstream MRI (the original Matz's Ruby Interpreter)
While YARV (1.9) switched to OS native threads, they still have a global interpreter lock making sure that only exactly one of these threads runs at a time.
There are a couple of Ruby implementations with OS native threads and without a GIL. The most prominent are JRuby (based on the JVM) and Rubinius (with its own VM). These implementations offer "real" concurrent threads.
Besides JRuby and Rubinius, who have got rid of an interpreter lock entirely, the state of affairs in CRuby/MRI has also made some progress with regard to concurrency.
One notable feature is that with the Bitmap Marking GC by Narihiro Nakamura, as of Ruby 2.0, another advantage of REE over CRuby will be gone: REE has a copy on write-friendly GC algorithm which made it attractive for achieving concurrency through processes (forking) rather than through threading. The new Bitmap Marking GC will have the same advantage of saving unnecessary copying of memory around when forking a new process.
The GIL (or GVL as it is officially called) is also not quite as bad as it sounds at first. For example, Ruby releases the interpreter lock when doing IO. Another feature that we see much more often lately is that C extension developers have the ability to manually release the lock by calling rb_thread_blocking_region, which will execute a C-level function with the GIL released. This can have huge effects if some operation in C is to be performed where we can rest assured that it will have no side effects. A nice example is RSA key generation - this runs completely in C with memory allocated by OpenSSL, so we can run it safely with the GIL released.
Fibers introduced in 1.9 or recent projects like Celluloid also cast a much more friendly light on the state of Ruby concurrency today as when compared to a few years ago.
Last not least, Koichi Sasada, the author of CRuby's VM, is actively working on the MVM technology, which will allow to run multiple VMs in a single Ruby process, and therefore achieving concurrency in yet another way.
Taking all the other performance improvements into account, there are less and less arguments for using REE, it's safe to switch to 1.9.3 or 2.0.0 once it's out, especially since the 1.8 series will no longer be actively developed and many popular projects have announced to quit their support for 1.8 sometime soon.
Edit:
As Holger pointed out, REE has also been EOLed, and there will be no port to 1.9 or further. So it's not only safe to switch, but also the right thing to do :)
Related
I read quite a lot on Google that ruby does not release back memory to the OS and I understand the as well because allocating memory from OS is a costly fair.
Which has led me to ask this
If a developer want the Ruby to release the memory back to OS from ruby how do they do it.
- I guess the answer is manually triggering a GC in Ruby.
But is it advisable to do (run manual GC) on a production application.
I'm thinking of creating a separate thread(or Actor) in Celluloid to start a GC at every 2 hours time.
Can anyone suggest. Is the above solution advisable for production.
Note: I think I also know GC.start (means stop the whole world in Ruby)
Note: Ruby Version 2.2.2 MRI Centos 4G RAM with 4 core.
And here is the reason for the question http://pastebin.com/rYdYwxLJ
I read quite a lot on Google that ruby does not release back memory to the OS and I understand the as well because allocating memory from OS is a costly fair.
That's simply not true. The Ruby Language doesn't say anything at all about releasing memory back to the OS. In fact, the Ruby Language doesn't say anything about memory, period.
There are a lot of different implementations of Ruby with a lot of different memory allocators and a lot of different garbage collectors. Some release memory back to the OS, some don't, some run on top of a managed runtime and don't even manage memory at all.
For example, JRuby running on J9 will release memory back to the OS. JRuby running on HotSpot with the G1 GC, ParallelScavenge GC, or ParallelOld GC will release memory back to the OS.
If a developer want the Ruby to release the memory back to OS from ruby how do they do it.
You don't. Either the implementation releases memory back to the OS, in which case you don't need to do anything, or it doesn't release memory back to the OS, in which you cannot do anything.
- I guess the answer is manually triggering a GC in Ruby.
If the implementation doesn't release memory back to the OS, then it doesn't release memory back to the OS. Period.
If the implementation does release memory back to the OS, then it will do that whether you manually trigger a GC or not.
By the way: you cannot manually trigger a GC in Ruby. You can suggest a GC (using GC::start), but not trigger it.
But is it advisable to do (run manual GC) on a production application.
Pretty much never. The GC knows better than you do whether it makes sense to run now or at a later time.
Note: I think I also know GC.start (means stop the whole world in Ruby)
No, GC::start simply suggests to do a GC. It does not force a GC. Also, GC does not mean stop-the-world. E.g. when running JRuby on the J9 JVM with the Metronome GC or running JRuby on the Azul Zing JVM with the Pauseless GC will never stop-the-world.
Note: Ruby Version 2.2.2
That's not the interesting bit. The interesting question is which implementation you are using, which garbage collector, and which OS. (And of course the versions and configuration settings for all of those.)
You likely misunderstood what you have read via Google links. The memory, that Ruby (here and after I mean MRI as default Ruby implementation) does not release back to OS (so-called Ruby Heaps,) is not released to OS. Period.
You have no ability to force Ruby to release Ruby Heaps back to OS (besides cold restart of the whole application.)
What might be released back to OS, is released by Ruby internally and it is a bad practice by all means to call GC manually, in 99% of cases that will slow down the application, in the rest 1% this will have no effect at all.
What I'm trying to understand is, practically speaking, how much benefit do I get from the parallel computing support in jRuby / Rubinius? A lot of ruby libraries keep track of global internal state. Is there any way to deal with these libraries, or do they just become unusable if I decide to parallelize my Ruby script? Maybe Rubinius automatically puts a mutex in front of all usage of unsafe libraries?
Whenever you want to use jruby or rubinius, you'll be forced to use gems that are compatible.
Many gems has been implemented using C extensions and you can not use them along jruby e.g.
Regarding others, mostly they clearly states their thread safety status.
So choosing jruby or rubinius will narrow down your options regarding the gems.
but a huge opportunity will be exposed, you'd be able to use many mature Java Libraries(in jruby case).
How is that JRuby's support for multithreading is any better than regular Ruby's support for it? What's wrong with threads in plain old Ruby?
"Normal" ruby (or mri) has a great big lock that prevents more than one thread from running ruby code at a time (known as the GIL or GVL).
Rubinius and jruby don't have this lock. In ruby 1.8.x the threads were green threads too, but as of ruby 1.9 ruby threads are mapped to native threads. The GVL stops you from gaining much benefit though.
Native extensions can run code outside of the lock so that, for example, multiple MySQL queries can run simultaneously from different threads but they can't call into the regular ruby api when they don't hold the lock
There is a very noticeable difference in application initiation time between running my specs from the command line with ruby 1.9.x vs. 1.8.7. My application initiates much faster with ruby 1.8.7 than with ruby 1.9.1 or 1.9.2. The application initiation difference is approximately 18 seconds. It takes about 5 seconds for my app to initialize with 1.8.7 and 23 seconds with 1.9.1 and 1.9.2.
Application initialization time is not a big deal for production, but it is a very big deal for BDD development. Every time I change my code and run my specs, I have to wait an additional 18 seconds per iteration.
I assume this application initialization time is attributed to YARV compiling bytecode as my application initializes.
Am I right about my YARV slowing down my application initialization, and is there a way to disable YARV on the command line. It would be very nice to be able to disable YARV only when I am running my specs.
YARV is a pure Ruby compiler. If you disable it, there's nothing left.
More precisely: YARV is a multi-phase implementation, where each of the phases is single-mode. It consists of a Ruby-to-YARV compiler and a YARV interpreter. If you remove the compiler, the only thing you are left with is the YARV bytecode interpreter. Unless you want to start writing your app in YARV bytecode, that interpreter is not going to be of much use to you.
This is in contrast to mixed-mode implementations such as JRuby and IronRuby which implement multiple execution modes (in particular, both a compiler and an interpreter) within a single phase. If you turn off the compiler in JRuby or IronRuby, you are still left with a usable execution engine, because they both also contain an interpreter. In fact, JRuby actually started out as a pure interpreter and added the compiler later and IronRuby started out as pure compiler and they added an interpreter exactly because of the same problem that you are seeing: compiling unit tests is simply a waste of time.
The only interpreted implementation of Ruby 1.9 right now is JRuby. Of course, there you have the whole JVM overhead to deal with. The best thing you can do is try how fast you can get JRuby to start up (use the nightly 1.6.0.dev builds from http://CI.JRuby.Org/snapshots/ since both 1.9 support and startup time are heavily worked on right this very moment) using either some very fast starting desktop-oriented JVM like IBM J9 or try JRuby's Nailgun support, which keeps a JVM running in the background.
You could also try to get rid of RubyGems, which generally eats up quite a lot of startup time, especially on YARV. (Use the --disable-gem commandline option to truly get rid of it.)
There's currently no way to disable YARV, simply because MRI 1.9 only includes the virtual machine, and not an interpreter. Maintaining both would be way too much job for the core team.
In the future there will probably be ways to cache the bytecode YARV generates (like Rubinius does). At the moment there is no way to load such bytecode through Ruby (see #971), but you could easily write a C extension which accomplishes it.
However, I would say that 18 seconds is way too much and it's probably a bug. I know there are some threads at ruby-core which discusses the slowness of require; maybe you find something interesting there!
the next RC of 1.9.2 out might be faster as it doesn't preload $: with all your gem lib dirs.
Background: I'm writing a 'standard' (nothing special) web application in Ruby (not Rails) and I need to start thinking about deployment.
So I've been hearing a lot of recommendations to use JRuby to deploy Ruby web applications, regardless of whether you actually need Java libraries or not. How true is this? Is it worth using the Java implementation just for speed? Would I gain anything else by doing so? Would I run into any issues?
PS: I don't know Java that well, so "you can write parts of it in Java" isn't very helpful.
JRuby is one of the most complete ruby implementations (there are a lot other ones out there such as IronRuby, Maglev, Rubinius, XRuby, YARV, MacRuby). It is very comprehensive, therefore, unless you use gems that use native C code, you will very very likely be just fine compatibility-wise.
JRuby is a bit faster than the actual C implementation, but it supports actual threads, whereas the official implementation is struggling a bit into getting it (it still uses Green Threads). Using Java threads from JRuby is quite trivial, even though it will require you to couple your code with Java (with a little DI, this coupling will only happen once, though).
Another benefit: runtime tools. Java, as a platform, instead of a language, has a lot of runtime tools to help you diagnose problems and check the state of the application (profilers, JConsole, and so on).
Twitter engineers also mentioned that the Ruby VM kinda has trouble being an environment for long lived processes, while the JVM is very good at that, because it’s been optimized for that over the last ten years.
Ruby also had a little security issue recently, which did not affect JRuby's implementation.
On the other hand, your project requires more artifacts (JVM, JRuby jars, etc). If you are using an application that will stay alive for long, and you want better runtime support, JRuby can be a great way to go. Otherwise, you can safely wait until you need these things to actually make the move (it is likely to go smoothly).
I use and love JRuby on daily basis, but I suggest you use MRI (a.k.a. C-Ruby) unless you have a actual need for JRuby.
Reasons for using JRuby:
Java integration
Restricted environment (your machine has Java installed by not ruby and you don't have root)
Restricted environment (you have ruby installed but don't have root so can't install gems you need)
You reached the limits of Ruby 1.8 performance and cannot use 1.9
From what you've described, you don't have any of the above reasons.
C-Ruby 1.9 has significant performance improvements over C-Ruby 1.8. I've yet to read (or find out for myself) how C-Ruby 1.9 compares with JRuby 1.8 or JRuby 1.9. In anycase, you don't have a performance problem (yet) so don't worry about it.
The good news is, you can start with either and convert later if needs be. It's all Ruby, and the Webrick and Mongrel gems work with both.
As mentioned above, ruby gems that have C extensions cannot be installed under JRuby. Hopefully this will change in the future if ruby C extensions utilize FFI.
http://kenai.com/projects/ruby-ffi/pages/Home
http://isitjruby.com/