Why does truffleruby need C extensions? - ruby

Current status of truffleruby says:
TruffleRuby is progressing fast but is currently probably not ready for you to try running your full Ruby application on. Support for critical C extensions such as OpenSSL and Nokogiri is missing.
Why does truffleruby need C extensions? It's built on GraalVM which is built on top of the JVM, it is in fact a fork of JRuby:
TruffleRuby is a fork of JRuby, combining it with code from the Rubinius project, and also containing code from the standard implementation of Ruby, MRI.
Can't they use JRuby world gems instead of depending on their C variants?
EDIT link to the issue on github

Running C extensions is hard because the C extension API is just the entire internals of MRI exposed as a header file. You aren't programming against a clean API - you're programming against all the implementation details and internal design decisions of MRI.
JRuby's Java extensions have exactly the same problem! The JRuby Java extension API is just the entire internals of JRuby, and you aren't programming against an API, instead you're programming against all the implementations details and design decisions of JRuby.
We plan to eventually tackle both problems in the same way - which is to add another level of abstraction over the C or Java code using an interpreter which we can intercept and redirect when required, so that it believes it is running against MRI or JRuby internals, but really we redirect these to our internals.
We think C extensions are more important, so we're tackling those first. We haven't really started on Java extensions yet, but we have started the underlying interpreter for Java that we'll use.
This video explains all
https://youtu.be/YLtjkP9bD_U?t=1562

You have already gotten a good answer by the project lead himself, but I want to offer a different point of view:
Why does truffleruby need C extensions?
It doesn't need them. But they do exist and there is code out there which uses them, and it would sure be nice to be able to run that code.

Related

Rubinius in RubySL

I use Ruby from a user level and really don't deal with the internals. I have know Rubinius as 'Ruby in Ruby' which I assumed was a generalization. Recently, I got an error with Rubinius in the RubySL (no, I don't have error msg).
I started looking at RubySL and was a little surprised to see Rubinius everywhere. I really like Ruby and was just curious why Rubinius is in most of the RubySL? It seems to be used with things like locks / unlocks (such as https://github.com/rubysl/rubysl-thread/blob/2.0/lib/rubysl/thread/thread.rb ). Definitely not questioning it, just curious.
RubySL is short for Ruby Standard Library. It is a basic part of the shipped code bundle which forms what is generally known as Ruby. The standards library provides rather basic stuff you often need but which doesn't need to be part of the core language.
For example, the implementation of the Hash or Array, the language keywords, how assignment works, ... are part of the core language. These are often implemented in a language other than Ruby. MRI (the common C-Ruby) implements this mostly in C, JRuby implements this in Java. Rubinius implements this patly in C++ but mostly in Ruby itself. It can do this by bootstrapping itself from a very simple base VM and gradually adding more stuff with Ruby.
The standards library however is mostly implemented in Ruby in all implementations (with some exceptions mostly for performance reasons). Now, all Ruby implementations right now have their own implementation of the Ruby standards library which can thus differ in minor details.
Rubinius' approach to implementing a standards library was to implement it as separate gems. The idea was to one day provide a common standards library which could be used by other implementations (including MRI). This is in line with the efforts of esp. the Rubinius community to drive the RubySpec project in order to provide a common language specification and test suite for all Ruby implementations.
The RubySpec project was eventually abandoned and right now, it doesn't seem as if other Ruby implementations seem to be moving to the RubySL gems for implementing their standards library.
Thus, (and this is the TL;DR), the RubySL gems implement the Ruby Standard Library for the Rubinius project. Thus, it is expected to see the Rubinius project all over the place there: it is their code which is generally not used by other Ruby implementations.

Is it possible to compile Ruby to byte code as with Python?

In Python, if I want to give out an application without sources I can compile it into bytecode .pyc, is there a way to do something like it in Ruby?
I wrote a much more detailed answer to this question in the question "Can Ruby, PHP, or Perl create a pre-compiled file for the code like Python?"
The answer is: it depends. The Ruby Language has no provisions for compiling to bytecode and/or running bytecode. It also has no specfication of a bytecode format. The reason for this is simple: it would be much too restricting for language implementors if they were forced to use a specific bytecode format, or even bytecodes at all. For example, XRuby and JRuby compile to JVM bytecode, Ruby.NET and IronRuby compile to CIL bytecode, Cardinal compiles to PAST, SmallRuby compiles to Smalltalk/X bytecode, MagLev compiles to GemStone/S bytecode. For all of these implementations it would be plain stupid to use any other bytecode format than the one they currently use, since their whole point is interoperating with other language implementations that use the same bytecode format.
Simlar for MacRuby: it compiles to native code, not bytecode. Again, using bytecode would be stupid, since one of the goals is to run Ruby on the iPhone, which pretty much requires native code.
And of course there is MRI, which is a pure AST-walking script interpreter and thus doesn't have a bytecode format.
That being said, there are some Ruby Implementations which allow compiling to and loading from bytecode. Rubinius allows that, for example. (Indeed, it has to have that functionality since its Ruby compiler is written in Ruby, and thus the compiler must be compiled to Rubinius bytecode first, in order to solve the Catch-22.)
YARV also can save and load bytecode, although the loading functionality is currently disabled until a bytecode verifier is implemented that prevents users from loading manipulated bytecode that could crash or otherwise subvert the interpreter.
But, of course, both of these have their own bytecode formats and don't understand each other's (nor tinyrb's or RubyGoLightly's or ...) Also, neither of those formats is understood by a JVM or a CLR and vice versa.
However, the whole point is irrelevant because, as Mark points out, you can always reverse engineer the byte code anyway, especially in cases like CPython, PyPy, Rubinius, YARV, tinyrb, RubyGoLightly, where the bytecode format was specifically designed to be very close to the source language.
In general it is simply impossible to protect code that way. The reason is simple: you want the machine to be able to execute the code. (Otherwise what's the point in writing it in the first place?) However, in order to execute the code, the machine must understand the code. Since machines are much dumber than humans, it follows that any code that can be understood by a machine can just as well be understood by a human, no matter whether that code happens to be in source form, bytecode, assembly, native code or a deck of punch cards.
There is only one workable technical solution: if you control the entire execution pipeline, i.e. build your own CPU, your own computer, your own operating system, your own compiler, your own interpreter, and so forth and use strong cryptography to protect all of those, then and only then might you be able to protect your code. However, as e.g. Microsoft found out the hard way with the XBox 360, even doing all of that and hiring some of the smartest cryptographers and mathematicians on the planet, doesn't guarantee success.
The only real solution is not a technical but a social one: as soon as you have written your code, it is automatically fully protected by copyright law, without you having to do one single thing. That's it. Your code is protected.
The short answer is "YES",
check rubini.us
It will solve your problem.
Here is how to compile ruby code:
http://rubini.us/2011/03/17/running-ruby-with-no-ruby/
Although Ruby's 1.9 YARV VM is a byte-code compiler I don't believe it can dump the byte-code to disk. You might want to look at the alternative compiler, Rubinius, I believe it has this ability. You should note though that byte-code pyc files (and I imagine the ruby equivalent) can be pretty easily "decompiled".
Not with the MRI interpretter, no.
Some newer VM's are being worked on where this is on the table, but these aren't widely used (or even ready to be used) at this point.
If you use Jruby, you can compile your Ruby code into Java .class files (including your Rails stuff) to execute them with (open)jdk out of the box!
You can even compile your complete stuff into a .war file to deploy it on Apache Tomcat or Jboss with a tool called "warbler"
https://rubygems.org/gems/warbler/
Depends on your ruby.
JRuby - https://github.com/jruby/jruby/wiki/JRubyCompiler
MRuby - http://mruby.org/docs/articles/executing-ruby-code-with-mruby.html
MRI (C)Ruby - https://devtechnica.com/ruby-language/compile-ruby-code-to-binary-and-execute-it

Use JRuby for Ruby web applications? Is it worth it?

Background: I'm writing a 'standard' (nothing special) web application in Ruby (not Rails) and I need to start thinking about deployment.
So I've been hearing a lot of recommendations to use JRuby to deploy Ruby web applications, regardless of whether you actually need Java libraries or not. How true is this? Is it worth using the Java implementation just for speed? Would I gain anything else by doing so? Would I run into any issues?
PS: I don't know Java that well, so "you can write parts of it in Java" isn't very helpful.
JRuby is one of the most complete ruby implementations (there are a lot other ones out there such as IronRuby, Maglev, Rubinius, XRuby, YARV, MacRuby). It is very comprehensive, therefore, unless you use gems that use native C code, you will very very likely be just fine compatibility-wise.
JRuby is a bit faster than the actual C implementation, but it supports actual threads, whereas the official implementation is struggling a bit into getting it (it still uses Green Threads). Using Java threads from JRuby is quite trivial, even though it will require you to couple your code with Java (with a little DI, this coupling will only happen once, though).
Another benefit: runtime tools. Java, as a platform, instead of a language, has a lot of runtime tools to help you diagnose problems and check the state of the application (profilers, JConsole, and so on).
Twitter engineers also mentioned that the Ruby VM kinda has trouble being an environment for long lived processes, while the JVM is very good at that, because it’s been optimized for that over the last ten years.
Ruby also had a little security issue recently, which did not affect JRuby's implementation.
On the other hand, your project requires more artifacts (JVM, JRuby jars, etc). If you are using an application that will stay alive for long, and you want better runtime support, JRuby can be a great way to go. Otherwise, you can safely wait until you need these things to actually make the move (it is likely to go smoothly).
I use and love JRuby on daily basis, but I suggest you use MRI (a.k.a. C-Ruby) unless you have a actual need for JRuby.
Reasons for using JRuby:
Java integration
Restricted environment (your machine has Java installed by not ruby and you don't have root)
Restricted environment (you have ruby installed but don't have root so can't install gems you need)
You reached the limits of Ruby 1.8 performance and cannot use 1.9
From what you've described, you don't have any of the above reasons.
C-Ruby 1.9 has significant performance improvements over C-Ruby 1.8. I've yet to read (or find out for myself) how C-Ruby 1.9 compares with JRuby 1.8 or JRuby 1.9. In anycase, you don't have a performance problem (yet) so don't worry about it.
The good news is, you can start with either and convert later if needs be. It's all Ruby, and the Webrick and Mongrel gems work with both.
As mentioned above, ruby gems that have C extensions cannot be installed under JRuby. Hopefully this will change in the future if ruby C extensions utilize FFI.
http://kenai.com/projects/ruby-ffi/pages/Home
http://isitjruby.com/

What is the significance for Ruby programmers of SAP's new implementation of Ruby?

SAP announced Blue Ruby, a version of Ruby that runs inside the ABAP Virtual Machine.
This seems to lend additional credibility to the Ruby language but, except for SAP developers, does this have any applicability to the rest of the Ruby community?
I'm just wondering what other significance this may have. Additional job opportunities, perhaps, for Ruby developers to be hired to work on SAP projects?
Any other potential benefits for Ruby programmers?
Also, something I'm not clear about: Besides MRI and JRuby, how many different implementations are there and why do I, as a Ruby programmer, need any implementation except the MRI? I understand I might need JRuby if I want to integrate with Java libraries. Are there any other times I might need to look into an implementation besides the MRI or JRuby?
I notice that Blue Ruby is compiled. Is that a big benefit and would this be the first compiled version of Ruby?
Sorry - maybe too many questions but this is being discussed as a big deal except I'm not sure why/if it really is. Would be grateful to get some feedback from people who understand Ruby (and I.T.) better than I do.
There are actually quite a few alternative implementations. The reason for so many versions is partly because while Ruby itself has a lot of fans, MRI is often considered to be not quite as nice as the language it hosts.
This is a list of all the other Ruby implementations I know of:
MRI
JRuby for Java. Its speed advantage isn't as great now that Ruby 1.9 is out, but it's still a big player for its integration with Java.
IronRuby (still in development) for .NET
MacRuby for making native Mac OS X applications. It's currently based on Ruby 1.9, but they're going to release a new LLVM-based virtual machine for the next version, which looks to offer a major speed boost and will hopefully be portable to other LLVM target platforms.
MagLev (still very much in development) for high-capacity Web apps. It will probably be a commercial product.
Rubinius is a ground-up reimplementation meant as an alternative to MRI with a much cleaner foundation.
I doubt Blue Ruby will have much of an impact on the general software development landscape. It will probably be nice for SAP programmers and allow them to cast a wider net for potential hires since Ruby is somewhat more popular than ABAP.
All of the current Ruby implementations are compiled to bytecode. Contrary to SAP's claims, as of Ruby 1.9, MRI itself includes a bytecode compiler, though the ability to save the compiled bytecode to disk disappeared somewhere in the process of merging the YARV virtual machine. JRuby is compiled into Java .class files. I don't have a lot of details on MagLev, but it seems safe to say it will take that road as well.
Each implementation has its own use cases. JRuby is useful for much the same reasons Jython is. IronRuby will be useful in a .NET environment, when it exists in a usable state.
Things like this make Ruby more acceptable on the Enterprise level, and hasten the spread of Good Programming Languages.
Of course, Blue Ruby is intended for SAP related people. I don't think is going to have much more of an impact on the Ruby community because most Ruby developers are not related to ABAP development.
The main purpose of Blue Ruby is to bring a powerful Scripting Language into the ABAP world, providing more tools for ABAP developers.
You can read my blog on Blue Ruby providing an SE16 (Data table explorer) emulator.
https://weblogs.sdn.sap.com/pub/wlg/14222
Greetings,
Alvaro "Blag" Tejada Galindo.
Senior ABAP Consultant - SAP Mentor.

Differences between Ruby VMs

What are the advantages/disadvantages of the major Ruby VMs (things like features, compatibility, performance, and quirks?) I know there are also some bonus features like being able to use Java interfaces through JRuby, too. Those would also be helpful to note. Does any VM have a clear advantage at this point, and in what contexts?
I've used both Matz's Ruby and JRuby, and they solve different tasks. If you are developing a straight Ruby or Rails app, then that will probably suffice, but if there are some powerful Java libraries that would help a lot, then JRuby might be worthwhile.
I haven't done anything overly complicated, but JRuby seemed to match up pretty well, at least as far as implementing the core language features (I haven't run into any differences yet, but they may exist).
One little anecdote I wish to share... I was writing a script to interact with a DB2 database. The DB2 support in Ruby is abysmal... you have to install the whole DB2 express version just to be able to compile the Ruby drivers, which didn't even work for me. I got fed up and switched to JRuby, using JDBC and a few small DB2 JDBC jars. It resolved my problem perfectly. The point? Well, if gaining access to some Java libraries will simplify the problem at hand, by all means go for it!
I hope this was helpful! Sorry I don't have any experience with other VMs....
One more caveat I have read about, but I don't know the details too well... JRuby I think supports threading via Java threads, instead of the "green" threads supported in Matz's implementation... so if you want multithreading on multicore systems, JRuby will probably serve you better... unless you want to do the threading in C.
Here's a bit of info I scrounged up on the main VMs: Ruby MRI, Ruby 1.9 (YARV), JRuby, XRuby, Rubinius, and IronRuby
There was a performance benchmark last year that compared the major VMs, but with how quickly VM development has been it probably is not as relevant today. Ruby 1.9 was generally the fastest, and still has the edge over JRuby for now, I believe.
Four VMs are currently capable of running Ruby on Rails: Ruby MRI, Ruby 1.9, JRuby, and Rubinius.
XRuby runs on the JVM, as does JRuby, and compiles the Ruby source files to a Java .class.
IronRuby runs on .NET, making use of their DLR, and allows you to integrate Ruby with the .NET libraries and infrastructure. It cannot yet run Ruby on Rails.
There is also a VM called HotRuby that lets you run Ruby source code in the browser or in Flash.

Resources