Is there a tool that can allow me to compile Ruby code so that it runs somewhat faster?
For example, I have heard that there is a tool for Python called "pyc" that allows us to compile the code, so that it runs 10 times faster.
The simple answer is that you can't, at least with MRI 1.8 (the standard). This is because 1.8 works by walking the Abstract Syntax Tree. Python, Ruby 1.9, JRuby, and Rubinius use byte code, which allows compilation to an Intermediate Representation (byte code). From MRI Ruby 2.3 it has become easy to do this, see this answer below.
With Rubinius, you can do something as described in this post: http://rubini.us/2011/03/17/running-ruby-with-no-ruby/
In JRuby you can use the "Ahead Of Time" compiler through, I believe, jrubyc.
This isn't really the standard way of doing things and you're generally better off just letting your Ruby implementation handle it like it wants to. Rubinius, at least, will cache byte code after the first compilation, updating it as it needs to.
From ruby 2.3.0 its so easy to compile your source code to bytecodes that the Ruby-VM understands.
byte_code = RubyVM::InstructionSequence.compile_file '/home/john/somefile.rb'
File.binwrite '/home/john/bytecode', byte_code.to_binary
and in Command Line
$ cat bytecode
YARB�
IUsx86_64-linux*.*1
+1�!AA*1
!qy��������yyQ� E/home/john/somefile.rbE<main>E <class:A>EshivaEhelloEAEputsEcore#define_methodu����� 5M
The content of the file
class A
def shiva
puts 'hello'
end
end
What is the purpose?
Well, ruby takes time to compile your source code into byte codes so you can load your bytecodes directly into ruby and execute. No overhead of grammar checking and compilation. It much faster than normal processes.
How to load byte code?
bytecode = File.binread('/home/john/bytecode')
instruction_from_byte_code = RubyVM::InstructionSequence.load_from_binary bytecode
instruction_from_byte_code.eval
# => :shiva
Note: This answer is tested in MRI only. It might or might not work in other Ruby Implementations
In the beginning of 2013 there is not a way to translate Ruby into C/C++ source and then compile it.
However, I heard Matz (Yukihiro Matsumoto) say that a researcher is creating this tool in Japan. The project should be founded by the Japanese government.
Otherwise you could use JRuby and compile it in Java byte-code or you could use Rubinius. Rubinius compiles automatically in byte-code (JIT compiler) for the Rubinius VM. It is possible to convert Rubinius in byte-code into LLVM IR and LLVM can generate machine code.
I know this is an old question but I found a very interesting project that may provide an answer to your question: http://crystal-lang.org/
It basically compiles Ruby to native machine code. That's not exactly true because Crystal is not exactly Ruby and you might have to make some modifications to your code. There are also libraries that are not supported (yet) but to me it all looks very promising.
Check the Unholy git repo
The following "selfcontained" ruby test-case is based on the examples from this very thread, from the comment/answer of the user named illusionist.
#!/usr/bin/env ruby
#==========================================================================
# This file is in public domain.
# The code of this file is based on the code fragments at the
# 2018_12_09 version of the:
#
# https://stackoverflow.com/questions/5902334/how-to-compile-ruby
#
# This file has been tested with the ruby version
#
# ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
#
#-----start--of--the--boilerplate------------------------------------------
s_fp_home=ENV["HOME"].to_s
s_fp_tmp=s_fp_home+"/tmp" # using /tmp can be a security risk
s_fp_demofolder=s_fp_tmp+"/ruby_bytecode_usage_demo_01"
def create_folder_if_needed(s_fp_in)
if !Dir.exists? s_fp_in
Dir.mkdir(s_fp_in)
if !Dir.exists? s_fp_in
raise(Exception.new("\n\n Folder creation failed.\n"+
"GUID=='d6e409cb-e072-4441-9421-22630190c2e7'\n"))
end # if
end # if
end # create_folder_if_needed
create_folder_if_needed(s_fp_tmp)
create_folder_if_needed(s_fp_demofolder)
s_rand=""
7.times{s_rand<<("_"+rand(100).to_s)}
s_fp_bytecode=s_fp_demofolder+"/awesome_bytecode"+s_rand
s_fp_src=s_fp_demofolder+"/x"+s_rand+".rb"
if File.exists? s_fp_src
raise(Exception.new("\n\n This file should not exist yet.\n"+
" s_fp_src=="+s_fp_src+"\n"+
"GUID=='43ab3d45-1324-47af-9441-22630190c2e7'\n"))
end # if
IO.write(s_fp_src,"puts('');puts('Greetings from bytecode!');puts('')")
if !File.exists? s_fp_src
raise(Exception.new("\n\n The file \n"+s_fp_src+"\n is missing.\n"+
"GUID=='4aeb5e54-efe0-4111-a851-22630190c2e7'\n"))
end # if
#-----start--of--the--core--of--the--demo----------------------------------
bytecode_out = RubyVM::InstructionSequence.compile_file(s_fp_src)
IO.binwrite(s_fp_bytecode, bytecode_out.to_binary)
bytecode_in = IO.binread(s_fp_bytecode)
instruction_from_byte_code = RubyVM::InstructionSequence.load_from_binary(bytecode_in)
instruction_from_byte_code.eval
#==========================================================================
Try ruby-packer which creates executables from Ruby and Ruby on Rails applications
I am assuming you're running CRuby.
You could experiment with the --jit flag when running Ruby to see if you gain any speed. Otherwise, I don't think you can compile Ruby scripts into native code yet. Only to bytecode via the RubyVM module but I don't think you'll gain anything from doing it.
Crystal-lang or GraalVM could be your alternative (with some possible changes) for higher performance.
Related
there are two Ruby environments on a system, normal ruby and Chef embedded ruby. I want to know, in a ruby script, which ruby executable is used to invoke the script itself. How can get that?
Recommended Solutions
Use the poorly-documented RbConfig module, if available:
RbConfig.ruby
#=> "/Users/foo/.rubies/ruby-2.7.0/bin/ruby"
Alternatively, you can use the easier-to-find Gem module from the standard library to do the same thing:
Gem.ruby
#=> "/Users/foo/.rubies/ruby-2.7.0/bin/ruby"
Other Approaches
The RbConfig and Gem modules are your best bet, but there may be times when you need to get at the version or path information another way. Here are some different approaches.
Get the Version
You can return the version of the executing Ruby as a String with:
RUBY_VERSION
#=> "2.7.0"
Get the Path
Ruby is usually installed to bin/ruby in the RUBY_ROOT. You can return the expected path to the running Ruby binary (and verify it actually exists, if necessary) as follows:
ENV["RUBY_ROOT"] + "/bin/ruby"
#=> "/Users/foo/.rubies/ruby-2.7.0/bin/ruby"
File.exist? ENV["RUBY_ROOT"] + "/bin/ruby"
#=> true
Alternatively, you can use Kernel#` to find the first Ruby in your PATH as follows:
`which ruby`.chomp
=> "/Users/foo/.rubies/ruby-2.7.0/bin/ruby"
There are certainly edge cases where either approach can be misleading, though. For example, Ruby may have been built in a non-standard way, or you may have invoked Ruby with a fully-qualified path rather than calling the first binary in PATH. That makes "roll your own" lookups less reliable, but if your environment is missing the RbConfig or Gem modules for some reason, this might be a reasonable alternative for you.
I've installed Ruby 2.1.1 via source.
I've seen suggestions to type ruby -v, which I assume would show that the binary isn't corrupted, but are there more comprehensive ways to ensure that it's working as expect? Unit-tests, benchmarks, etc to validate it's functional?
Run make test after compiling in the directory you’ve compiled in. (This might actually happen by default, I can’t remember. There’s also make test-all, among others.)
ruby -v will show you the current version of ruby installed on your machine.
If you want, just create a hello.rb with puts "hello" and run it using ruby hello.rb to check if it is interpreting the ruby code correctly. So you know that its functional.
I am developing a gem, which is currently pure Ruby, but I have also been developing a faster C variant for one of the features. The feature is usable, but sometimes slow, in pure Ruby. The slowness would only impact some of the potential users (depends which features they need, and how they use them), so it makes sense to have the gem available with graceful fallback to Ruby-only functions if it cannot compile on a target system.
I would like to maintain the Ruby and C variants of the feature in a single gem, and provide the best (i.e. fastest) experience from the gem on installation. That would allow me to support the widest set of potential users from a single project of mine. It would also allow other people's dependent gems and projects to use the best available dependency on a target system, as opposed to a lowest-common-denominator version for compatibility.
I would expect the require to fallback at runtime to appear in the main lib/foo.rb file simply like this:
begin
require 'foo/foo_extended'
rescue LoadError
require 'foo/ext_bits_as_pure_ruby'
end
However, I don't know how to get the gem installation to check (or try and fail) for native extension support so that the gem installs correctly whether or not it can build 'foo_extended'. When I researched how to do this, I mainly found discussions from a few years back e.g. http://permalink.gmane.org/gmane.comp.lang.ruby.gems.devel/1479 and http://rubyforge.org/pipermail/rubygems-developers/2007-November/003220.html that imply Ruby gems do not really support this feature. Nothing recent though, so I am hoping someone on SO has some more up-to-date knowledge?
My ideal solution would be a way to detect, prior to attempting a build of the extension, that the target Ruby did not support (or perhaps simply not want, at the project level) C native extensions. But also, a try/catch mechanism would be OK if not too dirty.
Is this possible, if so how? Or is the advice to have two gem variants published (e.g. foo and foo_ruby), that I am finding when I search, still current best practice?
This is my best result attempting to answer my own question to date. It appears to work for JRuby (tested in Travis and on my local installation under RVM), which was my main goal. However, I would be very interested in confirmations of it working in other environments, and for any input on how to make it more generic and/or robust:
The gem installation code expects a Makefile as output from extconf.rb, but has no opinion on what that should contain. Therefore extconf.rb can decide to create a do nothing Makefile, instead of calling create_makefile from mkmf. In practice that might look like this:
ext/foo/extconf.rb
can_compile_extensions = false
want_extensions = true
begin
require 'mkmf'
can_compile_extensions = true
rescue Exception
# This will appear only in verbose mode.
$stderr.puts "Could not require 'mkmf'. Not fatal, the extensions are optional."
end
if can_compile_extensions && want_extensions
create_makefile( 'foo/foo' )
else
# Create a dummy Makefile, to satisfy Gem::Installer#install
mfile = open("Makefile", "wb")
mfile.puts '.PHONY: install'
mfile.puts 'install:'
mfile.puts "\t" + '#echo "Extensions not installed, falling back to pure Ruby version."'
mfile.close
end
As suggested in the question, this answer also requires the following logic to load the Ruby fallback code in the main library:
lib/foo.rb (excerpt)
begin
# Extension target, might not exist on some installations
require 'foo/foo'
rescue LoadError
# Pure Ruby fallback, should cover all methods that are otherwise in extension
require 'foo/foo_pure_ruby'
end
Following this route also requires some juggling of rake tasks, so that the default rake task doesn't attempt to compile on Rubies that we're testing on that don't have capability to compile extensions:
Rakefile (excerpts)
def can_compile_extensions
return false if RUBY_DESCRIPTION =~ /jruby/
return true
end
if can_compile_extensions
task :default => [:compile, :test]
else
task :default => [:test]
end
Note the Rakefile part doesn't have to be completely generic, it just has to cover known environments we want to locally build and test the gem on (e.g. all the Travis targets).
I have noticed one annoyance. That is by default you will see Ruby Gems' message Building native extensions. This could take a while..., and no indication that the extension compilation was skipped. However, if you invoke the installer with gem install foo --verbose you do see the messages added to extconf.rb, so it's not too bad.
https://stackoverflow.com/posts/50886432/edit
I tried the other answers and could not get them to build on recent Rubies.
This worked for me:
Use mkmf#have_* methods in extconf.rb to check for everything you need. Then call #create_makefile, no matter what.
Use the preprocessor constants generated by #have_* to skip things in your C file.
Check which methods/modules are defined in Ruby.
If you want to support JRuby et al, you'll need a more complex release setup.
A simple example where the whole C extension is skipped if something is missing:
1.
ext/my_gem/extconf.rb
require 'mkmf'
have_struct_member('struct foo', 'bar')
create_makefile('my_gem/my_gem')
2.
ext/my_gem/my_gem.c
#ifndef HAVE_STRUCT_FOO_BAR
// C ext cant be compiled, ignore because it's optional
void Init_my_gem() {}
#else
#include "ruby.h"
void Init_my_gem() {
VALUE mod;
mod = rb_define_module("MyGemExt");
// attach methods to module
}
#endif
3.
lib/my_gem.rb
class MyGem
begin
require 'my_gem/my_gem'
include MyGemExt
rescue LoadError, NameError
warn 'Running MyGem without C extension, using slower Ruby fallback'
include MyGem::RubyFallback
end
end
4.
If you want to release the gem for JRuby, you need to adapt the gemspec before packaging. This will allow you to build and release multiple versions of the gem. The simplest solution I can think of:
Rakefile
require 'rubygems/package_task'
namespace :java do
java_gemspec = eval File.read('./my_gem.gemspec')
java_gemspec.platform = 'java'
java_gemspec.extensions = [] # override to remove C extension
Gem::PackageTask.new(java_gemspec) do |pkg|
pkg.need_zip = true
pkg.need_tar = true
pkg.package_dir = 'pkg'
end
end
task package: 'java:gem'
Then run $ rake package && gem push pkg/my_gem-0.1.0 && gem push pkg/my_gem-0.1.0-java to release a new version.
If you just want to run on JRuby, not distribute the gem for it, this will suffice (it will not work for releasing the gem, though, as it is evaluated before packaging):
my_gem.gemspec
if RUBY_PLATFORM !~ /java/i
s.extensions = %w[ext/my_gem/extconf.rb]
end
This approach has two advantages:
create_makefile should work in every environment
a compile task can remain prepended to other tasks (except on JRuby)
Here is a thought, based on info from http://guides.rubygems.org/c-extensions/ and http://yorickpeterse.com/articles/hacking-extconf-rb/.
Looks like you can put the logic in extconf.rb. For example, query the RUBY_DESCRIPTION constant and determine if you are in a Ruby that supports native extensions:
$ irb
jruby-1.6.8 :001 > RUBY_DESCRIPTION
=> "jruby 1.6.8 (ruby-1.8.7-p357) (2012-09-18 1772b40) (Java HotSpot(TM) 64-Bit Server VM
1.6.0_51) [darwin-x86_64-java]"
So you could try something like wrap the code in extconf.rb in a conditional (in extconf.rb):
unless RUBY_DESCRIPTION =~ /jruby/ do
require 'mkmf'
# stuff
create_makefile('my_extension/my_extension')
end
Obviously, you will want more sophisticated logic, grabbing parameters passed on "gem install", etc.
I want my app to not be able to use any installed gems. Is there a ruby 1.9 startup parameter or way of doing this programmatically?
ruby --disable-gems
is the MRI (1.9) commandline parameter. "It prevents the addition of gem installation directories to the default load path". (The Ruby Programming Language, p. 391)
Edit 25-10-2012: Ruby core had the same idea as #rogerdpack in the comments and added the more verbose ruby --help parameter. Ruby revision!
Looking at the rubygems configuration file, I would attempt to hack out gempath or gemhome to see if you can override (instead of just append to) defaults.
If, for example, setting gempath to be empty, or to point to /dev/null, prevents using system gems, then that would be the way to go.
The main advantage to this, as I see it, is that your anti-rubygems config file can be passed to ruby 1.9 as a startup parameter (so not coded in), well documented, and checked into your repository.
All of this is, of course, disregarding that rubygems is part of ruby 1.9's standard library - so ruby may choke and die if it can't have access to its gems, depending on how much of ruby's base install requires gem functionality. YMMV.
Is it possible to determine whether the implementation of ruby you're running on is capable of supporting fork, without running a regex against RUBY_PLATFORM that'll expand until it summons Cthulhu?
(Related question: Ruby - How can I find out on which system my program is running?)
Edit: I tried Marc-Andre's suggestion. It doesn't work for jruby with fork disabled by default:
192-168-1-7:~ agrimm$ jruby --1.9 -S jirb
irb(main):001:0> RUBY_VERSION
=> "1.9.2dev"
irb(main):002:0> Process.respond_to?(:fork)
=> true
irb(main):003:0> Process.fork
NotImplementedError: fork is unsafe and disabled by default on JRuby
Update: From Marc-Andre's link, it seems wiser heads than I have grappled with this problem from the perspective of creating ruby implementations, and failed.
From the perspective of someone who's writing a ruby library, what would be the most comprehensive incantation, short of running fork and seeing if it raises an exception?
In Ruby 1.9:
Process.respond_to?(:fork) # => true if fork is supported, false otherwise
For Ruby 1.8, or JRuby (which doesn't implement this currently) you'll have to actually test it.
See also this long discussion on ruby-core.
Instead of error-prone testing against RUBY_PLATFORM or other things you could test fork itself:
def can_fork?
pid = fork
exit unless pid # exit the child immediately
true
rescue NotImplementedError
false
end
One downside would be if fork is somehow emulated which could make this check expensive.
A recent commit to rails uses
Config::CONFIG['host_os'] !~ /mswin|mingw/)
to test whether it's not on Windows, and
RUBY_PLATFORM !~ /java/
to test whether it's not on JRuby. However, I know it's possible to enable fork as a command-line switch. I'd have to look into whether there's any way of telling whether it's been enabled.
I suspect macruby doesn't support, or strongly discourages, forking, and from the thread Marc-Andre refers to, you can test for it with
RUBY_ENGINE != "macruby"
Now for all the other platforms out there... (work in progress, marking as community wiki, please feel free to edit)