I would very much like to know what exactly makes Crystal faster than Ruby while code is so similar. The short answer could be that it is compiled, and Ruby is interpreted, yet I would like to understand more about the language specifications.
I guess it's a combination of things:
Ruby is interpreted, and the interpreter could be improved. For example other interpreted languages like JS or Java have a very good VM and JIT compiler.
Many Ruby checks that are done at runtime, in Crystal are done at compile time. For example a simple method call in Ruby ends up in a method lookup. Even with a cache it won't beat a native function call. Or when Ruby decides to do different things based on the type of an argument, these checks are done at runtime. In Crystal they are known at compile time so those checks disappear. Without those checks the compiler can inline calls and do some pretty crazy stuff (thanks to LLVM). Or, for example, looking up an instance varaibles is a hash lookup in Ruby (as far as I know), while in Crystal it's just a memory indirection and load.
In Crystal we try to avoid extra memory allocations. For example to_s(io) writes to an IO instead of converting the object to a string in memory. Or we have tuples for fixed-sized arrays that are allocated on the stack. Or you can declare a type as a struct to avoid heap allocations.
Calls to C are done directly, without wrappers. Well, you could have a wrapper but that will be inlined by LLVM. In Ruby it always has to resolve a Ruby method first.
Probably there are many more reasons, but they are related.
Related
Consider the following two programs:
unit module Comp;
say 'Hello, world!'
and
unit module Comp;
CHECK { if $*DISTRO.is-win { say 'compiling on Windows' }}
say 'Hello, world!'
Naively, I would have expected both programs to compile to exactly the same bytecode: the CHECK block specifies code to run at the end of compilation; checking a variable and then doing nothing has no effect on the run-time behavior of the program, and thus (I would have thought) shouldn't need to be included in the compiled bytecode.
However, compiling these two programs does not result in the same bytecode. Specifically, compiling the version without the CHECK block creates 24K of bytecode versus 60K for the version with it. Why is the bytecode different for these two versions? Does this difference in bytecode have (or potentially have) a runtime cost? (It seems like it must, but I want to be sure).
And one more related question: how do DOC CHECK blocks fit in with the above? My understanding is that even the compiler skips DOC CHECK blocks when it's not run with the --doc flag. Consistent with that, the bytecode for a hello-world program does not increase in size when given a DOC CHECK block like the one above. However, it does increase in size if the block includes a use statement. From that, I conclude that use is somehow special-cased and gets executed even in DOC CHECK blocks. Is that correct? If so, are there other simillarly special-cased forms I should know about?
A CHECK or BEGIN block (or other BEGIN-time constructs) may contain code that escapes. For example:
BEGIN SomeClass.^add_method('foo', anon method foo() { 42 })
Adds a method to a class, which exists beyond the bounds of the BEGIN block. That method's bytecode is therefore required in the compiled output. Currently, Rakudo conservatively includes the bytecode of everything in a BEGIN or CHECK block. It may be possible to avoid that for some simple cases in the future.
So far as the runtime cost goes, the implementation goes to some lengths to minimize the cost of bytecode that is never run (not so much for this case, but because the standard library is huge but many programs use only a fraction of it). For example:
Bytecode is mmap'd, so some unused parts of it may not actually be paged into memory
Bytecode is only validated on the first call to that frame
Frame meta-data (what lexicals does it have) is only deserialized on the first call to the frame
Unless something references it, the code object will not be deserialized
So far as use goes, its action is performed as soon as it is parsed. Being inside a DOC CHECK block does not suppress that - and in general can not, because the use might bring in things that need to be known in order to finish parsing the contents of that block.
Are there some tutorials or practical lessons on how to write an extension for Ruby in Go?
Go 1.5 added support for building shared libraries that are callable from C (and thus from Ruby via FFI). This makes the process easier than in pre-1.5 releases (when it was necessary to write the C glue layer), and the Go runtime is now usable, making this actually useful in real life (goroutines and memory allocations were not possible before, as they require the Go runtime, which was not useable if Go was not the main entry point).
goFuncs.go:
package main
import "C"
//export GoAdd
func GoAdd(a, b C.int) C.int {
return a + b
}
func main() {} // Required but ignored
Note that the //export GoAdd comment is required for each exported function; the symbol after export is how the function will be exported.
goFromRuby.rb:
require 'ffi'
module GoFuncs
extend FFI::Library
ffi_lib './goFuncs.so'
attach_function :GoAdd, [:int, :int], :int
end
puts GoFuncs.GoAdd(41, 1)
The library is built with:
go build -buildmode=c-shared -o goFuncs.so goFuncs.go
Running the Ruby script produces:
42
Normally I'd try to give you a straight answer but the comments so far show there might not be one. So, hopefully this answer with a generic solution and some other possibilities will be acceptable.
One generic solution: compile high level language program into library callable from C. Wrap that for Ruby. One has to be extremely careful about integration at this point. This trick was a nice kludge to integrate many languages in the past, usually for legacy reasons. Thing is, I'm not a Go developer and I don't know that you can compile Go into something callable from C. Moving on.
Create two standalone programs: Ruby and Go program. In the programs, use a very efficient way of passing data back and forth. The extension will simply establish a connection to the Go program, send the data, wait for the result, and pass the result back into Ruby. The communication channel might be OS IPC, sockets, etc. Whatever each supports. The data format can be extremely simple if there's no security issues and you're using predefined message formats. That further boosts speed. Some of my older programs used XDR for binary format. These days, people seem to use things like JSON, Protocol Buffers and ZeroMQ style wire protocols.
Variation of second suggestion: use ZeroMQ! Or something similar. ZeroMQ is fast, robust and has bindings for both languages. It manages the whole above paragraph for you. Drawbacks are that it's less flexible wrt performance tuning and has extra stuff you don't need.
The tricky part of using two processes and passing data between them is a speed penalty. The overhead might not justify leaving Ruby. However, Go has great native performance and concurrency features that might justify coding part of an application in it versus a scripting language like Ruby. (Probably one of your justifications for your question.) So, try each of these strategies. If you get a working program that's also faster, use it. Otherwise, stick with Ruby.
Maybe less appealing option: use something other than Go that has similar advantages, allows call from C, and can be integrated. Althought it's not very popular, Ada is a possibility. It's long been strong in native code, (restricted) concurrency, reliability, low-level support, cross-language development and IDE (GNAT). Also, Julia is a new language for high performance technical and parallel programming that can be compiled into a library callable from C. It has a JIT too. Maybe changing problem statement from Ruby+Go to Ruby+(more suitable language) will solve the problem?
As of Go 1.5, there's a new build mode that tells the Go compiler to output a shared library and a C header file:
-buildmode c-shared
(This is explained in more detail in this helpful tutorial: http://blog.ralch.com/tutorial/golang-sharing-libraries/)
With the new build mode, you no longer have to write a C glue layer yourself (as previously suggested in earlier responses). Once you have the shared-library and the header file, you can proceed to use FFI to call the Go-created shared library (example here: https://www.amberbit.com/blog/2014/6/12/calling-c-cpp-from-ruby/)
Short version: The default inspect method for a class displays the object's address.* How can I do this in a custom inspect method of my own?
*(To be clear, I want the 8-digit hex number you would normally get from inspect. I don't care about the actual memory address. I'm just calling it a memory address because it looks like one. I know Ruby is memory-safe.)
Long version: I have two classes, Thing and ThingList. ThingList is a subclass of Array specifically designed to hold Things. Due to the nature of Things and the way they are used in my program, Things have an instance variable #container that points back to the ThingList that holds the Thing.
It is possible for two Things to have exactly the same data. Therefore, when I'm debugging the application, the only way I can reliably differentiate between two Things is to use inspect, which displays their address. When I inspect a Thing, however, I get pages upon pages of output because inspect will recursively inspect #container, causing every Thing in the list to be inspected as well!
All I need is the first part of that output. How can I write a custom inspect method on Thing that will just display this?
#<Thing:0xb7727704>
EDIT: I just realized that the default to_s does exactly this. I didn't notice this earlier because I have a custom to_s that provides human-readable details about the object.
Assume that I cannot use to_s, and that I must write a custom inspect.
You can get the address using object_id and multiplying it by 2* and display it in hex using sprintf (aka %):
"#<Thing:0x%08x>" % (object_id * 2)
Of course, as long as you only need the number to be unique and don't care that it's the actual address, you can just leave out the * 2.
* For reasons that you don't need to understand (meaning: I don't understand them), object_id returns half the object's memory address, so you need to multiply by 2 to get the actual address.
This is impossible. There is no way in Ruby to get the memory address of an object, since Ruby is a memory-safe language which has (by design) no methods for accessing memory directly. In fact, in many implementations of Ruby, objects don't even have a memory address. And in most of the implementations that do map objects directly to memory, the memory address potentially changes after every garbage collection.
The reason why using the memory address as an identifier in current versions of MRI and YARV accidentally works, is because they have a crappy garbage collector implementation that never defragments memory. All other implementations have garbage collectors which do defragment memory, and thus move objects around in memory, thereby changing their address.
If you tie your implementation to the memory address, your code will only ever work on slow implementations with crappy garbage collectors. And it isn't even guaranteed that MRI and YARV will always have crappy garbage collectors, in fact, in both implementations the garbage collector has been identified as one of the major performance bottlenecks and it is safe to assume that there will be changes to the garbage collectors. There are already some major changes to YARV's garbage collector in the SVN, which will be part of YARV 1.9.3 and YARV 2.0.
If you want an ID for objects, use Object#object_id.
Instead of subclassing Array your class instances could delegate to one for the desired methods so that you don't inherit the overridden inspect method.
I am creating a Ruby On Rails website, and for one part it needs to be dynamic so that (sorta) trusted users can make parts of the website work differently. For this, I need a scripting language. In a sort of similar project in ASP.Net, I wrote my own scripting language/DSL. I can not use that source code(written at work) though, and I don't want to make another scripting language if I don't have to.
So, what choices do I have? The scripting must be locked down and not be able to crash my server or anything. I'd really like if I could use Ruby as the scripting language, but it's not strictly necessary. Also, this scripting part will be called on almost every request for the website, sometimes more than once. So, speed is a factor.
I looked at the RubyLuaBridge but it is Alpha status and seems dead.
What choices for a scripting language do I have in a Ruby project?
Also, I will have full control over where this project is deployed(root access), so there are no real limits..
There's also Rufus-lua though it's at version 0.1.0...
What about JRuby? You can use java implementation of many scripting language, such as javascript, scheme etc
Well, since it hasn't been suggested yet, there's Locking Ruby In The Safe as described by the Pickaxe book. This allows you to use Ruby as the language without significant slowdown AFAIK.
This technique is intended to allow safe sandboxing of untrusted Ruby code and bug fixes and discussions are directed toward keeping it that way, but infinite loops and some other things still allow malicious users to peg the CPU. (e.g. this discussion maybe.)
What I don't know is how you return data that is inherently safe to use from outside the safe thread. A singleton object (for instance) can mimic whatever class and then do something dangerous when any method is called in the returning thread. I'm still googling around about it. (The Ruby Programming Language says that level 4 "Prevents metaprogramming methods" which would allow you to safely verify the class of a returned object, which I suppose would make results safe to use.)
Barring that, it might not be hard (*snrk*) to implement a Lisp-1 with dynamic scope since you already have a garbage collector.
I have written a Ruby application which parses lots of data from sources in different formats html, xml and csv files. How can I find out what areas of the code are taking the longest?
Is there any good resources on how to improve the performance of Ruby applications? Or do you have any performance coding standards you always follow?
For example do you always join your string with
output = String.new
output << part_one
output << part_two
output << '\n'
or would you use
output = "#{part_one}#{part_two}\n"
Well, there are certain well known practices like string concatenation is way slower than "#{value}" but in order to find out where you script is consuming most of its time or more time than required, you need to do profiling. There is a ruby gem called ruby-prof. The profiler can bring to your notice even those performance issues that may rarely occur. I have been using it a lot and find it very helpful. Here is some information about it directly from its official site
ruby-prof is a fast code profiler for
Ruby. Its features include:
Speed - it is a C extension and therefore many times faster than the
standard Ruby profiler.
Modes - Ruby prof can measure a number of different parameters,
including
call times, memory usage and object allocations.
Reports - can generate text and cross-referenced html reports
Flat Profiles - similar to the reports generated by the standard Ruby
profiler
Graph profiles - similar to GProf, these show how long a method runs,
which methods call it and which
methods it calls.
Call tree profiles - outputs results in the calltree format
suitable for the KCacheGrind profiling
tool.
Threads - supports profiling multiple threads simultaneously
Recursive calls - supports profiling recursive method calls
You can test the performance of individual sections of code with the standard Benchmark module.
You could also test your code on different implementations of Ruby (eg 1.9, Rubinius) and see if that speeds things up.
Of course if your problems are algorithmic in nature then there's not too much point in worrying about things like string concatenation speeds...
The answer to the string concatenation is here: https://web.archive.org/web/20090122123342/http://blog.cbciweb.com/2008/06/10/ruby-performance-use-double-quotes-vs-single-quotes
Besides what's written above I also recommend watching the Scaling Ruby screencast. It has some interesting tips&tricks on how to write faster Ruby code.
There is also a set of dTrace tools for Ruby you can use in the dTraceToolkit