At the office, we had this little brain teaser:
class Bicycle
def spares
{tire_size: 21}.merge(local_spares)
end
def local_spares
{}
end
end
class RoadBike < Bicycle
def local_spares
{tape_color: 'red'}
end
end
roadbike = RoadBike.new
roadbike.spares
Most of us didn't get what roadbike.spares output is unless we ran the whole code in the console. We had our different hunch on the behaviour but can anyone break it down to me what really happened here?
If anyone's wondering, the output is {tire_size: 21, tape_color: 'red'}
It's quite obvious, RoadBike#spares (which is the same as Bicycle#spares, because RoadBike doesn't override this method) calls internally RoadBike#local_spares, merges its returned value to {tire_size: 21} hash and returns the result.
No surprise at all.
This is called method overriding. The RoadBike#local_spares method overrides the Bicycle#local_spares method because RoadBike inherits from Bicycle.
When you send a message to an object, Ruby will try to find a method with the same name as the message to execute. It first looks at the object's class, then at that class's superclass, then at that class's superclass and so on.
When you send a RoadBike object the spares message, it will first try (and fail) to find a method named spares in RoadBike. Then it will look into its superclass (Bicycle) and succeed.
The body of that method contains a message send of local_spares to the receiver object. Again, Ruby tries to find a method named local_spares in the class of the object (still RoadBike) and succeeds, so it executes that method.
This is all just standard inheritance and method overriding. There's nothing really special or surprising or "brain teaserish" about that. In fact, this is pretty much the whole point of inheritance and method overriding: that more specialized objects can provide more specialized implementations than their more general parents.
Note: the method lookup algorithm is in reality a bit more involved than that.
First off, what happens if there is no more superclass, and the method still hasn't been found? In that case, Ruby will send the message method_missing to the receiver and pass along the name of the method that it tried to look up. Only if the method_missing method also can't be found, will Ruby raise a NoMethodError.
And secondly, there are singleton classes, included modules, prepended modules, and Refinements to consider. So, really Ruby will look at the object's singleton class first, before it looks at the class and then the superclass and so on. And "look at the class" actually means look first at the prepended modules (in reverse order), then at the class itself, and then at the included modules (again in reverse order). Oh, and that has to be done recursively as well, so for each prepended module look first at the prepended modules of that module, then at the module itself, then the included modules and so forth.
(Oh, and Refinements throw a another wrinkle in this, obviously.)
Most of the Ruby implementations simplify this algorithm greatly by separating their internal notion of what a "class" is from the programmer's notion, by introducing the concept of "hidden classes" (YARV calls them "virtual classes") that exist inside the implementation but aren't exposed to the programmer. So, for example, the singleton class of an object would be a hidden class, and the class pointer of the object would simply point to the singleton class and the superclass pointer of the singleton class would point to the actual class of the object. When you include a module into a class, the implementation will synthesize a hidden class (which YARV calls an "include class") for the module and insert it as the superclass of the class and make the former superclass the superclass of the hidden class. Methods like Object#class and Class#superclass would then simply follow the superclass chain until it finds the first non-hidden class and return that, instead of returning the class/superclass pointer directly.
This makes methods like Object#class, Class#superclass and Module#ancestors slightly more complex, because they have to skip hidden classes, but it simplifies the method lookup algorithm, which is one of the most important performance bottlenecks in any object-oriented system.
Related
I've read stackoverflow posts on this topic as well as several articles which include A Primer on Ruby Method Lookup, What is the method lookup path in Ruby. In addition, I checked out the object model chapter in Ruby Metaprogramming 2, asked in a few chat rooms, and made this reddit thread. Short of learning C, I've done what I can to figure this out.
As described by the resources above, these 6 places are checked (in order) during method lookup on a receiving object like fido_instance:
singleton class of fido_instance
IClass (from an extended module)
IClass (from a prepended module)
class
IClass (from an included module)
superclass (if method isn't found here, repeat steps 4-6)
Obviously, the diagram is incomplete, and all of these singleton classes might not have been created in the real world. Still, those 6 steps leave a lot to be desired, and don't cover the following scenario. If there were no extended/prepended IClass above the singleton class of fido_instance, then there's no explanation of whether step 4 is executed on the singleton class of fido_instance. I have to assume not since the whole method lookup would short circuit.
If I were to guess a set of steps that could explain ruby's method lookup behavior, it might look like:
check fido_instance.class for the method. (obviously, ruby isn't going to use its own #class method to do the method lookup, but it conveys the logic of the process)
check fido_instance.class.superclass for the method. Keep adding .superclass and checking for the method until no superclasses are left. (again, ruby isn't going to use its own #superclass method)
method wasn't found. Start at step 1, looking for #method_missing this time.
I also recall reading that there's a separate method lookup process if the receiving object is a class, but I can't recall where.
So what's the correct, detailed explanation that doesn't involve knowing C?
There's a ... gem ... in that second ref that I think gets to the core of the answer: ancestors of the singleton class. Applied to your object, it would be:
fido_instance.singleton_class.ancestors
This will always give you the order of method lookup that Ruby uses. It's pretty simple when you view it this way, and that's the bottom line answer to your question. Ruby will start at the singleton_class and work its way up the ancestors looking for that method. Using your diagram:
fido.singleton_class.ancestors
=> [Fetch, WagTail, DogClass, Object, Kernel, BasicObject]
(Note1: Bark is not part of this output because you used extend instead of include. More on this in a second.)
(Note2: If it doesn't find it all the way up to BasicObject, then it will call method_missing up the same ancestry chain.)
It's no different when calling a method on a class, because in Ruby a class it just an instance of class Class. So DogClass.method1 will search for method1 on DogClass.singleton_class and then up its ancestry chain, just like before.
DogClass.singleton_class.ancestors
=> [Bark, Class, Module, Object, Kernel, BasicObject]
Since you used extend for Bark, this is where we find it! So if Bark defined a method bark, then you can call DogClass.bark because that method is defined in DogClass's singleton_class' ancestors.
To understand what that ancestry tree will be (instead of relying on printing it out every time), you simply need to know how the ancestry is modified by subclassing, extend, include, prepend, etc.
Subclassing gives the child class the entire ancestry chain of its superclass.
includeing a module in a class C adds that module into the ancestry chain after C and before everything else.
prepending a module in a class C adds that module into the ancestry chain before everything, including C and any currently prepended modules.
def x.method1 adds method1 to x.singleton_class. Similarly x.extend(M) will add M to the ancestry of x.singleton_class (but not to x.class). Note that the latter is exactly what happened with Bark and DogClass.singleton_class, but can equally apply to any object.
Leaving out extend from the above list because it does not modify the object's ancestry chain. It does modify the ancestry of that object's singleton_class -- as we saw, Bark was included in DogClass.singleton_class.ancestors.
Tangent:
The bit about class methods above is the key to me for understanding how important singleton classes are to Ruby. You obviously can't define bark on DogClass.class, because DogClass.class == Class and we don't want bark on Class! So how can we allow DogClass to be an instance of Class, allowing it to have a (class) method bark that is defined for DogClass but not unrelated classes? Using the singleton class! In this way, defining a "class method", like by def self.x inside class C, is sort of like C.singleton_class.send(:define_method, :x) {...}.
I understand that all classes in ruby are instances of metaclass Class. And that "regular" objects are instances of these classes (the instances of metaclass Class).
But I keep wondering, I mean classes are root of objects, classes are themselves instances of Class (called metaclass because its instances are classes). I saw in some blogs some overriding of method new, of class Class.
So Class behaves as a class, but its instances are classes. So it seems we have a circle, it looks likes class Class is an instance of itself.
I'm clearly missing a point here. What is the origin of class Class?
Here's an example that's confusing me:
class Class
def new
#something
end
end
But keyword class implies an instance of class Class. So how do this work?
how do this work
Easy: it doesn't. Not in Ruby, anyway.
Just like in most other languages, there are some core entities that are simply assumed to exist. They fall from the sky, materialize out of thin air, magically appear.
In Ruby, some of those magic things are:
Object doesn't have a superclass, but you cannot define a class with no superclass, the implicit direct superclass is always Object. [Note: there may be implementation-defined superclasses of Object, but eventually, there will be one which doesn't have a superclass.]
Object is an instance of Class, which is a subclass of Object (which means that indirectly Object is an instance of Object itself)
Class is a subclass of Module, which is an instance of Class
Class is an instance of Class
None of these things can be explained in Ruby.
BasicObject, Object, Module and Class all need to spring into existence at the same time because they have circular dependencies.
Just because this relationship cannot be expressed in Ruby code, doesn't mean the Ruby Language Specification can't say it has to be so. It's up to the implementor to figure out a way to do this. After all, the Ruby implementation has a level of access to the objects that you as a programmer don't have.
For example, the Ruby implementation could first create BasicObject, setting both its superclass pointer and its class pointer to null.
Then, it creates Object, setting its superclass pointer to BasicObject and its class pointer to null.
Next, it creates Module, setting its superclass pointer to Object and its class pointer to null.
Lastly, it creates Class, setting its superclass pointer to Module and its class pointer to null.
Now, we can overwrite BasicObject's, Object's, Module's, and Class's class pointer to point to Class, and we're done.
This is easy to do from outside the system, it just looks weird from the inside.
Once they do exist, however, it is perfectly possible to implement most of their behavior in plain Ruby. You only need very barebones versions of those classes, thanks to Ruby's open classes, you can add any missing functionality at a later time.
In your example, the class Class is not creating a new class named Class, it is reopening the existing class Class, which was given to us by the runtime environment.
So, it is perfectly possible to explain the default behavior of Class#new in plain Ruby:
class Class
def new(*args, &block)
obj = allocate # another magic thing that cannot be explained in Ruby
obj.initialize(*args, &block)
return obj
end
end
[Note: actually, initialize is private, so you need to use obj.send(:initialize, *args, &block) to circumvent the access restriction.]
BTW: Class#allocate is another one of those magic things. It allocates a new empty object in Ruby's object space, which is something that cannot be done in Ruby. So, Class#allocate is something that has to be provided by the runtime system as well.
There is a meta-circularity given by the "twist" link. It is the built-in superclass link from the root's eigenclass to the Class class. This can be expressed by
BasicObject.singleton_class.superclass == Class
A clue to understanding the .class map is seeing this map as derived from the eigenclass and superclass links: for an object x, x.class is the first class in the superclass chain of x's eigenclass. This can be expressed by
x.class == x.eigenclass.superclass(n)
where eigenclass is a "conceptual alias" of singleton_class
(resistant to issues with immediate values), y.superclass(i) means i-th superclass of y and n is smallest such that x.eigenclass.superclass(n) is a class. Equivalently, eigenclasses in the superclass chain of x.eigenclass are skipped (see rb_class_real which also reveals that in MRI, even superclass links are implemented indirectly – they arise by skipping "iclasses").
This results in that the class of every class (as well as of every eigenclass) is constantly the Class class.
A picture is provided by this diagram.
The metaclass confusion has 2 main sources:
Smalltalk. The Smalltalk-80 object model contains conceptual inconsistencies that are rectified by the Ruby object model. In addition, Smalltalk literature uses dialectics in terminology, which unfortunately has not been sufficiently remedied in the Ruby literature.
The definition of metaclass. At present, the definition states that metaclasses are classes of classes. However, for so called "implicit metaclasses" (the case of Ruby and Smalltalk-80) a much more fitting definition would be that of meta-objects of classes.
Yes, Class is an instance of itself. It's a subclass of Module, which is also an instance of class, and Module is a subclass of Object, which is also an instance of Class. It is indeed quite circular — but this is part of the core language, not something in a library. The Ruby runtime itself doesn't have the same limits thast you or I do when we're writing Ruby code.
I've never heard the word "metaclass" used to talk about Class, though. It isn't used much in Ruby at all, but when it is, it's usually a synonym for what's officially called a "singleton class of an object," which is an even more confusing topic than Object-Module-Class triangle.
Though it is a little out of date, this article by _why may help in understanding the behavior. You can find an even deeper dive into the subject in Paolo Perrotta's Metaprogramming Ruby.
In the book OO Design in Ruby, Sandi Metz says that the main use of modules is to implement duck types with them and include them in every class needed. Why is the Ruby Kernel a module included in Object? As far as I know it isn't used anywhere else. What's the point of using a module?
Ideally,
Methods in spirit (that are applicable to any object), that is, methods that make use of the receiver, should be defined on the Object class, while
Procedures (provided globally), that is, methods that ignore the receiver, should be collected in the Kernel module.
Kernel#puts, for example doesn't do anything with its receiver; it doesn't call private methods on it, it doesn't access any instance variables of it, it only acts on its arguments.
Procedures in Ruby are faked by using Ruby's feature that a receiver that is equal to self can be omitted. They are also often made private to prevent them from being called with an explicit receiver and thus being even more confusing. E.g., "Hello".puts would print a newline and nothing else since puts only cares about its arguments, not its receiver. By making it private, it can only be called as puts "Hello".
In reality, due to the long history of Ruby, that separation hasn't always been strictly followed. It is also additionally complicated by the fact that some Kernel methods are documented in Object and vice versa, and even further by the fact that when you define something which looks like a global procedure, and which by the above reasoning should then end up in Kernel, it actually ends up as a private instance method in Object.
As you already pointed out: Modules provide a way to collect and structure behavior, so does the Kernel module. This module is mixed in early into the class Object so every Ruby class will provide these methods. There is only a BasicObject before in hierarchy, it's child Objects purpose is only to get extended by the Kernel methods. BasicObject has only 7 methods that very very basic like new, __send__ or __id__.
class Object < BasicObject
include Kernel # all those many default methods we appreciate :)
end
I am new to Ruby and I saw methods defined like:
def method_one
puts "method 1"
end
class MyClass
method_one
def method_two
puts "method 2"
end
end
The way method_one is used reminds me of Python decorators.The output of
c = MyClass.new
c.method_two
is
method 1
method 2
I have been trying to search for more information about this syntax/language feature in the Ruby documentation on the web but I don't know what keywords to search for.
What this is thing called?
TL;DR
This code doesn't do what you think it does. Don't do stuff like this.
Ruby's Top-Level Object
Ruby lets you define methods outside a class. These methods exist on a top-level object, which you can (generally) treat as a sort of catch-all namespace. You can see various posts like What is the Ruby Top-Level? for more details, but you shouldn't really need to care.
In your original post, method_one is just a method defined in the top-level. It is therefore available to classes and methods nested within the top-level, such as MyClass.
Methods in Classes
Despite what you think, the following doesn't actually declare a :method_one class or instance method on MyClass:
class MyClass
method_one
def method_two; end
end
Instead, Ruby calls the top-level ::method_one during the definition of the class, but it never becomes a class method (e.g. MyClass::method_one) or an instance method (e.g. MyClass.new.method_one). There might be a few use cases for doing this (e.g. printing debugging information, test injection, etc.) but it's confusing, error-prone, and generally to be avoided unless you have a really strong use case for it.
Better Options
In general, when you see something like this outside an academic lesson, the programmer probably meant to do one of the following:
Extend a class.
Add a singleton method to a class.
Include a module in a class.
Set up a closure during class definition.
The last gets into murky areas of metaprogramming, at which point you should probably be looking at updating your class initializer, or passing Proc or lambda objects around instead. Ruby lets you do all sorts of weird and wonderful things, but that doesn't mean you should.
I think you're a little mislead; the output of:
c = MyClass.new
c.method_two
is
#<MyClass:0x007feda41acf18>
"method 2"
You're not going to see method one until the class is loaded or if you're in IRB you enter the last end statement.
I would suggest looking into ruby's initialize method.
I understand that all classes in ruby are instances of metaclass Class. And that "regular" objects are instances of these classes (the instances of metaclass Class).
But I keep wondering, I mean classes are root of objects, classes are themselves instances of Class (called metaclass because its instances are classes). I saw in some blogs some overriding of method new, of class Class.
So Class behaves as a class, but its instances are classes. So it seems we have a circle, it looks likes class Class is an instance of itself.
I'm clearly missing a point here. What is the origin of class Class?
Here's an example that's confusing me:
class Class
def new
#something
end
end
But keyword class implies an instance of class Class. So how do this work?
how do this work
Easy: it doesn't. Not in Ruby, anyway.
Just like in most other languages, there are some core entities that are simply assumed to exist. They fall from the sky, materialize out of thin air, magically appear.
In Ruby, some of those magic things are:
Object doesn't have a superclass, but you cannot define a class with no superclass, the implicit direct superclass is always Object. [Note: there may be implementation-defined superclasses of Object, but eventually, there will be one which doesn't have a superclass.]
Object is an instance of Class, which is a subclass of Object (which means that indirectly Object is an instance of Object itself)
Class is a subclass of Module, which is an instance of Class
Class is an instance of Class
None of these things can be explained in Ruby.
BasicObject, Object, Module and Class all need to spring into existence at the same time because they have circular dependencies.
Just because this relationship cannot be expressed in Ruby code, doesn't mean the Ruby Language Specification can't say it has to be so. It's up to the implementor to figure out a way to do this. After all, the Ruby implementation has a level of access to the objects that you as a programmer don't have.
For example, the Ruby implementation could first create BasicObject, setting both its superclass pointer and its class pointer to null.
Then, it creates Object, setting its superclass pointer to BasicObject and its class pointer to null.
Next, it creates Module, setting its superclass pointer to Object and its class pointer to null.
Lastly, it creates Class, setting its superclass pointer to Module and its class pointer to null.
Now, we can overwrite BasicObject's, Object's, Module's, and Class's class pointer to point to Class, and we're done.
This is easy to do from outside the system, it just looks weird from the inside.
Once they do exist, however, it is perfectly possible to implement most of their behavior in plain Ruby. You only need very barebones versions of those classes, thanks to Ruby's open classes, you can add any missing functionality at a later time.
In your example, the class Class is not creating a new class named Class, it is reopening the existing class Class, which was given to us by the runtime environment.
So, it is perfectly possible to explain the default behavior of Class#new in plain Ruby:
class Class
def new(*args, &block)
obj = allocate # another magic thing that cannot be explained in Ruby
obj.initialize(*args, &block)
return obj
end
end
[Note: actually, initialize is private, so you need to use obj.send(:initialize, *args, &block) to circumvent the access restriction.]
BTW: Class#allocate is another one of those magic things. It allocates a new empty object in Ruby's object space, which is something that cannot be done in Ruby. So, Class#allocate is something that has to be provided by the runtime system as well.
There is a meta-circularity given by the "twist" link. It is the built-in superclass link from the root's eigenclass to the Class class. This can be expressed by
BasicObject.singleton_class.superclass == Class
A clue to understanding the .class map is seeing this map as derived from the eigenclass and superclass links: for an object x, x.class is the first class in the superclass chain of x's eigenclass. This can be expressed by
x.class == x.eigenclass.superclass(n)
where eigenclass is a "conceptual alias" of singleton_class
(resistant to issues with immediate values), y.superclass(i) means i-th superclass of y and n is smallest such that x.eigenclass.superclass(n) is a class. Equivalently, eigenclasses in the superclass chain of x.eigenclass are skipped (see rb_class_real which also reveals that in MRI, even superclass links are implemented indirectly – they arise by skipping "iclasses").
This results in that the class of every class (as well as of every eigenclass) is constantly the Class class.
A picture is provided by this diagram.
The metaclass confusion has 2 main sources:
Smalltalk. The Smalltalk-80 object model contains conceptual inconsistencies that are rectified by the Ruby object model. In addition, Smalltalk literature uses dialectics in terminology, which unfortunately has not been sufficiently remedied in the Ruby literature.
The definition of metaclass. At present, the definition states that metaclasses are classes of classes. However, for so called "implicit metaclasses" (the case of Ruby and Smalltalk-80) a much more fitting definition would be that of meta-objects of classes.
Yes, Class is an instance of itself. It's a subclass of Module, which is also an instance of class, and Module is a subclass of Object, which is also an instance of Class. It is indeed quite circular — but this is part of the core language, not something in a library. The Ruby runtime itself doesn't have the same limits thast you or I do when we're writing Ruby code.
I've never heard the word "metaclass" used to talk about Class, though. It isn't used much in Ruby at all, but when it is, it's usually a synonym for what's officially called a "singleton class of an object," which is an even more confusing topic than Object-Module-Class triangle.
Though it is a little out of date, this article by _why may help in understanding the behavior. You can find an even deeper dive into the subject in Paolo Perrotta's Metaprogramming Ruby.