How to understand the work flow in ruby enumerator chain - ruby

Question

The code below produces two different results.
letters = %w(e d c b a)
letters.group_by.each_with_index { |item, index| index % 3 }
#=> {0=>["e", "b"], 1=>["d", "a"], 2=>["c"]}
letters.each_with_index.group_by { |item, index| index % 3 }
#=> {0=>[["e", 0], ["b", 3]], 1=>[["d", 1], ["a", 4]], 2=>[["c", 2]]}
I think the execution flow is from right to left, and the data flow is from the left to right. The block should be passed as parameter from right to left.
Using puts, I observed that the block is executed in the inner each.
In the first chain, group_by should ask each for data, each will return the result of index%3, and group_by should process the result and yield it to another block. But how is the block passed? If the block is executed in each, each would not pass two parameters item and index but only one parameter item.
In the second chain, in my understanding, each_with_index will receive the data from each method first; each yields to index%3. In that case, how can each_with_index process index%3?
It seems my understanding is somehow wrong. Can anyone illustrate theses two examples with details and give the general work flow in such cases?

Answer 1

Proxy objects
Both execution and data flows are from left to right, as with any method call in Ruby.
Conceptually, it can help to read Enumerators call chains from right to left, though, because they're a kind of a proxy object.
Called without block, they just remember in which order which method has been called. The method is then only really called when it's needed, for example when the Enumerator is converted back to an Array or the elements are printed on screen.
If no such method is called at the end of the chain, basically nothing happens:
[1,2,3].each_with_index.each_with_index.each_with_index.each_with_index
# #<Enumerator: ...>
[1,2,3].each_with_index.each_with_index.each_with_index.each_with_index.to_a
# [[[[[1, 0], 0], 0], 0], [[[[2, 1], 1], 1], 1], [[[[3, 2], 2], 2], 2]]
This behaviour makes it possible to work with very large streams of objects, without needing to pass huge arrays between method calls. If the output isn't needed, nothing is calculated. If 3 elements are needed at the end, only 3 elements are calculated.
The proxy pattern is heavily used in Rails, for example with ActiveRecord::Relation :
#person = Person.where(name: "Jason").where(age: 26)
It would be inefficient to launch 2 DB queries in this case. You can only know that at the end of the chained methods, though. Here's a related answer (How does Rails ActiveRecord chain “where” clauses without multiple queries?)
MyEnumerator
Here's a quick and dirty MyEnumerator class. It might help you understand the logic for the method calls in your question:
class MyEnumerator < Array
def initialize(*p)
#methods = []
#blocks = []
super
end
def group_by(&b)
save_method_and_block(__method__, &b)
self
end
def each_with_index(&b)
save_method_and_block(__method__, &b)
self
end
def to_s
"MyEnumerable object #{inspect} with methods : #{#methods} and #{#blocks}"
end
def apply
result = to_a
puts "Starting with #{result}"
#methods.zip(#blocks).each do |m, b|
if b
puts "Apply method #{m} with block #{b} to #{result}"
else
puts "Apply method #{m} without block to #{result}"
end
result = result.send(m, &b)
end
result
end
private
def save_method_and_block(method, &b)
#methods << method
#blocks << b
end
end
letters = %w[e d c b a]
puts MyEnumerator.new(letters).group_by.each_with_index { |_, i| i % 3 }.to_s
# MyEnumerable object ["e", "d", "c", "b", "a"] with methods : [:group_by, :each_with_index] and [nil, #<Proc:0x00000001da2518#my_enumerator.rb:35>]
puts MyEnumerator.new(letters).group_by.each_with_index { |_, i| i % 3 }.apply
# Starting with ["e", "d", "c", "b", "a"]
# Apply method group_by without block to ["e", "d", "c", "b", "a"]
# Apply method each_with_index with block #<Proc:0x00000000e2cb38#my_enumerator.rb:42> to #<Enumerator:0x00000000e2c610>
# {0=>["e", "b"], 1=>["d", "a"], 2=>["c"]}
puts MyEnumerator.new(letters).each_with_index.group_by { |_item, index| index % 3 }.to_s
# MyEnumerable object ["e", "d", "c", "b", "a"] with methods : [:each_with_index, :group_by] and [nil, #<Proc:0x0000000266c220#my_enumerator.rb:48>]
puts MyEnumerator.new(letters).each_with_index.group_by { |_item, index| index % 3 }.apply
# Apply method each_with_index without block to ["e", "d", "c", "b", "a"]
# Apply method group_by with block #<Proc:0x0000000266bd70#my_enumerator.rb:50> to #<Enumerator:0x0000000266b938>
# {0=>[["e", 0], ["b", 3]], 1=>[["d", 1], ["a", 4]], 2=>[["c", 2]]}