Idiomatic lazy sorting by multiple criteria - ruby

Question

In Ruby, the most common way to sort by multiple criteria is to use sort_by with the sorting function returning an array of the values corresponding to each sorting criterion, in order of decreasing importance, e.g.:
Dir["*"].sort_by { |f| [test(?s, f) || 0, test(?M, f), f] }
will sort the directory entries by size, then by mtime, then finally by the filename. This is efficient to the extent that it uses a Schwartzian transform to only calculate the size and mtime of each file once, not once per comparison. However it is not truly lazy, since it calculates the mtime for every single file, but if (say) every file in the directory had a different size, it should not be necessary to calculate any mtimes.
This is not a big problem in this case, since looking up the mtime immediately after looking up the size should be efficient due to caching at the kernel level (e.g. IIRC on Linux they both come from a stat(2) syscall), and I wouldn't be surprised if Ruby has its own optimizations too. But imagine if the second criterion was not the mtime, but (say) the number of occurrences of a string within the file, and the files in question are huge. In this case you'd really want lazy evaluation, to avoid reading the whole of these huge files if sorting by size is sufficient.
At the time of writing, the Wikibooks entry for Algorithm Implementation/Sorting/Schwartzian transform suggests this solution:
sorted_files =
Dir["*"]. # Get all files
# compute tuples of name, size, modtime
collect{|f| [f, test(?s, f), test(?M, f)]}.
sort {|a, b| # sort
a[1] <=> b[1] or # -- by increasing size
b[2] <=> a[2] or # -- by age descending
a[0] <=> b[0] # -- by name
}.collect{|a| a[0]} # extract original name
This kind of approach is copied from Perl, where
sort {
$a->[1] <=> $b->[1] # sort first numerically by size (smallest first)
or $b->[2] <=> $a->[2] # then numerically descending by modtime age (oldest first)
or $a->[0] cmp $b->[0] # then stringwise by original name
}
works beautifully because Perl has a quirk where 0 or $foo evaluates to $foo. But in Ruby, it's broken because 0 or foo evaluates to 0. So in effect, the Wikibooks implementation totally ignores mtimes and filenames, and only sorts by size. I've dusted off my Wikibooks account so that I can fix this, but I'm wondering: what is the cleanest way of combining the results of multiple <=> spaceship operator comparisons in Ruby?
I'll give a concrete-ish example to clarify the question. Let's assume we have two types of evaluation which may be required as criteria during the sort. The first is relatively cheap:
def size(a)
# get the size of file `a`, and if we're feeling keen,
# memoize the results
...
end
The second is expensive:
def matches(a)
# count the number of occurrences of a string
# in file `a`, which could be a large file, and
# memoize the results
...
end
And we want to sort first by size ascending, then descending by number of matches. We can't use a Schwartzian transform, because that would non-lazily call matches() on every item.
We could define a helper like
def nil_if_equal(result)
result == 0 ? nil : result
end
and then do:
sort {|a, b|
nil_if_equal(size(a) <=> size(b)) or
matches(b) <=> matches(a)
}
If there are n criteria to sort by then you'd need n-1 invocations of nil_if_equal here, since only the last sorting criteria doesn't require it.
So is there a more idiomatic way than this which can avoid the need for nil_if_equal?

Answer 1

No idea how idiomatic it is, but here's a way to use sort_by again. Instead of
for example
['bab', 'foo', 'so', 'bar'].sort_by { |s| [s.size, count_a(s), count_b(s)] }
do this to make count_a(s) and count_b(s) lazy and memoized:
['bab', 'foo', 'so', 'bar'].sort_by { |s| [s.size, lazy{count_a(s)}, lazy{count_b(s)}] }
My lazy makes the block act like a lazy and memoizing version of the value it yields.
Demo output, showing we only count what's necessary (i.e., don't count in 'so' since it has a unique size and don't count 'b' in 'foo' since its 'a'-count is unique among the size-3 strings):
Counting 'a' in 'bab'.
Counting 'a' in 'foo'.
Counting 'a' in 'bar'.
Counting 'b' in 'bab'.
Counting 'b' in 'bar'.
["so", "foo", "bar", "bab"]
Demo code:
def lazy(&block)
def block.value
(#value ||= [self.yield])[0]
end
def block.<=>(other)
value <=> other.value
end
block
end
def count_a(s)
puts "Counting 'a' in '#{s}'."
s.count('a')
end
def count_b(s)
puts "Counting 'b' in '#{s}'."
s.count('b')
end
p ['bab', 'foo', 'so', 'bar'].sort_by { |s| [s.size, lazy{count_a(s)}, lazy{count_b(s)}] }
A different way to make value memoizing: If it ever gets called, it immediately replaces itself with a method just returning the stored value:
def block.value
def self.value; #value end
#value = self.yield
end