Why this XML parsing Ruby code runs slower with GC disabled? - ruby

Question

I've got a piece of code parsing 500 MB XML file using libxml-ruby gem. What is surprising to me, this code runs slower with GC disabled, which seems counter-intuitive. What might be the reason? I've go plenty of memory available and the system is not swapping.
require 'xml'
#GC.disable
#reader = XML::Reader.file('books.xml', :options => XML::Parser::Options::NOBLANKS)
#reader.read
#reader.read
while #reader.name == 'book'
book_id = #reader.get_attribute('id')
#reader.read
until #reader.name == 'book' && #reader.node_type == XML::Reader::TYPE_END_ELEMENT
case #reader.name
when 'author'
author = #reader.read_string
when 'title'
title = #reader.read_string
when 'genre'
genre = #reader.read_string
when 'price'
price = #reader.read_string
when 'publish_date'
publish_date = #reader.read_string
when 'description'
description = #reader.read_string
end
#reader.next
end
#reader.read
end
#reader.close
Here are the results I got:
ruby gc on gc off
2.2.0 16.93s 18.81s
2.1.5 16.22s 18.58s
2.0.0 17.63s 17.99s
Why disable the garbage collector? I've read in Ruby Performance Optimization book that Ruby is slow mostly because programmers don't think about memory consumption, which makes garbage collector use a lot of execution time. Thus turning off the GC should instantly speed things up (at the cost of memory usage of course), as long as the system is not swapping.
I wanted to see if my XML parsing module can be improved, so I started experimenting with it by disabling the GC, which brought me to this problem. I expected a significant speed-up with GC disabled, but instead I got the opposite. I know that differences are not huge, but still this is strange to me.
libxml-ruby gem uses a native C LibXML implementation under the hood - can it be the reason?
The file that I used is manually multiplicated books.xml sample downloaded from some Microsoft documentation:
<catalog>
<book id="bk101">
<author>John Doe</author>
<title>XML for dummies</title>
<genre>Computer</genre>
<price>44.95</price>
<publish_date>2000-10-01</publish_date>
<description>Some description</description>
</book>
....
</catalog>
My setup: OS X Yosemite, Intel Core i5 2.6 GHz, 16GB of RAM.
Thanks for any suggestions.

Answer 1

You are forgetting the operating system - you've disabled GC in your MRI process but you have no control over the linux/unix kernel and how it allocates memory to your MRI application.
In fact, I believe that by disabling GC you have significantly hamstrung the behaviour of your application, making it likely that your program will need to continuously request more RAM from the kernel. This will likely involve some form of overhead in the kernel as it allocates swap or memory to you.
Your source data is a 500 mb xml file that you are reading, node by node, into your MRI program's memory footprint. It's likely your MRI process consumes several GB of data by the time it is done processing; and none of the values in your main reading block are discarded after each iteration - they simply hang around in memory, and are only finally cleaned up when your application exits and the memory is handed back to the operating system.
GC is in place to manage this; it is intended to prevent your application from requesting additional memory from the kernel unless it absolutely needs it, and to allow your application to run "well enough" within the memory allocated to it within reason.
So I'm not honestly surprised you see a slowdown with GC disabled. What would be telling is the load average and swap usage of your box during your benchmarks.