Parsing complex xml in Ruby with sax parsing - ruby

Question

I've been searching and searching for a couple of days on how to do this, but I can't seem to understand sax parsing in a way that will help me accomplish what I want to accomplish. I understand sax parsing on a basic level, but I can't wrap my mind around how to use it to extract the data I need to extract.
I'm currently using:
xml data
ruby
the saxerator gem (I'm not sold on this, it's just the easiest I've found so far that I'm able to understand clearly enough)
Here's a sample of the xml structure:
<result created="2015-08-26T09:42:35-05:00" host="testdata" status="
<items>
<client>
<clientid>00001</clientid>
<name>
<![CDATA[ ABC Company ]]>
</name>
<site>
<siteid>222222</siteid>
<name>
<![CDATA[ 123 Blvd ]]>
</name>
<workstations/>
<servers>
<server>
<id>333333</id>
<name>
<![CDATA[ 123BLVD-SRV ]]>
</name>
<failed_checks>
<check>
<checkid>4444444</checkid>
<check_type>0001</check_type>
<description>
<![CDATA[Critical Events Check - Application log]]>
</description>
<dsc_247>2</dsc_247>
<date>2015-08-26</date>
<time>06:03:44</time>
<consecutive_fails>2</consecutive_fails>
<startdate>2015-08-25</startdate>
<starttime>10:43:51</starttime>
<formatted_output>
<![CDATA[Event log issues[CLIENT:]]>
</formatted_output>
<checkstatus>
<![CDATA[ Status ]]>
</checkstatus>
</check>
</failed_checks>
</server>
</servers>
</site>
</client>
What I'm trying to extract is an array of clients. Each client will have a name, a clientid, an array of its workstations (and their properties), and an array of its servers (and their properties). Something like this:
clients_array = [
{
:name => 'ABC Company',
:clientid => '00001',
:workstations => [
{
:name => 'hostname',
:id => '00002',
:failed_checks => [
{
:description => 'description', :cause => 'cause'
}
]
},
{
:name => 'hostname2',
:id => '00003',
...
}
]
},
{
:name => 'Second Company',
:clientid => '...',
...
}
]
The problem I'm running into is I can extract the client node's information easily enough, but extracting the workstation and server information for each client node is difficult.
Side note: I would just use DOM parsing, which I've done in the past with great success, but the XML I'm working with is far too large and has crashed the server.
Here's what I've been working with so far. I keep getting stuck at the site/workstations/servers nodes because sometimes there will be one site (hash element) and sometimes there are multiple sites (array element). The same goes for workstations and servers.
Since this is sax parsing, I don't understand how I can point the workstations and servers back to each client. I don't need the site data, just the workstations and servers for each client:
require 'saxerator'
def parse_sax
clients_array = []
parser = Saxerator.parser(File.new("data.xml"))
parser.for_tag(:client).each do |client|
# Create a hash to store 'this' client's data in
client_hash = Hash.new
# Grab some data
client_hash[:name] = client['name']
client_hash[:clientid] = client['clientid']
# Here's where the workstation/server code would go
parser.for_tag(:site).each do |site|
# This just goes through and finds ALL sites
end
clients_array << client_hash
end
I thought I had figured it out when I thought about parsing clients, workstations, and servers separately:
parser.for_tag(:client).each do |client|
...
end
parser.for_tag(:workstation).each do |ws|
...
end
parser.for_tag(:server).each do |srv|
...
end
But then I end up with a bunch of separate client, workstation, and server objects with no way of relating the devices back to their respective clients.
It's very possible my grasp of sax parsing is such that I'm just missing something trivial that will accomplish what I want, but I can't seem to discover the solution.
I'm more than happy to provide clarification where needed and any help is more than appreciated.

Answer 1

Use XMLTextReader for huge xml files. Use code like this
namespace ConsoleApplication1
{
class Program
{
const string FILENAME = #"c:\temp\test.txt";
static void Main(string[] args)
{
List<Client> clients = new List<Client>();
XmlTextReader reader = new XmlTextReader(FILENAME);
while (!reader.EOF)
{
if (reader.Name == "client")
{
string xmlClient = reader.ReadOuterXml();
XElement xClient = XElement.Parse(xmlClient);
Client newClient = new Client();
clients.Add(newClient);
newClient.name = xClient.Element("name").Value;
newClient.clientid = xClient.Element("clientid").Value;
newClient.workstations = xClient.Descendants("server").Select(x => new WorkStation
{
name = x.Element("name").Value,
id = x.Element("id").Value
}).ToList();
}
else
{
reader.ReadToFollowing("client");
}
}
}
}
public class Client
{
public string name { get; set;}
public string clientid { get; set; }
public List<WorkStation> workstations { get; set; }
}
public class WorkStation
{
public string name { get; set; }
public string id { get; set; }
}
}
​