importing simplified XML file into mongodb using crack gem - ruby

Question

Here, I'm using mongodb driver for ruby. But after this will work perfect I want to run it as a scheduled task in Ruby on Rails 3 with Mongoid ODB.
So for now, I'm experimenting in ruby.
I've noticed crack gem is very convenient when it comes to convert XML file into the format that can be inserted into mongodb. When I use mongodb driver for ruby, crack converts to the format close to JSON (it's using "=>" instead of ":" columns), which is required condition before I will insert it into mondodb database as shown here.
The problem the way I'm using crack below it imports everything that is in XML file.
Please see below.
sample.xml
<?xml version="1.0" encoding="utf-8"?>
<ShipmentRequest>
<Envelope>
<TransmissionDateTime>05/08/2013 23:06:02</TransmissionDateTime>
</Envelope>
<Message>
<Comment />
<Header>
<MemberId>A00000001</MemberId>
<MemberName>Bruce</MemberName>
<DeliveryId>6377935</DeliveryId>
<ShipToAddress1>123-4567</ShipToAddress1>
<OrderDate>05/08/13</OrderDate>
<Payments>
<PayType>Credit Card</PayType>
<Amount>1000</Amount>
</Payments>
<Payments>
<PayType>Points</PayType>
<Amount>5390</Amount>
</Payments>
</Header>
<Line>
<LineNumber>3.1</LineNumber>
<Item>fruit-004</Item>
<Description>Peach</Description>
<Quantity>1</Quantity>
<UnitCost>1610</UnitCost>
<DeclaredValue>0</DeclaredValue>
<PointValue>13</PointValue>
</Line>
<Line>
<LineNumber>8.1</LineNumber>
<Item>fruit-001</Item>
<Description>Fruit Set</Description>
<Quantity>1</Quantity>
<UnitCost>23550</UnitCost>
<PointValue>105</PointValue>
<PickLine>
<PickLineNumber>8.1..1</PickLineNumber>
<PickItem>fruit-002</PickItem>
<PickDescription>Apple</PickDescription>
<PickQuantity>1</PickQuantity>
</PickLine>
<PickLine>
<PickLineNumber>8.1..2</PickLineNumber>
<PickItem>fruit-003</PickItem>
<PickDescription>Orange</PickDescription>
<PickQuantity>2</PickQuantity>
</PickLine>
</Line>
</Message>
</ShipmentRequest>
sample_crack.rb
#!/usr/bin/ruby
require "crack"
require 'mongo'
include Mongo
mongo_client = MongoClient.new("localhost", 27017)
db = mongo_client.db("somedb")
coll = db.collection("somecoll")
myXML = Crack::XML.parse(File.read("sample.xml"))
coll.insert(myXML)
puts myXML
It prints on console:
{"ShipmentRequest"=>{"Envelope"=>{"TransmissionDateTime"=>"05/08/2013 23:06:02"}, "Message"=>{"Comment"=>nil, "Header"=>{"MemberId"=>"A00000001", "MemberName"=>"Bruce", "DeliveryId"=>"6377935", "ShipToAddress1"=>"123-4567", "OrderDate"=>"05/08/13", "Payments"=>[{"PayType"=>"Credit Card", "Amount"=>"1000"}, {"PayType"=>"Points", "Amount"=>"5390"}]}, "Line"=>[{"LineNumber"=>"3.1", "Item"=>"fruit-004", "Description"=>"Peach", "Quantity"=>"1", "UnitCost"=>"1610", "DeclaredValue"=>"0", "PointValue"=>"13"}, {"LineNumber"=>"8.1", "Item"=>"fruit-001", "Description"=>"Fruit Set", "Quantity"=>"1", "UnitCost"=>"23550", "PointValue"=>"105", "PickLine"=>[{"PickLineNumber"=>"8.1..1", "PickItem"=>"fruit-002", "PickDescription"=>"Apple", "PickQuantity"=>"1"}, {"PickLineNumber"=>"8.1..2", "PickItem"=>"fruit-003", "PickDescription"=>"Orange", "PickQuantity"=>"2"}]}]}}, :_id=>BSON::ObjectId('51ad8d83a3d24b3b9f000001')}
In the mongodb, the converted XML file looks like:
{
"_id" : ObjectId("51ad8d83a3d24b3b9f000001"),
"ShipmentRequest" : {
"Envelope" : {
"TransmissionDateTime" : "05/08/2013 23:06:02"
},
"Message" : {
"Comment" : null,
"Header" : {
"MemberId" : "A00000001",
"MemberName" : "Bruce",
"DeliveryId" : "6377935",
"ShipToAddress1" : "123-4567",
"OrderDate" : "05/08/13",
"Payments" : [
{
"PayType" : "Credit Card",
"Amount" : "1000"
},
{
"PayType" : "Points",
"Amount" : "5390"
}
]
},
"Line" : [
{
"LineNumber" : "3.1",
"Item" : "fruit-004",
"Description" : "Peach",
"Quantity" : "1",
"UnitCost" : "1610",
"DeclaredValue" : "0",
"PointValue" : "13"
},
{
"LineNumber" : "8.1",
"Item" : "fruit-001",
"Description" : "Fruit Set",
"Quantity" : "1",
"UnitCost" : "23550",
"PointValue" : "105",
"PickLine" : [
{
"PickLineNumber" : "8.1..1",
"PickItem" : "fruit-002",
"PickDescription" : "Apple",
"PickQuantity" : "1"
},
{
"PickLineNumber" : "8.1..2",
"PickItem" : "fruit-003",
"PickDescription" : "Orange",
"PickQuantity" : "2"
}
]
}
]
}
}
}
But I'd like to import it like to eliminate not-needed nodes and ignore empty ones:
{
"_id" : ObjectId("51ad8d83a3d24b3b9f000001"),
"MemberId" : "A00000001",
"MemberName" : "Bruce",
"DeliveryId" : "6377935",
"ShipToAddress1" : "123-4567",
"OrderDate" : "05/08/13",
"Payments" : [
{
"PayType" : "Credit Card",
"Amount" : "1000"
},
{
"PayType" : "Points",
"Amount" : "5390"
}
],
"Line" : [
{
"LineNumber" : "3.1",
"Item" : "fruit-004",
"Description" : "Peach",
"Quantity" : "1",
"UnitCost" : "1610",
"DeclaredValue" : "0",
"PointValue" : "13"
},
{
"LineNumber" : "8.1",
"Item" : "fruit-001",
"Description" : "Fruit Set",
"Quantity" : "1",
"UnitCost" : "23550",
"PointValue" : "105",
"PickLine" : [
{
"PickLineNumber" : "8.1..1",
"PickItem" : "fruit-002",
"PickDescription" : "Apple",
"PickQuantity" : "1"
},
{
"PickLineNumber" : "8.1..2",
"PickItem" : "fruit-003",
"PickDescription" : "Orange",
"PickQuantity" : "2"
}
]
}
]
}
Can this be done with crack? Or this can be better done with nokogiri?
Update
Big thanks to #Alex Peachey, here I put the updated code.
sample_crack/rb (updated):
#!/usr/bin/ruby
require "crack"
require 'mongo'
include Mongo
mongo_client = MongoClient.new("localhost", 27017)
db = mongo_client.db("somedb")
coll = db.collection("somecoll")
myXML = Crack::XML.parse(File.read("sample.xml"))
myXML.merge!(myXML.delete("ShipmentRequest")) # not needed hash
myXML.merge!(myXML.delete("Message")) # not needed hash
myXML.merge!(myXML.delete("Header")) # not needed hash
myXML.delete("Envelope") # not needed hash
# planning to put here a code to remove hashes with empty values
coll.insert(myXML)
puts myXML

Answer 1

It's hard to say how you define "not-needed" nodes but empty ones are easy enough to understand. Either way though, Crack is very good at what's it's doing for you which is basically turning the XML into a Hash. Once you have the Hash just prune it as you wish based on whatever rules you have before you insert it into Mongo.
Based on your comment, I better understand what you are asking. My answer still holds true, just manipulate the hash. Specifically you could do this:
myXML.merge!(myXML.delete("ShipmentRequest"))
myXML.delete("Envelope")
myXML.merge!(myXML.delete("Message"))