Convert Hashes to CSV - ruby

Question

I have a CSV that I like to save all my hash values on it. I am using nokogiri sax to parse a xml document and then save it to a CSV.
The sax parser:
require 'rubygems'
require 'nokogiri'
require 'csv'
class MyDocument < Nokogiri::XML::SAX::Document
HEADERS = [ :titles, :identifier, :typeOfLevel, :typeOfResponsibleBody,
:type, :exact, :degree, :academic, :code, :text ]
def initialize
#infodata = {}
#infodata[:titles] = Array.new([])
end
def start_element(name, attrs)
#attrs = attrs
#content = ''
end
def end_element(name)
if name == 'title'
Hash[#attrs]["xml:lang"]
#infodata[:titles] << #content
#content = nil
end
if name == 'identifier'
#infodata[:identifier] = #content
#content = nil
end
if name == 'typeOfLevel'
#infodata[:typeOfLevel] = #content
#content = nil
end
if name == 'typeOfResponsibleBody'
#infodata[:typeOfResponsibleBody] = #content
#content = nil
end
if name == 'type'
#infodata[:type] = #content
#content = nil
end
if name == 'exact'
#infodata[:exact] = #content
#content = nil
end
if name == 'degree'
#infodata[:degree] = #content
#content = nil
end
if name == 'academic'
#infodata[:academic] = #content
#content = nil
end
if name == 'code'
Hash[#attrs]['source="vhs"']
#infodata[:code] = #content
#content = nil
end
if name == 'ct:text'
#infodata[:beskrivning] = #content
#content = nil
end
end
def characters(string)
#content << string if #content
end
def cdata_block(string)
characters(string)
end
def end_document
File.open("infodata.csv", "ab") do |f|
csv = CSV.generate_line(HEADERS.map {|h| #infodata[h] })
csv << "\n"
f.write(csv)
end
end
end
creating new an object for every file that is store in a folder(47.000xml files):
parser = Nokogiri::XML::SAX::Parser.new(MyDocument.new)
counter = 0
Dir.glob('/Users/macbookpro/Desktop/sax/info_xml/*.xml') do |item|
parser.parse(File.open(item, 'rb'))
counter += 1
puts "Writing file nr: #{counter}"
end
The issue:
I dont get a new line for every new set of values. Any ideas?
3 xml files for trying the code:
https://gist.github.com/2378898
https://gist.github.com/2378901
https://gist.github.com/2378904

Answer 1

You need to open the file using "a" mode (opening a file with "w" clears any previous content).
Appending an array to the csv object will automatically insert newlines. Hash#values returns an array of the values, but it would be safer to force the order. Flattening the array will potentially lead to misaligned columns (e.g. [[:title1, :title2], 'other-value'] will result in [:title1, :title2, 'other-value']). Try something like this:
HEADERS = [:titles, :identifier, ...]
def end_document
# with ruby 1.8.7
File.open("infodata.csv", "ab") do |f|
csv = CSV.generate_line(HEADERS.map { |h| #infodata[h] })
csv << "\n"
f.write(csv)
end
# with ruby 1.9.x
CSV.open("infodata.csv", "ab") do |csv|
csv << HEADERS.map { |h| #infodata[h] }
end
end
The above change can be verified by executing the following:
require "csv"
class CsvAppender
HEADERS = [ :titles, :identifier, :typeOfLevel, :typeOfResponsibleBody, :type,
:exact, :degree, :academic, :code, :text ]
def initialize
#infodata = { :titles => ["t1", "t2"], :identifier => 0 }
end
def end_document
#infodata[:identifier] += 1
# with ruby 1.8.7
File.open("infodata.csv", "ab") do |f|
csv = CSV.generate_line(HEADERS.map { |h| #infodata[h] })
csv << "\n"
f.write(csv)
end
# with ruby 1.9.x
#CSV.open("infodata.csv", "ab") do |csv|
# csv << HEADERS.map { |h| #infodata[h] }
#end
end
end
appender = CsvAppender.new
3.times do
appender.end_document
end
File.read("infodata.csv").split("\n").each do |line|
puts line
end
After running the above the infodata.csv file will contain:
"[""t1"", ""t2""]",1,,,,,,,,
"[""t1"", ""t2""]",2,,,,,,,,
"[""t1"", ""t2""]",3,,,,,,,,

Answer 2

I guess you need an extra loop. Something similar to
CSV.open("infodata.csv", "wb") do |csv|
csv << #infodata.keys
#infodata.each do |key, value|
csv << value
end
end