I have a nested array in ruby 2.4.4 that I need to flatten and send to a CSV.
A sample row of the data structure:
{:field1=>242,
:field2=>1950,
:field3=>"text",
:field4=>
{"sublevel1"=>{"detail1"=>"text", "detail2"=>24, "detail3"=>nil},
"sublevel2"=>{"anotherdetail"=>"text}}
The closest I've gotten is below but it doesn't unnest the deeper levels.
filename = "/home/myfile.csv"
headers = list.values[0].keys
data = list.values.map(&:values)
csvoutput = CSV.generate do |csv|
csv << headers
data.each do |single_row|
csv << single_row
end
end
File.write(filename, csv_output)
Thanks for your help
Related
I've been trying to use Ruby to create a CSV file from json data. I was able to create the file, but I need to add a few headers. I tried following suggestions and answers from similar questions posted here on Stack Overflow, but I keep getting errors. Can anyone give me some pointers?
Here's my code.
require 'csv'
require 'json'
CSV.open("your_csv.csv", "w") do |csv|
JSON.parse(File.open("tojson.txt").read).each do |hash|
csv << hash.values
#csv.each { |line| line['New_header'] = line[0].to_i + line[1].to_i }
end
end
And here is the error I'm getting:
Anyone have any suggestions?
This is not how you add headers to a csv file. When you generate csv content, a header row is just a regular row. And should be generated as such. Example:
CSV.open("your_csv.csv", "w") do |csv|
csv << ['new_header', 'value1', 'value2'] # the headers
JSON.parse(File.open("tojson.txt").read).each do |hash|
row = [generate, values, for, headers, above]
csv << row
end
end
You don't have a #csv variable. You have a csv one.
I have a bunch of data that needs to be written into a CSV file. Currently I'm doing this:
CSV.open("file.csv" , 'w' ) do |writer|
readfromCSV.each do |x|
writer << x
end
end
I need to serialize all the data and write it into the CSV file.
I'm new to serialization, I read about Marshal.dump(x) which serializes the array, but it throws an error when I try to do writer<<x.
I read about the option dump(ary_of_objs, io = "", options = Hash.new), but could not understand how to implement it in this case.
http://www.ruby-doc.org/stdlib-2.0.0/libdoc/csv/rdoc/CSV.html explains the functions in the csv module.
say you want to make the first column string in the csv lowercase;
CSV.open("file.csv" , 'w' ) do |writer|
readfromCSV.each do |row|
# row is a csv row, pretty much an array
row[0] = row[0].downcase
writer << row
#alternatively, you could do it manually eg. writer << ["this", "rocks"]
end
end
I have the following Ruby code:
require 'octokit.rb'
require 'csv.rb'
CSV.foreach("actors.csv") do |row|
CSV.open("node_attributes.csv", "wb") do |csv|
csv << [Octokit.user "userid"]
end
end
I have a csv called actors.csv where every row has one entry - a string with a userid.
I want to go through all the rows, and for each row do Octokit.user "userid", and then store the output from each query on a separate row in a CSV - node_attributes.csv.
My code does not seem to do this? How can I modify it to make this work?
require 'csv'
DOC = 'actors.csv'
DOD = 'new_output.csv'
holder = CSV.read(DOC)
You can navigate it by calling
holder[0][0]
=> data in the array
holder[1][0]
=> moar data in array
make sense?
#make this a loop
profile = []
profile[0] = holder[0][0]
profile[1] = holder[1][0]
profile[2] = 'whatever it is you want to store in the new cell'
CSV.open(DOD, "a") do |data|
data << profile.map
end
#end the loop here
That last bit of code will print whatever you want into a new csv file
I wrote a very simple program with Nokogiri to scrape a website and create a CSV file. It is getting the correct data and making the CSV, but the data is pushed into one cell (A1), and I would rather it come out as a column, with each value separated by a comma.
How do I tell CSV to make a column for each value separated by a comma instead of putting all the info into a single cell?
require 'open-uri'
require 'nokogiri'
require 'csv'
doc = Nokogiri::HTML(open('somewebpage.com'))
CSV.open("webpagedata.csv", "wb") do |csv|
data = doc.css('.information h3 a').map { |link| link['href'] }
puts data
csv << [data]
end
The result from doc.css('.information h3 a').map { |link| link['href'] } is already an array, so when you add it to your CSV file you don’t need to wrap it in [...].
Change the line
csv << [data]
to
csv << data
The CSV library deals mainly in rows, so if you want to create a column rather than a row, then you need to add a (single entry) row for each entry of the column:
CSV.open("webpagedata.csv", "wb") do |csv|
data = doc.css('.information h3 a').map { |link| link['href'] }
data.each do |entry|
csv << [entry]
end
end
Note that in this case you do need the [...] around the entry, as you need to add an array not a single item.
Here's what I'm trying to accomplish. I need to have a single CSV with headers and several rows. I'm iterating through the headers and storing then and then associating the row data to the header. I need to be able to iterate through each of the rows in the CSV to use for constructing an XML's data. The constructed XML is then dumped as a .xml file and the program starts on the next row in the CSV. Each row has a column that provides the name of the XML file.
Here's what I've got so far.
Read in the data from the CSV file. Collect the header and row data.
def get_rows
raw_data = CSV.read('test.csv', {:skip_blanks => false, :headers => true})
data = []
raw_data.each { |row| data << row}
return build_header(data, raw_data)
end
take the header and row data and marry them up.
def build_header(data, raw_data)
(0..(data.length - 1)).each do |ri|
h = {}
raw_data.headers.each_with_index do |v, i|
h[v] = data[ri].fields[i]
end
return build_ostruct(h)
end
end
take the hash h and make an ostruct of it.
def build_ostruct(h)
x = OpenStruct.new(h)
uniq = x.tc_name
y = uniq_name.to_s + ".xml"
#marshal dump for debugging
x.marshal_dump.each{ |k,v| puts "#{k} => #{v}" }
return xml_builder(x, y)
end
Below this I'm taking the new ostruct "x" and calling the column headers from the CSV to #populate the XML nodes
For example: x.column1, x.column2, x.column3
Now the part I'm getting hung up on is getting the ostruct to receive the new row of data per iteration run. The objective is to have the ostruct populate with each row from the CSV. Currently the hash is displaying the proper data set and my XML is populating as expected but only with the first row of data. How do I get this to iterate through all the rows and populate the ostruct with the data per iteration so I can create a bulk set of XML's?
Thanks in advance for any and all help!
Something like this should work:
require 'csv'
require 'nokogiri'
CSV.foreach('test.csv', :headers => true) do |row|
builder = Nokogiri::XML::Builder.new do |xml|
xml.root do |root|
row.each do |k, v|
root.send k, v
end
end
end
File.open("#{row['tc_name']}.xml", 'w'){|f| f << builder.to_xml}
end
you are calling return in build_header, which ends the call. you need to collect your results in some way without immediately returning the first one, so that build_header can run for the entire set of rows.