How to Export a Ruby Array into a single csv row? - ruby
I'm looking to pass an array into a csv as a single row. The code below showcases what I'm currently trying. The array is variable in length and will showcase something like this.
a = ["website url", "page link", "page link", "page link"]
The code essentially goes to a page using open-uri and scrapes out the links of each page. This is used for internal validation and checked against what we are expecting. If the link matches what we expect it is reported into the csv as a page link variable shown above.
I've currently tried using to_csv to format the data but inside the csv everything is in a single row including my comma separations.
def dwrite (array)
CSV.open("filename.csv", "ab") do |csv|
data = array.to_csv(row_sep: nil)
csv << [data]
end
end
This is an example of what the csv looks like in a single column:
https://www.url.com/example-page.html
#,
#,,
#,,,
#,,,,
#,,,,,
#,,,,,,
#,,,,,,,
#,,,,,,,,
#,,,,,,,,,
#,,,,,,,,,,
#,,,,,,,,,,,
#,,,,,,,,,,,,
#,,,,,,,,,,,,,
#,,,,,,,,,,,,,,
#,,,,,,,,,,,,,,,
https://anotherexamplepage.html
I was hoping that each array element passed into the method would showcase on the same line but this doesn't appear to be the case. Any help would be much appreciated.
You don't need to transform the array in any way. Just add it to the CSV:
def dwrite(array)
CSV.open("filename.csv", "ab") do |csv|
csv << array
end
end
Related
Parsing through excel using ruby gem "creek"
Hey guys so I am trying to parse through an excel file through the ruby gem "creek", it parses the the rows accurately but I want to just retrieve the Columns, such as only the data in the "A" cloumn. Outputs the whole excel documents correctly.
require 'creek'
creek = Creek::Book.new 'Final.xlsx'
sheet= creek.sheets[0]
sheet.rows.each do |row|
puts row # => {"A1"=>"Content 1", "B1"=>nil, C1"=>nil, "D1"=>"Content 3"}
end
Any suggestions will be much appreciated.
Creek doesn't make it easy to extract column information because it stores the column and row smashed together in a string hash key.
The more popular Roo allows you to do things like sheet.column(1) and get an entire column. Very simple.
If you absolutely must have creek, I noticed that there is an add-on to Creek called Ditch which adds some column-fetching capability. Example:
sheet.rows.each { |r|
puts "#{r.index} #{r.get('A')} - #{r.get('B')}"
}
Finally, if you want to do it with Creek and no add-ons, use Hash#select:
sheet.rows.each do |row|
puts row.select{ |k,v| ["A", "B"].include? k[0]}
end
To read the individual columns you can use Creek :: Sheet # simple_rows method
For example, to read the first and third columns:
require 'creek'
creek = Creek::Book.new 'Final.xlsx'
sheet_first = creek.sheets.first
# read first column A
col_first = sheet_first.simple_rows.map{|col| col['A']} #=> Array containing the first column
# read third column C
col_third = sheet_first.simple_rows.map{|col| col['C']} #=> Array containing the third column
Ruby equivalent to Python's DictWriter?
I have a Ruby script that goes through a CSV, determines some information, and then puts out a resulting CSV file. In Python, I'm able to open both my source file and my results file with DictReader and DictWriter respectively and write rows as dictionaries, where keys are the file header values. It doesn't appear that there is a manageable way to do this in Ruby, but I'm hoping somebody can point me to a better solution than storing all of my result hashes in an array and writing them after the fact.
The standard library "CSV" gives rows hash-like behavior when headers are enabled.
require 'csv'
CSV.open("file.csv", "wb") do |csv_out|
CSV.foreach("test.csv", headers: true) do |row|
row["header2"].upcase! # hashlike behaviour
row["new_header"] = 12 # add a new column
csv_out << row
end
end
(test.csv has a header1, a header2 and some random comma separated string lines.)
Ruby - Matching a value and returning a corresponding value from a separate file
I'm looking to match one value (A sku from a website) to the same value (A sku from lookup.csv) and return a corresponding model (From lookup.csv).
Here's sample data from lookup.csv:
SKU , Model
2520045 , DQ.SUNAA.002
7423599 , DA.MX00.1CC
9547543 , DX.MF01.2BM
Here's my code thus far:
url = "http://www.bestbuy.com/site/acer-aspire-23-touch-screen-all-in-one-intel-core-i5-8gb-memory-2tb-hard-drive-black/2520045.p?id=1219547718151&skuId=2520045"
page = Nokogiri::HTML(open(url))
sku = page.css('span#sku-value').text
#model = match the sku to the sku in lookup.csv and return corresponding model
puts model
I know that I can open the file with
open("lookup.csv", 'r')
but past that, I'm not quite sure how to match/return a corresponding value.
Any help is appreciated!
The suggestion of Aoukar would work but be slow with large data sets.
Here a better solution, read the CSV once, using the CSV gem (no need to reinvent the wheel) and store the data in a hash, after that you can just ask fort the right Model, here a working sample.
I'm using the CSV data in the DATA part of the script here so I don't need the CSV file itself.
require "csv"
lookup = {}
CSV.parse(DATA, col_sep: " , ", headers: true, force_quotes: false, :quote_char => "\x00").each do |row|
lookup.merge! Hash[row['SKU'], row['Model']]
end
lookup #{"2520045"=>"DQ.SUNAA.002", "7423599"=>"DA.MX00.1CC", "9547543"=>"DX.MF01.2BM"}
lookup['2520045'] #"DQ.SUNAA.002"
__END__
_ ,SKU , Model #the first element is to work around a bug in CSV used this way
2520045 , DQ.SUNAA.002
7423599 , DA.MX00.1CC
9547543 , DX.MF01.2BM
you can try this code, but I didn't test it so it might need some modifications, I've written it as a function since it's what I'm used to.
def search(path,key) #path to file, and word to search for
File.open(path,'r') do |file| #open file
file.readlines.each { |line| #read lines array
if line.split(' , ')[0] == key #match the SKU
return line.split(' , ')[1] #return the Model
end
}
end
end
Using excel to log results in ruby watir. how to keep values in different cells using puts
I am new to ruby watir and need your help.
I am using the following commands to log my script results into an excel sheet.
File.open('c:\log.txt', 'w') do |file|
file.puts("TEST PASSED" + "#{Time.now}")
end
Here the test passed and the time is getting displayed in a single cell itself.
I want to display both of it in different cells.
Please suggest a solution.
Thanks in advance!
you are logging to a file called log.txt which appears to be a plain text file. if you want your file to be an excel file you will need a format, the easiest one to write to is either .csv or .tsv which stands for comma separated variable and tab separated variables. You could then write in a few different ways. You could write as you were with:
File.open('c:\log.tsv', 'w') do |file|
file.puts("TEST PASSED\t" + "#{Time.now}")
end
for a tsv (note that it doesn't need to be called .tsv)
File.open('c:\log.csv', 'w') do |file|
file.puts("TEST PASSED," + "#{Time.now}")
end
for a csv
or you could use the standard csv library. like so:
CSV.open("c:\log.csv", "wb") do |csv|
csv << ["TEST PASSED", "#{Time.now}"]
end
which you can manipulate for tsv's:
CSV.open("c:\log.csv", "wb", { :col_sep => "\t" }) do |csv|
csv << ["TEST PASSED", "#{Time.now}"]
end
The easiest solution would probably be to log the results in a CSV (comma separated values) file. These can be written like a text file, as well as read by Excel as a grid.
For a CSV file, each line represents a row in the table. Within each line, the column values (or cells) are separated by commas. For example, the following will create 2 rows with 3 columns:
a1, b1, c1
a2, b2, c2
For your logging, you could:
Create a log.csv instead of log.txt
Output the values as comma separated
The code would be:
File.open('c:\log.csv', 'w') do |file|
file.puts("TEST PASSED, #{Time.now}")
end
How not to save to csv when array is empty
I'm parsing through a website and i'm looking for potentially many million rows of content. However, csv/excel/ods doesn't allow for more than a million rows.
That is why I'm trying to use a provisionary to exclude saving empty content. However, it's not working: My code keeps creating empty rows in csv.
This is the code I have:
# create csv
CSV.open("neverending.csv", "w") do |csv|
csv << ["kuk","date","name"]
# loop through all urls
File.foreach("neverendingurls.txt") do |line|
begin
doorzoekbarefile = Nokogiri::HTML(open(line))
for k in 1..999 do
# PROVISIONARY / CONDITIONAL
unless doorzoekbarefile.at_xpath("//td[contains(style, '60px')])[#{k}]").nil?
# xpaths
kuk = doorzoekbarefile.at_xpath("(//td[contains(#style,'60px')])[#{k}]")
date = doorzoekbarefile.at_xpath("(//td[contains(#style, '60px')])[#{k}]/following-sibling::*[1]")
name = doorzoekbarefile.at_xpath("(//td[contains(#style, '60px')])[#{k}]/following-sibling::*[2]")
# save to csv
csv << [kuk,date,name]
end
end
end
rescue
puts "error bij url #{line}"
end
end
end
Anybody have a clue what's going wrong or how to solve the problem? Basically I simply need to change the code so that it doesn't create a new row of csv data when the xpaths are empty.
This really doesn't have to do with xpath. It's simple Array#empty? row = [kuk,date,name] csv << row if row.compact.empty? BTW, your code is a mess. Learn how to indent at least beore posting again.