Okay so I have a hash with a keys where some contain multiple values per key. I am trying to create new files with key being the filename and values being written to the text while (one value per line). Here is what I got.
#agencyList.each do |domain, email|
File.open(domain.to_s, "w") { |file| file.write(email) }
end
The issue is only the first element of value set is being outputted to the file. Any ideas?
As I understood correctly #agencyList is an array of Hashes. For example:
#agencyList = [
{domain: 'domain1', email: 'email11'},
{domain: 'domain1', email: 'email12'},
{domain: 'domain2', email: 'email21'},
]
So in this case File.open(domain.to_s, "w") has incorrect file mode. w will recreate a new file so this file will contain always only one value - the last one.
Try to open files with a mode and write lines via puts to ensure that values will be on separate lines:
#agencyList.each do |hash|
File.open(hash[:domain].to_s, "a") { |file| file.puts(hash[:email]) }
end
OR
But if you are saying
I have a hash with a keys where some contain multiple values per key
A Hash cannot contain values with the same key. So your #agencyList is a Hash and should have values as an array:
#agencyList = {
'key1' => ['val11', 'val12'],
'key2' => ['val21'],
]
If so your code should be something like this:
#agencyList.each do |domain, emails|
File.open(domain.to_s, "w") do |file|
emails.each do |email|
file.puts(email)
end
end
end
You'd need to iterate over the emails set
domains.each { |domain, emails|
File.open(domain, 'w'){ |f|
emails.each { |email|
f.puts(email)
}
}
}
Related
I'm trying to access the csv data, which I recive if I make a http-request.
I don't save it to a csv file, so I save it to the variable.
Let's say this is the response I get, how can I print food?
uuid,event_id,category
12,1,food
13,2,cars
And this is the part of the ruby code which is important.
That's something I found, but it was originally used with a file, so it doesn't work.
csvdata = request(action,parameter)
#data_hash = {}
CSV.foreach(csvdata) do |row|
uuid, event_id, category = row
#data_hash[uuid] = event_id
end
Do I really need files for that or is there a easy way I can access the values?
Update
CSV.parse(csvdata,data = Hash.new) do |row|
puts data
end
The hash should look like this so I can use the column names
{"uuid" => "12,13", "event_id" => "323,3243", "category" => "food,cars"}
csv_data = Hash.new{|k, v| k[v] = []}
CSV.parse(csv_string, headers: true) do |row|
row.each{|k, v| csv_data[k] << v}
end
csv_data = Hash[csv_data.map{|k, v| [k, v.join(",")]}]
Update after specification Requested output.
Try this:
csvdata = request(action,parameter)
#data_hash = {}
CSV.parse(csvdata, headers: true) do |row|
#data_hash[row['uuid']] = row['event_id']
end
#data_hash
# => {"12"=>"1", "13"=>"2"}
When you parse a CSV, the seconds parameter (data = Hash.new in your code) is actually an options parameter. You can see the available options here:
:headers
If set to :first_row or true, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of ::parse_line with the same :col_sep, :row_sep, and :quote_char as this instance to produce an Array of headers. This setting causes #shift to return rows as CSV::Row objects instead of Arrays and #read to return CSV::Table objects instead of an Array of Arrays.
When passing headers: true - values are parsed into a Row object, where they can be accessed by name.
Given the following CSV file, how would you remove all rows that contain the word 'true' in the column 'foo'?
Date,foo,bar
2014/10/31,true,derp
2014/10/31,false,derp
I have a working solution, however it requires making a secondary CSV object csv_no_foo
#csv = CSV.read(#csvfile, headers: true) #http://bit.ly/1mSlqfA
#headers = CSV.open(#csvfile,'r', :headers => true).read.headers
# Make a new CSV
#csv_no_foo = CSV.new(#headers)
#csv.each do |row|
# puts row[5]
if row[#headersHash['foo']] == 'false'
#csv_no_foo.add_row(row)
else
puts "not pushing row #{row}"
end
end
Ideally, I would just remove the offending row from the CSV like so:
...
if row[#headersHash['foo']] == 'false'
#csv.delete(true) #Doesn't work
...
Looking at the ruby documentation, it looks like the row class has a delete_if function. I'm confused on the syntax that that function requires. Is there a way to remove the row without making a new csv object?
http://ruby-doc.org/stdlib-1.9.2/libdoc/csv/rdoc/CSV/Row.html#method-i-each
You should be able to use CSV::Table#delete_if, but you need to use CSV::table instead of CSV::read, because the former will give you a CSV::Table object, whereas the latter results in an Array of Arrays. Be aware that this setting will also convert the headers to symbols.
table = CSV.table(#csvfile)
table.delete_if do |row|
row[:foo] == 'true'
end
File.open(#csvfile, 'w') do |f|
f.write(table.to_csv)
end
You might want to filter rows in a ruby manner:
require 'csv'
csv = CSV.parse(File.read(#csvfile), {
:col_sep => ",",
:headers => true
}
).collect { |item| item[:foo] != 'true' }
Hope it help.
I have the following json...
{
"NumPages":"17",
"Page":"1",
"PageSize":"50",
"Total":"808",
"Start":"1",
"End":"50",
"FirstPageUri":"/v3/results?PAGE=1",
"LastPageUri":"/v3/results?PAGE=17",
"PreviousPageUri":"",
"NextPageUri":"/v3/results?PAGE=2",
"User":[
{
"RowNumber":"1",
"UserId":"86938",
"InternalId":"",
"CompletionPercentage":"100",
"DateTimeTaken":"2014-06-18T01:43:25Z",
"DateTimeLastUpdated":"2014-06-18T01:58:11Z",
"DateTimeCompleted":"2014-06-18T01:58:11Z",
"Account":{
"Id":"655",
"Name":"Technical Community College"
},
"FirstName":"Matthew",
"LastName":"Knice",
"EmailAddress":"knice#gmail.com",
"AssessmentResults":[
{
"Title":"Life Factors",
"Code":"LifeFactors",
"IsComplete":"1",
"AttemptNumber":"1",
"Percent":"58",
"Readiness":"fail",
"DateTimeCompleted":"2014-06-18T01:46:00Z"
},
{
"Title":"Learning Styles",
"Code":"LearnStyles",
"IsComplete":"0"
},
{
"Title":"Personal Attributes",
"Code":"PersonalAttributes",
"IsComplete":"1",
"AttemptNumber":"1",
"Percent":"52.08",
"Readiness":"fail",
"DateTimeCompleted":"2014-06-18T01:49:00Z"
},
{
"Title":"Technical Competency",
"Code":"TechComp",
"IsComplete":"1",
"AttemptNumber":"1",
"Percent":"100",
"Readiness":"pass",
"DateTimeCompleted":"2014-06-18T01:51:00Z"
},
{
"Title":"Technical Knowledge",
"Code":"TechKnowledge",
"IsComplete":"1",
"AttemptNumber":"1",
"Percent":"73.44",
"Readiness":"question",
"DateTimeCompleted":"2014-06-18T01:58:00Z"
},
{
"Title":"Reading Rate & Recall",
"Code":"Reading",
"IsComplete":"0"
},
{
"Title":"Typing Speed & Accuracy",
"Code":"Typing",
"IsComplete":"0"
}
]
},
{
"RowNumber":"2",
"UserId":"8654723",
"InternalId":"",
"CompletionPercentage":"100",
"DateTimeTaken":"2014-06-13T14:37:59Z",
"DateTimeLastUpdated":"2014-06-13T15:00:12Z",
"DateTimeCompleted":"2014-06-13T15:00:12Z",
"Account":{
"Id":"655",
"Name":"Technical Community College"
},
"FirstName":"Virginia",
"LastName":"Bustas",
"EmailAddress":"bigBusta#students.college.edu",
"AssessmentResults":[
{
...
I need to start processing where you see "User:" The stuff at the beginning (numpages, page, ect) I want to ignore. Here is the processing script I am working on...
require 'csv'
require 'json'
CSV.open("your_csv.csv", "w") do |csv| #open new file for write
JSON.parse(File.open("sample.json").read).each do |hash| #open json to parse
csv << hash.values
end
end
Right now this fails with the error:
convert.rb:6:in `block (2 levels) in <main>': undefined method `values' for ["NumPages", "17"]:Array (NoMethodError)
I have ran the json through a parser, and it seems to be valid. What is the best way to only process the "User" data?
You have to look at the structure of the JSON object being created. Here's a very small subset of your document being parsed, which makes it easier to see and understand:
require 'json'
foo = '{"NumPages":17,"User":[{"UserId":12345}]}'
bar = JSON[foo]
# => {"NumPages"=>17, "User"=>[{"UserId"=>12345}]}
bar['User'].first['UserId'] # => 12345
foo contains the JSON for a hash. bar contains the Ruby object created by the JSON parser after it reads foo.
User is the key pointing to an array of hashes. Because it's an array, you have to specify which of the hashes in the array you want to look at, which is what bar['User'].first does.
An alternate way to access that sub-hash is:
bar['User'][0]['UserId'] # => 12345
If there were multiple hashes inside the array, you could access them by using the appropriate index value. For example, if there are two hashes, and I want the second one:
foo = '{"NumPages":17,"User":[{"UserId":12345},{"UserId":12346}]}'
bar = JSON[foo]
# => {"NumPages"=>17, "User"=>[{"UserId"=>12345}, {"UserId"=>12346}]}
bar['User'].first['UserId'] # => 12345
bar['User'][0]['UserId'] # => 12345
bar['User'][1]['UserId'] # => 12346
I'm wondering if I am going down the wrong road with the JSON.parse(File.open("sample.json").read).each do |hash|?
Yes, you are. You need to understand what you're doing, and break your code into digestible pieces so they make sense to you. Consider this:
require 'csv'
require 'json'
json_object = JSON.parse(File.read("sample.json"))
CSV.open("your_csv.csv", "w") do |csv| #open new file for write
csv << %w[RowNumber UserID AccountID AccountName FirstName LastName EmailAddress]
json_object['User'].each do |user_hash|
puts 'RowNumber: %s' % user_hash['RowNumber']
puts 'UserID: %s' % user_hash['UserID']
account = user_hash['UserID']['Account']
puts 'Account->Id: %s' % account['Id']
puts 'Account->Name: %s' % account['Name']
puts 'FirstName: %s' % user_hash['FirstName']
puts 'LastName: %s' % user_hash['LastName']
puts 'EmailAddress: %s' % user_hash['EmailAddress']
csv << [
user_hash['RowNumber'],
user_hash['UserID'],
account['Id'],
account['Name'],
user_hash['FirstName'],
user_hash['LastName'],
user_hash['EmailAddress']
]
end
end
This reads the JSON file and parses it into a Ruby object immediately. There is no special magic or anything else that happens with the file, it's opened, read, closed, and its content is passed to the JSON parser and assigned to json_object.
Once parsed, the CSV file is opened and a header row is written. It could have been written as part of the open statement but this is clearer for explaining what's going on.
json_object is a hash, so to access the 'User' data you have to use a normal hash access json_object['User']. The value for the User key is an array of hashes, so those need to be iterated over, which is what json_object['User'].each does, passing the hash elements of that array into the block as user_hash.
Inside that block it's pretty much the same thing as access the value for 'User', each "element" is a key/value pair, except 'Account' which is an embedded hash.
Read the error message. each called on a hash is giving you a sequence of arrays with two members (the key and value together). There is no values method on an array. And in any case if what you have is a hash there seems little point cycling through it with each; if you want the "User" entry in the hash, why don't you ask for it up front?
Just for posterity and context this is the script I ended up using in its entity. I needed to pull from a url, and process the results and move them to a simple CSV. I needed to wite the student id, first name, last name, and the score from each of 4 assessments to the csv.
require 'csv'
require 'json'
require 'curb'
c = Curl::Easy.new('myURL/m/v3/results')
c.http_auth_types = :basic
c.username = 'myusername'
c.password = 'mypassword'
c.perform
json_object = JSON.parse(c.body_str)
CSV.open("your_csv.csv", "w") do |csv| #open new file for write
csv << %w[UserID FirstName LastName LifeFactors PersonalAttributes TechComp TechKnowledge]
json_object['User'].each do |user_hash|
csv << [
user_hash['UserId'],
user_hash['FirstName'],
user_hash['LastName'],
user_hash['AssessmentResults'][0]['Percent'],
user_hash['AssessmentResults'][2]['Percent'],
user_hash['AssessmentResults'][3]['Percent'],
user_hash['AssessmentResults'][4]['Percent']
]
end
end
I'm taking a file and reading in it's contents and creating a hash based on newlines. I've been able to make a hash based on the contents of each line, but how can I create a hash based on the content of everything before the next blank newline? Below is what I have so far.
Input:
Title 49th parallel
URL http://artsweb.bham.ac.uk/
Domain artsweb.bham.ac.uk
Title ABAA booknet
URL http://abaa.org/
Domain abaa.org
Code:
File.readlines('A.cfg').each do |line|
unless line.strip.empty?
hash = Hash[*line.strip.split("\t")]
puts hash
end
puts "\n" if line.strip.empty?
end
Outputs:
{"Title"=>"49th parallel"}
{"URL"=>"http://artsweb.bham.ac.uk/"}
{"Domain"=>"artsweb.bham.ac.uk"}
{"Title"=>"ABAA booknet"}
{"URL"=>"http://abaa.org/"}
{"Domain"=>"abaa.org"}
Desired Output:
{"Title"=>"49th parallel", "URL"=>"http://artsweb.bham.ac.uk/", "Domain"=>"artsweb.bham.ac.uk"}
{"Title"=>"ABAA booknet", "URL"=>"http://abaa.org/", "Domain"=>"abaa.org"}
Modifying your existing code, this does what you want:
hash = {}
File.readlines('A.cfg').each do |line|
if line.strip.empty?
puts hash if not hash.empty?
hash = {}
puts "\n"
else
hash.merge!(Hash[*line.strip.split("\t")])
end
end
puts hash
You can likely simplify that depending on what you're actually doing with the data.
open('A.cfg', &:read)
.strip.split(/#$/{2,}/)
.map{|s| Hash[s.scan(/^(\S+)\s+(\S+)/)]}
gives
[
{
"Title" => "49th",
"URL" => "http://artsweb.bham.ac.uk/",
"Domain" => "artsweb.bham.ac.uk"
},
{
"Title" => "ABAA",
"URL" => "http://abaa.org/",
"Domain" => "abaa.org"
}
]
read the whole content of the file using read:
contents = ""
File.open('A.cfg').do |file|
contents = file.read
end
And then split the contents on two newline characters:
contents.split("\n\n")
And lastly, create a function pretty similar to what you already have to parse those chunks.
Please note that if you are working on windows it may happen that you need to split on a different sequence because of the carriage return character.
I have a string with extra whitespace:
First,Last,Email ,Mobile Phone ,Company,Title ,Street,City,State,Zip,Country, Birthday,Gender ,Contact Type
I want to parse this line and remove the whitespaces.
My code looks like:
namespace :db do
task :populate_contacts_csv => :environment do
require 'csv'
csv_text = File.read('file_upload_example.csv')
csv = CSV.parse(csv_text, :headers => true)
csv.each do |row|
puts "First Name: #{row['First']} \nLast Name: #{row['Last']} \nEmail: #{row['Email']}"
end
end
end
#prices = CSV.parse(IO.read('prices.csv'), :headers=>true,
:header_converters=> lambda {|f| f.strip},
:converters=> lambda {|f| f ? f.strip : nil})
The nil test is added to the row but not header converters assuming that the headers are never nil, while the data might be, and nil doesn't have a strip method. I'm really surprised that, AFAIK, :strip is not a pre-defined converter!
You can strip your hash first:
csv.each do |unstriped_row|
row = {}
unstriped_row.each{|k, v| row[k.strip] = v.strip}
puts "First Name: #{row['First']} \nLast Name: #{row['Last']} \nEmail: #{row['Email']}"
end
Edited to strip hash keys too
CSV supports "converters" for the headers and fields, which let you get inside the data before it's passed to your each loop.
Writing a sample CSV file:
csv = "First,Last,Email ,Mobile Phone ,Company,Title ,Street,City,State,Zip,Country, Birthday,Gender ,Contact Type
first,last,email ,mobile phone ,company,title ,street,city,state,zip,country, birthday,gender ,contact type
"
File.write('file_upload_example.csv', csv)
Here's how I'd do it:
require 'csv'
csv = CSV.open('file_upload_example.csv', :headers => true)
[:convert, :header_convert].each { |c| csv.send(c) { |f| f.strip } }
csv.each do |row|
puts "First Name: #{row['First']} \nLast Name: #{row['Last']} \nEmail: #{row['Email']}"
end
Which outputs:
First Name: 'first'
Last Name: 'last'
Email: 'email'
The converters simply strip leading and trailing whitespace from each header and each field as they're read from the file.
Also, as a programming design choice, don't read your file into memory using:
csv_text = File.read('file_upload_example.csv')
Then parse it:
csv = CSV.parse(csv_text, :headers => true)
Then loop over it:
csv.each do |row|
Ruby's IO system supports "enumerating" over a file, line by line. Once my code does CSV.open the file is readable and the each reads each line. The entire file doesn't need to be in memory at once, which isn't scalable (though on new machines it's becoming a lot more reasonable), and, if you test, you'll find that reading a file using each is extremely fast, probably equally fast as reading it, parsing it then iterating over the parsed file.