Bizarre field switch between cli and file output in Ruby - ruby

I'm having such an strange issue with an ruby script which i'm working with... in this script i parse an iTunes Library xml file and form objects for Artists, Albums and Tracks. In my Album class, i have two numeric field, YEAR and TRACK_COUNT.
My script parses correctly the two fields, let's say, for example, the output of it:
#<Album:0x007f59b1472a18 #compilation=false, #title="Straight Out Of Hell", #year=2013, #track_count=13, #trackList=[], #coverList=[]>
when i output this same object to file, it get crippled, transforming to this, here in json format:
{"compilation":false,"title":"Straight Out Of Hell","year":13,"track_count":13,"trackList":[],"coverList":[]}]
as you can see, the field YEAR get overwritten with the value in TRACK_COUNT field... i'm getting crazy with this, as i don't do any change to this field between these outputs!
UPDATE
As asked by #Amadan...
http://pastebin.com/1FUuvaCr Biblioteca.xml (EXCERPT)
http://pastebin.com/F8wgu6bz Track.rb
http://pastebin.com/3qhd4TRU Song.rb
http://pastebin.com/RNf5S7AZ dependencies.rb
http://pastebin.com/haXPpJgN Cover.rb
http://pastebin.com/1JYtT1nn Artist.rb
http://pastebin.com/qsgLsAJa Album.rb
http://pastebin.com/eiUAMfwR app.rb (MAIN SCRIPT)

This is happening because your source file is not as clean as you believe it to be. In some albums in the source XML, "Track Count" and "Year" are appearing on the same line, without a recognized line break between them. So you might have a line like this:
<key>Track Count</key><integer>12</integer><key>Year</key><integer>2006</integer>
When your if-else-if ladder asks if "track count" appears in the line, it does, so you're grabbing the first <integer>something</integer> match on the line. This works fine. But when you try to extract the year out of this line, you're again asking for the first <integer> on the line, which is the Track Count.
The bigger problem is that you're attempting to parse an XML file line-by-line, and that's not how they're meant to be read. Install the nokogiri gem and call this:
data = Nokogiri::XML('Biblioteca.xml')
Now you can get to any information contained in the document. The official tutorials on user Nokogiri are here: http://www.nokogiri.org/tutorials/
Use this method to parse your file:
def parse filename
xml = Nokogiri::XML(filename)
songs = xml.css('dict key').select{|key| key.text =~ /^[0-9]{4}$/}
songs.map do |song|
info = {}
song.next_element.css('key').each do |attribute|
info[attribute.text] = attribute.next_element.text
end
info
end
end
This will create a list of song hashes. Here are some examples for how to use it:
# load the two songs in your example file
songs = parse('Biblioteca.xml')
# Get the year of the first song
songs[0]['Year'] #=> 2006
# Get the Track Count of the second song's album
songs[1]['Track Count'] #=> 12
# Get the Name of the second song
songs[1]['Name'] #=> 'Baby Come On'
# Get the Album name of the second song
songs[1]['Album'] #=> 'When Your Heart Stops Beating'
From here, you can easily put info into your song objects. Let me know if you have any more questions.

I've found a library for iTunes dodgy plist xml standart... Nokogiri-plist... working fine now :D

Related

Having a CSV file and letting a user edit

In ruby, if I have a CSV like this:
make,model,color,doors,email
dodge,charger,black,4,practice1#whatever.com
ford,focus,blue,5,practice2#whatever.com
nissan,350z,black,2,practice3#whatever.com
mazda,miata,white,2,practice4#whatever.com
honda,civid,brown,4,practice5#whatever.com
corvette,stingray,red,2,practice6#whatever.com
ford,fiesta,blue,5,practice7#whatever.com
bmw,m4,black,2,practice8#whatever.com
audi,a5,blue,2,practice9#whatever.com
subaru,brz,black,2,practice10#whatever.com
lexus,rc,black,2,practice11#whatever.com
I want to allow a user to enter an email and be able to edit any one of the options listed. For example, a user enters the email "practice11#whatever.com" and it will output "lexus,rc,black,2,practice11#whatever.com". Then from here the program will output some message that will tell the user to select to edit by "make,model,color,doors,email", and then be able to change whatever is there. Like lets say they choose "color", then they can change the color from "black" to "blue" of "practice11#whatever.com" line. I believe this can be done using a hash and using key-values but I am not sure how to exactly make the editing part work.
this is my current code:
require "csv"
csv = CSV.read('cars.csv', headers: true)
demo = gets.chomp
print csv.find {|row| row['email'] == demo}
all it does it takes in the csv file and allows a user to enter in an email and it will output that specific line.
So - your question is a bit vague and involves a number of implied questions, such as "how do I write code that can ask for different options and act accordingly" - so it might help if you clarify exactly what you are trying to ask.
From the looks of it, you seem most interested in understanding how to modify the CSV table, and to get info about the CSV fields/table/data etc..
And for this, you have two friends: The ruby 'p' method and the docs.
The 'p' method allows you to inspect objects. "p someObject" is the same as calling 'puts someObject.inspect' - and it's very handy, as is "puts someObject.class" to find out what type of object you're dealing with.
In this case, you can change the last line of your code a bit to get some info:
puts csv.class
got = csv.find {|row| row['email'] == demo}
p got
And suddenly we learn we are dealing with a CSV::Table
This is not surprising, let's head over to the docs. I don't know what version of ruby you're using, but 2.6.1 is current enough to have the info we need and is plenty old at this point, so you probably have access to it:
https://ruby-doc.org/stdlib-2.6.1/libdoc/csv/rdoc/CSV.html
Tells us that if we do the CSV.read using headers:
"If headers specified, reading methods return an instance of CSV::Table, consisting of CSV::Row."
So now we know we have a CSV::Table (which is much like an array/list but with some convenience methods (such as the 'find' that you are using).
And a CSV::Row is basically a hash that maintains it's order and is, as expected, keyed according to the headers.
So we can do:
p got.fields
p got['model']
got['model'] = 'edsel'
p got['model']
p got.fields
And not surprisingly, the CSV::Table has a 'to_s' method that let's us print out the CSV:
puts csv.to_s
You can probably take it from here.

How to import a column of a CSV file into a Ruby array?

My goal is to import a one column of a CSV file into a Ruby array. This is for a self-contained Ruby script, not an application. I'll just be running the script in Terminal and getting an output.
I'm having trouble finding the best way to import the file and finding the best way to dynamically insert the name of the file into that line of code. The filename will be different each time, and will be passed in by the user. I'm using $stdin.gets.chomp to ask the user for the filename, and setting it equal to file_name.
Can someone help me with this? Here's what I have for this part of the script:
require 'csv'
zip_array = CSV.read("path/to/file_name.csv")
and I need to be able to insert the proper file path above. Is this correct? And how do I get that path name in there? Maybe I'll need to totally re-structure my script, but any suggestions on how to do this?
There are two questions here, I think. The first is about getting user input from the command line. The usual way to do this is with ARGV. In your program you could do file_name = ARGV[0] so a user could type ruby your_program.rb path/to/file_name.csv on the command line.
The next is about reading CSVs. Using CSV.read will take the whole CSV, not just a single column. If you want to choose one column of many, you are likely better off doing:
zip_array = []
CSV.foreach(file_name) { |row| zip_array << row[whichever_column] }
Okay, first problem:
a) The file name will be different on each run (I'm supposing it will always be a CSV file, right?)
You can solve this problem with creating a folder, say input_data inside your Ruby script. Then do:
Dir.glob('input_data/*.csv')
This will produce an array of ALL files inside that folder that end with CSV. If we assume there will be only 1 file at a time in that folder (with a different name), we can do:
file_name = Dir.glob('input_data/*.csv')[0]
This way you'll dynamically get the file path, no matter what the file is named. If the csv file is inside the same directory as your Ruby script, you can just do:
Dir.glob('*.csv')[0]
Now, for importing only 1 column into a Ruby array (let's suppose it's the first column):
require 'csv'
array = []
CSV.foreach(file_name) do |csv_row|
array << csv_row[0] # [0] for the first column, [1] for the second etc.
end
What if your CSV file has headers? Suppose your column name is 'Total'. You can do:
require 'csv'
array = []
CSV.foreach(file_name, headers: true) do |csv_row|
array << csv_row['Total']
end
Now it doesn't matter if your column is the 1st column, the 3rd etc, as long as it has a header named 'Total', Ruby will find it.
CSV.foreach reads your file line-by-line and is good for big files. CSV.read will read it at once but using it you can make your code more concise:
array = CSV.read(, headers: true).map do |csv_row|
csv_row['Total']
end
Hope this helped.
First, you need to assign the returned value from $stdin.gets.chomp to a variable:
foo = $stdin.gets.chomp
Which will assign the entered input to foo.
You don't need to use $stdin though, as gets will use the standard input channel by default:
foo = gets.chomp
At that point use the variable as your read parameter:
zip_array = CSV.read(foo)
That's all basic coding and covered in any intro book for a language.

unwrapping an object returned from twitter api

While reading some data from the Twitter api, I inserted the data into the file like this
results.each do |f|
running_count += 1
myfile.puts "#{f.user_mentions}"
...
The results (2 sample lines below) look like this in the file
[#<Twitter::Entity::UserMention:0x007fda754035803485 #attrs={:screen_name=>"mr_blah_blah", :name=>"mr blah blah", :id=>2142450461, :id_str=>"2141354324324", :indices=>[3, 15]}>]
[#<Twitter::Entity::UserMention:0x007f490580928 #attrs={:screen_name=>"andrew_jackson", :name=>"Andy Jackson", :id=>1607sdfds, :id_str=>"16345435", :indices=>[3, 14]}>]
Since the only information I'm actually interested in is the :screen_name, I was wondering if there's a way that I could only insert the screen names into the file. Since each line is in array brackets and then I'm looking for the screen name inside the #attrs, I did this
myfile.puts "#{f.user_mentions[0]#attrs{"screen_name"}}"
This didn't work, and I didn't expect it to, as I'm not really sure if that's technically array etc. Can you suggest how it would be done?
You need to access the #attrs instance variable in the Twitter UserMention object. If you want to puts the screen name from the first object, based on your current output, I would write
myfile.puts "#{f.user_mentions[0].attrs[:screen_name]"
Also, putting the code on how results is returned would help get a definite answer quickly. Cheers!
Assuming that results is an array of Twitter::Entity::UserMention
results.each do |r|
myfile.puts r.screen_name
end

Extract JSON values from remote api with Ruby

I'm trying to grab some data from last.fm and use it in a simple sinatra app. I've worked out how to open the document but having issues extracting the data in ruby here is the first list of the API data I'd like to grab the name:
{"similarartists":{"artist":[{"name":"Sonny & Cher"}]}
This is just an extract of the return, I'm using this in my rb file:
require 'json'
require 'open-uri'
data = JSON.parse(open("http://ws.audioscrobbler.com/2.0/?method=artist.getsimilar&artist=editors&api_key=xxx&format=json").read)
puts data["similarartists"]["artist"]["name"]
It doesn't seem to be working I get can't convert String into Integer (TypeError) on ruby 1.9.3 but the name in the JSON isn't an integer? If I just put the following:
puts data["similarartists"]["artist"]
It returns the whole thing, but I want to grab inside of that and get the name.
"name"=>"Interpol"
I don't understand why it would complain about integers when the name is a string? Hope someone can help me!
Based on the comments thread, the issue is a misunderstanding of the structure of the data returned from the API call.
The exact issue was the structure had an array of artists under the artist key so to get at the name you need to do:
data['similarartists']['artist'][0]['name']
Note though that you should only do that if you are sure there will only be one artist. The nature of the return data suggests that won't always be the case so you might be better off pulling all names depending on your use doing something like:
data['similarartists']['artist'].map {|a| a['name']}.join(',')
That will join all of the artist names together comma separated.
In the future, you can track this issue down by looking at the full structure of the return data and making sure you see the correct structure. The docs on the API may indicate some help here too.
You also might check if someone has made a gem for accessing the API. Often a gem will up-level some of this raw output and give you a nice object to work with. I suggest searching GitHub for a last.fm gem.
The problem is that you are trying to access an Array with the index "name", Ruby tries to convert this to an Integer and fails which results in the Error message you are seeing.
If you test the class of data["similarartists"]["artist"].class you will see that it returns Array. So basically what is happening is that the JSON.parse() called created as the value of data["similarartists"]["artist"] an Array of Hashes. To access all of the artist names you can simply iterate through this array:
require 'json'
require 'open-uri'
data = JSON.parse(open("http://ws.audioscrobbler.com/2.0/?method=artist.getsimilar&artist=editors&api_key=29da5a0e01ca2d1524cac596d5462d67&format=jso\
n").read)
# iterate through the Array of returned artists and print their names
data["similarartists"]["artist"].each do |artist|
puts artist["name"]
end
# output
# Interpol
# White Lies
# The Cinematics
# Smith & Burrows
# The National
# Julian Plenti
# She Wants Revenge
# etc ...
If you only want the first entry for Interpol you can just use index [0]:
puts data["similarartists"]["artist"][0]["name"]

How can I QUICKLY get a string from one of the first couple lines of a long CSV at a remote URL?

I'm working on an assignment where I retrieve several stock prices from online, using Yahoo's stock price system. Unfortunately, the Yahoo API I'm required to use returns a .csv file that apparently contains a line for every single day that stock has been traded, which is at least 5 thousand lines for the stocks I'm working with, and over 10 thousand lines for some of them (example).
I only care about the current price, though, which is in the second line.
I'm currently doing this:
require 'open-uri'
def get_ticker_price(stock)
open("http://ichart.finance.yahoo.com/table.csv?s=#{stock}") do |io|
io.read.split(',')[10].to_f
end
end
…but it's really slow.
Is all the delay coming from getting the file, or is there some from the way I'm handling it? Is io.read reading the entire file?
Is there a way to download only the first couple lines from the Yahoo CSV file?
If the answers to questions 1 & 2 don't render this one irrelevant, is there a better way to process it that doesn't require looking at the entire file (assuming that's what io.read is doing)?
You can use query string parameters to reduce the data to the current date, by using date range parameters.
example for MO on 7/13/2012: (start/end month starts w/ a zero-index, { 00 - 11 } ).
http://ichart.finance.yahoo.com/table.csv?s=MO&a=06&b=13&c=2012&d=6&e=13&f=2012&g=d
api description here:
http://etraderzone.com/free-scripts/47-historical-quotes-yahoo.html

Resources