I want to preserve the order of the keys in a YAML file loaded from disk, processed in some way and written back to disk.
Here is a basic example of loading YAML in Ruby (v1.8.7):
require 'yaml'
configuration = nil
File.open('configuration.yaml', 'r') do |file|
configuration = YAML::load(file)
# at this point configuration is a hash with keys in an undefined order
end
# process configuration in some way
File.open('output.yaml', 'w+') do |file|
YAML::dump(configuration, file)
end
Unfortunately, this will destroy the order of the keys in configuration.yaml once the hash is built. I cannot find a way of controlling what data structure is used by YAML::load(), e.g. alib's orderedmap.
I've had no luck searching the web for a solution.
Use Ruby 1.9.x. Previous version of Ruby do not preserve the order of Hash keys, but 1.9 does.
If you're stuck using 1.8.7 for whatever reason (like I am), I've resorted to using active_support/ordered_hash. I know activesupport seems like a big include, but they've refactored it in later versions to where you pretty much only require the part you need in the file and the rest gets left out. Just gem install activesupport, and include it as shown below. Also, in your YAML file, be sure to use an !!omap declaration (and an array of Hashes). Example time!
# config.yml #
months: !!omap
- january: enero
- february: febrero
- march: marzo
- april: abril
- may: mayo
Here's what the Ruby behind it looks like.
# loader.rb #
require 'yaml'
require 'active_support/ordered_hash'
# Load up the file into a Hash
config = File.open('config.yml','r') { |f| YAML::load f }
# So long as you specified an !!omap, this is actually a
# YAML::PrivateClass, an array of Hashes
puts config['months'].class
# Parse through its value attribute, stick results in an OrderedHash,
# and reassign it to our hash
ordered = ActiveSupport::OrderedHash.new
config['months'].value.each { |m| ordered[m.keys.first] = m.values.first }
config['months'] = ordered
I'm looking for a solution that allows me to recursively dig through a Hash loaded from a .yml file, look for those YAML::PrivateClass objects, and convert them into an ActiveSupport::OrderedHash. I may post a question on that.
Someone came up with the same issue. There is a gem ordered hash. Note that it is not a hash, it creates a subclass of hash. You might give it a try, but if you see a problem dealing with YAML, then you should consider upgrading to ruby1.9.
Related
I'm trying to understand how YAML works and I'm working from examples in a book to learn it. Here's the code I'm using:
require "yaml"
def savelist(songs, playlist)
File.open playlist, 'w' do |f|
f.write(songs.to_yaml)
end
end
def openlist(playlist)
loadlist = File.read playlist
YAML::load loadlist
end
podcasts = Dir['C:/Users/7/Music/Amazon MP3/*.mp3']
puts "Welcome to the Playlist Maker!"
list = 'playlist.txt'
savelist(podcasts, list)
openlist(list)
I thought the second method, openlist, would actually call the file to the screen and print out its contents. I read it as "create a loadlist object then load the object to the screen with YAML". I expected it to print the file contents (YAML::load).
The program works, but I don't really understand how it works. I'm also not really sure what the second method is there for if it doesn't actually open the file.
Thanks for your insight. I sorta get what YAML does, but I don't really know how or why this works.
Consider this:
require "yaml"
def savelist(songs, playlist)
File.write(playlist, songs.to_yaml)
end
def openlist(playlist)
YAML::load_file(playlist)
end
Learning how YAML works is easier if you look at what it's emitting, and play with it in IRB:
>> require 'yaml'
false
>> songs = ['foo', 'bar']
[
[0] "foo",
[1] "bar"
]
>> puts songs.to_yaml
---
- foo
- bar
nil
>>
>> yaml = songs.to_yaml
"---\n- foo\n- bar\n"
>> YAML::load(yaml)
[
[0] "foo",
[1] "bar"
]
YAML is just text, so writing a YAML file is simple if you use write, as I did above. YAML has a convenient method load_file to read a file, then parse it back into the equivalent Ruby object. If you've already read the file into a variable, then use load to parse it and return the Ruby object.
Playing with YAML this way, by converting an array or hash to YAML, then using load to mess with it, is a great way to learn it. The YAML spec is useful once you start to get an idea how it works, but doing the "round-trip" thing is how to boot-strap yourself. (This is also useful when learning about JSON.)
We actually use a variation of this when initially defining complex YAML files. I'll write a bit of Ruby code with the object defined inside it, then save that to a file. From that point on we can tweak the YAML, or, as I prefer, we tweak the Ruby and re-create the YAML. Using the Ruby code to recreate the file means we've got a fall-back if someone hoses the configuration. At least we can rebuild our default setup.
Let's step through this and see how we might learn what a block of code does.
def openlist(playlist)
loadlist = File.read playlist
YAML::load loadlist
end
For starters we're passing some playlist object to File.read. We can look up the File class but we won't find a read method on it. We will however see that File is a subclass of IO and if we look at IO we will find IO.read
read(name, [length [, offset]] ) → string
So read takes a file name and an optional length and offset and returns a string. loadlist is then just a string containing the contents of this file. Useful but not easy for us to work with yet.
If we look up the YAML module we learn that it is a wrapper around Psych where we can finally find load
load(yaml, filename = nil)
Load yaml in to a Ruby data structure...
So load takes some string and returns a Ruby data structure created by parsing the YAML syntax it contains.
Now we see why this example is using two different methods, one to load the contents of a file and one to parse those into a Ruby data structure. While we're looking at Psych we might also notice the load_file method which gives us another way to do the same thing as #the-tin-man suggested.
I'm using Ruby 1.9.3 and I've discovered the CSV class, but I can't get it to work. Basically, I want to be able to manipulate the various options for the CSV, and then pull a .csv file into an array to work with, eventually pushing that array back out into a new file.
This is what I have currently:
require 'csv'
CSV_Definition = CSV.New(:header_converters => :symbol)
CSV_Total = CSV.Read(File.Path("C:\Scripts\SQL_Log_0.csv"))
However, I don't think this is the right way to change the :header_converters. Currently I can't get IRB working to parse these pieces of code (I'm not sure how to require 'csv' in IRB) so I don't have any particular error message. My expectations for this will be to create an array (CSV_Total) that has a header with no symbols in it. The next step is to put that array back into a new file. Basically it would scrub CSV files.
Ruby used to have it's own built in CSV library which has been replaced with FasterCSV as of version 1.9, click on the link for documentation.
All that's required on your part is to use to import the CSV class via require 'csv' statement wherever you want to use it and process accordingly. It's pretty easy to build an array with the foreach statement, e.g.,:
people.csv
Merry,Christmas
Hal,Apenyo
Terri,Aki
Willy,Byte
process_people.rb
require 'csv'
people = []
CSV.foreach(File.path("people.csv")) do |row|
# Where row[i] corresponds to a zero-based value/column in the csv
people << [row[0] + " " + row[1]]
end
puts people.to_s
=> [["Merry Christmas"], ["Hal Apenyo"], ["Terri Aki"], ["Willy Byte"]]
I'm trying to grab some data from last.fm and use it in a simple sinatra app. I've worked out how to open the document but having issues extracting the data in ruby here is the first list of the API data I'd like to grab the name:
{"similarartists":{"artist":[{"name":"Sonny & Cher"}]}
This is just an extract of the return, I'm using this in my rb file:
require 'json'
require 'open-uri'
data = JSON.parse(open("http://ws.audioscrobbler.com/2.0/?method=artist.getsimilar&artist=editors&api_key=xxx&format=json").read)
puts data["similarartists"]["artist"]["name"]
It doesn't seem to be working I get can't convert String into Integer (TypeError) on ruby 1.9.3 but the name in the JSON isn't an integer? If I just put the following:
puts data["similarartists"]["artist"]
It returns the whole thing, but I want to grab inside of that and get the name.
"name"=>"Interpol"
I don't understand why it would complain about integers when the name is a string? Hope someone can help me!
Based on the comments thread, the issue is a misunderstanding of the structure of the data returned from the API call.
The exact issue was the structure had an array of artists under the artist key so to get at the name you need to do:
data['similarartists']['artist'][0]['name']
Note though that you should only do that if you are sure there will only be one artist. The nature of the return data suggests that won't always be the case so you might be better off pulling all names depending on your use doing something like:
data['similarartists']['artist'].map {|a| a['name']}.join(',')
That will join all of the artist names together comma separated.
In the future, you can track this issue down by looking at the full structure of the return data and making sure you see the correct structure. The docs on the API may indicate some help here too.
You also might check if someone has made a gem for accessing the API. Often a gem will up-level some of this raw output and give you a nice object to work with. I suggest searching GitHub for a last.fm gem.
The problem is that you are trying to access an Array with the index "name", Ruby tries to convert this to an Integer and fails which results in the Error message you are seeing.
If you test the class of data["similarartists"]["artist"].class you will see that it returns Array. So basically what is happening is that the JSON.parse() called created as the value of data["similarartists"]["artist"] an Array of Hashes. To access all of the artist names you can simply iterate through this array:
require 'json'
require 'open-uri'
data = JSON.parse(open("http://ws.audioscrobbler.com/2.0/?method=artist.getsimilar&artist=editors&api_key=29da5a0e01ca2d1524cac596d5462d67&format=jso\
n").read)
# iterate through the Array of returned artists and print their names
data["similarartists"]["artist"].each do |artist|
puts artist["name"]
end
# output
# Interpol
# White Lies
# The Cinematics
# Smith & Burrows
# The National
# Julian Plenti
# She Wants Revenge
# etc ...
If you only want the first entry for Interpol you can just use index [0]:
puts data["similarartists"]["artist"][0]["name"]
I'm having an awful time trying to use a library to parse an XML File into a hash like object, modify it, then print it back out to another XML file in Ruby. For a class I'm taking, we're supposed to use a Java JAXB like library where we convert XML into an object. We've already done SAX and DOM methods so we can't use those methods of XML de-serialization. Nokogiri helped me with both of these in Ruby.
The only problem is that besides the SIMPLE modifications I'm making to the objects, when I write to file it has drastic differences. Is there a Ruby library meant for doing just this? I've tried: ROXML, XML::Mapping, and ActiveSupport::CoreExt. The only one I can get to even run is ActiveSupport, and even then it starts putting element attributes as child elements in the output XML.
I'm willing to try out XmlSimple, but I'm curious has anyone actually had to do this before/run into the same problems? Again, I can't read in lines one at a time like SAX or build a Tree like structure like DOM, it needs to be a hash like object.
Any help is much appreciated!
You should have a look into nokogiri: http://nokogiri.org/
Then you can parse the XML like this :
xml_file = "some_path"
#xml = Nokogiri::XML(File.open xml_file)
#xml.xpath('//listing').each do |node|
style = node.search("style").text
end
With Xpath, you can perform queries in the XML :
#xml.xpath("//listing[name='John']").first(10)
OK, I got it working. After looking at ActiveSupport::CoreExt 's source code I found it just uses a gem called xml-simple. What's obnoxious is the gem, library name in the require statement, and class name are a mixture of hyphenated and non hyphenated spellings. For future reference here's what I did:
# gem install xml-simple
# ^ all lowercase, hyphenated
require 'xmlsimple'
# ^ all lowercase, not hyphenated
doc = XmlSimple.xml_in 'hw3.xml', 'KeepRoot' => true
# ^ Camel cased (it's a class), not hyphenated
# doc.class => Hash
# manipulate doc as a hash
file = File.new('HW3a.xml', 'w')
file.write("<?xml version='1.0' encoding='utf-8'?>\n")
file.write(XmlSimple.xml_out doc, 'KeepRoot' => true)
I hope this helps someone. Also make sure you pay attention to case and hyphens with this gem!!!
I am trying to use the Yahoo Finance Gem, but am not able to get the information I want. When I try to get a quote, it creates a hash, but instead of the individual information (which I am trying to get), it gives a string will all the information in it. Is there a way to receive a single bit of information (such as % change) as a number? I am very new to ruby, so any help would be awesome.
require 'yahoofinance'
YahooFinance.get_quotes(YahooFinance::StandardQuote, 'yhoo') {|i|
puts i.change
puts i.changePoints
puts i.changePercent
puts i.time
}
Prints for me:
-0.03 - -0.17%
-0.03
-0.17
10:55am
or
r = yahooFinance.get_quotes(YahooFinance::StandardQuote, 'yhoo')
puts r[r.keys[0]].dayHigh
puts r["YHOO"].dayHigh
prints:
17.43
17.43
YahooFinance.get_quotes return a hash in which quote symbols are keys, and all data for each quote is a value. See YahooFinance::BaseQuote class to guess why it is possible to use getters like dayHigh() to auto parse data from the hash value.
I'm running Rails 3.2.8 along with the Ruby 1.9.3 and was having some problems with this gem.
So I just went straight to the source code and took that one file (its just a single file, and short too) and placed it in my /lib folder. In case you haven't been using your lib folder, you must add something like config.autoload_paths += Dir["#{config.root}/lib/**/"] to config/application.rb in order to load up lib folder classes from the rail console or elsewhere in rails.
Besides, its probably the simplest source code you will find and its always good to start reading the actual source that you rely on every day.