I am looking for a way to read and parse locally a remote CSV (hosted on a particular website).
I found on the Internet a couple of interesting examples that make use of FasterCSV, that in ruby 1.9.2 has been merged into CSV. I found that you can read a remote CSV using the gems 'csv' and 'open-uri' this way:
require 'csv'
require 'open-uri'
def read(url)
open(url) do |f|
f.each_line do |l|
CSV.parse(l) do |row|
puts row
end
end
end
end
But when I call this function, I get an exception:
ERROR IOError: closed stream
Anyone can explain me why? Is there anything wrong? Should I choose another approach for reading remote CSV's?
Update
The best solution I've found till now is this:
def read(url)
data = []
begin
open(url) do |f|
data = CSV.parse f
end
rescue IOError => e
# Silently catch the exception ...
end
return data
end
but it somewhat seems not so clean. I really do not like silently catching an exception where it shouldn't be ...
Update 2
I can reproduce the error using both
ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-darwin10.4.0]
and
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.7.0]
This is the code from my test.rb file:
require 'rubygems'
require 'open-uri'
require 'csv'
def read(url)
data = []
begin
open(url) do |f|
data = CSV.parse f
end
end
puts data
end
read("http://www.euribor-ebf.eu/assets/modules/rateisblue/processed_files/myav_EURIBOR_2011.csv")
And this is the output of the ruby test.rb command
/Users/marzu/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/open-uri.rb:152:in `close': closed stream (IOError)
from /Users/marzu/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/open-uri.rb:152:in `open_uri'
from /Users/marzu/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/open-uri.rb:671:in `open'
from /Users/marzu/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/open-uri.rb:33:in `open'
from test.rb:8:in `read'
from test.rb:16:in `<main>'
I am using rvm 1.6.9 on Mac OS X 10.6.7.
Any suggestions?
On Mac OS X 10.6.7, using ruby r1.9.2, I get the same error as displayed above. But using the following code to read CSV files works for the example URL provided:
require 'rubygems'
require 'open-uri'
require 'csv'
def read(url)
CSV.new(open(url), :headers => :first_row).each do |line|
puts line
puts line[0]
puts line['FEB11']
end
end
read("http://www.euribor-ebf.eu/assets/modules/rateisblue/processed_files/myav_EURIBOR_2011.csv")
Related
I have installed Ruby (not Rails) on a machine and am trying to run some code based off the koala framework for Facebook.
When I run
gem list
koala is mentioned but when I run the file, I get this error. Rubygems is already installed, I'm not sure what else to do. Any ideas?
Edit:
require 'koala'
require 'json'
#graph = Koala::Facebook::API.new("CAACEdEose0cBANb7YuygrBflSkBrpdalb4e70T5lJgdLPYEh0Uxy5JLPVdukKSiwZAK8g27DnwscSUWNaC0s53ogq6h562LETjYO4sB5lZAMAy8tC0SM9UzqXkk7GKYpaLrkQlgj1oLTdOJhBfq5KJtFxZBkOkpz8HaVPLYp66OnuGkaGOogVseR1tUNXVxToKl6ZCmwHL0i5RHNvMnd")
url = File.open("urls.txt","r")
url.each_line do |line|
id = /[\d]+/.match(line)
begin
temp = #graph.get_object(id)
list = File.open("working.txt", "a")
list.write(id)
list.write("\n")
puts "Worked for #{id}."
rescue
puts "Didn't work for #{id}."
end
end
I have a method that zips up files I pass in.
require 'zip/zip'
def zipup(aname, aloc="/tmp/")
Zip::ZipFile.open "#{aloc}"+File.basename(aname)+".zip", Zip::ZipFile::CREATE do |zipfile|
zipfile.add File.basename(aname), aname
end
end
I need to get a string object or array object from this method that has the archive.zip name of every file that has been compressed.
rubyzip does have a to_s method all though I have failed in getting the syntax correct.
http://rubyzip.sourceforge.net/classes/Zip/ZipEntry.html#M000131
thanks from a new rubyist.
Welcome Joey, do you use the 'zip/zip' gem or just 'zip' ? If you require something, better add it to the question next time. This gem needs some extra documentation and methods it seems to me.
This works
require 'zip' #or 'zip/zip' both work
def zip_list(filename)
zipfile = Zip::ZipFile.open(filename)
list = []
zipfile.each { |entry| list << entry.name }
list
end
puts zip_list("c:/temp/zip1.zip")
another way
require 'zip/zip'
Zip::ZipFile.open("c:/temp/zip1.rb.zip") do |zipfile|
zipfile.entries.each do |entry|
puts entry.name
end
end
I am trying to access the insides of an ODT file. I'll run it through IRB and it will work perfectly fine but when I try and write a script to do it, it fails with this error:
./replace_odf.rb:3:in `require': no such file to load -- rubygems (LoadError)
from ./replace_odf.rb:3
Here is my code when ran through IRB. As you can see towards the end, it can access the file.
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'zip/zip'
=> true
irb(main):003:0> odt = Zip::ZipFile.open('java.odt')
=> java.odt
irb(main):004:0> odt.entries.each do |entry|
irb(main):005:1* puts entry.name
irb(main):006:1> end
mimetype
Configurations2/statusbar/
Configurations2/accelerator/current.xml
Configurations2/floater/
... etc
Here is my script code. When ran, it gives the error posted above.
require 'rubygems'
require 'zip/zip'
require 'rexml/document'
odt = Zip::ZipFile.open('java.odt')
file1 = odt.entries[0]
odt.entries.each do |entry|
puts entry.name if entry.name =~ /\.xml$/
end
puts odt.read("mimetype")
xml = odt.read("content.xml")
doc = REXML::Document.new(xml)
doc.root.each_element do |o|
o.each_element do |i|
puts i
end
end
I'm learning Ruby (using version 1.8.6) on Windows 7.
When I try to run the stock_stats.rb program below, I get the following error:
C:\Users\Will\Desktop\ruby>ruby stock_stats.rb
stock_stats.rb:1: undefined method `require_relative' for main:Object (NoMethodE
rror)
I have three v.small code files:
stock_stats.rb
require_relative 'csv_reader'
reader = CsvReader.new
ARGV.each do |csv_file_name|
STDERR.puts "Processing #{csv_file_name}"
reader.read_in_csv_data(csv_file_name)
end
puts "Total value = #{reader.total_value_in_stock}"
csv_reader.rb
require 'csv'
require_relative 'book_in_stock'
class CsvReader
def initialize
#books_in_stock = []
end
def read_in_csv_data(csv_file_name)
CSV.foreach(csv_file_name, headers: true) do |row|
#books_in_stock << BookInStock.new(row["ISBN"], row["Amount"])
end
end
# later we'll see how to use inject to sum a collection
def total_value_in_stock
sum = 0.0
#books_in_stock.each {|book| sum += book.price}
sum
end
def number_of_each_isbn
# ...
end
end
book_in_stock.rb
require 'csv'
require_relative 'book_in_stock'
class CsvReader
def initialize
#books_in_stock = []
end
def read_in_csv_data(csv_file_name)
CSV.foreach(csv_file_name, headers: true) do |row|
#books_in_stock << BookInStock.new(row["ISBN"], row["Amount"])
end
end
# later we'll see how to use inject to sum a collection
def total_value_in_stock
sum = 0.0
#books_in_stock.each {|book| sum += book.price}
sum
end
def number_of_each_isbn
# ...
end
end
Thanks in advance for any help.
require_relative doesn't exist in your version of Ruby. You could upgrade Ruby, install the backports gem and require 'backports/1.9.1/kernel/require/relative' but the easiest fix will be to change your require to:
require File.join(File.dirname(__FILE__), 'csv_reader')
Edit:
Back in the days where this question was asked it referred to Ruby 1.8.6 where there was no require_relative. By now Ruby 1.8.6 is outdated and shouldn't be used anymore.
Original:
There is simply no method name require_relative. You can use require there aswell.
The require_relative function is included in an extension project to the Ruby core libraries, found here: http://www.rubyforge.org/projects/extensions
You should be able to install them with gem install extensions.
Then in your code add the following line before the require_relative:
require 'extensions/all'
I have a method that takes a block.
Obviously I don't know what is going to be passed in and for bizarre reasons that I won't go into here I want to print the contents of the block.
Is there a way to do this?
You can do this with Ruby2Ruby which implements a to_ruby method.
require 'rubygems'
require 'parse_tree'
require 'parse_tree_extensions'
require 'ruby2ruby'
def meth &block
puts block.to_ruby
end
meth { some code }
will output:
"proc { some(code) }"
I would also check out this awesome talk by Chris Wanstrath of Github http://goruco2008.confreaks.com/03_wanstrath.html He shows some interesting ruby2ruby and parsetree usage examples.
In Ruby 1.9+ (tested with 2.1.2), you can use https://github.com/banister/method_source
Print out the source via block#source:
#! /usr/bin/ruby
require 'rubygems'
require 'method_source'
def wait &block
puts "Running the following code: #{block.source}"
puts "Result: #{yield}"
puts "Done"
end
def run!
x = 6
wait { x == 5 }
wait { x == 6 }
end
run!
Note that in order for the source to be read you need to use a file and execute the file (testing it out from irb will result in the following error: MethodSource::SourceNotFoundError: Could not load source for : No such file or directory # rb_sysopen - (irb)
Building on Evangenieur's answer, here's Corban's answer if you had Ruby 1.9:
# Works with Ruby 1.9
require 'sourcify'
def meth &block
# Note it's to_source, not to_ruby
puts block.to_source
end
meth { some code }
My company uses this to display the Ruby code used to make carbon calculations... we used ParseTree with Ruby 1.8 and now sourcify with Ruby 1.9.
In Ruby 1.9, you can try this gem which extract the code from source file.
https://github.com/ngty/sourcify
In Ruby 2.5 the following works
puts block.source
In ruby 2.7, using the method_source gem (pry depends on it)
Set.instance_method(:merge).source.display
# =>
def merge(enum)
if enum.instance_of?(self.class)
#hash.update(enum.instance_variable_get(:#hash))
else
do_with_enum(enum) { |o| add(o) }
end
self
end
The repo says it works for procs, but I haven't tested it.