knowing if file is YAML or not - ruby

I would like to expect that YAML.load_file(foo) of ruby YAML module returns null if foo is not a YAML file. But I get exception:
did not find expected alphabetic or numeric character while scanning an alias at line 3 column 3 (Psych::SyntaxError)
from /usr/lib/ruby/2.4.0/psych.rb:377:in `parse_stream'
from /usr/lib/ruby/2.4.0/psych.rb:325:in `parse'
from /usr/lib/ruby/2.4.0/psych.rb:252:in `load'
from /usr/lib/ruby/2.4.0/psych.rb:473:in `block in load_file'
from /usr/lib/ruby/2.4.0/psych.rb:472:in `open'
from /usr/lib/ruby/2.4.0/psych.rb:472:in `load_file'
from ./select.rb:27:in `block in selecting'
from ./select.rb:26:in `each'
from ./select.rb:26:in `selecting'
from ./select.rb:47:in `block (2 levels) in <main>'
from ./select.rb:46:in `each'
from ./select.rb:46:in `block in <main>'
from ./select.rb:44:in `each'
from ./select.rb:44:in `<main>'
How can I triage if a file is a YAML file or not without a exception? In my case, I navigate to a directory and process markdown files: I add to a list markdown files with a key output: word and I return that list
mylist = Array.new
mylist = []
for d in (directory - excludinglist)
begin
info = YAML.load_file(d)
if info
if info.has_key?('output')
if info['output'].has_key?(word)
mylist.push(d)
end
end
end
rescue Psych::SyntaxError => error
return []
end
end
return mylist
When I catch exceptions, the bucle does not continue to push elements on my list.

The short answer: you can't.
Because YAML is just a text file, the only way to know whether a given text file is YAML or not is to parse it. The parser will try to parse the file, and if it is not valid YAML, it will raise an error.
Errors and exceptions are a common part of Ruby, especially in the world of IO. There's no reason to be afraid of them. You can easily rescue from them and continue on your way:
begin
yaml = YAML.load_file(foo)
rescue Psych::SyntaxError => e
# handle the bad YAML here
end
You mentioned that the following code will not work because you need to handle multiple files in a directory:
def foo
mylist = []
for d in (directory - excludinglist)
begin
info = YAML.load_file(d)
if info
if info.has_key?('output')
if info['output'].has_key?(word)
mylist.push(d)
end
end
end
rescue Psych::SyntaxError => error
return []
end
return mylist
end
The only issue here is that when you hit an error, you respond by returning from the function early. If you don't return, the for-loop will continue and you will get your desired functionality:
def foo
mylist = []
for d in (directory - excludinglist)
begin
info = YAML.load_file(d)
if info
if info.has_key?('output')
if info['output'].has_key?(word)
mylist.push(d)
end
end
end
rescue Psych::SyntaxError => error
# do nothing!
# puts "or your could display an error message!"
end
end
return mylist
end

Psych::SyntaxError gets raised by Psych::Parser#parse, the source for which is written in C. So unless you want to work with C, you can't write a patch for the method in Ruby to prevent the exception from getting raised.
Still, you could certainly rescue the exception, like so:
begin
foo = YAML.load_file("not_yaml.txt")
rescue Psych::SyntaxError => error
puts "bad yaml"
end

Related

Ruby CSV truncating backtrace

Given the following code:
test.rb
require 'csv'
def meth1
meth2
end
def meth2
begin
iter = CSV.foreach('').each # empty file path, will raise exception.
iter.next
rescue Exception => e
puts e
puts e.backtrace
end
end
meth1
Two questons.
First, why is the backtrace truncated and not showing meth1 or meth2 calls:
No such file or directory # rb_sysopen -
ruby test.rb
/Users/x/.rbenv/versions/2.7.2/lib/ruby/2.7.0/csv.rb:641:in `initialize'
/Users/x/.rbenv/versions/2.7.2/lib/ruby/2.7.0/csv.rb:641:in `open'
/Users/x/.rbenv/versions/2.7.2/lib/ruby/2.7.0/csv.rb:641:in `open'
/Users/x/.rbenv/versions/2.7.2/lib/ruby/2.7.0/csv.rb:510:in `foreach'
test.rb:in `each'
Second, the (truncated) backtrace points to line 641 of CSV (using Ruby v 2.7.2). However line 641 has no initialize() method. Where is this coming from?
.rbenv/versions/2.7.2/lib/ruby/2.7.0/csv.rb
...
begin
f = File.open(filename, mode, **file_opts) ## line 641
rescue ArgumentError => e
raise unless /needs binmode/.match?(e.message) and mode == "r"
mode = "rb"
file_opts = {encoding: Encoding.default_external}.merge(file_opts)
retry
end
If I try another test calling File.open('') directly (instead of through CSV), the resulting backtrace shows everything as expected (including the calls to meth1 and meth2).
Any Ruby gurus out there know what is going on?
I'm not sure I understand; #foreach:510 calls open, #open:641 calls File.open.
In any case, Ruby backtraces have always been a bit wonky, particularly with local top-level files.
The reason you see the initialize is because foreach is a class method of CSV, so there's some behind-the-scenes Ruby shenanigans.
You can use the private method caller_locations, however:
puts e.send(:caller_locations)
which outputs:
test.rb:7:in `meth2'
test.rb:4:in `meth1'
test.rb:20:in `<main>'
This is just the file's top level methods.
You can play games like ruby -d that'll at least get you the line of the script:
Exception `LoadError' at /Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems.rb:1424 - cannot load such file -- rubygems/defaults/operating_system
Exception `LoadError' at /Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems.rb:1432 - cannot load such file -- rubygems/defaults/ruby
Exception `SyntaxError' at /Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/forwardable/impl.rb:5 - /Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/forwardable/impl.rb:5: syntax error, unexpected end-of-input
Exception `SyntaxError' at /Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/forwardable/impl.rb:5 - /Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/forwardable/impl.rb:5: syntax error, unexpected end-of-input
Exception `Errno::ENOENT' at /Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/csv.rb:641 - No such file or directory # rb_sysopen -
Exception `Errno::ENOENT' at test.rb:10 - No such file or directory # rb_sysopen -
/Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/csv.rb:641:in `initialize'
/Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/csv.rb:641:in `open'
/Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/csv.rb:641:in `open'
/Users/dave/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/csv.rb:510:in `foreach'
test.rb:in `each'
Along with other noise.
If you really want to blow your mind, inside the rescue:
puts Thread.current.backtrace
Which outputs:
test.rb:21:in `backtrace'
test.rb:21:in `rescue in meth2'
test.rb:7:in `meth2'
test.rb:4:in `meth1'
test.rb:35:in `<main>'

Error handling for Ruby Kernel method not working

I have found myself in need to execute a string. Current method is to use the Kernel#eval() method. Everything is working fine, but my error handling isn't working. For example, a missing closing quotation mark will completely kill and exit the program.
Here's an excerpt. Any idea why I can't catch the error?
def process(str)
print "\n"
eval(str)
rescue => e
puts e
end
>> process('"')
console.rb:90:in `eval': (eval):1: unterminated string meets end of file (SyntaxError)
from console.rb:90:in `process'
from console.rb:81:in `bouncer'
from console.rb:14:in `block in prompt'
from console.rb:11:in `loop'
from console.rb:11:in `prompt'
from console.rb:97:in `<main>'
According to the documentation:
A rescue clause without an explicit Exception class will rescue all StandardErrors (and only those).
SyntaxError is not a StandardError. To catch it, you have to be explicit, e.g.:
def process(str)
print "\n"
eval(str)
rescue Exception => e
puts e
end
process('"')
Output:
(eval):1: unterminated string meets end of file

Ruby script raising unexpected backtrace

I have a method that should raise a custom error with a message. When I catch the error and raise my own custom error, it is still raising and printing the backtrace of the original error. I just want the custom error and message. Code below.
Method:
def load(configs)
begin
opts = {access_token: configs['token'],
api_endpoint: configs['endpoint'],
web_endpoint: configs['site'],
auto_paginate: configs['pagination']}
client = Octokit::Client.new(opts)
repos = client.org_repos(configs['org'])
repos.each do |r|
Project.create(name: r.name)
end
rescue Octokit::Unauthorized
raise GitConfigError, "boom"
end
#rescue Octokit::Unauthorized
end
class GitConfigError < StandardError
end
My test (which is failling):
context 'with incorrect git configs' do
before do
allow(loader).to receive(:load).and_raise Octokit::Unauthorized
end
it { expect{loader.load(configs)}.to raise_error(GitConfigError, "boom" ) }
end
Test Output:
GitProjectLoader#load with incorrect git configs should raise GitConfigError with "boom"
Failure/Error: it { expect{loader.load(configs)}.to raise_error(GitConfigError, "boom" ) }
expected GitConfigError with "boom", got #<Octokit::Unauthorized: Octokit::Unauthorized> with backtrace:
# ./spec/lib/git_project_loader_spec.rb:24:in `block (5 levels) in <top (required)>'
# ./spec/lib/git_project_loader_spec.rb:24:in `block (4 levels) in <top (required)>'
# ./spec/lib/git_project_loader_spec.rb:24:in `block (4 levels) in <top (required)>'
If you intend to test the handling of the Octokit::Unauthorized error, then raise the error anywhere before the rescue kicks in. Preferably, someplace where it would actually be raised.
Something like this, for example:
before do
allow(Octokit::Client).to receive(:new).and_raise(Octokit::Unauthorized)
end
And then:
expect{ loader.load(configs) }.to raise_error(GitConfigError, "boom" )
As a side note, I would discourage enclosing all lines of your method in a begin;rescue;end structure; you should enclose only the lines from which you are expecting errors.
You are not testing your code as you think. You have mocked it out.
The line
allow(loader).to receive(:load).and_raise Octokit::Unauthorized
replaces the load method on loader with a stub which just raises the named error.
Remove your before block, and it should test your code as intended. Note as written it will make a real request via Octokit, unless you mock that out instead.

how to handle mongodb's E11000 duplicate key error in ruby

Is there any good example of handling mongodb related exceptions in ruby?
In this case I have:
/home/askar/.rvm/gems/ruby-1.9.3-p429/gems/mongo-1.8.6/lib/mongo/networking.rb:89:in `send_message_with_gle': 11000: E11000 duplicate key error index: somedb.somecoll.$_id_ dup key: { : "some_id" } (Mongo::OperationFailure)
from /home/askar/.rvm/gems/ruby-1.9.3-p429/gems/mongo-1.8.6/lib/mongo/collection.rb:1108:in `block in insert_documents'
from /home/askar/.rvm/gems/ruby-1.9.3-p429/gems/mongo-1.8.6/lib/mongo/util/logging.rb:33:in `block in instrument'
from /home/askar/.rvm/gems/ruby-1.9.3-p429/gems/mongo-1.8.6/lib/mongo/util/logging.rb:65:in `instrument'
from /home/askar/.rvm/gems/ruby-1.9.3-p429/gems/mongo-1.8.6/lib/mongo/util/logging.rb:32:in `instrument'
from /home/askar/.rvm/gems/ruby-1.9.3-p429/gems/mongo-1.8.6/lib/mongo/collection.rb:1106:in `insert_documents'
from /home/askar/.rvm/gems/ruby-1.9.3-p429/gems/mongo-1.8.6/lib/mongo/collection.rb:375:in `insert'
from lib/tasks/getorders.rb:47:in `block in <main>'
from lib/tasks/getorders.rb:25:in `each'
from lib/tasks/getorders.rb:25:in `<main>'
I'm having this error because I'm trying to insert a document with the id that already exists in mongodb database, I just want to know how to handle mongodb related exceptions in ruby.
For example, if an exception occurs, then I would change the id of the hash and then re-try to insert it.
How rescue block would look like?
The ruby block would look something like :
begin
# your operation
rescue Mongo::OperationFailure => e
if e.message =~ /^11000/
puts "Duplicate key error #{$!}"
# do something to recover from duplicate
else
raise e
end
end
# the rest of the exceptions follow ..
# if you just care about the dup error
# then ignore them
#rescue Mongo::MongoRubyError
# #Mongo::ConnectionError, Mongo::ConnectionTimeoutError, Mongo::GridError, Mongo::InvalidSortValueError, Mongo::MongoArgumentError, Mongo::NodeWithTagsNotFound
# puts "Ruby Error : #{$!}"
#rescue Mongo::MongoDBError
# # Mongo::AuthenticationError, Mongo::ConnectionFailure, Mongo::InvalidOperation, Mongo::OperationFailure
# puts "DB Error : #{$!}"
#rescue Mongo::OperationTimeout
# puts "Socket operation timeout Error : #{$!}"
#rescue Mongo::InvalidNSName
# puts "invalid collection or database Error : #{$!}"
#end
But, if you are updating a record that already exists, why not you use an upsert.
If you are creating a new record then why not let the mongod create the _id ?
Can also use write concerns.
#mongo_client.save({:doc => 'foo'}, {:w => 0}) # writes are not acknowledged
This is not as good as the rescue though.
If your ruby driver is >= 2.0.0, you should use Mongo::Error class for most of mongodb related exceptions. Here is how your rescue block should look like:
begin
# Document insert code
rescue Mongo::Error => e
if e.message.include? 'E11000'
# Change the id of the hash & re-try to insert it
end
end

Why is kernel_required.rb in my stack trace?

I forgot to put the word end, at the end of a if statement,
and got the following error:
/home/***/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require': **/home/****/Desktop/ruby/food_finder/lib/restaurant.rb:84: syntax error, unexpected end-of-input, expecting keyword_end (SyntaxError)**
from /home/****/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
from /home/****/Desktop/ruby/food_finder/lib/guide.rb:1:in `<top (required)>'
from /home/****/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
from /home/****/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
from init.rb:14:in `<main>'
my code without errors:
def self.saved_restaurants
# read the restaurant file
restaurants = []
if file_usable?
file = File.new(##filepath, 'r')
file.each_line do |line|
restaurants << Restaurant.new.import_line(line.chomp)
end
file.close
**end** -- > forgotten end
# return instances of restaurant
return restaurants
end
my code with errors:
def self.saved_restaurants
# read the restaurant file
restaurants = []
if file_usable?
file = File.new(##filepath, 'r')
file.each_line do |line|
restaurants << Restaurant.new.import_line(line.chomp)
end
file.close
-- > forgotten end
# return instances of restaurant
return restaurants
end
My questions are:
Why do i get errors that has noting to do with my code?
like the following:
/home/***/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require':
What does this error mean?
When i use the correct syntax for the saved_restaurants method, i don't get any error.
Your file restaurant.rb is read by a method call require in guide.rb, which is defined in kernel_require.rb. Within its method definition, it has this part:
def require path
...
rescue LoadError => load_error
...
raise load_error
end
When you have a syntax error in the file that is read, that will raise a LoadError, which is rescued, and will be raised as an error of require.
If i understand correctly there is a file named guide.rb which does:
require restaurant
Basically, require is a function implemented in kernel_require.rb whose prototype is like:
require path
Here path is restaurant.rb and this function fails because the require function is unable to load the rb file because of syntax error.
Remember you are looking at the call stack so the function with missing end should not show up because that function is not called but only the ruby file is loaded.

Resources