Still new to Ruby - I've had a look at some of the answers to seemingly similar questions but, to be honest, I couldn't get my head around them.
I have some code that reads a .csv file. The data is split into groups of 40-50 rows per user record and validates data in the rows against a database accessed via a website.
A login is required for each record, but once that user has logged in each row in the .csv file can be checked until the next user is reached, at which point the user logs out.
All that's working, however, if an error occurs (e.g. a different result on the website than the expected result on the .csv file) the program stops.
I need something that will
a) at tell me which line on the file the error occurred
b) log the row to be output when it's finished running, and
iii) restart the program from the next line in the .csv file
The code I have so far is below
Thanks in advance,
Peter
require 'csv-mapper'
loginrequired = true
Given(/^I compare the User Details from rows "(.*?)" to "(.*?)"$/) do |firstrow, lastrow|
data = CsvMapper.import('C:/auto_test_data/User Data csv.csv') do
[dln, nino, pcode, endor_cd, ct_cd]
end
#Row number changed because Excel starts at 'row 1' and Ruby starts counting at 'row 0'
(firstrow.to_i-1..lastrow.to_i-1).each do |row|
#licnum1 = data.at(row).dln
#licnum2 = data.at(row+1).dln
#nino = data.at(row).nino
#postcode = data.at(row).pcode
#endor_cd = data.at(row).endor_cd
#ct_cd = data.at(row).ct_cd
#Login only required once for each new user-account
if
loginrequired == true
logon_to_vdr #def for this is in hooks
click_on 'P and D'
loginrequired = false
end
#This is the check against the database and is required for every line in the .csv file
check_ctcd #def for this is in hooks
#Need something in here to log errors and move on to the next line in the .csv file
#Compare the ID for the next record and logout if they're different
if #licnum1 == #licnum2
loginrequired = false
else
loginrequired = true`enter code here`
click_on 'Logout'
end
end
end
It seems like you need some error logging since you apparently don't know what type of error you're receiving or where. If this script is standalone you can redirect $stderr to file so that you can read what went wrong.
# put this line at the top of your script
$stderr = File.open("/path/to/your/logfile.log","a")
When an error occurs, ruby will automatically write the error message, class, and backtrace to the log file you specify so that you can trace back the line where things are not going as expected. (When you run a script from the command line, normally this information will just get blurted back to the terminal when an error happens.)
For example, on my desktop I created a file log_stderr.rb with the following (line numbers included):
1 $stderr = File.open("C:/Users/me/Desktop/my_log.log","w")
2
3 #require a file which will raise an error to see the backtrace
4 require_relative 'raise_error.rb'
5
6 puts "code that will never be reached"
Also on my desktop I created the file raise_error.rb with the following (to deepen the backtrace for better example output):
1 # call raise to generate an error arbitrarily
2 # to halt execution and exit the program.
3 raise RuntimeError, 'the program stopped working!'
When I run ruby log_stderr.rb from the command line, my_log.log is created on my desktop with the following:
C:/Users/me/Desktop/raise_error.rb:3:in `<top (required)>': the program stopped working! (RuntimeError)
from C:/Users/me/Desktop/log_stderr.rb:4:in `require_relative'
from C:/Users/me/Desktop/log_stderr.rb:4:in `<main>'
If you are working in a larger environment where your script is being called amidst other scripts then you probably do not want to redirect $stderr because this would affect everything else running in the environment. ($stderr is global as indicated by the $ variable prefix.) If this is the case you would want to implement a begin; rescue; end structure and also make your own logfile so that you don't affect $stderr.
Again, since you don't know where the error is happening you want to wrap the whole script with begin; end
# at the very top of the script, begin watching for weirdness
begin
logfile = File.open("/path/to/your/logfile.log", "w")
require 'csv-mapper'
#. . .
# rescue and end at the very bottom to capture any errors that have happened
rescue => e
# capture details about the error in your logfile
logfile.puts "ERROR:", e.class, e.message, e.backtrace
# pass the error along since you don't know what it is
# and there may have been a very good reason to stop the program
raise e
end
If you find that your error is happening only in the block (firstrow.to_i-1..lastrow.to_i-1).each do |row| you can place the begin; end inside of this block to have access to the local row variable, or else create a top level variable independent of the block and assign it during each iteration of the block to report to your logfile:
begin
logfile = File.open("/path/to/your/logfile.log", "w")
csv_row = "before csv"
#. . .
(firstrow.to_i-1..lastrow.to_i-1).each do |row|
csv_row = row
#. . .
end
csv_row = "after csv"
rescue => e
logfile.puts "ERROR AT ROW: #{csv_row}", e.class, e.message, e.backtrace
raise e
end
I hope this helps!
It doesn't seem like you need to rescue exception here. But what you could do is in your check_ctcd method, raise error if records doesn't match. Then you can rescue from it. In order to know which line it is, in your iteration, you could use #each_with_index and log the index when things go wrong.
(firstrow.to_i-1..lastrow.to_i-1).each_with_index do |row, i|
#licnum1 = data.at(row).dln
#licnum2 = data.at(row+1).dln
#nino = data.at(row).nino
#postcode = data.at(row).pcode
#endor_cd = data.at(row).endor_cd
#ct_cd = data.at(row).ct_cd
#Login only required once for each new user-account
if
loginrequired == true
logon_to_vdr #def for this is in hooks
click_on 'P and D'
loginrequired = false
end
#This is the check against the database and is required for every line in the .csv file
check_ctcd #def for this is in hooks
rescue => e
# log the error and index here
...
And you can make your own custom error, and rescue only the certain type so that you don't silently rescue other errors.
Related
Here is my unfinished code:
#When convert button is pressed
File.rename("*.osz", "*.zip$")
dialog.directory(
def extract_zip(file, destination) FileUtils.mkdir_p(destination)
file_path = "./convert_temp/*.zip"
destination = "./convert_temp/osz/"
extract_zip(file_path, destination)
until File.exists?( ".osu$" ) == false do
File.rename("./convert_temp/osz/*.osu$", "*.txt$")
File.foreach(filename) do |file|
file_string = File.read('./convert_temp/osz/*.txt$')
if file_string.include?('Mode: 1')
puts 'Yes'
else
puts 'No'
end
end
end
Robocop giving the following syntax error:
unexpected token $end (Using Ruby 2.2 parser; configure using `TargetRubyVersion` parameter, under `AllCops`)
Actually, Rubocop is not even able to parse the file, because it has syntax errors.
The error message syntax error: unexpected token $end means that the ruby parser was parsing along happily, but then it suddenly encountered an $end, which is the parser's way to say "the end of the file". It was expecting more code, but instead it found the end of the file.
This is what your code looks like with proper indentation:
#When convert button is pressed
File.rename("*.osz", "*.zip$")
dialog.directory(
def extract_zip(file, destination) FileUtils.mkdir_p(destination)
file_path = "./convert_temp/*.zip"
destination = "./convert_temp/osz/"
extract_zip(file_path, destination)
until File.exists?( ".osu$" ) == false do
File.rename("./convert_temp/osz/*.osu$", "*.txt$")
File.foreach(filename) do |file|
file_string = File.read('./convert_temp/osz/*.txt$')
if file_string.include?('Mode: 1')
puts 'Yes'
else
puts 'No'
end
end
end
Using this kind of indentation makes it easy to see that there are some missing ends/parentheses, because the last line is left hanging in the air instead of closing back to the left edge where it started from.
Additional notes:
dialog.directory(
def extract_zip(file, destination) FileUtils.mkdir_p(destination)
It's very unconventional to define a new method inside a method call. File.open(def hello_world(..)) Doesn't make a lot of sense.
until File.exists?( ".osu$" ) == false do
Are you using $ as a way to indicate "filename ends in .osu"? If yes, it does not work like that. This would look for a file that has .osu$ as name.
File.foreach(filename) do |file|
The file parameter is not used in the block that follows, you use file_string.
file_string = File.read('./convert_temp/osz/*.txt$')
You can't read multiple files at once like that. Also, File.foreach above would read the file line by line, so here you are trying to read it again, inside the loop that is reading it already.
I am using rspec to do some data driven testing. My test reads from a csv file, grabs an entry which is inserted into the text box on the page and is then compared to expected text which is also read from the csv file. All this is working as expected, I am able to read and compare without any issues.
Below is my code:
Method for reading csv file:
def user_data
user_data = CSV.read Dir.pwd + '/user_data.csv'
descriptor = user_data.shift
descriptor = descriptor.map { |key| key.to_sym }
user_data.map { |user| Hash[ descriptor.zip(user) ] }
end
Test:
describe "Text box tests" do
before :all do
#homepage = Homepage.new
end
it "should display the correct name" do
visit('http://my test url')
sleep 2
user_data.each do |entry|
#homepage.enter_name(entry[:name])
#homepage.click_go()
sleep 2
begin
expect(page).to have_css("#firstname", text: entry[:expected_name])
end
end
end
end
The problem is with failures. If I have a failure with one of the tests (i.e the expected text is not displayed on the page) then the test stops and all subsequent entries in the csv are not tested. If I put in a rescue after the expect statement like this:
rescue Exception => error
puts error.message
Then the error is logged to the console, however at the end of my test run it says no failures.
So basically I am looking for is, in the event of a failure for my test to keep running(until all entries in the csv have been covered), but for the test run to be marked as failed. Does anyone know how I can achieve this?
Try something like this:
context "when the user is on some page" do
before(:context) { visit('http://example.org/') }
user_data.each do |entry|
it "should display the correct name: #{entry[:name]}" do
#homepage.enter_name(entry[:name])
#homepage.click_go
expect(page).to have_css("#firstname", text: entry[:expected_name])
end
end
end
You will also need to change def user_data to def self.user_data
I would advise mapping over the entries and calling the regular capybara method has_css? instead of the rspec helper method. It would look like this:
results = user_data.map do |entry|
#homepage.enter_name(entry[:name])
#homepage.click_go()
sleep 2
page.has_css?("#firstname", text: entry[:expected_name])
end
expect(results.all?) to be_truthy
if you want to keep track of which ones failed, you cann modify it a bit:
missing_entries = []
user_data.each do |entry|
#homepage.enter_name(entry[:name])
#homepage.click_go()
sleep 2
has_entry = page.has_css?("#firstname", text: entry[:expected_name])
unless has_entry
missing_entries.push entry[:expected_name]
end
end
expect(missing_entries).to be_empty
I often find myself dealing with these kind of scenarios:
require 'nokogiri'
require "open-uri"
url = "https://www.random_website.com/contains_info_I_want_to_parse"
nokodoc = Nokogiri::HTML(open(url))
# Let's say one of the following line breaks the ruby script
# because the element I'm searching doesn't contain an attribute.
a = nokodoc.search('#element-1').attribute('href').text
b = nokodoc.search('#element-2').attribute('href').text.gsub("a", "A")
c = nokodoc.search('#element-3 h1').attribute('style').text.strip
What happens is that I'll be creating about 30 variables all searching for different elements in a page, and I'll be looping that code over multiple pages. However, a few of these pages may have an ever-so-slightly different layout and won't have one of those div. This will break my code (because you can't call .attribute or .gsub on nil for example). But I can never guess which line before-hand.
My go-to solution is usually surround each line with:
begin
line #n
rescue
puts "line #n caused an error"
end
I'd like to be able to do something like:
url = "https://www.random_website.com/contains_info_I_want_to_parse"
nokodoc = Nokogiri::HTML(open(url))
catch_error(a, nokodoc.search('#element-1').attribute('href').text)
catch_error(b, nokodoc.search('#element-2').attribute('href').text.gsub("a", "A"))
catch_error(c, nokodoc.search('#element-3 h1').attribute('style').text.strip)
def catch_error(variable_name, code)
begin
variable_name = code
rescue
puts "Code in #{variable_name} caused an error"
end
variable_name
end
I know that putting & before each new method works:
nokodoc.search('#element-1')&.attribute('href')&.text
But I want to be able to display the error with a 'puts' in my terminal to see when my code gives an error.
Is it possible?
You can't pass your code as a regular argument to a method because it'll be evaluated (and raise an exception) before it gets passed to your catch_error method. You could pass it as a block--something like
a = catch_error('element_1 href text') do
nokodoc.search('#element-1').attribute('href').text
end
def catch_error(error_description)
yield
rescue
puts "#{error_description} caused an error"
end
Note that you can't pass a to the method as variable_name: it hasn't been defined anywhere before calling that method, so you'll get an undefined local variable or method error. Even if you define a earlier, it won't work correctly. If your code works without raising an exception, the method will return the right value but the value won't get stored anywhere outside the method scope. If there is an exception, variable_name will have whatever value a had before the method (nil if you defined it without setting it), so your error message would output something like Code in caused an error. That's why I added an error_description parameter.
You could also try logging the message and backtrace if you didn't want to have to specify an error description every time.
a = catch_error(nokodoc) do |doc|
doc.search('#element-1').attribute('href').text
end
def catch_error(doc)
yield doc
rescue => ex
puts doc.title # Or something else that identifies the document
puts ex.message
puts ex.backtrace.join("\n")
end
I made one additional change here: passing the document in as a parameter so that rescue could easily log something that identifies the document, in case that's important.
I'm writing a vim plugin using the ruby interface.
When I execute VIM::command(...), how can I detect if vim raised an error during execution of this command, so that I can skip further commands and also present a better message to the user?
Vim's global variable v:errmsg will give you the last error. If you want to check whether an error occured, you can first set it to an empty string and then check for it:
let v:errmsg = ""
" issue your command
if v:errmsg != ""
" handle the error
endif;
I'll leave it up to you to transfer this to the Ruby API. Also see :h v:errmsg from inside Vim. Other useful global variables may be:
v:exception
v:throwpoint
Edit – this should work (caution: some magic involved):
module VIM
class Error < StandardError; end
class << self
def command_with_error *args
command('let v:errmsg=""')
command(*args)
msg = evaluate('v:errmsg')
raise ::VIM::Error, msg unless msg.empty?
end
end
end
# Usage
# use sil[ent]! or the error will bubble up to Vim
begin
VIM::command_with_error('sil! foobar')
rescue VIM::Error => e
puts 'Rescued from: ' + e.message;
end
# Output
Rescued from: E492: Not an editor command: sil! foobar
When I get exceptions, it is often from deep within the call stack. When this happens, more often than not, the actual offending line of code is hidden from me:
tmp.rb:7:in `t': undefined method `bar' for nil:NilClass (NoMethodError)
from tmp.rb:10:in `s'
from tmp.rb:13:in `r'
from tmp.rb:16:in `q'
from tmp.rb:19:in `p'
from tmp.rb:22:in `o'
from tmp.rb:25:in `n'
from tmp.rb:28:in `m'
from tmp.rb:31:in `l'
... 8 levels...
from tmp.rb:58:in `c'
from tmp.rb:61:in `b'
from tmp.rb:64:in `a'
from tmp.rb:67
That "... 8 levels..." truncation is causing me a great deal of trouble. I'm not having much success googling for this one: How do I tell ruby that I want dumps to include the full stack?
Exception#backtrace has the entire stack in it:
def do_division_by_zero; 5 / 0; end
begin
do_division_by_zero
rescue => exception
puts exception.backtrace
raise # always reraise
end
(Inspired by Peter Cooper's Ruby Inside blog)
You could also do this if you'd like a simple one-liner:
puts caller
This produces the error description and nice clean, indented stacktrace:
begin
# Some exception throwing code
rescue => e
puts "Error during processing: #{$!}"
puts "Backtrace:\n\t#{e.backtrace.join("\n\t")}"
end
IRB has a setting for this awful "feature", which you can customize.
Create a file called ~/.irbrc that includes the following line:
IRB.conf[:BACK_TRACE_LIMIT] = 100
This will allow you to see 100 stack frames in irb, at least. I haven't been able to find an equivalent setting for the non-interactive runtime.
Detailed information about IRB customization can be found in the Pickaxe book.
One liner for callstack:
begin; Whatever.you.want; rescue => e; puts e.message; puts; puts e.backtrace; end
One liner for callstack without all the gems's:
begin; Whatever.you.want; rescue => e; puts e.message; puts; puts e.backtrace.grep_v(/\/gems\//); end
One liner for callstack without all the gems's and relative to current directory
begin; Whatever.you.want; rescue => e; puts e.message; puts; puts e.backtrace.grep_v(/\/gems\//).map { |l| l.gsub(`pwd`.strip + '/', '') }; end
This mimics the official Ruby trace, if that's important to you.
begin
0/0 # or some other nonsense
rescue => e
puts e.backtrace.join("\n\t")
.sub("\n\t", ": #{e}#{e.class ? " (#{e.class})" : ''}\n\t")
end
Amusingly, it doesn't handle 'unhandled exception' properly, reporting it as 'RuntimeError', but the location is correct.
Almost everybody answered this. My version of printing any rails exception into logs would be:
begin
some_statement
rescue => e
puts "Exception Occurred #{e}. Message: #{e.message}. Backtrace: \n #{e.backtrace.join("\n")}"
Rails.logger.error "Exception Occurred #{e}. Message: #{e.message}. Backtrace: \n #{e.backtrace.join("\n")}"
end
I was getting these errors when trying to load my test environment (via rake test or autotest) and the IRB suggestions didn't help. I ended up wrapping my entire test/test_helper.rb in a begin/rescue block and that fixed things up.
begin
class ActiveSupport::TestCase
#awesome stuff
end
rescue => e
puts e.backtrace
end
[examine all threads backtraces to find the culprit]
Even fully expanded call stack can still hide the actual offending line of code from you when you use more than one thread!
Example: One thread is iterating ruby Hash, other thread is trying to modify it. BOOM! Exception! And the problem with the stack trace you get while trying to modify 'busy' hash is that it shows you chain of functions down to the place where you're trying to modify hash, but it does NOT show who's currently iterating it in parallel (who owns it)! Here's the way to figure that out by printing stack trace for ALL currently running threads. Here's how you do this:
# This solution was found in comment by #thedarkone on https://github.com/rails/rails/issues/24627
rescue Object => boom
thread_count = 0
Thread.list.each do |t|
thread_count += 1
err_msg += "--- thread #{thread_count} of total #{Thread.list.size} #{t.object_id} backtrace begin \n"
# Lets see if we are able to pin down the culprit
# by collecting backtrace for all existing threads:
err_msg += t.backtrace.join("\n")
err_msg += "\n---thread #{thread_count} of total #{Thread.list.size} #{t.object_id} backtrace end \n"
end
# and just print it somewhere you like:
$stderr.puts(err_msg)
raise # always reraise
end
The above code snippet is useful even just for educational purposes as it can show you (like x-ray) how many threads you actually have (versus how many you thought you have - quite often those two are different numbers ;)
You can also use backtrace Ruby gem (I'm the author):
require 'backtrace'
begin
# do something dangerous
rescue StandardError => e
puts Backtrace.new(e)
end