Crystal debugging with GDB - debugging

I am trying to learn to debug programs written in Crystal with GDB. Here is a sample:
class Demo
#array = [] of String
def bar(url)
ret = url.downcase * 2
if ret == "alsj"
return false
else
return ret
end
end
def do(foo)
#array.push(foo)
html = bar(foo)
puts "HI" # GDB breakpoint here
return html
end
end
a = Demo.new
puts a.do("HI")
I compiled the sample above with the --debug flag and loaded it into GDB. Then I let it run and stopped at the marked line (GDB breakpoint here). Now I have three four questions:
Printing string values (e.g. foo):
When I inspect a string variable, I often see something like $1 = (struct String *) 0x4b9f18. When I say printf "%s", foo, I get nothing back. How can I display the current value of a string variable?
optimized out.
Other times I just see $1 = <optimized out> when inspecting a variable. What does that mean and how can I see the value in that case?
Accessing object variables
How can I see the value of #array in the given situation? p array says no symbol in current context and p #array returns unknown address space modifier.
Edit: I found a way: use p self.array
Vanished variables
In the given situation (breaking at the line puts "HI") I can't see the variable html at all: p html returns no symbol in current context. Why is that and how do I solve it?

Crystal debugging capabilities are still on development so you can't see some symbols or symbols data is optimized by LLVM. About optimized output, sometimes crystal algorithms are optimized too much, even on --debug builds. By example:
Currently yield methods are inlined and optimized, so this isn't very debuggable.
3.times do |i|
pp i
end
However, you can use pp keyword to print name => value.
pp i
i => 1
i => 2
i => 3
Also you can set breakpoints using debugger keyword and inspect macro output using {% debug() %} or $ crystal tool expand command.
On the other hand compound statement are easy to debug using tools like GDB.
i = 0
while i < 3
debugger
i += 1 # i variable is listed by GDB
end
Finally you can try some tricks I found in my debugging daily tasks.
Printing string values: try #[NoInline] and p &foo.c this will enable args data and you will be able to print all struct string value
Using #[NoInline] attribute:
#[NoInline]
def do(foo)
debugger
end
On GDB:
(gdb) p &foo.c
$1 = (UInt8 *) 0x10008e6c4 "HI"
optimized out: maybe because LLVM optimizes some method calls. if you see <optimized out> use #[NoInline] and try assigning instance vars to local vars array = #array
Accessing object variables: use self.var for instance vars.
Also use p array.buffer[0]#size to print array values.
(gdb) p &array.buffer[0].c
$19 = (UInt8 *) 0x10008e7f4 "HI"
Vanished variables: this happens because debugging info is not complete enough.
Try adding debug info manually converting or casting values:
#[NoInline]
def do(foo)
html = bar(foo).as(String)
html = bar(foo).to_s
debugger
end
Now html var is visible on GDB thanks to .as or .to_s methods
(gdb) p &html.c
$1 = (UInt8 *) 0x1002fcfec "hihi"

Related

Is there an equivalent of shell scripting's xtrace option for Ruby?

When debugging shell scripts, I find it helpful to run with xtrace on:
-x xtrace Print commands and parameter
assignments when they are exe-
cuted, preceded by the value
of PS4.
For instance:
$ set -x
$ s='Say again?'
+ s='Say again?'
# Other commands that might mess with the value of $s
$ echo $s
+ echo Say 'again?'
Say again?
I know that Ruby has interactive debuggers such as pry and byebug, but I'm looking for something that will be easy to turn on for logging automated scripts.
I did find an xtrace gem, but it has something to do with a PHP format.
I also see there is a Tracer class and a TracePoint class which do seem to provide a way to print statements as they are executed. But I haven't found any way to print the value of variables (rather than just the variable name):
$ ruby -r tracer trace.rb
#0:/usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:Kernel:<: return gem_original_require(path)
#0:trace.rb:1::-: s='Say again?'
#0:trace.rb:2::-: puts s
Say again?
I'd like to have the penultimate line read:
#0:trace.rb:2::-: puts 'Say again?'
Is this possible? Or is there a better way with Ruby?
I was able to build a module that more or less does what I'm looking for:
require 'pry'
=begin
We are always one line behind because the value of assignment comes
_after_ the trace is called. Without this, every assignment would look
like:
x = 1 #=> {:x=>nil}
It would be nice if the trace happened after assignment, but what can
you do?
=end
module Xtrace
# Only run the trace on the main file, not on require'd files.
# Possible room for expansion later.
#files = {$0 => Pry::Code.from_file($0)}
def Xtrace.print_trace
if #prev_line then
if #files[#path] then
line = #files[#path].around(#prev_line, 0).chomp
# When calling a method, don't make it look like it's being defined.
line.gsub!(/^\s*def\s*\b/, '') if #event == :call
values = []
#bind.local_variables.each do |v|
values << {v => #bind.local_variable_get(v)} if line =~ /\b#{v}\b/
end
STDERR.printf "%5s: %s", #prev_line, line
STDERR.printf " #=> %s", values.join(', ') unless values.empty?
STDERR.printf "\n"
end
end
end
#xtrace = TracePoint.trace(:line, :call) do |tp|
tp.disable
#bind=tp.binding
Xtrace.print_trace
# Other than the binding, everything we need to print comes from the
# previous trace call.
#prev_line = tp.lineno
#event=tp.event
#path=tp.path
tp.enable
end
# Need to print the trace one last time after the last line of code.
at_exit do
# It doesn't matter what we do in this last line. Any statement works.
# Also, it's a bit inconvenient that the disable command is itself traced.
#xtrace.disable
end
end
If you put it in a file named xtrace.rb and put in in your library load path, you can begin tracing by adding require 'xtrace'. It prints the line number of each line and method call executed, the actual code and the values of any local variable in the line. For a simple factorial function, the output might look like:
3: def factorial(n)
8: puts factorial(3)
3: factorial(n) #=> {:n=>3}
4: return 1 if n <= 1 #=> {:n=>3}
5: return n*factorial(n-1) #=> {:n=>2}
3: factorial(n) #=> {:n=>2}
4: return 1 if n <= 1 #=> {:n=>2}
5: return n*factorial(n-1) #=> {:n=>1}
3: factorial(n) #=> {:n=>1}
4: return 1 if n <= 1
6
For the moment, it only looks at local variables. It also only traces the executed file and not any loaded files. There's no way to enable or disable traces just yet. The trace begins when you require the module and ends when the execution does. Trace output goes to STDERR and the format is hardcoded.
If you use this module, watch out that you don't leak sensitive information such as passwords, API keys or PII.

Rubocop rule: Never use 'do' with multi-line 'while

I have the following code
# colours a random cell with a correct colour
def colour_random!
while true do
col, row = rand(columns), rand(rows)
cell = self[row,col]
if cell.empty? then
cell.should_be_filled? ? cell.colour!(1) : cell.colour!(0)
break
end
end
end
it's not that important what's doing, although it should pretty obvious. The point is that Rubocop gives me a warning
Never use 'do' with multi-line 'while
Why should I not do that? How should I do it then?
while is a keyword,so you don't need to pass a block. Without do..end it will work fine. The below is fine
def colour_random!
while true
col, row = rand(columns), rand(rows)
cell = self[row,col]
if cell.empty? then
cell.should_be_filled? ? cell.colour!(1) : cell.colour!(0)
break
end
end
end
while is a keyword, and if you pass a block to it, like do..end, it still works as you asked it to do, by not throwing any error, rather just a warning. But it could be dangerous if you try to pass a Proc or Method object to it, and dynamically try to convert it to a block using & keyword, as we do generally. That means
# below code will work as expected just throwing an warning.
x = 2
while x < 2 do
#code
end
But if you try to do by mistake like below
while &block # booom!! error
The reason is while is a keyword, which don't support any to_proc method to satisfy your need. So it can be dangerous.
Ruby style guide also suggested that Never use while/until condition do for multi-line while/until
I think the reason is as Nobuyoshi Nakada said in the mailing list
loop is a kernel method which takes a block. A block introduces new local variable scope.
loop do
a = 1
break
end
p a #=> causes NameError
while doesn't.
while 1
a = 1
break
end
p a #=> 1
Ruby actually has a shortcut for while true: the loop statement.
def colour_random!
loop do
col, row = rand(columns), rand(rows)
cell = self[row,col]
if cell.empty? then
cell.should_be_filled? ? cell.colour!(1) : cell.colour!(0)
break
end
end
end

Using ruby-debug in for i in 0...5

I am learning ruby from 'Programming ruby 1.9'. I am learning to use the ruby-debug so I can understand what is going on underneath. I use rubymine since it integrates ruby-debug19 or something like that (it says I don't have the gem and installs it). Here is the question, I was able to step through the code and explore the variables and the stack. However, when it reaches a for i in 0...5, the debugger says
stack frame not available
I know that ruby don't use for loops much but I'd still like to know if there debug through for loops.
Code:
raw_text = %{
The problem breaks down into two parts. First, given some text as a
string, return a list of words. That sounds like an array. Then, build a
count for each distinct word. That sounds like a use for a hash---we can
index it with the word and use the corresponding entry to keep a count.}
word_list = words_from_string(raw_text)
counts = count_frequency(word_list)
sorted = counts.sort_by {|word, count| count}
top_five = sorted.last(5)
for i in 0...5 # (this is ugly code--read on
word = top_five[i][0] # for a better version)
count = top_five[i][1]
puts "#{word}: #{count}"
end
If you take a look at the Ruby Language Specification (clause 11.5.2.3.4 on p. 91), you will see that
for i in 0...5
word = top_five[i][0]
count = top_five[i][1]
puts "#{word}: #{count}"
end
is syntactic sugar for
(0...5).each do |i|
word = top_five[i][0]
count = top_five[i][1]
puts "#{word}: #{count}"
end
except that no new variable scope is created for the block. So, the code with for will be translated into the code with each and executed as if it were written that way, except that the variables used in the for loop leak into the surrounding scope.
To put it another way: for actually executes each but without allocating a new stack frame for the block. So, the error message is exactly right: there is a call to a block, but somehow there is no stack frame allocated for that block call. That obviously confuses the debugger.
Now, one might argue that this is a bug and that for loops should get special treatment inside the debugger. I guess that so far nobody has ever bothered to fix that bug, since nobody ever uses for loops, precisely because they leak their variables into the surrounding scope and are exactly equivalent to an idiomatic each which doesn't.
What do I mean by "leaking variables"? See here:
(1..2).each do |i|
t = true
end
i
# NameError: undefined local variable or method `i' for main:Object
t
# NameError: undefined local variable or method `t' for main:Object
for i in 1..2
t = true
end
i
# => 2
t
# => true

Is there a ruby hook for the variable that was just allocated?

Here is what I'd ideally like. The user does:
a="hello"
and the output would be
You just allocated "a" !
=> "Hello"
Order is irrelevant as long as I can make that message happen.
No, there's no straight-forward way to make this happen as local variable names are discarded by the Ruby bytecode compiler before your code is executed.
The only instructions YARV (the Ruby VM in use in MRI 1.9.2) provides regarding local variables are getlocal and setlocal, which both operate on integer indicies rather than variable names. Here's an excerpt from insns.def in the 1.9.2 source:
/**********************************************************/
/* deal with variables */
/**********************************************************/
/**
#c variable
#e get local variable value (which is pointed by idx).
#j idx
*/
DEFINE_INSN
getlocal
(lindex_t idx)
()
(VALUE val)
{
val = *(GET_LFP() - idx);
}
/**
#c variable
#e set local variable value (which is pointed by idx) as val.
#j idx
*/
DEFINE_INSN
setlocal
(lindex_t idx)
(VALUE val)
()
{
(*(GET_LFP() - idx)) = val;
}
It might be possible to hack the MRI source (or use set_trace_func and dive into a Binding object - see sarnold's answer) to inform you when a local variable is set, but there isn't any high-level way of doing it and you probably won't be able to retrieve the names of those local variables without diving into the interpreter internals.
I've come up with a solution that is based on set_trace_func. I can't actually counter the limitations Charlie points out but I believe what I have written should work more or less as you described:
#!/usr/bin/ruby
def hash_from_binding(bin)
h = Hash.new
bin.eval("local_variables").each do |i|
v = bin.eval(i)
v && h[i]=bin.eval(i)
end
bin.eval("instance_variables").each do |i|
v = bin.eval(i)
v && h[i]=bin.eval(i)
end
h
end
$old_binding = hash_from_binding(binding)
$new_binding = hash_from_binding(binding)
set_trace_func lambda {|event, file, line, id, bin, classname|
$old_binding = $new_binding
$new_binding = hash_from_binding(bin)
diff = $new_binding.reject {|k, v| $old_binding[k] == $new_binding[k]}
printf("%d:\n", line)
# $old_binding.each do |k,v|
# printf("%8s: %s\n", k, v)
# end
# $new_binding.each do |k,v|
# printf("%8s: %s\n", k, v)
# end
diff.each do |k,v|
printf("%8s: %s\n", k, v)
end
}
a = "hello"
b = "world"
c = "there"
d = nil
e = false
#a = "HELLO"
#b = "WORLD"
A="Hello"
B="World"
def foo
foo_a = "foo"
#foo_b = "foo"
end
foo
hash_from_binding(bin) will turn a Binding object into a Hash. You can remove the instance_variables portion if you don't want those. You can remove the local_variables portion if you don't want those. The complication of v && h[i]=bin.eval(i) is due to an oddity in the Binding objects -- even though the tracing function hasn't yet "parsed" through all the content, the Binding object passed to the tracing function does know about all the variables that are going to be defined in the scope. It's awkward. This at least filters out the variables that haven't been assigned a value. By consequence it also filters out variables assigned values nil or false. You might be content with the reject action in the tracing function to do the filtering work for you.
The set_trace_func API will call a tracing method for every source line that is parsed. (This might be a drastic limitation in the face of different execution environments.) So I wrote a tracing function that will compare the old bindings object against a new bindings object and report the variable definitions that are changed. You could also report the variable definitions that are new, but that would miss cases like:
a = 1
a = 2
One funny consequence is that the bindings reported change drastically across function calls as new variables are brought to life and old variables are removed from the environment. This may overly confuse the output, but perhaps the event parameter may be of use in determining whether to print new variable values. (Since the function call might modify variable values in the scope of the "returnee" code, printing them all seems like the safe approach.)
When the tool is run upon itself, it outputs the following:
$ ./binding.rb
38:
39:
a: hello
40:
b: world
41:
c: there
42:
43:
44:
#a: HELLO
45:
#b: WORLD
46:
48:
48:
48:
53:
#a: HELLO
#b: WORLD
48:
49:
50:
foo_a: foo
50:
#foo_b: foo
$
This is the most complicated piece of Ruby code I've run this tool upon, so it might break on something non-trivial.

Ruby forgets local variables during a while loop?

I'm processing a record-based text file: so I'm looking for a starting string which constitutes the start of a record: there is no end-of-record marker, so I use the start of the next record to delimit the last record.
So I have built a simple program to do this, but I see something which suprises me: it looks like Ruby is forgetting local variables exist - or have I found a programming error ? [although I don't think I have : if I define the variable 'message' before my loop I don't see the error].
Here's a simplified example with example input data and error message in comments:
flag=false
# message=nil # this is will prevent the issue.
while line=gets do
if line =~/hello/ then
if flag==true then
puts "#{message}"
end
message=StringIO.new(line);
puts message
flag=true
else
message << line
end
end
# Input File example:
# hello this is a record
# this is also part of the same record
# hello this is a new record
# this is still record 2
# hello this is record 3 etc etc
#
# Error when running: [nb, first iteration is fine]
# <StringIO:0x2e845ac>
# hello
# test.rb:5: undefined local variable or method `message' for main:Object (NameError)
#
From the Ruby Programming Language:
(source: google.com)
Blocks and Variable Scope
Blocks define a new variable scope: variables created within a block exist only within that block and are undefined outside of the block. Be cautious, however; the local variables in a method are available to any blocks within that method. So if a block assigns a value to a variable that is already defined outside of the block, this does not create a new block-local variable but instead assigns a new value to the already-existing variable. Sometimes, this is exactly the behavior we want:
total = 0
data.each {|x| total += x } # Sum the elements of the data array
puts total # Print out that sum
Sometimes, however, we do not want to alter variables in the enclosing scope, but we do so inadvertently. This problem is a particular concern for block parameters in Ruby 1.8. In Ruby 1.8, if a block parameter shares the name of an existing variable, then invocations of the block simply assign a value to that existing variable rather than creating a new block-local variable. The following code, for example, is problematic because it uses the same identifier i as the block parameter for two nested blocks:
1.upto(10) do |i| # 10 rows
1.upto(10) do |i| # Each has 10 columns
print "#{i} " # Print column number
end
print " ==> Row #{i}\n" # Try to print row number, but get column number
end
Ruby 1.9 is different: block parameters are always local to their block, and invocations of the block never assign values to existing variables. If Ruby 1.9 is invoked with the -w flag, it will warn you if a block parameter has the same name as an existing variable. This helps you avoid writing code that runs differently in 1.8 and 1.9.
Ruby 1.9 is different in another important way, too. Block syntax has been extended to allow you to declare block-local variables that are guaranteed to be local, even if a variable by the same name already exists in the enclosing scope. To do this, follow the list of block parameters with a semicolon and a comma-separated list of block local variables. Here is an example:
x = y = 0 # local variables
1.upto(4) do |x;y| # x and y are local to block
# x and y "shadow" the outer variables
y = x + 1 # Use y as a scratch variable
puts y*y # Prints 4, 9, 16, 25
end
[x,y] # => [0,0]: block does not alter these
In this code, x is a block parameter: it gets a value when the block is invoked with yield. y is a block-local variable. It does not receive any value from a yield invocation, but it has the value nil until the block actually assigns some other value to it. The point of declaring these block-local variables is to guarantee that you will not inadvertently clobber the value of some existing variable. (This might happen if a block is cut-and-pasted from one method to another, for example.) If you invoke Ruby 1.9 with the -w option, it will warn you if a block-local variable shadows an existing variable.
Blocks can have more than one parameter and more than one local variable, of course. Here is a block with two parameters and three local variables:
hash.each {|key,value; i,j,k| ... }
Contrary to some of the other answers, while loops don't actually create a new scope. The problem you're seeing is more subtle.
Prerequisite knowledge: a brief scoping demo
To help show the contrast, blocks passed to a method call DO create a new scope, such that a newly assigned local variable inside the block disappears after the block exits:
### block example - provided for contrast only ###
[0].each {|e| blockvar = e }
p blockvar # NameError: undefined local variable or method
But while loops (like your case) are different, because a variable defined in the loop will persist:
arr = [0]
while arr.any?
whilevar = arr.shift
end
p whilevar # prints 0
Summary of the "problem"
The reason you get the error in your case is because the line that uses message:
puts "#{message}"
appears before any code that assigns message.
It's the same reason this code raises an error if a wasn't defined beforehand:
# Note the single (not double) equal sign.
# At first glance it looks like this should print '1',
# because the 'a' is assigned before (time-wise) the puts.
puts a if a = 1
Not Scoping, but Parsing-visibility
The so-called "problem" - i.e. local-variable visibility within a single scope - is due to ruby's parser. Since we're only considering a single scope, scoping rules have nothing to do with the problem. At the parsing stage, the parser decides at which source locations a local variable is visible, and this visibility does not change during execution.
When determining if a local variable is defined (i.e. defined? returns true) at any point in the code, the parser checks the current scope to see if any code has assigned it before, even if that code has never run (the parser can't know anything about what has or hasn't run at the parsing stage). "Before" meaning: on a line above, or on the same line and to the left-hand side.
An exercise to determine if a local is defined (i.e. visible)
Note that the following only applies to local variables, not methods. (Determining whether a method is defined in a scope is more complex, because it involves searching included modules and ancestor classes.)
A concrete way to see the local variable behavior is to open your file in a text editor. Suppose also that by repeatedly pressing the left-arrow key, you can move your cursor backward through the entire file. Now suppose you're wondering whether a certain usage of message will raise the NameError. To do this, position your cursor at the place you're using message, then keep pressing left-arrow until you either:
reach the beginning of the current scope (you must understand ruby's scoping rules in order to know when this happens)
reach code that assigns message
If you've reached an assignment before reaching the scope boundary, that means your usage of message won't raise NameError. If you don't reach any assignment, the usage will raise NameError.
Other considerations
In the case a variable assignment appears in the code but isn't run, the variable is initialized to nil:
# a is not defined before this
if false
# never executed, but makes the binding defined/visible to the else case
a = 1
else
p a # prints nil
end
While loop test case
Here's a small test case to demonstrate the oddness of the above behavior when it happens in a while loop. The affected variable here is dest_arr.
arr = [0,1]
while n = arr.shift
p( n: n, dest_arr_defined: (defined? dest_arr) )
if n == 0
dest_arr = [n]
else
dest_arr << n
p( dest_arr: dest_arr )
end
end
which outputs:
{:n=>0, :dest_arr_defined=>nil}
{:n=>1, :dest_arr_defined=>nil}
{:dest_arr=>[0, 1]}
The salient points:
The first iteration is intuitive, dest_arr is initialized to [0].
But we need to pay close attention in the second iteration (when n is 1):
At the beginning, dest_arr is undefined!
But when the code reaches the else case, dest_arr is visible again, because the interpreter sees that it was assigned 2 lines above.
Notice also, that dest_arr is only hidden at the start of the loop; its value is never lost (as evidenced by the final contents being [0, 1]).
This also explains why assigning your local before the while loop fixes the problem. The assignment doesn't need to be executed; it only needs to appear in the source code.
Lambda example
f1 = ->{ f2 }
f2 = ->{ f1 }
p f2.call()
# The following fails because the body of f1 tries to access f2 before an assignment for f2 was seen by the parser.
p f1.call() # undefined local variable or method `f2'.
Fix this by putting an f2 assignment before f1's body. Remember, the assignment doesn't actually need to be executed!
f2 = nil # Could be replaced by: if false; f2 = nil; end
f1 = ->{ f2 }
f2 = ->{ f1 }
p f2.call()
p f1.call() # ok
Method-masking gotcha
Things get really hairy if you have a local variable with the same name as a method:
def dest_arr
:whoops
end
arr = [0,1]
while n = arr.shift
p( n: n, dest_arr: dest_arr )
if n == 0
dest_arr = [n]
else
dest_arr << n
p( dest_arr: dest_arr )
end
end
Outputs:
{:n=>0, :dest_arr=>:whoops}
{:n=>1, :dest_arr=>:whoops}
{:dest_arr=>[0, 1]}
A local variable assignment in a scope will "mask"/"shadow" a method call of the same name. (You can still call the method by using explicit parentheses or an explicit receiver.) So this is similar to the previous while loop test, except that instead of becoming undefined above the assignment code, the dest_arr method becomes "unmasked"/"unshadowed" so that the method is callable w/o parentheses. But any code after the assignment will see the local variable.
Some best-practices we can derive from all this
Don't name local variables the same as method names in the same scope
Don't put the initial assignment of a local variable inside the body of a while or for loop, or anything that causes execution to jump around within a scope (calling lambdas or Continuation#call can do this too). Put the assignment before the loop.
I think this is because message is defined inside the loop. At the end of the loop iteration "message" goes out of scope. Defining "message" outside of the loop stops the variable from going out of scope at the end of each loop iteration. So I think you have the right answer.
You could output the value of message at the beginning of each loop iteration to test whether my suggestion is correct.
Why do you think this is a bug? The interpreter is telling you that message may be undefined when that particular piece of code executes.
I'm not sure why you're surprised: on line 5 (assuming that the message = nil line isn't there), you potentially use a variable that the interpreter has never heard of before. The interpreter says "What's message? It's not a method I know, it's not a variable I know, it's not a keyword..." and then you get an error message.
Here's a simpler example to show you what I mean:
while line = gets do
if line =~ /./ then
puts message # How could this work?
message = line
end
end
Which gives:
telemachus ~ $ ruby test.rb < huh
test.rb:3:in `<main>': undefined local variable or method `message' for main:Object (NameError)
Also, if you want to prepare the way for message, I would initialize it as message = '', so that it's a string (rather than nil). Otherwise, if your first line doesn't match hello, you end up tring to add line to nil - which will get you this error:
telemachus ~ $ ruby test.rb < huh
test.rb:4:in `<main>': undefined method `<<' for nil:NilClass (NoMethodError)
you can simply do this :
message=''
while line=gets do
if line =~/hello/ then
# begin a new record
p message unless message == ''
message = String.new(line)
else
message << line
end
end
# hello this is a record
# this is also part of the same record
# hello this is a new record
# this is still record 2
# hello this is record 3 etc etc

Resources