Does Elixir infinite recursion ever overflow the stack? - stack-overflow

A number of different how-tos on Elixir programming express the view that storing state or running an infinite loop is done idiomatically either by spinning the data off into an Agent or Task, or by infinite recursion of the function that needs state. They don't mention any limits on how deep the recursion can go or any other caveats.
Since searching for "Elixir stack overflow" just results in hits to this website, let me remove the ambiguity and ask here: What implementation guarantees are there in Elixir to make sure that infinite recursion as a method of 'looping' won't result in a stack overflow, especially when state information is being carried around along the way?

To summarize excellent comments by Hristo, the general mechanism is called "Tail Call Optimization" (TCO) and it ensures that if the last thing a function does is invocation of another function (or itself), then there won't be stack push. Instead, a simple jump will occur.
There are some subtle nuances as to what is a tail call. Let's see a few example. The simplest one is:
def foo do
# ...
bar(...) # tail call -> nothing is pushed to the stack
end
TCO will also apply for conditional expressions:
def foo do
# ...
if (...) do
# ...
bar(...) # tail call
else
# ...
baz(...) # tail call
end
end
This works because again the last thing a function does is an invocation of a function. The result of if is the result of either bar or baz so there's no need to push anything onto stack.
In contrast, if the caller function does something after calling another function, it's not a tail call, and TCO won't happen:
def foo do
# ...
# Not a tail call since we're doing something after bar returns
# (increment the result by 1)
1 + bar(...)
end
Another example of breaking TCO is calling the function in try:
def foo do
try do
bar(...) # not a tail call
rescue
# ...
end
end
It's also worth mentioning that due to TCO you won't see some functions in the stack trace when an exception occurs:
def foo do
# ...
bar(...) # at this point foo "disappears" from stack trace
end
def bar(...) do
# ...
raise("error")
end
The stack dump of this error won't include foo since it is not on the stack anymore (it is effectively replaced with bar).

Related

Evaluate argument at call site

This is somewhat of an esoteric situation that I am not even sure is possible. I have a method that is called by a lot of methods:
def called_by_a_lot_of_methods
# Work
end
I am introducing a feature in which the called_by_a_lot_of_methods will do stuff depends on the caller method, and my current solution is to change all these methods to pass their names when calling called_by_a_lot_of_methods:
def some_method
called_by_a_lot_of_methods(method_name: some_method)
end
Or, alternatively:
def some_method
called_by_a_lot_of_methods(method_name: __method__)
end
But this is becoming tedious, and I was wondering if I can give the method_name parameter a default value:
def called_by_a_lot_of_methods(method_name: __method__)
# Work
end
This does not work because __method__ evaluates immediately giving called_by_a_lot_of_methods, which is obviously not what I want. The question then is, is there a way in Ruby to defer the evaluation of the argument until a useful time when I know it should give the correct result, which is the outer caller method? Thus saving me from having to pass the argument everywhere?

Is it possible to create "SystemStackError: stack level too deep" errors without recursion?

Consider the following irb interaction:
2.1.1 :001 > def do_it
2.1.1 :002?> do_it
2.1.1 :003?> end
=> :do_it
2.1.1 :004 > do_it
SystemStackError: stack level too deep
In this example, does it detect the certainty of running out of stack or does it actually run out of stack? Is it possible to produce this error without using recursion?
In this example, does it detect the certainty of running out of stack or does it actually run out of stack?
It actually runs out of stack. It is, in fact, impossible to "detect the certainty of running out of stack" in the general case because of the halting problem, one of the core problems in computer science.
Is it possible to produce this error without using recursion?
Sure. Just define a lot of methods, each of which calls the next:
20000.times do |n|
define_method :"foo_#{n}" do
puts "Called #{__method__}"
send :"foo_#{n+1}"
end
end
foo_0
# -> Called foo_0
# -> Called foo_1
# -> Called foo_2
# -> Called foo_3
# ...
# -> Called foo_4931
# -> SystemStackError: stack level too deep
It does actually reach its maximum stack limit...
def do_it x=1
puts "Do it #{x}"
do_it x+1
end
do_it
Here's a reasonable explanation of Ruby's stack limitations: How to increase stack size for a ruby app. Recursive app getting: Stack level too deep (SystemStackError)
Summary of Ruby stack details
courtesy #gregspurrier:
v1.8.x used C stack
v1.9.x used virtual memory with its own stack
v2.0.0 this VM limit can be set via RUBY_THREAD_VM_STACK_SIZE
It actually runs out of space on the stack.
Every method invocation needs some space on the stack. You could have code like this:
def method_1
method_2
end
def method_2
method_3
end
...
...
def method_123456
method_123457
end
method_1
There is no recursion involved but still it will run out of stack space at some point.

Shortest Ruby one-liner to execute a statement exactly once

I'm looking for the shortest, most simple Ruby one-liner to execute a statement exactly once. Idea is to use this while debugging to quickly add a debug statement to a loop that gets executed exactly once.
Best I've come up with so far:
puts "do something interesting exactly once!" if (once ||= "0").next! == "1"
Can you come up with something even shorter?
Added for clarification:
The idea for the questions was to focus on the "do it once" part and not so much on the "do something interesting" part. It should be assumed that the code do be executed once could be anything, not just a puts statement.
The ideal solution would also work in different kinds of loop constructs. E.g. as was pointed out my initial solution only works if the once variable is already defined outside the loop context or if the loop context used doesn't create a new lexical scope.
The original use case that triggered this question was slightly different - it looked more like below. But I though the above, simpler example would more easily explain the kind of solution I was looking for.
def process
do_some_preprocessing()
raise SomeError if <once> # added temp. for debugging purposes - the <once> part is what this question is about!
dangerous_operation() # this can raise SomeError under certain conditions
rescue SomeError
attempt_to_rescue() and retry
end
Well, you could abuse lambdas and closures.
->{puts "do something interesting exactly once!";x=->{x}}[][]
#=> do something interesting exactly once!
#=> #<Proc:0x5465282c#(irb):10 (lambda)>
The original contents of the lambda are only run once; any subsequent invocations will simply return an empty proc.
You could alternately abuse globals for a more true "one-liner", but it's awful.
$x ||= puts("do something interesting exactly once!") || 1
debug = ["do something interesting exactly once!"]
puts debug.pop # "do something interesting exactly once!"
puts debug.pop # nil
(answer edited to reflect the discussion in comments)
Your code won't do what you want it to do, it will depend of the looping construct you use.
This will work:
puts "do something interesting exactly once!" if once = once.nil?
But with this one, you'll have to define once before: once = nil (same thing for your own code). This is because otherwise, the scope of the once variable will be restrained to the block within an each loop, causing it to fail. This would work just fine within a for loop (the way you must have tested it):
(1..3).each do # 3.times would behave just the same
puts "once has not been defined before! Won't work!" if once = once.nil?
end
# >once has not been defined before! Won't work!
# once has not been defined before! Won't work!
# once has not been defined before! Won't work!
for i in 1..3 do
puts "Works because of the way for loops treat vars within loop" if once = once.nil?
end
# >Works because of the way for loops treat vars within loop
To avoid that problem without having to initialize the variable first, you can make once global:
(1..3).each do
puts "$once's scope is not restrained to the 'each' loop! Works!" if $once = $once.nil?
end
# >$once's scope is not restrained to the 'each' loop! Works!
The original idea generates code-smell. It results in code that will leave someone else scratching their head, which isn't a good thing. Generating code that is obvious and easy to understand will make your, and other programmer's, job easier.
Writing code that takes a while to figure out will take you a while to figure out in the future if you're debugging so be kind to your future self.
I'd stick with the standard way, using a simple flag:
once = false
2.times do
puts "do something interesting exactly once!" unless once
once ||= true
end
Which results in this output:
# >> do something interesting exactly once!

Ruby 1.9.3: Recieving Stack Level Too Deep errors when creating a new object

I'm not exactly an expert in Ruby but I'm attempting to trace the issue in some scripts and haven't been able to reach the expert on this particular script to get an idea of why we're receiving this error. I've narrowed things down to this particular function and from what I can tell, the first print statement happens but the end print statement does not. The stack level too deep error only happens when this function exists exactly as below:
def load_power_ports(io_info)
return if !io_info
io_info.each_key do |key|
print key
if !#power_controllers[key.to_s.downcase]
#power_controllers[key.to_s.downcase] = Object.const_get($equipment_table['power_controller'][key.to_s.downcase][0].driver_class_name).new($equipment_table['power_controller'][key.to_s.downcase][0])
end
end
print "end of equipment power block"
rescue Exception => e
raise e.to_s + "\nUnable to create power controller: " + io_info.to_s
end
The print statements are only to see which information is being passed, how far the program is getting and iterations being performed.
Does the constructor of driver_class_name call load_power_ports directly or indirectly? If so, you'd end up having infinite recursion which would give you your stack overflow (oh the irony). That's the only place I can see where you'd likely have that issue.

How to get argument names using reflection

I would like to do some fairly heavy-duty reflection in Ruby. I want to create a function that returns the names of the arguments of various calling functions higher up the call stack (just one higher would be enough but why stop there?). I could use Kernel.caller, go to the file and parse the argument list but that would be ugly and unreliable.
The function that I would like would work in the following way:
module A
def method1( tuti, fruity)
foo
end
def method2(bim, bam, boom)
foo
end
def foo
print caller_args[1].join(",") #the "1" mean one step up the call stack
end
end
A.method1
#prints "tuti,fruity"
A.method2
#prints "bim, bam, boom"
I would not mind using ParseTree or some similar tool for this task but looking at Parsetree, it is not obvious how to use it for this purpose. Creating a C extension like this is another possibility but it would be nice if someone had already done it for me.
I can see that I'll probably need some kind of C extension. I suppose that means my question is what combination of C extension would work most easily. I don't think caller+ParseTree would be enough by themselves.
As far as why I would like to do this goes, rather than saying "automatic debugging", perhaps I should say that I would like to use this functionality to do automatic checking of the calling and return conditions of functions:
def add x, y
check_positive
return x + y
end
Where check_positive would throw an exception if x and y weren't positive. Obviously, there would be more to it than that but hopefully this gives enough motivation.
In Ruby 1.9.2, you can trivially get the parameter list of any Proc (and thus of course also of any Method or UnboundMethod) with Proc#parameters:
A.instance_method(:method1).parameters # => [[:req, :tuti], [:req, :fruity]]
The format is an array of pairs of symbols: type (required, optional, rest, block) and name.
For the format you want, try
A.instance_method(:method1).parameters.map(&:last).map(&:to_s)
# => ['tuti', 'fruity']
Of course, that still doesn't give you access to the caller, though.
I suggest you take a look at Merb's action-args library.
require 'rubygems'
require 'merb'
include GetArgs
def foo(bar, zed=42)
end
method(:foo).get_args # => [[[:bar], [:zed, 42]], [:zed]]
If you don't want to depend on Merb, you can choose and pick the best parts from the source code in github.
I have a method that is quite expensive and only almost works.
$shadow_stack = []
set_trace_func( lambda {
|event, file, line, id, binding, classname|
if event == "call"
$shadow_stack.push( eval("local_variables", binding) )
elsif event == "return"
$shadow_stack.pop
end
} )
def method1( tuti, fruity )
foo
end
def method2(bim, bam, boom)
foo
x = 10
y = 3
end
def foo
puts $shadow_stack[-2].join(", ")
end
method1(1,2)
method2(3,4,4)
Outputs
tuti, fruity
bim, bam, boom, x, y
I'm curious as to why you'd want such functionality in such a generalized manner.
I'm curious how you think this functionality would allow for automatic debugging? You'd still need to inject calls to your "foo" function. In fact, something based on set_trace_func is more able to be automatic, as you don't need to touch existing code. Indeed this is how debug.rb is implemented, in terms of set_trace_func.
The solutions to your precise question are indeed basically, as you outlined. use caller + parsetree, or open the file and grab the data that way. There is no reflection capability that I am aware of that will let you get the names of arguments. You can approve upon my solution by grabbing the associated method object and calling #arity to then infer what of local_variables are arguments, but though it appears the result of that function is ordered, I'm not sure it is safe to rely on that. If you don't mind me asking, once you have the data and the interface you describe, what are you going to do with it? Automatic debugging was not what initially came to mind when I imagined uses for this functionality, although perhaps it is failing of imagination on my part.
Aha!
I would approach this differently then. There are several ruby libraries for doing design by contract already, including ruby-contract, rdbc, etc.
Another option is to write something like:
def positive
lambda { |x| x >= 0 }
end
def any
lambda { |x| true }
end
class Module
def define_checked_method(name, *checkers, &body)
define_method(name) do |*args|
unless checkers.zip(args).all? { |check, arg| check[arg] }
raise "bad argument"
end
body.call(*args)
end
end
end
class A
define_checked_method(:add, positive, any) do |x, y|
x + y
end
end
a = A.new
p a.add(3, 2)
p a.add(3, -1)
p a.add(-4, 2)
Outputs
5
2
checked_rb.rb:13:in `add': bad argument (RuntimeError)
from checked_rb.rb:29
Of course this can be made much more sophisticated, and indeed that's some of what the libraries I mentioned provided, but perhaps this is a way to get you where you want to go without necessarily taking the path you planned to use to get there?
if you want the value for the default values, too, there's the "arguments" gem
$ gem install rdp-arguments
$ irb
>> require 'arguments'
>> require 'test.rb' # class A is defined here
>> Arguments.names(A, :go)
In fact, the method you describe clearly fails to distinguish arguments from local variables while also failing to work automatically
That's because what you're trying to do is not something which is supported. It's possible (everything is possible in ruby), but there's no documented or known way to do it.
Either you can eval the backtrace like what logan suggested, or you can bust out your C compiler and hack sourcecode for ruby. I'm reasonably confident there aren't any other ways to do this.

Resources