My script gets elements from redis database end doing some work with it. I need to be sure that if script finishes with ^C or other Signal, the element will be returned back in the database.
I'm trying to do it
require "redis"
class Test
attr_reader :quit
def initialize
#redis = Redis.new
end
def trap_signal
trap("INT") {
puts "get ready to exit"
#redis.rpush "TestQueue", #elem # I need to be sure that #emelent always puts back in the database
#quit = true}
end
def run!
trap_signal
#elem = "test string"
#redis.rpush "TestQueue", #elem
while !quit
#redis.blpop "TestQueue", #elem
# Do some work whith #element
sleep 1
# And put it back in the database
#redis.rpush "TestQueue", #elem
end
end
end
Test.new.run!
but get this error
^Cget ready to exit
/usr/lib/ruby/2.1.0/monitor.rb:185:in `lock': can't be called from trap context (ThreadError)
from /usr/lib/ruby/2.1.0/monitor.rb:185:in `mon_enter'
from /usr/lib/ruby/2.1.0/monitor.rb:209:in `mon_synchronize'
from /home/kusayu/.gem/ruby/2.1.0/gems/redis-3.2.0/lib/redis.rb:37:in `synchronize'
from /home/kusayu/.gem/ruby/2.1.0/gems/redis-3.2.0/lib/redis.rb:991:in `rpush'
from test.rb:13:in `block in trap_signal'
from test.rb:24:in `call'
from test.rb:24:in `sleep'
from test.rb:24:in `run!'
from test.rb:32:in `<main>'
Your code already works properly, just remove the #redis.rpush from the signal handler.
You shouldn't run "heavy" operations in the signal handler (and you get an exception because of it anyway). It's much better to use a variable like #quit = true to signal the main loop that it's time to finish, and then let the main loop handle the proper cleanup.
So if you just remove the #redis.rpush from your INT signal handler, then you will ensure that the element is returned back into the database because the main loop will only finish once #quit is true.
Related
I have created a class which I want to hang on to a file descriptor and close it when the instance is GC-ed.
I created a class that looks something like this:
class DataWriter
def initialize(file)
# open file
#file = File.open(file, 'wb')
# create destructor
ObjectSpace.define_finalizer(self, self.class.finalize(#file))
end
# write
def write(line)
#file.puts(line)
#file.flush
end
# close file descriptor, note, important that it is a class method
def self.finalize(file)
proc { file.close; p "file closed"; p file.inspect}
end
end
I then tried to test the destructor method like so:
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
data_writer = DataWriter.new('/tmp/example.txt')
expect(DataWriter).to receive(:finalize)
data_writer = nil
GC.start
end
end
end
When running this test, even though the "file closed" is printed along with the file.inspect, the test fails with the following output:
1) DataWriter it should call its destructor calls the destructor
Failure/Error: expect(DataWriter).to receive(:finalize)
(DataWriter (class)).finalize(*(any args))
expected: 1 time with any arguments
received: 0 times with any arguments
# ./spec/utils/data_writer_spec.rb:23:in `block (3 levels) in <top (required)>'
finalize is called in initialize, returns the proc, and is never called again, so you can't expect it to be called at finalization time. It's the proc that's called when the instance is finalized. To check that, have the proc call a method instead of doing the work itself. This passes:
class DataWriter
# initialize and write same as above
def self.finalize(file)
proc { actually_finalize file }
end
def self.actually_finalize(file)
file.close
end
end
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
data_writer = DataWriter.new('/tmp/example.txt')
expect(DataWriter).to receive(:actually_finalize)
data_writer = nil
GC.start
end
end
end
even though the "file closed" is printed along with the file.inspect, the test fails with the following output
I threw your code into a single file and ran it. It appears that the finalize code isn't being cleaned up until rspec exits given the output I'm receiving:
Failures:
F
1) DataWriter it should call its destructor calls the destructor
Failure/Error: expect(DataWriter).to receive(:finalize)
(DataWriter (class)).finalize(*(any args))
expected: 1 time with any arguments
received: 0 times with any arguments
# /scratch/data_writer.rb:27:in `block (3 levels) in <top (required)>'
Finished in 0.01066 seconds (files took 0.16847 seconds to load)
1 example, 1 failure
Failed examples:
rspec /scratch/data_writer.rb:25 # DataWriter it should call its destructor calls the destructor
"file closed"
"#<File:/tmp/example.txt (closed)>"
As to the why of it, I can't tell right now. Dave is right you're asserting on something that's already happened, so your test is never going to pass. You can observe this by changing your test to:
it 'calls the destructor' do
expect(DataWriter).to receive(:finalize).and_call_original
data_writer = DataWriter.new('/tmp/example.txt')
data_writer = nil
GC.start
end
IMHO you should not rely on the finalizer to run exactly when GC runs. They will run, eventually. But perhaps only when the Process finishes. As far as I can tell this is also dependent on the Ruby implementation and the GC implementation.
1.8 has different behavior than 1.9+, Rubinius and JRuby might be different as well.
Making sure that a resource is released can be achieved by a block, which will also take care that the resource is released as soon as not needed anymore.
Multiple APIs have the same style in Ruby:
File.open('thing.txt', 'wb') do |file| # file is passed to block
# do something with file
end # file will be closed when block ends
Instead of doing this (as you showed in your gist)
(1..100_000).each do |i|
File.open(filename, 'ab') do |file|
file.puts "line: #{i}"
end
end
I'd do it this way:
File.open(filename, 'wb') do |file|
(1..100_000).each do |i|
file.puts "line: #{i}"
end
end
I rewrote my working solution below, bit I have not ran this code.
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
# creating pipe for IPC to get result from child process
# after it garbaged
# http://ruby-doc.org/core-2.0.0/IO.html#method-c-pipe
rd, wr = IO.pipe
# forking
# https://ruby-doc.org/core-2.1.2/Process.html#method-c-fork
if fork
wr.close
called = rd.read
Process.wait
expect(called).to eq('/tmp/example.txt')
rd.close
else
rd.close
# overriding DataWriter.actually_finalize(file)
DataWriter.singleton_class.class_eval do
define_method(:actually_finalize) do |arg|
wr.write arg
wr.close
end
end
data_writer = DataWriter.new('/tmp/example.txt')
data_writer = nil
GC.start
end
end
end
end
The main thing is I catch that GC.start call performs real work exactly when exiting from process. I have tried blocks and threads but in my case (ruby 2.2.4p230 # Ubuntu x86_64) it works only when a process finished.
I suggest, that there may exist a better way to get results from child process, but I used inter-process communication (IPC).
And I have not got result with building the rspec expectation on destructor method call in form like expect(DataWriter).to receive(:actually_finalize).with('/tmp/example.txt') - I don't know why, but I suppose that wrappers created by Rspec have been garbaged or infringed before calling of destructor of a class.
Hope this helps!
I am using AirBnb Nerve service. It's service code looks like this:
require 'logger'
require 'json'
require 'timeout'
require 'nerve/version'
require 'nerve/utils'
require 'nerve/log'
require 'nerve/ring_buffer'
require 'nerve/reporter'
require 'nerve/service_watcher'
module Nerve
class Nerve
include Logging
def initialize(opts={})
log.info 'nerve: starting up!'
# set global variable for exit signal
$EXIT = false
...some code...
# Any exceptions in the watcher threads should wake the main thread so
# that we can fail fast.
Thread.abort_on_exception = true
log.debug 'nerve: completed init'
end
def run
log.info 'nerve: starting run'
#services.each do |name, config|
launch_watcher(name, config)
end
begin
sleep
rescue StandardError => e
log.error "nerve: encountered unexpected exception #{e.inspect} in main thread"
raise e
ensure
$EXIT = true
log.warn 'nerve: reaping all watchers'
#watchers.each do |name, watcher_thread|
reap_watcher(name)
end
end
log.info 'nerve: exiting'
ensure
$EXIT = true
end
def launch_watcher(name, config)
... some code ...
end
def reap_watcher(name)
... some code ...
end
end
end
I do not see any stop method. What is the right way of stopping such a service? I am using JRuby and intend to write a JSVC adapter for this service.
There is no way to do this via the current API, short of sending it a signal.
If sending a signal isn't going to work and you want to handle stop explicitly, it looks like you will need to change the following things:
Add a #stop method to Nerve that sets $EXIT = true.
Modify #run so that rather than sleeping forever (sleep) it wakes up and checks $EXIT.
I have the following code:
# Connect to neighbors and send hello message
current_node.neighbors_addresses.each do |address|
for i in 0..10
begin
sock = TCPSocket.new(address, 3000)
break
rescue Errno::EHOSTUNREACH, Errno::ECONNREFUSED
puts "before sleep"
sleep(2)
puts "after sleep"
end
end
sock.write("Hi from node #{current_node.name}\n")
end
The current_node is a node object with neighbors_addresses as an instance variable that stores the ip addresses of the neighbors of the current node.
The problem is, the line sock.write("Hi from node #{current_node.name}\n")gives the error:
server1.rb:211:in `block in main': undefined method `write' for nil:NilClass (NoMethodError)
from server1.rb:200:in `each'
from server1.rb:200:in `main'
from server1.rb:276:in `<main>'
If write is not a method in the socket class then how do you write to that socket?
Put sock.write just after sock = TCPSocket... ?
– SeanNieuwoudt
if the code of rescue gets executed 10 times, sock will be nil after the for-loop ends. You need to make sure sock is a Socket object before calling any method or accessing any attribute of the object itself. SeanNieuwoudt is right.
– yeyo
What I actually trying to see when no 'initialize' method is given to an class definition then the class as you said should call the "Object#initialize",which here I tried to customize and see if it has been called or not. With that approach I reached to a conclusion(although that's wrong), when I typed "ob = A .new" that yes I can overload the Object#initialize method.But all has been ended up with the below exception. Then I thought I did something wrong in my customization.So I tried to create the object creation within an exception block and when I typed "begin" and pressed "ENTER" - i got the same error.
>> class A
>> def Object.new initialize
>> p "hi"
>> rescue
>> end
>> end
=> nil
>> begin # <~~~ Here I have pressed on ENTER
"hi" #<~~~~ How was it print out?
/usr/lib/ruby/1.9.1/irb/ruby-token.rb:94:in `Token': undefined method `set_backtrace' for "hi":String (NoMethodError)
from /usr/lib/ruby/1.9.1/irb/ruby-lex.rb:348:in `block in lex_init'
from /usr/lib/ruby/1.9.1/irb/slex.rb:236:in `call'
from /usr/lib/ruby/1.9.1/irb/slex.rb:236:in `match_io'
from /usr/lib/ruby/1.9.1/irb/slex.rb:221:in `match_io'
from /usr/lib/ruby/1.9.1/irb/slex.rb:75:in `match'
from /usr/lib/ruby/1.9.1/irb/ruby-lex.rb:286:in `token'
from /usr/lib/ruby/1.9.1/irb/ruby-lex.rb:262:in `lex'
from /usr/lib/ruby/1.9.1/irb/ruby-lex.rb:233:in `block (2 levels) in each_top_level_statement'
from /usr/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `loop'
from /usr/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `block in each_top_level_statement'
from /usr/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `catch'
from /usr/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `each_top_level_statement'
from /usr/lib/ruby/1.9.1/irb.rb:155:in `eval_input'
from /usr/lib/ruby/1.9.1/irb.rb:70:in `block in start'
from /usr/lib/ruby/1.9.1/irb.rb:69:in `catch'
from /usr/lib/ruby/1.9.1/irb.rb:69:in `start'
from /usr/bin/irb:12:in `<main>'
#ubuntu:~$
Now my questions are -
How has the "hi" been printed?
What is the cause of the error as printed above?
If such initialize definition is not allowed,then why has the error not come after I ended with the class definition?
EDIT
As per #casper I tried below:
>> def Object.new
>> p "hi"
>> end
=> nil
>> begin
/usr/lib/ruby/1.9.1/irb/ruby-token.rb:96: stack level too deep (SystemStackError)
But here no "hi" printed back.
So what made the "hi" to print back in the first case?
What exactly are you trying to do? You just redefined Object.new, so there is no surprise you make everything go haywire.
You can basically get the same effect by just:
>> def Object.new
>> end
>> [press enter]
KABOOM
The reason "hi" is printed is that someone just called Object.new, probably the irb REPL loop, and it expected an object, but instead it gets gobledygook.
You can also try this:
def Object.new *args
p args
end
And you will see funny stuff. However you won't be able to quit irb or do anything useful with it after that. Again: you just broke Object.
To make some sense of it you should read this:
In Ruby, what's the relationship between 'new' and 'initialize'? How to return nil while initializing?
And then you can try this:
class Object
class << self
alias :old_new :new
end
end
Now you can do:
def Object.new *args
p args
old_new *args
end
This won't break new because you are still calling the old version of it. However you will now be printing out stuff every time someone calls new.
I've got a Ruby TCPSocket client that works great except when I'm trying to close it. When I call the disconnect method in my code below, I get this error:
./smartlinc.rb:70:in `start_listen': stream closed (IOError)
from ./smartlinc.rb:132:in `initialize'
from ./smartlinc.rb:132:in `new'
from ./smartlinc.rb:132:in `start_listen'
from bot.rb:45:in `initialize'
from bot.rb:223:in `new'
from bot.rb:223
Here's the (simplified) code:
class Smartlinc
def initialize
#socket = TCPSocket.new(HOST, PORT)
end
def disconnect
#socket.close
end
def start_listen
# Listen on a background thread
th = Thread.new do
Thread.current.abort_on_exception = true
# Listen for Ctrl-C and disconnect socket gracefully.
Kernel.trap('INT') do
self.disconnect
exit
end
while true
ready = IO.select([#socket])
readable = ready[0]
readable.each do |soc|
if soc == #socket
buf = #socket.recv_nonblock(1024)
if buf.length == 0
puts "The socket connection is dead. Exiting."
exit
else
puts "Received Message"
end
end
end # end each
end # end while
end # end thread
end # end message callback
end
Is there a way I can prevent or catch this error? I'm no expert in socket programming (obviously!), so all help is appreciated.
Your thread is sitting in IO.select() while the trap code happily slams the door in its face with #socket.close, hence you get some complaining.
Don't set abort_on_exception to true, or then handle the exception properly in your code:
Something along these lines...
Kernel.trap('INT') do
#interrupted = true
disconnect
exit
end
...
ready = nil
begin
ready = IO.select(...)
rescue IOError
if #interrupted
puts "Interrupted, we're outta here..."
exit
end
# Else it was a genuine IOError caused by something else, so propagate it up..
raise
end
...