Spent hours trying to get a test case working. From my understanding, after the watch block is started, if that key is changed between then and the exec time, the multi block should do nothing.
redis = Redis.new
thread = Thread.new do
redis.watch("test_case") do
sleep 20
redis.multi do |setup|
redis.set "test_case", "foo"
end
end
end
and then during the 20 second sleep in IRB
redis.set "test_case", "bar"
The result is "test_case" => "foo". It should be "bar" because the watch should not allow execution of the multi statement. I have also tested this in redis-cli and the commands work as expected.
Issue was I had a module defining:
def redis
#redis = Redis.new
end
Everytime I called redis, a new object was made.
Related
I'm a DBA, and I stumbled upon such case: developers run irb sessions (from Ruby on Rails app). This irb keeps database connection open. Sometimes - they forget about it, and it keeps on "running" - not doign anything, but still using one db connection.
I'd like to add some kind of "idle timeout" to their irb config. Is it possible? How to do it?
Here's a quick hack how you might implement this.
Note that this does not take into account that the user might be executing some long-running task inside the irb session. It simply looks at the time stamp of the last input; if it has not changed then it just flat out kills the process:
Update: it now checks if irb is currently running a command and ignores any timeouts if that is the case.
# Add some methods to IRB::Context and IRB::Irb
# for easier timeout implementation.
class IRB::Irb
def status
#signal_status
end
end
class IRB::Context
attr_reader :irb
attr_reader :line_no
def is_evaluating?
self.irb.status == :IN_EVAL
end
end
# Implement an auto exit timer thread. Timeout is given in seconds.
module IRB
def self.auto_exit_after(timeout = 60)
Thread.new {
context = IRB.conf[:MAIN_CONTEXT]
last_input = Time.now
last_line = context.line_no
loop {
sleep 10
# Check if irb is running a command
if context.is_evaluating?
# Reset the input time and ignore any timeouts
last_input = Time.now
next
end
# Check for new input
if last_line != context.line_no
# Got new input
last_line = context.line_no
last_input = Time.now
next
end
# No new input, check if idle time exceeded
if Time.now - last_input > timeout
$stderr.puts "\n** IRB exiting due to idle timeout. Goodbye..."
Process.kill("KILL", Process.pid)
end
}
}
end
end
To use it add the code to .irbrc, or some other place that auto-loads when irb is started, and then just start the timer:
IRB.auto_exit_after(60)
Is it possible to create a "worker thread" so to speak that is on standby until it receives a function to execute asynchronously?
Is there a way to send a function like
def some_function
puts "hi"
# write something
db.exec()
end
to an existing thread that's just sitting there waiting?
The idea is I'd like to pawn off some database writes to a thread which runs asynchronously.
I thought about creating a Queue instance, then have a thread do something like this:
$command = Queue.new
Thread.new do
while trigger = $command.pop
some_method
end
end
$command.push("go!")
However this does not seem like a particularly good way to go about it. What is a better alternative?
The thread gem looks like it would suit your needs:
require 'thread/channel'
def some_method
puts "hi"
end
channel = Thread.channel
Thread.new do
while data = channel.receive
some_method
end
end
channel.send("go!")
channel.send("ruby!") # Any truthy message will do
channel.send(nil) # Non-truthy message to terminate other thread
sleep(1) # Give other thread time to do I/O
The channel uses ConditionVariable, which you could use yourself if you prefer.
I am trying to make my Rails app use Resque for managing workers. But, I would like to continue using the ConnectionPool gem.
I have this in an initializer:
puts ENV["REDISTOGO_URL"]
uri = (not ENV["REDISTOGO_URL"].nil?) ? URI.parse(ENV["REDISTOGO_URL"]) : nil
# at this point, debugger confirms $redis is nil
$redis = ConnectionPool::Wrapper.new(:size => 5, :timeout => 3) {
if uri.nil?
Redis.connect
else
Redis.connect(:host => uri.host, :port => uri.port, :password => uri.password)
end
}
$redis # just put this in here for the debugger
# At this point, $redis is #<Redis:0x007fb1b0036bf0>
# when it should be an instance of ConnectionPool::Wrapper
Does anyone have an idea why $redis would not be returned as an instance ConnectionPool::Wrapper?
I've searched in all the gems source code, nowhere does it set the value of $redis. In ConnectionPool's source code, I did not find anything where it would return an instance of Redis instead of itself.
This only happens when I switched from DelayedJob to Resque. So, it would seem that is the problem. However, I'm at a loss.
I am using Unicorn. Here's that file in config.
worker_processes 2
timeout 30
preload_app true
before_fork do |server, worker|
# Replace with MongoDB or whatever
if defined?(ActiveRecord::Base)
ActiveRecord::Base.connection.disconnect!
Rails.logger.info('Disconnected from ActiveRecord')
end
# If you are using Redis but not Resque, change this
if defined?(Resque)
Resque.redis.quit
Rails.logger.info('Disconnected from Redis')
end
sleep 1
end
after_fork do |server, worker|
# Replace with MongoDB or whatever
if defined?(ActiveRecord::Base)
ActiveRecord::Base.establish_connection
Rails.logger.info('Connected to ActiveRecord')
end
# If you are using Redis but not Resque, change this
if defined?(Resque)
# Yes, commented the Resque out for debugging, still get the same problem.
#Resque.redis = ENV['REDISTOGO_URL']
Rails.logger.info('Connected to Redis')
end
end
And finally, the Procfile:
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
worker: env TERM_CHILD=1 QUEUE=* bundle exec rake resque:work
I'm using foreman in my develop environment.
Any help is greatly appreciated.
From the docs:
You can use ConnectionPool::Wrapper to wrap a single global connection.
I looks like the ConnectionPool::Wrapper is meant to wrap a single connection to Redis as a convenience for migrating large applications from using Redis directly to using ConnectionPools
If you call $redis.with, you get the #with defined by ConnectionPool
To get an actual connection pool, just change your
ConnectionPool::Wrapper.new(:size => 5, :timeout => 3) { #redis logic }
to
ConnectionPool.new(:size => 5, :timeout => 3) { #redis logic }
Internally ConnectionPool::Wrapper creates a normal ConnectionPool object, and uses method_missing to automatically checkout/checkin from that pool whenever any method is called on the wrapper.
This use of method_missing includes calls to inspect, or class, or any number of methods which are normally used to try to look at the object or figure out its type.
require 'connection_pool'
class MyClass
def foo
'bar'
end
end
obj = MyClass.new
obj.respond_to?(:foo) # true
obj.respond_to?(:with) # false
wrapper = ConnectionPool::Wrapper.new { MyClass.new }
wrapper.respond_to?(:foo) # true
wrapper.respond_to?(:with) # also true! 'with' is a method on ConnectionPool::Wrapper
You do have an instance of ConnectionPool::Wrapper, it just is a bit hard to tell.
I'm trying to do something like this, where I have two loops going in seperate threads. The problem I am having is that in the main thread, when I use gets and the script is waiting for user input, the other thread is stopped to wait as well.
class Server
def initialize
#server = TCPServer.new(8080)
run
end
def run
#thread = Thread.new(#server) { |server|
while true
newsock = server.accept
puts "some stuff after accept!"
next if !newsock
# some other stuff
end
}
end
end
def processCommand
# some user commands here
end
test = Server.new
while true do
processCommand(STDIN.gets)
end
The above is just a sample of what I want to do.
Is there a way to make the main thread block while waiting for user input?
You might want to take a look at using the select method of the IO class. Take a look at
good select example for handling select with asynchronous input. Depending upon what version of ruby you're using you might have issues with STDIN though, I'm pretty sure it always triggers the select in 1.8.6.
I'm not sure if this is what you are looking for, but I was looking for something similar and this example does exactly what I wanted. The thread will continue processing until the user hits enter, and then the thread will be able to handle your user input as desired.
user_input = nil
t1 = Thread.new do
while !user_input
puts "Running"
end
puts "Stopping per user input: #{user_input}"
end
user_input = STDIN.gets
t1.join
I'm writing a delayed_job clone for DataMapper. I've got what I think is working and tested code except for the thread in the worker process. I looked to delayed_job for how to test this but there are now tests for that portion of the code. Below is the code I need to test. ideas? (I'm using rspec BTW)
def start
say "*** Starting job worker #{#name}"
t = Thread.new do
loop do
delay = Update.work_off(self) #this method well tested
break if $exit
sleep delay
break if $exit
end
clear_locks
end
trap('TERM') { terminate_with t }
trap('INT') { terminate_with t }
trap('USR1') do
say "Wakeup Signal Caught"
t.run
end
see also this thread
The best approach, I believe, is to stub the Thread.new method, and make sure that any "complicated" stuff is in it's own method which can be tested individually. Thus you would have something like this:
class Foo
def start
Thread.new do
do_something
end
end
def do_something
loop do
foo.bar(bar.foo)
end
end
end
Then you would test like this:
describe Foo
it "starts thread running do_something" do
f = Foo.new
expect(Thread).to receive(:new).and_yield
expect(f).to receive(:do_something)
f.start
end
it "do_something loops with and calls foo.bar with bar.foo" do
f = Foo.new
expect(f).to receive(:loop).and_yield #for multiple yields: receive(:loop).and_yield.and_yield.and_yield...
expect(foo).to receive(:bar).with(bar.foo)
f.do_something
end
end
This way you don't have to hax around so much to get the desired result.
You could start the worker as a subprocess when testing, waiting for it to fully start, and then check the output / send signals to it.
I suspect you can pick up quite a few concrete testing ideas in this area from the Unicorn project.
Its impossible to test threads completely. Best you can do is to use mocks.
(something like)
object.should_recieve(:trap).with('TERM').and yield
object.start
How about just having the thread yield right in your test.
Thread.stub(:new).and_yield
start
# assertions...