Testing sidekiq perform_in with RSpec 3 - ruby

RSpec 3 and sidekiq 3.2.1. And I have setup sidekiq and rspec-sidekiq properly.
Suppose I have a worker called WeatherJob, which will change the weather status from sunny to rainy:
class WeatherJob
include Sidekiq::Worker
def perform record_id
weather = Weather.find record_id
weather.update status: 'rainy'
end
end
I use this worker like this:
WeatherJob.perform_in 15.minutes, weather.id.
In the spec, I use Timecop to mock time:
require 'rails_helper'
describe WeatherJob do
let(:weather) { create :weather, status: 'sunny' }
let(:now) { Time.current }
it 'enqueue a job' do
expect {
WeatherJob.perform_async weather.id
}.to change(WeatherJob.jobs, :size).by 1
end
context '15 mins later' do
before do
Timecop.freeze(now) do
Weather.perform_in 15.minutes, weather.id
end
end
it 'update to rainy' do
Timecop.freeze(now + 16.minutes) do
expect(weather.status).to eq 'rainy'
end
end
end
end
I could see there is job in Weather.jobs array. And time is correctly 16 mins after. But it did not execute the job? Any advices? Thanks!

Sidekiq has three testing modes: disabled, fake, and inline. The default is fake, which just pushes all jobs into a jobs array and is the behavior you are seeing. The inline mode runs the job immediately instead of enqueuing it.
To force Sidekiq to run the job inline during the test, wrap your test code in a Sidekiq::Testing.inline! block:
before do
Sidekiq::Testing.inline! do
Timecop.freeze(now) do
Weather.perform_in 15.minutes, weather.id
end
end
end
For more info on testing Sidekiq, refer to the official Testing wiki page.

Do it in two steps. First test that the job was scheduled, then execute a job inline without time delay. Here is an example
it "finishes auction (async)" do
auction = FactoryGirl.create(:auction)
auction.publish!
expect(AuctionFinishWorker).to have_enqueued_sidekiq_job(auction.id).at(auction.finishes_at)
end
it "finishes auction (sync)" do
auction = FactoryGirl.create(:auction)
auction.publish!
Sidekiq::Testing.inline! do
AuctionFinishWorker.perform_async(auction.id)
end
auction.reload
expect(auction).to be_finished
end
have_enqueued_sidekiq_job method is coming from rspec-sidekiq gem. They have active development going on at develop branch. Make sure you include it like that
gem 'rspec-sidekiq', github: "philostler/rspec-sidekiq", branch: "develop"

If you want to test the job whether it should executes 15 minutes later or not then you should split you test cases into two parts. First part, you should test that whether it inserts job which would be active in 15 minutes(using mocks). Second part, whether the job has been executed properly or not.

Weather.drain can be a hack for issue
require 'rails_helper'
describe WeatherJob do
let(:weather) { create :weather, status: 'sunny' }
let(:now) { Time.current }
it 'enqueue a job' do
expect {
WeatherJob.perform_async weather.id
}.to change(WeatherJob.jobs, :size).by 1
end
context '15 mins later' do
before do
Timecop.freeze(now) do
Weather.perform_in 15.minutes, weather.id
end
end
it 'update to rainy' do
Timecop.freeze(now + 16.minutes) do
Weather.drain
expect(weather.status).to eq 'rainy'
end
end
end
end

Related

Periodically checking if a sidekiq job has been cancelled

Jobs in sidekiq are suppose to check if they have been cancelled, but if I have a long running job, I'd like for it to check itself periodically. This example does not work : I've not wrapped the fake work in any sort of future within which I can raise an exception -- which I'm not sure is even possible. How might I do this?
class ThingWorker
def perform(phase, id)
thing = Thing.find(id)
# schedule the initial check
schedule_cancellation_check(thing.updated_at, id)
# maybe wrap this in something I can raise an exception within?
sleep 10 # fake work
#done = true
return true
end
def schedule_cancellation_check(initial_time, thing_id)
Concurrent.schedule(5) {
# just check right away...
return if #done
# if our thing has been updated since we started this job, kill this job!
if Thing.find(thing_id).updated_at != initial_time
cancel!
# otherwise, schedule the next check
else
schedule_cancellation_check(initial_time, thing_id)
end
}
end
# as per sidekiq wiki
def cancelled?
#cancelled
Sidekiq.redis {|c| c.exists("cancelled-#{jid}") }
end
def cancel!
#cancelled = true
# not sure what this does besides marking the job as cancelled tho, read source
Sidekiq.redis {|c| c.setex("cancelled-#{jid}", 86400, 1) }
end
end
You're thinking about this way too hard. Your worker should be a loop and check for cancellation every iteration.
def perform(thing_id, updated_at)
thing = Thing.find(thing_id)
while !cancel?(thing, updated_at)
# do something
end
end
def cancel?(thing, last_updated_at)
thing.reload.updated_at > last_updated_at
end

Is it possible to write a spec for infinite-loop issues using Rspec?, Ruby

Listen I've an interesting question here, the other day I ran into an "infinite-loop" problem using Rspec, Rspec couldn't even go through the spec related to other methods inside the loop and even the comp was almost crashing. Very funny.
I'd like to test my future loops (While-loop in this case) against infinite loop-code.
How I can test this while-loop and catch up this problem like this one and make the proper correction?
Thanks!
This is my code from other day:
i = 0
while i <= Video.all.count do
if ( #sampler = Video.find_next_sampler(#samplers[-1].end_time, #samplers[-1].end_point) )
#samplers << #sampler
else
flash[:error] = 'There is not any more match for this video-sampler'
end
i + 1 #Now Here is the bug!! IT should be: i += 1
end
require 'timeout'
it 'should not take too long' do
Timeout.timeout(20) do
... blah ...
end
end
Or even
# spec_helper.rb
require 'timeout'
RSpec.configure do |c|
c.around(:example, finite: true) do |example|
Timeout.timeout(20) do
example.run
end
end
end
# my_spec.rb
it "should work really fast", finite: true do
... blah ...
end
In this particular example is doesn't make sense to run the loop more often that the total number of all videos in the database.
Therefore I would try something like this:
let(:videos_count) { Video.count }
before do
allow(Video).to receive(:find_next_sampler).and_call_original
end
it 'is not an infinite loop' do
except(Video).to receive(:find_next_sampler).at_most(videos_count).times
# call your method
end

Resque - Not Processing Queue

I have an web based app that I built using Sinatra. From recent I was needing to collect data at regular intervals and store them in a database. For this I was told that I could use Resque and Clockwork gems in combine.
Every hour or so I need to do nearly 15 calculations based on the database and store the results in a database.
So this is the approach I took. I decided to make 15 classes that have the perform method ( the exact file I used for testing is below ). Then to do some thing similar to Resque.enqueue( GraphData ) for all 15 classes.
class GraphData
#queue = :graph_data
def self.init()
end
def self.perform()
File.open( '/home/ziyan/Desktop/resque.txt', 'a' ) { | file | file.write( "Resqueu - performed - #{Time.now}\n" ) }
end
end
To trigger the operation for testing purposes, I created a rake task.
desc "Start Resque Workers for Queue" # {{{
task :graph_data do |t|
ENV["QUEUE"] = "*"
ENV["VVERBOSE"] = "1"
ENV["INTERVAL"] = "5"
Resque.enqueue( GraphData )
#resque = Resque.new
#resque << AdminWorker.new
end # }}}
As you see, in the GraphData class, under self.perform method I am writing to a file.
My problem is it doesn't! Have I done some thing wrong?
rake graph_data will show no output. The web interface will show a job in the Queue.
Additional Information
I added another Rake task for running the web interface.
desc "Start Resque Web Frontend" # {{{
task :resque_web_frontend do |t|
sh "resque-web -p 8282"
puts "Resque Web Frontend is running on http://localhost:8282"
end # }}
Over their, some thing interesting is seen. As I run the rake task, under stats the pending value increases but not the processed.
Under queues -> graph_data I see some thing like this.
Class Args
GraphData []
GraphData []
GraphData []
What I ended up with:
desc "Start Resque Workers for Queue" # {{{
task :graph_data do |t|
ENV["QUEUE"] = "*"
ENV["VVERBOSE"] = "1"
ENV["INTERVAL"] = "5"
Rake::Task[ "resque:work" ].invoke
Resque.enqueue( GraphData )
#resque = Resque.new
#resque << AdminWorker.new
end # }}}
Is this in development or production? From what you describe, it looks like you are not lauching the Resque process. How are you running your app? I'm more familiar with using Resque with Rails, but you should have to run something like:
rake resque:work QUEUE='*'
To have the resque worker start.

How do you have threads in Ruby send strings back to a parent thread

I want to be able to call a method that repeats x amount of times on a separate thread that sends messages such as "still running" every few moments to the console while I am free to call other methods that do the same thing.
This works in my test environment and everything checks out via rspec - but when I move the code into a gem and call it from another script, it appears that the code is working in additional threads, but the strings are never sent to my console (or anywhere that I can tell).
I will put the important parts of the code below, but for a better understanding it is important to know that:
The code will check stock market prices at set intervals with the intent of notifying the user when the value of said stock reaches a specific price.
The code should print to the console a message stating that the code is still running when the price has not been met.
The code should tell the user that the stock has met the target price and then stop looping.
Here is the code:
require "trade_watcher/version"
require "market_beat"
module TradeWatcher
def self.check_stock_every_x_seconds_for_value(symbol, seconds, value)
t1 = Thread.new{(self.checker(symbol, seconds, value))}
end
private
def self.checker(symbol, seconds, value)
stop_time = get_stop_time
pp stop_time
until is_stock_at_or_above_value(symbol, value) || Time.now >= stop_time
pp "#{Time.now} #{symbol} has not yet met your target of #{value}."
sleep(seconds)
end
if Time.now >= stop_time
out_of_time(symbol, value)
else
reached_target(symbol, value)
end
end
def self.get_stop_time
Time.now + 3600 # an hour from Time.now
end
def self.reached_target(symbol, value)
pp "#{Time.now} #{symbol} has met or exceeded your target of #{value}."
end
def self.out_of_time(symbol, value)
pp "#{Time.now} The monitoring of #{symbol} with a target of #{value} has expired due to the time limit of 1 hour being rached."
end
def self.last_trade(symbol)
MarketBeat.last_trade_real_time symbol
end
def self.is_stock_at_or_above_value(symbol, value)
last_trade(symbol).to_f >= value
end
end
Here are the tests (that all pass):
require 'spec_helper'
describe "TradeWatcher" do
context "when comparing quotes to targets values" do
it "can report true if a quote is above a target value" do
TradeWatcher.stub!(:last_trade).and_return(901)
TradeWatcher.is_stock_at_or_above_value(:AAPL, 900).should == true
end
it "can report false if a quote is below a target value" do
TradeWatcher.stub!(:last_trade).and_return(901)
TradeWatcher.is_stock_at_or_above_value(:AAPL, 1000).should == false
end
end
it "checks stock value multiple times while stock is not at or above the target value" do
TradeWatcher.stub!(:last_trade).and_return(200)
TradeWatcher.should_receive(:is_stock_at_or_above_value).at_least(2).times
TradeWatcher.check_stock_every_x_seconds_for_value(:AAPL, 1, 400.01)
sleep(2)
end
it "triggers target_value_reahed when the stock has met or surpassed the target value" do
TradeWatcher.stub!(:last_trade).and_return(200)
TradeWatcher.should_receive(:reached_target).exactly(1).times
TradeWatcher.check_stock_every_x_seconds_for_value(:AAPL, 1, 100.01)
sleep(2)
end
it "returns a 'time limit reached' message once a stock has been monitored for the maximum of 1 hour" do
TradeWatcher.stub!(:last_trade).and_return(200)
TradeWatcher.stub!(:get_stop_time).and_return(Time.now - 3700)
TradeWatcher.check_stock_every_x_seconds_for_value(:AAPL, 1, 100.01)
TradeWatcher.should_receive(:out_of_time).exactly(1).times
sleep(2)
end
end
And here is a very simple script that (in my understanding) should print "{Time.now} AAPL has not yet met your target of 800.54." every 1 second that the method is still running and should at least be visible for 20 seconds (I test this using sleep in rspec and am able to see the strings printed to the console):
require 'trade_watcher'
TradeWatcher.check_stock_every_x_seconds_for_value(:AAPL, 1, 800.54)
sleep (20)
However I get no output - although the program does wait 20 seconds to finish. If I add other lines to print out to the console they work just fine, but nothing within the thread triggered by my TradeWatcher method call actually work.
In short, I'm not understanding how to have threads communicate with each other appropriately - or how to sync them up with each other (I don't think thread.join is appropriate here because it would leave the main thread hanging and unable to accept another method call if I chose to send one at a time in the future). My understanding of Ruby multithreading is weak anyone able to understand what I'm trying to get at here and nudge me in the right direction?
It looks like the pp function is simply not yet loaded by ruby when you go to print. By adding:
require 'pp'
to the top of trade_watcher.rb I was able to get the output you're expecting. You might also want to consider adding:
$stdout.sync = $stderr.sync = true
to your binary/executable script so that your output is not buffered internally by the IO class and instead passed directly to the os.

Downloading multiple Files at the same Time (Multithreading)

I'm trying to use multithreading in ruby for heaving a lot of Network-Connections at the same time but I really stuck at the basics.
I tried this:
for i in 1..1000 do
Thread.new{load(i)}
end
def load(i)
File.open(filePath, "w") do |output|
open(imageURL) do | input |
output << input.read
end
end
end
This is only a part of the Download-Script for showing what I'm doing. At real I first load some HTML, parse it with Nokogiri and so on...
I'm running this script in Terminal with "ruby script.rb" - and nothing - really nothing happened.
Any idea how to solve this?
Thanks allot
Chris
Your script will immediately exit, because you are not waiting for your threads to finish.
Consider the first example of this section of the Pickaxe - you need to join your threads, in order to actually wait for all of them to finish their jobs.
So you should rather try this:
def load(i)
...
end
threads = []
for i in 1..1000 do
threads << Thread.new { load(i) }
end
threads.each { |t| t.join }

Resources