I'm working with delayed job for the first time and I am not having any luck getting it to work and I don't understand why. The job gets queued, but never executes. It does get removed from the database though, so the worker seems to be processing it, but it doesn't ever run the perform method it seems.
I've set up the following initalizer:
require 'delayed/worker'
Dir.glob("./app/jobs/*.rb").each { |f| require f }
Delayed::Worker.logger = ActiveSupport::BufferedLogger.new("log/dj.log", Rails.logger.level)
module Delayed
class Worker
def self.dj_say(text)
self.logger.info "#{Time.now.strftime('%F %T %z')}: #{text}"
end
end
end
The initializer works, and dj.log does get written to when the work starts/exits.
My job consists of the following:
class UpdateServices
attr_accessor :options
def initialize(options)
self.options = options
end
def perform
#Debugging code
Delayed::Worker.dj_say "starting"
File.open("tmp/test.txt","w").close
#End debugging code
#bunch-o-code-here
Delayed::Worker.dj_say "completed"
end
end
None of this code ever runs. I've tried adding debugging code into the perform method, and it never gets executed, so there is definitely something going on. I call the job from a model like this:
class Service < ActiveRecord::Base
def start_updating!(options)
Delayed::Job.enqueue(UpdateServices.new(options), :priority => 0, :run_at => 30.seconds.from_now, :queue => 'update_services')
end
end
That model is called from my controller like this:
Service.new.start_updating!(params)
My only guess is that I'm called Delayed::Job.enqueue incorrectly, but this seems to be how it should be done based on everything I've read.
It's possible for this to happen if DJ can't deserialize a job for whatever reason since the worker will force the job to permafail. This appears not to be logged by default. Try disabling failed job deletion Delayed::Worker.destroy_failed_jobs = false and seeing if there's an error.
Related
While working on a Rails app and investigating a bug, I narrowed it down to the following situation, where line AAA would cause events to return nil, running no code after points XXX or YYYY, but line BBB worked as expected. I switched back and forth several times making a clean environment every time, and it always behaved the same odd way.
Ruby 2.5.0, haven't tried downgrading yet but I suspect the issue was present in 2.3 (I skipped 2.4 with this app).
# this is the method that has the issue when called. It's in a PORO service object instantiated in a helper method & called from a Rails view.
def events
#events ||= begin
events = []
# some code building up events
events << :whatever if fulfillment.ready_to_process?
#XXX -- execution never reaches here with #AAA. But no error is thrown, and events returns nil
# some more code building up events
end
end
# Fulfillment & LineItem are ActiveRecords
class Fulfillment
def ready_to_process?
# some "return false if" assertions
result = line_items.any?(&:fulfillment_needed?) #AAA
result = line_items.any?{ |line_item| line_item.fulfillment_needed? } #BBB
# YYY -- exececution never reaches here with #AAA
result
end
end
class LineItem
def fulfillment_needed?
# various assertions
true
end
end
EDIT: note that in the context where the bug was appearing, fulfillment_needed? was always true.
EDIT 2: actually, I found another something in fulfillment_needed? that can toggle the bug:
class LineItem
has_many :vendor_line_items, dependent: :destroy
def fulfillment_needed?
return true if vendor_line_items.empty? # Displays the bug, AAA fails but BBB works
return true # Does not display the bug AAA && BBB work
end
end
vendor_line_items.empty? behaves normally elsewhere.
I have a Volt Framework Task that checks and stores information on a directory, e.g.
class DirectoryHelperTask < Volt::Task
def list_contents()
contents = []
Dir.glob("/path/to/files").each do |f|
contents << f
end
return contents
end
end
I would like to call this from a different task, e.g.
class DirectoryRearrangerTask < Volt::Task
dir_contents = DirectoryHelperTask.list_contents()
end
The code above (DirectoryRearranger) throws an error, as does a promise call
DirectoryHelperTask.list_contents().then do |r|
dir_conents = r
end.fail do |e|
puts "Error: #{e}"
end
Could not find a way to call a task from another task in the Volt Framework documentation.
Thanks a lot!
From what I gather, tasks are meant to be run on the server side and then called on the client side, hence the use of the promise object. The promise object comes from OpalRb, so trying to call it from MRI won't work. If you have a "task" that will only be used on the server side, then it doesn't really fit with Volt's concept of a task.
Your first approach to the problem actually does work, except that DirectoryRearrangerTask can't inherit from Volt::Task.
directory_helper_task.rb
require_relative "directory_rearranger_task"
class DirectoryHelperTask < Volt::Task
def list_contents
contents = []
Dir.glob("*").each do |file|
contents << file
end
DirectoryRearrangerTask.rearrange(contents)
contents
end
end
directory_rearranger_task.rb
class DirectoryRearrangerTask
def self.rearrange(contents)
contents.reverse!
end
end
Here is a GitHub repo with my solution to this problem.
You can call tasks from the client or server, but keep in mind that you call instance methods on the class. (So they get treated like singletons) And all methods return a Promise. I think your issue here is that your doing dir_contents = DirectoryHelperTask.list_contents() inside of the class. While you could do this in ruby, I'm not sure its what you want.
Also, where you do dir_contents = r, unless dir_contents was defined before the block, its going to get defined just in the block.
Is there a way to keep track of variables that are created when using let?
I have a series of tests, some of which use let(:server) { #blah blah }. Part of the blah is to wait for the server to start up so that it is in a decent state before it is used.
The issue comes when I'm done with that test. I want to kill the server using server.kill(). This would be almost perfect if I could say something to the effect of
after(:each) { server.kill }
But this would create the server and waste all the resources/time to create it when it is referenced, only to kill it immediately if the server hadn't been used in the preceding test. Is there a way to keep track of and only clean up the server if it has been used?
I've come across a similar problem. A simple way to solve this is to set a instance variable in the let method to track if the object was created:
describe MyTest do
before(:each) { #created_server = false }
let(:server) {
#created_server = true
Server.new
}
after(:each) { server.kill if #created_server }
end
What I would do is something like this:
describe MyTest do
let(:server) { Server.new }
context "without server" do
## dont kill the server in here.
end
context "with server" do
before do
server
end
after(:each) { server.kill }
it {}
it {}
end
end
This is definitely a hack:
describe "cleanup for let" do
let(:expensive_object) {
ExpensiveObject.new
}
after(:context) {
v = __memoized[:expensive_object]
v.close if v
}
end
I figured that rspec had to be storing these lazy values somewhere the instance could access them, and __memoized is that place.
With a helper, it becomes a bit tidier:
def cleanup(name, &block)
after(:context) do
v = __memoized[name]
instance_exec(v, &block) if v
end
end
describe "cleanup for let" do
let(:expensive_object) {
ExpensiveObject.new
}
cleanup(:expensive_object) { |v|
v.close
}
end
There's still room for improvement, though. I think I would rather not have to type the object's name twice, so something like this would be nicer:
describe "cleanup for let" do
let(:expensive_object) {
ExpensiveObject.new
}.cleanup { |v|
v.close
}
end
I'm not sure I can do that without hacking rspec to pieces, but maybe if rspec themselves saw the benefit of it, something could be done in core...
Edit: Changed to using instance_exec because rspec started whining if things were called from the wrong context, and changed cleanup to be after(:context), because apparently this is the level it's memoising at.
Just write a small decorator to handle both the explicit and implicit starting of the server and which allows you to determine if the server has been started.
Imagine this to be the real server that needs to be started:
class TheActualServer
def initialize
puts 'Server starting'
end
def operation1
1
end
def operation2
2
end
def kill
puts 'Server stopped'
end
end
The reusable decorator could look like this:
class ServiceWrapper < BasicObject
def initialize(&start_procedure)
#start_procedure = start_procedure
end
def started?
!!#instance
end
def instance
#instance ||= #start_procedure.call
end
alias start instance
private
def method_missing(method_name, *arguments)
instance.public_send(method_name, *arguments)
end
def respond_to?(method_name)
super || instance.respond_to?(method_name)
end
end
Now you can apply this in your specs like the following:
describe 'something' do
let(:server) do
ServiceWrapper.new { TheActualServer.new }
end
specify { expect(server.operation1).to eql 1 }
specify { expect(server.operation2).to eql 2 }
specify { expect(123).to be_a Numeric }
context 'when server is running' do
before(:each) { server.start }
specify { expect('abc').to be_a String }
specify { expect(/abc/).to be_a Regexp }
end
after(:each) { server.kill if server.started? }
end
When a method is called on the decorator, it will run it's own implementation if one exists. For example if #started? is called, it will answer whether the actual server has been started or not. If it doesn't have an own implementation of that method, it will delegate the method call to the server object returned by that. If it doesn't have a reference to an instance of the actual server at that point, it will run the provided start_procedure to get one and memoize that for future calls.
If you put all the posted code into a file called server_spec.rb you can then run it with:
rspec server_spec.rb
The output will be like this:
something
Server starting
Server stopped
should eql 1
Server starting
Server stopped
should eql 2
should be a kind of Numeric
when server is running
Server starting
Server stopped
should be a kind of String
Server starting
Server stopped
should be a kind of Regexp
Finished in 0.00165 seconds (files took 0.07534 seconds to load)
5 examples, 0 failures
Note that in the examples 1 and 2, methods on the server are called, and therefore you see the output of the server that is implicitly started by the decorator.
In example 3 there is no interaction with the server at all, therefore you don't see the server's output in the log.
Then again in examples 4 and 5, there is not direct interaction with the server object in the example code, but the server is explicitly started through a before block, which can also be seen in the output.
I found another question very similar to mine with a solution that worked for me when I wrote it all in one simple script. I even wrote a second simple example sort of simulating what I'm trying to do, and it seemed to still work.
My simulation was:
class A
def looper(&block)
Thread.new do
loop do
exit if gets.chomp == 'q'
end
end
loop do
block.call
end
end
end
class B < A
def looper
super do
puts 'howddyyyy from B'
end
end
end
This works fine, exiting when you press q<Enter>. However, when I tried to implement this into my actual project, it fails to work. I'll post the code from the method in question in the child class, as the parent class is literally exactly the same as the example above.
def looper
super do
if obj = Object.first(:process_status => STATUS_UNPROCESSED)
puts "[Object ##{obj.id}] Processing..."
puts "-" * 60
obj.set_failed
if #obj.process(obj)
obj.set_processed
end
puts "-" * 60
puts "[Object ##{obj.id}] Finished!"
puts
puts
else
sleep 10
end
end
end
So, for some reason, this doesn't work. I put a puts into the new Thread (listening for q), and it seems to output the puts before every loop of block.call. Maybe it just isn't able to get the key, by which I mean, maybe the timeframe in which you have to enter q<Enter> is way too small? I'm not sure, which is why I'm asking some advice here. My only other guess is that it has something to do with the methods called within this method (process, or possible the Sequel calls to the database) blocking the other thread(s)?
I'm new to threading, so I have no clue.
Okay, everybody. I feel a little stupid for typing all that up, as I came to a solution not five minutes later (and one I had overlooked here on Stack Overflow).
For anyone facing a similar issue in the future, this is what I ended up doing (in the parent class):
def looper(&block)
interrupted = false
trap("INT") { interrupted = true }
until interrupted do
block.call
end
exit
end
This manages to achieve what I was essentially trying to do.
Thanks for reading!
This below app saves some data to the db and I want to test that it saves properly.
require 'goliath'
class App < Goliath::API
def response(env)
db = EM::Mongo::Connection.new('localhost').db('hello')
db.collection('coll').insert({'identifier => 1'})
[204, {}, {}]
end
end
require 'goliath/test_helper'
Goliath.env = :test
describe App do
include Goliath::TestHelper
it do
with_api(described_class) do
get_request do |req|
db = EM::Mongo::Connection.new('localhost').db('hello')
db.collection('coll').first.callback do |rec|
rec['identifier'].should == 100
end
end
end
end
end
The above spec passes since reactor ends before callback returns. I thought about manually starting a reactor like:
EM.run do
db = EM::Mongo::Connection.new('localhost').db('hello')
db.collection('coll').first.callback do |rec|
rec['identifier'].should == 100
EM.stop
end
end
Though I'm not sure if starting the reactor for every spec is good practice. Help please?
The problem is that when the get_request is setup we add a callback on the request that stops the event loop. So, as soon as your block finishes (which will be before the connection is even created), it will stop the reactor.
I'm not sure the best solution, but a crappy one would be to override:
def hookup_request_callbacks(req, errback, &blk)
req.callback &blk
req.callback { stop }
req.errback &errback if errback
req.errback { stop }
end
in your test class after you include Goliath::TestHelper. Then, I think, you should be able to write your own that just has something like:
def hookup_request_callbacks(req, errback, &blk)
req.callback &blk
req.errback &errback if errback
req.errback { stop }
end
You'll just have to make sure you call stop in your callback from Mongo.
I haven't actually tested this, so let me know if something doesn't work and I can dig in further.
#dj2's solution works great, but I decided instead of use mongo gem in specs, instead of em-mongo. Since mongo blocks, I don't have to worry about Goliath stopping the reactor before database returns results.