I'm running into a strange issue that is causing my Heroku workers to crash. We're using Ruby on Rails and delayed_job for background jobs. I'm passing a job to delayed_job using the Vero gem.
This is the call I make to "identify" the user to Vero:
after_save { self.identify! }
Then it puts a job in the queue that looks like this:
--- !ruby/object:Vero::Api::Workers::Users::TrackAPI
domain: https://api.getvero.com
options:
:email: ******#gmail.com
:data:
:email: ******#gmail.com
:name: ? ?
:first_name: ?
:last_name: ?
:school_id: -1
The issue seems to be those question marks. I'm not sure why they are showing up there instead of a string of text. This is the error that comes up:
Psych::SyntaxError: (<unknown>): mapping keys are not allowed in this context at line 7 column 14
Unfortunately, instead of the job just failing.. it actually crashes the worker.. not allowing for another jobs to be processed.
Has anyone run into this issue in the past? How can I format the YAML in a way that it won't crash the worker?
Thanks!
Check out this user. Seems that he entered some data not accept by the encoding of the fields in the db.
Are you using utf-8? If he entered utf-16 you can translitirate it in ruby to utf-8
Related
I am getting a return code from my memcached cluster of value 44120, which is breaking my memcached-northscale ruby client because it does not know what this return code value means.
Can anyone shed light on what this code means and which ruby memcached clients are capable of handling it?
I am using an AWS ElastiCache cluster that is returning this value.
Thanks!
Update: I just tried to substitute memcached for memcached-northscale, and just got this exception:
TypeError: can't convert Fixnum into String
Where:
[GEM_ROOT]/gems/memcached-1.4.6/lib/memcached/memcached.rb, line 306
I don't know what the root is but my suspicion is that it's related to the earlier issue with memcached-northscale.
I also got an exception:
Memcached::Error: Unknown return code: 4832
Where:
[GEM_ROOT]/gems/memcached-1.4.6/lib/memcached/memcached.rb, line 631
I just discovered that the default memcached timeout was being overridden with a very short timeout. Increasing the timeout removed these errors. I do not know why they were being caused by a short timeout however, it seems like a bug in the library someplace.
I have a Daemon I am trying to start but I would like to set a few variables in the daemon when starting it. Here is the script I am using to control my daemons locates in RAILSAPP/script/daemon
#!/usr/bin/env ruby
require 'rubygems'
require 'daemons'
ENV["APP_ROOT"] ||= File.expand_path("#{File.dirname(__FILE__)}/..")
ENV["RAILS_ENV_PATH"] ||= "#{ENV["APP_ROOT"]}/config/environment.rb"
script = "#{ENV["APP_ROOT"]}/daemons/#{ARGV[1]}"
Daemons.run(script, dir_mode: :normal, dir: "#{ENV["APP_ROOT"]}/tmp/pids")
When I start this daemon I would like to pass a variable to it like a reference to an active record so I can base the daemon's initial run off of it.
If you want to fetch a specific ActiveRecord object, you can either pass just the id, or the classname + id as an additional parameter on the commandline. As you're already using ARGV[1] for the script name, you could pass it as ARGV[2] and something like Product_123 that you then parse via split, and do a Product.find(123) to get the actual record.
Another approach would be to put the object information into a queue like memcached or redis, and have the daemon fetch the information out of the queue. That would keep your daemon startup a bit simpler, and would allow you to queue up multiple records for the daemon to process. (Something just processing a single record would probably be better written as a script anyway.)
The other concern I have about your script is using ENV["APP_ROOT"]. Does that really need to go in the environment? What if you have a second daemon? It seems that it would be better as a local variable, and if you need it in the daemon, you can always get it relative to where the daemon's file is located anyway.
This question already has an answer here:
tailable cursor in mongo db timing out
(1 answer)
Closed 9 years ago.
How to specify a no-timeout option on the cursor?
I can run the job manually and from my laptop but something is going on the server and all the time I am getting this error:
MONGODB cursor.refresh() for cursor xxx
Query response returned CURSOR_NOT_FOUND. Either an invalid cursor was specified, or the cursor may have timed out on the server.
MONGODB cursor.refresh() for cursor yyy
The job is ran from a ruby scheduler file that and is specified as a namespace in rake
rake is calling for another ruby module in the middle, and the job dies during the execution of this module
I asked this question earlier and it got downvoted. Please, instead of downvoting explain what is so stupid about it, because I really need to solve this problem and can't figure out what is going on.
The server is kind of experimental and does not have any monitoring tools. But it seems to be reliable. And there are no other jobs running.
See the FAQ for the Ruby MongoDB driver for details on how to turn off the cursor timeout.
Example from there:
#collection.find({}, :timeout => false) do |cursor|
cursor.each do |document
# Process documents here
end
end
I have this in my initializer:
Delayed::Job.const_set( "MAX_ATTEMPTS", 1 )
However, my jobs are still re-running after failure, seemingly completely ignoring this setting.
What might be going on?
more info
Here's what I'm observing: jobs with a populated "last error" field and an "attempts" number of more than 1 (10+).
I've discovered I was reading the old/wrong wiki. The correct way to set this is
Delayed::Worker.max_attempts = 1
Check your dbms table "delayed_jobs" for records (jobs) that still exist after the job "fails". The job will be re-run if the record is still there. -- If it shows that the "attempts" is non-zero then you know that your constant setting isn't working right.
Another guess is that the job's "failure," for some reason, is not being caught by DelayedJob. -- In that case, the "attempts" would still be at 0.
Debug by examining the delayed_job/lib/delayed/job.rb file. Esp the self.workoff method when one of your jobs "fail"
Added #John, I don't use MAX_ATTEMPTS. To debug, look in the gem to see where it is used. Sounds like the problem is that the job is being handled in the normal way rather than limiting attempts to 1. Use the debugger or a logging stmt to ensure that your MAX_ATTEMPTS setting is getting through.
Remember that the DelayedJobs jobs runner is not a full Rails program. So it could be that your initializer setting is not being run. Look into the script you're using to run the jobs runner.
I've used Delayed_job in the past. I have an old project that runs on a server where I can't upgrade from Ruby 1.8.6 to 1.8.7, and therefore can't use Delayed Job, so I'm trying BackgroundJobs http://codeforpeople.rubyforge.org/svn/bj/trunk/README
I have it working so that my job runs, but something doesn't seem right. For example, if I run the job like this:
jobs = Bj.submit "echo hi", :is_restartable => false, :limit => 1, :forever => false
Then I see the job in the bj_job table and I see that it completed along with 'hi' in stdout. I also see only one job in the table and it doesn't keep re-running it.
For some reason if I do this:
jobs = Bj.submit "./script/runner ./jobs/calculate_mean_values.rb #{self.id}", :is_restartable => false, :limit => 1, :forever => false
The job still completes as expected, however, it keeps inserting new rows in the bj_job table, and the method gets run over and over until I stop my dev server. Is that how it is supposed to work?
I'm using Ruby 1.8.6 and Rails 2.1.2 and I don't have the option of upgrading. I'm using the plugin flavor of Bj.
Because I just need to run the process once after the model is saved, I have it working by using script/runner directly like this:
system " RAILS_ENV=#{RAILS_ENV} ruby #{RAILS_ROOT}/script/runner 'CompositeGrid.calculate_values(#{self.id})' & "
But would like to know if I'm doing something wrong with Background Jobs,
OK, this was stupid user error. As it turns out, I had a call back that was restarting the process and creating an endless loop. After fixing the call back it is working exactly as expected.