Resque has a great web interface on which we can see the pending and failed jobs.
But where are the 'proccessed jobs'? How can I know what I have done?
There is a nice plugin called resque-status that might be of use to you.
Related
I probably have found a similar question (and answer) but I wanted to know if any better alternative is available.
Link to similar question:
http://mqseries.net/phpBB/viewtopic.php?t=72601&sid=f62d9730d61ee2ee2a59986dd79defd1
I want to schedule a particular message flow every 5 seconds (or so). I'm using IIB 10 and it's not associated with MQ. So, Timer nodes are non functional.
I've read about scheduling it with cronjob but again it's getting dependent on the OS which is not my preference. Is there any alternative to the timeout notification node?
Can we use java.util.TimerTask or something similar to to it? Any helping hands please?
I don't know of any solution that does not require a cron job or other external scheduler.
Many organisations use a distributed scheduler like Ctrl-M for a wide range of tasks, and adding a couple of jobs to support the integration layer is not seen as a problem.
You can write you own timer flow using an infinite WHILE loop with SLEEP and PROPAGATE TO TERMINAL functions and sending HTTP requests or configure a "CallableFlow".
Is it possible to somehow view sidekiq completed job list - for example, find all PurchaseWorkers with params (1)? Yesterday in my app delayed method that was supposed to run didn't and associated entity (lets say 'purchase') got stuck in limbo with state "processing". I am trying to understand whats the reason: job wasn't en-queued at all or was en-queued but for some reason exited unexpectedly. There were no errors in sidekiq log.
Thanks.
This is old but I wanted to see the same thing since I'm not sure if jobs I scheduled ran or not!
Turns out, Sidekiq doesn't have anything built in to see jobs that completed and still doesn't seem to.
If it err'd and never completes it should be in the 'dead' queue. But to check that something actually ran seems to be beyond Sidekiq by default.
The FAQ suggests installing 3rd party plugins to track and log information: https://github.com/mperham/sidekiq/wiki/FAQ#how-can-i-tell-when-a-job-has-finished One of them allows for having a callback to do follow up (maybe add a record for completed jobs elsewhere?)
You can also setup Sidekiq to log to somewhere other than STDOUT (default) so you can output log information about your jobs. In this case, logging that it's complete or catching errors if for some reason it is never landing in the retrying or dead jobs queue when there is a problem. See https://github.com/mperham/sidekiq/wiki/Logging
To see jobs still in queue you can use the Rails console and look at the queue by queue name https://www.rubydoc.info/gems/sidekiq/Sidekiq/Queue
One option is the default stats provided by sidekiq - https://github.com/mperham/sidekiq/wiki/Monitoring#using-the-built-in-dashboard
The best options is to use the Web UI provided here - https://github.com/mperham/sidekiq/wiki/Monitoring#web-ui
Is there a way to manually launch a Sidekiq process without using Ruby, but by posting the appropriate message into Redis? There must be some sort of convention for the message that it expects.
This is already covered in the FAQ: https://github.com/mperham/sidekiq/wiki/FAQ#how-do-i-push-a-job-to-sidekiq-without-ruby
Not sure why you would do this, but from its documentation: "Sidekiq is compatible with Resque. It uses the exact same message format as Resque so it can integrate into an existing Resque processing farm." I know that Resque enqueues a hash of data as a string:
"{\"class\":\"NoOpWorker\",\"args\":[]}"
You can manually verify this by enqueuing a job at a console with:
Resque.enqueue_to "foo", NoOpWorker
And then see what the data is with a redis-cli command
redis-cli lrange resque:queue:foo 0 100
But before proceeding, why would you want to do this? Why not just run a script or a rake task that would use enqueue the job through Sidekiq's normal API instead of hacking around it?
EDIT: Are you trying to interop between technologies?
Redis doesn't know anything about Ruby or Sidekiq.
So yeah, it's possible. It might require some work, and you might have to take versioning of the non-public (well, it is open source after all, so anything is public) API into account.
You could write a separate client process in any programming language, and analyze the redis keyspace. Read up on the implementation of the Sidekiq serialization. A quick look (I don't use Sidekiq) reveals that it uses simple JSON serialization: sidekiq/api.rb .
Hope this helps, TW
Is there any way to know if worker has finished the particular job/process in Resque.
Scenario: I have 5 worker doing some specific process, I want to know whether process is done to proceed with other part of code.
I am using Ruby 1.8.7 and Rails 3.1.1 if that is of any help.
you can try gearman if you need know this
log your information in your code
use redis-cli to check if the key of your job has value
resque-web and resque-status can also help you
You probably want to use something like resque-status: https://github.com/quirkey/resque-status .
If that doesn't quite meet your needs, you can always check the wiki plugin page for more possibilities: https://github.com/defunkt/resque/wiki/plugins
It is also not hard to store the fact of job completion as an extra field in your database.
We use DelayedJob to run some of our long running processes and would like to test with Cucumber/Webrat.
Currently, we are calling Delayed::Job.work_off in a Ruby thread to get work done in the background, but are looking for a more robust solution
What is the best approach for this?
Thanks.
The main problem I see with the Delayed:Job.work_off approach is that you are making explicit in your Cucumber scenarios something that belongs to the internals of your system. Mixing both concerns is against the spirit of functional testing:
When I click some link # Some operation is launched in the background
And Jobs are dispatched # Delayed:Job.work_off invoked here
Then I should see the results...
Another problem is that you populate your Cucumber scenarios with repetitive steps for dispatching jobs when needed.
The approach I am currently using is launching delayed_job in the background while cucumber scenarios are being executed. You can check the Cucumber hooks I am using in that link.