Is there any way to know if worker has finished the particular job/process in Resque.
Scenario: I have 5 worker doing some specific process, I want to know whether process is done to proceed with other part of code.
I am using Ruby 1.8.7 and Rails 3.1.1 if that is of any help.
you can try gearman if you need know this
log your information in your code
use redis-cli to check if the key of your job has value
resque-web and resque-status can also help you
You probably want to use something like resque-status: https://github.com/quirkey/resque-status .
If that doesn't quite meet your needs, you can always check the wiki plugin page for more possibilities: https://github.com/defunkt/resque/wiki/plugins
It is also not hard to store the fact of job completion as an extra field in your database.
Related
I am designing a reminder type app. Somebody enters into a form, for example, "call me at such and such a time on such and such a day in the future to remind me of something".
This is put into a postgres database. Obviously lots of people (hopefully) will be doing this and scheduling different things at different times in the future.
So, my question is how can ensure that my app checks the database for things it has to do at the right time? Can I:
a) should I, when the entry is made, create an automated script to execute at the time necessary to perform the reminder function? If so, how?
b) get my app to check the database every minute, again if so how? This would seem a huge waste of resources.
Sorry I cannot provide any code for this but I have no idea where to begin. all help gratefully received.
Thank you!
You might wanna use a background job framework like Sidekiq. Sidekiq supports scheduled jobs like this:
Notifier.perform_at(a_time_object, message_to_send)
It seems like Sidekiq also works with Sinatra.
For that particular task I prefer whenever https://github.com/javan/whenever
Is there a way to manually launch a Sidekiq process without using Ruby, but by posting the appropriate message into Redis? There must be some sort of convention for the message that it expects.
This is already covered in the FAQ: https://github.com/mperham/sidekiq/wiki/FAQ#how-do-i-push-a-job-to-sidekiq-without-ruby
Not sure why you would do this, but from its documentation: "Sidekiq is compatible with Resque. It uses the exact same message format as Resque so it can integrate into an existing Resque processing farm." I know that Resque enqueues a hash of data as a string:
"{\"class\":\"NoOpWorker\",\"args\":[]}"
You can manually verify this by enqueuing a job at a console with:
Resque.enqueue_to "foo", NoOpWorker
And then see what the data is with a redis-cli command
redis-cli lrange resque:queue:foo 0 100
But before proceeding, why would you want to do this? Why not just run a script or a rake task that would use enqueue the job through Sidekiq's normal API instead of hacking around it?
EDIT: Are you trying to interop between technologies?
Redis doesn't know anything about Ruby or Sidekiq.
So yeah, it's possible. It might require some work, and you might have to take versioning of the non-public (well, it is open source after all, so anything is public) API into account.
You could write a separate client process in any programming language, and analyze the redis keyspace. Read up on the implementation of the Sidekiq serialization. A quick look (I don't use Sidekiq) reveals that it uses simple JSON serialization: sidekiq/api.rb .
Hope this helps, TW
I have a django application in heroku and one thing I need to do sometimes that take a little bit of time is sending emails.
This is a typical use case of using workers. Heroku offers support for workers, but I have to leave them running all the time (or start and stop them manually), which is annoying.
I would like to use a one-off process to send every email. One possibility I first thought of was using IronWorker, since I thought that I could simply add the job to ironworker's queue and it would be exectuted with a mex of 15 min delay, which is ok for me.
The problem is that with ironworker, I need to put in a zip file all the modules and their dependencies in order to run the job, so in my email use case, as I use "EmailMultiAlternatives" from "django.core.mail.message", I would need to include all the django framework in my zip file in order to be able to use it.
According to this link, it's possible to add/remove workers from the app. Is it possible to start one-off processes from the app?
Does anyone has a better solution?
Thanks in advance
In my ruby on rails project, I have to take pull from sql-server to my mysql database.
When I run my project on port 3000, it makes system busy when I want to take pull.
I want such method or way which system can detect, how many ports are running for ruby application and how to close if it is not in use ?
Thanks in advance.
Hard to understand exactly what you're asking for, but I'm assuming that when you are synchronizing databases, the system becomes busy and you can't serve any pages. This is a perfect example for the use of a background job that allows you to do tasks like this without affecting the rails application. The two gems that come to mind that will allow you to do this is Delayed_job and Resque. An outstanding screencast for doing this type of stuff is listed below as well.
http://github.com/collectiveidea/delayed_job
https://github.com/defunkt/resque/
http://railscasts.com/episodes/171-delayed-job
http://railscasts.com/episodes/271-resque
Resque has a great web interface on which we can see the pending and failed jobs.
But where are the 'proccessed jobs'? How can I know what I have done?
There is a nice plugin called resque-status that might be of use to you.