I'm using Phusion Passenger with my nginx to deploy rails/sinatra applications, and I'm currently having a problem.
I want to run a class that checks for new submissions to reddit.com every 30 seconds. But since passenger shuts down the application after x seconds of idle time, it won't keep checking.
Yes, I've tried to set passenger_pool_idle_time to 0, but it still shuts it down.
If you want more details, see the application at github
Thanks in advance.
you could use cron to call into your server ever so often, to make sure it's still running. What may be happening is that passenger is starting up an initial process, then forking it for each worker process it needs later. After awhile, it kills the initial process (thinking it has spawned all the children it ever needs), so setting it to not do that might fix it.
Related
I am new to Go and I am using go routines in my app in Heroku, which are long (up to 7 minutes), and cannot be interrupted.
I saw that the auto scaler sometimes kills the Heroku dyno which is running the routine. I need a way of running this routine independently from the dynos so I know that it will not get shutdown. I read articles and still don't understand how to perform a go routine in a background worker. It is hard for me to believe I am the only one experiencing this.
My go routines use my redis database.
Could someone please point me to an example of how to setup a background worker in heroku for go and how to send my go routine to that worker?
Thank you very much
I need a way of running this routine independently from the dynos so I
know that it will not get shutdown.
If you don't want to run your worker code on a dyno then you'll need to use a different provider from Heroku, like Amazon AWS, Digital Ocean, Linode etc.
Having said that, you should design your workers, especially those that are mission critical, to be able to recover from a shutdown. Either to be able to continue where they left off or to start over. Heroku's dyno manager restarts the dynos at least once a day but I wouldn't be surprised if the other cloud providers also restart their virtual instances once in a while, probably not once a day but still... And even if you decide to deploy your workers on a physical machine that you control and never turn off, you cannot prevent things like hardware failure or power outage from happening.
If your workers need to perform some task till it's done you need to make them be aware of possible shutdowns and have them handle such scenarios gracefully. Do not ever rely on a machine, physical or virtual, to keep running while your worker is doing it's job.
For example if you're on Heroku, use a worker dyno and make your worker listen for the SIGTERM signal, after your worker receives such a signal...
The application processes have 30 seconds to shut down cleanly
(ideally, they will do so more quickly than that). During this time
they should stop accepting new requests or jobs and attempt to finish
their current requests, or put jobs back on the queue for other worker
processes to handle. If any processes remain after that time period,
the dyno manager will terminate them forcefully with SIGKILL.
... continue reading here.
But keep in mind, as I mentioned earlier, if there is an outage and Heroku goes down, which is something that happens from time to time, your worker won't even have those 30 seconds to clean up.
Is it possible to create a script that is always running on my VPS server? And what need i to do to run it the hole time? (I haven't yet a VPS server, but if this is possible i wants to buy one!
Yes you can, there are many methods to get your expected result.
Supervisord
Supervisord is a process control system that keeps any process running. It automatically start or restart your process whenever necessary.
When to use it: Use it when you need a process that run continuously, eg.:
A queue worker that reads a database continuously waiting for a job to run.
A node application that acts like a daemon
Cron
Cron allow you running processes regularly, in time intervals. You can for example run a process every 1 minute, or every 30 minutes, or any time interval you need.
When to use it: Use it when your process is not long running, it do a task and end, and you do not need it beign restarted automatically like on Supervisord, eg.:
A task that collects logs everyday and send it on a gzip by email
A backup routine.
Whatever you choose, there are many tutorials on the internet on how configuring both, so I'll not go into this details.
I have a Heroku worker setup to do a long running job which iterates over long periods. However whenever I do an update & deploy of other files in the repo this worker restarts, which is annoying, any way to avoid this?
No. This behaviour is part of Heroku's Automatic Dyno Restarting.
You can't work around this. Instead, you need to build all parts of your app to be able to function properly despite the fact that all dynos will restart at least once every 24 hours or so, whether or not you deploy updates in your repo.
Most significantly, you need to build support for Graceful Shutdown into all your processes (e.g. web process and worker processes).
I have two web dynos in my heroku app, and at times get a dyno automatic restart (as per heroku policy). Is the function that was going on during the restart automatically restored in the new restarted dyno? If not, is there a way I can control this restart?
Is the function that was going on during the restart automatically restored in the new restarted dyno?
no
If not, is there a way I can control this restart?
no
What you can do, is trap the SIGTERM signal that is sent to your process 10 seconds before it is SIGKILLed. This would give you time to finish current computation, stop taking web requests, do cleanup, etc. More details on the process is in the Heroku Devcenter.
A ruby application is showing multiple processes in the server, though it's the same application. It's a windows server.
How can I remove all but one process for this application without manually closing them from the windows task manager.
Pleas help.
Depending on how you're running your app (Passenger? Thin? Mongrel? mod_ruby?), this could actually be normal. As in, the app keeps a pool of processes running until they time out, each awaiting new requests, much like a dynamic php/fastcgi pool would do.
Along the same lines, and per Peter's comment, might it be using threads? If so, it could be equally normal, as in it launches some background jobs before returning and the processes remain around until those jobs are completed.
It is reported that thin opens multiple threads per connection over time and i suppose a thread will be a process.
Try if
thin restart -C /etc/thin/app.yml
helps.
See http://jordanhollinger.com/2011/04/22/how-to-use-thin-effectivly
Limitting your maximum connections, timeout to the minimum will also help.