I'm a little bit confused about the whole dyno business with heroku . On their site, they define a dyno as:
A dyno is a lightweight Linux container that runs a single
user-specified command. A dyno can run any command available in its
default environment (what we supply in the Cedar stack) or in your
app’s slug (a compressed and pre-packaged copy of your application and
its dependencies).
This (somewhat?) makes sense in my head, but when I think about dynos in the context of multiple dynos running for one web app, my brain gets twisted.
Lets I'm building a web app with a server which runs a very important task only once every 3 hours for all users. If I am running multiple web dynos for this site, does that mean there is a separate server instance running on each dyno? And would that very-important-task that is run every 3 hours be run on each dyno every 3 hours?
Thanks a lot for any clarification!
Each dyno is an LXC container running in one of the heroku instances.
Depending of your dyno size, there might be other containers on the same instance or not.
But the rough idea is that yes, each dyno you have running is a different instance, and a task set to run on any web dyno each 3 hours will run on all of them.
You may want to look into the heroku scheduler addon. It will run a one-off dyno at a specified interval, allowing you to run cron-like tasks.
Related
I am currently using a Hobby dyno to run my web process -- frontend website as well as an API. To execute jobs, I need to run a background process that processes these jobs. Do I need to spin up another dyno that specifically runs background workers? I ask this because I see that Standard 1X/2x dynos include "unlimited background workers" which makes me think that multiple process types can run on a single dyno. It looks like I can run 2 hobby dynos -- one for web, one for workers or upgrade to one of the standard dynos...is this correct?
I believe that yes, if you want to have a dedicated worker app, that needs to be a separate dyno. You can run the processes in memory though and that saves you doing it with a separate binary/dyno
I have deployed my node app on heroku free web dyno plan.I want to know how much free hours are remaining and how much are left so for that I am using
heroku ps -a <AppName>
After running above command I got something like this below:
As per the result everything is clear but what does Web(Free) mean written in green color. Someone please let me know any help would be appreciated.
THANKS
It means your app is running on a single web dyno and free dyno type.
Dyno configurations
Every dyno belongs to one of the three following configurations:
Web: Web dynos are dynos of the “web” process type that is defined in your Procfile. Only web dynos receive HTTP traffic from the routers.
Worker: Worker dynos can be of any process type declared in your Procfile, other than “web”. Worker dynos are typically used for background jobs, queueing systems, and timed jobs. You can have multiple kinds of worker dynos in your application. For example, one for urgent jobs and another for long-running jobs. For more information, see Worker Dynos, Background Jobs and Queueing.
One-off: One-off dynos are temporary dynos that can run detached, or with their input/output attached to your local terminal. They’re loaded with your latest release. They can be used to handle administrative tasks, such as database migrations and console sessions. They can also be used to run occasional background work, as with Heroku Scheduler. For more information, see One-Off Dynos.
Once a web or worker dyno is started, the dyno formation of your app will change (the number of running dynos of each process type) - and subject to dyno lifecycle, Heroku will continue to maintain that dyno formation until you change it. One-off dynos, on the other hand, are only expected to run a short-lived command and then exit, not affecting your dyno formation.
Dyno Types
Heroku provides a number of different dyno types each with a set of unique properties and performance characteristics. Free, Hobby, Standard and Performance dynos are available in the Common Runtime to all Heroku customers. Private Dynos only run in Private Spaces and are available in Heroku Enterprise.
I subscribed a Hobby plan in Heroku.
The details of the plan specifies that it allows up to 10 Process Types.
So I developed an app with the following Procfile:
backend-dev: node ./backend-dev/backend.js
backend-prod: node ./backend-prod/backend.js
Which represents 2 Process Types, right ?
But when I run it with:
heroku ps:scale backend-dev=1
heroku ps:scale backend-prod=1
I end up with two Hobby Dynos...
As the plan also specifies 7€/month/Dyno I am billed 14€/month.
So my questions are:
What is the difference between Process Types and Dynos?
Can I run 2 Process Types within a single Dyno?
Can I run for instance 1 free Dyno (for backend-dev) and 1 Hobby Dyno (for backend-prod)?
Consider this simple example of web application with background worker, so it has web process and worker process. When such app receives a lot of web traffic, but processes very few background jobs, you can increase the number of dynos for your web process, but have only one dyno for worker process. It is also possible to have different dyno size per process. Instead of using more dynos, you can use performance-l dyno for web process and standard-1x for worker process. In other words, Process Types describe different processes that are working together within one application. They are not supposed to be different applications like in your case.
No. You can run one Process Type on multiple dynos.
Technically you can run one process on free dyno and another on hobby, but it won't work in your case. When you upgrade to professional dynos, then all processes must run on professional dynos.
Your Procfile is all wrong. You must have Process Type name web to receive web traffic. If you start your current setup, you will be running two processes, but they will never receive any web requests. It is described in Heroku docs, only web process can receive web traffic and you can only have one such process. So to run two versions of your app, you need to create two different Heroku applications. And ideally you should allow to configure your app via environmental variables so you can deploy the same code to both apps.
If I have an app on Heroku that consists of one worker and one or no web dynos, will it run? I'm unsure if the absent or idling web dynos will cause the worker dyno not to run.
Heroku doesn't just run web dynos, in fact, it makes no assumptions at all with regards to the processes you're running. There's absolutely nothing wrong with launching a single worker process.
This is actually a common scenario for me to deploy single cron-like tasks to Heroku, I've written about it here http://blog.y3xz.com/blog/2012/11/16/deploying-periodical-tasks-on-heroku/
If you are looking for cron-like tasks for simple jobs (like I am), now you have another alternative: Heroku Scheduler. It is easy to configure in a dashboard.
Advantage:
No need to choose and learn a new scheduler library. Configure it in seconds.
Same way for different platforms: Python, Ruby, etc.
Save Dyno Hours for Free Plan user. Only the actual working time counts. Some scheduler library (like Rufus Scheduler) will keep running between launches (so that it does not rely on cron to work).
Disadvantage:
Trivial options. You can only choose among "Daily"/"Hourly"/"Every 10 minutes".
Conclusion: Best for basic use.
I'm hoping the community can clarify something for me, and that others can benefit.
My understanding is that gunicorn worker processes are essentially virtual replicas of Heroku web dynos. In other words, Gunicorn's worker processes should not be confused with Heroku's worker processes (e.g. Django Celery Tasks).
This is because Gunicorn worker processes are focused on handling web requests (basically throttling up the performance of the Heroku Web Dyno) while Heroku Worker Dynos specialize in Remote API calls, etc that are long-running background tasks.
I have a simple Django app that makes decent use of Remote APIs and I want to optimize the resource balance. I am also querying a PostgreSQL database on most requests.
I know that this is very much an oversimplification, but am I thinking about things correctly?
Some relevant info:
https://devcenter.heroku.com/articles/process-model
https://devcenter.heroku.com/articles/background-jobs-queueing
https://devcenter.heroku.com/articles/django#running-a-worker
http://gunicorn.org/configure.html#workers
http://v3.mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/
https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/gunicorn/
Other Quasi-Related Helpful SO Questions for those researching this topic:
Troubleshooting Site Slowness on a Nginx + Gunicorn + Django Stack
Performance degradation for Django with Gunicorn deployed into Heroku
Configuring gunicorn for Django on Heroku
Troubleshooting Site Slowness on a Nginx + Gunicorn + Django Stack
To provide an answer and prevent people from having to search through the comments, a dyno is like an entire computer. Using the Procfile, you give each of your dynos one command to run, and it cranks away on that command, re-running it periodically to refresh it and re-running it when it crashes. As you can imagine, it's rather wasteful to waste an entire computer running a single-threaded webserver, and that's where Gunicorn comes in.
The Gunicorn master thread does nothing but act as a proxy server, spawning a given number of copies of your application (workers), distributing HTTP requests amongst them. It takes advantage of the fact that each dyno actually has multiple cores. As someone mentioned, the number of workers you should choose depends on how much memory your app takes to run.
Contrary to what Bob Spryn said in the last comment, there are other ways of exploiting this opportunity for parallelism to run separate servers on the same dyno. The easiest way is to make a separate sub-procfile and run the all-Python Foreman equivalent, Honcho, from your main Procfile, following these directions. Essentially, in this case your single dyno command is a program that manages multiple single commands. It's kind of like being granted one wish from a genie, and making that wish be for 4 more wishes.
The advantage of this is you get to take full advantage of your dynos' capacity. The disadvantage of this approach is that you lose the ability scale individual parts of your app independently when they're sharing a dyno. When you scale the dyno, it will scale everything you've multiplexed onto it, which may not be desired. You will probably have to use diagnostics to decide when a service should be put on its own dedicated dyno.