There is an application in Django + Gunicorn (with Uvicorn workers) + Nginx. The application is built WITHOUT the use of multiprocessing and microservices (yet).
As expected, the use of multiple workers leads to errors due to the lack of shared RAM.
Is it possible to configure Gunicorn (or similar servers) in such a way that different workers only handle their own groups of urls?
For example:
Worker №1 handle only urls started with '/group1/...' (for big files, static).
Worker №2 handle only urls started with '/group2/...' (for user data calculations).
Feel free to call the question stupid. And thanks for any ideas :)
You need some kind of WSGI middleware to handle the mapping of routes.
Here is an example: https://github.com/lukearno/selector
Gunicorn just handles management of worker processes for a WSGI server.
Related
Is there a way to deploy a FastAPI application so that memory state cannot be persisted between requests? The goal is to avoid leaking any data between requests in a multi-tenant application.
Starting up the application from scratch for every request seems not feasible since it takes too long. Is there a way in which the application is launched for every instance of the service but individual requests are handled by workers or threads that get purged after the request is handled so that any static property, singleton instance and such is destroyed and the next request is handled with clean memory?
FastAPI is basically stateless by default. It actually takes extra work to persist data across requests through methods such connection pooling, reading a value from Redis, and so on. If you consider things such as starting up the server, loading a configuration, setting up path redirects, and so on to be "state", then FastAPI will not work for your purposes.
When you say "memory state", it sounds like you are trying to partition off instances of FastAPI server from each other so that they do not even use the same memory. This is not going to be a viable solution because most web servers, FastAPI included, are not designed for this type of segregating. By default, the requests from one tenant will not have anything to do with the requests from another tenant unless you write additional code that allows them to become related; so separating the concerns of the different tenants becomes a matter for the programmer, not the server's memory.
Instead, if you absolutely cannot let requests from multiple tenants inhabit the same memory, you'd be better off giving different tenants their own subdomain on the DNS level. Spin up a VPS and instance of your FastAPI program for each of them. That will truly prevent the requests from one tenant share any memory or state with the others.
I am building a web api service that has two components: node.js and java.
Every request reaches the node service which in turn makes an API call to the java service. I implemented this architecture using two different heroku apps, with both services implemented as "web" dynos. It works, but it will be easier to manage as a single app.
I don't fully understand what are the options and the process for combining the two components in the same application. I guess I can make two entries in the Procfile, but I don't understand how the request routing could work. How can the node "web" dyno make requests to the java dyno? Is there some mechanism for inter-dyno requests?
Heroku only allows the "web" dyno to accept network connections (well, HTTP at least...). The docs say specifically "The web process type is special as it’s the only process type that will receive HTTP traffic from Heroku’s routers. Other process types can be named arbitrarily."
Looking at a similar question, my $.02 would be that an MQ-based solution (or something MQ-ish, like Redis pub/sub) would be the way to go. The rub is that it works best if the API call is asynchronous. For example, you could have the Node app publish to a Redis channel, and have your Java processes (defined via a 'worker' dyno in your Procfile). If the Node app actually needs the results as part of it's response... hm... I suppose you'd have to build something going the opposite direction, and include enough data in the message or channel structure to match up responses with the originating request.
In that case, you might just be better off sticking with the multi-app configuration. Though I have not tested the FIFO solution mentioned there, I'm not clear how it would work since the dynos are isolated from each other.
Is it possible to run 2 different web process in one instance, like
for /url1/ some process
and
for /url2/ another
or some port manipulation?
Heroku has the concept of Dynos. They can roughly be thought of as processes.
It is possible for multiple processes to run under each Dyno. Requests are load balanced by Heroku. No special configuration is required.
For example, the Unicorn web server can be configured to spawn "worker" processes. Each "worker" serves a HTTP request. So, if you configure Unicorn to have 3 workers, 2 Dynos would give you 6 processes to serve HTTP requests.
These articles should help you understand how Heroku works:
Dynos and the Dyno Manifold
The Process Model
I'm hoping the community can clarify something for me, and that others can benefit.
My understanding is that gunicorn worker processes are essentially virtual replicas of Heroku web dynos. In other words, Gunicorn's worker processes should not be confused with Heroku's worker processes (e.g. Django Celery Tasks).
This is because Gunicorn worker processes are focused on handling web requests (basically throttling up the performance of the Heroku Web Dyno) while Heroku Worker Dynos specialize in Remote API calls, etc that are long-running background tasks.
I have a simple Django app that makes decent use of Remote APIs and I want to optimize the resource balance. I am also querying a PostgreSQL database on most requests.
I know that this is very much an oversimplification, but am I thinking about things correctly?
Some relevant info:
https://devcenter.heroku.com/articles/process-model
https://devcenter.heroku.com/articles/background-jobs-queueing
https://devcenter.heroku.com/articles/django#running-a-worker
http://gunicorn.org/configure.html#workers
http://v3.mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/
https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/gunicorn/
Other Quasi-Related Helpful SO Questions for those researching this topic:
Troubleshooting Site Slowness on a Nginx + Gunicorn + Django Stack
Performance degradation for Django with Gunicorn deployed into Heroku
Configuring gunicorn for Django on Heroku
Troubleshooting Site Slowness on a Nginx + Gunicorn + Django Stack
To provide an answer and prevent people from having to search through the comments, a dyno is like an entire computer. Using the Procfile, you give each of your dynos one command to run, and it cranks away on that command, re-running it periodically to refresh it and re-running it when it crashes. As you can imagine, it's rather wasteful to waste an entire computer running a single-threaded webserver, and that's where Gunicorn comes in.
The Gunicorn master thread does nothing but act as a proxy server, spawning a given number of copies of your application (workers), distributing HTTP requests amongst them. It takes advantage of the fact that each dyno actually has multiple cores. As someone mentioned, the number of workers you should choose depends on how much memory your app takes to run.
Contrary to what Bob Spryn said in the last comment, there are other ways of exploiting this opportunity for parallelism to run separate servers on the same dyno. The easiest way is to make a separate sub-procfile and run the all-Python Foreman equivalent, Honcho, from your main Procfile, following these directions. Essentially, in this case your single dyno command is a program that manages multiple single commands. It's kind of like being granted one wish from a genie, and making that wish be for 4 more wishes.
The advantage of this is you get to take full advantage of your dynos' capacity. The disadvantage of this approach is that you lose the ability scale individual parts of your app independently when they're sharing a dyno. When you scale the dyno, it will scale everything you've multiplexed onto it, which may not be desired. You will probably have to use diagnostics to decide when a service should be put on its own dedicated dyno.
I have an ajax application built in catalyst running through fastcgi. I am noticing that all of my ajax requests spin up another process and that process sticks around indefinitely.
Ideally I would like to have my main page stick around but the ajax pages should just stop and restart as needed.
Is this a common issue and how does one get around this.
How do I tell the server to shut down inactive processes quicker?
The answer is web server dependent (and fastcgi module dependent for apache - as there are at least 2 different fcgi modules). You haven't told us which web server etc, so I can't really answer. (But consult the documentation for your web server / fastcgi module)
An alternative approach would of course be to run the server with external fastcgi, rather than in dynamic mode, which would mean you start a fixed number of processes up-front, but then never create more or less than this number to serve requests..