I am building a small application with dash and flask. Where my user can upload his csv/excel file and have a look at the graphs being generated.
I assume the size of each excel file could be around hardly 50MB max / week.
I have 'ZERO' knowledge on servers and deployment etc. Can anyone guide or enlighten me on this area. Also this application is just for an internal purpose so we are not allowed to go easy on the budget.
My random google searches gave me options like,
1. AWS
2. Heroku
Which would be a right option and why ? Considering price and ease of use.
Thanks !
I will share some of mine web dev knowledge, so.. in my company we use flask for all server dev, using many of his libs(like marshmallow, sqlalchemy, etc) and making improvements to them, flask offers you a big flexibility and fast development, but your request thread is poor, so i highly recommend to use a load balancer, the most famous load balancer for flask is Gunicorn, is easy to set and use. For Http server we use Nginx, its like Apache, but make to work with Websockets more easy, and to use with Gunicorn just make a proxy. For the Host, we use AWS, and work very fine for big and little applications, but your application is small and your budget too, so i recommend use the pythonanywhere server, its easy to use and optimized for python webservers. And for frontend we use Vue.js framework, makes our page more beautiful and fast to dev.
Related
I'm trying out HTTP/2 and would like to know if there's any best practices for making a web application take advantages of the new protocol in both:
Server side and
Client side
And... where (and when?) should I start? What are the least effort changes to take effort of HTTP/2
Update:
Assumption : Users are already using browser that support HTTP/2.
Update 2:
Some good resources on this topic:
https://http2.github.io/faq/
https://www.nginx.com/blog/7-tips-for-faster-http2-performance/
https://www.smashingmagazine.com/2016/02/getting-ready-for-http2/
You don't need any changes to your apps to deploy HTTP/2: most things will work out of the box.
That said, here are a few tips that apply mostly if you are starting from scratch:
Start developing your application using HTTPS. You don't want to have problems with your URLs when you deploy your application because fetching some resources over http:// (e.g. Google fonts or any other CDN resources) does not work anymore. It takes five minutes with some tools (just google "Five minutes to HTTPS development URLs").
Consider if you want to host some resources that you would normally link to a public CDN in your own server. The main performance reason with HTTP/2 for linking to some assets in a CDN, if you are not hosting your entire site on that CDN, is that those resources may be already in the user's cache. If they are not, opening a new connection takes in average half a second.
Don't set up bundling. Set up instead HTTP/2 Push. And then go and grab your favourite module manager. Give a try also to web components and PolymerJS. They are simply awesome when you are not concerned about bundling and round-trips.
If you are using a server that uses machine learning for configuring HTTP/2 Push automatically (e.g, ShimmerCat), then a third tip is to keep the structure of your web pages predictable.
We're currently evaluating Heroku to replace the initial workflow of renting a VPS for a small Web App (since we're working on NodeJS, cPanel hosting plans aren't enough, ergo, VPS).
The confusion lies in Heroku's actual usage as even though it's clear it's used as a platform as a service, there is no Disk (HDD/SSD) limit described.
Web App requirement includes file upload capabilities (profile picture, etc) so I'm not sure Heroku is what we need. Can I get a clear explanation on this?
Not a Heroku expert, but...
You could always use one of the various add-ins that offer database support for storing your images until that no longer works
As the usage of your site scales out, you'd probably want to place static content into a CDN.
I wouldn't consider placing files into Heroku that weren't related to running code and honestly I don't even know if you can.
(I originally just wanted to comment, but need a higher rep :/)
Web applications frameworks such as sinatra (ruby), play (scala), lift (scala) produces a web server listening to a specific port.
I know there are some reasons like security, clustering and, in some cases, performance, that may lead me to use an apache web server in front of my web application one.
Do you have any reasons for this from your experience?
Part of any web application is fully standardized and commoditized functionality. The mature web servers like nginx or apache can do the following things. They can do the following things in a way that is very likely more correct, more efficient, more stable, more secure, more familiar to sysadmins, and more easy to configure than anything you could rewrite in your application server.
Serve static files such as HTML, images, CSS, javascript, fonts, etc
Handle virtual hosting (multiple domains on a single IP address)
URL rewriting
hostname rewriting/redirecting
TLS termination (thanks #emt14)
compression (thanks #JacobusR)
A separate web server provides the ability to serve a "down for maintenance" page while your application server restarts or crashes
Reverse proxies can provide load balancing and fault tolerance for you application framework
Web servers have built-in and tested mechanisms for binding to privileged ports (below 1024) as root and then executing as a non-privileged user. Most web application frameworks do not do this by default.
Mature web servers are battle hardened and stable. By stable, I mean that they quite literally almost never crash. Your web application is almost certainly far less stable. This gives you the ability to at least serve a pretty error page to the user saying your application is down instead of the web browser just displaying a generic "could not connect" error.
Anecdotal case in point: nginx handles attack that would otherwise DoS node.js: http://blog.nodejs.org/2013/10/22/cve-2013-4450-http-server-pipeline-flood-dos/
And just in case you want the semi-official answer from Isaac Schluetter at the Airbnb tech talk on January 30, 2013 around 40 minutes in he addresses the question of whether node is stable & secure enough to serve connections directly to the Internet. His answer is essentially "yes" it is fine. So you can do it and you will probably be fine from a stability and security standpoint (assuming you are using cluster to handle unexpected termination of an app server process), but as detailed above the reality of current operations is that still almost everybody runs node behind a separate web server or reverse proxy/cache.
I would add:
ssl handling
for some servers like apache lots of modules (i.e.
ntml/kerberos authentication)
Web servers are much better for some things compared to your application, like serving static.
Quite often the frameworks do everything you need, but sometimes, adding a layer on top of that can give you seemingly free functionality like compression, security, session management, load balancing, etc. Still, adding a web server may also introduce security issues, for example, chances are your web server security may be compromised easier than Lift by itself. Also, some of the web frameworks are extremely scalable and may even be hampered by an ill chosen web server.
In summary, if you require web server like functionality that is not provided by the framework, then a web server may be a very good option, but keep in mind that it's one more thing to configure properly and update regularly with security patches, etc.
If for example, you just need encryption, or compression, then you may find that adding the correct library or plug-in to your framework may do just that (and only that)
With a proxy http server, the framework doesn't need to keep an http connection open for giving the computed content and can then start serving some other request. It acts as a buffer.
It's an issue of reinventing the wheel. Most frameworks will give you a development environment but for production it's usually good practice to use a commercial/open source project that is able to deal with all issues that arise during production.
Guys building a Framework will have the framework to concentrate on whilst guys building a server are doing just the same(perfecting).
When we do single page application, the webserver basically does only one things, it gives some data when the client asks them (using JSON format for example). So any server side language (php, ror) or tool (apache, ningx) can do it.
But is there a language/tool that works better with this sorts of single page applications that generates lot of small requests that need low latency and sometimes permanent connection (for realtime and push things)?
SocketStream seems like it matches your requirements quite well: "A phenomenally fast real-time web framework for Node.js ... dedicated to creating single-page real time websites."
SocketStream uses WebSockets to get lowest latency for the real-time portion. There are several examples on the site to build from.
If you want a lot of small requests in realtime by pushing data - you should take a look at socket type connections.
Check out Node.js with Socket.io.
If you really want to optimize for speed, you could try implementing a custom HTTP server that just fits your needs, for example with the help of Netty.
It's blazingly fast and has examples for HTTP and WebSocket servers included.
Also, taking a look at GWAN may be worthwile (though I have not tried that one yet).
http://en.wikipedia.org/wiki/Nginx could be appropriate
How does one go about testing the server-side performance of a web application?
I'm working on a small web application (specifically, it will solely be responding to AJAX requests with database rows). I want to see how it performs under load. However, I cannot upload it to an internet host right away. The development environment is, however, part of the local intranet, so I can use as many machines as I want to hammer the development server, presumably using Python in conjunction with urllib2.
My question is, is such an approach really accurate for determining the high-load performance of a server-side script? Is there a better way to do this? Am I missing something here?
Have you tried a tool like Apache JMeter? http://jmeter.apache.org/