There is this example on amazon, a high traffic web application. I noticed that they are using S3 as their content delivery method. I was wondering if I need to have a Web Server for the content delivery and a Web App for my application. I don't understand why they have 2 web servers and 2 web app in the diagram.
And what is the best way to set up a website that serves images and static contents through S3 and the rest of the content through the regular storage.
My last question is, can I consider S3 as a main storage, reliable enough that I can only keep my static content there and don't have a normal storage as a backup ?
That is a very general diagram, specific diagrams will vary depending on the specifics of the overall architecture.
Having said that, I believe the Web Server represents something like Apache or Nginx and the App Server represent something like Rails, Rack Server, Unicorn, Gunicorn, Django, Sinatra, Flask, Jetty, Tomcat, etc. In some cases you can merge the Web Server and the App Server together like for example deploying Apache with python mod_wsgi to run your Django app. (So depends on Architecture)
what is the best way to set up a website that serves images and static
contents through S3 and the rest of the content through the regular
storage.
There's no really best way other than just point your dynamic content to your Databases (SQL and NoSQL) and point your static files to an S3 bucket (images, css, Jquery code, etc) You can also use third party modules depending on your application stack. For example you can accomplish this in Django with the django-storages module. You can find similar modules for other app stacks like Rails.
My last question is, can I consider S3 as a main storage, reliable
enough that I can only keep my static content there and don't have a
normal storage as a backup ?
S3 is pretty reliable, they provide a 99.999999999% reliability of your data. That goes down if you use their RRS (Reduced Redundancy Storage), but if you want to use it you probably want to back up your data in a non RRS bucket anyways. Anyhow, if it's extremely critical data, you are more than free to backup your data somewhere else just in case.
Notice in the diagram that they also recommend using CloudFront for your static files and this is especially useful if your users will be accessing your application from different geographical areas.
Hope this helps.
Related
What is the preferred way of serving static files for an application that is deployed in a microservices architecture (in production)?
Let's say for simplicity that I have 3 application servers and one load-balancer that forwards requests to these servers.
Should the load-balancer store the files and serve them imminently upon request? OR..
Should the load-balancer forward static files requests to the different application instances (each request to a different instance)?
Is there a best practice for this?
As stated in other comments/answers, there are a lot of ways to handle this. This largely depends on what you actually need (version control, access control, http headers, CDN).
Specifically to your question, if it was me, I wouldn't deploy these files on the load balancers, because a newer version of the static files would require a downtime on the load balancers. Instead, I would build very simple Nginx/Caddy containers that their sole purpose is to serve these files, and have the LB route the traffic to those containers.
Best practice would be to store it in a service meant for static content like blob storage (in the cloud this would be Azure Storage, S3, etc.). If necessary, leverage a CDN to improvement latency and throughput to end users.
But as someone else commented, there are many ways to handle this depending on your particular requirements.
I have a project using MEAN stack that uploads imagefiles to a server and the names of the images to db. Then the images are shown for users of the applications kinda like an image gallery.
I have been trying to figure out an effiecent way of storing the imagefiles. atm im storing them under the angular application in a folder /var/www/app/files
What are the usual ways of storing them in a cloud server like digital ocean, heroku and many others.
Im a bit thrown off by the fact they offer many options for datastorage.
Lets say that hundres of thousands of images were uploaded by the application to the server.
Saving all of them in inside your front end app in a subfolder might not be the best solution? or am i wrong with this.
I am very new to these webserver cloud services and how they actually operate.
Can someone clarify on what would be the optimal solution.
Thanks!
Saving all of them in inside your front end app in a subfolder might not be the best solution?
You're very right about this. Over time this will get cluttered, and unless you use some very convoluted logic, will slow down your server.
If you're using Angular and this is in the public folder sent to every user, this is even worse.
The best solution to this is using something like an AWS S3 Bucket (DigitalOcean has Block Storage and I believe Heroku has something a bit different). These services offer storage of files, and essentially act as logic-less servers. You can set some privacy policies and other settings, but there's no runtime like NodeJS that can do logic for you.
Your Node server (or any other server setup) interfaces with this storage server, and handles most of the fetching and storing of files. You can optionally limit these storage services so they can only communicate with your Node server, so any file traffic would be done through your Node server.
I made a mobile application in static html, which is equal to my site wordpress site
The first version was completely static, all texts were in the mobile HTML application.
Today, I updated my application to pull data from the wordpress with AJAX.
The problem is that now, with so many requests being made, the S3 bucket is not being enough.
Despite having decreased from 6kb to 83kb, but it is still more slow because of AJAX..
is it possible put static applications in some other service from Amazon?
For the static content, you should probably be looking at AWS CloudFront instead of S3. As per the page itself:
Amazon CloudFront is a content delivery web service. It integrates with other Amazon Web Services products to give developers and businesses an easy way to distribute content to end users with low latency, high data transfer speeds, and no minimum usage commitments.
Other thing you can leverage is the AJAX caching. That will make your webpage load much faster from the next time. You may also want to using nginx on your server for caching (this will reduce your server load)
I have a bunch of markdown documents in Riak, which I'm exposing via a small Sinatra API with basic search functionality etc.
Each document has an associated image, also stored in Riak (in a different bucket). I'd like to have a client app display the documents alongside their associated images - so I need some way to make the images available, but as I'm only ever going to be requesting them by key it seems wasteful to serve them via a Sinatra app as I'm doing with the documents.
However I'm uneasy with serving them directly from Riak, because a) even using nginx to limit the acceptable requests, I worry about exposing more functionality than we want to and b) Riak throws a 403 for any request where the referrer is set, so by default using a direct-to-Riak url as the src of an img tag doesn't work.
So my question is - what's a good approach to take for serving the images? Add another endpoint to the Sinatra app? Direct from Riak using some Nginx wizardry that is currently beyond me? Or some other approach I haven't considered yet? This would ideally use Ruby as that's what the team I'm working with are more comfortable with.
Not sure if this question might be better suited to Server Fault - if so I'll move it over.
You're right to be concerned about exposing Riak to any direct connectivity. Until 2.0 arrives early next year, there is no security in the system (although the 403 for requests with a referrer is a security mechanism to protect against XSS), and even with security exposing any database directly to the Internet invites disaster.
I've not done anything with nginx, but all you'd really need to use it properly, I'd think, would be two features:
Ability to restrict requests to GET
Ability to restrict (or rewrite) requests to the proper bucket
Ability to strip out all HTTP headers that Riak includes in its result (which, since nginx is a proxy server and not a straight load balancer, seems like it should be straightforward)
Assuming that your images are the only content in that bucket, nginx feels like a reasonable choice here.
For my new web project I am considering to abadon server-side processing of web pages in favor of just using static HTML5 pages. All dynamic content of the page will be loaded using ajax from a REST service. No need for php, jsp, jsf.
I came across this post and it seems I am not the only one.
What are the advantages and disadvantages using this approach?
I can imagine there are more client-server requests since many REST calls have to be made in order to gather all the information needed to display the web page.
I believe that we have much more PROS than CONS. One of the good ideias to give HTML pages trought a WEB app server, like apache, nginx, or ISS, is that you can apply more security and control to the container that is delivered.
BUT, HOWEVER
Use static content, like JS, CSS, and HTML5 to consume only services, is the next goal in software development. When you start to divide things like, API and UX, you can test then separatly, you can develop at same time, service and interface, and you have much more speed and quality to development.
When we look at a web page, and the weight that the DOM have, and how much cost for the app server to give all that container to the user, and sometimes less then 10% of this is JSON from a service, we need to start to rethink the architecture of our web app.
I have been developing apps like this since one year and a half now, all the projects, and be sure, we re not to come back to the past. Its very important to work oriented to services, and how to consume this services.
If you use for example, Amazon S3 to host your HTML,JS, CSS, IMAGES, files, you dont need a app server, only the REST api to consume and give the content to the user. Costless and very,very faster.