What is the benefits of seperating css,js and media folders under subdomains like
css.domain-name.com
js.domain-name.com
media.domain-name.com
I know that scalibilty begin from static/media files but does serving them from subdomain has any advantage ?
If so, in which degree should I do that ? For example, if I allowed to photo uploads, should I put my "uploads" folder under media subdomain ?
Thanks
I'd separate uploads from static files used in the generic layout (e.g. logos, icons, etc.) so its a lot easier to clear the existing files to upload a new design without having to care for the uploads to not be deleted/overwritten.
As for the domain names, I wouldn't split the files that way. One sub domain for static files, one for uploads - fine. But I wouldn't go as far as adding one for scripts or stylesheets.
Using sub domains can have advantages though, depending on the web server you can configure the whole virtual host to adhere to specific rules, e.g. not providing directory listings or not allowing access to any files other than images - or refusing to deliver hotlinked files (without having to worry about specific sub directories). It can as well make it easier to move the files to another host later on, e.g. for media files or downloads to a cloud hosting service.
Considering your example I'd use the following sub domains:
www.domain-name.com (basic web presence)
static.domain-name.com or media.domain-name.com (serving support files like js, css, images, etc. - stuff that doesn't change and can be cached for a long time)
uploads.domani-name.com (serving uploaded files)
Don't overcomplicate it as you're not gaining any additional performance that way (unless utilizing different servers and you're expecting heavy load). In fact page load might be slower (due to additional DNS lookups) and you might encounter security limitations, e.g. regarding cookie validity/accessibility or cross domain scripting.
There are mainly two reasons for doing this
Scaling - static content and dynamic content has other scaling parameters. The more you may differ between webservers serving dynamic and static contents. Based on this you may scale different based on your websites requirements. E.g if you host a photo site you will end up having 10times more static servers than dynamic sites. Static servers are usually much more lightweight than full featured application servers.
Cookies - Cookies are always sent to the domain they are assigned to. So cookies will be sent to e.g. www.xyz.com and not to sub.xyz.com
Probably it makes no sense to go more into detail than static[1-n].xyz.com. But that really depends on what you want to do.
To you "upload" folder question. Preferable the images uploaded to your main domain will be served by a static server (serving contents on your subdomain).
For JS, this seems like a bad idea. There are security restrictions to what you can do using JS when dealing with a different domain, which would be the case here.
For media files and downloads, basically BLOB storage, the story is a bit different. For high-traffic sites, it may be required to separate this in order to create a good load-balancing structure and to avoid putting unnecessary load on the web application servers (the media subdomain can point to different servers, thus reducing the load on the app servers while allowing to create massive load balancing for just the binary data serving).
Related
I've made a website for an arts organisation. The website allows people to browse a database of artists' work. The database is large and the image files for the artists' work come to about 150Gb. I have my own server that is currently just being used to keep the images on its hard-drive.
I'm going to purchase hosting so I don't have to worry about bandwidth etc... but would it be better to purchase hosting that allows me to upload my entire image database or should I use the website to get the images from my server? If so how would I do that?
Sorry I am very new to this
I think it could be better to have the data on the same server so you avoid calls to another server for images which are quite big as you say and this can slow you down overall.
I assume you will need to set up some API on your server to deliver the images or at least URLs for them but then you must make sure they are accessible.
You'll want the image files on the same server as your website, as requests elsewhere to pull in images will definitely hinder your site's performance - especially if you have large files.
Looking at large size of database and consideration of bandwidth, dedicated server will be suitable as they includes large disk spaces and bandwidth. You can install webserver as well as database server on same server inspite of managing them separately. Managing database backups and service monitoring becomes much more easier.
For an instance, you can review dedicated server configuration and resources here :- https://www.accuwebhosting.com/dedicated-servers
I have this small technical question about caching...
I am planning to use caching for my website and I was wondering if the cached file where save on visitors personal computers !?
I asked somebody and told me that they are saved on HTML files, and these are not on visitors Personal PC
Regards
That depends on what you mean by Cache. Most sites use caching to save bandwidth by reducing the hits to the database or other server resources by not having to re-generate dynamic content on every request. On the other hand, browsers will cache JavaScript and CSS files from websites on the local computer as a part of their normal process. Cookies are 'caching' important information specific to that computer / user and are also stored by the browser locally.
I am assuming that your talking about pages on a server, reusing them for multiple requests. That can be stored as tmp files or as entries in a database on the server (CakePHP and CSP comes to mind here). It really depends on your configuration and what you decide you want to do.
There is this example on amazon, a high traffic web application. I noticed that they are using S3 as their content delivery method. I was wondering if I need to have a Web Server for the content delivery and a Web App for my application. I don't understand why they have 2 web servers and 2 web app in the diagram.
And what is the best way to set up a website that serves images and static contents through S3 and the rest of the content through the regular storage.
My last question is, can I consider S3 as a main storage, reliable enough that I can only keep my static content there and don't have a normal storage as a backup ?
That is a very general diagram, specific diagrams will vary depending on the specifics of the overall architecture.
Having said that, I believe the Web Server represents something like Apache or Nginx and the App Server represent something like Rails, Rack Server, Unicorn, Gunicorn, Django, Sinatra, Flask, Jetty, Tomcat, etc. In some cases you can merge the Web Server and the App Server together like for example deploying Apache with python mod_wsgi to run your Django app. (So depends on Architecture)
what is the best way to set up a website that serves images and static
contents through S3 and the rest of the content through the regular
storage.
There's no really best way other than just point your dynamic content to your Databases (SQL and NoSQL) and point your static files to an S3 bucket (images, css, Jquery code, etc) You can also use third party modules depending on your application stack. For example you can accomplish this in Django with the django-storages module. You can find similar modules for other app stacks like Rails.
My last question is, can I consider S3 as a main storage, reliable
enough that I can only keep my static content there and don't have a
normal storage as a backup ?
S3 is pretty reliable, they provide a 99.999999999% reliability of your data. That goes down if you use their RRS (Reduced Redundancy Storage), but if you want to use it you probably want to back up your data in a non RRS bucket anyways. Anyhow, if it's extremely critical data, you are more than free to backup your data somewhere else just in case.
Notice in the diagram that they also recommend using CloudFront for your static files and this is especially useful if your users will be accessing your application from different geographical areas.
Hope this helps.
If i would want to deploy a site, like ebay, where people will be able to sell something, i would need to store large amounts of images, like many thausends of them and i would need to display about 30 images on some pages.
Normaly i would make a separate server and store them in there, but what else can i do ?
I was trying to store my images on some image hosts like ImageShack, but i have noticed that if i try to request 30 images at same time, some of the images are not showing, looks like imageshack dosnt like to be used this way.
Still there must be some other ways to do it, maybe cloud servers, or other services like imageshack. Does anyone have experience with this ? what would be the best practice beside having a second server ?
Have a file server served by a HTTPD server (e.g. Apache HTTPD) for these static contents.
You can store into the DBMS file locations of the static contents instead of saving as a BLOB this way.
For redirecting traffic, you can store all images (even static files like CSS) in a separate server , like http://staticimg.yourdomain.com . With this way, your web server will be less busy serving download requests.
I can see that all big sites store the images on a complete different server. What are the benefits of this practice?
Load balancing.
Separation of dynamic and static content.
Static content is served from servers which are geographically (or in network "length") close to the client.
(Update) forgot to mention that browsers used to limit the number of concurrent requests to the same server or domain (don't know if it's still used) and using different domain names allowed the server to bypass this limitation.
This way each kind of server serves resources it's tuned up for so clients get pages faster.
This way, the browser won't send cookies when requesting images.
It also enables the use of location-aware CDNs for images only.