If i would want to deploy a site, like ebay, where people will be able to sell something, i would need to store large amounts of images, like many thausends of them and i would need to display about 30 images on some pages.
Normaly i would make a separate server and store them in there, but what else can i do ?
I was trying to store my images on some image hosts like ImageShack, but i have noticed that if i try to request 30 images at same time, some of the images are not showing, looks like imageshack dosnt like to be used this way.
Still there must be some other ways to do it, maybe cloud servers, or other services like imageshack. Does anyone have experience with this ? what would be the best practice beside having a second server ?
Have a file server served by a HTTPD server (e.g. Apache HTTPD) for these static contents.
You can store into the DBMS file locations of the static contents instead of saving as a BLOB this way.
For redirecting traffic, you can store all images (even static files like CSS) in a separate server , like http://staticimg.yourdomain.com . With this way, your web server will be less busy serving download requests.
Related
I have a project using MEAN stack that uploads imagefiles to a server and the names of the images to db. Then the images are shown for users of the applications kinda like an image gallery.
I have been trying to figure out an effiecent way of storing the imagefiles. atm im storing them under the angular application in a folder /var/www/app/files
What are the usual ways of storing them in a cloud server like digital ocean, heroku and many others.
Im a bit thrown off by the fact they offer many options for datastorage.
Lets say that hundres of thousands of images were uploaded by the application to the server.
Saving all of them in inside your front end app in a subfolder might not be the best solution? or am i wrong with this.
I am very new to these webserver cloud services and how they actually operate.
Can someone clarify on what would be the optimal solution.
Thanks!
Saving all of them in inside your front end app in a subfolder might not be the best solution?
You're very right about this. Over time this will get cluttered, and unless you use some very convoluted logic, will slow down your server.
If you're using Angular and this is in the public folder sent to every user, this is even worse.
The best solution to this is using something like an AWS S3 Bucket (DigitalOcean has Block Storage and I believe Heroku has something a bit different). These services offer storage of files, and essentially act as logic-less servers. You can set some privacy policies and other settings, but there's no runtime like NodeJS that can do logic for you.
Your Node server (or any other server setup) interfaces with this storage server, and handles most of the fetching and storing of files. You can optionally limit these storage services so they can only communicate with your Node server, so any file traffic would be done through your Node server.
I've made a website for an arts organisation. The website allows people to browse a database of artists' work. The database is large and the image files for the artists' work come to about 150Gb. I have my own server that is currently just being used to keep the images on its hard-drive.
I'm going to purchase hosting so I don't have to worry about bandwidth etc... but would it be better to purchase hosting that allows me to upload my entire image database or should I use the website to get the images from my server? If so how would I do that?
Sorry I am very new to this
I think it could be better to have the data on the same server so you avoid calls to another server for images which are quite big as you say and this can slow you down overall.
I assume you will need to set up some API on your server to deliver the images or at least URLs for them but then you must make sure they are accessible.
You'll want the image files on the same server as your website, as requests elsewhere to pull in images will definitely hinder your site's performance - especially if you have large files.
Looking at large size of database and consideration of bandwidth, dedicated server will be suitable as they includes large disk spaces and bandwidth. You can install webserver as well as database server on same server inspite of managing them separately. Managing database backups and service monitoring becomes much more easier.
For an instance, you can review dedicated server configuration and resources here :- https://www.accuwebhosting.com/dedicated-servers
I got a page, with not that much bandwidth, therefore I want to store the images externally on another server, that offers unlimited bandwidth. Any suggestions on how to do this, or maybe a better solution?
Image storage on different server
Check out this similar post, Facebook does it, Google does it, so it's a preferred solution to store images on another server. You can assign links to the images dynamically or statically from the external server and that's all you need to do! You need to take care of the hierarchy how the images are to be stored in the external server.
I'm programming a application with mvc2. Users should be able to upload images to their profile.
The best way to save the images is to save them in a database, but I think it is the most expensive one too. (I'm using MSSql)
I thought the best way would be to save them on the server. I thought about: a User uploads images, the server resize them and save the Image on the server and the image-path in a database.
But what about if I do not have any more capacities on my server an I have to use a second one or third one?
My question: what is the best way to handle images on a server? What is the best way to be flexible?
Thanks for your answers!!!
There are a number of factors to consider.
The best way isn't necessarily storing images in a database. That can be a good choice, especially if you want to implement access control on the images. However, this comes at the cost of having to pull the image from the database and loading it into memory so that it can be streamed out by a server.
However, as these are profile images, and presumably visible by anyone who visits that user's profile, I'd advocate storing the file as a file on the server and storing a reference to that file in your database.
When it comes down to it, web servers are very good at serving files efficiently. If you can make use of that, you should.
Finally, you have concerns about space. In the first instance, you can prevent images of a certain size from being uploaded in the first place. You can also, as you suggest, auto-crop to a selected size.
If sheer volume of users becomes a problem, you can always store your media on a separate server, storing a fully qualified link to each resource, eg :-
<img src="http://images2.mydomain.com/image/profile_123.png" alt="A profile pic" />
What is the benefits of seperating css,js and media folders under subdomains like
css.domain-name.com
js.domain-name.com
media.domain-name.com
I know that scalibilty begin from static/media files but does serving them from subdomain has any advantage ?
If so, in which degree should I do that ? For example, if I allowed to photo uploads, should I put my "uploads" folder under media subdomain ?
Thanks
I'd separate uploads from static files used in the generic layout (e.g. logos, icons, etc.) so its a lot easier to clear the existing files to upload a new design without having to care for the uploads to not be deleted/overwritten.
As for the domain names, I wouldn't split the files that way. One sub domain for static files, one for uploads - fine. But I wouldn't go as far as adding one for scripts or stylesheets.
Using sub domains can have advantages though, depending on the web server you can configure the whole virtual host to adhere to specific rules, e.g. not providing directory listings or not allowing access to any files other than images - or refusing to deliver hotlinked files (without having to worry about specific sub directories). It can as well make it easier to move the files to another host later on, e.g. for media files or downloads to a cloud hosting service.
Considering your example I'd use the following sub domains:
www.domain-name.com (basic web presence)
static.domain-name.com or media.domain-name.com (serving support files like js, css, images, etc. - stuff that doesn't change and can be cached for a long time)
uploads.domani-name.com (serving uploaded files)
Don't overcomplicate it as you're not gaining any additional performance that way (unless utilizing different servers and you're expecting heavy load). In fact page load might be slower (due to additional DNS lookups) and you might encounter security limitations, e.g. regarding cookie validity/accessibility or cross domain scripting.
There are mainly two reasons for doing this
Scaling - static content and dynamic content has other scaling parameters. The more you may differ between webservers serving dynamic and static contents. Based on this you may scale different based on your websites requirements. E.g if you host a photo site you will end up having 10times more static servers than dynamic sites. Static servers are usually much more lightweight than full featured application servers.
Cookies - Cookies are always sent to the domain they are assigned to. So cookies will be sent to e.g. www.xyz.com and not to sub.xyz.com
Probably it makes no sense to go more into detail than static[1-n].xyz.com. But that really depends on what you want to do.
To you "upload" folder question. Preferable the images uploaded to your main domain will be served by a static server (serving contents on your subdomain).
For JS, this seems like a bad idea. There are security restrictions to what you can do using JS when dealing with a different domain, which would be the case here.
For media files and downloads, basically BLOB storage, the story is a bit different. For high-traffic sites, it may be required to separate this in order to create a good load-balancing structure and to avoid putting unnecessary load on the web application servers (the media subdomain can point to different servers, thus reducing the load on the app servers while allowing to create massive load balancing for just the binary data serving).