Writing an application for a client that is very photo focused. Users will be uploading hundreds of images to their account. These images are only accessible to the owners. Our first thought was to throw them all out on AmazonS3 but after doing some research we have found that accessing protected files on S3 is too slow for the kind of response we need. Also, from further research, we're not sure that non-US users are going to have a good experience because of S3 (still researching that one).
I'm wondering if I'm finding the right information or not. Is it possible to host images on S3 that have specific user rights and are pulled down quickly? Would we need to implement some sort of caching proxy to speed things up? Should I be looking at hosting the images on our own servers and delivering them that way? Thanks for any feedback.
Are you using the CloudFront CDN in front of S3? That might be why it's a bit slow.
If you don't like Amazon, you could use CloudFiles from Rackspacecloud. That uses Limelight as a CDN.
Related
first post here although i've been reading for about 7 years :). Please take it easy on me if there are things that are wrong with my post.
I am working on a social network/marketplace for online education. The business problem i am dealing with is how to delegate the cost of hosting (most importantly video content) to the vendors themselves, in order to simplify the platform's relationship with them.
As an example, Patreon.com integrates with vimeo to upload videos from patreon website straight to vimeo, thus the platform is not incurring the cost of hosting creator videos.
My question is if it would be possible to do the same thing with amazon s3 or digital ocean s3. Instead of our app (laravel+vue) to have it's own hosting on amazon s3, each user of our app would be able to connect their own s3 drives, access his buckets/objects and share them on our platform. That way they can pay their own bills for hosting and we don't have to deal with "re-selling" hosting.
Has anyone gone the route of allowing their app users to enter their own S3 hosting credentials, store them in the db, and when retrieving the file, setting those credentials on the fly, as ml59 suggested, and accessing the user's s3 files.
Thanks,
hopefull someone can shed some light on this topic and here are some of my questions:
How do big ecommerce websites manage their images for their websites?
Is there any Best Practices should be consider when deciding where to keep websites images?
I have heard to keep the images in multiple folder structure on the same server where the website is hosted so websites can render them easily and fast since they are all on same server - Is this the idea solution?
How do professionals or big ecommerce handle images storage and maintain website images reliability and stability?
Is Azure or AWS etc best place to store images for websites rendering?
Thanks in advance!
Keeping the files in the same server comes with more risk, if your server crashes or region goes down your application stopped as well as your files not gonna render if you are using those files separately for different applications like mobile applications.
In this case, users will also face a high loading time for those media files if the users are not in the same zone as your application hosted.
The best practice to store the image/media files on some cloud storage like S3 or Azure Blob then connect it with some CDN like CloudFront or Azure CDN.
Now you can serve your media files via CDN which will act as a global caching system for your media files.
I've come across this issue where I don't know of any reliable places where I could upload all these images for my personal GitHub page.
I don't want to upload them directly to the GitHub repo and take up storage space.
AWS and Google Cloud seem too expensive for this small website.
Has anyone had any experience using Google Photos, Dropbox or Imgur for hosting images that get referenced on their websites?
Did that scale well?
Are there any other solutions out there I'm unaware of?
Questions asking to find an off-site resource are off-topic to Stack Overflow, but I'll throw a couple ideas out there:
I'd start by just doing a Google search for "static file hosting" for a ton of results. Any of the options you mentioned will probably work, but it might be easier if you use something specifically designed to host files that are used by a website. Put another way, you could also upload all of your images to Facebook and use the URLs of the images on your site. But that's not really what it's designed for.
You could use Amazon S3, which you can use to host static files. See also:
Hosting a Static Website on Amazon S3
How Do I Configure an S3 Bucket for Static Website Hosting?
Moving a Static Site to S3 Before My Girlfriend Got Out of the Shower
Similarly, you could use Google Cloud Storage. See also:
Hosting a static website with Cloud Storage
Hosting a Static Website
Static Website Examples, Troubleshooting and Tips
I found all of the above using pretty basic Google searches, so that's where you should start. The only way to really answer this question is for you to try out a bunch of different things and see which one you like best.
This a very simple question, but I haven't found the answer anywhere.
I am thinking about moving one of my websites, a joomla website, to the cloud, more specifically to a EC2 instance on Amazon. I have been watching some videos and the process seems rather simple. However, I haven't found any information about where the files are stored.
Are they in a S3 bucket? Are they saved somewhere else? How do I access the files?
Can I use Cloudfront to serve images and other files?
How does the whole integration process between EC2, S3 and Cloudfront work for a hosted website?
Thank you!
In my experience With EC2, you interact with the instance just as you would with any headless server in the basement. You can connect over SSH and install software, store files, etc, as normal.
It's great if you want a whole server to play with, but if it's just website hosting you're after, it might be overkill.
S3 can serve up static pages, but you can't install any code on there.
I don't know about cloudfront.
It seems to be though that what you need is standard website hosting.
I'd like to save some of my site monthly bandwidth allocation and I'm wondering if I can use Flickr PRO or I should rely on Amazon S3 as an hosting service for my web site images. (My Web Application allows users to upload their own pictures and at the moment it's managing around 40GB of data)
I've never used Amazon's services and I like the idea of using Flickr REST Api do dynamically upload images from my webApp.
I like also the idea of having virtually unlimited space to store images on Flickr for only 25$/year but I'm not sure if I can use their service on my web site.
I think that my account can be banned if I use Flickr services to store images (uploaded by users of my website) that are not only for 'personal use'.
What's your experience and would you suggest other services rather than Amazon's S3 or is this the only available option at the moment?
Thanks
edit: Flickr explicitly says 'Don’t use Flickr for commercial purpose', you could always contact them to ask to evaluate your request but it sounds to me like I can't use their services to achieve what I want. S3 looks like the way to go then...
Even though a rough estimate of what I'm going to spend every month is still scaring
5000 visit/day
* 400 img/user (avg 50kB/image)
* 30 days
= ~3TB of traffic
* 0.15$/GB (Amazon S3)
= 429$/month
is there any cheaper place to host my images?
400 images per user seems high? Is that figure from actual stats?
Amazon S3 is great and it just works!
A possible cheaper option is Google. Google docs now supports all file types, so you can load the images up to a Google docs folder, and share the folder for public access. The URL's are kind of long e.g.
http://lh6.ggpht.com/VMLEHAa3kSHEoRr7AchhQ6HEzHVTn1b7Mf-whpxmPlpdrRfPW216UhYdQy3pzIe4f8Q7PKXN79AD4eRqu1obC7I
Add the =s paramter to scale the image, cool! e.g. for 200 pixels wide
http://lh6.ggpht.com/VMLEHAa3kSHEoRr7AchhQ6HEzHVTn1b7Mf-whpxmPlpdrRfPW216UhYdQy3pzIe4f8Q7PKXN79AD4eRqu1obC7I=s200
Google only charge USD5/year for 20GB. There is a full API for uploading docs etc
I love amazon S3. There are so many great code libraries (LitS3) and browser plugins (S3Fox) and upload widgets (Flajaxian) that make it really easy to use.
And you only pay for what you use. I use it a lot and have only ever experienced down time on one occasion.
Nivanix is an s3 competitor. I haven't used them, but they have a bit more functionality (image resizing) etc.
Edit:The link about Nivanix is dead now(2015/07/21), because Nivanix was dead.