I have an Umbraco 8.2.2 application which is hosted in AWS EC2 server.
Recently, I encounter server availability issues that caused downtimes once in a while.
One of the solutions I've thought about is to maintain an additional AWS EC2 server which hosts the same application (Same code, same database) and configure load balancing between them.
It will host both client and server.
To what extent is this possible, in your experience?
How can I handle obstacles like shared media & cache folders, as they should be the same?
I've heard about S3 as an option.
What additional obstacles may I face, and what should I put my focus on?
Thanks.
This sounds like a good use case for Amazon EFS which offers you a shared POSIX file system. You can mount the directories where your media and cache folders are located to the EFS share and then mount the EFS share to the backend EC2 instances that are behind the load balancer. This solution requires very little or no changes to your application itself, you will just be changing the storage media for certain files in your application.
As for obstacles, EFS is a network filesystem, therefore, it is generally not recommended to execute code from your EFS share or to use it for applications that require very low storage latency. If that's the case then you can consider Amazon FSx but that's a very expensive solution.
If you can't avoid executing your code from EFS, just try it out and see how it affects your application's performance. EFS works fine for plenty of web application use cases. Here is a tutorial on how to host a simple website using EFS behind a load balanced environment to get you started.
If EFS is not an option, then you could try to offload your static content to Amazon S3 and serve it through CloudFront. This is probably a cheaper option and offloads a lot of traffic from your load balancer and EC2 instances but it is also probably more work because you have to refractor your application to serve your content through CloudFront. Here is a tutorial (there are plenty more online) on how to create a static website that serves content through CloudFront. In your case, you would be serving the content (i.e. your media files) through S3/CloudFront and then update the links used in your applications to retrieve that content from the CloudFront endpoint instead of retrieving them directly from your application/load balancer endpoint... so the work you need to do is on two fronts, setting up the S3/CloudFront environment, and configuring your application to offload the content to S3 and serve it through CloudFront.
Related
This a very simple question, but I haven't found the answer anywhere.
I am thinking about moving one of my websites, a joomla website, to the cloud, more specifically to a EC2 instance on Amazon. I have been watching some videos and the process seems rather simple. However, I haven't found any information about where the files are stored.
Are they in a S3 bucket? Are they saved somewhere else? How do I access the files?
Can I use Cloudfront to serve images and other files?
How does the whole integration process between EC2, S3 and Cloudfront work for a hosted website?
Thank you!
In my experience With EC2, you interact with the instance just as you would with any headless server in the basement. You can connect over SSH and install software, store files, etc, as normal.
It's great if you want a whole server to play with, but if it's just website hosting you're after, it might be overkill.
S3 can serve up static pages, but you can't install any code on there.
I don't know about cloudfront.
It seems to be though that what you need is standard website hosting.
I've already got an RDS instance configured and running, but currently we're still running on our old web host. We'd like to decrease latency by hosting the code within AWS.
Two concerns:
1) Future scalability
2) Redundancy ... not a huge concern but AWS does occasionally go down.
Has anyone had this problem where they just need to cheaply run what is essentially a database interface via a language such as PHP/Ruby, in 2 regions? (with one as a failover)
Does Amazon offer something that automatically manages resources, that's also cost effective?
Amazon's Elastic Beanstalk service supports both PHP and Ruby apps natively, and allows you to scale your app servers automatically.
In a second region, run a slave RDS instance off of your master (easy to set up in RDS) and have another beanstalk setup there ready as a failover.
The app I am currently hosting on Heroku allows users to submit photos. Initially, I was thinking about storing those photos on the filesystem, as storing them in the database is apparently bad practice.
However, it seems there is no permanent filesystem on Heroku, only an ephemeral one. Is this true and, if so, what are my options with regards to storing photos and other files?
It is true. Heroku allows you to create cloud apps, but those cloud apps are not "permanent" - they are instances (or "slugs") that can be replicated multiple times on Amazon's EC2 (that's why scaling is so easy with Heroku). If you were to push a new version of your app, then the slug will be recompiled, and any files you had saved to the filesystem in the previous instance would be lost.
Your best bet (whether on Heroku or otherwise) is to save user submitted photos to a CDN. Since you are on Heroku, and Heroku uses AWS, I'd recommend Amazon S3, with optionally enabling CloudFront.
This is beneficial not only because it gets around Heroku's ephemeral "limitation", but also because a CDN is much faster, and will provide a better service for your webapp and experience for your users.
Depending on the technology you're using, your best bet is likely to stream the uploads to S3 (Amazon's storage service). You can interact with S3 with a client library to make it simple to post and retrieve the files. Boto is an example client library for Python - they exist for all popular languages.
Another thing to keep in mind is that Heroku file systems are not shared either. This means you'll have to be putting the file to S3 with the same application as the one handling the upload (instead of say, a worker process). If you can, try to load the upload into memory, never write it to disk and post directly to S3. This will increase the speed of your uploads.
Because Heroku is hosted on AWS, the streams to S3 happen at a very high speed. Keep that in mind when you're developing locally.
I am setting up a website that uses Amazon EC2 as Webserver, EBS to store the data of the website and another instance of Wowza for Video on Demand streaming and it seems hard to find answers for my questions:
If my webserver instance is terminated, am I loosing the apache settings/modules on my ec2 instance as well?
Can I have mysql on my webserver instance to save cost of RDS or is it a bad idea?
If I am using RDS for the database, is it deployed on edge locations also (like what cloudfront is doing)?
If have a Wowza Media server instance also running on ec2, can Wowza make use of Cloudfront as well, just so if somebody from somewhere in the world will get a VOD stream right from the next edge location?
Thanks!
If your instance is terminated, the instance and the EBS storage associated with that instance is lost. If you want to remove an instance without losing the state of that server, create an image (AMI) of the server before terminating it.
Depends on how much IT admin you want. The whole idea of Amazon is it takes this admin away from you for a cost higher than if you do it yourself. They do the backing up of databases and load balancing for you (that's the trickier part). That said, Amazon isn't fail-proof, you have to do backups outside of the Amazon system for everything, I've had instances crash and trash the disk, it does happen.
Their database instances are deployed in their main data center locations, which is different to their cloud distribution locations. Having the servers and databases in the same zone will save you network costs.
To use cloudfront you first create one (obviously) and then use the cloudfront domain instead of your own, the cloudfront cache maps through to your domain and caches at the edge location. If it the content is accessible from the server, it's possible to put a cloudfront in front of it. Note that cloudfront charges you not only slightly more than for traffic directly from your server, it also charges to get traffic from your server to the edge locations as instance traffic cost and you'll be charged per 10,000 requests aside from bandwidth (larger content works out cheaper per MB).
It's also possible to map your domain to the cloudfront URL if you want a pretty looking domain.
I want to publish a web site on Amazon's servers, that:
Runs CakePHP
Uses MySQL to store data
Lets users upload audio through flash (currently using a hosted Flash Media Server), and listen to the files later
Do I need Amazon's EC2 for the website, RDS for the MySQL database, and CloudFront for the FMS? I'd really like a walkthrough of which services I should use.
Thanks.
First of all you need EC2 service in order to have a virtual machine, where you can install Apache, PHP and your Web Application.
Then you also need a database server and data repository for the media files. The recommended way is exactly what you suggest: RDS for MySQL and CloudFront as the file repository.
Initially none of the above services (RDS, CloudFront and even EBS) were available. Developers have no way to use a MySQL database, because even if it was installed in an EC2 instance, the instance isn't guaranteed to stay up and running and if the instance is lost, the data is also lost. For this reason EBS was introduced. It created a mounted storage with guaranteed persistence that you could access from the EC2 instance. Theoretically you could install MySQL there and use it to store the flash files. If you only want to serve files through the HTTP protocol, there is no problem using EBS.
CloudFront however has some advantages:
Users are automatically routed to the nearest edge location for high performance delivery of your content.
You can also use it to stream content through the the RTMP protocol.
You don't have to worry about the size of the storage. With EBS you create a storage with a specific size. This could be a problem if you later find out that you need more storage. With CloudFront the files are installed in S3 and you do not need to worry about their size.
You do not waste web server capacity. If you use EBS, the files will be served by the server in EC2.
You could also use S3, but you wouldn't able to use the RTMP protocol and you would need to manually create links to your files. Also, it wouldn't be possible to use your domain name for the files.
RDS also has some advantages over installing MySQL in EC2, EBS:
automated database backups
You can monitor your database with Amazon CloudWatch (free service)
You need EC2 to launch instance and create your LAMP server. RDS is good if you don't need to manage MySql db yourself, but one limiting factor of RDS is you can't have DB replication.
For persistent storage, you can make use EBS or S3 for data file.
One thing not mentioned in any of these replies is the security that may (or may not) need to go around your file access. Cloud networks are good for publicly accessible data, but I haven't seen a cloud network yet that will provide a granular level of file access on a per user basis. While you may be able to obfuscate the url's to access files so that it isn't easy to sequentially guess audio file IDs, that may not be enough if people are keeping private audio. Not saying don't do it, just make the decision with care.