For bandwidth of 400GB per month, what EC2 instance should I use if I want to create a video streaming infrastructure to different regions?
You won't get any specific answers on questions like this. It is totally dependent on your application.
If you stored the videos in Amazon S3 and streamed videos through Amazon CloudFront, then Amazon EC2 would purely be handling user interactions and web pages, without having to serve video content at all.
For any application, the only way to know how much compute is required is to test the application under many different workloads and instance types and measure the performance. Alternatively, an application can be designed to use serverless microservices using AWS Lambda, which can automatically scale without using EC2 instances.
Related
I use three services in my docker-compose file: Django API, Celery, and Redis. I can run this using AWS ECS for which I have to create 3 services (one for each service) and at least three tasks for each service created which seems to be expensive when compared to AWS EC2 instance where I can deploy 3 services by running docker-compose command and also I can attach Auto-scaling groups for horizontal scaling by creating an Image of my instance and attach a Load balancer to it.
I don't understand why should one use ECS which is mainly used for auto-scaling multi-service dockerized applications when I can do the same with simple EC2 auto-scaling groups. Am I missing something?
The "why should one use ECS" is a question of "why to use an abstraction" at its heart. To answer it, ECS bills itself as a "fully managed container orchestration service" and provides an abstraction on top of a cluster of EC2 instances (run and managed by you or by AWS, depending upon the choices you make). It will come down to these choices which you want to consider before choosing to use (or NOT choosing ) ECS:
Do you want to do mapping of containers to EC2 instances yourself OR do you want your container orchestration engine to take care of it for you?
Do you want to be even aware of underlying EC2 instances and hence be responsible for running and patching them OR do you want your container orchestration engine to take care of it for you?
Do you want to own responsibility of redeploying newly built and tested containers on all your EC2 instances individually OR do you want your container orchestration engine and associated compute platform (eg. Fargate) to take care of it for you?
I have an application which requires a strong GPU, and it runs on an EC2 instance of type p2.xlarge which is ideal for that kind of tasks. Because the p2.xlarge instances are quiet expensive though, I keep them offline and only start them when necessary.
Sometimes I do multiple calculations on 1 instance, and sometimes I even use multiple instances at the same time.
I've written an application in Angular that can visualize the results of these calculations. Which I've only tested in an environment where the angular application is hosted on that same instance.
But since I have multiple instances, it would be ideal to visualize them all on a single webpage. So that leads me to the diagram below, where a single instance is a like a portal or management console that controls the other instances.
Now, to get things moving, I would like to setup this front-end server as soon as possible. But there are so many instance types to choose from. What would be the best instance type for this front-end server for a dashboard / portal that controls other aws instances. The only requirements are:
of course it should be able to run a nodejs server (and a minimalistic db for storing logins).
it should be able to start/stop other aws instances.
it should be able to communicate to other aws instances using websockets, and as far as I'm concerned, that shouldn't even really be over the internet, that can be within the aws network.
Well ,
of course it should be able to run a nodejs server (and a minimalistic db for storing logins).
Sounds like you need a small machine .
I would suggest using the T2/T3 family . very cheap and can be configured without bursting limits which gives you all the power you need for a very low price .
it should be able to start/stop other aws instances.
Not a problem ,
Create an IAM role which have permissions to EC2 and when you
launch your instance , give it that IAM role.
It will be able to do what ever you grant it to do with the api.
Pay attention to the image you use ,
if you take the Amazon Linux 2 you get the aws-cli preinstalled ,
it's pretty nice .
Read more about IAM roles here.
it should be able to communicate to other aws instances using websockets, and as far as I'm concerned, that shouldn't even really be over the internet, that can be within the aws network.
Just make sure you launch all instances in the same VPC .
when machines are in the same vpc they can communicate with each other only with internal ips .
You can create a new VPC like here
Or , just use the default one .
after you launch the instance you will
get it's internal IP.
My company uses AWS heavily and has several Amazon Direct Connect network links from our points of presence into Amazon. These reduce our latency and costs.
http://aws.amazon.com/directconnect/
We would like to be able to use Heroku more extensively with our internal applications, but the dynos would need to exist inside our Amazon VPCs in order for us to get the latency and cost benefits. I can't see a way to do this.
Is there any way for Heroku customers to run their dynos inside specific Amazon VPCs?
I've already got an RDS instance configured and running, but currently we're still running on our old web host. We'd like to decrease latency by hosting the code within AWS.
Two concerns:
1) Future scalability
2) Redundancy ... not a huge concern but AWS does occasionally go down.
Has anyone had this problem where they just need to cheaply run what is essentially a database interface via a language such as PHP/Ruby, in 2 regions? (with one as a failover)
Does Amazon offer something that automatically manages resources, that's also cost effective?
Amazon's Elastic Beanstalk service supports both PHP and Ruby apps natively, and allows you to scale your app servers automatically.
In a second region, run a slave RDS instance off of your master (easy to set up in RDS) and have another beanstalk setup there ready as a failover.
I want to publish a web site on Amazon's servers, that:
Runs CakePHP
Uses MySQL to store data
Lets users upload audio through flash (currently using a hosted Flash Media Server), and listen to the files later
Do I need Amazon's EC2 for the website, RDS for the MySQL database, and CloudFront for the FMS? I'd really like a walkthrough of which services I should use.
Thanks.
First of all you need EC2 service in order to have a virtual machine, where you can install Apache, PHP and your Web Application.
Then you also need a database server and data repository for the media files. The recommended way is exactly what you suggest: RDS for MySQL and CloudFront as the file repository.
Initially none of the above services (RDS, CloudFront and even EBS) were available. Developers have no way to use a MySQL database, because even if it was installed in an EC2 instance, the instance isn't guaranteed to stay up and running and if the instance is lost, the data is also lost. For this reason EBS was introduced. It created a mounted storage with guaranteed persistence that you could access from the EC2 instance. Theoretically you could install MySQL there and use it to store the flash files. If you only want to serve files through the HTTP protocol, there is no problem using EBS.
CloudFront however has some advantages:
Users are automatically routed to the nearest edge location for high performance delivery of your content.
You can also use it to stream content through the the RTMP protocol.
You don't have to worry about the size of the storage. With EBS you create a storage with a specific size. This could be a problem if you later find out that you need more storage. With CloudFront the files are installed in S3 and you do not need to worry about their size.
You do not waste web server capacity. If you use EBS, the files will be served by the server in EC2.
You could also use S3, but you wouldn't able to use the RTMP protocol and you would need to manually create links to your files. Also, it wouldn't be possible to use your domain name for the files.
RDS also has some advantages over installing MySQL in EC2, EBS:
automated database backups
You can monitor your database with Amazon CloudWatch (free service)
You need EC2 to launch instance and create your LAMP server. RDS is good if you don't need to manage MySql db yourself, but one limiting factor of RDS is you can't have DB replication.
For persistent storage, you can make use EBS or S3 for data file.
One thing not mentioned in any of these replies is the security that may (or may not) need to go around your file access. Cloud networks are good for publicly accessible data, but I haven't seen a cloud network yet that will provide a granular level of file access on a per user basis. While you may be able to obfuscate the url's to access files so that it isn't easy to sequentially guess audio file IDs, that may not be enough if people are keeping private audio. Not saying don't do it, just make the decision with care.