I want to run a simple Ruby Rack app (a REST API) that takes Internet requests (from iPhone clients), talks to a PostgreSQL database, and responds with JSON.
How exactly should I set this up on Amazon Web Services (AWS)?
I want the app to be able to scale to serve a growing number of clients, so I think I should use Auto Scaling with Elastic Load Balancing.
Should I use Elastic Beanstalk or manually set everything up myself?
How does the question Manual deployment vs. Amazon Elastic Beanstalk apply when setting up a Ruby Rack server with PostgreSQL?
Default vs Custom VPC
Should I just use the default VPC and use security groups to prevent direct Internet access to the EC2 & DB instances? Or, should I create a custom VPC and use private subnets, as described in Example: Launching an Elastic Beanstalk in a VPC with Amazon RDS?
Using the concept of public and private subnets adds a fantastic layer of security to your AWS application. By placing your database and application server instances in private subnets you can by design protect them from external penetration and accidental exposure.
I would recommend that you start by provisioning a VPC in 2 AZs with 1 public and 1 private subnet in each Availability Zone (4 subnets in all).
Place a NAT instance in each public subnet and update the main route table for your private subnets to send all non-vpc traffic to the NAT. This will allow instances launched into your private subnets to communicate with the WAN Internet even though they are not publicly addressable themselves.
I would recommend that you use a Multi-AZ RDS deployment for your Postgres deployment with the RDS instances in your private subnets within each AZ. This will maximize security (Postgres is not publicly accessible) and will provide you with fault tolerance (an AZ failure will not take down your app).
I would setup your Ruby app on Elastic Beanstalk. This will provide you with fault tolerance and auto-scaling. Your Elastic Beanstalk load balancers will reside in the public subnet of each AZ and your Elastic Beanstalk EC2 instances will reside in the private subnets.
Related
I am planning to migrate from Ec2 classic to EC2 VPC. My application reads messages from SQS, download assets from S3 and perform actions mentioned in the SQS messages and then updates RDS. I have following queries
Is it beneficial for me to migrate to Amazon VPC from Classic
I create my EC2 machines using ruby scripts, and deploy code on them using capistrano. In classic mode I used the IP address to deploy code using capistrano. But in VPC there is a concept of private IP address and you cannot access a machine inside a subnet.So my question is:
How should I deploy code on the EC2 instances or rather how should I connect to them?
Thank You.
This questions is pretty broad but I'll take stab at it:
Is it beneficial for me to migrate to Amazon VPC from Classic
It's beneficial if you care about security of your data in transit and at rest. In a VPC none of your traffic is exposed to the outside and you can chose which components you want to expose in case you want to receive traffic/data from the outside. i.e Your ELB or ELBs.
I create my EC2 machines using ruby scripts, and deploy code on them using capistrano. In classic mode I used the IP address to deploy
code using capistrano. But in VPC there is a concept of private IP
address and you cannot access a machine inside a subnet. So my question
is: How should I deploy code on the EC2 instances or rather how should
I connect to them?
You can actually assign a public IP to your EC2 machines in a VPC if you choose to. You can use that IP to deploy your code from the outside.
You can read about it here: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-ip-addressing.html
If you want more security you can always deploy from a machine in your VPC (that has SSH access to the outside). You can ssh to that machine and then run cap deploy from there.
I am planning to have a multi server architecture in amazon EC2 where the servers need to talk to each other. These servers need to be located in different amazon regions (different datacenters). Can I just use the internal network of the amazon ec2? What are the security issues? Should I mandatorily use Amazon VPC in this setup.
Jam ,
If you are planning to create instances on different regions then go for VPC ,because VPC gives you more security .You can restrict these machines for limited external access also .
As security part , VPC is better than classic EC2 instances ,as you can even only allow VPC to VPC connections also .
Is there any way to make new instances added to an autoscaling group associate with an elastic IP? I have a use case where the instances in my autoscale group need to be whitelisted on remote servers, so they need to have predictable IPs.
I realize there are ways to do this programmatically using the API, but I'm wondering if there's any other way. It seems like CloudFormation may be able to do this.
You can associate an Elastic IP to ASG instances using manual or scripted API calls just as you would any other instance -- however, there is no automated way to do this. ASG instances are designed to be ephemeral/disposable, and Elastic IP association goes against this philosophy.
To solve your problem re: whitelisting, you have a few options:
If the system that requires predictable source IPs is on EC2 and under your control, you can disable IP restrictions and use EC2 security groups to secure traffic instead
If the system is not under your control, you can set up a proxy server with an Elastic IP and have your ASG instances use the proxy for outbound traffic
You can use http://aws.amazon.com/vpc/ to gain complete control over instance addressing, including network egress IPs -- though this can be time consuming
There are 3 approaches I could find to doing this. Cloud Formation will just automate it but you need to understand what's going on first.
1.-As #gabrtv mentioned use VPC, this lends itself to two options.
1.1-Within a VPC use a NAT Gateway to route all traffic in and out of the Gateway. The Gateway will have an Elastic IP and internet traffic then whitelist the NAT Gateway on your server side. Look for NAT gateway on AWS documentation.
1.2-Create a Virtual Private Gateway/VPN connection to your backend servers in your datacenter and route traffic through that.
1.2.a-Create your instances within a DEDICATED private subnet.
1.2.b-Whitelist the entire subnet on your side, any request from that subnet will be allowed in.
1.2.c Make sure your routes in the Subnet are correct.
(I'm skipping 2 on purpose since that is 1.2)
3.-The LAZY way:
Utilize AWS Opsworks to do two things:
1st: Allocate a RESOURCE Pool of Elastic IPs.
2nd: Start LOAD instances on demand and AUTO assign them one elastic ip from the Pool.
For the second part you will need to have the 24/7 instances be your minimum and the Load instances be your MAX. AWS Opsworks now allows Cloud Watch alarms to trigger instance startup so it is very similar to ASG.
The only disadvantage of Opsworks is that instances aren't terminated but stopped instead when the load goes down and that you must "create" instances beforehand. Also you depend on Chef solo to initiate your instances but is the only way to get auto assigning EIPs to your newly created instances that I could find.
Cheers!
I have an EC2 instance, which is able to connect to my RDS instance, yet its elastic IP does not appear in the DB security group of whitelisted IP's.
How might this be?
I ask because I have created a new instance, which I also want to whitelist and just entering its elastic IP does not seem like the way to do things since none of the other servers have their elastic IP listed.
Thanks in advance,
There might be two causes here:
Traffic Sources
Security Group Rules do not necessarily specify IP addresses as traffic sources alone, rather regularly will refer to other security groups as well:
The source can be an individual IP address (203.0.113.1), a range of
addresses (e.g., 203.0.113.0/24), or an EC2 security group. The
security group can be another group in your AWS account, a group in
another AWS account, or the security group itself.
By specifying a security group as the source, you allow incoming
traffic from all instances that belong to the source security group.
[...] You might specify another security group in your account if you're creating a
three-tier web service (see Creating a Three-Tier Web Service).
[emphasis mine]
Consequently, the DB security group of your Amazon RDS instance might refer to the EC2 security group used for your Amazon EC2 instance, implying respective access rights already. See my answer to AWS - Configuring access to EC2 instance from Beanstalk App for more details regarding this concept/approach.
Public vs. Private IP Addresses
You might see the effect of a little known, but nonetheless important and quite helpful feature of the AWS DNS infrastructure, see section Public and Private Addresses on page Using Instance IP Addresses:
Amazon EC2 also provides an internal DNS name and a public DNS name
that map to the private and public IP addresses respectively. The
internal DNS name can only be resolved within Amazon EC2. The public
DNS name resolves to the public IP address outside the Amazon EC2
network and the private IP address within the Amazon EC2 network. [emphasis mine]
That is, it's resolving the public DNS (e.g. ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com) to the private IP address when you are using it inside the Amazon EC2 network, and to the public or elastic IP address when using it outside the Amazon EC2 network.
Accordingly, the various AWS products are usually wired up between each other by means of their private IP Addresses rather than external ones for a variety of reasons, most importantly network speed and cost (see my answer to AWS EC2 Elastic IPs bandwidth usage and charges for details).
Consequently, the DB security group of your Amazon RDS instance might refer to the private IP address of your Amazon EC2 instance, implying respective access rights accordingly.
I am currently using AWS CloudFormation for my application. Right now I am able to auto scale the instances. Now, I want to put every thing on an Amazon VPC. Can we create a VPC using CloudFormation? And how can we manage the Elastic IP address via CloudFormation, when we have an ELB in the template? I have found a VPC related example on AWS CloudFormation Sample Templates, but it only provisions resources into an existing VPC and doesn't create a new one in the template.
Update
As pointed out by Jeff already (+1), AWS has just announced AWS CloudFormation Support for Creating VPC Resources as of April 25, 2012, covering the missing piece of their initial VPC support:
We are excited to announce that AWS CloudFormation now supports the
creation of Amazon Virtual Private Cloud (VPC) resources. [...]
Now, you can create new Virtual Private Clouds (VPC), subnets,
gateways, network ACLs, routes and route tables using CloudFormation
templates. [...]
[...] A CloudFormation can now fully represent your VPC configuration
along with all the resources needed to run your application in the
VPC.
See Jeff Barr's introductory post AWS CloudFormation Can Now Create Virtual Private Clouds for more details and examples. In particular, the AWS CloudFormation Sample Templates feature two new sample templates [...] to get you started as well:
VPC with a single EC2 Instance - Sample template showing how to create a VPC and add an EC2 instance with an Elastic IP address and a security group.
VPC with public and private subnets, an Elastic load Balancer, and an EC2 instance - Sample template showing how to create a VPC with multiple subnets. The first subnet is public and contains the load balancer, the second subnet is private and contains an EC2 instance behind the load balancer.
Initial Answer
I don't think creating an Amazon VPC with AWS CloudFormation is already supported.
While AWS has just announced AWS CloudFormation Support For VPC as of February 12, 2012 indeed, this covers existing resource types only:
All resource types such as Amazon EC2 instances, security groups and
Elastic IP addresses, Elastic Load Balancers, Auto Scaling Groups and
Amazon RDS Database instances can now be deployed into any existing
Amazon VPC using CloudFormation templates. The templates allow you to
run multi-tier web applications and corporate applications in a
private network. With Amazon VPC and CloudFormation, you can easily
control which resources you want to expose publicly and which ones
should be private.
Amazon VPC is notably absent from this list, which matches the fact that it isn't listed in the supported AWS Resource Types Reference either.
It's supported now: see AWS CloudFormation Support for Creating VPC Resources for details.