Amazon ELB in VPC - amazon-ec2

We're using Amazon EC2, and we want to put an ELB (load balancer) to 2 instances on a private subnet. If we just add the private subnet to the ELB, it will not get any connections, if we attach both subnets to the ELB then it can access the instances, but it often will get time-outs. Has anyone successfully implemented an ELB within the private subnet of their VPC? If so, could you perhaps explain the procedure to me?
Thanks

My teammate and I just have implemented ELB in a VPC with 2 private subnets in different availability zones. The reason you get timeouts is that for each subnet you add to the load balancer, it gets one external IP address. (try 'dig elb-dns-name-here' and you will see several IP addresses). If one of these IP address maps a private subnet, it will timeout. The IP that maps into your public subnet will work. Because DNS may give you any one of the IP addresses, sometimes it works, sometimes it times out.
After some back and forth with amazon, we discovered that the ELB should only be placed in 'public' subnets, that is subnets that have a route out to the Internet Gateway. We wanted to keep our web servers in our private subnets but allow the ELB to talk to them. To solve this, we had to ensure that we had a corresponding public subnet for each availability zone in which we had private subnets. We then added to the ELB, the public subnets for each availability zone.
At first, this didn't seem to work, but after trying everything, we recreated the ELB and everything worked as it should. I think this is a bug, or the ELB was just in an odd state from so many changes.
Here is more or less what we did:
WebServer-1 is running in PrivateSubnet-1 in availability zone us-east-1b with security group called web-server.
WebServer-2 is running in PrivateSubnet-2 in availability zone us-east-1c with security group called web-server.
Created a public subnet in zone us-east-1b, we'll call it PublicSubnet-1. We ensured that we associated the routing table that includes the route to the Internet Gateway (ig-xxxxx) with this new subnet. (If you used the wizard to create a public/private VPC, this route already exists.)
Created a public subnet in zone us-east-1c, we'll call it PublicSubnet-2. We ensured that we associated the routing table that includes the route to the Internet Gateway (ig-xxxxx) with this new subnet. (If you used the wizard to create a public/private VPC, this route already exists.)
Created a new ELB, adding to it PublicSubnet-1 and PublicSubnet-2 (not the PrivateSubnet-X). Also, picked the instances to run in the ELB, in this case WebServer-1 and WebServer-2. Made sure to assign a security group that allows incoming port 80 and 443. Lets call this group elb-group.
In the web-server group, allow traffic from port 80 and 443 from the elb-group.

The key here is understanding, that you are not "Adding subnets/availability zones" to ELB, but rather specifying what subnets to put ELB instances into.
Yes, ELB is a software load balancer and when you create ELB object, a custom loadbalancing EC2 instance is put into the all subnets that you specified. So for the ELB (its instances) to be accessible, they have to be put into the subnets that have default route configured via IGW (most likely you classified these subnets as public).
So as already was answered above, you have to specify "public" networks for ELB, and those networks should be from the AZs where your EC2 instances are running. In this case ELB instances will be able to reach your EC2 instances (as long as security groups are configured correctly)

We've implemented ELB in a private subnet so the statement that all ELB's need to be public isn't completely true. You do need a NAT. Create a private subnet for the private ELB's, turn on VPC DNS and then make sure the private routing table is configured to go through the NAT. The subnet security groups also need to be setup to allow traffic between ELB and App, and App to DB subnets.
Beanstalk health checks won't work as they can't reach the load balancer, but for services that need to be outside of the public reach this is a good compromise.
Suggested reading to get your VPC architecture started: http://blog.controlgroup.com/2013/10/14/guided-creation-of-cloudformation-templates-for-vpc/.

You must add the following settings.
Public subnet zone b = Server NAT
Private subnet zone c = Server Web
Public subnet zone c = ELB
The trick is routing:
The router to NAT is attach with gateway A.
The router to Server Web is attach to NAT.
The router to Public subnet is attach with gateway A.
ELB details:
1.Zone: Public subnet zone c
2.Instance: Server Web
3.Security Groups: enable ports
http://docs.amazonaws.cn/en_us/ElasticLoadBalancing/latest/DeveloperGuide/UserScenariosForVPC.html

Adding a diagram to Nathan's answer. Full medium post here: https://nav7neeet.medium.com/load-balance-traffic-to-private-ec2-instances-cb07058549fd

Related

How do I look up the IP address range used by my AWS load balancer for creating instances?

My API accesses a service that wants to know what the possible IP address range is for the API so they can whitelist us. How do I know what IP addresses AWS may assign when the load balancer creates new instances?
Direct answer: AWS publishes its IP address ranges in a publicly accessible JSON file. IP ranges are associated with a region and a service, and as you can imagine, there are a lot of them. At the current moment there are 71 CIDR blocks associated with EC2 in us-west-2, for example. That's as specific as you can get, though. So you'd have to supply your partner with all of the EC2 IP address ranges for your region in order to guarantee that they would permit any IP address assigned to an instance your autoscale group might start up. Oh, and they do change periodically, so your partner would have to maintain that list.
Suggestion: Sounds like you have your EC2 instances in public subnets and are assigning public IPs to them. Instead, you can create your EC2 instances in private subnets and use a NAT Gateway to allow them to access the Internet. If you do this, the source IP for outbound traffic will the be the elastic IP address you assign to the NAT gateway. So you can give your partner a definite IP address they can permit to access their service.
Using a private subnet for your EC2 instances also reduces your attack surface area, since the EC2 instances won't have individual public IP addresses.

Network load balancer with Target group having targets inside Private VPC configuration not working

I am having VPC with four subnets out of which 2 are private and 2 are public subnets.I have created Network load balancer into 2 public subnets present in different availability zones.I have also created target group to have target EC2 instances in private subnets.
I have whitelisted the primary private IP's of loadbalancer node's subnets as mentioned in documentation.
https://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-register-targets.html#target-security-groups
The health check calls works fine with those instances marked as 'Healthy'. But the NLB endpoint is not accessible for TCP communication to outside world.
But when I whitelist EC2 instance to allowed access to all(0.0.0.0/0) for particular TCP port,then I am able to access that NLB outside of VPC.
Am I misconfiguring EC2 access?
Public Network Load Balancers balance traffic by modifying the behavior of the Internet Gateway... or, at least, that is how it appears from the user perspective.
The documented need to allow traffic from the load balancer's private IP is solely for the purpose of allowing health checker access -- health check requests appear to be coming from the address of the balancer.
All other traffic appears to come from the actual IP address of the client. Any client address that needs to connect to the instances via the balancer needs to be allowed in the security group of the instance.

EC2 t2.micro instance has no public DNS

I launched an Amazon Web Service (AWS) EC2 Instance, t2.micro, which must be launched into a VPC.
The VPC has Auto-assign Public IP set to Yes.
DNS resolution: Yes
DNS hostnames: Yes
But on the EC2 Dashboard, the instance still has a blank Public DNS and Public IP. I have tried to restart the instance several times, but it still has not been assigned a Public IP. The 5 Elastic IPs that came with our AWS account have already been used. Is it possible to get a Public IP assigned to a t2.micro instance without using Elastic IP?
I have read the post: EC2 instance has no public DNS,
but I do not have reputation points to be able to add a comment, so I am posting this as new question.
Rightclick on the VPC row in the VPC management console page and select "EDIT DNS Hostname". Set it to "Yes". It´s necessary to allow all the instances with the same VPC.
When you create the new instance in the "Step 3: Configure Instance Details", you need to enable "Auto-assign Public IP".
That´s it! :-)
The most common cause of no public IP address for your EC2 instance is that you're launching your EC2 instance in a private subnet. A private subnet means that any EC2 instances located in that subnet are not directly addressable from the public Internet. In other words, by definition, EC2 instances in a private subnet cannot have a public IP address.
This would explain why checking "public IP address" has no effect, and why you're unable to assign an Elastic IP address.
You can't just relocate an instance from one subnet to another. If you need to do that, you can create an AMI of your instance (right-click on the EC2 instance and click create image), and then launch a new instance from that AMI in a different subnet.
To determine if your subnet is private, look at the Route Table and see if you have an Internet Gateway route. Go to VPC > Subnets > Select a Subnet > Route Table tab. Look for an entry that has something like igw-***. If you see this, it's a public subnet. If you see something like eni-*** / i-***, it's a private subnet.
Also check:
VPC -> Subnets -> Subnet Actions -> Modify Auto-Assign Public IP
Face the same issue today. My EC2 instance has no public DNS thus I'm unable to connect via ssh.
I tried and success with these steps:
Go to VPC > Internet Gateways: make sure an Internet Gateway is created and attached to the EC2's VPC
Goto VPC > Route Tables, select a VPC route, navigate to Routes tab: add a new rule with
++ Destination: 0.0.0.0/0
++ Target: select the created Internet Gateway
Goto VPC > Subnet > Route Table tab: click edit, change to the Route Table with destination 0.0.0.0/0 above
Done.
Hmm. So many responses. All of them on the order of "you did something wrong."
Newsflash: AWS doesn't always work correctly. I've used AWS for a very long time. I've personally witnessed instances that do not start, instances that do not stop, disk corruptions on deployed instances and network failures on running instances.
I've never seen a case where a public IP was not created. Until this morning. Now I can add that to the list.
For the record - here's what I verified :)
Three identical instances in the cluster
All instances are in the same availability zone
All instances have same VPC
VPC DNS settings are correct (resolution / hostnames enabled)
All instances have same subnet
Subnet has: a) public routing table; b) option enabled to create public IP
Plenty of IP space available in the subnet
Two of the three instances receive a public IP. The third does not.
So for any others in the future getting to this post: No, you are not insane. Yes, it is possible that AWS screws up.
In our case, manually terminating the problem instance and issuing a new cluster up..."fixed" the problem.
And - I upvoted the answer that indicated a "launch more like this" from a STOPPED instance had an impact on public IP. Not because it is the correct answer (it is not) but because it demonstrates an admirable response to an otherwise inexplicable situation: trial and error / experimentation. The good old "Gee, what happens if I try this...". As cloud professionals: If all other standard troubleshooting steps fail and the only alternative is to blow away the instance (or subnet, or Lambda function, or DynamoDb, or SNS queue; whatever the failing resource) then it's wise to think outside the box and try other actions.
In other words: keep an open mind.
Go to VPC -> Subnets
And make sure that the Auto-assign public IPv4 address is set to YES
There are many possible reasons. Check the follow.
You need to have a VPC created.
The DNS resolution and DNS hostnames should be enabled.
Choose your VPC -> Actions -> Edit DNS resolution -> enable
Choose your VPC -> Actions -> Edit DNS hostnames -> enable
Into the VPC maybe you need a private and public subnet.
In the private subnet, you need to have a NAT Gateway associate to this.
In the public subnet, you need to have an Internet Gateway associate to this.
You need to enable the auto-assign IP for your public subnet.
Choose the public subnet -> Actions -> Modify auto-assign IP settings -> enable
Later when you launch a new instance in
Step 3: Configure Instance Details.
You should choose your VPC and your public subnet. And in the "Auto-assign Public IP" section choose "Use subnet setting (Enabled)"
I think that that should solve your problem...
I had the same issue. The reason of my issue turned out to be that I was using a route table which was not associated with a subnet.
enter image description here
After I changed my subnet, my instances were assigned public ips.
After creating a Subnet - make sure the Auto-assign public IPv4 setting is set to Yes or Enabled.
After making sure the above setting is turned on - then launch the EC2 instance.
If the above setting is not enabled after Subnet creation - the EC2 instance will be treated as Private and won't have a public IPV4 address.
When I use "launch more like this" option from a STOPPED instance, I'll get a new instance without a public ip. But if I "launch more like this" from a running instance, the new instance has a public ip.
Most likely, the public subnet has no enabled feature for "Auto-assign IPv4". It is selected as "No". And during your instance creation process the default option is "Use subnet setting (Enabled)". That's why newly issued instances cannot get public IP address.
Go VPC dashboard and click Subnets. Select a public subnet and select Modify auto-assign IP settings from Actions list and check Auto-assign IPv4. After saving your changes, your instances will get public IP automatically.
My big "gotcha" on this was when creating a VPC & Subnets from a CloudFormation stack, my Subnets were missing the Property "MapPublicIpOnLaunch" : true.
My Observation :
You need to enable the auto-assign IP for your public subnet. Choose the public subnet -> Actions -> Modify auto-assign IP settings -> enable
Only after the above is done, then launch an EC2 instance and you will start seeing public IP assigned.
Once an EC2 instance created without above setting enabled, that EC2 will not have public IP assigned even after reboot , it already considered that subnet to be private.
Hope this helps!
My Answer:
Please check if you attached a secondary network interface with the instance.
As per AWS Documentations, If you attach another network interface to your instance, your current public IP address is released when you restart your instance. Please read the third point from below.
You cannot manually associate or disassociate a public IP (IPv4) address from your instance. Instead, in certain cases, we release the public IP address from your instance, or assign it a new one:
We release your instance's public IP address when it is stopped,
hibernated, or terminated. Your stopped or hibernated instance receives a
new public IP address when it is started.
We release your instance's public IP address when you associate an
Elastic IP address with it. When you disassociate the Elastic IP address
from your instance, it receives a new public IP address.
If the public IP address of your instance in a VPC has been released, it
will not receive a new one if there is more than one network interface
attached to your instance.
If your instance's public IP address is released while it has a secondary
private IP address that is associated with an Elastic IP address, the
instance does not receive a new public IP address.
AWS Documentation Link for more reference: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html?icmpid=docs_ec2_console#concepts-public-addresses

Why might the CIDR/IP in DB security group be different from instance elastic IP?

I have an EC2 instance, which is able to connect to my RDS instance, yet its elastic IP does not appear in the DB security group of whitelisted IP's.
How might this be?
I ask because I have created a new instance, which I also want to whitelist and just entering its elastic IP does not seem like the way to do things since none of the other servers have their elastic IP listed.
Thanks in advance,
There might be two causes here:
Traffic Sources
Security Group Rules do not necessarily specify IP addresses as traffic sources alone, rather regularly will refer to other security groups as well:
The source can be an individual IP address (203.0.113.1), a range of
addresses (e.g., 203.0.113.0/24), or an EC2 security group. The
security group can be another group in your AWS account, a group in
another AWS account, or the security group itself.
By specifying a security group as the source, you allow incoming
traffic from all instances that belong to the source security group.
[...] You might specify another security group in your account if you're creating a
three-tier web service (see Creating a Three-Tier Web Service).
[emphasis mine]
Consequently, the DB security group of your Amazon RDS instance might refer to the EC2 security group used for your Amazon EC2 instance, implying respective access rights already. See my answer to AWS - Configuring access to EC2 instance from Beanstalk App for more details regarding this concept/approach.
Public vs. Private IP Addresses
You might see the effect of a little known, but nonetheless important and quite helpful feature of the AWS DNS infrastructure, see section Public and Private Addresses on page Using Instance IP Addresses:
Amazon EC2 also provides an internal DNS name and a public DNS name
that map to the private and public IP addresses respectively. The
internal DNS name can only be resolved within Amazon EC2. The public
DNS name resolves to the public IP address outside the Amazon EC2
network and the private IP address within the Amazon EC2 network. [emphasis mine]
That is, it's resolving the public DNS (e.g. ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com) to the private IP address when you are using it inside the Amazon EC2 network, and to the public or elastic IP address when using it outside the Amazon EC2 network.
Accordingly, the various AWS products are usually wired up between each other by means of their private IP Addresses rather than external ones for a variety of reasons, most importantly network speed and cost (see my answer to AWS EC2 Elastic IPs bandwidth usage and charges for details).
Consequently, the DB security group of your Amazon RDS instance might refer to the private IP address of your Amazon EC2 instance, implying respective access rights accordingly.

Why am I unable to associate an Elastic IP to an EC2 instance in a second VPC on AWS?

I have for a long time a VPC (with 1 subnet) on Amazon Web Services (AWS) with several instances each having an Elastic IP address.
For new needs, I have defined a second VPC (with 1 subnet also) on my same account: for some reasons, I can't associate EIP (which is allocated with no problem) to instances launched in VPC #2: the interactive wizard of the console only presents me the instances of the first VPC.
Is it a known limitation or am I doing something wrong?
Two questions:
How many EIP's do you have on your account?
Is the 2nd VPC using a NAT instance to access the Internet?
EIP addresses should only be used on instances in subnets configured to route their traffic directly to the Internet Gateway. EIPs cannot be used on instances in subnets configured to use a NAT instance to access the Internet. (aws.amazon.com)

Resources