I have two EC2 instances from an Ubuntu image, they are located in different regions.
I just want to ask, whether they can communicate over the private IP addresses?
I have opened the required ports with a security group. I use netcat to test the communication, but it only works, when I use the public IP addresses.
It is not possible to communicate between Amazon EC2 regions via the private IP addresses (except if you setup a VPN and respective routing for this, see section VPN Solution below), traffic between regions is in fact passing the public internet and is not distinguishable from any other internet traffic, see e.g. the following FAQs from the Region and Availability Zone FAQ:
Can instances use group-based firewall rules across Regions? - No. Group-based firewall rules only work within a Region. If you need instances to communicate with each other across Regions, you should use CIDR based firewall rules. [...]
What is the cost for data transfer between Regions? - Data transferred from one Region to another is charged at both sides at the Internet data transfer rate.
This applies to an Amazon VPC as well, see e.g. the FAQ Can Amazon EC2 instances within a VPC in one region communicate with Amazon EC2 instances within a VPC in another region?:
Yes, as long as all communication takes place over the Internet
Gateway of each VPC and uses the Elastic IP addresses assigned to the
instances in each VPC. Please note: security groups cannot span
regions. All traffic filtering between instances in one VPC and
instances in another VPC must use the Elastic IP addresses as the
specified source or destination address. [emphasis mine]
VPN Solution
AWS has meanwhile released two walkthroughs describing a solution for Connecting Multiple VPCs with EC2 Instances based on either IPsec or OpenVPN:
Connecting Multiple VPCs with EC2 Instances (IPSec)
please note that this tutorial facilitates Openswan, but you can achieve the same with strongSwan (or even the Linux IPsec stack built in as of kernel 2.6+, see e.g. IPsec L2TP VPN server)
Connecting Multiple VPCs with EC2 Instances (SSL)
Now you can do it with AWS VPC peer connection.
It enables the resources in two VPCs that are in different regions, even in different accounts, to be able to communicate with the private IP address, just like in the same LAN.
One thing you need to know is the CIDR blocks you choose for your both VPCs, they must not be in conflict. Otherwise, the peer connection can't be made successfully.
See the official doc about VPC peer connection.
Related
My API accesses a service that wants to know what the possible IP address range is for the API so they can whitelist us. How do I know what IP addresses AWS may assign when the load balancer creates new instances?
Direct answer: AWS publishes its IP address ranges in a publicly accessible JSON file. IP ranges are associated with a region and a service, and as you can imagine, there are a lot of them. At the current moment there are 71 CIDR blocks associated with EC2 in us-west-2, for example. That's as specific as you can get, though. So you'd have to supply your partner with all of the EC2 IP address ranges for your region in order to guarantee that they would permit any IP address assigned to an instance your autoscale group might start up. Oh, and they do change periodically, so your partner would have to maintain that list.
Suggestion: Sounds like you have your EC2 instances in public subnets and are assigning public IPs to them. Instead, you can create your EC2 instances in private subnets and use a NAT Gateway to allow them to access the Internet. If you do this, the source IP for outbound traffic will the be the elastic IP address you assign to the NAT gateway. So you can give your partner a definite IP address they can permit to access their service.
Using a private subnet for your EC2 instances also reduces your attack surface area, since the EC2 instances won't have individual public IP addresses.
Is there any way to make new instances added to an autoscaling group associate with an elastic IP? I have a use case where the instances in my autoscale group need to be whitelisted on remote servers, so they need to have predictable IPs.
I realize there are ways to do this programmatically using the API, but I'm wondering if there's any other way. It seems like CloudFormation may be able to do this.
You can associate an Elastic IP to ASG instances using manual or scripted API calls just as you would any other instance -- however, there is no automated way to do this. ASG instances are designed to be ephemeral/disposable, and Elastic IP association goes against this philosophy.
To solve your problem re: whitelisting, you have a few options:
If the system that requires predictable source IPs is on EC2 and under your control, you can disable IP restrictions and use EC2 security groups to secure traffic instead
If the system is not under your control, you can set up a proxy server with an Elastic IP and have your ASG instances use the proxy for outbound traffic
You can use http://aws.amazon.com/vpc/ to gain complete control over instance addressing, including network egress IPs -- though this can be time consuming
There are 3 approaches I could find to doing this. Cloud Formation will just automate it but you need to understand what's going on first.
1.-As #gabrtv mentioned use VPC, this lends itself to two options.
1.1-Within a VPC use a NAT Gateway to route all traffic in and out of the Gateway. The Gateway will have an Elastic IP and internet traffic then whitelist the NAT Gateway on your server side. Look for NAT gateway on AWS documentation.
1.2-Create a Virtual Private Gateway/VPN connection to your backend servers in your datacenter and route traffic through that.
1.2.a-Create your instances within a DEDICATED private subnet.
1.2.b-Whitelist the entire subnet on your side, any request from that subnet will be allowed in.
1.2.c Make sure your routes in the Subnet are correct.
(I'm skipping 2 on purpose since that is 1.2)
3.-The LAZY way:
Utilize AWS Opsworks to do two things:
1st: Allocate a RESOURCE Pool of Elastic IPs.
2nd: Start LOAD instances on demand and AUTO assign them one elastic ip from the Pool.
For the second part you will need to have the 24/7 instances be your minimum and the Load instances be your MAX. AWS Opsworks now allows Cloud Watch alarms to trigger instance startup so it is very similar to ASG.
The only disadvantage of Opsworks is that instances aren't terminated but stopped instead when the load goes down and that you must "create" instances beforehand. Also you depend on Chef solo to initiate your instances but is the only way to get auto assigning EIPs to your newly created instances that I could find.
Cheers!
I have an EC2 instance, which is able to connect to my RDS instance, yet its elastic IP does not appear in the DB security group of whitelisted IP's.
How might this be?
I ask because I have created a new instance, which I also want to whitelist and just entering its elastic IP does not seem like the way to do things since none of the other servers have their elastic IP listed.
Thanks in advance,
There might be two causes here:
Traffic Sources
Security Group Rules do not necessarily specify IP addresses as traffic sources alone, rather regularly will refer to other security groups as well:
The source can be an individual IP address (203.0.113.1), a range of
addresses (e.g., 203.0.113.0/24), or an EC2 security group. The
security group can be another group in your AWS account, a group in
another AWS account, or the security group itself.
By specifying a security group as the source, you allow incoming
traffic from all instances that belong to the source security group.
[...] You might specify another security group in your account if you're creating a
three-tier web service (see Creating a Three-Tier Web Service).
[emphasis mine]
Consequently, the DB security group of your Amazon RDS instance might refer to the EC2 security group used for your Amazon EC2 instance, implying respective access rights already. See my answer to AWS - Configuring access to EC2 instance from Beanstalk App for more details regarding this concept/approach.
Public vs. Private IP Addresses
You might see the effect of a little known, but nonetheless important and quite helpful feature of the AWS DNS infrastructure, see section Public and Private Addresses on page Using Instance IP Addresses:
Amazon EC2 also provides an internal DNS name and a public DNS name
that map to the private and public IP addresses respectively. The
internal DNS name can only be resolved within Amazon EC2. The public
DNS name resolves to the public IP address outside the Amazon EC2
network and the private IP address within the Amazon EC2 network. [emphasis mine]
That is, it's resolving the public DNS (e.g. ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com) to the private IP address when you are using it inside the Amazon EC2 network, and to the public or elastic IP address when using it outside the Amazon EC2 network.
Accordingly, the various AWS products are usually wired up between each other by means of their private IP Addresses rather than external ones for a variety of reasons, most importantly network speed and cost (see my answer to AWS EC2 Elastic IPs bandwidth usage and charges for details).
Consequently, the DB security group of your Amazon RDS instance might refer to the private IP address of your Amazon EC2 instance, implying respective access rights accordingly.
I want to use the Amazon Web Service free micro-instance for my different projects for testing and personal purpose. But I required some static-public IP on which I can run my server.
Is that possible? From where I can buy just IP and use it with my AWS?
EC2 Elastic IP Addresses
Elastic IPs are tied to an account, not an instance.
You need to look at AWS VPC for this.
Whilst VPC is free outside of the usual instance pricing, it doesn't work with Micro instances (the cheapest ones).
When not using VPC, you're assigned IP addresses through DHCP. When the DHCP lease expires, or you restart, your IP is released back to the pool.
VPC lets you use private IP addressing, you can use it with Elastic IPs and is much easier to integrate with a physical infrastructure setup.
If you're only testing/investigating AWS and have little or no budget to use anything other than a Micro instance, I'd just suck it up and deal with the changing of IPs.
If you've got a budget that lets you use instances other than Micro, then go for VPC.
Also, if you're doing more than testing/investigating I'd recommend starting with VPC straight away as trying to migrate from a non VPC to a VPC infrastructure is a massive PITA.
For every AWS account, 5 free elastic ips are provided. You have to just allocate them to required instance. But make sure that the allocate address(newly created elastic ip) in in use, because you will billed if the Elastic ip is not in use.
Looks like they have configured ARP statically so you can only use the IP address on an instance that was bound to that instance through the EC2 management console.
I just configured one of my instances to use a static IP address other than the one assigned through the management console and rebooted the instance.
I'm still receiving ARP responses on the old address but not receiving ARP responses on the new address at all.
Unfortunately for me, I have a not responding instance (NFS File Server) stuck in a stopping state while I attempt to terminate it.
The IP Address bound to that instance cannot be re-assigned to a replacement instance so now I have to reconfigure
On the whole pricing delima: When you come to think of it, there is a limited amount of static IPs so there must some pricing (supply and demand). This pricing is two fold: 1) for upto a limited number (5 per account) you don't have to pay. 2) if you created one you need to use it if you don't you'll be billed (to prevent every user to get 5 static IPs)
I have for a long time a VPC (with 1 subnet) on Amazon Web Services (AWS) with several instances each having an Elastic IP address.
For new needs, I have defined a second VPC (with 1 subnet also) on my same account: for some reasons, I can't associate EIP (which is allocated with no problem) to instances launched in VPC #2: the interactive wizard of the console only presents me the instances of the first VPC.
Is it a known limitation or am I doing something wrong?
Two questions:
How many EIP's do you have on your account?
Is the 2nd VPC using a NAT instance to access the Internet?
EIP addresses should only be used on instances in subnets configured to route their traffic directly to the Internet Gateway. EIPs cannot be used on instances in subnets configured to use a NAT instance to access the Internet. (aws.amazon.com)