Deploying an number of instances in AWS using terraform. The code is written to make use of specific private ip ranges. The code iterates through a range to provide the last two digits on an ip.
IP_AA_MGMT_Windows = [for i in range(1, var.Number_of_AA_Candidates +1 ) : format("%s%02d", "10.10.8.1", i)]
For information the subnet this belongs to has the following CIDR allocation
cidr_block = "10.10.8.0/22"
This gives an ip range of 10.10.8.0 - 10.10.11.255
The instance is created with no real problems. Expected private ip is allocated in an identical manner as the network interface.
resource "aws_instance" "Windows" {
instance_type = "t2.large"
subnet_id = aws_subnet.windows.id
vpc_security_group_ids = [aws_security_group.AA_Eng_Windows[count.index].id]
key_name = aws_key_pair.ENG-DEV.id
count = var.Number_of_AA_Candidates
private_ip = local.IP_AA_WINLAN_Windows[count.index]
associate_public_ip_address = false
An additional network interface is created and attached to the instance.
resource "aws_network_interface" "Windows_Access_Interface" {
subnet_id = aws_subnet.management.id
private_ip = local.IP_AA_MGMT_Windows[count.index]
security_groups = [aws_security_group.Windows.id]
count = var.Number_of_AA_Candidates
attachment {
instance = aws_instance.Windows[count.index].id
device_index = 1
}
All deploys correctly according to terraform. Its not until you check the private ip's in AWS or through terraform state show that you realise the network interface resource is created but with an incorrect private ip, not the one provisioned in the code. NOTE. Terraform plan provides output suggesting no problems with ip allocation.
Below is some of the output from the terrafrom show command.
# aws_network_interface.Windows_Access_Interface[0]:
resource "aws_network_interface" "Windows_Access_Interface" {
interface_type = "interface"
private_ip = "10.10.10.72"
private_ip_list = [
"10.10.10.72",
]
private_ip_list_enabled = false
private_ips = [
"10.10.10.72",
]
NOTE Some of the details in the show have intentionally been removed for security.
The question now is, what is causing this?
There are some important points to take into account here. One of the first being that there are five reserved addresses in each VPC [1]:
The first four IP addresses and the last IP address in each subnet CIDR block are not available for your use, and they cannot be assigned to a resource, such as an EC2 instance.
So that means you would have to start counting the addresses that are assignable from 10.10.8.4. That further means that in the range function, the counting would have to start after 3:
IP_AA_MGMT_Windows = [for i in range(4, var.Number_of_AA_Candidates + 1 ) : format("%s%02d", "10.10.8.1", i)]
Since IP addresses are not really strings, the format function along with the %02d will only add 0 and depending on the Number_of_AA_Candidates a certain number of decimal numbers at the end of the string. So for example, if Number_of_AA_Candidates was equal to 2, that would yield the following IP addresses:
> local.IP_AA_MGMT_Windows
[
"10.10.8.101",
"10.10.8.102",
]
Note that this is for the original range starting from 1. This looks like it is fine, but consider the case where you would add a double-digit number (or even triple-digit number to drive the point home). Additionally, the second part of the range is fine unless you set the Number_of_AA_Candidates to a value greater than or equal to the maximum number of IP addresses. If you were somehow to miscalculate, the range would go over and the IP addresses that would be created would not be valid IP addresses. To make sure you do not overstep the maximum number of available IPs in the CIDR range, you can calculate that number with:
2^10 - 5
10 is the number of bits that remains after subnet bits are deducted from the maximum number of bits which is 32. The 5 is the number of IP addresses that cannot be used. This leaves you with 1019 possible host addresses. To make sure that does not happen, you could introduce the ternary operator for the second part of the range function:
IP_AA_MGMT_Windows = [for i in range(4, (var.Number_of_AA_Candidates > 1019 ? 1019 : var.Number_of_AA_Candidates + 1) ) : format("%s%02d", "10.10.8.1", i)]
Now, those are two issues resolved. The third and final issue is the format function. To enable the usage of only available IP addresses and avoid using format, I suggest trying the cidrhost built-in function [2]. The cidrhost syntax is:
cidrhost(prefix, hostnum)
The hostnum part represents the wanted host IP address in the CIDR range. So for example, if you were to do:
cidrhost("10.10.8.0/22", 1)
This would return the first IP address in the range. For hostnum equal to 2 it would return 2nd, and so on.
To use this properly, you would have to modify the local variable to look like this:
IP_AA_MGMT_Windows = [for i in range(4, (var.Number_of_AA_Candidates > 1019 ? 1019 : var.Number_of_AA_Candidates + 1)) : cidrhost("10.10.8.0/22", i)]
This works well with any number up to the maximum number of host IP addresses. Finally, even though we know there are 5 IP addresses we cannot use, cidrhost does not know anything about that and always starts counting from the first to the last number in a CIDR range, so the last expression would have to use 1023 addresses, as we don't want to include the broadcast one (the start IP address is covered because we start from 4):
IP_AA_MGMT_Windows = [for i in range(4, (var.Number_of_AA_Candidates > 1023 ? 1023 : var.Number_of_AA_Candidates + 1)) : cidrhost("10.10.8.0/22", i)]
EDIT: After having a discussion in chat we have identified that there is an issue with the argument in the aws_network_interface (even though terraform did not complain about it). The argument in the question is private_ip while the provider lists that as private_ips which is a list of strings [3]. After changing that to:
private_ips = [ local.IP_AA_MGMT_Windows[count.index] ]
The apply worked as expected.
[1] https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html#subnet-sizing
[2] https://www.terraform.io/language/functions/cidrhost
[3] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_interface#private_ips
Related
Terraform variable validation for CIDR, looking alternative for regex
Below is the tested code in Terraform version 13.0
is there any alternative way to achieve same thing with not using regex?
cidr block - start 172.28.0.0.0/16
variable "vpc_cidr" {
description = "Kubernetes cluster CIDR notation for vpc."
validation {
condition = can(regex("^([0-9]{1,3}\\.){3}[0-9]{1,3}($|/(16))$", var.pod_cidr))
error_message = "Vpc_cidr value must be greater than 172.0.0.0/16."
}
}
how to validate CIDR block 172.28.x.x/16 with cidrsubnet function?
https://www.terraform.io/docs/language/functions/cidrsubnet.html
validation condition - if IP range is outof 172.28.x.x/16 then validation will be failed
I can't think of a direct way to achieve what you want with the functions in today's Terraform, but I think we can break the check into two conditions that, taken together, lead to the same effect:
The zeroth address in the subnet must be 172.28.0.0.
The netmask of the range must be 255.255.0.0.
We can test these two rules using two expressions:
cidrhost(var.vpc_cidr, 0) == "172.28.0.0"
cidrnetmask(var.vpc_cidr) == "255.255.0.0"
Both of these functions can fail if given something that isn't valid CIDR block syntax at all, and cidrnetmask will additionally fail if given an IPv6 address, so we can add try guards to both of them to turn that error into a false result as condition expects:
try(cidrhost(var.vpc_cidr, 0), null) == "172.28.0.0"
try(cidrnetmask(var.vpc_cidr), null) == "255.255.0.0"
The try function will return the result of the first expression that doesn't produce a dynamic error, so in the above examples an invalid input to the cidrhost or cidrnetmask function will cause an expression like null == "172.28.0.0", which will always be false and thus the condition will still not be met.
Finally, we can combine these together using the && operator to get the complete condition expression:
condition = (
try(cidrhost(var.vpc_cidr, 0), null) == "172.28.0.0" &&
try(cidrnetmask(var.vpc_cidr), null) == "255.255.0.0"
)
You have one 0 to many and it should be greater then 172.0.0.0/16. For example:
172.1.0.0/16
not:
172.0.0.0.0/16
A bit of hack, but still:
variable "vpc_cidr" {
validation {
condition = cidrsubnet("${var.pod_cidr}/16", 0, 0) == "172.28.0.0/16"
}
}
I prefer the solution from https://dev.to/drewmullen/terraform-variable-validation-with-samples-1ank
variable "string_like_valid_ipv4_cidr" {
type = string
default = "10.0.0.0/16"
validation {
condition = can(cidrhost(var.string_like_valid_ipv4_cidr, 32))
error_message = "Must be valid IPv4 CIDR."
}
}
Also note, as commented there, that the condition requires a modification to work for /32 addresses.
I want to set random destinations for an array of 100 nodes in Udp basic app
*.host[*].udpApp[0].destAddresses = "host[${intuniform(0,99)}]"
I need to all source nodes to select a random destination and start sending traffic. But omnet++ is giving error in above statement. Already tried
*.host[*].udpApp[0].destAddresses = "host[${0..99}]" but it is only selecting first node for all nodes for 1 simulation run.
You cannot achieve your goal this way because according to OMNeT++ Simulation Manual in INI file:
Variables are substituted textually, and the result is normally not evaluated as an arithmetic expression.
As a matter of fact, the manipulation with the value of destAddresses is unnecessary, because UDP Basic App does choose the destination address randomly from the set given in destAddresses. Take a look at that method in UdpBasicApp.cc:
L3Address UdpBasicApp::chooseDestAddr()
{
int k = intrand(destAddresses.size());
if (destAddresses[k].isUnspecified() || destAddresses[k].isLinkLocal()) {
L3AddressResolver().tryResolve(destAddressStr[k].c_str(), destAddresses[k]);
}
return destAddresses[k];
}
What you should to do is to add all hosts into destAddresses. For example, assuming that there are five hosts in the network:
*.host[*].udpApp[0].destAddresses = "host[0] host[1] host[2] host[3] host[4]"
I need to be able to output all the ranges of IP addresses that are not in a given list of IP addresses ranges.
There is some sort of algorithm that I can use for this kind of task that I can transform into working code?
Basically I will use Salesforce Apex code, so any JAVA like language will do if a given example is possible.
I think the key for an easy solution is to remember IP addresses can be treated as a number of type long, and so they can be sorted.
I assumed the excluded ranges are given in a "nice" way, meaning no overlaps, no partial overlaps with global range and so on. You can of course add such input checks later on.
In this example I'll to all network ranges (global, included, excluded) as instances of NetworkRange class.
Following is the implementation of NetworkRange. Pay attention to the methods splitByExcludedRange and includes.
public class NetworkRange {
private long startAddress;
private long endAddress;
public NetworkRange(String start, String end) {
startAddress = addressRepresentationToAddress(start);
endAddress = addressRepresentationToAddress(end);
}
public NetworkRange(long start, long end) {
startAddress = start;
endAddress = end;
}
public String getStartAddress() {
return addressToAddressRepresentation(startAddress);
}
public String getEndAddress() {
return addressToAddressRepresentation(endAddress);
}
static String addressToAddressRepresentation(long address) {
String result = String.valueOf(address % 256);
for (int i = 1; i < 4; i++) {
address = address / 256;
result = String.valueOf(address % 256) + "." + result;
}
return result;
}
static long addressRepresentationToAddress(String addressRep) {
long result = 0L;
String[] tokens = addressRep.split("\\.");
for (int i = 0; i < 4; i++) {
result += Math.pow(256, i) * Long.parseLong(tokens[3-i]);
}
return result;
}
public List<NetworkRange> splitByExcludedRange(NetworkRange excludedRange) {
if (this.startAddress == excludedRange.startAddress && this.endAddress == excludedRange.endAddress)
return Arrays.asList();
if (this.startAddress == excludedRange.startAddress)
return Arrays.asList(new NetworkRange(excludedRange.endAddress+1, this.endAddress));
if (this.endAddress == excludedRange.endAddress)
return Arrays.asList(new NetworkRange(this.startAddress, excludedRange.startAddress-1));
return Arrays.asList(new NetworkRange(this.startAddress, excludedRange.startAddress-1),
new NetworkRange(excludedRange.endAddress+1, this.endAddress));
}
public boolean includes(NetworkRange excludedRange) {
return this.startAddress <= excludedRange.startAddress && this.endAddress >= excludedRange.endAddress;
}
public String toString() {
return "[" + getStartAddress() + "-" + getEndAddress() + "]";
}
}
Now comes the class that calculates the network ranges left included. It accepts a global range in constructor.
public class RangeProducer {
private NetworkRange global;
public RangeProducer(NetworkRange global) {
this.global = global;
}
public List<NetworkRange> computeEffectiveRanges(List<NetworkRange> excludedRanges) {
List<NetworkRange> effectiveRanges = new ArrayList<>();
effectiveRanges.add(global);
List<NetworkRange> effectiveRangesSplitted = new ArrayList<>();
for (NetworkRange excludedRange : excludedRanges) {
for (NetworkRange effectiveRange : effectiveRanges) {
if (effectiveRange.includes(excludedRange)) {
effectiveRangesSplitted.addAll(effectiveRange.splitByExcludedRange(excludedRange));
} else {
effectiveRangesSplitted.add(effectiveRange);
}
}
effectiveRanges = effectiveRangesSplitted;
effectiveRangesSplitted = new ArrayList<>();
}
return effectiveRanges;
}
}
You can run the following example:
public static void main(String[] args) {
NetworkRange global = new NetworkRange("10.0.0.0", "10.255.255.255");
NetworkRange ex1 = new NetworkRange("10.0.0.0", "10.0.1.255");
NetworkRange ex2 = new NetworkRange("10.1.0.0", "10.1.1.255");
NetworkRange ex3 = new NetworkRange("10.6.1.0", "10.6.2.255");
List<NetworkRange> excluded = Arrays.asList(ex1, ex2, ex3);
RangeProducer producer = new RangeProducer(global);
for (NetworkRange effective : producer.computeEffectiveRanges(excluded)) {
System.out.println(effective);
}
}
Output should be:
[10.0.2.0-10.0.255.255]
[10.1.2.0-10.6.0.255]
[10.6.3.0-10.255.255.255]
First, I assume you mean that you get one or more disjoint CIDR ranges as input, and need to produce the list of all CIDR ranges not including any of the ones given as input. For convenience, let's further assume that the input does not include the entire IP address space: i.e. 0.0.0.0/0. (That can be accommodated with a single special case but is not of much interest.)
I've written code analogous to this before and, though I'm not at liberty to share the code, I can describe the methodology. It's essentially a binary search algorithm wherein you bisect the full address space repeatedly until you've isolated the one range you're interested in.
Think of the IP address space as a binary tree: At the root is the full IPv4 address space 0.0.0.0/0. Its children each represent half of the address space: 0.0.0.0/1 and 128.0.0.0/1. Those, in turn, can be sub-divided to create children 0.0.0.0/2 / 64.0.0.0/2 and 128.0.0.0/2 / 192.0.0.0/2, respectively. Continue this all the way down and you end up with 2**32 leaves, each of which represents a single /32 (i.e. a single address).
Now, consider this tree to be the parts of the address space that are excluded from your input list. So your task is to traverse this tree, find each range from your input list in the tree, and cut out all parts of the tree that are in your input, leaving the remaining parts of the address space.
Fortunately, you needn't actually create all the 2**32 leaves. Each node at CIDR N can be assumed to include all nodes at CIDR N+1 and above if no children have been created for it (you'll need a flag to remember that it has already been subdivided -- i.e. is no longer a leaf -- see below for why).
So, to start, the entire address space is present in the tree, but can all be represented by a single leaf node. Call the tree excluded, and initialize it with the single node 0.0.0.0/0.
Now, take the first input range to consider -- we'll call this trial (I'll use 14.27.34.0/24 as the initial trial value just to provide a concrete value for demonstration). The task is to remove trial from excluded leaving the rest of the address space.
Start with current node pointer set to the excluded root node.
Start:
Compare the trial CIDR with current. If it is identical, you're done (but this should never happen if your input ranges are disjoint and you've excluded 0.0.0.0/0 from input).
Otherwise, if current is a leaf node (has not been subdivided, meaning it represents the entire address space at this CIDR level and below), set its sub-divided flag, and create two children for it: a left pointer to the first half of its address space, and a right pointer to the latter half. Label each of these appropriately (for the root node's children, that will be 0.0.0.0/1 and 128.0.0.0/1).
Determine whether the trial CIDR falls within the left side or the right side of current. For our initial trial value, it's to the left. Now, if the pointer on that side is already NULL, again you're done (though again that "can't happen" if your input ranges are disjoint).
If the trial CIDR is exactly equivalent to the CIDR in the node on that side, then simply free the node (and any children it might have, which again should be none if you have only disjoint inputs), set the pointer to that side NULL and you're done. You've just excluded that entire range by cutting that leaf out of the tree.
If the trial value is not exactly equivalent to the CIDR in the node on that side, set current to that side and start over (i.e. jump to Start label above).
So, with the initial input range of 14.27.34.0/24, you will first split 0.0.0.0/0 into 0.0.0.0/1 and 128.0.0.0/1. You will then drop down on the left side and split 0.0.0.0/1 into 0.0.0.0/2 and 64.0.0.0/2. You will then drop down to the left again to create 0.0.0.0/3 and 32.0.0.0/3. And so forth, until after 23 splits, you will then split 14.27.34.0/23 into 14.27.34.0/24 and 14.27.35.0/24. You will then delete the left-hand 14.27.34.0/24 child node and set its pointer to NULL, leaving the other.
That will leave you with a sparse tree containing 24 leaf nodes (after you dropped the target one). The remaining leaf nodes are marked with *:
(ROOT)
0.0.0.0/0
/ \
0.0.0.0/1 128.0.0.0/1*
/ \
0.0.0.0/2 64.0.0.0/2*
/ \
0.0.0.0/3 32.0.0.0.0/3*
/ \
0.0.0.0/4 16.0.0.0/4*
/ \
*0.0.0.0/5 8.0.0.0/5
/ \
*8.0.0.0/6 12.0.0.0/6
/ \
*12.0.0.0/7 14.0.0.0/7
/ \
14.0.0.0/8 15.0.0.0/8*
/ \
...
/ \
*14.27.32.0/23 14.27.34.0/23
/ \
(null) 14.27.35.0/24*
(14.27.34.0/24)
For each remaining input range, you will run through the tree again, bisecting leaf nodes when necessary, often resulting in more leaves, but always cutting out some part of the address space.
At the end, you simply traverse the resulting tree in whatever order is convenient, collecting the CIDRs of the remaining leaves. Note that in this phase you must exclude those that have previously been subdivided. Consider for example, in the above tree, if you next processed input range 14.27.35.0/24, you would leave 14.27.34.0/23 with no children, but both its halves have been separately cut out and it should not be included in the output. (With some additional complication, you could of course collapse nodes above it to accommodate that scenario as well, but it's easier to just keep a flag in each node.)
First, what you describe can be simplified to:
you have intervals of the form x.x.x.x - y.y.y.y
you want to output the intervals that are not yet "taken" in this range.
you want to be able to add or remove intervals efficiently
I would suggest the use of an interval tree, where each node stores an interval, and you can efficiently insert and remove nodes; and query for overlaps at a given point (= IP address).
If you can guarantee that there will be no overlaps, you can instead use a simple TreeSet<String>, where you must however guarantee (for correct sorting) that all strings use the xxx.xxx.xxx.xxx-yyy.yyy.yyy.yyy zero-padded format.
Once your intervals are in a tree, you can then generate your desired output, assuming that no intervals overlap, by performing a depth-first pre-order traversal of your tree, and storing the starts and ends of each visited node in a list. Given this list,
pre-pend 0.0.0.0 at the start
append 255.255.255.255 at the end
remove all duplicate ips (which will forcefully be right next to each other in the list)
take them by pairs (the number will always be even), and there you have the intervals of free IPs, perfectly sorted.
Note that 0.0.0.0 and 255.255.255.255 are not actually valid, routable IPs. You should read the relevant RFCs if you really need to output real-world-aware IPs.
I'm new with VBScript, and I need to find a way to fill with zero a group of an IP address when it does not have three characters per group. For example, I got an IP address, "10.67.131.1", and I need to save it in a variable as "010.067.131.001".
I already have a function which gets the IP address, but I'll never know how many characters it'll have in which group. So I need a function that fills it automatically.
#KenWhite asks a good question in the comments, but assuming that you have a good reason for wanting to store IP addresses in a left-padded manner then you can certainly use VBScript to convert to that form. Here is a simple function which does so:
Option Explicit
Function PadGroups(s)
Dim A,i, group
A = Split(s,".")
For i = 0 To UBound(A)
group = A(i)
If Len(group) < 3 Then
group = String(3-Len(group),"0") & group
A(i) = group
End If
Next
PadGroups = Join(A,".")
End Function
'test
MsgBox PadGroups("10.67.131.1" )
Does anyone know of a way in Ruby, or even through a web service, if there's a tool that will take a bunch of ip addresses (currently about 2 million of them) in a file and convert them to ip ranges like 192.168.0.1 - 192.168.0.10 ?
Convert the IP address to 32 bit integer (Assume you're dealing with IPv4 address according to your post), remove the duplicates, sort them, and do the merge. After that, convert the integers back to IP string:
require 'ipaddr'
def to_ranges(ips)
ips = ips.map{|ip| IPAddr.new(ip).to_i }.uniq.sort
prev = ips[0]
ips
.slice_before {|e|
prev2, prev = prev, e
prev2 + 1 != e
}
.map {|addrs| if addrs.length > 1 then [addrs[0], addrs[-1]] else addrs end }
.map {|addrs| addrs.map{|ip| IPAddr.new(ip, Socket::AF_INET)}.join("-") }
end
# some ip samples
ips = (0..255).map{|i| ["192.168.0.#{i}", "192.168.1.#{i}", "192.168.2.#{i}"] }.reduce(:+)
ips += ["192.168.3.0", "192.168.3.1"]
ips += ["192.168.3.5", "192.168.3.6"]
ips += ["192.168.5.1"]
ips += ["192.168.6.255", "192.168.7.0", "192.168.7.1"]
p to_ranges(ips)
# => ["192.168.0.0-192.168.3.1", "192.168.3.5-192.168.3.6", "192.168.5.1", "192.168.6.255-192.168.7.1"]
Reading IP addresses from file and storing them in an array should be relatively easy. 2 million IP addresses is a small set. You don't need to worry to much about the memory usage. (If it really matters, you may need to implement a algorithm to incrementally convert and merge the addresses)
BTW, I found the handy method Enumerable#slice_before when solving your problem.