How this can be done? I would like to know how to parse a string into a net.Addr in Go. As I see there are only parsers for CIDR, IP, and MAC.
Use the net.Resolve* functions to parse addresses. The functions return values that satisfy the net.Addr interface.
The net.ResolveIPAddr function parses the address for an IP endpoint:
ip, err := net.ResolveIPAddr("ip", "203.0.113.0")
if err != nil {
// handle error
}
Use net.ResolveUPDAddr to parse the address of a UDP endpoint (the scenario mentioned in the question comments). The addresss of a UDP endpoint is an IP endpoint and port.
ip, err := net.ResolveUDPAddr("udp", "203.0.113.0:53")
if err != nil {
// handle error
}
The variables ip in the snippets above are typed to the specific address type, but can be used anywhere a net.Addr is needed.
Use net.ResolveIPAddr
var ip net.Addr
ip, _ = net.ResolveIPAddr("ip", "1.1.1.1")
Related
I have a GCP cloud run service that posts data to external URLs. These outgoing requests are routed thru a NAT to ensure requests originate from a static IP address. When I post the data to https://curlmyip.org, CurlmyIP correctly identifies the static IP address.
I have another Cloud Run service that can receive the same data. However, this service is unable to retrieve the static IP address. I am using this code to try and retrieve the static IP address of the incoming request
func getIP(r *http.Request) (string, error) {
//Get IP from the X-REAL-IP header
ip := r.Header.Get("X-REAL-IP")
log.Printf("X-REAL-IP: %s", ip)
netIP := net.ParseIP(ip)
if netIP != nil {
return ip, nil
}
//Get IP from X-FORWARDED-FOR header
ips := r.Header.Get("X-FORWARDED-FOR")
log.Printf("X-FORWARDED-FOR: %s", ips)
splitIps := strings.Split(ips, ",")
for _, ip := range splitIps {
netIP := net.ParseIP(ip)
if netIP != nil {
return ip, nil
}
}
//Get IP from RemoteAddr
ip, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return "", err
}
log.Printf("r.RemoteAddr: %s", ip)
netIP = net.ParseIP(ip)
if netIP != nil {
return ip, nil
}
return "", fmt.Errorf("no valid ip found")
}
The above always returns 0.0.0.0 which is in r.RemoteAddr and not the configured static IP address. I added logs to print all the headers in the incoming request and the configured IP address is nowhere to be found.
Depending on what routers/proxies/load balancers/content delivery networks/etc. the request passes through before it gets to you (and how those boxen are configured), the header could be found in any of
True-Client-IP
X-Forwarded-For
X-Real-IP
Others?
Or it could simply not be there at all.
In the past we use the header x-forwarded-for but we skip the gcp ip ranges via nginx module.
There are some ip blocks that we found in the web but you must include the load balancer ip too. Then extract the first non-private ip from header.
You can do with pure go. Just need to find the ip ranges to skip
I'm trying to configure a Go program (namely, Prometheus) to only look up IPv4 addresses, but seems the LookupIP function that the program uses for DNS resolution doesn't have an option for that.
Is there any way I can work around it, or am I getting something wrong?
the LookupIP function as in the src:
// LookupIP looks up a host using the local resolver.
// It returns a slice of that host's IPv4 and IPv6 addresses.
func LookupIP(host string) ([]IP, error) {
addrs, err := DefaultResolver.LookupIPAddr(context.Background(), host)
if err != nil {
return nil, err
}
ips := make([]IP, len(addrs))
for i, ia := range addrs {
ips[i] = ia.IP
}
return ips, nil
}
I think you're looking for LookupIP. It's the same function but your pass the ip network you are targeting.
// LookupIP looks up host for the given network using the local resolver.
// It returns a slice of that host's IP addresses of the type specified by
// network.
// network must be one of "ip", "ip4" or "ip6".
func (r *Resolver) LookupIP(ctx context.Context, network, host string) ([]IP, error) {
Usage:
net.DefaultResolver.LookupIP(context.Background(), "ip4", "host")
I have an IP address and want to check if I can send requests to the internet from it.
I want to check this both for IPv4 addresses and IPv6 addresses.
The check should work for any IP address (local, public, docker's, lo, etc.).
From another angle, I want to see if I can bind to the address. For example, I have IPs set on my machine to which I can't bind (Tested via curl --interface) and want to filter those from my available addresses.
Usage: user inputs an IP address to send (http get) requests from, and I want to check if the IP is valid/usable.
EDIT on trying to bind and seeing if I can:
I've tried this:
tcpAddr, _ := net.ResolveTCPAddr("tcp", ipString)
client := &http.Client{
Transport: &http.Transport{
DialContext: (&net.Dialer{
Timeout: 10 * time.Second,
LocalAddr: tcpAddr,
}).DialContext,
},
}
_, err = client.Get("http://www.google.com/")
if err != nil {
fmt.Println("Unusable")
} else {
fmt.Println("Usable")
}
It shows all available addresses (even loopback) as usable (Even checked response bodies; All were valid). Many of those aren't bindable through curl --interface.
Try binding it and see if you get an error:
l,err := net.Listen("tcp", address)
if err != nil {
// address is not bindable
} else {
l.Close()
}
I have grpc server 192.168.1.12:8800 and 192.168.1.13:8800, I want to connect them use grpc.Dial with ip list, not server discover, How can I do?
conn, err = grpc.Dial("192.168.1.12:8800,192.168.1.13:8800", grpc.WithInsecure())
with error
rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp: too many colons in address 192.168.1.12:8800,192.168.1.13:8800
Unfortunately, you aren't able to pass multiple IP addresses using grpc.Dial(...), it only takes a single argument.
gRPC in Go does have an "experimental" load balancer api that you should be able to take advantage of.
An example of the resolver you would need to write can be found here. It creates a fake resolver that will load balance across the multiple IP addresses.
So once you have a resolver such as that, the code you would want would look something like this:
conn, err := grpc.Dial(
"",
grpc.WithInsecure(),
grpc.WithBalancer(grpc.RoundRobin(resolver.NewPseudoResolver([]string{
"10.0.0.1:10000",
"10.0.0.2:10000",
}))),
)
I wrote a library for this: https://github.com/Jille/grpc-multi-resolver
Usage is easy:
import _ "github.com/Jille/grpc-multi-resolver"
grpc.Dial("multi:///192.168.1.12:8800,192.168.1.13:8800")
Note the triple slash at the beginning.
I recently implemented this kind of gRPC resolver in https://github.com/letsencrypt/boulder:
resolver implementation:
package grpc
import (
"fmt"
"net"
"strings"
"google.golang.org/grpc/resolver"
)
// staticBuilder implements the `resolver.Builder` interface.
type staticBuilder struct{}
// newStaticBuilder creates a `staticBuilder` used to construct static DNS
// resolvers.
func newStaticBuilder() resolver.Builder {
return &staticBuilder{}
}
// Build implements the `resolver.Builder` interface and is usually called by
// the gRPC dialer. It takes a target containing a comma separated list of
// IPv4/6 addresses and a `resolver.ClientConn` and returns a `staticResolver`
// which implements the `resolver.Resolver` interface.
func (sb *staticBuilder) Build(target resolver.Target, cc resolver.ClientConn, _ resolver.BuildOptions) (resolver.Resolver, error) {
var resolverAddrs []resolver.Address
for _, address := range strings.Split(target.Endpoint, ",") {
parsedAddress, err := parseResolverIPAddress(address)
if err != nil {
return nil, err
}
resolverAddrs = append(resolverAddrs, *parsedAddress)
}
return newStaticResolver(cc, resolverAddrs), nil
}
// Scheme returns the scheme that `staticBuilder` will be registered for, for
// example: `static:///`.
func (sb *staticBuilder) Scheme() string {
return "static"
}
// staticResolver is used to wrap an inner `resolver.ClientConn` and implements
// the `resolver.Resolver` interface.
type staticResolver struct {
cc resolver.ClientConn
}
// newStaticResolver takes a `resolver.ClientConn` and a list of
// `resolver.Addresses`. It updates the state of the `resolver.ClientConn` with
// the provided addresses and returns a `staticResolver` which wraps the
// `resolver.ClientConn` and implements the `resolver.Resolver` interface.
func newStaticResolver(cc resolver.ClientConn, resolverAddrs []resolver.Address) resolver.Resolver {
cc.UpdateState(resolver.State{Addresses: resolverAddrs})
return &staticResolver{cc: cc}
}
// ResolveNow is a no-op necessary for `staticResolver` to implement the
// `resolver.Resolver` interface. This resolver is constructed once by
// staticBuilder.Build and the state of the inner `resolver.ClientConn` is never
// updated.
func (sr *staticResolver) ResolveNow(_ resolver.ResolveNowOptions) {}
// Close is a no-op necessary for `staticResolver` to implement the
// `resolver.Resolver` interface.
func (sr *staticResolver) Close() {}
// parseResolverIPAddress takes an IPv4/6 address (ip:port, [ip]:port, or :port)
// and returns a properly formatted `resolver.Address` object. The `Addr` and
// `ServerName` fields of the returned `resolver.Address` will both be set to
// host:port or [host]:port if the host is an IPv6 address.
func parseResolverIPAddress(addr string) (*resolver.Address, error) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
return nil, fmt.Errorf("splitting host and port for address %q: %w", addr, err)
}
if port == "" {
// If the port field is empty the address ends with colon (e.g.
// "[::1]:").
return nil, fmt.Errorf("address %q missing port after port-separator colon", addr)
}
if host == "" {
// Address only has a port (i.e ipv4-host:port, [ipv6-host]:port,
// host-name:port). Keep consistent with net.Dial(); if the host is
// empty (e.g. :80), the local system is assumed.
host = "127.0.0.1"
}
if net.ParseIP(host) == nil {
// Host is a DNS name or an IPv6 address without brackets.
return nil, fmt.Errorf("address %q is not an IP address", addr)
}
parsedAddr := net.JoinHostPort(host, port)
return &resolver.Address{
Addr: parsedAddr,
ServerName: parsedAddr,
}, nil
}
// init registers the `staticBuilder` with the gRPC resolver registry.
func init() {
resolver.Register(newStaticBuilder())
}
Usage example:
return grpc.Dial(
"static:///192.168.1.12:8800,192.168.1.13:8800",
grpc.WithBalancerName("round_robin"),
)
I have a program in go which accepts URLs from clients and gets them using the net/http package. Before doing further processing, I would like to check if the URL maps to private (non-routable / RFC1918 networks) address space.
The straight-forward way would be to perform an explicit DNS request and check the address for the known private ranges. After that, perform the HTTP GET request for the URL.
Is there a better way to accomplish this? Preferably integrating with http.Client so it can be performed as a part of the GET request.
You might also want to include checks for loopback (IPv4 or IPv6) and/or IPv6 link-local or unique-local addresses. Here is an example with a list of RFC1918 address plus these others and a simple check against them as isPrivateIP(ip net.IP):
var privateIPBlocks []*net.IPNet
func init() {
for _, cidr := range []string{
"127.0.0.0/8", // IPv4 loopback
"10.0.0.0/8", // RFC1918
"172.16.0.0/12", // RFC1918
"192.168.0.0/16", // RFC1918
"169.254.0.0/16", // RFC3927 link-local
"::1/128", // IPv6 loopback
"fe80::/10", // IPv6 link-local
"fc00::/7", // IPv6 unique local addr
} {
_, block, err := net.ParseCIDR(cidr)
if err != nil {
panic(fmt.Errorf("parse error on %q: %v", cidr, err))
}
privateIPBlocks = append(privateIPBlocks, block)
}
}
func isPrivateIP(ip net.IP) bool {
if ip.IsLoopback() || ip.IsLinkLocalUnicast() || ip.IsLinkLocalMulticast() {
return true
}
for _, block := range privateIPBlocks {
if block.Contains(ip) {
return true
}
}
return false
}
That should be easier to do with Go 1.17 (Q4 2021, 5 years later), as reported by Go 101:
See commit c73fccc and CL 272668:
net: add IP.IsPrivate()
Add IsPrivate() helper to check if an IP is private according to RFC 1918 & RFC 4193
That fixes golang issue 29146 raised by Aaran McGuire:
The net package seems to have many helpers to report what an IP is. e.g:
IsLoopback()
IsMulticast()
IsInterfaceLocalMulticast()
However there are no helpers to report if a IP address is in the private ranges (RFC 1918 & RFC 4193).
package main
import (
"fmt"
"net"
)
func main() {
fmt.Println(privateIPCheck("1.1.1.1")) // False since this is not a private IP
fmt.Println(privateIPCheck("10.8.0.1")) // True: Since this is a private ip.
}
// Check if a ip is private.
func privateIPCheck(ip string) bool {
ipAddress := net.ParseIP(ip)
return ipAddress.IsPrivate()
}
This requires Go 1.17.
It seems there's no better way to accomplish than the one I described. Combining code from #MichaelHausenblas with the suggestion from #JimB, my code ended up kind of like this.
func privateIP(ip string) (bool, error) {
var err error
private := false
IP := net.ParseIP(ip)
if IP == nil {
err = errors.New("Invalid IP")
} else {
_, private24BitBlock, _ := net.ParseCIDR("10.0.0.0/8")
_, private20BitBlock, _ := net.ParseCIDR("172.16.0.0/12")
_, private16BitBlock, _ := net.ParseCIDR("192.168.0.0/16")
private = private24BitBlock.Contains(IP) || private20BitBlock.Contains(IP) || private16BitBlock.Contains(IP)
}
return private, err
}