Check if IP address is in private network space - go

I have a program in go which accepts URLs from clients and gets them using the net/http package. Before doing further processing, I would like to check if the URL maps to private (non-routable / RFC1918 networks) address space.
The straight-forward way would be to perform an explicit DNS request and check the address for the known private ranges. After that, perform the HTTP GET request for the URL.
Is there a better way to accomplish this? Preferably integrating with http.Client so it can be performed as a part of the GET request.

You might also want to include checks for loopback (IPv4 or IPv6) and/or IPv6 link-local or unique-local addresses. Here is an example with a list of RFC1918 address plus these others and a simple check against them as isPrivateIP(ip net.IP):
var privateIPBlocks []*net.IPNet
func init() {
for _, cidr := range []string{
"127.0.0.0/8", // IPv4 loopback
"10.0.0.0/8", // RFC1918
"172.16.0.0/12", // RFC1918
"192.168.0.0/16", // RFC1918
"169.254.0.0/16", // RFC3927 link-local
"::1/128", // IPv6 loopback
"fe80::/10", // IPv6 link-local
"fc00::/7", // IPv6 unique local addr
} {
_, block, err := net.ParseCIDR(cidr)
if err != nil {
panic(fmt.Errorf("parse error on %q: %v", cidr, err))
}
privateIPBlocks = append(privateIPBlocks, block)
}
}
func isPrivateIP(ip net.IP) bool {
if ip.IsLoopback() || ip.IsLinkLocalUnicast() || ip.IsLinkLocalMulticast() {
return true
}
for _, block := range privateIPBlocks {
if block.Contains(ip) {
return true
}
}
return false
}

That should be easier to do with Go 1.17 (Q4 2021, 5 years later), as reported by Go 101:
See commit c73fccc and CL 272668:
net: add IP.IsPrivate()
Add IsPrivate() helper to check if an IP is private according to RFC 1918 & RFC 4193
That fixes golang issue 29146 raised by Aaran McGuire:
The net package seems to have many helpers to report what an IP is. e.g:
IsLoopback()
IsMulticast()
IsInterfaceLocalMulticast()
However there are no helpers to report if a IP address is in the private ranges (RFC 1918 & RFC 4193).

package main
import (
"fmt"
"net"
)
func main() {
fmt.Println(privateIPCheck("1.1.1.1")) // False since this is not a private IP
fmt.Println(privateIPCheck("10.8.0.1")) // True: Since this is a private ip.
}
// Check if a ip is private.
func privateIPCheck(ip string) bool {
ipAddress := net.ParseIP(ip)
return ipAddress.IsPrivate()
}
This requires Go 1.17.

It seems there's no better way to accomplish than the one I described. Combining code from #MichaelHausenblas with the suggestion from #JimB, my code ended up kind of like this.
func privateIP(ip string) (bool, error) {
var err error
private := false
IP := net.ParseIP(ip)
if IP == nil {
err = errors.New("Invalid IP")
} else {
_, private24BitBlock, _ := net.ParseCIDR("10.0.0.0/8")
_, private20BitBlock, _ := net.ParseCIDR("172.16.0.0/12")
_, private16BitBlock, _ := net.ParseCIDR("192.168.0.0/16")
private = private24BitBlock.Contains(IP) || private20BitBlock.Contains(IP) || private16BitBlock.Contains(IP)
}
return private, err
}

Related

ICMP Golang with raw socket, control message has nil value

I'm playing around with the ICMP raw socket of Golang. I'd like to read the TTL which is part the control message returned by ReadFrom(buffer).
Weirdly this value is always nil, is there something I'm missing.
Please find below my playground code:
package main
import (
"fmt"
"golang.org/x/net/icmp"
"golang.org/x/net/ipv4"
)
func main() {
c, _ := icmp.ListenPacket("ip4:icmp", "")
rb := make([]byte, 1500)
for true {
n, cm, peer, _ := c.IPv4PacketConn().ReadFrom(rb)
rm, _ := icmp.ParseMessage(ipv4.ICMPTypeEchoReply.Protocol(), rb[:n])
switch rm.Type {
case ipv4.ICMPTypeEchoReply:
{
fmt.Printf("received answer from %s\n", peer)
if cm != nil {
println(cm.TTL)
} else {
println("empty control message")
}
}
default:
}
}
}
Finally, I found out what was missing.
Before reading, it is required to set IP socket options.
In my case, I was interested in TTL, so:
_ = c.IPv4PacketConn().SetControlMessage(ipv4.FlagTTL, true)

Any way to tell DNS resolver in go to lookup IPV4 address only?

I'm trying to configure a Go program (namely, Prometheus) to only look up IPv4 addresses, but seems the LookupIP function that the program uses for DNS resolution doesn't have an option for that.
Is there any way I can work around it, or am I getting something wrong?
the LookupIP function as in the src:
// LookupIP looks up a host using the local resolver.
// It returns a slice of that host's IPv4 and IPv6 addresses.
func LookupIP(host string) ([]IP, error) {
addrs, err := DefaultResolver.LookupIPAddr(context.Background(), host)
if err != nil {
return nil, err
}
ips := make([]IP, len(addrs))
for i, ia := range addrs {
ips[i] = ia.IP
}
return ips, nil
}
I think you're looking for LookupIP. It's the same function but your pass the ip network you are targeting.
// LookupIP looks up host for the given network using the local resolver.
// It returns a slice of that host's IP addresses of the type specified by
// network.
// network must be one of "ip", "ip4" or "ip6".
func (r *Resolver) LookupIP(ctx context.Context, network, host string) ([]IP, error) {
Usage:
net.DefaultResolver.LookupIP(context.Background(), "ip4", "host")

Memcached Ping() doesn't return an error on an invalid server

I use memcache for caching and the client I use is https://github.com/bradfitz/gomemcache. When I tried initiate new client with dummy/invalid server address and then pinging to it, it return no error.
package main
import (
"fmt"
m "github.com/bradfitz/gomemcache"
)
func main() {
o := m.New("dummy_adress")
fmt.Println(o.Ping()) // return no error
}
I think it suppose to return error as the server is invalid. What do I miss?
It looks like the New() call ignores the return value for SetServers:
func New(server ...string) *Client {
ss := new(ServerList)
ss.SetServers(server...)
return NewFromSelector(ss)
}
The SetServers() function will only set the server list to valid servers (in
your case: no servers) and the Ping() funtion will only ping servers that are
set, and since there are no servers set it doesn't really do anything.
This is arguably a feature; if you have 4 servers and one is down then that's
not really an issue. Even with just 1 server memcache is generally optional.
You can duplicate the New() logic with an error check:
ss := new(memcache.ServerList)
err := ss.SetServers("example.localhost:11211")
if err != nil {
panic(err)
}
c := memcache.NewFromSelector(ss)
err = c.Ping()
if err != nil {
panic(err)
}
Which gives:
panic: dial tcp 127.0.0.1:11211: connect: connection refused

bind: The requested address is not valid in its context

I develop a bot on golang. Started it on vds with ubuntu 20.01 OS and it works great, but it's a problem when I'm starting to debug my code. Because of it I decide to use my PC as a VDS: I have opened a 8443 port and etc. But when main.go is started I get an error:
listen tcp [ip]:8443: bind: The requested address is not valid in its context.
My code:
package main
import (
"./configuration"
"./get_data"
"encoding/json"
"fmt"
tgBotApi "github.com/go-telegram-bot-api/telegram-bot-api"
"io/ioutil"
"net/http"
"strings"
"time"
)
var (
NewBot, BotErr = tgBotApi.NewBotAPI(configuration.BOT_TOKEN)
)
func setWebhook(bot *tgBotApi.BotAPI) {
webHookInfo := tgBotApi.NewWebhookWithCert(fmt.Sprintf("https://%s:%s/%s",
configuration.BOT_HOST, configuration.BOT_PORT, configuration.BOT_TOKEN), configuration.CERT_FILE)
_, err := bot.SetWebhook(webHookInfo)
if err != nil {
fmt.Println(err)
}
}
func main () {
fmt.Println("OK", time.Now().Unix(), time.Now(), time.Now().Weekday())
setWebhook(NewBot)
message := func (w http.ResponseWriter, r *http.Request) {
text, _ := ioutil.ReadAll(r.Body)
var botText = get_data.BotMessage{}
_ = json.Unmarshal(text, &botText)
chatGroup := int64(botText.Message.Chat.Id)
botCommand := strings.Split(botText.Message.Text, "#")[0]
/*markup := tgBotApi.InlineKeyboardMarkup{
InlineKeyboard: [][]tgBotApi.InlineKeyboardButton{
[]tgBotApi.InlineKeyboardButton{
tgBotApi.InlineKeyboardButton{Text: "start"},
},
},
}*/
//reply := tgBotApi.NewEditMessageReplyMarkup(chatGroup, messageId, markup)
var msg tgBotApi.MessageConfig
switch botCommand {
case "/start":
msg = tgBotApi.NewMessage(chatGroup, helloCommandMsg())
msg.ReplyMarkup = tgBotApi.NewReplyKeyboard(
[]tgBotApi.KeyboardButton{catalogBtn.Btn},
[]tgBotApi.KeyboardButton{myProfileBtn.Btn, supportBtn.Btn},
[]tgBotApi.KeyboardButton{getLuckyBtn.Btn, rulesBtn.Btn},
[]tgBotApi.KeyboardButton{addFundsBtn.Btn},
)
break
case getLuckyBtn.Btn.Text:
getLuckyBtn.Action(botText, NewBot)
break
case myProfileBtn.Btn.Text:
myProfileBtn.Action(botText, NewBot)
break
case addFundsBtn.Btn.Text:
addFundsBtn.Action(botText, NewBot)
break
default:
msg = tgBotApi.NewMessage(chatGroup, unknownCommandMsg())
}
NewBot.Send(msg)
}
http.HandleFunc("/", message)
errListenTLS := http.ListenAndServeTLS(fmt.Sprintf("%s:%s", configuration.BOT_HOST, configuration.BOT_PORT), configuration.CERT_FILE, configuration.CERT_KEY, nil)
if errListenTLS != nil {
fmt.Println(errListenTLS)
}
}
configuration/main_config.go:
package configuration
const (
TELEGRAM_URL = "https://api.telegram.org/bot"
BOT_TOKEN = MYTOKEN
BOT_HOST = "[ip]"
BOT_PORT = "8443"
)
configuration/cert_config.go:
package configuration
const (
CERT_FILE = "C:\\Users\\USER\\Desktop\\GOlocal\\BOTlocal\\certificates\\mybot.pem"
CERT_KEY = "C:\\Users\\USER\\Desktop\\GOlocal\\BOTlocal\\certificates\\mybot.key"
)
I have generated the certificate like in this topic: What is easy way to create and use a Self-Signed Certification for a Telegram Webhook?
Moreover, I'm trying to set 0.0.0.0 instead my public ip and then this error comes out:
Bad Request: bad webhook: IP address 0.0.0.0 is reserved
The error bind: The requested address is not valid in its context. indicates that the address does not belong to a network interface on your machine. Likely, there is a router/load balancer/etc... with the actual public address and it is forwarding traffic to your machine.
You need to split the addresses you use:
your local address (see ifconfig) or 0.0.0.0 for all interface as the address passed to http.ListenAndServeTLS
the public address as the callback address passed in NewWebhookWithCert

Dial multiple IP addresses with golang gRPC client

I have grpc server 192.168.1.12:8800 and 192.168.1.13:8800, I want to connect them use grpc.Dial with ip list, not server discover, How can I do?
conn, err = grpc.Dial("192.168.1.12:8800,192.168.1.13:8800", grpc.WithInsecure())
with error
rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp: too many colons in address 192.168.1.12:8800,192.168.1.13:8800
Unfortunately, you aren't able to pass multiple IP addresses using grpc.Dial(...), it only takes a single argument.
gRPC in Go does have an "experimental" load balancer api that you should be able to take advantage of.
An example of the resolver you would need to write can be found here. It creates a fake resolver that will load balance across the multiple IP addresses.
So once you have a resolver such as that, the code you would want would look something like this:
conn, err := grpc.Dial(
"",
grpc.WithInsecure(),
grpc.WithBalancer(grpc.RoundRobin(resolver.NewPseudoResolver([]string{
"10.0.0.1:10000",
"10.0.0.2:10000",
}))),
)
I wrote a library for this: https://github.com/Jille/grpc-multi-resolver
Usage is easy:
import _ "github.com/Jille/grpc-multi-resolver"
grpc.Dial("multi:///192.168.1.12:8800,192.168.1.13:8800")
Note the triple slash at the beginning.
I recently implemented this kind of gRPC resolver in https://github.com/letsencrypt/boulder:
resolver implementation:
package grpc
import (
"fmt"
"net"
"strings"
"google.golang.org/grpc/resolver"
)
// staticBuilder implements the `resolver.Builder` interface.
type staticBuilder struct{}
// newStaticBuilder creates a `staticBuilder` used to construct static DNS
// resolvers.
func newStaticBuilder() resolver.Builder {
return &staticBuilder{}
}
// Build implements the `resolver.Builder` interface and is usually called by
// the gRPC dialer. It takes a target containing a comma separated list of
// IPv4/6 addresses and a `resolver.ClientConn` and returns a `staticResolver`
// which implements the `resolver.Resolver` interface.
func (sb *staticBuilder) Build(target resolver.Target, cc resolver.ClientConn, _ resolver.BuildOptions) (resolver.Resolver, error) {
var resolverAddrs []resolver.Address
for _, address := range strings.Split(target.Endpoint, ",") {
parsedAddress, err := parseResolverIPAddress(address)
if err != nil {
return nil, err
}
resolverAddrs = append(resolverAddrs, *parsedAddress)
}
return newStaticResolver(cc, resolverAddrs), nil
}
// Scheme returns the scheme that `staticBuilder` will be registered for, for
// example: `static:///`.
func (sb *staticBuilder) Scheme() string {
return "static"
}
// staticResolver is used to wrap an inner `resolver.ClientConn` and implements
// the `resolver.Resolver` interface.
type staticResolver struct {
cc resolver.ClientConn
}
// newStaticResolver takes a `resolver.ClientConn` and a list of
// `resolver.Addresses`. It updates the state of the `resolver.ClientConn` with
// the provided addresses and returns a `staticResolver` which wraps the
// `resolver.ClientConn` and implements the `resolver.Resolver` interface.
func newStaticResolver(cc resolver.ClientConn, resolverAddrs []resolver.Address) resolver.Resolver {
cc.UpdateState(resolver.State{Addresses: resolverAddrs})
return &staticResolver{cc: cc}
}
// ResolveNow is a no-op necessary for `staticResolver` to implement the
// `resolver.Resolver` interface. This resolver is constructed once by
// staticBuilder.Build and the state of the inner `resolver.ClientConn` is never
// updated.
func (sr *staticResolver) ResolveNow(_ resolver.ResolveNowOptions) {}
// Close is a no-op necessary for `staticResolver` to implement the
// `resolver.Resolver` interface.
func (sr *staticResolver) Close() {}
// parseResolverIPAddress takes an IPv4/6 address (ip:port, [ip]:port, or :port)
// and returns a properly formatted `resolver.Address` object. The `Addr` and
// `ServerName` fields of the returned `resolver.Address` will both be set to
// host:port or [host]:port if the host is an IPv6 address.
func parseResolverIPAddress(addr string) (*resolver.Address, error) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
return nil, fmt.Errorf("splitting host and port for address %q: %w", addr, err)
}
if port == "" {
// If the port field is empty the address ends with colon (e.g.
// "[::1]:").
return nil, fmt.Errorf("address %q missing port after port-separator colon", addr)
}
if host == "" {
// Address only has a port (i.e ipv4-host:port, [ipv6-host]:port,
// host-name:port). Keep consistent with net.Dial(); if the host is
// empty (e.g. :80), the local system is assumed.
host = "127.0.0.1"
}
if net.ParseIP(host) == nil {
// Host is a DNS name or an IPv6 address without brackets.
return nil, fmt.Errorf("address %q is not an IP address", addr)
}
parsedAddr := net.JoinHostPort(host, port)
return &resolver.Address{
Addr: parsedAddr,
ServerName: parsedAddr,
}, nil
}
// init registers the `staticBuilder` with the gRPC resolver registry.
func init() {
resolver.Register(newStaticBuilder())
}
Usage example:
return grpc.Dial(
"static:///192.168.1.12:8800,192.168.1.13:8800",
grpc.WithBalancerName("round_robin"),
)

Resources