SSRF vul should forbidden local ip request.
How did go net.LookupIP work with Base 8 (octal )?
#go code
net.LookupIP("0177.0.0.01") // NOT_OK 0177.0.0.01
net.LookupIP("0266.075.0310.07") // OK 182.61.200.7
#shell
ping 0177.0.0.01 //OK 127.0.0.1
ping 0266.075.0310.07 // OK 182.61.200.7
input http:address to chrome is the same as ping ,but go not stablize
debug-snapshot-img net.LookupIP("0177.0.0.01") Has Not Decode Base8
debug-snapshot-img ping 0177.0.0.01 in shell Has Decode Base8
debug-snapshot-img net.LookupIP("0266.075.0310.07") Has Decode Base8
The problem
I think you cannot for a simple reason: that A.B.C.D format of IPv4 addresses is called "dot-decimal" (or "dotted-decimal") for a reason: the numbers representing the octets of an address are in the decimal format, and no "special modifiers"—such as 0x, 0, 0o etc—to make it "not decimal" are allowed.
I'm inclined to think that the fact ping is OK to parse the address with some octets in octal is some weird artefact of its implementation; for instance, it also parses hexadecimal:
$ ping 0xAB.0.0.0x01
PING 0xAB.0.0.0x01 (171.0.0.1) 56(84) bytes of data.
^C
Also note that there's no such thing as "the standard ping tool": any OS can have any implementation of it; for instance, ping available on Windows definitely shares no source code with ping available on a typical GNU/Linux-based OS, and that one is probably different from ping implementations available on systems tracing their heritage to *BSD.
To further illustrate my point, consider host:
$ host -t PTR 0177.0.0.01
Host 0177.0.0.01 not found: 3(NXDOMAIN)
This tool, as you can see, failed to cope with the leading 0s.
So, I think it's sensible to assume that the fact net.LookupIP (and net.ParseIP for that matter) definitely cannot be blamed for not supporting "octals" when they parse the dotted-decimal representation of an IPv4 address.
Possible solutions
Possible solutions depend on what really you're after.
If you have to accept IP addresses as strings in such a "dotted-not-so-decimal" format, I'd propose to write a simplistic custom parser which would "preprocess" the strings before calling net.LookupIP on them.
Such a preprocessor would split the string and try to parse each octet, then assemble the address back.
Go has strconv.ParseInt in its standard library, which accepts the "base" argument telling it which base the digits forming the string representation to parse are, and if you pass 0 as the base, the function uses the usual heuristics to guess the base; in particular, the leading 0 signalizes the base 8—just as you'd want it to.
So, we could roll something like this:
package main
import (
"errors"
"fmt"
"net"
"strconv"
"strings"
)
func normalizeIPv4Addr(s string) (string, error) {
parts := strings.SplitN(s, ".", 4)
if len(parts) != 4 || strings.IndexByte(parts[3], '.') != -1 {
return "", errors.New("invalid IPv4 address: invalid number of octets")
}
var dst [4]byte
for i, p := range parts {
n, err := strconv.ParseInt(p, 0, 8)
if err != nil {
return "", fmt.Errorf("invalid IPv4 address: invalid octet %d: %s", i+1, err)
}
if n < 0 || 255 < n {
return "", fmt.Errorf("invalid IPv4 address: invalid octet %d: out of range", i+1)
}
dst[i] = byte(n)
}
return net.IP(dst[:]).String(), nil
}
func main() {
fmt.Println(normalizeIPv4Addr("0177.0.0.01"))
}
Playground.
Related
I'm trying to do direct i/o on linux, so I need to create memory aligned buffers. I copied some code to do it, but I don't understand how it works:
package main
import (
"fmt"
"golang.org/x/sys/unix"
"unsafe"
"yottaStore/yottaStore-go/src/yfs/test/utils"
)
const (
AlignSize = 4096
BlockSize = 4096
)
// Looks like dark magic
func Alignment(block []byte, AlignSize int) int {
return int(uintptr(unsafe.Pointer(&block[0])) & uintptr(AlignSize-1))
}
func main() {
path := "/path/to/file.txt"
fd, err := unix.Open(path, unix.O_RDONLY|unix.O_DIRECT, 0666)
defer unix.Close(fd)
if err != nil {
panic(err)
}
file := make([]byte, 4096*2)
a := Alignment(file, AlignSize)
offset := 0
if a != 0 {
offset = AlignSize - a
}
file = file[offset : offset+BlockSize]
n, readErr := unix.Pread(fd, file, 0)
if readErr != nil {
panic(readErr)
}
fmt.Println(a, offset, offset+utils.BlockSize, len(file))
fmt.Println("Content is: ", string(file))
}
I understand that I'm generating a slice twice as big than what I need, and then extracting a memory aligned block from it, but the Alignment function doesn't make sense to me.
How does the Alignment function works?
If I try to fmt.Println the intermediate steps of that function I get different results, why? I guess because observing it changes its memory alignment (like in quantum physics :D)
Edit:
Example with fmt.println, where I don't need any more alignment:
package main
import (
"fmt"
"golang.org/x/sys/unix"
"unsafe"
)
func main() {
path := "/path/to/file.txt"
fd, err := unix.Open(path, unix.O_RDONLY|unix.O_DIRECT, 0666)
defer unix.Close(fd)
if err != nil {
panic(err)
}
file := make([]byte, 4096)
fmt.Println("Pointer: ", &file[0])
n, readErr := unix.Pread(fd, file, 0)
fmt.Println("Return is: ", n)
if readErr != nil {
panic(readErr)
}
fmt.Println("Content is: ", string(file))
}
Your AlignSize has a value of a power of 2. In binary representation it contains a 1 bit followed by full of zeros:
fmt.Printf("%b", AlignSize) // 1000000000000
A slice allocated by make() may have a memory address that is more or less random, consisting of ones and zeros following randomly in binary; or more precisely the starting address of its backing array.
Since you allocate twice the required size, that's a guarantee that the backing array will cover an address space that has an address in the middle somewhere that ends with as many zeros as the AlignSize's binary representation, and has BlockSize room in the array starting at this. We want to find this address.
This is what the Alignment() function does. It gets the starting address of the backing array with &block[0]. In Go there's no pointer arithmetic, so in order to do something like that, we have to convert the pointer to an integer (there is integer arithmetic of course). In order to do that, we have to convert the pointer to unsafe.Pointer: all pointers are convertible to this type, and unsafe.Pointer can be converted to uintptr (which is an unsigned integer large enough to store the uninterpreted bits of a pointer value), on which–being an integer–we can perform integer arithmetic.
We use bitwise AND with the value uintptr(AlignSize-1). Since AlignSize is a power of 2 (contains a single 1 bit followed by zeros), the number one less is a number whose binary representation is full of ones, as many as trailing zeros AlignSize has. See this example:
x := 0b1010101110101010101
fmt.Printf("AlignSize : %22b\n", AlignSize)
fmt.Printf("AlignSize-1 : %22b\n", AlignSize-1)
fmt.Printf("x : %22b\n", x)
fmt.Printf("result of & : %22b\n", x&(AlignSize-1))
Output:
AlignSize : 1000000000000
AlignSize-1 : 111111111111
x : 1010101110101010101
result of & : 110101010101
So the result of & is the offset which if you subtract from AlignSize, you get an address that has as many trailing zeros as AlignSize itself: the result is "aligned" to the multiple of AlignSize.
So we will use the part of the file slice starting at offset, and we only need BlockSize:
file = file[offset : offset+BlockSize]
Edit:
Looking at your modified code trying to print the steps: I get an output like:
Pointer: 0xc0000b6000
Unsafe pointer: 0xc0000b6000
Unsafe pointer, uintptr: 824634466304
Unpersand: 0
Cast to int: 0
Return is: 0
Content is:
Note nothing is changed here. Simply the fmt package prints pointer values using hexadecimal representation, prefixed by 0x. uintptr values are printed as integers, using decimal representation. Those values are equal:
fmt.Println(0xc0000b6000, 824634466304) // output: 824634466304 824634466304
Also note the rest is 0 because in my case 0xc0000b6000 is already a multiple of 4096, in binary it is 1100000000000000000100001110000000000000.
Edit #2:
When you use fmt.Println() to debug parts of the calculation, that may change escape analysis and may change the allocation of the slice (from stack to heap). This depends on the used Go version too. Do not rely on your slice being allocated at an address that is (already) aligned to AlignSize.
See related questions for more details:
Mix print and fmt.Println and stack growing
why struct arrays comparing has different result
Addresses of slices of empty structs
I want to develop a UDP server based on Linux. There are some IP set on that host machine (such as 1.1.1.1,1.1.1.2, 2001::1:1:1:1), and I want server listen on all IP as follows (9090 as sample)
udp6 0 0 :::9090 :::*
The server code as follows
package main
import (
"fmt"
"net"
)
func main() {
udpAddr, err := net.ResolveUDPAddr("udp", ":9090")
conn, err := net.ListenUDP("udp", udpAddr)
if err != nil {
fmt.Println(err)
return
}
var data [1024]byte
n, addr, err := conn.ReadFromUDP(data[:])
if err != nil {
fmt.Println(err)
}
fmt.Println(n)
fmt.Println(addr)
// this is not my wanted result. it will print [::]:9090
fmt.Println(conn.LocalAddr())
}
When the client dial this server (dst_string is 1.1.1.1:9090);
Actual result:
the server will print conn.LocalAddr() with
[::]:9090
excepted result:
the server should print
1.1.1.1:9090
How to achieve that?
BTW: I know if UDP server only listen 1.1.1.1:9090 can make that. But server has many IP, I want the server listen all IP and LocalAddr() can print 1.1.1.1:9090
Let's cite this post from a PowerDNS developer:
There are two ways to listen on all addresses, one of which is to enumerate all interfaces, grab all their IP addresses, and bind to all of them. Lots of work, and non-portable work too. We really did not want to do that. You also need to monitor new addresses arriving.
Secondly, just bind to 0.0.0.0 and ::! This works very well for TCP and other connection-oriented protocols, but can fail silently for UDP and other connectionless protocols. How come? When a packet comes in on 0.0.0.0, we don’t know which IP address it was sent to. And this is a problem when replying to such a packet – what would the correct source address be? Because we are connectionless (and therefore stateless), the kernel doesn’t know what to do.
So it picks the most appropriate address, and that may be the wrong one. There are some heuristics that make some kernels do the right thing more reliably, but there are no guarantees.
When receiving packets on datagram sockets, we usually use recvfrom(2), but this does not provide the missing bit of data: which IP address the packet was actually sent to. There is no recvfromto(). Enter the very powerful recvmsg(2). Recvmsg() allows for the getting of a boatload of parameters per datagram, as requested via setsockopt().
One of the parameters we can request is the original destination IP address of the packet.
<…>
IPv4
<..> For Linux, use the setsockopt() called IP_PKTINFO, which will get you a parameter over recvmsg() called IP_PKTINFO, which carries a struct in_pktinfo, which has a 4 byte IP address hiding in its ipi_addr field.
It appears, that the closest to recvmsg() thing there exists in the net package is net.IPConn.ReadMsgIP, and its documentation states that
The packages golang.org/x/net/ipv4 and golang.org/x/net/ipv6 can be used to manipulate IP-level socket options in oob.
Hence, looks like a way forward.
I'd also make the following points explicit (they are not obvious from the text above):
It seems, the net package of the Go's stdlib does not have a standard and easy-to-use way to have what you want.
It appears that the approach to getting the destination address of a datagram when receiving them on a wildcard address is not really standardized, and hence implementations vary between different kernels.
While it looks that net.IPConn.ReadMsgIP wraps recvmsg(2), I'd first verify that in the source code of the Go standard library. Pay attention to the fact that the stdlib contains code for all platforms it supports, so make sure you understand what build constraints are.
https://godoc.org/golang.org/x/net/ may help. And so do the syscall package and https://godoc.org/golang.org/x/sys — if the stock one falls short.
Thanks response of kostix very much.
And according to IP_PKTINFO prompt, I found the following code can resolve my ipv4 issue directly
https://gist.github.com/omribahumi/5da8517497042152691e
But for ipv6, the result is still NOT excepted
package main
import (
"bytes"
"encoding/binary"
"fmt"
"net"
"syscall"
)
func main() {
serverAddr, _ := net.ResolveUDPAddr("udp", ":9999")
sConn, _ := net.ListenUDP("udp", serverAddr)
file, _ := sConn.File()
syscall.SetsockoptInt(int(file.Fd()), syscall.IPPROTO_IPV6, syscall.IP_PKTINFO, 1)
data := make([]byte, 1024)
oob := make([]byte, 2048)
sConn.ReadMsgUDP(data, oob)
oob_buffer := bytes.NewBuffer(oob)
msg := syscall.Cmsghdr{}
binary.Read(oob_buffer, binary.LittleEndian, &msg)
if msg.Level == syscall.IPPROTO_IPV6 && msg.Type == syscall.IP_PKTINFO {
packet_info := syscall.Inet6Pktinfo{}
binary.Read(oob_buffer, binary.LittleEndian, &packet_info)
fmt.Println(packet_info)
// the ipv6 address is not my wanted result
fmt.Println(packet_info.Addr)
}
}
The result as follows
root#test-VirtualBox:/home/test/mygo/src/udp# go run s2.go
{[64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] 0}
[64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
the tcpdump sniffer as follows
22:20:47.009712 IP6 ::1.43305 > ::1.1025: UDP, length 6
I resolve this ipv6 issue by Setting the source IP for a UDP socket via set ipv6 options
err = syscall.SetsockoptInt(int(fd), syscall.IPPROTO_IPV6, syscall.IPV6_RECVPKTINFO, 1)
thanks everyone
How can I get the start and end for v4 and v6 ip addresses of a CIDR? I don't care about addresses which are between.
I have checked the net library and parseCIDR does not return this information. Is there an idiomatic way to calculate the ip range of a CIDR which will work for v6 and v4 addresses alike?
For example, given the CIDR 2001:db8:a0b:12f0::1/32 I would expect
2001:0db8:0000:0000:0000:0000:0000:0000 and 2001:0db8:ffff:ffff:ffff:ffff:ffff:ffff returned as the start and end addresses respectively.
Using the IPAddress Go library, this code will do it for either IPv4 or IPv6. Disclaimer: I am the project manager.
import (
"fmt"
"github.com/seancfoley/ipaddress-go/ipaddr"
)
func main() {
ipRange("2001:db8:a0b:12f0::1/32")
ipRange("1.2.3.1/16")
}
func ipRange(cidr string) {
block := ipaddr.NewIPAddressString(cidr).GetAddress().ToPrefixBlock()
addr := block.WithoutPrefixLen()
fmt.Printf("addr %s block %s lower %s upper %s\n",
cidr, block, addr.GetLower(), addr.GetUpper())
}
Output:
addr 2001:db8:a0b:12f0::1/32 block 2001:db8::/32 lower 2001:db8:: upper 2001:db8:ffff:ffff:ffff:ffff:ffff:ffff
addr 1.2.3.1/16 block 1.2.0.0/16 lower 1.2.0.0 upper 1.2.255.255
There are many things I don't understand about ipv6 and networking in general, which is why I need some further clarification on some the answers already posted to other questions. I'll list my questions, what I grasped from other answers, and what I'm still confused about.
Say I have a VPS with a /56 ipv6 subnet (256 * residential /64 subnets) allotted to it. How can I programmatically find the range (prefix?) of the ip's I "own".
How to get IPv4 and IPv6 address of local machine?. This is the answer I saw for this question: and what I think I understand is that I get the DNS hostname for the machine, then look up that same hostname to find the range. I'm wondering two things: How do I do this in Go, and
How do I transfer this range ^ into a slice (array) of ipv6 addresses. For this specific use case: the ideal solution would be to only get one ipv6 address per \64 subnet, resulting in 256 seperate ips
DNS is not very helpful in determining the local IP addresses, because a DNS entry is not required to make an IP address work, nor is it required to point to (only) the machine that you happen to run your program on.
Instead, inspect the network interfaces and their configuration:
package main
import (
"fmt"
"log"
"net"
"os"
"text/tabwriter"
)
func main() {
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
ifaces, err := net.Interfaces()
if err != nil {
log.Fatal(err)
}
for _, iface := range ifaces {
addrs, err := iface.Addrs()
if err != nil {
log.Fatal(err)
}
for _, addr := range addrs {
addr, ok := addr.(*net.IPNet)
if !ok {
// Not an IP interface
continue
}
if addr.IP.To4() != nil {
// Skip IPv4 addresses
continue
}
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\n",
iface.Name, addr.String(), addr.IP, addr.Mask)
}
}
tw.Flush()
}
For my local machine the output is:
lo ::1/128 ::1 ffffffffffffffffffffffffffffffff
enp2s0 fe80::52e5:49ff:fe3b:107a/64 fe80::52e5:49ff:fe3b:107a ffffffffffffffff0000000000000000
docker0 fe80::42:afff:fedb:7389/64 fe80::42:afff:fedb:7389 ffffffffffffffff0000000000000000
tun0 fe80::f22c:2d3b:a5a0:1b61/64 fe80::f22c:2d3b:a5a0:1b61 ffffffffffffffff0000000000000000
vethd176f0c fe80::1cc1:65ff:fe39:feff/64 fe80::1cc1:65ff:fe39:feff ffffffffffffffff0000000000000000
Note that these addresses are not necessarily reachable from the Internet. This all depends on how the routing of the hoster works. In any kind of cloud setup, you are almost always better off querying the providers APIs.
To list all /64 subnets in a particular /56 subnet, you have to leave the 56 upper bits of the subnet address as they are and permute the following 64-56 = 8 bits (which happens to be the eigth byte):
package main
import (
"fmt"
"net"
)
func main() {
_, subnet, _ := net.ParseCIDR("2001:db8::/56")
fmt.Println(subnet)
subnet.Mask = net.CIDRMask(64, 128) // change mask to /64
for i := 0; i <= 0xff; i++ {
subnet.IP[7] = byte(i) // permute the 8th byte
fmt.Println("\t", subnet)
}
// Output:
// 2001:db8::/56
// 2001:db8::/64
// 2001:db8:0:1::/64
// 2001:db8:0:2::/64
// 2001:db8:0:3::/64
// 2001:db8:0:4::/64
// 2001:db8:0:5::/64
// 2001:db8:0:6::/64
// 2001:db8:0:7::/64
// 2001:db8:0:8::/64
// [...]
}
Regarding the second part of your question, "How do I transfer this range ^ into a slice (array) of ipv6 addresses"
The IPAddress Go library can do this with polymorphic code that works with both IPv4 and IPv6 addresses and all prefix lengths. Repository here. Disclaimer: I am the project manager.
addrStr := "2001:db8::/56"
addr := ipaddr.NewIPAddressString(addrStr).GetAddress()
addrAdjusted := addr.SetPrefixLen(64) // adjust prefix
iterator := addrAdjusted.PrefixIterator()
var blocks []*ipaddr.IPAddress
for iterator.HasNext() {
blocks = append(blocks, iterator.Next())
}
// print the details
fmt.Println("first and last blocks are",
addrAdjusted.GetLower().ToPrefixBlock(), "and",
addrAdjusted.GetUpper().ToPrefixBlock())
fmt.Print("list: ")
for i, addr := range blocks {
if i < 3 || len(blocks)-i <= 3 {
if i > 0 {
fmt.Print(", ")
}
fmt.Print(addr)
} else if i == 3 {
fmt.Print(", ...")
}
}
Output:
first and last blocks are 2001:db8::/64 and 2001:db8:0:ff::/64
list: 2001:db8::/64, 2001:db8:0:1::/64, 2001:db8:0:2::/64, ..., 2001:db8:0:fd::/64, 2001:db8:0:fe::/64, 2001:db8:0:ff::/64
When I am starting the docker daemon I am modifying the dns server so that the containers have a modified /etc/resolv.conf. Looking at the usage message I see:
$ docker --help
Usage: docker [OPTIONS] COMMAND [arg...]
A self-sufficient runtime for linux containers.
Options:
--api-enable-cors=false Enable CORS headers in the remote API
-b, --bridge="" Attach containers to a prexisting network bridge
use 'none' to disable container networking
--bip="" Use this CIDR notation address for the network bridge's IP, not compatible with -b
-D, --debug=false Enable debug mode
-d, --daemon=false Enable daemon mode
--dns=[] Force Docker to use specific DNS servers
--dns-search=[] Force Docker to use specific DNS search domains
-e, --exec-driver="native" Force the Docker runtime to use a specific exec driver
... etc ...
The --dns is what I want to pass, it shows a 'list' with the [], which after much trial and error I finally got this to work:
--dns 127.0.0.1 --dns 8.8.8.8
which deposits :
nameserver 127.0.0.1
nameserver 8.8.8.8
in to the /etc/resolv.conf file.
Is this the correct way to provide a list to docker (and presumably any go) program?
This is a way of passing multiple arguments to a program in Go but certainly not the only way. This is accomplished by defining a type that implements the Value interface. The flag package on flag.Parse() iterates though the argument list matching the name to a registered Value and calling the Set(string) function on the Value. You can use this to append each value of a given name to a slice.
type numList []int
func (l *numList) String() string {
return "[]"
}
func (l *numList) Set(value string) error {
number, err := strconv.Atoi(value)
if err != nil {
return fmt.Errorf("Unable to parse number from value \"%s\"", value)
}
*l = append(*l, number)
return nil
}
This new type can be registered as a flag variable. In the following example the application takes n num command line arguments that are converted to an integer and added to a list.
var numbers numList
func main() {
flag.Var(&numbers, "num", "A number to add to the summation"
flag.Parse()
sum := 0
for _, num := range numbers {
sum += num
}
fmt.Printf("The sum of your flag arguments is %d.\n", sum)
}
This could have been easily done with a string flag and have users pass a delimited list. There is no established convention within the Go language and each application can provide whatever implementation of best fit.