slice bounds out of range when using unsafe.Pointer - go

I am faced up with a curious panic when I trim a byte array using syntax like b[1:2] which is converted from []byte to string and then back to []byte.
My go version is go1.7.3 darwin/amd64. What belows is the detail code.
package main
import (
"reflect"
"unsafe"
"fmt"
)
func BytesToString(b []byte) string {
bh := (*reflect.SliceHeader)(unsafe.Pointer(&b))
sh := reflect.StringHeader{bh.Data, bh.Len}
return *(*string)(unsafe.Pointer(&sh))
}
func StringToBytes(s string) []byte {
sh := (*reflect.StringHeader)(unsafe.Pointer(&s))
bh := reflect.SliceHeader{sh.Data, sh.Len, 0}
return *(*[]byte)(unsafe.Pointer(&bh))
}
func main() {
b := []byte{'b', 'y', 't', 'e'}
// No1 here you can trim []byte using b[1:2]
_ = b[1:2]
fmt.Println("No1")
// convert []byte to string
s := BytesToString(b)
// convert string to []byte
b = StringToBytes(s)
// create new []byte variant using content of b
bb := make([]byte, len(b))
for i := 0; i < len(b); i++ {
bb[i] = b[i]
}
// No2 here you also can trim []byte using bb[1:2]
_ = bb[1:2]
fmt.Println("No2")
// No3 here you can not trim []byte. I don't know why. why?
_ = b[1:2]
fmt.Println("No3")
}
Run this code, and get error as follows:
No1
No2
panic: runtime error: slice bounds out of range
goroutine 1 [running]:
panic(0x8f060, 0xc42000a100)
/usr/local/Cellar/go/1.7.3/libexec/src/runtime/panic.go:500 +0x1a1
main.main()
/tmp/unsafe.go:45 +0x274
exit status 2
I'm curious about what caused this panic?

The capacity of the slice created by StringToBytes is zero.
The for loop does not panic because index expressions check len(b).
The expression b[1:2] panics because slice expressions check cap(b).
One fix is to set capacity to the length of the string:
func StringToBytes(s string) []byte {
sh := (*reflect.StringHeader)(unsafe.Pointer(&s))
bh := reflect.SliceHeader{sh.Data, sh.Len, sh.Len}
return *(*[]byte)(unsafe.Pointer(&bh))
}

Related

How to convert []byte to C hex format 0x...?

func main() {
str := hex.EncodeToString([]byte("go"))
fmt.Println(str)
}
this code return 676f. How I can print C-like 0x67, 0x6f ?
I couldn't find any function in the hex module that would achieve what you want. However, we can use a custom buffer to write in our desired format.
package main
import (
"bytes"
"fmt"
)
func main() {
originalBytes := []byte("go")
result := make([]byte, 4*len(originalBytes))
buff := bytes.NewBuffer(result)
for _, b := range originalBytes {
fmt.Fprintf(buff, "0x%02x ", b)
}
fmt.Println(buff.String())
}
Runnable example: https://goplay.space/#fyhDJ094GgZ
Here's a solution that produces the result as specified in the question. Specifically, there's a ", " between each byte and no trailing space.
p := []byte("go")
var buf strings.Builder
if len(p) > 0 {
buf.Grow(len(p)*6 - 2)
for i, b := range p {
if i > 0 {
buf.WriteString(", ")
}
fmt.Fprintf(&buf, "0x%02x", b)
}
}
result := buf.String()
The strings.Builder type is used to avoid allocating memory on the final conversion to a string. Another answer uses bytes.Buffer that does allocate memory at this step.
The the builder is initially sized large enough to hold the representation of each byte and the separators. Another answer ignores the size of the separators.
Try this on the Go playground.

Convert binary value as string to uint32 in Golang

Hello i am trying to convert 00000000000000000000000000001011 to uint32 in golang using
var v = "00000000000000000000000000001011"
fmt.Printf("%T\n", v)
c := []byte(v)
u := binary.LittleEndian.Uint32(c)
However it is not working.
You can't use encoding/binary for this, as that is to serialize and deserialize the (memory) bytes of different values (e.g. numbers). What you have is the base 2 string representation of the number.
To get its integer value you have to parse it. For that, use strconv.ParseUint():
s := "00000000000000000000000000001011"
u, err := strconv.ParseUint(s, 2, 32)
if err != nil {
panic(err)
}
fmt.Println(u)
This outputs (try it on the Go Playground):
11
Note that strconv.ParseUint() returns a value of type uint64, so if you need uint32, you have to manually convert it, e.g.:
u32 := uint32(u)
There are more options for parsing numbers from strings, for an overview, check Convert string to integer type in Go?
For example,
package main
import (
"fmt"
"strconv"
)
func main() {
s := "00000000000000000000000000001011"
fmt.Println(s)
u64, err := strconv.ParseUint(s, 2, 32)
u32 := uint32(u64)
if err == nil {
fmt.Println(u32)
}
}
Playground: https://play.golang.org/p/yiicgWsb7B_M
Output:
00000000000000000000000000001011
11

Serialize struct fields to pre existing slice of bytes

I have a setup where I receive data over the network and serialize it to my struct. It works fine, but now I need to serialize the data to a slice buffer to send it across the network.
I am trying to avoid having to allocate more than needed so I have already set up a buffer which I like to write to for all my serializing. But am not sure how to do this.
My setup is like this:
recieveBuffer := make([]byte, 1500)
header := recieveBuffer[0:1]
message := recieveBuffer[1:]
So I am trying to write fields from a struct to message and the total number of bytes for all the fields as a value for header.
This was how I deserialized to the struct:
// Deserialize ...
func (userSession *UserSession) Deserialize(message []byte) {
userSession.UID = int64(binary.LittleEndian.Uint32(message[0:4]))
userSession.UUID = string(message[4:40])
userSession.Username = string(message[40:])
}
I don't really know how to do the reverse of this, however. Is it possible without creating buffers for each field I want to serialize before copying to message?
Given the preallocated buffer buf, you can reverse the process like this:
buf[0] = byte(40+len(userSession.Username))
binary.LittleEndian.PutUint32(buf[1:], uint32(int32(userSession.UID)))
copy(buf[5:41], userSession.UUID)
copy(buf[41:], userSession.Username)
Given two helper functions.
One to encode a primitive to a byte slice:
func EncodeNumber2NetworkOrder(v interface{}) ([]byte, error) {
switch v.(type) {
case int: // int is at least 32 bits
b := make([]byte, 4)
binary.BigEndian.PutUint32(b, uint32(v.(int)))
return b, nil
case int8:
b := []byte{byte(v.(int8))}
return b, nil
// ... truncated
and one to convert primitive, non-byte slices to a byte slice
func EncodeBigEndian(in []float64) []byte {
var out []byte = make([]byte, len(in)*8)
var wg sync.WaitGroup
wg.Add(len(in))
for i := 0; i < len(in); i++ {
go func(out *[]byte, i int, f float64) {
defer wg.Done()
binary.BigEndian.PutUint64((*out)[(i<<3):], math.Float64bits(f))
}(&out, i, in[i])
}
wg.Wait()
return out
}
your binary serialization might look like this for a bogus struct like
type Foo struct {
time int64
data []float64
}
func Encode(f *Foo) []byte {
da := encoder.EncodeBigEndian(f.data)
bytes := make([]byte,0)
bytes = append(bytes, da...)
return bytes
}

How do you set a value to a pointer containing nil using reflection

I'm trying to set a value to a nil pointer in a struct like so.
// https://play.golang.org/p/jPTMNC_ZQ9
package main
import (
"fmt"
"reflect"
)
type T struct {
A *int
}
func main() {
fmt.Println("Hello, playground")
t := &T{}
v := 1
vptr := &v
CopyValue(vptr, t.A) // I want to set t.A to contain 1
}
func CopyValue(src interface{}, dest interface{}) {
srcRef := reflect.ValueOf(src)
if srcRef.Kind() == reflect.Ptr {
srcRef = srcRef.Elem()
}
destRef := reflect.New(srcRef.Type()).Elem()
destRef.Set(srcRef)
reflect.ValueOf(dest).Elem().Set(destRef)
}
However, I encounter the following error:
panic: reflect: call of reflect.Value.Set on zero Value
goroutine 1 [running]:
reflect.flag.mustBeAssignable(0x0, 0x1040a128)
/usr/local/go/src/reflect/value.go:221 +0x260
reflect.Value.Set(0x0, 0x0, 0x0, 0xdefc0, 0x1040a128, 0x182)
/usr/local/go/src/reflect/value.go:1339 +0x40
main.CopyValue(0xd7860, 0x1040a124, 0xd7860, 0x0)
/tmp/sandbox487854080/main.go:30 +0x1a0
main.main()
/tmp/sandbox487854080/main.go:19 +0x100
What am I doing wrong?
In order to be able to modify what t.A points to, you need to send a reference to it to your CopyValue function.
CopyValue(vptr, &t.A) // (note the &)
You can then assign the pointer to the new address:
func CopyValue(src interface{}, dest interface{}) {
srcRef := reflect.ValueOf(src)
vp := reflect.ValueOf(dest)
vp.Elem().Set(srcRef)
}
See the 3rd "law of reflection" here: https://blog.golang.org/laws-of-reflection
Full working code:
package main
import (
"fmt"
"reflect"
)
type T struct {
A *int
}
func main() {
t := &T{}
v := 1
vptr := &v
CopyValue(vptr, &t.A) // we pass a reference to t.A since we want to modify it
fmt.Printf("%v\n", *t.A)
}
func CopyValue(src interface{}, dest interface{}) {
srcRef := reflect.ValueOf(src)
vp := reflect.ValueOf(dest)
vp.Elem().Set(srcRef)
}
reflect.ValueOf(dest).Elem().Set(destRef)
If you look into this line, reflect.ValueOf(dest) will give you nil, since you passed in a nil pointer. Calling .Elem() on this is invalid, since there is no element to the nil pointer.
t.A is a nil pointer when you pass it in, so CopyValue is being asked to copy a value into an invalid (nil) location. You need to allocate space for an int for it to point to, and then CopyValue will be able to do the copy into the location pointed at.
This resolves the error and allows the value to be copied:
t := &T{}
t.A = new(int) // Add this line

Is there an equivalent of os.Args() for functions?

To help debug GO programs, I want to write two generic functions that will be called on entry and exit, which will print the values of input and output parameters respectively:
printInputParameters(input ...interface{})
printOutputParameters(output ...interface{})
Is there an equivalent of os.Args() for functions? I looked at runtime package and didn't find such functions.
For example lets say I have two functions with different input parameters and output parameters
func f1(int i, float f) (e error) {
... some code here
}
func f2(s string, b []byte) (u uint64, e error) {
.. some code here
}
I want to be able to do the following
func f1(int i, float f) (e error) {
printInputparameters( ? )
defer func() {
printOutputParameters( ? )
}()
... some code here
}
func f2(s string, b []byte) (u uint64, e error) {
printInputparameters( ? )
defer func() {
printOutputParameters( ? )
}()
... some code here
}
You cannot do this in Go since there is no way you can get the stack frame of the currently active function in the current goroutine. It is not impossible to do this as I'll show further below but the problem is that there is no public API to get this done reliably. That it can be done can be seen in the stack traces printed when a panic is raised: all values on the stack are dumped in that case.
Should you be interested in how the stack trace is actually generated then have a look at genstacktrace in the runtime package.
As for a solution to your problem, you can the source code parsing route as already suggested. If you feel adventurous, you can parse the stack trace provided by runtime.Stack. But beware, there are so many drawbacks that you will quickly realize that any solution is better than this one.
To parse the stack trace, just get the line of the previously called function (from the viewpoint of printInputParameters), get the name of that function and parse the parameter values according to the parameter types provided by reflection. Some examples of stack trace outputs of various function invocations:
main.Test1(0x2) // Test1(int64(2))
main.Test1(0xc820043ed5, 0x3, 0x3) // Test1([]byte{'A','B','C'})
main.Test1(0x513350, 0x4) // Test1("AAAA")
You can see that complex types (those which do not fit into a register) may use more than one 'parameter'. A string for example is a pointer to the data and the length. So you have to use the unsafe package to access these pointers and reflection to create values from this data.
If you want to try yourself, here's some example code:
import (
"fmt"
"math"
"reflect"
"runtime"
"strconv"
"strings"
"unsafe"
)
// Parses the second call's parameters in a stack trace of the form:
//
// goroutine 1 [running]:
// main.printInputs(0x4c4c60, 0x539038)
// /.../go/src/debug/main.go:16 +0xe0
// main.Test1(0x2)
// /.../go/src/debug/main.go:23
//
func parseParams(st string) (string, []uintptr) {
line := 1
start, stop := 0, 0
for i, c := range st {
if c == '\n' {
line++
}
if line == 4 && c == '\n' {
start = i + 1
}
if line == 5 && c == '\n' {
stop = i
}
}
call := st[start:stop]
fname := call[0:strings.IndexByte(call, '(')]
param := call[strings.IndexByte(call, '(')+1 : strings.IndexByte(call, ')')]
params := strings.Split(param, ", ")
parsedParams := make([]uintptr, len(params))
for i := range params {
iv, err := strconv.ParseInt(params[i], 0, 64)
if err != nil {
panic(err.Error())
}
parsedParams[i] = uintptr(iv)
}
return fname, parsedParams
}
func fromAddress(t reflect.Type, addr uintptr) reflect.Value {
return reflect.NewAt(t, unsafe.Pointer(&addr)).Elem()
}
func printInputs(fn interface{}) {
v := reflect.ValueOf(fn)
vt := v.Type()
b := make([]byte, 500)
if v.Kind() != reflect.Func {
return
}
runtime.Stack(b, false)
name, params := parseParams(string(b))
pidx := 0
fmt.Print(name + "(")
for i := 0; i < vt.NumIn(); i++ {
t := vt.In(i)
switch t.Kind() {
case reflect.Int64:
case reflect.Int:
// Just use the value from the stack
fmt.Print(params[pidx], ",")
pidx++
case reflect.Float64:
fmt.Print(math.Float64frombits(uint64(params[pidx])), ",")
pidx++
case reflect.Slice:
// create []T pointing to slice content
data := reflect.ArrayOf(int(params[pidx+2]), t.Elem())
svp := reflect.NewAt(data, unsafe.Pointer(params[pidx]))
fmt.Printf("%v,", svp.Elem())
pidx += 3
case reflect.String:
sv := fromAddress(t, params[pidx])
fmt.Printf("%v,", sv)
pidx += 2
case reflect.Map:
// points to hmap struct
mv := fromAddress(t,params[pidx])
fmt.Printf("%v,", mv)
pidx++
} /* switch */
}
fmt.Println(")")
}
Test:
func Test1(in int, b []byte, in2 int, m string) {
printInputs(Test1)
}
func main() {
b := []byte{'A', 'B', 'C'}
s := "AAAA"
Test1(2, b, 9, s)
}
Output:
main.Test1(2,[65 66 67],9,"AAAA",)
A slightly advanced version of this can be found on github:
go get github.com/githubnemo/pdump
To generically print your functions' arguments, you can do this:
func printInputParameters(input ...interface{}) {
fmt.Printf("Args: %v", input)
}
printInputParameters is a variadic function, and input is of type []interface{}.

Resources