I've found that function in the golang's source code and want to know whether it's truly a perfect hash function or not.
Is it the correct way to test that?
package main
import (
"fmt"
"strconv"
"unsafe"
)
//go:linkname strhash runtime.strhash
func strhash(p unsafe.Pointer, h uintptr) uintptr
const seed = 666
func main() {
m := make(map[uintptr]string)
for i := 0; i < 1000000000; i++ {
key := strconv.Itoa(i)
hash := strhash(unsafe.Pointer(&key), seed)
_, exist := m[hash]
if exist {
fmt.Println("collision")
break
}
m[hash] = key
}
fmt.Println("finish")
}
As far as I know/can tell, it is not. It uses the AES instructions to create the hash. You might want to check out something like https://github.com/cespare/mph.
I got this code off some blog, I want to know why the reassignment of a particular index in the slice b is denoted using a 2d indexing, example
b[0][0] = byte('*')
Isn't it supposed to be
b[0]=byte('*')
package main
import (
"bytes"
"fmt"
)
func splitDemo() {
fmt.Println("Split demo")
a := []byte("a,b,c")
b := bytes.Split(a, []byte(","))
b[0][0] = byte('*')
fmt.Printf("%q",b)
fmt.Printf("%q",a)
}
func main() {
splitDemo()
}
bytes.Split returns a [][]byte. That's a slice of []byte.
They seem to be the same:
package main
import "fmt"
type S struct {
i int
}
func main() {
var s1 *S = new(S)
fmt.Println(s1)
var s2 *S = &S{}
fmt.Println(s2) // Prints the same thing.
}
Update:
Hm. I just realized that there's no obvious way to initialize S.i using new. Is there a way to do that? new(S{i:1}) does not seem to work :/
From Effective Go:
As a limiting case, if a composite literal contains no fields at all, it creates a zero value for the type. The expressions new(File) and &File{} are equivalent.
Not only do they give the same resulting value, but if we allocate something both ways and look at their values...
// Adapted from http://tour.golang.org/#30
package main
import "fmt"
type Vertex struct {
X, Y int
}
func main() {
v := &Vertex{}
v2 := new(Vertex)
fmt.Printf("%p %p", v, v2)
}
...we'll see that they are in fact allocated in consecutive memory slots. Typical output: 0x10328100 0x10328108. I'm not sure if this is an implementation detail or part of the specification, but it does demonstrate that they're both being allocated from the same pool.
Play around with the code here.
As for initializing with new, according to the language spec: The built-in function new takes a type T and returns a value of type *T. The memory [pointed to] is initialized as described in the section on initial values. Because functions in go can't be overloaded, and this isn't a variadic function, there's no way to pass in any initialization data. Instead, go will initialize it with whatever version of 0 makes sense for the type and any member fields, as appropriate.
Case 1: package main
import (
"fmt"
)
type Drink struct {
Name string
Flavour string
}
func main() {
a := new(Drink)
a.Name = "Maaza"
a.Flavour = "Mango"
b := a
fmt.Println(&a)
fmt.Println(&b)
b.Name = "Frooti"
fmt.Println(a.Name)
}//This will output Frooti for a.Name, even though the addresses for a and b are different.
Case 2:
package main
import (
"fmt"
)
type Drink struct {
Name string
Flavour string
}
func main() {
a := Drink{
Name: "Maaza",
Flavour: "Mango",
}
b := a
fmt.Println(&a)
fmt.Println(&b)
b.Name = "Froti"
fmt.Println(a.Name)
}//This will output Maaza for a.Name. To get Frooti in this case assign b:=&a.
in the main package i have:
var foo C.int
foo = 3
t := fastergo.Ctuner_new()
fastergo.Ctuner_register_parameter(t, &foo, 0, 100, 1)
in the fastergo package i have:
func Ctuner_register_parameter(tuner unsafe.Pointer, parameter *C.int, from C.int, to C.int, step C.int) C.int {
...
}
if i try to run it, i get:
demo.go:14[/tmp/go-build742221968/command-line-arguments/_obj/demo.cgo1.go:21]: cannot use &foo (type *_Ctype_int) as type *fastergo._Ctype_int in function argument
i am not really sure what go is trying to tell me here, but somehow i think it wants to tell me, that all C.int are not equal? why is this the case? how can i solve this / work around?
Since _Ctype_int doesn't begin with a Unicode upper case letter, the type is local to the package. Use Go types, except in the C wrapper package where you convert them to C types. The wrapper package should hide all the implementation details.
You don't provide sufficient information for us to create sample code which compiles and runs. Here's a rough outline of what I expected to see:
package main
import "tuner"
func main() {
var foo int
foo = 3
t := tuner.New()
t.RegisterParameter(&foo, 0, 100, 1)
}
.
package tuner
import (
"unsafe"
)
/*
#include "ctuner.h"
*/
import "C"
type Tuner struct {
ctuner uintptr
}
func New() *Tuner {
var t Tuner
t.ctuner = uintptr(unsafe.Pointer(C.ctuner_new()))
return &t
}
func (t *Tuner) RegisterParameter(parameter *int, from, to, step int) error {
var rv C.int
rv = C.ctuner_register_parameter(
(*C.ctuner)(unsafe.Pointer(t.ctuner)),
(*C.int)(unsafe.Pointer(parameter)),
C.int(from),
C.int(to),
C.int(step),
)
if rv != 0 {
// handle error
}
return nil
}
As explained by peterSO, you can't pass C.int between packages. However, you can pass pointers between packages by converting the pointer type. To do this, you would define a named type in the target package, import that type into the calling package and covert via unsafe.Pointer. There isn't any point in doing this with a single int.
However, it is helpful if you keep code to convert complex types in a package; for example an array of strings (or any sort of nested array).
The example below is for exporting a go function to be called in C, but this works in reverse, ie. if you want to call a C functions which a returns nested array.
package convert
import "C"
type PP_char **C.char
func From_c_to_go(arr_str PP_char, length int) []string {
// Some operation on the Ctype
var slice []string
for _, s := range unsafe.Slice(arr_str, length) {
if s == nil {
break
}
x := C.GoString(s)
slice = append(slice, x)
}
return slice
}
package main
import "C"
import "convert"
//export myFunc
func myFunc(arr_str **C.char, length int){
retyped_arr_str := convert.PP_char(unsafe.Pointer(arr_str))
slice := convert.From_c_to_go(retyped_arr_str, length)
// Do something with slice
}
You could instead decide to pass instance of unsafe.Pointer as an argument to the go function in the target package and perform the type conversion in that function.
I get a byte slice ([]byte) from a UDP socket and want to treat it as an integer slice ([]int32) without changing the underlying array, and vice versa. In C(++) I would just cast between pointer types; how would I do this in Go?
As others have said, casting the pointer is considered bad form in Go. Here are examples of the proper Go way and the equivalent of the C array casting.
WARNING: all code untested.
The Right Way
In this example, we are using the encoding/binary package to convert each set of 4 bytes into an int32. This is better because we are specifying the endianness. We are also not using the unsafe package to break the type system.
import "encoding/binary"
const SIZEOF_INT32 = 4 // bytes
data := make([]int32, len(raw)/SIZEOF_INT32)
for i := range data {
// assuming little endian
data[i] = int32(binary.LittleEndian.Uint32(raw[i*SIZEOF_INT32:(i+1)*SIZEOF_INT32]))
}
The Wrong Way (C array casting)
In this example, we are telling Go to ignore the type system. This is not a good idea because it may fail in another implementation of Go. It is assuming things not in the language specification. However, this one does not do a full copy. This code uses unsafe to access the "SliceHeader" which is common in all slices. The header contains a pointer to the data (C array), the length, and the capacity. Instead of just converting the header to the new slice type, we first need to change the length and capacity since there are less elements if we treat the bytes as a new type.
import (
"reflect"
"unsafe"
)
const SIZEOF_INT32 = 4 // bytes
// Get the slice header
header := *(*reflect.SliceHeader)(unsafe.Pointer(&raw))
// The length and capacity of the slice are different.
header.Len /= SIZEOF_INT32
header.Cap /= SIZEOF_INT32
// Convert slice header to an []int32
data := *(*[]int32)(unsafe.Pointer(&header))
You do what you do in C, with one exception - Go does not allow to convert from one pointer type to another. Well, it does, but you must use unsafe.Pointer to tell compiler that you are aware that all rules are broken and you know what you are doing. Here is an example:
package main
import (
"fmt"
"unsafe"
)
func main() {
b := []byte{1, 0, 0, 0, 2, 0, 0, 0}
// step by step
pb := &b[0] // to pointer to the first byte of b
up := unsafe.Pointer(pb) // to *special* unsafe.Pointer, it can be converted to any pointer
pi := (*[2]uint32)(up) // to pointer to the first uint32 of array of 2 uint32s
i := (*pi)[:] // creates slice to our array of 2 uint32s (optional step)
fmt.Printf("b=%v i=%v\n", b, i)
// all in one go
p := (*[2]uint32)(unsafe.Pointer(&b[0]))
fmt.Printf("b=%v p=%v\n", b, p)
}
Obviously, you should be careful about using "unsafe" package, because Go compiler is not holding your hand anymore - for example, you could write pi := (*[3]uint32)(up) here and compiler wouldn't complain, but you would be in trouble.
Also, as other people pointed already, bytes of uint32 might be layout differently on different computers, so you should not assume these are layout as you need them to be.
So safest approach would be to read your array of bytes one by one and make whatever you need out of them.
Alex
The short answer is you can't. Go wont let you cast a slice of one type to a slice of another type. You will have loop through the array and create another array of the type you want while casting each item in the array. This is generally regarded as a good thing since typesafety is an important feature of go.
Since Go 1.17, there is a simpler way to do this using the unsafe package.
import (
"unsafe"
)
const SIZEOF_INT32 = unsafe.Sizeof(int32(0)) // 4 bytes
func main() {
var bs []byte
// Do stuff with `bs`. Maybe do some checks ensuring that len(bs) % SIZEOF_INT32 == 0
data := unsafe.Slice((*int32)(unsafe.Pointer(&bs[0])), len(bs)/SIZEOF_INT32)
// A more verbose alternative requiring `import "reflect"`
// data := unsafe.Slice((*int32)(unsafe.Pointer((*reflect.SliceHeader)(unsafe.Pointer(&bs)).Data)), len(bs)/SIZEOF_INT32)
}
Go 1.17 and beyond
Go 1.17 introduced the unsafe.Slice function, which does exactly this.
Converting a []byte to a []int32:
package main
import (
"fmt"
"unsafe"
)
func main() {
theBytes := []byte{
0x33, 0x44, 0x55, 0x66,
0x11, 0x22, 0x33, 0x44,
0x77, 0x66, 0x55, 0x44,
}
numInts := uintptr(len(theBytes)) * unsafe.Sizeof(theBytes[0]) / unsafe.Sizeof(int32(0))
theInts := unsafe.Slice((*int32)(unsafe.Pointer(&theBytes[0])), numInts)
for _, n := range theInts {
fmt.Printf("%04x\n", n)
}
}
Playground.
I had the size unknown problem and tweaked the previous unsafe method with the following code.
given a byte slice b ...
int32 slice is (*(*[]int)(Pointer(&b)))[:len(b)/4]
The array to slice example may be given a fictional large constant and the slice bounds used in the same way since no array is allocated.
You can do it with the "unsafe" package
package main
import (
"fmt"
"unsafe"
)
func main() {
var b [8]byte = [8]byte{1, 2, 3, 4, 5, 6, 7, 8}
var s *[4]uint16 = (*[4]uint16)(unsafe.Pointer(&b))
var i *[2]uint32 = (*[2]uint32)(unsafe.Pointer(&b))
var l *uint64 = (*uint64)(unsafe.Pointer(&b))
fmt.Println(b)
fmt.Printf("%04x, %04x, %04x, %04x\n", s[0], s[1], s[2], s[3])
fmt.Printf("%08x, %08x\n", i[0], i[1])
fmt.Printf("%016x\n", *l)
}
/*
* example run:
* $ go run /tmp/test.go
* [1 2 3 4 5 6 7 8]
* 0201, 0403, 0605, 0807
* 04030201, 08070605
* 0807060504030201
*/
Perhaps it was not available when the earlier answers were given, but it would seem that the binary.Read method would be a better answer than "the right way" given above.
This method allows you to read binary data from a reader directly into the value or buffer of your desired type. You can do this by creating a reader over your byte array buffer. Or, if you have control of the code that is giving you the byte array, you can replace it to read directly into your buffer without the need for the interim byte array.
See https://golang.org/pkg/encoding/binary/#Read for the documentation and a nice little example.
http://play.golang.org/p/w1m5Cs-ecz
package main
import (
"fmt"
"strings"
)
func main() {
s := []interface{}{"foo", "bar", "baz"}
b := make([]string, len(s))
for i, v := range s {
b[i] = v.(string)
}
fmt.Println(strings.Join(b, ", "))
}
func crackU32s2Bytes(us []uint32) []byte {
var bs []byte
var ptrBs = (*reflect.SliceHeader)(unsafe.Pointer(&bs))
var ptrUs = (*reflect.SliceHeader)(unsafe.Pointer(&us))
ptrBs.Data = ptrUs.Data
ptrBs.Len = ptrUs.Len*4
ptrBs.Cap = ptrBs.Len
return bs
}
func crackBytes2U32s(bs []byte) []uint32 {
var us []uint32
var ptrBs = (*reflect.SliceHeader)(unsafe.Pointer(&bs))
var ptrUs = (*reflect.SliceHeader)(unsafe.Pointer(&us))
ptrUs.Data = ptrBs.Data
ptrUs.Len = ptrBs.Len/4
ptrUs.Cap = ptrUs.Len
return us
}