I've found that function in the golang's source code and want to know whether it's truly a perfect hash function or not.
Is it the correct way to test that?
package main
import (
"fmt"
"strconv"
"unsafe"
)
//go:linkname strhash runtime.strhash
func strhash(p unsafe.Pointer, h uintptr) uintptr
const seed = 666
func main() {
m := make(map[uintptr]string)
for i := 0; i < 1000000000; i++ {
key := strconv.Itoa(i)
hash := strhash(unsafe.Pointer(&key), seed)
_, exist := m[hash]
if exist {
fmt.Println("collision")
break
}
m[hash] = key
}
fmt.Println("finish")
}
As far as I know/can tell, it is not. It uses the AES instructions to create the hash. You might want to check out something like https://github.com/cespare/mph.
Related
I'm creating a structure where a developer can store a reference to something and retrieve it when needed using a reference key, but not delete the reference.
How to create a unique key for a map that is generated at the point of insertion?
So far I'm using a not exported pointer to an empty *struct{} , but wondering if there there is a better way.
package main
import "fmt"
type referenceKey **struct{}
type reference[K referenceKey, R any] struct {
s map[K]R
}
func (ref *reference[K, R]) Set(reference R) *K {
if ref.s == nil {
ref.s = make(map[K]R)
}
key := new(struct{})
refKey := K(&key)
ref.s[refKey] = reference
return &refKey
}
func (ref *reference[K, R]) Get(key *K) R {
return ref.s[*key]
}
func main() {
ref := &reference[referenceKey, int]{}
key1 := ref.Set(77)
key2 := ref.Set(15345351)
fmt.Println(ref.Get(key2))
fmt.Println(ref.Get(key1))
}
https://go.dev/play/p/SF6S5BNlP7N
EDIT:
With int I have to keep a reference to it and increment it. To reduce code I tried to use the pointer to a instance of a empty struct.
Basically I need to create a unique key that can't be created outside internal package nor can Get method be called with an invalid key.
I've also fixed the code to now allow anyone to create a new invalid key.
Based on the comments I would make it with a counter and protect the store with a mutex for concurrent use:
package main
import (
"fmt"
"sync"
)
type reference[T any] struct {
mutex sync.RWMutex
store map[uint64]T
counter uint64
}
func (r *reference[T]) Set(item T) uint64 {
r.mutex.Lock()
defer r.mutex.Unlock()
if r.store == nil {
r.store = make(map[uint64]T)
}
r.counter++
r.store[r.counter] = item
return r.counter
}
func (r *reference[T]) Get(key uint64) T {
r.mutex.RLock()
defer r.mutex.RUnlock()
return r.store[key]
}
func main() {
ref := reference[int]{}
key1 := ref.Set(77)
key2 := ref.Set(15345351)
fmt.Println(ref.Get(key2))
fmt.Println(ref.Get(key1))
}
But if in the future you need to be secure, and the keys should be random, then change the map's key to string, remove the counter and use uuid.NewString() for the new key value.
package main
import (
"fmt"
"encoding/json"
"reflect"
)
func someFunc( data interface{}, out interface{} ) {
v := reflect.ValueOf(out).Elem();
fmt.Printf("Incoming type: %s\n",reflect.ValueOf(v).Kind())
v.SetCap(reflect.ValueOf(data).Len())
}
func main() {
expected := []int{1,2,3}
jsonRaw, _ := json.Marshal(expected)
var tmpData interface{}
json.Unmarshal(jsonRaw, &tmpData)
fmt.Printf("%s\n",string(jsonRaw))
fmt.Printf("%#v\n",tmpData)
result := []int{}
var tmp interface{}
tmp = result
fmt.Printf("Outcoming type: %s\n",reflect.TypeOf(&tmp))
someFunc(tmpData,&tmp)
}
I would like to operate on v parameter inside someFunc as if it were
a slice, i.e. "Incoming type"-debug message should output slice.
However, it outputs struct, as is shown here.
The ultimate goal is that I use reflection to analyze the data-parameter's contents and recover everything into out, but for now I would like to
know how to make sure the correct type of v is detected,
so that I can use it as a slice.
EDIT: It appears to be impossible (as of 2013 at least): https://groups.google.com/forum/#!topic/golang-nuts/bldM9tIL-JM
to set the size of a slice for stuff discovered at runtime.
One of the authors says something to the effect that "you have to be able
to sort the elements, i.e. implement Less()" for the values...
EDIT: In any case, I did try to use MakeSlice in this Playgound link,
and it says reflect.MakeSlice of non-slice type
EDIT: I apologize and thank you all for your comments.
What I ended up doing is the following (after an illuminating read of the source code of MakeSlice):
package main
import (
"fmt"
"encoding/json"
"reflect"
)
func someFunc( data interface{}, out interface{} ) {
v := reflect.ValueOf(out).Elem();
fmt.Printf("Incoming type: %s\n",v.Kind())
//v.SetCap(reflect.ValueOf(data).Len()) <-- doesn't work
n := reflect.ValueOf(data).Len()
s := reflect.MakeSlice(reflect.TypeOf(data),n,n)
fmt.Printf("Len= %d\n",s.Len())
}
func main() {
expected := []int{1,2,3}
jsonRaw, _ := json.Marshal(expected)
var tmpData interface{}
json.Unmarshal(jsonRaw, &tmpData)
fmt.Printf("%s\n",string(jsonRaw))
fmt.Printf("%#v\n",tmpData)
result := []int{}
someFunc(tmpData,&result)
}
What I ended up doing is the following (after an illuminating read of the source code of MakeSlice):
package main
import (
"fmt"
"encoding/json"
"reflect"
)
func someFunc( data interface{}, out interface{} ) {
v := reflect.ValueOf(out).Elem();
fmt.Printf("Incoming type: %s\n",v.Kind())
//v.SetCap(reflect.ValueOf(data).Len()) <-- doesn't work
n := reflect.ValueOf(data).Len()
s := reflect.MakeSlice(reflect.TypeOf(data),n,n)
fmt.Printf("Len= %d\n",s.Len())
}
func main() {
expected := []int{1,2,3}
jsonRaw, _ := json.Marshal(expected)
var tmpData interface{}
json.Unmarshal(jsonRaw, &tmpData)
fmt.Printf("%s\n",string(jsonRaw))
fmt.Printf("%#v\n",tmpData)
result := []int{}
someFunc(tmpData,&result)
}
It appears that there are convenience functions such as SliceOf, too.
the bottomline is that the first argument of MakeSlice is not the type
of the arguments that the slice holds, but the slice-type, e.g. []int rather than int.
I'm using golang to call a Dll function like char* fn(), the dll is not written by myself and I cannot change it. Here's my code:
package main
import (
"fmt"
"syscall"
"unsafe"
)
func main() {
dll := syscall.MustLoadDLL("my.dll")
fn := dll.MustFindProc("fn")
r, _, _ := fn.Call()
p := (*byte)(unsafe.Pointer(r))
// define a slice to fill with the p string
data := make([]byte, 0)
// loop until find '\0'
for *p != 0 {
data = append(data, *p) // append 1 byte
r += unsafe.Sizeof(byte(0)) // move r to next byte
p = (*byte)(unsafe.Pointer(r)) // get the byte value
}
name := string(data) // convert to Golang string
fmt.Println(name)
}
I have some questions:
Is there any better way of doing this? There're hundred of dll functions like this, I'll have to write the loop for all functions.
For very-long-string like 100k+ bytes, will append() cause performance issue?
Solved. the unsafe.Pointer(r) causes linter govet shows warning possible misuse of unsafe.Pointer, but the code runs fine, how to avoid this warning? Solution: This can be solved by adding -unsafeptr=false to govet command line, for vim-ale, add let g:ale_go_govet_options = '-unsafeptr=false'.
Casting uintptr as upointer is haram.
You must read the rules:
https://golang.org/pkg/unsafe/#Pointer
But there's hacky way, that shouldn't produce warning:
//go:linkname gostringn runtime.gostringn
func gostringn(p uintptr, l int) string
//go:linkname findnull runtime.findnull
//go:nosplit
func findnull(s uintptr) int
// ....
name := gostringn(r, findnull(r))
Functions takes pointer, but we link them from runtime as uintptr because they have same sizeof.
Might work in theory. But is also frowned upon.
Getting back to your code, as JimB said, you could do it one line with:
name := C.GoString((*C.char)(unsafe.Pointer(r)))
I got the following solution by tracking the os.Args of the go source code, But I am based on go1.17. If you are in another version, you can read the source code to solve it.
func UintPtrToString(r uintptr) string {
p := (*uint16)(unsafe.Pointer(r))
if p == nil {
return ""
}
n, end, add := 0, unsafe.Pointer(p), unsafe.Sizeof(*p)
for *(*uint16)(end) != 0 {
end = unsafe.Add(end, add)
n++
}
return string(utf16.Decode(unsafe.Slice(p, n)))
}
Does Go allow functions to add array length constraints to the signature, or would length still require a runtime check?
For arrays it is more than possible, it is required. For slices it is impossible.
package main
import (
"fmt"
)
func main() {
d := [2]int{1, 2}
fmt.Println(sum(d))
}
func sum(data [2]int) int {
return data[0] + data[1]
}
https://play.golang.org/p/-VMxyDvwUt
For example, I want to do something like this,
package main
import (
"fmt"
"reflect"
)
func main() {
var f func(int) int
v := reflect.ValueOf(f)
fmt.Println(v.ReturnType() == reflect.TypeOf(1)) // => true
}
ReturnType method doesn't exist in reflect package.
My question is, can I implement such a function without using cgo.
Rather than using reflect.ValueOf(f) try reflect.TypeOf(f). The Type type has two relevant methods named NumOut and Out(int). To get the types of all return values in order, you can use the following loop
typ := reflect.TypeOf(f)
for i := 0; i < typ.NumOut(); i++ {
returnType := typ.Out(i)
// do something with returnType
}
If you're certain your function only has one return value, you can always get it with Out(0), the standard disclaimers about making sure your input is correct apply.