Hashing reflect.Type - go

I'm trying to find a quick way of performing comparisons between two []reflect.Type. Right now I have the following:
func Equal(left, right []reflect.Type) bool {
if len(left) != len(right) {
return false
}
for i := 0; i < len(left); i++ {
if left[i] != right[i] {
return false
}
}
return true
}
Most of the slices don't change. So if I can find a way to hash them, I'd get a huge perf boost.
Background
I'm trying to (for fun) implement a form of function overloading in Go using the reflect package. The first thing I did is to convert each specialised/overloaded function into a signature type.
type Signature struct {
Variadic bool
In, Out []reflect.Type
}
The idea is that, when the overloaded function gets called, I'll convert the arguments into a slice of reflect.Type and then find a Signature where the In types match.
This works, but for each comparison, it's a linear scan which is pretty slow. If I could hash the slice of []reflect.Type I could stick that in a map and get constant time lookups.

I ended up abusing the built-in map to assign unique ids to each reflect.Type. Then I hash those using djb2.
type TypeCode struct {
seq int64
codes map[reflect.Type]int64
}
func (td *TypeCode) TypeID(t reflect.Type) int64 {
if code, ok := td.codes[t]; ok {
return code
}
td.seq++
td.codes[t] = td.seq
return td.seq
}
func (td *TypeCode) SliceTypeID(tt []reflect.Type) int64 {
id := int64(5381)
for _, t := range tt {
id = ((id << 5) + id) + td.TypeID(t)
}
return id
}
edit: I switched to a string based approach which is less efficient, but removes any potential for collisions.
type TypeCode struct {
seq int64
codes map[reflect.Type]string
}
func (td *TypeCode) TypeID(t reflect.Type) string {
if code, ok := td.codes[t]; ok {
return code
}
td.seq++
id := strconv.FormatInt(td.seq, 10)
td.codes[t] = id
return id
}
func (td *TypeCode) SliceTypeID(tt []reflect.Type) string {
ids := make([]string, len(tt))
for i := 0; i < len(tt); i++ {
ids[i] = td.TypeID(tt[i])
}
return strings.Join(ids, ".")
}

Related

Basic + Slice + Map Type Compatible Generics?

Is there a way to create a generic function that can adjust its operation when passed a map or a slice type vs a basic type?
Goal
Create a slice reading function generator with a flexible return type:
func ValueReader[T <probably something fancy>](i int) func ([]ProtoConvertable) T {
return func (row []ProtoConvertable) T {
return ...
}
}
row := []ProtoConvertable{
&Data[int]{Value: 333},
&ListData{Values: []ProtoConvertable{
&Data[string]{Value: "hello"},
&Data[string]{Value: "world"},
}},
&MapData{Values: map[ProtoConvertable]ProtoConvertable{
&Data[int]{Value: 22}: &Data[string]{Value: "world"},
&Data[int]{Value: 11}: &Data[string]{Value: "hello"},
}},
}
dataReader := ValueReader[int](0) // A function that converts the first element to an int
listDataReader := ValueReader[[]string](1) // A function that converts the second element to a slice
mapDataReader := ValueReader[map[int]string](2) // A function that converts the third element to a map
data := dataReader(row) // 333
listData := listDataReader(row) // []string{"hello", "world"}
mapData := mapDataReader(row) // map[int]string{11: "hello", 22: "world"}
Types
type ValueType interface {
int | string
}
type ProtoConvertable interface {
ToProto() *pb.GenericMessage
}
type Data[T ValueType] struct {
Value T
}
func (d *Data) ToProto() *pb.GenericMessage{
...
}
type ListData struct {
Values []ProtoConvertable
}
func (d *ListData) ToProto() *pb.GenericMessage {
...
}
type MapData struct {
Values map[ProtoConvertable]ProtoConvertable
}
func (d *MapData) ToProto() *pb.GenericMessage {
...
}
Current Solution
func ValueReader[T ValueType](i int) func([]ProtoConvertable) T {
return func(row []ProtoConvertable) T {
return row[i].(*Data[T]).Value
}
}
func ListValueReader[T ValueType](i int) func([]ProtoConvertable) []T {
return func(row []ProtoConvertable) []T {
vs := row[i].(*ListData).Values
res := make([]T, len(vs))
for i, v := range vs {
res[i] = v.(*Data[T]).Value
}
return res
}
}
func MapValueReader[K ValueType, V ValueType](i int) func([]ProtoConvertable) map[K]V {
return func(row []ProtoConvertable) map[K]V {
vs := row[i].(*MapData).Values
res := make(map[K]V, len(vs))
for k, v := range vs {
res[k.(*Data[K]).Value] = v.(*Data[V]).Value
}
return res
}
}
dataReader := ValueReader[int](0)
listDataReader := ListValueReader[string](1)
mapDataReader := MapValueReader[int, string](2)
Note: all of this code is an untested simplification of a more complicated library. It might need some tweaking to get to actually work.
The <probably something fancy> doesn't exist.
The main issue is that you want to model a type parameter that matches a base value and two composite types, one of which is a map type where you want to capture both K and V.
Even if it existed, the body of ValueReader would be a type-switch on T to return each specialized reader function, so your existing solution that involves a small amount of code duplication seems just a better strategy overall.
My advice is to use generics when the operations on the different concrete types of T are really identical. You can read more at: https://go.dev/blog/when-generics

when should I return value instead of modifying receiver pointer?

I have a method for the struct ProofOfWork which should modify the struct members Nonce and Hash. So I wonder whether it should modify these two members of the given instance inside the method Run or should make these two variables as a return.
So here is the method Run with return variables:
// Run performs a proof-of-work
func (pow *ProofOfWork) Run() (int, []byte) {
var hashInt big.Int
var hash [32]byte
nonce := 0
fmt.Printf("Mining the block containing \"%s\"\n", pow.block.Data)
for nonce < maxNonce {
data := pow.prepareData(nonce)
hash = sha256.Sum256(data)
fmt.Printf("\r%x", hash)
hashInt.SetBytes(hash[:])
if hashInt.Cmp(pow.target) == -1 {
break
} else {
nonce++
}
}
fmt.Print("\n\n")
return nonce, hash[:]
}
Then the version without any return variables:
func (pow *ProofOfWork) Run() {
var hashInt big.Int
var hash [32]byte // the type of hash value is defined by result of the sha256 function
nonce := 0
for nonce < MaxNonce {
data := pow.prepareData(nonce)
hash := sha256.Sum256(data)
hashInt.SetBytes(hash[:])
if hashInt.Cmp(pow.target) == -1 {
// the nonce found
break
} else {
nonce++
}
}
pow.block.Hash = hash[:]
pow.block.Nonce = nonce
}
Both options you show might be useful sometimes. May I propose another possibility. In Go we should use functions much more often then in other languages. A plain function might be exactly what you are looking for:
// Run performs a proof-of-work
func Run(pow *ProofOfWork) (int, []byte) {
var hashInt big.Int
var hash [32]byte
nonce := 0
fmt.Printf("Mining the block containing \"%s\"\n", pow.block.Data)
for nonce < maxNonce {
data := pow.prepareData(nonce)
hash = sha256.Sum256(data)
fmt.Printf("\r%x", hash)
hashInt.SetBytes(hash[:])
if hashInt.Cmp(pow.target) == -1 {
break
} else {
nonce++
}
}
fmt.Print("\n\n")
return nonce, hash[:]
}
I would probably make ProofOfWork an interface, and abstract Run that way.

Making code more generic

I have a program where many functionalities are similar across different structures, however, I end up writing these functions again and again, esp because the variable that are being dealt inside are of different structures.
I have written a sample code here.
In Go Playgroud
package main
import "fmt"
func (a *Match) Add(v Match) {
a.Runs += v.Runs
a.Points += v.Points
}
type Match struct {
Runs uint64
Points uint64
}
func (a *Activity) Add(v Activity) {
a.Walk += v.Walk
a.Jog += v.Jog
}
type Activity struct {
Walk uint64
Jog uint64
}
func GetDailyMatches() map[string]Match {
var dailyMatches map[string]Match
Match1, Match2 := Match{5, 10}, Match{1, 2}
dailyMatches = make(map[string]Match)
dailyMatches["01"] = Match1
dailyMatches["02"] = Match2
dailyMatches["03"] = Match1
dailyMatches["04"] = Match2
return dailyMatches
}
func GetDailyActivities() map[string]Activity {
var dailyActivities map[string]Activity
Activity1, Activity2 := Activity{5, 10}, Activity{1, 2}
dailyActivities = make(map[string]Activity)
dailyActivities["01"] = Activity1
dailyActivities["02"] = Activity2
dailyActivities["03"] = Activity1
dailyActivities["04"] = Activity2
return dailyActivities
}
func main() {
fmt.Println(CalculateMatchSummary("01", "03"))
fmt.Println(CalculateActivitySummary("02", "04"))
fmt.Println(CalculateMatchSummary("01", "03"))
fmt.Println(CalculateActivitySummary("02", "04"))
}
func CalculateMatchSummary(start, end string) (total Match) {
dailyMatches := GetDailyMatches()
for day, value := range dailyMatches {
if day < start {
continue
} else if day > end {
continue
} else {
total.Add(value)
}
}
return
}
func CalculateActivitySummary(start, end string) (total Activity) {
dailyActivities := GetDailyActivities()
for day, value := range dailyActivities {
if day < start {
continue
} else if day > end {
continue
} else {
total.Add(value)
}
}
return
}
If you notice, both Match and Activity has the same functions and same structures, except that internally they are of different structures.
Is there a easy way to make the code more generic (Go generics, which is not there in Go??) in Golang itself.
Go has a pretty package "reflect". You can not do genericity strictly speaking but you can get unification of code for the same behavior.
I've changed your playground a bit : https://play.golang.org/p/bfqZsFOgVQ
The main part :
func AddTwo(a, b interface{}) interface{} {
va := reflect.ValueOf(a)
vb := reflect.ValueOf(b)
res := reflect.New(reflect.TypeOf(a)).Elem()
if va.Kind() != reflect.Struct && vb.Kind() != reflect.Struct {
return nil
}
na, nb := va.NumField(), vb.NumField()
if na != nb {
return nil
}
for i := 0; i < na; i++ {
// additional verification needed here
fa := va.Field(i).Uint()
fb := vb.Field(i).Uint()
fr := fa + fb
res.Field(i).SetUint(fr)
}
return res.Interface()
}
I use reflect to check the fields of the struct I am given. If both are uint64, I can add them reflectively. If your structs contains many uint64, it can add them all !
Note that you must convert the resulting interface to the type of the struct given after calling this function. That is why this is not strictly generic, because the returning type is a interface, and not a Match or Activity.
EDIT: No need even to return a new struct. You can simply update the field of the "a" struct by calling .SetUint() method.

3 variable map in Go

I am trying to make a 3 variable map in go so that you can do something like.
var postlist = make(map[int][int]bool)
postlist[postid][userid] = true
if postid[postid][userid] = true {
//do something
}
I have tried to make my own using a struct like
var postlist = make(map[int]cusmap)
type cusmap struct {
userid int
seen bool
}
but then I don't know how to check to check both the userid and seen bool condition.
I am not sure what you are trying to do, but a map is only a key/value. You can't have key/key/value. In order to do this, you need the value to be a map, so you would do:
http://play.golang.org/p/dOAXNAI4CO
package main
func main() {
var postlist = make(map[int]map[int]bool)
postid, userid := 0, 0
postlist[postid] = make(map[int]bool)
postlist[postid][userid] = true
if postlist[postid][userid] == true {
println("ok")
} else {
println("ko")
}
}
If you want to implement a set of int pairs, you can use a structure as the map key:
type postData struct {
userID int
postID int
}
Then make a map with postData keys and bool values:
postSet := map[postData]bool{
postData{1234, 7284}: true,
postData{7777, 1212}: true}
We can exploit the fact that if we give a non-existent index to the map, it will just return a zero-value.
For bool, the zero-value is false.
if postSet[postData{7777, 1212}] {
fmt.Println("post found.")
} else {
fmt.Println("no such post!")
}
Here's a full working example: http://play.golang.org/p/VJw9Vm8gHA
An alternative to #creack's approach is to define funcs on the map itself that way you don't have to manually check every time you want to set / unset something:
func main() {
cm := CustMap{}
pid, uid := 0, 0
cm.Set(pid, uid)
fmt.Println(cm.Exists(pid, uid))
fmt.Println(cm.Exists(pid, 10))
fmt.Println(cm.Exists(10, 10))
}
type CustMap map[int]map[int]struct{} //struct makes the map use 0 bytes for values instead of 1 bytes for bools, handy as the map grows
func (cm CustMap) Set(pid, uid int) {
if _, ok := cm[pid]; !ok {
cm[pid] = make(map[int]struct{})
}
cm[pid][uid] = struct{}{}
}
func (cm CustMap) Exists(pid, uid int) (ok bool) {
if u := cm[pid]; u != nil {
_, ok = u[uid]
}
return
}
playground

Generic variadic argument in Go?

I know that Go doesn't support templates or overloaded functions, but I'm wondering if there's any way to do some kind of generic programming for variadic functions anyway?
I have many functions such as these:
func (this Document) GetString(name string, defaults ...string) string {
v, ok := this.GetValueFromDb(name)
if !ok {
if len(defaults) >= 1 {
return defaults[0]
} else {
return ""
}
}
return v.asString
}
func (this Document) GetInt(name string, defaults ...int) int {
v, ok := this.GetValueFromDb(name)
if !ok {
if len(defaults) >= 1 {
return defaults[0]
} else {
return 0
}
}
return v.asInt
}
// etc. for many different types
Is there any way to do this without having so much redundant code?
The most of what you can achieve is usage of interface{} type, something like this:
func (this Document) Get(name string, defaults ...interface{}) interface{} {
v, ok := this.GetValueFromDb(name)
if !ok {
if len(defaults) >= 1 {
return defaults[0]
} else {
return 0
}
}
return v
}
GetValueFromDb function should also be tweaked to return interface{} value and not some wrapper like now.
Then in the client code you can do the following:
value := document.Get("index", 1).(int) // Panics when the value is not int
or
value, ok := document.Get("index", 1).(int) // ok is false if the value is not int
This will yield some runtime overhead though. I'd better stick with separate functions and try to restructure the code somehow.
Here's a working example of how you could change your code.
package main
import (
"fmt"
)
type Document struct{
getSucceeds bool
}
func (d *Document) GetValueFromDb(name string) (interface{}, bool) {
return 1, d.getSucceeds
}
func (this Document) Get(name string, def ...int) interface{} {
v, ok := this.GetValueFromDb(name)
if !ok {
if len(def) >= 1 {
return def[0]
} else {
return 0
}
}
return v
}
func main() {
d1 := Document{true}
d2 := Document{false}
var int1, int2 int
int1 = d1.Get("foo", 2).(int)
int2 = d2.Get("foo", 2).(int)
fmt.Println(int1, int2)
}
Since you know what type you expect for the given name, you can write your Get method in a generic way, returning interface{}, and then assert the type at the call site. See the spec about type assertions.
There are different ways to emulate some aspects of generics in Go. There were lots of discussions on the mailing list. Often, there's a way to restructure code so it's less dependent on generics.
In the client code you can do like this :
res := GetValue("name", 1, 2, 3)
// or
// res := GetValue("name", "one", "two", "three")
if value, ok := res.(int); ok {
// process int return value
} else if value, ok := res.(string); ok {
// process string return value
}
// or
// res.(type) expression only work in switch statement
// and 'res' variable's type have to be interface type
switch value := res.(type) {
case int:
// process int return value
case string:
// process string return value
}

Resources