Generics type inference when cascaded calls of generic functions - go

While building a (my first) generics heavy library, and I'm bumping on some apparent limitations on the generics type checking implementation -- more likely my lack of knowledge.
Any ideas how to get something like below to work ?
package main
import (
"fmt"
"reflect"
)
type Number interface {
int | float32
}
type MultiDimensionSlice interface {
int | float32 | []int | []float32 | [][]int | [][]float32
}
func dimensions[S MultiDimensionSlice](s S) int {
dims := 0
t := reflect.TypeOf(s)
for t.Kind() == reflect.Slice {
dims += 1
t = t.Elem()
}
return dims
}
func indirection[T Number](v T) int {
var slice2D [][]T
return dimensions(slice2D)
}
func main() {
x := [][]float32{{1}, {2}, {3}}
fmt.Printf("x=%v, dims=%d\n", x, dimensions(x))
fmt.Printf("indirection should return 2, got %d\n", indirection(0))
}
This fails to compile with the message [][]T does not implement MultiDimensionSlice ([][]T missing in int | float32 | []int | []float32 | [][]int | [][]float32)
But within the function indirection() all the allowed values of T will have an implementation in dimensions().
Any help or pointers would be mostly appreciated!
(Playground link)
ps.: My problem is a bit more complex than that, but the issue is that one generic function (indirection() in this example) is not able to invoke the other (dimensions() here) because (apparently) Go compiler is not able to resolve the type parameter constraints (the information is there in compile time...).

As mentioned in the comments go has some restrictions in its generics perspective. You can achieve what you require with a workaround.
First, you need to change the interface you define. (Make it generic too)
type Number interface {
int | float32
}
type MultiDimensionSlice[T Number] interface {
Number | []T | [][]T
}
Then we need to change dimension methods type arguments. It would be much cleaner if go would let us define dimensions method like this
func dimensions[S Number](s MultiDimensionSlice[S]) int {
But all we can do is :
func dimensions[S Number, K MultiDimensionSlice[S]](s K) int {
dims := 0
t := reflect.TypeOf(s)
for t.Kind() == reflect.Slice {
dims += 1
t = t.Elem()
}
return dims
}
And then we need to change how we call dimensions method. We need to provide an extra type argument in order to go can infer type argument S
func indirection[T Number](v T) int {
var slice2D [][]T
return dimensions[T](slice2D)
}
func main() {
x := [][]float32{{1}, {2}, {3}}
fmt.Printf("x=%v, dims=%d\n", x, dimensions[float32](x))
fmt.Printf("indirection should return 2, got %d\n", indirection(0))
}
Playground

Related

Generics range over union of maps

There is a simple example of use of generics in which we want to copy a map
package main
import "fmt"
type myMap interface {
map[string]int | map[string]float64
}
func copyMap[T myMap](m T) T {
newMap := make(T)
for key, elem := range m {
newMap[key] = elem
}
return newMap
}
func main() {
m := map[string]int{"seven": 7}
fmt.Println(copyMap(m))
}
demo here
This code fails to compile returning error
./prog.go:12:17: invalid argument: cannot make T: no core type
./prog.go:13:25: cannot range over m (variable of type T constrained by myMap) (T has no core type)
./prog.go:14:18: invalid operation: cannot index m (variable of type T constrained by myMap)
How can I circumvent this issue and have a working generic copyMap function working for types map[string]int and map[string]float64?
Do
func copyMap[T ~map[string]V, V any](m T) T {/* ... */}
demo
Or indeed just use https://cs.opensource.google/go/x/exp/+/062eb4c6:maps/maps.go;l=65 (who uses a similar construct) as #jubObs mentioned.

go generics: processing different struct types with same data member types [duplicate]

This question already has answers here:
How can I access a struct field with generics (type T has no field or method)?
(1 answer)
Generic function to work on different structs with common members from external package?
(1 answer)
Closed 3 months ago.
There are two struct types, Foo and Bar, with an int data member val. I am trying to write a generic function that can handle both types. I tried the following and this did not work.
package main
import "fmt"
type Foo struct {
val int
}
type Bar struct {
val int
}
func Add[T any](slice []T) int {
var sum int
for _, elem := range slice {
sum += elem.val
}
return sum
}
func Test() {
f1 := Foo{val: 2}
f2 := Foo{val: 2}
fslice := []Foo{f1, f2}
fsum := Add(fslice)
fmt.Printf("fsum = %d\n", fsum)
b1 := Bar{val: 3}
b2 := Bar{val: 3}
bslice := []Bar{b1, b2}
bsum := Add(bslice)
fmt.Printf("bsum = %d\n", bsum)
}
func main() {
Test()
}
The compiler throws the following error.
$ go run generics1.go
# command-line-arguments
./generics1.go:16:15: elem.val undefined (type T has no field or method val)
Go playground link: https://go.dev/play/p/mdOMH3xuwu7
What could be a possible way to approach this?
Per golang 1.18 release note
The Go compiler does not support accessing a struct field x.f where x is of type parameter type even if all types in the type parameter's type set have a field f. We may remove this restriction in a future release.
You could define one GetVal() interface method to retrieve the val, and use this method as part of type constraint of generic.
Sample codes
type Foo struct {
val int
}
func (f Foo) GetVal() int {
return f.val
}
type Bar struct {
val int
}
func (b Bar) GetVal() int {
return b.val
}
type MyType interface {
Foo | Bar
GetVal() int
}
func Add[T MyType](slice []T) int {
var sum int
for _, elem := range slice {
sum += elem.GetVal()
}
return sum
}
https://go.dev/play/p/0eJZpqy7q8f

struct type as map key [duplicate]

This question already has an answer here:
golang how can I use struct name as map key
(1 answer)
Closed 9 months ago.
We have a following function:
func (h *Handler) Handle(message interface{}) error {
//here there is a switch for different messages
switch m := message.(type) {
}
}
This signature is given and can't be changed. There are around 20 different message types the handler processes.
Now, there are some of these messages (around 4) which need special post-processing. In a different package.
Thus, I am thinking to do this like this:
func (h *Handler) Handle(message interface{}) error {
//here there is a switch for different messages
switch m := message.(type) {
}
//only post-process if original message processing succeeds
postProcessorPkg.Process(message)
}
Now, in the Process function, I want to quickly lookup if the message type is indeed of the ones we need postprocessing for. I don't want to do a switch again here. There are many handlers, in different packages, with varying amount of message types, and it should be generic.
So I was thinking of registering the message type in the postprocessor and then just do a lookup:
func (p *Postprocessor) Register(msgtype interface{}) {
registeredTypes[msgtype] = msgtype
}
and then
func (p *Postprocessor) Process(msgtype interface{}) error {
if ok := registeredTypes[msgtype]; !ok {
return errors.New("Unsupported message type")
}
prop := GetProp(registeredTypes[msgtype])
doSmthWithProp(prop)
}
This will all not work now because I can only "register" instances of the message, not the message type itself, as far as I know. Thus the map would only match a specific instance of a message, not its type, which is what I need.
So I guess this needs redesign. I can completely ditch the registering and the map lookup, but
I can't change the Handle function to a specific type (signature will need to remain message interface{}
I would like to avoid to have to use reflect, just because I will have a hard time defending such a solution with some colleagues.
As there is no possibility to set a type as the map key, I finally decided to implement the following solution, which is based on #Chrono Kitsune 's solution:
type Postprocess interface {
NeedsPostprocess() bool
}
type MsgWithPostProcess struct {}
func (p *MsgWithPostProcess) NeedsPostprocess() bool {
return true
}
type Msg1 struct {
MsgWithPostProcess
//other stuff
}
type Msg2 struct {
MsgWithPostProcess
//other stuff
}
type Msg3 struct {
//no postprocessing needed
}
func (p *Postprocessor) Process(msgtype interface{}) error {
if _, ok := msgtype.(Postprocess); ok {
//do postprocessing
}
}
As of my simple test I did, only Msg1 and Msg2 will be postprocessed, but not Msg3, which is what I wanted.
This question was the first hit I found on Google but the title is somewhat misleading. So I'll leave this here to add some food for thought with the title of the question in mind.
First, the issue with maps is that its key must be a comparable value. This is why for example a slice cannot be used is a map key. A slice is not comparable and is therefore not allowed. You can use an array (fixed sized slice) but not a slice for the same reason.
Second, you have in the reflect.TypeOf(...).String()a way to get a canonical string representation for types. Though it is not unambiguous unless you include the package path, as you can see here.
package main
import (
"fmt"
s2 "go/scanner"
"reflect"
s1 "text/scanner"
)
type X struct{}
func main() {
fmt.Println(reflect.TypeOf(1).String())
fmt.Println(reflect.TypeOf(X{}).String())
fmt.Println(reflect.TypeOf(&X{}).String())
fmt.Println(reflect.TypeOf(s1.Scanner{}).String())
fmt.Println(reflect.TypeOf(s2.Scanner{}).String())
fmt.Println(reflect.TypeOf(s1.Scanner{}).PkgPath(), reflect.TypeOf(s1.Scanner{}).String())
fmt.Println(reflect.TypeOf(s2.Scanner{}).PkgPath(), reflect.TypeOf(s2.Scanner{}).String())
}
int
main.X
*main.X
scanner.Scanner
scanner.Scanner
text/scanner scanner.Scanner
go/scanner scanner.Scanner
https://play.golang.org/p/NLODZNdik6r
With this information, you can (if you feel so inclined) create a map which let's go from a reflect.Type to a key and back again, like this.
package main
import (
"fmt"
s2 "go/scanner"
"reflect"
s1 "text/scanner"
)
type TypeMap struct {
m []reflect.Type
}
func (m *TypeMap) Get(t reflect.Type) int {
for i, x := range m.m {
if x == t {
return i
}
}
m.m = append(m.m, t)
return len(m.m) - 1
}
func (m *TypeMap) Reverse(t int) reflect.Type {
return m.m[t]
}
type X struct{}
func main() {
var m TypeMap
fmt.Println(m.Get(reflect.TypeOf(1)))
fmt.Println(m.Reverse(0))
fmt.Println(m.Get(reflect.TypeOf(1)))
fmt.Println(m.Reverse(0))
fmt.Println(m.Get(reflect.TypeOf(1)))
fmt.Println(m.Reverse(0))
fmt.Println(m.Get(reflect.TypeOf(X{})))
fmt.Println(m.Reverse(1))
fmt.Println(m.Get(reflect.TypeOf(&X{})))
fmt.Println(m.Reverse(2))
fmt.Println(m.Get(reflect.TypeOf(s1.Scanner{})))
fmt.Println(m.Reverse(3).PkgPath(), m.Reverse(3))
fmt.Println(m.Get(reflect.TypeOf(s2.Scanner{})))
fmt.Println(m.Reverse(4).PkgPath(), m.Reverse(4))
}
0
int
0
int
0
int
1
main.X
2
*main.X
3
text/scanner scanner.Scanner
4
go/scanner scanner.Scanner
In the above case I'm assuming that N is small. Also note the use of the identity of reflect.TypeOf, it will return the same pointer for the same type on subsequent calls.
If N is not small, you may want to do something a bit more complex.
package main
import (
"fmt"
s2 "go/scanner"
"reflect"
s1 "text/scanner"
)
type PkgPathNum struct {
PkgPath string
Num int
}
type TypeMap struct {
m map[string][]PkgPathNum
r []reflect.Type
}
func (m *TypeMap) Get(t reflect.Type) int {
k := t.String()
xs := m.m[k]
pkgPath := t.PkgPath()
for _, x := range xs {
if x.PkgPath == pkgPath {
return x.Num
}
}
n := len(m.r)
m.r = append(m.r, t)
xs = append(xs, PkgPathNum{pkgPath, n})
if m.m == nil {
m.m = make(map[string][]PkgPathNum)
}
m.m[k] = xs
return n
}
func (m *TypeMap) Reverse(t int) reflect.Type {
return m.r[t]
}
type X struct{}
func main() {
var m TypeMap
fmt.Println(m.Get(reflect.TypeOf(1)))
fmt.Println(m.Reverse(0))
fmt.Println(m.Get(reflect.TypeOf(X{})))
fmt.Println(m.Reverse(1))
fmt.Println(m.Get(reflect.TypeOf(&X{})))
fmt.Println(m.Reverse(2))
fmt.Println(m.Get(reflect.TypeOf(s1.Scanner{})))
fmt.Println(m.Reverse(3).PkgPath(), m.Reverse(3))
fmt.Println(m.Get(reflect.TypeOf(s2.Scanner{})))
fmt.Println(m.Reverse(4).PkgPath(), m.Reverse(4))
}
0
int
1
main.X
2
*main.X
3
text/scanner scanner.Scanner
4
go/scanner scanner.Scanner
https://play.golang.org/p/2fiMZ8qCQtY
Note the subtitles of pointer to type, that, X and *X actually are different types.

Golang: convert struct to embedded at offset 0 struct

I have some different structs like Big with Small embedded at offset 0.
How can I access Small's structure fields from code, that doesn't know anything about Big type, but it is known that Small is at offset 0?
type Small struct {
val int
}
type Big struct {
Small
bigval int
}
var v interface{} = Big{}
// here i only know about 'Small' struct and i know that it is at the begining of variable
v.(Small).val // compile error
It seems that compiler is theoretically able to operate such expression, because it knows that Big type has Small type embedded at offset 0. Is there any way to do such things (maybe with unsafe.Pointer)?
While answer with reflection is working but it has performance penalties and is not idiomatic to Go.
I believe you should use interface. Like this
https://play.golang.org/p/OG1MPHjDlQ
package main
import (
"fmt"
)
type MySmall interface {
SmallVal() int
}
type Small struct {
val int
}
func (v Small) SmallVal() int {
return v.val
}
type Big struct {
Small
bigval int
}
func main() {
var v interface{} = Big{Small{val: 3}, 4}
fmt.Printf("Small val: %v", v.(MySmall).SmallVal())
}
Output:
Small val: 3
Avoid using unsafe whenever possible. The above task can be done using reflection (reflect package):
var v interface{} = Big{Small{1}, 2}
rf := reflect.ValueOf(v)
s := rf.FieldByName("Small").Interface()
fmt.Printf("%#v\n", s)
fmt.Printf("%#v\n", s.(Small).val)
Output (try it on the Go Playground):
main.Small{val:1}
1
Notes:
This works for any field, not just the first one (at "offset 0"). This also works for named fields too, not just for embedded fields. This doesn't work for unexported fields though.
type Small struct {
val int
}
type Big struct {
Small
bigval int
}
func main() {
var v = Big{Small{10},200}
print(v.val)
}

How to check variable type at runtime in Go language

I have few C functions declared like this
CURLcode curl_wrapper_easy_setopt_long(CURL* curl, CURLoption option, long param);
CURLcode curl_wrapper_easy_setopt_str(CURL* curl, CURLoption option, char* param);
I would like to expose those as one Go function like this
func (e *Easy)SetOption(option Option, param interface{})
so I need to be able to check param type at runtime. How do I do that and is this good idea (if not what is good practice in this case)?
It seems that Go have special form of switch dedicate to this (it is called type switch):
func (e *Easy)SetOption(option Option, param interface{}) {
switch v := param.(type) {
default:
fmt.Printf("unexpected type %T", v)
case uint64:
e.code = Code(C.curl_wrapper_easy_setopt_long(e.curl, C.CURLoption(option), C.long(v)))
case string:
e.code = Code(C.curl_wrapper_easy_setopt_str(e.curl, C.CURLoption(option), C.CString(v)))
}
}
The answer by #Darius is the most idiomatic (and probably more performant) method. One limitation is that the type you are checking has to be of type interface{}. If you use a concrete type it will fail.
An alternative way to determine the type of something at run-time, including concrete types, is to use the Go reflect package. Chaining TypeOf(x).Kind() together you can get a reflect.Kind value which is a uint type: http://golang.org/pkg/reflect/#Kind
You can then do checks for types outside of a switch block, like so:
import (
"fmt"
"reflect"
)
// ....
x := 42
y := float32(43.3)
z := "hello"
xt := reflect.TypeOf(x).Kind()
yt := reflect.TypeOf(y).Kind()
zt := reflect.TypeOf(z).Kind()
fmt.Printf("%T: %s\n", xt, xt)
fmt.Printf("%T: %s\n", yt, yt)
fmt.Printf("%T: %s\n", zt, zt)
if xt == reflect.Int {
println(">> x is int")
}
if yt == reflect.Float32 {
println(">> y is float32")
}
if zt == reflect.String {
println(">> z is string")
}
Which prints outs:
reflect.Kind: int
reflect.Kind: float32
reflect.Kind: string
>> x is int
>> y is float32
>> z is string
Again, this is probably not the preferred way to do it, but it's good to know alternative options.
quux00's answer only tells about comparing basic types.
If you need to compare types you defined, you shouldn't use reflect.TypeOf(xxx). Instead, use reflect.TypeOf(xxx).Kind().
There are two categories of types:
direct types (the types you defined directly)
basic types (int, float64, struct, ...)
Here is a full example:
type MyFloat float64
type Vertex struct {
X, Y float64
}
type EmptyInterface interface {}
type Abser interface {
Abs() float64
}
func (v Vertex) Abs() float64 {
return math.Sqrt(v.X*v.X + v.Y*v.Y)
}
func (f MyFloat) Abs() float64 {
return math.Abs(float64(f))
}
var ia, ib Abser
ia = Vertex{1, 2}
ib = MyFloat(1)
fmt.Println(reflect.TypeOf(ia))
fmt.Println(reflect.TypeOf(ia).Kind())
fmt.Println(reflect.TypeOf(ib))
fmt.Println(reflect.TypeOf(ib).Kind())
if reflect.TypeOf(ia) != reflect.TypeOf(ib) {
fmt.Println("Not equal typeOf")
}
if reflect.TypeOf(ia).Kind() != reflect.TypeOf(ib).Kind() {
fmt.Println("Not equal kind")
}
ib = Vertex{3, 4}
if reflect.TypeOf(ia) == reflect.TypeOf(ib) {
fmt.Println("Equal typeOf")
}
if reflect.TypeOf(ia).Kind() == reflect.TypeOf(ib).Kind() {
fmt.Println("Equal kind")
}
The output would be:
main.Vertex
struct
main.MyFloat
float64
Not equal typeOf
Not equal kind
Equal typeOf
Equal kind
As you can see, reflect.TypeOf(xxx) returns the direct types which you might want to use, while reflect.TypeOf(xxx).Kind() returns the basic types.
Here's the conclusion. If you need to compare with basic types, use reflect.TypeOf(xxx).Kind(); and if you need to compare with self-defined types, use reflect.TypeOf(xxx).
if reflect.TypeOf(ia) == reflect.TypeOf(Vertex{}) {
fmt.Println("self-defined")
} else if reflect.TypeOf(ia).Kind() == reflect.Float64 {
fmt.Println("basic types")
}
See type assertions here:
http://golang.org/ref/spec#Type_assertions
I'd assert a sensible type (string, uint64) etc only and keep it as loose as possible, performing a conversion to the native type last.
func (e *Easy)SetOption(option Option, param interface{}) {
if s, ok := param.(string); ok {
// s is string here
}
// else...
}
What's wrong with
func (e *Easy)SetStringOption(option Option, param string)
func (e *Easy)SetLongOption(option Option, param long)
and so on?

Resources