I am trying to write the following function:
func Fill[X any](slice []*X){
for i := range slice {
slice[i] = new(X)
}
}
xs := make([]*int, 10) // fill with nils
Fill(xs) // now fill with new(int)
That works fine but… if I want to use a slice of interfaces and provide a concrete type?
func Fill[X, Y any](slice []X){
for i := range slice {
slice[i] = new(Y) // not work!
}
}
xs := make([]sync.Locker, 10) // fill with nils
Fill[sync.Locker,sync.Mutex](xs) // ouch
I try some combinations without success, is there a way or go1.18 does not support such relations?
When you constrain both X and Y to any, you lose all interface-implementor relationship. The only thing that is known at compile time is that X and Y are different types, and you can't assign one to the another within the function body.
A way to make it compile is to use an explicit assertion:
func Fill[X, Y any](slice []X) {
for i := range slice {
slice[i] = any(*new(Y)).(X)
}
}
But this panics if Y doesn't really implement X, as in your case, since it is *sync.Mutex (pointer type) that implements sync.Locker.
Moreover, when Y is instantiated with a pointer type, you lose information about the base type, and therefore the zero value, including *new(Y) would be nil, so you don't really have a baseline improvement over make (just typed nils vs. nil interfaces).
What you would like to do is to constrain Y to X, like Fill[X any, Y X](slice []X) but this is not possible because 1) a type parameter can't be used as a constraint; and/or 2) a constraint can't embed a type parameter directly. It also initializes nils as the above.
A better solution is to use a constructor function instead of a second type parameter:
func main() {
xs := make([]sync.Locker, 10)
Fill(xs, func() sync.Locker { return &sync.Mutex{} })
}
func Fill[X any](slice []X, f func() X) {
for i := range slice {
slice[i] = f()
}
}
Related
I am testing out generics in go 1.18 and took a look at this example.
I would like to recreate that example but instead be able to pass in a slice of int or slice of float instead, and in the function I'll just sum up everything in the slice.
This is when I ran into some issues just iterating the slice. This is what I tried:
package main
import "fmt"
// NumberSlice constraint
type NumberSlice interface {
[]int64 | []float64
}
func add[N NumberSlice](n N) {
// want: to range over n and print value of v
for _, v := range n {
fmt.Println(v)
}
}
func main() {
ints := []int64{1, 2}
add(ints)
}
I got the error:
cannot range over n (variable of type N constrained by NumberSlice) (N has no core type)
How do I accomplish this?
A core type, for an interface (including an interface constraint) is defined as follows:
An interface T has a core type if one of the following conditions is
satisfied:
There is a single type U which is the underlying type of all types in the type set of T
or the type set of T contains only channel types with identical element type E, and all directional channels have the same direction.
Your interface constraint has no core type, because it has two underlying types: []int64 and []float64.
Therefore you can't use it where a core type is required. Notably range and make.
You can change the interface to require the base types, and then specify the slice in the function signature:
// still no core type...
type Number interface {
int64 | float64
}
// ...but the argument will be instantiated with either int64 or float64
func add[N Number](n []N) {
for _, v := range n {
fmt.Println(v)
}
}
This also works, but it's way more verbose:
type NumberSlice[N int64 | float64] interface {
// one core type []N
~[]N
}
func add[S NumberSlice[N], N int64 | float64](n S) {
for _, v := range n {
fmt.Println(v)
}
}
Could something like this work for you?
package main
import "fmt"
type NumberOrFloat interface {
int64 | float64
}
func add[N NumberOrFloat](n []N) {
for _, v := range n {
fmt.Println(v)
}
}
func main() {
ints := []int64{1, 2}
add(ints)
}
The difference here is that you define type constraints on array elements (not on array types): []N
I am try to do a matrix subtraction with unknow size of matrix. here is the code:
type ArrayLike interface {
[]interface{} | [][]interface{} | [][][]interface{} | [][][][]interface{}
}
func subMatrix[T ArrayLike](A, B T, shape []int) interface{} {
dim := shape[0]
if len(shape) == 1 {
retObj := make([]interface{}, dim)
for i := 0; i < dim; i++ {
Av := A[i].(float64)
Bv := B[i].(float64)
retObj[i] = Av - Bv
}
return retObj
} else {
retObj := make([]interface{}, dim)
for i := 0; i < dim; i++ {
retObj[i] = subMatrix(Av[i], Bv[i], shape[1:])
}
return retObj
}
}
It complains about
invalid operation: cannot index A (variable of type T constrained by []interface{}|[][]interface{}|[][][]interface{}|[][][][]interface{})compilerNonIndexableOperand
Does anyone know how to do this job?
You can't do this, not with generics and not with interface{}/any alone. The main issue is that you can't statically model slices with arbitrary dimensions — i.e. arbitrary types. Let's go in order:
The error message you got is because you can't index a type parameter with a union constraint like that. Specs, Index expressions:
For a of type parameter type P:
[...]
The element types of all types in P's type set must be
identical. [...]
The element types of ArrayLike's type set are not identical. The element type of []interface{} is interface{}, the one of [][]interface{} is []interface{} and so on.
There is a placebo solution to the indexing error, outlined here, basically it involves changing the type of the arguments A and B to a slice []T. Then you can index A[i] and B[i].
However this isn't enough. At this point, there's no way to pass correct arguments to the recursive call. The type of the expression A[i] is now T, but subMatrix wants []T.
Even dropping type parameters and declaring the args A and B as any doesn't work, because on the recursive call you still want to index them. In order to index, you have to assert any to something that's indexable, but what would that be? At each recursion the types would have one less dimension, so a statically typed assertion would eventually panic.
The only way to solve this is (probably?) with reflection, but honestly I don't see a practical utility to do that.
You can, and should, write a generic function for each matrix dimension:
import "golang.org/x/exp/constraints"
type Number interface {
constraints.Integer | constraints.Float
}
func Sub2DMatrix[T Number](a, b [][]T) {
for i := range a {
for j := range a[i] {
a[i][j] -= b[i][j]
}
}
}
I am trying to write the following function:
func Fill[X any](slice []*X){
for i := range slice {
slice[i] = new(X)
}
}
xs := make([]*int, 10) // fill with nils
Fill(xs) // now fill with new(int)
That works fine but… if I want to use a slice of interfaces and provide a concrete type?
func Fill[X, Y any](slice []X){
for i := range slice {
slice[i] = new(Y) // not work!
}
}
xs := make([]sync.Locker, 10) // fill with nils
Fill[sync.Locker,sync.Mutex](xs) // ouch
I try some combinations without success, is there a way or go1.18 does not support such relations?
When you constrain both X and Y to any, you lose all interface-implementor relationship. The only thing that is known at compile time is that X and Y are different types, and you can't assign one to the another within the function body.
A way to make it compile is to use an explicit assertion:
func Fill[X, Y any](slice []X) {
for i := range slice {
slice[i] = any(*new(Y)).(X)
}
}
But this panics if Y doesn't really implement X, as in your case, since it is *sync.Mutex (pointer type) that implements sync.Locker.
Moreover, when Y is instantiated with a pointer type, you lose information about the base type, and therefore the zero value, including *new(Y) would be nil, so you don't really have a baseline improvement over make (just typed nils vs. nil interfaces).
What you would like to do is to constrain Y to X, like Fill[X any, Y X](slice []X) but this is not possible because 1) a type parameter can't be used as a constraint; and/or 2) a constraint can't embed a type parameter directly. It also initializes nils as the above.
A better solution is to use a constructor function instead of a second type parameter:
func main() {
xs := make([]sync.Locker, 10)
Fill(xs, func() sync.Locker { return &sync.Mutex{} })
}
func Fill[X any](slice []X, f func() X) {
for i := range slice {
slice[i] = f()
}
}
Consider the experimental package slices. The package is experimental, so I understand the signatures may change; I'm using it to illustrate the issue.
Consider the signatures of two functions from this package, slices.Contains and slices.Grow:
func Contains[E comparable](s []E, v E) bool
func Grow[S ~[]E, E any](s S, n int) S
The first argument to Contains has type []E (slice of Es) with E constrained by comparable (types that are comparable).
The first argument to Grow instead has type S (just S), with S constrained by ~[]E (types whose underlying type is a slice of E)
However it looks like there isn't any practical difference between what operations are allowed inside functions with such type params. If we declare some fake funcs with the same type parameters, we can see that both compile just fine:
As expected, in both functions we can len/cap, append, range, allocate with make, and index with [ ].
func fakeContains[E comparable](s []E, v E) {
fmt.Println(len(s), cap(s))
var e E
fmt.Println(append(s, e))
fmt.Println(make([]E, 4))
for _, x := range s {
fmt.Println(x)
}
fmt.Println(s[0])
fmt.Println(reflect.TypeOf(s).Kind())
}
func fakeGrow[S ~[]E, E any](s S, n int) {
fmt.Println(len(s), cap(s))
var e E
fmt.Println(append(s, e))
fmt.Println(make(S, 4))
for _, x := range s {
fmt.Println(x)
}
fmt.Println(s[0])
fmt.Println(reflect.TypeOf(s).Kind())
}
Even reflect.TypeOf(s).Kind() gives reflect.Slice in all cases.
The functions can also be tested with different types, and all compile:
// compiles just fine
func main() {
type MyUint64 uint64
type MyUint64Slice []uint64
foo := []uint64{0, 1, 2}
fakeContains(foo, 0)
fakeGrow(foo, 5)
bar := []MyUint64{3, 4, 5}
fakeContains(bar, 0)
fakeGrow(bar, 5)
baz := MyUint64Slice{6, 7, 8}
fakeContains(baz, 0)
fakeGrow(baz, 5)
}
The only actual difference in my understanding is that in slices.Grow the argument s S is not a slice. It's just constrained to slice types. And as a matter of fact reflect.TypeOf(s) gives a different output when the arg is an instance of type MyUint64Slice []uint64:
Contains with arg s []E gives reflect.TypeOf(s) -> []uint64
Grow with arg s S gives reflect.TypeOf(s) -> main.MyUint64Slice
However it's not immediately apparent to me what's the practical difference between the two.
Playground with the code: https://gotipplay.golang.org/p/zg2dGtSJwuI
Question
Are these two declarations equivalent in practice? If not, when should I choose one over the other?
It matters if you have to return a slice of the same (possibly named) type as the argument.
If you do not have to return a slice (just some other info e.g. a bool to report if the value is contained), you do not need to use a type parameter that itself constraints to a slice, you may use a type parameter for the element only.
If you have to return a slice of the same type as the input, you must use a type parameter that itself constraints to a slice (e.g. ~[]E).
To demonstrate, let's see these 2 implementations of Grow():
func Grow[S ~[]E, E any](s S, n int) S {
return append(s, make(S, n)...)[:len(s)]
}
func Grow2[E any](s []E, n int) []E {
return append(s, make([]E, n)...)[:len(s)]
}
If you pass a slice of a custom type having a slice as its underlying type, Grow() can return a value of the same type. Grow2() cannot: it can only return a value of an unnamed slice type: []E.
And the demonstration:
x := []int{1}
x2 := Grow(x, 10)
fmt.Printf("x2 %T len=%d cap=%d\n", x2, len(x2), cap(x2))
x3 := Grow2(x, 10)
fmt.Printf("x3 %T len=%d cap=%d\n", x3, len(x3), cap(x3))
type ints []int
y := ints{1}
y2 := Grow(y, 10)
fmt.Printf("y2 %T len=%d cap=%d\n", y2, len(y2), cap(y2))
y3 := Grow2(y, 10)
fmt.Printf("y3 %T len=%d cap=%d\n", y3, len(y3), cap(y3))
Output (try it on the Go Playground):
x2 []int len=1 cap=12
x3 []int len=1 cap=12
y2 main.ints len=1 cap=12
y3 []int len=1 cap=12
As you can see Grow2(y, 10) receives a value of type main.ints and yet it returns a value of type []int. This is not what we would want from it.
I'm curious why Go does't implicitly convert []T to []interface{} when it will implicitly convert T to interface{}. Is there something non-trivial about this conversion that I'm missing?
Example:
func foo([]interface{}) { /* do something */ }
func main() {
var a []string = []string{"hello", "world"}
foo(a)
}
go build complains
cannot use a (type []string) as type []interface {} in function argument
And if I try to do it explicitly, same thing: b := []interface{}(a) complains
cannot convert a (type []string) to type []interface {}
So every time I need to do this conversion (which seems to come up a lot), I've been doing something like this:
b = make([]interface{}, len(a), len(a))
for i := range a {
b[i] = a[i]
}
Is there a better way to do this, or standard library functions to help with these conversions? It seems kind of silly to write 4 extra lines of code every time I want to call a function that can take a list of e.g. ints or strings.
In Go, there is a general rule that syntax should not hide complex/costly operations.
Converting a string to an interface{} is done in O(1) time. Converting a []string to an interface{} is also done in O(1) time since a slice is still one value. However, converting a []string to an []interface{} is O(n) time because each element of the slice must be converted to an interface{}.
The one exception to this rule is converting strings. When converting a string to and from a []byte or a []rune, Go does O(n) work even though conversions are "syntax".
There is no standard library function that will do this conversion for you. Your best option though is just to use the lines of code you gave in your question:
b := make([]interface{}, len(a))
for i := range a {
b[i] = a[i]
}
Otherwise, you could make one with reflect, but it would be slower than the three line option. Example with reflection:
func InterfaceSlice(slice interface{}) []interface{} {
s := reflect.ValueOf(slice)
if s.Kind() != reflect.Slice {
panic("InterfaceSlice() given a non-slice type")
}
// Keep the distinction between nil and empty slice input
if s.IsNil() {
return nil
}
ret := make([]interface{}, s.Len())
for i:=0; i<s.Len(); i++ {
ret[i] = s.Index(i).Interface()
}
return ret
}
The thing you are missing is that T and interface{} which holds a value of T have different representations in memory so can't be trivially converted.
A variable of type T is just its value in memory. There is no associated type information (in Go every variable has a single type known at compile time not at run time). It is represented in memory like this:
value
An interface{} holding a variable of type T is represented in memory like this
pointer to type T
value
So coming back to your original question: why go does't implicitly convert []T to []interface{}?
Converting []T to []interface{} would involve creating a new slice of interface {} values which is a non-trivial operation since the in-memory layout is completely different.
Here is the official explanation: https://github.com/golang/go/wiki/InterfaceSlice
var dataSlice []int = foo()
var interfaceSlice []interface{} = make([]interface{}, len(dataSlice))
for i, d := range dataSlice {
interfaceSlice[i] = d
}
In Go 1.18 or later, use the following function to convert an arbitrary slice type to []interface{} or its alias any:
func ToSliceOfAny[T any](s []T) []any {
result := make([]any, len(s))
for i, v := range s {
result[i] = v
}
return result
}
The Go 1.18 generics feature does not eliminate the need to convert an arbitrary slice to []any. Here's an example of where the conversion is required: The application wants to query a database using the elements of a []string as the variadic query arguments declared as args ...any. The function in this answer allows the application to query the database in a convenient one-liner:
rows, err := db.Query(qs, ToSliceOfAny(stringArgs)...)
Try interface{} instead. To cast back as slice, try
func foo(bar interface{}) {
s := bar.([]string)
// ...
}
In case you need more shorting your code, you can creating new type for helper
type Strings []string
func (ss Strings) ToInterfaceSlice() []interface{} {
iface := make([]interface{}, len(ss))
for i := range ss {
iface[i] = ss[i]
}
return iface
}
then
a := []strings{"a", "b", "c", "d"}
sliceIFace := Strings(a).ToInterfaceSlice()
I was curious how much slower it is convert interface arrays via reflection vs. doing it inside a loop, as described in Stephen's answer. Here's a benchmark comparison of the two approaches:
benchmark iter time/iter bytes alloc allocs
--------- ---- --------- ----------- ------
BenchmarkLoopConversion-12 2285820 522.30 ns/op 400 B/op 11 allocs/op
BenchmarkReflectionConversion-12 1780002 669.00 ns/op 584 B/op 13 allocs/op
So using a loop is ~20% faster than doing it via reflection.
Here's my test code in case you'd like to verify if I did things correctly:
import (
"math/rand"
"reflect"
"testing"
"time"
)
func InterfaceSlice(slice interface{}) []interface{} {
s := reflect.ValueOf(slice)
if s.Kind() != reflect.Slice {
panic("InterfaceSlice() given a non-slice type")
}
// Keep the distinction between nil and empty slice input
if s.IsNil() {
return nil
}
ret := make([]interface{}, s.Len())
for i := 0; i < s.Len(); i++ {
ret[i] = s.Index(i).Interface()
}
return ret
}
type TestStruct struct {
name string
age int
}
var letters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
func randSeq(n int) string {
b := make([]rune, n)
for i := range b {
b[i] = letters[rand.Intn(len(letters))]
}
return string(b)
}
func randTestStruct(lenArray int, lenMap int) map[int][]TestStruct {
randomStructMap := make(map[int][]TestStruct, lenMap)
for i := 0; i < lenMap; i++ {
var testStructs = make([]TestStruct, 0)
for k := 0; k < lenArray; k++ {
rand.Seed(time.Now().UnixNano())
randomString := randSeq(10)
randomInt := rand.Intn(100)
testStructs = append(testStructs, TestStruct{name: randomString, age: randomInt})
}
randomStructMap[i] = testStructs
}
return randomStructMap
}
func BenchmarkLoopConversion(b *testing.B) {
var testStructMap = randTestStruct(10, 100)
b.ResetTimer()
for i := 0; i < b.N; i++ {
obj := make([]interface{}, len(testStructMap[i%100]))
for k := range testStructMap[i%100] {
obj[k] = testStructMap[i%100][k]
}
}
}
func BenchmarkReflectionConversion(b *testing.B) {
var testStructMap = randTestStruct(10, 100)
b.ResetTimer()
for i := 0; i < b.N; i++ {
obj := make([]interface{}, len(testStructMap[i%100]))
obj = InterfaceSlice(testStructMap[i%100])
_ = obj
}
}
Though you can use a generic function to convert a slice to a slice of interface{}, it may be most appropriate and cheapest in terms of execution time to change foo to a generic function if possible.
For example:
func foo[T any](slice []T) { /* do something */ }
func main() {
var a []string = []string{"hello", "world"}
foo(a)
}
Now there is no conversion necessary at all.
Convert interface{} into any type.
Syntax:
result := interface.(datatype)
Example:
var employee interface{} = []string{"Jhon", "Arya"}
result := employee.([]string) //result type is []string.