With the addition of generics in Go 1.18, would it now be possible to come up with an equivalent of C#'s LINQ to Objects?
Or are Go's generics lacking something in principle, compared to C# generics, that will make that difficult or impossible?
For example, the first of the original 101 LINQ samples ("LowNumbers") could now be implemented in Go with generics roughly like this:
package main
import (
"fmt"
)
type collection[T comparable] []T
func (input collection[T]) where(pred func(T) bool) collection[T] {
result := collection[T]{}
for _, j := range input {
if pred(j) {
result = append(result, j)
}
}
return result
}
func main() {
numbers := collection[int]{5, 4, 1, 3, 9, 8, 6, 7, 2, 0}
lowNums := numbers.where(func(i int) bool { return i < 5 })
fmt.Println("Numbers < 5:")
fmt.Println(lowNums)
}
Yes and no.
You can almost get there using a chained APIs.
This works for many of the standard LINQ methods, such as Skip, Take, Where, First, Last etc.
What doesn't work, is when you need to switch to another generic type within a flow/stream.
Go Generics does not allow methods to have other type argument than the interface/struct for which they are defined.
e.g. you cannot have a struct Foo[T any] that then has a method Bar[O any]
This is needed for methods like Select where you have one type for the input and another type for the output.
However, if you don't use chaining and just go for plain functions. then you can get pretty close functionality-wise.
I've done that here: https://github.com/asynkron/gofun
This is a fully lazy enumerable implementation by simulating co-routines.
What doesn't work here is functions like Zip which needs to enumerate two enumerables at the same time. (although there are ways to hack that. but nothing pretty)
(Disclaimer: I'm not a C# expert)
A conspicuous difference between Go's parametric polymorphism and the implementation of generics in C# or Java is that Go (still) has no syntax for co-/contra-variance over type parameters.
For example in C# you can have code that implements IComparer<T> and pass derived container classes; or in Java the typical Predicate<? super T> in the stream API. In Go, types must match exactly, and instantiating a generic type with different type parameters yields different named types that just can't be assigned to each other. See also: Why does Go not allow assigning one generic to another?
Also Go is not OO, so there's no concept of inheritance. You may have types that implement interfaces, and even parametrized interfaces. A contrived example:
type Equaler[T any] interface {
Equals(T) bool
}
type Vector []int32
func (v Vector) Equals(other Vector) bool {
// some logic
}
So with this code, Vector implements a specific instance of Equaler which is Equaler[Vector]. To be clear, the following var declaration compiles:
var _ Equaler[Vector] = Vector{}
So with this, you can write functions that are generic in T and use T to instantiate Equaler, and you will be able to pass anything that does implement that specific instance of Equaler:
func Remove[E Equaler[T], T any](es []E, v T) []E {
for i, e := range es {
if e.Equals(v) {
return append(es[:i], es[i+1:]...)
}
}
return es
}
And you can call this function with any T, and therefore with any T that has an Equals(T) method:
// some other random type that implements Equaler[T]
type MyString string
// implements Equaler[string]
func (s MyString) Equals(other string) bool {
return strings.Split(string(s), "-")[0] == other
}
func main() {
vecs := []Vector{{1, 2}, {3, 4, 5}, {6, 7}, {8}}
fmt.Println(Remove(vecs, Vector{6, 7}))
// prints [[1 2] [3 4 5] [8]]
strs := []MyString{"foo-bar", "hello-world", "bar-baz"}
fmt.Println(Remove(strs, "hello"))
// prints [foo-bar bar-baz]
}
The only problem is that only defined types can have methods, so this approach already excludes all composite non-named types.
However, to a partial rescue, Go has higher-order functions, so writing a stream-like API with that and non-named types is not impossible, e.g.:
func Where[C ~[]T, T any](collection C, predicate func(T) bool) (out C) {
for _, v := range collection {
if predicate(v) {
out = append(out, v)
}
}
return
}
func main() {
// vecs declared earlier
filtered := Where(vecs, func(v Vector) bool { return v[0] == 3})
fmt.Printf("%T %v", filtered, filtered)
// prints []main.Vector [[3 4 5]]
}
In particular here you use a named type parameter C ~[]T instead of just defining collection []T so that you can use it with both named and non-named types.
Code available in the playground: https://gotipplay.golang.org/p/mCM2TJ9qb3F
(Choosing the parametrized interfaces vs. higher-order functions probably depends on, among others, if you want to chain methods, but method chaining in Go isn't very common to begin with.)
Conclusion: whether that is enough to mimic LINQ- or Stream-like APIs, and/or enable large generic libraries, only practice will tell. The existing facilities are pretty powerful, and could become even more so in Go 1.19 after the language designers gain additional experience with real-world usage of generics.
Related
I had a function implemented to obtain the keys in a map (a few versions actually, for different types), which I updated to use generics in Go 1.18. Then I found out that the experimental library was extended to include that functionality, and while my implementation was nearly identical, the function declaration has some differences that I would like to better understand.
Here is my original generic version (I renamed the variables to match the standard library, to better highlight was is actually different):
func mapKeys[K comparable, V any](m map[K]V) []K {
r := make([]K, 0, len(m))
for k := range m {
r = append(r, k)
}
return r
}
And here is the standard-library version:
func Keys[M ~map[K]V, K comparable, V any](m M) []K {
r := make([]K, 0, len(m))
for k := range m {
r = append(r, k)
}
return r
}
As you can see, the main difference is the extra M ~map[K]V type parameter, which I omitted and directly used map[K]V for the function's argument type. My function works, so why would I need to go through the extra trouble of adding a third parameterized type?
As I was writing my question, I thought I had figured out the answer: to be able to invoke the function on types that are really maps under the covers, but are not directly declared as such, like maybe on this DataCache type:
type DataCache map[string]DataObject
My thinking was that this probably required the ~map notation, and the ~ can only be used in a type constraint, not in an actual type. Only problem with this theory: my version works fine on such map types. So I am at a loss as to what it's useful for.
tl;dr it is relevant in the very uncommon case where you need to declare a variable of function type (without calling it), and you instantiate that function with a named map type from another package that uses unexported types in its definition.
Using a named type parameter in function signatures is mostly relevant when you need to accept and return defined types, as you correctly guessed, and as #icza answered here with respect to the x/exp/slices package.
Your remark that "tilde types" can only be used in interface constraints is also correct.
Now, almost all functions in the x/exp/maps package do not actually return the named type M. The only one that actually does is maps.Clone with signature:
func Clone[M ~map[K]V, K comparable, V any](m M) M
However declaring the signature without the approximate constraint ~map[K]V still works for defined types thanks to type unification. From the specs:
[...], because a defined type D and a type literal L are never equivalent, unification compares the underlying type of D with L instead
And a code example:
func Keys[K comparable, V any](m map[K]V) []K {
r := make([]K, 0, len(m))
for k := range m {
r = append(r, k)
}
return r
}
type Dictionary map[string]int
func main() {
m := Dictionary{"foo": 1, "bar": 2}
k := Keys(m)
fmt.Println(k) // it just works
}
Playground: https://go.dev/play/p/hzb2TflybZ9
The case where the additional named type parameter M ~map[K]V is relevant is when you need to pass around an instantiated value of the function:
func main() {
// variable of function type!
fn := Keys[Dictionary]
m := Dictionary{"foo": 1, "bar": 2}
fmt.Println(fn(m))
}
Playground: https://go.dev/play/p/hks_8bnhgsf
Without the M ~map[K]V type parameter, it would not be possible to instantiate such a function value with defined types. Of course you could instantiate your function with K and V separately like
fn := Keys[string, int]
But this is not viable when the defined map type belongs to a different package and references unexported types:
package foo
type someStruct struct{ val int }
type Dictionary map[string]someStruct
and:
package main
func main() {
// does not compile
// fn := Keys[string, foo.someStruct]
// this does
fn := maps.Keys[foo.Dictionary]
}
Though, this seems a rather exoteric use case.
You can see the final playground here: https://go.dev/play/p/B-_RBSqVqUD
However keep in mind that x/exp/maps is an experimental package, so the signatures may be changed with future Go releases, and/or when these functions get promoted into the standard library.
This question already has answers here:
Go Generics - Unions
(2 answers)
Closed 11 months ago.
Can you write a function in Go whose parameter can be two different types? For example, if I write a simple function that takes in an array/slice of int and simply returns the first value:
func First(array []int) int {
return array[0]
}
Is there a way to type this such that we can also pass in a []string, etc.? In TypeScript for example, we can do it like such without having to type the array as any:
const first = (array: (number | string)[]): number | string => {
return array[0];
};
I've seen answers explaining the use of interface{} for situations like this... and maybe that's the only way, but it seems to close to any in TS and it feels like there might be a better way of doing this.
(I haven't used Go for a few years, long before they introduced generics - so I'm basing my answer off their documentation for generics)
TypeScript's "generics" (in quotes) aren't really comparable to Go's generics. TypeScript is all about being able to describe an interface (in an abstract sense) for a runtime system built around type-erasure (i.e. JavaScript), while Go's is... honestly, I have no idea. I just don't know how Go's generics are implemented nor their runtime characteristics: Articles on Go's generics, even from the language authors blog or their own documentation site fail to mention key-terms like type erasure, reified generics, (template) instantiation or monomorph, so if anyone has a better understanding please edit this post, or let me know in a comment!
Anyway, the good news is that as-of the Go 1.18 Beta, its support for generics includes support for generic constraints, but also support for union types as a constraint for generic type parameters (though I haven't yet found any information regarding support for other ADTs like product types and intersection types).
(Note that, at least for now, Go won't support union types as concrete types, but in practice that shouldn't be an issue)
In your case, if you want a function that returns the first element of a slice that could be either []int or []string (or returns some default-value if the slice is empty), then you can do this:
func First[T int | string](arr []T, ifEmpty T) T {
for _, v := range arr {
return v
}
return ifEmpty
}
While at first-glance you might think that this would allow for the arr slice to simultaneously contain both int and string values, this is not allowed (see below). Remember that generic parameters arguments are supplied by the caller and have to be valid concrete types, so First can only be instantiated as either First[int] or First[string], which in-turn implies that arr can be either []int or []string.
Anyway, the full example below compiles and runs in the Go dev branch on the Go playground:
package main
import "fmt"
func First[T int | string](arr []T, ifEmpty T) T {
for _, v := range arr {
return v
}
return ifEmpty
}
func main() {
// First[int]:
arrayOfInts := []int{2, 3, 5, 7, 11, 13}
firstInt := First(arrayOfInts, -1)
fmt.Println(firstInt) // 2
// First[string]:
arrayOfStrings := []string{"life", "is short", "and love is always over", "in the morning"}
firstString := First(arrayOfStrings, "empty")
fmt.Println(firstString) // "life"
}
You can also extract the constraint T int | string and move it to an interface, and then use that as the constraint, which I personally think is easier to read, especially when you might need to repeat the same constraint in multiple places:
type IntOrString interface {
int | string
}
func First[T IntOrString](arr []T, ifEmpty T) T {
for _, v := range arr {
return v
}
return ifEmpty
}
Things you can't do...
Note that Go does not (currently, at least) allow using a type that describes a union as a variable's type by itself (nor can you use as a slice's element type either); you can only use a union as a constraint, otherwise you'll get the "interface contains type constraints" error. Which means you can't describe an array that can contain both int and string values and then use that interface for a concrete array type:
package main
import "fmt"
type IntOrString interface {
int | string
}
func First[T IntOrString](arr []T, ifEmpty T) T {
for _, v := range arr {
return v
}
return ifEmpty
}
func main() {
arrayOfIntsOrStrings := []IntOrString{2, "foo", 3, "bar", 5, "baz", 7, 11, 13} // ERROR: interface contains type constraints
firstValue := First(arrayOfIntsOrStrings, -1)
fmt.Println(firstValue)
}
./prog.go:19:28: interface contains type constraints
Go build failed.
Let's say I have a bunch of structs (around 10).
type A struct {
ID int64
... other A-specific fields
}
type B struct {
ID int64
... other B-specific fields
}
type C struct {
ID int64
... other C-specific fields
}
If I have an array of these structs at any given time (either []A, []B, or []C), how can I write a single function that pulls the IDs from the array of structs without writing 3 (or in my case, 10) separate functions like this:
type AList []A
type BList []B
type CList []C
func (list *AList) GetIDs() []int64 { ... }
func (list *BList) GetIDs() []int64 { ... }
func (list *CList) GetIDs() []int64 { ... }
With general method on the slice itself
You can make it a little simpler if you define a general interface to access the ID of the ith element of a slice:
type HasIDs interface {
GetID(i int) int64
}
And you provide implementation for these:
func (x AList) GetID(i int) int64 { return x[i].ID }
func (x BList) GetID(i int) int64 { return x[i].ID }
func (x CList) GetID(i int) int64 { return x[i].ID }
And then one GetID() function is enough:
func GetIDs(s HasIDs) (ids []int64) {
ids = make([]int64, reflect.ValueOf(s).Len())
for i := range ids {
ids[i] = s.GetID(i)
}
return
}
Note: the length of the slice may be a parameter to GetIDs(), or it may be part of the HasIDs interface. Both are more complex than the tiny reflection call to get the length of the slice, so bear with me on this.
Using it:
as := AList{A{1}, A{2}}
fmt.Println(GetIDs(as))
bs := BList{B{3}, B{4}}
fmt.Println(GetIDs(bs))
cs := []C{C{5}, C{6}}
fmt.Println(GetIDs(CList(cs)))
Output (try it on the Go Playground):
[1 2]
[3 4]
[5 6]
Note that we were able to use slices of type AList, BList etc, we did not need to use interface{} or []SomeIface. Also note that we could also use e.g. a []C, and when passing it to GetIDs(), we used a simple type conversion.
This is as simple as it can get. If you want to eliminate even the GetID() methods of the slices, then you really need to dig deeper into reflection (reflect package), and it will be slower. The presented solution above performs roughly the same as the "hard-coded" version.
With reflection completely
If you want it to be completely "generic", you may do it using reflection, and then you need absolutely no extra methods on anything.
Without checking for errors, here's the solution:
func GetIDs(s interface{}) (ids []int64) {
v := reflect.ValueOf(s)
ids = make([]int64, v.Len())
for i := range ids {
ids[i] = v.Index(i).FieldByName("ID").Int()
}
return
}
Testing and output is (almost) the same. Note that since here parameter type of GetIDs() is interface{}, you don't need to convert to CList to pass a value of type []C. Try it on the Go Playground.
With embedding and reflection
Getting a field by specifying its name as a string is quite fragile (think of rename / refactoring for example). We can improve maintainability, safety, and somewhat the reflection's performance if we "outsource" the ID field and an accessor method to a separate struct, which we'll embed, and we capture the accessor by an interface:
type IDWrapper struct {
ID int64
}
func (i IDWrapper) GetID() int64 { return i.ID }
type HasID interface {
GetID() int64
}
And the types all embed IDWrapper:
type A struct {
IDWrapper
}
type B struct {
IDWrapper
}
type C struct {
IDWrapper
}
By embedding, all the embedder types (A, B, C) will have the GetID() method promoted and thus they all automatically implement HasID. We can take advantage of this in the GetIDs() function:
func GetIDs(s interface{}) (ids []int64) {
v := reflect.ValueOf(s)
ids = make([]int64, v.Len())
for i := range ids {
ids[i] = v.Index(i).Interface().(HasID).GetID()
}
return
}
Testing it:
as := AList{A{IDWrapper{1}}, A{IDWrapper{2}}}
fmt.Println(GetIDs(as))
bs := BList{B{IDWrapper{3}}, B{IDWrapper{4}}}
fmt.Println(GetIDs(bs))
cs := []C{C{IDWrapper{5}}, C{IDWrapper{6}}}
fmt.Println(GetIDs(cs))
Output is the same. Try it on the Go Playground. Note that in this case the only method is IDWrapper.GetID(), no other methods needed to be defined.
As far as I know, there is no easy way.
You might be tempted to use embedding, but I'm not sure there's any way to make this particular task any easier. Embedding feels like subclassing but it doesn't give you the power of polymorphism.
Polymorphism in Go is limited to methods and interfaces, not fields, so you can't access a given field by name across multiple classes.
You could use reflection to find and access the field you are interested in by name (or tag), but there are performance penalties for that and it will make your code complex and hard to follow. Reflection is not really intended to be a substitute for Polymorphism or generics.
I think your best solution is to use the polymorphism that Go does give you, and create an interface:
type IDable interface {
GetId() int64
}
and make a GetId method for each of your classes. Full example.
Generic methods require the use of interfaces and reflection.
In Go, in order to iterate over an array/slice, you would write something like this:
for _, v := range arr {
fmt.Println(v)
}
However, I want to iterate over array/slice which includes different types (int, float64, string, etc...). In Python, I can write it out as follows:
a, b, c = 1, "str", 3.14
for i in [a, b, c]:
print(i)
How can I do such a work in Go? As far as I know, both array and slice are supposed to allow only same-type object, right? (say, []int allows only int type object.)
Thanks.
As Go is a statically typed language, that won't be as easy as in Python. You will have to resort to type assertions, reflection or similar means.
Take a look at this example:
package main
import (
"fmt"
)
func main() {
slice := make([]interface{}, 3)
slice[0] = 1
slice[1] = "hello"
slice[2] = true
for _, v := range slice {
switch v.(type) {
case string:
fmt.Println("We have a string")
case int:
fmt.Println("That's an integer!")
// You still need a type assertion, as v is of type interface{}
fmt.Printf("Its value is actually %d\n", v.(int))
default:
fmt.Println("It's some other type")
}
}
}
Here we construct a slice with the type of an empty interface (any type implements it), do a type switch and handle the value based on the result of that.
Unfortunately, you'll need this (or a similar method) anywhere where you'll be dealing with arrays of unspecified type (empty interface). Moreover, you'll probably need a case for every possible type, unless you have a way to deal with any object you could get.
One way would be to make all of the types you want to store implement some interface of yours and then only use those objects through that interface. That's kind of how fmt handles generic arguments – it simply calls String() on any object to get its string representation.
I have some golang code that manipulates slices of an interface type (Comparable). To test my code, I want to create some fake data and operate on it. However, I'm having trouble doing this in a way that is not incredibly tedious. The only thing I can think to do is create a new type for testing (in this case an alias of type int) that satisfies the Comparable interface, and then feed my tests literal slices of that type. I envision it looking something like the following:
type Comparable interface {
LT(Comparable) bool
AsFloat() float64
}
type testInt int
func (self testInt) LT(other Comparable) bool {
return float64(self) < other.AsFloat()
}
func (self testInt) AsFloat() float64 {
return float64(self)
}
func TestAFunction(t *testing.T) {
FunctionToTest([]Comparable{7, 4, 2, 1})
....
}
However, with this example, the compiler will complain that type int cannot be used as a Comparable. I understand why this is happening, but I'm not sure how to solve it. First, I don't know how to create a literal of type testInt. Second, I have to write a significant number of these functions. Working with literal ints is far more convenient for my purposes.
Is there a way to work with type aliases of builtin types such that the compiler can correctly infer the correct type of literals with a minimum of code?
Additionally, is there perhaps a better way to accomplish what I am trying to do, i.e., generate hard data that satisfies an interface for use in testing?
func NewWhatevers(a ...int) (r []Whatever) {
r = make([]Whatever, len(a))
for i, v := range a {
r[i] = Whatever(v)
}
return
}
...
myWhatevers := NewWhatevers(7, 4, 2, 1)
There are a number of ways to accomplish this. The problem, as you correctly state, is that the Go compiler cannot automatically convert int to Comparable (since doing so would require finding all possible equivalent types, and figuring out which of those equivalent types satisfy the Comparable interface, and then if there are more than one... you get the idea). Thus, you'll have to do one of two things:
Write an explicit type conversion:
FunctionToTest([]Comparable{ testInt(7), testInt(4), testInt(2), testInt(1) })
However, if you need a lot of literals, this could get really annoying. Thus, you could also:
Write a function to convert []int to []Comparable:
func intToComparable(i []int) []Comparable {
c := make([]Comparable, len(i))
for i, v := range i {
c[i] = testInt(v)
}
return c
}
and then you'd only have to do:
FunctionToTest(intToComparable([]int{ 7, 4, 2, 1 }))
Additionally, is there perhaps a better way to accomplish what I am trying to do, i.e., generate hard data that satisfies an interface for use in testing?
Maybe. The problem you encountered is that []Comparable and []testInt are fundamentally different and cannot be exchanged as the underlying representation in memory is different.
If your code is less about individual item which are Comparable but more about slices of items which can be compared than you could refactor your code to work on whole Slices.
Have a look at how package sort does this: It doesn't operate on a slice of comparables but on a "comparable slice".
// FloatOrder is a slice with comparable and float-convertible elements
type FloatOrder interface {
Less(i, j int) bool // Compare element i and j and return true first is less than the other
Float(i int) float64 // Return element i as a float64
}
type testInts []int
func (n testInts) Less(i, j int) bool {return n[i] < n[j]}
func (n testInts) Float(i int) float64 { return float64(n[i]) }
func FunctionTotest(fo FloatOrder) { ... }
func TestAFunction(t *testing.T) {
FunctionToTest(testInts{1,2,3,4})
....
}
(Completely untested, illustration-only code)