I have some golang code that manipulates slices of an interface type (Comparable). To test my code, I want to create some fake data and operate on it. However, I'm having trouble doing this in a way that is not incredibly tedious. The only thing I can think to do is create a new type for testing (in this case an alias of type int) that satisfies the Comparable interface, and then feed my tests literal slices of that type. I envision it looking something like the following:
type Comparable interface {
LT(Comparable) bool
AsFloat() float64
}
type testInt int
func (self testInt) LT(other Comparable) bool {
return float64(self) < other.AsFloat()
}
func (self testInt) AsFloat() float64 {
return float64(self)
}
func TestAFunction(t *testing.T) {
FunctionToTest([]Comparable{7, 4, 2, 1})
....
}
However, with this example, the compiler will complain that type int cannot be used as a Comparable. I understand why this is happening, but I'm not sure how to solve it. First, I don't know how to create a literal of type testInt. Second, I have to write a significant number of these functions. Working with literal ints is far more convenient for my purposes.
Is there a way to work with type aliases of builtin types such that the compiler can correctly infer the correct type of literals with a minimum of code?
Additionally, is there perhaps a better way to accomplish what I am trying to do, i.e., generate hard data that satisfies an interface for use in testing?
func NewWhatevers(a ...int) (r []Whatever) {
r = make([]Whatever, len(a))
for i, v := range a {
r[i] = Whatever(v)
}
return
}
...
myWhatevers := NewWhatevers(7, 4, 2, 1)
There are a number of ways to accomplish this. The problem, as you correctly state, is that the Go compiler cannot automatically convert int to Comparable (since doing so would require finding all possible equivalent types, and figuring out which of those equivalent types satisfy the Comparable interface, and then if there are more than one... you get the idea). Thus, you'll have to do one of two things:
Write an explicit type conversion:
FunctionToTest([]Comparable{ testInt(7), testInt(4), testInt(2), testInt(1) })
However, if you need a lot of literals, this could get really annoying. Thus, you could also:
Write a function to convert []int to []Comparable:
func intToComparable(i []int) []Comparable {
c := make([]Comparable, len(i))
for i, v := range i {
c[i] = testInt(v)
}
return c
}
and then you'd only have to do:
FunctionToTest(intToComparable([]int{ 7, 4, 2, 1 }))
Additionally, is there perhaps a better way to accomplish what I am trying to do, i.e., generate hard data that satisfies an interface for use in testing?
Maybe. The problem you encountered is that []Comparable and []testInt are fundamentally different and cannot be exchanged as the underlying representation in memory is different.
If your code is less about individual item which are Comparable but more about slices of items which can be compared than you could refactor your code to work on whole Slices.
Have a look at how package sort does this: It doesn't operate on a slice of comparables but on a "comparable slice".
// FloatOrder is a slice with comparable and float-convertible elements
type FloatOrder interface {
Less(i, j int) bool // Compare element i and j and return true first is less than the other
Float(i int) float64 // Return element i as a float64
}
type testInts []int
func (n testInts) Less(i, j int) bool {return n[i] < n[j]}
func (n testInts) Float(i int) float64 { return float64(n[i]) }
func FunctionTotest(fo FloatOrder) { ... }
func TestAFunction(t *testing.T) {
FunctionToTest(testInts{1,2,3,4})
....
}
(Completely untested, illustration-only code)
Related
I decided that now that generics have been introduced into Go that something like map/reduce should be possible. So, I took a naive stab at it and I get the error:
./prog.go:18:36: cannot use thing (variable of type int) as type I in argument to mapper
Which doesn't explain if the problem is fundamental or I am simply doing something wrong syntactically. Can generic map/reduce be implemented in Go?
package main
import "fmt"
func main() {
things := []int{1, 2, 3, 4}
results := Map(things, func(t int) int {
return t + 1
})
fmt.Printf("%v", results)
}
func Map[I interface{}, O interface{}](things []I, mapper func(thing I) O) []O {
results := make([]O, 0, len(things))
for thing := range things {
results = append(results, mapper(thing))
}
return results
}
You have incorrect use of range. A single variable extracted from range will be the index (type int), not the value (type I, which is only coincidentally int in this case).
Try
for _, thing := range things{...}
This can be done quite easily. You have an error in your code, though right here:
for thing := range things {
You are iterating over the index values (int), not the values of type I. You're also specifying 2 constraints (types I and O) both set to be interface{}. You can just use any instead (it's shorthand for interface{})
So simply write:
func Map[T any, O any](things []T, mapper func(thing T) O) []O {
result := make([]O, 0, len(things))
for _, thing := range things {
result = append(result, mapper(thing))
}
return result
}
Demo
This is quite closely related to some code I reviewed on codereview exchange here. After going through the code, and writing snippets with a ton of suggestions, I decided to just create a package and throw it up on github instead. You can find the repo here.
In it, there's some examples that may come in handy, or help you work through some other quirks WRT generics in golang. I wsa specifically thinking about this bit, where you can filter a generic map type using callbacks like so:
// given the sMap type
type sMap[K comparable, V any] struct {
mu *sync.RWMutex
m map[K]V
}
// Filter returns a map containing the elements that matched the filter callback argument
func (s *sMap[K, V]) Filter(cb func(K, V) bool) map[K]V {
s.mu.RLock()
defer s.mu.RUnlock()
ret := make(map[K]V, len(s.m))
for k, v := range s.m {
if cb(k, v) {
ret[k] = v
}
}
return ret
}
With the addition of generics in Go 1.18, would it now be possible to come up with an equivalent of C#'s LINQ to Objects?
Or are Go's generics lacking something in principle, compared to C# generics, that will make that difficult or impossible?
For example, the first of the original 101 LINQ samples ("LowNumbers") could now be implemented in Go with generics roughly like this:
package main
import (
"fmt"
)
type collection[T comparable] []T
func (input collection[T]) where(pred func(T) bool) collection[T] {
result := collection[T]{}
for _, j := range input {
if pred(j) {
result = append(result, j)
}
}
return result
}
func main() {
numbers := collection[int]{5, 4, 1, 3, 9, 8, 6, 7, 2, 0}
lowNums := numbers.where(func(i int) bool { return i < 5 })
fmt.Println("Numbers < 5:")
fmt.Println(lowNums)
}
Yes and no.
You can almost get there using a chained APIs.
This works for many of the standard LINQ methods, such as Skip, Take, Where, First, Last etc.
What doesn't work, is when you need to switch to another generic type within a flow/stream.
Go Generics does not allow methods to have other type argument than the interface/struct for which they are defined.
e.g. you cannot have a struct Foo[T any] that then has a method Bar[O any]
This is needed for methods like Select where you have one type for the input and another type for the output.
However, if you don't use chaining and just go for plain functions. then you can get pretty close functionality-wise.
I've done that here: https://github.com/asynkron/gofun
This is a fully lazy enumerable implementation by simulating co-routines.
What doesn't work here is functions like Zip which needs to enumerate two enumerables at the same time. (although there are ways to hack that. but nothing pretty)
(Disclaimer: I'm not a C# expert)
A conspicuous difference between Go's parametric polymorphism and the implementation of generics in C# or Java is that Go (still) has no syntax for co-/contra-variance over type parameters.
For example in C# you can have code that implements IComparer<T> and pass derived container classes; or in Java the typical Predicate<? super T> in the stream API. In Go, types must match exactly, and instantiating a generic type with different type parameters yields different named types that just can't be assigned to each other. See also: Why does Go not allow assigning one generic to another?
Also Go is not OO, so there's no concept of inheritance. You may have types that implement interfaces, and even parametrized interfaces. A contrived example:
type Equaler[T any] interface {
Equals(T) bool
}
type Vector []int32
func (v Vector) Equals(other Vector) bool {
// some logic
}
So with this code, Vector implements a specific instance of Equaler which is Equaler[Vector]. To be clear, the following var declaration compiles:
var _ Equaler[Vector] = Vector{}
So with this, you can write functions that are generic in T and use T to instantiate Equaler, and you will be able to pass anything that does implement that specific instance of Equaler:
func Remove[E Equaler[T], T any](es []E, v T) []E {
for i, e := range es {
if e.Equals(v) {
return append(es[:i], es[i+1:]...)
}
}
return es
}
And you can call this function with any T, and therefore with any T that has an Equals(T) method:
// some other random type that implements Equaler[T]
type MyString string
// implements Equaler[string]
func (s MyString) Equals(other string) bool {
return strings.Split(string(s), "-")[0] == other
}
func main() {
vecs := []Vector{{1, 2}, {3, 4, 5}, {6, 7}, {8}}
fmt.Println(Remove(vecs, Vector{6, 7}))
// prints [[1 2] [3 4 5] [8]]
strs := []MyString{"foo-bar", "hello-world", "bar-baz"}
fmt.Println(Remove(strs, "hello"))
// prints [foo-bar bar-baz]
}
The only problem is that only defined types can have methods, so this approach already excludes all composite non-named types.
However, to a partial rescue, Go has higher-order functions, so writing a stream-like API with that and non-named types is not impossible, e.g.:
func Where[C ~[]T, T any](collection C, predicate func(T) bool) (out C) {
for _, v := range collection {
if predicate(v) {
out = append(out, v)
}
}
return
}
func main() {
// vecs declared earlier
filtered := Where(vecs, func(v Vector) bool { return v[0] == 3})
fmt.Printf("%T %v", filtered, filtered)
// prints []main.Vector [[3 4 5]]
}
In particular here you use a named type parameter C ~[]T instead of just defining collection []T so that you can use it with both named and non-named types.
Code available in the playground: https://gotipplay.golang.org/p/mCM2TJ9qb3F
(Choosing the parametrized interfaces vs. higher-order functions probably depends on, among others, if you want to chain methods, but method chaining in Go isn't very common to begin with.)
Conclusion: whether that is enough to mimic LINQ- or Stream-like APIs, and/or enable large generic libraries, only practice will tell. The existing facilities are pretty powerful, and could become even more so in Go 1.19 after the language designers gain additional experience with real-world usage of generics.
This question already has answers here:
Go Generics - Unions
(2 answers)
Closed 11 months ago.
Can you write a function in Go whose parameter can be two different types? For example, if I write a simple function that takes in an array/slice of int and simply returns the first value:
func First(array []int) int {
return array[0]
}
Is there a way to type this such that we can also pass in a []string, etc.? In TypeScript for example, we can do it like such without having to type the array as any:
const first = (array: (number | string)[]): number | string => {
return array[0];
};
I've seen answers explaining the use of interface{} for situations like this... and maybe that's the only way, but it seems to close to any in TS and it feels like there might be a better way of doing this.
(I haven't used Go for a few years, long before they introduced generics - so I'm basing my answer off their documentation for generics)
TypeScript's "generics" (in quotes) aren't really comparable to Go's generics. TypeScript is all about being able to describe an interface (in an abstract sense) for a runtime system built around type-erasure (i.e. JavaScript), while Go's is... honestly, I have no idea. I just don't know how Go's generics are implemented nor their runtime characteristics: Articles on Go's generics, even from the language authors blog or their own documentation site fail to mention key-terms like type erasure, reified generics, (template) instantiation or monomorph, so if anyone has a better understanding please edit this post, or let me know in a comment!
Anyway, the good news is that as-of the Go 1.18 Beta, its support for generics includes support for generic constraints, but also support for union types as a constraint for generic type parameters (though I haven't yet found any information regarding support for other ADTs like product types and intersection types).
(Note that, at least for now, Go won't support union types as concrete types, but in practice that shouldn't be an issue)
In your case, if you want a function that returns the first element of a slice that could be either []int or []string (or returns some default-value if the slice is empty), then you can do this:
func First[T int | string](arr []T, ifEmpty T) T {
for _, v := range arr {
return v
}
return ifEmpty
}
While at first-glance you might think that this would allow for the arr slice to simultaneously contain both int and string values, this is not allowed (see below). Remember that generic parameters arguments are supplied by the caller and have to be valid concrete types, so First can only be instantiated as either First[int] or First[string], which in-turn implies that arr can be either []int or []string.
Anyway, the full example below compiles and runs in the Go dev branch on the Go playground:
package main
import "fmt"
func First[T int | string](arr []T, ifEmpty T) T {
for _, v := range arr {
return v
}
return ifEmpty
}
func main() {
// First[int]:
arrayOfInts := []int{2, 3, 5, 7, 11, 13}
firstInt := First(arrayOfInts, -1)
fmt.Println(firstInt) // 2
// First[string]:
arrayOfStrings := []string{"life", "is short", "and love is always over", "in the morning"}
firstString := First(arrayOfStrings, "empty")
fmt.Println(firstString) // "life"
}
You can also extract the constraint T int | string and move it to an interface, and then use that as the constraint, which I personally think is easier to read, especially when you might need to repeat the same constraint in multiple places:
type IntOrString interface {
int | string
}
func First[T IntOrString](arr []T, ifEmpty T) T {
for _, v := range arr {
return v
}
return ifEmpty
}
Things you can't do...
Note that Go does not (currently, at least) allow using a type that describes a union as a variable's type by itself (nor can you use as a slice's element type either); you can only use a union as a constraint, otherwise you'll get the "interface contains type constraints" error. Which means you can't describe an array that can contain both int and string values and then use that interface for a concrete array type:
package main
import "fmt"
type IntOrString interface {
int | string
}
func First[T IntOrString](arr []T, ifEmpty T) T {
for _, v := range arr {
return v
}
return ifEmpty
}
func main() {
arrayOfIntsOrStrings := []IntOrString{2, "foo", 3, "bar", 5, "baz", 7, 11, 13} // ERROR: interface contains type constraints
firstValue := First(arrayOfIntsOrStrings, -1)
fmt.Println(firstValue)
}
./prog.go:19:28: interface contains type constraints
Go build failed.
Let's say I have a bunch of structs (around 10).
type A struct {
ID int64
... other A-specific fields
}
type B struct {
ID int64
... other B-specific fields
}
type C struct {
ID int64
... other C-specific fields
}
If I have an array of these structs at any given time (either []A, []B, or []C), how can I write a single function that pulls the IDs from the array of structs without writing 3 (or in my case, 10) separate functions like this:
type AList []A
type BList []B
type CList []C
func (list *AList) GetIDs() []int64 { ... }
func (list *BList) GetIDs() []int64 { ... }
func (list *CList) GetIDs() []int64 { ... }
With general method on the slice itself
You can make it a little simpler if you define a general interface to access the ID of the ith element of a slice:
type HasIDs interface {
GetID(i int) int64
}
And you provide implementation for these:
func (x AList) GetID(i int) int64 { return x[i].ID }
func (x BList) GetID(i int) int64 { return x[i].ID }
func (x CList) GetID(i int) int64 { return x[i].ID }
And then one GetID() function is enough:
func GetIDs(s HasIDs) (ids []int64) {
ids = make([]int64, reflect.ValueOf(s).Len())
for i := range ids {
ids[i] = s.GetID(i)
}
return
}
Note: the length of the slice may be a parameter to GetIDs(), or it may be part of the HasIDs interface. Both are more complex than the tiny reflection call to get the length of the slice, so bear with me on this.
Using it:
as := AList{A{1}, A{2}}
fmt.Println(GetIDs(as))
bs := BList{B{3}, B{4}}
fmt.Println(GetIDs(bs))
cs := []C{C{5}, C{6}}
fmt.Println(GetIDs(CList(cs)))
Output (try it on the Go Playground):
[1 2]
[3 4]
[5 6]
Note that we were able to use slices of type AList, BList etc, we did not need to use interface{} or []SomeIface. Also note that we could also use e.g. a []C, and when passing it to GetIDs(), we used a simple type conversion.
This is as simple as it can get. If you want to eliminate even the GetID() methods of the slices, then you really need to dig deeper into reflection (reflect package), and it will be slower. The presented solution above performs roughly the same as the "hard-coded" version.
With reflection completely
If you want it to be completely "generic", you may do it using reflection, and then you need absolutely no extra methods on anything.
Without checking for errors, here's the solution:
func GetIDs(s interface{}) (ids []int64) {
v := reflect.ValueOf(s)
ids = make([]int64, v.Len())
for i := range ids {
ids[i] = v.Index(i).FieldByName("ID").Int()
}
return
}
Testing and output is (almost) the same. Note that since here parameter type of GetIDs() is interface{}, you don't need to convert to CList to pass a value of type []C. Try it on the Go Playground.
With embedding and reflection
Getting a field by specifying its name as a string is quite fragile (think of rename / refactoring for example). We can improve maintainability, safety, and somewhat the reflection's performance if we "outsource" the ID field and an accessor method to a separate struct, which we'll embed, and we capture the accessor by an interface:
type IDWrapper struct {
ID int64
}
func (i IDWrapper) GetID() int64 { return i.ID }
type HasID interface {
GetID() int64
}
And the types all embed IDWrapper:
type A struct {
IDWrapper
}
type B struct {
IDWrapper
}
type C struct {
IDWrapper
}
By embedding, all the embedder types (A, B, C) will have the GetID() method promoted and thus they all automatically implement HasID. We can take advantage of this in the GetIDs() function:
func GetIDs(s interface{}) (ids []int64) {
v := reflect.ValueOf(s)
ids = make([]int64, v.Len())
for i := range ids {
ids[i] = v.Index(i).Interface().(HasID).GetID()
}
return
}
Testing it:
as := AList{A{IDWrapper{1}}, A{IDWrapper{2}}}
fmt.Println(GetIDs(as))
bs := BList{B{IDWrapper{3}}, B{IDWrapper{4}}}
fmt.Println(GetIDs(bs))
cs := []C{C{IDWrapper{5}}, C{IDWrapper{6}}}
fmt.Println(GetIDs(cs))
Output is the same. Try it on the Go Playground. Note that in this case the only method is IDWrapper.GetID(), no other methods needed to be defined.
As far as I know, there is no easy way.
You might be tempted to use embedding, but I'm not sure there's any way to make this particular task any easier. Embedding feels like subclassing but it doesn't give you the power of polymorphism.
Polymorphism in Go is limited to methods and interfaces, not fields, so you can't access a given field by name across multiple classes.
You could use reflection to find and access the field you are interested in by name (or tag), but there are performance penalties for that and it will make your code complex and hard to follow. Reflection is not really intended to be a substitute for Polymorphism or generics.
I think your best solution is to use the polymorphism that Go does give you, and create an interface:
type IDable interface {
GetId() int64
}
and make a GetId method for each of your classes. Full example.
Generic methods require the use of interfaces and reflection.
Is there a way to write a generic array/slice deduplication in go, for []int we can have something like (from http://rosettacode.org/wiki/Remove_duplicate_elements#Go ):
func uniq(list []int) []int {
unique_set := make(map[int] bool, len(list))
for _, x := range list {
unique_set[x] = true
}
result := make([]int, len(unique_set))
i := 0
for x := range unique_set {
result[i] = x
i++
}
return result
}
But is there a way to extend it to support any array? with a signature like:
func deduplicate(a []interface{}) []interface{}
I know that you can write that function with that signature, but then you can't actually use it on []int, you need to create a []interface{} put everything from the []int into it, pass it to the function then get it back and put it into a []interface{} and go through this new array and put everything in a new []int.
My question is, is there a better way to do this?
While VonC's answer probably does the closest to what you really want, the only real way to do it in native Go without gen is to define an interface
type IDList interface {
// Returns the id of the element at i
ID(i int) int
// Returns the element
// with the given id
GetByID(id int) interface{}
Len() int
// Adds the element to the list
Insert(interface{})
}
// Puts the deduplicated list in dst
func Deduplicate(dst, list IDList) {
intList := make([]int, list.Len())
for i := range intList {
intList[i] = list.ID(i)
}
uniques := uniq(intList)
for _,el := range uniques {
dst.Insert(list.GetByID(el))
}
}
Where uniq is the function from your OP.
This is just one possible example, and there are probably much better ones, but in general mapping each element to a unique "==able" ID and either constructing a new list or culling based on the deduplication of the IDs is probably the most intuitive way.
An alternate solution is to take in an []IDer where the IDer interface is just ID() int. However, that means that user code has to create the []IDer list and copy all the elements into that list, which is a bit ugly. It's cleaner for the user to wrap the list as an ID list rather than copy, but it's a similar amount of work either way.
The only way I have seen that implemented in Go is with the clipperhouse/gen project,
gen is an attempt to bring some generics-like functionality to Go, with some inspiration from C#’s Linq and JavaScript’s underscore libraries
See this test:
// Distinct returns a new Thing1s slice whose elements are unique. See: http://clipperhouse.github.io/gen/#Distinct
func (rcv Thing1s) Distinct() (result Thing1s) {
appended := make(map[Thing1]bool)
for _, v := range rcv {
if !appended[v] {
result = append(result, v)
appended[v] = true
}
}
return result
}
But, as explained in clipperhouse.github.io/gen/:
gen generates code for your types, at development time, using the command line.
gen is not an import; the generated source becomes part of your project and takes no external dependencies.
You could do something close to this via an interface. Define an interface, say "DeDupable" requiring a func, say, UniqId() []byte, which you could then use to do the removing of dups. and your uniq func would take a []DeDupable and work on it