I want to express a function that can take any slice. I thought that I could do this:
func myFunc(list []interface{}) {
for _, i := range list {
...
some_other_fun(i)
...
}
}
where some_other_fun(..) itself takes an interface{} type. However, this doesn't work because you can't pass []DEFINITE_TYPE as []interface{}. See: https://golang.org/doc/faq#convert_slice_of_interface which notes that the representation of an []interface{} is different. This answer sums up why but with respect to pointers to interfaces instead of slices of interfaces, but the reason is the same: Why can't I assign a *Struct to an *Interface?.
The suggestion provided at the golang.org link above suggests rebuilding a new interface slice from the DEFINITE_TYPE slice. However, this is not practical to do everywhere in the code that I want to call this function (This function is itself meant to abbreviate only 9 lines of code, but those 9 lines appear quite frequently in our code).
In every case that I want to invoke the function I would be passing a []*DEFINITE_TYPE which I at first thought would be easier to abstract until, again, I discovered Why can't I assign a *Struct to an *Interface? (also linked above).
Further, everytime I want to invoke the function it is with a different DEFINITE_TYPE so implementing n examples for the n types would not save me any lines of code or make my code any clearer (quite the contrary!).
It is frustrating that I can't do this since the 9 lines are idiomatic in our code and a mistype could easily introduce a bug. I'm really missing generics. Is there really no way to do this?!!
In the case you provided, you would have to create your slice as a slice of interface e.g. s := []interface{}{}. At which point you could literally put any type you wanted into the slice (even mixing types). But then you would have to do all sorts of type assertions and everything gets really nasty.
Another technique that is commonly used by unmarshalers is a definition like this:
func myFunc(list interface{})
Because a slice fits an interface, you can indeed pass a regular slice into this. You would still need to do some validation and type assertions in myFunc, but you would be doing single assertions on the entire list type, instead of having to worry about a list that could possibly contain mixed types.
Either way, due to being a statically typed language, you eventually have to know the type that is passed in via assertions. It's just the way things are. In your case, I would probably use the func signature as above, then use a type switch to handle the different cases. See this document https://newfivefour.com/golang-interface-type-assertions-switch.html
So, something like this:
func myFunc(list interface{}) {
switch v := list.(type) {
case []string:
// do string thing
case []int32, []int64:
// do int thing
case []SomeCustomType:
// do SomeCustomType thing
default:
fmt.Println("unknown")
}
}
No there is no easy way to deal with it. Many people miss generics in Go.
Maybe you can get inspired by sort.Sort function and sort.Interface to find a reasonable solution that would not require copying slices.
Probably the best thing to do is to define an interface that encapsulates what myFunc needs to do with the slice (i.e., in your example, get the nth element). Then the argument to the function is that interface type and you define the interface method(s) for each type you want to pass to the function.
You can also do it with the reflect package, but that's probably not a great idea since it will panic if you pass something other than a slice (or array or string).
func myFunc(list interface{}) {
listVal := reflect.ValueOf(list)
for i := 0; i < listVal.Len(); i++ {
//...
some_other_fun(listVal.Index(i).Interface())
//...
}
}
See https://play.golang.org/p/TyzT3lBEjB.
Now with Go 1.18+, you can use the generics feature to do that:
func myFunc[T any](list []T) {
for _, item := range list {
doSomething(item)
}
}
Related
At first, I want to introduce you because I feel like I am missing some core Golang concept.
In my application, many models will have a method called GetByUserId. I created interface(UserCreatedEntity) that requires this method so then I will be able to create Controller's GetUserRecords method factory for each type of records with just:
router.Handle("/ideas/mine",
middlewares.AuthUser(controllers.GetMineFactory(&models.Idea{}))).Methods("POST")
router.Handle("/votes/mine",
middlewares.AuthUser(controllers.GetMineFactory(&models.Vote{}))).Methods("POST")
router.Handle("/someNewType/mine",
middlewares.AuthUser(controllers.GetMineFactory(&models.SomeNewType{}))).Methods("POST")
This is how my interface looks like:
type UserCreatedEntity interface {
GetByUserId(userId uint) []UserCreatedEntity
}
And implementation:
func (idea *Idea) GetByUserId(userId uint) []UserCreatedEntity {
ideas := []Idea{}
GetDB().
Table("ideas").
/** Query removed to make code less confusing **/
Scan(ideas)
return ideas
}
Obviously, it does not work (version with slice of pointers neither do). The thing is - This code would work if I return only one record - like that (obviously with changing signature in interface also):
func (idea *Idea) GetByUserId(userId uint) UserCreatedEntity {
idea := &Idea{}
GetDB().
Table("ideas").
/** Query removed to make code less confusing **/
First(idea)
return idea
}
How to make it work as slice? As I said I suspect that I am missing some important knowledge. So deep explaination would be awesome.
Solution:
func (idea *Idea) GetByUserId(userId uint) []UserCreatedEntity {
ideas := []*Idea{}
GetDB().
Table("ideas").
Select("problems.name AS problem_name, ideas.id, ideas.problem_id, ideas.action_description, ideas.results_description, ideas.money_price, ideas.time_price, ideas.is_published").
Joins("INNER JOIN problems ON ideas.problem_id = problems.id").
Where("ideas.user_id = ?", userId).
Scan(&ideas)
uces := make([]UserCreatedEntity, len(ideas))
for i, idea := range ideas {
uces[i] = idea
}
return uces
}
In programming language theory this is called variance, and it is not supported in Go. For much more details see this proposal.
Specifically, return types are not covariant. A slice of T does not implement a slice of I even if T implements I.
The FAQ entry linked above proposes this workaround:
It is necessary to copy the elements individually to the destination
slice. This example converts a slice of int to a slice of interface{}:
t := []int{1, 2, 3, 4}
s := make([]interface{}, len(t))
for i, v := range t {
s[i] = v
}
Though in your case the right solution may be different.
Interfaces are dynamic. Composite types that involve interfaces are not.
UserCreatedEntity is an interface, and Idea satisfies the interface, so you can return an Idea from a function whose signature has a return type of UserCreatedEntity.
[]UserCreatedEntity is a slice of UserCreatedEntity, not an interface. The only type that can be returned is []UserCreatedEntity. []Idea is a different type (slice of Idea). You can fill a []UserCreatedEntity with Idea elements, because each element is of type UserCreatedEntity, which again is an interface and Idea is allowed there.
Similarly, func() UserCreatedEntity is a type "function which returns UserCreatedEntity". You cannot subsitute a func() Idea because that is a different type. But you can return an Idea from a func() UserCreatedEntity because an Idea is a UserCreatedEntity.
If you weren't using Scan here, which presumably uses reflection, the fix would be to declare your local slice as []UserCreatedEntity instead of []Idea. Since you are using Scan, you instead must scan into a []Idea, then iterate over it to copy all the elements to a []UserCreatedEntity and return that.
I am trying to create a Map with String and functions as key and Value. It works if all functions are of same signature but my requirement is to store functions of different signature in the same map. Is this possible in Go?
package main
import "fmt"
func main() {
functions := buildFunctions()
f := functions["isInValid"]
// f("hello")
}
func buildFunctions() map[string]func() bool {
functions := map[string]func() bool{
"isInValid": isInValid,
"isAvailable": isAvailable,
}
return functions
}
func isInValid(s string) bool {
fmt.Println("Invalid ", s)
return true
}
func isAvailable(s string, s1 string) bool {
return true
}
https://play.golang.org/p/ocwCgEpa_0G
Go is a strongly typed language. So, it's not possible the way, it is possible with, say python. But just like python, once you do this, you loose the benefit of compile time error checks, and your runtime error checking has to be full-proof.
Here's what you can do:
Use map[string]interface{} type for your function map, which enables you to store anything. But then you are responsible to correctly type assertion at the time of calling. Problem is that, in most cases, if while calling a function, you could know the type of function, may be you might not need a map in the first place.
Use a map[string]string or map[string]interface{} as the argument, and return type in all the functions that are supposed to go into this map. Or at least put all the variable arguments into this map.
eg.
map[string](func (name string, age int, other_attributes
map[string]interface{}) (map[string]interface{}, error))
But again, each function call should provide the correct arguments, and there should also be checks inside the functions, to see (with non-panic version of map lookup), if the parameters are correctly provided, if not, you can return a custom error like ErrInvalidParametersPassed. (playing the role of an interpreter). But you will still have lesser chances of messing up, compared to first option. As the burden of type assertion will be on the function implementation, and not the caller. Caller just needs to fetch it's required values, which it anyways needs to know about.
But yet, best option would be to redesign your actual solution in a way, so that it can be done without going this road. As #bereal suggested in comments, it's good to have separate maps if possible, or maybe use a superset of arguments if they aren't too different, or too many. If there are just a few arguments, even switch case could be clean enough. Look for ways that cheat/bypass compile time checks, when you are truly convinced that there is no other elegant way.
Suppose I want to have a method that should either return a chan or a slice. For example, I need a chan if I want to "follow" a file as new lines come, and a slice if I just want to read and return existing lines.
In both cases I will only have to iterate through this return value. Here is an abstract example in Python (which has nothing to do with files but sort of shows the idea):
def get_iterable(self):
if self.some_flag:
return (x for x in self.some_iterable)
return [x for x in self.some_iterable]
def do_stuff(self):
items = self.get_iterable()
for item in items:
self.process(item)
Now, I have a difficulty doing this in Go. I suppose I should look for something like an "iterable interface" which I should return, but I failed to google up some ready-to-use solutions (sorry if it's just my poor googling skills).
What is the best way to do what I want? Or, maybe, the whole design is "bad" for Go and I should consider something else?
Or, maybe, the whole design is "bad" for Go and I should consider something else?
While you could build some interface on top of the types so that you can deal with them as if they were the same I would say it's a poor choice. The far simpler one is to take advantage of multiple return types and define your func with chan myType, []myType, error for it's return then just use 3 way if-else to check for error, followed by chan or slice. Read of the channel like you normally would, iterate the slice like you normally would. Put the code that does work on myType in a helper method so you can call it from both control flows.
My money says this is no more code and it's also far more straight forward. I don't have to read through some abstraction to understand that I have a channel and the inherit complications that come along with it (chan and a slice are incongruous so trying to model them the same sounds like a nightmare), instead you just have an extra step in the programs control flow.
I'm kinda late to the party, but if you really need some "abstract iterable", you could create an interface like this:
type Iterable interface {
Next() (int, error)
}
(Inspired by sql.Rows.)
Then, you could use it like this:
for n, err := iter.Next(); err != nil; n, err = iter.Next() {
fmt.Println(n)
}
For iteration I usually follow pattern found in sql.Rows and bufio.Scanner. Both have a next-equivalent function returning bool, indicating whether next item has been successfully fetched. Then there's a separate method to access the value and error. This pattern lets you write very clean for loops without complex conditions (and without using break or continue statements) and moves error handling outside of the loop.
If you were to abstract your line input, you could for example create an interface like this:
type LineScanner interface {
Scan() bool
Text() string
Err() error
}
This would give you and abstract line source reader. As a bonus, by using exactly these method names you would make bufio.Scanner instantly implementing your interface, so you could use it along with your own types, for example tail-like reader mentioned in your question.
Fuller example:
package main
import (
"bufio"
"fmt"
"strings"
)
type LineScanner interface {
Scan() bool
Text() string
Err() error
}
func main() {
var lr LineScanner
// Use scanner from bufio package
lr = bufio.NewScanner(strings.NewReader("one\ntwo\nthree!\n"))
// Alternatively you can provide your own implementation of LineScanner,
// for example tail-like, blocking on Scan() until next line appears.
// Very clean for loop, isn't it?
for lr.Scan() {
// Handle next line
fmt.Println(lr.Text())
}
// Check if no error while reading
if lr.Err() != nil {
fmt.Println("Error:", lr.Err())
}
}
http://play.golang.org/p/LRbGWj9_Xw
Go has no unions. But unions are necessary in many places. XML makes excessive use of unions or choice types. I tried to find out, which is the preferred way to work around the missing unions. As an example I tried to write Go code for the non terminal Misc in the XML standard which can be either a comment, a processing instruction or white space.
Writing code for the three base types is quite simple. They map to character arrays and a struct.
type Comment Chars
type ProcessingInstruction struct {
Target *Chars
Data *Chars
}
type WhiteSpace Chars
But when I finished the code for the union, it got quite bloated with many redundant functions. Obviously there must be a container struct.
type Misc struct {
value interface {}
}
In order to make sure that the container holds only the three allowed types I made the value private and I had to write for each type a constructor.
func MiscComment(c *Comment) *Misc {
return &Misc{c}
}
func MiscProcessingInstruction (pi *ProcessingInstruction) *Misc {
return &Misc{pi}
}
func MiscWhiteSpace (ws *WhiteSpace) *Misc {
return &Misc{ws}
}
In order to be able to test the contents of the union it was necessary to write three predicates:
func (m Misc) IsComment () bool {
_, itis := m.value.(*Comment)
return itis
}
func (m Misc) IsProcessingInstruction () bool {
_, itis := m.value.(*ProcessingInstruction)
return itis
}
func (m Misc) IsWhiteSpace () bool {
_, itis := m.value.(*WhiteSpace)
return itis
}
And in order to get the correctly typed elements it was necessary to write three getters.
func (m Misc) Comment () *Comment {
return m.value.(*Comment)
}
func (m Misc) ProcessingInstruction () *ProcessingInstruction {
return m.value.(*ProcessingInstruction)
}
func (m Misc) WhiteSpace () *WhiteSpace {
return m.value.(*WhiteSpace)
}
After this I was able to create an array of Misc types and use it:
func main () {
miscs := []*Misc{
MiscComment((*Comment)(NewChars("comment"))),
MiscProcessingInstruction(&ProcessingInstruction{
NewChars("target"),
NewChars("data")}),
MiscWhiteSpace((*WhiteSpace)(NewChars(" \n")))}
for _, misc := range miscs {
if (misc.IsComment()) {
fmt.Println ((*Chars)(misc.Comment()))
} else if (misc.IsProcessingInstruction()) {
fmt.Println (*misc.ProcessingInstruction())
} else if (misc.IsWhiteSpace()) {
fmt.Println ((*Chars)(misc.WhiteSpace()))
} else {
panic ("invalid misc");
}
}
}
You see there is much code looking almost the same. And it will be the same for any other union. So my question is: Is there any way to simplify the way to deal with unions in Go?
Go claims to simplify programing work by removing redundancy. But I think the above example shows the exact opposite. Did I miss anything?
Here is the complete example: http://play.golang.org/p/Zv8rYX-aFr
As it seems that you're asking because you want type safety, I would firstly argue that your initial
solution is already unsafe as you have
func (m Misc) Comment () *Comment {
return m.value.(*Comment)
}
which will panic if you haven't checked IsComment before. Therefore this solution has no benefits over
a type switch as proposed by Volker.
Since you want to group your code you could write a function that determines what a Misc element is:
func IsMisc(v {}interface) bool {
switch v.(type) {
case Comment: return true
// ...
}
}
That, however, would bring you no compiler type checking either.
If you want to be able to identify something as Misc by the compiler then you should
consider creating an interface that marks something as Misc:
type Misc interface {
ImplementsMisc()
}
type Comment Chars
func (c Comment) ImplementsMisc() {}
type ProcessingInstruction
func (p ProcessingInstruction) ImplementsMisc() {}
This way you could write functions that are only handling misc. objects and get decide later
what you really want to handle (Comments, instructions, ...) in these functions.
If you want to mimic unions then the way you wrote it is the way to go as far as I know.
I think this amount of code might be reduced, e.g. I personally do not think that safeguarding type Misc against containing "illegal" stuff is really helpful: A simple type Misc interface{} would do, or?
With that you spare the constructors and all the Is{Comment,ProcessingInstruction,WhiteSpace} methods boil down to a type switch
switch m := misc.(type) {
Comment: fmt.Println(m)
...
default: panic()
}
Thats what package encoding/xml does with Token.
I am not sure to understand your issue. The 'easy' way to do it would be like the encoding/xml package with interface{}. If you do not want to use interfaces, then you can do something like you did.
However, as you stated, Go is a typed language and therefore should be use for typed needs.
If you have a structured XML, Go can be a good fit, but you need to write your schema. If you want a variadic schema (one given field can have multiple types), then you might be better off with an non-typed language.
Very useful tool for json that could easily rewritten for xml:
http://mholt.github.io/json-to-go/
You give a json input and it gives you the exact Go struct. You can have multiple types, but you need to know what field has what type. If you don't, you need to use the reflection and indeed you loose a lot of the interest of Go.
TL;DR You don't need a union, interface{} solves this better.
Unions in C are used to access special memory/hardware. They also subvert the type system. Go does not have the language primitives access special memory/hardware, it also shunned volatile and bit-fields for the same reason.
In C/C++ unions can also be used for really low level optimization / bit packing. The trade off: sacrifice the type system and increase complexity in favor of saving some bits. This of course comes with all the warnings about optimizations.
Imagine Go had a native union type. How would the code be better? Rewrite the code with this:
// pretend this struct was a union
type MiscUnion struct {
c *Comment
pi *ProcessingInstruction
ws *WhiteSpace
}
Even with a builtin union accessing the members of MiscUnion requires a runtime check of some kind. So using an interface is no worse off. Arguably the interface is superior as the runtime type checking is builtin (impossible to get wrong) and has really nice syntax for dealing with it.
One advantage of a union type is static type check to make sure only proper concrete types where put in a Misc. The Go way of solving this is "New..." functions, e.g. MiscComment, MiscProcessingInstruction, MiscWhiteSpace.
Here is a cleaned up example using interface{} and New* functions: http://play.golang.org/p/d5bC8mZAB_
A package I am using, gosqlite, has a method with a variadic parameter where its type is the empty interface.
func (s *Stmt) Exec(args ...interface{}) os.Error
I can call this fine if explicitly pass individual parameters:
statement := blah()
error := statement.Exec("hello", 3.0, true) // works fine
However, as the variadic parameter corresponds to placeholders within the in operator of my SQL statement's select, the number of these placeholders is not known at compile time but dynamically changes at run time depending upon what the user is doing. E.g. I end up with SQL akin to the following if the user enters four values:
SELECT * FROM sky WHERE name IN (?,?,?,?)
So naturally I would like to call the Exec method with a slice of strings:
var values []string = getValuesFromUser()
statement := createStatementWithSufficientNumberOfPlaceholders(len(values))
_ := statement.Exec(values...) // compiler doesn't like this
This does not compile. I can get around this problem by creating an empty interface slice and copying the references over:
values2 := make([]interface{}, len(values))
for index, value := range values { values2[index] = value }
_ := statement.Exec(values2...) // compiler happy but I'm not
And this works fine but it feels a bit clunky. I was wondering if there was some trick to be able to pass values directly to this function or, failing that, a neater way of converting the string slice to an empty interface one?
Many thanks.
There is no way to pass a []string directly to a ...interface{} parameter. Doing this requires a linear time copy (with n + 1 allocations!). If the language hid this from you, it would be a significant hidden cost. Normally, passing a slice to a variadic argument just passes the slice into the function.
As for other ways of doing this, you could make it cleaner by writing a function that takes a []string and returns the corresponding []interface{}. Of course, you'll have to write it again for each []T -> []interface{} conversion you want to do, but its a rather short function, and all that changes is the signature. You could use reflection, which comes with an inherent runtime cost, to make the function "generic", such as in:
valuesVal := reflect.ValueOf(values)
...
for i := range values2 { values2[i] = valuesVal.Index(i).Interface() }
I don't have an answer. And I don't suppose there is one since even built-in and variadic copy and append have the same (or compatible concrete) element type "blockhead", but I have two obvious suggestions:
do not return []string from getValuesFromUser() (i.e. pass still unadorned []interface{}),
on the other type end wrap calls to statement.Exec() with a func making []string to []interface{} conversion.
Or on the same, third, obvious note extend type statement with Exec(args ...string).
P.S. I haven't made any benchmarks myself but I don't think this kind of conversion is highly expensive as interface{} feels like a reference type and compiler is probably doing some dirty trickstery behind the curtain... then again perhaps not, though, I'd be happy, too, to learn of an actual solution.
You need to pass a varargs slice of interface{} type like this to the method.
var paramArray []interface{}
paramArray = append(paramArray, "test1")
paramArray = append(paramArray, "test2")
varargsFunc(paramArray...)