I'm writing code that allows data access from a database. However, I find myself repeating the same code for similar types and fields. How can I write generic functions for the same?
e.g. what I want to achieve ...
type Person{FirstName string}
type Company{Industry string}
getItems(typ string, field string, val string) ([]interface{}) {
...
}
var persons []Person
persons = getItems("Person", "FirstName", "John")
var companies []Company
cs = getItems("Company", "Industry", "Software")
So you're definitely on the right track with the idea of returning a slice of nil interface types. However, you're going to run into problems when you try accessing specific members or calling specific methods, because you're not going to know what type you're looking for. This is where type assertions are going to come in very handy. To extend your code a bit:
getPerson(typ string, field string, val string) []Person {
slice := getItems(typ, field, val)
output := make([]Person, 0)
i := 0
for _, item := range slice {
// Type assertion!
thing, ok := item.(Person)
if ok {
output = append(output, thing)
i++
}
}
return output
}
So what that does is it performs a generic search, and then weeds out only those items which are of the correct type. Specifically, the type assertion:
thing, ok := item.(Person)
checks to see if the variable item is of type Person, and if it is, it returns the value and true, otherwise it returns nil and false (thus checking ok tells us if the assertion succeeded).
You can actually, if you want, take this a step further, and define the getItems() function in terms of another boolean function. Basically the idea would be to have getItems() run the function pass it on each element in the database and only add that element to the results if running the function on the element returns true:
getItem(critera func(interface{})bool) []interface{} {
output := make([]interface{}, 0)
foreach _, item := range database {
if criteria(item) {
output = append(output, item)
}
}
}
(honestly, if it were me, I'd do a hybrid of the two which accepts a criteria function but also accepts the field and value strings)
joshlf13 has a great answer. I'd expand a little on it though to maintain some additional type safety. instead of a critera function I would use a collector function.
// typed output array no interfaces
output := []string{}
// collector that populates our output array as needed
func collect(i interface{}) {
// The only non typesafe part of the program is limited to this function
if val, ok := i.(string); ok {
output = append(output, val)
}
}
// getItem uses the collector
func getItem(collect func(interface{})) {
foreach _, item := range database {
collect(item)
}
}
getItem(collect) // perform our get and populate the output array from above.
This has the benefit of not requiring you to loop through your interface{} slice after a call to getItems and do yet another cast.
Related
I'm trying to test around an SQL query wherein one of the arguments is a gosnowflake.Array (essentially a wrapper to a slice) using the go-sqlmock package. Normally, something like this requires me to create a value converter, which I have included:
func (opt arrayConverterOption[T]) ConvertValue(v any) (driver.Value, error) {
casted, ok := v.(*[]T)
if ok {
Expect(*casted).Should(HaveLen(len(opt.Expected)))
for i, c := range *casted {
Expect(c).Should(Equal(opt.Expected[i]))
}
} else {
fmt.Printf("Type: %T\n", v)
return v, nil
}
return "TEST_RESULT", nil
}
Now, this function is called for every argument submitted to the query. I use it to test the correctness of the values in the slice or pass the argument through if it isn't one. The problem I'm having is that, when I create a arrayConverterOption[string] and give it a gosnowflake.Array(["A", "B", "C"]) as an argument, the type assertion fails because gosnowflake.Array returns an internal dynamic type, *stringArray, which is defined as a *[]string.
So you can see my dilemma here. On the one hand, I can't convert v because it's an interface{} and I can't alias v because the inner type is not *[]string, but *stringArray. So then, what should I do here?
I didn't find a way to do this without resulting to reflection. However, with reflction I did manage it:
var casted []T
var ok bool
value := reflect.ValueOf(v)
if value.Kind() == reflect.Pointer {
if inner := value.Elem(); inner.Kind() == reflect.Slice {
r := inner.Convert(reflect.TypeOf([]T{})).Interface()
casted, ok = r.([]T)
}
}
So, this code checks specifically for anything that is a pointer to a slice, which my dynamic type is. Then it uses reflection to convert the inner object to the slice type I was expecting. After that, I call Interface() on the result to get the interface{} from the reflected value and then cast it to a []T. This succeeds. If it doesn't then I'm not working with one of those dynamically typed slices and I can handle the type normally.
I'm trying to write in Go custom cache for Google DataStore (more precisely - a wrapper around one of existing cache libraries). At cache initialisation, it should accept any custom type of struct (with appropriately-defined datastore fields), which then would be the basis for all items stored. The idea is that cache can be created/initialised for various types which reflect the structure of a particular DataStore entry (CustomEntry)
Approach 1 - store reflect.Type and use it. Problem encountered - can't iterate over a slice of a custom type
type CustomEntry struct {
Data struct {
name string `datastore:"name,noindex"`
address []string `datastore:"address,noindex"`
} `datastore:"data,noindex"`
}
func (cache *MyCache) CacheData(dataQuery string, dataType reflect.Type) {
slice := reflect.MakeSlice(reflect.SliceOf(dataType), 10, 10)
if keys, err := DataStoreClient.GetAll(cache.ctx, datastore.NewQuery(dataQuery), &slice); err != nil {
//handle error
} else {
for i, dataEntry:= range slice {
// ERROR: Cannot range over 'slice' (type Value)
cache.Set(keys[i].Name, dataEntry)
}
}
//usage: Cache.CacheData("Person", reflect.TypeOf(CustomEntry{})
Approach 2 - accept an array of interfaces as arguments. Problem encountered = []CustomEntry is not []interface{}
func (cache *MyCache) CacheData(dataQuery string, dataType []interface{}) {
if keys, err := DataStoreClient.GetAll(cache.ctx, datastore.NewQuery(dataQuery), &dataType); err != nil {
//handle error
} else {
for i, dataEntry:= range slice {
// this seems to work fine
cache.Set(keys[i].Name, dataEntry)
}
}
//usage:
var dataType []CustomEntry
Cache.CacheData("Person", data)
// ERROR: Cannot use 'data' (type []CustomEntry) as type []interface{}
Any suggestions would be highly appreciated.
I have found a solution and thought it might be worth sharing in case anyone else has a similar problem.
The easiest way is to initiate a slice of structs which the DataStore is expected to receive, and then to pass a pointer to it as an argument (interface{}) into the desired function. DataStore, similarly to a few unmarshaling functions (I have tried with JSON package) will be able to successfully append the data to it.
Trying to dynamically create the slice within the function, given a certain Type, which would be then accepted by a function (such as DataStore client) might be quite difficult (I have not managed to find a way to do it). Similarly, passing a slice of interfaces (to allow for easy iteration) only complicates things.
Secondly, in order to iterate over the data (e.g. to store it in cache), it is necessary to:
(1) retrieve the underlying value of the interface (i.e. the pointer itself) - this can be achieved using reflect.ValueOf(pointerInterface),
(2) dereference the pointer so that we obtain access to the underlying, iterable slice of structs - this can be done by invoking .Elem(),
(3) iterate over the underlying slice using .Index(i) method (range will not accept an interface, even if the underlying type is iterable).
Naturally, adding a number of switch-case statements might be appropriate to ensure that any errors are caught rather than cause a runtime panic.
Hence the following code provides a working solution to the above problem:
In main:
var data []customEntry
c.CacheData("Person",&data)
And the function itself:
func (cache *MyCache) CacheData(dataQuery string, data interface{}) error {
if keys, err := DataStoreClient.GetAll(cache.ctx, datastore.NewQuery(dataQuery), data); err != nil {
return err
} else {
s := reflect.ValueOf(data).Elem()
for i := 0; i < s.Len(); i++ {
cache.Set(keys[i].Name, s.Index(i), 1)
}
}
}
Cannot Range Over List Type Interface {} In Function Using Go.
for me is important then i execute for in a function.
How can fix?
package main
import (
"fmt"
)
type MyBoxItem struct {
Name string
}
type MyBox struct {
Items []MyBoxItem
}
func (box *MyBox) AddItem(item MyBoxItem) []MyBoxItem {
box.Items = append(box.Items, item)
return box.Items
}
func PrintCustomArray(list interface{}) interface{} {
//items := reflect.ValueOf(list)
for _, v := range list {
fmt.Println(v.Key,v.Value)
}
return 0
}
func main() {
items := []MyBoxItem{}
item := MyBoxItem{Name: "Test Item 1"}
box := MyBox{items}
box.AddItem(item)
fmt.Println((box.Items))
PrintCustomArray(box.Items)
}
https://play.golang.org/p/ZcIBLMliq3
Error : cannot range over list (type interface {})
How can fix?
Note
The answer below describes, in broad strokes, 2 possible approaches: using interfaces, and using specific types. The approach focusing on interfaces is mentioned for completeness sake. IMHO, the case you've presented is not a viable use-case for interfaces.
Below, you'll find a link to a playground example that uses both techniques. It should be apparent to anyone that the interface approach is too cumbersome if for this specific case.
Quite apart from the fact that you don't really seem to be too familiar with how loops work in go (v.Key and v.Value are non-existent fields for example), I'll attempt to answer your question.
You are passing a list to your function, sure enough, but it's being handled as an interface{} type. That means your function accepts, essentially, any value as an argument. You can't simply iterate over them.
What you can do is use type assertions to convert the argument to a slice, then another assertion to use it as another, specific interface:
type Item interface{
key() string
val() string
}
func (i MyBoxItem) key() string {
return i.Key
}
func (i MyBoxItem) val() string {
return i.Value
}
func PrintCustomArray(list interface{}) error {
listSlice, ok := list.([]interface{})
if !ok {
return fmt.Errorf("Argument is not a slice")
}
for _, v := range listSlice {
item, ok := v.(Item)
if !ok {
return fmt.Errorf("element in slice does not implement the Item interface")
}
fmt.Println(item.key(), item.val())
}
return nil
}
But let's be honest, a function like this only works if a slice is passed as an argument. So having that first type assertion in there makes no sense whatsoever. At the very least, changing the function to something like this makes a lot more sense:
func PrintCustomArray(list []interface{})
Then, because we're not expecting an array as such, but rather a slice, the name should be changed to PrintCustomSlice.
Lastly, because we're using the same type assertion for every value in the slice, we might as well change the function even more:
// at this point, we'll always return 0, which is pointless
// just don't return anything
func PrintCustomSlice(list []Item) {
for _, v := range list {
fmt.Println(v.key(), v.val())
}
}
The advantages of a function like this is that it can still handle multiple types (all you have to do is implement the interface). You don't need any kind of expensive operations (like reflection), or type assertions.
Type assertions are very useful, but in a case like this, they merely serve to hide problems that would otherwise have resulted in a compile-time error. Go's interface{} type is a very useful thing, but you seem to be using it to get around the type system. If that's what you want to achieve, why use a typed language in the first place?
Some closing thoughts/remarks: If your function is only going to be used to iterate over specific "thing", you don't need the interfaces at all, simply specify the type you're expecting to be passed to the function in the first place. In this case that would be:
func PrintCustomSlice(list []MyBoxItem) {
for _, v := range list {
fmt.Println(v.Key, v.Value)
}
}
Another thing that I've noticed is that you seem to be exporting everything (all functions, types, and fields start with a capital letter). This, in go, is considered bad form. Only export what needs to be public. In the main package, that usually means you're hardly export anything.
Lastly, as I mentioned at the start: you don't seem to have a firm grasp on the basics just yet. I'd strongly recommend you go through the interactive tour. It covers the basics nicely, but shows you the features of the language at a decent pace. It doesn't take long, and is well worth taking a couple of hours to complete
Playground demo
It's possible to implement PrintCustomArray using the reflect package, but most experienced Go programmers will write a simple for loop:
for _, i := range box.Items {
fmt.Println("Name:", i.Name)
}
https://play.golang.org/p/RhubiCpry0
You can also encapsulate it in a function:
func PrintCustomArray(items []MyBoxItem) {
for _, i := range items {
fmt.Println("Name:", i.Name)
}
}
https://play.golang.org/p/c4EPQIx1AH
Here since you are returning box.Items from AddItem(), Items is of the type []MyBoxItem , so list should be of type []MyBoxItem .Moreover you are returning 0 in PrintCustomArray and the return type you have set is {}interface.
func PrintCustomArray(list []MyBoxItem) {
//items := reflect.ValueOf(list)
for i, v := range list {
fmt.Println(i, v)
}
//return 0
}
Again, MyBoxItem struct has only one variable named Name so v.key v.value won't make any sense.
This is what the proper code should look like https://play.golang.org/p/ILoUwEWv6Y .
You need to clear your understanding about interfaces in go. This might help https://golang.org/doc/effective_go.html#interfaces_and_types .
I have custom types Int64Array, Channel and ChannelList like:
type Int64Array []int64
func (ia *Int64Array) Scan(src interface{}) error {
rawArray := string(src.([]byte))
if rawArray == "{}" {
*ia = []int64{}
} else {
matches := pgArrayPat.FindStringSubmatch(rawArray)
if len(matches) > 1 {
for _, item := range strings.Split(matches[1], ",") {
i, _ := strconv.ParseInt(item, 10, 64)
*ia = append(*ia, i)
}
}
}
return nil
}
func (ia Int64Array) Value() (driver.Value, error) {
var items []string
for _, item := range ia {
items = append(items, strconv.FormatInt(int64(item), 10))
}
return fmt.Sprintf("{%s}", strings.Join(items, ",")), nil
}
type Channel int64
type ChannelList []Channel
How can I embed Int64Array to ChannelList such that I can call Scan and Value methods on it? I tried the following:
type ChannelList []Channel {
Int64Array
}
but I'm getting syntax error. What's important is to make sure ChannelList items are of type Channel, if this isn't possible via embedding I might just create stand-alone functions to be called by both ChannelList and Int64Array.
An anonymous (or embedded field) is found in a struct (see struct type), not in a type alias (or "type declaration").
You cannot embed a type declaration within another type declaration.
Plus, as illustrated by the answers to "Go: using a pointer to array", you shouldn't be using pointers to slice, use directly the slice themselves (passed by value).
Wessie kindly points out in the comments that (ia *Int64Array) Scan() uses pointer to a slice in order to mutate the underlying array referenced by said slice.
I would prefer returning another slice instead of mutating the existing one.
That being said, the Golang Code Review does mention:
If the receiver is a struct, array or slice and any of its elements is a pointer to something that might be mutating, prefer a pointer receiver, as it will make the intention more clear to the reader.
Is there a way to write a generic array/slice deduplication in go, for []int we can have something like (from http://rosettacode.org/wiki/Remove_duplicate_elements#Go ):
func uniq(list []int) []int {
unique_set := make(map[int] bool, len(list))
for _, x := range list {
unique_set[x] = true
}
result := make([]int, len(unique_set))
i := 0
for x := range unique_set {
result[i] = x
i++
}
return result
}
But is there a way to extend it to support any array? with a signature like:
func deduplicate(a []interface{}) []interface{}
I know that you can write that function with that signature, but then you can't actually use it on []int, you need to create a []interface{} put everything from the []int into it, pass it to the function then get it back and put it into a []interface{} and go through this new array and put everything in a new []int.
My question is, is there a better way to do this?
While VonC's answer probably does the closest to what you really want, the only real way to do it in native Go without gen is to define an interface
type IDList interface {
// Returns the id of the element at i
ID(i int) int
// Returns the element
// with the given id
GetByID(id int) interface{}
Len() int
// Adds the element to the list
Insert(interface{})
}
// Puts the deduplicated list in dst
func Deduplicate(dst, list IDList) {
intList := make([]int, list.Len())
for i := range intList {
intList[i] = list.ID(i)
}
uniques := uniq(intList)
for _,el := range uniques {
dst.Insert(list.GetByID(el))
}
}
Where uniq is the function from your OP.
This is just one possible example, and there are probably much better ones, but in general mapping each element to a unique "==able" ID and either constructing a new list or culling based on the deduplication of the IDs is probably the most intuitive way.
An alternate solution is to take in an []IDer where the IDer interface is just ID() int. However, that means that user code has to create the []IDer list and copy all the elements into that list, which is a bit ugly. It's cleaner for the user to wrap the list as an ID list rather than copy, but it's a similar amount of work either way.
The only way I have seen that implemented in Go is with the clipperhouse/gen project,
gen is an attempt to bring some generics-like functionality to Go, with some inspiration from C#’s Linq and JavaScript’s underscore libraries
See this test:
// Distinct returns a new Thing1s slice whose elements are unique. See: http://clipperhouse.github.io/gen/#Distinct
func (rcv Thing1s) Distinct() (result Thing1s) {
appended := make(map[Thing1]bool)
for _, v := range rcv {
if !appended[v] {
result = append(result, v)
appended[v] = true
}
}
return result
}
But, as explained in clipperhouse.github.io/gen/:
gen generates code for your types, at development time, using the command line.
gen is not an import; the generated source becomes part of your project and takes no external dependencies.
You could do something close to this via an interface. Define an interface, say "DeDupable" requiring a func, say, UniqId() []byte, which you could then use to do the removing of dups. and your uniq func would take a []DeDupable and work on it