I want to use Scan() in package sql, but the number of columns, and hence the number of arguments, will change at runtime. This is the signature of Scan():
func (rs *Rows) Scan(dest ...interface{}) error
According to the documentation, *interface{} is one of the types accepted by Scan(). So I want to create a slice of []*interface{} and that expand as arguments.
This is what I thought would work:
func query(database *sql.DB) {
rows, _ := database.Query("select * from testTable")
for rows.Next() {
data := make([]*interface{}, 2)
err := rows.Scan(data...) // Compilation error
fmt.Printf("%v%v\n", *data[0], *data[1])
if err != nil {
fmt.Println(err.Error())
}
}
}
Compilation fails with cannot use data (type []*interface {}) as type []interface {} in argument to rows.Scan. I thought that data... would expand to &data[0], &data[1], but apparently not. I don't understand the error message. *interface{} is compatible with interface{}, so why can't I expand the slice of pointers to interface types?
This works:
func query(database *sql.DB) {
rows, _ := database.Query("select * from testTable")
for rows.Next() {
data := make([]*interface{}, 2)
err := rows.Scan(&data[0], &data[1]) // Only changed this line
fmt.Printf("%v%v\n", *data[0], *data[1]) // Outputs "[48][116 101 120 116]"
if err != nil {
fmt.Println(err.Error())
}
}
}
I can't use this however, because the number of columns is unknown at compile time. How can I write this code so that I can pass a variable number of *interface{} to rows.Scan()?
First, you must not use []*interface{} slice of pointers to interface rather than []interface{} where the interfaces are pointers. []*interface{} is different from []interface{}. Just create a slice of interfaces where each element is a pointer to a concrete type.
Here is a snippet how you would do this.
var x int
var s string
data := []interface{}{&x, &s}
rows.Scan(data...)
Note on the use of the ... spread operator.
Here are some related questions that will explain a bit more:
golang: slice of struct != slice of interface it implements?
Cannot convert []string to []interface {}
If you really want to pass a []*interface{} (perhaps you don't know the concrete types of the output) you must first wrap each *interface{} in a interface{}:
values := make([]interface{}, columnsCount)
for i := range values {
values[i] = new(interface{})
}
Individual values passed into a ...interface{} parameter are automatically wrapped in a interface{}, but just like []int... won't satisfy ...interface{}, neither will []*interface{}....
Related
I'm trying to extract all of the values for a struct into a string slice.
func structValues(item Item) []string {
values := []string{}
e := reflect.ValueOf(&item).Elem()
for i := 0; i < e.NumField(); i++ {
fieldValue := e.Field(i).Interface()
values = append(values, fmt.Sprintf("%#v", fieldValue))
}
return values
}
I'd like to use this function with any struct, so I thought I could just change the type signature to func structValues(item interface{}) but then I got a panic:
panic: reflect: call of reflect.Value.NumField on interface Value
Working example: https://repl.it/#fny/stackoverflow61719532
I'd like to use this function with any struct ...
You can do this, but note that it gives up type-safety. Moreover, the only way to do this is to allow a call with any type, not just any type that is some structure type, so you have to check that what you got was in fact some struct type:
func structValues(item interface{}) {
if reflect.ValueOf(item).Kind() != reflect.Struct {
... do something here ...
}
Having made that check—or deferring it slightly, or omitting it to allow reflect to panic instead—you then need to replace reflect.ValueOf(&item).Elem() with the simpler reflect.ValueOf(item).
If you wish to allow pointers to structures as well as actual structures, you can make that happen pretty simply by using reflect.Indirect first. The result is:
func structValues(item interface{}) []string {
e := reflect.Indirect(reflect.ValueOf(item))
if e.Kind() != reflect.Struct {
panic("not a struct")
}
values := []string{}
for i := 0; i < e.NumField(); i++ {
fieldValue := e.Field(i).Interface()
values = append(values, fmt.Sprintf("%#v", fieldValue))
}
return values
}
Leave out the reflect.Indirect if you want to make sure that callers do their own indirection when they have a pointer.
(Note that the panic here is not very friendly. If you want proper debugging, consider either just printing the struct directly with %v or %#v, or for something much more thorough, the spew package.)
Complete example here on the Go Playground uses your type Item struct from your own link.
This question already has answers here:
Can I type assert a slice of interface values?
(2 answers)
Closed 3 years ago.
I have a type that implements the stringer interface
// RowID stores the ID of a single row in a table
type RowID []string
// String implements Stringer interface for RowID
func (r RowID) String() string {
return fmt.Sprintf("[%s]", strings.Join(r, ", "))
}
And I have a function that I want to pass a slice of this type (or any other type that implements the Stringer interface) to.
// PrintChanges ...
func PrintChanges(ids []fmt.Stringer) {
for _, id := range ids {
fmt.Println(id)
}
}
However, The go compiler gives me an error:
cannot use rowIDs (type []RowID) as type []fmt.Stringer in argument to PrintChanges
I can pass a RowID to a func that accepts a single fmt.Stringer
func PrintChange(id fmt.Stringer) {
fmt.Println(id)
}
...
PrintChange(RowID{"1", "24"})
But for some reason I am not able to pass a slice of RowID to a func that accepts a slice of fmt.Stringer. What am I missing?
Go Playground
Keep it simple
It is considered okay by professional Go programmers to repeat functions like this for every type, or to have a for loop over every slice you want to print. This is because Go aims to be as easy to read as possible, i.e. a person who reads a chunk of code for the first time should not be asking questions like "which function overload will this function call go to" (common pitfall in C++, Go does not have function overloads). So you can just write in main():
Playground: https://ideone.com/IL3rGR
for _, id := range rowIDs { fmt.Println(id) }
Simple and concise.
Note that fmt.Println(id) does not call your String() function
This is because the fmt library uses the reflect library and hardcodes behavior for the string type, which you are trying to replace. RowID instances are also string instances, the library always prefers string over its type aliases. I would say it is a bug in the library:
Library source: https://golang.org/src/fmt/print.go#L649
// Some types can be done without reflection.
switch f := arg.(type) {
...
case string:
p.fmtString(f, verb)
If you really want to
You can use a function that takes an interface{} and makes a runtime reflect type cast to a slice of Stringers. Note that this means you will not see type mismatches during compilation, only in runtime:
Playground: https://ideone.com/vlrBP9
func castToStringerSlice(iface interface{}) ([]fmt.Stringer, bool /* ok */) {
if reflect.TypeOf(iface).Kind() != reflect.Slice {
return nil, false
}
v := reflect.ValueOf(iface)
stringers := make([]fmt.Stringer, v.Len())
for i := 0; i < v.Len(); i++ {
stringers[i] = v.Index(i)
}
return stringers, true
}
func PrintChanges(iface_ids interface{}) {
ids, ok := castToStringerSlice(iface_ids)
if !ok {
log.Fatal(errors.New("the argument to PrintChanges must be a slice of Stringers"))
}
for _, id := range ids {
fmt.Println(id)
}
}
Resources:
Go documentation: Why does Go not support overloading of methods and operators?
Go documentation: Why does Go not have generic types?
Stack Overflow: Express function that takes any slice
Stack Overflow: Range over interface{} which stores a slice
Go documentation: Package reflect
I need some help with unmarshaling. I have this example code:
package main
import (
"encoding/json"
"fmt"
)
type Obj struct {
Id string `json:"id"`
Data []byte `json:"data"`
}
func main() {
byt := []byte(`{"id":"someID","data":["str1","str2"]}`)
var obj Obj
if err := json.Unmarshal(byt, &obj); err != nil {
panic(err)
}
fmt.Println(obj)
}
What I try to do here - convert bytes to the struct, where type of one field is []byte. The error I get:
panic: json: cannot unmarshal string into Go struct field Obj.data of
type uint8
That's probably because parser already sees that "data" field is already a slice and tries to represent "str1" as some char bytecode (type uint8?).
How do I store the whole data value as one bytes array? Because I want to unmarshal the value to the slice of strings later. I don't include a slice of strings into struct because this type can change (array of strings, int, string, etc), I wish this to be universal.
My first recommendation would be for you to just use []string instead of []byte if you know the input type is going to be an array of strings.
If data is going to be a JSON array with various types, then your best option is to use []interface{} instead - Go will happily unmarshal the JSON for you and you can perform checks at runtime to cast those into more specific typed variables on an as-needed basis.
If []byte really is what you want, use json.RawMessage, which is of type []byte, but also implements the methods for JSON parsing. I believe this may be what you want, as it will accept whatever ends up in data. Of course, you then have to manually parse Data to figure out just what actually IS in there.
One possible bonus is that this skips any heavy parsing because it just copies the bytes over. When you want to use this data for something, you use a []interface{}, then use a type switch to use individual values.
https://play.golang.org/p/og88qb_qtpSGJ
package main
import (
"encoding/json"
"fmt"
)
type Obj struct {
Id string `json:"id"`
Data json.RawMessage `json:"data"`
}
func main() {
byt := []byte(`{"id":"someID","data":["str1","str2", 1337, {"my": "obj", "id": 42}]}`)
var obj Obj
if err := json.Unmarshal(byt, &obj); err != nil {
panic(err)
}
fmt.Printf("%+v\n", obj)
fmt.Printf("Data: %s\n", obj.Data)
// use it
var d []interface{}
if err := json.Unmarshal(obj.Data, &d); err != nil {
panic(err)
}
fmt.Printf("%+v\n", d)
for _, v := range d {
// you need a type switch to deterine the type and be able to use most of these
switch real := v.(type) {
case string:
fmt.Println("I'm a string!", real)
case float64:
fmt.Println("I'm a number!", real)
default:
fmt.Printf("Unaccounted for: %+v\n", v)
}
}
}
Your question is:
convert bytes array to struct with a field of type []byte
But you do not have a bytearray but a string array. Your question is not the same as your example. So let answer your question, there are more solutions possible depending in how far you want to diverge from your original requirements.
One string can be converted to one byte-slice, two strings need first to be transformed to one string. So that is problem one. The second problem are the square-brackets in your json-string
This works fine, it implicitly converts the string in the json-string to a byte-slice:
byt := []byte(`{"id":"someID","data":"str1str2"}`)
var obj Obj
if err := json.Unmarshal(byt, &obj); err != nil {
panic(err)
}
fmt.Println(obj)
I have an interface{} variable and I know it's a pointer to slice:
func isPointerToSlice(val interface{}) bool {
value := reflect.ValueOf(val)
return value.Kind() == reflect.Ptr && value.Elem().Kind() == reflect.Slice
}
But I'm finding difficult to type cast it into an []interface{} variable:
if isPointerToSlice(val) {
slice, worked := reflect.ValueOf(val).Elem().Interface().([]interface{})
// 'worked' is false :(
}
This doesn't work. Any idea how can I solve this?
If you just want to convert a slice to []interface{} you can use something like this:
func sliceToIfaceSlice(val interface{}) []interface{} {
rf := reflect.Indirect(reflect.ValueOf(val)) // skip the pointer
if k := rf.Kind(); k != reflect.Slice && k != reflect.Array {
// panic("expected a slice or array")
return nil
}
out := make([]interface{}, rf.Len())
for i := range out {
out[i] = rf.Index(i).Interface()
}
return out
}
playground
You can simply use type assertion to obtain the value stored in an interface, e.g.
if isPointerToSlice(val) {
var result []interface{}
result = *val.(*[]interface{})
fmt.Println(result)
} else {
fmt.Println("Not *[]interface{}")
}
The type of the value stored in the interface as you claim is pointer to []interface{}, which is *[]interface{}. The result of the type assertion will be a pointer, just dereference it to get the slice []interface{}.
Using short variable declaration:
result := *val.(*[]interface{}) // type of result is []interface{}
Try it on the Go Playground.
Also your attempt also works:
slice, worked := reflect.ValueOf(val).Elem().Interface().([]interface{})
fmt.Println(slice, worked)
Here's the edited the Playground example which proves your solution works.
But using reflection is unnecessary (as it can be done with type assertion).
Also note that *[]interface{} and *[]someOtherType are 2 different types and you can't obtain a value of *[]interface{} if there is something else in val.
Icza's answer is great and will work especially if you can't know for sure you are getting an interface slice, however if you don't want to bother with the reflect package at all and want to keep imported code low, you can use type switching to obtain the same functionality using only built-in methods.
Using this method, you can shorten your code to just:
package main
import (
"fmt"
)
func main() {
s := []interface{}{"one", 2}
p := &s
do(p)
}
func do(val interface{}) {
switch val.(type){
case *[]interface{}:
var result []interface{}
result = *val.(*[]interface{})
fmt.Println(result)
}
}
Playground: http://play.golang.org/p/DT_hb8JcVt
The downside is if you don't know the exact type of slice you are receiving beforehand, then this will not work unless you list all possible types for handling and assertion.
I have custom types Int64Array, Channel and ChannelList like:
type Int64Array []int64
func (ia *Int64Array) Scan(src interface{}) error {
rawArray := string(src.([]byte))
if rawArray == "{}" {
*ia = []int64{}
} else {
matches := pgArrayPat.FindStringSubmatch(rawArray)
if len(matches) > 1 {
for _, item := range strings.Split(matches[1], ",") {
i, _ := strconv.ParseInt(item, 10, 64)
*ia = append(*ia, i)
}
}
}
return nil
}
func (ia Int64Array) Value() (driver.Value, error) {
var items []string
for _, item := range ia {
items = append(items, strconv.FormatInt(int64(item), 10))
}
return fmt.Sprintf("{%s}", strings.Join(items, ",")), nil
}
type Channel int64
type ChannelList []Channel
How can I embed Int64Array to ChannelList such that I can call Scan and Value methods on it? I tried the following:
type ChannelList []Channel {
Int64Array
}
but I'm getting syntax error. What's important is to make sure ChannelList items are of type Channel, if this isn't possible via embedding I might just create stand-alone functions to be called by both ChannelList and Int64Array.
An anonymous (or embedded field) is found in a struct (see struct type), not in a type alias (or "type declaration").
You cannot embed a type declaration within another type declaration.
Plus, as illustrated by the answers to "Go: using a pointer to array", you shouldn't be using pointers to slice, use directly the slice themselves (passed by value).
Wessie kindly points out in the comments that (ia *Int64Array) Scan() uses pointer to a slice in order to mutate the underlying array referenced by said slice.
I would prefer returning another slice instead of mutating the existing one.
That being said, the Golang Code Review does mention:
If the receiver is a struct, array or slice and any of its elements is a pointer to something that might be mutating, prefer a pointer receiver, as it will make the intention more clear to the reader.