golang reflect into []interface{} - go

i wanna create a mini framework which takes a simple struct and creates a full crud out of it. i already started and the "findOne, update,create,delete" is working. not i have a problem to create a findAll method. to be more clear, i dont know how to use reflect to address my ptr to an array of struct.
Here a small example for the findOne function.
type company struct {
Id int
Name string
}
comp.InitModel(newDbConnection(), &comp)
In InitModel i can fill the pointer to the company with the following:
//m.caller = pointer to the ptr to comp (struct)
callerV := reflect.ValueOf(m.caller)
CallerField := callerV.Elem()
var values []interface{}
for _, e := range m.columns {
values = append(values, CallerField.FieldByName(e.name).Addr().Interface())
}
err := r.Scan(values...)
if err != nil {
return err
}
Now i wanna create a findAll method which would be called like this
var companies []company
comp.InitModel(newDbConnection(), &comp)
comp.FindAll(&companies) //in this is the db query and scan
fmt.Println(companies) //here should be the result
But i have a problem to get the reflect with a []interface working.
func (m *Model) FindAll(test []interface{}, c *Condition) error {
//get the sql statement from the struct
stmt := PrepairStmt(m, c)
rows, err := m.db.Query(stmt.selectParse(), c.arguments...)
if err != nil {
return err
}
defer rows.Close()
callerV := reflect.ValueOf(m.caller)
CallerField := callerV.Elem()
for rows.Next() {
var values []interface{}
for _, e := range m.columns {
values = append(values, CallerField.FieldByName(e.name).Addr().Interface())
}
err = rows.Scan(values...)
if err != nil {
return err
}
valuePtr := reflect.New(reflect.TypeOf(test).Elem())
test = reflect.Append(test,reflect.ValueOf(values))
}
return nil
}
Thats were my latest tries.
maybe someone can help me with this.
i would be really thankful

Use interface{} instead of []interface{} as the argument type:
func (m *Model) FindAll(result interface{}, c *Condition) error {
stmt := PrepairStmt(m, c)
rows, err := m.db.Query(stmt.selectParse(), c.arguments...)
if err != nil {
return err
}
defer rows.Close()
// resultv is the result slice
resultv := reflect.ValueOf(result).Elem()
// rowt is the struct type
rowt := resultv.Type().Elem()
// allocate a value for the row
rowv := reflect.New(rowt).Elem()
// collect values for scan
var values []interface{}
for _, e := range m.columns {
values = append(values, rowv.FieldByName(e.name).Addr().Interface())
}
for rows.Next() {
err = rows.Scan(values...)
if err != nil {
return err
}
// Append struct to result slice. Because the struct
// is copied in append, we can reuse the struct in
// this loop.
resultv.Set(reflect.Append(resultv, rowv))
}
return nil
}
Use it like this:
var companies []company
comp.InitModel(newDbConnection(), &comp)
comp.FindAll(&companies) //in this is the db query and scan

Related

Go: Implementing a ManyDecode for a "set" of individual results

I have implemented a very simple Decode method (using gob.Decoder for now) - this works well for single responses - it would even work well for slices, but I need to implement a DecodeMany method where it is able to decode a set of individual responses (not a slice).
Working Decode method:
var v MyType
_ = Decode(&v)
...
func Decode(v interface{}) error {
buf, _ := DoSomething() // func DoSomething() ([]byte, error)
// error handling omitted for brevity
return gob.NewDecoder(bytes.NewReader(buf)).Decode(v)
}
What I'm trying to do for a DecodeMany method is to deal with a response that isn't necessarily a slice:
var vv []MyType
_ = DecodeMany(&vv)
...
func DecodeMany(vv []interface{}) error {
for _, g := range DoSomething() { // func DoSomething() []struct{Buf []bytes}
// Use g.Buf as an individual "interface{}"
// want something like:
var v interface{} /* Somehow create instance of single vv type? */
_ = gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(v)
vv = append(vv, v)
}
return
}
Besides not compiling the above also has the error of:
cannot use &vv (value of type *[]MyType) as type []interface{} in argument to DecodeMany
If you want to modify the passed slice, it must be a pointer, else you must return a new slice. Also if the function is declared to have a param of type []interface{}, you can only pass a value of type []interface{} and no other slice types... Unless you use generics...
This is a perfect example to start using generics introduced in Go 1.18.
Change DecodeMany() to be generic, having a T type parameter being the slice element type:
When taking a pointer
func DecodeMany[T any](vv *[]T) error {
for _, g := range DoSomething() {
var v T
if err := gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(&v); err != nil {
return err
}
*vv = append(*vv, v)
}
return nil
}
Here's a simple app to test it:
type MyType struct {
S int64
}
func main() {
var vv []MyType
if err := DecodeMany(&vv); err != nil {
panic(err)
}
fmt.Println(vv)
}
func DoSomething() (result []struct{ Buf []byte }) {
for i := 3; i < 6; i++ {
buf := &bytes.Buffer{}
v := MyType{S: int64(i)}
if err := gob.NewEncoder(buf).Encode(v); err != nil {
panic(err)
}
result = append(result, struct{ Buf []byte }{buf.Bytes()})
}
return
}
This outputs (try it on the Go Playground):
[{3} {4} {5}]
When returning a slice
If you choose to return the slice, you don't have to pass anything, but you need to assign the result:
func DecodeMany[T any]() ([]T, error) {
var result []T
for _, g := range DoSomething() {
var v T
if err := gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(&v); err != nil {
return result, err
}
result = append(result, v)
}
return result, nil
}
Using it:
vv, err := DecodeMany[MyType]()
if err != nil {
panic(err)
}
fmt.Println(vv)
Try this one on the Go Playground.

How to set slice interface values with reflection

I would like to build a function that takes a generic pointer array and fill that list based on mongo results.
I don't know how to set the value I got from mongo into my pointer array. In the below attempt, program panics with following error : reflect.Set: value of type []interface {} is not assignable to type []Person
When I print total / documents found, it corresponds to what I am expecting. So I think question is about reflection.
func getListWithCount(ctx context.Context, receiver interface{}) (int, error) {
//my mongo query here
var mongoResp struct {
Total int `bson:"total"`
Documents interface{} `bson:"documents"`
}
if err := cursor.Decode(&mongoResp); err != nil {
return 0, err
}
receiverValue := reflect.ValueOf(receiver)
docs := []interface{}(mongoResp.Documents.(primitive.A))
receiverValue.Elem().Set(reflect.ValueOf(docs))
return mongoResp.Total, nil
}
type Person struct {
Name string `bson:"name"`
}
func main() {
var persons []Person
count, err := getListWithCount(context.Background(), &persons)
if err != nil {
log.Fatal(err)
}
fmt.Println(count)
fmt.Println(persons)
}
You should be able to decode first into bson.RawValue and then Unmarshal it into the receiver.
func getListWithCount(ctx context.Context, receiver interface{}) (int, error) {
//my mongo query here
var mongoResp struct {
Total int `bson:"total"`
Documents bson.RawValue `bson:"documents"`
}
if err := cursor.Decode(&mongoResp); err != nil {
return 0, err
}
if err := mongoResp.Documents.Unmarshal(receiver); err != nil {
return 0, err
}
return mongoResp.Total, nil
}
You can also implement it as a custom bson.Unmarshaler.
type MongoResp struct {
Total int `bson:"total"`
Documents interface{} `bson:"documents"`
}
func (r *MongoResp) UnmarshalBSON(data []byte) error {
var temp struct {
Total int `bson:"total"`
Documents bson.RawValue `bson:"documents"`
}
if err := bson.Unmarshal(data, &temp); err != nil {
return err
}
r.Total = temp.Total
return temp.Documents.Unmarshal(r.Documents)
}
With that you would use it in the function like so:
func getListWithCount(ctx context.Context, receiver interface{}) (int, error) {
//my mongo query here
mongoResp := MongoResp{Documents: receiver}
if err := cursor.Decode(&mongoResp); err != nil {
return 0, err
}
return mongoResp.Total, nil
}
Dynamically create a struct type that matches the queried document. See commentary below for details.
func getListWithCount(receiver interface{}) (int, error) {
dst := reflect.ValueOf(receiver).Elem()
// Your mongo query here
// Create a struct type that matches the document.
doct := reflect.StructOf([]reflect.StructField{
reflect.StructField{Name: "Total", Type: reflect.TypeOf(0), Tag: `bson:"total"`},
reflect.StructField{Name: "Documents", Type: dst.Type(), Tag: `bson:"documents"`},
})
// Decode to a value of the type.
docp := reflect.New(doct)
if err := cursor.Decode(docp.Interface()); err != nil {
return 0, err
}
docv := docp.Elem()
// Copy the Documents field to *receiver.
dst.Set(docv.Field(1))
// Return the total
return docv.Field(0).Interface().(int), nil
}
there is no need to use reflect here, you can decode it directly to your Person slices
func getPersons(ctx context.Context, coll *mongo.Collection, results interface{}) error {
cur, err := coll.Find(ctx, bson.D{})
if err != nil {
return err
}
err = cur.All(ctx, results)
if err != nil {
return err
}
return nil
}
and the len is the count of the results.
err = getPersons(ctx, coll, &persons)
require.NoError(t, err)
t.Logf("Got %d persons: %v", len(persons), persons)
see https://gist.github.com/xingyongtao/459f92490bdcbf7d5afe9f5d1ae6c04a

Decode into struct when passed as interface{} returns map

When using mongo-driver, I have this struct which I pass to the get function as interface{}. The problem is when I use the Decode method, it returns a map while I was expecting a struct of the proper type. I found a similar question
here but the solution didnt work for me, the program crashes.
type Item struct {
//whatever
}
type mymodel struct {
I interface{}
F interface{}
DatabaseName string
Collection string
}
func Do(){
var item Item
var filter Item
m:= mymodel{I: item, F: filter}
res,_ := get(m)
}
func get(m mymodel) (*interface{}, error) {
c := database.DBCon.Database("whatever").Collection("whatever")
ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
err := c.FindOne(ctx, m.F).Decode(&m.I)
if err != nil {
log.Log.Info(err)
return nil, err
}
return &m.I, nil
}
You need to make the interface fields I and F store a pointer to the instance into which you want to decode the data (maybe not F since you're not decoding anything into that), and then pass them directly to Decode, without any more address operations (&x).
For example:
func Do(){
var item Item
var filter Item
m:= mymodel{I: &item, F: filter}
res,_ := get(m)
}
func get(m mymodel) (*interface{}, error) {
c := database.DBCon.Database("whatever").Collection("whatever")
ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
err := c.FindOne(ctx, m.F).Decode(m.I)
if err != nil {
log.Log.Info(err)
return nil, err
}
return &m.I, nil
}

Fill a slice with field pointers of a struct

I'm creating a function which receives an interface{}, which will always be a struct, but it could be any struct.
I need to fill one slice with pointers of all the fields of the struct received. The place in the code below where I need to pick the field pointer is flagged with **FIELD POINTER**
My final goal is create a function to receive a struct equivalent to the return of the query sent in the parameter sqlQuery. I want to create a dynamic function to perform any type of query to select, always using the struct received for .Scan.
It may be that I'm thinking the wrong way, I'm still starting Golang.
func QuerySelect(entity interface{}, sqlQuery string) {
val := reflect.Indirect(reflect.ValueOf(entity))
psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
"password=%s dbname=%s sslmode=disable",
host, port, user, password, dbname)
// Validate database params
db, err := sql.Open(driver, psqlInfo)
// If returned any error
if err != nil {
panic(err)
}
// Validate connection with database
err = db.Ping()
if err != nil {
panic(err)
}
// Execute query
rows, err := db.Query(sqlQuery)
countColumns := val.Type().NumField()
var allRows []interface{}
for rows.Next() {
columnsPointers := make([]interface{}, countColumns)
for i := 0; i < countColumns; i++ {
columnsPointers[i] = **FIELD POINTER (entity struct)**
}
if err := rows.Scan(columnsPointers...); err != nil {
log.Fatal(err)
}
allRows = append(allRows, entity)
}
}
Per mkopriva's commentary on the question, use the following to get the address of the field:
for i := 0; i < countColumns; i++ {
columnsPointers[i] = val.Field(i).Addr().Interface()
}

How to append objects to a slice?

I am new to golang and I'd like to aggregaet query results into a results slice to be pushed to the browser. Here is the code:
type Category struct {
Id bson.ObjectId `bson:"_id,omitempty"`
Name string
Description string
Tasks []Task
}
type Cats struct {
category Category
}
func CategoriesCtrl(w http.ResponseWriter, req *http.Request) {
session, err := mgo.Dial("localhost")
if err != nil {
panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
c := session.DB("taskdb").C("categories")
iter := c.Find(nil).Iter()
result := Category{}
results := []Cats //Here is the problem
for iter.Next(&result) {
results = append(results, result)
fmt.Printf("Category:%s, Description:%s\n", result.Name, result.Description)
tasks := result.Tasks
for _, v := range tasks {
fmt.Printf("Task:%s Due:%v\n", v.Description, v.Due)
}
}
if err = iter.Close(); err != nil {
log.Fatal(err)
}
fmt.Fprint(w, results)
}
But instead I get
type []Cats is not an expression
How can I fix this?
You can say
results := make([]Cats, 0)
or
var results []Cats
or
results := []Cats{}
instead.
You can use results := make([]Cats, len) instead, where len is the initial length of slice.
results := []Cats{} will also work.
If you use var results []Cats, its initial value is nil so you'd need to initialize it before using append.

Resources