How to set slice interface values with reflection - go

I would like to build a function that takes a generic pointer array and fill that list based on mongo results.
I don't know how to set the value I got from mongo into my pointer array. In the below attempt, program panics with following error : reflect.Set: value of type []interface {} is not assignable to type []Person
When I print total / documents found, it corresponds to what I am expecting. So I think question is about reflection.
func getListWithCount(ctx context.Context, receiver interface{}) (int, error) {
//my mongo query here
var mongoResp struct {
Total int `bson:"total"`
Documents interface{} `bson:"documents"`
}
if err := cursor.Decode(&mongoResp); err != nil {
return 0, err
}
receiverValue := reflect.ValueOf(receiver)
docs := []interface{}(mongoResp.Documents.(primitive.A))
receiverValue.Elem().Set(reflect.ValueOf(docs))
return mongoResp.Total, nil
}
type Person struct {
Name string `bson:"name"`
}
func main() {
var persons []Person
count, err := getListWithCount(context.Background(), &persons)
if err != nil {
log.Fatal(err)
}
fmt.Println(count)
fmt.Println(persons)
}

You should be able to decode first into bson.RawValue and then Unmarshal it into the receiver.
func getListWithCount(ctx context.Context, receiver interface{}) (int, error) {
//my mongo query here
var mongoResp struct {
Total int `bson:"total"`
Documents bson.RawValue `bson:"documents"`
}
if err := cursor.Decode(&mongoResp); err != nil {
return 0, err
}
if err := mongoResp.Documents.Unmarshal(receiver); err != nil {
return 0, err
}
return mongoResp.Total, nil
}
You can also implement it as a custom bson.Unmarshaler.
type MongoResp struct {
Total int `bson:"total"`
Documents interface{} `bson:"documents"`
}
func (r *MongoResp) UnmarshalBSON(data []byte) error {
var temp struct {
Total int `bson:"total"`
Documents bson.RawValue `bson:"documents"`
}
if err := bson.Unmarshal(data, &temp); err != nil {
return err
}
r.Total = temp.Total
return temp.Documents.Unmarshal(r.Documents)
}
With that you would use it in the function like so:
func getListWithCount(ctx context.Context, receiver interface{}) (int, error) {
//my mongo query here
mongoResp := MongoResp{Documents: receiver}
if err := cursor.Decode(&mongoResp); err != nil {
return 0, err
}
return mongoResp.Total, nil
}

Dynamically create a struct type that matches the queried document. See commentary below for details.
func getListWithCount(receiver interface{}) (int, error) {
dst := reflect.ValueOf(receiver).Elem()
// Your mongo query here
// Create a struct type that matches the document.
doct := reflect.StructOf([]reflect.StructField{
reflect.StructField{Name: "Total", Type: reflect.TypeOf(0), Tag: `bson:"total"`},
reflect.StructField{Name: "Documents", Type: dst.Type(), Tag: `bson:"documents"`},
})
// Decode to a value of the type.
docp := reflect.New(doct)
if err := cursor.Decode(docp.Interface()); err != nil {
return 0, err
}
docv := docp.Elem()
// Copy the Documents field to *receiver.
dst.Set(docv.Field(1))
// Return the total
return docv.Field(0).Interface().(int), nil
}

there is no need to use reflect here, you can decode it directly to your Person slices
func getPersons(ctx context.Context, coll *mongo.Collection, results interface{}) error {
cur, err := coll.Find(ctx, bson.D{})
if err != nil {
return err
}
err = cur.All(ctx, results)
if err != nil {
return err
}
return nil
}
and the len is the count of the results.
err = getPersons(ctx, coll, &persons)
require.NoError(t, err)
t.Logf("Got %d persons: %v", len(persons), persons)
see https://gist.github.com/xingyongtao/459f92490bdcbf7d5afe9f5d1ae6c04a

Related

Implement a struct-to-csv writer in Go

The following code attempt to implement a generic CSV writer for any simple struct. By "simple", I mean field value of the struct are of standard, simple types (int, string etc).
type (
CSV interface {
Header() []string
String([]string) (string, error)
}
CSVArray []CSV
)
func CSVOutput(w io.Writer, data CSVArray, cols []string) error {
if len(data) == 0 {
return nil
}
_, err := fmt.Fprintln(w, data[0].Header())
if err != nil {
return err
}
for _, d := range data {
str, err := d.String(cols)
if err != nil {
return err
}
_, err = fmt.Fprintln(w, str)
if err != nil {
return err
}
}
return nil
}
The problem is CSVOutput() does not actually work. e.g.:
var data []Employee //the Employee struct implements CSV interface
CSVOutput(w, data, nil)
Compilation failed: cannot use data (type []Employee) as type CSVArray in argument to CSVOutput
I understand that []CSV is not same as []Employee, as explained here, and many other resources available online.
That said, is it possible to rewrite the CSVOutput() function by using reflection:
func CSVOutput(w io.Writer, data interfac{}, cols []string) error {
sliceOfIntf = castToSlice(data) //how to do this?
if !implementedCSV(sliceOfIntf[0]) { //and how to do this?
return errors.New("not csv")
}
... ...
}
is it possible to rewrite the CSVOutput() function by using reflection
Yes
// if data is []Employee{...}, then you can do the following:
rv := reflect.ValueOf(data)
if rv.Kind() != reflect.Slice {
return fmt.Errorf("data is not slice")
}
if !rv.Type().Elem().Implements(reflect.TypeOf((*CSV)(nil)).Elem()) {
return fmt.Errorf("slice element does not implement CSV")
}
csvArr := make(CSVArray, rv.Len())
for i := 0; i < rv.Len(); i++ {
csvArr[i] = rv.Index(i).Interface().(CSV)
}
// now csvArr is CSVArray containing all the elements of data
https://go.dev/play/p/gcSOid533gx

How to list all the items in a table with pagination

I'm trying to list all the items in a DynamoDB table with pagination, and here below is my attempt:
const tableName = "RecordingTable"
type Recording struct {
ID string `dynamodbav:"id"`
CreatedAt string `dynamodbav:"createdAt"`
UpdatedAt string `dynamodbav:"updatedAt"`
Duration int `dynamodbav:"duration"`
}
type RecordingRepository struct {
ctx context.Context
svc *dynamodb.Client
}
func NewRecordingRepository(ctx context.Context) (*RecordingRepository, error) {
cfg, err := config.LoadDefaultConfig(ctx)
if err != nil {
return nil, err
}
return &RecordingRepository{ctx, dynamodb.NewFromConfig(cfg)}, nil
}
func (r *RecordingRepository) List(page int, size int) ([]Recording, error) {
size32 := int32(size)
queryInput := &dynamodb.QueryInput{
TableName: aws.String(tableName),
Limit: &size32,
}
recordings := []Recording{}
queryPaginator := dynamodb.NewQueryPaginator(r.svc, queryInput)
for i := 0; queryPaginator.HasMorePages(); i++ {
result, err := queryPaginator.NextPage(r.ctx)
if err != nil {
return nil, err
}
if i == page {
if result.Count > 0 {
for _, v := range result.Items {
recording := Recording{}
if err := attributevalue.UnmarshalMap(v, &recording); err != nil {
return nil, err
}
recordings = append(recordings, recording)
}
}
break
}
}
return recordings, nil
}
When I run the code above, I get the following error message:
api error ValidationException: Either the KeyConditions or KeyConditionExpression parameter must be specified in the request.
But why should I specify a KeyConditionExpression when I want to get all the items? Is there another way to go or a workaround this?
Query does need your keys. It is meant to find specific items in your DynamoDB. To get all items in your DynamoDB, you need to use the Scan operation.
This should be easily fixed in your code.
Instead of QueryInput use ScanInput and instead of NewQueryPaginator use NewScanPaginator.
Just replaced QueryInput with ScanInput and QueryPaginator with ScanPaginator.

Custom unmarshall to a type than input out different type

I'm learning Go and I have a question:
I have an Info type that is defined like that:
type Info struct {
ID ID `json:"id,omitempty"`
DisplayName string `json:"display_name,omitempty"`
}
I made a custom UnmarshallJSON function to unmarshall this struct because as an input I have either:
An []interface{} with at position [0] an int and [1] a string
A boolean always equals to false meaning that the field is null
I want that when the input is false, the Info is nil.
Here's the UnmarshallJSON function
func (i *Info) UnmarshalJSON(data []byte) error {
var v []interface{}
if err := json.Unmarshal(data, &v); err != nil {
var v bool
if err = json.Unmarshal(data, &v); err != nil {
return err
}
return nil
}
i.ID = ID(v[0].(float64))
i.DisplayName = v[1].(string)
return nil
}
It's ugly, and I would like to know if there's a better option.
Thank you very much.
Fist you should be more defensive about unexpected types and length to avoid a panic. Then you can unmarshal into a []json.RawMessage to defer unmarshaling of the elements until you are ready. Finally you should guard against your invalid true.
Here is my best effort, please others feel free to edit (here is a playground):
func (i *Info) UnmarshalJSON(data []byte) error {
var raw interface{}
if err := json.Unmarshal(data, &raw); err != nil {
return err
}
switch r := raw.(type) {
case []interface{}:
case bool:
if r {
return errors.New("unexpected true, must be array or false")
}
return nil
default:
return fmt.Errorf("unexpected type %T, must be array or false", r)
}
var v []json.RawMessage
if err := json.Unmarshal(data, &v); err != nil {
return err
}
if len(v) != 2 {
return fmt.Errorf("unexpected length %d, must be 2", len(v))
}
if err := json.Unmarshal(v[0], &i.ID); err != nil {
return err
}
if err := json.Unmarshal(v[1], &i.DisplayName); err != nil {
return err
}
return nil
}

Difference in func with receiver and param golang

I try to create marshal func for struct Item.
So the question is, why the first example gives stackoverflow for goroutine and second works correctly?
Call method with receiver
type Item struct{
a int
}
func some(items []Item){
for _,i:=range items{
buf,err:=i.MarhsalBinary()
fmt.Println(buf.Bytes(), err)
}
func (i Item) MarshalBinary() ([]byte, error) {
var (
buf bytes.Buffer
err error
)
if err = gob.NewEncoder(&buf).Encode(i); err != nil { // endless loop here
return nil, err
}
return buf.Bytes(), err
}
This is pass by value.
type Item struct{
a int
}
func some(items []Item){
for _,i:=range items{
buf,err:=MarhsalBinary(i)
fmt.Println(buf.Bytes(), err)
}
func MarshalBinary(i Item) ([]byte, error) {
var (
buf bytes.Buffer
err error
)
if err = gob.NewEncoder(&buf).Encode(i); err != nil { // works correctly
return nil, err
}
return buf.Bytes(), err
}

How to judge unmarshal json interface{} type in golang?

I want to judge json type,but it always return "I don't know about type map[string]interface {}!",How to resolve it.
=========================================================================
type getRemoteCardInfo struct {
Code int
Msg string
Data []*remoteCardInfo
}
type remoteCardInfo struct {
Sn string
RemoteCardIp string
RemoteCardMac string
}
func Get_json_data(url string) (interface{}, error) {
client := &http.Client{}
req, err := http.NewRequest("GET", url, nil)
req.Header.Add("X-MYCMDB-Auth-Token", "sPf98SMBWzOZJEJB8KWltbJyKvFYPauu")
if err != nil {
return nil, err
}
resp, _ := client.Do(req)
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
return nil, fmt.Errorf("query failed: %s", resp.Status)
}
var result interface{}
body, err := ioutil.ReadAll(resp.Body)
if err := json.Unmarshal(body, &result); err != nil {
log.Fatalf("JSON unmarshaling failed: %s", err)
}
resp.Body.Close()
fmt.Println(result)
return result, nil
}
func main() {
jsondata, err := Get_json_data(DHCPURL)
if err != nil {
log.Fatal(err)
}
switch v := jsondata.(type) {
case getRemoteCardInfo:
fmt.Println("aaaa")
default:
fmt.Printf("I don't know about type %T!\n", v)
}
The go JSON unmarshaler doesn't know about types, as you can tell by the fact that it stores the result into an interface{} value:
func Unmarshal(data []byte, v interface{}) error
// "v" can be any type -------^
So it's up to you to use the unmarshaler to populate your structure and determine if the result is valid or not.
In your example it looks like you're trying to unmarshal a remoteCardInfo from an HTTP response. To do this you should unmarshal into an empty remoteCardInfo struct and determine if the required fields were populated.
For example, suppose you expect a JSON document like so:
{
"sn": "123",
"ip": "0.0.0.0",
"mac": "ff:ff:ff:ff:ff:ff"
}
Then you should define your "remoteCardInfo" struct as below:
type remoteCardInfo struct {
Sn string `json:"sn"`
RemoteCardIp string `json:"ip"`
RemoteCardMac string `json:"mac"`
}
And then unmarshal and validate it like so:
func getRemoteCardInfo(bs []byte) (*remoteCardInfo, error) {
rci := remoteCardInfo{}
err := json.Unmarshal(bs, &rci)
if err != nil {
return nil, err
}
// Validate the expected fields
if rci.Sn == "" {
return nil, fmt.Errorf(`missing "sn"`)
}
if rci.RemoteCardIp == "" {
return nil, fmt.Errorf(`missing "ip"`)
}
if rci.RemoteCardMac == "" {
return nil, fmt.Errorf(`missing "mac"`)
}
return &rci, nil
}
Of course, you can validate the fields any way you like but the main thing to remember is that the unmarshaler only does the job of ensuring that the input byte array is a valid JSON document and populates the fields from the document into the fields defined by the value.
It cannot tell you what "type" of object the JSON document represents.

Resources