I have a problem with making dynamic model of a struct. I mean that I want to assert or cast, or just change the type of struct according to the incoming data strut.
if sourceName variable would be type_x , than the type of deserializedData should be type_x, if type_y, than type_y. How to set the variable deserializedData dynamicly for this ?
I have this part in my code:
....
var cacheData []byte
var deserializedData models.NoaggModel
cache_err := cache.Get(string(string(sourceName) + "_" + string(t.Date)), &cacheData);
if cache_err != nil {
fmt.Println("cache_error: ", cache_err)
panic("the cache is empty")
}
err2 := json.Unmarshal([]byte(cacheData), &deserializedData)
if err2 == nil {
fmt.Println("deserialized data: " + string(sourceName), deserializedData)
}
for _, chart := range charts {
w.Name = chart.Name
if err2 == nil {
w.Data = countDataByName(sourceName, deserializedData, t.Request.Filters, string(chart.Name))
}
out <- w
}
....
How to modify it, to avoid setting models.Noagg Model type in a strict way?
Creating an instance of a type dynamically during runtime can be done using the reflect package. You can use a map to store the different types that you should be able to create:
Example:
package main
import (
"fmt"
"reflect"
)
type Foo struct {
Foo string
}
type Bar struct {
Bar int
}
func main() {
var sourceTypes = map[string]reflect.Type{
"foo": reflect.TypeOf(Foo{}),
"bar": reflect.TypeOf(Bar{}),
}
sourceName := "foo"
var deserializedData interface{}
deserializedData = reflect.New(sourceTypes[sourceName]).Interface()
fmt.Printf("%#v", deserializedData)
}
Output:
&main.Foo{Foo:""}
Playground: http://play.golang.org/p/qeDA4cu5et
Related
package main
import (
"fmt"
"reflect"
)
type Config struct {
App *AppConfig
}
type AppConfig struct {
Name string
}
func (a *AppConfig) Parse() {
a.Name = "111"
}
var (
config = &Config{
App: &AppConfig{},
}
)
func main() {
v := reflect.ValueOf(*config)
typeOf := reflect.TypeOf(*config)
for i := 0; i < typeOf.NumField(); i++ {
method := v.Field(i).MethodByName("Parse")
method.Call([]reflect.Value{})
}
fmt.Println(config)
}
look at this, it can run successfully
but when i change
var (
config = &Config{
App: &AppConfig{},
}
)
to
var (
config = &Config{}
)
it will be failed with error #panic: runtime error: invalid memory address or nil pointer dereference#
how can i run successfully with
var (
config = &Config{}
)
Given f := v.Field(i) where f represents a pointer type, you first need to check whether the value represented by f isn't nil, there's a method for that in the reflect package. And if you get back true, i.e. it is nil, then, before you can call a method on that field, you need to initialize that field so that is it not nil anymore. i.e. f.Set(reflect.New(f.Type().Elem())).
And you also need to make sure that the f is addressable, to do that, pass a pointer to the config to reflect, i.e. *Config, not Config.
v := reflect.ValueOf(config).Elem()
typeOf := reflect.TypeOf(config).Elem()
for i := 0; i < typeOf.NumField(); i++ {
f := v.Field(i)
if f.Kind() == reflect.Ptr && f.IsNil() {
f.Set(reflect.New(f.Type().Elem()))
}
method := f.MethodByName("Parse")
method.Call([]reflect.Value{})
}
fmt.Println(config)
fmt.Println(config.App)
https://go.dev/play/p/DMRFS41NXFS
I have a struct that looks like this:
type Type int
const (
A Type = iota
B
C
)
type Character struct {
Type Type `json:"type"`
}
When I call json.Marshal(...) on the struct, is there a way that the json:"type" representation is a string called either "A", "B", or "C"?
When this is presented in JSON, nobody is going to know what 0, 1, or 2 is, so the name of the constant is more useful.
Apologies if this has been asked before. I googled all over and couldn't find anything.
Here is an example:
type Type int
const (
A Type = iota
B
C
)
type Character struct {
Type Type `json:"type,string"`
}
func main() {
c := Character{}
c.Type = A
j, err := json.Marshal(c)
if err != nil {
panic(err)
}
fmt.Println(string(j))
}
I want fmt.Println(string(j)) to print {"type":"A"}, not {"type":0}.
You need to define a custom marshaller for your type.
type Type int
const (
A Type = iota
B
C
)
var typeToString = map[Type]string{
A: "A",
B: "B",
C: "C",
}
func (t Type) MarshalJSON() ([]byte, error) {
return json.Marshal(typeToString[t])
}
type Character struct {
Type Type `json:"type"`
}
func main() {
c := Character{}
c.Type = A
j, err := json.Marshal(c)
if err != nil {
panic(err)
}
fmt.Println(string(j))
}
The function MarshalJSON defines how json.Marshal should marshal your type. You can do something similar for unmarshaling if you need to go the other direction as well.
See https://play.golang.org/p/mLxThWA19by.
Short reply: there's NO direct option to achieve your goal.
Long reply: truth is, you can actually override the way Go can encode structs.
Here's the working code:
https://play.golang.org/p/i92pUpNG-Wr
package main
import (
"encoding/json"
"fmt"
)
// please consider to rename this!
type Type int
const (
A Type = iota
B
C
)
type Character struct {
Type Type `json:"-"`
}
func (c *Character) mapTypeToString() string {
switch c.Type {
case B:
return "B"
case C:
return "C"
}
// defaults on A
return "A"
}
// MarshalJSON overwrites the standard JSON marshal.
func (c *Character) MarshalJSON() ([]byte, error) {
return json.Marshal(&struct {
Type string `json:"type"`
}{
Type: c.mapTypeToString(),
})
}
func main() {
c := &Character{}
c.Type = A
j, err := json.Marshal(c)
if err != nil {
panic(err)
}
fmt.Println(string(j))
}
Be aware of the c := &Character{} initialization.
The tradeoff would be to have the mapTypeToString() method that needs to be updated according to the different types of ... Type.
Please, also consider to avoid naming structs and vars like Type :)
So your JSON API gives you the type A:
I want fmt.Println(string(j)) to print {"type":"A"}, not {"type":0}.
You could change your code like this, then your API works:
https://play.golang.org/p/ypvFvQpBw-C
type JSONType string
const (
A JSONType = "A"
B JSONType = "B"
C JSONType = "C"
)
type Character struct {
JSONType JSONType `json:"type,string"`
}
func main() {
c := Character{}
c.JSONType = A
j, err := json.Marshal(c)
if err != nil {
panic(err)
}
fmt.Println(string(j))
}
I'm trying to set-up an AWS-lambda using aws-sdk-go that is triggered whenever a new user is added to a certain dynamodb table.
Everything is working just fine but I can't find a way to unmarshal a map map[string]DynamoDBAttributeValue like:
{
"name": {
"S" : "John"
},
"residence_address": {
"M": {
"address": {
"S": "some place"
}
}
}
}
To a given struct, for instance, a User struct. Here is shown an example of unsmarhaling a map[string]*dynamodb.AttributeValue into a given interface, but I can't find a way to do the same thing with map[string]DynamoDBAttributeValue even though these types seem to fit the same purposes.
map[string]DynamoDBAttributeValue is returned by a events.DynamoDBEvents from package github.com/aws/aws-lambda-go/events. This is my code:
package handler
import (
"context"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"github.com/aws/aws-sdk-go/service/dynamodb"
)
func HandleDynamoDBRequest(ctx context.Context, e events.DynamoDBEvent) {
for _, record := range e.Records {
if record.EventName == "INSERT" {
// User Struct
var dynamoUser model.DynamoDBUser
// Of course this can't be done for incompatible types
_ := dynamodbattribute.UnmarshalMap(record.Change.NewImage, &dynamoUser)
}
}
}
Of course, I can marshal record.Change.NewImage to JSON and unmarshal it back to a given struct, but then, I would have to manually initialize dynamoUser attributes starting from the latter ones.
Or I could even write a function that parses map[string]DynamoDBAttributeValue to map[string]*dynamodb.AttributeValue like:
func getAttributeValueMapFromDynamoDBStreamRecord(e events.DynamoDBStreamRecord) map[string]*dynamodb.AttributeValue {
image := e.NewImage
m := make(map[string]*dynamodb.AttributeValue)
for k, v := range image {
if v.DataType() == events.DataTypeString {
s := v.String()
m[k] = &dynamodb.AttributeValue{
S : &s,
}
}
if v.DataType() == events.DataTypeBoolean {
b := v.Boolean()
m[k] = &dynamodb.AttributeValue{
BOOL : &b,
}
}
// . . .
if v.DataType() == events.DataTypeMap {
// ?
}
}
return m
}
And then simply use dynamodbattribute.UnmarshalMap, but on events.DataTypeMap it would be quite a tricky process.
Is there a way through which I can unmarshal a DynamoDB record coming from a events.DynamoDBEvent into a struct with a similar method shown for map[string]*dynamodb.AttributeValue?
I tried the function you provided, and I met some problems with events.DataTypeList, so I managed to write the following function that does the trick:
// UnmarshalStreamImage converts events.DynamoDBAttributeValue to struct
func UnmarshalStreamImage(attribute map[string]events.DynamoDBAttributeValue, out interface{}) error {
dbAttrMap := make(map[string]*dynamodb.AttributeValue)
for k, v := range attribute {
var dbAttr dynamodb.AttributeValue
bytes, marshalErr := v.MarshalJSON(); if marshalErr != nil {
return marshalErr
}
json.Unmarshal(bytes, &dbAttr)
dbAttrMap[k] = &dbAttr
}
return dynamodbattribute.UnmarshalMap(dbAttrMap, out)
}
I was frustrated that the type of NewImage from the record wasn't map[string]*dynamodb.AttributeValue so I could use the dynamodbattribute package.
The JSON representation of events.DynamoDBAttributeValue seems to be the same as the JSON represenation of dynamodb.AttributeValue.
So I tried creating my own DynamoDBEvent type and changed the type of OldImage and NewImage, so it would be marshalled into map[string]*dynamodb.AttributeValue instead of map[string]events.DynamoDBAttributeValue
It is a little bit ugly but it works for me.
package main
import (
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"fmt"
)
func main() {
lambda.Start(lambdaHandler)
}
// changed type of event from: events.DynamoDBEvent to DynamoDBEvent (see below)
func lambdaHandler(event DynamoDBEvent) error {
for _, record := range event.Records {
change := record.Change
newImage := change.NewImage // now of type: map[string]*dynamodb.AttributeValue
var item IdOnly
err := dynamodbattribute.UnmarshalMap(newImage, &item)
if err != nil {
return err
}
fmt.Println(item.Id)
}
return nil
}
type IdOnly struct {
Id string `json:"id"`
}
type DynamoDBEvent struct {
Records []DynamoDBEventRecord `json:"Records"`
}
type DynamoDBEventRecord struct {
AWSRegion string `json:"awsRegion"`
Change DynamoDBStreamRecord `json:"dynamodb"`
EventID string `json:"eventID"`
EventName string `json:"eventName"`
EventSource string `json:"eventSource"`
EventVersion string `json:"eventVersion"`
EventSourceArn string `json:"eventSourceARN"`
UserIdentity *events.DynamoDBUserIdentity `json:"userIdentity,omitempty"`
}
type DynamoDBStreamRecord struct {
ApproximateCreationDateTime events.SecondsEpochTime `json:"ApproximateCreationDateTime,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
Keys map[string]*dynamodb.AttributeValue `json:"Keys,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
NewImage map[string]*dynamodb.AttributeValue `json:"NewImage,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
OldImage map[string]*dynamodb.AttributeValue `json:"OldImage,omitempty"`
SequenceNumber string `json:"SequenceNumber"`
SizeBytes int64 `json:"SizeBytes"`
StreamViewType string `json:"StreamViewType"`
}
I have found the same problem and the solution is to perform a simple conversion of types. This is possible because in the end the type received by lambda events events.DynamoDBAttributeValue and the type used by the SDK V2 of AWS DynamoDB types.AttributeValue are the same. Next I show you the conversion code.
package aws_lambda
import (
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
func UnmarshalDynamoEventsMap(
record map[string]events.DynamoDBAttributeValue, out interface{}) error {
asTypesMap := DynamoDbEventsMapToTypesMap(record)
err := attributevalue.UnmarshalMap(asTypesMap, out)
if err != nil {
return err
}
return nil
}
func DynamoDbEventsMapToTypesMap(
record map[string]events.DynamoDBAttributeValue) map[string]types.AttributeValue {
resultMap := make(map[string]types.AttributeValue)
for key, rec := range record {
resultMap[key] = DynamoDbEventsToTypes(rec)
}
return resultMap
}
// DynamoDbEventsToTypes relates the dynamo event received by AWS Lambda with the data type that is
// used in the Amazon SDK V2 to deal with DynamoDB data.
// This function is necessary because Amazon does not provide any kind of solution to make this
// relationship between types of data.
func DynamoDbEventsToTypes(record events.DynamoDBAttributeValue) types.AttributeValue {
var val types.AttributeValue
switch record.DataType() {
case events.DataTypeBinary:
val = &types.AttributeValueMemberB{
Value: record.Binary(),
}
case events.DataTypeBinarySet:
val = &types.AttributeValueMemberBS{
Value: record.BinarySet(),
}
case events.DataTypeBoolean:
val = &types.AttributeValueMemberBOOL{
Value: record.Boolean(),
}
case events.DataTypeList:
var items []types.AttributeValue
for _, value := range record.List() {
items = append(items, DynamoDbEventsToTypes(value))
}
val = &types.AttributeValueMemberL{
Value: items,
}
case events.DataTypeMap:
items := make(map[string]types.AttributeValue)
for k, v := range record.Map() {
items[k] = DynamoDbEventsToTypes(v)
}
val = &types.AttributeValueMemberM{
Value: items,
}
case events.DataTypeNull:
val = nil
case events.DataTypeNumber:
val = &types.AttributeValueMemberN{
Value: record.Number(),
}
case events.DataTypeNumberSet:
val = &types.AttributeValueMemberNS{
Value: record.NumberSet(),
}
case events.DataTypeString:
val = &types.AttributeValueMemberS{
Value: record.String(),
}
case events.DataTypeStringSet:
val = &types.AttributeValueMemberSS{
Value: record.StringSet(),
}
}
return val
}
There is a package that allows conversion from events.DynamoDBAttributeValue to dynamodb.AttributeValue
https://pkg.go.dev/github.com/aereal/go-dynamodb-attribute-conversions/v2
From there one can unmarshal AttributeValue into struct
func Unmarshal(attribute map[string]events.DynamoDBAttributeValue, out interface{}) error {
av := ddbconversions.AttributeValueMapFrom(attribute)
return attributevalue.UnmarshalMap(av, out)
}
I want to use some external code that requires a pointer to a struct. At the point that the code is called, I have an interface variable.
When creating a pointer off of that variable, the pointer's type is interface{}* when I need it to be the pointer type of the struct's type.
Image the code in TestCanGetStructPointer does not know about the Cat class, and that it exists in some external package.
How can I cast it to this?
Here is a code sample:
import (
"reflect"
"testing"
)
type Cat struct {
name string
}
type SomethingGeneric struct {
getSomething func() interface{}
}
func getSomeCat() interface{} {
return Cat{}
}
var somethingForCats = SomethingGeneric{getSomething: getSomeCat}
func TestCanGetStructPointer(t *testing.T) {
interfaceVariable := somethingForCats.getSomething()
pointer := &interfaceVariable
interfaceVarType := reflect.TypeOf(interfaceVariable)
structPointerType := reflect.PtrTo(interfaceVarType)
pointerType := reflect.TypeOf(pointer)
if pointerType != structPointerType {
t.Errorf("Pointer type was %v but expected %v", pointerType, structPointerType)
}
}
The test fails with:
Pointer type was *interface {} but expected *parameterized.Cat
#dyoo's example does work, but it relies on you to manually cast Dog and Cat.
Here's a bit of a convoluted/verbose example which avoids that constraint somewhat:
package main
import (
"fmt"
"reflect"
)
type Cat struct {
name string
}
type SomethingGeneric struct {
getSomething func() interface{}
}
func getSomeCat() interface{} {
return Cat{name: "Fuzzy Wuzzy"}
}
var somethingForCats = SomethingGeneric{getSomething: getSomeCat}
func main() {
interfaceVariable := somethingForCats.getSomething()
castVar := reflect.ValueOf(interfaceVariable)
castVar.Convert(castVar.Type())
// If you want a pointer, do this:
fmt.Println(reflect.PtrTo(castVar.Type()))
// The deref'd val
if castVar.Type() != reflect.TypeOf(Cat{}) {
fmt.Printf("Type was %v but expected %v\n", castVar, reflect.TypeOf(&Cat{}))
} else {
fmt.Println(castVar.Field(0))
}
}
Playground Link
I found this thread: https://groups.google.com/forum/#!topic/golang-nuts/KB3_Yj3Ny4c
package main
import (
"fmt"
"reflect"
)
type Cat struct {
name string
}
//
// Return a pointer to the supplied struct via interface{}
//
func to_struct_ptr(obj interface{}) interface{} {
fmt.Println("obj is a", reflect.TypeOf(obj).Name())
// Create a new instance of the underlying type
vp := reflect.New(reflect.TypeOf(obj))
// Should be a *Cat and Cat respectively
fmt.Println("vp is", vp.Type(), " to a ", vp.Elem().Type())
vp.Elem().Set(reflect.ValueOf(obj))
// NOTE: `vp.Elem().Set(reflect.ValueOf(&obj).Elem())` does not work
// Return a `Cat` pointer to obj -- i.e. &obj.(*Cat)
return vp.Interface()
}
//
// Dump out a pointer ...
//
func test_ptr(ptr interface{}) {
v := reflect.ValueOf(ptr)
fmt.Println("ptr is a", v.Type(), "to a", reflect.Indirect(v).Type())
}
func main() {
cat := Cat{name: "Fuzzy Wuzzy"}
// Reports "*main.Cat"
test_ptr(&cat)
// Get a "*Cat" generically via interface{}
sp := to_struct_ptr(cat)
// *should* report "*main.Cat" also
test_ptr(sp)
fmt.Println("sp is",sp)
}
The following may help: http://play.golang.org/p/XkdzeizPpP
package main
import (
"fmt"
)
type Cat struct {
name string
}
type Dog struct {
name string
}
type SomethingGeneric struct {
getSomething func() interface{}
}
func getSomeCat() interface{} {
return Cat{name: "garfield"}
}
func getSomeDog() interface{} {
return Dog{name: "fido"}
}
var somethings = []SomethingGeneric{
SomethingGeneric{getSomething: getSomeCat},
SomethingGeneric{getSomething: getSomeDog},
}
func main() {
for _, something := range somethings {
interfaceVariable := something.getSomething()
cat, isCat := interfaceVariable.(Cat)
dog, isDog := interfaceVariable.(Dog)
fmt.Printf("cat %v %v\n", cat, isCat)
fmt.Printf("dog %v %v\n", dog, isDog)
}
}
What is the way of printing "Foo" here? In this example, what prints is "string".
http://play.golang.org/p/ZnK6PRwEPp
type A struct {
Foo string
}
func (a *A) PrintFoo() {
fmt.Println("Foo value is " + a.Foo)
}
func main() {
a := &A{Foo: "afoo"}
val := reflect.Indirect(reflect.ValueOf(a))
fmt.Println(val.Field(0).Type().Name())
}
You want val.Type().Field(0).Name. The Field method on reflect.Type will return a struct describing that field, which includes the name, among other information.
There is no way to retrieve the field name for a reflect.Value representing a particular field value, since that is a property of the containing struct.
I think the better way to get the fields' name in the struct is
func main() {
a := &A{Foo: "afoo"}
val := reflect.ValueOf(a).Elem()
for i:=0; i<val.NumField();i++{
fmt.Println(val.Type().Field(i).Name)
}
}
There are two tips:
use .Elem() after you reflect.ValueOf(a), because in your case, a is a pointer.
val.Field(i).Type().Name is totally different from val.Type().Field(i).Name. The latter one can get the name of the field in the struct
Hope that it is helpful..
If you want to have a look at more cases, please check my 2mins article
You need to Get the Field of the Type Definition not of the Value.
http://play.golang.org/p/7Bc7MJikbJ
package main
import "fmt"
import "reflect"
type A struct {
Foo string
}
func (a *A) PrintFoo() {
fmt.Println("Foo value is " + a.Foo)
}
func main() {
a := &A{Foo: "afoo"}
val := reflect.Indirect(reflect.ValueOf(a))
fmt.Println(val.Type().Field(0).Name)
}
With the new Names method of the structs package it's even more easier:
package main
import (
"fmt"
"github.com/fatih/structs"
)
type A struct {
Foo string
Bar int
}
func main() {
names := structs.Names(&A{})
fmt.Println(names) // ["Foo", "Bar"]
}
You can also use https://github.com/fatih/structs
// Convert the fields of a struct to a []*Field
fields := s.Fields()
for _, f := range fields {
fmt.Printf("field name: %+v\n", f.Name())
}
package main
import "fmt"
import "reflect"
type A struct {
Foo string
}
func (a *A) PrintFoo() {
fmt.Println("Foo value is " + a.Foo)
}
func main() {
a := &A{Foo: "afoo"}
//long and bored code
t := reflect.TypeOf(*a)
if t.Kind() == reflect.Struct {
for i := 0; i < t.NumField(); i++ {
fmt.Println(t.Field(i).Name)
}
} else {
fmt.Println("not a stuct")
}
//shorthanded call
fmt.Println(reflect.TypeOf(*a).Field(0).Name)//can panic if no field exists
}
You can use this function, which takes the struct as the first parameter, and then its fields. It returns the map type, which is convenient to use
If you use fields from another structure, nothing will happen
If you try to use a different type, it will cause panic
Note that the field has an ordinal number according to the list (starting from 0). All fields in the structure must start with uppercase
func GetStructFieldName(Struct interface{}, StructField ...interface{}) (fields map[int]string) {
fields = make(map[int]string)
s := reflect.ValueOf(Struct).Elem()
for r := range StructField {
f := reflect.ValueOf(StructField[r]).Elem()
for i := 0; i < s.NumField(); i++ {
valueField := s.Field(i)
if valueField.Addr().Interface() == f.Addr().Interface() {
fields[i] = s.Type().Field(i).Name
}
}
}
return fields
}
Full example and playground
package main
import (
"fmt"
"reflect"
)
type Example struct {
Apple bool
Pear int
}
func GetStructFieldName(Struct interface{}, StructField ...interface{}) (fields map[int]string) {
fields = make(map[int]string)
for r := range StructField {
s := reflect.ValueOf(Struct).Elem()
f := reflect.ValueOf(StructField[r]).Elem()
for i := 0; i < s.NumField(); i++ {
valueField := s.Field(i)
if valueField.Addr().Interface() == f.Addr().Interface() {
fields[i] = s.Type().Field(i).Name
}
}
}
return fields
}
func main() {
e := Example{}
names := GetStructFieldName(&e, &e.Apple, &e.Pear)
fmt.Println(names)
fmt.Println(names[0], names[1])
for i := range names {
fmt.Println(names[i])
}
/* Output:
map[0:Apple 1:Pear]
Apple Pear
Apple
Pear
*/
}