I need to create a map from a Json array , I started with below in GO , kind of stuck , any pointers ?
package main
import (
"encoding/json"
"fmt"
"strconv"
)
func main() {
jsonStr := `{
"employee": [
{
"id": 14325,
"grpname": "senior"
},
{
"id": 234,
"grpname": "junior"
}
]
}`
type Group struct {
Employee []struct {
GroupName string
GroupId int
}
}
var group []Group
var groupMap []map[string]interface{}
err := json.Unmarshal([]byte(jsonStr), &groupMap)
if err != nil {
panic(err)
}
for _, groupData := range groupMap {
// convert map to array of Group struct
var g Group
g.GroupName = fmt.Sprintf("%s", groupData["grpname"])
g.GroupId, _ = strconv.Atoi(fmt.Sprintf("%v", groupData["id"]))
group = append(group, g)
}
fmt.Println(group)
}
Error:
./prog.go:45:4: g.GroupName undefined (type Group has no field or method GroupName)
./prog.go:46:4: g.GroupId undefined (type Group has no field or method GroupId)
Output expected:
{"senior": 14325,"junior": 234}
Tried few things like below , but getting error : There was an error:%!(EXTRA string=json: cannot unmarshal array into Go struct field GetEmpResponse.employee of type map[string][]model.Employee)
type GetEmpResponse struct {
Employee map[string][]Employee json:"employee"
}
Tried to simplify the Json for my testing , please refer to play.golang.org
empResponse.Employee is an array, so you have to access its elements by index, e.g.
empResponse.Employee[0].ID
Related
I am trying to create the new API with Go and Fiber framework. I am using MongoDB.
Please refer to my below structure
type Product struct {
Name string `json:"name"`
Code string `json:"code"`
Description string `json:"description"`
Brand string `json:"brand"`
Variants []map[string]string `json:"variants"`
Categories []int `json:"categories"`
}
The Variants field will have another set array of objects. But the fields will be dynamic. So I can't define the structure. Now this one I solved with []map[string][string].
My task here I have to form the CSV file with the below format.
[{
Name: "Rock",
Code: "RRR"
Description: "This is rock"
Brand: "R"
...variants
},
{
Name: "Rock",
Code: "RRR"
Description: "This is rock"
Brand: "R"
...variants
},
{
Name: "Rambo",
Code: "RAM"
Description: "This is Rambo"
Brand: "RA"
...variants
},
{
Name: "Rambo",
Code: "RAM"
Description: "This is Rambo"
Brand: "RA"
...variants
}]
Only variants will change for every row. Others will be remaining same
I formed the other fields except for the Variants. I can't create the struct because the data will be dynamic.
I have a lot of confusion
How to create the struct for dynamic fields
How to spread the Variants at the root level
I am from javascript. So, we used the spread operator. Here I am confused about how to do that in Golang.
Help me out, guys
#mkopriva is right
However you can use reflection to dynamically construct a new struct if you like
follow is a example:
package main
import (
"encoding/json"
"fmt"
"reflect"
)
type DynamicMap map[string]interface{}
type Product struct {
Name string `json:"name"`
Code string `json:"code"`
Description string `json:"description"`
Brand string `json:"brand"`
Variants map[string]string `json:"variants"`
Categories []int `json:"categories"`
}
func (j Product) MarshalJSON() ([]byte, error) {
m := DynamicMap{}
t := reflect.TypeOf(j)
v := reflect.ValueOf(j)
for i := 0; i < t.NumField(); i++ {
if t.Field(i).Tag.Get("json") == "variants" {
dyn := v.Field(i).Interface().(map[string]string)
for key, val := range dyn {
m[key] = val
}
} else if t.Field(i).Type.Kind() == reflect.String {
m[t.Field(i).Tag.Get("json")] = v.Field(i).String()
} else {
m[t.Field(i).Tag.Get("json")] = v.Field(i)
}
}
return json.Marshal(m)
}
func (j Product) UnmarshalJSON(data []byte) error {
fmt.Println("Unmarshal...")
return json.Unmarshal(data, &j)
}
func main() {
p := Product{Name: "aa", Variants: map[string]string{"a": "a", "b": "b"}}
marshal, err := json.Marshal(p)
if err != nil {
fmt.Printf("%v\n", err)
}
fmt.Printf("%s\n", marshal)
}
{"a":"a","b":"b","brand":"","categories":{},"code":"","description":"","name":"aa"}
I want to insert datastruct like below to MongoDB
{
"rolecode": "DHBK1_ROLE_04",
"functioncodelist": [
"DHBK1_FUNC_1",
"DHBK1_FUNC_2",
.....
"DHBK1_FUNC_n"]
"productid": "ABC_Platform",
"comid": "DHBK1"
}
Here is my code:
package main
import (
"context"
"gopkg.in/mgo.v2"
)
func main() {
insertObj()
}
type CompanyRoleFunction struct {
Rolecode string `json:"rolecode"`
Productid string `json:"productid"`
Functioncodelist []string
Comid string `json:"comid"`
}
func insertObj() {
session, err := mgo.Dial("mongodb://casuser:Mellon#222.255.102.145:27017/users")
if err != nil {
panic(err)
}
c := session.DB("users").C("company_role_function")
var companyRoleFunction CompanyRoleFunction
companyRoleFunction.Rolecode = "DHBK1_ROLE_05"
companyRoleFunction.Productid = "XYZ_Platform"
companyRoleFunction.Comid = "DHBK1"
err = c.Insert(companyRoleFunction)
if err != nil {
fmt.Println(err)
}
}
My code have run but it only insert struct like this (of course, because I don't know how to handle array Functioncodelist )
[![enter image description here][1]][1]
Here is my data, I hope to insert
{"_id":{"$oid":"5ed0c2e2402a000037004c84"},"rolecode":"DHBK1_ROLE_07","functioncodelist":["DHBK1_FUNC_1","DHBK1_FUNC_2"],"productid":"ABC_Platform","comid":"DHBK1"}
It doesn't show up in the database because the document you inserted did not contain any values.
Set a value in your go code and it should show up just fine in the database, unless I misinterpreted your question.
...
type Functioncode string
...
companyRoleFunction.Functioncodelist = make(map[string]Functioncode)
...
companyRoleFunction.Functioncodelist["foo"] = "bar"
...
See https://tour.golang.org/moretypes/19 for a quick introduction to go maps.
I'm trying to create the following json below, but I'm not getting it:
json
{
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "foo1",
"displayText": "foo2"
}
},
{
"basicCard": {
"formattedText": "foo3",
"imageDisplayOptions": "CROPPED"
}
}
]
}
}
The problem is that I have a []struct called Items (slice of structs) and it has 2 different structs SimpleResponse and BasicCard and I am not able to mount this json.
The errors appear:
can not use literal SimpleResponse
can not use BasicCard literal
main.go
https://play.golang.org/p/Gbl0UNWhqko
package main
import (
"encoding/json"
"fmt"
"os"
)
func main() {
type SimpleResponse struct {
TextToSpeech string `json:"textToSpeech"`
DisplayText string `json:"displayText"`
}
type BasicCard struct {
FormattedText string `json:"formattedText"`
ImageDisplayOptions string `json:"imageDisplayOptions"`
}
type Items []struct {
SimpleResponse SimpleResponse `json:"simpleResponse"`
BasicCard BasicCard `json:"basicCard"`
}
type RichResponse struct{
Items Items `json:"items"`
}
group := RichResponse{
Items: Items{
SimpleResponse{"foo1", "foo2"},
BasicCard{
FormattedText: "foo3",
ImageDisplayOptions: "CROPPED",
},
},
}
b, err := json.Marshal(group)
if err != nil {
fmt.Println("error:", err)
}
os.Stdout.Write(b)
}
Could you help me mount this json?
Items is a slice of structs but you're using it as if it's a single Item struct. Being a slice with elements of type Item the correct syntax would be:
group := RichResponse{
Items: Items{ // This is a slice
Item{ // This is an element in the slice
SimpleResponse{"foo1", "foo2"},
BasicCard{
FormattedText: "foo3",
ImageDisplayOptions: "CROPPED",
},
},
},
}
Note that you need to define Item as a named type if you don't want some really messy struct literals. Here's a working example on Playground: https://play.golang.org/p/pzO_w2cIeOJ
I am trying to Unmarshal some json data to a proto message.
JSON
{
"id": 1,
"first_name": "name",
"phone_numbers": []
}
Proto
message Item {
uint32 id=1;
string name=2;
repeated string numbers=3;
}
Proto.go
type Item struct {
Id uint32 `protobuf:"varint,1,opt,name=id" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name" json:"name,omitempty"`
Numbers []string `protobuf:"bytes,4,rep,name=numbers" json:"numbers,omitempty"`
}
How can I map the above JSON to my proto Message (from what I can see there is no way to specify tags in proto atm)?
Your JSON document doesn't match the proto definition; name != first_name and numbers != phone_numbers.
You can define another type that has the same fields as Item but different struct tags and then convert to Item:
var x struct {
Id uint32 `json:"id,omitempty"`
Name string `json:"first_name,omitempty"`
Numbers []string `json:"phone_numbers,omitempty"`
}
if err := json.Unmarshal(jsonDoc, &x); err != nil {
log.Fatal(err)
}
var i = Item(x)
If every JSON document you want to decode has this structure, it may be more convenient to let Item implement json.Unmarshaler:
package main
import (
"encoding/json"
"fmt"
"log"
)
var jsonDoc = []byte(`
{
"id": 1,
"first_name": "name",
"phone_numbers": [
"555"
]
}
`)
type Item struct {
Id uint32 `protobuf:"varint,1,opt,name=id" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name" json:"name,omitempty"`
Numbers []string `protobuf:"bytes,4,rep,name=numbers" json:"numbers,omitempty"`
}
// You can define this function is item_json.go or so, then it
// isn't removed if you re-generate your types.
func (i *Item) UnmarshalJSON(b []byte) error {
type item struct {
Id uint32 `json:"id,omitempty"`
Name string `json:"first_name,omitempty"`
Numbers []string `json:"phone_numbers,omitempty"`
}
var x item
if err := json.Unmarshal(jsonDoc, &x); err != nil {
return err
}
*i = Item(x)
return nil
}
func main() {
var i Item
if err := json.Unmarshal(jsonDoc, &i); err != nil {
log.Fatal(err)
}
fmt.Printf("%#v\n", i)
}
Try it on the playground: https://play.golang.org/p/0qibavRJbwi
I'm trying to set-up an AWS-lambda using aws-sdk-go that is triggered whenever a new user is added to a certain dynamodb table.
Everything is working just fine but I can't find a way to unmarshal a map map[string]DynamoDBAttributeValue like:
{
"name": {
"S" : "John"
},
"residence_address": {
"M": {
"address": {
"S": "some place"
}
}
}
}
To a given struct, for instance, a User struct. Here is shown an example of unsmarhaling a map[string]*dynamodb.AttributeValue into a given interface, but I can't find a way to do the same thing with map[string]DynamoDBAttributeValue even though these types seem to fit the same purposes.
map[string]DynamoDBAttributeValue is returned by a events.DynamoDBEvents from package github.com/aws/aws-lambda-go/events. This is my code:
package handler
import (
"context"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"github.com/aws/aws-sdk-go/service/dynamodb"
)
func HandleDynamoDBRequest(ctx context.Context, e events.DynamoDBEvent) {
for _, record := range e.Records {
if record.EventName == "INSERT" {
// User Struct
var dynamoUser model.DynamoDBUser
// Of course this can't be done for incompatible types
_ := dynamodbattribute.UnmarshalMap(record.Change.NewImage, &dynamoUser)
}
}
}
Of course, I can marshal record.Change.NewImage to JSON and unmarshal it back to a given struct, but then, I would have to manually initialize dynamoUser attributes starting from the latter ones.
Or I could even write a function that parses map[string]DynamoDBAttributeValue to map[string]*dynamodb.AttributeValue like:
func getAttributeValueMapFromDynamoDBStreamRecord(e events.DynamoDBStreamRecord) map[string]*dynamodb.AttributeValue {
image := e.NewImage
m := make(map[string]*dynamodb.AttributeValue)
for k, v := range image {
if v.DataType() == events.DataTypeString {
s := v.String()
m[k] = &dynamodb.AttributeValue{
S : &s,
}
}
if v.DataType() == events.DataTypeBoolean {
b := v.Boolean()
m[k] = &dynamodb.AttributeValue{
BOOL : &b,
}
}
// . . .
if v.DataType() == events.DataTypeMap {
// ?
}
}
return m
}
And then simply use dynamodbattribute.UnmarshalMap, but on events.DataTypeMap it would be quite a tricky process.
Is there a way through which I can unmarshal a DynamoDB record coming from a events.DynamoDBEvent into a struct with a similar method shown for map[string]*dynamodb.AttributeValue?
I tried the function you provided, and I met some problems with events.DataTypeList, so I managed to write the following function that does the trick:
// UnmarshalStreamImage converts events.DynamoDBAttributeValue to struct
func UnmarshalStreamImage(attribute map[string]events.DynamoDBAttributeValue, out interface{}) error {
dbAttrMap := make(map[string]*dynamodb.AttributeValue)
for k, v := range attribute {
var dbAttr dynamodb.AttributeValue
bytes, marshalErr := v.MarshalJSON(); if marshalErr != nil {
return marshalErr
}
json.Unmarshal(bytes, &dbAttr)
dbAttrMap[k] = &dbAttr
}
return dynamodbattribute.UnmarshalMap(dbAttrMap, out)
}
I was frustrated that the type of NewImage from the record wasn't map[string]*dynamodb.AttributeValue so I could use the dynamodbattribute package.
The JSON representation of events.DynamoDBAttributeValue seems to be the same as the JSON represenation of dynamodb.AttributeValue.
So I tried creating my own DynamoDBEvent type and changed the type of OldImage and NewImage, so it would be marshalled into map[string]*dynamodb.AttributeValue instead of map[string]events.DynamoDBAttributeValue
It is a little bit ugly but it works for me.
package main
import (
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"fmt"
)
func main() {
lambda.Start(lambdaHandler)
}
// changed type of event from: events.DynamoDBEvent to DynamoDBEvent (see below)
func lambdaHandler(event DynamoDBEvent) error {
for _, record := range event.Records {
change := record.Change
newImage := change.NewImage // now of type: map[string]*dynamodb.AttributeValue
var item IdOnly
err := dynamodbattribute.UnmarshalMap(newImage, &item)
if err != nil {
return err
}
fmt.Println(item.Id)
}
return nil
}
type IdOnly struct {
Id string `json:"id"`
}
type DynamoDBEvent struct {
Records []DynamoDBEventRecord `json:"Records"`
}
type DynamoDBEventRecord struct {
AWSRegion string `json:"awsRegion"`
Change DynamoDBStreamRecord `json:"dynamodb"`
EventID string `json:"eventID"`
EventName string `json:"eventName"`
EventSource string `json:"eventSource"`
EventVersion string `json:"eventVersion"`
EventSourceArn string `json:"eventSourceARN"`
UserIdentity *events.DynamoDBUserIdentity `json:"userIdentity,omitempty"`
}
type DynamoDBStreamRecord struct {
ApproximateCreationDateTime events.SecondsEpochTime `json:"ApproximateCreationDateTime,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
Keys map[string]*dynamodb.AttributeValue `json:"Keys,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
NewImage map[string]*dynamodb.AttributeValue `json:"NewImage,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
OldImage map[string]*dynamodb.AttributeValue `json:"OldImage,omitempty"`
SequenceNumber string `json:"SequenceNumber"`
SizeBytes int64 `json:"SizeBytes"`
StreamViewType string `json:"StreamViewType"`
}
I have found the same problem and the solution is to perform a simple conversion of types. This is possible because in the end the type received by lambda events events.DynamoDBAttributeValue and the type used by the SDK V2 of AWS DynamoDB types.AttributeValue are the same. Next I show you the conversion code.
package aws_lambda
import (
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
func UnmarshalDynamoEventsMap(
record map[string]events.DynamoDBAttributeValue, out interface{}) error {
asTypesMap := DynamoDbEventsMapToTypesMap(record)
err := attributevalue.UnmarshalMap(asTypesMap, out)
if err != nil {
return err
}
return nil
}
func DynamoDbEventsMapToTypesMap(
record map[string]events.DynamoDBAttributeValue) map[string]types.AttributeValue {
resultMap := make(map[string]types.AttributeValue)
for key, rec := range record {
resultMap[key] = DynamoDbEventsToTypes(rec)
}
return resultMap
}
// DynamoDbEventsToTypes relates the dynamo event received by AWS Lambda with the data type that is
// used in the Amazon SDK V2 to deal with DynamoDB data.
// This function is necessary because Amazon does not provide any kind of solution to make this
// relationship between types of data.
func DynamoDbEventsToTypes(record events.DynamoDBAttributeValue) types.AttributeValue {
var val types.AttributeValue
switch record.DataType() {
case events.DataTypeBinary:
val = &types.AttributeValueMemberB{
Value: record.Binary(),
}
case events.DataTypeBinarySet:
val = &types.AttributeValueMemberBS{
Value: record.BinarySet(),
}
case events.DataTypeBoolean:
val = &types.AttributeValueMemberBOOL{
Value: record.Boolean(),
}
case events.DataTypeList:
var items []types.AttributeValue
for _, value := range record.List() {
items = append(items, DynamoDbEventsToTypes(value))
}
val = &types.AttributeValueMemberL{
Value: items,
}
case events.DataTypeMap:
items := make(map[string]types.AttributeValue)
for k, v := range record.Map() {
items[k] = DynamoDbEventsToTypes(v)
}
val = &types.AttributeValueMemberM{
Value: items,
}
case events.DataTypeNull:
val = nil
case events.DataTypeNumber:
val = &types.AttributeValueMemberN{
Value: record.Number(),
}
case events.DataTypeNumberSet:
val = &types.AttributeValueMemberNS{
Value: record.NumberSet(),
}
case events.DataTypeString:
val = &types.AttributeValueMemberS{
Value: record.String(),
}
case events.DataTypeStringSet:
val = &types.AttributeValueMemberSS{
Value: record.StringSet(),
}
}
return val
}
There is a package that allows conversion from events.DynamoDBAttributeValue to dynamodb.AttributeValue
https://pkg.go.dev/github.com/aereal/go-dynamodb-attribute-conversions/v2
From there one can unmarshal AttributeValue into struct
func Unmarshal(attribute map[string]events.DynamoDBAttributeValue, out interface{}) error {
av := ddbconversions.AttributeValueMapFrom(attribute)
return attributevalue.UnmarshalMap(av, out)
}