There is the following field on dynamo
{
"config": {
"BASE_AUTH_URL_KEY": "https://auth.blab.bob.com",
"BASE_URL": "https://api.dummy.data.com",
"CONN_TIME_OUT_SECONDS": "300000",
"READ_TIME_OUT_SECONDS": "300000"
},
"id": "myConfig"
}
and getting the element with dynamodbattribute
import(
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute")
result, err := svc.GetItem(&dynamodb.GetItemInput{
TableName: aws.String(tableName),
Key: map[string]*dynamodb.AttributeValue{
"id": {
S: aws.String(configId),
},
},
})
this code its working but when i try to retrieve the object its rendered like this
map[config:{
M: {
BASE_AUTH_URL_KEY: {
S: "https://auth.blab.bob.com"
},
CONN_TIME_OUT_SECONDS: {
S: "300000"
},
READ_TIME_OUT_SECONDS: {
S: "300000"
},
BASE_URL: {
S: "https://api.dummy.data.com"
}
}
} id:{
S: "myConfig"
}]
for that reason when i try to unmarshal my object the object unmarshalled returns as {}
type Config struct {
id string
baseAuthUrlKey string
baseUrl string
connectTimeOutSecs string
readTimeOutSecs string
}
item := Config{}
err = dynamodbattribute.UnmarshalMap(result.Item, &item)
how can i assign the value return from the GetItem that seems to be a map to my struct ?
The root of the issue is that your Config struct is incorrectly structured.
I recommend using json-to-go when converting JSON to Go structs; this tool will help you catch issues like this in the future.
Once you get your struct constructed correctly, you'll also notice that your struct fields are not capitalized, meaning they will not be exported (i.e. able to be used by other packages), which is another reason that your UnmarshalMap code will not return the result you are expecting.
Here is a good answer on struct field visibility and its importance, briefly summarized above.
Below is a corrected version of your struct that, combined with your UnmarshalMap code, will correctly allow you to print your item and not receive a {} which is no fun.
type Item struct {
Config struct {
BaseAuthUrlKey string `json:"BASE_AUTH_URL_KEY"`
BaseUrl string `json:"BASE_URL"`
ConnTimeoutSeconds string `json:"CONN_TIME_OUT_SECONDS"`
ReadTimeoutSeconds string `json:"READ_TIME_OUT_SECONDS"`
} `json:"config"`
ID string `json:"id"`
}
Related
I'm trying to get values/index/key of a JSON but no success. I found some answers and tutorials but I couldn't get them working for me.
I have the following JSON:
{
"id": "3479",
"product": "Camera",
"price": "",
"creation": 04032020,
"products": [
{
"camera": "Nikon",
"available": true,
"freeshipping": false,
"price": "1,813",
"color": "black"
},
{
"camera": "Sony",
"available": true,
"freeshipping": true,
"price": "931",
"color": "black"
}
],
"category": "eletronics",
"type": "camera"
}
I have tried several examples but none worked for this type of Json.
The error I'm getting:
panic: interface conversion: interface {} is nil, not
map[string]interface {}
I believe it's because of "products[]" I tried map[string]interface{} and []interface{} it compiles but gives me that error afterwards.
Could you give me an example on how I can extract these values?
The code I'm using:
//gets the json
product,_:=conn.GetProductData(shop.Info.Id)
// assing a variable
productInformation:=<-product
//prints the json
fmt.Printf(productInformation)
//try to get json values
type Product struct {
Id string
Product string
}
var product Product
json.Unmarshal([]byte(productInformation), &product)
fmt.Printf("Id: %s, Product: %s", product.Id, product.Product)
This code does not panic but it doesn't print all results either so I tried
this one below (which was supposed to give me all results) but it panics
var result map[string]interface{}
json.Unmarshal([]byte(productInformation), &result)
// The object stored in the "birds" key is also stored as
// a map[string]interface{} type, and its type is asserted from
// the interface{} type
products := result["products"].(map[string]interface{})
for key, value := range products {
// Each value is an interface{} type, that is type asserted as a string
fmt.Println(key, value.(string))
}
You need to add json tag to specify the field name in json as it is in lowercase
type Product struct {
Id string `json:"id"`
Product string `json:"product"`
}
And in the second case according to json it is a slice not a map so you need to cast it to []interface{}
var result map[string]interface{}
json.Unmarshal([]byte(productInformation), &result)
// The object stored in the "birds" key is also stored as
// a map[string]interface{} type, and its type is asserted from
// the interface{} type
products := result["products"].([]interface{})
for key, value := range products {
// Each value is an interface{} type, that is type asserted as a string
fmt.Println(key, value)
}
(Note, Dgraph isn't required for this problem; it's just the tool I am using where I am encountering the problem)
Let's say you have some structs
type AccountSettings struct {
Uid string `json:"uid"`
DType string `json:"dgraph.type"`
Name string `json:"name"`
Username string `json:"username"`
}
type Settings struct {
Uid string `json:"uid"`
DType string `json:"dgraph.type"`
Account AccountSettings `json:"account"`
}
type Tag struct {
Uid string `json:"uid"`
DType string `json:"dgraph.type"`
Label Label `json:"label"`
}
type User struct {
Uid string `json:"uid"`
DType string `json:"dgraph.type"`
Settings Settings `json:"settings"`
Tags []Tag `json:"tags"`
}
and you create a user and add it to your Dgraph database
user = User{
DType: "User",
Settings: Settings{
DType: "Settings",
Account: SettingsAccount{
DType: "SettingsAccount",
Name: "Billy Bob"
Username: "billyboy123"
},
Tags: []Tag{{
DType: "Tag",
Label: "admin"
}, {
DType: "Tag",
Label: "user"
}}
},
}
And then you query your Dgraph database like (assuming you have the uid of the user node)
{
user(func: uid("0x1234")) {
uid
settings {
account {
name
username
}
}
tags {
label
}
}
}
the response will come back as
{
"data": {
"user": [
{
"uid": "0x1234"
"settings": [
{
"account": [
{
"name": "Billy Bob",
"username": "billyboy123"
}
]
}
]
"tags": [
{
label: "admin"
},
{
label: "user"
},
]
}
]
}
}
So the problem is that the JSON response of Dgraph will return fields of struct types as arrays (presumably because there could potentially be multiple nodes pointing to a single other node), but since it's an array, you can't immediately marshal it with the User struct (because User.Settings is of type Settings, not []Settings).
What would you do in order to marshal the Dgraph JSON response with the User struct (or any other kind of struct)?
It's important to note that the settings array in the JSON response should just be the first element in that array, but the tags array should still remain an array because that's what's specified in the User struct.
You can do this with a custom Unmarshal as described here:
http://choly.ca/post/go-json-marshalling/
You use an alias by embedding the original User in an anonymous type that has a slice of Settings inside it that you Umarshal into like so:
func (u *User) UnmarshalJSON(data []byte) error {
type Alias User
aux := &struct {
Settings []*Settings `json:"settings"`
*Alias
}{
Alias: (*Alias)(u),
}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
u.Settings = aux.Settings[0]
return nil
}
A full working example is here (you need the same trick for the AccountSettings):
https://play.golang.org/p/FQWakX8B7OF
The problem:
I've got a map of structs in another struct and I'd like to initialize the nested map of structs, but apparently that is not possible.
Code:
type Exporter struct {
TopicsByName map[string]Topic
}
type Topic struct {
Name string
Partitions map[int32]Partition
}
type Partition struct {
PartitionID int32
HighWaterMark int64
}
// Eventually I want to do something like:
e := Exporter{ TopicsByName: make(map[string]Topic) }
for _, topicName := range topicNames {
// This does not work because "cannot assign to struct field e.TopicsByName[topicName].Partitions in map"
e.TopicsByName[topicName].Partitions = make(map[int32]Partition)
}
// I wanted to initialize all these maps so that I can do
e.TopicsByName[x.TopicName].Partitions[x.PartitionID] = Partition{...}
I don't understand why I can not initialize the nested struct map above. Is it so bad to nest maps with struct as values? How can I fix this?
It is not possible to assign to a field in a map value. The fix is to
assign a struct value to the map value:
for _, topicName := range []string{"a"} {
e.TopicsByName[topicName] = Topic{Partitions: make(map[int32]Partition)}
}
You can initialize it like you'd expect:
e := Exporter{
TopicsByName: map[string]Topic{
"my-topic": Topic{
Name: "my-topic",
Partitions: map[int32]Partition{
int32(1): Partition{
PartitionID: int32(1),
HighWaterMark: int64(199),
},
},
},
},
}
This isn't directly answering the question, since you wanted to iterate over a list of topics, but if this is used in kafka testing, I highly recommend the more verbose/literal format above.
https://play.golang.org/p/zm3A7CIvbyu
If you've known an initialed value. Why don't do it like
val := `
{
"topics_by_name": {
"name1": {
"name":"topicname1",
"partions": {
1: {
"partion_id":1,
"high_water_mark":12,
},
2: {}
}
},
"name2": {}
}
}
`
func main() {
var m map[string]interface{}
// if m has be well designed, use your designed map is better
// init
json.Unmarshal(val, &m)
}
I am getting nested data from mongo and I want to flatten that out in a structure to store it in a csv file.
The data looks like this:
{
"_id" : "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
"data" : {
"driver" : {
"etaToStore" : 156
},
"createdAt" : 1532590052,
"_id" : "07703a33-a3c3-4ad5-9e06-d05063474d8c"
}
}
And the structure I want to eventually get should be something like this
type EventStruct struct {
Id string `bson:"_id"`
DataId string `bson:"data._id"`
EtaToStore string `bson:"data.driver.etaToStore"`
CreatedAt int `bson:"data.createdAt"`
}
This doesn't work, so following some SO answers I broke it down into multiple structures:
// Creating a structure for the inner struct that I will receive from the query
type DriverStruct struct {
EtaToStore int `bson:"etaToStore"`
}
type DataStruct struct {
Id string `bson:"_id"`
Driver DriverStruct `bson:"driver"`
CreatedAt int `bson:"createdAt"`
}
// Flattenning out the structure & getting the fields we need only
type EventStruct struct {
Id string `bson:"_id"`
Data DataStruct `bson:"data"`
}
This gets all the data from the Mongo query result but it's nested:
{
"Id": "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
"Data": {
"Id": a33-a3c3-4ad5-9e06-d05063474d8c,
"Driver": {
"EtaToStore": 156
},
"CreatedAt": 1532590052
}
}
What I want to end up with is:
{
"Id": "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
"DataId": "a33-a3c3-4ad5-9e06-d05063474d8c",
"EtaToStore": 156,
"CreatedAt": 1532590052
}
I'm sure there's an easy way to do this but I can't figure it out, help!
You can implement the json.Unmarshaler interface to add a custom method to unmarshal the json. Then in that method, you can use the nested struct format, but return the flattened one at the end.
func (es *EventStruct) UnmarshalJSON(data []byte) error {
// define private models for the data format
type driverInner struct {
EtaToStore int `bson:"etaToStore"`
}
type dataInner struct {
ID string `bson:"_id" json:"_id"`
Driver driverInner `bson:"driver"`
CreatedAt int `bson:"createdAt"`
}
type nestedEvent struct {
ID string `bson:"_id"`
Data dataInner `bson:"data"`
}
var ne nestedEvent
if err := json.Unmarshal(data, &ne); err != nil {
return err
}
// create the struct in desired format
tmp := &EventStruct{
ID: ne.ID,
DataID: ne.Data.ID,
EtaToStore: ne.Data.Driver.EtaToStore,
CreatedAt: ne.Data.CreatedAt,
}
// reassign the method receiver pointer
// to store the values in the struct
*es = *tmp
return nil
}
Runnable example: https://play.golang.org/p/83VHShfE5rI
This question is a year and a half old, but I ran into it today while reacting to an API update which put me in the same situation, so here's my solution (which, admittedly, I haven't tested with bson, but I'm assuming the json and bson field tag reader implementations handle them the same way)
Embedded (sometimes referred to as anonymous) fields can capture JSON, so you can compose several structs into a compound one which behaves like a single structure.
{
"_id" : "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
"data" : {
"driver" : {
"etaToStore" : 156
},
"createdAt" : 1532590052,
"_id" : "07703a33-a3c3-4ad5-9e06-d05063474d8c"
}
}
type DriverStruct struct {
EtaToStore string `bson:"etaToStore"`
type DataStruct struct {
DriverStruct `bson:"driver"`
DataId string `bson:"_id"`
CreatedAt int `bson:"createdAt"`
}
type EventStruct struct {
DataStruct `bson:"data"`
Id string `bson:"_id"`
}
You can access the nested fields of an embedded struct as though the parent struct contained an equivalent field, so e.g. EventStructInstance.EtaToStore is a valid way to get at them.
Benefits:
You don't have to implement the Marshaller or Unmarshaller interfaces, which is a little overkill for this problem
Doesn't require any copying fields between intermediate structs
Handles both marshalling and unmarshalling for free
Read more about embedded fields here.
You can use basically the same logic as:
package utils
// FlattenIntegers flattens nested slices of integers
func FlattenIntegers(slice []interface{}) []int {
var flat []int
for _, element := range slice {
switch element.(type) {
case []interface{}:
flat = append(flat, FlattenIntegers(element.([]interface{}))...)
case []int:
flat = append(flat, element.([]int)...)
case int:
flat = append(flat, element.(int))
}
}
return flat
}
(Source: https://gist.github.com/Ullaakut/cb1305ede48f2391090d57cde355074f)
By adapting it for what's in your JSON. If you want it to be generic, then you'll need to support all of the types it can contain.
I have two struct having the same members, I want to copy one struct to another, see the pseudo code below:
type Common struct {
Gender int
From string
To string
}
type Foo struct {
Id string
Name string
Extra Common
}
type Bar struct {
Id string
Name string
Extra Common
}
Then I have foo of struct Foo, and bar of struct Bar, Is there any way to copy bar from foo?
Use a conversion to change the type. The following code uses a conversion to copy a value of type Foo to a value of type Bar:
foo := Foo{Id: "123", Name: "Joe"}
bar := Bar(foo)
playground example
The conversion only works when the underlying types are identical except for struct tags.
https://github.com/jinzhu/copier (same author of gorm) is also quite a good one, I have nested structs and all I do is:
copier.Copy(&employees, &user)
works great
If you would like to copy or clone to a different struct, I would suggest using deepcopier
It provides nice features like skipping, custom mapping, and forcing. below is an example from github:
Install:
go get -u github.com/ulule/deepcopier
Example:
package main
import (
"fmt"
"github.com/ulule/deepcopier"
)
// Model
type User struct {
// Basic string field
Name string
// Deepcopier supports https://golang.org/pkg/database/sql/driver/#Valuer
Email sql.NullString
}
func (u *User) MethodThatTakesContext(ctx map[string]interface{}) string {
// do whatever you want
return "hello from this method"
}
// Resource
type UserResource struct {
//copy from field "Name"
DisplayName string `deepcopier:"field:Name"`
//this will be skipped in copy
SkipMe string `deepcopier:"skip"`
//this should call method named MethodThatTakesContext
MethodThatTakesContext string `deepcopier:"context"`
Email string `deepcopier:"force"`
}
func main() {
user := &User{
Name: "gilles",
Email: sql.NullString{
Valid: true,
String: "gilles#example.com",
},
}
resource := &UserResource{}
deepcopier.Copy(user).To(resource)
//copied from User's Name field
fmt.Println(resource.DisplayName)//output: gilles
fmt.Println(resource.Email) //output: gilles#example.com
fmt.Println(resource.MethodThatTakesContext) //output: hello from this method
}
Also, some other way you could achieve this is by encoding the source object to JSON and then decode it back to the destination object.
This is another possible answer
type Common struct {
Gender int
From string
To string
}
type Foo struct {
Id string
Name string
Extra Common
}
type Bar struct {
Id string
Name string
Extra Common
}
foo:=Foo{
Id:"123",
Name:"damitha",
Extra: struct {
Gender int
From string
To string
}{Gender:1 , From:"xx", To:"yy" },
}
bar:=*(*Bar)(unsafe.Pointer(&foo))
fmt.Printf("%+v\n",bar)
If you would like to copy or clone to a different struct, I would suggest using deepcopier
It provides nice features like skipping, custom mapping, and forcing.
You can achieve nested struct copy following way.
Install:
go get -u github.com/ulule/deepcopier
Example:
package main
import (
"fmt"
"strconv"
"github.com/ulule/deepcopier"
)
//FieldStruct - Field Struct
type FieldStruct struct {
Name string `deepcopier:"field:TargetName"`
Type string `deepcopier:"field:TargetType"`
}
//SourceStruct - Source Struct
type SourceStruct struct {
Name string `deepcopier:"field:TargetName"`
Age int `deepcopier:"field:TargetAge"`
StringArray []string `deepcopier:"field:TargetStringArray"`
StringToInt string `deepcopier:"context"`
Field FieldStruct
Fields []FieldStruct
}
//TargetFieldStruct - Field Struct
type TargetFieldStruct struct {
TargetName string
TargetType string
}
//TargetStruct - Target Struct
type TargetStruct struct {
TargetName string
TargetAge int
TargetStringArray []string
TargetInt int
TargetField TargetFieldStruct
TargetFields []TargetFieldStruct
}
//write methods
//TargetInt - StringToInt
func (s *SourceStruct) TargetInt() int {
i, _ := strconv.Atoi(s.StringToInt)
return i
}
func main() {
s := &SourceStruct{
Name: "Name",
Age: 12,
StringArray: []string{"1", "2"},
StringToInt: "123",
Field: FieldStruct{
Name: "Field",
Type: "String",
},
Fields: []FieldStruct{
FieldStruct{
Name: "Field1",
Type: "String1",
},
FieldStruct{
Name: "Field2",
Type: "String2",
},
},
}
t := &TargetStruct{}
//coping data into inner struct
deepcopier.Copy(&t.TargetField).From(&s.Field)
// copied array of Struct
for i := range s.Fields {
// init a struct
t.TargetFields = append(t.TargetFields, TargetFieldStruct{})
// coping the data
deepcopier.Copy(&t.TargetFields[i]).From(&s.Fields[i])
}
//Top level copy
deepcopier.Copy(t).From(s)
fmt.Println(t)
}
Output:
&{Name 12 [1 2] 123 {Field String} [{Field1 String1} {Field2 String2}]}