I have a Dynamodb Table that has the following struct:
type StatusItem struct {
requestStatus string
timestamp string
RequestId string
}
I have the code as such:
items := []StatusItem{}
err = dynamodbattribute.UnmarshalListOfMaps(result.Items, &items)
if err != nil {
exitWithError(fmt.Errorf("failed to unmarshal Query result items, %v", err))
}
// Print out the items returned
for i, item := range items {
fmt.Println(i)
fmt.Printf(item.RequestId, item.requestStatus)
}
However, items is empty.
when i put a watcher in items, i can see the it unmarshals to the struct, but all the values are empty.
if i do:
log.println(result.items) i can see the values there (though a bit ugly)
%!(EXTRA string=)[map[timestamp:{
S: "2018-04-18 12:04:43.761"
} RequestId:{
S: "9"
} requestStatus:{
S: "REQUEST_RECEIVED"
}]
what am i doing wrong?
Thanks to #Peter for pointing me in the right direction.
The solution is simple (though rather dumb that that was the case).
as noted in : https://til.hashrocket.com/posts/3512417fb0-private-vs-public-struct-members
lowercase starting members are private, and Capital was public.
in order to get it working correctly, i needed to update my struct from:
type StatusItem struct {
requestStatus string
timestamp string
RequestId string
}
to:
type StatusItem struct {
RequestStatus string
Timestamp string
RequestId string
}
Related
Let's say I have the first struct as
type Person struct {
Name string `json:"person_name"`
Age int `json:"person_age"`
Data map[string]interface{} `json:"data"`
}
and I am trying to marshal an array of the above struct
Things work well till here and a sample response I receive is
[
{
"person_name":"name",
"person_age":12,"data":{}
},
{
"person_name":"name2",
"person_age":12,"data":{}
}
]
Now, I need to append another struct over here and the final response should like
[
{
"person_name":"name",
"person_age":12,"data":{}
},
{
"person_name":"name2",
"person_age":12,"data":{}
},
{
"newData":"value"
}
]
So can someone help on this and how i can achieve this ?
I tried by creating an []interface{} and then iterating over person to append each data, but the issue in this approach is that it makes the Data as null if in case it's an empty string.
I would need it be an empty map only.
Let me prefix this by saying this looks to me very much like you might be dealing with an X-Y problem. I can't really think of many valid use-cases where one would end up with a defined data-type that has to somehow be marshalled alongside a completely different, potentially arbitrary/freeform data structure. It's possible, though, and this is how you could do it:
So you just want to append a completely different struct to the data-set, then marshal it and return the result as JSON? You'll need to create a new slice for that:
personData := []Person{} // person 1 and 2 here
more := map[string]string{ // or some other struct
"newdata": "value",
}
allData := make([]any, 0, len(personData) + 1) // the +1 is for the more, set cap to however many objects you need to marshal
for _, p := range personData {
allData = append(allData, p) // copy over to this slice, because []Person is not compatible with []any
}
allData = append(allData, more)
bJSON, err := json.Marshal(allData)
if err != nil {
// handle
}
fmt.Println(string(bJSON))
Essentially, because you're trying to marshal a slice containing multiple different types, you have to add all objects to a slice of type any (short for interface{}) before marshalling it all in one go
Cleaner approaches
There are much, much cleaner approaches that allow you to unmarshal the data, too, assuming the different data-types involved are known beforehand. Consider using a wrapper type like so:
type Person struct {} // the one you have
type NewData {
NewData string `json:"newdata"`
}
type MixedData struct {
*Person
*NewData
}
In this MixedData type, both Person and NewData are embedded, so MixedData will essentially act as a merged version of all embedded types (fields with the same name should be overridden at this level). With this type, you can marshal and unmarshal the JSON accordingly:
allData := []MixedData{
{
Person: &person1,
},
{
Person: &person2,
},
{
NewData: &newData,
},
}
Similarly, when you have a JSON []byte input, you can unmarshal it same as you would any other type:
data := []MixedData{}
if err := json.Unmarshal(&data, in); err != nil {
// handle
}
fmt.Printf("%#v\n", data) // it'll be there
It pays to add some functions/getters to the MixedData type, though:
func (m MixedData) IsPerson() bool { return m.Person != nil }
func (m MixedData) Person() *Person {
if m.Person == nil {
return nil
}
cpy := *m.Person // create a copy to avoid shared pointers
return &cpy // return pointer to the copy
}
Do the same for all embedded types and this works like a charm.
As mentioned before, should your embedded types contain fields with the same name, then you should override them in the MixedData type. Say you have a Person and Address type, and both have an ID field:
type MixedData struct {
ID string `json:"id"`
*Person
*Address
}
This will set the ID value on the MixedData type, and all other (non-shared) fields on the corresponding embedded struct. You can then use the getters to set the ID where needed, or use a custom unmarshaller, but I'll leave that to you to implement
I'm relatively new to Go and am working on building out a request decoder. The request comes in JSON format and we decode that to a map[string]interface{}. We then pass that object data in to be decoded to our own ProcessRequest struct. As I said I'm new so I am reusing some logic in similar parts of the code wrote by previous developers. Essentially we are checking the map for the necessary pieces and then setting and returning those. Can someone explain to me why I am getting the titled error? Would I have to set the items all the way down to base structs that no longer have any nested? Is there a better way to accomplish what I want? Here is the code and the related structs. It is highlighting the error on the return of the model.ProcessRequest. TYIA
type ProcessRequest struct {
RequestID string
Message *message.Message
Rule *Rule
Options *ProcessOptions
//TODO: Context EvaluatorContext
//TODO: Links
}
type Message struct {
ID int
Key string
Source string
Headers *HeaderSet
Categories *CategorySet
Properties *PropertySet
Streams *StreamSet
}
type RuleAction struct {
Name string
Expression map[string]interface{}
}
type RuleLink struct {
LinkID int
Condition map[string]interface{}
TargetRuleID int
}
type Rule struct {
Name string
Code string
Actions []RuleAction
Links []RuleLink
}
type object = map[string]interface{}
func DecodeProcessRequest(dataObject map[string]interface{}) (*model.ProcessRequest, error) {
var (
requestID string
message *message.Message
rule *model.Rule
options *model.ProcessOptions
err error
)
if reqIDSrc, ok := dataObject["requestId"]; ok {
if requestID, err = converter.ToString(reqIDSrc); err != nil {
return nil, errors.Wrapf(err, "Error reading property 'requestID'")
}
if requestID == "" {
return nil, errors.Errorf("Property 'requestID' is an empty string")
}
} else {
return nil, errors.Errorf("Missing required property 'requestID'")
}
if messageSrc, ok := dataObject["message"]; ok {
messageData, ok := messageSrc.(object)
if !ok {
return nil, errors.Errorf("Error reading property 'message': Value is not an object")
}
if message, err = DecodeMessage(messageData); err != nil {
return nil, errors.Wrapf(err, "Error reading property 'message'")
}
} else {
return nil, errors.Errorf("Missing required property 'message'")
}
if ruleSrc, ok := dataObject["rule"]; ok {
ruleObj, ok := ruleSrc.(object)
if !ok {
return nil, errors.Errorf("Error reading property 'rule': Value is not an object")
}
if rule, err = DecodeRule(ruleObj); err != nil {
return nil, errors.Wrapf(err, "Error reading 'rule' during decoding")
}
} else {
return nil, errors.Errorf("Missing required property 'requestID'")
}
// Parse plain object to a Message struct
return &model.ProcessRequest{
requestID,
message,
rule,
options,
}, nil
}
super said in this comment:
In general, the warning says that you should prefer to use the syntax ProcessRequest{ RequestID: requestID, ... }. Naming the keys instead of unkeyed values.
That worked for me. Also the explanation by kostix in this comment really helped.
Basically the idea is that if you use "unkeyed" way of defining struct literals, the meaning of your definitions depends on the way the fields of the underlying type are layed out. Now consider that your type has three fields of type string in a certain order. Now a couple of iterations down the road some programmer moves the second field to the 1st position—your literal will still compile but will end up defining a completely different value at runtime.
I have two structs in golang as below
type Data struct {
Name string
Description string
HasMore bool
}
type DataWithItems struct {
Name string
Description string
HasMore bool
Items []Items
}
At most DataWithItems struct can be rewritten as
type DataWithItems struct {
Info Data
Items []Items
}
But the above make it difficult when decoding a json object into DataWithItems. I know this can be solved with inheritance in other programming languages but Is there a way I can solve this in Go?
You can "embed" the one struct into the other:
type Items string
type Data struct {
Name string
Description string
HasMore bool
}
type DataWithItems struct {
Data // Notice that this is just the type name
Items []Items
}
func main() {
d := DataWithItems{}
d.Data.Name = "some-name"
d.Data.Description = "some-description"
d.Data.HasMore = true
d.Items = []Items{"some-item-1", "some-item-2"}
result, err := json.Marshal(d)
if err != nil {
panic(err)
}
println(string(result))
}
this prints
{"Name":"some-name","Description":"some-description","HasMore":true,"Items":["some-item-1","some-item-2"]}
Just use one struct - DataWithItems and sometimes leave items blank
All I looking to do is to create an array of struct Response from a json encoded file.
the file that contains json data looks like this.
cat init.txt
{"events": [{"action":"cpr","id":69,"sha1":"abc","cpr":"cpr_data0"},{"action":"cpr","id":85,"sha1":"def","cpr":"cpr_data1"}]}
The way I have gone about approaching this is
I created a response of type map[string][]Response
.. decoded the JSON from the file
.. created a responseStruct of type []Response
But somehow when I inspect the Value they all look 0 or empty
map[events:[{ 0 } { 0 }]
What is wrong with the approach mentioned above.
type Response struct {
action string `json:"action"`
id int64 `json:"id"`
sha1 string `json:"sha1"`
cpr string `json:"cpr"`
}
func main() {
file, err := os.Open("init.txt")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
var response map[string][]Response
err = json.NewDecoder(file).Decode(&response)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
var responseArray []Response
responseArray = response["events"]
for _, responseStruct := range responseArray {
fmt.Println("id =", responseStruct.id)
fmt.Println("action =", responseStruct.action)
fmt.Println("sha1 = ", responseStruct.sha1)
fmt.Println("cpr =", responseStruct.cpr)
fmt.Println("==========")
}
fmt.Println(response)
}
Well If I modify the struct to look like this it works
type Response struct {
Action string `json:"action"`
ID int64 `json:"id"`
Sha1 string `json:"sha1"`
Cpr string `json:"cpr"`
}
So my question is this how the stuff would work, can't I get the above code to work in the way it is?
Go has the notion of public and private fields in objects, and the only distinction is whether they start with an initial capital letter or not. This applies to not just code modules but also the reflect package and things that use it. So in your initial version
type Response struct {
action string `json:"action"`
...
}
nothing outside your source package, not even the "encoding/json" module, can see the private fields, so it can't fill them in. Changing these to public fields by capitalizing Action and the other field names makes them visible to the JSON decoder.
Lowercase struct elements in golang are private, Hence json decoder (which is an external package) can't access them.
Its able to create the struct object but not able to set values. They appear zero because they 0 is default value.
I was wondering if someone could explain how to unmarshal my changefeed cursor value to a specific struct type.
var message map[string]interface{}
for chatFeedCursor.Next(&message) {
fmt.Println(message)
}
map[new_val:map[club_id:ea2eb6e2-755f-4dad-922d-e3693b6e55c6
date:2017-04-07 14:48:17.714 +0100 +01:00
id:e389ab54-963e-4b33-9b34-adcb6ec5b17e message:what is the meaning of life?
user_id:00ff679f-9421-4b8b-ae7f-d11cf2adaee2] old_val:]
However, I would like the response to be mapped to struct ChatMessage.
Update:
I've tried:
var message ChatMessage
However, it doesn't seem like any of my data gets set in the struct.
{ 0001-01-01 00:00:00 +0000 UTC}
My struct:
type ChatMessage struct {
ID string `json:"id" gorethink:"id,omitempty"`
UserID string `json:"user_id" gorethink:"user_id"`
ClubID string `json:"club_id" gorethink:"club_id"`
Message string `json:"message" gorethink:"message"`
Date time.Time `json:"date" gorethink:"date"`
}
Thanks.
I figured it out!
The problem was that I didn't specify a field on the rethinkdb change request.
Previous code:
chatFeedCursor, _ := gorethink.Table("club_chat").Changes().Run(gorethinkSession)
Working Code:
chatFeedCursor, _ := gorethink.Table("club_chat").Changes().Field("new_val").Run(gorethinkSession)
Now the .Next() value maps to my struct with no issues.
If you want to unmarshal the changefeed cursor value to a specific struct type, but keep both the new and the old value, you could use a library for decoding generic map values into native Go structures, like mapstructure, to obtain a neat result:
Import mapstructure:
import (
...
"github.com/mitchellh/mapstructure"
...
)
Now you can do something like:
cursor, err := r.Table("club_chat").
Changes().
Run(db.Session)
if err != nil {
// manage error
}
var changeFeed map[string]interface{}
for cursor.Next(&changeFeed) {
var oldVal, newVal ChatMessage
if changeFeed["old_val"] != nil {
mapstructure.Decode(changeFeed["old_val"], &oldVal)
}
if changeFeed["new_val"] != nil {
mapstructure.Decode(changeFeed["new_val"], &newVal)
}
// Do what you want with the values