Better way of decoding json values - go

Assume a JSON object with the general format
"accounts": [
{
"id": "<ACCOUNT>",
"tags": []
}
]
}
I can create a struct with corresponding json tags to decode it like so
type AccountProperties struct {
ID AccountID `json:"id"`
MT4AccountID int `json:"mt4AccountID,omitempty"`
Tags []string `json:"tags"`
}
type Accounts struct {
Accounts []AccountProperties `json:"accounts"`
}
But the last struct with just one element seems incorrect to me. Is there a way I could simply say type Accounts []AccountProperties `json:"accounts"` instead of creating an entire new struct just to decode this object?

You need somewhere to store the json string accounts. Using a:
var m map[string][]AccountProperties
suffices, though of course you then need to know to use the string literal accounts to access the (single) map entry thus created:
type AccountProperties struct {
ID string `json:"id"`
MT4AccountID int `json:"mt4AccountID,omitempty"`
Tags []string `json:"tags"`
}
func main() {
var m map[string][]AccountProperties
err := json.Unmarshal([]byte(data), &m)
fmt.Println(err, m["accounts"])
}
See complete Go Playground example (I had to change the type of ID to string and fix the missing { in the json).
As Dave C points out in comments, this is no shorter than just using an anonymous struct type:
var a struct{ Accounts []AccountProperties }
in terms of the Unmarshall call (and when done this way it's more convenient to use). Should you want to use an anonymous struct like this in a json.Marshall call, you'll need to tag its single element to get a lowercase encoding: without a tag it will be called "Accounts" rather than "accounts".
(I don't claim the map method to be better, just an alternative.)

Related

Dynamically Create Structs in Golang

So I am working with an external API, whose responses I wanted to parse. The incoming responses are of a fixed format i.e.
type APIResponse struct {
Items []interface{} `json:"items"`
QuotaMax int `json:"quota_max"`
QuotaRemaining int `json:"quota_remaining"`
}
So for each response I am parsing the items. Now the items can be of diff types as per the request. It can be a slice of sites, articles, etc. Which have their individual models. like:
type ArticleInfo struct {
ArticleId uint64 `json:"article_id"`
ArticleType string `json:"article_type"`
Link string `json:"link"`
Title string `json:"title"`
}
type SiteInfo struct {
Name string `json:"name"`
Slug string `json:"slug"`
SiteURL string `json:"site_url"`
}
Is there any way, when parsing the input define the type of Items in APIResponse. I don't want to create separate types for individual responses.
Basically want to Unmarshall any incoming response into the APIResponse struct.
Change type of the Items field to interface{}:
type APIResponse struct {
Items interface{} `json:"items"`
...
}
Set the response Items field to pointer of the desired type. Unmarshal to the response:
var articles []ArticleInfo
response := APIResponse{Items: &articles}
err := json.Unmarshal(data, &response)
Access the articles using variable articles.
Run an example on the playground.

Marshal struct field to JSON without field name

I need to marshal into this JSON format:
{"messageProtocolHandshake":[{"handshakeType":"announceMax"},{"version":[{"major":1},{"minor":0}]}]}
Problem is matching the handshakeType. My struct is
type MessageProtocolHandshake struct {
HandshakeType HandshakeType `json:"handshakeType"`
Version []Version `json:"version"`
}
type HandshakeType struct {
HandshakeType string
}
Marshaling can be done using slice of interface:
func (h MessageProtocolHandshake) MarshalJSON() ([]byte, error) {
res := make([]interface{}, 3)
res[0] = struct {
HandshakeType string `json:"handshakeType"`
}{h.HandshakeType.HandshakeType}
res[1] = struct {
Version []Version `json:"version"`
}{h.Version}
return json.Marshal(res)
}
Using a simple marshaler/unmarshaler takes away the surrounding curly brackets from the handshakeType, so that doesn't work:
{"messageProtocolHandshake":[{"handshakeType":"announceMax","version":[{"major":1,"minor":0}],"formats":[{"format":"JSON-UTF8"}]}]}
Seems as if Go applies some heuristic in that case on the retuned byte array (undocumented?).
Is there a more elegant way of omitting the structs outer field name?
--
UPDATE To summarise the answers: key is to think about different structs for marshalling and unmarshalling if nothing else works, potentially a using a 3rd presentation for working internally with the data.
When custom (Un)Marshalers come into play remember that promoted fields inherit their methods and hence influence parent structs.
The JSON that you specified has a different model from that of your struct.
There are a few approaches to aligning these: Change the specification of the JSON data to match your structs, change the structs to match the specification of the JSON, or create a new struct that is only used for marshaling.
I omit the last example, because it's very similar to the second method.
Changing the specification of the JSON
The following model stays the same:
type MessageProtocolHandshake struct {
HandshakeType HandshakeType `json:"handshakeType"`
Version []Version `json:"version"`
}
type HandshakeType struct {
HandshakeType string
}
The JSON for this would be:
{"handshakeType":{"HandshakeType":""},"version":[]}
You did not specify the Version type so I don't know how one would change the JSON for that.
Changing the structs
The following JSON stays the same:
{"messageProtocolHandshake":[{"handshakeType":"announceMax"},{"version":[{"major":1},{"minor":0}]}]}
The structs for this would be:
type Model struct {
MessageProtocolHandshake []interface{} `json:"messageProtocolHandshake"`
}
type HandshakeType struct {
HandshakeType string `json:"handshakeType"`
}
type Versions struct {
Version []Version `json:"version"`
}
type Version struct {
Major *int `json:"major,omitempty"`
Minor *int `json:"minor,omitempty"`
}
Unmarshaling would not be trivial.
https://play.golang.org/p/89WUhcMFM0B
As is obvious from the results, the models you are using are not good. If there's a way to change all of this, I would recommend starting from scratch, using the data that is necessary and creating the JSON specification from the structs.
I recommend reading up on JSON: https://www.json.org/json-en.html
Also, I recommend this introduction to Go and JSON: https://blog.golang.org/json

Use different json keys for marshalling and unmarshalling

What do I have: two structs for some kind of API
type BaseUser struct {
ID int64 `json:"user_id"`
Name string `json:"user_name"`
Email string `json:"user_email"`
}
and
type UserWithAddress struct {
BaseUser
Postal string `json:"user_postal"`
City string `json:"user_city"`
Street string `json:"user_street"`
}
What do I want to do: convert json keys from snake_case to camelCase.
Lets say, this is a request body
{
"user_id": 123,
"user_name": "test",
"user_email": "test#mail.com",
"user_postal": "12312",
"user_city": "city",
"user_street": "street"
}
So as a result, after some kind of transformation, I'd like to have this output
{
"userId": 123,
"userName": "test",
"userEmail": "test#mail.com",
"userPostal": "12312",
"userCity": "city",
"userStreet": "street"
}
How do I handle this at the moment: I made another two structs with camelCase json tag
type BaseUserCamelCase struct {
ID int64 `json:"userId"`
Name string `json:"userName"`
Email string `json:"userEmail"`
}
and
type UserWithAddressCamelCase struct {
BaseUserCamelCase
Postal string `json:"userPostal"`
City string `json:"userCity"`
Street string `json:"userStreet"`
}
My transformation looks like
var userWithAddressCamelCase UserWithAddressCamelCase
userWithAddressCamelCase.BaseUserCamelCase = BaseUserCamelCase(userWithAddress.BaseUser)
//I can't cast whole userWithAddressCamelCase object to another type because of different field names - BaseUser and BaseUserCamelCase
userWithAddressCamelCase.Name = userWithAddress.Name
userWithAddressCamelCase.Email = userWithAddress.Email
userWithAddressCamelCase.Postal = userWithAddress.Postal
//and so on
and I don't like it, because if BaseUser or UserWithAddress will grow up, I have to add appropriate field to %CamelCase structs.
My question: is there another more efficient way to handle keys transformation?
Is there another more efficient way to handle keys transformation?
No.
(Well, based on your definition of "efficient". You could use reflection, but I will not recommend this. Your code is perfectly fine. If any struct grows you add a few lines of simple code. There is nothing wrong with simple code which is not going to produce errors and is fast during execution. Just because it doesn't look fancy it doesn't mean that there is anything to "improve" here.)
If the need to maintain the field list is what concerns you the most, then I'd suggest making type aliases for your user types and implementing the json.Marshaler interface for those aliases, where you'd implement custom JSON encoding. You can even introduce an alternative set of tags and use those there.
Something along these lines:
type BaseUser struct {
ID int64 `json:"user_id" jsonCC:"userId"`
Name string `json:"user_name" jsonCC:"userName"`
Email string `json:"user_email" jsonCC:"userEmail"`
}
type BaseUserCamelCase BaseUser
func (bucc BaseUserCamelCase) MarshalJSON() ([]byte, error) {
buccVal := reflect.ValueOf(bucc)
kvpairs := []string{}
for i := 0; i < buccVal.NumField(); i++ {
k := buccVal.Type().Field(i).Tag.Get("jsonCC")
v := buccVal.Field(i).Interface() //TODO: proper JSON encoding of things
kvpairs = append(kvpairs, fmt.Sprintf("\"%s\":%#v", k, v))
}
return []byte(fmt.Sprintf("{%s}", strings.Join(kvpairs, ","))), nil
}
Then you can choose marshaling style:
user := BaseUser{
ID: 123,
Name: "Johnny D03",
Email: "j#example.com",
}
json.Marshal(user)
// {"user_id":123,"user_name":"Johnny D03","user_email":"j#example.com"}
json.Marshal(BaseUserCamelCase(user))
// {"userId":123,"userName":"Johnny D03","userEmail":"j#example.com"}

Conditional (Dynamic) Struct Tags

I'm trying to parse some xml documents in Go. I need to define a few structs for this purpose, and my struct tags depend on a certain condition.
Imagine the following code (even though I know it won't work)
if someCondition {
type MyType struct {
// some common fields
Date []string `xml:"value"`
}
} else {
type MyType struct {
// some common fields
Date []string `xml:"anotherValue"`
}
}
var t MyType
// do the unmarshalling ...
The problem is that these two structs have lots of fields in common. The only difference is in one of the fields and I want to prevent duplication. How can I solve this problem?
You use different types to unmarshal. Basically, you write the unmarshaling code twice and either run the first version or the second. There is no dynamic solution to this.
The simplest is probably to handle all possible fields and do some post-processing.
For example:
type MyType struct {
DateField1 []string `xml:"value"`
DateField2 []string `xml:"anotherValue"`
}
// After parsing, you have two options:
// Option 1: re-assign one field onto another:
if !someCondition {
parsed.DateField1 = parsed.DateField2
parsed.DateField2 = nil
}
// Option 2: use the above as an intermediate struct, the final being:
type MyFinalType struct {
Date []string `xml:"value"`
}
if someCondition {
final.Date = parsed.DateField1
} else {
final.Date = parsed.DateField2
}
Note: if the messages are sufficiently different, you probably want completely different types for parsing. The post-processing can generate the final struct from either.
As already indicated, you must duplicate the field. The question is where the duplication should exist.
If it's just a single field of many, one option is to use embedding, and field shadowing:
type MyType struct {
Date []string `xml:"value"`
// many other fields
}
Then when Date uses the other field name:
type MyOtherType struct {
MyType // Embed the original type for all other fields
Date []string `xml:"anotherValue"`
}
Then after unmarshaling of MyOtherType, it's easy to move the Date value into the original struct:
type data MyOtherType
err := json.Unmarshal(..., &data)
data.MyType.Date = data.Date
return data.MyType // will of MyType, and fully populated
Note that this only works for unmarshaling. If you need to also marshal this data, a similar trick can be used, but the mechanics around it must be essentially reversed.

How to omit some parameters of structure Gin gonic

I have big structure with more than 50 params
type Application struct {
Id int64 `json:"id"`
FullName string `json:"fullName,omitempty"`
ActualAddress string `json:"actualAddress,omitempty"`
.....
}
I use gin-gonic and when I return application I need to omit some params I've created a function which makes empty some params (playLink) and then gin returns me correct json (without unnecessary values). I heard that reflection isn't fast operation so in our case we can use a lot of ugly if-else or switch-cases. Is there any other solutions faster than reflecting and more beautiful than if-elses?
The thing is that structure params have non-empty values, so they wont by omitted by gin. That's why I've created function to make some params empty before return
The thing is, if you only want to zero a few fields, it's more readable to do it without a function, e.g.
app := Application{}
app.FullName, app.ActualAddress = "", ""
If you want to create a function for it, at least use variadic parameter, so it's easier to call it:
func zeroFields(application *Application, fields ...string) {
// ...
}
So then calling it:
zeroFields(&app, "FullName", "ActualAddress")
Yes, this will have to use reflection, so it's slower than it could be, and error prone (mistyped names can only be detected at runtime). If you want to avoid using reflection, pass the address of the fields:
func zeroFields(ps ...*string) {
for _, p := range ps {
*p = ""
}
}
This way you have compile-time guarantee that you type field names correctly, and that they have string type.
Calling it:
zeroFields(&application.FullName, &application.ActualAddress)
Try it on the Go Playground.
If I understand correctly: you want to return some values from your struct but not all of them? Perhaps a nested struct?
type Application struct {
ID struct {
ID int64 `json:"id"`
} `json:"id"`
Person struct {
Fullname string `json:"Fullname"
} `json:"person"
}
That should let you filter out the fields you want to use.

Resources