I'm trying to parse some xml documents in Go. I need to define a few structs for this purpose, and my struct tags depend on a certain condition.
Imagine the following code (even though I know it won't work)
if someCondition {
type MyType struct {
// some common fields
Date []string `xml:"value"`
}
} else {
type MyType struct {
// some common fields
Date []string `xml:"anotherValue"`
}
}
var t MyType
// do the unmarshalling ...
The problem is that these two structs have lots of fields in common. The only difference is in one of the fields and I want to prevent duplication. How can I solve this problem?
You use different types to unmarshal. Basically, you write the unmarshaling code twice and either run the first version or the second. There is no dynamic solution to this.
The simplest is probably to handle all possible fields and do some post-processing.
For example:
type MyType struct {
DateField1 []string `xml:"value"`
DateField2 []string `xml:"anotherValue"`
}
// After parsing, you have two options:
// Option 1: re-assign one field onto another:
if !someCondition {
parsed.DateField1 = parsed.DateField2
parsed.DateField2 = nil
}
// Option 2: use the above as an intermediate struct, the final being:
type MyFinalType struct {
Date []string `xml:"value"`
}
if someCondition {
final.Date = parsed.DateField1
} else {
final.Date = parsed.DateField2
}
Note: if the messages are sufficiently different, you probably want completely different types for parsing. The post-processing can generate the final struct from either.
As already indicated, you must duplicate the field. The question is where the duplication should exist.
If it's just a single field of many, one option is to use embedding, and field shadowing:
type MyType struct {
Date []string `xml:"value"`
// many other fields
}
Then when Date uses the other field name:
type MyOtherType struct {
MyType // Embed the original type for all other fields
Date []string `xml:"anotherValue"`
}
Then after unmarshaling of MyOtherType, it's easy to move the Date value into the original struct:
type data MyOtherType
err := json.Unmarshal(..., &data)
data.MyType.Date = data.Date
return data.MyType // will of MyType, and fully populated
Note that this only works for unmarshaling. If you need to also marshal this data, a similar trick can be used, but the mechanics around it must be essentially reversed.
Related
The code gives me error: DB redeclared.
Is there any idiomatic way to solve it? Or any work-around?
TIA
type a struct {
DB[int64]
DB[string]
}
type b interface {
DB[int64]
DB[string]
}
type DB[T any] interface {
GetList(query string) ([]T, error)
}
You can't embed the same interface, even with different type parameters. Regardless of how it is instantiated, you are trying to promote into the b interface two methods with the same name GetList and different signatures — given by the different instantiations of DB.
The situation is similar, although not technically the same, for embedding into a struct. In structs, the name of the embedded field is the name of the type — DB —, and a struct can't have two non-blank fields with the same name.
About how to solve this issue, it depends what you want to accomplish.
If you want to convey that "a implements DB with either type parameter" you can embed DB[T] and make a itself generic, and restrict a's type parameters:
type a[T int64 | string] struct {
DB[T]
}
// illustrative implementation of the DB[T] interface
// if you embed DB[T] you likely won't use `a` itself as receiver
func (r *a[T]) GetList(query string) ([]T, error) {
// also generic method
}
This is okay because DB's type parameter is constrained to any, and a's type parameter is more restrictive. This allows you to use a in other generic methods, or choose a specific type upon instantiation, but the implementation of GetList has to be parametrized too.
Otherwise if you need a to have separated methods that return int64 or string, you must give it different names.
Finally, you can embed instances of DB into two interfaces with different names, and then embed those into a instead.
type a struct {
DBStr
DBInt
}
type DBStr interface {
DB[string]
}
type DBInt interface {
DB[int64]
}
This way though the top-level selector isn't available because the method names are still the same. The program compiles, but you'll have to explicitly choose which field to call the method on:
myA := a{ /* init the two fields */ }
res, err = myA.DBStr.GetList(query)
// res is type []string
// or
res, err = myA.DBInt.GetList(query)
// res is type []int64
I need to marshal into this JSON format:
{"messageProtocolHandshake":[{"handshakeType":"announceMax"},{"version":[{"major":1},{"minor":0}]}]}
Problem is matching the handshakeType. My struct is
type MessageProtocolHandshake struct {
HandshakeType HandshakeType `json:"handshakeType"`
Version []Version `json:"version"`
}
type HandshakeType struct {
HandshakeType string
}
Marshaling can be done using slice of interface:
func (h MessageProtocolHandshake) MarshalJSON() ([]byte, error) {
res := make([]interface{}, 3)
res[0] = struct {
HandshakeType string `json:"handshakeType"`
}{h.HandshakeType.HandshakeType}
res[1] = struct {
Version []Version `json:"version"`
}{h.Version}
return json.Marshal(res)
}
Using a simple marshaler/unmarshaler takes away the surrounding curly brackets from the handshakeType, so that doesn't work:
{"messageProtocolHandshake":[{"handshakeType":"announceMax","version":[{"major":1,"minor":0}],"formats":[{"format":"JSON-UTF8"}]}]}
Seems as if Go applies some heuristic in that case on the retuned byte array (undocumented?).
Is there a more elegant way of omitting the structs outer field name?
--
UPDATE To summarise the answers: key is to think about different structs for marshalling and unmarshalling if nothing else works, potentially a using a 3rd presentation for working internally with the data.
When custom (Un)Marshalers come into play remember that promoted fields inherit their methods and hence influence parent structs.
The JSON that you specified has a different model from that of your struct.
There are a few approaches to aligning these: Change the specification of the JSON data to match your structs, change the structs to match the specification of the JSON, or create a new struct that is only used for marshaling.
I omit the last example, because it's very similar to the second method.
Changing the specification of the JSON
The following model stays the same:
type MessageProtocolHandshake struct {
HandshakeType HandshakeType `json:"handshakeType"`
Version []Version `json:"version"`
}
type HandshakeType struct {
HandshakeType string
}
The JSON for this would be:
{"handshakeType":{"HandshakeType":""},"version":[]}
You did not specify the Version type so I don't know how one would change the JSON for that.
Changing the structs
The following JSON stays the same:
{"messageProtocolHandshake":[{"handshakeType":"announceMax"},{"version":[{"major":1},{"minor":0}]}]}
The structs for this would be:
type Model struct {
MessageProtocolHandshake []interface{} `json:"messageProtocolHandshake"`
}
type HandshakeType struct {
HandshakeType string `json:"handshakeType"`
}
type Versions struct {
Version []Version `json:"version"`
}
type Version struct {
Major *int `json:"major,omitempty"`
Minor *int `json:"minor,omitempty"`
}
Unmarshaling would not be trivial.
https://play.golang.org/p/89WUhcMFM0B
As is obvious from the results, the models you are using are not good. If there's a way to change all of this, I would recommend starting from scratch, using the data that is necessary and creating the JSON specification from the structs.
I recommend reading up on JSON: https://www.json.org/json-en.html
Also, I recommend this introduction to Go and JSON: https://blog.golang.org/json
Assume a JSON object with the general format
"accounts": [
{
"id": "<ACCOUNT>",
"tags": []
}
]
}
I can create a struct with corresponding json tags to decode it like so
type AccountProperties struct {
ID AccountID `json:"id"`
MT4AccountID int `json:"mt4AccountID,omitempty"`
Tags []string `json:"tags"`
}
type Accounts struct {
Accounts []AccountProperties `json:"accounts"`
}
But the last struct with just one element seems incorrect to me. Is there a way I could simply say type Accounts []AccountProperties `json:"accounts"` instead of creating an entire new struct just to decode this object?
You need somewhere to store the json string accounts. Using a:
var m map[string][]AccountProperties
suffices, though of course you then need to know to use the string literal accounts to access the (single) map entry thus created:
type AccountProperties struct {
ID string `json:"id"`
MT4AccountID int `json:"mt4AccountID,omitempty"`
Tags []string `json:"tags"`
}
func main() {
var m map[string][]AccountProperties
err := json.Unmarshal([]byte(data), &m)
fmt.Println(err, m["accounts"])
}
See complete Go Playground example (I had to change the type of ID to string and fix the missing { in the json).
As Dave C points out in comments, this is no shorter than just using an anonymous struct type:
var a struct{ Accounts []AccountProperties }
in terms of the Unmarshall call (and when done this way it's more convenient to use). Should you want to use an anonymous struct like this in a json.Marshall call, you'll need to tag its single element to get a lowercase encoding: without a tag it will be called "Accounts" rather than "accounts".
(I don't claim the map method to be better, just an alternative.)
I have big structure with more than 50 params
type Application struct {
Id int64 `json:"id"`
FullName string `json:"fullName,omitempty"`
ActualAddress string `json:"actualAddress,omitempty"`
.....
}
I use gin-gonic and when I return application I need to omit some params I've created a function which makes empty some params (playLink) and then gin returns me correct json (without unnecessary values). I heard that reflection isn't fast operation so in our case we can use a lot of ugly if-else or switch-cases. Is there any other solutions faster than reflecting and more beautiful than if-elses?
The thing is that structure params have non-empty values, so they wont by omitted by gin. That's why I've created function to make some params empty before return
The thing is, if you only want to zero a few fields, it's more readable to do it without a function, e.g.
app := Application{}
app.FullName, app.ActualAddress = "", ""
If you want to create a function for it, at least use variadic parameter, so it's easier to call it:
func zeroFields(application *Application, fields ...string) {
// ...
}
So then calling it:
zeroFields(&app, "FullName", "ActualAddress")
Yes, this will have to use reflection, so it's slower than it could be, and error prone (mistyped names can only be detected at runtime). If you want to avoid using reflection, pass the address of the fields:
func zeroFields(ps ...*string) {
for _, p := range ps {
*p = ""
}
}
This way you have compile-time guarantee that you type field names correctly, and that they have string type.
Calling it:
zeroFields(&application.FullName, &application.ActualAddress)
Try it on the Go Playground.
If I understand correctly: you want to return some values from your struct but not all of them? Perhaps a nested struct?
type Application struct {
ID struct {
ID int64 `json:"id"`
} `json:"id"`
Person struct {
Fullname string `json:"Fullname"
} `json:"person"
}
That should let you filter out the fields you want to use.
I am new to Go and am running into a situation that I am unsure how to solve. I am working on some code that takes a DNS packet in raw bytes and returns a struct called DNSPacket.
The struct looks like the following
type DNSPacket struct {
...some fields
Questions []Question
Answers []Answer
...some more fields
}
The issue I am having is with the Answers type which looks like this.
type Answer struct {
Name string
Type int
Class int
TTL uint32
RdLength int
Data []byte
}
Depending on the type of Answer the Data field must be decoded differently. For example if the Answer is an A record (Type 1) the data is simply an ipv4 address. However if the Answer is an SRV record (Type 33) then the data is contains port, priority, weight and target encoded in the byte slice.
I thought it would be great if I could have a method on Answer called DecodeData() that returns the correct data depending on the type, however since there is no overriding or inheritance in Go I am unsure how to solve this. I tried using an interface to solve this, but it would not compile. I tried something like
type DNSRecordType interface {
Decode(data []byte)
}
type RecordTypeSRV struct {
target string
...more fields
}
//to 'implement' the DNSRecordType interface
func (record *RecordTypeSRV) Decode(data []byte) {
//do the work to decode appropriately and set
//the fields on the record
}
Then in the Answer method
func (a *Answer) DecodeData() DNSRecordType {
if a.Type === SRVType {
record := RecordTypeSRV{}
record.Decode(a.Data)
return record
}
//do something similar for other record types
}
What would be the correct Go way of having a single Answer type, but be able to return different types of Answer Data depending on their type?
Sorry, if this is a completely beginner question as I am still very new to Go.
Thanks!
Let me summarize your question.
You have a DNS Packet with the list of Answers. Based on the type of answer you have to process the data in the answer.
type DNSPacket struct {
...some fields
Questions []Question
Answers []Answer
...some more fields
}
type Answer struct {
Name string
Type int
Class int
TTL uint32
RdLength int
Data []byte
}
Answer
Let's create an interface that should be implemented to process data.
type PacketProcessor interface {
Process(Answer)
}
Let SRV implements the PacketProcessor
type SRV struct {
...
}
func (s *SRV) Process(a Answer) {
...
}
Your processing logic should be as follows
func (a *Answer) Process() {
var p PacketProcessor
switch a.Type {
case SRVType:
p = &SRV{}
...
//other cases
}
//finally
p.Process(*a)
}
Hope it helps :).
There is a Gurgaon based golang community that is always ready to help developers with their problems.
You can join the community via slack
As I know, to return different types, the return param must be an interface. So you can simply declare the function like this:
func (a *Answer) DecodeData() (mode modeType, value interface{}) {}
mode means the value is A record or SRV record, and you can return anything you want with the value field.
The function caller can handle the value depending on mode
If you want the code be more elegant, you can define different value structs for each mode. Then the caller may act as below:
type modeType int
const (
ARecord modeType = 1
SRVRecord modeType = 2
)
switch mode {
case ARecord:
// do something
case SRVRecord:
// do something
}