Possibly related: How to use interface type as a model in mgo (Go)?
I have a struct like so:
type Game struct {
ID bson.ObjectId
Type string
Location string
Details interface{}
}
type FeudDetails struct {
...
}
type TriviaDetails struct {
...
}
type BingoDetails struct {
...
}
I want to use the Type field of Game to unserialize Details into a specific type (like an instance of FeudDetails or BingoDetails). It would still be an interface{} in Game, but then I could do this:
feudDetails, ok := game.Details.(FeudDetails)
if ok {
// we know this is a Feud game, and we have the details
feudDetails.Round++
}
The docs say that "it is possible to unmarshal or marshal values partially" using bson.Raw, but they don't provide any examples I've been able to find.
I've tried using:
func (game *Game) SetBSON(r bson.Raw) error {
err := r.Unserialize(game)
if err != nil {
return nil
}
games[game.Type].LoadDetails(game) // this calls a function based on the Type to
// create a concrete value for that game.
return nil
}
I get a (ahem) stack overflow here. I assume this is because r.Unserialize is recursively calling SetBSON.
My goal is to use standard unserializing on all fields except Details, and then be able to use game.Type to determine how to process Details. If I do something like this:
type GameDetails interface{}
type Game struct {
...
Details GameDetails
}
func (details *GameDetails) SetBSON(r bson.Raw) error {
// game isn't defined
games[game.Type].LoadDetails(r, details)
}
then how can I access the outer Type field to know which game type to unserialize it to?
I'll also accept the answer "you're doing it all wrong, and a better pattern in Go is XYZ..."
EDIT: I also tried deserializing normally, and then casting the interface{} version of Details using game.Details.(FeudDetails), but the conversion failed. I guess I can't do it that way because the underlying type after unserialization is not a FeudDetails but rather probably map[string]interface{}.
EDIT 2: I thought I'd be clever and pre-populate an object with the right types when retrieving from the database (game := Game{Details: FeudDetails: {}} prior to calling db...One(&game)) but my trickery did not work:
DEBU[Mar 31 22:19:09.442] Caching show gid=5e814448ef5b9858b7ff4e57
TRAC[Mar 31 22:19:09.442] Before database call dtype=main.FeudDetails
TRAC[Mar 31 22:19:09.446] After database call dtype=bson.M
Ignore Details during (un)marshalling
Change the definition of Game so bson doesn't try to do anything with the Details field:
type Game struct {
...
Details interface{} `json:"details" bson:"-"`
}
Unmarshal Details manually
func (game *Game) SetBSON(r bson.Raw) error {
// Unmarshall everything except Details
type tempGame Game
err := r.Unmarshal((*tempGame)(game)) // this is necessary to prevent recursion
if err != nil {
return err
}
// Get the raw data for Details
var d struct {
Details bson.Raw
}
if err := r.Unmarshal(&d); err != nil {
return err
}
gameType, ok := games[game.Type]
if ok {
// Use individual processing based on game Type
game.Details, err = gameType.LoadDetails(d.Details)
if err != nil {
return err
}
// fmt.Sprintf("%T", game.Details) => main.FeudDetails
}
return nil
}
Marshal Details manually
As far as I can tell, the only way to get bson to include Details after we told it to omit it in the original struct, is to create a whole other structure, copy the data over one by one, and use that type in GetBSON. It seems like there must be a better way.
Related
This question already has answers here:
Pointer to an interface vs interface holding a pointer when unmarshaling JSON
(1 answer)
How to tell json.Unmarshal to use struct instead of interface
(1 answer)
Closed 3 months ago.
I have constructed a small example of what I am trying to do:
type Service interface {
Transform()
}
type XYZ struct {
ID string `json:"id"`
}
func (s XYZ) Transform() {}
func main() {
x := "could be anything"
msg := []byte(`{"id": "123"}`)
var msgS Service
switch x {
case "could be anything":
msgS = XYZ{}
break
case "could be something else":
// another type that implements Transform()
break
}
err := json.Unmarshal(msg, &msgS)
if err != nil {
fmt.Println(err)
}
// msgS.Transform()
}
Go Playground
This returns an error:
json: cannot unmarshal object into Go value of type main.Service
Essentially, I need to be flexible with whatever x might hold, hence the switch. If I instantitate msgSto the actual type that implements Transform(), all works as expected.
I realise I might need to reconsider my whole implementation - Any ideas would be welcome!
You should unmarshal to the concrete type and then you can assign the result to the interface type.
A simple example:
var service Service
var err error
switch x {
case "something":
var something Something // this implements Service
err = json.Unmarshal(data, &something)
service = something
case "something else":
var somethingElse SomethingElse // this implements Service
err = json.Unsmarshal(data, &somethingElse)
service = somethingElse
// etc...
}
if err != nil {
// handle error (here or inside of each case if you need/want to)
}
// use service as needed here
I'm new to golang generics and have the following setup.
I've gathered loads of different kinds of reports.
Each report has enclosing fields
So I wrapped it in a ReportContainerImpl
I've used a type argument of [T Reportable] where the Reportable is defined as follows
type Reportable interface {
ExportDataPointReport | ImportDataPointReport | MissingDataPointReport | SensorThresoldReport
}
Each of the type in the type constraint is structs that is to be embedded in the container.
type ReportContainerImpl[T Reportable] struct {
LocationID string `json:"lid"`
Provider string `json:"pn"`
ReportType ReportType `json:"m"`
Body T `json:"body"`
}
I use a discriminator ReportType to determine the concrete type when Unmarshal.
type ReportType string
const (
ReportTypeExportDataPointReport ReportType = "ExportDataPointReport"
ReportTypeImportDataPointReport ReportType = "ImportDataPointReport"
ReportTypeMissingDataPointReport ReportType = "MissingDataPointReport"
ReportTypeSensorThresoldReport ReportType = "SensorThresoldReport"
)
Since go does not support type assertion for struct (only interfaces) it is not possible to cast the type when Unmarshal. Also go does not support pointer to the "raw" generic type. Hence, I've created a interface that the ReportContainerImpl implements.
type ReportContainer interface {
GetLocationID() string
GetProvider() string
GetReportType() ReportType
GetBody() interface{}
}
The problem I then get is that I cannot do type constrains on the return type in any form or shape and am back at "freetext semantics" on the GetBody() function to allow for type assertion when Unmarshal is done.
container, err := UnmarshalReportContainer(data)
if rep, ok := container.GetBody().(ExportDataPointReport); ok {
// Use the ReportContainerImpl[ExportDataPointReport] here...
}
Maybe I'm getting this wrong? - but however I do this, I always end up with somewhere needs a interface{} or to know the exact type before Unmarshal
Do you have a better suggestion how to solve this in a type (safer) way?
Cheers,
Mario :)
For completeness I add the UnmarshalReportContainer here
func UnmarshalReportContainer(data []byte) (ReportContainer, error) {
type Temp struct {
LocationID string `json:"lid"`
Provider string `json:"pn"`
ReportType ReportType `json:"m"`
Body *json.RawMessage `json:"body"`
}
var temp Temp
err := json.Unmarshal(data, &temp)
if err != nil {
return nil, err
}
switch temp.ReportType {
case ReportTypeExportDataPointReport:
var report ExportDataPointReport
err := json.Unmarshal(*temp.Body, &report)
return &ReportContainerImpl[ExportDataPointReport]{
LocationID: temp.LocationID,
Provider: temp.Provider,
ReportType: temp.ReportType,
Body: report,
}, err
// ...
}
}
but however I do this, I always end up with somewhere needs a interface{} or to know the exact type before Unmarshal
Precisely.
The concrete types needed to instantiate some generic type or function like ReportContainerImpl or UnmarshalReportContainer must be known at compile time, when you write the code. JSON unmarshalling instead occurs at run-time, when you have the byte slice populated with the actual data.
To unmarshal dynamic JSON based on some discriminatory value, you still need a switch.
Do you have a better suggestion how to solve this in a type (safer) way?
Just forgo parametric polymorphism. It's not a good fit here. Keep the code you have now with json.RawMessage, unmarshal the dynamic data conditionally in the switch and return the concrete structs that implement ReportContainer interface.
As a general solution — if, and only if, you can overcome this chicken-and-egg problem and make type parameters known at compile time, you can write a minimal generic unmarshal function like this:
func unmarshalAny[T any](bytes []byte) (*T, error) {
out := new(T)
if err := json.Unmarshal(bytes, out); err != nil {
return nil, err
}
return out, nil
}
This is only meant to illustrate the principle. Note that json.Unmarshal already accepts any type, so if your generic function actually does nothing except new(T) and return, like in my example, it is no different than "inlining" the entire thing as if unmarshalAny didn't exist.
v, err := unmarshalAny[SomeType](src)
functionally equivalent as
out := &SomeType{}
err := json.Unmarshal(bytes, out)
If you plan to put more logic in unmarshalAny, its usage may be warranted. Your mileage may vary; in general, don't use type parameters when it's not actually necessary.
I've got some REST API with my models defined as Go structs.
type User struct {
FirstName string
LastName string
}
Then I've got my database methods for getting data.
GetUserByID(id int) (*User, error)
Now I'd like to replace my REST API with https://github.com/twitchtv/twirp .
Therefore I started defining my models inside .proto files.
message User {
string first_name = 2;
string last_name = 3;
}
Now I've got two User types. Let's call them the native and the proto type.
I've also got a service defined in my .proto file which returns a user to the frontend.
service Users {
rpc GetUser(Id) returns (User);
}
This generates an interface that I have to fill in.
func (s *Server) GetUser(context.Context, id) (*User, error) {
// i'd like to reuse my existing database methods
u, err := db.GetUserByID(id)
// handle error
// do more stuff
return u, nil
}
Unfortunately this does not work. My database returns a native User but the interface requires a proto user.
Is there an easy way to make it work? Maybe using type aliases?
Thanks a lot!
One way you can solve your problem is by doing the conversion manually.
type User struct {
FirstName string
LastName string
}
type protoUser struct {
firstName string
lastName string
}
func main() {
u := db() // Retrieve a user from a mocked db
fmt.Println("Before:")
fmt.Printf("%#v\n", *u) // What db returns (*protoUser)
fmt.Println("After:")
fmt.Printf("%#v\n", u.AsUser()) // What conversion returns (User)
}
// Mocked db that returns pointer to protoUser
func db() *protoUser {
pu := protoUser{"John", "Dough"}
return &pu
}
// Conversion method (converts protoUser into a User)
func (pu *protoUser) AsUser() User {
return User{pu.firstName, pu.lastName}
}
The key part is the AsUser method on the protoUser struct.
There we simply write our custom logic for converting a protoUser into a User type we want to be working with.
Working Example
As #Peter mentioned in the comment section.
I've seen a project which made it with a custom Convert function. It converts the Protobuf to local struct via json.Unmarshal, not sure how's the performance but it's a way to go.
Preview Code PLAYGROUND
// Convert converts the in struct to out struct via `json.Unmarshal`
func Convert(in interface{}, out interface{}) error {
j, err := json.Marshal(in)
if err != nil {
return err
}
err = json.Unmarshal(j, &out)
if err != nil {
return err
}
return nil
}
func main() {
// Converts the protobuf struct to local struct via json.Unmarshal
var localUser User
if err := convert(protoUser, &localUser); err != nil {
panic(err)
}
}
Output
Before:
main.ProtoUser{FirstName:"John", LastName:"Dough"}
After:
main.User{FirstName:"John", LastName:"Dough"}
Program exited.
I suspect I am trying to shoehorn go into behaving in an OOP way, but I don't know the go idiom to do what I want.
I have a Message struct that I use to pass data around in a client-server application:
type Message struct {
ID string `json:"id,omitempty"`
Type string `json:"type"`
Data interface{} `json:"data"`
}
the Data here can be different things, for example a number of Commands:
type Command struct {
User *types.UserInfo `json:"user"`
}
type CommandA struct {
Command
A *AData `json:"a_data"`
}
type CommandB struct {
CommandB
B *BData `json:"b_data"`
}
What I want to do is to check that the message data type is a Command and perform actions that are common to all commands, for example authorisation, all in one place and not having to type assert what type of command, calling the appropriate handler function and then do the auth, as this would result in massive code duplication.
The code below reflects what I would like to happen.
for {
select {
case m := <-in:
// what I would like to do, obviously not working as
// m.Data is not of type Command but the actual command type
if c, ok := m.Data.(msg.Command); ok {
// do auth and other common stuff
}
switch t := m.Data.(type) {
case *msg.CommandA:
go srv.handleCommandA(m.ID, t)
case *msg.CommandB:
go srv.handleCommandB(m.ID, t)
// etc etc
default:
// do something
}
}
}
How do I solve this go idiomatically?
You can define common command stuff in interface
type Commander interface{
DoCommonStuff()
}
implement it for Command struct
func (c Command) DoCommonStuff(){
//do stuff
}
and then assert
if c, ok := m.Data.(Commander); ok {
c.DoCommonStuff()
}
your other code should work unchanged
One approach is using reflection to extract common field value from the Data. In your example, since all Command has User field, we can use it to identify whether Message.Data is a command or not. If Command is not embedded to the data, simply return nil. Example code:
func GetUserInfo(v interface{}) *types.UserInfo {
vt := reflect.ValueOf(v)
if vt.Kind() == reflect.Ptr {
vt = vt.Elem()
}
if vt.Kind() != reflect.Struct {
return nil
}
u := vt.FieldByName("User")
if !u.IsValid() {
return nil
}
user, ok := u.Interface().(*types.UserInfo)
if !ok {
return nil
}
return user
}
//Call GetUserInfo then perform common operation
user := GetUserInfo(m.Data)
if user != nil {
//Do auth and other common stuff
}
I have a list of objects (olievere/Elastic SearchResult.Hits to be exact). Each of these has a json.RawMessage object and I'm looking to create an abstractable method that takes in an interface slice of any struct, Unmarshal's each individual hits' json.RawMessage into said struct, and appends it to the passed in []interface.
This func is not supposed to have any logic or insight into the desired business layer struct, and the DB Call is interfaced pretty heavily, and as such has no visibility into the Elastic package mentioned above. Example of what I was attempting to do...
foo.go
import (bar, package)
type TestStruct struct {
Slice []*package.Struct // package.Struct has a value of Source which is a
// json.RawMessage
}
func GetData() bar.Test {
return &TestStruct{*package.GetData()}
}
func (result TestStruct) UnmarshalStruct(v []interface{}) {
for _, singleStruct := range result.Slice {
append(json.Unmarshal(singleStruct, &v))
}
Second File
bar.go
type Handler interface {
GetData() Test
}
type Test interface {
UnmarshalStruct
}
type OtherType struct {
foo string
bar string
}
func RetrieveData() []OtherType {
handler := New(Handler)
test := handler.GetData()
var typeSlice []OtherType
test.UnmarshalStruct(&typeSlice)
}
I'm looking to hand something of type []OtherType, or any other new struct I decide to create, to UnmarshalStruct, and have it return to me that same struct, just full of data
As an example, I have two different types of data I'll be searching for from Elastic. I will be receiving a list of ONE of the following two objects.
{ 'foo': '',
'id':
}
And in a different index
{ 'bar': '',
'baz': '',
'eee': ''
}
Each of these will naturally require two different structs.
However, I desire a single method to be able to decode either of these lists. I'll be given below, and using the same function I want to be able to convert this to a bar struct, and the other type to a foo struct.
{ 'source': [
{ 'bar': '',
'baz': '',
'eee': ''
},
{ 'bar': '',
'baz': '',
'eee': ''
},
{ 'bar': '',
'baz': '',
'eee': ''
}
]
}
There's really no way to do what you want without reflection. I would personally structure this differently, so that you unmarshal into more generic types, like a map[string]string, or as #ThunderCat shows, get rid of the intermediary state and unamrshal directly into the correct types. But it can be done...
(I moved the json.RawMessage directly into TestStruct to get rid of one level of indirection and make the example more clear)
type TestStruct struct {
Slice []json.RawMessage
}
func (t TestStruct) UnmarshalStruct(v interface{}) error {
// get the a Value for the underlying slice
slice := reflect.ValueOf(v).Elem()
// make sure we have adequate capacity
slice.Set(reflect.MakeSlice(slice.Type(), len(t.Slice), len(t.Slice)))
for i, val := range t.Slice {
err := json.Unmarshal(val, slice.Index(i).Addr().Interface())
if err != nil {
return err
}
}
return nil
}
You can then call it like so
var others []OtherType
err := ts.UnmarshalStruct(&others)
if err != nil {
log.Fatal(err)
}
http://play.golang.org/p/dgly2OOXDG
If I understand correctly, you want to unmarshal data to slices of two types:
type A struct {
Foo string `json:"foo"`
ID string `json:"id"`
}
type B struct {
Bar string `json:"bar"`
Baz string `json:"baz"`
Eee string `json:"eee"`
}
from a SearchHit Source.
The JSON package can do most of the work for you:
func executeQuery(q Query, v interface{}) error {
// Get a SearchHit. I am making this up.
// I have no idea how the package works.
searchHit, err := getHit(q)
if err != nil {
return err
}
// This is the important part. Convert the raw message to
// a slice of bytes and decode to the caller's slice.
return json.Unmarshal([]byte(*searchHit.Source), v)
}
You can call this function to decode to a slice of the types or a slice of pointers to the types.
// Slice of type
var s1 []TypeA
if err := executeQuery(q1, &s1); err != nil {
// handle error
}
// Slice of pointer to type
var s2 []*TypeB
if err := error(q2, &s2); err != nil {
// handle error
}
I know that this is not a direct answer to the question, but this is how this scenario is typically handled.
I don't believe this is easy to do. In the Raw Message Example in the godocs they use a value within the json, "Space" in their example, to determine which struct type to unmarshal into.
For this to work, the function would have to have some way of getting every struct that has been defined ever for the program, and then it would have to examine each json object and compare it to each struct using reflection to figure out which type it most likely is. And what if there are multiple structs that "could be it"? Then conflict resolution complicates things.
In short, I don't think you can do this.