Golang: Unmarshal function that can handle 2 different structs - go

looking for help.
Say I have the following structs..
type phoneStruct struct {
PhoneID string `json:"id"`
Carrier string `json:"carrier"`
}
type carStruct struct {
CarID string `json:"id"`
Model string `json:"model"`
}
Two very different structs.. but predictable.
Now in my code what I'm doing is doing a POST request and then parsing the body of what I get back so that I can do some test assertions on it. But for simplicity lets say I have this JSON
var jsonPhone = `{
"id": "123",
"carrier": "Rogers"
}`
var jsonCar = `{
"id": "foobar-fewf-434-fewf",
"model": "Civic"
}`
And I want to associate that json with my structs, so I have the following calls:
p, err := parsePhone(jsonPhone)
c, err := parseCar(jsonCar)
And the functions looks something like
func parsePhone(body []byte) (*phoneStruct, error) {
var p = new(phoneStruct)
err := json.Unmarshal(body, p)
return p, err
}
func parseCar(body []byte) (*carStruct, error) {
var c = new(carStruct)
err := json.Unmarshal(body, c)
return c, err
}
.. I am wondering if there is a way to just have one struct. I know there is if there is some commonality between two or more structs .. or how to parse unknown structs.. but haven't been able to find a resource on if I know the structures.. how can I have a function that looks something like..
func parseAnything(body []byte (<pointer to the struct>, error)
var c = new(someStruct)
err := json.Unmarshal(body, c)
return c, err
}
Note that I will always know what I am calling so I could do a conditional switch statement in my parse function that says 'if car.. if phone..' but I'm not sure how to define the return.
Thank you!

Write the function to take a pointer to the result:
func parseAnything(body []byte, c interface{}) error {
return json.Unmarshal(body, c)
}
Use it like this:
var p phoneStruct
if err := parseAnything(jsonPhone, &p); err != nil {
// handle error
}
// p has unmarshaled phone
var c carStruct
if err := parseAnything(jsonCar, &c); err != nil {
// handle error
}
// c has unmarshaled car

I don't know exactly what are you trying to build, but will try to give some insight based on what I understood.
If you are trying to use the same parser for both structs, they probably have something in common. Probably they are being used together for some use case of you application.
So, if they are being use together, they should implement some interface that represents the set of features that these structs have in common (with may or maybe not be related with just the parsing of data).
And, as you said, you know beforehand of what type of struct you need, so this should come easy with something like this:
//The interface that represents what the structs have in common.
//I named it "Parser" because it's the info I have. It probably should have another (better) name
type Parser interface {
Parse([]byte) (Parser, error)
}
type Phone struct {
PhoneID string `json:"id"`
Carrier string `json:"carrier"`
}
type Car struct {
CarID string `json:"id"`
Model string `json:"model"`
}
func (p *Phone) Parse (body []byte) (Parser, error) {
err := json.Unmarshal(body, p)
return p, err
}
func (c *Car) Parse (body []byte) (Parser, error) {
err := json.Unmarshal(body, c)
return c, err
}
func main() {
c := Car{}
var jsonCar = `{
"id": "123",
"model": "Civic"
}`
res, _ := c.Parse([]byte(jsonCar))
fmt.Println(res)
p := Phone{}
var jsonPhone = `{
"id": "123",
"carrier": "Rogers"
}`
res, _ = p.Parse([]byte(jsonPhone))
fmt.Println(res)
}

Related

Go interface for MongoDB store for fields like bson.D

I've got a Go app which uses MongoDB data storage, but still want it to be easy to switch out drivers (to PostgresSQL, who knows) and to test.
So I'm passing around a Storage interface which describes what a storage should do.
type Storage interface {
InsertOne(ctx context.Context, collection string, document interface{}) error
FindOne(ctx context.Context, collection string, filters map[string]string) (map[string]interface{}, error)
}
Then I've got a MongoDB implementation of that:
type Mongo struct {
Client *mongo.Client
Database string
}
func (m *Mongo) InsertOne(ctx context.Context, collection string, document interface{}) error {
_, err := m.Client.Database(m.Database).Collection(collection).InsertOne(ctx, document)
if err != nil {
return err
}
return nil
}
func (m *Mongo) FindOne(ctx context.Context, collection string, filters map[string]string) (map[string]interface{}, error) {
var result bson.M
var filter bson.D
for k, v := range filters {
filter = append(filter, bson.E{
Key: k,
Value: v,
})
}
err := m.Client.Database(m.Database).Collection(collection).FindOne(ctx, filter).Decode(&result)
if err != nil {
return nil, err
}
resultMap := make(map[string]interface{}, len(result))
for k, v := range result {
resultMap[k] = v
}
return resultMap, nil
}
I'm using bson.D to filter Mongo, so passing in filters map[string]string and converting it to bson.D.
This works fine when the filter is something simple like bson.D{{"title", "The Room"}}, simply pass it to the function like map[string]string{"title": "The Room"}.
But it's not so simple when it's a more complex filter like bson.D{{"population", bson.D{{"$lte", 500}}}}
This approach is not looking that good at the moment, feels like I'm very limited in what I can do.
So I'm looking for advice on a better way to do this.

Why Json Unmarshall changing array type in Golang?

I'm trying to seed my Postgres database as functionally. In my case, SeedSchema() function can take any type struct. So I define a interface and create functions to my structs which will seed. I tried with generics and without.
When I unmarshall any json array from file as byte array, json.Unmarshall method change my tempMember member of struct. Exp, models.Term to map[string]interface{}. I've used unmarshall before this function and I've not seen like this situation.
Here is my SeedSchema() function:
func (db *Database) SeedSchema(models ...globals.Seeder[any]) error {
var (
subjects []globals.Seeder[any]
fileByte []byte
err error
// tempMember any
)
if len(models) == 0 {
subjects = seederModelList
} else {
subjects = models
}
for _, model := range subjects {
fileName, tempMember := model.Seed()
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //1
if fileByte, err = os.ReadFile("db/seeds/" + fileName); err != nil {
fmt.Println(err)
return err
}
if err = json.Unmarshal(fileByte, &tempMember); err != nil {
fmt.Println(err)
return err
}
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //2
}
return nil
}
First print returns []models.AirportCodes and the second []interface {}.
Here is my interface and model:
func (AirportCodes) Seed() (string, any) {
return "airport_codes.json", []AirportCodes{}
}
type Seeder[T any] interface {
Seed() (string, T)
// Seed(*gorm.DB) error
TableName() string
}
seederModelList = []globals.Seeder[any]{
m.AirportCodes{},
m.Term{},
}
After a few weeks, I have looking for solve this problem and look unmarshaler interfaces and examples. Then Like what icza said, I started to look over the my code that convention between types and I solved like this. If you guys have better answer than mine, please add answer.
Data:
[
{
"id":1,
"name":"Term 1",
"content": [
"a1",
"a2",
"a3"
]
}
]
Result:
[{ID:1 Name:Term 1 Content:[a1 a2 a3]}]
UnmarshalJSON Function:
func (term *Term) UnmarshalJSON(data []byte) error {
tempMap := map[string]interface{}{}
if err := json.Unmarshal(data, &tempMap); err != nil {
return err
}
*term = Term{
Name: tempMap["name"].(string),
}
if tempMap["content"] != nil {
for _, v := range tempMap["content"].([]interface{}) {
(*term).Content = append((term).Content, v.(string))
}
}
return nil
}
Thank you for comments.

Error while trying to fetch queryresult.KV object in JSON.Unmarshal

I am a little bit confused here and although I have searched a lot on this, something is clearly missing from my knowledge and I am asking your help.
I have created a Hyperledger Fabric Network and installed a chaincode in it. And I want to make a function that retrieves all the World State inputs about the Keys. I have done it already with the bytes.Buffer and it worked. But what I want to do is to do it with a struct.
So, I created the following struct that has only the key:
type WSKeys struct {
Key string `json: "key"`
Namespace string `json: "Namespace"`
}
And this is my code function:
func (s *SmartContract) getAllWsDataStruct(APIstub shim.ChaincodeStubInterface , args []string) sc.Response {
var keyArrayStr []WSKeys
resultsIterator, err := APIstub.GetQueryResult("{\"selector\":{\"_id\":{\"$ne\": null }} }")
if err != nil {
return shim.Error("Error occured when trying to fetch data: "+err.Error())
}
for resultsIterator.HasNext() {
// Get the next record
queryResponse, err := resultsIterator.Next()
if err != nil {
return shim.Error(err.Error())
}
fmt.Println(queryResponse)
var qry_key_json WSKeys
json.Unmarshal([]byte(queryResponse), &qry_key_json)
keyArray = append(keyArray, qry_key_json)
}
defer resultsIterator.Close()
all_bytes, _ := json.Marshal(keyArray)
fmt.Println(keyArray)
return shim.Success(all_bytes)
}
When executing the above I get the following error:
cannot convert queryResponse (type *queryresult.KV) to type []byte
I can get the results correctly if I, for example do this:
func (s *SmartContract) getAllWsDataStruct(APIstub shim.ChaincodeStubInterface , args []string) sc.Response {
var keyArray []string
resultsIterator, err := APIstub.GetQueryResult("{\"selector\":{\"_id\":{\"$ne\": null }} }")
if err != nil {
return shim.Error("Error occured when trying to fetch data: "+err.Error())
}
for resultsIterator.HasNext() {
// Get the next record
queryResponse, err := resultsIterator.Next()
if err != nil {
return shim.Error(err.Error())
}
fmt.Println(queryResponse)
keyArray = append(keyArray, queryResponse.Key)
}
defer resultsIterator.Close()
all_bytes, _ := json.Marshal(keyArray)
fmt.Println(keyArray)
return shim.Success(all_bytes)
}
But, why I get the above error when trying to add the queryResponse into a custom struct?
Do I need to add it to a struct that is only its type?
Please someone can explain what I am missing here?
The error statement is verbose enough to indicate, that your []byte conversion failed for the type queryResponse which, with a bit of lookup seems to be a struct type. In Go you cannot natively convert a struct instance to its constituent bytes without encoding using gob or other means.
Perhaps your intention was to use the Key record in the struct for un-marshalling
json.Unmarshal([]byte(queryResponse.Key), &qry_key_json)

Strange behaviour when Unmarshalling into struct in Go

I'm developing a tool that can be implemented to simplify the process of creating simple CRUD operations/endpoints. Since my endpoints don't know what kind of struct they'll be receiving, I've created an interface that users can implement, and return an empty object to be filled.
type ItemFactory interface {
GenerateEmptyItem() interface{}
}
And the users would implement something like:
type Test struct {
TestString string `json:"testString"`
TestInt int `json:"testInt"`
TestBool bool `json:"testBool"`
}
func (t Test) GenerateEmptyItem() interface{} {
return Test{}
}
When the Test object gets created, its type is "Test", even though the func returned an interface{}. However, as soon as I try to unmarshal some json of the same format into it, it strips it of its type, and becomes of type "map[string]interface {}".
item := h.ItemFactory.GenerateEmptyItem()
//Prints "Test"
fmt.Printf("%T\n", item)
fmt.Println(reflect.TypeOf(item))
err := ConvertRequestBodyIntoObject(r, &item)
if err != nil {...}
//Prints "map[string]interface {}"
fmt.Printf("%T\n", item)
Func that unmarshalls item:
func ConvertRequestBodyIntoObject(request *http.Request, object interface{}) error {
body, err := ioutil.ReadAll(request.Body)
if err != nil {
return err
}
// Unmarshal body into request object
err = json.Unmarshal(body, object)
if err != nil {
return err
}
return nil
}
Any suggestions as to why this happens, or how I can work around it?
Thanks
Your question lacks an example showing this behavior so I'm just guessing this is what is happening.
func Generate() interface{} {
return Test{}
}
func GeneratePointer() interface{} {
return &Test{}
}
func main() {
vi := Generate()
json.Unmarshal([]byte(`{}`), &vi)
fmt.Printf("Generate Type: %T\n", vi)
vp := GeneratePointer()
json.Unmarshal([]byte(`{}`), vp)
fmt.Printf("GenerateP Type: %T\n", vp)
}
Which outputs:
Generate Type: map[string]interface {}
GenerateP Type: *main.Test
I suggest you return a pointer in GenerateEmptyItem() instead of the actual struct value which is demonstrated in the GenerateP() example.
Playground Example

Idiomatic way to pass any type in Go (while keeping compile time type-checking?)

I am parsing a form and have written a number of functions func parseAndValidateX(val string) (T, err) where T is any type.
Now I would like to write a closure func catchError(T, Error) T, so that I can do something like:
errors []Error
func catchError(val T, err Error) T {
if err != nil {
//append err to errors
}
return val
}
data = MyStruct {
Age = catchError(parseAndValidateAge("5")) // Age is a int
DistanceFromHome = catchError(parseAndValidatePi("3.14")) // DistanceFromHome is a float
Location = catchError(parseAndValidatePi("3.14,2.0")) //Location is a custom Struct
}
if len(errors) > 0 {
// o, oh
}
Is this possible in Go? How can this be done easily/idiomatically?
Nope; you cannot do that since Go has no parametric polymorphism for user-defined functions. All you can do is take and return interface{} and add type assertions at the call sites.
Everything in Go is type-safe if you don't use the unsafe package, so you don't have to worry about that. A type assertion will fail at runtime instead of at compile-time, though.
If you are willing to violate DRY, though:
type errorList []Error
func (l *errorList) addIfNotNil(err Error) {
if err != nil {
*l = append(*l, err)
}
}
func (l *errorList) int(x int, err Error) int {
l.addIfNotNil(err)
return x
}
func (l *errorList) float32(x float32, err Error) float32 {
l.addIfNotNil(err)
return x
}
list := errorList([]Error{})
data := MyStruct{
Age: list.int(parseAndValidateAge("5")),
DistanceFromHome: list.float32(parseAndValidatePi("3.14")),
}

Resources